Look, communicating awkwardly doesn’t make you a horrible person. I think part of it is you have a confidence problem. It sounds like you always cowtow to your coworkers’ reactions, even going so far as apologizing when you haven’t objectively done anything wrong (your responses don’t seem that weird, but maybe it is the way you say things). It seems odd to me that anyone would be offended by an offer to go home early, unless you’re making it seem like you specifically don’t want them to be around.
At the end of the day, your coworkers are just people with their own issues and imperfections. They are probably not experts at communication either, so don’t treat them as such. I would not take any social cues from coworker 1.
You may not be able to change the dynamic at work, so my advice is to find a way to socialize outside of work (with people who actually like & respect you!). That way you won’t have to rely on your workplace for those needs.
Slightly unrelated, but last time I vomited during a flight they refused to take the bag and throw it away. I had to sit there for 6 hours, holding an uncloseable plastic bag full of my own vomit. Next to other passengers. Like, I get that it’s technically some kind of hazard waste that flight attendants shouldn’t have to handle, but the alternative is me potentially accidentally spilling it on or near other passengers. I had to carry it off the plane with me like it was my carry-on. It was absolutely ridiculous.