This is what I love about this place. We've all had different experiences that have led us here, but thank god there's a place we all kind of sort of understand each other a little bit.
Anthony, here's my two cents. I'm 13 years younger than you, but I tend to do the same thing. Yes, I have had quite a bit of confusion throughout my life, starting pretty much immediately after I figured out that what my cousin had been having me do with him was, by definition, "gay" in the sense that there were sexual acts occurring between two males (I was ages 6-9, he would have been 9-12). I never did act out with other people sexually after that, and when I finally did have sex and start hooking up, it was with women. That was what I enjoyed. I could never make the move though unless they made the move first. I feel like a million dollars when I'm with one, and I feel like a billionaire if I do something in bed that makes them happy (need for self validation much?). Before I got into therapy though, I used to frequently check out gay porn and got off to it. That was what started my questioning, and ultimately what led me into therapy. I guess to relate this to Don64, that was the signal from my insides that I had some stuff to figure out.
After college it's been much harder to meet people, and so I've been alone, and when you go literally years without having sex, you start to wonder, at least I did given my past. That's the thing though. I get confused NOT because I find myself attracted to other men or fantasizing about other men, I question because I'm not with a woman. What sense does that make?! When you think about it, we live in a culture that doesn't really know what to do with single adult men. I think people assume there must be something really wrong with us. Maybe there is. I don't know. Where I'm at though, I live in the middle of nowhere. All the cute girls I have crushes on are either unavailable, sisters of my best friends (NOT going near those ones), or I'm just too chicken sh*t to approach. I think I make excuses, but I don't know why. I'm confident at this point that it isn't me being a repressed gay guy. I think it's because I'm really embarrassed to admit to people that I'm just too nervous or anxious to put myself out there, and embarrassed that I'm 26, my friends are getting married, and I don't even know how to have a normal relationship.
I started seeing a third therapist a while back for insurance purposes (namely, she took my insurance). We didn't really click, but she seized on something that the other two I think knew about, but didn't emphasize. It wasn't just the SA - I was horribly abused verbally and emotionally by my sister when we were younger (she feels terrible now, but the damage is done) and also my mother, so my relationships with women aren't very good. I tend to assume that if I get close, I will be manipulated or unable to trust them, because they'll want something from me, or that they're going to get close to me and then use my vulnerabilities to try and crush me emotionally. Alternately, I feel like if I took the initiative, I would be acting overly aggressive, disrespectful, and objectifying.
The CSA and a combination of bullying throughout elementary, middle, and a decent chunk of high school (my nickname was Faggot for like, 7 years) emasculated the everloving tar out of me. I've only just begun to recover from that NOW. I feel not manly all the damned time. The bullies have been out of my life for years, but in their absence I bully myself now. And wouldn't you know that whenever a prospective date comes up and I chicken out, or something falls through, the first place I go is to check gay porn to see if I react? My therapist seized on this, and offered "Did you ever consider that you do that now to trip yourself up, pull the carpet out from you so to speak?" meaning that I will fall into that habit to distract myself from whatever "real" reason there is that I'm staying single, or I'll do it to self sabotage.
Our brains are tricky, tricky things. Be patient. For what it's worth, you don't sound gay to me, but as others have said, if that's what it comes down to, then own it. You are what you are, and that's all you are! Best of luck.