You may be right in thinking that you did not identify him correctly because you idolized him.
I knew for decades what my first perp did to me, but I never thought of it as bad or damaging during all that time because of the good feelings, the love, that I also remembered.
On the other hand, after being violently raped at age 12, I couldn't even think about that fact. The very day after it happened, I had no words for it. Without the words, it had no reality.
It wasn't until 5 years ago when my life fell apart, that I was able to say it and fully ( damn TOO fully) remember what happened.
Your mind, your brain, will do what it must to protect you. Fear and pain will alter and disguise, even obliterate memories.
Here is something that may help. It is taken from a site I have found helpful: http://www.kalimunro.com/article_survivor_memories.html
While all memory, especially declarative memory -- the story or details of the event -- is reconstructive (recreated over time) this does not diminish the truthfullness of the memory itself. Sometimes because recovered memories can be hazy, it can be hard to be sure of all the details. Sometimes, because of how memories are categorized by our minds, it is possible to remember two different incidents as having occurred at the same time. Again, just because this happens doesn't mean that what happened isn't true, only that the events may not have happened in that exact way.
A good analogy is people are in a bank when a bank robbery occurs. They are frightened by the robbers and their guns, and are afraid for their safety. After a robbery, it is not uncommon for witnesses to contradict each other about the colour of the robbers' clothing, even what race they were, and the total numbers of robbers present. But, no one is uncertain about the fact that there was a robbery and that they were scared for their lives.
While no survivor can be certain that every single detail of their memories of sexual abuse are precisely accurate, it is possible to be confident that the crime of sexual abuse occurred, to know who did it and to know approximately what age you were.
Your sanity is not in question. I believe it is a sign of strength and growth that you now remember.