I struggle with this to a degree as well. Not exactly the same way, but still very much tied to my relationship with my father.
My father was very distant. Hugs from my father, which were infrequent except for before bed, were always awkward and uncomfortable. Not sure why, but maybe it was because I never felt that love.
So my struggle is that I always feel like God is distance and cold.
Our relationship with our biological father will definitely effect and color our relationship with God. I don't think it's easy to overcome, but it can be overcome with time and trying to trust God and take His word and promises as they are, because God never breaks a promise and is always true to His Word.