OK, I'll chime in on this one.
For years I avoided the damn apartment building where I was brutally raped. I couldn't drive on the road where it was, nothing. Eventually the issue came up in therapy. My T said that he would go back with me if I wanted. I told him no, because that was a hurdle I wanted to overcome on my own.
It took me months. I'm a big bike rider, and one day I rode by. My body was shaking so hard I thought I would cause an accident. I didn't.
Next, I went into the parking lot, looked at the building, and had a bit of a breakdown. Couldn't hold it in. But I got through.
Finally I got the right idea in my head. The real point is that you're safe. Nothing can happen anymore - there, or anywhere else. It's all overwith. The bad shit is in the past. I faced the demon. And I won.
There are still times when I get those bad twinges when I go near there. But they go away eventually, and I realize that the building no longer has any power over me. It's just a building, nothing more. And it can't hurt me now.
Nor can the house where you were attacked. It's just a building, and once you master your fear of it, I think you'll progress even more in your recovery.
Hope this helps.