The memories can be glimpses in time and may not reflect all that happened or at particular times you may not be able to see who was there. Memories can be extremely unsettling. Details emerge over time--was it a new incident or part of another. It is difficult to put them into perspective at times. But over time, at least for me, clarity does emerge. With clarity comes new emotions and feelings that have been extremely unsettling and hurtful.
I only hope your memories one day bring the clarity you need to understand your older half brother and what happened to him as well as what he did to you. I am sorry you have lost someone you saw as your hero--a guiding light in your life. I do hope one day you resolve these memories and you have your own internal peace. Please stay strong and time will help you heal. Also, your twin maybe help to help--but only approach him when you are ready and feel you can handle whatever he may say.