You're obviously a caring brother who wants to help, but it sounds like you two are on the same airplane that's crashing.
Remember you need to put on your own oxygen mask before assisting others. If I'm interpreting your post correctly then it sounds like you're having memories of your own.
This doesn't make you weak and it's not your fault. It doesn't mean you're letting your brother down. Survivors of abuse are incredibly strong...they've survived! Maybe you could confide in each other and help validate each others' memories? Maybe that would make it less scary for you both. Just a thought.