The key point in all this is that you were eleven years old, bro, and being abused and threatened by adults whom you knew would hurt you if you didn't do what they wanted. However you reacted, and whether or not any of those younger kids were involved, all of this is the fault of the adult abusers - you were not to blame either then or now.
Stefan, you may have other memories of doing things - apparently willingly - with these abusers or according to their orders. Here what you have to remember is that after a time an abused boy just gives up; I know I did, and I now understand why.
By the time I was 12 I had been abused and hurt so many times and in so many (to me) unimaginable ways that I was beyond caring what happened to me anymore. What I cared about was that no one should discover what kind of boy I was. As for the abuse, well, I thought I deserved it for being so worthless, not realizing that this feeling of worthlessness had come from the abuse itself. I went with the abuser willingly, even got in his car, stood at his front door while he parked, and then went upstairs to wait for him. I was lost and doomed, I thought; this was never going to end.
It's important not to judge yourself in any way, Stefan. The situation you are in now allows you to look back from a position of safety; I know it isn't an ideal position moving around so much, but at least you are safe. Back then you had no idea when the next beating or rape would come or how badly you would be treated. Whatever happened, or whoever it may have involved - none of it was your fault. The important thing now is that you survived.
Nobody living can ever stop me
As I go walking my freedom highway.
Nobody living can make me turn back:
This land was made for you and me. (Woody Guthrie)