I think your presence will help to dispel those false beliefs that a man cannot be a victim of sexual abuse or rape. It may also prove to be empowering for some women to see men showing their solidarity and shared experiences with sexual violence.
If you have supportive friends or family, maybe consider bringing them with you to make you feel more at ease, especially if you feel the experience could be triggering for you.
I hope that if you do participate, that it will be a positive and empowering experience for you and those affected by sexual violence in your community.
"Don't be ashamed to need help. Like a soldier storming a wall, you have a mission to accomplish. And if you've been wounded and you need a comrade to pull you up? So what?" Meditations of Marcus Aurelius, Book 7:7