Was I really in the wrong?

I have 7 siblings, and 4 of them are in the same school as me, three older sisters (15, 15 and 17) and one younger brother (11). Last friday I was at school and my older sister (15) came up to me and said that our little brother (11) was being bullied and picked in his class (they had stabbed him with a compass in his finger), and that we should go and sort the bullies out. So I went with them and it basically ended up in fight between my three older sisters and these 4 boys (they haven't ever been violent before, but they were really angry). I didn't get involved, I don't like that sort of thing; (and its not becuse I'm weak, I have my own punch bag and dumbells at home), and afterwards my sisters were giving me greif about it and telling me that I was a wimp and wasn't really part of the family. Was I really in the wrong, I'm starting to think that maybe I should have joined in and beat up the guys. What do you think, I feel terrible?