I outed my bully, but I feel bad now - is there a way to undo the damage I caused?

I think you did the right thing. He's responsible for his own actions, and also with coping the consequences his actions cause. No need to restore karma as I see it.