Who should apologize in this situation?

I think you should care much about what you have done. Because it reveals that you are a polite gentle man, while the women is affectation. You shouldn't think over too much, I should say you have done the right thing. You avoid a dispute. You should be pride of what you have done but not to think what the woman hadn't done.