Ex wants to change custody schedule last minute again

This is the third time in two months she's asking to swap weekends. I get that stuff comes up but I've already made plans with the kids both times and had to cancel. How do I set boundaries without being the bad guy? My lawyer said I have the right to say no but I don't want to hurt the kids or seem unreasonable.