Ex won't stick to custody schedule

This is driving me nuts. We agreed on alternating weekends and she gets them Wed nights. Last month she kept them an extra 3 days without asking because she had a work thing come up. I said ok the first time but now it's happening every other week. I don't want to be the guy who says no to extra time with my kids but I also have plans, work stuff, and frankly I need predictability. Do I need to go back to court over this or just talk to her again? We're trying to keep things civil but I'm getting frustrated.