Boss asked me to stay late, had to pick up kids from daycare

He knows I have a hard stop at 5:30 for pickup. Asked me to stay for an 'important meeting' at 5pm. Had to tell him I couldn't make it. Felt like I was letting work down even though my kids are my priority. This is the balance nobody warns you about I guess.