Career growth vs being present - am I crazy for turning down a promotion?

Got offered a director role that would mean more travel and longer hours. My kids are 8, 6, and 4. I turned it down because I'd barely see them during the week. My dad worked constantly and I swore I wouldn't do that. But now I'm second guessing myself. Is this a mistake? Will I regret not taking the opportunity later? Just looking for some perspective from guys who've been here.