Travel for work with a newborn at home?

Wife just had our second kid 3 weeks ago. I have a conference in Denver in 4 weeks that's pretty important for my career. It's 3 days.

Part of me thinks I should just skip it. Part of me thinks I'll be miserable being stuck at home (I'm not super hands-on during the newborn phase, if I'm honest). Part of me knows my company is counting on me.

How did you guys handle this? Is going to the conference selfish or is it okay to still advance your career when you have a newborn?