Taking paternity leave and terrified about my job

Got offered 12 weeks paternity leave and I want to take it so bad but my company culture is weird about it. They say it's cool but I know my boss will think I'm not committed. Already got passed over for a promotion last year. This baby is due in 6 weeks and I don't want to miss those first months. Anyone actually took extended leave and came back fine? Or am I being paranoid?