Taking paternity leave and don't want to go back

I'm 4 weeks into my 8 weeks of paternity leave and honestly I'm dreading going back to the office. I never thought I'd feel this way about work but staying home with my newborn has been incredible. Anyone else struggle with this transition? How do you handle the guilt of leaving them?