OMG I know exactly how you feel. I have the same issue - when I'm spending time doing music I feel guilty that I'm not doing something "productive" either in my profession or around the house, with my family, etc. You're not alone in that.
For me I think it all comes down to giving myself permission. I'll decide for example that one day is a music day. That's what I'm doing that day. It's pre-allocated and potentially even scheduled on my calendar. Just like making a date to go see a movie or visit a friend or do anything else I want to do in life. And it helps if I also have permission from my significant other, although she would think it silly of me to ask... it helps my own subconscious to ask her if she minds that I take a "day off" to spend doing music.
As for inspiration... listening to other music in a genre I like certainly doesn't hurt. Exploring my absurdly-large Refill collection also makes it pretty easy to get ideas. Also advice I heard from Olivia Broadfield in an interview was that most of your music is going to be crap, and occasionally something brilliant comes out. But you have to get the crap out of the way to allow the brilliant to come through. So just create... create lots of crap as quickly as you can and the brilliant stuff will come faster.
I hope that helps, but just know that you're not alone in this for sure...