I know. I hear the excuses too. And for those of us who have other things competing for our time, like the work of our day jobs, it is all too easy - seductive even - to believe our excuses. We think the lack of singular focus gives us an out.
Michael Tilson Thomas said, "The most important thing about music is what happens when it stops, what remains with the listener, what they take away. A melody, rhythm, some understanding of another person or another culture. The way those experiences add up, in the soul of a person over the years....You want to shake people even when they're not listening to the music." And he's right!
So, when you have the opportunity to share, and especially if you are offered the opportunity to play with someone else - do it! Don't let your fear get in your way.
And by the way, it's not about you. Don't focus on how you're not good enough, or you haven't practiced enough, or you're not ready. It's about them - focus on the person that you will touch, that person that you don't even know. the person who, after hearing you will be brave enough to come up to you later and tell you how much they enjoyed it...and the person who loved it but didn't work up the courage to come tell you so.