Perhaps the main problem is the idea that you have to be good as something to do it. I think there's a lot of this in school. Being afraid to fail at something, especially in front of other kids. So a lot of people don't try, because how can anyone take the piss when you don't care yourself?
I have a friend who's pretty sporty and good at most sports, but he'd always refuse to come rock climbing with us, and I'm convinced that the main reason was because he didn't want to be the shit one in the group at a sport, because he was so used to being one of the best.
We need to make it okay to be in a shit band, to be an actor who's only good enough to be in an amateur dramatics version of Dick Whittington, to write crap poems, to make a shit spice rack in woodwork class, to play five-a-side with another bunch of overweight, middle-aged blokes with no co-ordination. We have this attitude, particularly in education, that if you're never going to make a living doing it professionally, that it's worthless. That's why no-one has a hobby any more.