You're not gonna become a master in your spare time, but you can get decently good at photography in five years of dedicated but non-full-time practice. It's taken me about twice that to get decent, but I fucked around a lot along the way.

You don't need a ton of fancy equipment, but photography does require some gear. I'll limit the gear-buying here to once a year (not counting books).