As with ANY excersize, its only doing you any serious good if you can hear your heart pounding in your chest and you are red and sweaty... anything else is just moving about a bit.
Whilst that's true up to a point, I do *not* recommend you immediately start trying to do that sort of thing if you've not done regular exercise for a long time because you're just going to knacker yourself out and probably de-motivating yourself.
When you start riding to work, don't try to over-do it, just take it easy and make a record of how long it takes you to get there. Work out the average time for the first week, then the second week try to knock a little off the time. It doesn't have to be much, all you want is a steady improvement over the course of a month.
Once you're into the habit of exercising, *then* you can start pushing the pace a bit, but not before.
It might also be worth buying a pulse monitor. Initially you want to start exercising at around 65% of your maximum heart rate (Multiply your age by 0.7 and subtract that figure from 208), later on you can start pushing up to 75% to 80% but not initially.
As for joining gyms, I go twice a week, but instead of just doing a work-out I do a class called Body Combat (non-contact martial arts moves set to music). This is pretty high-intensity exercise (if you do it flat out you can burn 700-800 calories an hour!) but you don't have to do it that hard straight away, of course.
I find it's much better than working out on your own because there's more encouragement doing it with other people (and the fact that I'm one of only two or three guys in a class full of ladies getting sweaty... ahem, moving on...)
If martial arts moves (not fighting, I re-iterate) aren't your thing, there's classes like Body Pump (an all-over weights programme) or RPM (a class full of exercise bikes, although I find that boring because I cycle most places anyway!) or if you want to work on flexibility and your core, Pilates is excellent for that.
Mostly the important thing is to get into a routine, apart from cycling to work, put the exercise into your calendar and do it regularly for the best benefit and because that will make you less inclined to start skipping it!