I have a pretty wide set of musical tastes that includes more than just the Jersey Boys, and Frankie Valli & The Four Seasons.
There were recently several interesting music related podcasts from San Diego's KPBS These Days:
When I'm in the mood for the blues I always turn to The Roadhouse podcast.
When I'm in the mood for classical music, I find WGBH Classical Performance Podcast to be great. I also find the classical feed on CBC Radio 2 to be a nice streaming option.
The Magnatune "great pianist" podcast is very good if you want something 'lite' playing in the background.
The Alternative Classix Podcast is a well produced and researched 80s 'new wave' radio show you will not be disappointed with.
For those that like electronica and/or techno ... I highly recommend:
If you want something more mellow but still pretty electronic then check out the Ultima Thule Ambient Music podcast.
I also enjoy the Coverville podcast and the Rock'N'Roll Geek podcast.