I only buy used as I'm not interested in new music, so if you're looking for new stuff I can't really help. I mostly buy rock as well.
As above, Flashback and Haggle on Essex Road are good. I think there's even a third on Essex Rd?
There's a good one on Camden High St, good selection of used stuff though not cheap.
There are quite a few on Berwick St in Central London as well. There are also some near Brick Lane but that tends to be more electronic/dub stuff.
Also as above there are some great used places in Glasgow -- that's actually where I started my record collection, before I even had a hifi! I think it was in the West End of Glasgow (Ruthven Lane?). Cheap and good assortment, not in the best nick though.
My personal favourite is a place in Clifton Arcade, Bristol, near the Gorge. He's always got new stuff in, cheap, and in great condition!
I find that used vinyl is much cheaper than used CDs; you can find stuff in really good nick if you're willing to look. I've never paid more than about 7 quid (for an Ex copy of Bowie's Low).
Oh, online: http://www.retrovinyl.co.uk/ The guy is based in Angel, very friendly, and often has a sale on.