+1 for San Quentin. It's great. JC at the height of his powers.
Early stuff is available in many,many incarnations. There's stacks of Sun recordings and rockabilly compilations out there. Usually with Get Rhythm, I Walk The Line and all the early hits on.
That said, my favourite recordings are probably the American Recordings, party referred to above. Volumes 1,3, and 5 are my personal favourites. They're all pretty fantastic. Late period Cash in pristine recordings. There's a big out-takes box called Unearthed which features many great tracks not on the American Recoordings series but recorded at the same time.
If you want to go bonkers there's a huge Cash Box set on Amazon, can't remember the price and exact details but it's loads of discs (50-60 or so) for £100 approx.
I've got a few fairly dreadfully bland Cash albums from the fallow early 70's period and one ghastly re-hash of his own back catalogue from the early 80s.
He's great duetting on Dylan's Nashville Skyline album from 1969 though.