Unfortunately, as you may be aware, the few large manufacturers who still produce them only sell cheapies. I'm not aware of anyone making high quality portable CDPs for the last few years.
A couple of years ago I picked up two new-in-box players from ebay - a Sony D-NE830 and a Panasonic SL-CT810, both had obviously been out of production for some time, I think they cost me around the £60 mark each.
In terms of sound quality, I'd rate the Panasonic as the best sounding player that I've owned, a little bright but much more musically involving then the many players that I've owned over the years, also it has optical digital output and I think it's just better designed overall. The Sony machines, especially the later ones, always have issues e.g. buttons placed underneath the unit, all the various outputs using the same jack, etc.
I've never been that terribly impressed with the sound quality of them either, the D-NE20 being the worst I've ever heard, very weak sound with a ridiculously high noise floor, shame as it was quite a nice little machine otherwise.
Just keep looking around on eBay or Amazon Resellers as many bargains pop up there all the time.