I'm very picky about what I purchase on any physical media now, but especially vinyl. It is proportionately cheaper than it was in the 80s, but still expensive when there's so much pressure on our pockets and when we also have the cheaper options of CD, download and streaming.
My most recent vinyl purchase was 'Here It Is: A Tribute To Leonard Cohen'. It's one of the best tribute sets I've come across, beautifully recorded and sounds wonderful on my turntable. It's special enough to justify the price, as are the Cat Stevens deluxe editions (for example) and new pressings of Crosby, Stills Nash (& Young) albums I've purchased previously. There's a new gatefold of 'Once Again' (Barclay James Harvest) due soon, and I love that album enough to buy it.
@DCarmi mentions CD box sets that can represent bargains, especially the case with classical music. My collection includes many of these including complete recordings of Menuhin (Warner), Bernstein (DG), Solti (Decca), Perahia (Sony), Brendel (Decca), Stern (Sony), Richter (multiple), Rachmaninov (RCA), Bergonzi (Decca), Perlmann (Sony) etc... Then there's the 'label' and orchestra sets I have, including RCA Living Stereo, Mercury Living Presence, DG111, The Decca Sound, Berlin Phil 100 years, Vienna Phil anniversary set etc... These are all beautifully presented and offer a ridiculous amount of music recorded by the cream of musicians. The relatively low prices are achieved because many of these recordings have paid for themselves several times over already. In many cases it's about squeezing whatever remaining revenue possible out of these recordings before putting many of them out of print, with the possibility of doing similar sometime in the future. Most of these sets are much better value new than secondhand because they have limited runs and demand is high. I've seen some of the sets I have being offered secondhand for hundreds of pounds. This isn't a completely new situation, because similar happened years ago with Universals' 'Great Pianists of the 20th Century series', Philips Classics' 'Complete Mozart Edition' etc.
Special editions of pop/rock albums show similar trends - Jethro Tull book editions, for example, £30 new and then often selling for £100+ once they've gone out of print. It sounds crazy, but this actually works in the labels' favour - once fans cotton onto the short lifecycle of the new product, demand is increased at point of release, and there's little to no risk of excess stock floating around. I have a big box set of Caravan's complete output which was circa £200 new but prices are going through the roof now.
When these things are reissued on vinyl they command higher prices than the alternatives because the format is trendy again, and there's a whole new audience for it - newcomers buying music for the first time, and those who sold their collections with the advent of CD. Add to this the limited capacity for pressing vinyl, and there you have it.