As I built my own classical cd collection I started buying bits and pieces, but then wanted to be more comprehensive and bought box sets of, say, all a composers symphonies, or all their piano concertos. This resulted in a degree of duplication because I had already made many "ad hoc" purchases at the start. Of couse, it is sometimes nice to compare recordings, but just an issue to think about.
In terms of cds, if you buy those, there are many excellent box sets for sale, new or secondhand on the likes of amazon, and it has never been cheaper to build a good collection. Although I haven't bought it myself, as I already had recordings of the same music, you might consider the box set "Great symphonies the zurich years, David Zinman" which has complete symphony cycles by Beethoven, Brahms, Schubert and Mahler. 50 cds for c£60...a bargain starter pack.
As you dip in into classical music more generally, you will notice a steady evolution from chamber music to the "classical" symphonic works of Haydh and Mozart, to the Heroic style of beethoven, to the romantics such as Rachmaninov and eventually the more introspective 20C work of composers such as Shostokovich. You might find it helpful to have this process in mind as you listen to the different composers.
You really have a lot to discover and I hope you enjoy the process as much as I have.