I'll probably get hung for this opinion but I question the validity of hi-res audio. The Nyquist Shannon theorem has yet to be disproved. That means that a sample rate of 44.1khz is sufficient to produce 20khz audio. Bit depth might affect dynamic range but 16 bit should be sufficient.
I did toy with hi-res audio. That was until I bought a 1997 Rotel DAC which can only cope with 44.1 and 48khz. It may have it's limitations but as far as audio quality is concerned it wiped the floor with my 768khz capable modern DAC.
The other factor is the quality of other components. In all honesty, is your other audio equipment of sufficient quality to differentiate between an iphone or a network streamer as the source. You have to remember we're talking about comparison of two digital sources. Not easily discernible. The differences are going to be due to jitter, electromagnetics and perhaps microphonics.
You'll find sites on the internet which will allow you access the same music in 320kbps and in lossless FLAC file. Test yourself, if you haven't already. See if you can tell the difference. If you can't , then definitely stick with the iphone for the time being. I used a reasonable pair of planar headphones and only managed 80% success. But then at age 63, I was over the moon.
Whatever you decide - enjoy the music! That's what it's all about.