It's mostly about jitter and the requirement to produce the audio signal in realtime. FLAC 24/96 is a lot of data to decode at both steps in the process (tops 3200kbps or an order of magnitude more information than the highest resolution MP3). I realised the sheer impact of the quality of the source material and D/A conversion when I heard two Dynaudio demos at the Bristol show - same speakers, one demo with an iPod, one with Chord Red Reference CD player...
A general purpose operating system e.g. Windows, MacOSX and your home/office equivalents in the Linux world are built to do lots of things, not one thing well. Then there is the issue that S/PDIF is a compromised form of the original pro audio setup - at the time the compromises were about reducing cost, but now of course the cost difference is minimal which is sort of ironic as despite this the consumer audio world seems to have stuck with it. So we have a compromised compromise when we try and use a general purpose operating system to provide information over S/PDIF, with the two types of connection having different issues to overcome.
Generally the issues surrounding optical TOSLINK are easier to mitigate than those with coaxial insofar as the TOSLINK issues are based on the quality of the cable, its length and the paths through which you make it travel. In particular keeping the connection short and keeping it from having to go round any tight bends or being squashed are the most important. A 6m cable neatly routed around the outside of a room is likely to be an issue, a 0.5m cable running in a constant radius of 150mm will be good. Even better (marginal, but better) would be .1m cable in a completely straight run - but for the life of me I couldn't figure out how to do that and still be able to switch both devices on...
Then you need an operating system that has been optimised for the purpose, not built to do lots of things reasonably well. And ideally hardware that has been optimised, which is where the challenges start and why I want to try going down the USB route. So the operating system is not that much of a challenge. Run a Linux realtime low-latency kernel, preferably headless (without a graphical interface device nor producing a graphical output) to prevent any other processes getting involved, and cut down the other processes to a minimum. If you must have a local graphical interface make it monochrome or utterly optimised for minimal graphics processing overhead, you can do the pretty bit somewhere else (and should). Keep it simple, the software that plays music should be just good at you know, playing music (unlike UPnP...), the method you use to get the media store available should also be simple - I use SMB, NFS would also be an option, and an FTP mount would be an interesting approach (very, very low overhead, very fast).
Optimised hardware is harder. The first priority to me was to remove noise; electrical noise was reasonably easy to mitigate, use a TOSLINK. The other noise is the fan found in most computers, but there a number of fanless options now, particularly if you don't need a lot of processing power because you've optimised the operating system; use an SSD rather than traditional hard drive and shortly afterwards you have a computer that acts a lot like a piece of hifi - you press the button on the front to turn it on and off and you only know it is on because there is a light on the front. It makes no noise at all, even you put your ear to it.
But optimisation is about selecting the best possible components and putting them together in an optimised way. Not having my own PCB assembly plant I can't do that, so the sound card which decodes the FLAC, the motherboard quality, etc. is a compromise that I cannot surmount, hence my desire to push the full audio processing part to the purpose-built DAC.
OTOH I think when my EB2s arrive I'll have a setup that would make most audiophiles want to listen some more, I just can't resist looking for further (cost-effective) optimisations...