In conventional connections, one of the two conductors forms the outer shield of the cable. In a balanced connection, each conductor is just a conductor, and the outer shield is entirely separate. In addition, the two channels are sent 180 degrees out of phase with each other, so any hum picked up on one conductor is picked up in reverse phase on the other. In the receiving equipment, the in-phase and out-of-phase noise cancel each other out, due to what's called 'common-mode rejection'.
Balanced tends to give better rejection of interference, which is why it's so widely used in professional applications such as recording studios and live stage-work. In the hi-fi arena, balanced connections between components are generally preferable where the equipment at either end supports them.