Initially I moved to Bluray because I was sick of edge enhancement and (visible) video compression. I wasn't expecting the difference in audio that I experienced. So I would say both are important, but I would err on the side of video, as that annoys me the most.
Also, I would say that the size of the audio soundstage needs to match the picture size - a projector just doesn't match with an all in one 5.1 system, and a 32" screen doesn't go with a top notch audio system. It's just so much more engrossing and believeable when the size of the audio matches the video.