The wikipedia article on room modes seems fairly accurate from a brief glance, and should get you started,
Here's a room mode calculator: http://www.bobgolds.com/Mode/RoomModes.htm
And here's a website of a guy who takes acoustics very seriously: http://www.ethanwiner.com/acoustics.html
Basically, standing waves set up between parallel surfaces in your room at frequencies whose wavelengths are: twice the dimension in question, equal to the dimension, 2/3 the dimension, 1/2 the dimension, etc, giving a harmonic series of resonances. More complex modes are set up by sound bouncing obliquely around the room in 2 and 3 dimensions, making the calculations pretty complex.
You only need to worry about room modes in small (ish) rooms and at low frequencies,
If your system is halfway decent, then sorting out room acoustics is the best way to fix muddy bass and imaging problems.
I hope this helps!