Main sign of overdriving speakers is distortion, harshening of treble, bass notes causing the bass unit to sound more wooly and less detailed as the cone acts less like a piston with the whole cone moving in and out together. If pushed further increased distortion and the bass units hitting their maximum excursion making clacking sounds. Next stage is when serious permanent damage occurs as crossovers are driven beyond their design limits and components break, the tweeter gets too hot and burns out its coil and the bass unit either burns its coil or the coil goes permanently of centre and rubs against it's magnet.
Once you start to lose quality, that's generally a good sign that you should reduce the volume a notch or two. It's easy to avoid undue distortion and damage.
That said you can damage speakers by using an underpowered amp driven beyond it's limits (clipping). Generally you'd kill the tweeters first. Again you can hear things are not right well before damage occurs.