No, you should be fine as long as ‘normal music’ is at a comfortable level then unusual sounds won’t bother speakers. Only continuous tones at high levels, at very low or high frequencies would potentially cause issues.
I don't think you would play tracks with sub 20hz bass loud to hear or feel it on a small pair of harbeth p3esr, most tweeters go way above 20khz so you should be find as long as you don't overload the amp/speakers
Ooh, I dunno, I had my little P3's wound up during the week and I could actually feel the bass through my chair. Tested with an SPL meter and I was hitting nearly 90dB peak , neighbour was away at the time