Noise regulations and their enforcement varies between local authorities, but in general terms if you are annoying your neighbours then you may face a visit from the noise police.
How to test? Get a sound pressure meter and go into your neighbours' place and measure the sound levels, I guess, after checking with your local environmental health people what limits they enforce. But whether your neighbours will be up for this is a diffrerent matter.
If the sound is 'going through the walls' it's probably actually going through the floor joists, as mechanical transmission of bass is usually more efficient than airborne - you could try putting the speakers on solid paving slabs to damp out some of this by sheer mass, or a more hi-tech approach such as the isolation products from
Auralex, sold in the UK by
Audio Agency, but these aren't a cheap option.
Yes, smaller speakers might help a bit, but the 9.30 effect you mention is as much about the speakers shifting air as it is about the volume level setting - that's why some equipment has a loudness button to boost bass and treble at low levels.
However, most amps do start sounding better at about the 9-10 o'clock position, so you could try using devices such as the
Rothwell Attenuators, which reduce the signal coming from your source component and thus let you run the amp at a higher volume setting for a given sound level.
Good luck - and try to keep friendly with the neighbours.