Sure you could spend some money on better speakers, and no doubt, your setup will sound better.
But, it will just be a better version of what you have now, if that is what you want, go for it.
There are a few things that you can do in hi-fi that are complete game changers, active speakers, planar speaker, vinyl platback and in this case valve amplifiers. It depends on what you want and where you want to go.
Personally, if I wanted a hi-fi 'experience' and was spending £2-4k, the last thing I would buy is a conventional source > amp > speaker combination from any of the 'usual suspects', I just would not do it.
Partly of course it is a case of 'been there, done that' but it is also a desire for a setup that, in a number of ways is simply more 'involving'. Involving in that time needs to be spent on the choice, the setup, involving in the sense that playing music becomes an event and, above all, involving in the sense of getting closer yo the music.
I don't really do this myself anymore, but when I did, this was the period and the method by which I got the most out of recorded music, it wasn't forever, but for a while it was bl**dy good.