d_a_n1979:I agree The Rotel RA04SE is too much of a sideway step (if not a little backwards IMO) I think you'd be best to upgrade the CDP first with the likes of a Marantz CD6003 or a Rega Apollo if you can find one within budget. The likes of an Arcam CD72 or CD73 would be ideal. As for amp upgrade; minimum spec you should look at IMO would be the NAD C355BEE and then up from there (Creek Evo II, Arcam FMJ A28, Leema Pulse, Rotel RA06SE, Rotel RA1520, Rega Mira, Marantz PM7003 etc...)
Given that the OP mentions "budget amp" I still reckon Arcam A85 (or depending on size of listening room, A65+). Otherwise, Creek Evo MKI or Roksan Kandy LIII would make a cracking combo.