From what I'm reading, the bass isn't something that blows you away on the 535s either. The N40 works well, particularly with the DAC you've got.
Curious, as someone who hasn't done that on a phone, what's it like using the Dragonfly on your iPhone? Is the added bulk inconvenient to any degree?