CDs don't take up a lot of space, I have 500 CDs on a 512 GB flash drive with lots of room to spare. I prefer the USB to a drive so you don't have to worry about powering the drive. As for 'HD' the only thing to consider is that you use at least a USB 2.0 device; but those go back to 2000, so unless you're pulling one out of your drawer that's *really* old you'll be fine.
One last "trick", if you get a USB that's at least twice the size of your CD collection, then you can just copy the entire folder over and tell the computer to skip like-named folders/files. This comes in handy when you get lazy and forget what CDs you've placed on the USB.