USB 3 drives are faster than SD cards and also way more reliable. An SSD drive could be powered from the Pi itself. But a nano size USB stick is also worth considering. Some good advice in the article below on how to get going with it.
USB boot is enabled by default on the Pi 4B (and the Pi 400, by the way), but there is a small caveat to that. It seems that some early bootloader firmware versions did not properly support this, so you have to be sure that your Raspberry Pi 4 has bootloader eeprom firmware dated Sep 3 2020 or later. If you happen to have an older version, the instructions for updating are given in the USB mass storage boot section of the Raspberry Pi Hardware Documentation.
See Booting my Raspberry Pi 4 from a USB device | ZDNet
USB boot has been possible since the Raspberry Pi 2 Model B (v1.2), but it has only become really worthwhile with the Pi 4. Here is some information, tips, tricks and opinions explaining why and how.