Former audio engineer here. Things that touch your body are highly subjective. Add in the audio angle, and the variables spiral out of control. Driver size is a guideline, but not an absolute litmus. This is a leaky comparison, but a great set of IEMs might have 10mm drivers. A high-end smaller driver will be more accurate than a cheap larger driver.
When I'm considering a new set of headphones or earbuds, I start by looking for aspects that generally outline a quality set of audio gear, such as AptX HD support, serviceability/rebuildable/parts availability, driver type and manufacturer, DAC manufacturer, wired option, etc. None of these by themselves are indicators of quality, but they are typically common to good gear. A published response curve from an independent lab is a nice-to-have. Once I approach the pull-the-trigger phase, I'll consider reviews from SoundGuys and people I trust in my circle.
ANC nukes audio quality. There are plenty of decent sounding cans with ANC, but they take an instant hit to sound and massive bump in price. In my experience, better ANC means lower sound quality. I prefer passive noise reduction for noisy environs. Comply foam eartips on your IEMs go a long way to improving isolation. Yes, this would mean a set of headphones and a set of IEM, and picking the right tool for the job. I've tried a bunch of ANC gear: Sony WF-1000XM4, Sony WH-1000XM4/5, Bose QC, Sennheiser HDB 630, and so on. In the end, I always go back to my wired hardware because it just sounds that much better.
