What are you trying to do that an older, cheap IPad (or other tablet) can’t accomplish?
BudgetAudiophile
A place for AUDIO enthusiast to share, discuss and listen to others people setups
A phone or tablet would work great. Last week, when I was trying to figure this all out, I pulled out a Samsung that was a few years old. Threw it on the charger and it swelled up like a balloon within 15 minutes. That made me think twice about leaving a phone or tablet plugged in all day at home, while I'm not there.
Yeah, non-portable audio players don't really seem to exist commercially as far as I can tell. I think a raspberry pi zero 2 would fit your needs rather well (it costs like $15), although the initial setup might constitute more hacking than you're willing to put up with.
I've been eyeballing the Zero 2. I'm no pro but I've been messing with Pis for years.. I'll probably give it a try. I've found so many small computers for signage.. then I found rack mount media players for in-store music. I'll look into that now.