will all my Jellyfin traffic go through the VPS
Yes
and count as bandwidth used?
Yes, twice (download from home to the server, and upload from the server to the client)
I do the same thing - I have a 3TB limit, but usually only use 300GB, sharing Jellyfin to a dozen or so users.
Edit: I'm sure there are plenty of good VPS providers out there, I personally have been using NerdRack for a few years now (got a VPS on special and the rate is locked as long as I keep it). Looks like they'll do $11/year right now for a KVM VPS that's sufficient.