So I've firmed up some part selections and elements of the design, enough to start dropping things into CAD for some rough sketching.
I have a gen 4 ipad display I can salvage, which was the original "iPad with Retina Display". 9.7", 2048x1536 resolution, and it seems a good number of available driver boards for it. Haven't selected one yet, but there are loads of options:

What isn't clear is if any of these support the digitizer. I'm assuming not, but thats the next bit to look into.
That said, the sizing and res is great for what I want, so I think thats settled.
Next up: inputs!
So I really like the MNT Pocket Reform, but I'd like to go a bit different in the design. I think their keyboard layout is straightforward, and I've got plenty of switches and caps for that. Since its all open hardware, the design is here and the repo is here.
For the mouse input, I was thinking something a bit different. Taking some inspiration from ploopy, I was thinking it would be neat to have a combo of the trackpad and the knob. If I get creative, I could set it up so the trackpad can be inserted into the bottom, and removed for cabled use. Or give the option to make it modular with a small version of the trackball (or the MNT designed trackball). The other idea I like is maybe sticking in a thinkpad style trackpoint, which I also really like as a concept. There was a ton of great info up on deskthority, but the wiki is dead - it seems they were bought and the new owners don't actually care about it. Thats ok though, the wiki is dead, long live the archive of the wiki!
I was considering grabbing a Framework motherboard, but the recent shenanigans there ruled that out for me. Still would really like to use something like a 3588, but with Vulkan support. It seems there has been movement there with Mesa and Vulkan 1.4 support, so I'm going to be looking at it more closely to see what I can do.
Going to be sketching the rough shell this out this weekend (as long as the children cooperate enough to let me do it).
Edit: Almost forgot! Given the size I think there will be room for a meshstick to bring in some meshtastic to it is well. Should I consider a more modular base so I can make this more flexible in the future? Probably. Is it maybe worth it to take from framework? Definitely! This is why I like open hardware designs. Even if the company is going down a path I don't like, that doesn't mean everything before is ruined.
I think its going to be repo time pretty soon, will have to set it up on codeberg.

It went... OK.
It works fine and all, though I may change a few oids around after an snmpwalk and realizing I made a few spots messier than I like. Its just the most mind-numbing creation possible.