I'm considered an expert in my field, and with almost 20 years experience I tried what you are suggesting.
The thing all us tech geeks forget about when starting a business, is all the stuff that actually goes into the running a business part of it.
You want to focus on the work, but there are bills, invoices, client and project management, etc.
I had clients, I had projects, but I didn't have nearly enough time in the day to handle all of it.
Then clients started paying late, causing all kinds of fees to stack up. So even when I had time to do the work, I was distracted and nothing got done the way I wanted.
I tried hiring an admin, but it was too little too late. In the end I went back to being the in house guy at a large company, where I get to focus on the stuff I am good at and enjoy.
I'm not saying don't do it, I'm saying be very very aware about all the other elements before going off on your own.