So, the problem isn't Rottnest. It's that hospitality in regional towns doesn't qualify as remote work. If cultural exchange is the point, having backpackers doing hospitality in towns where people have time for a chat sounds like the right thing.
As to farm work, that's a tough one. We want that to be an attractive option. Not knowing anything on the topic, I can only assume it isn't attractive because workers are not compensated enough for the work.