I have strong opinions here. I view moderating a small community as trying to grow a garden and set the tone such that other people feel comfortable contributing to that garden.
The core problem with negative only participation is it makes the community hostile to new contributors. Most people on lemmy are lurkers, and if they feel that their post will be met with overwhelming negativity they simply wont post.
Downvoting is a form of participation, its a negative signal by design.
If someone hates a community so much they feel they need to downvote it every time they see it, but they don't want to block the community, its totally reasonable for a moderator to help them block the community so it doesn't ruin their lemmy experience (i.e. ban them from the community so they don't see it anymore)