I disagree. I think upvotes/downvotes being public contribute overall to a high-trust culture. Mods absolutely can be stupid about it and ban people for single downvotes, but some people really do mass downvote entire communities and/or spin up alternative accounts purely to downvote.