I think it's possible to have both in a federated system. Let the instance the user is on keep the tally of who upvoted what, and let the instance the post is on to know only the tally from other instances. Should be up to instances whether to show this data to users or not. This way it'd be easy to find and defederate single user instances manipulating votes.
But, on the other hand, I don't see a reason to care about privacy of votes if you can't even delete a post or comment. Fediverse is, by design, not very private, why bother with just this one aspect of it then.
At first I was going to say "сука бля, nyet, we have enough shitheads already". But then I realized that they would honestly just fit right in. So you have yourself a deal... as long as you could also find a safe place for Snowden - the dude does not deserve to be in this hellhole.