Thinking more about the PEP 13 change, do we really need to change it? Very few people will care (current company excluded of course :-). Also fudging with election rules shortly before the election can feel suspicious (certainly in the current US political climate).
If possible I would rather amend PEP 8107 to clarify STAR’s “pre-established tiebreaker” to use randomness except in the final round, when PEP 13 takes over.
PS. Maybe someone can clarify the following to me. If there’s a tie in round N of Block-STAR, where N is not the last round, isn’t the loser of the tie going to be the winner of the next round?
PPSS. If the answer to that PS is yes, maybe if there’s a K-way tie we should resort to manual resolution if N + K - 1 is larger than the total number of rounds. Of course this gets more and more academic.
Nobody has shown the slightest suspicion here, and I don’t expect anyone to. The PEP 13 text was suitable for Block Approval, but is underspecified for Bloc STAR. The change is clearly motivated and very straightforward.
Seems like needless complication to me. @zware’s proposed change is easy, simple, and uniform.
The most obvious hole is that ties can be K-way for K > 2. In which case there are K-1 > 1 losers, so “the” loser doesn’t apply. Whether the next K-1 winners must be those is something I don’t want to burn time thinking about.
Mostly because I don’t see value to finding “an answer”. A tie means the electorate expressed no preference among those tied. The time to express preferences was when casting the ballots.
Or, to my eyes, more needless complication .
In any case, BetterVoting supports none of that stuff. Part of the 'voting experience" is picking a service and letting it tell us who “the winners are”. When voting ends, the results are available instantly, with a nice graphically illustrated account (on the site) of how the winners were picked.
An alternative would be to use BetterVoting only to collect the ballots, and ignore its idea who the winners are. We could then download the ballot data and use @larry’s software to implement whatever tiebreaking scheme we like (including manual intervention).
Nor attractive to me, but wouldn’t fight hard against it.
In each round we use total scores (6, 6 and 5) to choose a pair of candidates and then do the runoff where we count how many times one was preferred to the other by each of the voters. In a runoff Cand1 vs Cand2 would tie while otherwise Cand3 would always win. In the first round Cand3 is not selected because Cand1 and Cand2 have higher total scores so the rounds are:
Cand1 vs Cand2: tie break (choose e.g. Cand1)
Cand2 vs Cand3: Cand3 wins
The same thing could happen without ties that Cand3 is not selected in the first round even though they would beat either of the candidates that was. The reason is that the scores are treated as numeric values to be summed linearly when selecting who goes into the runoff but then the same scores are only treated as ordinal values for deciding the outcome of the runoff. The big number 5 given by some voters is decisive in the first step but becomes irrelevant in the runoff.
It is possible I have misunderstood the intended tallying method though…
Nice, @oscarbenjamin! You’re right. The loser of the first-round 2-way tie goes on to lose in the 2nd round too. Here’s how @larry’s package scores it:
$ py -m starvote oscar.starvote
[Bloc STAR]
Tabulating 3 ballots.
Maximum score is 5.
Want to fill 2 seats.
[Bloc STAR: Round 1: Scoring Round]
The two highest-scoring candidates advance to the next round.
Cand1 -- 6 (average 2) -- First place
Cand2 -- 6 (average 2) -- Second place
Cand3 -- 5 (average 1+2/3)
Cand1 and Cand2 advance.
[Bloc STAR: Round 1: Automatic Runoff Round]
The candidate preferred in the most head-to-head matchups wins.
Cand1 -- 1 -- Tied for first place
Cand2 -- 1 -- Tied for first place
No Preference -- 1
There's a two-way tie for first.
[Bloc STAR: Round 1: Automatic Runoff Round: First tiebreaker]
The highest-scoring candidate wins.
Cand1 -- 6 -- Tied for first place
Cand2 -- 6 -- Tied for first place
There's still a two-way tie for first.
[Bloc STAR: Round 1: Automatic Runoff Round: Second tiebreaker]
The candidate with the most votes of score 5 wins.
Cand1 -- 1 -- Tied for first place
Cand2 -- 1 -- Tied for first place
There's still a two-way tie for first.
[Bloc STAR: Round 1: Automatic Runoff Round: Predefined permutation tiebreaker]
Permutation was defined in 'oscar.starvote'.
Pre-permuted list of candidates:
1. Cand1
2. Cand2
3. Cand3
Tiebreaker candidates will be selected from this list, preferring candidates with lower numbers.
Choosing the earliest of these candidates from the permuted list:
Cand1
Cand2
Selected winner: Cand1
[Bloc STAR: Round 2: Scoring Round]
The two highest-scoring candidates advance to the next round.
Cand2 -- 6 (average 2) -- First place
Cand3 -- 5 (average 1+2/3) -- Second place
Cand2 and Cand3 advance.
[Bloc STAR: Round 2: Automatic Runoff Round]
The candidate preferred in the most head-to-head matchups wins.
Cand3 -- 2 -- First place
Cand2 -- 1
No Preference -- 0
Cand3 wins.
[Bloc STAR: Winners]
Cand1
Cand3
Thanks! Even if itt urns out to be even subtler, this is why I asked a question about it instead of posting my guess as fact (even though the latter would be the modern internet convention :-).
Regardless of that the answer was “no”, there’s a different way to handle this that doesn’t require changing anything, if it comes up. Suppose Ada, Brian, and Charlene tie for the 5th open seat. Suppose Ada wins the virtual 3-sided coin toss.
But Ada would prefer that Brian or Charlene get the seat. Fine! Ada can resign immediately, and what remains of the SC can pick anyone else they like (although one would hope they pick either Brian or Charlene).
From PEP 13:
Council members may resign their position at any time.
Whenever there is a vacancy during the regular council term, the council may vote to appoint a replacement to serve out the rest of the term.
Which hints at deeper things in play: cardinal and ordinal systems have different weaknesses, and it’s supremely intentional that STAR combines some of each. It’s not so much that it’s trying to combine the best of both than that t’s trying to use each to counter the worst of the other.
For example, a weakness of cardinal systems is that they’ve vulnerable to distortion via “strategic” (insincere) voters, who exaggerate their honest scores to try to game the system. But, as you observed, the magnitude of scores is irrelevant in the runoff stage. If you don’t express your true preferences in the scoring stage, it can backfire on you in the runoff stage.
The updatable ballots feature is live! I appreciate your patience. It can be enabled as an additional option under the extra settings page.
We’ve run our own testing and internal mock elections verifying that everything is working properly, but if you notice anything while using the feature be sure to let me know .
Great! @EWDurbin, please add this new item to your checklist:
Check the “Allow Voters To Edit Vote” box on the extra settings page.
@ArendPeter, I was surprised to see that, when I do change a ballot, it starts over from scratch, with a blank ballot. I expected it would show me the state as of the last time I submitted a ballot.
When people change a ballot, it’s not typically because their view of the world has massively changed . It’s more that their opinion of a single candidate (or at worst a few) has gotten higher or lower due to some recent action.
Minor: when I created a test election, I got an error box when trying to cast a test ballot before finalizing the election. It complained that it couldn’t find a previous ballot for me.
ah yes, this was a deliberate decision after much discussion. Showing the previous ballot is better UX, but it also means that you can reveal someone’s ballot by only knowing their voter id.
Voter ID is how users are authenticated to vote and it’s usually only available to the voter through the unique voting link in their email, but there’s also a break-glass flow for the admin to reveal the link in case the voter is having trouble with their email. If we show the voter their previous ballot, then an admin could if theory also see how they voted using that same break-glass flow, however currently it’s impossible for an admin to see how someone voted.
We may change our mind in the future, but we went with the conservative approach for now.
Thank you! We’ve recreated this and we’re working on a fix.
I don’t need to understand this - but I’d like to .
After I vote, I get an email receipt, containing two links:
“You can verify your ballot and ballot status at any time.”
That link appears to contain a ballot ID (which I’ll call a BID here).
“While the election is still open, you can update your ballot”
And that link appears to contain a voter ID (VID).
If I’m following you, so long as an election is open, there is a way to connect email addresses to VIDs, although only possible for the people running the software. But there is no persistent mapping between VIDs and BIDs. That connection exists only for so long as it takes the software to create the voter’s email receipt.
If so, then two things come to mind:
Couldn’t the receipt’s “update the ballot” link contain both the VID and the BID? The VID would be used to authenticate the voter, and the BID to retrieve the ballot they cast. We already know that’s possible, because showing the ballot is what the “verify the ballot” link does.
If there is no persistent connection on your end between VID and BID, when the voter uses the “update the ballot” link to cast a new ballot, how can you know which older ballot to throw out?
Revised guesses, based on playing with test elections:
Because the ballot ID (“BID”) doesn’t change after revising a vote, there is a persistent record mapping VID (voter ID) to BID. But there’s nothing in the admin UI that exposes it.
I found the “break glass” part of the admin UI the allows the admin to discover the VID attached to an email address. But indeed nothing beyond that to discover the associated BID.
It still seems possible to me to create a voting receipt that encodes both the VID and the BID into a single link, so that the existing ballot could be shown to the voter when/if they want to change their vote, and without exposing any of that to the election admin (it would only exist in the email sent to the voter, just as the BID is currently sent in the email without exposing it to the election admin). But that would certainly require new code to create and process.
So, in short, I don’t see a reason for why the current state of the ballot couldn’t be shown to a voter who wants to revise their vote - that doing so would require new code, but wouldn’t require exposing anything to the election admin that they don’t already have easy access to.
I agree it’s good that the admin can’t tell how users voted, and I appreciate that you all are worrying about this kind of stuff .
Your revised guess is spot on, but I’ll post my original response anyway since I typed it
After I vote, I get an email receipt, containing two links:
“You can verify your ballot and ballot status at any time.”
That link appears to contain a ballot ID (which I’ll call a BID here).
“While the election is still open, you can update your ballot”
And that link appears to contain a voter ID (VID).
If I’m following you, so long as an election is open, there is a way to connect email addresses to VIDs, although only possible for the people running the software…
Yep, all correct so far.
But there is no persistent mapping between VIDs and BIDs. That connection exists only for so long as it takes the software to create the voter’s email receipt.
Almost. The database DOES have a persistent mapping between VID and BID for ballot editing purposes, but there’s no way for the election admin to find that connection.
Couldn’t the receipt’s “update the ballot” link contain both the VID and the BID? The VID would be used to authenticate the voter, and the BID to retrieve the ballot they cast. We already know that’s possible, because showing the ballot is what the “verify the ballot” link does.
This approach was also discussed, and we may opt for this in the future. Currently it’s nice that the user can update the ballot using any vote link (whether it’s their original vote link, or the one in the ballot receipt). If we updated it to require VID and BID for ballot updates, then the original link (with only the VID) wouldn’t work. We want to minimize the amount of support tickets for voters trying to figure out how the edit their ballot.
Perhaps we could update this to generate the VID + BID mapping before the voter casts a vote, and then both vote links could have the mapping? Then again we’d still want links generated by the election admin to work without revealing the mapping. Perhaps the first vote can be cast using only the VID, or VID+BID? But then all email links include VID+BID, and ballot updates require VID+BID?
Lot’s to think about
I agree it’s good that the admin can’t tell how users voted, and I appreciate that you all are worrying about this kind of stuff .
@ArendPeter, I submit that most-desirable behavior would be clearer if we stepped back from a developer-centric view of the world .
For example, you think of changing a ballot as “updating” or “editing” because the implementation has at most one ballot object per voter, which persists across updates/edits.
But that’s not what it feels like to a voter: to them, “update/edit” means casting an entirely new ballot, because nothing they did before is visible to them. Developer-speak doesn’t make sense to them, based on all they can see.
On general principle, I suppose I’d create a voter object and a ballot object at the very start, for each email address in the list. So code could assume both are always available, without fiddly (and so prone to error) conditional logic propagating to figure out which one(s) are available.
How and when to reveal all that to the voter then writes itself .
At worst (I don’t think it would come to this). different links could be explained with different words, like
“Click this to create a new ballot” (a lie-to-children, to avoid saying “we only have a VID for you but not a BID - it’s not really a new ballot, but its content will entirely replace whatever ballot (if any) you already cast)”.
“Click this to edit your existing ballot”.
I expect it would be better to have a single combined VID+BID link from the start, and each voter starts life with a blank ballot. Then all voter operations are just “edits” to the voter’s eyes.
This breaks down if the admin has to “break the glass” to give a user a VID-only link, but that doesn’t bother me. That has to be rare, right? A voter can’t really expect to have an ideal experience with an email-based voting system if they don’t supply a functioning email address.
Words for user consumption take care. Another: if a voter happens to give every candidate the same number of stars, a message tells them that’s an “abstention”.
But that word doesn’t make sense to voters. They didn’t abstain (unless they left the ballot entirely blank) - they voted! Perhaps they gave everyone 5 stars because they really believe they’re all very qualified - or perhaps everyone 0 stars because they believe nobody running is qualified at all. Either way, that’s valuable information for the organization running the election to get - “we’re attracting a lot of great candidates” or “we’re attracting nothing but losers” .
Now you (& I) may think of it as an abstention because their ballot has no effect on the outcome in such cases, the same in that narrower respect as if they hadn’t voted at all. But then better to say so plainly than fall back on jargon:
“Caution: the ballot gives all candidates the same rating, so will have no effect on the outcome.”
To be clear, while this is fun to think about, and I hope will eventually add real value, this is not a pressing concern for the upcoming Steering Council election. Far as I’m concerned, we’re good to go on that .
@ArendPeter, this appears to be a buglet of sorts. If I create a test election, and cast some all-the-same-rank ballots, they really are treated as abstentions - partly. The Voters page shows that those voters did vote (which I expect), but they’re apparently not counted as having voted in the “N voters” line of the results page when the election is closed. Which I didn’t expect - while their votes didn’t affect the outcome, they did vote. And while their ballots appear in the downloaded CSV, the stars they gave are not counted in any of the details given on the results page.
It’s conceivable this could have consequences, beyond just giving a distorted picture of actual support. Most serious I can think of is that the “N voters” result line may be used by an organization to decide whether participation reached a minimum level for the election to count.
So I think it would be a real improvement to make the code less “clever” about this .It’s fine to caution people about that expressing no preferences has no effect on who wins, but I don’t see a case for special-casing such ballots beyond just giving the voter an informative message. They “should be” scored and reported on the same way as all other ballots.
This is nerd city, but an interesting puzzle for those so inclined.
As a last resort, automated voting systems break ties “at random”. This is typically done (in both academic papers and in software) by creating a random permutation of the candidates, and picking the earliest of the tied candidates in that permuted list.
But for reproducibility, you want the same permutation every time a given collection of ballots is scored, and across implementations in various languages. BetterVoting currently uses a PRNG relying on the vagaries of exactly what the version of JavaScript in use happens to return when evaluating the sine(!) of small integers. Not high quality, but probably good enough for the purpose. Portability is out the window, though.
Behind the scenes, I created a “portable” good-quality small PRNG sticking strictly to operations on ints with fewer than 54 bits, which runs fine even under Python 2, and for which a JavaScript version has already been contributed:
But that leaves puzzles:
How to seed it in a portable, reproducible, and “fair” way?
How to ensure all implementations start with the same list to be shuffled?
Those have yet to be solved. I floated the idea of using a NIST “randomness beacon”:
which gives crypro-strength guarantees of fairness and that the seed wasn’t manipulated.
But that’s a pile of fiddly code nobody has had the heart to write in any language yet . Plus it adds reliance on an external service that may go away. Plus it feels like gross overkill in context.
@larry has a very different approach in his “starvote” library:
That works by deriving a cryptographic hash from the ballots themselves, and using it to seed Python’s Mersenne Twister PRNG. It appears equally impossible to manipulate, but is quite Python-specific, and is quite RAM-hungry if there are many ballots (think upwards of tens of thousands here).
So in the background I’ve been thinking instead of “what’s the simplest thing that could possibly work?”.
Here’s my current thinking. It’s time- and space- efficient, and very short and easy to code in Python. I believe (but don’t know) that the “hard parts” are also easily gotten at from JavaScript - or any other modern language.
The idea is not to “shuffle” at all (so no PRNG dependence), but instead use “crypto-strength” keys to sort a list of candidate names (Unicode strings).
A key is the SHA-512 hash of
the name encoded in utf-8. catenated
with that candidate's total score (number of stars across all ballots)
as a 4-byte little-endian int
This is the entire code needed in Python, where score is a dict mapping a candidate’s name to their total score:
The chances of key collision are too small to fret over. Really, the universe will end before that happens .
To try to manipulate it, you have to know in advance not just the candidate names, but also their total scores across all ballots. But you can’t know that. Get anything off by even one, and you’ll get an unrelated permutation.
Other high-order bits:
It doesn’t care what order the ballots appear in.
Or the order of candidates within a ballot.
It’s wholly deterministic: give it a score dict, and you get the same output every time.
It’s not at all resistant against things “crypto geeks” care about, but that don’t matter here. For example, it’s dead easy to contrive many distinct collections of ballots that result in the same score dict, and so also the same permutation, but who cares? That’s not a relevant “attack”.
A relevant attack would be a way for coordinated voters, and/or election admins, to increase the odds of creating a permutation in which their favored candidates’ names appear early.
Best I can think of along those lines is for the election admin to add “sock puppet” email addresses to the voter list, and cast votes using those accounts last-minute to manipulate the score dict. Still poke-and-hope, though.
Anyone see a reasonable way to do better? Alas, the only wholly un-manipulable “even in theory” ways I know of require relying on things not even the people with root access to the election software’s servers can influence. Hence “NIST randomness beacon” (and, @ArendPeter, I hope the above is more palatable to you too than that is).
I’m happy to settle for trusting the election admin not to cheat. In which respect, I’m apparently more trusting than the BetterVoting folks .
Big fan of the mental image of an election admin that comes up with a plan to very carefully stuff the ballot box with handcrafted fake votes to manipulate the hash values to change the winner of a tie-breaker… But doesn’t realize they could just stuff one more fake ballot and win directly
Except they can’t realize that in advance . While the election is open, the admin has no access to the ballots (nobody does, except for people running the hardware, and the internals of BetterVoting’s software), neither directly nor in any form of summary. In particular, they have no idea what the score dict contains before the election ends, and the results are made available to the public.
Which suggests a different form of attack that appears impossible to stop “in theory” no matter what we do:
An admin can temporarily close the election and make the results public.
Then they (anyone) can see all the ballots (although nothing about who voted for what).
Then make the results private again and re-open the election. Nobody (but the admin) will know this happened.
Then they can add any number of new sock puppet voters, and change the then-current outcome to anything they like.
And there are variations on the last point. For example, even if the service disallowed adding new voters after an election first starts, the admin could add a bunch of sock puppets before it starts, and hold off on casting ballots until they’re “needed” to sway the outcome.
This all boils down to that we trust the election admin to supply a valid list of voter email addresses to begin with. There is no external, independent verification of that possible now. And, for privacy, we can’t make that list of addresses publicly discoverable.
We could make a crypto hash of the addresses available, and pester BetterVotiing to disclose that hash value in the results. I’d rather not bother, and bet they wouldn’t either . In part because we’d still have no way to verify that the hash didn’t also include sock puppet addresses.
One part that can be verified: when the election ends, you can download the anonymized ballots. That info contains the “ballot id” (BID) of each ballot. You can know your own BID by picking apart the URL in the voting receipt email’ed to you(*), and verify that the ballot with that BID in the download matches what you voted for.
I’ll be running @larry’s library on that download to verify that his results match what BetterVoting reports, So, if you trust me and Larry, you can be sure that your ballot was counted.
Although nothing is actually simple: Larry’s library no longer works with the current format of BetterVoting’s CSV downloads. I have to parse those myself before invoking Larry’s code (easy with Python’s csv module, but necessary). And none of Larry’s tiebreaker methods match BetterVoting’s current randomization. I have to override Larry’s code for that. And just yesterday I discovered that BetterVoting doesn’t include ballots that give all candidates the same number of stars in its tallying (which I consider to be “a bug”, but they’re nothing special to Larry’s code). So I may have to worm around that too.
(*) How to do that: the receipt contains a line saying " You can verify your ballot and ballot status at any time"
Which links to a URL like:
https://bettervoting.com/4xh9p2/ballot/b-yd8vtr2x
Your BID is the trailing part, 'b-yd8vtr2x" in that URL.
Sorry, that’s an extended hallucination. Since the admin has no access to the score dict, they’re flying blind too until the results are made public. Although, as in the last msg, it appears there’s nothing to stop them from temporarily closing the election and making the results public, just long enough for them to download the current (anonymized) ballots. Then they know everything relevant at the time.
.
.
.
.
. Plus it adds reliance on an external service that may go away. Plus it feels like gross overkill in context.