Detecting election fraud: is it possible to identify manipulated vote counts?

In the quest to tackle electoral malpractice, election observers and political scientists have sought ways to identify when vote counts have been manipulated. One proposed method is to analyse non-random patterns in the last digits of vote counts. Verena Mack and Lukas F. Stoetzer tested this technique and found that it cannot reliably identify fraud, but that such experiments are a useful way to build up a set of potential identifiers of vote manipulation.

Photo by dylan nolte on Unsplash

Election fraud is a widespread phenomenon. In many elections held over the world, political actors are suspected of illegally interfering in the election process to tilt the result in their direction. This includes recent cases in the Democratic Republic of Congo’s presidential elections and the Russian presidential elections of 2018, but also cases in the US. For firm believers in democracy this is troublesome, and such incidents contribute to a public loss of trust in the democratic process.

The prevention of election fraud goes hand in hand with the detection of it. When a controversial election is expected, international organisations often send independent observers to assess the conduct of an election process. While monitoring does not directly prevent fraud, it helps to document misconduct measured against national and international election standards. With those facts at hand, the promise is that political actors can be held accountable, helping to keep elections clean by preventing fraudulent activities because of the anticipated costs of manipulation.

What can political scientists add to the detection of election fraud? They can conduct systematic analysis of information partially provided by the election observers after an election. And in fact, political scientists have developed an arsenal of forensic methods to detect election fraud. A particularly promising fraud-detection method is based on the unusual distribution of digits in vote-count data (see e.g. Mebane 2008, Beber and Scacco 2012, Medzihorsky 2015). The idea is simple: the digits on a return sheet that has been manipulated by humans look different to last digits in a ‘normal election’. Why is that? In an influential paper, Beber and Scacco (2012) argue that the inability of human counterfeiters to produce random-looking numbers (watch this video), means the last digit will deviate from a distribution, where all digits appear at the same rate. The method is promising as it requires relatively little to detect manipulations. The returned vote sheets are enough to detect interference in the election result. Digit-based election forensic methods have been applied to several elections, including Nigeria’s 20013 presidential elections, the Afghanistan election of 2009, the 2008 election in Azerbaijan, Swiss referendums and Russian elections.

This sounds very promising but unfortunately it is generally difficult to evaluate if election forensic methods work. To make sure that the last digit is accurate in detecting fraudulent interferences, we tested this in a laboratory experiment, presented in our recent paper. We invited participants at LakeLab in Konstanz to manipulate fraud-free election results from the 2008 Canadian federal election in favour of a specific party. 104 participants received the results from 90 polling stations and we asked them to conduct marginal fraud (such that the party wins with up to 5% points overall), considerable fraud (10–20%) or extreme fraud (40–50%). To achieve these results, participants were able to replace each of the vote counts with a new number. In addition, we provided them with financial incentives to fake the sheets inconspicuously. The laboratory experiment helps us to evaluate the last-digit test under different intensities of fraud: we can track participants’ manipulation and therefore identify under which conditions the test can detect misconduct.

Overall it was difficult to detect the fraudulent behaviour of the participants in our laboratory experiment using the last-digit test. Across all conditions, only 21% of manipulations were detected. This indicates that the test is not very sensitive in detecting fraudulent activities, even in an ideal setting in which one person has the power to fake the final results. When participants conducted marginal fraud we were only able to find irregularities in one of the 26 cases. Even though participants are truly not able to come up with random numbers for the last digits, especially shying away from using zeros and fives, the overall patterns that emerge from the process are still not unusual enough to make a clear case for manipulations. Only if manipulators exchange a large share of last digits, could the test identify their manipulations.

Part of this has to do with the variety of methods participants used to falsify the election sheets. First, participants did not manipulate all polling stations but focused solely on a few important results to change the outcome in their favour. It was, for example, possible to change the results from just one polling station and make the party win with a small margin. This level of change does not result in unusual patterns in the last digits. Second, participants often only altered the leading digit, leaving the last digit untouched. This holds especially true when we asked respondents to conduct extreme fraud and makes it more difficult for them to find irregularities in this condition than in the marginal fraud condition. Third, participants used a simple trick to reach their goal. They swapped the votes of the winning party with their party, to increase their vote share. This strategy leaves no traces in the distribution of digits, making it impossible for the test to detect any irregularities.

In our experiment, the last-digit test was not powerful enough to detect irregularities. What does this mean for the detection of election fraud in general? It’s most likely that no ideal method for fraud detection exists, but we can keep on trying to develop advanced methods. Our research, first of all, contributes to this development. The results, for example, show that it is implausible to treat all election return sheets as being manipulated. Recently, Medzihorsky (2015) developed an extension of the last-digit test that takes this consideration into account. Second, the results point out that different indicators of election fraud should be combined to build an election forensic toolkit. For example, Rozenas (2017) shows that spikes in vote share distribution can give an indication of fraudulent activities. It might be promising to combine different insights (see e.g. Montgomery et al., 2015, Cantú and Saiegh). Third, our paper suggests laboratory experiments can guide the development of powerful detection methods.

The post gives the views of its authors, not the position of Democratic Audit. It draws on the authors’ article, ‘Election fraud, digit tests and how humans fabricate vote counts – An experimental approach’, published in Electoral Studies.

About the authors

Verena Mack works at the Stiftung Risiko-Dialog in Winterthur, Switzerland. She graduated from the University of Konstanz with a dissertation on “The Fingerprints of Fraud: An In-depth Study of Election Forensics with Digit Tests”. Her research on election forensics has been published in the Jahrbücher für Nationalökonomie und Statistik.

Lukas F. Stoetzer is Post-Doc at the Department of Political Science at the University of Zurich. He graduated from University of Mannheim at the Graduate School of Economic & Social Sciences in 2015. His research focuses on comparative political behaviour, public opinion as well as political methodology. His articles have been published among others in Political Analysis, Journal of Politics, British Journal of Political Science, Electoral Studies, and the European Journal of Political Research.