Published on June 1, 2023

Application of Benford's Law to United States Election Results (2020)

A Statistical Approach to Assessing the Integrity of the Voting Process

Table of Contents

• Authors and Affiliation
• Table of Contents
• Abstract
• Introduction
• Methods
• Functions
• Inverse Cumulative Distribution Function (ICDF)
• Chi-Squared Test
• Results
• Discussion
• Significance Level α = 0.05
• Electoral Votes
• Outliers
• Breakdown by County
• Wisconsin
• Virginia
• Iowa
• Nevada
• Vermont
• Arkansas
• Missouri
• Nebraska
• Works Cited

Abstract

This research paper applies Benford's Law to the 2020 US election results at the state levels, with references to county results. Using a dataset obtained from the Election Administration and Voting Survey (EAVS) Datasets, Codebooks, and Survey Instruments, we analyze the first digit frequencies of the vote counts using the programming language C# and the MathNet.Numerics code library for statistical analysis. We compare the observed frequencies to the expected frequencies predicted by Benford's Law and use a chi-squared test to identify any potential irregularities in the vote count. The results of this study could help identify any potential issues with the accuracy and transparency of the electoral process and provide insights into how to ensure fair and transparent elections.

Introduction

Elections are a cornerstone of democratic societies, and their legitimacy rests on the accuracy and transparency of the vote count. However, allegations of electoral fraud are not uncommon, and can cast doubt on the integrity of the democratic process. In recent years, concerns about the accuracy and fairness of election results have been heightened by the prevalence of electronic voting systems and the potential for hacking or other forms of tampering.

One statistical tool that can be used to detect irregularities in organic numerical data is Benford's Law¹. Benford's Law, also known as the first-digit law, states that in many naturally occurring datasets, the frequency of the first digit of each number follows a predictable pattern. Specifically, the probability of the first digit being "d" is log10(1 + 1/d), where "d" is any digit from 1 to 9. This means that the first digit "1" should occur more frequently than any other digit, followed by "2", then "3", and so on, with "9" being the least frequent.

Benford's Law has been applied to a wide range of datasets, including financial statements, scientific measurements, and sports statistics. More recently, it has been used to analyze election results in various countries to assess their legitimacy. The rationale behind this approach is that if the election results are organic, then they should follow the expected pattern predicted by Benford's Law.

The purpose of this paper is to apply Benford's Law to the 2020 election results in the United States, specifically at the state levels, to determine if the results are consistent with what would be expected under normal circumstances. This study will provide insight into the reliability and accuracy of the electoral process in the United States and will contribute to ongoing discussions about the importance of ensuring fair and transparent elections.

Methods

The dataset used in this research was obtained from the Election Administration and Voting Survey (EAVS) Datasets, Codebooks, and Survey Instruments², and included the vote counts from the 2020 elections at the state and county levels in the United States. The data was provided in a structured format, including variables such as the number of registered voters, the number of votes cast, with breakdowns for absentee, in-person, and other voting methods.

To process the data and apply Benford's Law calculations, we used the programming language C#. C# is a widely used language for data analysis, and it offers libraries and tools that allow for easy data manipulation and statistical analysis.

We used the MathNet.Numerics code library to calculate the expected and observed frequencies for each digit from 1 to 9. The library provides a suite of mathematical tools and functions, including chi-squared and critical chi-squared tests, which are necessary for Benford analysis.

The first step in the analysis was to extract the first digit of each vote count and calculate the observed frequency of occurrence for each digit. We then compared the observed frequency of the first digit to the expected frequency predicted by Benford's Law using the MathNet.Numerics library. The expected frequency was calculated using the formula log10(1 + 1/d), where "d" is the digit from 1 to 9, or 8 degrees of freedom.

To determine if the observed frequencies were consistent with the expected frequencies, we used a chi-squared test. The chi-squared test is a statistical test that measures the degree of deviation between the observed and expected frequencies. The critical chi-squared value is the threshold value that is compared to the calculated chi-squared value. If the calculated chi-squared value exceeds the critical value, it suggests a significant deviation from the expected distribution and could indicate potential irregularities in the vote count.

In statistical hypothesis testing, the significance level, often denoted as α (alpha), is the probability threshold used to determine whether to accept or reject a null hypothesis. It represents the maximum acceptable probability of observing a result as extreme as, or more extreme than, the one observed, assuming that the null hypothesis is true.

Regarding Benford's Law, significance level is a mathematical principle that describes the expected distribution of leading digits in many real-world datasets. According to Benford's Law, the leading digits 1 to 9 should appear with specific probabilities in naturally occurring datasets, where the digit 1 should occur most frequently (around 30.1% of the time) and the digit 9 should occur least frequently (around 4.6% of the time).

To apply Benford's Law to a dataset, one common approach is to compare the observed distribution of leading digits in the dataset with the expected distribution based on Benford's Law. Statistical hypothesis testing can be used to determine if the dataset conforms to Benford's Law or if there is a significant deviation.
Here's where the significance level comes into play. When testing the conformity to Benford's Law, we establish a null hypothesis and an alternative hypothesis:

• Null hypothesis (H0): The dataset conforms to Benford's Law.
• Alternative hypothesis (Ha): The dataset does not conform to Benford's Law.

To determine whether to accept or reject the null hypothesis, we perform a statistical test, such as the chi-square test or the Kolmogorov-Smirnov³ test, using the observed and expected digit frequencies.

The significance level is then used to set a threshold below which one rejects the null hypothesis. For example, if one chooses a significance level of α = 0.05, one is saying that if the probability of observing a result as extreme as, or more extreme than, the one observed is less than 0.05 (5%), then reject the null hypothesis. In other words, one would consider the deviation from Benford's Law to be statistically significant.

For the purposes of this study, we utilized a significance level of α = 0.05.

Overall, using the MathNet.Numerics⁴ code library and C# programming language allowed us to perform a rigorous analysis of the 2020 election results using Benford's Law. The results of this study could help identify any potential issues with the accuracy and transparency of the electoral process and provide insights into how to ensure fair and transparent elections.

Functions

The following Benford functions were implemented (C#/Visual Studio) to calculate the required values when reading the EAVS dataset.

Inverse Cumulative Distribution Function (ICDF)

A quantile function, also known as a percent point function or inverse cumulative distribution function⁵ (ICDF), is a mathematical function that gives the value below which a certain proportion of the probability distribution falls. In other words, it provides the value at a specific percentile or quantile of a distribution.

For a given probability p, the quantile function returns the corresponding value x such that the cumulative probability up to x is p. It is the inverse of the cumulative distribution function (CDF), which gives the probability that a random variable takes on a value less than or equal to a given value.

The quantile function is widely used in statistics and probability theory for various purposes, such as estimating percentiles, constructing confidence intervals, and generating random samples from a specific distribution.

using System;
using System.Collections.Generic;
using System.Linq;
using MathNet.Numerics.Distributions;

public static class Benford
{
    public static Dictionary<int, double> GetExpectedFrequencies()
    {
        var expected = new Dictionary<int, double>();

        for (int i = 1; i <= 9; i++)
        {
            expected[i] = Math.Log10(1 + 1.0 / i);
        }

        return expected;
    }

    public static Dictionary<int, int> GetActualFrequencies(IEnumerable<int> data)
    {
        var actual = new Dictionary<int, int>();

        foreach (int d in data)
        {
            if (d > 0) // Don't agg negative values
            {
                if (int.TryParse(d.ToString()[0].ToString(), out int firstDigit))
                {
                    if (actual.TryGetValue(firstDigit, out int value))
                    {
                        value++;
                    }
                    else
                    {
                        actual[firstDigit] = 1;
                    }
                }
            }
        }

        return actual;
    }

    public static double CalculateCriticalChiSquared(double significanceLevel = 0.05, int degreesOfFreedom = 8)
    {
        // Calculate the critical chi-squared value using the inverse cumulative distribution function (ICDF)
        var chiSquaredDist = new ChiSquared(degreesOfFreedom);
        return chiSquaredDist.InverseCumulativeDistribution(1 - significanceLevel);
    }

    public static double CalculateChiSquared(IEnumerable<int> data)
    {
        var expected = GetExpectedFrequencies();
        var actual = GetActualFrequencies(data);
        double chiSquared = 0;
        int sampleSize = data.Count();

        foreach (int digit in expected.Keys)
        {
            double expectedCount = expected[digit] * sampleSize;
            int actualCount = actual.ContainsKey(digit) ? actual[digit] : 0;
            double diff = actualCount - expectedCount;
            chiSquared += diff * diff / expectedCount;
        }

        return chiSquared;
    }
}

Chi-Squared Test

In the context of Benford's Law and statistical analysis, the Chi-squared test⁶ is a statistical test used to assess the goodness-of-fit⁷ between an observed set of data and the expected values based on Benford's Law. It allows us to determine if the observed data significantly deviate from the expected distribution.

To apply the Chi-squared test to Benford's Law, we compare the observed frequencies of the first digits in a dataset to the expected frequencies predicted by Benford's Law. The test calculates a Chi-squared statistic, which measures the discrepancy between the observed and expected frequencies. If the Chi-squared statistic is sufficiently large, it suggests that there may be a significant departure from Benford's Law.

Results

Discussion

Significance Level α = 0.05

Of the fifty-five (55) sampled U.S. states (including District of Columbia, Guam, Northern Mariana Islands, Puerto Rico, and the U.S. Virgin Islands), the passing percentile (the percentile within the significance level of Benford’s Law) was 85.45%.
Eight (8) U.S. states were above the calculated critical Chi Squared value of 15.51. Ordering the highest to lowest variance:

1. Wisconsin (43.3) [Democrat won, 0.63% margin]⁸
2. Virginia (20.26) [Democrat won, 10.11% margin]⁹
3. Iowa (18.03) [Republican won, 8.2% margin]¹⁰
4. Nevada (17.45) [Democrat won, 2.39% margin]¹¹
5. Vermont (16.97) [Democrat won, 35.42% margin]¹²
6. Arkansas (16.94) [Republican won, 27.62% margin]¹³
7. Missouri (16.14) [Republican won, 15.39% margin]¹⁴
8. Nebraska (15.71) [Republican won, 19.15% margin]¹⁵

Electoral Votes

1. Wisconsin - 10 (Democrat won)
2. Virginia - 13 (Democrat won)
3. Iowa - 6 (Republican won)
4. Nevada - 6 (Democrat won)
5. Vermont - 3 (Democrat won)
6. Arkansas - 6 (Republican won)
7. Missouri - 10 (Republican won)
8. Nebraska - 4 (Republican won) /1 (Democrat)

Outliers

In statistics, the treatment of outliers is a topic of debate and there is no consensus on a universally accepted approach. The decision to remove or handle outliers depends on various factors, such as the nature of the data, the research question, and the specific analysis being conducted.
Some statisticians argue that outliers should not be removed or altered without a strong justification, as they can carry valuable information or may be indicative of interesting phenomena in the data. Others suggest that outliers should be examined carefully to determine whether they are genuine extreme values or the result of data entry errors or measurement problems. If outliers are identified as erroneous data, they may be excluded from the analysis.
A common practice is to use robust statistical methods that are less influenced by outliers, such as non-parametric methods or robust regression techniques. These methods aim to provide more robust estimates and inference even in the presence of outliers.

A non-parametric method¹⁶ is a statistical technique that does not make any assumptions about the underlying probability distribution or parameters of the population from which the data are sampled. Non-parametric methods are often used when the data do not meet the assumptions of parametric methods or when the research question does not require specific assumptions about the population distribution.

Non-parametric methods are more flexible and can be applied to a wider range of data. They are based on ranks, orders, or other distribution-free properties of the data. These methods often involve permutation tests, resampling techniques (such as bootstrap), or rank-based statistics.

Non-parametric methods can be valuable in situations where the data do not adhere to assumptions of normality, have outliers, or are on ordinal or categorical scales. These methods offer robustness against violations of assumptions but may have lower statistical power compared to parametric methods when the assumptions are met.

It's worth noting that non-parametric methods do have their own assumptions and limitations, but they are generally less stringent compared to parametric methods. The choice between parametric and non-parametric methods depends on the specific research question, the nature of the data, and the assumptions that can reasonably be made.

Breakdown by County

While county metrics are available within this dataset, it’s important to note that Benford’s Law applies to sequences. Since a county metric only contains a single numeric value for total reporting, any reporting number with a significant leading digit of 9 would automatically be reported as “failed”. For this reason, it cannot be reasonably deduced that the failed counties are necessarily “fraudulent” or “suspicious”, and the reader is cautioned from coming to any conclusions regarding these data without additional due diligence. Nonetheless, the result sets are provided to show the county outliers.

Wisconsin

Virginia

Iowa

Nevada

Vermont

Arkansas

Missouri

Nebraska

Works Cited

1. Nigrini, M. J. (2012). Benford's Law: Applications for Forensic Accounting, Auditing, and Fraud Detection. Wiley.
2. Berger, A., Hill, T. P., & Hill, I. D. (2015). An Introduction to Benford's Law. Princeton University Press.
3. Diekmann, A. (2007). Not the First Digit! Using Benford's Law to Detect Fraudulent Scientific Data. Journal of Applied Statistics, 34(3), 321-329.
4. Johnson, N. L., Kotz, S., & Balakrishnan, N. (1994). Continuous Univariate Distributions (Vol. 1). Wiley.
5. Hahn, G. J., & Shapiro, S. S. (1967). Statistical Models in Engineering. Wiley.
6. Wilcox, R. R. (2017). Introduction to Robust Estimation and Hypothesis Testing (4th ed.). Academic Press.
7. Hyndman, R. J., & Fan, Y. (1996). Sample Quantiles in Statistical Packages. American Statistician, 50(4), 361-365.
8. Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
9. Barnett, V., & Lewis, T. (1994). Outliers in Statistical Data (3rd ed.). Wiley.
10. Hawkins, D. M. (1980). Identification of Outliers. Chapman & Hall.
11. Hoaglin, D. C., Mosteller, F., & Tukey, J. W. (1983). Understanding Robust and Exploratory Data Analysis. Wiley.
12. Aggarwal, C. C. (2017). Outlier Analysis (3rd ed.). Springer.
13. Wasserman, L. (2004). All of statistics: A concise course in statistical inference. Springer Science & Business Media.
14. Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis. John Wiley & Sons.
15. Rice, J. A. (2007). Mathematical statistics and data analysis. Cengage Learning.
16. Gibbons, J. D., & Chakraborti, S. (2010). Nonparametric statistical inference. CRC Press.