Colorado Secretary of State logo - cube with a C in it

Colorado Secretary
of State Jena Griswold
www.coloradosos.gov
www.sos.state.co.us

Colorado Secretary of State logo - Colorado Secretary of State Jena Griswold

Picture of Secretary of State Jena Griswold

×
Colorado Secretary of State logo - cube with a C in it

Colorado
Secretary of State
Jena Griswold

×

Risk-Limiting Audit (RLA) FAQs

Q1. What is a risk-limiting audit?

A1. A risk-limiting audit is a post-election audit that gives a statistical level of confidence that the outcome of an election is correct. In other words, after performing a risk-limiting audit, we can say that there is a high probability that the reported winners accurately reflect how voters marked their ballots. For more information, click here to read the Colorado Election Fact Sheet on Post-Election Audits Overview (PDF).

Q2. What is the risk-limit?

A2. The risk-limit tells us the probability that the audit would correct a wrong outcome. For example, an audit with a 5% risk-limit means that if an election has the incorrect reported outcome, on average 95 times out of 100 the audit would catch that incorrect outcome and correct it. If the reported outcome of the election is correct, then the audit offers a very strong level of evidence of that outcome.

Colorado began RLAs with a 9% risk-limit in comparison audits, and has reduced that number to 3% as counties got more comfortable performing the audit.

Technical information on audit can be found on our risk-limiting audit resources page.

Q3. What kind of risk-limiting audits are there?

A3. In Colorado, all counties participating in the audit conduct comparison audits. Historically, some counties with voting systems incapable of exporting a cast vote record conducted a second type of RLA called a ballot polling audit. Any county that hand counts all of their ballots instead of using a voting system to tabulate does not participate in the RLA because it is an audit of the voting system.

Q4. How does Colorado perform the audit?

A4. During a comparison audit, counties are given a list of specific ballots to examine and then replicate the voter markings in an open source software application created by the Colorado Department of State specifically to facilitate the audit. Further details about the audit can be found in the next few questions.

Q5. How are the ballots to audit chosen?

A5. Each county creates a ballot manifest that describes how many ballot cards have been counted, in terms of batches and the number of ballots in each batch. A pseudo-random number generator with a random seed, generated by rolling 20 ten-sided dice during a public meeting, is used by the audit software to randomly select ballots from the ballot manifests. Since the audit is based on statistical confidence, the number of ballots that are chosen to audit is based on the margin of the race that is targeted. The closer the margin, the more ballots are audited to confirm the outcome.

Q6. What are target contests?

A6. Since the number of ballots chosen to audit in a contest is based on the margin of victory in a race, to perform a risk-limiting audit the auditor needs to designate what races will be used to drive the audit. The Secretary of State chooses the races to target (at least one in each county and one across the whole state) by the Friday after an election. The results of the audit in these targeted contests also determine if counties must do more than one round of auditing. However, even though targeted contests drive the audit, audit boards examine and reproduce voter markings in all of the contests on the ballot and staff at the Department of State look at all choices to make sure nothing unexpected is occurring. 

Q7. What occurs during the audit?

A7. Bipartisan audit boards examine the ballots selected by the audit software and reproduce the voter markings in the software. If all of the markings in the targeted contest match between what the audit board reports and what the voting system recorded, the audit has gathered enough evidence to say with a high level of statistical confidence (based on the risk limit) that the reported outcome of the targeted race (what the voting system reported) match how the voters actually marked their ballots.

Q8. What if there isn’t a match?

A8. If there isn’t a match it’s called a discrepancy. There are two types of discrepancies: overstatements and understatements. Understatements happen when the audit board reports a vote for the reported winner of a contest that wasn’t originally tabulated for the winner. This further confirms the reported results so there is no increase in ballots needed to be examined. An overstatement happens when the audit board reports a vote for the loser of a contest that wasn’t originally tabulated for the loser. This makes the margin between the winner and loser smaller than what was originally reported, so the audit may require the county to audit additional ballots to gather more evidence. Counties may be required to continue doing additional rounds of auditing if discrepancies continue to be reported.

Q9. Why do discrepancies happen?

A9. There are many reasons that discrepancies may occur. The most common reason is that the audit board retrieved a different ballot than the one required by the software, and therefore reported the wrong voter choices. Another common reason for a discrepancy is that the audit board made a mistake when inputting the voter choices into the software. In addition to these two common discrepancy types there are other less common reasons. Each audit, staff in the Colorado Department of State examine all of the discrepancies reported during the audit, whether overstatements or understatements, or whether they are in targeted or non-targeted contests, to determine the probable cause of the discrepancy. Since the 2020 State Primary a report has been posted to the Audit Center on the Secretary of State’s website that explains the cause of each discrepancy. From audits before the 2020 State Primary all data from each audit has been posted in full as well.

There have not been any cases since Colorado began doing risk-limiting audits where a discrepancy was the product of the voting system switching voters or otherwise not working as intended.

Q10. Why do some counties have to audit more ballots in subsequent rounds if their audit boards didn’t report any discrepancies in a target contest?  

A10. There are two type of target contests: county and state. County contests are targeted contests that are wholly contained in one county—the evidence gathered for the contest is done on ballots cast within the county only. If discrepancies are reported in this kind of targeted contest that results in more ballots having to be audited, the additional ballots are only selected from ballots cast in that county. State contests are multi-jurisdictional, meaning they take place across more than one county (typically the whole state). Since the evidence of the audit is being gathered in all the participating counties together, if a discrepancy is reported in a state targeted contest that results in more ballots having to be audited, all of the counties that share that race will have a chance of having additional ballots selected to audit.

Q11. What is the difference between a ballot and a ballot card?

A11. A ballot card is a page of the ballot that has two sides. A ballot may consist of one, two, or more ballot cards. For the RLA this is an important distinction because the cast vote record file will have a separate record for each ballot card, not one record for each set of ballot cards. Therefore, the number of ballot cards can be much greater than the number of ballots cast in a county.

Q12. Why does Colorado conduct a risk-limiting audit?

A12. In 2009, the Colorado General Assembly began the process of requiring the implementation of risk-limiting audits by enacting 1-7-515, C.R.S. The first audit was delayed a few years as technology caught up to the task. In the 2017 Coordinated Election, all of the counties that use machines to tally votes performed the first statewide risk-limiting audit. Risk- limiting audits are considered by election experts to be the highest standard post-election audits.

Q13. How can I get involved with the audit?

A13. Since the audit is performed at the county level, you can contact your County Clerk and Recorder's office to get involved in the next audit. If you're interested in reports from former audits, you can go to the Secretary of State's audit center.