Benchmarks
Data quality is three separate questions and they fail independently. Status answers whether a county endpoint is alive. This page answers whether the answer it gave was right. Legality, meaning whether we are permitted to serve a given field for a given county, is tracked in the atlas and is not a number. We do not collapse the three into one green checkmark. The measurements below are published per metric, with the sample size on every one and a list of what each could not see.
146 reference parcels across 7 counties. 146 rows scored across 7 counties. Measured 2026-08-24.
This snapshot is refreshed by hand. A weekly job and a staleness check that fails when this date falls more than ten days behind are both written and merged, and neither runs until the account's monthly CI allowance resets on 1 September 2026. Until then the honest description of the date above is that a person produced it.
7 of 151 monitored counties have enough reference data to produce a correctness number. The other 144 have none, and this page says nothing about them. 0 rows are excluded from every accuracy metric because the only source that established their parcel identifier was the endpoint they would have graded.
What was measured
The share where the pipeline named a different parcel than the roll does. Kept apart from accuracy because declining to answer and answering wrong are not the same failure: one costs a caller a lookup, the other sends a legal notice to a stranger.
What it cannot see: Cases where the pipeline returned nothing. Those count against accuracy below, not here.
The share of reference addresses where the pipeline named the same parcel the county's own assessment roll names. This is the number the product is selling.
What it cannot see: Whether the county's record is itself correct. A row is graded against what an independent county source publishes, so this measures agreement with the county, not truth about the world.
The share where the correct parcel was anywhere in the 30 metre buffered candidate set, before disambiguation picked one. The gap between this and accuracy is what disambiguation loses. The gap between this and 100 percent is what the buffer never reached.
What it cannot see: Why a parcel was missed. A geocode landing on the wrong block and a parcel layer with a genuine hole look identical from here.
The share of addresses the Census geocoder placed at all. Census is the only geocoder in this pipeline, so an address it cannot place cannot be looked up at any accuracy.
What it cannot see: Whether the point it returned is on the right parcel. Census interpolates along a street centreline, so a successful geocode still lands in the road. Address agreement below is the check on that.
When the pipeline did name a parcel, the share where that parcel's site address agrees with the address Census matched, scored with the same house number and street token test that gates a billed answer.
What it cannot see: Anything about the rows where no parcel was named, which are excluded rather than counted as disagreement. The denominator here is smaller than the others and is labelled with what it counts.
Excluded: 62 no parcel was named, so there was no site address to compare
Of every parcel the buffered query returned, the share that was a neighbour rather than the one being looked for. The denominator counts candidates, not addresses: a 30 metre buffer on a city block pulls in several neighbours and only one of them is right, so this number is high by design and says how much sorting disambiguation has to do. It is not an error rate; the wrong parcel rate above is.
What it cannot see: Anything about the pipeline being wrong. A layer that returns more candidates is doing more work, not worse work, and a high number here with a zero wrong parcel rate means the sorting is doing its job.
Every match needed normalizing
Of the 84 correct answers, 38 matched the reference identifier exactly and 46 matched only after punctuation was stripped. Cook County's assessment roll writes a PIN as fourteen digits and the GIS layer writes the same PIN with dashes, so a plain string comparison would have scored every correct answer as wrong. Normalizing is the right call, but it is a heuristic, so a match that needed it is counted here rather than quietly promoted.
By county
| County | State | Accuracy | Recall | Scored rows | Declined | Confidence |
|---|---|---|---|---|---|---|
| Denver County | Colorado | 86% | 86% | 14 of 14 | 2 | reliable |
| Cook County | Illinois | 63% | 66% | 73 of 73 | 27 | reliable |
| Hennepin County | Minnesota | 69% | 69% | 13 of 13 | 4 | reliable |
| Wake County | North Carolina | 46% | 54% | 13 of 13 | 7 | reliable |
| Franklin County | Ohio | 55% | 64% | 11 of 11 | 5 | reliable |
| Travis County | Texas | 63% | 88% | 8 of 8 | 3 | limited |
| King County | Washington | 0% | 29% | 14 of 14 | 14 | reliable |
Pooled numbers above sum numerators and denominators across counties, so a county with more reference rows counts for more. Averaging the county percentages instead would let a thin county swing the headline.
That weighting is kept, so here is what it comes to: Cook County contributes 73 of the 146 scored rows, which makes every pooled figure above substantially that county's number until the thinner counties gain reference data.
The declined column is not a softer way of saying wrong. A declined row returned no parcel, so the caller got nothing rather than somebody else's land. Of the 62 declined rows, 14 are counties whose parcel layer publishes no site address at all. The other 48 have an address field, which is not the same thing as a usable address: one county's holds a house number with no street name, so a candidate can only be checked on the number. 61 of all 62 returned more than one candidate parcel, and that is the case this pipeline refuses on purpose, because naming one owner in a building of many without a unit number is a guess wearing a confident face.
That county is the reason this run reports a second kind of match. Where the layer publishes a number and no street, the pipeline resolves the row only if exactly one parcel in the buffer carries that number, and records the result as house-only rather than exact, because a house number that is unique among neighbours is weaker evidence than an address that matched in full. 5 of the named parcels rest on that basis. Where two parcels inside the buffer share the number, which happens on corner lots and split parcels, the row still declines: the alternative is a coin flip on whose name gets published.
What this page cannot yet tell you is how that remainder splits between nothing matching and several things matching equally well. The recorded observations carry which parcels were candidates but not their addresses, so the distinction is not recoverable after the run. Recording candidate addresses is the next thing to instrument, and until that is done this paragraph is the honest limit of what the evidence supports.
What this page does not measure
- Owner name correctness. Entirely unmeasured. The reference file holds no owner names: this product keeps no owner database, and a committed file of names in a public repository would be one. So the field most likely to be wrong in a way that matters is the field with no number next to it.
- Latency, and what this number is not. Over the 146 rows the layer answered in the recorded run, it took 130 ms at the median and 2420 ms at the 95th percentile, with a slowest answer of 4413 ms. Percentiles are nearest rank, so each figure is a duration that was measured rather than an average of two that were. The figures cover the parcel query alone: the Census geocode that precedes it is excluded, and so is everything this site does after the answer arrives. They were measured from one machine at one moment, so read them as evidence that these endpoints respond in this range, not as a service level objective and not as a percentile over user traffic. The liveness prober still does not persist per request timing, so the status page publishes no latency of its own.
- 144 of 151 monitored counties. They have no reference parcels, so they have no correctness number. Their presence on the status page means their endpoint answers, and nothing more than that.
- The hardest addresses. A parcel is only admitted as reference data when its address maps to exactly one parcel in the source. Condo stacks and split parcels, which are where address to parcel resolution breaks, are absent from the sample by construction.
- Whether the county is right. Every row is graded against what an independent county source publishes. If the county's own roll is wrong, this page will score us as correct for reproducing it.
How this is measured, and how to check it
Each reference row records the address, the parcel identifier a named independent source publishes for it, a URL that returns that identifier, who verified it and when, and the exact layer the row grades. A run puts every row through the same path a paying caller gets: geocode the address with Census, query the county layer with a 30 metre buffer, classify the layer's fields, and disambiguate by site address. The run records what came back and scores nothing. Scoring is a separate pure module with unit tests, so the published numbers are arithmetic over evidence that ships next to them.
Both files are readable: ground-truth.json holds the reference parcels and observations.json holds what the last run saw. Every observation carries a fingerprint of the row it measured; if a row is edited afterwards, the observation is dropped as stale rather than scored, and the build fails until the run is repeated.
There is no single correctness score. Which of these numbers is weak is the useful information, and one blended figure is exactly the shape that hides it.
Endpoint liveness, which is a different question with a different answer, is on the status page. Whether an AI agent handed this data picks the right tool for a question, and avoids the paid ones when a free one answers, is a third question again and is on the agent evals page. The commitment behind all three is on the data availability page.