Engineering notes
Data quality here is three separate questions, and they fail independently. An endpoint can be alive and wrong. It can be alive, correct, and legally barred from serving the field you need. So status measures liveness, benchmarks measure correctness, and agent evals measure whether a model handed this server behaves. The operating point under all three: the pipeline declines rather than guesses, and the wrong parcel rate is published first, because naming the wrong parcel mails a legal notice to a stranger.
The county that publishes house numbers with no streets
Every Travis County reference row declined. No wrong answers, no right ones. The matcher scores a geocoded address against the county's site-address field by house number and street tokens, and Travis registers a field named "Situs Address", so this looked like a tuning problem.
The live layer said otherwise. That field holds a bare house number: the value for a house on Oak Ledge Drive is the string 8701, and no street column exists anywhere in the layer. A test that requires street tokens to agree was unsatisfiable there by construction. No threshold could fix it, because the county publishes what it publishes.
The fix is a guarded fallback. It engages only when no candidate parcel carries letters after its house number, it requires exactly one match, and it reports a distinct, weaker confidence class so a caller who needs street-level proof can refuse it. Before shipping, we ran it against the full known-answer reference set: zero wrong picks, and every adversarial row, the ones whose correct parcel was not retrievable at all, declined.
An adversarial review then caught three deeper defects. A street named entirely of generic suffix words tokenized to nothing and could open the fallback on a county that does publish streets. Two spellings of one house number slipped past the shared-number guard through raw string comparison. And the public OpenAPI schema promised confidence values that never existed in any version of the code. The benchmark caught the original failure, the review gate hardened the fix, and the weaker evidence class is published rather than promoted.
The churned customer
The first paying subscription ran its trial and cancelled without one successful billable query, because the purchase path knew quota and not capability. The fix and the full writeup are at /postmortem.
Process, stated as facts
- CI gates are proven able to fail before being trusted. We break the guarded thing, predict which assertion goes red, watch it, restore. A gate that has never gone red is a decoration. The gates live in the repo's scripts and test suite.
- Published prose that quotes data is pinned by tests, the pattern visible on /agent-evals: a re-run that changes the data turns the narrative stale loudly instead of silently.
- We caught two of our own audit errors and corrected them in the public record, commit history included, rather than quietly editing them away.
- One reference row was excluded against the incentive to grow the sample: a single address carrying two parcels in the county's own roll, which no grader can score fairly. The reasoning is recorded in ground-truth.json.
- One spelling-drift specimen is kept on purpose: the Census geocoder renders a street name singular, the county spells it plural, and the pipeline declines it honestly. The row is in ground-truth.json with the drift documented.
What is deliberately not built
- No owner database. The reference file holds no owner names, because a committed file of names would itself be the privacy problem.
- No telemetry on the free browser tools, by promise on /about. Measurement is server-side only.
- No composite scores anywhere. Which number is weak is the information.
- The application repo is private; the two npm packages on /developers are MIT. A read-only collaborator invite is available to interviewers on request.
Reading order
The declined column, and the "what it cannot see" note under every metric.
2The first churned subscription
The section titled "What this still does not fix".
The county shown as indexed, publishing data, and down, all at once.
Two models, never averaged, and the metrics that spend a caller's money.
Repair or retirement within 7 business days, in writing.