Appearance
Tier 1 — Zip → County Resolution Audit
Last run: April 14, 2026 Duration: 16.2 minutes Total zips: 18,801 CMS calls: 18,930 (with retries) Avg latency: 124 ms Success rate: 99.17%
Purpose
Verify our zip_county collection returns the same set of counties as CMS's /counties/by/zip endpoint for every zip in our service area. If a user's zip spans multiple counties, both we and CMS should return the same county options for them to choose from.
Method
For each unique zip, query CMS and compare the FIPS code set to ours. Match = same set, regardless of order. Mismatch = any difference (extra counties, missing counties, wrong FIPS).
Results — April 14, 2026
| Metric | Count | % |
|---|---|---|
| Exact matches | 18,415 | 97.95% |
| County count mismatches | 352 | 1.87% |
| CMS errors (max retries) | 29 | 0.15% |
| Issues to fix | 381 | 2.03% |
Breakdown of mismatches
| Type | Count | Description |
|---|---|---|
| Extra in CMS (we're missing) | 310 | CMS returns counties our DB doesn't have for this zip |
| Extra in ours (CMS doesn't have) | 59 | We have counties CMS doesn't return |
Missing counties by state
Most concentrated in:
| State | Missing zips |
|---|---|
| AL | 143 |
| NC | 77 |
| SC | 33 |
| FL | 13 |
| TX | 12 |
| AK | 10 |
| SD | 9 |
| NE | 7 |
| WY | 6 |
| AZ | 6 |
| OR | 6 |
| Others | 30 |
Root cause analysis
Type 1: Cross-state border zips (most "extra in ours")
Examples:
21912(MD): we have DE 10003 entry, CMS doesn't return DE for this zip21874(MD): we have DE 10005 entry, CMS doesn't return DE for this zip
Why: During the CMS enrichment sweep, we asked CMS about every zip, including ones near state borders. CMS sometimes returned multiple states. We stored those, but CMS doesn't always return the same set when re-queried.
Action: Cross-state entries for non-service-area states already deleted. Remaining cross-state entries within our 30 federal states are flagged for cleanup.
Type 2: AL/NC/SC missing border counties (most "extra in CMS")
Examples:
27249(NC): missing 37001, 37033, 3715727265(NC): missing 37057, 3706730165(GA-bordering): we have AL 01019 but missing GA counties
Why: Our enrichment captured the primary county per zip but missed secondary counties for zips that span 3+ counties. The original CMS sweep had query-result limits.
Action: Run scripts/db/fix-tier-1-gaps.js to query CMS and add missing entries.
Type 3: CMS errors (29 zips)
All "max retries exceeded" — CMS API timeouts. These zips should be retried in a follow-up pass.
Action plan
- Run
scripts/db/fix-tier-1-gaps.js— re-query CMS for the 352 mismatched zips and reconcile - Re-run Tier 1 — verify match rate ≥99.5%
- Document remaining residual — any zips that still don't match get individual investigation
Production impact
The 310 "missing in ours" zips affect users whose zip spans 2+ counties — they'll see fewer county options in the picker than they should. This is a user-facing functional gap but doesn't produce wrong premiums (if they pick the right county, the rest of the flow is correct).
The 59 "extra in ours" zips are mostly cross-state border noise. Users would see counties from states we don't serve, which would fail at the plan search step. Functional impact: minor confusion, no wrong data.
CMS API stats
total: 18930
success: 18772
retried: 158 (backoff events)
failed: 29
avgLatencyMs: 124
successRate: 99.17%