Post-deploy smoke tests

The post-deploy smoke (scripts/audit/post-deploy-smoke.ts) runs at the end of every staging and prod deploy. If it fails, the deploy workflow turns red and the deploy is NOT marked green.

Two anti-pattern classes motivate the script:

1. ENG-272 reads — silent fallback when an Atlas user was narrowed and MONGODB_REFERENCE_URI lost FIND on the reference collections. /api/providers/covered + /api/drugs/covered 500'd for days before a real user noticed.
2. ENG-275 writes — silent no-op (or try/catch-swallow) when a new env var was added in code but never bound to ECS (ENG-272 RESUME_TOKEN_SECRET, ENG-274 SCHEDULER_*, GitHub #171 home-page capture). Write paths were intentionally excluded from the original 6-check script and are now covered by this Phase 1 expansion.

What runs (11 checks)

#	Check	What it proves
1	GET /api/health	Task is responding
2	POST /api/eligibility	CMS API key valid + primary Mongo cluster
3	POST /api/providers/autocomplete	CMS API only (no DB)
4	POST /api/providers/covered	CMS API + getReferenceDb() — the ENG-272 catcher
5	POST /api/drugs/autocomplete	CMS API only
6	POST /api/drugs/covered	CMS API + getReferenceDb() — the ENG-272 catcher
7	POST /api/waitlist (agent)	Agent signup write-path → Mongo row + cleanup
8	POST /api/agents/discovery (partial)	Partial-save write-path → Mongo row + cleanup
9	POST /api/agents/discovery/resume	Token verify — valid (404 expected since no row), tampered (401), expired (401), missing (400)
10	POST /api/agents/discovery/send-reminder	Auth boundary — 401 without token, 200 with token
11	GET /, /plans, /agents, /agent-onboarding, /agent-discovery, /updates, /team/how-we-work	Every primary route 200 + Content-Type text/html

Checks 1-6 always run. Checks 7-10 require additional env vars (see below); each skips with a clear message if its dependencies are missing.

When it runs

Workflow	Trigger	URL targeted
deploy-staging.yml	Push to staging branch (or workflow_dispatch)	https://origin.stage.askflorence.health
deploy-prod.yml	workflow_dispatch only	https://origin.askflorence.health

Both target the origin.* subdomain (bypasses CloudFront + WAF), since AWS managed-rule false-positives block GitHub Actions runner IPs.

Synthetic test data

Every write check uses an email matching:

taha+ci-smoke-<runId>-<checkN>@askflorence.health

runId = GITHUB_RUN_ID in CI, ISO timestamp locally. The plus-address pattern lets us GDPR-delete the resulting HubSpot contacts safely (per CLAUDE.md May 9 precedent — never on real work emails).

Test inputs (when applicable):

• full_name = CI Smoke
• role = individual
• npn = 99999999
• phone = 5555550100

These match the manual smoke flow inputs documented in CLAUDE.md → "Standard smoke tests".

Cleanup

Each write check runs a cleanup in its finally block:

Resource	How it's cleaned
agent_waitlist_submissions row	MongoClient.deleteOne({ email }) via MONGODB_WRITE_URI
agent_survey_responses row	Same pattern
EBS schedule	SchedulerClient.send(DeleteScheduleCommand) against askflorence-<env>-agent-reminders group
HubSpot contact	POST /crm/v3/objects/contacts/gdpr-delete — gated on the synthetic-email regex (defense in depth)

End-of-run sweep deletes any orphans matching the synthetic pattern older than 5 minutes (handles crashed prior runs). The 5-minute floor is to avoid clobbering in-flight rows from a parallel CI run.

Required env vars

scripts/audit/post-deploy-smoke.ts reads from process.env. In CI, the deploy workflows inject these via aws secretsmanager get-secret-value shell calls in a step before the smoke step. Each value is ::add-mask::-ed so it doesn't echo to the workflow log.

Env var	Used by	Source
SMOKE_TARGET_URL	All checks	Hardcoded per workflow (origin.* URL)
MONGODB_WRITE_URI	Checks 7, 8 + cleanup + end-of-run sweep	staging/mongodb/app-write (or prod/)
RESUME_TOKEN_SECRET	Check 9 (token minting)	staging/resume-token-secret (or prod/)
INTERNAL_REMINDER_TOKEN	Check 10 (200-case)	staging/internal-reminder-token (or prod/)
SCHEDULER_GROUP_NAME	Check 7 cleanup	Hardcoded per workflow
AWS_REGION	Check 7 cleanup default	Hardcoded us-east-1
HUBSPOT_ACCESS_TOKEN	HubSpot cleanup (optional)	staging/hubspot-access-token (or prod/); skip if unset
SMOKE_TIMEOUT_MS	Per-request timeout	Optional, default 15000

Locally, set these in .env.local or export them in your shell before running.

Running locally

Against local dev:

SMOKE_TARGET_URL=http://localhost:3004 \
MONGODB_WRITE_URI="mongodb+srv://..." \
RESUME_TOKEN_SECRET="..." \
INTERNAL_REMINDER_TOKEN="..." \
npx tsx scripts/audit/post-deploy-smoke.ts

Against staging:

SMOKE_TARGET_URL=https://origin.stage.askflorence.health \
MONGODB_WRITE_URI="mongodb+srv://..." \
RESUME_TOKEN_SECRET="..." \
INTERNAL_REMINDER_TOKEN="..." \
SCHEDULER_GROUP_NAME=askflorence-staging-agent-reminders \
npx tsx scripts/audit/post-deploy-smoke.ts

Or via npm run smoke:post-deploy (same script).

Or via the preflight orchestrator in --full mode:

npm run preflight -- --full --base-url=https://origin.stage.askflorence.health

Known limitations

• HTTP-only, not DOM-level. The browser-page smoke (Check 11) asserts HTTP 200 + text/html; it does NOT exercise React hydration, click handlers, or form interactions. For that, use the manual flows in CLAUDE.md → "Standard smoke tests" (member 9-step + agent 7-step). Full Playwright E2E is Phase 3a (separate Linear ticket).
• No cross-region edge probes (CloudFront / DNS / WAF). Those happen out-of-band via the Phase 11 follow-up canaries.
• HubSpot cleanup uses GDPR-delete which permanently blocklists the email at the portal level. Safe ONLY for +ci-smoke-* addresses. The script enforces this with a regex gate (defense in depth); the call still requires HUBSPOT_ACCESS_TOKEN to be set. If HUBSPOT_ACCESS_TOKEN is unset, HubSpot cleanup is skipped — manual cleanup via the portal is needed in that case.

How to interpret a failure

Failing check	Likely root cause
/api/providers/covered / /api/drugs/covered 500	MONGODB_REFERENCE_URI binding or role privileges (ENG-272)
/api/eligibility 500	Primary cluster MONGODB_URI binding or CMS_API_KEY
/api/waitlist (agent) non-200	Waitlist write privileges (ENG-279) or schema-validation regression
agent_waitlist_submissions row not found	Route returned 200 but didn't persist; check ECS CloudWatch logs
Resume-token tests fail	RESUME_TOKEN_SECRET drift between Secrets Manager + ECS + smoke env
send-reminder 401 with valid token	INTERNAL_REMINDER_TOKEN drift between ECS + reminder Lambda + smoke env
HTML 4xx/5xx on a browser route	Route deleted, build incomplete, or runtime crash; check ECS logs

Cross-references

• scripts/audit/post-deploy-smoke.ts — the script (in the repo root)
• .github/workflows/deploy-staging.yml — staging deploy + smoke (in the repo root)
• .github/workflows/deploy-prod.yml — prod deploy + smoke (in the repo root)
• CLAUDE.md → "Standard smoke tests" section — the manual browser-flow companion (repo root)
• docs/development/preflight.md — the local mirror of PR-time CI
• ENG-275 — original Phase 1 issue (closed docs-only; Phase 1 actually shipped under ENG-284)
• ENG-284 — the issue this script + doc shipped under
• GitHub #156 — PR-level smoke expansion tracking