Persist discovery answers to CSV for reprocessing and recovery

Save each discovery conversation's prompts and answers to durable CSV
files (per-conversation + append-only master log) on both save and
completion, so answers survive an extraction error, can be re-fed to the
AI, and can be reviewed/resumed by the user.

- app/services/answer_store.py: canonical prompt list + atomic CSV writes,
  master append, and read-back helpers (DB stays system of record; CSV
  failures are logged, never fatal).
- discovery router: write CSV on /respond and /complete; new endpoints
  GET /answers, GET /answers.csv, POST /reprocess (shared extraction
  helper; locked profiles return 409).
- discovery.html: prefill/resume from saved answers after an error and a
  "Re-run analysis" button wired to /reprocess.
- scripts/reprocess_csv.py: offline CLI to re-run extraction from a CSV
  (print or --write-db).
- QUESTIONS_DIR / QUESTIONS_MASTER_CSV config, .gitignore, README, tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dg6XWUwprmP5QCL18HxssY

README.md

@@ -196,6 +196,7 @@ Important files:
 | `app/models.py` | SQLAlchemy ORM models for users, refresh tokens, activity log, conversations, and profiles |
 | `app/database.py` | Async database engine, session factory, SQLite directory setup |
 | `app/services/extractor.py` | Anthropic client wrapper, prompt, JSON parsing, retry logic |
+| `app/services/answer_store.py` | Durable CSV persistence of discovery questions/answers (per-conversation + master log); canonical prompt list |
 | `app/migration_bootstrap.py` | Stamps pre-Alembic SQLite DBs as revision `001` so `alembic upgrade head` succeeds on older local databases |
 | `app/static/discovery.html` | Browser-based seven-prompt flow |
 | `app/static/profile.html` | Browser-based profile display, edit, and confirm actions; links to reflection |
@@ -212,6 +213,7 @@ Important files:
 | `app/services/tagging.py` | `FoundationTagger`: Anthropic smart-tagging suggestion (Phase 5) |
 | `app/routers/coaching.py` | Phase 3 coaching routes: preferences, check-ins, weekly batch `/run` |
 | `app/routers/integration.py` | Phase 4/5 integration: foundations, task-mappings, work-patterns, suggest-foundation |
+| `scripts/reprocess_csv.py` | Offline CLI to re-run AI extraction over a saved answers CSV (print or `--write-db`) |
 | `alembic/versions/001_initial.py` | Initial database schema migration |
 | `alembic/versions/002_add_auth.py` | Adds `users`, `refresh_tokens`, and `activity_log` tables |
 | `alembic/versions/003_add_goals.py` | Adds the goal columns to `discovery_conversation` and `discovery_profile` |
@@ -229,6 +231,8 @@ Important files:
 | `tests/test_phase5.py` | Tests for goal-history and smart-tagging endpoints |
 | `tests/test_auth.py` | Tests for the dual-auth dependency (JWT + cookie + API key), token refresh/logout, admin enforcement, and domain allow-list |
 | `tests/test_profile_edit.py` | Tests for `PATCH /discovery/profile/me` (edit, partial update, lock/`409`) |
+| `tests/test_answer_store.py` | Unit tests for the CSV answer store (round-trip, atomic overwrite, master header, fallbacks) |
+| `tests/test_discovery_csv.py` | Tests for the CSV/reprocess endpoints (respond writes CSV, answers, download, reprocess, lock/`409`) |
 | `tests/test_reflection.py` | Tests for the reflection endpoints (turns, applied revisions, lock/`409`, history) |
 | `tests/test_migration_bootstrap.py` | Unit tests for the pre-Alembic SQLite stamping helper |
 | `tests/test_static_discovery.py` | Guard tests for the static pages' cookie-session and edit contract |
@@ -393,6 +397,10 @@ It stores the responses on the existing conversation and returns:
 
 If the conversation id does not exist, it returns `404`.
 
+On every save the answers are also written to a durable CSV copy (see
+[Answer CSV persistence](#answer-csv-persistence)) so they survive an
+extraction error and can be reviewed or re-processed.
+
 ### 4. Completing Analysis
 
 `POST /discovery/{conversation_id}/complete` loads the conversation, builds a
@@ -409,6 +417,55 @@ The route rejects completion with:
 On success, it stores a new `DiscoveryProfile`, marks the conversation
 `completed_at`, and returns the profile.
 
+### Answer CSV persistence
+
+Every discovery conversation's prompts and answers are mirrored to CSV on
+disk so the answers are durable beyond the database, recoverable after an
+extraction error, and re-feedable to the AI. The SQLite database stays the
+system of record — CSV write failures are logged and never break a request.
+
+Two artifacts are written, on both save (`/respond`) and completion
+(`/complete`):
+
+- **Per-conversation file** — `data/questions/{conversation_id}.csv`,
+  rewritten in full on each save (latest answers, atomic write).
+- **Master log** — `data/questions_master.csv`, an append-only record of
+  every save/complete event across all conversations, for batch
+  re-processing.
+
+Both use a long format (one row per prompt) with columns:
+`conversation_id, user_id, user_email, status, saved_at, prompt_key,
+prompt_title, answer`. Paths are configurable via `QUESTIONS_DIR` and
+`QUESTIONS_MASTER_CSV`.
+
+Related endpoints:
+
+- `GET /discovery/{conversation_id}/answers` — saved prompts + answers as
+  JSON (used by the browser flow to resume/review original answers).
+- `GET /discovery/{conversation_id}/answers.csv` — download the answers as a
+  CSV file (built from the DB, so it works even if the on-disk copy is
+  missing).
+- `POST /discovery/{conversation_id}/reprocess` — re-run extraction over the
+  saved answers, producing a fresh profile. Returns `409` if the latest
+  profile is locked, `400` if there is nothing to analyze, `502` on an
+  extractor failure. The discovery page's error screen offers a **Re-run
+  analysis** button wired to this endpoint.
+
+Offline/bulk re-processing is available via a CLI that needs no running
+server:
+
+```bash
+# Print the regenerated profile JSON for inspection (default, no DB write):
+python scripts/reprocess_csv.py data/questions/<conversation_id>.csv
+
+# Re-process a single conversation out of the master log and persist it:
+python scripts/reprocess_csv.py data/questions_master.csv \
+    --conversation-id <conversation_id> --write-db
+```
+
+It reads `ANTHROPIC_API_KEY` (and optional `ANTHROPIC_MODEL`) from the
+environment.
+
 ### 5. Loading The Profile
 
 `GET /discovery/profile/me` fetches the newest profile for the authenticated