impactflow_discovery/README.md

# ImpactFlow Self-Discovery Module

ImpactFlow Self-Discovery is a standalone FastAPI service that turns seven
short reflection responses into a structured Enneagram + Ikigai + goals
profile. It is
designed to run independently from the main ImpactFlow app on port `8011` and
can be integrated into the larger product later.

The service presents a lightweight browser flow, stores the user's narrative
answers in SQLite, sends those answers to Anthropic for structured extraction,
persists the resulting profile, and displays a plain-language profile page.

## Quick Explanation For AI Assistants

If you need to explain this app in detail, use this mental model:

1. A user opens `/static/discovery.html`.
2. The browser sends them through Google sign-in at `GET /api/auth/login`;
   `GET /api/auth/callback` mints an access + refresh token, sets them as
   httpOnly cookies, and redirects back into the app. The browser holds no
   tokens itself — the cookies are sent automatically on later requests.
3. The browser starts a discovery conversation with `POST /discovery/start` —
   the backend derives the `user_id` from the session cookie, not the body.
4. The user answers seven open-ended prompts: five discovery prompts plus a
   near-term (6–12 month) and a long-term (3–5 year) goal prompt.
5. The browser saves all answers with
   `PUT /discovery/{conversation_id}/respond`.
6. The browser asks the backend to analyze the saved answers with
   `POST /discovery/{conversation_id}/complete`.
7. The backend calls Anthropic through `DiscoveryExtractor`.
8. The extractor asks for JSON containing Enneagram, instinctual variant,
   Ikigai summaries, articulated short- and long-term goals, confidence flags,
   and optional extraction notes.
9. The backend stores that JSON as a `DiscoveryProfile` row owned by the
   authenticated user.
10. The browser redirects to `/static/profile.html`.
11. The profile page loads the newest profile with `GET /discovery/profile/me`.
    While it is unlocked the user can revise their words with
    `PATCH /discovery/profile/me`, then lock it with
    `PUT /discovery/profile/me/confirm`.
12. Optionally (Phase 2) the user opens `/static/reflect.html` and refines the
    profile through an AI-coach reflection loop (`POST /discovery/profile/me/reflect`)
    — the coach mirrors the profile back and applies the person's own
    corrections — before affirming with the same confirm/lock.
13. After affirming (Phase 3) the user opens `/static/coaching.html` to tune
    coaching preferences (auto-derived from their profile) and receive periodic
    check-ins that quote their own words and ask if their direction still holds.
    A weekly cron calls `POST /discovery/coaching/run` to generate due check-ins.
14. (Phase 4) The ImpactFlow core time-tracker maps each logged task to a
    profile foundation via `POST /discovery/integration/task-mappings`. The
    rolled-up work patterns drive `/static/dashboard.html` and are fed into the
    coaching check-ins so they can reflect where time has actually gone.

Machine-to-machine callers (e.g. the MCP server) skip the OAuth dance and
authenticate with `X-API-Key: $IMPACTFLOW_API_KEY` instead. That header
resolves to a synthetic admin user (`api-key-admin`) so foreign keys stay
valid and admin-only routes work without a real Google sign-in.

The app is intentionally small: static HTML/CSS for the UI, FastAPI for the
API, async SQLAlchemy for persistence, Alembic for migrations, SQLite for local
storage, Google OAuth + httpOnly cookie sessions for browser auth, and Anthropic for the analysis
step.

## What The App Does

The module collects seven narrative prompts — five discovery prompts plus two
goal prompts:

| Stored field | User-facing prompt | Purpose in extraction |
| --- | --- | --- |
| `prompt_alive` | The Alive Moment | Reveals energy, motivation, strengths, and core need |
| `prompt_friction` | The Friction Moment | Strongest signal for Enneagram triad |
| `prompt_pull` | The Natural Pull | Helps infer instinctual variant and recurring interests |
| `prompt_recognition` | The Recognition Moment | Reveals what the person values being seen for |
| `prompt_future` | The Future Pull | Helps infer mission, vocation, and ideal future direction |
| `prompt_goals_short` | The Near Horizon | The person's own near-term (6–12 month) goals, articulated back |
| `prompt_goals_long` | The Long Horizon | The person's own long-term (3–5 year) goals, articulated back |

The generated profile includes:

| Field | Meaning |
| --- | --- |
| `triad` | One of `gut`, `heart`, or `head` |
| `probable_type` | Likely Enneagram type number, `1` through `9` |
| `wing` | Adjacent Enneagram wing type |
| `instinctual_variant` | One of `sp`, `so`, or `sx` |
| `instinctual_stack` | Ordered stack such as `sp/so/sx` |
| `love_summary` | Ikigai: what the user loves |
| `strength_summary` | Ikigai: what the user is good at |
| `mission_summary` | Ikigai: what the world needs from the user |
| `vocation_summary` | Ikigai: what the user can be paid for |
| `overlap_narrative` | Plain-language convergence narrative |
| `short_term_goals` | The user's own near-term (6–12 month) goals, articulated back |
| `long_term_goals` | The user's own long-term (3–5 year) goals, articulated back |
| `confidence` | Confidence flags for triad, type, variant, and Ikigai |
| `extraction_notes` | Optional ambiguity or caveat from the model |
| `locked` | Whether the user has confirmed the profile |

The profile page deliberately avoids showing the raw Enneagram type number to
the user. It translates the triad into plain-language pattern descriptions and
shows the Ikigai summaries as cards.

## Architecture

```text
Browser static UI         MCP server / other machines
  discovery.html            X-API-Key: $IMPACTFLOW_API_KEY
  profile.html
        |                          |
        | Google OAuth +           |
        | session cookie           |
        v                          v
+--------------------------------------------+
| FastAPI app                                |
|   app/main.py        (middleware stack)    |
|   app/auth.py        (dual-auth dependency)|
|   app/routers/auth.py     /api/auth, /api/me
|   app/routers/activity.py /api/activity
|   app/routers/discovery.py /discovery/*    |
|   app/tracking.py    (activity middleware) |
+--------------------------------------------+
        |
        v
Async SQLAlchemy + SQLite
  app/database.py
  app/models.py     (users, refresh_tokens, activity_log,
                     discovery_conversation, discovery_profile)
  data/discovery.db
        |
        v
Anthropic extraction
  app/services/extractor.py
```

Important files:

| File | Role |
| --- | --- |
| `app/main.py` | FastAPI entrypoint, middleware wiring, lifespan hooks, health check, static file mount, root redirect |
| `app/auth.py` | Google OAuth registration, JWT issue/decode, dual-auth dependency, refresh-token hashing, domain allow-list |
| `app/tracking.py` | `ActivityTrackingMiddleware` and `log_activity` helper |
| `app/routers/auth.py` | OAuth endpoints, `/api/me`, sessions, refresh, logout, `/api/me/stats` |
| `app/routers/activity.py` | Activity feed, per-user summary, admin activity view, 90-day retention pruner |
| `app/routers/discovery.py` | Discovery API routes; all routes user-scoped via the auth dependency |
| `app/schemas.py` | Pydantic request and response models |
| `app/models.py` | SQLAlchemy ORM models for users, refresh tokens, activity log, conversations, and profiles |
| `app/database.py` | Async database engine, session factory, SQLite directory setup |
| `app/services/extractor.py` | Anthropic client wrapper, prompt, JSON parsing, retry logic |
| `app/migration_bootstrap.py` | Stamps pre-Alembic SQLite DBs as revision `001` so `alembic upgrade head` succeeds on older local databases |
| `app/static/discovery.html` | Browser-based seven-prompt flow |
| `app/static/profile.html` | Browser-based profile display, edit, and confirm actions; links to reflection |
| `app/static/reflect.html` | Phase 2 AI-coach reflection chat (mirror loop, applies revisions, affirm) |
| `app/static/coaching.html` | Phase 3 coaching preferences form + check-in feed |
| `app/static/dashboard.html` | Phase 4 goal dashboard: time per foundation + neglected ones |
| `app/static/auth.js` | Shared `authedFetch` helper: sends session cookies, silently refreshes on `401`, redirects to login |
| `app/static/style.css` | Shared UI styling |
| `app/services/reflector.py` | `ReflectionCoach`: Anthropic-backed mirror loop, JSON parsing, revision filtering |
| `app/services/coaching.py` | Deterministic preference generator + `CheckinCoach` (Anthropic check-in text) |
| `app/services/foundations.py` | The six foundations + pure work-pattern aggregator (Phase 4) |
| `app/routers/coaching.py` | Phase 3 coaching routes: preferences, check-ins, weekly batch `/run` |
| `app/routers/integration.py` | Phase 4 task-to-goal integration: foundations, task-mappings, work-patterns |
| `alembic/versions/001_initial.py` | Initial database schema migration |
| `alembic/versions/002_add_auth.py` | Adds `users`, `refresh_tokens`, and `activity_log` tables |
| `alembic/versions/003_add_goals.py` | Adds the goal columns to `discovery_conversation` and `discovery_profile` |
| `alembic/versions/004_add_reflection.py` | Adds `reflection_message` and `profile_revision` tables (Phase 2) |
| `alembic/versions/005_add_coaching.py` | Adds `coaching_preferences` and `coaching_checkin` tables (Phase 3) |
| `alembic/versions/006_add_task_mapping.py` | Adds the `task_mapping` table (Phase 4) |
| `tests/conftest.py` | Shared `app_client` fixture (isolated app + temp DB) |
| `tests/test_extractor.py` | Unit tests for extraction plumbing, goals, and retry behavior |
| `tests/test_reflector.py` | Unit tests for `ReflectionCoach` (mirror, revision filtering, retry) |
| `tests/test_coaching_prefs.py` | Unit tests for the deterministic coaching-preference generator |
| `tests/test_coaching.py` | Tests for coaching endpoints (preferences, check-ins, due-logic batch, work-pattern wiring) |
| `tests/test_foundations.py` | Unit tests for the pure work-pattern aggregator |
| `tests/test_integration.py` | Tests for the task-to-goal integration endpoints |
| `tests/test_auth.py` | Tests for the dual-auth dependency (JWT + cookie + API key), token refresh/logout, admin enforcement, and domain allow-list |
| `tests/test_profile_edit.py` | Tests for `PATCH /discovery/profile/me` (edit, partial update, lock/`409`) |
| `tests/test_reflection.py` | Tests for the reflection endpoints (turns, applied revisions, lock/`409`, history) |
| `tests/test_migration_bootstrap.py` | Unit tests for the pre-Alembic SQLite stamping helper |
| `tests/test_static_discovery.py` | Guard tests for the static pages' cookie-session and edit contract |
| `tests/fixtures/{gut,head,heart}_type_responses.json` | Synthetic seven-prompt responses used by extractor tests |
| `smoke_test.py` | In-process end-to-end API smoke test with fake extraction; also exercises the auth and activity tracking surface |

## Authentication

The `get_current_user` dependency in `app/auth.py` resolves a request from
three credential sources, tried in order: `X-API-Key`, then an
`Authorization: Bearer` access token, then the session cookie.

| Caller | Mechanism | Notes |
| --- | --- | --- |
| Browser users | Google OAuth → httpOnly session cookies | Set by `/api/auth/callback`; the access cookie is refreshed silently via `/api/auth/refresh` |
| Scripts / API clients | `Authorization: Bearer <access token>` | The same access token presented manually instead of via cookie |
| Machine-to-machine (MCP server) | `X-API-Key: $IMPACTFLOW_API_KEY` | Resolves to a synthetic admin user `api-key-admin` so FK constraints stay valid |

The first real Google user to sign in is auto-promoted to `role=admin`;
every subsequent user defaults to `role=user`. Admin-only routes (e.g.
`/api/admin/activity`) check the role on the resolved user.

### One-Time Google Cloud Setup

1. In <https://console.cloud.google.com> open APIs & Services → Credentials.
2. Create an **OAuth 2.0 Client ID** of type **Web application**.
3. Add authorized redirect URIs that match `OAUTH_REDIRECT_URI` in `.env`
   exactly (scheme, host, port, path — Google treats `localhost` and
   `127.0.0.1` as distinct):
   - `http://localhost:8011/api/auth/callback` for local http dev (set `COOKIE_SECURE=false`)
   - `https://impactflow.teamci.org:8011/api/auth/callback` for the deployed instance (HTTPS; set `COOKIE_SECURE=true`)
4. Copy the client id and client secret into `.env` as `GOOGLE_CLIENT_ID` and
   `GOOGLE_CLIENT_SECRET`.
5. Generate two random 64-character strings and put them in `.env` as
   `JWT_SECRET` and `IMPACTFLOW_API_KEY`:

   ```bash
   .venv/Scripts/python.exe -c "import secrets; print(secrets.token_urlsafe(48))"
   ```

### OAuth Endpoints

| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/api/auth/login` | 302s the browser to Google's consent screen |
| `GET` | `/api/auth/callback` | Google redirects here with `code`; we exchange it for a Google access token, find-or-create the user, then return `{access_token, refresh_token, user}` |
| `POST` | `/api/auth/refresh` | Body `{refresh_token}` → new short-lived access token |
| `POST` | `/api/auth/logout` | Body `{refresh_token}` → revokes that refresh token (idempotent) |

`access_token` lifetime defaults to 15 minutes; `refresh_token` lifetime
defaults to 7 days. Both are configurable through `.env`. Refresh tokens are
stored as SHA-256 hashes — the raw value only exists in the response from
`/api/auth/callback` and `/api/auth/refresh`.

### Profile And Session Endpoints

| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/api/me`, `/api/auth/me` | Current user profile |
| `PATCH` | `/api/me` | Update `display_name` (email is owned by Google) |
| `GET` | `/api/me/stats` | Conversations / profiles / locked-profiles / 30-day activity counts |
| `GET` | `/api/me/sessions` | List active refresh tokens (`id`, `device`, `created_at`, `expires_at`) |
| `DELETE` | `/api/me/sessions/{id}` | Revoke a refresh token |

### Activity Tracking

`ActivityTrackingMiddleware` records one `activity_log` row per authenticated,
successful (`status < 400`), non-noisy request. The `source` column is set to
`mcp` when the request carries an `X-API-Key` header and `web` otherwise, so
Claude-initiated calls are distinguishable from browser activity. Rows older
than 90 days are pruned on app startup.

| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/api/activity?page=…&limit=…` | Paginated activity feed for the current user |
| `GET` | `/api/activity/summary?days=30` | Per-day counts, top resources, web/mcp ratio |
| `GET` | `/api/admin/activity?user_id=…` | Admin-only cross-user feed |

## Runtime Flow

### 1. App Startup

`app/main.py` creates the FastAPI app, wires the middleware stack
(`ActivityTrackingMiddleware` innermost, then `SessionMiddleware` for the
OAuth state cookie, `CORSMiddleware` outermost), includes the auth, activity,
and discovery routers, mounts `/static`, and redirects `/` to
`/static/discovery.html`.

The lifespan hook:

1. Calls `Base.metadata.create_all()` as a local development safety net.
   Alembic remains the source of truth for schema changes.
2. Calls `ensure_api_key_admin()` so the synthetic admin user backing
   `X-API-Key` exists before the first request arrives.
3. Calls `prune_old_activity()` to drop activity log rows older than 90 days.

In Docker, the container's `CMD` runs three steps in order before serving:

1. `python -m app.migration_bootstrap` — if `data/discovery.db` already
   exists with the app tables but no `alembic_version` row (older local DBs
   created via `create_all` before Alembic existed), stamp it as revision
   `001` so step 2 does not try to recreate existing tables.
2. `alembic upgrade head` — apply any outstanding migrations.
3. `uvicorn app.main:app --host 0.0.0.0 --port 8011` — serve the app.

The Dockerfile also runs `alembic upgrade head` at build time against a
throwaway in-image DB as a sanity check that migrations apply cleanly. At
runtime the `./data` volume shadows `/app/data`, so the runtime migration
step above is what populates the persistent database.

### 2. Starting A Conversation

`POST /discovery/start` requires authentication (session cookie, Bearer token,
or `X-API-Key`). It takes no body — the `user_id` is derived from the
authenticated user.

It creates a `DiscoveryConversation` row with:

- a UUID conversation id
- `user_id` set to the authenticated user's id
- `started_at` in UTC
- empty prompt fields

It returns:

```json
{
  "conversation_id": "uuid"
}
```

The static UI starts this conversation once the browser has a session cookie
from the OAuth callback, and retries on submit if the first start call failed.

### 3. Saving Answers

`PUT /discovery/{conversation_id}/respond` accepts all seven prompt responses
(five discovery prompts plus the two goal prompts):

```json
{
  "prompt_alive": "I felt most alive when...",
  "prompt_friction": "Something felt wrong when...",
  "prompt_pull": "I naturally keep returning to...",
  "prompt_recognition": "I felt seen when...",
  "prompt_future": "If I could not fail...",
  "prompt_goals_short": "In the next 6–12 months I want to...",
  "prompt_goals_long": "In the next 3–5 years I want to..."
}
```

All fields default to empty, so a partial save is accepted.

It stores the responses on the existing conversation and returns:

```json
{
  "conversation_id": "uuid",
  "status": "responses_saved"
}
```

If the conversation id does not exist, it returns `404`.

### 4. Completing Analysis

`POST /discovery/{conversation_id}/complete` loads the conversation, builds a
compact response dictionary with keys `alive`, `friction`, `pull`,
`recognition`, `future`, `goals_short`, and `goals_long`, then calls
`DiscoveryExtractor.extract()`.

The route rejects completion with:

- `404` if the conversation does not exist
- `400` if all seven responses are blank
- `502` if Anthropic extraction fails or returns unusable output after retry

On success, it stores a new `DiscoveryProfile`, marks the conversation
`completed_at`, and returns the profile.

### 5. Loading The Profile

`GET /discovery/profile/me` fetches the newest profile for the authenticated
user by descending `generated_at`.

The profile page uses this route after redirect. This means one user can have
multiple completed conversations, but the UI always displays the latest one.

### 6. Editing The Profile

`PATCH /discovery/profile/me` edits the prose of the newest profile. The body
is a partial update — only the fields supplied are changed — over the seven
editable text fields: `love_summary`, `strength_summary`, `mission_summary`,
`vocation_summary`, `overlap_narrative`, `short_term_goals`, and
`long_term_goals`. The AI's structural read (`triad`, `probable_type`, `wing`,
`instinctual_variant`, `instinctual_stack`) and confidence are not editable
here.

It rejects with:

- `404` if the user has no profile
- `409` if the profile is locked (affirming makes it final)
- `400` if the body contains no editable fields

### 7. Confirming The Profile

`PUT /discovery/profile/me/confirm` locks the newest profile by setting
`locked = true`. This is the "This is me" affirmation on the profile page;
once locked, the profile can no longer be edited (`PATCH` returns `409`). It
does not prevent future conversations from generating newer profiles.

### 8. Reflecting With The Coach (Phase 2)

The AI-coach reflection loop lets the person refine their profile through a
conversation before affirming it. The coach is a **mirror, not a compass**: it
reflects the profile back, asks whether it fits, and — only when the person
explicitly corrects or adds something — proposes revised text in the person's
own direction. It never prescribes goals or invents direction. Affirmation is
still the `/confirm` lock above; the loop is what happens before it.

- `POST /discovery/profile/me/reflect` advances the loop by one turn. An empty
  `message` starts it (the coach's opening reflection); a non-empty `message`
  is recorded as the person's turn before the coach replies. When the coach
  proposes revisions, they are applied to the editable prose fields (never the
  structural Enneagram read) and snapshotted. Returns the coach turn, the
  (possibly revised) profile, and `revised`/`revision_note`. `409` once locked.
- `GET /discovery/profile/me/reflection` returns the dialogue (ordered
  `coach`/`person` turns) plus the current profile.
- `GET /discovery/profile/me/revisions` returns the profile's edit/iteration
  history, newest first. Every change is snapshotted in `profile_revision`
  with a `source` of `extraction` (initial), `reflection`, or `manual_edit`.

### 9. Coaching Preferences & Check-ins (Phase 3)

How the person wants to be coached, plus a periodic check-in engine.

Coaching **preferences** are auto-generated from the profile's Enneagram centre
(a deterministic mapping — gut → direct/higher-friction, heart → warm/sensitive
to drift, head → reflective/question-led) and are fully overridable. Fields:
`coaching_frequency`, `coaching_style`, `misalignment_threshold`,
`friction_tolerance`, `prefer_questions_over_directives`,
`time_of_day_preference`.

- `GET /discovery/coaching/preferences` returns the preferences, deriving
  defaults from the latest profile on first access (`404` if no profile yet).
- `PUT /discovery/coaching/preferences` overrides any field (validated against
  the allowed value sets) and marks them user-customized.
- `POST /discovery/coaching/preferences/regenerate` re-derives the defaults
  from the latest profile, discarding overrides.

A **check-in** quotes the person's own words and asks whether their stated
direction still feels valid — mirror, not compass: it asks, it never judges or
prescribes. The person's answer is recorded in `still_valid`.

- `POST /discovery/coaching/checkins` generates a check-in now (on demand).
- `GET /discovery/coaching/checkins` lists them, newest first.
- `PUT /discovery/coaching/checkins/{id}/respond` records the self-assessment
  (`still_valid` + optional note).
- `POST /discovery/coaching/run` is the **weekly batch job** (admin-only,
  intended for a cron): it generates a check-in for every eligible user whose
  cadence is due. Eligible = coaching cadence not `off` and an affirmed
  (locked) profile; due = no prior check-in or the cadence interval has elapsed.

### 10. Task-to-Goal Integration (Phase 4)

This is the boundary the ImpactFlow **core time-tracker** plugs into. A task
maps to a **foundation** — one of the six stable profile elements: `love`,
`strength`, `mission`, `vocation`, `short_term`, `long_term`.

**Core-tracker contract:** when a user logs time, the tracker (1) fetches the
options from `GET /discovery/integration/foundations`, (2) asks the person
"which goal does this build toward?", and (3) posts the answer to
`POST /discovery/integration/task-mappings` with `{external_task_id,
foundation, minutes, task_label?, occurred_at?}`. It calls these endpoints as
the user (forwarded session/JWT) or service-to-service with `X-API-Key`.

- `GET /discovery/integration/foundations` — the six foundations with the
  person's own text (what the tracker shows). `404` if no profile.
- `POST /discovery/integration/task-mappings` — record one logged unit of work.
- `GET /discovery/integration/task-mappings?days=N` — the user's mappings.
- `GET /discovery/integration/work-patterns?days=N` — per-foundation rollup
  (minutes, share, task count, last activity) plus `neglected` foundations.
  Powers the goal dashboard and feeds the coaching reminder engine: a check-in
  is given a plain-language summary of the last 14 days so it can reflect where
  time has and hasn't gone — as an observation to check against the person's
  own words, never a verdict (mirror, not compass).

## API Reference

All `/discovery/*`, `/api/me*`, `/api/activity*`, and `/api/admin/*` routes
require authentication. The browser is authenticated by the session cookie set
at the OAuth callback; machine callers send `X-API-Key`. (A raw
`Authorization: Bearer` access token is also still accepted, e.g. for scripted
clients.) `/api/auth/login`, `/api/auth/callback`, `/health`, `/`, and
`/static/*` are public.

| Method | Path | Auth | Purpose |
| --- | --- | --- | --- |
| `GET` | `/` | public | Redirects to `/static/discovery.html` |
| `GET` | `/health` | public | Liveness check, returns `{"status": "ok"}` |
| `GET` | `/api/auth/login` | public | 302 to Google consent screen |
| `GET` | `/api/auth/callback` | public | OAuth callback; sets session cookies and redirects into the app |
| `POST` | `/api/auth/refresh` | public (refresh cookie or body) | Mint a new access token; refreshes the access cookie |
| `POST` | `/api/auth/logout` | public (refresh cookie or body) | Revoke the refresh token and clear session cookies |
| `GET` | `/api/me`, `/api/auth/me` | yes | Current user profile |
| `PATCH` | `/api/me` | yes | Update `display_name` |
| `GET` | `/api/me/stats` | yes | Per-user usage stats |
| `GET` | `/api/me/sessions` | yes | List active refresh tokens |
| `DELETE` | `/api/me/sessions/{id}` | yes | Revoke a refresh token |
| `GET` | `/api/activity` | yes | Paginated activity feed for current user |
| `GET` | `/api/activity/summary` | yes | Aggregate activity stats |
| `GET` | `/api/admin/activity` | admin | All-users activity feed |
| `POST` | `/discovery/start` | yes | Begin a conversation (user derived from auth) |
| `PUT` | `/discovery/{conversation_id}/respond` | yes | Save all seven responses |
| `POST` | `/discovery/{conversation_id}/complete` | yes | Run extraction, store profile, return profile |
| `GET` | `/discovery/profile/me` | yes | Fetch newest profile for the authenticated user |
| `PATCH` | `/discovery/profile/me` | yes | Edit the newest profile's prose (`409` if locked) |
| `PUT` | `/discovery/profile/me/confirm` | yes | Lock newest profile for the authenticated user |
| `POST` | `/discovery/profile/me/reflect` | yes | Advance the AI-coach reflection loop (Phase 2); applies revisions (`409` if locked) |
| `GET` | `/discovery/profile/me/reflection` | yes | The reflection dialogue for the latest profile |
| `GET` | `/discovery/profile/me/revisions` | yes | Profile edit/iteration history (newest first) |
| `GET` | `/discovery/conversation/{conversation_id}` | yes | Fetch stored conversation responses (owner only) |
| `GET` | `/discovery/coaching/preferences` | yes | Coaching preferences (auto-derived on first access) |
| `PUT` | `/discovery/coaching/preferences` | yes | Override coaching preferences |
| `POST` | `/discovery/coaching/preferences/regenerate` | yes | Re-derive preference defaults from the profile |
| `GET` | `/discovery/coaching/checkins` | yes | List the user's check-ins (newest first) |
| `POST` | `/discovery/coaching/checkins` | yes | Generate a check-in now |
| `PUT` | `/discovery/coaching/checkins/{id}/respond` | yes | Record "is your direction still valid?" |
| `POST` | `/discovery/coaching/run` | admin | Weekly batch: generate due check-ins for eligible users |
| `GET` | `/discovery/integration/foundations` | yes | The six mappable foundations with the person's own text |
| `POST` | `/discovery/integration/task-mappings` | yes | Record a logged unit of work mapped to a foundation |
| `GET` | `/discovery/integration/task-mappings` | yes | List the user's task mappings in a window |
| `GET` | `/discovery/integration/work-patterns` | yes | Per-foundation work-pattern rollup over a window |

## Data Model

### `users`

One row per signed-in human, plus a single synthetic `api-key-admin` row
backing the `X-API-Key` dual-auth path.

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key; `api-key-admin` for the synthetic row |
| `email` | string | Unique, from Google |
| `display_name` | string | From Google `name` claim; editable via `PATCH /api/me` |
| `avatar_url` | string nullable | Google profile picture |
| `google_id` | string nullable, unique | Google `sub` claim; null only for the API-key row |
| `role` | string | `user` or `admin`; first real user is auto-promoted |
| `created_at` | datetime | UTC |
| `last_login_at` | datetime nullable | Updated on every OAuth callback |

### `refresh_tokens`

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `token_hash` | string | SHA-256 of the raw token; unique |
| `device` | string nullable | Captured from `User-Agent` at issue time |
| `created_at` | datetime | UTC |
| `expires_at` | datetime | UTC |
| `revoked_at` | datetime nullable | Set by logout or `DELETE /api/me/sessions/{id}` |

### `activity_log`

Append-only audit trail. Pruned to 90 days on app startup.

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `action` | string | `view`, `create`, `update`, `delete`, or `other` |
| `resource` | string | Coarse resource name parsed from the URL path |
| `resource_id` | string nullable | When a specific record id is identifiable |
| `metadata_json` | text nullable | JSON-encoded extra context |
| `source` | string | `web` or `mcp` (set from presence of `X-API-Key`) |
| `ip_address` | string nullable | From `request.client.host` |
| `user_agent` | text nullable | Truncated to 1000 chars |
| `created_at` | datetime | UTC, indexed |

### `discovery_conversation`

Stores one seven-prompt response set.

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `started_at` | datetime | UTC timestamp |
| `completed_at` | datetime nullable | Set after successful profile generation |
| `prompt_alive` | text nullable | First narrative response |
| `prompt_friction` | text nullable | Second narrative response |
| `prompt_pull` | text nullable | Third narrative response |
| `prompt_recognition` | text nullable | Fourth narrative response |
| `prompt_future` | text nullable | Fifth narrative response |
| `prompt_goals_short` | text nullable | Near-term (6–12 month) goal response |
| `prompt_goals_long` | text nullable | Long-term (3–5 year) goal response |

### `discovery_profile`

Stores one extracted profile for one conversation.

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `conversation_id` | string | Foreign key to `discovery_conversation.id` |
| `generated_at` | datetime | UTC timestamp |
| `triad` | string nullable | `gut`, `heart`, or `head` |
| `probable_type` | integer nullable | Enneagram type number |
| `wing` | integer nullable | Adjacent Enneagram wing |
| `instinctual_variant` | string nullable | `sp`, `so`, or `sx` |
| `instinctual_stack` | string nullable | Ordered stack |
| `love_summary` | text nullable | Ikigai love summary |
| `strength_summary` | text nullable | Ikigai strength summary |
| `mission_summary` | text nullable | Ikigai mission summary |
| `vocation_summary` | text nullable | Ikigai vocation summary |
| `overlap_narrative` | text nullable | Convergence narrative |
| `short_term_goals` | text nullable | Articulated near-term (6–12 month) goals |
| `long_term_goals` | text nullable | Articulated long-term (3–5 year) goals |
| `confidence_json` | text nullable | JSON string for confidence flags |
| `locked` | boolean | Defaults to false |

### `reflection_message`

One turn in the Phase 2 AI-coach reflection loop (migration `004`).

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `profile_id` | string | FK to `discovery_profile.id`, indexed |
| `user_id` | string | FK to `users.id`, indexed |
| `role` | string | `coach` (AI mirror) or `person` (the human) |
| `content` | text | The turn's text |
| `sequence` | integer | Monotonic order within a profile's thread |
| `created_at` | datetime | UTC |

### `profile_revision`

A snapshot of a profile's editable prose at a point in time, so edits and
iterations are captured rather than overwritten (migration `004`).

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `profile_id` | string | FK to `discovery_profile.id`, indexed |
| `user_id` | string | FK to `users.id`, indexed |
| `source` | string | `extraction` (initial), `reflection`, or `manual_edit` |
| `fields_json` | text | JSON snapshot of the seven editable prose fields |
| `note` | text nullable | What changed (e.g. the coach's revision note) |
| `created_at` | datetime | UTC |

### `coaching_preferences`

How the person wants to be coached; one row per user (migration `005`).

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, unique, indexed |
| `profile_id` | string nullable | The profile the defaults were derived from |
| `coaching_frequency` | string | `weekly`, `biweekly`, `monthly`, or `off` |
| `coaching_style` | string | `direct`, `warm`, or `reflective` |
| `misalignment_threshold` | string | `low`, `medium`, or `high` |
| `friction_tolerance` | string | `low`, `medium`, or `high` |
| `prefer_questions_over_directives` | boolean | Lead with questions vs statements |
| `time_of_day_preference` | string | `morning`, `afternoon`, or `evening` |
| `auto_generated` | boolean | True until the user edits a field |
| `created_at` / `updated_at` | datetime | UTC |

### `coaching_checkin`

A periodic coaching check-in and the person's response (migration `005`).

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `profile_id` | string | FK to `discovery_profile.id` |
| `body` | text | The check-in text (quotes the person's own words) |
| `created_at` | datetime | UTC, indexed |
| `still_valid` | boolean nullable | The person's answer: is their direction still valid? |
| `response_note` | text nullable | Optional note with their response |
| `acknowledged_at` | datetime nullable | When they responded |

### `task_mapping`

One logged unit of work from the core tracker, mapped to a foundation
(migration `006`).

| Column | Type | Notes |
| --- | --- | --- |
| `id` | string | UUID primary key |
| `user_id` | string | FK to `users.id`, indexed |
| `external_task_id` | string | Opaque task id from the core tracker (not an FK) |
| `task_label` | string nullable | Human label of the task |
| `foundation` | string | `love`/`strength`/`mission`/`vocation`/`short_term`/`long_term`, indexed |
| `minutes` | integer | Time logged toward it |
| `occurred_at` | datetime | When the work happened, indexed |
| `created_at` | datetime | UTC |

## Extraction Details

`DiscoveryExtractor` is intentionally responsible for plumbing, not business
logic hidden elsewhere. It:

1. Validates that `ANTHROPIC_API_KEY` exists.
2. Builds one user message from the seven responses.
3. Calls Anthropic with `SYSTEM_PROMPT`.
4. Parses the model response as JSON.
5. Strips Markdown code fences if present.
6. Validates that all required top-level keys exist.
7. Validates that `confidence` contains `triad`, `type`, `variant`, and
   `ikigai`.
8. Retries once with an explicit JSON-only reminder if the first response
   cannot be parsed or is missing required keys.
9. Raises `DiscoveryExtractionError` if the API call fails or retry also fails.

Default extraction settings:

| Setting | Value |
| --- | --- |
| Default model | `claude-sonnet-4-6` |
| Max tokens | `2000` |
| Required response type | Single JSON object |
| Retry count | One retry after invalid JSON or missing keys |

The system prompt instructs the model to infer:

- Enneagram triad and type from emotional center signals
- instinctual variant from recurring attention patterns
- Ikigai love, strength, mission, and vocation from repeated story themes
- confidence levels based on consistency and strength of evidence

## Frontend Behavior

The frontend is static HTML with embedded JavaScript, served from `/static`.
The browser session is a server-set httpOnly cookie (issued by the OAuth
callback), so the pages hold no tokens of their own.

`auth.js` (shared helper):

- `authedFetch()` sends every request with `credentials: "include"` so the
  session cookies ride along
- on a `401` it makes a single silent `POST /api/auth/refresh` (the refresh
  cookie is scoped to `/api/auth`) and retries the original request
- if the refresh also fails, it redirects to `GET /api/auth/login` — which is
  also how an unauthenticated first visit gets bounced through Google sign-in

`discovery.html`:

- shows the seven prompts one at a time (five discovery prompts plus near-term
  and long-term goal prompts)
- keeps answers in memory while navigating back and next
- starts a conversation on page load via `authedFetch("/discovery/start")`
- saves all responses on submit (`PUT /discovery/{id}/respond`), then triggers
  extraction (`POST /discovery/{id}/complete`)
- redirects to `/static/profile.html` on success — no user id in the URL, since
  the backend derives the user from the session cookie
- shows an error box and reload button if submission fails

`profile.html`:

- fetches the authenticated user's newest profile via `GET /discovery/profile/me`
- escapes all model-generated text before rendering
- shows the overlap narrative, Ikigai cards, a triad description, and the
  person's near-term/long-term goals, with confidence dots for triad and Ikigai
- while the profile is unlocked, offers two actions: "Edit my words" and
  "This is me"
- edit mode turns the narrative, the four Ikigai summaries, and both goal
  fields into textareas and saves with `PATCH /discovery/profile/me`; the AI's
  structural read (triad/type/wing/variant) is shown but not editable here
- "This is me" locks the profile with `PUT /discovery/profile/me/confirm`; a
  locked profile shows the confirmed state and no longer offers edit
- links to `reflect.html` ("talk it through with your coach") while unlocked

`reflect.html` (Phase 2 AI-coach reflection):

- loads the dialogue + profile via `GET /discovery/profile/me/reflection`, and
  auto-starts the coach's opening reflection when the thread is empty
- shows a chat thread (coach / person bubbles) plus a live profile summary that
  refreshes when the coach applies a revision
- each turn posts to `POST /discovery/profile/me/reflect`; when `revised` is
  true it updates the summary and notes what changed
- "This is me — affirm" locks the profile via the same confirm endpoint

`coaching.html` (Phase 3 coaching):

- loads preferences via `GET /discovery/coaching/preferences` (auto-derived on
  first visit) and renders the six fields as selects + a toggle; Save
  (`PUT`) marks them customized, "Reset to suggested" re-derives from the
  profile
- lists check-ins and can generate one on demand (`POST .../checkins`); each
  unanswered check-in offers "still feels true" / "it's shifted" which posts to
  `.../respond`
- linked from `profile.html` ("Coaching preferences & check-ins")

`dashboard.html` (Phase 4 goal dashboard):

- reads `GET /discovery/integration/work-patterns?days=N` and renders minutes
  and share per foundation as bars, with a selectable window
- surfaces foundations with no logged time and asks whether that matches where
  the person wants their energy — it observes, it does not prescribe
- linked from `profile.html` ("Where your time goes")

## Configuration

Populate `.env` with at minimum the Anthropic key and the auth-related
secrets:

```text
ANTHROPIC_API_KEY=your-key-here
GOOGLE_CLIENT_ID=xxxx.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=GOCSPX-xxxx
JWT_SECRET=random-64-char-string
IMPACTFLOW_API_KEY=random-64-char-string
```

Environment variables:

| Env var | Default | Notes |
| --- | --- | --- |
| `ANTHROPIC_API_KEY` | required | Used by `DiscoveryExtractor` |
| `DATABASE_URL` | `sqlite+aiosqlite:///./data/discovery.db` | Async SQLAlchemy database URL |
| `HOST_BIND_IP` | `0.0.0.0` | Docker host IP for publishing port `8011`; use this when Docker runs inside WSL |
| `HOST_PORT` | `8011` | WSL/Docker host port; Windows portproxy exposes the same port |
| `ANTHROPIC_MODEL` | `claude-sonnet-4-6` | Override extraction model |
| `GOOGLE_CLIENT_ID` | required for browser auth | OAuth 2.0 client id from Google Cloud Console |
| `GOOGLE_CLIENT_SECRET` | required for browser auth | OAuth 2.0 client secret |
| `OAUTH_REDIRECT_URI` | derived from request | Override the callback URL Google redirects to; must be registered in the Cloud Console |
| `JWT_SECRET` | required | HS256 signing key for access tokens; also used for the OAuth `state` session cookie |
| `JWT_ACCESS_MINUTES` | `15` | Access-token lifetime |
| `JWT_REFRESH_DAYS` | `7` | Refresh-token lifetime |
| `COOKIE_SECURE` | `true` | Set `false` for local `http://localhost` dev (Secure cookies aren't sent over plain http); must be `true` in production over HTTPS |
| `POST_LOGIN_REDIRECT` | `/static/discovery.html` | Path the OAuth callback redirects to after setting the session cookies |
| `IMPACTFLOW_API_KEY` | required for MCP/machine auth | Header value for `X-API-Key`; resolves to the synthetic admin user |
| `CORS_ALLOWED_ORIGINS` | `http://localhost:8011` | Comma-separated allow-list of browser origins |
| `ALLOWED_EMAIL_DOMAINS` | empty (any) | Comma-separated allow-list of email domains; empty means accept any verified Google email |

`app/database.py` creates the SQLite directory automatically when the URL uses
a local SQLite file path.

### Production deployment

The service is deployed (Docker Compose on the `ci-gpu` host) behind TLS at:

```
https://impactflow.teamci.org:8011
```

The container reads its env from `.env` (`env_file` in `docker-compose.yml`),
which is gitignored and lives only on the host — it does not travel with the
repo. For this HTTPS deployment the env must set:

- `OAUTH_REDIRECT_URI=https://impactflow.teamci.org:8011/api/auth/callback`
  (and the same URI registered as an Authorized redirect URI on the Google
  OAuth client), and
- `COOKIE_SECURE=true` so session cookies carry the `Secure` flag.

After changing `.env`, apply with `docker compose up -d` (recreates the
container with the new env).

## Quick Start With Docker

This path requires Docker Desktop or another Docker Engine installation with
the Compose plugin available as `docker compose`.

If Docker is installed inside WSL, run these commands from your WSL shell and
change into the Windows-mounted project directory first:

```bash
cd /mnt/c/SyncData/impactflow-discovery
```

```bash
cp .env.example .env
# edit .env and add ANTHROPIC_API_KEY
docker compose up --build
```

Open:

- Discovery flow: <http://100.103.206.4:8011/static/discovery.html>
- Health check: <http://100.103.206.4:8011/health>

When Docker runs inside WSL and you want the app to behave like the other
WSL-published services, keep `HOST_BIND_IP=0.0.0.0` in `.env`, then create a
Windows portproxy from an elevated PowerShell prompt:

```powershell
powershell -ExecutionPolicy Bypass -File .\scripts\setup-wsl-bridge.ps1
```

That creates an entry like `0.0.0.0 8011 -> <WSL-IP> 8011`, so the app is
available on localhost, LAN addresses, and the Tailscale address
`100.103.206.4` without colliding with any existing WSL service on port `8001`.

If you cannot run an elevated PowerShell prompt, use the non-admin TCP bridge
fallback while the app is running:

```powershell
powershell -ExecutionPolicy Bypass -File .\scripts\start-tcp-bridge.ps1
```

Keep that window open for as long as you need `http://100.103.206.4:8011` to
forward to the app.

The SQLite file lives in `./data/discovery.db` and persists across restarts
because `docker-compose.yml` mounts `./data` into the container.

## Local Development

Windows:

```bash
python -m venv .venv
.venv/Scripts/python.exe -m pip install -r requirements.txt
cp .env.example .env
alembic upgrade head
uvicorn app.main:app --reload --port 8011
```

macOS/Linux:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
alembic upgrade head
uvicorn app.main:app --reload --port 8011
```

## Tests And Verification

The extractor tests mock Anthropic, so they do not need a real API key or
network access:

```bash
.venv/Scripts/python.exe -m pytest
```

`smoke_test.py` drives the whole API in process with `httpx.ASGITransport` and
patches `DiscoveryExtractor` so it also avoids a real Anthropic call:

```bash
.venv/Scripts/python.exe smoke_test.py
```

The smoke test verifies:

- `/health`
- unauthenticated requests return `401`
- `X-API-Key` resolves to the synthetic admin user via `/api/me`
- a malformed bearer token is rejected
- `/api/auth/login` redirects to `accounts.google.com`
- conversation creation, response saving, conversation fetching
- profile completion, fetching, and confirmation
- the activity log captures the calls and tags them `source=mcp`
- `/api/me/stats` reports the correct conversation, profile, and locked counts
- key `404` paths

## Error Handling And Edge Cases

The backend currently handles:

| Scenario | Result |
| --- | --- |
| Missing conversation on respond, complete, or fetch | `404` |
| Completing a conversation with all blank answers | `400` |
| Missing Anthropic API key | `502` from complete route |
| Anthropic transport or SDK error | `502` from complete route |
| Invalid model JSON on first try | One retry |
| Invalid model JSON after retry | `502` from complete route |
| Missing profile for user | `404` |

The frontend currently handles submission and profile-load failures by showing a
simple error box. It does not persist partially typed answers across a full page
reload, except for the browser's normal form restoration behavior.

## Integration Notes

This service is ready to be called from a larger ImpactFlow app. The
authenticated user identity comes from Google OAuth (httpOnly session cookies)
on the browser side and `X-API-Key` on the machine-to-machine side; the
per-call `user_id` body parameter is gone.

Likely integration points:

- route users into `/static/discovery.html` (or recreate the flow in the main
  UI); the session cookie set by `/api/auth/callback` is sent automatically on
  subsequent calls, and `auth.js` handles silent token refresh on `401`
- the MCP server should send `X-API-Key: $IMPACTFLOW_API_KEY` on every
  request — no OAuth dance needed
- use `locked` as the user's confirmation signal, and `PATCH
  /discovery/profile/me` if the surrounding app wants its own edit affordance
- decide whether future profiles should supersede locked profiles or be
  versioned in the main product experience

## Privacy And Data Notes

The app stores personal narrative answers and model-generated personality
summaries in SQLite. Treat `data/discovery.db` as sensitive user data.

Do not commit:

- `.env`
- real API keys
- production SQLite databases
- exported user response data

The repository includes test fixtures with synthetic responses for extractor
tests.

## Common Changes

When changing prompts:

1. Update the prompt text in `app/static/discovery.html`.
2. Keep the request body keys aligned with `RespondRequest` in
   `app/schemas.py`.
3. Update `PROMPT_LABELS` in `app/services/extractor.py` if the extraction
   labels should change.
4. Adjust tests or fixtures if the extractor prompt expectations change.

When changing the profile schema:

1. Update `app/schemas.py`.
2. Update `app/models.py`.
3. Add a new Alembic migration.
4. Update `REQUIRED_KEYS` and `SYSTEM_PROMPT` in `app/services/extractor.py`.
5. Update `app/routers/discovery.py` mapping logic.
6. Update `app/static/profile.html` rendering.
7. Add or update tests.

When changing extraction behavior:

1. Update `SYSTEM_PROMPT` in `app/services/extractor.py`.
2. Keep the JSON contract explicit.
3. Update `REQUIRED_KEYS` or `REQUIRED_CONFIDENCE_KEYS` only when the response
   contract changes.
4. Add extractor tests for parsing, retry, or validation changes.

## Known Limitations

- The static discovery page keeps in-progress answers in memory only.
- The static profile page always loads the newest profile for a user.
- Confirming a profile does not prevent a newer profile from being generated.
- The database is SQLite by default and intended for standalone/local service
  operation.
- Extraction quality depends on model output and the clarity of the user's
  stories.
- The system prompt asks for structured interpretation, but personality results
  should be treated as reflective guidance rather than clinical or diagnostic
  truth.