04 — Data Collection (Health Sync & Conversational Capture)

1. The collection philosophy

NataCoach is not a quantified-self product. Whoop and Garmin already are; we do not compete with their dashboards, we consume their conclusions. Health Sync exists for exactly one purpose: to let @NataCoachBot make the same call Nata would make if she saw the client walk into the gym — "you look wrecked, we're going lighter today" — without asking the client anything.

Three consequences of that stance:

1. Decisions first, metrics second. Every row in the data inventory names the coaching decision it improves. No decision, no collection. This keeps the pipeline small, the privacy surface small, and the Wiki Brain free of noise.
2. The wearable is the default input; the user is the fallback, not the other way around. Manual logging is a last resort that costs the user taps. Per the prime directive (the user does less), a question to the user is a budgeted expense: at most one recovery-related question per day, and only when it changes a decision.
3. Everything lands as an event. Whether a fact arrives via webhook, poll, or button tap, it becomes an immutable event in Postgres (and raw/ in the user's wiki repo), then a normalized metric, then — nightly — distilled context in the Wiki Brain. Analytics never read LLM output; they read events (locked decision 6 in the brief).

2. The connect flow

Connecting a wearable is a first-session onboarding step (see 02-user-experience.md) and takes the user under 60 seconds. The bot sends a deep link; the provider handles consent; our callback binds the provider account to the Telegram user; then we backfill 30 days and switch to webhooks.

sequenceDiagram
 participant M as Marta (Telegram)
 participant B as NataCoachBot
 participant HS as Health Sync service
 participant P as Whoop OAuth + API

 M->>B: taps "Connect Whoop" button
 B->>HS: create connect intent (telegram_user_id, provider)
 HS-->>B: signed deep link (state token, 15 min TTL, single use)
 B->>M: message with "Authorize Whoop" URL button
 M->>P: opens link, logs in, grants scopes
 P->>HS: redirect to /oauth/callback with code and state
 HS->>HS: verify state, bind provider account to Telegram user
 HS->>P: exchange code for tokens
 P-->>HS: access token (1 h) + rotating refresh token
 HS-->>B: connection confirmed
 B->>M: "Whoop connected. Pulling your last 30 days now..."
 HS->>P: backfill jobs via BullMQ (sleep, recovery, workouts, 30 days)
 P-->>HS: paged historic data
 HS->>HS: normalize to events, then daily_metrics
 P->>HS: ongoing webhooks (recovery.updated, sleep.updated, workout.updated)
 HS->>P: fetch full object by id (thin webhook pattern)
 B->>M: next morning, first recovery-aware Morning Brief

Design details that matter:

• The state token is a signed JWT carrying telegram_user_id, provider, and a nonce; 15-minute TTL, single use. This is the entire account-linking mechanism — no passwords, no email matching. If the token is expired the callback page shows one line ("Link expired — tap Connect again in Telegram") rather than an error dump.
• Tokens are stored encrypted (AES-256-GCM, key held outside the DB) in a provider_accounts table: user_id, provider, provider_user_id, access_token_enc, refresh_token_enc, scopes, status (active | needs_reauth | revoked), last_event_at. Privacy treatment is detailed in 03-system-architecture.md.
• Backfill is 30 days, not 90. Thirty days is enough to establish the user's recovery baseline and weekly load rhythm for the weight-proposal engine; more history adds API cost and privacy weight without changing a decision. Backfill runs as a BullMQ job chain, ~120 paged requests per user, completes in under 5 minutes at pilot scale.
• The bot narrates the outcome, not the plumbing. After backfill: "Got it — 30 days of sleep and recovery in. Your average night is 6 h 41 m and your recovery baseline is 58%. Tomorrow's Morning Brief will use this." The numbers make the connection feel immediately worth it.

3. Whoop API v2

Whoop is the launch-priority provider (Marta wears one) because it hands us the exact abstraction we want: a daily recovery score already normalized to 0–100.

4. Garmin Health API

Garmin covers Denys (Forerunner) and is architecturally the easier integration: the Health API is push-based — Garmin POSTs summary payloads to our endpoint, no polling loop needed.

⚠ Project risk: Garmin partner program

The Garmin Health API is not self-serve. Access requires an application to Garmin's developer/partner program (company details, use-case description, data-handling answers), and approval historically takes 2–6 weeks with no SLA. This is a real schedule risk flagged in 11-roadmap.md.

Mitigation ladder, in order:

1. Apply on day one of the build, before any Garmin code is written; Whoop (self-serve) ships first regardless.
2. Interim fallback — conversational mode. A Garmin user without API access still gets full coaching: the one-question morning fallback (§7) replaces the recovery feed. Denys loses automatic run import, not coaching quality.
3. Interim fallback — manual export. Power users can export activities from Garmin Connect (FIT/CSV) and forward the file to the bot; Health Sync parses it as a batch of activity events. Documented but not promoted — it violates the user does less.
4. Phase-2 structural fallback — Apple Health bridge. Garmin syncs to Apple Health on iPhone; the phase-2 bridge (e.g. Health Auto Export pushing JSON to our webhook, per the brief locked decision) reaches Garmin data without Garmin's program. Denys is an iPhone user; this path fully covers him if approval stalls.

5. Normalization

5.1 Canonical daily metrics

Every provider payload is reduced to one canonical row per user per local day (materialized from events into daily_metrics). Coaching logic and analytics read only this table plus the event log — never provider-specific fields.

Each field carries per-field provenance (source, source_event_id, is_proxy) so a proxy value from a button tap is never mistaken for a measured one.

5.2 Provider mapping

An honest caveat, encoded in the system rather than hidden: Whoop's recovery (HRV-driven) and Garmin's Body Battery (energy model) are not the same physiology, so cross-provider comparison of raw scores is banned. What is canonical is the band: green ≥ 67, yellow 34–66, red ≤ 33 (Whoop's own bands, adopted product-wide). All coaching decisions key off the band; the raw score appears only in trends within one provider. The Coach Console shows Marta and Denys in bands, never in falsely comparable percentages.

5.3 Conflict resolution

Conflicts are rare at pilot (each persona wears exactly one device) but the rules are fixed now:

6. How recovery data changes coaching — same day

This is the payoff for the whole pipeline. Recovery data feeds exactly two same-day decisions:

1. The Morning Brief — tone and plan proposal are chosen by band:

2. Session Mode weight proposals — the engine in 06-training-experience.md takes (exercise history, last RPEs, recovery_band, day_load trend) and emits per-set proposals. Recovery is an input, never an override: the user can always tap through to the original plan.

Realistic red-day transcript (Marta, baby night, Whoop recovery 31%):

07:12  @NataCoachBot
Morning, Marta ☀️
Rough night — 4 h 50 m of sleep, recovery 31% (red).
Today is Pull Day. Proposal: same exercises, one set
less each, and we hold last week's weights instead of
adding. About 35 min instead of 45.

[ ✓ Do the lighter version ]  [ Keep full plan ]  [ Skip today ]

07:14  Marta
✓ Do the lighter version

07:14  @NataCoachBot
Smart call. Lat pulldown will open at 38 kg (not 40).
See you at the gym — tap Start when you're there. 💪

Timing contract (identical in 02 §2.2 and 03 §2): the Brief is built by a per-user pre-dawn job (06:30 local) and delivered 30–45 minutes after detected wake, falling back to the user's chosen time (default 07:15 local) when no wake signal has arrived — never inside quiet hours. Sleep/recovery webhooks usually land 20–60 min after wake; the readiness check below decides whether the Brief is data-driven or question-driven.

7. Failure UX

Health Sync fails quietly and degrades to conversation — the user never sees an error they cannot fix in one tap.

flowchart TD
 A["Morning Brief scheduler fires (user local time)"] --> B{"Fresh sleep or recovery data for today?"}
 B -- "yes" --> C["Compute recovery band"]
 B -- "no" --> D["Poll provider once (reconciliation sweep)"]
 D --> E{"Data arrived?"}
 E -- "yes" --> C
 E -- "no" --> F["Ask one question instead"]
 F --> G["Map answer to proxy values (is_proxy = true)"]
 C --> H["Send recovery-aware Morning Brief"]
 G --> H

Token expiry / revocation. A failed refresh (invalid_grant) flips the account to needs_reauth and triggers exactly one message with a fresh deep link:

@NataCoachBot
Your Whoop link expired (they rotate these for security —
not your fault). One tap fixes it:

[ 🔗 Reconnect Whoop ]

No nagging: if ignored, the reconnect button is silently appended to the next Morning Brief for up to 5 days, after which the bot switches to conversational mode and Nata sees a "Whoop disconnected 5+ days" flag in the Coach Console roster.

Provider outage. Three consecutive failed fetches (or webhook silence past the expected window plus a failed poll) opens a circuit breaker for that provider. The Morning Brief still ships on time — it just opens with one question:

@NataCoachBot
Morning! Whoop is having a slow day, so quick check —
how did you sleep?

[ 😴 Great ]  [ 🙂 OK ]  [ 😩 Rough ]

One tap yields proxy sleep_quality/recovery_score (mapping in §5.2), the day proceeds normally, and when the provider recovers, backfill fills the gap: measured values replace proxies in daily_metrics, but per rule 4 the decisions already taken stand. The user experiences an outage as one extra tap, which is the entire point.

8. Conversational capture as a data source

The chat itself is Health Sync's third provider — the highest-trust one for subjective data. Everything captured conversationally becomes an event, exactly like a webhook payload.

Denys, Friday morning, travel week:

07:05  @NataCoachBot
Friday weigh-in 📉 Last week: 91.2 kg. Trend says ~90.9.
What does the hotel scale say?

[ ✓ 90.9 ]  [ ✏️ Enter number ]  [ No scale here — skip ]

07:07  Denys
✏️ 91.4

07:07  @NataCoachBot
91.4 logged. Up 0.2 on the week — travel weeks usually
read +0.5 from water and restaurant sodium, so the trend
line still points down. Nothing to fix. ✈️

The "skip" option is load-bearing: a skipped travel weigh-in generates a weight.skipped event, which teaches the Weekly Review to interpolate rather than nag — and tells the Wiki Brain that Denys's data goes quiet in hotels, which is itself part of his model.

9. Data inventory

The complete list. Column five enforces the rule: if it does not change a coaching decision, we do not collect it.

Deliberately not collected (each fails the rule at pilot scale): SpO₂, respiratory rate, skin temperature, stress scores, GPS routes, continuous heart-rate streams, wearable calorie-burn estimates (misleading enough to be actively harmful next to Meal Lens numbers), VO₂max, and menstrual-cycle data (would genuinely change coaching for Marta, but the privacy weight demands its own consent design — revisit in phase 2, noted in 11-roadmap.md).

Every inventory row flows the same pipeline: event (Postgres + raw/) → normalized metric (daily_metrics) → nightly distillation into the Wiki Brain — one architecture, regardless of whether the fact arrived from a Whoop webhook or a thumb tap.