07 — Video Feedback (Form Check)

Form Check is the feature that most convincingly says "a coach is watching." It is also where a wrong answer costs the most — bad technique advice can hurt someone. So the design is deliberately conservative: a frontier vision LLM graded against checklists Nata authored, a hard cap of 2–3 cues, and a low threshold for putting the clip in front of the real Nata.

1. User flow: record, send, keep training

No special mode, no upload form. During or after a set — often prompted by a set card in Session Mode — the client props their phone, records one set, and sends the video into the same chat:

NataCoachBot  (footer under a set card, once per exercise per week)
🎥 Want a form check on this one? Film one set — side-on, whole body in frame.

The moment the video lands:

1. Instant ack (< 2 s): Got it 🎥 Checking your form — about 2 minutes. Keep going, I'll ping you. The session runner is not blocked; feedback arrives asynchronously.
2. The clip is queued as a BullMQ job (locked stack: Redis + BullMQ, see 03-system-architecture.md).

File-size reality (assumption checked against Telegram docs): Telegram clients can send videos up to 2 GB, but the hosted Bot API only lets bots download files ≤ 20 MB — a 45-second 1080p clip often exceeds that. We therefore run a self-hosted Bot API gateway (supported by grammY via a custom API root), which raises the download cap to 2 GB. Guidance to users stays simple: "15–60 seconds, one set, whole body in frame."

2. The pipeline

flowchart TD
 A["User sends video in chat"] --> B["Instant ack + BullMQ job"]
 B --> C["Download via self-hosted Bot API -> S3 (user's media prefix)"]
 C --> D["ffmpeg: 12 evenly spaced keyframes, 768 px, timestamped"]
 D --> E["Vision LLM (Opus-tier) grades frames vs Nata's exercise checklist"]
 E --> F{"Quality gate: exercise visible? angle usable?"}
 F -->|"unusable"| G["Ask for one better angle, with example image"]
 F -->|"usable"| H{"Confidence + red-flag scan"}
 H -->|"low confidence or red flag"| I["Nata queue in Coach Console"]
 H -->|"confident, no flags"| J["Persona layer: 1 praise + max 2 cues"]
 J --> K{"Autonomy dial for form_check"}
 K -->|"draft"| I
 K -->|"auto-send with audit"| L["Send feedback in chat"]
 I --> M["Nata approves, edits, or replies with voice note"]
 M --> L
 L --> N["form_check event -> Wiki Brain distillation"]

Keyframe sampling. ffmpeg extracts 12 evenly spaced frames (plus first/last), resized to 768 px on the long edge, each stamped with its timestamp. Twelve frames cover a 6–10-rep set at roughly one frame per rep — enough to catch positional faults (the only faults we claim to catch; see §4). Cost per check ≈ 20k input tokens ≈ $0.12 at Opus-tier (row 5 of the cost model in 03 §5) — comfortably inside the $2/user/day budget at a typical 2–3 checks/week.

Grading against Nata's checklist. Every exercise in the library carries a technique checklist Nata wrote once in the Coach Console. Example — squat pattern:

The vision LLM returns structured JSON — per checkpoint: rating (good / minor fault / major fault / not visible), evidence_frames (timestamps), confidence (0–1). The persona layer then renders the human message under hard output rules:

• Max 2–3 cues, prioritized safety > efficiency > aesthetics.
• Say what was GOOD first — always, and specifically ("reps 1–5 are textbook"), never generic praise.
• Nata's voice via the persona layer + the user's Wiki Brain (Denys gets blunt-but-kind; Marta gets encouraging).
• Continuity: the prompt includes the user's last form-check page from the Wiki Brain, so feedback can say "hips are staying higher than last time — that cue landed" and un-fixed faults from last time get re-checked before new ones are introduced. Surplus faults are stored in the wiki as a cue backlog, not dumped on the user.
• Optional annotated keyframe: pilot ships the raw evidence frame ("this is the rep I mean, 0:19"). Drawing overlays (spine line, load path) from LLM-specified coordinates is Phase 2 — coordinate reliability from vision LLMs is not yet good enough to draw lines on someone's spine (see 11-roadmap.md).

Quality gate. If the exercise isn't clearly gradable — cropped body, face-on angle for a hinge, gym mirror confusion — the pipeline never guesses. It asks for exactly one better take, with a reference image, and remembers the good angle per exercise so the ask happens once:

Bot  21:14  I can see the top half but your hips are out of frame, so I won't
            guess on this one. One more take from the side, phone ~3 m away,
            like this 👇 [reference image: side-on RDL framing]

3. Confidence and escalation: when the real Nata steps in

Per the Coach-in-the-loop autonomy dial, form_check starts the pilot at draft (Nata approves everything) and graduates to auto-send with audit around week 4 if edit rates fall below ~1 in 5. Independent of the dial, these conditions always route the clip to Nata's queue:

In the queue, Nata sees the clip, the 12 frames, the checklist JSON, and the drafted reply — approve / edit / discard takes under 30 seconds. Her third option is the high-value one: reply with a voice note. A 40-second "I watched your deadlift, here's the thing…" voice message is the strongest retention moment in the product and cannot be faked. Budget (per 09 §6.1 and 02 §3): one ~2-minute voice note per day, rotating or triggered, plus the weekly ~15-minute batch covering all 10 clients — roughly 3 minutes per client per week, tracked in the Console so it's spent on the moments that matter — first videos, red flags, PRs. Target SLA: escalated clips answered within 4 waking hours; auto-sent feedback within ~2 minutes.

Mini-example of the escalation path:

Marta  19:41  [video, 28 s]  "front rack felt weird on my wrist today"
Bot    19:41  Got it 🎥 — and since you mentioned your wrist, I'm having Nata look
              at this one personally. Skip the last front-rack set for today, ok?
Nata   20:55  [voice note, 0:37]  "Marta, привіт! Watched it — nothing scary, your
              wrist is just bending back because the elbows drop. Try two fingers
              under the bar… ok not bar, dumbbell 🙂 Show me Thursday."

How we know Form Check is working (pilot targets, measured weekly in the Console — all from events, per locked decision 6):

4. Explicitly not building: pose-estimation models (pilot)

Locked technical decision (brief §6): no custom pose-estimation / rep-counting models in the pilot. Why the checklist-plus-vision-LLM route wins now:

1. Coaching insight ≠ keypoints. MediaPipe-style skeletons give joint angles; a coach's judgment ("that rounding is fatigue on rep 7, stop at mid-shin") is a reasoning task the frontier vision model does well when anchored to Nata's checklist.
2. Robustness economics. Pose models need per-exercise tuning across camera angles, lighting, and hotel-room framing. At 10 users × ~2 clips/week (~20 clips/week), that engineering never pays back; the LLM handles messy real-world video natively.
3. Nata stays the author. Checklists are markdown she edits in the Console; improving the product means editing text, not retraining a model.
4. Graceful degradation. When the model can't see, it says so and asks for one better angle — a fine coaching interaction in itself.

Revisit trigger: > 50 clips/day, or a roadmap need for tempo/rep-count metrics that positional keyframes genuinely can't provide.

5. Privacy: the strictest media rules in the product

Videos of people training in bedrooms and hotel rooms are the most sensitive data we hold. Rules (enforced in 03-system-architecture.md, summarized here because users will ask in chat and the bot must answer accurately):

6. Worked example: Denys, hotel-room Romanian deadlift

Tuesday, travel week, hotel_gym profile active (see 06-training-experience.md §5). Autonomy dial for Denys's form checks: auto-send with audit — this clip was confident (lowest safety checkpoint 0.82, spine rated minor fault, no pain language), so it auto-sent and appeared in Nata's next audit feed without blocking.

Denys  21:12  [video, 34 s — hotel room, dumbbell RDL]
              "hotel gym maxes out at 24s. RDLs felt off, check?"
Bot    21:12  Got it 🎥 Checking your RDLs — about 2 minutes.
Bot    21:15  Form check — Dumbbell RDL, 2 × 24 kg
              Good news first: the hinge is real. Hips travel back, knees hold
              their angle, and reps 1–5 are textbook. Two things:
              1) From rep 6 the dumbbells drift ~10 cm in front of your thighs
                 (frame at 0:19 👇). Drag them down your legs like you're shaving
                 your shins — that alone takes the load off your lower back.
              2) On the last 2 reps your lower back rounds slightly at the bottom.
                 With the 24s and travel fatigue, stop at mid-shin. Depth is not
                 the goal here — tension is.
              [evidence keyframe, 0:19]
              Same weight Thursday, cue #1 only. That's the whole homework.
              [ 👍 Helpful ] [ Ask Nata ] [ 📌 Save this clip ]
Denys  21:17  shaving my shins 😂 ok

What the system recorded: a form_check event (exercise, checklist JSON, cues sent, confidence), and at nightly distillation wiki/training/form/rdl.md gains "fault: DBs drift forward under fatigue — cue: shave the shins (2026-08-04)" plus a cue-backlog entry for the minor rounding. Thursday's RDL set card will append: "Last time: shave the shins 🪒" — and his next video is checked against that cue first. That loop — cue, remember, verify — is what makes ten users feel like each one has Nata's full attention.

Related: 02-user-experience.md (where Form Check appears in daily flows) · 03-system-architecture.md (queues, storage, privacy enforcement) · 06-training-experience.md (Session Mode prompts and safety triage) · 08-llm-wiki-brain.md (cue backlog and continuity) · 09-admin-analytics.md (Nata's queue, autonomy dial, voice-note budget).