# Script / transcript — text-based editing & navigation (cues)

Cues are the **text layer of a project**: an array of `{ id, from, to, text, kind, speaker, clip, sfx }`
on the `.mvideo.json` (`project.cues`), each bound to a timeline frame range. They power the editor's
**script mode** — a list of timestamped lines you can click to seek, that highlights the active line during
playback, and whose text you can edit inline. This is how you do podcast / "edit the words, not the
timeline" workflows. It's the same project file, not a separate one.

## The cue

```jsonc
{ "id": "q1", "from": 0, "to": 90, "text": "Every morning starts here.",
  "kind": "speech",           // speech | sfx | note
  "speaker": "ana",           // library entity id (character) → face + voice
  "clip": "c6",               // the audio clip that renders this line (once assembled)
  "sfx": "door creak" }        // for kind:"sfx"
```

Edit cues with `timeline_edit`: `add_cue{cue,at?}`, `update_cue{cueId,patch}`, `move_cue{cueId,from?,to?}`,
`remove_cue{cueId}`. (The editor's script mode also edits text/timing directly via the UI.)

## Staying in sync with the timeline

The script view and the visual timeline are two views of **one** project, kept consistent by a single rule:
**a cue's linked audio clip owns its timing.** A cue with `clip` set has its `from`/`to` snapped to that
clip after *every* edit (a `reconcileCues` pass runs at the end of each `timeline_edit` batch). So:

- Move / trim / split a clip in the timeline → the matching script line's timestamp updates.
- Retime a line in script mode (`move_cue`) → it retimes the linked clip, then snaps back to it — both move.
- Delete a clip → its cue is **unlinked** (the text stays as an authored, un-voiced line, never a silent orphan).
- Editing a line's **text/speaker** is metadata — it never moves anything.

An **unlinked** cue (no `clip` yet — authored text you haven't voiced) owns its own `from`/`to` until you
generate its audio, which creates a clip and sets `cue.clip`; from then on the clip is truth. Because both
views render from the same file and the editor auto-reloads on change, they can't drift.

## Two directions

### Audio → text (transcribe, then navigate/edit)
1. `transcribe_media { projectPath, path, clipId?, speaker? }` runs **local Whisper with word-level
   timestamps** (`-ml 1 -sow -oj`) and adds the text as cues on **real word boundaries** — each cue also
   carries per-word timings (`words[]`). (If the engine is text-only — a cloud STT without offsets — it
   falls back to proportional timing, marked `estimated`.)
2. The operator navigates by clicking lines (seek) and edits text inline.
3. To **cut** by text: `split_cue { cueId, at }` — because the cue has word timings, the split lands
   exactly on the chosen word and splits the linked audio clip at the same frame; then `remove_cue` +
   `remove_clip` the unwanted piece and later clips ripple. Accurate word-level clipping.

### Text → audio (write, assign voices, voice it — podcast assembly)
1. Write the lines as cues, each with a `speaker` (a character entity that carries a `voice`).
2. **`voice_cue { projectPath, cueId? }`** does the whole voicing in one call: generate the speech, add it
   as a `role:"voice"` audio-track clip, link the cue's `clip`, and **snap** the cue to the real spoken
   length (clearing the estimate). Omit `cueId` to voice **every** un-voiced speech line at once (whole-script
   assembly), laid out back-to-back without overlap (`sequential`, default true).
   - **Any available TTS engine.** It routes over the platform's TTS: local **Piper / Kokoro / macOS say**
     (free, offline), **Grok / xAI** (free cloud), or cloud **OpenAI / ElevenLabs**. With nothing specified
     it prefers free engines (local, then Grok) so voicing a script doesn't silently spend; force one with
     `engine` (piper/kokoro/say/grok/openai/elevenlabs).
   - **Consistent character voice.** Each cue's `speaker` (a library entity) carries a `voice`
     (`provider` + `voiceId` + model/settings); `voice_cue` uses it so that character sounds the **same**
     every line. Set it once in the library. **Grok default voices:** when voicing via Grok with no voice
     set, each character is auto-assigned a **distinct** preset (eve/ara/rex/sal/leo, skipping ones other
     characters already took) and it's persisted — so a cast gets consistent, differentiated voices for
     free without any setup. **Voice cloning:** give a character a `voice.sample` (a
     reference clip) with no `voiceId` and `voice_cue` clones it automatically the first time — on **Grok
     (xAI)** if `voice.provider` is xai/grok or that's the available free engine, else **ElevenLabs** —
     then writes the resulting `voiceId` (and provider) back to the library so every later line reuses that
     voice. (Grok's clone API is Enterprise-gated and US-only; if it 403s it falls back / you can create the
     voice in the xAI console and set its `voiceId` directly.)
   - **Cost.** Local engines are free; cloud is billed per character. It reports the estimate (summed over
     the cloud engines actually used) — **state that before voicing a long script**.
   - In the editor's **script view**, each un-voiced line has a **voice** button, and a **voice all** bar
     sits above the script; both route here.
3. **Inline SFX:** add `kind:"sfx"` cues (`sfx: "whoosh"`, or just text) between lines, then
   **`sfx_cue { projectPath, cueId? }`** — the mirror of `voice_cue` for the sound layer: it generates the
   foley (local Stable Audio model, **free**), drops it on a `role:"sfx"` track **at the cue's time**
   (timed events, overlap allowed — not laid out back-to-back), links the cue, and snaps it to the real
   length. Omit `cueId` to fill every un-filled sfx cue. The script view shows **make sfx** per cue and a
   **make all sfx** button.
4. **Assign a voice to a selection:** when the operator selects text and says "have Ana say this", set that
   cue's `speaker` (so its `voice` is used) and `voice_cue { cueId }` to (re)generate it.
5. **Consistent character voice:** a character entity's `voice.voiceId` (+ provider/model) is reused for
   every line they speak, so the same person always sounds the same. `voice_cue` picks it up from the cue's
   `speaker` automatically; set it once in the library.

### Music bed (scores a range, not a point)
Music is a **bed over a span**, not a cue. **`music_bed { projectPath, prompt, from?, to?, instrumental?,
lyrics? }`** generates it (local — free: Stable Audio instrumental, or ACE-Step for a vocal song) over a
frame range (default the **whole project**), lays it on a `role:"music"` track at a dialogue-safe level
(`gainDb`, default -18) with a ~1s fade in/out.
- **Ducking (default on):** one bed clip with **gain automation** (`gainKeyframes`) that ramps **smoothly**
  down to `duckDb` (default `gainDb - 8`) around each voice region and back up — a proper sidechain, with
  ~250ms attack/release. Interpolated identically by the preview and the Remotion renderer.
- If the range is longer than the model's max clip, the bed **loops** (tiles the track to fill the range,
  with a short fade at each seam). Trigger it from the composer (the mood is a freeform prompt), e.g.
  "score this with tense low strings, duck under the VO".

## Script entity references (the pre-production `.script.md`)

The Markdown script uses a light convention so people/places link to library entities and are clickable:

```markdown
## Scene 2 — The pour
**@ana:** Fresh, daily.            <!-- speaker → character entity "ana" -->
> [set: @cafe] warm oak interior   <!-- place → location entity "cafe" -->
```

`@<id>` (and `[set: @<id>]`) resolve against the nearest `library.json`. In script mode the speaker of each
cue shows as a clickable chip that opens the character in the Cast panel. When compiling a script to cues,
carry the `@speaker` into each cue's `speaker` and the `[set: …]` into the shot/clip `entities`.