trace-avatar-render 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/LICENSE +21 -0
- package/README.md +175 -2
- package/dist/chunk-BM26YQ4L.js +1729 -0
- package/dist/chunk-BM26YQ4L.js.map +1 -0
- package/dist/cli.js +324 -0
- package/dist/cli.js.map +1 -0
- package/dist/index.d.ts +829 -0
- package/dist/index.js +139 -0
- package/dist/index.js.map +1 -0
- package/package.json +71 -4
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 (first release)
|
|
4
|
+
|
|
5
|
+
- Render a trace-avatar guide to a **GIF** or an **MP4** by playing it on your real app in headless Chromium, in
|
|
6
|
+
Show me or Do it for me (`--allow-perform`). Signed-in sessions with `--storage-state`.
|
|
7
|
+
- Long waits are cut (`--max-wait`); a stalled guide fails the render (exit 1, with a screenshot) so docs GIFs that no
|
|
8
|
+
longer match the app fail CI; masked fields only ever get dots; guides that cross full page loads keep playing.
|
|
9
|
+
- Polish: a browser frame, a camera that zooms in on each action, burned-in or VTT captions and chapters, a poster,
|
|
10
|
+
title and end cards, `--hide` and `--blur`.
|
|
11
|
+
- AI (only with `--ai`; DeepSeek first through a gateway, or OpenAI-compatible, or your own `provider.js`): `--qa`
|
|
12
|
+
(a visual check of frames), `--alt`, and `suggest` / `apply` for captions, waits, a title and narration lines.
|
|
13
|
+
- Narration in the MP4 (`--voice`: a gateway's Gemini speech, OpenAI-compatible speech, macOS `say`, or your own
|
|
14
|
+
`voice.js`), with holds so speech never talks over an action, and a cache by voice and text.
|
|
15
|
+
- Repair (`--repair`): a step that can't find its element is matched to what the page shows now (code first, AI only
|
|
16
|
+
when unsure), re-recorded, and verified by playing the guide again; fixes go to `<out>.repair.json` for `apply`.
|
|
17
|
+
- `render()` from code, with `RenderError` (and its screenshot).
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Saidi Adegoke
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,176 @@
|
|
|
1
|
-
#
|
|
1
|
+
# trace-avatar-render
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Turn a [trace-avatar](https://github.com/saidiadegoke/trace-avatar) guide into a **GIF** or an **MP4** by playing it
|
|
4
|
+
on your real app in headless Chromium.
|
|
5
|
+
|
|
6
|
+
```bash
|
|
7
|
+
npm i -D trace-avatar-render # or run it with npx
|
|
8
|
+
npx playwright install chromium # once
|
|
9
|
+
npx trace-avatar-render invoice.trace.json --url http://localhost:3000/app --out invoice.gif
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Node 20 or later. GIFs need nothing else; MP4 needs ffmpeg (below).
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
invoice.gif 800×500 12 fps 9.6 s 210 KB
|
|
16
|
+
cut 4.2 s of waiting
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Because the guide is known before it plays, the render can do what a screen recorder can't:
|
|
20
|
+
|
|
21
|
+
- **Long waits are cut.** While the app loads or saves ("Saving…" → "Saved"), the player is waiting; anything
|
|
22
|
+
longer than `--max-wait` (default 0.8 s) keeps its first and last 0.4 s and loses the middle.
|
|
23
|
+
- **A stalled guide fails the render** (exit code 1, with a screenshot of the page at that moment). Run it in CI and
|
|
24
|
+
a GIF in your docs that no longer matches the app fails the build. `--allow-stall` renders it anyway.
|
|
25
|
+
- **Masked fields only ever get dots.** Do it for me hands a masked field to the viewer; the render types the
|
|
26
|
+
trace's bullets there, so no real text is ever in the app or the file.
|
|
27
|
+
- **A guide that crosses full page loads** keeps playing: the player is injected into every page.
|
|
28
|
+
|
|
29
|
+
## Options
|
|
30
|
+
|
|
31
|
+
| | |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `--url <url>` | the page the guide starts on (required) |
|
|
34
|
+
| `--out <file>` | `.gif` or `.mp4`, repeatable (required) |
|
|
35
|
+
| `--mode overlay\|perform` | Show me (default) or Do it for me |
|
|
36
|
+
| `--allow-perform` | needed for Do it for me: it clicks and types in the app for real. **Use a sandbox account.** |
|
|
37
|
+
| `--width <px>` | output width (GIF default 800, MP4 the capture's) |
|
|
38
|
+
| `--fps <n>` | GIF default 12, MP4 30 |
|
|
39
|
+
| `--viewport <WxH>` | default: the trace's `meta.viewport`, else 1280x800 |
|
|
40
|
+
| `--scale <n>` | device pixel ratio of the capture; 2 makes video text crisp |
|
|
41
|
+
| `--storage-state <file>` | a signed-in Playwright session ([how to save one](https://playwright.dev/docs/auth)) |
|
|
42
|
+
| `--max-wait <ms>` | waits longer than this are cut to it; 0 keeps them |
|
|
43
|
+
| `--hide <selector>` | hidden before capture (your dev toolbars); repeatable |
|
|
44
|
+
| `--blur <selector>` | blurred before capture, as well as `[data-trace-blur]` in your app; repeatable |
|
|
45
|
+
| `--allow-stall` | render a stalled guide instead of failing |
|
|
46
|
+
| `--player <file>` | the `trace-perform.min.js` to use (default: the installed trace-avatar; or set `TRACE_AVATAR_PLAYER`) |
|
|
47
|
+
| `--timeline <file>` | write the player's events and the cuts as JSON |
|
|
48
|
+
| `--timeout <ms>` | give up after this long (default 120000) |
|
|
49
|
+
| `--headed` | watch the browser |
|
|
50
|
+
|
|
51
|
+
**Polish** (all off by default):
|
|
52
|
+
|
|
53
|
+
| | |
|
|
54
|
+
|---|---|
|
|
55
|
+
| `--frame browser` | the app in a browser window (title bar with the page path and title) on a background |
|
|
56
|
+
| `--background <css>` | a colour or gradient behind the window and on the cards |
|
|
57
|
+
| `--camera` | zoom in on each click and typing step before it happens, pan between close ones, back out between bursts; never on scrolls, at most one zoom change per 1.5 s. Pair with `--scale 2` for crisp zooms |
|
|
58
|
+
| `--zoom <n>` | how far the camera goes in (default 1.6); implies `--camera` |
|
|
59
|
+
| `--captions burn\|vtt\|both` | `burn` draws the steps' words low in the picture (the player's own bubble is then off); `vtt` writes `<out>.vtt` and `<out>.chapters.vtt` beside the first output |
|
|
60
|
+
| `--poster [file]` | a PNG at the first click (default `<out>.png`) |
|
|
61
|
+
| `--title-card [title]` | 1.5 s first: the trace's title and the avatar, or your title; fades into the guide |
|
|
62
|
+
| `--end-card [title]` | 2 s last: "Done", or your text (e.g. a URL) |
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
trace-avatar-render invoice.trace.json --url http://localhost:3000/app --out invoice.mp4 \
|
|
66
|
+
--scale 2 --frame browser --camera --captions both --poster --title-card --end-card "acme.com/invoices"
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Exit codes: 0 rendered, 1 the render failed, 2 wrong usage.
|
|
70
|
+
|
|
71
|
+
**MP4 needs ffmpeg** on the PATH (or `FFMPEG=/path/to/ffmpeg`): `brew install ffmpeg`, `apt install ffmpeg`. GIF
|
|
72
|
+
needs nothing else (it uses [gifenc](https://github.com/mattdesl/gifenc), pure JS).
|
|
73
|
+
|
|
74
|
+
**AI** (nothing calls a model without `--ai`):
|
|
75
|
+
|
|
76
|
+
| | |
|
|
77
|
+
|---|---|
|
|
78
|
+
| `--ai <provider>` | `gateway`: an AI gateway that forwards each provider's own API (`AI_GATEWAY_URL`, `AI_GATEWAY_KEY`), DeepSeek first (usually the cheapest), then Gemini, then OpenAI; `gateway:<provider>[/<model>][,<provider>…]` for another order; `openai:<model>` (`OPENAI_API_KEY`; `OPENAI_BASE_URL` for any OpenAI-compatible server, e.g. Ollama), or `./provider.js` whose default export is `(task, input, schema, { instructions, images }) => json` |
|
|
79
|
+
| `--ai-vision <model>` | the model for `--qa`'s frames (the gateway's DeepSeek default is `deepseek-flash`) |
|
|
80
|
+
| `--qa [strict]` | sends a frame after each step and the last one (640 px) to the AI, which flags error messages, empty states, a stuck "Looking for …", personal data on screen or a wrong page. `strict` exits 1 when anything is flagged. Needs a vision model |
|
|
81
|
+
| `--alt` | writes `<out>.alt.txt`; with `--captions vtt` the AI names the chapters |
|
|
82
|
+
|
|
83
|
+
The renderer never changes a trace. Suggestions go to a file, and a person picks what goes in:
|
|
84
|
+
|
|
85
|
+
```sh
|
|
86
|
+
trace-avatar-render suggest invoice.trace.json --task captions --ai gateway --out suggestions.json
|
|
87
|
+
trace-avatar-render apply invoice.trace.json suggestions.json --accept 1,3 --out invoice.trace.json
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
What is sent and never sent is the same as in the studio ([trace-avatar's AI suggestions](https://github.com/saidiadegoke/trace-avatar#ai-suggestions-optional));
|
|
91
|
+
frames are sent only with `--qa`.
|
|
92
|
+
|
|
93
|
+
**Repair** (`--repair`): when a step can't find its element, the renderer looks at the page as it is now, picks
|
|
94
|
+
what the step means, and re-records that element as the step's target. Code scores the candidates first; an AI
|
|
95
|
+
is asked only when unsure (with `--ai`, DeepSeek first; `--repair-vision` adds a screenshot). Each fix is verified by
|
|
96
|
+
playing the guide again, up to 5 stalls in one run. The fixes go to `<out>.repair.json` with
|
|
97
|
+
`verified`, how each was chosen and, for an unresolved step, the best 3 candidates. The render still exits 1:
|
|
98
|
+
|
|
99
|
+
```sh
|
|
100
|
+
trace-avatar-render invoice.trace.json --url … --out invoice.gif --repair --ai gateway
|
|
101
|
+
trace-avatar-render apply invoice.trace.json invoice.repair.json --accept 1,2 --out invoice.trace.json
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
**Narration** (MP4; nothing speaks without `--voice`):
|
|
105
|
+
|
|
106
|
+
| | |
|
|
107
|
+
|---|---|
|
|
108
|
+
| `--voice <voice>` | `gateway`: Gemini speech through an AI gateway (`AI_GATEWAY_URL`, `AI_GATEWAY_KEY`; `gateway:gemini/<model>:<voice>`, default `gemini-2.5-flash-preview-tts`, Kore). DeepSeek would go first, but it has no speech API. `openai:<model>[:<voice>]`: `/audio/speech` at `OPENAI_BASE_URL` (OpenAI, or a local server such as Kokoro). `say[:<voice>]`: macOS, offline and free. Or `./voice.js`, whose default export is `(text) => ({ pcm: Int16Array, sampleRate })` |
|
|
109
|
+
| `--script <file.json>` | the spoken lines, `{ "lines": [{ "n": 1, "text": "…" }] }`; by default each step's caption is spoken. An empty line silences a step |
|
|
110
|
+
| `--voice-out <file.wav>` | also write the narration track |
|
|
111
|
+
| `--voice-strict` | fail when a line needs the video to hold longer than 8 s (otherwise a warning) |
|
|
112
|
+
| `--no-voice-cache` | spoken lines are kept in `.trace-avatar-cache/voice` by (voice, text), so a re-render pays only for changed lines |
|
|
113
|
+
|
|
114
|
+
Each line plays while the cursor travels to its step. Where a line is longer than that, the video holds on the frame
|
|
115
|
+
just before the step until the line ends, so the voice never talks over an action. Captions, chapters and the QA
|
|
116
|
+
frames move with the holds. GIFs from the same render keep the original timing. The script goes to
|
|
117
|
+
`<out>.narration.txt`. `suggest --task narration` drafts spoken lines from the steps (nothing typed is sent) for a
|
|
118
|
+
person to edit and pass with `--script`.
|
|
119
|
+
|
|
120
|
+
## From code
|
|
121
|
+
|
|
122
|
+
```js
|
|
123
|
+
import { render } from 'trace-avatar-render';
|
|
124
|
+
|
|
125
|
+
const r = await render({
|
|
126
|
+
trace: './invoice.trace.json', // or the trace object
|
|
127
|
+
url: 'http://localhost:3000/app',
|
|
128
|
+
mode: 'overlay',
|
|
129
|
+
out: ['invoice.gif', { file: 'invoice.mp4', fps: 30 }],
|
|
130
|
+
storageState: './auth.json',
|
|
131
|
+
frame: 'browser', camera: true, captions: 'both', poster: true, titleCard: true, // optional polish
|
|
132
|
+
});
|
|
133
|
+
r.durationMs; r.rawMs; r.outputs; // the video's length, as played, the files written
|
|
134
|
+
r.poster; r.vtt; r.chapters; // with those options
|
|
135
|
+
r.offsetMs + outputAt(r.segments, t); // where a moment of the capture lands in the video
|
|
136
|
+
// With an AI: render({ …, ai, qa: true, alt: true }) → r.qa (flags with times), r.alt (the .alt.txt file)
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
`render` rejects with a `RenderError` (with `.screenshot`) when the guide stalls, errors, or can't start.
|
|
140
|
+
|
|
141
|
+
## How it works
|
|
142
|
+
|
|
143
|
+
```
|
|
144
|
+
trace ──► headless Chromium at meta.viewport, trace-perform.min.js + an event logger in every page
|
|
145
|
+
│ player events (state, step, stalled, handoff, end) and page loads, timestamped
|
|
146
|
+
│ frames from Chromium's screencast, on the same clock
|
|
147
|
+
▼
|
|
148
|
+
edit list: keep everything except the middle of each long wait
|
|
149
|
+
▼
|
|
150
|
+
frames at the target fps
|
|
151
|
+
▼
|
|
152
|
+
compose: background + window (drawn once per page by Chromium), the camera's view of the capture,
|
|
153
|
+
a burned-in caption, title / end cards with a fade
|
|
154
|
+
▼
|
|
155
|
+
GIF (one palette, unchanged pixels transparent, written frame by frame) · MP4 (ffmpeg, raw RGBA in, H.264)
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
The camera is a pure function of the steps' positions and times, so the same trace gives the same camera.
|
|
159
|
+
|
|
160
|
+
A guide's own pause steps (and Do it for me's hand-offs) are held for 1.2 s, then it plays on: a video can't wait
|
|
161
|
+
for a viewer.
|
|
162
|
+
|
|
163
|
+
## Not yet
|
|
164
|
+
|
|
165
|
+
Planned: click sounds, other languages, a phone frame, both modes in one file. For a video without a server, trace-avatar's own
|
|
166
|
+
[Save as video](https://github.com/saidiadegoke/trace-avatar#save-as-video-in-the-browser) records the guide in the viewer's browser.
|
|
167
|
+
|
|
168
|
+
## Developing
|
|
169
|
+
|
|
170
|
+
`npm test` (unit), `npm run test:e2e` (real Chromium against `tests/e2e/fixtures`; the MP4 and narration tests run only
|
|
171
|
+
where ffmpeg is installed, as in CI). `npm run live:ai [-- deepseek openai gemini]` makes real calls through the AI
|
|
172
|
+
gateway (`AI_GATEWAY_URL`, `AI_GATEWAY_KEY`, or a git-ignored `.env.live`): captions, a title, alt text and a visual
|
|
173
|
+
check, per provider. `node scripts/readme-gifs.mjs ../trace-avatar` rebuilds trace-avatar's README GIFs from its
|
|
174
|
+
fresh build. `node scripts/gif-frames.mjs file.gif dir 2 6` dumps frames at those seconds to look at.
|
|
175
|
+
|
|
176
|
+
MIT
|