@analitikisler/ai-tag-debugger 0.8.2 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.10.0 (2026-10-16)
4
+
5
+ - Plan, then audit. `aitd plan --site <url>` drafts a measurement plan in its own folder, `aitd/<domain>/plans/<id>/`: the events a site should send, with a description, the page and element that trigger each one, its parameters (custom ones marked) with example values, and suggested user properties. Events the site already sends are marked `seen`. It browses one page of each type (home, a category, a product, cart, first checkout step, search, sign-up and contact forms) without paying or submitting personal details, and prints the page types it found and the ones the site doesn't have.
6
+ - `aitd run --against <plan>` checks a run against a plan. `<plan>` is `latest`, `first`, a plan ID or a path to a plan file or folder; with `--against` alone the journeys are derived from the plan's trigger pages (at most six), and with `--journey` or `--goal` only the events expected on that flow are checked. The run records the plan ID in its report. The same option works on the `run_audit` and `finish_audit` MCP tools.
7
+ - The report has a "Plan vs. site" section for runs with `--against`, at the top of `report.html` and `report.md`, with a coverage score (`Coverage 7 of 11 (64%), 2 events not checked`). Each planned event is marked Implemented, Missing, Missing params, Wrong values, or Not checked, and unplanned events are listed. The same results are written to `plan-check.json`. Without a plan the report is unchanged.
8
+ - Each plan folder has a `plan.html` page showing its events, parameters, user properties and the consent-mode flag.
9
+ - `aitd plan list --site <url>` lists the plan IDs for a site. `aitd plan export <plan> --site <url>` writes a plan to `plan.xlsx` (or `events.csv` and `user_properties.csv` with `--format csv`), and `aitd plan import <file> --site <url>` saves an edited spreadsheet as a new plan. Import never changes an existing plan; errors name the tab, row and fix.
10
+ - Runs are saved in `aitd/<domain>/runs/<id>/` (report, timeline, capture and plan check) with IDs in local time. `--data-dir <dir>` moves the `aitd/` folder; it replaces `run --out`, which still works as a hidden alias for one minor version.
11
+ - The help is redesigned: a short `aitd --help` with the commands and three examples, and one help text per command that lists the essential options first. `--no-ai` now implies `--scripted`, and the missing-key error names the environment variables to set.
12
+ - `aitd plan --site` prints the page types it found, and `aitd run` without `--journey`, `--goal`, `--against` or journeys in the `--config` file stops before the browser or the model starts, so a run can't wander and spend API usage.
13
+ - Fixed: `npm test` no longer keeps running for about 15 minutes after the MCP tests. A refused `finish_audit` now says the audit is still open and to call it again.
14
+
15
+ **Breaking changes**
16
+
17
+ - The default output folder is now `./aitd`. Runs and plans are no longer written to `reports/`.
18
+ - `aitd plan --from` is removed. Draft a plan with `aitd plan --site <url>` instead.
19
+ - `plan --out` and `plan --force` are removed. Each plan is saved in a new folder, so nothing is replaced.
20
+ - `run --out` is replaced by `--data-dir <dir>`.
21
+ - `--providers` is renamed `--platforms`. The old name still works for one minor version and prints a note.
22
+
23
+ ## 0.9.0 (2026-10-15)
24
+
25
+ - `aitd run ... --save-steps <file>` writes the steps each journey took into `journeys[].steps` of that config file (creating it if needed, and adding the run's plan when the config has none). A later `aitd run --config <file> --scripted --no-ai` repeats them with no AI model or API key. A journey run on several viewports keeps the steps of its first viewport.
26
+ - New purchase checks: a purchase sent with a value that is 0, negative or not a number (`purchase_value_invalid`), a currency that isn't a 3-letter ISO code such as USD (`purchase_currency_invalid`), or no items (`purchase_items_empty`). GA4 and Meta reporting the same order with a different value or currency is flagged as `purchase_mismatch`; purchases are paired by order ID, or taken as one order when the journey has a single purchase on each.
27
+
3
28
  ## 0.8.2 (2026-10-15)
4
29
 
5
30
  - A journey that fails with an error, such as the AI provider's API refusing a request or the browser crashing, no longer ends the whole audit. That journey is marked unfinished with the error as its note, what it captured until then is kept, the other journeys still run, and the report is written. This applies to the CLI and to `run_audit` over MCP.
package/README.md CHANGED
@@ -25,7 +25,7 @@ Built by [Analitik İşler](https://analitikisler.com). MIT licensed.
25
25
  ✓ All scripted steps completed. · 20 tracking hits captured
26
26
 
27
27
  2 broken · 1 risk · 1 warning · 4 ok
28
- Report: reports/demo/report.html
28
+ Report: reports/demo/localhost/runs/2026-10-16_0800/report.html
29
29
  ```
30
30
 
31
31
  ## Why
@@ -40,18 +40,23 @@ Manual tag QA catches this, but it's tedious, so it rarely happens after every r
40
40
  |---|---|
41
41
  | Missing events | `purchase` never fired on the checkout journey |
42
42
  | Missing parameters | `purchase` sent without `value` and `currency` on mobile |
43
+ | Invalid purchase value | `purchase` sent with `value` `0`, a negative number or text such as `"free"` |
44
+ | Invalid purchase currency | `purchase` sent with currency `usd` or `TL` instead of a 3-letter ISO code such as `USD` or `TRY` |
45
+ | Empty purchase items | `purchase` sent with an empty `items` list |
46
+ | Purchase mismatch | GA4 sends `value` 120 and Meta sends 115 for the same order ID |
43
47
  | Duplicates | `add_to_cart` sent twice by one click |
44
48
  | Consent | Meta `PageView` fired before the visitor accepted cookies, Consent Mode default missing or set too late, Consent Mode v2 signals (`ad_user_data`, `ad_personalization`) missing, tracking cookies set before consent, tags or cookies still active after the `decline_cookies` journey rejects the banner, a reject button that does nothing |
45
49
  | Personal data | An email address sent to an ad platform in plain text |
46
50
  | Unplanned events | Events firing that aren't in your measurement plan |
51
+ | Plan coverage | Each planned event sorted into implemented, missing, missing parameters, wrong values or not checked (see [Plan vs. site](#plan-vs-site)) |
47
52
 
48
53
  Platforms: GA4 (including server-side GTM endpoints and Google tag gateway paths on the site's own domain; hits to a server container are marked server-side, and Stape and Elevar requests are recognised), Google Ads conversions, Meta Pixel, TikTok Pixel, and basic X Pixel events.
49
54
 
50
- ### Choose the providers to check
55
+ ### Choose the platforms to check
51
56
 
52
- By default an audit checks GA4, Google Ads, Meta and TikTok. Pick others with `--providers ga4,meta,x`, `"providers"` in the config, or `providers` on the MCP `start_audit` and `run_audit` tools. You can choose from `ga4`, `google_ads`, `meta`, `tiktok`, `x`, `linkedin`, `microsoft_ads`, `pinterest`, `snapchat`, `clarity`, `hotjar`, `doubleclick` and `adsense`. Events are parsed for GA4, Google Ads, Meta, TikTok and X; the others show their network requests.
57
+ By default an audit checks GA4, Google Ads, Meta and TikTok. Pick others with `--platforms ga4,meta,x`, `"providers"` in the config, or `providers` on the MCP `start_audit` and `run_audit` tools. You can choose from `ga4`, `google_ads`, `meta`, `tiktok`, `x`, `linkedin`, `microsoft_ads`, `pinterest`, `snapchat`, `clarity`, `hotjar`, `doubleclick` and `adsense`. Events are parsed for GA4, Google Ads, Meta, TikTok and X; the others show their network requests. `--providers` still works as the old name of `--platforms` for one more minor version and prints a note.
53
58
 
54
- Requests to providers you didn't choose are still answered locally, so they never reach those accounts either. They are left out of the timeline, the CSV and the checks, and only counted: the report lists them with their request counts (for example "61 out-of-scope requests were skipped: Microsoft Clarity, DoubleClick"), and each step says how many it skipped. Without `--providers`, any provider your measurement plan names is added to the default list. Planned events for providers you did leave out are listed in one info finding instead of being reported as missing. Tag manager and consent platform requests are always shown.
59
+ Requests to platforms you didn't choose are still answered locally, so they never reach those accounts either. They are left out of the timeline, the CSV and the checks, and only counted: the report lists them with their request counts (for example "61 out-of-scope requests were skipped: Microsoft Clarity, DoubleClick"), and each step says how many it skipped. Without `--platforms`, any platform your measurement plan names is added to the default list. Planned events for platforms you did leave out are listed in one info finding instead of being reported as missing. Tag manager and consent platform requests are always shown.
55
60
 
56
61
  ## How it works
57
62
 
@@ -114,7 +119,7 @@ The global install also adds `aitd`, a short alias: `aitd run --site … --journ
114
119
 
115
120
  Or run it without installing: `npx -y @analitikisler/ai-tag-debugger run --site … --journey add_to_cart`.
116
121
 
117
- Run `ai-tag-debugger --help` to see every command and option, such as `--providers`, `--scripted` and `--no-ai`.
122
+ Run `aitd --help` to see the commands, and `aitd run --help` (or `plan`, `replay`, `mcp`) for each command's options.
118
123
 
119
124
  ### Try the demo shop (from source)
120
125
 
@@ -126,21 +131,107 @@ cd ai-tag-debugger
126
131
  npm install
127
132
  npx playwright install chromium # run again after upgrading, so the browser matches Playwright
128
133
  npm run demo:serve # in one terminal: http://localhost:4321
129
- node src/cli.js run --config examples/demo.config.json --out reports/demo
130
- open reports/demo/report.html
134
+ node src/cli.js run --config examples/demo.config.json --data-dir reports/demo
135
+ open reports/demo/localhost/runs/*/report.html
131
136
  ```
132
137
 
133
138
  No API key yet? Replay the fixed steps without any AI model:
134
139
 
135
140
  ```bash
136
- node src/cli.js run --config examples/demo.config.json --scripted --no-ai --out reports/demo
141
+ node src/cli.js run --config examples/demo.config.json --scripted --no-ai --data-dir reports/demo
137
142
  ```
138
143
 
144
+ ## Plan, then audit
145
+
146
+ A run checks what a site sends against a **plan**: the events, parameters and user properties you expect. Three steps:
147
+
148
+ ```bash
149
+ # 1. Draft a plan. The AI explores the site and proposes what should be tracked.
150
+ aitd plan --site https://yoursite.com
151
+
152
+ # 2. Optional: edit it in a spreadsheet, then save the edited copy as a new plan.
153
+ aitd plan export latest --site https://yoursite.com # writes plan.xlsx next to the plan
154
+ aitd plan import plan.xlsx --site https://yoursite.com # saves a new plan; the old one is untouched
155
+
156
+ # 3. Audit the site against the plan.
157
+ aitd run --site https://yoursite.com --against latest
158
+ ```
159
+
160
+ **Drafting a plan** needs no tracking code. The AI explores one page of each type: the home page, a category, a product, the cart, the first checkout step, search, and any sign-up or contact forms. It never pays, and it never submits a form with personal details. `--provider`, `--model` and `--max-steps` (default 25, the most AI steps) work as they do for `run`. The output prints "Page types found" and "Not present on this site", so you can see which page types the AI reached. A small site, with only 2 pages, is fine.
161
+
162
+ The plan is saved in a new folder, `aitd/<domain>/plans/<id>/`, as `measurement-plan.json` and a readable `plan.html`. For each event it lists:
163
+
164
+ - `name` and `platform`. Events specific to your site, such as a size guide or a store locator, are marked `custom: true`.
165
+ - `description` and `trigger`: the page type and the visible label of the element that fires the event.
166
+ - `parameters`: each with a description and an example value. Custom parameters are marked. `required_params` lists the ones that must be present.
167
+ - `user_properties`: suggested ones such as `customer_type`, each with a description and an example.
168
+
169
+ Events your site already sends are marked `"seen": true`.
170
+
171
+ Review the plan before you use it. The examples and suggestions come from the pages the AI saw, so check that each event fits your business. The extra fields are ignored by the checks.
172
+
173
+ **Spreadsheet import and export.** `aitd plan export <plan> --site <url>` writes `plan.xlsx` next to the plan, with one row per event parameter on an `events` tab (`event, platform, parameter, required, example, trigger, description`) and one row per user property on a `user_properties` tab. Add `--format csv` to get `events.csv` and `user_properties.csv` instead; CSV needs no extra package. `.xlsx` uses one small library, `fflate`, which is installed with the tool as an optional dependency. `aitd plan import <file> --site <url>` reads the spreadsheet back and saves it as a new plan folder. Import never edits an existing plan. Unknown columns are ignored with a warning, and an error names the tab, row and fix, for example `events, row 7: "required" must be yes or no (found "maybe").`
174
+
175
+ `measurement-plan.json` looks like this:
176
+
177
+ ```json
178
+ {
179
+ "consent_mode": true,
180
+ "events": [
181
+ { "platform": "ga4", "name": "add_to_cart", "required_params": ["currency", "value", "items"] },
182
+ { "platform": "ga4", "name": "purchase", "journeys": ["checkout"], "required_params": ["transaction_id", "currency", "value", "items"] },
183
+ { "platform": "meta", "name": "Purchase", "journeys": ["checkout"], "required_params": ["value", "currency"] }
184
+ ]
185
+ }
186
+ ```
187
+
188
+ `platform` is `ga4`, `google_ads`, `meta` or `tiktok`. `journeys` limits where an event is expected (default: every journey). Set `allow_multiple: true` for events that may legitimately fire more than once per action.
189
+
190
+ Event names are matched loosely, so `dl_add_to_cart` (Elevar), `addToCart` and `Add To Cart` count as `add_to_cart`, and `transaction` or `order_completed` as `purchase`. Add your own with `"aliases": ["my_event"]` on a planned event. Required parameters are also found under their usual other names: `products` for `items`, `revenue` for `value`, `order_id` for `transaction_id`.
191
+
192
+ **Auditing against a plan.** `run --against <plan>` takes `latest` (or `last`), `first`, a plan ID, or a path to a plan file or its folder. When `--site` is not given, the site comes from the plan. Give the run a journey as well and it walks only that flow, checking only the plan events expected there; the rest of the plan is reported as not checked, never as missing. Used alone, `--against` derives up to six journeys from the plan: it groups the planned events by the page they are triggered on and makes one journey per page, at most six (the smallest groups share the last journey), with no extra AI call. The chosen journeys are printed before the browser starts. Every report header then says `Checked against plan <id>`.
193
+
194
+ `run` needs something to test: `--against`, `--journey`, `--goal`, or journeys in your config file. Without one, it stops before opening the browser with `Nothing to test. Add --against latest, --journey <id> or --goal "<text>".`, so it can't wander and spend API usage. If a site has no plan yet, `run --against` says `No plan yet for example.com. Draft one with: aitd plan --site https://example.com`.
195
+
196
+ `aitd plan list --site <url>` lists the site's plan IDs, oldest first.
197
+
198
+ ### Plan vs. site
199
+
200
+ When a run uses `--against`, the report starts with a **Plan vs. site** section, in `report.html` and `report.md`. Each planned event lands in one bucket:
201
+
202
+ | Bucket | Meaning |
203
+ |---|---|
204
+ | Implemented | Planned, fired, all required parameters present, no value problems |
205
+ | Missing | Planned for a journey the run walked, but never fired |
206
+ | Missing params | Fired, but a required parameter is absent |
207
+ | Wrong values | Fired with a value problem, from the same checks as purchase value, currency and items |
208
+ | Unplanned | Fired, but isn't in the plan |
209
+ | Not checked | Planned, but its journey or platform wasn't part of this run |
210
+
211
+ The coverage score is implemented divided by the planned events that were checkable (all except not checked), shown overall and per platform, for example `Coverage 7 of 11 (64%), 2 events not checked`. The same entries are in `plan-check.json`, one per planned event with its bucket. Exit codes don't change: `--fail-on` still looks at finding severity. Without `--against`, the report is exactly as before.
212
+
213
+ ### Folder layout and plan IDs
214
+
215
+ Plans and runs are kept in `aitd/` in the folder you run the tool from:
216
+
217
+ ```
218
+ aitd/
219
+ example.com/ the site's host, without www.
220
+ plans/2026-10-16_0900/ measurement-plan.json, plan.html, plan.xlsx (after an export)
221
+ runs/2026-10-16_0912/ report.html, report.md, timeline.csv, capture.json
222
+ and plan-check.json (with --against)
223
+ ```
224
+
225
+ An ID is the local time, `YYYY-MM-DD_HHMM`. When that minute is already taken, seconds are added (`_HHMMSS`). Nothing is ever replaced: each draft, import and run gets a new folder. Use `--data-dir <dir>` to keep the `aitd/` folder somewhere else. `replay` and `--save-site` folders are not affected.
226
+
139
227
  ## Choose what to test
140
228
 
141
- You don't need a config file to start. Pick ready-made journeys, describe your own in plain language, and narrow the check to the events you care about:
229
+ You don't need a config file to start. Pick ready-made journeys, describe your own in plain language, check against a plan, and narrow the check to the events you care about:
142
230
 
143
231
  ```bash
232
+ # Check the site against its latest saved plan.
233
+ aitd run --site https://staging.yourshop.com --against latest
234
+
144
235
  # Ready-made journeys come with a goal and the GA4 and Meta events to expect.
145
236
  ai-tag-debugger run --site https://staging.yourshop.com --journey purchase --journey sign_up
146
237
 
@@ -161,6 +252,8 @@ ai-tag-debugger run --config ai-tag-debugger.config.json --events purchase,add_t
161
252
 
162
253
  `--journey` also picks journeys from your config by id. With a config, ready-made journeys add their events to your plan.
163
254
 
255
+ Not sure what to test? `aitd plan --site https://yoursite.com` drafts a measurement plan, and `--against latest` checks the site against it.
256
+
164
257
  ### Keeping API usage down
165
258
 
166
259
  Each browsing step is one model call, so usage grows with **journeys × viewports × steps**, plus one call at the end to explain the findings. The checks themselves never call the model.
@@ -170,7 +263,7 @@ Each browsing step is one model call, so usage grows with **journeys × viewport
170
263
  - **Cap the steps.** `--max-steps 15` stops a journey that wanders.
171
264
  - **Expect bigger pages to cost more.** Each step sends the page view (up to 250 elements and 3,000 characters of text) to the model, so busy pages use more tokens per step than simple ones.
172
265
  - **Use `--events`** to keep the report focused. It doesn't shorten browsing, so it barely changes usage.
173
- - **Replay without AI.** After one AI run, copy the steps into `journeys[].steps` and use `--scripted` (with `--no-ai` for zero model calls) on every deploy.
266
+ - **Replay without AI.** After one AI run, add `--save-steps <file>` to save its steps, then run that config with `--scripted --no-ai` on every deploy. See [Save the steps and replay them without AI](#save-the-steps-and-replay-them-without-ai).
174
267
  - **Use a chat app.** Over MCP, the chat app's own model does the browsing, so no API key is used at all.
175
268
 
176
269
  ### Check a new version against saved sites
@@ -186,6 +279,17 @@ npx @analitikisler/ai-tag-debugger replay saved/*
186
279
 
187
280
  The first run's findings become the expected ones. When a change is right, for example a false alarm that is fixed, `replay --update` makes the new findings the expected ones. Use it to check a new version of the tool against sites you know, or a site change against its last good run. A saved site is usually a few MB. It leaves out tracking requests, images and video, HttpOnly and login cookies and the cookies the browser sent, authorization headers, form data, and requests with an email or other user data in their address. It still holds the site's pages and scripts, so keep sites you don't own out of public repositories.
188
281
 
282
+ ### Save the steps and replay them without AI
283
+
284
+ `--save-steps <file>` writes the steps each journey took into `journeys[].steps` of a config file. It creates the file if it's missing, and it also saves the run's plan when the config has none. The same config then repeats the journey with no AI model and no API key:
285
+
286
+ ```bash
287
+ ai-tag-debugger run --site https://www.yourshop.com --journey purchase --save-steps ai-tag-debugger.config.json
288
+ ai-tag-debugger run --config ai-tag-debugger.config.json --scripted --no-ai
289
+ ```
290
+
291
+ When a journey ran on both viewports, only the first viewport's steps are kept. Keep the config in your repository and run the second command on every deploy to check the same journey without any model calls.
292
+
189
293
  ## Configure your site
190
294
 
191
295
  `ai-tag-debugger.config.json`:
@@ -208,15 +312,15 @@ The first run's findings become the expected ones. When a change is right, for e
208
312
  | Field | Meaning |
209
313
  |---|---|
210
314
  | `site` | Base URL. Use staging when you can. |
211
- | `plan` | Measurement plan: a file path or an inline object (below). Optional with ready-made journeys. |
315
+ | `plan` | Measurement plan: a file path or an inline object (below). Optional with ready-made journeys. A run with `--against` uses that plan instead. |
212
316
  | `gtm_container` | Optional GTM container export (Admin → Export container), so explanations can name tags and triggers. |
213
317
  | `viewports` | `desktop` and/or `mobile`. |
214
- | `journeys` | Optional when you pick ready-made journeys with `--journey` or describe one with `--goal`. |
318
+ | `journeys` | Optional when you pick ready-made journeys with `--journey`, describe one with `--goal`, or check against a plan with `--against`. |
215
319
  | `journeys[].goal` | What the AI should do, in plain language. |
216
320
  | `journeys[].start` | Optional start path (default `/`). |
217
321
  | `journeys[].steps` | Optional fixed steps for `--scripted` runs: `{ "action": "click", "selector": "#add-to-cart" }`. |
218
322
  | `allowed_hosts` | Other hosts journeys may visit, e.g. `["checkout.payments-provider.com"]` for a hosted checkout. The browser otherwise stays on the `site` host and its subdomains. Same as `--allow-host`. |
219
- | `providers` | Tracking providers to check, e.g. `["ga4", "meta"]`. Default `["ga4", "google_ads", "meta", "tiktok"]`. Same as `--providers`. |
323
+ | `providers` | Tracking platforms to check, e.g. `["ga4", "meta"]`. Default `["ga4", "google_ads", "meta", "tiktok"]`. Same as `--platforms`. |
220
324
  | `block_hits` | Default `true`. Set to `false` to let hits reach your real endpoints. |
221
325
  | `language` | Report language, `en` (default) or `tr`. Same as `--lang`. |
222
326
  | `keep_bodies` | Default `false`. Set to `true` to store tracking requests as sent, without redacting user data. Same as `--keep-bodies`. |
@@ -224,55 +328,76 @@ The first run's findings become the expected ones. When a change is right, for e
224
328
  | `provider` | Optional: `anthropic`, `openai`, `gemini` or `openai-compatible`. Same as `--provider`. |
225
329
  | `base_url`, `model` | Optional: the API address and model id, e.g. for `openai-compatible`. Same as `--base-url` and `--model`. |
226
330
 
227
- `measurement-plan.json`:
228
-
229
- ```json
230
- {
231
- "consent_mode": true,
232
- "events": [
233
- { "platform": "ga4", "name": "add_to_cart", "required_params": ["currency", "value", "items"] },
234
- { "platform": "ga4", "name": "purchase", "journeys": ["checkout"], "required_params": ["transaction_id", "currency", "value", "items"] },
235
- { "platform": "meta", "name": "Purchase", "journeys": ["checkout"], "required_params": ["value", "currency"] }
236
- ]
237
- }
238
- ```
239
-
240
- `platform` is `ga4`, `google_ads`, `meta` or `tiktok`. `journeys` limits where an event is expected (default: every journey). Set `allow_multiple: true` for events that may legitimately fire more than once per action.
241
-
242
- Event names are matched loosely, so `dl_add_to_cart` (Elevar), `addToCart` and `Add To Cart` count as `add_to_cart`, and `transaction` or `order_completed` as `purchase`. Add your own with `"aliases": ["my_event"]` on a planned event. Required parameters are also found under their usual other names: `products` for `items`, `revenue` for `value`, `order_id` for `transaction_id`.
243
-
244
331
  ## CLI
245
332
 
246
333
  ```
247
- ai-tag-debugger run [--site <url>] [--config <file>] [--journey <id>] [--goal "<text>"] [--events <names>]
248
- [--max-steps <n>] [--out <dir>] [--viewport desktop|mobile] [--providers <list>]
249
- [--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--base-url <url>]
250
- [--lang en|tr] [--allow-host <host>] [--keep-bodies] [--scripted] [--no-ai] [--fail-on broken|risk|warning|never] [--headed]
251
- [--save-site <dir>]
252
- ai-tag-debugger replay <dir>... [--update] [--out <dir>]
253
- ai-tag-debugger mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
254
- [--allow-private-network]
334
+ aitd plan --site <url> [--headed] [--provider <name>] [--model <id>] [--max-steps <n>] [--data-dir <dir>]
335
+ aitd plan list --site <url> [--data-dir <dir>]
336
+ aitd plan import <file> --site <url> [--data-dir <dir>]
337
+ aitd plan export <plan> --site <url> [--format xlsx|csv] [--data-dir <dir>]
338
+ aitd run --site <url> (--against <plan> | --journey <id> | --goal "<text>") [--config <file>] [--events <names>]
339
+ [--max-steps <n>] [--viewport desktop|mobile] [--platforms <list>] [--data-dir <dir>]
340
+ [--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--lang en|tr] [--headed]
341
+ [--fail-on broken|risk|warning|never] [--scripted] [--no-ai] [--save-site <dir>] [--save-steps <file>]
342
+ aitd replay <dir>... [--update] [--out <dir>]
343
+ aitd mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
344
+ [--allow-private-network]
255
345
  ```
256
346
 
257
- `aitd` is a short alias for `ai-tag-debugger` and takes the same commands and options.
347
+ `aitd` is a short alias for `ai-tag-debugger` and takes the same commands and options. Run `aitd <command> --help` for the essential options of each command.
348
+
349
+ ### All options
350
+
351
+ | Option | Commands | Meaning |
352
+ |---|---|---|
353
+ | `--site <url>` | plan, run | The site. Required for `plan`; for `run`, needed unless `--against` or the config gives it. |
354
+ | `--against <plan>` | run | Check against a plan: `latest` (or `last`), `first`, a plan ID, or a path to a plan file or folder. |
355
+ | `--journey <id>` | run | A ready-made journey (repeatable). |
356
+ | `--goal "<text>"` | run | A journey in plain language (repeatable). |
357
+ | `--config <file>` | run | Take the site, journeys and plan from a config file. |
358
+ | `--events <names>` | run | Only check these events, e.g. `purchase,add_to_cart`. |
359
+ | `--viewport <name>` | run | `desktop` or `mobile` (repeatable). |
360
+ | `--platforms <list>` | run | Platforms to check, e.g. `ga4,meta`. Replaces `--providers`, which still works for one more minor version. |
361
+ | `--max-steps <n>` | plan, run | The most AI steps per journey (default 25 for `plan`). |
362
+ | `--lang <code>` | run | Report language: `en` or `tr`. |
363
+ | `--fail-on <level>` | run | Exit with 1 at this level: `broken` (default), `risk`, `warning` or `never`. |
364
+ | `--headed` | plan, run | Show the browser window. |
365
+ | `--provider <name>`, `--model <id>` | plan, run | The AI provider and model. Default: whichever API key is set. |
366
+ | `--scripted` | run | Replay each journey's fixed steps, with no AI navigation. |
367
+ | `--no-ai` | run | No AI model at all. Implies `--scripted`. |
368
+ | `--save-site <dir>` | run | Save the site's responses for `replay`. |
369
+ | `--save-steps <file>` | run | Write the steps each journey took into a config file. |
370
+ | `--data-dir <dir>` | plan, run | Where `aitd/` (plans and runs) is kept. Default: `./aitd`. Replaces `run --out`. |
371
+ | `--format xlsx\|csv` | plan export | The spreadsheet format. Default: `xlsx`. |
372
+ | `--update` | replay | Make this replay's findings the expected ones. |
373
+ | `--out <dir>` | replay | Where to write the replay reports. For `run`, `--out` still works as a hidden alias for one minor version and prints a note; use `--data-dir`. |
374
+ | `--base-url <url>` | run | The API address for an OpenAI-compatible provider. |
375
+ | `--allow-host <host>` | run | Another host journeys may visit (repeatable). Same as `allowed_hosts`. |
376
+ | `--keep-bodies` | run | Store tracking requests as sent, without redacting user data. |
377
+ | `--no-branding` | plan, run | Leave the logo and credit line out of the reports and `plan.html`. |
378
+ | `--http`, `--port`, `--host`, `--token`, `--public-url`, `--reports`, `--allow-private-network` | mcp | See the MCP section below. |
379
+
380
+ Removed in 0.10.0: `plan --from`, `plan --out` and `plan --force`. Drafting a plan always saves a new folder, and a removed flag prints what to run instead.
258
381
 
259
382
  The exit code is `1` when findings reach `--fail-on` (default `broken`), so it can gate a deploy. See [`examples/github-actions/ai-tag-debugger.yml`](examples/github-actions/ai-tag-debugger.yml) for running it after every deployment.
260
383
 
261
384
  Output:
262
385
 
263
386
  - `report.html`: one file with four tabs. Links work across tabs (a finding's "Where" chip opens that step), and every view has its own URL you can bookmark or share.
264
- - **Overview**: problem counts, the AI summary (most important problem first, plus things worth checking that the findings don't cover), the journeys with their problem counts, and the providers checked.
387
+ - **Overview**: problem counts, the AI summary (most important problem first, plus things worth checking that the findings don't cover), the journeys with their problem counts, the providers checked, and, with `--against`, the [Plan vs. site](#plan-vs-site) coverage.
265
388
  - **Findings**: grouped by severity and filterable. Each one has its impact, likely cause, evidence, numbered fix steps and the steps where it happened.
266
389
  - **Journeys**: one journey at a time, with its steps on the left and the selected step on the right. It opens on the first step with a problem. Each step has three tabs:
267
390
  - **Events**: each tracking hit with its parameters, consent state, the dataLayer push and request behind it, and a note when a finding points at it.
268
391
  - **dataLayer**: every push in the step.
269
- - **Network requests**: every request to the chosen providers, decoded into a parameter table with a `?` tip on each parameter. A batched GA4 request shows each of its events.
270
- - **Scope and plan**: whether each planned event was sent on each journey, and the providers checked or left out, with request counts.
392
+ - **Network requests**: every request to the chosen platforms, decoded into a parameter table with a `?` tip on each parameter. A batched GA4 request shows each of its events.
393
+ - **Scope and plan**: whether each planned event was sent on each journey, and the platforms checked or left out, with request counts.
271
394
 
272
395
  Events and requests can be filtered by source. GA4 item strings (`pr1`…) and dataLayer `ecommerce.items` are shown as item cards, with custom item parameters in their own group. The report has no scripts: tabs and filters are plain HTML and CSS (they need Chrome 105, Safari 15.4 or Firefox 121 or newer), and printing shows every tab. The logos are embedded, so nothing is loaded from the internet. `--no-branding` leaves out the Analitik İşler logo and credit line.
273
396
  - `timeline.csv`: the same timeline as one row per hit, dataLayer push and network call, ready for Excel or Google Sheets.
274
397
  - `report.md`: the report in Markdown, e.g. for a pull request comment.
275
398
  - `capture.json`: everything captured, for your own analysis.
399
+ - `plan-check.json`: with `--against`, one entry per planned event with its bucket.
400
+ - `plan.html` (in the plan's folder): a readable page of the plan's events, parameters and user properties.
276
401
 
277
402
  Reports end with a credit line, "Generated by AI Tag Debugger · analitikisler.com", where the domain is a plain link (no images, no tracking, and no referrer, so report URLs aren't leaked). `--no-branding` removes it along with the logo.
278
403
 
@@ -280,17 +405,19 @@ Use `--lang tr` for a Turkish report. Finding texts, headings, and the model's e
280
405
 
281
406
  ## Use it from your AI chat app (MCP)
282
407
 
283
- `ai-tag-debugger mcp` starts a [Model Context Protocol](https://modelcontextprotocol.io) server. Your chat app's model then runs the audit itself: you ask "check the tracking on the checkout of staging.myshop.com" and it opens the browser, clicks through, and explains the findings in the chat. No API key is needed, because the app's own model does the thinking.
408
+ `aitd mcp` starts a [Model Context Protocol](https://modelcontextprotocol.io) server. Your chat app's model then runs the audit itself: you ask "check the tracking on the checkout of staging.myshop.com" and it opens the browser, clicks through, and explains the findings in the chat. No API key is needed, because the app's own model does the thinking.
284
409
 
285
410
  Tools the server offers:
286
411
 
287
412
  | Tool | What it does |
288
413
  |---|---|
289
- | `start_audit` | Opens a fresh browser on a URL and starts recording all tracking. Pass a ready-made journey (`purchase`, `sign_up`, …) to get its goal and expected events, and `providers` to choose what to check. The assistant asks you which platforms to check first. |
414
+ | `start_audit` | Opens a fresh browser on a URL and starts recording all tracking. Pass a ready-made journey (`purchase`, `sign_up`, …) to get its goal and expected events, and `providers` to choose the platforms to check. The assistant asks you which platforms to check first. |
290
415
  | `browser_action` | One click, fill, select, goto, scroll or wait. Returns the page and the tracking hits that step sent. |
291
416
  | `get_page` | Shows the current page. |
292
- | `finish_audit` | Checks everything against your measurement plan (JSON) or the ready-made journey's events, optionally only the `events` you name, and writes the full HTML report with the timeline. |
293
- | `run_audit` | Runs a config file end to end (local only). |
417
+ | `finish_audit` | Checks everything against your measurement plan (JSON), a saved one given as `against` (`latest`, `first`, a plan ID, or on a local server a path), or the ready-made journey's events, optionally only the `events` you name, and writes the full HTML report with the timeline. The site comes from the audit. |
418
+ | `run_audit` | Runs a config file end to end. With `against`, it checks the run against a saved plan and, when the config has no journeys, walks the journeys derived from that plan. Local only for file paths. |
419
+
420
+ `against` takes the same values as the CLI's `--against`. Over HTTP it takes only `latest`, `first` or a plan ID, never a path. Plans are the ones saved by `aitd plan --site <url>`.
294
421
 
295
422
  The browser runs on the computer where the server runs, so it can reach your staging and `localhost` sites. Tracking requests are answered locally as usual.
296
423
 
@@ -335,7 +462,7 @@ The server prints a URL with a random secret path, like `https://<your-tunnel-ad
335
462
 
336
463
  In HTTP mode, the server is more locked down than locally:
337
464
 
338
- - It doesn't read measurement plans or configs from your disk (pass the plan as JSON text in the chat), and `run_audit` isn't offered. It only serves the reports it wrote itself.
465
+ - It doesn't read measurement plans or configs from your disk (pass the plan as JSON text in the chat, or name a saved plan with `against: latest`, `first` or its ID), and `run_audit` isn't offered. It only serves the reports it wrote itself.
339
466
  - The browser opens only `http` and `https` pages, and refuses `localhost` and private network addresses (such as `192.168.x.x` or `169.254.169.254`), so the URL can't be used to look into your network. All browser traffic, WebSockets included, goes through a small built-in proxy that resolves each host once and connects only to the address it checked, so a DNS answer that changes mid-audit can't get around the check. To audit a local staging site this way anyway, start the server with `--allow-private-network`.
340
467
  - Audits left open for 15 minutes are closed, and request bodies are limited to 1 MB.
341
468