@analitikisler/ai-tag-debugger 0.0.0-stage → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +104 -0
- package/LICENSE +21 -0
- package/README.md +327 -2
- package/package.json +56 -4
- package/src/agent.js +191 -0
- package/src/audit.js +100 -0
- package/src/browser.js +579 -0
- package/src/checks.js +213 -0
- package/src/cli.js +155 -0
- package/src/egress.js +134 -0
- package/src/glossary.js +139 -0
- package/src/guard.js +129 -0
- package/src/i18n.js +207 -0
- package/src/journeys.js +138 -0
- package/src/mcp.js +425 -0
- package/src/parsers.js +234 -0
- package/src/providers.js +250 -0
- package/src/redact.js +130 -0
- package/src/report.js +827 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.5.2 (2026-10-09)
|
|
4
|
+
|
|
5
|
+
Fixes from the review of 0.5.1.
|
|
6
|
+
|
|
7
|
+
- Meta's newer advanced matching fields (`udff[em]`, `udff[ph]`, …) are redacted like `ud[…]`.
|
|
8
|
+
- dataLayer pushes are redacted too: email and phone fields, everything under `user_data` (including `gtag("set", "user_data", …)`), and email-like values. Page data with similar names, such as `page.city`, is kept. Unhashed user data in the dataLayer is reported as a warning: it is a valid setup for Google enhanced conversions, but every other tag on the page can read it. `--keep-bodies` keeps pushes as they were.
|
|
9
|
+
- Any value that looks like an email address is redacted, whatever its field is called (for example `ep.customer_email`).
|
|
10
|
+
- In bodies cut at the 32 KB limit, a value left open at the end is dropped, and numbers (such as a phone number sent as a number) are redacted by field name.
|
|
11
|
+
|
|
12
|
+
## 0.5.1 (2026-10-09)
|
|
13
|
+
|
|
14
|
+
Fixes from the review of 0.5.0.
|
|
15
|
+
|
|
16
|
+
- **User data for ad matching is redacted** before requests are stored or shown: emails, phone numbers, names, addresses and user ids in GA4 and Google Ads enhanced conversions, Meta advanced matching (`ud[*]`) and TikTok `context.user`, in URLs, POST bodies and hit parameters. Each value becomes `[redacted: N chars, looks hashed]` or `[redacted: N chars, plain]`, and plain values are reported as a personal data risk. `--keep-bodies` or `"keep_bodies": true` stores requests as sent.
|
|
17
|
+
- Stored request bodies are capped at 2 MB per journey on top of 32 KB per request. Requests past the limit say so in the report.
|
|
18
|
+
- Without `--providers`, the providers named in the measurement plan are added to the defaults. Plan platforms are compared in lower case, so `"GA4"` matches. Planned events for providers that were left out are listed in an info finding instead of disappearing.
|
|
19
|
+
- `report.md` escapes the report title and journey ids like the other fields.
|
|
20
|
+
|
|
21
|
+
## 0.5.0 (2026-10-09)
|
|
22
|
+
|
|
23
|
+
- **Report v2.** Every step has three tabs: Events, dataLayer and Network requests. They replace the per-event tabs and the "Other dataLayer pushes and network calls" box. Events and requests can be filtered by source (GA4, Google Ads, Meta, TikTok, …), with only the sources in the step offered. Still no scripts.
|
|
24
|
+
- Each event shows its parameters, consent, a Source box (the matched dataLayer push and the request with host, method and status) and, when a finding points at it, a note saying what is wrong.
|
|
25
|
+
- GA4 item strings (`pr1`…, including `k0`/`v0` custom item parameters) and dataLayer `ecommerce.items` are shown as item cards.
|
|
26
|
+
- Network requests are decoded into parameter tables with a `?` tip on each parameter (a GA4, Google Ads, Meta, TikTok and X glossary in English and Turkish). Event parameters come first; session, device, page and consent parameters are collapsed. The raw request is kept under "Raw request".
|
|
27
|
+
- GA4 batched POST bodies and TikTok JSON bodies are now recorded (up to 32 KB per request), so the Network tab shows every event in a batch.
|
|
28
|
+
- The AI summary is structured: a headline, a ranked list of issues with severity tags, and "Not in the findings list, but worth checking". Each explanation has an impact line and numbered fix steps. Findings are grouped by severity and are collapsible, and evidence is shown as a table. Older captures with a plain-text summary still render.
|
|
29
|
+
- **Choose the providers to check** with `--providers`, `"providers"` in the config, or `providers` on the MCP `start_audit` and `run_audit` tools (default GA4, Google Ads, Meta, TikTok). Requests to the others are still answered locally but are left out of the timeline, the CSV and the checks, and only counted. The report shows which providers were checked and how many requests were skipped, overall and per step. The MCP server asks the assistant to check with the user before starting.
|
|
30
|
+
- Basic X (Twitter) Pixel parsing: event name, pixel id, value and quantity.
|
|
31
|
+
|
|
32
|
+
## 0.4.9 (2026-10-09)
|
|
33
|
+
|
|
34
|
+
- The project moved to the `analitikisler` GitHub organization: the npm package is now `@analitikisler/ai-tag-debugger`, the repository is github.com/analitikisler/ai-tag-debugger, and `package.json` lists the repository, homepage, issue tracker and author.
|
|
35
|
+
|
|
36
|
+
## 0.4.8 (2026-10-09)
|
|
37
|
+
|
|
38
|
+
- Reports (HTML and Markdown, English and Turkish) end with a credit line, "Generated by AI Tag Debugger · analitikisler.com". The domain is a link with `rel="noopener noreferrer"`, and there are no images or tracking. `--no-branding`, `"branding": false` in the config, or `branding: false` on the MCP `finish_audit` and `run_audit` tools removes it together with the Analitik İşler logo.
|
|
39
|
+
- The README ends with a short pointer to the hosted version at analitikisler.com.
|
|
40
|
+
|
|
41
|
+
## 0.4.7 (2026-10-09)
|
|
42
|
+
|
|
43
|
+
- **Redesigned HTML report** in the Analitik İşler design system. Findings are cards linked to their step, there is a journey overview, and events are grouped by platform (GA4, Meta, Google Ads, TikTok, others). Each event has tabs for its parameters, its dataLayer push and its network calls. The report has no scripts, so it works under the report server's CSP and on phones down to 320px wide.
|
|
44
|
+
- dataLayer pushes are linked to events by name, including gtag.js array pushes and pixel aliases (Meta `AddToCart` to `add_to_cart`, `Contact`, `SubmitForm`, `Subscribe`, `StartTrial` and more). Duplicate hits are paired with their pushes in order, and finding links pick the right step when step labels repeat.
|
|
45
|
+
|
|
46
|
+
## 0.4.6 (2026-10-09)
|
|
47
|
+
|
|
48
|
+
- Fix: elements whose class merely contains "cmp" (such as Adobe AEM's `cmp-navigation` headers) no longer count as consent banners. Real consent tools are still recognized by name.
|
|
49
|
+
|
|
50
|
+
## 0.4.5 (2026-10-09)
|
|
51
|
+
|
|
52
|
+
- Cookie and consent banners (OneTrust, Cookiebot, Usercentrics, Didomi, CookieYes and fixed elements named cookie, consent, GDPR or KVKK) block clicks underneath even when they cover less than half the screen.
|
|
53
|
+
- Errors and notes that quote a covering element's text get the same untrusted-content treatment as the page view.
|
|
54
|
+
- Ad slots in `about:blank` iframes (`google_ads_iframe_*`, `aswift_*`) are skipped.
|
|
55
|
+
|
|
56
|
+
## 0.4.4 (2026-10-09)
|
|
57
|
+
|
|
58
|
+
Fixes from the review of 0.4.3's element finding and clicking.
|
|
59
|
+
|
|
60
|
+
- **No clicking through consent walls.** When something covers the element, the click is refused if the cover is a dialog, a modal, or covers at least half the screen, and the agent is told to deal with it first. Otherwise (a sticky header, a small badge), the agent clicks a part of the element nothing covers, with real mouse events. A synthetic click is only the last resort. Either way, the agent is told, and the report marks the step.
|
|
61
|
+
- **Exact text matches first.** Clicking by `text` tries exact matches before partial ones, only visible elements, in the order button, link, label, placeholder, then any text, on the main page before iframes. When several elements match, the agent is asked to use a ref instead of a guess being made.
|
|
62
|
+
- **Text reaches iframes.** Clicking by text also searches visible iframes.
|
|
63
|
+
- **Fewer, better iframes.** Hidden, tiny and ad, analytics or consent vendor iframes are skipped, so they don't use up the list or get clicked. "Not in view" for iframe elements accounts for the frame's position on the page.
|
|
64
|
+
- **Shadow DOM refs.** Old refs are cleared inside shadow roots too, so a stale element can't share a ref with a new one.
|
|
65
|
+
- Faster page views on big pages (the pointer-cursor scan only looks near the visible area). Fewer duplicates: `label[for]` is no longer listed separately, and the shadow-root parent check is fixed. "More elements not listed" only shows when visible elements were actually left out. A non-integer `ref` is rejected.
|
|
66
|
+
|
|
67
|
+
## 0.4.3 (2026-10-09)
|
|
68
|
+
|
|
69
|
+
The agent finds the buttons real sites build.
|
|
70
|
+
|
|
71
|
+
- **Custom controls are listed.** Besides links, buttons and form fields, the page view now includes controls built from divs and spans: role attributes, `tabindex`, and elements with a pointer cursor (such as a styled "Rezervasyon Yap" div with a JavaScript click handler). Labels also come from `aria-labelledby`, `<label for>`, image alt text and icon titles.
|
|
72
|
+
- **On-screen elements come first.** Elements that are actually visible at their position are listed before those out of view, and covered or clipped ones (collapsed mega menus, overlays) come last. A long menu can no longer push the button the agent needs out of the list. Up to 250 elements are listed, up from 150.
|
|
73
|
+
- **Iframes and shadow DOM.** Elements inside embedded frames (booking and payment widgets) and open shadow roots get refs too.
|
|
74
|
+
- **Click by visible text.** `browser_action` accepts `text` instead of `ref`, e.g. `text: "Rezervasyon Yap"`, matching a button or link name, a label, a placeholder, or any visible text.
|
|
75
|
+
- **More reliable clicks and scrolling.** A click blocked by a sticky header or overlay falls back to a direct click event. `scroll` takes `up` or `down`, and scrolls the window directly when the page ignores the mouse wheel.
|
|
76
|
+
|
|
77
|
+
## 0.4.2 (2026-10-09)
|
|
78
|
+
|
|
79
|
+
Follow-up hardening from the 0.4.1 re-review, for the HTTP MCP server.
|
|
80
|
+
|
|
81
|
+
- **DNS rebinding.** When private addresses are blocked (`mcp --http`), all browser traffic goes through a small local proxy (`src/egress.js`). It resolves each host once, refuses private addresses, and connects to exactly the address it checked, so Chromium never resolves hosts itself.
|
|
82
|
+
- **WebSockets.** `ws://` and `wss://` connections go through the same proxy, so the private-address block applies to them too. Non-proxied WebRTC UDP is turned off in this mode.
|
|
83
|
+
- **IPv6 forms of private IPv4 addresses** (NAT64 `64:ff9b::/96`, 6to4 `2002::/16`, IPv4-compatible `::a.b.c.d`, hex-form IPv4-mapped) count as private, and Teredo addresses are refused.
|
|
84
|
+
- README: the prompt-injection line no longer promises more than a prompt rule can, and hosted setups are advised to block private ranges at the network level as well.
|
|
85
|
+
|
|
86
|
+
## 0.4.1 (2026-10-09)
|
|
87
|
+
|
|
88
|
+
Security and robustness fixes from the first full code and security review, before the first public release.
|
|
89
|
+
|
|
90
|
+
- **Navigation limits.** The audit browser now opens only `http` and `https` pages and stays on the audited site and its subdomains. Other hosts can be allowed with `allowed_hosts` in the config, `--allow-host`, or `allowed_hosts` on the MCP `start_audit` tool. Redirects and links that leave the site are blocked and the browser goes back.
|
|
91
|
+
- **Private addresses blocked over HTTP.** In `mcp --http` mode, audits refuse `localhost`, private, link-local and similar addresses, for the start page, navigations and every request. `--allow-private-network` turns this off.
|
|
92
|
+
- **Page content is untrusted.** The page view the model sees is wrapped in `<page_content>` tags, and the prompts say that this content is data, never instructions.
|
|
93
|
+
- **No secrets in the browser.** Chromium starts without API keys, tokens and other secrets in its environment.
|
|
94
|
+
- **MCP HTTP hardening.** Request bodies are limited to 1 MB, `--token` needs at least 16 characters, the secret is compared in constant time, binding to a non-loopback host prints a warning, and reports are served with security headers.
|
|
95
|
+
- **Abandoned audits close.** MCP audits idle for 15 minutes are closed, and per-audit logs are capped.
|
|
96
|
+
- **More beacons answered locally.** Google Ads remarketing (`viewthroughconversion`, `1p-user-list`, `ccm/collect`, `td.doubleclick.net`), Meta privacy sandbox, Microsoft Ads, LinkedIn, Pinterest, Snapchat and X pixels no longer reach real accounts during audits.
|
|
97
|
+
- **Fixes.** A malformed `%` in a tracking URL no longer crashes the checks. An unknown viewport no longer leaks a browser. The Gemini key is read from the injected environment.
|
|
98
|
+
- **Report safety.** CSV cells that start with `=`, `+`, `-` or `@` are prefixed so spreadsheets don't run them as formulas, and `report.md` escapes HTML in page-controlled values.
|
|
99
|
+
- **Reproducible installs.** `package-lock.json` is committed and Playwright is pinned to 1.56.1. The CI example pins a released npm version, installs the matching browser, sets `permissions: contents: read` and passes one API key. The repository has its own test workflow.
|
|
100
|
+
- The demo shop server listens on `127.0.0.1` only and checks paths correctly.
|
|
101
|
+
|
|
102
|
+
## 0.4.0
|
|
103
|
+
|
|
104
|
+
First complete version: CLI and MCP server, Anthropic, OpenAI, Gemini and OpenAI-compatible providers, ready-made journeys, Turkish reports, full event timeline and CSV.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Analitik İşler
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,328 @@
|
|
|
1
|
-
#
|
|
1
|
+
# AI Tag Debugger
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**An open-source AI agent that QA-tests your website's analytics and ad tracking.**
|
|
4
|
+
|
|
5
|
+
An AI model drives a real browser through your key journeys (product → cart → checkout, lead forms) like a real visitor. While it browses, `ai-tag-debugger` records every GA4, Google Ads, Meta Pixel and TikTok hit and every dataLayer push. It then checks them against your measurement plan and Consent Mode setup and writes a plain-language report of what broke and how to fix it, in English or Turkish.
|
|
6
|
+
|
|
7
|
+
Use it in two ways:
|
|
8
|
+
|
|
9
|
+
- **From your terminal or CI**, with your own API key for Anthropic (Claude), OpenAI, Google Gemini, or any OpenAI-compatible API (Kimi, GLM, DeepSeek, OpenRouter, a local Ollama and more).
|
|
10
|
+
- **From your AI chat app** (Claude, ChatGPT, Gemini and other apps that support MCP). The app's own model drives the audit, so you don't need a separate API key.
|
|
11
|
+
|
|
12
|
+
Built by [Analitik İşler](https://analitikisler.com). MIT licensed.
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
▶ checkout (mobile)
|
|
16
|
+
Accept cookies
|
|
17
|
+
Open the first product
|
|
18
|
+
Add to cart
|
|
19
|
+
Open the cart
|
|
20
|
+
Go to checkout
|
|
21
|
+
…
|
|
22
|
+
Place the order
|
|
23
|
+
✓ All scripted steps completed. · 20 tracking hits captured
|
|
24
|
+
|
|
25
|
+
2 broken · 1 risk · 1 warning · 4 ok
|
|
26
|
+
Report: reports/demo/report.html
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Why
|
|
30
|
+
|
|
31
|
+
Tracking breaks silently. A redesign renames a button, a new template forgets a field, a consent banner update lets pixels fire too early. Nobody gets an alert. You find out weeks later, after ad platforms have optimized on bad conversions.
|
|
32
|
+
|
|
33
|
+
Manual tag QA catches this, but it's tedious, so it rarely happens after every release. `ai-tag-debugger` makes it a command you can run in CI.
|
|
34
|
+
|
|
35
|
+
## What it checks
|
|
36
|
+
|
|
37
|
+
| Check | Example finding |
|
|
38
|
+
|---|---|
|
|
39
|
+
| Missing events | `purchase` never fired on the checkout journey |
|
|
40
|
+
| Missing parameters | `purchase` sent without `value` and `currency` on mobile |
|
|
41
|
+
| Duplicates | `add_to_cart` sent twice by one click |
|
|
42
|
+
| Consent | Meta `PageView` fired before the visitor accepted cookies, Consent Mode default missing or set too late |
|
|
43
|
+
| Personal data | An email address sent to an ad platform in plain text |
|
|
44
|
+
| Unplanned events | Events firing that aren't in your measurement plan |
|
|
45
|
+
|
|
46
|
+
Platforms: GA4 (including server-side GTM endpoints), Google Ads conversions, Meta Pixel, TikTok Pixel, and basic X Pixel events.
|
|
47
|
+
|
|
48
|
+
### Choose the providers to check
|
|
49
|
+
|
|
50
|
+
By default an audit checks GA4, Google Ads, Meta and TikTok. Pick others with `--providers ga4,meta,x`, `"providers"` in the config, or `providers` on the MCP `start_audit` and `run_audit` tools. You can choose from `ga4`, `google_ads`, `meta`, `tiktok`, `x`, `linkedin`, `microsoft_ads`, `pinterest`, `snapchat`, `clarity`, `hotjar`, `doubleclick` and `adsense`. Events are parsed for GA4, Google Ads, Meta, TikTok and X; the others show their network requests.
|
|
51
|
+
|
|
52
|
+
Requests to providers you didn't choose are still answered locally, so they never reach those accounts either. They are left out of the timeline, the CSV and the checks, and only counted: the report shows "Providers checked: … · Out of scope: Microsoft Clarity, DoubleClick (61 requests skipped)", and each step says how many it skipped. Without `--providers`, any provider your measurement plan names is added to the default list. Planned events for providers you did leave out are listed in one info finding instead of being reported as missing. Tag manager and consent platform requests are always shown.
|
|
53
|
+
|
|
54
|
+
## How it works
|
|
55
|
+
|
|
56
|
+
1. **Browse:** the AI model gets a goal in plain language ("add any product to the cart and complete checkout with the test payment") and a text view of the page. It acts through two tools, `browser_action` (goto, click, fill, select, scroll, wait) and `finish`, one step at a time with Playwright.
|
|
57
|
+
2. **Capture:** every request to a tracking endpoint is parsed into normalized hits and tagged with the step that caused it. Every dataLayer push is recorded through an init script, including `gtag('consent', ...)` commands. By default, the collection requests of GA4, Google Ads, Meta, TikTok, Microsoft Ads, LinkedIn, Pinterest, Snapchat and X are answered locally, so **test traffic doesn't reach those real accounts**. Tags from other vendors still send their requests; the network part of the timeline shows each call and whether it was answered locally.
|
|
58
|
+
3. **Check:** deterministic rules compare the hits with your measurement plan. This part needs no API calls, so results are reproducible.
|
|
59
|
+
4. **Explain:** the model reads the findings, the hits per step and (optionally) your GTM container export. It writes the likely cause and fix for each problem, naming the tags and triggers involved.
|
|
60
|
+
|
|
61
|
+
## AI providers
|
|
62
|
+
|
|
63
|
+
Bring your own API key. `ai-tag-debugger` uses the official SDK of each provider:
|
|
64
|
+
|
|
65
|
+
| Provider | API key variable | Default model |
|
|
66
|
+
|---|---|---|
|
|
67
|
+
| Anthropic (Claude) | `ANTHROPIC_API_KEY` | `claude-opus-5-5` |
|
|
68
|
+
| OpenAI | `OPENAI_API_KEY` | `gpt-5.5` |
|
|
69
|
+
| Google Gemini | `GEMINI_API_KEY` | `gemini-pro-latest` |
|
|
70
|
+
|
|
71
|
+
The provider is picked from whichever key is set (Anthropic first if several are). Choose explicitly with `--provider anthropic|openai|gemini` or `"provider"` in the config, and pick another model with `--model`. Navigation asks for low reasoning effort to keep runs cheap; explanations ask for high effort and structured JSON output.
|
|
72
|
+
|
|
73
|
+
### Any OpenAI-compatible API: Kimi, GLM, DeepSeek, OpenRouter, Ollama and more
|
|
74
|
+
|
|
75
|
+
Many providers offer an API that works like OpenAI's. Point `ai-tag-debugger` at it with its address and a model id:
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
export OPENAI_COMPATIBLE_API_KEY=... # leave unset for a local Ollama
|
|
79
|
+
ai-tag-debugger run --site https://yourshop.com --journey add_to_cart \
|
|
80
|
+
--provider openai-compatible --base-url <API address> --model <model id>
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
| Provider | `--base-url` |
|
|
84
|
+
|---|---|
|
|
85
|
+
| Moonshot (Kimi) | `https://api.moonshot.ai/v1` |
|
|
86
|
+
| Z.ai (GLM) | `https://api.z.ai/api/paas/v4` |
|
|
87
|
+
| DeepSeek | `https://api.deepseek.com` |
|
|
88
|
+
| Mistral | `https://api.mistral.ai/v1` |
|
|
89
|
+
| Groq | `https://api.groq.com/openai/v1` |
|
|
90
|
+
| OpenRouter (hundreds of models) | `https://openrouter.ai/api/v1` |
|
|
91
|
+
| Ollama on your computer | `http://localhost:11434/v1` |
|
|
92
|
+
|
|
93
|
+
Use the model id from your provider's documentation. You can also set `OPENAI_COMPATIBLE_BASE_URL` and `OPENAI_COMPATIBLE_MODEL` instead of the flags, or `"provider"`, `"base_url"` and `"model"` in the config. When the API can't return strict JSON schemas, the explanations fall back to plain JSON mode.
|
|
94
|
+
|
|
95
|
+
**A word on model choice.** The AI browses by calling tools, one step at a time, for 5 to 30 steps. Models differ a lot in how reliably they do that: smaller and local models may click the wrong thing, loop, or stop early. The checks stay correct whatever the model does, because they come from the captured hits, but a journey that doesn't finish can't show its later events. If a model struggles, try a larger one, a ready-made journey with a short goal, or `--max-steps`. The Anthropic, OpenAI and Gemini defaults are the tested ones.
|
|
96
|
+
|
|
97
|
+
Adding a native provider means implementing two functions in [`src/providers.js`](src/providers.js): a tool-calling conversation and a structured JSON call.
|
|
98
|
+
|
|
99
|
+
## Quick start
|
|
100
|
+
|
|
101
|
+
Requires Node 20+.
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
git clone https://github.com/analitikisler/ai-tag-debugger.git
|
|
105
|
+
cd ai-tag-debugger
|
|
106
|
+
npm install
|
|
107
|
+
npx playwright install chromium # run again after upgrading, so the browser matches Playwright
|
|
108
|
+
export ANTHROPIC_API_KEY=... # or OPENAI_API_KEY, or GEMINI_API_KEY
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Try it on the included demo shop. It has three deliberate tracking bugs:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
npm run demo:serve # in one terminal: http://localhost:4321
|
|
115
|
+
node src/cli.js run --config examples/demo.config.json --out reports/demo
|
|
116
|
+
open reports/demo/report.html
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
No API key yet? Replay the fixed steps without any AI model:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
node src/cli.js run --config examples/demo.config.json --scripted --no-ai --out reports/demo
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Choose what to test
|
|
126
|
+
|
|
127
|
+
You don't need a config file to start. Pick ready-made journeys, describe your own in plain language, and narrow the check to the events you care about:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
# Ready-made journeys come with a goal and the GA4 and Meta events to expect.
|
|
131
|
+
ai-tag-debugger run --site https://staging.yourshop.com --journey purchase --journey sign_up
|
|
132
|
+
|
|
133
|
+
# Describe a journey in your own words. The events you name are expected on it.
|
|
134
|
+
ai-tag-debugger run --site https://yoursite.com --goal "Subscribe to the newsletter in the footer" --events newsletter_signup
|
|
135
|
+
|
|
136
|
+
# Only check some events.
|
|
137
|
+
ai-tag-debugger run --config ai-tag-debugger.config.json --events purchase,add_to_cart
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
| Ready-made journey | What the AI does | Events it expects |
|
|
141
|
+
|---|---|---|
|
|
142
|
+
| `purchase` (or `checkout`) | Product → cart → checkout → order with the test payment option | GA4 `view_item`, `add_to_cart`, `begin_checkout`, `purchase`; Meta `AddToCart`, `InitiateCheckout`, `Purchase` |
|
|
143
|
+
| `add_to_cart` (or `cart`) | Opens a product and adds it to the cart | GA4 `view_item`, `add_to_cart`; Meta `AddToCart` |
|
|
144
|
+
| `sign_up` (or `signup`, `register`) | Creates an account with test data | GA4 `sign_up`; Meta `CompleteRegistration` |
|
|
145
|
+
| `lead` (or `contact`, `form`) | Fills in and sends a contact, quote or demo form | GA4 `generate_lead`; Meta `Lead` |
|
|
146
|
+
| `search` | Searches the site and opens a result | GA4 `search` with `search_term`; Meta `Search` |
|
|
147
|
+
|
|
148
|
+
`--journey` also picks journeys from your config by id. With a config, ready-made journeys add their events to your plan.
|
|
149
|
+
|
|
150
|
+
### Keeping API usage down
|
|
151
|
+
|
|
152
|
+
Each browsing step is one model call, so usage grows with **journeys × viewports × steps**, plus one call at the end to explain the findings. The checks themselves never call the model.
|
|
153
|
+
|
|
154
|
+
- **Run only the journeys you need.** This is the biggest saving: `--journey add_to_cart` instead of the full `purchase` journey takes a few steps instead of ten or more.
|
|
155
|
+
- **Run one viewport.** `--viewport mobile` halves the calls compared with desktop and mobile.
|
|
156
|
+
- **Cap the steps.** `--max-steps 15` stops a journey that wanders.
|
|
157
|
+
- **Expect bigger pages to cost more.** Each step sends the page view (up to 250 elements and 3,000 characters of text) to the model, so busy pages use more tokens per step than simple ones.
|
|
158
|
+
- **Use `--events`** to keep the report focused. It doesn't shorten browsing, so it barely changes usage.
|
|
159
|
+
- **Replay without AI.** After one AI run, copy the steps into `journeys[].steps` and use `--scripted` (with `--no-ai` for zero model calls) on every deploy.
|
|
160
|
+
- **Use a chat app.** Over MCP, the chat app's own model does the browsing, so no API key is used at all.
|
|
161
|
+
|
|
162
|
+
## Configure your site
|
|
163
|
+
|
|
164
|
+
`ai-tag-debugger.config.json`:
|
|
165
|
+
|
|
166
|
+
```json
|
|
167
|
+
{
|
|
168
|
+
"site": "https://staging.yourshop.com",
|
|
169
|
+
"plan": "measurement-plan.json",
|
|
170
|
+
"gtm_container": "GTM-XXXX_workspace.json",
|
|
171
|
+
"viewports": ["desktop", "mobile"],
|
|
172
|
+
"journeys": [
|
|
173
|
+
{
|
|
174
|
+
"id": "checkout",
|
|
175
|
+
"goal": "Accept cookies, add any product to the cart and complete checkout using the test payment method."
|
|
176
|
+
}
|
|
177
|
+
]
|
|
178
|
+
}
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
| Field | Meaning |
|
|
182
|
+
|---|---|
|
|
183
|
+
| `site` | Base URL. Use staging when you can. |
|
|
184
|
+
| `plan` | Measurement plan: a file path or an inline object (below). Optional with ready-made journeys. |
|
|
185
|
+
| `gtm_container` | Optional GTM container export (Admin → Export container), so explanations can name tags and triggers. |
|
|
186
|
+
| `viewports` | `desktop` and/or `mobile`. |
|
|
187
|
+
| `journeys` | Optional when you pick ready-made journeys with `--journey` or describe one with `--goal`. |
|
|
188
|
+
| `journeys[].goal` | What the AI should do, in plain language. |
|
|
189
|
+
| `journeys[].start` | Optional start path (default `/`). |
|
|
190
|
+
| `journeys[].steps` | Optional fixed steps for `--scripted` runs: `{ "action": "click", "selector": "#add-to-cart" }`. |
|
|
191
|
+
| `allowed_hosts` | Other hosts journeys may visit, e.g. `["checkout.payments-provider.com"]` for a hosted checkout. The browser otherwise stays on the `site` host and its subdomains. Same as `--allow-host`. |
|
|
192
|
+
| `providers` | Tracking providers to check, e.g. `["ga4", "meta"]`. Default `["ga4", "google_ads", "meta", "tiktok"]`. Same as `--providers`. |
|
|
193
|
+
| `block_hits` | Default `true`. Set to `false` to let hits reach your real endpoints. |
|
|
194
|
+
| `language` | Report language, `en` (default) or `tr`. Same as `--lang`. |
|
|
195
|
+
| `keep_bodies` | Default `false`. Set to `true` to store tracking requests as sent, without redacting user data. Same as `--keep-bodies`. |
|
|
196
|
+
| `branding` | Default `true`. Set to `false` to leave the Analitik İşler logo and the "Generated by AI Tag Debugger · analitikisler.com" line out of the reports. Same as `--no-branding`, or `branding: false` on the MCP `finish_audit` and `run_audit` tools. |
|
|
197
|
+
| `provider` | Optional: `anthropic`, `openai`, `gemini` or `openai-compatible`. Same as `--provider`. |
|
|
198
|
+
| `base_url`, `model` | Optional: the API address and model id, e.g. for `openai-compatible`. Same as `--base-url` and `--model`. |
|
|
199
|
+
|
|
200
|
+
`measurement-plan.json`:
|
|
201
|
+
|
|
202
|
+
```json
|
|
203
|
+
{
|
|
204
|
+
"consent_mode": true,
|
|
205
|
+
"events": [
|
|
206
|
+
{ "platform": "ga4", "name": "add_to_cart", "required_params": ["currency", "value", "items"] },
|
|
207
|
+
{ "platform": "ga4", "name": "purchase", "journeys": ["checkout"], "required_params": ["transaction_id", "currency", "value", "items"] },
|
|
208
|
+
{ "platform": "meta", "name": "Purchase", "journeys": ["checkout"], "required_params": ["value", "currency"] }
|
|
209
|
+
]
|
|
210
|
+
}
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
`platform` is `ga4`, `google_ads`, `meta` or `tiktok`. `journeys` limits where an event is expected (default: every journey). Set `allow_multiple: true` for events that may legitimately fire more than once per action.
|
|
214
|
+
|
|
215
|
+
## CLI
|
|
216
|
+
|
|
217
|
+
```
|
|
218
|
+
ai-tag-debugger run [--site <url>] [--config <file>] [--journey <id>] [--goal "<text>"] [--events <names>]
|
|
219
|
+
[--max-steps <n>] [--out <dir>] [--viewport desktop|mobile] [--providers <list>]
|
|
220
|
+
[--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--base-url <url>]
|
|
221
|
+
[--lang en|tr] [--allow-host <host>] [--keep-bodies] [--scripted] [--no-ai] [--fail-on broken|risk|warning|never] [--headed]
|
|
222
|
+
ai-tag-debugger mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
|
|
223
|
+
[--allow-private-network]
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
The exit code is `1` when findings reach `--fail-on` (default `broken`), so it can gate a deploy. See [`examples/github-actions/ai-tag-debugger.yml`](examples/github-actions/ai-tag-debugger.yml) for running it after every deployment.
|
|
227
|
+
|
|
228
|
+
Output:
|
|
229
|
+
|
|
230
|
+
- `report.html`: a summary with the most important problem first, findings grouped by severity (each with its impact, likely cause, evidence and fix steps), and a **full event timeline**. Every step of every journey has three tabs:
|
|
231
|
+
- **Events**: each tracking hit with its parameters, consent state, the dataLayer push and request behind it, and a note when a finding points at it.
|
|
232
|
+
- **dataLayer**: every push in the step.
|
|
233
|
+
- **Network requests**: every request to the chosen providers, decoded into a parameter table with a `?` tip on each parameter. A batched GA4 request shows each of its events.
|
|
234
|
+
|
|
235
|
+
Events and requests can be filtered by source. GA4 item strings (`pr1`…) and dataLayer `ecommerce.items` are shown as item cards, with custom item parameters in their own group. The report has no scripts.
|
|
236
|
+
- `timeline.csv`: the same timeline as one row per hit, dataLayer push and network call, ready for Excel or Google Sheets.
|
|
237
|
+
- `report.md`: the report in Markdown, e.g. for a pull request comment.
|
|
238
|
+
- `capture.json`: everything captured, for your own analysis.
|
|
239
|
+
|
|
240
|
+
Reports end with a credit line, "Generated by AI Tag Debugger · analitikisler.com", where the domain is a plain link (no images, no tracking, and no referrer, so report URLs aren't leaked). `--no-branding` removes it along with the logo.
|
|
241
|
+
|
|
242
|
+
Use `--lang tr` for a Turkish report. Finding texts, headings, and the model's explanations and fixes are all written in Turkish, and the agent describes its steps in Turkish too.
|
|
243
|
+
|
|
244
|
+
## Use it from your AI chat app (MCP)
|
|
245
|
+
|
|
246
|
+
`ai-tag-debugger mcp` starts a [Model Context Protocol](https://modelcontextprotocol.io) server. Your chat app's model then runs the audit itself: you ask "check the tracking on the checkout of staging.myshop.com" and it opens the browser, clicks through, and explains the findings in the chat. No API key is needed, because the app's own model does the thinking.
|
|
247
|
+
|
|
248
|
+
Tools the server offers:
|
|
249
|
+
|
|
250
|
+
| Tool | What it does |
|
|
251
|
+
|---|---|
|
|
252
|
+
| `start_audit` | Opens a fresh browser on a URL and starts recording all tracking. Pass a ready-made journey (`purchase`, `sign_up`, …) to get its goal and expected events, and `providers` to choose what to check. The assistant asks you which platforms to check first. |
|
|
253
|
+
| `browser_action` | One click, fill, select, goto, scroll or wait. Returns the page and the tracking hits that step sent. |
|
|
254
|
+
| `get_page` | Shows the current page. |
|
|
255
|
+
| `finish_audit` | Checks everything against your measurement plan (JSON) or the ready-made journey's events, optionally only the `events` you name, and writes the full HTML report with the timeline. |
|
|
256
|
+
| `run_audit` | Runs a config file end to end (local only). |
|
|
257
|
+
|
|
258
|
+
The browser runs on the computer where the server runs, so it can reach your staging and `localhost` sites. Tracking requests are answered locally as usual.
|
|
259
|
+
|
|
260
|
+
### Apps that start a local server: Claude Desktop, Gemini CLI, Cursor, VS Code, Claude Code
|
|
261
|
+
|
|
262
|
+
Add the server to the app's MCP config, pointing at your copy of the repository. For Claude Desktop that is Settings → Developer → Edit Config (`claude_desktop_config.json`):
|
|
263
|
+
|
|
264
|
+
```json
|
|
265
|
+
{
|
|
266
|
+
"mcpServers": {
|
|
267
|
+
"ai-tag-debugger": {
|
|
268
|
+
"command": "node",
|
|
269
|
+
"args": ["/path/to/ai-tag-debugger/src/cli.js", "mcp"]
|
|
270
|
+
}
|
|
271
|
+
}
|
|
272
|
+
}
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
Gemini CLI uses the same `mcpServers` block in `~/.gemini/settings.json`. Claude Code: `claude mcp add ai-tag-debugger -- node /path/to/ai-tag-debugger/src/cli.js mcp`. Restart the app, and the ai-tag-debugger tools appear. If the app can't find `node`, use its full path (`which node`). Reports are written to `~/ai-tag-debugger-reports`.
|
|
276
|
+
|
|
277
|
+
### Apps that connect to a URL: ChatGPT, Gemini
|
|
278
|
+
|
|
279
|
+
These apps connect to MCP servers over HTTPS (in ChatGPT, through developer mode connectors, depending on your plan). Start the server in HTTP mode and expose it with a tunnel:
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
cloudflared tunnel --url http://localhost:8787 # or: ngrok http 8787; note the https address it prints
|
|
283
|
+
node src/cli.js mcp --http --port 8787 --public-url https://<your-tunnel-address>
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
The server prints a URL with a random secret path, like `https://<your-tunnel-address>/mcp/<secret>`. Add that URL as a connector in the app. Anyone with the URL can use the browser on your machine, so keep it private and stop the server when you're done. A secret of your own (`--token`) must be at least 16 characters.
|
|
287
|
+
|
|
288
|
+
In HTTP mode, the server is more locked down than locally:
|
|
289
|
+
|
|
290
|
+
- It doesn't read measurement plans or configs from your disk (pass the plan as JSON text in the chat), and `run_audit` isn't offered. It only serves the reports it wrote itself.
|
|
291
|
+
- The browser opens only `http` and `https` pages, and refuses `localhost` and private network addresses (such as `192.168.x.x` or `169.254.169.254`), so the URL can't be used to look into your network. All browser traffic, WebSockets included, goes through a small built-in proxy that resolves each host once and connects only to the address it checked, so a DNS answer that changes mid-audit can't get around the check. To audit a local staging site this way anyway, start the server with `--allow-private-network`.
|
|
292
|
+
- Audits left open for 15 minutes are closed, and request bodies are limited to 1 MB.
|
|
293
|
+
|
|
294
|
+
A hosted server, so you don't need a tunnel, is part of the [Analitik İşler](https://analitikisler.com) Cloud version.
|
|
295
|
+
|
|
296
|
+
## Safety
|
|
297
|
+
|
|
298
|
+
- The agent is told to use obvious test data and **never to enter real payment details**. If a journey needs a real payment or account, it stops and reports that instead.
|
|
299
|
+
- Tracking requests to the platforms listed in [How it works](#how-it-works) are answered locally by default, so those analytics and ad accounts don't receive test conversions.
|
|
300
|
+
- The browser stays on the audited site and its subdomains, plus any `allowed_hosts`. It opens only `http` and `https` pages.
|
|
301
|
+
- The model is told that page content is data, never instructions. A prompt rule can't guarantee a model is never misled, so the navigation limits above contain what a misled model can do.
|
|
302
|
+
- The browser runs without your API keys and other secrets in its environment.
|
|
303
|
+
- **User data sent for ad matching is redacted** before it is stored in `capture.json` or shown in the report: emails, phone numbers, names, addresses and user ids in GA4 and Google Ads enhanced conversions (`em`, `ph`, `uid`, …), Meta advanced matching (`ud[*]`, `udff[*]`), TikTok `context.user`, and the `user_data` the site pushes to the dataLayer for enhanced conversions. Any value that looks like an email address is redacted whatever its field is called. Each value becomes a marker such as `[redacted: 64 chars, looks hashed]`, and a value that wasn't hashed is reported as a personal data risk. `--keep-bodies` (`"keep_bodies": true`) stores requests as sent, for debugging.
|
|
304
|
+
- Stored request bodies are capped at 32 KB per request and 2 MB per journey. Past that, the report says the body wasn't stored.
|
|
305
|
+
- Only run it against sites you own or are authorized to test.
|
|
306
|
+
- Hosting the HTTP server for other people? Also block private and cloud metadata address ranges at the network level (an egress firewall or an isolated network), as defense in depth.
|
|
307
|
+
|
|
308
|
+
## Roadmap
|
|
309
|
+
|
|
310
|
+
- Consent Check: deeper audits of cookie banners and Consent Mode v2 against GDPR/KVKK.
|
|
311
|
+
- Measurement Plan: draft a plan and dataLayer spec from a URL.
|
|
312
|
+
- Hosted version with scheduled runs, deploy webhooks, a hosted MCP server and multi-site dashboards for agencies at [analitikisler.com](https://analitikisler.com).
|
|
313
|
+
|
|
314
|
+
## Development
|
|
315
|
+
|
|
316
|
+
```bash
|
|
317
|
+
npm test # parsers, checks, each AI provider and the MCP server, end to end against the demo shop
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
The end-to-end tests use fake model clients, so they run without an API key.
|
|
321
|
+
|
|
322
|
+
## Hosted version
|
|
323
|
+
|
|
324
|
+
For hosted runs, scheduling and agency dashboards, see [analitikisler.com](https://analitikisler.com).
|
|
325
|
+
|
|
326
|
+
## License
|
|
327
|
+
|
|
328
|
+
MIT © Analitik İşler
|
package/package.json
CHANGED
|
@@ -1,6 +1,58 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@analitikisler/ai-tag-debugger",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.5.2",
|
|
4
|
+
"description": "Open-source AI agent that QA-tests your website's analytics, ad and consent tracking (GA4, GTM, Google Ads, Meta, TikTok). Bring your own model (Claude, OpenAI, Gemini) or use it from your chat app over MCP.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"ai-tag-debugger": "src/cli.js"
|
|
8
|
+
},
|
|
9
|
+
"files": [
|
|
10
|
+
"src",
|
|
11
|
+
"README.md",
|
|
12
|
+
"LICENSE",
|
|
13
|
+
"CHANGELOG.md"
|
|
14
|
+
],
|
|
15
|
+
"scripts": {
|
|
16
|
+
"test": "node --test test/*.test.js",
|
|
17
|
+
"demo:serve": "node examples/serve.js",
|
|
18
|
+
"demo": "node src/cli.js run --config examples/demo.config.json --scripted --out reports/demo"
|
|
19
|
+
},
|
|
20
|
+
"engines": {
|
|
21
|
+
"node": ">=20"
|
|
22
|
+
},
|
|
23
|
+
"keywords": [
|
|
24
|
+
"analytics",
|
|
25
|
+
"ga4",
|
|
26
|
+
"google-tag-manager",
|
|
27
|
+
"tracking",
|
|
28
|
+
"qa",
|
|
29
|
+
"consent-mode",
|
|
30
|
+
"ai-agent",
|
|
31
|
+
"mcp",
|
|
32
|
+
"mcp-server",
|
|
33
|
+
"claude",
|
|
34
|
+
"openai",
|
|
35
|
+
"gemini",
|
|
36
|
+
"playwright",
|
|
37
|
+
"tag-debugger",
|
|
38
|
+
"gtm-debugger"
|
|
39
|
+
],
|
|
40
|
+
"license": "MIT",
|
|
41
|
+
"author": "Analitik İşler (https://analitikisler.com)",
|
|
42
|
+
"homepage": "https://github.com/analitikisler/ai-tag-debugger#readme",
|
|
43
|
+
"repository": {
|
|
44
|
+
"type": "git",
|
|
45
|
+
"url": "git+https://github.com/analitikisler/ai-tag-debugger.git"
|
|
46
|
+
},
|
|
47
|
+
"bugs": {
|
|
48
|
+
"url": "https://github.com/analitikisler/ai-tag-debugger/issues"
|
|
49
|
+
},
|
|
50
|
+
"dependencies": {
|
|
51
|
+
"@anthropic-ai/sdk": "^0.110.0",
|
|
52
|
+
"@google/genai": "^2.28.0",
|
|
53
|
+
"@modelcontextprotocol/sdk": "^1.32.1",
|
|
54
|
+
"openai": "^7.30.1",
|
|
55
|
+
"playwright": "1.56.1",
|
|
56
|
+
"zod": "^4.6.5"
|
|
57
|
+
}
|
|
58
|
+
}
|