@stacknyu/pi-glitch-lab 0.0.0-stage → 0.5.1

package/CHANGELOG.md

@@ -0,0 +1,17 @@
+# pi-glitch-lab changelog
+
+## 0.5.1
+
+- Cancel interaction approvals through the execution signal in native Pi and RPC/Pi Web dialogs; preserve explicit headless interaction/write authorization.
+- Reject pre-aborted, killed or truncated browser installation attempts instead of reporting incomplete installation as success.
+- Add native/RPC approval, headless authorization and browser-install cancellation regressions.
+
+## 0.5.0
+
+- First independent `@stacknyu/pi-glitch-lab` npm release, with Pi-native manifest, host peers, release notes and MIT license.
+- Extend the rendered-control scan with bounded state exploration, scrolling, observable outcomes, before/after evidence and honest coverage/limit reports.
+- Add deterministic UAT assertions for URL, visibility, values, checked/enabled state, counts and attributes, including reload/persistence journeys.
+- Require explicit interaction/write authorization; enforce origin/request guards, strict targets and positive login verification. Keep browser auth in memory rather than checkout files.
+- Reject evidence/baseline overwrites and propagate failed/incomplete runs as tool errors; prevent stale results from appearing successful.
+- Correct cross-document sensor timing, React root traversal, text-flap timing and DOM evidence frames. Report sampling limitations rather than claiming exhaustive frame coverage.
+- Add synthetic browser, failure, sensor and standalone-package regressions. No production targets, credentials or model calls are used by the tests.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Stacknyu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,88 @@
-# Temporary Holding Version
+# pi-glitch-lab
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Deterministic browser UI/UX checks, interaction coverage and scripted UAT for [Pi](https://pi.dev). Uses Playwright/Chromium; scripts do not call AI. The agent interprets evidence, not invents acceptance criteria.
+
+## Install
+
+```sh
+pi install npm:@stacknyu/pi-glitch-lab
+# Later: pi update npm:@stacknyu/pi-glitch-lab
+# Remove: pi remove npm:@stacknyu/pi-glitch-lab
+```
+
+Reload Pi. This package is independent of pi-flow and pi-check. Requires Pi (tested with 1.5.0), Node 22.19+, macOS/Linux and Chromium or Google Chrome. `/glitch-lab-doctor` checks the browser; `/glitch-lab-doctor install` explicitly downloads Playwright Chromium. No browser is downloaded on plugin load. Unpinned installs use Pi's native update notifications and this package's versioned changelog.
+
+## Three passes
+
+| Pass/tool | What it establishes |
+|---|---|
+| `ux_affordance_scan` | Observes rendered controls: labels, focusability, disabled/selected/expanded/checked states, target sizes, covering elements, overflow and visual affordance candidates. Scrolls the page and bounded nested containers. No clicks by default. |
+| `ux_affordance_scan` with `probeClicks=true`, `interact=true` | Explores menu/tab/dialog/navigation states with bounded sequential clicks, replaying the initial state for independent probes. Records responded, no-observable-response, inconclusive (background noise), error, blocked and skipped outcomes, plus before/after screenshots. Does not automatically fill forms. |
+| `ux_journeys` | Executes explicit acceptance steps/assertions, optionally establishing or comparing a visual baseline. Can test approved writes and persistence after reload. |
+| `glitch_hunt` | Samples painted frames, React commits, DOM mutations, text flaps, layout shifts and network timing. Modes: nav, idle, loads, journey. Optional paint-flashing video. |
+| `ux_report` | Builds a private HTML evidence report including coverage, limits, skipped actions, UAT failures and glitch findings. |
+
+**A control responding is not proof of a correct business outcome.** A network request alone is not success. A control with no observed response is inconclusive: clipboard/hover actions and longer waits may need explicit journeys. Inventories and fake-button heuristics are candidates, not an accessibility certification.
+
+Example requests:
+
+- "Inventory controls on http://localhost:3000/dashboard without clicking."
+- "On this disposable test app, explore the dashboard's controls. Report skipped controls and coverage limits."
+- "Run the approved profile-save UAT journey, including a reload to verify persistence."
+- "Find flicker while navigating the test dashboard."
+
+## Authorization and data safety
+
+**Use disposable test accounts and a staging/synthetic environment. Never point automatic exploration at production.** Loading a page itself can have side effects; this is not a sandbox or transaction rollback system.
+
+- Observation is the default. Any click/typing/navigation exploration needs `interact=true` (`--interact` on CLI), reflecting explicit user authorization. Pi UI sessions additionally ask for confirmation.
+- `allowWrites=true` (`--allow-writes`) additionally authorizes HTTP writes, login, typing, selecting/checking values and potentially submitting Enter presses. It requires `interact=true`. Replays/repeats can repeat server writes; there is no automatic cleanup or rollback.
+- Without write authorization, non-GET/HEAD/OPTIONS requests and WebSockets are blocked. Unsafe-looking URL paths are also blocked. These guards cannot make unsafe GET endpoints or arbitrary application JavaScript harmless.
+- Automatic exploration always skips obvious submit/destructive controls; authorized **scripted journeys** are required for those. Labels are not a security boundary.
+- Document navigation is limited to the configured base origin plus explicit `allowedOrigins` on the cell. Writes and WebSockets can only reach those origins. HTTP redirects on document/write requests currently fail closed (use direct canonical targets and SPA-style login); chained redirect flows are not supported. Read-only subresources such as CDN assets may use other origins. Popups/dialogs/downloads encountered by the explorer are closed/dismissed/cancelled, but prior requests cannot be undone.
+- Credentials use `env:VARIABLE` in login fields. Missing credentials or failed positive authentication checks fail the run. Set `login.authenticated` to a visible authenticated-only selector, and/or `login.waitFor` to an exact pathname other than `/`. Browser login state stays in memory; it is not cached in `.ux/auth` or written to disk.
+- Evidence can contain application data. Review masks and keep `.ux/` private/ignored; do not publish reports automatically. Password inputs and `[data-private]` are useful mask defaults. Raw sensor/network evidence is not a secret-redaction system.
+
+## Interaction coverage and limits
+
+Default exploration limits per configured screen: 40 actual click attempts (including parent-state replays), 12 states, 5 URLs, depth 4, 120 seconds and 1000 ms observation per action. Tool options: `maxActions`, `maxStates`, `maxPages`, `maxDepth`, `seconds`, `observeMs`; CLI uses kebab-case flags. Each action has a bounded timeout. Reset/replay and evidence capture consume wall time; in-flight browser operations may finish after the exploration deadline. Unvisited states and limit hits are reported, not passed.
+
+Every coverage file records control-state pairs, exercised/skipped/blocked results, states visited, pending states and limits. Same controls in different states count separately. A completed observation/exploration run means collection succeeded, **not** that the application passed UAT.
+
+Coverage is limited to rendered light DOM. Closed shadow roots, iframe contents, hover-only widgets, infinitely scrolling/virtualized content outside the scroll budget, keyboard-only flows and unknown states are not exhaustively explored. Mobile viewports emulate a browser; they are not native-device testing. Use explicit journeys for keyboard, inputs and important business flows.
+
+## Configure UAT
+
+Copy `skills/ai-agentic-usability-testing/assets/matrix.web.example.json` to your project's `.ux/matrix.json`. Replace the URL, journey selectors and expected values with the approved contract. Older education/finance persona examples are optional templates, not universal defaults; their login placeholders must be replaced with positive authentication checks.
+
+Targets use exact roles/names, labels, placeholders, text or explicit selectors. Ambiguous matches fail instead of silently clicking the first result.
+
+Step types:
+
+- `goto`, `reload`, `click`, `fill`, `press`, `select`, `check`, `wait`, `screenshot`.
+- `expectText`, `expectNoText`, `expectURL`, `expectVisible`, `expectHidden`, `expectValue`, `expectChecked`, `expectEnabled`, `expectCount`, `expectAttribute`.
+- `waitForText`, `measureResponse` (requires observed output and stable completion before the deadline).
+
+For object steps, put the target and expected value together, e.g. `{"expectValue":{"label":"Display name","value":"QA test user"}}`. A count uses `{"expectCount":{"selector":"[role=row]","count":3}}`. Attributes use target + `attribute` + `value`. Positive assertions retry within the step timeout. Negative text checks are point-in-time; place them after a positive loaded-state assertion. A script without at least one successful required explicit acceptance assertion fails rather than claiming UAT acceptance.
+
+Required-step failures and empty/missing journeys fail (exit 1). Comparisons fail with exit 2 for current failures, missing baselines or regressions. Optional-step failures are reported but do not alone fail the journey. All run output directories must be new; existing evidence/baselines are never overwritten. Create a new baseline directory to intentionally rebaseline.
+
+## CLI
+
+From the installed package directory (or this workspace):
+
+```sh
+node scripts/affordance_scan.mjs --url http://localhost:3000/dashboard --out .ux/observe-1
+node scripts/affordance_scan.mjs --url http://localhost:3000/dashboard --probe-clicks --interact --max-actions 30 --out .ux/explore-1
+node scripts/journey_runner.mjs --config .ux/matrix.json --all --journey profile-persistence --interact --allow-writes --out .ux/uat-1
+node scripts/journey_runner.mjs --config .ux/matrix.json --all --journey navigation --interact --baseline .ux/baseline-1
+node scripts/journey_runner.mjs --config .ux/matrix.json --all --journey navigation --interact --compare .ux/baseline-1 --out .ux/compare-1
+node scripts/glitch_lab.mjs nav --url http://localhost:3000/dashboard --interact --out .ux/glitch-1
+node scripts/report.mjs .ux --out .ux/report.html --embed
+```
+
+Use production-like builds: development hot reload/double rendering creates noise. React 18/19 sensors use DevTools internals and bounded traversal; minified names and unavailable sensors limit attribution. Screencast frames are sampled, not a guarantee of every painted frame. A detected correlation is not proof of root cause. DOM evidence frames may surround rather than capture the transient. A clean run means nothing detected under the reported conditions, not no defects.
+
+## Development
+
+From the repository root: `pnpm install --frozen-lockfile`, then `pnpm test` through your check broker. Browser tests use an ephemeral loopback fixture, not your apps/accounts or AI. Install Chromium explicitly or use the installed Chrome fallback. See the [repository release guide](https://github.com/stacknyu/stack-pi-plugins#release-one-plugin) for independent npm publishing. No GitHub Actions required.

package/examples/glitchy-app/README.md

@@ -0,0 +1,14 @@
+# Glitchy example app
+
+A small React 19 app with three planted glitches, for checking that Glitch Lab works on your machine:
+
+1. **Sidebar rebuilt on every navigation.** Each page renders its own `<Sidebar>`, which refetches the user and flashes a skeleton.
+2. **Chat box rebuilt every 1.5 s.** `ChatBox` is defined inside `ChatPage`, and a poll clears the messages before refetching them.
+3. **Header credits flash "…".** They're reset on every route change.
+
+```bash
+node examples/glitchy-app/serve.mjs          # http://localhost:4321/dashboard
+```
+
+Then ask Pi: "glitch hunt http://localhost:4321/dashboard", and "idle watch http://localhost:4321/chat".
+You should get all three as HIGH findings. Source is in `app.jsx`; the prebuilt bundle is in `public/app.js`.

package/examples/glitchy-app/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,27 @@
+# Third-party notices
+
+The prebuilt `public/app.js` demonstration bundle contains React, React DOM and Scheduler code from Meta, under the MIT license. These are embedded demonstration dependencies, not Glitch Lab runtime dependencies. Original license headers remain in the bundle.
+
+Source/license: https://github.com/facebook/react/blob/main/LICENSE
+
+MIT License
+
+Copyright (c) Meta Platforms, Inc. and affiliates.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/examples/glitchy-app/app.jsx

@@ -0,0 +1,62 @@
+import React, { useEffect, useState } from "react";
+import { createRoot } from "react-dom/client";
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+const api = async (path, ms = 150) => { await sleep(ms); return fetch(path).then((r) => r.json()).catch(() => ({})); };
+
+function useRoute() {
+  const [path, setPath] = useState(location.pathname);
+  useEffect(() => { const f = () => setPath(location.pathname); addEventListener("popstate", f); return () => removeEventListener("popstate", f); }, []);
+  const go = (p) => { history.pushState({}, "", p); setPath(p); };
+  return [path, go];
+}
+
+// BUG 3: header re-fetches credits on every route change and shows "…" meanwhile (text flap)
+function Header({ path }) {
+  const [credits, setCredits] = useState(120);
+  useEffect(() => { setCredits(null); api("/api/credits.json", 120).then(() => setCredits(120)); }, [path]);
+  return <header style={{ background: "#1e3a8a", color: "#fff", padding: 14, height: 24 }}>Platform · Credits: <b id="credits">{credits ?? "…"}</b></header>;
+}
+
+// BUG 1: each page renders its own Sidebar -> unmount+mount on every navigation, and it fetches the user on mount (skeleton flash)
+function Sidebar({ go }) {
+  const [user, setUser] = useState(null);
+  useEffect(() => { api("/api/me.json", 180).then(() => setUser({ name: "Arjun" })); }, []);
+  const item = (p, label) => <a href={p} onClick={(e) => { e.preventDefault(); go(p); }} style={{ display: "block", padding: "8px 0", color: "#111" }}>{label}</a>;
+  return (
+    <nav aria-label="Main" style={{ position: "absolute", left: 0, top: 52, width: 180, bottom: 0, background: "#f1f5f9", padding: 10 }}>
+      {user ? <div style={{ fontWeight: 700 }}>{user.name}</div> : <div style={{ height: 18, width: 90, background: "#cbd5e1" }} />}
+      {item("/dashboard", "Dashboard")}{item("/chat", "Chat")}{item("/notes", "Notes")}{item("/research", "Research")}
+    </nav>
+  );
+}
+
+function Main({ children }) { return <main style={{ marginLeft: 210, padding: 16 }}>{children}</main>; }
+
+function DashboardPage({ go }) {
+  return <><Sidebar go={go} /><Main><h1>Dashboard</h1><div className="card" style={{ background: "#fde68a", padding: 20 }}>Market overview: NIFTY 24,310</div></Main></>;
+}
+function NotesPage({ go }) {
+  return <><Sidebar go={go} /><Main><h1>Notes</h1><p>Germany checklist</p></Main></>;
+}
+function ResearchPage({ go }) {
+  return <><Sidebar go={go} /><Main><h1>Research</h1><p>No reports yet</p></Main></>;
+}
+
+// BUG 2: ChatBox is defined inside ChatPage (new component type every render) + a 1.5s poll re-renders ChatPage,
+// and messages are reset to [] before refetch -> list flashes empty and the input box remounts
+function ChatPage({ go }) {
+  const [messages, setMessages] = useState(["Hi! How can I help with admissions?"]);
+  const [tick, setTick] = useState(0);
+  useEffect(() => { const t = setInterval(() => setTick((x) => x + 1), 1500); return () => clearInterval(t); }, []);
+  useEffect(() => { if (!tick) return; setMessages([]); api("/api/messages.json", 90).then(() => setMessages(["Hi! How can I help with admissions?"])); }, [tick]);
+  function ChatBox() { return <div className="chatbox" style={{ border: "1px solid #999", padding: 8, marginTop: 8 }}><input aria-label="Message" placeholder="Ask anything" style={{ width: 300 }} /><button>Send</button></div>; }
+  return <><Sidebar go={go} /><Main><h1>Chat</h1><div style={{ minHeight: 40, background: "#eef2ff", padding: 8 }}>{messages.map((m, i) => <div key={i}>{m}</div>)}</div><ChatBox /></Main></>;
+}
+
+function App() {
+  const [path, go] = useRoute();
+  const Page = { "/chat": ChatPage, "/notes": NotesPage, "/research": ResearchPage }[path] || DashboardPage;
+  return <><Header path={path} /><Page go={go} /></>;
+}
+createRoot(document.getElementById("root")).render(<App />);

package/examples/glitchy-app/public/api/credits.json

@@ -0,0 +1 @@
+{"ok":1}

package/examples/glitchy-app/public/api/me.json

@@ -0,0 +1 @@
+{"ok":1}

package/examples/glitchy-app/public/api/messages.json

@@ -0,0 +1 @@
+{"ok":1}