site-agent-pro 1.0.0

package/docs/01-installation.md

@@ -0,0 +1,134 @@
+# 01 - Installation
+
+There are two ways to use site-agent-pro. Choose the one that fits your workflow.
+
+---
+
+## Track A — devDependency (recommended for most developers)
+
+This is the right choice if you want to audit your own product as you build it, run audits in CI, or call site-agent-pro from scripts and test files.
+
+### 1. Prerequisites
+
+- Node.js 20.10 or newer
+- npm 10 or newer
+
+```bash
+node -v
+npm -v
+```
+
+### 2. Install in your project
+
+```bash
+npm install --save-dev site-agent-pro
+```
+
+### 3. Install the Playwright browser
+
+```bash
+npx playwright install chromium
+```
+
+This only needs to be done once per machine or CI environment.
+
+### 4. Set your API key
+
+Create a `.env` file (or set environment variables) in your project root:
+
+```bash
+OPENAI_API_KEY=your_real_key_here
+```
+
+Or use Ollama for local/offline development (no API key needed):
+
+```bash
+LLM_PROVIDER=ollama
+OLLAMA_MODEL=llama3.1:8b
+```
+
+### 5. Run your first audit
+
+```bash
+# Against your running dev server
+site-agent-pro --url http://localhost:3000 --task "Check the homepage CTA"
+
+# Or add it to your package.json scripts
+# "audit": "site-agent-pro --url http://localhost:3000 --task 'Check the homepage'"
+# npm run audit
+```
+
+### 6. Or use it programmatically
+
+```ts
+import { runAudit } from "site-agent-pro";
+
+const result = await runAudit({
+  url: "http://localhost:3000",
+  tasks: ["Check the homepage CTA", "Try the signup flow"],
+});
+
+console.log(`Score: ${result.report.overall_score}/10`);
+
+if (result.report.overall_score < 7) {
+  process.exit(1); // Fail CI
+}
+```
+
+---
+
+## Track B — Clone and run (for contributors and self-hosting)
+
+Use this if you want to run the full web dashboard, modify the source, or self-host the submission server.
+
+### 1. Prerequisites
+
+- Node.js 20.10 or newer
+- npm 10 or newer
+- Git
+
+```bash
+node -v
+npm -v
+```
+
+### 2. Clone the repository
+
+```bash
+git clone https://github.com/your-org/site-agent-pro.git
+cd site-agent-pro
+```
+
+### 3. Install dependencies
+
+```bash
+npm install
+```
+
+### 4. Install the Playwright browser
+
+```bash
+npm run browser:install
+```
+
+### 5. Create your environment file
+
+```bash
+cp .env.example .env
+```
+
+### 6. Add your OpenAI API key
+
+Open `.env` and set:
+
+```bash
+OPENAI_API_KEY=your_real_key_here
+```
+
+### 7. Confirm TypeScript builds cleanly
+
+```bash
+npm run check
+```
+
+If this fails, do not keep going and pretend everything is fine. Fix the error first.

package/docs/02-running-your-first-audit.md

@@ -0,0 +1,136 @@
+# 02 - Running Your First Audit
+
+## Which command to use?
+
+| Setup | Command |
+|---|---|
+| Installed as npm devDependency | `site-agent-pro --url ... --task "..."` |
+| Cloned from source | `npm run dev -- --url ... --task "..."` |
+
+All examples below use the `site-agent-pro` command. If you cloned the repo, replace it with `npm run dev --`.
+
+---
+
+## 1. Start with a simple public site
+
+```bash
+site-agent-pro --url https://example.com --task "Open pricing and compare the visible plans before signup"
+```
+
+This creates a new run directory in `runs/`.
+If you want the full local product flow, start the app with `npm run dashboard` and submit the URL through `http://localhost:4173/`.
+
+---
+
+## 2. Run against your own localhost dev server
+
+```bash
+# Start your app first
+npm run dev
+
+# Then in another terminal
+site-agent-pro --url http://localhost:3000 --task "Check the homepage CTA"
+```
+
+This is the core use case for side-by-side development: catch UX issues as you build, not after.
+
+---
+
+## 3. Inspect the output
+
+Each run produces a timestamped directory in `runs/` containing:
+
+- `inputs.json`
+- `raw-events.json`
+- `task-results.json`
+- `accessibility.json`
+- `report.json`
+- `report.html`
+- `report.md`
+- `click-replay.webp` (animated replay)
+- `*.webm` (full video recording if `RECORD_VIDEO=true`)
+
+Open `report.html` in your browser for a readable, standalone report.
+
+---
+
+## 4. Run in a visible browser
+
+Use this while debugging interaction issues:
+
+```bash
+site-agent-pro --url https://example.com --task "Open pricing" --headed
+```
+
+---
+
+## 5. Run as a mobile user
+
+```bash
+site-agent-pro --url https://example.com --task "Check the mobile nav" --mobile
+```
+
+---
+
+## 6. Bootstrap an authenticated session
+
+When the site requires signup, email verification, OTP, or login before the important content is visible:
+
+```bash
+site-agent-pro --url https://example.com \
+  --task "Reach the account dashboard and confirm billing is visible" \
+  --auth-flow --signup-url /register --login-url /login --access-url /dashboard \
+  --headed
+```
+
+If you only want the authenticated Playwright session file and not the task run:
+
+```bash
+site-agent-pro --url https://example.com \
+  --auth-only --signup-url /register --login-url /login --access-url /dashboard
+```
+
+This writes `auth-flow.json` into the run directory and saves the authenticated `storageState` so future runs can reuse it directly.
+
+---
+
+## 7. Use it programmatically (devDependency users)
+
+Instead of the CLI, call site-agent-pro from a script or test file:
+
+```ts
+import { runAudit } from "site-agent-pro";
+
+const result = await runAudit({
+  url: "http://localhost:3000",
+  tasks: [
+    "Open pricing and compare the visible plans before signup",
+    "Click the sign-up button and check the form fields",
+  ],
+});
+
+console.log(`Score: ${result.report.overall_score}/10`);
+console.log(`Strengths: ${result.report.strengths.join(", ")}`);
+console.log(`Top fixes: ${result.report.top_fixes.join(", ")}`);
+```
+
+---
+
+## 8. Read the task output correctly
+
+Do not treat the overall score as objective truth.
+Use the output to answer:
+- what users could do
+- where they got stuck
+- what broke trust
+- what to fix first
+
+---
+
+## 9. Use the hosted output flow (self-hosted setup only)
+
+When you run the local app server:
+- submit a public URL from `/`
+- check status at `/submissions/<submission-id>`
+- open the unique public task-output link at `/r/<token>`
+- download the finished output from `/dashboard`

package/docs/03-configuration.md

@@ -0,0 +1,233 @@
+# 03 - Configuration
+
+## Environment variables
+
+### `OPENAI_API_KEY`
+Required. Your API key.
+
+### `OPENAI_MODEL`
+Default: `gpt-5`
+
+Change this if you want a different compatible model.
+
+### `APP_BASE_URL`
+Default: `http://localhost:4173`
+
+Used when building hosted task-output links.
+
+### `HEADLESS`
+Default: `true`
+
+Set to `false` if you want the browser visible by default.
+
+### `MAX_SESSION_DURATION_MS`
+Default: `600000`
+
+Caps a single audit at 10 minutes in V1.
+The code enforces a hard ceiling of 600 seconds even if you set a larger value.
+
+### `MAX_STEPS_PER_TASK`
+Default: `32`
+
+The default now leans toward a forensic investigation across multiple focused coverage lanes instead of one vague exploration pass.
+The runner also preserves time for later tasks and supplemental site checks, so increasing this does not guarantee more useful coverage.
+Raise this only when tasks genuinely require even longer flows. Bigger numbers can still make the agent wander if the site has poor signals.
+
+### `ACTION_DELAY_MS`
+Default: `600`
+
+Extra delay between actions. Useful when sites animate heavily.
+
+### `NAVIGATION_TIMEOUT_MS`
+Default: `25000`
+
+Increase this for painfully slow sites.
+This timeout also affects the supplemental site probes that power performance, SEO, security, mobile, and content coverage.
+
+### `REPORT_TTL_DAYS`
+Default: `30`
+
+Hosted public task-output links expire after this many days.
+
+### `RECORD_VIDEO`
+Default: `false`
+
+When set to `true`, Playwright captures a full video recording of every browser session. These recordings are saved in the run directory and are viewable in the dashboard alongside the animated WebP replays.
+
+### `PLAYWRIGHT_STORAGE_STATE_PATH`
+Default: unset
+
+Optional path to a Playwright `storageState` JSON file.
+Use this when your approved test lane already has a legitimate verified or authenticated session and you want the CLI or local app to reuse it automatically.
+
+## Coverage playbook
+
+If you want the fewest possible `blocked` metrics:
+
+- Prefer sites or QA lanes that are reachable without CAPTCHA, Cloudflare challenges, or geo/IP throttling.
+- Reuse a legitimate session with `PLAYWRIGHT_STORAGE_STATE_PATH` or `--storage-state` when important paths sit behind login or verification.
+- Raise `NAVIGATION_TIMEOUT_MS` for slow sites before raising `MAX_STEPS_PER_TASK`.
+- Keep `MAX_SESSION_DURATION_MS` near the 10-minute ceiling for deeper task runs.
+- Use multiple agent perspectives in the submission form when you want broader behavioral coverage, not just deeper repetition from one agent.
+
+### `DASHBOARD_PORT`
+Default: `4173`
+
+Port used by the local app server.
+
+### `DASHBOARD_HOST`
+Default: `127.0.0.1`
+
+Host binding used by the local app server.
+
+## Auth bootstrap variables
+
+These are needed when you bootstrap a new auth identity with `--auth-flow` or `--auth-only`.
+After a successful auth run, the runner also caches the working credentials in `.auth/credentials.json` keyed by target origin, so later runs against the same site can reuse them automatically.
+
+### `AUTH_TEST_EMAIL`
+Required for auth bootstrap.
+
+The base mailbox address the runner uses for signup and login.
+On the first signup attempt it uses this exact address.
+If the site says the account already exists, the runner now retries with fresh plus-address aliases such as `name+siteagent-...@domain.com` so it can keep registering without manual edits.
+
+### `AUTH_TEST_PASSWORD`
+Required for auth bootstrap.
+
+The password the runner uses for both signup and login.
+
+### `AUTH_TEST_USERNAME`
+Optional.
+
+Use this when the site expects a username field that is different from the email address. If omitted, the runner derives a fallback username from the configured email address.
+
+### `AUTH_TEST_FIRST_NAME` through `AUTH_TEST_COMPANY`
+Defaults are provided in `.env.example`.
+
+These values are used to fill visible signup fields such as name, phone, address, city, state, postal code, country, and company.
+When the runner has to retry signup with a fresh identity, it also adds small numeric variations to these details so sites that enforce uniqueness beyond email are less likely to reject the retry.
+
+### `AUTH_IMAP_HOST`, `AUTH_IMAP_PORT`, `AUTH_IMAP_SECURE`, `AUTH_IMAP_USER`, `AUTH_IMAP_PASSWORD`, `AUTH_IMAP_MAILBOX`
+
+Configure the real inbox the runner should poll for OTP or verification emails.
+The auth bootstrap uses IMAP mailbox access, not a browser-driven webmail tab.
+
+### `AUTH_EMAIL_POLL_TIMEOUT_MS`
+Default: `180000`
+
+How long to wait for the verification email before failing the auth bootstrap.
+
+### `AUTH_EMAIL_POLL_INTERVAL_MS`
+Default: `5000`
+
+How frequently to poll the inbox for a new message.
+
+### `AUTH_OTP_LENGTH`
+Default: `6`
+
+Expected OTP length for numeric code extraction.
+
+### `AUTH_EMAIL_FROM_FILTER`
+Optional.
+
+Use this when the mailbox receives lots of unrelated email and you want to constrain matching to a specific sender.
+
+### `AUTH_EMAIL_SUBJECT_FILTER`
+Optional.
+
+Use this when the mailbox receives lots of unrelated email and you want to constrain matching to a specific subject fragment.
+
+### `AUTH_GENERATED_IDENTITY_MAX_ATTEMPTS`
+Default: `5`
+
+How many signup identities the runner should try before giving up when the site keeps reporting that the account already exists.
+
+### `AUTH_SIGNUP_URL`, `AUTH_LOGIN_URL`, `AUTH_ACCESS_URL`
+Optional.
+
+Default auth flow URLs used by the CLI when you do not pass `--signup-url`, `--login-url`, or `--access-url`.
+
+If auth credentials are configured and a normal task run lands on a real login or registration wall, the runner can also attempt an automatic in-session signup/login recovery using the current blocked page as the protected destination to re-open.
+
+### `AUTH_SESSION_STATE_PATH`
+Default: `.auth/session.json`
+
+Where the authenticated Playwright session is saved if you do not explicitly pass `--save-storage-state`.
+
+## CLI flags
+
+### `--url`
+Required website URL.
+
+### `--task`
+Required for task runs. Repeat it for each accepted task you want the agent to perform.
+
+Example:
+
+```bash
+npm run dev -- --url https://example.com --task "Open pricing and compare the visible plans" --task "Reach the signup page without creating an account"
+```
+
+### `--headed`
+Shows the browser.
+
+### `--mobile`
+Uses a mobile browser profile.
+
+### `--ignore-https-errors`
+Allows invalid or self-signed HTTPS certificates.
+
+Useful for local development sites such as:
+
+```bash
+npm run dev -- --url https://localhost:3000 --ignore-https-errors
+```
+
+### `--storage-state`
+Loads a Playwright `storageState` JSON file for a single run.
+
+Example:
+
+```bash
+npm run dev -- --url https://example.com --storage-state .auth/session.json
+```
+
+### `--save-storage-state`
+Saves the Playwright `storageState` JSON after the run finishes.
+
+Example:
+
+```bash
+npm run dev -- --url https://example.com --storage-state .auth/session.json --save-storage-state .auth/session.json
+```
+
+### `--auth-flow`
+Runs the auth bootstrap first, then continues the accepted task run with the authenticated session.
+
+Example:
+
+```bash
+npm run dev -- --url https://example.com --auth-flow --signup-url /register --login-url /login --access-url /app
+```
+
+### `--auth-only`
+Runs only the auth bootstrap and saves the authenticated session without generating a task output.
+
+### `--signup-url`
+Optional absolute or relative signup URL for auth bootstrap.
+
+### `--login-url`
+Optional absolute or relative login URL for auth bootstrap.
+
+### `--access-url`
+Optional absolute or relative protected URL used to confirm the session can reach authenticated content after login.
+
+## Local app routes
+
+After running `npm run dashboard`:
+- `/` is the public submission form
+- `/dashboard` is the internal run dashboard
+- `/submissions/<id>` is the submission status page
+- `/r/<token>` is the public task-output link
+- `/outputs/<run-id>` is the standalone HTML output route

package/docs/04-how-the-agent-thinks.md

@@ -0,0 +1,41 @@
+# 04 - How the Agent Thinks
+
+## The execution loop
+
+For each task, the system does this:
+
+1. capture visible page state
+2. ask the model for the next realistic user action
+3. execute the action with guarded locators
+4. log what happened
+5. repeat until the task ends or the step limit is hit
+
+## Why the planner and evaluator are separate
+
+If one model both acts and judges, it will flatter itself and invent success.
+That is weak design.
+
+This project separates:
+- **planner**: chooses the next action
+- **evaluator**: reviews the evidence afterward
+
+## What the planner sees
+
+The planner gets:
+- page title and URL
+- visible body text excerpt
+- visible interactive elements
+- headings
+- modal hints
+- previous action history
+
+## What the planner does not get
+
+It does not get:
+- hidden DOM content
+- fake claims that something succeeded
+
+## Why this matters
+
+You wanted a system that behaves like a regular user.
+Regular users do not inspect invisible elements or parse the entire DOM perfectly.

package/docs/05-extending-personas-and-tasks.md

@@ -0,0 +1,42 @@
+# 05 - Extending Accepted Tasks
+
+## Define tasks from explicit input
+
+There are no built-in personas or default task files anymore.
+Each run is driven by accepted tasks submitted from the dashboard or passed to the CLI with repeated `--task` flags.
+
+Example CLI input:
+
+```bash
+npm run dev -- --url https://example.com \
+  --task "Find the pricing page and compare the visible plans" \
+  --task "Open the contact path and confirm whether support is easy to reach"
+```
+
+For game-oriented runs, write the requested behavior directly into the accepted tasks. Example: read the visible how-to-play section, reach a playable state, and play five rounds while recording wins and losses.
+
+## Good task design
+
+A good task is:
+- concrete
+- time-bounded
+- observable
+- easy to judge from evidence
+- complementary with the other tasks in the suite
+
+Good task sets usually split coverage into a few lanes such as:
+- main journey and orientation
+- discovery and information architecture
+- conversion and trust
+- suspicious interactions and recovery states
+
+That gives the runner broader coverage without asking one task to explain the whole site alone.
+
+## Bad task design
+
+Trash tasks look like this:
+- “Explore the site”
+- “See if it is good”
+- “Understand everything”
+
+Those are vague, hard to score, and guaranteed to produce mush.

package/docs/06-hardening-for-production.md

@@ -0,0 +1,92 @@
+# 06 - Hardening for Production
+
+## 0. CI integration (devDependency users)
+
+This is the natural endpoint of side-by-side development: once you trust the audit scores on your own product, automate them.
+
+### Run site-agent-pro in CI against a preview URL
+
+```yaml
+# Example: GitHub Actions
+- name: Run site-agent-pro audit
+  env:
+    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+  run: |
+    site-agent-pro --url ${{ env.PREVIEW_URL }} \
+      --task "Check the homepage CTA" \
+      --task "Open pricing and compare the visible plans"
+
+> [!TIP]
+> You can also pass secrets directly via CLI flags (e.g., `--openai-api-key ${{ secrets.OPENAI_API_KEY }}`) if you prefer not to map them as environment variables.
+```
+
+### Gate on a minimum score using the programmatic API
+
+Create an `audit.mjs` script in your project:
+
+```ts
+import { runAudit } from "site-agent-pro";
+
+const result = await runAudit({
+  url: process.env.PREVIEW_URL ?? "http://localhost:3000",
+  tasks: [
+    "Check the homepage CTA",
+    "Open pricing and compare the visible plans",
+  ],
+});
+
+console.log(`Score: ${result.report.overall_score}/10`);
+result.report.top_fixes.forEach((fix) => console.log(`Fix: ${fix}`));
+
+if (result.report.overall_score < 7) {
+  console.error("Audit score below threshold. Failing build.");
+  process.exit(1);
+}
+```
+
+```yaml
+- name: Run audit gate
+  run: node audit.mjs
+```
+
+> **Important:** Do not add CI gating until you have manually reviewed enough runs on your own product to understand the score range. Arbitrary thresholds will create false failures and destroy trust in the tool.
+
+---
+
+## 1. Add retries carefully
+
+Retrying every failed action blindly is lazy and dangerous.
+Add retries only for:
+- network hiccups
+- delayed rendering
+- slow client-side routing
+
+## 2. Improve task completion checks
+
+The current completion logic is conservative but still heuristic.
+Production systems should add explicit validators per task.
+
+Examples:
+- pricing check should detect real price patterns
+- contact check should verify actual support/contact details
+- signup check should verify the next-step form is real and usable
+
+## 3. Improve event-aware evaluation
+
+Right now the evaluator relies on interaction logs, task outcomes, and accessibility findings.
+If you want better judgment later, enrich the structured events instead of adding guesswork.
+
+## 4. Add category-specific personas
+
+Use different task sets for:
+- SaaS marketing sites
+- ecommerce stores
+- docs portals
+- recruiting pages
+- local business websites
+
+## 5. Add CI only after manual trust is earned
+
+Do not turn this into a pipeline gate until you have manually reviewed enough runs to understand its failure modes.
+
+