samey-cli 0.0.0-stage → 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+## Unreleased
+
+### Renamed
+- The project is now **samey** (was unslop): the npm name `unslop` belongs to an unrelated tool. The command, report branding, output folder (`.samey/`), and config file (`samey.config.json`) all changed. The npm package is `samey-cli` (npm rejects `samey` as too similar to existing packages); the installed command is `samey`.
+
+### Security
+- All browser requests are intercepted and performed by a Node client that validates the URL, resolves DNS, rejects names with any non-public address, and pins the connection to the validated address (DNS rebinding).
+- Redirects are re-validated hop by hop; loops and long chains are stopped. Per-response (20 MB) and per-crawl (250 MB, 3,000 requests) limits.
+- Loopback, private, link-local, reserved, and cloud metadata addresses are blocked, including IPv4-mapped, NAT64, and 6to4 forms and numeric host tricks.
+- Service workers, downloads, pop-ups, and (in normal mode) WebSockets are blocked; stray Chromium traffic goes to a dead proxy.
+- New `--dev` flag required to audit localhost or private `--url` targets. Metadata addresses stay blocked.
+- Secrets in excerpts are redacted; source paths in findings must be relative, inside the project, and in the scanned file set.
+
+### Added
+- Every audit writes `.samey/fix-prompt.md`: a ready-to-paste prompt for Claude Code, Cursor or Codex, measured defects first, then design observations, each with page, selector, file:line and the fix. `samey prompt` prints it (`--only-defects` to skip observations). The report's copy buttons use the same text.
+- `samey setup` installs Chromium with samey's bundled Playwright.
+- The NovaFlow before/after example (`examples/novaflow`) and a demo GIF in the README.
+
+### Fixed
+- HTML document boilerplate (doctype, head, meta tags) shared by every page was reported as copy-pasted component markup.
+- Sites that redirect (bare domain to `www`, `http` to `https`, moved pages) failed every page with `ERR_PROXY_CONNECTION_FAILED`: Chromium followed redirects from intercepted responses outside the guard. Navigation redirects are now re-checked and followed by the crawler, subresource redirects are followed hop by hop in Node, and a redirecting start URL adopts its final origin.
+- Requests samey itself blocked (for example a video over the 20 MB response cap) were reported as failed assets and console errors on the site. They are now listed only under coverage.
+- Redirected pages are recorded under the URL they landed on and are not audited twice.
+- Reports and progress output show local paths relative to home or the working directory instead of the full path.
+
+### Changed (breaking)
+- Score is inverted to match the brief: 0 = no slop indicators, 100 = many or severe. Lower is better.
+- Scoring uses `p = s × c × w` per finding and `100 × (1 − e^(−P/100))` per category, with brief weights: visual 40%, content 20%, functional 15%, accessibility 15%, technical 10%. Unmeasured categories are excluded and the rest renormalized.
+- Grade labels ("Distinct", "Generic", ...) removed until calibrated.
+- Finding schema: `description` (was `message`), numeric `confidence` 0..1, `fingerprint`, `verified`, `ruleVersion`, `routes`. Evidence uses `type`, `route`, `excerpt`, `screenshotPath`, `alsoOn`.
+- Report adds `schemaVersion`, `scanId`, `project`, `versions`, `viewports`, `failedRoutes`, `coverage.limitations`, `coverage.blockedRequests`.
+- `--fail-under` replaced by `--fail-above`. `--report` added as an alias for `--format`.
+
+### Added
+- Deduplication by fingerprint; identical evidence across routes collapses into one item, penalized once.
+- Per-rule penalty weights; convention-based rules weigh less.
+- A single modest stock gradient is reported as `info` with no penalty.
+
+## 0.1.0
+
+- First vertical slice: CLI, repository scan, Playwright crawl at two viewports, 37 rules, scoring, self-contained HTML and JSON reports, optional local Ollama critique, three fixtures, tests.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 samey contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,225 @@
-# Temporary Holding Version
+# samey
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Is your AI-built site samey?** samey audits a website built with Claude Code, Cursor, Codex or any other coding agent, shows you what makes it look like every other one, and writes a prompt your agent can use to fix it.
+
+```bash
+npx samey-cli audit .
+```
+
+Or install it once with `npm i -g samey-cli` and run `samey audit .`.
+
+![samey auditing a generic landing page, writing a fix prompt, and re-auditing the fixed site](docs/samey-demo.gif)
+
+> **Early release.** The rules work and are tested, but the scoring weights have not been calibrated against human judgment yet. If samey flags something that looks fine to you, or misses something obvious, please [open an issue](https://github.com/azkaiftikhar01/samey/issues) with the URL or a screenshot. That feedback is exactly what will make it accurate.
+
+## What it catches
+
+- **Stock visuals:** the blue-violet-pink gradient, gradient headline text, glows and blurred blobs, emoji used as icons, everything centered.
+- **Sameness:** the same three-card grid in every section, the hero-logos-features-testimonials-pricing-FAQ sequence, different pages built from one skeleton.
+- **Template copy:** "Unlock the power of seamless...", lorem ipsum, Jane Doe at Acme, unsourced "Trusted by 10,000+ teams".
+- **Things that are simply broken:** links to `#`, buttons that do nothing, broken links and images, console errors, pages that scroll sideways on a phone.
+- **Accessibility gaps:** buttons without names, images without alt text, unlabeled inputs, tiny touch targets.
+
+Every finding has evidence you can check: the page and viewport, the CSS selector, a screenshot crop with the element outlined, and the source file and line when samey can trace it. Measured defects are kept separate from design observations, which are heuristics that need your judgment. Popular conventions are not penalized on their own.
+
+samey runs entirely on your machine: no account, no API key, no cloud. It measures what is on the page; it does not try to guess who or what wrote it.
+
+## Before and after
+
+![The same NovaFlow landing page before and after fixing samey's findings](docs/before-after.jpg)
+
+| | Slop score | Findings |
+| --- | ---: | ---: |
+| [Before](examples/novaflow/before) | 53.3 | 26 |
+| [After](examples/novaflow/after) | 0 | 0 |
+
+The before site is a deliberately generic landing page. samey produced [this fix prompt](examples/novaflow/fix-prompt.md) for it, and the after site is that site rewritten from the prompt by a coding agent: one deliberate accent colour instead of the gradient, a product preview and a worked example instead of three-card grids, a pricing table, real links, and the invented testimonials and user counts removed rather than reworded. Run both yourself:
+
+```bash
+npx samey-cli audit examples/novaflow/before
+npx samey-cli audit examples/novaflow/after
+```
+
+A score of 0 means samey found nothing to flag, not that a site is perfect. It is a small demo site; real sites will rarely reach zero.
+
+## Quick start
+
+```bash
+npx samey-cli setup                    # once: downloads the Chromium build samey uses (about 150 MB)
+
+npx samey-cli audit .                  # a static build (out/, dist/, build/) or plain HTML
+npx samey-cli audit . --serve "npm run dev" --port 3000   # samey starts your dev server, audits, stops it
+npx samey-cli audit --url https://your-site.com           # a deployed site
+npx samey-cli audit . --url http://localhost:3000 --dev   # a dev server you already started
+```
+
+Each run writes three files to `.samey/` (add it to your `.gitignore`):
+
+- `report.html`: the full report, self-contained, open it in any browser
+- `fix-prompt.md`: a ready-to-paste prompt for your coding agent
+- `report.json`: everything, for scripts and agents
+
+## Fix it with your coding agent
+
+```bash
+npx samey-cli prompt | pbcopy          # macOS; or open .samey/fix-prompt.md
+```
+
+Paste it into Claude Code, Cursor or Codex. The prompt lists measured defects first, then design observations to weigh, each with the page, selector, file and line, and the fix. It tells the agent to ask you for anything only you can provide, like real testimonials or a brand colour, instead of inventing it. Then run `npx samey-cli audit` again to confirm.
+
+Prefer to pick? Tick findings in the HTML report and use "Copy fix prompt", or run `npx samey-cli prompt --only-defects` to skip the design observations.
+
+## Commands
+
+```
+samey audit [path] [options]   audit a project, a URL, or both (npx samey-cli audit ...)
+samey prompt [path]            print the fix prompt from the last audit
+samey setup                    install the browser samey uses
+samey rules                    list every rule
+```
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `-u, --url <url>` | | Audit a running site. Overrides any local server. |
+| `--dev` | | Development mode: lets `--url` point at localhost or a private network address. |
+| `--serve [command]` | | Start the dev server with this command (`npm run dev` if omitted). This runs project code, so it only happens when you pass the flag. |
+| `--port <port>` | 4173 | Port the dev server listens on. `PORT` is set in its environment. |
+| `--max-routes <n>` | 10 | Maximum pages to render. Skipped pages are listed in the report. |
+| `--timeout <seconds>` | 30 | Navigation timeout per page. |
+| `--route <path...>` | | Extra routes to audit. |
+| `--format <html\|json\|both>` | both | Which reports to write. `--report` is an alias. |
+| `-o, --out <dir>` | `<path>/.samey` | Output directory. |
+| `--static-only` | | Only scan source files; do not launch a browser. |
+| `--no-screenshots` | | Skip screenshots. |
+| `--no-interact` | | Do not click any buttons. |
+| `--ignore-robots` | | Crawl pages disallowed by robots.txt. Only for sites you own. |
+| `--ignore <rules...>` | | Rule ids to skip (see `samey rules`). |
+| `--fail-above <score>` | | Exit with code 1 if the slop score is above this. For CI. |
+| `--model <name>` | | Local Ollama vision model for an advisory critique. |
+| `--model-endpoint <url>` | `http://127.0.0.1:11434` | Must be a loopback address. |
+| `--no-ai` | | Never use a model, even if one is configured. |
+| `--json` | | Print the JSON report to stdout. |
+| `-q, --quiet` | | Only print the final summary. |
+
+Exit codes: `0` success, `1` score above `--fail-above`, `2` the audit could not run (bad arguments, a blocked or unreachable URL, no page could be rendered, dev server failed to start).
+
+### Config file
+
+`samey.config.json` in the project root sets defaults. Command-line flags win.
+
+```json
+{
+  "maxRoutes": 15,
+  "routes": ["/pricing", "/docs"],
+  "ignoreRules": ["centered-everything"],
+  "failAbove": 30,
+  "serve": "pnpm dev",
+  "port": 5173
+}
+```
+
+`serve` in the config file is only used when you also pass `--serve` on the command line, so cloning an untrusted repository and running `samey audit .` never executes its code.
+
+## How the target is chosen
+
+1. `--url` if given. Public addresses only, unless you pass `--dev`.
+2. Otherwise, with `--serve`, the dev server samey starts.
+3. Otherwise, a static build in the project (`out/`, `dist/`, `build/`, `_site/`), or the project root if it is plain HTML. samey serves it read-only on 127.0.0.1.
+4. Otherwise only static analysis runs, and the report says so. Categories that need a rendered page show n/a rather than a perfect score.
+
+Routes are discovered from links, `sitemap.xml`, `--route`, and route files in the project (Next.js `app/` and `pages/`, SvelteKit, Astro, React Router paths). Dynamic routes like `/blog/[slug]` are skipped.
+
+## What it checks
+
+Run `samey rules` for the full list. In summary:
+
+**Visual genericness**: blue-violet-pink gradients (including gradient text), decorative glows, blur blobs and glass panels, one card composition reused across sections, consecutive sections with identical layout, different pages built from the same skeleton, the stock hero-logos-features-testimonials-pricing-FAQ-CTA sequence, centered everything, emoji used as icons.
+
+**Copy and credibility**: stock marketing phrases ("unlock the power", "seamless", "next-generation"), placeholder content (lorem ipsum, Jane Doe, Acme, example.com, placeholder image services), repeated paragraphs, unsourced user counts and ratings, testimonials with only a name and title.
+
+**Functionality**: pages that fail to load, broken internal links, console errors, failed requests, links to `#`, buttons that do nothing when clicked, broken images, missing titles and descriptions.
+
+**Responsive and accessibility**: horizontal scrolling, touch targets under 24px, text under 12px on mobile, controls without accessible names, images without alt, unlabeled inputs, heading structure, vague link text.
+
+**Design consistency**: too many font sizes, radii, shadows, or font families; spacing off a 4px scale; hard-coded values bypassing tokens; markup copy-pasted instead of extracted into components.
+
+Static checks on the source tree mirror several rendered checks. When both fire, the source lines are attached to the rendered finding as evidence instead of being reported twice.
+
+### Safe click testing
+
+On each page, samey clicks up to six visible buttons to see whether anything happens. It never clicks links (those are checked by the crawler), anything inside a form, submit buttons, or anything labelled like delete, buy, pay, checkout, subscribe, log out, send, confirm, and similar. Use `--no-interact` to turn this off.
+
+## Scoring
+
+The score counts detected quality problems. It is not a probability that AI wrote the site.
+
+Each finding adds a penalty `p = s × c × w`:
+
+- `s`, severity: info 0, low 0.25, medium 0.5, high 0.75, critical 1
+- `c`, confidence: a number from 0 to 1
+- `w`, the rule's weight: 40 by default, lower for rules about conventions (centered layouts, the landing-page sequence, unverified claims), higher for hard failures (a page that does not load: 80)
+
+Within a category the penalties add up to `P`, and the category score is `100 × (1 − e^(−P/100))`, so a pile of minor findings approaches 100 but never passes it. The overall score is the weighted average of the measured categories:
+
+| Category | Weight |
+| --- | ---: |
+| Visual genericness | 40% |
+| Content genericness | 20% |
+| Functional defects | 15% |
+| Accessibility and usability | 15% |
+| Technical quality | 10% |
+
+Categories that could not be measured (for example functional defects in a static-only run) are shown as "not measured" and left out, with the remaining weights renormalized. Identical evidence on several pages, such as a broken link in a shared footer, counts once. The same set of findings always produces the same score. Weights and penalties are starting hypotheses, versioned in the report (`versions.scoring`), and have not yet been calibrated against human judgments, so there are deliberately no labels like "good" or "bad".
+
+## Network safety
+
+Every request the browser makes is intercepted and performed by samey itself, so the audited page cannot use the browser to reach places it should not:
+
+- Only `http` and `https`. URLs with embedded credentials are refused.
+- In normal mode only public addresses are reachable. Loopback, private (including carrier-grade NAT), link-local, multicast, documentation, and other reserved ranges are blocked, including IPv4-mapped, NAT64, and 6to4 IPv6 forms and numeric tricks like `http://2130706433/`.
+- Hostnames are resolved before connecting, every returned address must be allowed, and the connection is pinned to the checked address, which defeats DNS rebinding.
+- Every redirect hop is checked again. Redirect loops and chains longer than 5 hops are stopped.
+- Responses are capped at 20 MB each and 250 MB per crawl, with 3,000 requests at most.
+- Service workers, downloads, and pop-up windows are blocked. WebSockets are blocked in normal mode. Anything Chromium tries to send outside the interception (preconnects, WebRTC) goes to a closed local port.
+- Cloud metadata addresses (169.254.169.254 and others) are never reachable, even with `--dev`.
+- With `--dev`, loopback and private addresses are allowed so you can audit a local server. When samey starts the server itself (static build or `--serve`), only loopback is allowed.
+- Crawling stays on the starting origin. Pages that redirect to another site are skipped.
+
+Blocked requests are listed in the report's coverage section. Text on the audited page is treated as content, never as instructions.
+
+## Optional local model
+
+```bash
+ollama pull qwen2.5vl:7b
+samey audit . --model qwen2.5vl:7b
+```
+
+samey sends up to three desktop screenshots to the model and asks for specific visual observations. Only loopback endpoints are accepted, so screenshots never leave your machine. Model observations are marked advisory, given confidence 0.3, and never change the score. No model has been benchmarked with samey yet; the model name is recorded in the report. If no model is configured or the server is not running, the audit runs normally and the report notes it.
+
+## For coding agents
+
+`report.json` is designed to be read by agents directly. See [AGENTS.md](AGENTS.md) for its shape and a suggested fixing order.
+
+## Limits
+
+- Ten routes and two viewports by default. Large sites are sampled, not exhausted.
+- Accessibility checks are samey's own, not axe-core yet, and technical quality covers page metadata only (no Lighthouse yet). See PLAN.md.
+- Heuristics are tuned to avoid false positives on plain, well-built pages, but they are heuristics. Low-confidence findings are worth a look, not an automatic fix.
+- Single-page apps that render routes without real URLs only get their first screen audited.
+- samey never modifies your source files.
+
+## Development
+
+```bash
+npm install
+npx playwright install chromium
+npm test            # unit, security, and end-to-end tests
+npm run typecheck
+npm run dev -- audit fixtures/sloppy-site   # run from source
+```
+
+`fixtures/sloppy-site` is a deliberately generic multi-page site, `fixtures/clean-site` is a plain site that should produce no genericness findings, and `fixtures/react-app` is a source tree for static analysis. See [PLAN.md](PLAN.md) for status and next steps.
+
+## License
+
+MIT

package/dist/ai/local-model.d.ts

@@ -0,0 +1,26 @@
+import type { DraftFinding } from '../rules/types.js';
+import type { CrawlResult } from '../types.js';
+/**
+ * Optional visual critique by a local, open-weight vision model served by Ollama.
+ * Only loopback endpoints are accepted, so screenshots never leave the machine.
+ * Model output is advisory: it is shown in the report but never changes the score.
+ */
+export interface ModelOptions {
+    model: string;
+    endpoint: string;
+    timeoutMs: number;
+    maxPages: number;
+}
+export declare function assertLoopback(endpoint: string): URL;
+interface ModelFinding {
+    title: string;
+    observation: string;
+    recommendation: string;
+}
+export declare function parseModelOutput(raw: string): ModelFinding[];
+export declare function checkModel(opts: ModelOptions): Promise<{
+    ok: boolean;
+    note?: string;
+}>;
+export declare function critiqueScreens(crawl: CrawlResult, shotsDir: string, opts: ModelOptions): Promise<DraftFinding[]>;
+export {};

package/dist/ai/local-model.js

@@ -0,0 +1,91 @@
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+const LOOPBACK = new Set(['localhost', '127.0.0.1', '::1', '[::1]']);
+export function assertLoopback(endpoint) {
+    let u;
+    try {
+        u = new URL(endpoint);
+    }
+    catch {
+        throw new Error(`Invalid model endpoint: ${endpoint}`);
+    }
+    if (!LOOPBACK.has(u.hostname)) {
+        throw new Error(`Model endpoint must be on this machine (localhost/127.0.0.1/::1). Refusing to send screenshots to ${u.hostname}.`);
+    }
+    return u;
+}
+const PROMPT = `You are reviewing a screenshot of a web page for design quality. Focus only on what is visible.
+List up to 5 specific, observable problems that make the page look generic, template-like, or unfinished:
+for example weak visual hierarchy, stock gradients, interchangeable hero layout, decorative effects without purpose,
+cramped or inconsistent spacing, unclear primary action. Do not guess who made the site. Do not praise.
+Respond with JSON only: {"findings":[{"title":"short title","observation":"what you see and where","recommendation":"concrete fix"}]}`;
+export function parseModelOutput(raw) {
+    const start = raw.indexOf('{');
+    const end = raw.lastIndexOf('}');
+    if (start < 0 || end <= start)
+        return [];
+    try {
+        const parsed = JSON.parse(raw.slice(start, end + 1));
+        if (!Array.isArray(parsed.findings))
+            return [];
+        return parsed.findings
+            .filter((f) => !!f && typeof f === 'object' && typeof f.title === 'string' && typeof f.observation === 'string')
+            .map((f) => ({ title: f.title.slice(0, 100), observation: f.observation.slice(0, 500), recommendation: String(f.recommendation ?? '').slice(0, 400) }))
+            .slice(0, 5);
+    }
+    catch {
+        return [];
+    }
+}
+export async function checkModel(opts) {
+    const base = assertLoopback(opts.endpoint);
+    try {
+        const res = await fetch(new URL('/api/tags', base), { signal: AbortSignal.timeout(3000) });
+        if (!res.ok)
+            return { ok: false, note: `endpoint returned HTTP ${res.status}` };
+        const body = (await res.json());
+        const names = (body.models ?? []).map((m) => m.name);
+        if (!names.some((n) => n === opts.model || n.split(':')[0] === opts.model)) {
+            return { ok: false, note: `model "${opts.model}" is not pulled (available: ${names.join(', ') || 'none'}). Run: ollama pull ${opts.model}` };
+        }
+        return { ok: true };
+    }
+    catch {
+        return { ok: false, note: `no local model server at ${opts.endpoint}` };
+    }
+}
+export async function critiqueScreens(crawl, shotsDir, opts) {
+    const base = assertLoopback(opts.endpoint);
+    const findings = [];
+    const pages = crawl.pages.filter((p) => p.desktop?.screenshot).slice(0, opts.maxPages);
+    for (const p of pages) {
+        const img = readFileSync(path.join(shotsDir, p.desktop.screenshot)).toString('base64');
+        const res = await fetch(new URL('/api/generate', base), {
+            method: 'POST',
+            headers: { 'content-type': 'application/json' },
+            body: JSON.stringify({ model: opts.model, prompt: PROMPT, images: [img], stream: false, format: 'json', options: { temperature: 0.2 } }),
+            signal: AbortSignal.timeout(opts.timeoutMs),
+        });
+        if (!res.ok)
+            continue;
+        const body = (await res.json());
+        for (const mf of parseModelOutput(body.response ?? '')) {
+            findings.push({
+                ruleId: 'model-critique',
+                ruleVersion: 1,
+                verified: false,
+                title: mf.title,
+                category: 'visual',
+                severity: 'info',
+                confidence: 0.3,
+                source: 'model',
+                advisory: true,
+                description: mf.observation,
+                recommendation: mf.recommendation || 'Review the screenshot and decide whether this applies.',
+                evidence: [{ type: 'screenshot', route: p.path, viewport: 'desktop', screenshotPath: `shots/${p.desktop.screenshot}`, detail: `${opts.model} reviewing the desktop screenshot` }],
+                routes: [p.path],
+            });
+        }
+    }
+    return findings;
+}

package/dist/audit.d.ts

@@ -0,0 +1,18 @@
+import type { AuditOptions } from './config.js';
+import type { AuditReport } from './types.js';
+export declare function toolVersion(): string;
+export interface AuditResult {
+    report: AuditReport;
+    files: {
+        html?: string;
+        json?: string;
+        prompt?: string;
+    };
+    ruleErrors: {
+        ruleId: string;
+        message: string;
+    }[];
+}
+export declare class AuditError extends Error {
+}
+export declare function runAudit(opts: AuditOptions): Promise<AuditResult>;