linersh 0.5.4 → 0.5.6

package/README.md

@@ -23,7 +23,15 @@ For JavaScript-rendered pages, run the setup command from the same npm install:
 liner setup-js
 ```
 
-That command is idempotent and downloads the Playwright browser used for `render: js` and automatic JS-wall fallback. For debugging or local development, `LINER_BIN=/path/to/liner liner` overrides the bundled binary.
+That command is idempotent and downloads the Playwright browser used for `render: js` and automatic JS-wall fallback (about 150MB). The TUI also offers this during first-run onboarding, and again from compile results if a source needs JS rendering. For debugging or local development, `LINER_BIN=/path/to/liner liner` overrides the bundled binary.
+
+To reset a machine for clean testing, run:
+
+```sh
+liner uninstall --yes
+```
+
+This removes `~/.liner`, Playwright's Chromium cache, and npm's `_npx` execution cache. It does not delete your mixtape project folders. If you installed Liner globally, also run `npm uninstall -g linersh`.
 
 ## Run
 
@@ -46,7 +54,8 @@ LINER_DIR=~/mixtapes liner
 | Key | Action |
 |---|---|
 | `↑` `↓` or `j` `k` | move selection |
-| `enter` or `e` | open in editor |
+| `enter` or `o` | open the selected mixtape hub |
+| `e` | open the selected mixtape source editor |
 | `n` | new mixtape (prompts for slug) |
 | `c` | compile selected |
 | `s` | share selected (zip to `.mixtape`) |
@@ -55,7 +64,21 @@ LINER_DIR=~/mixtapes liner
 | `r` | refresh |
 | `q` | quit |
 
-### Editor
+### Project Hub
+| Key | Action |
+|---|---|
+| `↑` `↓` | select phase |
+| `enter` | run/open the selected phase's primary action |
+| `e` | edit the selected artifact by hand |
+| `p` | open the process manifest |
+| `s` | share the mixtape |
+| `o` | open the project folder |
+| `esc` | back |
+| `q` | quit |
+
+Completed mixtapes land in a ready state. If `MIXTAPE.md` is current, pressing `enter` on Compile opens the existing compile result instead of fetching sources again. Press `r` from the compile result screen to explicitly re-run/retry.
+
+### Source Editor
 | Key | Action |
 |---|---|
 | `t` `d` `u` | edit title / description / curator |
@@ -107,24 +130,29 @@ For `web` sources the URL is the load-bearing field; the type is auto-detected a
 ### Compile
 | Key | Action |
 |---|---|
+| `↑` `↓` | scroll sources |
+| `pgup` `pgdn` | jump through sources |
+| `j` | install JS rendering support when the result says it is needed |
 | `esc` | cancel running compile |
 | `y` | copy compiled `MIXTAPE.md` to clipboard (`pbcopy` / `xclip`) |
 | `s` | share the compiled mixtape (zip to `.mixtape`) |
-| `r` | re-run compile |
-| `b` | back to editor |
+| `o` | open the project folder |
+| `r` | retry failed sources / re-run compile |
+| `b` | back to hub |
+| `q` | quit |
 
-The compile writes `MIXTAPE.md` and `sources/NN-<slug>.md` files directly into the project folder. There's nothing to "save" afterwards — the artifacts are already on disk.
+The compile writes `MIXTAPE.md` and `sources/NN-<slug>.md` files directly into the project folder. A partial compile is still treated as a usable finished state when `MIXTAPE.md` is written; the hub shows "compiled with warnings" and the compile result screen lists the remaining warnings. There's nothing to "save" afterwards — the artifacts are already on disk.
 
 ## Uninstalling
 
-The TUI ships in the same `linersh` package as the CLI. The cleanest path is the repo's uninstaller:
+The TUI ships in the same `linersh` package as the CLI. For npm/npx installs, use the package uninstaller:
 
 ```sh
-./scripts/uninstall.sh         # interactive
-./scripts/uninstall.sh --yes   # no prompts
+liner uninstall
+liner uninstall --yes
 ```
 
-It removes the optional Playwright browser, the local cache, any pipx-installed CLI, and any globally installed TUI (`npm install -g linersh`). Your mixtape project folders are left alone unless you pass `--purge-workspace`. See the root [README — Uninstalling](../../README.md#uninstalling) for the manual recipe.
+It removes `~/.liner`, Playwright's Chromium cache, and npm's `_npx` execution cache. Your mixtape project folders are left alone. If you installed Liner globally, also run `npm uninstall -g linersh`. See the root [README — Uninstalling](../../README.md#uninstalling) for the manual recipe.
 
 ## Develop
 
@@ -149,16 +177,17 @@ LINER_BIN=/path/to/liner npm run dev
 ## Architecture
 
 ```
-TapeBrowser ────► NewProjectPrompt ─┐
+BootSplash ─────► TapeBrowser ──────► ProjectHub ──────┐
        │                            │
        │          ImportPrompt ─────┤
        │                            │
+       ├────► NewProjectWizard ─────┤
+       │                            │
        ├────► TapeEditor ⇄ SourceModal
-       │             │
-       │             ▼
-       │       CompileView (streams NDJSON from `liner compile --emit-events`)
-       │
-       └────► (share/delete inline)
+       │                            │
+       ├────► PhaseRunner / gates / draft review
+       │                            │
+       └────► CompileView (streams NDJSON from `liner compile --emit-events`)
 ```
 
 State is global via `AppContext`. YAML is parsed and re-written in Node via the `yaml` package's `Document` API so user-written block comments survive edits. All folder operations (`init`, `compile`, `share`, `import`, `list`) go through the Python CLI as subprocesses — the TUI never reimplements them.

package/bin/liner.js

@@ -9,7 +9,8 @@
 import { fileURLToPath } from "node:url";
 import { createRequire } from "node:module";
 import { spawn, spawnSync } from "node:child_process";
-import { existsSync, readFileSync } from "node:fs";
+import { existsSync, readFileSync, rmSync } from "node:fs";
+import { homedir } from "node:os";
 import { dirname, join } from "node:path";
 
 const here = dirname(fileURLToPath(import.meta.url));
@@ -26,6 +27,11 @@ if (argv[0] === "--version" || argv[0] === "-v") {
   process.exit(0);
 }
 
+if (argv[0] === "uninstall") {
+  const code = await runUninstall(argv.slice(1));
+  process.exit(code);
+}
+
 const CLI_COMMANDS = new Set([
   "init",
   "replay",
@@ -129,6 +135,124 @@ function resolveCliBinary() {
   process.exit(1);
 }
 
+async function runUninstall(args) {
+  if (args.includes("--help") || args.includes("-h")) {
+    console.log(
+      [
+        "Usage: liner uninstall [--yes] [--dry-run]",
+        "",
+        "Remove local Liner traces for a clean reinstall/test:",
+        "  - ~/.liner config and source cache",
+        "  - Playwright's Chromium browser cache",
+        "  - npm's npx execution cache (_npx)",
+        "",
+        "This does not delete your mixtape project folders.",
+        "For global installs, also run: npm uninstall -g linersh",
+      ].join("\n"),
+    );
+    return 0;
+  }
+
+  const known = new Set(["--yes", "-y", "--dry-run"]);
+  const unknown = args.filter((arg) => !known.has(arg));
+  if (unknown.length > 0) {
+    console.error(`Unknown option: ${unknown.join(", ")}`);
+    console.error("Run `liner uninstall --help` for usage.");
+    return 1;
+  }
+
+  const yes = args.includes("--yes") || args.includes("-y");
+  const dryRun = args.includes("--dry-run");
+  const targets = uninstallTargets();
+
+  console.error("liner uninstall will remove:");
+  for (const target of targets) {
+    console.error(`  - ${target.label}: ${target.path}`);
+  }
+  console.error("");
+  console.error("It will not delete your mixtape project folders.");
+
+  if (!yes && !dryRun) {
+    if (!process.stdin.isTTY) {
+      console.error("Refusing to run without a TTY prompt. Re-run with --yes.");
+      return 1;
+    }
+    const { createInterface } = await import("node:readline/promises");
+    const rl = createInterface({ input: process.stdin, output: process.stderr });
+    const answer = (await rl.question("Continue? [y/N] ")).trim().toLowerCase();
+    rl.close();
+    if (answer !== "y" && answer !== "yes") {
+      console.error("Cancelled.");
+      return 1;
+    }
+  }
+
+  let failed = false;
+  for (const target of targets) {
+    if (dryRun) {
+      console.error(`[dry-run] would remove ${target.path}`);
+      continue;
+    }
+    try {
+      rmSync(target.path, { recursive: true, force: true });
+      console.error(`Removed ${target.path}`);
+    } catch (err) {
+      failed = true;
+      console.error(`Failed to remove ${target.path}: ${err.message}`);
+    }
+  }
+
+  if (dryRun) {
+    console.error("Dry run complete.");
+    return 0;
+  }
+  if (failed) return 1;
+  console.error("Liner local state and caches removed.");
+  console.error("For a global install, finish with: npm uninstall -g linersh");
+  return 0;
+}
+
+function uninstallTargets() {
+  const home = homedir();
+  return [
+    {
+      label: "Liner config and source cache",
+      path: join(home, ".liner"),
+    },
+    {
+      label: "Playwright Chromium cache",
+      path: playwrightCachePath(home),
+    },
+    {
+      label: "npm npx execution cache",
+      path: join(npmCachePath(home), "_npx"),
+    },
+  ];
+}
+
+function npmCachePath(home) {
+  const npm = spawnSync("npm", ["config", "get", "cache"], {
+    encoding: "utf8",
+    stdio: ["ignore", "pipe", "ignore"],
+  });
+  const configured = npm.status === 0 ? npm.stdout.trim() : "";
+  if (configured) return configured;
+  if (process.platform === "win32") {
+    return join(process.env.LOCALAPPDATA || join(home, "AppData", "Local"), "npm-cache");
+  }
+  return join(home, ".npm");
+}
+
+function playwrightCachePath(home) {
+  if (process.platform === "darwin") {
+    return join(home, "Library", "Caches", "ms-playwright");
+  }
+  if (process.platform === "win32") {
+    return join(process.env.LOCALAPPDATA || join(home, "AppData", "Local"), "ms-playwright");
+  }
+  return join(process.env.XDG_CACHE_HOME || join(home, ".cache"), "ms-playwright");
+}
+
 function findBundledBinary() {
   const require = createRequire(import.meta.url);
   const target = `linersh-${process.platform}-${process.arch}`;

package/cli-update-docs/ABOUT.md

@@ -62,7 +62,7 @@ A curator who learns the methodology can apply it with or without Liner's tools.
 
 ## Project status
 
-Active development. The CLI and TUI are both functional and published on npm (current release v0.5.3, Mac + Linux). Both support the full v1.1 tape format — `youtube`, `web`, and `local_file` sources, with optional Playwright-backed JavaScript rendering for sites that need it (`pipx install 'linersh[js]'`, then `liner setup-js`). The library is empty pending its first methodology-mode entries. The `curating-mixtapes` skill and the MCP server are planned. Maintained as a side project — see [PLATFORM.md](./PLATFORM.md) for the maintenance contract.
+Active development. The CLI and TUI are both functional and published on npm (current release v0.5.5, Mac, Linux, and Windows x64). Both support the full v1.1 tape format — `youtube`, `web`, and `local_file` sources, with optional Playwright-backed JavaScript rendering for sites that need it (`liner setup-js`, surfaced by the TUI during onboarding and compile warnings). The TUI now treats compilation as the completion milestone and opens existing compile results instead of rerunning by default. The library is empty pending its first methodology-mode entries. The `curating-mixtapes` skill ships inside the TUI package; the MCP server is planned. Maintained as a side project — see [PLATFORM.md](./PLATFORM.md) for the maintenance contract.
 
 ## License
 

package/cli-update-docs/AI-HANDOFF.md

@@ -0,0 +1,227 @@
+# Liner AI Handoff
+
+Last updated: 2026-06-01
+
+This handoff summarizes the current Liner state after the 0.5.5 TUI/core release,
+marketing-site launch-prep pass, and 0.5.6 source-extraction hardening prep. It
+is intended for any AI agent continuing the work without the full chat history.
+
+## Current Product State
+
+- `linersh@0.5.5` is published on npm and promoted to `latest`.
+- The published Python core/platform packages are `0.5.5`; matching
+  `linersh-*@0.5.5` platform packages are published on npm and promoted to
+  `latest`.
+- The repo is prepared for `0.5.6` (`pyproject.toml`, `src/liner/__init__.py`,
+  and `packages/tui/package.json`). It needs rebuilt platform packages because
+  the Python core changed.
+- Platform packages are available for macOS arm64/x64, Linux arm64/x64, and Windows x64.
+- The TUI is installed/tested locally from the packed tarball in `/tmp/liner-playwright-test`.
+- The main tested mixtape is `staff-design-engenieer` (typo preserved in the folder name).
+- Clean npm smoke passed with `npx linersh@latest --version`: `liner 0.5.5 (tui)  ·  0.5.5 (core)`.
+
+## Important Local Paths
+
+- Active repo/worktree for this thread: `/Users/arturo/.codex/worktrees/875c/liner`
+- Previous v0.5.5 implementation worktree: `/Users/arturo/.codex/worktrees/44f8/liner`
+- Original Liner project folder: `/Users/arturo/Documents/Projects/Youtube Grabber/liner`
+- Original mixtapes folder: `/Users/arturo/Documents/Projects/Youtube Grabber/liner/mixtapes`
+- CMD-UX copied mixtapes folder: `/Users/arturo/Documents/Projects/CMD-UX/Case Studies/mixtapes`
+- Local TUI test install: `/tmp/liner-playwright-test`
+- 0.5.5 release artifacts: `/tmp/liner-release-artifacts-0.5.5`
+- 0.5.6 release artifacts: `/tmp/liner-release-artifacts-0.5.6`
+- Tested mixtape now moved back to: `/Users/arturo/Documents/Projects/Youtube Grabber/liner/mixtapes/staff-design-engenieer`
+
+## What Was Fixed In This Pass
+
+0.5.6 source-extraction hardening:
+
+- Remote web PDFs are detected before HTML extraction and extracted through the
+  shared PDF helper. This prevents raw `%PDF-` bytes from entering generated
+  source Markdown.
+- Bad web cache entries are purged and refetched when they contain raw PDF
+  bytes, stale PDF URL fallbacks, tiny soft/playwright fallback bodies, or
+  cookie-consent-only text.
+- Server HTML that looks like an empty client-side app shell now raises
+  `JsRenderingRequired`, so existing auto-fallback can try Playwright.
+- Cookie-consent-only output is rejected as unusable, including after Playwright
+  rendering. This is intentionally an honest failure, not an automatic
+  cookie-clicker.
+- Partial compile output now says how many source files are usable versus
+  unavailable placeholders.
+
+0.5.5 TUI/core release:
+
+- Skill bundle resolution for npm installs:
+  - TUI now resolves the bundled `cli-update-docs/SKILL.md` reliably from the installed package.
+  - `LINER_SKILL_PATH` can point either at the skill directory or at `SKILL.md`.
+
+- Playwright/JS rendering onboarding:
+  - First-run TUI flow now asks whether to install JS rendering support after agent/model setup.
+  - Compile results that detect missing Chromium show a `JS rendering needed` callout.
+  - The callout says: `Some pages require headless Chromium to fetch correctly. Press j to run liner setup-js --yes (~150MB).`
+  - Pressing `j` runs setup and retries compile.
+
+- Compile result UX:
+  - Long source lists use an internal viewport with `↑↓` and `pgup/pgdn`.
+  - PhaseRunner done screens use an internal viewport for the final response.
+  - Partial compiles now count as complete when `MIXTAPE.md` is written.
+  - The hub shows `Ready to use — compiled with warnings` when some sources failed.
+  - Pressing Enter on completed Compile opens the existing compile result instead of fetching again.
+  - Recompile/retry is explicit with `r` from CompileView.
+
+- Web extraction fallback:
+  - `trafilatura` crashes are caught.
+  - Web handlers keep useful fallback text where possible and surface a soft warning instead of raw Python errors like `'NoneType' object is not subscriptable`.
+
+- Clean reinstall:
+  - `liner uninstall` was added to the npm shim.
+  - It removes `~/.liner`, npm `_npx`, and Playwright Chromium cache.
+  - It does not remove mixtape project folders.
+
+## Known Gotchas
+
+- The tested mixtape folder is misspelled `staff-design-engenieer`; do not silently rename it unless the user asks, because existing references and copied paths use that spelling.
+- `LINER_BIN` matters during local testing. To test TUI changes plus local Python handler fixes, run the TUI with:
+
+```sh
+cd /tmp/liner-playwright-test
+LINER_BIN=/Users/arturo/.codex/worktrees/875c/liner/.venv/bin/liner ./node_modules/.bin/liner
+```
+
+- Running `./node_modules/.bin/liner` without `LINER_BIN` uses the bundled platform binary, which may not include local Python source edits until platform packages are rebuilt.
+- If Python source changed, publishing only the main `linersh` package is not
+  enough. Rebuild and publish all platform packages at the same version first.
+- Cookie-banner auto-dismissal is not implemented in 0.5.6. The safety net is
+  to reject cookie-only output; a future dismissal pass should keep that final
+  rejection in place.
+- The TUI package generated `packages/tui/cli-update-docs/` during `prepack`; the canonical docs live at repo-root `cli-update-docs/`.
+- The old root `docs/` folder was removed. `LINER-MASTER.md` now lives at `cli-update-docs/LINER-MASTER.md`.
+- The Jhey source URL currently 404s on the live site. The useful archived version is:
+
+```text
+https://web.archive.org/web/20250408204038id_/https://jhey.dev/cheep/rotating-gallery-with-css-scroll-driven-animations/
+```
+
+## Verification Already Run
+
+TUI package:
+
+```sh
+cd /Users/arturo/.codex/worktrees/875c/liner/packages/tui
+npm run typecheck
+npm run build
+npm test
+npm pack --json
+```
+
+Python suite:
+
+```sh
+cd /Users/arturo/.codex/worktrees/875c/liner
+uv run --extra dev pytest
+```
+
+Result from the latest full Python run in this pass: `190 passed, 1 deselected`.
+
+Release-prep verification:
+
+```sh
+cd /Users/arturo/.codex/worktrees/875c/liner/packages/tui
+npm ci
+npm run typecheck
+npm test
+npm run build
+npm pack --dry-run
+
+cd /Users/arturo/.codex/worktrees/875c/liner
+python3 -m compileall -q src/liner
+uv run --extra dev pytest
+```
+
+Results: TUI `92 passed`; Python `179 passed, 1 skipped, 1 deselected`.
+
+Platform artifact workflow:
+
+```text
+Status: latest post-release-prep run is green; check `gh run list --workflow platform-bundles.yml --limit 3` if you need the current run ID.
+Artifacts downloaded to: /tmp/liner-release-artifacts-0.5.5
+Local npm bundle smoke: liner 0.5.5 (tui)  ·  0.5.5 (core)
+```
+
+npm release smoke:
+
+```sh
+rm -rf ~/.npm/_npx
+npx linersh@latest --version
+# liner 0.5.5 (tui)  ·  0.5.5 (core)
+```
+
+## Useful Test Commands
+
+Install the local TUI tarball into the scratch project:
+
+```sh
+cd /tmp/liner-playwright-test
+npm install /tmp/liner-release-artifacts-0.5.5/linersh-darwin-arm64/linersh-darwin-arm64-0.5.5.tgz /tmp/liner-release-artifacts-0.5.5/linersh-tui/linersh-0.5.5.tgz
+```
+
+Run the TUI against the local Python core:
+
+```sh
+cd /tmp/liner-playwright-test
+LINER_BIN=/Users/arturo/.codex/worktrees/875c/liner/.venv/bin/liner ./node_modules/.bin/liner
+```
+
+Force a fresh compile/fetch:
+
+```sh
+cd /tmp/liner-playwright-test
+LINER_BIN=/Users/arturo/.codex/worktrees/875c/liner/.venv/bin/liner ./node_modules/.bin/liner cache clear --yes
+LINER_BIN=/Users/arturo/.codex/worktrees/875c/liner/.venv/bin/liner ./node_modules/.bin/liner
+```
+
+Run JS setup explicitly:
+
+```sh
+cd /tmp/liner-playwright-test
+LINER_BIN=/Users/arturo/.codex/worktrees/875c/liner/.venv/bin/liner ./node_modules/.bin/liner setup-js --yes
+```
+
+Clean local state for reinstall testing:
+
+```sh
+liner uninstall --yes
+npm uninstall -g linersh 2>/dev/null || true
+```
+
+## Documentation State
+
+Canonical docs now live in `cli-update-docs/`:
+
+- `LINER-MASTER.md` — current conceptual and technical overview.
+- `AI-HANDOFF.md` — this document.
+- `TUI-BACKLOG.md` — backlog and done log.
+- `MIXTAPE-FORMAT.md` — tape/mixtape format.
+- `CURATION.md` — methodology.
+- `SKILL.md` and companion files — bundled curation skill.
+
+Package-specific docs still live in `packages/tui/`:
+
+- `packages/tui/README.md`
+- `packages/tui/PUBLISH-CHECKLIST.md`
+- `packages/tui/PLATFORM-BUNDLES.md`
+
+## Suggested Next Work
+
+- Deploy the marketing site from `marketing/site` once DNS/Vercel details are ready.
+- Update the public website docs and changelog for 0.5.6 after the release
+  artifacts are ready.
+- Consider a conservative Playwright cookie-banner dismissal pass for common,
+  low-risk labels, while still rejecting cookie-only output if dismissal fails.
+- Consider surfacing extraction mode/body length in the TUI compile details.
+- Consider adding `liner doctor-source <url>` or equivalent single-source
+  diagnostics.
+- Consider adding Wayback as an automatic fetcher tier between Playwright and agent-summary fallback.
+- Add per-source failure actions in CompileView: open URL, drop source, retry one source.
+- Add a regression test for completed Compile opening existing results rather than running `streamCompile`.

package/cli-update-docs/JOB-STORY-MIGRATION.md

@@ -26,7 +26,7 @@ Until a real validator exists, the JTBD master prompt's internal rubric is the a
 | `cli-update-docs/SKILL.md` | The `curating-mixtapes` skill's Phase 1 paragraph defines what a tight JTBD looks like. Replace examples and any embedded house-form templates. |
 | `cli-update-docs/JTBD-MASTER-PROMPT.md` | Drop in the revised version from the previous turn. |
 | `cli-update-docs/JTBD-MASTER-PROMPT-ABOUT.md` | Substantial rewrite. The current doc explicitly defends choosing the house form over Job Story as a deliberate deviation from the mixtape's synthesis. The new framing inverts: "we initially diverged, then realigned." Update §"Where it came from," the Liner's-house-form choice in §"Design choices worth understanding," and any other instances. |
-| `docs/LINER-MASTER.md` | Project-level overview. Sweep for house-form references and example JTBDs. |
+| `cli-update-docs/LINER-MASTER.md` | Project-level overview. Sweep for house-form references and example JTBDs. |
 
 ## Files to check (categories — likely exist, names not confirmed)
 

package/cli-update-docs/JTBD-MASTER-PROMPT-ABOUT.md

@@ -130,4 +130,4 @@ A few decisions in the prompt aren't obvious and have been pushed back on during
 - The Liner curation methodology that places this work in Phase 1: [`cli-update-docs/CURATION.md`](./CURATION.md)
 - The skill that will likely embed this prompt: [`cli-update-docs/SKILL.md`](./SKILL.md)
 - The mixtape format spec defining the `jtbd:` field the prompt's output lands in: [`cli-update-docs/MIXTAPE-FORMAT.md`](./MIXTAPE-FORMAT.md)
-- Project-level overview of Liner: [`docs/LINER-MASTER.md`](../docs/LINER-MASTER.md)
+- Project-level overview of Liner: [`LINER-MASTER.md`](./LINER-MASTER.md)