@only-cli/oc 0.1.0-dev.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,93 @@
1
+ # only-cli
2
+
3
+ Turn most website into a command line interface, so AI agents like Claude Code, Codex, and Antigravity can browse without burning tokens on raw HTML or screenshots.
4
+
5
+ A typical page is tens of thousands of tokens of markup. The signal on it fits in a few hundred. only-cli fetches the page, distills it into a compact text view with numbered actions, and lets an agent drive the site by number:
6
+
7
+ ```
8
+ $ oc open news.ycombinator.com
9
+ # Hacker News
10
+ [1] Show HN: I built a tiny CSV toolkit
11
+ [2] 312 comments
12
+ ...
13
+ actions: do <n> | raw <url>
14
+
15
+ $ oc do 1
16
+ ```
17
+
18
+ No per-site adapters required, no browser extension, no daemon. One generic distillation engine, three runtime dependencies, and a hard token budget on everything it prints.
19
+
20
+ ## Install
21
+
22
+ ```
23
+ npm install -g @only-cli/oc
24
+ ```
25
+
26
+ Requires Node 20+. Requests go through [impers](https://github.com/lexiforest/impers) impersonating Chrome; if impers is unavailable the tool falls back to native fetch.
27
+
28
+ ## For AI agents
29
+
30
+ The fastest setup is one line in your agent's instructions file (CLAUDE.md, AGENTS.md, or equivalent):
31
+
32
+ > When you need content from a web page, run `npx @only-cli/oc open <url>` instead of fetching raw HTML. Run `npx @only-cli/oc --help` once to learn the commands.
33
+
34
+ Claude Code users can install the skill instead: copy `skills/only-cli/` into your project's `.claude/skills/` directory (or `~/.claude/skills/` to enable it everywhere). A skill costs almost no tokens until the agent actually invokes it, which fits how this whole project thinks. The same skill also installs through the [skills.sh](https://skills.sh) directory into Claude Code, Cursor, Codex, Copilot, and others:
35
+
36
+ ```
37
+ npx skills add only-cli/oc
38
+ ```
39
+
40
+ No setup at all also works: `npx @only-cli/oc` runs without a global install, and the tool teaches its own command surface through `--help`, the `actions:` line at the bottom of every render, and error messages that name the next command to run.
41
+
42
+ ## Commands
43
+
44
+ ```
45
+ oc open <url> fetch and render a page with numbered actions
46
+ oc raw <url> distilled markdown of the whole page
47
+ oc do <n> activate a numbered element (v0.2)
48
+ oc fill <n> <text> type into a numbered input (v0.2)
49
+ oc submit [n] submit a form (v0.2)
50
+ ```
51
+
52
+ Flags: `--budget <tokens>` (default 500), `--json`, `--html` (raw as cleaned HTML instead of markdown), `--verbose`/`-v` (metrics on stderr: tokens saved vs the page HTML, HTTP status and client identity, timing, transfer size, memory; alias `--stats`, or export `OC_VERBOSE=1`). Metrics are off by default because they cost tokens too; agents should pass `--verbose` only when running verbosely.
53
+
54
+ ## Supported websites
55
+
56
+ only-cli works on any mostly-static website with no per-site setup: news sites, blogs, documentation, forums, search engines. One generic engine distills whatever HTML comes back. On top of that, `clis/` ships tuned command shortcuts for:
57
+
58
+ | website | domain | shortcuts |
59
+ | --- | --- | --- |
60
+ | Hacker News | news.ycombinator.com | `top`, `new`, `item <id>`, `user <name>` |
61
+ | Reddit | reddit.com (via old.reddit.com) | `sub <name>`, `post <id>`, `user <name>`, `search <query>` |
62
+ | GitHub | github.com | `repo <owner> <name>`, `user <name>`, `search <query>`, `trending`, `issues <owner> <name>` |
63
+ | LinkedIn | linkedin.com | `profile <name>`, `company <name>`, `jobs <query>` (public guest views) |
64
+ | DuckDuckGo | duckduckgo.com | `search <query>`, `lite <query>` |
65
+ | Bing | bing.com | `search <query>`, `news <query>` |
66
+
67
+ Not supported yet: pages that only render with JavaScript (a headless fallback is planned for v0.3), sites behind logins (sessions land in v0.2), and sites with hard bot challenges. Adding a site shortcut is a small JSON file; see [CONTRIBUTING.md](CONTRIBUTING.md).
68
+
69
+ ## Benchmarks
70
+
71
+ Measured against live sites in [only-cli/benchmarks](https://github.com/only-cli/benchmarks) (only-cli 0.1.0, 2026-08-18):
72
+
73
+ | method | success | total tokens | avg ms |
74
+ | --- | ---: | ---: | ---: |
75
+ | `oc open` | 4/4 | 1,444 | 591 |
76
+ | `oc raw` | 4/4 | 15,170 | 659 |
77
+ | raw HTML fetch | 4/4 | 65,372 | 247 |
78
+
79
+ Across four real tasks (an article, a news front page, a Reddit discussion, a search) the compact view hands the agent 45x fewer tokens than raw HTML for roughly 350ms more per page. The heaviest single page, a Reddit thread, went from 52,899 tokens to 477. The benchmark repo has per-task numbers, methodology, and instructions for adding other tools and models.
80
+
81
+ ## Status
82
+
83
+ Early. v0.1 covers static pages, budget-aware rendering, and offline tests. Sessions and actions land in v0.2, a lazy headless fallback for script-heavy pages in v0.3. The design principles and how to contribute are in [CONTRIBUTING.md](CONTRIBUTING.md).
84
+
85
+ Known limits, honestly: no JavaScript rendering yet, no sites behind logins yet, and pages behind hard bot challenges may still refuse the tool.
86
+
87
+ ## Contributors
88
+
89
+ - [only-cli](https://github.com/only-cli), creator and maintainer
90
+
91
+ ## License
92
+
93
+ MIT
@@ -0,0 +1,7 @@
1
+ {
2
+ "domain": "bing.com",
3
+ "commands": {
4
+ "search": { "open": "https://www.bing.com/search?q={query}", "args": ["query"] },
5
+ "news": { "open": "https://www.bing.com/news/search?q={query}", "args": ["query"] }
6
+ }
7
+ }
@@ -0,0 +1,7 @@
1
+ {
2
+ "domain": "duckduckgo.com",
3
+ "commands": {
4
+ "search": { "open": "https://html.duckduckgo.com/html/?q={query}", "args": ["query"] },
5
+ "lite": { "open": "https://lite.duckduckgo.com/lite/?q={query}", "args": ["query"] }
6
+ }
7
+ }
@@ -0,0 +1,10 @@
1
+ {
2
+ "domain": "github.com",
3
+ "commands": {
4
+ "repo": { "open": "https://github.com/{owner}/{repo}", "args": ["owner", "repo"] },
5
+ "user": { "open": "https://github.com/{name}", "args": ["name"] },
6
+ "search": { "open": "https://github.com/search?q={query}&type=repositories", "args": ["query"] },
7
+ "trending": { "open": "https://github.com/trending" },
8
+ "issues": { "open": "https://github.com/{owner}/{repo}/issues", "args": ["owner", "repo"] }
9
+ }
10
+ }
@@ -0,0 +1,8 @@
1
+ {
2
+ "domain": "linkedin.com",
3
+ "commands": {
4
+ "profile": { "open": "https://www.linkedin.com/in/{name}", "args": ["name"] },
5
+ "company": { "open": "https://www.linkedin.com/company/{name}", "args": ["name"] },
6
+ "jobs": { "open": "https://www.linkedin.com/jobs/search?keywords={query}", "args": ["query"] }
7
+ }
8
+ }
@@ -0,0 +1,9 @@
1
+ {
2
+ "domain": "news.ycombinator.com",
3
+ "commands": {
4
+ "top": { "open": "https://news.ycombinator.com" },
5
+ "new": { "open": "https://news.ycombinator.com/newest" },
6
+ "item": { "open": "https://news.ycombinator.com/item?id={id}", "args": ["id"] },
7
+ "user": { "open": "https://news.ycombinator.com/user?id={name}", "args": ["name"] }
8
+ }
9
+ }
@@ -0,0 +1,9 @@
1
+ {
2
+ "domain": "reddit.com",
3
+ "commands": {
4
+ "sub": { "open": "https://old.reddit.com/r/{sub}", "args": ["sub"] },
5
+ "post": { "open": "https://old.reddit.com/comments/{id}", "args": ["id"] },
6
+ "user": { "open": "https://old.reddit.com/user/{name}", "args": ["name"] },
7
+ "search": { "open": "https://old.reddit.com/search?q={query}", "args": ["query"] }
8
+ }
9
+ }
package/package.json ADDED
@@ -0,0 +1,51 @@
1
+ {
2
+ "name": "@only-cli/oc",
3
+ "version": "0.1.0-dev.8",
4
+ "description": "Turn websites into a compact CLI so AI agents can browse without burning tokens.",
5
+ "type": "module",
6
+ "bin": {
7
+ "oc": "src/cli.js"
8
+ },
9
+ "files": [
10
+ "src",
11
+ "clis",
12
+ "skills"
13
+ ],
14
+ "engines": {
15
+ "node": ">=20"
16
+ },
17
+ "scripts": {
18
+ "test": "node --test"
19
+ },
20
+ "keywords": [
21
+ "cli",
22
+ "ai-agents",
23
+ "web-browsing",
24
+ "token-efficient",
25
+ "claude",
26
+ "claude-code",
27
+ "chatgpt",
28
+ "codex",
29
+ "llm",
30
+ "scraping",
31
+ "html-to-markdown",
32
+ "agent-tools"
33
+ ],
34
+ "author": "only-cli (https://github.com/only-cli)",
35
+ "license": "MIT",
36
+ "repository": {
37
+ "type": "git",
38
+ "url": "git+https://github.com/only-cli/oc.git"
39
+ },
40
+ "homepage": "https://github.com/only-cli/oc#readme",
41
+ "bugs": {
42
+ "url": "https://github.com/only-cli/oc/issues"
43
+ },
44
+ "dependencies": {
45
+ "linkedom": "^0.18.12",
46
+ "turndown": "^7.2.4"
47
+ },
48
+ "optionalDependencies": {
49
+ "impers": "^0.1.0"
50
+ }
51
+ }
@@ -0,0 +1,37 @@
1
+ ---
2
+ name: only-cli
3
+ description: Browse websites from the terminal in a few hundred tokens. Use when you need content from a web page, want to check a link, or would otherwise fetch raw HTML or reach for a browser.
4
+ ---
5
+
6
+ # only-cli
7
+
8
+ Turns a web page into a compact terminal view instead of a raw HTML dump. A typical page renders in under 500 tokens.
9
+
10
+ No install needed, run it with npx:
11
+
12
+ ```
13
+ npx @only-cli/oc open <url> compact view with numbered elements
14
+ npx @only-cli/oc raw <url> whole page as markdown (add --html for cleaned HTML)
15
+ ```
16
+
17
+ ## Reading the output
18
+
19
+ - The first line is the page title, then headings, text, and interactive elements in page order.
20
+ - `[n]` marks a link, button, or input.
21
+ - The `actions:` line at the bottom lists valid next commands.
22
+ - `... N more blocks over budget` means content was cut to stay cheap. Rerun with `--budget 1500` if you need more, or use `raw` for everything.
23
+
24
+ ## Following links (current version)
25
+
26
+ `do <n>` and the other action commands land in v0.2. Until then: run `raw <url>` to see each link's href in markdown form, then `open` that URL directly.
27
+
28
+ ## Flags
29
+
30
+ - `--budget <tokens>` raise or lower the render budget (default 500)
31
+ - `--json` machine-stable JSON of the distilled page
32
+ - `--html` with raw: cleaned HTML instead of markdown, if markup suits your task better
33
+ - `--verbose` (`-v`, alias `--stats`) metrics on stderr: tokens saved vs the page HTML, HTTP status and which client identity got the page, fetch and processing time, bytes transferred, memory use. Only pass this when you are running in verbose mode or diagnosing a problem; the metrics line costs tokens like everything else. Users can export `OC_VERBOSE=1` to turn it on globally.
34
+
35
+ ## When not to use it
36
+
37
+ Pages that require login or heavy client-side JavaScript are not supported yet. If a page comes back empty or blocked, say so and fall back to another method rather than retrying.
package/src/act.js ADDED
@@ -0,0 +1,44 @@
1
+ /**
2
+ * Actions against a live session: do, fill, submit, read, find, back, next.
3
+ * All of this ships in v0.2 together with session state. The signatures
4
+ * exist now so cli.js wires up once and the command surface stays stable.
5
+ */
6
+
7
+ export class NotImplemented extends Error {
8
+ constructor(command) {
9
+ super(`'oc ${command}' lands in v0.2. Until then use 'oc open' and 'oc raw'.`);
10
+ }
11
+ }
12
+
13
+ /** @param {number} n */
14
+ export function activate(n) {
15
+ throw new NotImplemented('do');
16
+ }
17
+
18
+ /** @param {number} n @param {string} text */
19
+ export function fill(n, text) {
20
+ throw new NotImplemented('fill');
21
+ }
22
+
23
+ /** @param {number} [n] */
24
+ export function submit(n) {
25
+ throw new NotImplemented('submit');
26
+ }
27
+
28
+ /** @param {number} [n] */
29
+ export function read(n) {
30
+ throw new NotImplemented('read');
31
+ }
32
+
33
+ /** @param {string} query */
34
+ export function find(query) {
35
+ throw new NotImplemented('find');
36
+ }
37
+
38
+ export function back() {
39
+ throw new NotImplemented('back');
40
+ }
41
+
42
+ export function next() {
43
+ throw new NotImplemented('next');
44
+ }
package/src/cli.js ADDED
@@ -0,0 +1,112 @@
1
+ #!/usr/bin/env node
2
+ import { parseArgs } from 'node:util';
3
+ import { fetchPage } from './fetch.js';
4
+ import { distill, toMarkdown, toHTML } from './distill.js';
5
+ import { render, estimateTokens } from './render.js';
6
+ import * as act from './act.js';
7
+
8
+ const HELP = `only-cli: the web as a compact terminal, built for AI agents.
9
+
10
+ usage: oc <command> [args] [flags]
11
+
12
+ open <url> fetch and render a page with numbered actions
13
+ raw <url> distilled markdown of the whole page
14
+ do <n> activate a numbered element (v0.2)
15
+ fill <n> <text> type into a numbered input (v0.2)
16
+ submit [n] submit a form (v0.2)
17
+ read [n] full text of one region (v0.2)
18
+ find <query> search visible text on the current page (v0.2)
19
+ back | next history and pagination (v0.2)
20
+ session ls|rm manage saved sessions (v0.2)
21
+
22
+ flags:
23
+ --budget <tokens> tighten or loosen the render budget (default 500)
24
+ --json machine-stable JSON output
25
+ --html raw only: cleaned HTML instead of markdown
26
+ --verbose, -v metrics on stderr: tokens saved vs the page HTML, HTTP
27
+ status and client identity, timing, transfer size, and
28
+ memory. --stats is an alias; OC_VERBOSE=1 turns it on
29
+ globally. Off by default because metrics cost tokens too.
30
+ --session <name> named session (v0.2)`;
31
+
32
+ // What browsing costs without this tool is the raw page HTML in context.
33
+ const savings = (out, raw) =>
34
+ `~${out} tokens vs ~${raw} for the page HTML (${Math.max(0, 100 - Math.round((out / Math.max(raw, 1)) * 100))}% saved)`;
35
+
36
+ async function main() {
37
+ const { values, positionals } = parseArgs({
38
+ allowPositionals: true,
39
+ options: {
40
+ json: { type: 'boolean', default: false },
41
+ html: { type: 'boolean', default: false },
42
+ stats: { type: 'boolean', default: false },
43
+ verbose: { type: 'boolean', short: 'v', default: false },
44
+ budget: { type: 'string' },
45
+ session: { type: 'string' },
46
+ help: { type: 'boolean', short: 'h', default: false },
47
+ },
48
+ });
49
+
50
+ // Metrics cost tokens too, so they are opt-in: agents pass --verbose only
51
+ // when their own verbose mode is on, or the user exports OC_VERBOSE=1.
52
+ const verbose = values.stats || values.verbose || process.env.OC_VERBOSE === '1';
53
+
54
+ const [command, ...args] = positionals;
55
+ if (values.help || !command) {
56
+ console.log(HELP);
57
+ return;
58
+ }
59
+
60
+ switch (command) {
61
+ case 'open':
62
+ case 'raw': {
63
+ const url = args[0];
64
+ if (!url) throw new Error(`usage: oc ${command} <url>`);
65
+ const budget = values.budget ? Number(values.budget) : 500;
66
+ if (!Number.isFinite(budget) || budget <= 0) throw new Error('--budget must be a positive number');
67
+ const t0 = performance.now();
68
+ const { url: finalUrl, html, status, via } = await fetchPage(url);
69
+ const fetchMs = performance.now() - t0;
70
+ const resources = () => {
71
+ const processMs = performance.now() - t0 - fetchMs;
72
+ const rss = process.memoryUsage().rss;
73
+ return `HTTP ${status} via ${via}, fetch ${Math.round(fetchMs)}ms, process ${Math.round(processMs)}ms, `
74
+ + `${Math.round(html.length / 1024)}KB transferred, ${Math.round(rss / 1048576)}MB memory`;
75
+ };
76
+ if (values.json) {
77
+ console.log(JSON.stringify(distill(html, finalUrl)));
78
+ if (verbose) console.error(resources());
79
+ return;
80
+ }
81
+ const htmlTokens = estimateTokens(html);
82
+ if (command === 'raw') {
83
+ const out = values.html ? toHTML(html) : toMarkdown(html);
84
+ console.log(out);
85
+ if (verbose) console.error(`${savings(estimateTokens(out), htmlTokens)}; ${resources()}`);
86
+ return;
87
+ }
88
+ const page = distill(html, finalUrl);
89
+ const { text, stats } = render(page, { budget });
90
+ console.log(text);
91
+ if (verbose) {
92
+ console.error(`~${stats.tokens} tokens, ${stats.rendered}/${stats.blocks} blocks rendered, ${savings(stats.tokens, htmlTokens)}; ${resources()}`);
93
+ }
94
+ return;
95
+ }
96
+ case 'do': return act.activate(Number(args[0]));
97
+ case 'fill': return act.fill(Number(args[0]), args.slice(1).join(' '));
98
+ case 'submit': return act.submit(args[0] ? Number(args[0]) : undefined);
99
+ case 'read': return act.read(args[0] ? Number(args[0]) : undefined);
100
+ case 'find': return act.find(args.join(' '));
101
+ case 'back': return act.back();
102
+ case 'next': return act.next();
103
+ case 'session': throw new act.NotImplemented('session');
104
+ default:
105
+ throw new Error(`unknown command '${command}', run oc --help`);
106
+ }
107
+ }
108
+
109
+ main().catch((err) => {
110
+ console.error(`oc: ${err.message}`);
111
+ process.exit(1);
112
+ });
package/src/distill.js ADDED
@@ -0,0 +1,167 @@
1
+ import { parseHTML } from 'linkedom';
2
+ import TurndownService from 'turndown';
3
+
4
+ /**
5
+ * @typedef {Object} Block
6
+ * @property {'heading'|'text'|'link'|'input'|'button'} type
7
+ * @property {string} text
8
+ * @property {number} [n] - action handle, only on interactive blocks
9
+ * @property {number} [level] - heading level 1..6
10
+ * @property {string} [href] - links only
11
+ * @property {string} [name] - inputs only
12
+ *
13
+ * @typedef {Object} Page
14
+ * @property {string} url
15
+ * @property {string} title
16
+ * @property {Block[]} blocks
17
+ */
18
+
19
+ // Dropped wholesale, subtree included. Nav and footer stay in v0.1: on many
20
+ // sites they carry the only working links, and the budget in render.js is
21
+ // what keeps them from costing anything.
22
+ const DROP = new Set([
23
+ 'script', 'style', 'noscript', 'template', 'svg', 'iframe',
24
+ 'link', 'meta', 'head', 'canvas', 'video', 'audio', 'object',
25
+ ]);
26
+
27
+ const clean = (s) => s.replace(/\s+/g, ' ').trim();
28
+
29
+ /**
30
+ * Reduce raw HTML to an interaction tree: readable text plus numbered
31
+ * interactive elements, in document order. Handles are assigned during a
32
+ * single deterministic walk, so the same page always yields the same numbers.
33
+ * @param {string} html
34
+ * @param {string} url
35
+ * @returns {Page}
36
+ */
37
+ export function distill(html, url = '') {
38
+ const { document } = parseHTML(html);
39
+ const title = clean(document.querySelector('title')?.textContent ?? '');
40
+ /** @type {Block[]} */
41
+ const blocks = [];
42
+ let handle = 0;
43
+
44
+ const hidden = (el) =>
45
+ el.getAttribute('hidden') !== null ||
46
+ el.getAttribute('aria-hidden') === 'true' ||
47
+ /display:\s*none/.test(el.getAttribute('style') ?? '');
48
+
49
+ const walk = (node) => {
50
+ if (node.nodeType === 3) {
51
+ const text = clean(node.textContent);
52
+ if (text) blocks.push({ type: 'text', text });
53
+ return;
54
+ }
55
+ if (node.nodeType !== 1) return;
56
+ const tag = node.localName;
57
+ if (DROP.has(tag) || hidden(node)) return;
58
+
59
+ if (/^h[1-6]$/.test(tag)) {
60
+ const text = clean(node.textContent);
61
+ if (text) blocks.push({ type: 'heading', level: Number(tag[1]), text });
62
+ return;
63
+ }
64
+ if (tag === 'a' && node.getAttribute('href')) {
65
+ const text = clean(node.textContent);
66
+ if (text) {
67
+ blocks.push({ type: 'link', n: ++handle, text, href: node.getAttribute('href') });
68
+ }
69
+ return;
70
+ }
71
+ if (tag === 'input' || tag === 'textarea' || tag === 'select') {
72
+ const kind = node.getAttribute('type') ?? 'text';
73
+ if (kind === 'hidden') return;
74
+ if (kind === 'submit' || kind === 'button') {
75
+ blocks.push({ type: 'button', n: ++handle, text: node.getAttribute('value') ?? 'submit' });
76
+ return;
77
+ }
78
+ const name = node.getAttribute('name') ?? node.getAttribute('placeholder') ?? tag;
79
+ blocks.push({ type: 'input', n: ++handle, text: kind, name });
80
+ return;
81
+ }
82
+ if (tag === 'button') {
83
+ const text = clean(node.textContent) || 'button';
84
+ blocks.push({ type: 'button', n: ++handle, text });
85
+ return;
86
+ }
87
+ for (const child of node.childNodes) walk(child);
88
+ };
89
+
90
+ const body = bodyOf(document);
91
+ if (body) walk(body);
92
+ return { url, title, blocks: mergeText(blocks) };
93
+ }
94
+
95
+ /**
96
+ * linkedom's document.body getter comes back empty on some real pages (Bing)
97
+ * while querySelector finds the populated element, so always resolve the body
98
+ * this way.
99
+ * @returns {any}
100
+ */
101
+ const bodyOf = (document) => document.querySelector('body') ?? document.documentElement;
102
+
103
+ /**
104
+ * Shared cleanup for the raw modes: parse, then delete the same noise
105
+ * distill() skips, so neither raw output ever leaks scripts, styles, or
106
+ * hidden content.
107
+ * @param {string} html
108
+ */
109
+ function cleanDocument(html) {
110
+ const { document } = parseHTML(html);
111
+ for (const tag of DROP) {
112
+ for (const el of [...document.querySelectorAll(tag)]) el.remove();
113
+ }
114
+ for (const el of [...document.querySelectorAll('[hidden], [aria-hidden="true"], input[type="hidden"]')]) el.remove();
115
+ for (const el of [...document.querySelectorAll('[style]')]) {
116
+ if (/display:\s*none/.test(el.getAttribute('style') ?? '')) el.remove();
117
+ }
118
+ return document;
119
+ }
120
+
121
+ /**
122
+ * Whole-page markdown for `oc raw`, produced by turndown so lists, emphasis,
123
+ * links, and code blocks come out as real markdown instead of flat lines.
124
+ * @param {string} html
125
+ * @returns {string}
126
+ */
127
+ export function toMarkdown(html) {
128
+ const document = cleanDocument(html);
129
+ const title = clean(document.querySelector('title')?.textContent ?? '');
130
+ const turndown = new TurndownService({ headingStyle: 'atx', codeBlockStyle: 'fenced' });
131
+ const el = bodyOf(document);
132
+ const body = el ? turndown.turndown(el.innerHTML).trim() : '';
133
+ return title && !body.startsWith(`# ${title}`) ? `# ${title}\n\n${body}` : body;
134
+ }
135
+
136
+ /**
137
+ * Whole-page cleaned HTML for `oc raw --html`, for agents that would rather
138
+ * work with markup than markdown. Same noise removal, no other rewriting.
139
+ * @param {string} html
140
+ * @returns {string}
141
+ */
142
+ export function toHTML(html) {
143
+ const document = cleanDocument(html);
144
+ const el = bodyOf(document);
145
+ return el ? el.innerHTML.trim() : '';
146
+ }
147
+
148
+ /**
149
+ * Adjacent text nodes arrive fragmented (one per inline element boundary).
150
+ * Merging them is what turns DOM noise into readable lines.
151
+ * @param {Block[]} blocks
152
+ * @returns {Block[]}
153
+ */
154
+ function mergeText(blocks) {
155
+ /** @type {Block[]} */
156
+ const out = [];
157
+ for (const b of blocks) {
158
+ const prev = out[out.length - 1];
159
+ if (b.type === 'text' && prev?.type === 'text') {
160
+ prev.text = `${prev.text} ${b.text}`;
161
+ } else {
162
+ out.push(b);
163
+ }
164
+ }
165
+ // Single stray characters (list bullets, separators) cost tokens and say nothing.
166
+ return out.filter((b) => b.type !== 'text' || b.text.length > 1);
167
+ }
package/src/fetch.js ADDED
@@ -0,0 +1,68 @@
1
+ /**
2
+ * HTTP layer. impers (libcurl-impersonate) presents a real browser TLS and
3
+ * HTTP/2 fingerprint so ordinary public pages load the way they would in
4
+ * Chrome. It is loaded lazily and native fetch is the silent fallback, so a
5
+ * bare `npm install --omit=optional` still gives a working tool. The headless
6
+ * fallback for script-gated pages lands in v0.3 and must stay lazy too:
7
+ * never import a browser here.
8
+ */
9
+
10
+ // The fetch fallback can't fake a TLS fingerprint like impers does, but it
11
+ // should at least send the same Chrome identity in its headers.
12
+ const UA = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36';
13
+
14
+ /** @type {Promise<any> | null} */
15
+ let impersPromise = null;
16
+ const loadImpers = () => {
17
+ impersPromise ??= import('impers').catch(() => null);
18
+ return impersPromise;
19
+ };
20
+
21
+ /**
22
+ * Fetch a page.
23
+ * @param {string} url - with or without a scheme, https is assumed
24
+ * @returns {Promise<{url: string, html: string, status: number, via: string}>}
25
+ * final URL after redirects, the body, the HTTP status, and which client
26
+ * identity got the page (impers:chrome, impers:firefox, or fetch)
27
+ */
28
+ export async function fetchPage(url) {
29
+ const target = /^https?:\/\//i.test(url) ? url : `https://${url}`;
30
+ const impers = await loadImpers();
31
+ return impers ? viaImpers(impers, target) : viaFetch(target);
32
+ }
33
+
34
+ async function viaImpers(impers, target) {
35
+ // Some sites (Reddit) 403 the chrome fingerprint but accept firefox, so a
36
+ // blocked first attempt gets one cheap retry with a second identity.
37
+ let via = 'impers:chrome';
38
+ let res = await impers.get(target, { impersonate: 'chrome' });
39
+ // impers mirrors the curl_cffi response shape, not the WHATWG one.
40
+ let status = res.status ?? res.statusCode ?? 0;
41
+ if (status >= 400) {
42
+ via = 'impers:firefox';
43
+ res = await impers.get(target, { impersonate: 'firefox' });
44
+ status = res.status ?? res.statusCode ?? 0;
45
+ }
46
+ if (status >= 400) throw new Error(`fetch failed: ${status} for ${target}`);
47
+ const html = typeof res.text === 'function' ? await res.text() : String(res.text ?? res.body ?? '');
48
+ return { url: res.url ?? target, html, status, via };
49
+ }
50
+
51
+ async function viaFetch(target) {
52
+ const res = await fetch(target, {
53
+ redirect: 'follow',
54
+ headers: {
55
+ 'user-agent': UA,
56
+ accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
57
+ 'accept-language': 'en-US,en;q=0.9',
58
+ },
59
+ });
60
+ if (!res.ok) {
61
+ throw new Error(`fetch failed: ${res.status} ${res.statusText} for ${target}`);
62
+ }
63
+ const type = res.headers.get('content-type') ?? '';
64
+ if (type && !type.includes('html')) {
65
+ throw new Error(`not an HTML page (${type.split(';')[0]}), nothing to distill`);
66
+ }
67
+ return { url: res.url, html: await res.text(), status: res.status, via: 'fetch' };
68
+ }
package/src/render.js ADDED
@@ -0,0 +1,115 @@
1
+ /**
2
+ * Rendering is where the token budget is enforced. Everything printed here
3
+ * gets read by a paying model, so the default view is dense and anything
4
+ * skipped says so in one line.
5
+ */
6
+
7
+ const TEXT_CAP = 200;
8
+
9
+ /**
10
+ * Rough but stable token estimate. Close enough for budgets; the point is
11
+ * that it never changes between runs, not that it matches any one tokenizer.
12
+ * @param {string} s
13
+ * @returns {number}
14
+ */
15
+ export const estimateTokens = (s) => Math.ceil(s.length / 4);
16
+
17
+ /**
18
+ * Budget-aware compact view of a distilled page.
19
+ * @param {import('./distill.js').Page} page
20
+ * @param {{budget?: number}} [opts]
21
+ * @returns {{text: string, stats: {tokens: number, blocks: number, rendered: number}}}
22
+ */
23
+ export function render(page, { budget = 500 } = {}) {
24
+ const lines = page.title ? [`# ${page.title}`] : [];
25
+ let spent = estimateTokens(lines.join('\n'));
26
+ let skipped = 0;
27
+ let hasLinks = false;
28
+ let hasInputs = false;
29
+
30
+ for (const block of collapseRuns(page.blocks)) {
31
+ // Never print the same content twice. Pages often repeat the title as
32
+ // their first heading.
33
+ if (block.type === 'heading' && block.text === page.title) continue;
34
+ const line = formatBlock(block);
35
+ if (!line) continue;
36
+ const cost = estimateTokens(line) + 1;
37
+ if (spent + cost > budget) {
38
+ skipped++;
39
+ continue;
40
+ }
41
+ spent += cost;
42
+ lines.push(line);
43
+ if (block.type === 'link' || block.type === 'button') hasLinks = true;
44
+ if (block.type === 'input') hasInputs = true;
45
+ }
46
+
47
+ if (skipped) lines.push(`... ${skipped} more blocks over budget, raise --budget or use oc raw`);
48
+ const actions = [
49
+ hasLinks && 'do <n>',
50
+ hasInputs && 'fill <n> <text>',
51
+ hasInputs && 'submit',
52
+ 'raw <url>',
53
+ ].filter(Boolean);
54
+ lines.push(`actions: ${actions.join(' | ')}`);
55
+
56
+ const text = lines.join('\n');
57
+ return {
58
+ text,
59
+ stats: { tokens: estimateTokens(text), blocks: page.blocks.length, rendered: page.blocks.length - skipped },
60
+ };
61
+ }
62
+
63
+ /**
64
+ * Collapse repeated siblings. Long runs of short
65
+ * links are almost always nav chrome (subreddit bars, tag clouds, footers)
66
+ * and would otherwise eat the whole budget before the content starts. Handles
67
+ * are assigned in distill, so the hidden links keep their numbers and the
68
+ * marker names the range.
69
+ * @param {import('./distill.js').Block[]} blocks
70
+ * @returns {import('./distill.js').Block[]}
71
+ */
72
+ function collapseRuns(blocks) {
73
+ const SHORT = 20;
74
+ const RUN = 8;
75
+ const KEEP = 5;
76
+ const out = [];
77
+ let i = 0;
78
+ while (i < blocks.length) {
79
+ let j = i;
80
+ while (j < blocks.length && blocks[j].type === 'link' && blocks[j].text.length <= SHORT) j++;
81
+ const run = j - i;
82
+ if (run > RUN) {
83
+ out.push(...blocks.slice(i, i + KEEP));
84
+ const first = blocks[i + KEEP];
85
+ const last = blocks[j - 1];
86
+ out.push({ type: 'text', text: `[${first.n}-${last.n}] ${run - KEEP} similar links, expand with oc raw` });
87
+ i = j;
88
+ } else {
89
+ out.push(blocks[i]);
90
+ i++;
91
+ }
92
+ }
93
+ return out;
94
+ }
95
+
96
+ /**
97
+ * @param {import('./distill.js').Block} b
98
+ * @returns {string}
99
+ */
100
+ function formatBlock(b) {
101
+ switch (b.type) {
102
+ case 'heading':
103
+ return `${'#'.repeat(Math.min(b.level ?? 2, 3))} ${b.text}`;
104
+ case 'link':
105
+ return `[${b.n}] ${truncate(b.text)}`;
106
+ case 'button':
107
+ return `[${b.n}] button "${truncate(b.text)}"`;
108
+ case 'input':
109
+ return `[${b.n}] input ${b.name} (${b.text})`;
110
+ default:
111
+ return truncate(b.text);
112
+ }
113
+ }
114
+
115
+ const truncate = (s) => (s.length > TEXT_CAP ? `${s.slice(0, TEXT_CAP)} ...` : s);
package/src/session.js ADDED
@@ -0,0 +1,16 @@
1
+ /**
2
+ * Sessions are plain JSON files on disk, one per name: current URL, cookies,
3
+ * the last distilled page so actions can resolve handles, and history. No
4
+ * daemon, no background process. Read/write lands in v0.2 with act.js.
5
+ */
6
+
7
+ import { homedir } from 'node:os';
8
+ import { join } from 'node:path';
9
+
10
+ export const SESSION_DIR = join(homedir(), '.only-cli', 'sessions');
11
+
12
+ /**
13
+ * @param {string} name
14
+ * @returns {string}
15
+ */
16
+ export const sessionPath = (name) => join(SESSION_DIR, `${name}.json`);