@only-cli/oc 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 only-cli
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  ![A tangle of raw HTML being funneled into a small, tidy terminal window](docs/hero.jpg)
4
4
 
5
- [![npm](https://img.shields.io/npm/v/%40only-cli%2Foc)](https://www.npmjs.com/package/@only-cli/oc) [![node](https://img.shields.io/node/v/%40only-cli%2Foc)](https://nodejs.org) [![license: MIT](https://img.shields.io/badge/license-MIT-green)](#license) [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/only-cli/oc/badge)](https://scorecard.dev/viewer/?uri=github.com/only-cli/oc)
5
+ [![npm](https://img.shields.io/npm/v/%40only-cli%2Foc)](https://www.npmjs.com/package/@only-cli/oc) [![node](https://img.shields.io/node/v/%40only-cli%2Foc)](https://nodejs.org) [![license: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE) [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/only-cli/oc/badge)](https://scorecard.dev/viewer/?uri=github.com/only-cli/oc)
6
6
 
7
7
  Turns websites into a command line interface for AI agents. `oc open <url>` fetches a page and hands back a compact, numbered view instead of raw HTML or a screenshot, so agents like Claude Code, Codex, and Antigravity can browse without burning tokens. It also gets past blocks that stop naive fetchers on some sites, by talking to the page the way a real browser would.
8
8
 
@@ -27,7 +27,7 @@ If you are an LLM reading this repository, [llms.txt](llms.txt) is the short ver
27
27
  npm install -g @only-cli/oc
28
28
  ```
29
29
 
30
- Requires Node 20+. Requests impersonate Chrome via [impers](https://github.com/lexiforest/impers); falls back to native fetch if impers is unavailable.
30
+ Requires Node 20+. Requests impersonate Chrome via [impers](https://github.com/lexiforest/impers); falls back to native fetch if impers is unavailable. Outbound fetches honor `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` when set.
31
31
 
32
32
  ### Agent skill
33
33
 
@@ -64,6 +64,8 @@ oc find <query> where a string appears on the page already open, or
64
64
  oc read <n> full text of the region at [n], up to 2000 tokens
65
65
  oc next the next budget worth of the page already open
66
66
  oc raw [url] distilled markdown of the whole page
67
+ oc <site> <verb> ... site shortcut: 'oc hn top', 'oc reddit sub ClaudeAI'
68
+ oc sites the site shortcuts that ship with oc
67
69
  oc fill <n> <text> type into a numbered input (planned)
68
70
  oc submit [n] submit a form (planned)
69
71
  ```
@@ -72,25 +74,30 @@ Flags: `--budget <tokens>` (default 500), `--json`, `--html` (raw as cleaned HTM
72
74
 
73
75
  `oc open` remembers the page it rendered in a JSON file per session under `~/.only-cli` (override with `OC_HOME`), so `oc do 3` follows `[3]` without the agent ever handling a URL. A result title on a search page is a link, so `oc do` on it opens the result rather than repeating the title. Pages longer than the budget say what they left out; `oc find`, `oc read <n>`, and `oc next` read the rest without refetching the page, and a `find` with a single match prints that region instead of the number to read it with. The budget is a target rather than a hard cap: a page that would only run a little long is printed whole rather than cut, since one extra tool call costs far more than the tokens it would have saved.
74
76
 
77
+ When a page comes back with no readable text (JavaScript-only, a consent wall, a bot challenge), `oc` says so in one line on stderr and exits 2 instead of printing a title and calling it a render. That is a different exit code from every other failure, and `--json` carries the same verdict as an `empty` field, so an agent can tell "this page has nothing on it" from "oc could not read this page" and pay for a browser only when it is worth it.
78
+
75
79
  ## Supported websites
76
80
 
77
- Works on any mostly-static site with no per-site setup: news sites, blogs, documentation, forums, search engines. A JSON API is a page here too: `oc open` on an endpoint that answers with JSON renders one numbered item per record, keeps the fields that actually differ between items, and says once what every item shares. On top of that, `clis/` ships tuned shortcuts for:
81
+ Works on any mostly-static site with no per-site setup: news sites, blogs, documentation, forums, search engines. A JSON API is a page here too: `oc open` on an endpoint that answers with JSON renders one numbered item per record, keeps the fields that actually differ between items, and says once what every item shares. On top of that, `clis/` ships tuned shortcuts, so `oc hn item 4711` or `oc gh repo only-cli oc` gets there without the agent knowing how that site spells its URLs. Name the site by its short name, its bare name, or its domain (`oc hn`, `oc ycombinator`, `oc news.ycombinator.com`), and `oc sites` prints the whole list with its verbs:
78
82
 
79
- | website | domain | shortcuts |
83
+ | website | command | shortcuts |
80
84
  | --- | --- | --- |
81
- | Hacker News | news.ycombinator.com | `top`, `new`, `item <id>`, `user <name>` |
82
- | Reddit | reddit.com (via old.reddit.com) | `sub <name>`, `post <id>`, `user <name>`, `search <query>` |
83
- | GitHub | github.com | `repo <owner> <name>`, `user <name>`, `search <query>`, `trending`, `issues <owner> <name>` |
84
- | X | x.com | `user <name>`, `post <id>` |
85
- | LinkedIn | linkedin.com | `profile <name>`, `company <name>`, `jobs <query>` (public guest views) |
86
- | DuckDuckGo | duckduckgo.com | `search <query>`, `lite <query>` |
87
- | Bing | bing.com | `search <query>`, `news <query>` |
88
- | Stack Overflow | stackoverflow.com (via Atom feeds and the Stack Exchange API) | `search <query>`, `question <id>`, `tag <name>`, `user <id>`, `recent` |
89
- | Yahoo Finance | finance.yahoo.com | `quote <symbol>`, `news <symbol>`, `history <symbol>`, `lookup <query>`, `markets`, `gainers`, `losers`, `trending` |
90
- | YouTube | youtube.com | `video <id>`, `channel <name>` |
91
- | AWS docs | docs.aws.amazon.com (search via DuckDuckGo) | `guide <service> <page>`, `page <service> <guide> <page>`, `cli <command>`, `search <query>` |
92
- | Google Cloud docs | cloud.google.com (via docs.cloud.google.com, search via DuckDuckGo) | `docs <product>`, `page <product> <page>`, `gcloud <command>`, `search <query>` |
93
- | Microsoft Learn | learn.microsoft.com (search via its RSS API) | `azure <page>`, `doc <path>`, `cli <command>`, `search <query>` |
85
+ | Hacker News | `oc hn` | `top`, `new`, `item <id>`, `user <name>` |
86
+ | Reddit | `oc reddit` (via old.reddit.com) | `sub <name>`, `post <id>`, `user <name>`, `search <query>` |
87
+ | GitHub | `oc gh` | `repo <owner> <name>`, `user <name>`, `search <query>`, `trending`, `issues <owner> <name>` |
88
+ | X | `oc x` | `user <name>`, `post <id>` |
89
+ | LinkedIn | `oc linkedin` | `profile <name>`, `company <name>`, `jobs <query>` (public guest views) |
90
+ | DuckDuckGo | `oc ddg` | `search <query>`, `lite <query>` |
91
+ | Bing | `oc bing` | `search <query>`, `news <query>` |
92
+ | Stack Overflow | `oc so` (via Atom feeds and the Stack Exchange API) | `search <query>`, `question <id>`, `tag <name>`, `user <id>`, `recent` |
93
+ | Yahoo Finance | `oc yahoo` | `quote <symbol>`, `news <symbol>`, `history <symbol>`, `lookup <query>`, `markets`, `gainers`, `losers`, `trending` |
94
+ | YouTube | `oc yt` | `video <id>`, `channel <name>` |
95
+ | Wikipedia | `oc wiki` (via `action=render`) | `article <title>`, `search <query>`, `lang <code> <title>` |
96
+ | AWS docs | `oc aws` (search via DuckDuckGo) | `guide <service> <page>`, `page <service> <guide> <page>`, `cli <command>`, `search <query>` |
97
+ | Google Cloud docs | `oc gcp` (via docs.cloud.google.com, search via DuckDuckGo) | `docs <product>`, `page <product> <page>`, `gcloud <command>`, `search <query>` |
98
+ | Microsoft Learn | `oc learn` (search via its RSS API) | `azure <page>`, `doc <path>`, `cli <command>`, `search <query>` |
99
+
100
+ A shortcut only ever resolves to a URL and then takes the same path `oc open` does, so it changes nothing about what a page costs or how it reads. The last argument takes every word after it, so `oc ddg search claude code cli` and `oc aws search s3 lifecycle rules` need no quoting, and a path argument keeps its slashes, so `oc learn doc azure/aks/what-is-aks` reaches that page.
94
101
 
95
102
  A few of these (X, Stack Overflow, YouTube, Microsoft Learn search) read pages that look login-gated or JS-only from the outside, by finding the server-rendered HTML, feed, inline data, or public API the page already ships without a login. Stack Overflow search goes through the Stack Exchange API, and each result prints its `question_id`: read one with the `question <id>` feed rather than following its link, since the question page itself answers a bot challenge instead of the question. AWS and Google Cloud render docs search purely client-side with no feed, so their `search` goes through DuckDuckGo with a baked-in `site:` filter instead. Not supported yet: pages that only render with JavaScript, sites behind logins, and sites with hard bot challenges that expose no feed.
96
103
 
@@ -106,6 +113,27 @@ Full methodology, per-task numbers, and other agents/models live in [only-cli/be
106
113
  | Jina Reader | 16,402 | blocked on the Reddit page |
107
114
  | raw HTML fetch | 177,685 | blocked on the search page |
108
115
 
116
+ Read cost is one thing, but what an agent actually spends is another, so a
117
+ second suite runs whole tasks end to end in Claude Code and compares `oc`
118
+ against the tools the agent already has. Five Wikipedia lookups, one tool per
119
+ run, Sonnet driving:
120
+
121
+ | tool | answered correctly | input tokens | cost | turns | avg time |
122
+ | --- | ---: | ---: | ---: | ---: | ---: |
123
+ | `oc wiki` | 5/5 | 5,535 | $0.27 | 22 | 11s |
124
+ | built-in `WebFetch` | 5/5 | 128,792 | $0.37 | 25 | 14s |
125
+ | built-in `WebSearch` | 5/5 | 160,431 | $0.52 | 27 | 22s |
126
+
127
+ All three got every answer right, so this is a cost result, not an accuracy one.
128
+ Input tokens are the fresh context each tool put in front of the model, which is
129
+ the number the page size drives; totals including cache reads sit closer together
130
+ because the agent's own prompt dominates them. The spread widens with the page:
131
+ `oc` cost 5.7x less than `WebFetch` on a short stub and 35x less on a long
132
+ article, because the 500 token budget makes it flat at about 1,100 tokens per
133
+ page while a full fetch pays for whatever the page weighs. `WebSearch` was given
134
+ only the question, not the article URL, which is the honest way to use it and
135
+ part of why it costs the most.
136
+
109
137
  ## Status
110
138
 
111
139
  Early. Reading works and is covered by offline tests: static pages, XML feeds, JSON APIs, budget-aware rendering, sessions, and the numbered actions `do`, `find`, `read`, `next`, and `raw`. Writing does not: `fill`, `submit`, and `back` report that they are not implemented rather than pretending, and a lazy headless fallback for script-heavy pages comes after them.
@@ -118,4 +146,4 @@ Known limits, honestly: no JavaScript rendering yet, no sites behind logins yet,
118
146
 
119
147
  ## License
120
148
 
121
- MIT
149
+ MIT, see [LICENSE](LICENSE).
@@ -1,10 +1,10 @@
1
1
  {
2
2
  "domain": "github.com",
3
3
  "commands": {
4
- "repo": { "open": "https://github.com/{owner}/{repo}", "args": ["owner", "repo"] },
4
+ "repo": { "open": "https://github.com/{owner}/{name}", "args": ["owner", "name"] },
5
5
  "user": { "open": "https://github.com/{name}", "args": ["name"] },
6
6
  "search": { "open": "https://github.com/search?q={query}&type=repositories", "args": ["query"] },
7
7
  "trending": { "open": "https://github.com/trending" },
8
- "issues": { "open": "https://github.com/{owner}/{repo}/issues", "args": ["owner", "repo"] }
8
+ "issues": { "open": "https://github.com/{owner}/{name}/issues", "args": ["owner", "name"] }
9
9
  }
10
10
  }
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "domain": "reddit.com",
3
3
  "commands": {
4
- "sub": { "open": "https://old.reddit.com/r/{sub}", "args": ["sub"] },
4
+ "sub": { "open": "https://old.reddit.com/r/{name}", "args": ["name"] },
5
5
  "post": { "open": "https://old.reddit.com/comments/{id}", "args": ["id"] },
6
6
  "user": { "open": "https://old.reddit.com/user/{name}", "args": ["name"] },
7
7
  "search": { "open": "https://old.reddit.com/search?q={query}", "args": ["query"] }
@@ -0,0 +1,8 @@
1
+ {
2
+ "domain": "wikipedia.org",
3
+ "commands": {
4
+ "article": { "open": "https://en.wikipedia.org/w/index.php?title={title}&action=render", "args": ["title"] },
5
+ "search": { "open": "https://en.wikipedia.org/w/index.php?search={query}&fulltext=1&ns0=1", "args": ["query"] },
6
+ "lang": { "open": "https://{code}.wikipedia.org/w/index.php?title={title}&action=render", "args": ["code", "title"] }
7
+ }
8
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@only-cli/oc",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "Turn websites into a compact CLI so AI agents can browse without burning tokens.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -8,17 +8,32 @@ description: Token-efficient web browsing and web content extraction for AI agen
8
8
  Renders a web page as a compact, numbered terminal view instead of raw HTML. A typical page is under 500 tokens.
9
9
 
10
10
  ```
11
- npx --yes @only-cli/oc@0.3.0 open <url> compact view, numbered elements
12
- npx --yes @only-cli/oc@0.3.0 do <n> follow link [n], or read it if [n] is text
13
- npx --yes @only-cli/oc@0.3.0 find <query> where a string appears, or that place itself
11
+ npx --yes @only-cli/oc@0.4.0 open <url> compact view, numbered elements
12
+ npx --yes @only-cli/oc@0.4.0 do <n> follow link [n], or read it if [n] is text
13
+ npx --yes @only-cli/oc@0.4.0 find <query> where a string appears, or that place itself
14
14
  when only one matches
15
- npx --yes @only-cli/oc@0.3.0 next next ~500 tokens of the page already open
16
- npx --yes @only-cli/oc@0.3.0 read <n> full text of region [n]
17
- npx --yes @only-cli/oc@0.3.0 raw [url] whole page as markdown (--html for cleaned HTML)
15
+ npx --yes @only-cli/oc@0.4.0 next next ~500 tokens of the page already open
16
+ npx --yes @only-cli/oc@0.4.0 read <n> full text of region [n]
17
+ npx --yes @only-cli/oc@0.4.0 raw [url] whole page as markdown (--html for cleaned HTML)
18
18
  ```
19
19
 
20
20
  None of these except `open`/`do`/`raw <url>` fetch anything — they replay the page `open` already saved.
21
21
 
22
+ ## Site shortcuts
23
+
24
+ `oc <site> <verb> [args]` resolves to a URL and then behaves exactly like `open` on it, so it costs the same and reads the same. It saves guessing a URL shape and, on a few sites, points at the feed or public API that answers without a login.
25
+
26
+ ```
27
+ oc hn top oc reddit sub ClaudeAI oc gh repo only-cli oc
28
+ oc wiki article Eiffel Tower oc wiki search anthropic oc wiki lang de Berlin
29
+ oc ddg search claude code oc so question 231767 oc learn doc azure/aks/what-is-aks
30
+ ```
31
+
32
+ Sites: `hn`, `reddit`, `gh`, `x`, `linkedin`, `ddg`, `bing`, `so`, `finance`, `yt`, `aws`, `gcp`, `learn`, `wiki`. Name one by short name, bare name, or domain (`oc hn`, `oc ycombinator`, `oc news.ycombinator.com`). The last argument takes every word after it, so a query or title needs no quoting. `oc sites` lists every site with its verbs, which is cheaper than guessing one.
33
+
34
+ Prefer a shortcut over a hand-built URL when one exists for the site, and prefer `oc wiki article <title>` over a search when you already know the article's name.
35
+
36
+
22
37
  ## Output
23
38
 
24
39
  - Line 1 is the title, then main content (article/thread/results); nav/sidebar/footer follow after `--- rest of page ---`, still numbered.
@@ -56,7 +71,9 @@ None of these except `open`/`do`/`raw <url>` fetch anything — they replay the
56
71
 
57
72
  ## When not to use it
58
73
 
59
- Pages needing login or heavy client-side JS aren't supported yet. If a page comes back empty or blocked, say so and fall back rather than retrying.
74
+ Pages needing login or heavy client-side JS aren't supported yet. A page with no readable text (JavaScript-only, a consent wall, a bot challenge) prints one line on stderr and exits 2, which is distinct from the exit 1 every other failure uses, so exit 2 means "oc cannot read this one" rather than "this page is empty". Take it at its word: say so and fall back to another tool rather than retrying the same URL.
75
+
76
+ Outbound fetches honor `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`, so a sandbox that only reaches the network through a proxy needs no extra flags.
60
77
 
61
78
  ## Untrusted content
62
79
 
package/src/cli.js CHANGED
@@ -2,7 +2,8 @@
2
2
  import { parseArgs } from 'node:util';
3
3
  import { fetchPage } from './fetch.js';
4
4
  import { distill, toMarkdown, toHTML } from './distill.js';
5
- import { render, estimateTokens } from './render.js';
5
+ import { render, estimateTokens, contentTokens, contentFailure, MIN_CONTENT } from './render.js';
6
+ import { resolveSite, listSites } from './sites.js';
6
7
  import * as act from './act.js';
7
8
  import { DEFAULT_SESSION, loadSession, saveSession, sessionFromPage } from './session.js';
8
9
 
@@ -11,6 +12,8 @@ const HELP = `only-cli: the web as a compact terminal, built for AI agents.
11
12
  usage: oc <command> [args] [flags]
12
13
 
13
14
  open <url> fetch and render a page with numbered actions
15
+ <site> <verb> ... site shortcut: 'oc hn top', 'oc reddit sub ClaudeAI'
16
+ sites the site shortcuts that ship with oc
14
17
  find <query> where a string appears on the page already open, or
15
18
  the region itself when only one place matches
16
19
  next the next budget worth of the page already open
@@ -35,6 +38,10 @@ flags:
35
38
  globally. Off by default because metrics cost tokens too.
36
39
  --session <name> keep separate page state under a name (default: default)
37
40
 
41
+ A page that comes back with no readable text (JavaScript-only, a consent wall,
42
+ a bot challenge) says so in one line on stderr and exits 2, so a caller can
43
+ tell an empty page from a page oc could not read and fall back to a browser.
44
+
38
45
  'oc open' remembers the page it printed, so 'oc do 3' follows link [3] without
39
46
  you ever handling its URL, and 'oc next' or 'oc read 12' picks up what the
40
47
  budget left behind without fetching it again. 'oc do' on a search result title
@@ -60,6 +67,23 @@ const remember = (page, name, cursor) => {
60
67
  const savings = (out, raw) =>
61
68
  `~${out} tokens vs ~${raw} for the page HTML (${Math.max(0, 100 - Math.round((out / Math.max(raw, 1)) * 100))}% saved)`;
62
69
 
70
+ // Nonzero, and distinct from the exit 1 that every other failure uses, so a
71
+ // caller can branch on 'oc could not read this' without parsing prose.
72
+ const NO_CONTENT_EXIT = 2;
73
+
74
+ const noContent = (url, detail, hint = "; 'oc raw' has the page's markdown if there is any, otherwise this one needs a browser") => {
75
+ console.error(`oc: no readable content at ${url} (${detail}), so it is JavaScript-only, gated, or challenged${hint}`);
76
+ // Not process.exit: stdout may still be draining, and whatever did render is
77
+ // worth printing even when the render failed.
78
+ process.exitCode = NO_CONTENT_EXIT;
79
+ };
80
+
81
+ // Anything else in the first position is tried as a site shortcut before it is
82
+ // called unknown, so a new clis/ definition needs no change here.
83
+ const COMMANDS = new Set([
84
+ 'open', 'do', 'raw', 'read', 'next', 'find', 'fill', 'submit', 'back', 'session', 'sites',
85
+ ]);
86
+
63
87
  async function main() {
64
88
  const { values, positionals } = parseArgs({
65
89
  allowPositionals: true,
@@ -78,12 +102,22 @@ async function main() {
78
102
  // when their own verbose mode is on, or the user exports OC_VERBOSE=1.
79
103
  const verbose = values.stats || values.verbose || process.env.OC_VERBOSE === '1';
80
104
 
81
- const [command, ...args] = positionals;
105
+ let [command, ...args] = positionals;
82
106
  if (values.help || !command) {
83
107
  console.log(HELP);
84
108
  return;
85
109
  }
86
110
 
111
+ // A first word that is not a command may still be a site oc ships a
112
+ // definition for, and a shortcut is only ever a URL, so it resolves to one
113
+ // here and the rest of this function never learns it was not typed.
114
+ if (!COMMANDS.has(command)) {
115
+ const site = resolveSite(command, args);
116
+ if (!site) throw new Error(`unknown command '${command}', run oc --help`);
117
+ args = [site.url];
118
+ command = 'open';
119
+ }
120
+
87
121
  const sessionName = values.session || DEFAULT_SESSION;
88
122
  // Zero means "whatever this command's default is", which differs: the
89
123
  // compact view targets 500 tokens, read targets 2000.
@@ -123,27 +157,49 @@ async function main() {
123
157
  return `HTTP ${status} via ${via}, fetch ${Math.round(fetchMs)}ms, process ${Math.round(processMs)}ms, `
124
158
  + `${Math.round(html.length / 1024)}KB transferred, ${Math.round(rss / 1048576)}MB memory`;
125
159
  };
160
+ const htmlTokens = estimateTokens(html);
126
161
  if (values.json) {
127
162
  const page = distill(html, finalUrl);
128
163
  remember(page, sessionName);
129
- console.log(JSON.stringify(page));
164
+ const failure = contentFailure(contentTokens(page), htmlTokens);
165
+ // Always present, so a caller can branch on the field rather than on
166
+ // whether a field it was hoping for turned up.
167
+ console.log(JSON.stringify({ ...page, empty: failure != null }));
130
168
  if (verbose) console.error(resources());
169
+ if (failure) noContent(finalUrl, failure);
131
170
  return;
132
171
  }
133
- const htmlTokens = estimateTokens(html);
134
172
  if (command === 'raw') {
135
173
  const out = values.html ? toHTML(html, finalUrl) : toMarkdown(html, finalUrl);
174
+ const outTokens = estimateTokens(out);
136
175
  console.log(out);
137
- if (verbose) console.error(`${savings(estimateTokens(out), htmlTokens)}; ${resources()}`);
176
+ if (verbose) {
177
+ const cost = outTokens < MIN_CONTENT
178
+ ? `nothing distilled out of ~${htmlTokens} tokens of page HTML`
179
+ : savings(outTokens, htmlTokens);
180
+ console.error(`${cost}; ${resources()}`);
181
+ }
182
+ // Only the blank case here. `raw` is the fallback the compact view's
183
+ // failure line names, so it must not fail on the same pages: a page
184
+ // whose only text is its menu still has markup, and printing it is the
185
+ // whole point of `raw`.
186
+ if (outTokens < MIN_CONTENT) noContent(finalUrl, `~${outTokens} tokens of markdown`, '');
138
187
  return;
139
188
  }
140
189
  const page = distill(html, finalUrl);
190
+ const failure = contentFailure(contentTokens(page), htmlTokens);
141
191
  const { text, stats } = render(page, { budget });
142
192
  remember(page, sessionName, stats.next);
143
193
  console.log(text);
144
194
  if (verbose) {
145
- console.error(`~${stats.tokens} tokens, ${stats.rendered}/${stats.blocks} blocks rendered, ${savings(stats.tokens, htmlTokens)}; ${resources()}`);
195
+ // Reporting '100% saved' of a render that extracted nothing is the one
196
+ // place this line lies, and it lies in the tool's own favour.
197
+ const cost = failure
198
+ ? `no content distilled out of ~${htmlTokens} tokens of page HTML`
199
+ : savings(stats.tokens, htmlTokens);
200
+ console.error(`~${stats.tokens} tokens, ${stats.rendered}/${stats.blocks} blocks rendered, ${cost}; ${resources()}`);
146
201
  }
202
+ if (failure) noContent(finalUrl, failure);
147
203
  return;
148
204
  }
149
205
  case 'read': return console.log(act.read(Number(args[0]), { session: sessionName, budget: asked || 2000 }));
@@ -152,6 +208,7 @@ async function main() {
152
208
  case 'fill': return act.fill(Number(args[0]), args.slice(1).join(' '));
153
209
  case 'submit': return act.submit(args[0] ? Number(args[0]) : undefined);
154
210
  case 'back': return act.back();
211
+ case 'sites': return console.log(listSites());
155
212
  case 'session': throw new act.NotImplemented('session');
156
213
  default:
157
214
  throw new Error(`unknown command '${command}', run oc --help`);
package/src/fetch.js CHANGED
@@ -8,7 +8,10 @@
8
8
  */
9
9
 
10
10
  import dns from 'node:dns/promises';
11
+ import http from 'node:http';
12
+ import https from 'node:https';
11
13
  import net from 'node:net';
14
+ import tls from 'node:tls';
12
15
 
13
16
  // The fetch fallback can't fake a TLS fingerprint like impers does, but it
14
17
  // should at least send the same Chrome identity in its headers.
@@ -23,6 +26,8 @@ const loadImpers = () => {
23
26
 
24
27
  const BLOCKED_MESSAGE = 'blocked: private or internal URL';
25
28
  const MAX_REDIRECTS = 20;
29
+ // Match undici's default so proxy transport does not hang indefinitely.
30
+ const PROXY_TIMEOUT_MS = 300_000;
26
31
 
27
32
  // What oc can turn into text: any text/* type, plus the application/* types
28
33
  // that are really text (json, xml, and the +json / +xml families a feed or an
@@ -139,13 +144,304 @@ async function assertSafeTarget(urlStr) {
139
144
  return;
140
145
  }
141
146
  if (hostname === 'localhost') throw new Error(BLOCKED_MESSAGE);
142
- const addresses = await dns.lookup(hostname, { all: true }).catch(() => []);
147
+ const addresses = await dns.lookup(hostname, { all: true }).catch(() => {
148
+ // With a proxy the client never resolves the target; the proxy does, often
149
+ // on corporate DNS where internal names are NXDOMAIN locally. Fail closed.
150
+ if (resolveProxy(urlStr)) throw new Error(BLOCKED_MESSAGE);
151
+ return [];
152
+ });
143
153
  for (const { address, family } of addresses) {
144
154
  if (family === 4 && isBlockedIPv4(address)) throw new Error(BLOCKED_MESSAGE);
145
155
  if (family === 6 && isBlockedIPv6(address)) throw new Error(BLOCKED_MESSAGE);
146
156
  }
147
157
  }
148
158
 
159
+ function envFirst(env, ...names) {
160
+ for (const name of names) {
161
+ const value = env[name];
162
+ if (value) return value;
163
+ }
164
+ }
165
+
166
+ function normalizeProxy(value) {
167
+ if (value == null) return null;
168
+ const trimmed = String(value).trim();
169
+ if (!trimmed) return null;
170
+ return /^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed) ? trimmed : `http://${trimmed}`;
171
+ }
172
+
173
+ function assertHttpProxyProtocol(proxyUrl) {
174
+ if (!/^https?:$/.test(proxyUrl.protocol)) {
175
+ throw new Error(`unsupported proxy protocol (${proxyUrl.protocol.slice(0, -1)}), oc honors HTTP and HTTPS proxies`);
176
+ }
177
+ }
178
+
179
+ // Split host[:port], including [IPv6]:port. Unbracketed IPv6 literals never
180
+ // carry a port suffix (use [addr]:port); a trailing :digits on ::1 is part of
181
+ // the address, not a port.
182
+ function splitHostPort(entry) {
183
+ if (entry.startsWith('[')) {
184
+ const end = entry.indexOf(']');
185
+ if (end === -1) return { host: entry, port: '' };
186
+ const rest = entry.slice(end + 1);
187
+ return { host: entry.slice(1, end), port: rest.startsWith(':') ? rest.slice(1) : '' };
188
+ }
189
+ if (net.isIP(entry) === 6) return { host: entry, port: '' };
190
+ const colon = entry.lastIndexOf(':');
191
+ if (colon !== -1 && /^\d+$/.test(entry.slice(colon + 1))) {
192
+ const host = entry.slice(0, colon);
193
+ if (net.isIP(host) === 6) return { host: entry, port: '' };
194
+ return { host, port: entry.slice(colon + 1) };
195
+ }
196
+ return { host: entry, port: '' };
197
+ }
198
+
199
+ function ipv4InCidr(ip, base, bits) {
200
+ const mask = bits === 0 ? 0 : (0xffffffff << (32 - bits)) >>> 0;
201
+ return (ipv4ToInt(ip) & mask) === (ipv4ToInt(base) & mask);
202
+ }
203
+
204
+ function ipv6InCidr(ip, base, bits) {
205
+ const g = expandIPv6(ip);
206
+ const b = expandIPv6(base);
207
+ if (!g || !b) return false;
208
+ let remaining = bits;
209
+ for (let i = 0; i < 8 && remaining > 0; i++) {
210
+ if (remaining >= 16) {
211
+ if (g[i] !== b[i]) return false;
212
+ remaining -= 16;
213
+ } else {
214
+ const mask = (0xffff << (16 - remaining)) & 0xffff;
215
+ if ((g[i] & mask) !== (b[i] & mask)) return false;
216
+ remaining = 0;
217
+ }
218
+ }
219
+ return true;
220
+ }
221
+
222
+ function ipInCidr(ip, base, bits) {
223
+ const family = net.isIP(ip);
224
+ if (family === 4) return ipv4InCidr(ip, base, bits);
225
+ if (family === 6) return ipv6InCidr(ip, base, bits);
226
+ return false;
227
+ }
228
+
229
+ function bypassesProxy(target, noProxy) {
230
+ const list = noProxy.trim();
231
+ if (!list) return false;
232
+ const hostname = target.hostname.toLowerCase().replace(/^\[|\]$/g, '');
233
+ const port = target.port || (target.protocol === 'https:' ? '443' : '80');
234
+ for (let entry of list.split(',')) {
235
+ entry = entry.trim();
236
+ if (!entry) continue;
237
+ if (entry === '*') return true;
238
+ const { host: rawHost, port: entryPort } = splitHostPort(entry);
239
+ if (entryPort && entryPort !== port) continue;
240
+ let pattern = rawHost.toLowerCase().replace(/^\[|\]$/g, '');
241
+ const wildcard = pattern.startsWith('*.');
242
+ if (wildcard) pattern = pattern.slice(2);
243
+ const slash = pattern.indexOf('/');
244
+ if (slash !== -1 && net.isIP(pattern.slice(0, slash))) {
245
+ const bits = Number(pattern.slice(slash + 1));
246
+ if (Number.isInteger(bits) && net.isIP(hostname) && ipInCidr(hostname, pattern.slice(0, slash), bits)) {
247
+ return true;
248
+ }
249
+ continue;
250
+ }
251
+ const host = pattern.replace(/^\./, '');
252
+ if (!host) continue;
253
+ if (wildcard) {
254
+ if (hostname.endsWith(`.${host}`)) return true;
255
+ continue;
256
+ }
257
+ if (hostname === host || hostname.endsWith(`.${host}`)) return true;
258
+ }
259
+ return false;
260
+ }
261
+
262
+ /**
263
+ * Pick a proxy for this URL from HTTP_PROXY / HTTPS_PROXY / NO_PROXY (and
264
+ * their lowercase forms). The proxy host is not run through assertSafeTarget:
265
+ * corporate proxies live on loopback or RFC 1918 addresses, and they are not
266
+ * the page being fetched. Redirect hops still go through that check.
267
+ * @param {string} url
268
+ * @param {NodeJS.ProcessEnv} [env]
269
+ * @returns {string | null} proxy URL, or null to connect directly
270
+ */
271
+ export function resolveProxy(url, env = process.env) {
272
+ const target = new URL(url);
273
+ if (bypassesProxy(target, envFirst(env, 'NO_PROXY', 'no_proxy') ?? '')) return null;
274
+ const httpsProxy = envFirst(env, 'HTTPS_PROXY', 'https_proxy');
275
+ const httpProxy = envFirst(env, 'HTTP_PROXY', 'http_proxy');
276
+ const chosen = target.protocol === 'https:' ? (httpsProxy || httpProxy) : httpProxy;
277
+ const normalized = normalizeProxy(chosen);
278
+ if (!normalized) return null;
279
+ assertHttpProxyProtocol(new URL(normalized));
280
+ return normalized;
281
+ }
282
+
283
+ function decodeCredential(value) {
284
+ try {
285
+ return decodeURIComponent(value);
286
+ } catch {
287
+ return value;
288
+ }
289
+ }
290
+
291
+ function proxyAuthHeader(proxy) {
292
+ if (!proxy.username) return undefined;
293
+ const token = Buffer.from(`${decodeCredential(proxy.username)}:${decodeCredential(proxy.password)}`).toString('base64');
294
+ return `Basic ${token}`;
295
+ }
296
+
297
+ function authority(target) {
298
+ const host = net.isIP(target.hostname) === 6 ? `[${target.hostname}]` : target.hostname;
299
+ const port = target.port || (target.protocol === 'https:' ? '443' : '80');
300
+ return `${host}:${port}`;
301
+ }
302
+
303
+ function wrapNodeResponse(res, url) {
304
+ const headers = {
305
+ get(name) {
306
+ const v = res.headers[name.toLowerCase()];
307
+ if (v == null) return null;
308
+ return Array.isArray(v) ? v.join(', ') : v;
309
+ },
310
+ };
311
+ const text = () => new Promise((resolve, reject) => {
312
+ const chunks = [];
313
+ res.on('data', (c) => chunks.push(c));
314
+ res.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
315
+ res.on('error', reject);
316
+ });
317
+ const status = res.statusCode ?? 0;
318
+ return {
319
+ status,
320
+ statusText: res.statusMessage || '',
321
+ headers,
322
+ ok: status >= 200 && status < 300,
323
+ url,
324
+ text,
325
+ };
326
+ }
327
+
328
+ function proxyTransport(proxy) {
329
+ return proxy.protocol === 'https:' ? https : http;
330
+ }
331
+
332
+ function proxyPort(proxy) {
333
+ return Number(proxy.port) || (proxy.protocol === 'https:' ? 443 : 80);
334
+ }
335
+
336
+ function armRequestTimeout(req, reject, label) {
337
+ req.setTimeout(PROXY_TIMEOUT_MS, () => {
338
+ req.destroy();
339
+ reject(new Error(`proxy timed out after ${PROXY_TIMEOUT_MS / 1000}s for ${label}`));
340
+ });
341
+ }
342
+
343
+ function pickTlsCa(tlsOpts) {
344
+ return tlsOpts.ca != null ? { ca: tlsOpts.ca } : {};
345
+ }
346
+
347
+ function httpViaProxy(target, proxy, headers) {
348
+ const auth = proxyAuthHeader(proxy);
349
+ return new Promise((resolve, reject) => {
350
+ const req = proxyTransport(proxy).request({
351
+ hostname: proxy.hostname,
352
+ port: proxyPort(proxy),
353
+ method: 'GET',
354
+ path: `${target.protocol}//${target.host}${target.pathname}${target.search}`,
355
+ headers: {
356
+ ...headers,
357
+ host: target.host,
358
+ ...(auth && { 'proxy-authorization': auth }),
359
+ },
360
+ }, (res) => resolve(wrapNodeResponse(res, target.href)));
361
+ armRequestTimeout(req, reject, target.href);
362
+ req.on('error', (err) => reject(new Error(`proxy failed: ${err.message} for ${target.href}`)));
363
+ req.end();
364
+ });
365
+ }
366
+
367
+ function httpsViaConnect(target, proxy, headers, tlsOpts = {}) {
368
+ const dest = authority(target);
369
+ const auth = proxyAuthHeader(proxy);
370
+ return new Promise((resolve, reject) => {
371
+ let settled = false;
372
+ const fail = (err) => {
373
+ if (settled) return;
374
+ settled = true;
375
+ req.destroy();
376
+ reject(err instanceof Error ? err : new Error(String(err)));
377
+ };
378
+ const req = proxyTransport(proxy).request({
379
+ hostname: proxy.hostname,
380
+ port: proxyPort(proxy),
381
+ method: 'CONNECT',
382
+ path: dest,
383
+ headers: {
384
+ host: dest,
385
+ ...(auth && { 'proxy-authorization': auth }),
386
+ },
387
+ });
388
+ armRequestTimeout(req, fail, target.href);
389
+ req.on('connect', (res, socket, head) => {
390
+ if (res.statusCode !== 200) {
391
+ socket.destroy();
392
+ fail(new Error(`proxy CONNECT failed: ${res.statusCode} for ${target.href}`));
393
+ return;
394
+ }
395
+ if (head.length) socket.unshift(head);
396
+ // tls.connect already opened the tunnel. https.request would wrap TLS
397
+ // again, and the origin would see a second ClientHello as garbage.
398
+ // SNI is a hostname; an IP literal is only used for the cert check.
399
+ const hostname = target.hostname;
400
+ const tlsSocket = tls.connect({
401
+ socket,
402
+ host: hostname,
403
+ ...(net.isIP(hostname) ? {} : { servername: hostname }),
404
+ ...pickTlsCa(tlsOpts),
405
+ }, () => {
406
+ const tunneled = http.request({
407
+ createConnection: () => tlsSocket,
408
+ path: `${target.pathname}${target.search}`,
409
+ method: 'GET',
410
+ headers: { ...headers, host: target.host },
411
+ }, (httpsRes) => {
412
+ if (settled) return;
413
+ settled = true;
414
+ resolve(wrapNodeResponse(httpsRes, target.href));
415
+ });
416
+ armRequestTimeout(tunneled, fail, target.href);
417
+ tunneled.on('error', (err) => fail(new Error(`proxy failed: ${err.message || err.code} for ${target.href}`)));
418
+ tunneled.end();
419
+ });
420
+ tlsSocket.on('error', (err) => fail(new Error(`proxy failed: ${err.message || err.code} for ${target.href}`)));
421
+ });
422
+ req.on('error', (err) => fail(new Error(`proxy failed: ${err.message || err.code} for ${target.href}`)));
423
+ req.end();
424
+ });
425
+ }
426
+
427
+ /**
428
+ * One GET through an HTTP(S) proxy. HTTP targets use the absolute-URI form;
429
+ * HTTPS targets open a CONNECT tunnel first. Redirects are not followed:
430
+ * followRedirects owns that so each hop still goes through assertSafeTarget.
431
+ * @param {string} url
432
+ * @param {string} proxy
433
+ * @param {Record<string, string>} [headers]
434
+ * @param {import('node:tls').ConnectionOptions} [tlsOpts]
435
+ */
436
+ export function proxyGet(url, proxy, headers = {}, tlsOpts = {}) {
437
+ const target = new URL(url);
438
+ const proxyUrl = new URL(proxy);
439
+ assertHttpProxyProtocol(proxyUrl);
440
+ return target.protocol === 'https:'
441
+ ? httpsViaConnect(target, proxyUrl, headers, tlsOpts)
442
+ : httpViaProxy(target, proxyUrl, headers);
443
+ }
444
+
149
445
  /**
150
446
  * Fetch a page.
151
447
  * @param {string} url - with or without a scheme, https is assumed
@@ -190,10 +486,17 @@ export async function followRedirects(get, start) {
190
486
  }
191
487
  }
192
488
 
489
+ const FETCH_HEADERS = {
490
+ 'user-agent': UA,
491
+ accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
492
+ 'accept-language': 'en-US,en;q=0.9',
493
+ };
494
+
193
495
  async function viaImpers(impers, target) {
194
496
  // Some sites (Reddit) 403 the chrome fingerprint but accept firefox, so a
195
497
  // blocked first attempt gets one cheap retry with a second identity.
196
- const asking = (impersonate) => (url) => impers.get(url, { impersonate, allowRedirects: false });
498
+ const asking = (impersonate) => (url) =>
499
+ impers.get(url, { impersonate, allowRedirects: false, proxy: resolveProxy(url) ?? '' });
197
500
  let via = 'impers:chrome';
198
501
  let { res } = await followRedirects(asking('chrome'), target);
199
502
  let status = res.status ?? res.statusCode ?? 0;
@@ -209,14 +512,12 @@ async function viaImpers(impers, target) {
209
512
  }
210
513
 
211
514
  async function viaFetch(target) {
212
- const { res, url: current } = await followRedirects((url) => fetch(url, {
213
- redirect: 'manual',
214
- headers: {
215
- 'user-agent': UA,
216
- accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
217
- 'accept-language': 'en-US,en;q=0.9',
218
- },
219
- }), target);
515
+ const { res, url: current } = await followRedirects((url) => {
516
+ const proxy = resolveProxy(url);
517
+ return proxy
518
+ ? proxyGet(url, proxy, FETCH_HEADERS)
519
+ : fetch(url, { redirect: 'manual', headers: FETCH_HEADERS });
520
+ }, target);
220
521
  if (!res.ok) {
221
522
  throw new Error(`fetch failed: ${res.status} ${res.statusText} for ${current}`);
222
523
  }
package/src/render.js CHANGED
@@ -16,6 +16,62 @@ export const estimateTokens = (s) => Math.ceil(s.length / 4);
16
16
 
17
17
  const num = (v) => v.toLocaleString('en-US');
18
18
 
19
+ // A page that distilled to nothing must not read like a page with nothing on
20
+ // it. JS-only pages, consent walls, and bot challenges all answer HTTP 200 with
21
+ // markup carrying no text, and "100% saved" is technically true of a render
22
+ // that saved every token by extracting none. From the output alone an agent
23
+ // cannot tell that from a genuinely empty page, so it never falls back to
24
+ // something heavier and an empty result travels on as evidence. oc has to fail
25
+ // loud and cheap there instead, per principle 5 in CONTRIBUTING.
26
+ //
27
+ // The numbers below are measured, not guessed. Against live pages, the thinnest
28
+ // render the README claims (an X profile) carries about 460 tokens of content,
29
+ // while a JS-only or gated one carries under 55 (reddit.com/r/*,
30
+ // instagram.com), so a threshold in between never has to be a close call.
31
+
32
+ // Text the page itself wrote, in tokens: prose and headings, plus link and
33
+ // button labels long enough to be content rather than furniture. Nav chrome is
34
+ // short by nature ("Help", "Log in") and a headline or a post title is not,
35
+ // which is what tells a link-list page (Hacker News, search results) from a
36
+ // page whose only links are its own menu.
37
+ const CONTENT_LABEL = 25;
38
+ // Below this there is nothing to read whatever the page is, so how much markup
39
+ // it arrived in does not matter.
40
+ export const MIN_CONTENT = 25;
41
+ // Below this, with markup that large behind it, the fetch worked and the render
42
+ // did not: a real page of that weight always distills to more. A genuinely
43
+ // short page is exempt, because its HTML never reaches THIN_HTML.
44
+ const THIN_CONTENT = 100;
45
+ const THIN_HTML = 2500;
46
+
47
+ /**
48
+ * How much of a distilled page is text the page itself wrote.
49
+ * @param {import('./distill.js').Page} page
50
+ * @returns {number}
51
+ */
52
+ export const contentTokens = (page) =>
53
+ page.blocks.reduce((sum, b) => {
54
+ if (b.type === 'heading' || b.type === 'text') return sum + estimateTokens(b.text ?? '');
55
+ const label = (b.type === 'link' || b.type === 'button') && (b.text ?? '').length > CONTENT_LABEL;
56
+ return label ? sum + estimateTokens(b.text) : sum;
57
+ }, 0);
58
+
59
+ /**
60
+ * Why this render carries no content worth printing, as a phrase for the
61
+ * failure line, or null when it does carry some. Both figures are token counts,
62
+ * which is the unit the caller is paying in.
63
+ * @param {number} content
64
+ * @param {number} htmlTokens
65
+ * @returns {string|null}
66
+ */
67
+ export function contentFailure(content, htmlTokens) {
68
+ if (content < MIN_CONTENT) return `~${content} tokens of text on the whole page`;
69
+ if (content < THIN_CONTENT && htmlTokens > THIN_HTML) {
70
+ return `~${content} tokens of text out of ~${htmlTokens} of HTML`;
71
+ }
72
+ return null;
73
+ }
74
+
19
75
  // How far past the budget a page may run and still be printed whole. Cutting a
20
76
  // page that was nearly done costs the agent a second command, and a command is
21
77
  // dear: measured inside Claude Code, one tool call is 23,000 to 33,000 tokens of
package/src/sites.js ADDED
@@ -0,0 +1,148 @@
1
+ /**
2
+ * Site shortcuts. `clis/*.json` names the URLs on a site worth reaching
3
+ * directly, so `oc hn item 4711` gets there without the agent knowing that
4
+ * Hacker News spells it /item?id=. A shortcut is only ever a URL: it resolves
5
+ * to one and hands off to the same fetch and render path `oc open` uses, so
6
+ * nothing here can change what a page costs or how it reads.
7
+ */
8
+
9
+ import { readdirSync, readFileSync } from 'node:fs';
10
+ import { fileURLToPath } from 'node:url';
11
+
12
+ const DIR = fileURLToPath(new URL('../clis/', import.meta.url));
13
+
14
+ // Short names an agent is likely to reach for. The domain itself and its
15
+ // registrable label always resolve, so this only covers what those miss.
16
+ const ALIASES = {
17
+ hn: 'news.ycombinator.com',
18
+ gh: 'github.com',
19
+ so: 'stackoverflow.com',
20
+ ddg: 'duckduckgo.com',
21
+ yt: 'youtube.com',
22
+ finance: 'finance.yahoo.com',
23
+ twitter: 'x.com',
24
+ aws: 'docs.aws.amazon.com',
25
+ gcp: 'cloud.google.com',
26
+ learn: 'learn.microsoft.com',
27
+ wiki: 'wikipedia.org',
28
+ };
29
+
30
+ /** @typedef {{open: string, args?: string[]}} Shortcut */
31
+ /** @typedef {{domain: string, commands: Record<string, Shortcut>}} Site */
32
+
33
+ /** @type {Map<string, Site>|null} */
34
+ let cache = null;
35
+
36
+ /**
37
+ * Every site definition that ships with oc, keyed by each name that resolves
38
+ * to it. A definition that will not parse is skipped rather than fatal: a bad
39
+ * file costs its own shortcuts and leaves every other command working.
40
+ * @returns {Map<string, Site>}
41
+ */
42
+ export function sites() {
43
+ if (cache) return cache;
44
+ cache = new Map();
45
+ /** @type {string[]} */
46
+ let files = [];
47
+ try {
48
+ files = readdirSync(DIR).filter((f) => f.endsWith('.json')).sort();
49
+ } catch {
50
+ return cache;
51
+ }
52
+ for (const file of files) {
53
+ /** @type {Site} */
54
+ let site;
55
+ try {
56
+ site = JSON.parse(readFileSync(DIR + file, 'utf8'));
57
+ } catch {
58
+ continue;
59
+ }
60
+ if (!site?.domain || !site.commands) continue;
61
+ const labels = site.domain.split('.');
62
+ for (const key of [site.domain, labels[labels.length - 2]]) {
63
+ // First definition wins, in sorted filename order, so which site owns a
64
+ // shared label never depends on how the directory happens to be read.
65
+ if (key && !cache.has(key)) cache.set(key, site);
66
+ }
67
+ }
68
+ for (const [alias, domain] of Object.entries(ALIASES)) {
69
+ const site = cache.get(domain);
70
+ if (site && !cache.has(alias)) cache.set(alias, site);
71
+ }
72
+ return cache;
73
+ }
74
+
75
+ const verbs = (site) =>
76
+ Object.entries(site.commands)
77
+ .map(([name, def]) => (def.args?.length ? `${name} <${def.args.join('> <')}>` : name))
78
+ .join(' | ');
79
+
80
+ /**
81
+ * Resolve `oc <site> <verb> [args]` to a URL, or null when the first word
82
+ * names no site oc ships, which is the caller's cue to report an unknown
83
+ * command. A site that exists with a verb that does not is an error here
84
+ * instead, since the agent has the right site and only needs the verb list.
85
+ * @param {string} name
86
+ * @param {string[]} args
87
+ * @returns {{url: string, domain: string, command: string}|null}
88
+ */
89
+ export function resolveSite(name, args) {
90
+ const site = sites().get(name.toLowerCase());
91
+ if (!site) return null;
92
+ const [verb, ...rest] = args;
93
+ if (!verb) throw new Error(`usage: oc ${name} <verb>, one of: ${verbs(site)}`);
94
+ const def = site.commands[verb];
95
+ if (!def) throw new Error(`'${verb}' is not a ${site.domain} shortcut, try: ${verbs(site)}`);
96
+
97
+ const need = def.args ?? [];
98
+ if (rest.length < need.length) {
99
+ throw new Error(`usage: oc ${name} ${verb} <${need.join('> <')}>`);
100
+ }
101
+ // The last argument soaks up everything left, so a query an agent typed as
102
+ // separate words ('oc ddg search claude code cli') works unquoted.
103
+ const values = need.map((_, i) =>
104
+ i === need.length - 1 ? rest.slice(i).join(' ') : rest[i]);
105
+ const url = need.reduce(
106
+ (open, arg, i) => open.replaceAll(`{${arg}}`, encode(values[i], def.open, arg)),
107
+ def.open);
108
+ return { url, domain: site.domain, command: verb };
109
+ }
110
+
111
+ /**
112
+ * Percent-encode one value for the slot it fills. A value in the query string
113
+ * is encoded outright, but a docs path is often several segments deep, so
114
+ * 'oc learn doc azure/aks/intro' has to keep its slashes: escaping them would
115
+ * ask the site for one impossible segment instead of the page.
116
+ * @param {string} value
117
+ * @param {string} template
118
+ * @param {string} arg
119
+ * @returns {string}
120
+ */
121
+ function encode(value, template, arg) {
122
+ const query = template.includes('?') && template.indexOf(`{${arg}}`) > template.indexOf('?');
123
+ const encoded = encodeURIComponent(value);
124
+ return query ? encoded : encoded.replaceAll('%2F', '/');
125
+ }
126
+
127
+ /**
128
+ * One line per site for `oc sites`, shortest name first, since that is what an
129
+ * agent will type. Discovery has to cost less than a wrong guess does, so this
130
+ * stays one line each rather than a table.
131
+ * @returns {string}
132
+ */
133
+ export function listSites() {
134
+ const byDomain = new Map();
135
+ for (const [key, site] of sites()) {
136
+ if (!byDomain.has(site.domain)) byDomain.set(site.domain, { site, keys: [] });
137
+ byDomain.get(site.domain).keys.push(key);
138
+ }
139
+ const lines = [...byDomain.values()].map(({ site, keys }) => {
140
+ const names = keys
141
+ .filter((k) => k !== site.domain)
142
+ .sort((a, b) => a.length - b.length || a.localeCompare(b));
143
+ return `oc ${names[0] ?? site.domain} <verb> (${[...names.slice(1), site.domain].join(', ')}): ${verbs(site)}`;
144
+ });
145
+ return lines.length
146
+ ? `${lines.join('\n')}\nany other site: oc open <url>`
147
+ : 'no site definitions found; oc open <url> works on any site';
148
+ }