@only-cli/oc 0.5.2 → 0.5.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -107,7 +107,7 @@ Works on any mostly-static site with no per-site setup: news sites, blogs, docum
107
107
 
108
108
  A shortcut only ever resolves to a URL and then takes the same path `oc open` does, so it changes nothing about what a page costs or how it reads. The last argument takes every word after it, so `oc ddg search claude code cli` and `oc aws search s3 lifecycle rules` need no quoting, and a path argument keeps its slashes, so `oc learn doc azure/aks/what-is-aks` reaches that page.
109
109
 
110
- A few of these (X, Stack Overflow, YouTube, Microsoft Learn search) read pages that look login-gated or JS-only from the outside, by finding the server-rendered HTML, feed, inline data, or public API the page already ships without a login. Stack Overflow search goes through the Stack Exchange API, and each result prints its `question_id`: read one with the `question <id>` feed rather than following its link, since the question page itself answers a bot challenge instead of the question. AWS, Google Cloud, Rust, Java, TypeScript, PHP, and cppreference render docs search client-side, or as a page too bare for oc to read, so their `search` goes through DuckDuckGo with a baked-in `site:` filter instead; Go needs no such fallback, because pkg.go.dev renders its search results on the server and `oc go search` simply opens them. Python's docs are built with Sphinx, which publishes the site's full-text search index as one static file, so `oc py search` fetches that index (cached on disk for a day), ranks it locally, and prints a numbered result list; a query that names a symbol exactly, like `json.dumps`, links straight to its anchor. The same backend will work for any Sphinx site, including most Read the Docs projects. MDN also renders its search client-side, but the page gets its results from a public JSON endpoint, so `oc mdn search` asks that endpoint directly and prints the site's own ranking; that `api` shape in a site definition works for any site whose search answers as JSON. Node.js ships no search endpoint at all, but publishes its whole API reference as one static JSON file, so `oc node search` ranks that file locally the same way the Sphinx backend does, under the same day cache, and every module, class, method, property, and event heading links to its own anchor. Ruby's docs are built with RDoc, which also ships its search index as one static file, so `oc ruby search` ranks every class, method, and guide page locally the same way. PHP's manual has a lookup endpoint that sends an exact function name straight to its page, which is what `oc php fn` rides. Not supported yet: pages that only render with JavaScript and sites with hard bot challenges that expose no feed. Sites that genuinely require your account can be reached with `oc login` (bring your own cookies).
110
+ A few of these (X, Reddit, Stack Overflow, YouTube, Microsoft Learn search) read pages that look login-gated or JS-only from the outside, by finding the server-rendered HTML, feed, inline data, or public API the page already ships without a login. Stack Overflow search goes through the Stack Exchange API, and each result prints its `question_id`: read one with the `question <id>` feed rather than following its link, since the question page itself answers a bot challenge instead of the question. Reddit goes through the Atom feeds on www.reddit.com: old.reddit.com has sent logged-out readers to a login page since June 2026 and the `.json` views answer 403 without an OAuth token, so a reddit.com page URL handed to `oc open` still meets that wall, where `oc reddit post <id>`, or the same URL with `/.rss` on the end, reads the post and its comments. oc asks reddit.com with its Firefox fingerprint first, because Reddit's edge refuses the Chrome one more often than not. The feeds carry titles, authors, dates, and bodies but no scores or comment counts, and anonymous reddit.com allows roughly ten requests a minute per address, so a burst of Reddit shortcuts ends in a 429 that takes minutes to clear. AWS, Google Cloud, Rust, Java, TypeScript, PHP, and cppreference render docs search client-side, or as a page too bare for oc to read, so their `search` goes through DuckDuckGo with a baked-in `site:` filter instead; Go needs no such fallback, because pkg.go.dev renders its search results on the server and `oc go search` simply opens them. Python's docs are built with Sphinx, which publishes the site's full-text search index as one static file, so `oc py search` fetches that index (cached on disk for a day), ranks it locally, and prints a numbered result list; a query that names a symbol exactly, like `json.dumps`, links straight to its anchor. The same backend will work for any Sphinx site, including most Read the Docs projects. MDN also renders its search client-side, but the page gets its results from a public JSON endpoint, so `oc mdn search` asks that endpoint directly and prints the site's own ranking; that `api` shape in a site definition works for any site whose search answers as JSON. Node.js ships no search endpoint at all, but publishes its whole API reference as one static JSON file, so `oc node search` ranks that file locally the same way the Sphinx backend does, under the same day cache, and every module, class, method, property, and event heading links to its own anchor. Ruby's docs are built with RDoc, which also ships its search index as one static file, so `oc ruby search` ranks every class, method, and guide page locally the same way. PHP's manual has a lookup endpoint that sends an exact function name straight to its page, which is what `oc php fn` rides. Not supported yet: pages that only render with JavaScript and sites with hard bot challenges that expose no feed. Sites that genuinely require your account can be reached with `oc login` (bring your own cookies).
111
111
 
112
112
  Want a website on that list? Open a pull request, or an issue naming the site; see [CONTRIBUTING.md](CONTRIBUTING.md).
113
113
 
@@ -171,7 +171,7 @@ Private and internal addresses are refused whether or not a proxy is set. With a
171
171
  Full methodology, per-task rows, and the Codex runs live in [only-cli/benchmarks](https://github.com/only-cli/benchmarks). Where things stand (oc 0.5.1, September 2026, live sites):
172
172
 
173
173
  - **125x fewer tokens than raw HTML** across the twelve real pages both could read: 8,519 against 1,064,474. 15x fewer than Jina Reader, 59x fewer than Playwright MCP's accessibility snapshot.
174
- - **Real content on every page it could reach, and an honest failure on the two it could not.** Reddit now sends logged-out readers to a login wall, and every other tool returned that wall, or a 403 block page, as a success. Yahoo Finance refuses plain fetch outright and DuckDuckGo still blocks lynx; oc's Chrome impersonation read both.
174
+ - **Real content on every page it could reach, and an honest failure on the two it could not.** Reddit now sends logged-out readers to a login wall, and every other tool returned that wall, or a 403 block page, as a success. The suite hands every tool the old.reddit.com page URLs, which still end at that wall; since 0.5.2 the `oc reddit` shortcuts read the same threads through Reddit's Atom feeds, a route the suite does not measure yet. Yahoo Finance refuses plain fetch outright and DuckDuckGo still blocks lynx; oc's Chrome impersonation read both.
175
175
  - **Half the cost of Claude Code's built-in `WebSearch`** on Wikipedia lookups: $0.23 against $0.45 for five questions, both 5/5 correct, on 25x less fresh input.
176
176
  - **21% cheaper than `WebFetch` and 34% cheaper than `WebSearch`** on eleven language docs lookups, at equal or better accuracy.
177
177
  - **10% cheaper than `WebFetch` and 49% cheaper than `WebSearch`** on twelve dependency lookups across GitHub, npm, PyPI, RubyGems, crates.io, Docker Hub, Stack Overflow and an RFC, 12/12 correct with no tuned shortcut for most of those sites.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@only-cli/oc",
3
- "version": "0.5.2",
3
+ "version": "0.5.3",
4
4
  "description": "Turn websites into a compact CLI so AI agents can browse without burning tokens.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -8,15 +8,15 @@ description: Token-efficient web browsing and web content extraction for AI agen
8
8
  Renders a web page as a compact, numbered terminal view instead of raw HTML. A typical page is under 500 tokens.
9
9
 
10
10
  ```
11
- npx --yes @only-cli/oc@0.5.2 open <url> compact view, numbered elements
12
- npx --yes @only-cli/oc@0.5.2 do <n> follow link [n], or read it if [n] is text
13
- npx --yes @only-cli/oc@0.5.2 find <query> where a string appears, or that place itself
11
+ npx --yes @only-cli/oc@0.5.3 open <url> compact view, numbered elements
12
+ npx --yes @only-cli/oc@0.5.3 do <n> follow link [n], or read it if [n] is text
13
+ npx --yes @only-cli/oc@0.5.3 find <query> where a string appears, or that place itself
14
14
  when only one matches
15
- npx --yes @only-cli/oc@0.5.2 next next ~500 tokens of the page already open
16
- npx --yes @only-cli/oc@0.5.2 read <n> full text of region [n]
17
- npx --yes @only-cli/oc@0.5.2 raw [url] whole page as markdown (--html for cleaned HTML)
18
- npx --yes @only-cli/oc@0.5.2 login seed cookies (--cookie, --domain, --expires)
19
- npx --yes @only-cli/oc@0.5.2 logout [session] forget a session: cookies and saved page
15
+ npx --yes @only-cli/oc@0.5.3 next next ~500 tokens of the page already open
16
+ npx --yes @only-cli/oc@0.5.3 read <n> full text of region [n]
17
+ npx --yes @only-cli/oc@0.5.3 raw [url] whole page as markdown (--html for cleaned HTML)
18
+ npx --yes @only-cli/oc@0.5.3 login seed cookies (--cookie, --domain, --expires)
19
+ npx --yes @only-cli/oc@0.5.3 logout [session] forget a session: cookies and saved page
20
20
  ```
21
21
 
22
22
  None of these except `open`/`do`/`raw <url>` fetch anything; they replay the page `open` already saved.
@@ -32,7 +32,7 @@ oc ddg search claude code oc so question 231767 oc learn doc azur
32
32
  oc py library json oc mdn js Array/map oc node api fs
33
33
  ```
34
34
 
35
- Sites: `hn`, `reddit`, `gh`, `x`, `linkedin`, `ddg`, `bing`, `so`, `yahoo`, `yt`, `aws`, `gcp`, `learn`, `wiki`, `py`, `mdn`, `node`, `ruby`, `go`, `rust`, `java`, `php`, `cpp`, `ts`. Name one by short name, bare name, or domain (`oc hn`, `oc ycombinator`, `oc news.ycombinator.com`). The last argument takes every word after it, so a query or title needs no quoting. `oc sites` lists every site with its verbs, which is cheaper than guessing one.
35
+ Sites: `hn`, `reddit`, `gh`, `x`, `linkedin`, `ddg`, `bing`, `so`, `yahoo`, `yt`, `aws`, `gcp`, `learn`, `wiki`, `py`, `mdn`, `node`, `ruby`, `go`, `rust`, `java`, `php`, `cpp`, `ts`. Name one by short name, bare name, or domain (`oc hn`, `oc ycombinator`, `oc news.ycombinator.com`). The last argument takes every word after it, so a query or title needs no quoting. `oc sites` lists every site with its verbs, which is cheaper than guessing one. Reddit reads through its Atom feeds: a reddit.com page URL given to `open` meets a login wall or a 403, so use `oc reddit post <id>` or add `/.rss` to the URL, and keep Reddit calls under about ten a minute or the site answers 429.
36
36
 
37
37
  Prefer a shortcut over a hand-built URL when one exists for the site, and prefer `oc wiki article <title>` over a search when you already know the article's name.
38
38
 
package/src/fetch.js CHANGED
@@ -555,6 +555,29 @@ function captureSetCookie(jar, url, res) {
555
555
  jar.storeFromResponse(url, getSetCookieHeaders(res));
556
556
  }
557
557
 
558
+ // Hosts whose edge answers the chrome fingerprint with a 403 or a 429 while
559
+ // letting firefox through. reddit.com started doing this in 2026 (#52), so
560
+ // starting with chrome there would turn every read into two requests against
561
+ // a per-address rate limit of about ten a minute. Subdomains inherit the
562
+ // entry.
563
+ const FIREFOX_FIRST_HOSTS = ['reddit.com'];
564
+
565
+ /**
566
+ * The order in which impers identities are tried for a URL.
567
+ * @param {string} target
568
+ * @returns {['chrome', 'firefox'] | ['firefox', 'chrome']}
569
+ */
570
+ export function identityOrder(target) {
571
+ let host = '';
572
+ try {
573
+ host = new URL(target).hostname.toLowerCase();
574
+ } catch {
575
+ return ['chrome', 'firefox'];
576
+ }
577
+ const firefoxFirst = FIREFOX_FIRST_HOSTS.some((h) => host === h || host.endsWith(`.${h}`));
578
+ return firefoxFirst ? ['firefox', 'chrome'] : ['chrome', 'firefox'];
579
+ }
580
+
558
581
  /**
559
582
  * Fetch a page through impers, downgrading identity when one is refused.
560
583
  * Exported so the downgrade chain can be proven against a fake impers; the
@@ -565,14 +588,15 @@ function captureSetCookie(jar, url, res) {
565
588
  * @returns {Promise<{url: string, html: string, status: number, via: string}>}
566
589
  */
567
590
  export async function viaImpers(impers, target, jar) {
568
- // Some sites (Reddit) 403 the chrome fingerprint but accept firefox, so a
569
- // blocked first attempt gets one cheap retry with a second identity. An
591
+ // A blocked first attempt gets one cheap retry with the other identity. An
570
592
  // ImpersonateError is the same story one layer down: impers resolves the
571
593
  // 'chrome' alias to its newest fingerprint, but the native library it loads
572
594
  // can be an older system copy of libcurl-impersonate that predates that
573
595
  // fingerprint and refuses it before any request leaves. Firefox aliases to
574
596
  // an older target that such a library usually still knows, and when both
575
597
  // identities are refused the plain fetch transport still gets the page.
598
+ // Hosts that are known to refuse chrome outright start with firefox, so the
599
+ // usual case there costs one request instead of a 403 and a retry.
576
600
  const asking = (impersonate) => (url) =>
577
601
  impers.get(url, {
578
602
  impersonate,
@@ -590,11 +614,12 @@ export async function viaImpers(impers, target, jar) {
590
614
  return null;
591
615
  }
592
616
  };
593
- let via = 'impers:chrome';
594
- let got = await attempt('chrome');
617
+ const [first, second] = identityOrder(target);
618
+ let via = `impers:${first}`;
619
+ let got = await attempt(first);
595
620
  if (!got || got.status >= 400) {
596
- via = 'impers:firefox';
597
- got = (await attempt('firefox')) ?? got;
621
+ via = `impers:${second}`;
622
+ got = (await attempt(second)) ?? got;
598
623
  }
599
624
  if (!got) return viaFetch(target, jar);
600
625
  const { res, status } = got;