sf-docs-agent 0.1.0 → 0.1.1

package/README.md

@@ -1,5 +1,7 @@
 # sf-docs-agent
 
+[![npm](https://img.shields.io/npm/v/sf-docs-agent)](https://www.npmjs.com/package/sf-docs-agent)
+
 A Claude Code subagent that answers Salesforce questions from the official docs and hands back quotes, not pages.
 
 It reads developer.salesforce.com, architect.salesforce.com, help.salesforce.com, admin.salesforce.com and
@@ -30,29 +32,31 @@ Here is how the two compare:
 | Checking an answer | re-read the page | `grep` one line |
 
 The trade-off: your main session trusts the subagent. That is why the model only picks line numbers and the script
-copies the lines. Every quote is the page's own text, never the model's memory, and you can check it with one `grep`.
+copies the lines. Every quote is copied word for word from the page, not written from the model's memory, and you can
+check it with one `grep` against the cached copy, which saves curly quotes as straight ones and drops image links and
+the padding in table rows.
 
-## 86% less in your context, 64% fewer tokens overall
+## 84% less in your context, 63% fewer tokens overall
 
 On 10 questions against Salesforce's own platform-docs-get skill, the agent
-added 86% fewer tokens to your context and used 64% fewer tokens over the whole session, with the right answer on
+added 84% fewer tokens to your context and used 63% fewer tokens over the whole session, with the right answer on
 10 of 10.
 
 Below are the measurements, each read as platform-docs-get → sf-docs.
 
 | Question | Site | Tokens added to context | Saved | Session token usage | Saved |
 |---|---|---|---|---|---|
-| 1. Scratch org editions | developer | 906 → **242** | **73.3%** | 236,882 → **86,172** | **63.6%** |
-| 2. Platform event allocation | developer | 3,120 → **471** | **84.9%** | 387,764 → **126,998** | **67.2%** |
-| 3. QueryLocator batch limits | developer | 692 → **458** | **33.8%** | 287,129 → **105,756** | **63.2%** |
-| 4. Integration patterns | architect | 1,223 → **528** | **56.8%** | 154,149 → **102,635** | **33.4%** |
-| 5. Salesforce Connect editions | help | 5,591 → **649** | **88.4%** | 477,647 → **311,396** | **34.8%** |
-| 6. Flow version limits | help | 5,940 → **293** | **95.1%** | 354,017 → **171,752** | **51.5%** |
-| 7. Headless 360 | admin | 4,247 → **341** | **92.0%** | 675,859 → **121,840** | **82.0%** |
-| 8. SLDS tooltips | lightningdesignsystem | 5,983 → **463** | **92.3%** | 360,244 → **112,618** | **68.7%** |
-| 9. Bulk API batch size | developer | 2,328 → **476** | **79.6%** | 454,224 → **91,811** | **79.8%** |
-| 10. Pub/Sub fetch limit | developer | 2,212 → **625** | **71.7%** | 448,205 → **141,343** | **68.5%** |
-| **All** | | 32,242 → **4,546** | **85.9%** | 3,836,120 → **1,372,321** | **64.2%** |
+| 1. Scratch org editions | developer | 2,178 → **289** | **86.7%** | 338,852 → **119,355** | **64.8%** |
+| 2. Platform event allocation | developer | 4,107 → **490** | **88.1%** | 387,598 → **120,396** | **68.9%** |
+| 3. QueryLocator batch limits | developer | 2,568 → **581** | **77.4%** | 406,878 → **113,611** | **72.1%** |
+| 4. Integration patterns | architect | 1,104 → **484** | **56.2%** | 154,068 → **125,540** | **18.5%** |
+| 5. Salesforce Connect editions | help | 1,948 → **322** | **83.5%** | 382,751 → **157,746** | **58.8%** |
+| 6. Flow version limits | help | 4,578 → **290** | **93.7%** | 339,464 → **164,888** | **51.4%** |
+| 7. Headless 360 | admin | 3,772 → **380** | **89.9%** | 427,542 → **117,258** | **72.6%** |
+| 8. SLDS tooltips | lightningdesignsystem | 3,876 → **414** | **89.3%** | 340,914 → **112,252** | **67.1%** |
+| 9. Bulk API batch size | developer | 2,134 → **411** | **80.7%** | 506,340 → **125,870** | **75.1%** |
+| 10. Pub/Sub fetch limit | developer | 1,199 → **716** | **40.3%** | 236,766 → **130,110** | **45.0%** |
+| **All** | | 27,464 → **4,377** | **84.1%** | 3,521,173 → **1,287,026** | **63.4%** |
 
 <details>
 <summary>The 10 questions</summary>
@@ -256,24 +260,30 @@ official URLs, and the session turned them into the answer.
 
 ## Install
 
-User scope:
+Needs Node 22+ and Claude Code. The installer is on npm as
+[`sf-docs-agent`](https://www.npmjs.com/package/sf-docs-agent).
+
 ```
 npx sf-docs-agent            # for all your projects
-```
-Project scope:
-```
 npx sf-docs-agent --project  # for the project in the current folder only
 ```
 
-Needs Node 18.17+ and Claude Code. The installer adds `playwright-core` (8.5 MB). It also needs a headless Chromium
-(about 180 MB). On Linux, also run `npx playwright-core install-deps chromium` once.
+The installer adds `playwright-core` (13 MB) and asks before it downloads headless Chromium (about 200 MB). Pass
+`--yes` to skip the question in scripts. If you say no, it prints the command to run before first use. On Linux, also
+run `node "${CLAUDE_CONFIG_DIR:-$HOME/.claude}/sf-docs/node_modules/playwright-core/cli.js" install-deps chromium` once
+for the system libraries. It asks for your password itself.
+
+Then start a new Claude Code session. The agent is listed as `sf-docs`.
+
+**Update:** `npx sf-docs-agent@latest`. Without `@latest`, npx runs any copy already installed globally or in the
+current project.
 
-To update, run it again. To remove it, delete `~/.claude/agents/sf-docs.md` (or `.claude/agents/sf-docs.md`
-in the project) and `~/.claude/sf-docs/`. If `CLAUDE_CONFIG_DIR` is set, the installer uses that instead of
-`~/.claude`.
+**Remove:** delete `~/.claude/agents/sf-docs.md` (or `.claude/agents/sf-docs.md` in the project). The script and cache
+live in `~/.claude/sf-docs/` for every scope, so delete that folder last, once no project uses the agent. If
+`CLAUDE_CONFIG_DIR` is set, the installer uses that instead of `~/.claude`.
 
-Pages stay cached for 24 hours, so a follow-up question does not fetch them again. To change that, set `SF_DOCS_CACHE_HOURS` in your Claude Code `settings.json`
-(`0` fetches every time):
+**Cache:** pages stay cached for 24 hours, so a follow-up question does not fetch them again. To change that, set
+`SF_DOCS_CACHE_HOURS` in your Claude Code `settings.json` (`0` fetches every time):
 
 ```json
 { "env": { "SF_DOCS_CACHE_HOURS": "72" } }
@@ -289,6 +299,8 @@ Pages stay cached for 24 hours, so a follow-up question does not fetch them agai
 - No Help search. Help's `robots.txt` disallows it. Help's sitemaps list article ids but not titles.
 - It reads the current docs, which don't always name their release.
 - It follows at most one link from a page it read. An answer that sits further away comes back as a gap.
+- On a long list or table, the agent now and then stops a row early, for example five of six integration patterns.
+  Every quote it gives is still right; the missing row is the problem.
 - If a site starts blocking headless browsers, the agent stops and reports it. It does not try to get around the block.
 - What leaves your machine: the pages the agent fetches from those sites (and each site's `robots.txt`, which raw
   fetches load first). No telemetry. Installing downloads `playwright-core` from npm and, if you agree, Chromium from

package/agents/sf-docs.md

@@ -17,7 +17,7 @@ you do not fill gaps from memory. If the docs do not say it, the answer is "not
 | developer.salesforce.com, new docs (`/docs/platform/...` and other paths in a per-product index) | `.md` twin, fetched raw | `developer.salesforce.com/docs/llms.txt` lists one `/docs/llms-<product>.txt` per product (see Find for the two that terms can miss) |
 | developer.salesforce.com, atlas docs (`/docs/atlas.en-us.*`: Apex Developer Guide, Metadata API, Object Reference and more) | `.htm` page, rendered. There is no `.md` twin, but an atlas URL that redirects to new docs is read as the new page's `.md` (`[twin ...]` in the output) | `developer.salesforce.com/docs/llms-product-docs.txt` (3 MB, 19,000+ `.htm` links) |
 | architect.salesforce.com | `.md`, fetched raw | `https://architect.salesforce.com/llms.txt` |
-| help.salesforce.com | rendered, about 10 s a page | `'https://help.salesforce.com/s/articleView?id=<id>.htm&type=5'`. With an id from the brief, a doc, or a fetched Help page's `.links.txt` (`/apex/HTViewHelpDoc?id=<id>.htm` lines; knowledge articles are `?id=<number>&type=1`), fetch it. With no id, use the sitemaps: fetch `'https://help.salesforce.com/apex/Help_SiteMapIndexExternal?producttype=<platform|service|xcloud|sales|ai|data|experience|analytics|mktg|commerce|cc|ind|sfdo|slack|release-notes>'` with `--terms`; an index prints every page sitemap it lists, whatever the terms (46 for platform). Fetch those 20 a call with `--terms '<word_word>'`. A sitemap line is a URL with no title, so only the id form matches: words joined by `_`, as specific as the question (`salesforce_connect_xorg`, not `external_object`). Each match prints the article's `<loc>` line, up to 40 a page. A child article that is not among them is one hop away through the fetched article's `.links.txt`. Do not use Help search: robots.txt disallows `/s/search-result` |
+| help.salesforce.com | rendered, about 10 s a page | `'https://help.salesforce.com/s/articleView?id=<id>.htm&type=5'`. With an id from the brief, a doc, or a fetched Help page's `.links.txt` (`/s/articleView?id=<id>.htm&language=en_US&type=5` lines; knowledge articles are `?id=<number>&type=1`), fetch it. With no id, use the sitemaps: fetch `'https://help.salesforce.com/apex/Help_SiteMapIndexExternal?producttype=<platform|service|xcloud|sales|ai|data|experience|analytics|mktg|commerce|cc|ind|sfdo|slack|release-notes>'` with `--terms`; an index prints every page sitemap it lists, whatever the terms (46 for platform). Fetch those 20 a call with `--terms '<word_word>'`. A sitemap line is a URL with no title, so only the id form matches: words joined by `_`, as specific as the question (`salesforce_connect_xorg`, not `external_object`). Each match prints the article's `<loc>` line, up to 40 a page. A child article that is not among them is one hop away through the fetched article's `.links.txt`. Do not use Help search: robots.txt disallows `/s/search-result` |
 | admin.salesforce.com | rendered | `admin.salesforce.com/sitemap_index.xml` lists the post sitemaps (`/post-sitemap.xml`, `/post-sitemap2.xml` ...); fetch them all in one call with `--terms` and use the matching `<loc>` URLs. `/blog` shows only the latest 3 posts |
 | lightningdesignsystem.com | `.md` twin, fetched raw (the home page itself is rendered) | the `.links.txt` of the home page, as `/2e1ef8501/p/<id>-<slug>` |
 | v1.lightningdesignsystem.com (SLDS 1, still used by orgs not on SLDS 2) | rendered | the `.links.txt` of `https://v1.lightningdesignsystem.com/`, as `/components/<name>/` and `/utilities/<name>/` |
@@ -85,7 +85,9 @@ Every tool call costs you the whole conversation again, so use as few Bash calls
    next page.
 3. **Look.** To see the lines around a match, use Read with `offset` and `limit`, or `grep -n -C3 -e '<term>' <file>`:
    both number lines as `--quote` counts them. `sed -n` and `head` do not, so never take a line number from them.
-   Never read a page whole.
+   Never read a page whole. When the answer is a list or a table (patterns, editions, values), Read from its first
+   item to the next heading before you quote: a `--terms` match list stops at its cap, and the last rows may not
+   hold your term.
 4. **One hop.** If a matched line, a `.links.txt` match or a sentence you will quote names another page that holds
    part of the answer (a child article, a "Limitations for ..." heading with no text under it, an index entry whose
    title matches the question, another post), fetch it now with `--terms`. One hop only: a page named by that page
@@ -93,7 +95,8 @@ Every tool call costs you the whole conversation again, so use as few Bash calls
 5. **Quote.** Pick the line numbers that carry the answer, from the matches or the Read. Then let the script copy them:
    `node "/path/to/sf-docs-agent/fetch-docs.mjs" --quote <cache file>:<line>,<line> [<cache file>:<line> ...]`
    (cache file relative to the cache dir or absolute). It prints the Evidence and Sources blocks verbatim, with the
-   official URL from each file's first line. Never type a quote yourself.
+   official URL from each file's first line. Never type a quote yourself, and paste the script's output unchanged:
+   not one character, apostrophes included.
 
 ## Hard limits
 

package/bin/install.mjs

@@ -47,7 +47,7 @@ const ask = async q => {
   return /^y(es)?$/i.test(a.trim());
 };
 if (present) console.log('Headless Chromium is already installed.');
-else if (process.argv.includes('--yes') || await ask('Download headless Chromium (about 180 MB)? [y/N] ')) {
+else if (process.argv.includes('--yes') || await ask('Download headless Chromium (about 200 MB)? [y/N] ')) {
   try { execFileSync(process.execPath, [cli, 'install', '--only-shell', 'chromium'], { stdio: 'inherit' }); }
   catch { console.error(`\nBrowser download failed (see above). Retry later with:\n  node "${cli}" install --only-shell chromium`); }
 } else console.log(`Skipped the browser download. Before first use, run:\n  node "${cli}" install --only-shell chromium`);

package/fetch-docs.mjs

@@ -23,8 +23,10 @@ if (oi >= 0) out = args.splice(oi, 2)[1];
 // No page starts after the budget: each one left is printed 'not fetched (time limit)', so the caller can rerun it.
 const bi = args.indexOf('--budget');
 const deadline = bi >= 0 ? Date.now() + Number(args.splice(bi, 2)[1]) * 1000 : Infinity;
+// Curly quotes are saved straight: a model that retypes a quote writes ' and ", and the grep check must still match.
+const plain = text => text.replace(/[\u2018\u2019]/g, "'").replace(/[\u201C\u201D]/g, '"');
 // --terms a,b: after the fetch, print each saved page's lines that hold a term, so the caller needs no grep.
-const termsOf = v => (v ?? '').split(',').map(s => s.trim().toLowerCase()).filter(Boolean);
+const termsOf = v => plain(v ?? '').split(',').map(s => s.trim().toLowerCase()).filter(Boolean); // the cache holds straight quotes
 const ti = args.indexOf('--terms');
 const terms = ti >= 0 ? termsOf(args.splice(ti, 2)[1]) : [];
 if (!args.length || !out || Number.isNaN(deadline) || (ti >= 0 && !terms.length)) { console.error('usage: node fetch-docs.mjs [--out dir] [--budget seconds] [--terms a,b] <url | /docs/path.md> ...\n       node fetch-docs.mjs [--out dir] --search a,b [--site <host>]\n       node fetch-docs.mjs [--out dir] --quote <cache file>:<line>[,<line>...] ...'); process.exit(2); }
@@ -82,7 +84,8 @@ const slash = p => p.split(sep).join('/');
 // page order. Caps set on a clean 20-question sample (looser caps printed twice the old grep's 2k chars a call).
 const MAX_HITS = 20, MAX_LINKS = 10, SHORT_LINES = 40, SHORT_CHARS = 4000, PER_FILE = 3;
 const matches = (name, text, t) => text.split('\n').flatMap((line, i, ls) => {
-  const low = i || !ls[0].startsWith('Source: ') ? line.toLowerCase() : '', hit = t.filter(s => low.includes(s)); // a page's line 1 is its Source line; a sidecar has none
+  // plain(): a sidecar and a page cached before quotes were saved straight still hold curly ones.
+  const low = i || !ls[0].startsWith('Source: ') ? plain(line).toLowerCase() : '', hit = t.filter(s => low.includes(s)); // a page's line 1 is its Source line; a sidecar has none
   return hit.length ? [{ name, n: i + 1, line, hit, at: Math.min(...hit.map(s => low.indexOf(s))) }] : [];
 });
 const rank = (ms, max = MAX_HITS) => {
@@ -215,8 +218,11 @@ function extract([sels, fallback]) {
   for (let i = 0; i < roots.length; i++) for (const el of roots[i].querySelectorAll('*')) if (el.shadowRoot) roots.push(el.shadowRoot);
   // Same-site links (nav and TOC included) so pages without an llms.txt index can still be discovered.
   const links = new Set();
+  // Path and query: every Help article sits at /s/articleView and differs only by ?id=, and it links itself with
+  // &language= added, so a link is the page itself when its id matches.
+  const self = u => u.pathname + (new URLSearchParams(u.search).get('id') ?? u.search), here = self(location);
   for (const r of roots) for (const a of r.querySelectorAll('a[href]')) {
-    if (a.host === location.host && a.pathname !== location.pathname) links.add(`${a.pathname}${a.search} ${(a.innerText || '').trim().replace(/\s+/g, ' ').slice(0, 80)}`);
+    if (a.host === location.host && self(a) !== here) links.add(`${a.pathname}${a.search} ${(a.innerText || '').trim().replace(/\s+/g, ' ').slice(0, 80)}`);
   }
   const page = t => `# ${document.title}\n\n${t}\n`;
   const list = [...links].join('\n');
@@ -321,7 +327,7 @@ const emit = r => {
   // Markdown only (a .md URL, or a twin read as r.md): rendered text has no code fences, so lean could reach a code
   // sample. Cached pages are lean already.
   const md = r.md || /\.md$/.test(new URL(key).pathname);
-  const body = status === 200 && !cached && md ? lean(r.body) : r.body;
+  const body = status === 200 && !cached ? plain(md ? lean(r.body) : r.body) : r.body;
   const h = createHash('sha1').update(body).digest('hex').slice(0, 12);
   const u = new URL(key);
   const file = fileFor(u);
@@ -367,7 +373,7 @@ if (!urls.every(isFresh) && Date.now() < deadline) { // a spent budget skips eve
     // The exact command, not "reinstall": a reinstall without a terminal skips the browser again.
     const cli = slash(join(dirname(fileURLToPath(import.meta.url)), 'node_modules/playwright-core/cli.js'));
     console.error(`Install the browser with: node "${cli}" install --only-shell chromium`);
-    console.error('On Linux, also run once: npx playwright-core install-deps chromium');
+    console.error(`On Linux, also run once: node "${cli}" install-deps chromium`);
     process.exit(3);
   }
 }

package/package.json

@@ -1,10 +1,10 @@
 {
   "name": "sf-docs-agent",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "type": "module",
   "description": "Claude Code subagent that answers from official Salesforce docs (developer, architect, help, admin, lightningdesignsystem.com) with grep-checked quotes",
   "devDependencies": {
-    "playwright-core": "1.56.1"
+    "playwright-core": "1.64.1"
   },
   "bin": {
     "sf-docs-agent": "bin/install.mjs"
@@ -20,7 +20,7 @@
   },
   "license": "MIT",
   "engines": {
-    "node": ">=18.17"
+    "node": ">=22"
   },
   "keywords": [
     "claude-code",