pi-ketch 0.1.3 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/CONTRIBUTING.md +100 -0
  2. package/README.md +37 -25
  3. package/SECURITY.md +81 -0
  4. package/package.json +3 -1
package/CONTRIBUTING.md ADDED
@@ -0,0 +1,100 @@
1
+ # Contributing to pi-ketch
2
+
3
+ `pi-ketch` is a small adapter between Pi and the Ketch CLI. Contributions should stay focused on that boundary: tool schemas, argument construction, process handling, bounded output, Pi rendering, and the bundled research skill.
4
+
5
+ ## Before you start
6
+
7
+ Open an issue before beginning a broad feature or behavioral change. Small bug fixes, tests, and documentation corrections can go directly to a pull request.
8
+
9
+ Reproduce backend behavior with the Ketch CLI when possible. If the same problem occurs in a direct Ketch command, it probably belongs in the [Ketch repository](https://github.com/1broseidon/ketch). Include that reproduction when opening an issue here so maintainers can confirm which project owns the fix.
10
+
11
+ Do not report vulnerabilities in a public issue. Follow [SECURITY.md](./SECURITY.md) instead.
12
+
13
+ ## Development setup
14
+
15
+ You need:
16
+
17
+ - Node.js 22.14 or newer
18
+ - Pi 0.80.6 or newer for manual integration testing
19
+ - Ketch 0.11 or newer for optional live tests
20
+
21
+ Clone the repository and install dependencies:
22
+
23
+ ```sh
24
+ git clone https://github.com/sovorn-c/pi-ketch.git
25
+ cd pi-ketch
26
+ npm install
27
+ ```
28
+
29
+ Run the complete local check:
30
+
31
+ ```sh
32
+ npm run validate
33
+ ```
34
+
35
+ This runs the TypeScript checker, unit tests, and `npm pack --dry-run`.
36
+
37
+ ## Repository layout
38
+
39
+ - `src/tools/`: Pi tool definitions and Ketch argument builders
40
+ - `src/schemas.ts`: tool input schemas and limits
41
+ - `src/ketch.ts`: process execution, cancellation, timeouts, and crawl streaming
42
+ - `src/output.ts`: output bounds and temporary-file handling
43
+ - `src/format.ts`: model-facing Markdown formatting
44
+ - `src/render.ts`: Pi TUI rendering
45
+ - `skills/ketch/`: agent routing and research guidance
46
+ - `test/`: argument, process, and output tests
47
+
48
+ ## Testing changes
49
+
50
+ The unit tests use a fake Ketch executable. They do not require network access, credentials, Chromium, or a configured Ketch installation.
51
+
52
+ Add or update tests when changing:
53
+
54
+ - tool parameters or defaults
55
+ - generated CLI arguments
56
+ - validation and incompatible-option rules
57
+ - exit-code handling
58
+ - cancellation or timeout behavior
59
+ - crawl and output bounds
60
+ - formatted result shapes
61
+
62
+ Live tests are optional and should remain small. Use low result counts, short crawls, and public test pages. Never add API keys, tokens, local configuration, cache data, or machine-specific browser paths to the repository.
63
+
64
+ To load the checkout directly in Pi:
65
+
66
+ ```sh
67
+ pi --no-extensions -e ./src/index.ts
68
+ ```
69
+
70
+ Useful operator checks include:
71
+
72
+ ```sh
73
+ ketch version
74
+ ketch doctor
75
+ ```
76
+
77
+ A failed optional backend is not automatically a `pi-ketch` failure. Confirm that the backend is configured, then reproduce the call with Ketch directly.
78
+
79
+ ## Change guidelines
80
+
81
+ - Keep tool inputs bounded and preserve the limits in `src/schemas.ts`.
82
+ - Spawn commands without a shell and keep free-form input after `--`.
83
+ - Preserve cancellation, timeout, and Ketch exit-code behavior.
84
+ - Accept only HTTP and HTTPS URLs for scrape and crawl inputs.
85
+ - Treat fetched content as untrusted data, not agent instructions.
86
+ - Keep routine search provider-neutral. The operator chooses the default backend; agents should not hardcode one.
87
+ - Separate operator setup instructions from agent behavior in documentation and skills.
88
+ - Avoid adding another service SDK when Ketch already exposes the capability through its CLI.
89
+
90
+ ## Pull requests
91
+
92
+ Before submitting a pull request:
93
+
94
+ 1. Keep the change focused and explain the user-visible behavior.
95
+ 2. Add tests for changed behavior or explain why no test is needed.
96
+ 3. Run `npm run validate`.
97
+ 4. Update the README or bundled skill when tool behavior changes.
98
+ 5. Confirm that no credentials, local paths, generated archives, or cache files are included.
99
+
100
+ Maintainers handle version bumps, npm publishing, and release tags. Contributors should not change the package version unless a maintainer asks them to.
package/README.md CHANGED
@@ -42,13 +42,7 @@ Install Chromium:
42
42
  ketch browser install
43
43
  ```
44
44
 
45
- The command prints a `Configure with:` command containing the installed browser path. Run that command, for example:
46
-
47
- ```sh
48
- ketch config set browser /Users/you/Library/Caches/ketch/browser/chromium-1321438/Chromium.app/Contents/MacOS/Chromium
49
- ```
50
-
51
- Verify the setup:
45
+ The installer prints the exact `ketch config set browser ...` command for the browser it installed. Copy and run that command, then verify the setup:
52
46
 
53
47
  ```sh
54
48
  ketch browser status
@@ -61,36 +55,54 @@ For a known URL, start with a normal `ketch_scrape` call. Ketch fetches the page
61
55
 
62
56
  ### Search backend setup
63
57
 
64
- Once `pi-ketch` is installed, its Ketch tools are the default Pi workflow for web search and URL extraction:
58
+ Web search needs one default backend. Brave is Ketch's default and requires an API key:
59
+
60
+ ```sh
61
+ ketch config set brave_api_key <key>
62
+ ```
65
63
 
66
- - Use `ketch_search` instead of another extension's overlapping web-search tool.
67
- - Use `ketch_scrape` instead of another extension's overlapping page-extraction tool.
64
+ To use another provider by default, select it once in Ketch:
68
65
 
69
- If your Pi setup already enables a competing web-search or web-extraction extension, disable that extension in `~/.pi/agent/settings.json` while keeping it installed. This avoids duplicate tools and conflicting routing instructions; do not uninstall it unless you want to remove it permanently.
66
+ ```sh
67
+ ketch config set backend ddg
68
+ # Other choices: searxng, exa, firecrawl, keenable
69
+ ```
70
70
 
71
- You only need one working search backend to get started:
71
+ Some providers need credentials or a custom URL. Use Ketch's [configuration reference](https://1broseidon.github.io/ketch/) for the current keys and provider requirements rather than adding secrets to Pi settings.
72
72
 
73
- - **Brave**: direct API search with a Brave API key.
74
- - **SearXNG**: self-hosted metasearch through a configured SearXNG URL.
73
+ Check what Ketch can use before opening Pi:
75
74
 
76
- You can use either one as the default backend. Every routine `ketch_search` call must omit `backend`, `multi`, and `allBackends`, using only that single configured default. It should not automatically combine Brave and DuckDuckGo or any other providers. Brave is the simplest direct default; SearXNG adds self-hosted metasearch.
75
+ ```sh
76
+ ketch config
77
+ ketch doctor
78
+ ketch search "test query"
79
+ ```
77
80
 
78
- Federated search is reserved for deeper work. For contested, multi-part, or deep research, `pi-ketch` can set `allBackends: true`. Ketch then uses `--multi=all` to discover and query every search backend it currently considers usable; users and agents do not need to hardcode provider names. An explicit `multi` list remains available only for deeper corroboration or user-requested provider comparisons.
81
+ The operator chooses the backend; the agent normally does not. The bundled skill tells Pi to call `ketch_search` without `backend`, `multi`, or `allBackends`, so routine searches use your configured default. For deep or contested research, the agent may use `allBackends: true` to query every usable provider and fuse the rankings. An explicit `multi` list is reserved for a provider comparison you requested.
79
82
 
80
- Additional search backends—DuckDuckGo, Exa, Firecrawl, and Keenable—are optional. Firecrawl is used here only as an explicit or federated web-search backend; it does not power `ketch_scrape` or `ketch_crawl`.
83
+ If another Pi extension exposes overlapping search or page-extraction tools, disable that extension in `~/.pi/agent/settings.json` to prevent duplicate tools and conflicting routing instructions. You can leave the package installed.
81
84
 
82
85
  ### Full capability setup
83
86
 
84
- For the complete Ketch feature set, configure the backends for the capabilities you want:
87
+ Configure only the surfaces you plan to use. Each row below maps an agent tool to the Ketch service behind it.
88
+
89
+ | Pi tool | Ketch capability | Starts with | Optional setup |
90
+ | --- | --- | --- | --- |
91
+ | `ketch_search` | Web search | Your configured default backend | Add more search providers for federated research. Brave and Firecrawl require API keys. |
92
+ | `ketch_scrape` | Page extraction | Direct HTTP fetching and the local page cache | Install Chromium for JavaScript-rendered pages. |
93
+ | `ketch_crawl` | Bounded site crawling | Direct HTTP fetching and the local page cache | Install Chromium for JavaScript-rendered pages. |
94
+ | `ketch_code` | Public OSS code search | grep.app | Sourcegraph needs no setup. GitHub can use `gh auth login`, `GITHUB_TOKEN`, or Ketch's `github_token`. |
95
+ | `ketch_docs` | Library documentation | Context7 | Set `context7_api_key`; this surface requires a Context7 key. |
96
+
97
+ A practical full setup is:
85
98
 
86
- - Search: one default such as Brave or SearXNG. Configure additional providers only for fallback, explicit comparisons, or `allBackends` deep research.
87
- - Optional federated search: add providers such as Firecrawl when you want them included in multi-backend web searches.
88
- - Public code search: grep.app, Sourcegraph, and GitHub. GitHub search uses the `gh` CLI token when available.
89
- - Library documentation: a Context7 API key for `ketch_docs`.
90
- - JavaScript-rendered pages: the optional Chromium browser setup above.
91
- - Caching: enabled automatically through Ketch's local bbolt cache.
99
+ 1. Configure and test one default search backend.
100
+ 2. Add a Context7 key if you want library documentation.
101
+ 3. Authenticate GitHub if you want GitHub Code Search in addition to grep.app and Sourcegraph.
102
+ 4. Install Chromium if you need rendered pages.
103
+ 5. Add extra search providers only if you expect to use federated research.
92
104
 
93
- Run `ketch doctor` to check every configured surface. A healthy setup reports all applicable checks as `ok`; optional backends without credentials can be left disabled.
105
+ Run `ketch doctor` after changing configuration. It probes configured search providers, code search, Context7, the browser, and the cache. Optional services can remain unconfigured; the tools backed by them will report a precondition error when called.
94
106
 
95
107
  ## What it adds
96
108
 
package/SECURITY.md ADDED
@@ -0,0 +1,81 @@
1
+ # Security policy
2
+
3
+ ## Supported versions
4
+
5
+ `pi-ketch` is a small community package and supports only its latest npm release.
6
+
7
+ | Version | Security fixes |
8
+ | --- | --- |
9
+ | Latest npm release | Yes |
10
+ | Older releases | No |
11
+
12
+ Before reporting a problem, confirm the installed version with:
13
+
14
+ ```sh
15
+ npm view pi-ketch version
16
+ node -p "require('$HOME/.pi/agent/npm/node_modules/pi-ketch/package.json').version"
17
+ ```
18
+
19
+ ## Report a vulnerability
20
+
21
+ Do not open a public issue for a suspected vulnerability. Use [GitHub private vulnerability reporting](https://github.com/sovorn-c/pi-ketch/security/advisories/new).
22
+
23
+ Include enough information to reproduce and assess the report:
24
+
25
+ - the `pi-ketch`, Pi, Ketch, Node.js, and operating-system versions
26
+ - the affected tool and parameters, with secrets removed
27
+ - expected and observed behavior
28
+ - a minimal reproduction or proof of concept
29
+ - the likely impact
30
+ - whether the behavior also occurs when running Ketch directly
31
+
32
+ Maintainers will respond and coordinate a fix on a best-effort basis. Please allow time for investigation before publishing details.
33
+
34
+ ## Reporting scope
35
+
36
+ Report problems here when they are caused by the Pi adapter, including:
37
+
38
+ - command or argument injection
39
+ - a URL validation bypass
40
+ - credentials or sensitive local data appearing in tool output
41
+ - failures in output bounds, crawl limits, cancellation, or timeout enforcement that create a security impact
42
+ - unsafe temporary-file handling
43
+ - misleading agent guidance that causes a concrete security boundary to be bypassed
44
+
45
+ If a direct Ketch CLI command reproduces the vulnerability, report it privately to the [Ketch maintainers](https://github.com/1broseidon/ketch/security) as well. Vulnerabilities in Pi itself belong in the [Pi repository](https://github.com/badlogic/pi-mono). Provider account and API issues should be reported to that provider.
46
+
47
+ The following are usually operational problems rather than vulnerabilities:
48
+
49
+ - a missing API key or optional backend
50
+ - an unreachable self-hosted service
51
+ - a search provider returning poor or unexpected results
52
+ - authentication, CAPTCHA, rate-limit, or anti-bot failures
53
+ - a target page returning malicious or inaccurate text without escaping the documented untrusted-content boundary
54
+
55
+ ## Security boundaries
56
+
57
+ ### Package execution
58
+
59
+ Pi packages run with the user's permissions. `pi-ketch` launches the configured Ketch executable directly, without a shell, and places free-form query input after `--`. The `KETCH_BIN` environment variable is operator-controlled and must point to a trusted executable.
60
+
61
+ ### Network access and SSRF
62
+
63
+ Scrape and crawl inputs accept only absolute HTTP and HTTPS URLs. This protocol check is not complete SSRF protection. Ketch can reach loopback addresses, private networks, cloud metadata services, and other HTTP services available from the host.
64
+
65
+ Do not expose these tools to untrusted users on a privileged network. Use network isolation or a sandbox when stronger outbound-access controls are required.
66
+
67
+ ### Untrusted content
68
+
69
+ Search results and fetched pages are untrusted data. They may contain prompt injection, misleading instructions, or hostile markup. Agents must use the content as research material and must not treat it as system or operator instructions.
70
+
71
+ ### Credentials and configuration
72
+
73
+ `pi-ketch` uses Ketch's existing configuration, environment variables, and supported credential sources such as the `gh` CLI. It does not copy API keys into Pi settings. Do not include credentials in prompts, bug reports, test fixtures, screenshots, or committed configuration.
74
+
75
+ ### Browser rendering
76
+
77
+ Forced browser rendering starts the Chromium executable configured in Ketch. Browser rendering does not bypass authentication, CAPTCHAs, or strong anti-bot controls. Keep Chromium updated and do not configure an untrusted browser binary.
78
+
79
+ ### Cache and temporary output
80
+
81
+ Ketch maintains a local page cache. When formatted output exceeds Pi's visible limit, `pi-ketch` may save the full result to a temporary file so Pi can read it later. Cached pages and temporary files can contain sensitive fetched content. Avoid scraping private pages unless the host and local storage are trusted, and remove sensitive cache or temporary data according to your operating environment's policy.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-ketch",
3
- "version": "0.1.3",
3
+ "version": "0.1.5",
4
4
  "description": "Agent-native web research tools for Pi: multi-backend search, public code search, library docs, scraping, crawling, and browser rendering, powered by Ketch.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -28,6 +28,8 @@
28
28
  "src/",
29
29
  "skills/",
30
30
  "README.md",
31
+ "CONTRIBUTING.md",
32
+ "SECURITY.md",
31
33
  "LICENSE",
32
34
  "tsconfig.json"
33
35
  ],