pi-twitterapi.io 0.1.1

package/CHANGELOG.md

@@ -0,0 +1,51 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.1] - 2026-10-03
+
+First version published to npm.
+
+### Added
+
+- `twitter` tool backed by the [twitterapi.io](https://twitterapi.io) REST API, with 17 read modes:
+  `posts` (default), `users`, `thread`, `user`, `trends`, `replies`, `quotes`, `mentions`, `followers`,
+  `followings`, `profile`, `about`, `tweets`, `retweeters`, `community`, `list` and `space`.
+- Answer synthesis with a pi model: `twitter.synthesisModel` when configured, otherwise the model running
+  the current session, with a per-failure-kind fallback chain back to the session model.
+- Citations derived from fetched permalinks only; invented X links are dropped and counted in the notes.
+- Best-effort media understanding (`enableImageUnderstanding`, `enableVideoUnderstanding` with poster
+  frames), bounded paging with early-stop disclosure, and per-mode parameter validation.
+- Strict settings parsing for the `twitter` block in `~/.pi/agent/settings.json` and
+  `<cwd>/.pi/settings.json`.
+
+### Fixed
+
+- README no longer claims a peer range of `>=0.99.2 <2`; the host packages are declared as peers with
+  `"*"`, and the `ModelRegistry.complete` requirement is documented separately from that range.
+- The release workflow decides publishing and GitHub-release creation separately, so a rerun after a
+  failed release step skips the immutable npm version instead of failing on it and repairs the release.
+  A registry lookup that fails for any reason other than "not found" now fails the run loudly.
+- The `mode`, `queryType` and `replySort` tool parameters are closed literal unions rather than free-form
+  strings, and the tests assert both the accepted and rejected values.
+
+### Changed
+
+- Package layout: the pi manifest loads `./src/index.ts` directly and the tarball ships the TypeScript
+  sources (tests excluded), so there is no build step; `tsdown` and `dist/` are gone.
+- Release workflow actions are pinned to commit SHAs, checkouts do not persist credentials, `ci.yml` runs
+  with `contents: read`, and the publishing job does not use a package-manager cache.
+
+## [0.1.0] - 2026-10-03
+
+Tagged but never published. Superseded by [0.1.1](#011---2026-10-03), which carries the same features
+plus the fixes above; the entry is kept because the git tag exists.
+
+[Unreleased]: https://github.com/jagaliano/pi-twitterapi.io/compare/v0.1.1...HEAD
+[0.1.1]: https://github.com/jagaliano/pi-twitterapi.io/compare/v0.1.0...v0.1.1
+[0.1.0]: https://github.com/jagaliano/pi-twitterapi.io/releases/tag/v0.1.0

package/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2026 José Antonio Galiano Sandoval (pi-twitterapi.io)
+Copyright (c) 2026 anthod0 (upstream: https://github.com/anthod0/pi-lab)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,278 @@
+# pi-twitterapi.io
+
+A [twitterapi.io](https://twitterapi.io)-backed X/Twitter extension for the
+[pi coding agent](https://pi.dev). It registers a `twitter` tool that reads
+posts, accounts, threads, timelines, trends, conversations and more through
+twitterapi.io, then synthesizes an answer with citation URLs using a model from
+pi's own registry.
+
+This extension is self-contained: twitterapi.io is the only retrieval source and
+pi's model registry performs the synthesis. It is designed to cover the same
+surface as xAI's official `x_search` tool and, where twitterapi.io exposes more,
+to go beyond it — see [Compared with xAI `x_search`](#compared-with-xai-x_search).
+
+## Install
+
+```bash
+pi install npm:pi-twitterapi.io
+```
+
+Or load the local source directly (pi compiles TypeScript extensions on the
+fly, so there is no build step):
+
+```bash
+pi -e ./src/index.ts
+```
+
+## Configure
+
+Set the twitterapi.io API key. Synthesis uses `twitter.synthesisModel` when set, and otherwise falls back to the model running the current pi session, so a key-only setup already works:
+
+```bash
+export TWITTERAPI_IO_API_KEY="your-twitterapi.io-key"
+```
+
+The settings file is strict JSON (no comments). Add the `twitter` block to
+`~/.pi/agent/settings.json` — or `<cwd>/.pi/settings.json` for a project-scoped
+override — and set only the keys you need:
+
+```json
+{
+  "twitter": {
+    "synthesisModel": "anthropic/claude-haiku-4-5-20251001",
+    "enableImageUnderstanding": false,
+    "enableVideoUnderstanding": false,
+    "maxMediaPerSearch": 4,
+    "maxPages": 5,
+    "maxPagesCeiling": 20,
+    "minRequestIntervalMs": 5000,
+    "retryBaseDelayMs": 5000
+  }
+}
+```
+
+| Key | Required | Meaning |
+|---|---|---|
+| `synthesisModel` | no | A pi model id (`provider/model`) used to turn retrieved posts into an answer. When unset, the session model is used; if it is set but fails at runtime, the session model answers instead with a note. |
+| `enableImageUnderstanding` | no | Attach post images to the synthesis request when the model accepts image input. |
+| `enableVideoUnderstanding` | no | Attach video poster frames (chat models cannot ingest video). |
+| `maxMediaPerSearch` | no | Upper bound on media attachments per search (max 20, default 4). |
+| `maxPages` | no | Base page budget per search (default 5). |
+| `maxPagesCeiling` | no | Hard cap that `maxPages` is clamped to (default 20). |
+| `minRequestIntervalMs` | no | Minimum spacing between upstream requests (default 5000). twitterapi.io allows 0.2 QPS on unpaid accounts; raise it if you are being throttled, lower it for a higher-QPS tier, or set it to 0 to disable pacing. |
+| `retryBaseDelayMs` | no | Base delay for retry backoff (default 5000). |
+
+> **Important:** the extension needs a pi version whose `ModelRegistry.complete`
+> exists — it is absent on pi 0.80.6, present from pi 0.99.2, and verified on
+> pi 1.0.2. The host packages are declared as peers with a `"*"` range because pi
+> supplies them at runtime, so that range does not enforce the capability: check
+> your pi version if you are on an old release. Synthesis is a real model call, so
+> it consumes tokens on the configured model.
+
+## The `twitter` tool
+
+| Parameter | Type | Notes |
+|---|---|---|
+| `query` | string (required) | Natural-language question. Required for every mode. |
+| `mode` | `posts` \| `users` \| `thread` \| `user` \| `trends` \| `replies` \| `quotes` \| `mentions` \| `followers` \| `followings` \| `profile` \| `about` \| `tweets` \| `retweeters` \| `community` \| `list` \| `space` | Defaults to `posts`. |
+| `tweet` | string | Post id or X permalink. Required for `thread`, `replies`, `quotes`, `retweeters`; refused in any other mode. |
+| `user` | string | Handle (no `@`) for `mode=user`, `mentions`, `followers`, `followings`, `profile`, `about`. |
+| `userId` | string | Numeric user id for `mode=user`; preferred over `user` when known. |
+| `ids` | string[] | `mode=tweets`: post ids or permalinks to fetch (max 100). |
+| `pageSize` | number | `mode=followers`/`followings`: accounts per page (20–200). |
+| `communityId` | string | `mode=community`: the community id. |
+| `listId` | string | `mode=list`: the list id. |
+| `spaceId` | string | `mode=space`: the X Space id. |
+| `woeid` | number | `mode=trends` location id (1=Worldwide, 23424977=USA). |
+| `includeReplies` | boolean | `mode=user` (timeline) and `mode=quotes`. |
+| `sinceTime` / `untilTime` | number | `mode=quotes` and `mode=mentions`: unix timestamps (seconds) bounding the results. |
+| `limit` | number | `mode=user`/`mentions`/`followers`/`followings`/`replies`/`quotes`/`retweeters`/`community`/`list`: stop after this many items (max 1000). |
+| `replySort` | `"Relevance"` \| `"Latest"` \| `"Likes"` | `mode=replies` sort order (default `Relevance`). |
+| `allowed_x_handles` | string[] | `mode=posts`: only these handles (max 20, no `@`). |
+| `excluded_x_handles` | string[] | `mode=posts`: exclude these handles (max 20, no `@`). |
+| `from_date` / `to_date` | `YYYY-MM-DD` | `mode=posts` date range. |
+| `queryType` | `"Latest"` \| `"Top"` | `mode=posts`: newest-first (default) or ranked. |
+| `count` | number | `mode=posts`/`users`: max items (posts default 10, accounts default 20; max 50). `mode=trends`: number of trends (min 30). |
+
+Parameters that cannot apply in a given mode are rejected with an error rather
+than silently ignored.
+
+### Modes
+
+| Mode | Endpoint | What it reads |
+|---|---|---|
+| `posts` (default) | `/twitter/tweet/advanced_search` | Keyword/operator post search; handle filters, dates, `Latest`/`Top`, item count. Optionally attaches media. |
+| `users` | `/twitter/user/search` | Account discovery; profile URLs as sources. |
+| `thread` | `/twitter/tweet/thread_context` | A referenced post's whole thread context. |
+| `user` | `/twitter/user/last_tweets` | A specific account's most recent posts (`user`/`userId`, `includeReplies`, `limit`). |
+| `trends` | `/twitter/trends` | Trending topics for a `woeid`; sources are X search URLs. |
+| `replies` | `/twitter/tweet/replies/v2` | Replies to a post (`replySort`, `limit`). |
+| `quotes` | `/twitter/tweet/quotes` | Quote-posts of a post (`sinceTime`/`untilTime`, `includeReplies`, `limit`). |
+| `mentions` | `/twitter/user/mentions` | Posts mentioning an account (`user`, `sinceTime`/`untilTime`, `limit`). |
+| `followers` | `/twitter/user/followers` | Who follows an account (`user`, `pageSize`, `limit`); profile URLs as sources. |
+| `followings` | `/twitter/user/followings` | Who an account follows (`user`, `pageSize`, `limit`). |
+| `profile` | `/twitter/user/info` | A single account profile (`user`). |
+| `about` | `/twitter/user_about` | Extended profile-page metadata (`user`): account-based-in, creation source, handle changes, identity verification; cited as the profile URL. |
+| `tweets` | `/twitter/tweets` | Specific posts by id (`ids`, max 100). |
+| `retweeters` | `/twitter/tweet/retweeters` | Accounts that reposted a post (`tweet`, `limit`); profile URLs as sources. |
+| `community` | `/twitter/community/tweets` | Posts from a community (`communityId`, `limit`). |
+| `list` | `/twitter/list/tweets_timeline` | Posts from a list (`listId`, `limit`). |
+| `space` | `/twitter/spaces/detail` | An X Space's detail (`spaceId`); cited as `https://x.com/i/spaces/<id>`. Note: twitterapi.io currently returns HTTP 404 for this endpoint even for Spaces that are live in the X app, so the mode often reports the upstream error verbatim. |
+
+## Behavior and disclosures
+
+- **Citations are derived from fetched permalinks, never from model output.**
+  An X/Twitter link the model invents that was not among the retrieved posts is
+  dropped from `Sources` and counted in the `## Notes` section. Non-X links are
+  outside the citation contract: they are neither published as sources nor
+  counted as invented citations.
+- **Media is best-effort.** Video cannot be sent to a chat model, so video posts
+  are represented by their poster frame and this limitation is disclosed.
+- **Partial retrieval is disclosed.** If paging stops early (page cap, cursor
+  cycle, or a missing cursor while more results remain), the answer carries a
+  note saying the results may be incomplete.
+- **`mode=space` depends on a flaky upstream.** twitterapi.io's
+  `/twitter/spaces/detail` returned HTTP 404 "Space not found or API error" for
+  a Space that X itself displayed as live, so this mode frequently reports the
+  upstream error rather than a summary.
+- **Date windows are resolved at 04:00 UTC by twitterapi.io**, so posts outside
+  the requested local window are trimmed while paging, with a note when that
+  happens. Two consequences worth knowing: far-west offsets (for example
+  UTC-8/-10) can lose the last few hours of the requested day because the
+  upstream window ends before local midnight (disclosed as a note), and far-east
+  offsets (UTC+10 and beyond) can spend free paging on the newer trim band before
+  results begin. Start padding is deliberately conservative (one extra hour) so a
+  winter boundary shift cannot silently drop the first hour of the local day.
+- **Images and the fallback.** Media is attached only when the model chosen to
+  synthesize accepts image input; if that model fails and a different model
+  answers, images are omitted rather than sent to a model that cannot read them,
+  and the answer says so.
+- **Synthesis fallback.** If `twitter.synthesisModel` fails at runtime, the
+  answer is retried with the model running the current session and the result
+  carries a note naming the model that answered. Failures are classified so the
+  chain reacts per kind rather than treating every error alike:
+
+  | Failure | Examples | Reaction |
+  |---|---|---|
+  | Quota / billing | `402`, `insufficient_quota`, quota exceeded, subscription limit | next model |
+  | Authentication | `401`, `403`, invalid API key | next model |
+  | Unknown model | `404`, "does not exist" | next model |
+  | Invalid request | `400`, `422`, malformed | next model |
+  | Rate limit | `429`, "too many requests", overloaded | next model, retried once when it is the last one |
+  | Server | `5xx`, bad gateway, unavailable | next model, retried once when it is the last one |
+  | Transport | timeouts, connection drops, premature stream endings | next model, retried once when it is the last one |
+  | Empty response | model returned no usable text | next model, retried once when it is the last one |
+  | Unclassified | anything else | next model, never retried |
+  | Cancelled | aborted signal / `AbortError`, or pi's "was cancelled" wording | stop, no fallback |
+
+  Deterministic failures never retry the same model. The bounded retry (500 ms,
+  doubling to a 4 s cap) is spent only on the last available model, since an
+  untried model is the better bet while one remains.
+
+## Compared with xAI `x_search`
+
+xAI's official [`x_search`](https://docs.x.ai/developers/tools/x-search) is a
+server-side tool that bundles four underlying operations — keyword search,
+semantic search, user search and thread fetch — and lets Grok choose which to
+run. `pi-twitterapi.io` targets the same read surface through twitterapi.io's REST
+API and adds the endpoints twitterapi.io exposes that `x_search` has no
+equivalent for. The table below is the honest capability comparison; the full
+version, including parameter mapping, lives in
+[`docs/x-search-comparison.md`](docs/x-search-comparison.md).
+
+| Capability | xAI `x_search` | `pi-twitterapi.io` (`twitter` tool) |
+|---|---|---|
+| Retrieval source | xAI's server-side X index | twitterapi.io REST API |
+| Credential | `XAI_API_KEY` | `TWITTERAPI_IO_API_KEY` |
+| Keyword post search | ✅ `x_keyword_search` | ✅ `mode=posts` (`advanced_search`) |
+| Semantic search | ✅ `x_semantic_search` | ❌ **no equivalent** (twitterapi.io has no semantic endpoint) |
+| User/account search | ✅ `x_user_search` | ✅ `mode=users` |
+| Thread fetch | ✅ `x_thread_fetch` | ✅ `mode=thread` |
+| Account timeline | ❌ | ✅ `mode=user` (`last_tweets`) |
+| Trends by location | ❌ | ✅ `mode=trends` |
+| Replies to a post | ❌ | ✅ `mode=replies` |
+| Quote-posts | ❌ | ✅ `mode=quotes` |
+| Mentions of an account | ❌ | ✅ `mode=mentions` |
+| Followers / followings | ❌ | ✅ `mode=followers`, `mode=followings` |
+| Single profile lookup | partial (via user search) | ✅ `mode=profile` |
+| Extended profile ("about") metadata | ❌ | ✅ `mode=about` |
+| Fetch posts by id | ❌ | ✅ `mode=tweets` |
+| Users who reposted a post | ❌ | ✅ `mode=retweeters` |
+| Communities / lists / Spaces | ❌ | ✅ `mode=community`, `mode=list`, `mode=space` |
+| Handle filters | `allowed_x_handles` / `excluded_x_handles` (max 20, mutually exclusive) | same, `mode=posts` |
+| Date range | `from_date` / `to_date` (`YYYY-MM-DD`) | same, `mode=posts`; plus unix windows for `quotes`/`mentions` |
+| Result order | chosen by the model | `queryType` (`Latest`/`Top`), `replySort` |
+| Item-count control | ❌ | ✅ `count`, `limit` |
+| Image understanding | ✅ `enable_image_understanding` | ✅ `enableImageUnderstanding` (attached when the model accepts images) |
+| Video understanding | ✅ `enable_video_understanding` | ⚠️ poster frame only (chat models cannot ingest video) |
+| Answer generation | Grok (xAI) | any pi model: `twitter.synthesisModel`, else the session model |
+| Citations | xAI annotations/citations | derived from fetched permalinks; unmatched X links dropped and disclosed |
+| Cost | xAI tokens + per post/profile | twitterapi.io credits + your model's tokens |
+| Shape | one `x_search` request | one `twitter` tool with 17 modes |
+
+**Summary.** `pi-twitterapi.io` matches `x_search` on keyword search, user search,
+thread fetch, handle filters, date ranges and image understanding, and adds a
+dedicated account timeline, trends, replies, quotes, mentions, followers,
+followings, profile, about, posts-by-id, retweeters, community, list and Space
+modes. The one
+capability it cannot match is **semantic search**, because twitterapi.io exposes
+only keyword/operator search. It also differs structurally: `x_search` is one
+server-side call answered by Grok, while this extension retrieves through
+twitterapi.io and synthesizes with a pi model of your choice.
+
+## Development
+
+```bash
+pnpm install
+pnpm typecheck
+pnpm test
+```
+
+There is no build step. Pi loads the TypeScript sources in `src/` directly
+through [jiti](https://github.com/unjs/jiti), and the published package ships
+those sources, so `pi install npm:pi-twitterapi.io` and
+`pi install git:github.com/jagaliano/pi-twitterapi.io` both work without a
+compile step. `src/*.test.ts` files are excluded from the npm tarball.
+
+The dev dependencies are version ranges (pi 1.0.2, typebox 1.3.x) resolved by
+[`pnpm-lock.yaml`](pnpm-lock.yaml); CI installs them with `--frozen-lockfile`,
+so a build is reproducible from the lockfile rather than from the manifest.
+
+## Releases
+
+1. `pnpm typecheck && pnpm test`
+2. Update [`CHANGELOG.md`](CHANGELOG.md), then bump the version with
+   `npm version patch` (or `minor`), which also creates the git tag
+3. `git push origin main --tags` — a `v*` tag makes `.github/workflows/release.yml`
+   publish it (needs an `NPM_TOKEN` repository secret) and open the GitHub release
+4. Or publish by hand: `npm publish --access public`
+
+### Rehearsing a release
+
+The pipeline can be checked without releasing anything. It runs the tag/version
+check, install, typecheck, tests and a token check, then stops unless the publish
+is explicitly requested:
+
+```bash
+gh workflow run verify-npm-token.yml --repo jagaliano/pi-twitterapi.io
+gh workflow run release.yml --ref vX.Y.Z                  # dry run
+gh workflow run release.yml --ref vX.Y.Z -f publish=true  # the real publish
+```
+
+`workflow_dispatch` runs the workflow as it exists at the given ref, so a manual
+run has to target a tag that already contains the revision you want. The `v0.1.0`
+tag was created before the dry-run mode existed, so dispatching against it
+publishes directly.
+
+Once the package exists on npm you can switch to
+[trusted publishing](https://docs.npmjs.com/trusted-publishers/) (OIDC) and delete
+`NPM_TOKEN`; the workflow works with either.
+
+The `pi-package` keyword makes the published package eligible for the
+[pi package gallery](https://pi.dev/packages); the gallery indexes npm, so there
+is nothing to submit separately. `.github/workflows/ci.yml` typechecks and tests
+every push and pull request.
+
+## License
+
+MIT

package/docs/x-search-comparison.md

@@ -0,0 +1,119 @@
+# `pi-twitterapi.io` vs xAI's official `x_search`
+
+This document compares the `twitter` tool from `pi-twitterapi.io` with xAI's
+official **X Search** (`x_search`) server-side tool. It is meant to be read
+alongside the [README](../README.md).
+
+## How each one works
+
+**xAI `x_search`** is a server-side tool on the Grok API. You enable it with
+`{ "type": "x_search" }` and Grok decides which of four underlying operations to
+run:
+
+- `x_keyword_search` — literal/keyword post retrieval
+- `x_semantic_search` — retrieval by meaning
+- `x_user_search` — profile/account discovery
+- `x_thread_fetch` — a post's whole thread
+
+xAI runs the retrieval and Grok writes the answer, so one request does
+everything. It is only available on xAI models; there is no non-xAI fallback.
+
+**`pi-twitterapi.io`** registers one `twitter` tool with 17 modes. Each mode maps
+to a twitterapi.io REST endpoint; the retrieved posts, accounts or metadata are
+then handed to a pi model (`twitter.synthesisModel`, else the session model) that
+writes the answer with citations. Retrieval and synthesis are two separate steps,
+which is why the two are priced differently.
+
+## Capability comparison
+
+| Capability | xAI `x_search` | `pi-twitterapi.io` |
+|---|---|---|
+| Retrieval source | xAI server-side X index | twitterapi.io REST API |
+| Credential | `XAI_API_KEY` | `TWITTERAPI_IO_API_KEY` |
+| Keyword post search | ✅ `x_keyword_search` | ✅ `mode=posts` → `/twitter/tweet/advanced_search` |
+| Semantic search | ✅ `x_semantic_search` | ❌ no equivalent |
+| User/account search | ✅ `x_user_search` | ✅ `mode=users` → `/twitter/user/search` |
+| Thread fetch | ✅ `x_thread_fetch` | ✅ `mode=thread` → `/twitter/tweet/thread_context` |
+| Account timeline | ❌ | ✅ `mode=user` → `/twitter/user/last_tweets` |
+| Trends by location | ❌ | ✅ `mode=trends` → `/twitter/trends` |
+| Replies to a post | ❌ | ✅ `mode=replies` → `/twitter/tweet/replies/v2` |
+| Quote-posts | ❌ | ✅ `mode=quotes` → `/twitter/tweet/quotes` |
+| Mentions of an account | ❌ | ✅ `mode=mentions` → `/twitter/user/mentions` |
+| Followers / followings | ❌ | ✅ `mode=followers` / `mode=followings` |
+| Single profile lookup | partial (via user search) | ✅ `mode=profile` → `/twitter/user/info` |
+| Extended profile ("about") metadata | ❌ | ✅ `mode=about` → `/twitter/user_about` |
+| Fetch posts by id | ❌ | ✅ `mode=tweets` → `/twitter/tweets` |
+| Users who reposted a post | ❌ | ✅ `mode=retweeters` → `/twitter/tweet/retweeters` |
+| Communities | ❌ | ✅ `mode=community` → `/twitter/community/tweets` |
+| Lists | ❌ | ✅ `mode=list` → `/twitter/list/tweets_timeline` |
+| Spaces | ❌ | ✅ `mode=space` → `/twitter/spaces/detail` |
+| Handle filters | ✅ `allowed_x_handles` / `excluded_x_handles`, max 20, mutually exclusive | ✅ same names, `mode=posts` |
+| Date range | ✅ `from_date` / `to_date` (`YYYY-MM-DD`) | ✅ same names, `mode=posts`; unix `sinceTime`/`untilTime` for `quotes`/`mentions` |
+| Result order | chosen by the model | ✅ `queryType` (`Latest`/`Top`), `replySort` (`Relevance`/`Latest`/`Likes`) |
+| Item-count control | ❌ | ✅ `count` (posts/users/trends), `limit` (many modes) |
+| Image understanding | ✅ `enable_image_understanding` | ✅ `enableImageUnderstanding` (attached only when the model accepts images) |
+| Video understanding | ✅ `enable_video_understanding` | ⚠️ poster frame only — chat models cannot ingest video, and this is disclosed |
+| Answer generation | Grok (xAI) | any pi model: `twitter.synthesisModel`, else the session model, with runtime fallback |
+| Citations | xAI annotations/citations | derived from fetched permalinks; unmatched X links dropped and disclosed |
+| Billing | xAI model tokens + per post/profile fetched | twitterapi.io credits + your synthesis model's tokens |
+| Tool shape | one `x_search` call | one `twitter` tool, 17 modes |
+| Availability | xAI models only | any pi session with a twitterapi.io key |
+
+## Parameter mapping
+
+| xAI `x_search` | `pi-twitterapi.io` |
+|---|---|
+| `allowed_x_handles` | `allowed_x_handles` (`mode=posts`) |
+| `excluded_x_handles` | `excluded_x_handles` (`mode=posts`) |
+| `from_date` | `from_date` (`mode=posts`) |
+| `to_date` | `to_date` (`mode=posts`) |
+| `enable_image_understanding` | `twitter.enableImageUnderstanding` setting |
+| `enable_video_understanding` | `twitter.enableVideoUnderstanding` setting (poster frames) |
+| — | `mode`, `tweet`, `user`, `userId`, `woeid`, `ids`, `communityId`, `listId`, `spaceId`, `queryType`, `replySort`, `count`, `limit`, `pageSize`, `includeReplies`, `sinceTime`, `untilTime` |
+
+## Where `x_search` is stronger
+
+**Semantic search.** `x_semantic_search` retrieves posts by meaning, not word
+match. twitterapi.io exposes no semantic endpoint — `advanced_search` is
+keyword/operator based. The closest approximation is to let the synthesis model
+expand a natural-language question into advanced-search operators
+(`"a" OR "b"`, `min_faves:`, `lang:`, `-filter:replies`), which is a heuristic,
+not true semantic retrieval.
+
+## Where `pi-twitterapi.io` is stronger
+
+Everything below has no `x_search` equivalent and is a first-class mode here:
+account timeline, trends, replies, quotes, mentions, followers, followings,
+profile, about, fetch-by-id, retweeters, communities, lists and Spaces.
+
+## Behavioral differences worth knowing
+
+- **Two steps, not one.** `x_search` bills xAI tokens plus per-item retrieval;
+  this extension bills twitterapi.io credits plus tokens on the pi model you
+  choose. Choosing a cheap synthesis model for summaries keeps cost down.
+- **Citations are reconstructed.** This extension never trusts model output for
+  sources: citations are derived from the permalinks actually fetched, and an X
+  link the model invents is dropped from `Sources` and disclosed in `## Notes`.
+- **Retrieval honesty.** Paging can stop early (page cap, cursor cycle, missing
+  cursor); when it does, the answer carries a note saying the results may be
+  incomplete. Trends have no permalink, so their sources are X search URLs.
+- **Cancellation is not retried** by the synthesis fallback. A cancellation is
+authoritative from the aborted signal or `AbortError`, not from provider text;
+a message such as "connection aborted" is classified as a transport failure and
+routed like any other.
+- **Images respect the answering model.** Media is attached only for a model that
+accepts image input; if the primary fails and a text-only fallback answers, the
+images are dropped (and the answer discloses it) instead of being sent to a model
+that would reject them.
+- **Date boundaries are 04:00 UTC as observed, with conservative padding.** The
+upstream resolves `since:` day boundaries at 04:00 UTC in the probed season; if
+that is really US-Eastern local midnight it shifts to 05:00 UTC in winter, so the
+start padding is one hour wider to avoid silently dropping the first hour of the
+local day. Far-west offsets can lose the tail of the requested day (disclosed),
+and far-east offsets can spend free pages on the newer trim band.
+
+## Sources
+
+- xAI X Search docs: https://docs.x.ai/developers/tools/x-search
+- xAI tool usage details: https://docs.x.ai/developers/tools/tool-usage-details
+- twitterapi.io docs: https://docs.twitterapi.io/introduction

package/package.json

@@ -0,0 +1,56 @@
+{
+  "name": "pi-twitterapi.io",
+  "version": "0.1.1",
+  "description": "twitterapi.io-backed X/Twitter search extension for pi coding agent",
+  "author": "José Antonio Galiano Sandoval",
+  "keywords": [
+    "pi-package",
+    "pi",
+    "twitter",
+    "twitterapi.io",
+    "x"
+  ],
+  "license": "MIT",
+  "homepage": "https://github.com/jagaliano/pi-twitterapi.io#readme",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/jagaliano/pi-twitterapi.io.git"
+  },
+  "bugs": {
+    "url": "https://github.com/jagaliano/pi-twitterapi.io/issues"
+  },
+  "type": "module",
+  "files": [
+    "src",
+    "!src/**/*.test.ts",
+    "README.md",
+    "CHANGELOG.md",
+    "docs"
+  ],
+  "pi": {
+    "extensions": [
+      "./src/index.ts"
+    ],
+    "image": "https://raw.githubusercontent.com/jagaliano/pi-twitterapi.io/main/preview.png"
+  },
+  "scripts": {
+    "typecheck": "tsc --noEmit",
+    "test": "tsx --test src/**/*.test.ts"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "devDependencies": {
+    "@earendil-works/pi-coding-agent": "^1.0.2",
+    "@earendil-works/pi-tui": "^1.0.2",
+    "@types/node": "^26.6.4",
+    "tsx": "^4.23.15",
+    "typebox": "^1.3.34",
+    "typescript": "^6.0.3"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*",
+    "@earendil-works/pi-tui": "*",
+    "typebox": "*"
+  }
+}

package/src/backend/media.ts

@@ -0,0 +1,101 @@
+import { toBase64, type ImageAttachment } from "../synthesize.js";
+
+const MAX_MEDIA_BYTES = 8 * 1024 * 1024;
+const MEDIA_TIMEOUT_MS = 20_000;
+
+/** Read a body while enforcing a byte cap, so an oversized response is never fully buffered. */
+async function readCapped(response: Response, limit: number): Promise<Uint8Array | undefined> {
+  const body = response.body;
+  if (!body) return undefined;
+  const reader = body.getReader();
+  const chunks: Uint8Array[] = [];
+  let total = 0;
+  try {
+    for (;;) {
+      const { done, value } = await reader.read();
+      if (done) break;
+      if (!value) continue;
+      total += value.byteLength;
+      if (total > limit) {
+        await reader.cancel().catch(() => undefined);
+        return undefined;
+      }
+      chunks.push(value);
+    }
+  } finally {
+    reader.releaseLock();
+  }
+  const bytes = new Uint8Array(total);
+  let offset = 0;
+  for (const chunk of chunks) {
+    bytes.set(chunk, offset);
+    offset += chunk.byteLength;
+  }
+  return bytes;
+}
+
+export interface FetchMediaLimits {
+  /** Maximum accepted image size in bytes (default 8 MiB). */
+  maxBytes?: number;
+  /** Per-download deadline covering the body read (default 20s). */
+  timeoutMs?: number;
+}
+
+/** Twitter media CDN domains whose URLs may be downloaded for synthesis. */
+export const ALLOWED_MEDIA_HOSTS = ["twimg.com"] as const;
+
+/**
+ * True for an HTTPS URL on a known Twitter media host.
+ *
+ * Post media URLs come from the upstream API and are therefore attacker-
+ * influenceable: without this check a crafted URL could point the download at
+ * localhost or a private service (SSRF). Redirects are additionally refused at
+ * fetch time so an allowed host cannot bounce the request inward.
+ */
+export function isAllowedMediaUrl(raw: string): boolean {
+  let parsed: URL;
+  try {
+    parsed = new URL(raw);
+  } catch {
+    return false;
+  }
+  if (parsed.protocol !== "https:") return false;
+  const host = parsed.hostname.toLowerCase();
+  return ALLOWED_MEDIA_HOSTS.some((suffix) => host === suffix || host.endsWith(`.${suffix}`));
+}
+
+export function createFetchMedia(fetcher: typeof fetch, callerSignal?: AbortSignal, limits: FetchMediaLimits = {}) {
+  const maxBytes = limits.maxBytes ?? MAX_MEDIA_BYTES;
+  const timeoutMs = limits.timeoutMs ?? MEDIA_TIMEOUT_MS;
+  return async function fetchMedia(url: string, timeoutMsOverride?: number): Promise<ImageAttachment | undefined> {
+    const budget = timeoutMsOverride === undefined ? timeoutMs : Math.max(1, Math.min(timeoutMsOverride, timeoutMs));
+    if (callerSignal?.aborted) return undefined;
+    // Media URLs are attacker-influenceable upstream data, so the host and
+    // scheme are checked before any request is made.
+    if (!isAllowedMediaUrl(url)) return undefined;
+    // A stalled image must not outlive the tool call: the download carries the
+    // caller's signal and its own deadline, covering the body read too. A caller
+    // may shorten that deadline (the media phase passes what remains of its
+    // budget) but never extend it.
+    const controller = new AbortController();
+    const onAbort = () => controller.abort();
+    callerSignal?.addEventListener("abort", onAbort, { once: true });
+    const timer = setTimeout(() => controller.abort(), budget);
+    try {
+      const response = await fetcher(url, { signal: controller.signal, redirect: "error" });
+      if (!response.ok) return undefined;
+      const mimeType = response.headers.get("content-type")?.split(";")[0]?.trim().toLowerCase() ?? "";
+      if (!mimeType.startsWith("image/")) return undefined;
+      const declared = Number(response.headers.get("content-length") ?? Number.NaN);
+      if (Number.isFinite(declared) && declared > maxBytes) return undefined;
+      const bytes = await readCapped(response, maxBytes);
+      if (!bytes) return undefined;
+      return { data: toBase64(bytes), mimeType };
+    } catch {
+      return undefined;
+    } finally {
+      clearTimeout(timer);
+      callerSignal?.removeEventListener("abort", onAbort);
+    }
+  };
+}