pi-inferhub 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Damn Mersault
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -7,9 +7,10 @@ InferHub **combos** as selectable chat models.
7
7
  InferHub pools AI subscriptions behind one OpenAI-compatible endpoint. Combos
8
8
  are its server-side routed bundles — failover, cache-sticky, billed in USDC —
9
9
  and they are the primary consumption surface. This extension exposes **combos
10
- only**: each combo on your account appears as exactly one model named
11
- `combo/<slug>`. Upstream `prefix/model` entries, bare-name aliases, and
12
- free-tier models from InferHub's model list are intentionally not exposed.
10
+ only by default**: each combo on your account appears as exactly one model
11
+ named `combo/<slug>`. Upstream `prefix/model` entries, bare-name aliases, and
12
+ free-tier models are not exposed unless you opt in with
13
+ [`publishUpstream`](#full-upstream-catalog-optional).
13
14
 
14
15
  ## Install
15
16
 
@@ -51,9 +52,30 @@ points you to `/login inferhub` if not.
51
52
  ## Model selection
52
53
 
53
54
  Send `/model` and filter by provider name `inferhub` or by `combo/`. Each
54
- model id is `combo/<slug>` — the routable inference name on
55
+ combo model id is `combo/<slug>` — the routable inference name on
55
56
  `https://api.inferhub.dev/v1`.
56
57
 
58
+ ## Configuration
59
+
60
+ Optional `~/.pi/agent/pi-inferhub.json` (or `$PI_AGENT_DIR/pi-inferhub.json`).
61
+ A missing file means defaults; a malformed one logs an issue and keeps the
62
+ defaults — it never breaks startup.
63
+
64
+ ```json
65
+ { "publishUpstream": true }
66
+ ```
67
+
68
+ ### Full upstream catalog (optional)
69
+
70
+ `publishUpstream` (default `false`) additionally publishes every chat model
71
+ from `/v1/models` as a direct-selectable `prefix/model` entry, listed after
72
+ the combos. This bypasses combo routing — you pick one specific serving — and
73
+ is intended for pinning a particular upstream. Alias rows, `free/` tiers, and
74
+ image/video generation models are never published, even with the option on.
75
+
76
+ The `INFERHUB_PUBLISH_UPSTREAM` environment variable overrides the file in
77
+ both directions (`1/true/yes/on`, `0/false/no/off`).
78
+
57
79
  ## Catalog derivation
58
80
 
59
81
  On refresh the extension makes two authenticated GETs —
@@ -63,19 +85,29 @@ combo to one model:
63
85
 
64
86
  - **Context/output limits** — minimum across resolvable members
65
87
  (`input_token_limit` / `max_output_tokens`), so a combo is only as large as
66
- its smallest member. When no resolvable member supplies limits, conservative
67
- fallbacks apply: **128k context / 8k max output**.
88
+ its smallest member. When no resolvable member supplies limits, fallbacks
89
+ apply: **256k context / 128k max output**. A limit of `0` is treated as
90
+ missing data rather than a real ceiling.
91
+ - **Output cap enforcement** — `max_tokens` is sent rather than
92
+ `max_completion_tokens`, because InferHub ignores the latter (a request
93
+ capped at 20 tokens still produced 107).
68
94
  - **Image input** — offered (`text` + `image`) only when *every* resolvable
69
95
  member's modality includes image; otherwise text-only.
70
- - **Reasoning** — the combo is marked reasoning-capable when any resolvable
71
- member advertises `reasoning_levels`; Pi's thinking levels map onto the
72
- advertised ladder (levels a combo doesn't advertise are unavailable).
96
+ - **Reasoning** — combos are always marked reasoning-capable: a false
97
+ negative hides Pi's thinking knob (the worse error), while a false positive
98
+ only shows an inert one. Where members advertise `reasoning_levels`, Pi's
99
+ levels map onto that union; when none advertise a ladder the map is omitted
100
+ so every level stays selectable.
73
101
  - **Cost** — per-direction minimum of resolvable members' *official* rates
74
102
  (see below). `cacheRead`/`cacheWrite` are shown as `0`: the catalog carries
75
103
  no per-combo cache pricing, and none is fabricated.
76
104
 
77
- Combos whose members are all unresolvable aliases get safe defaults:
78
- text-only, reasoning-capable, `0` cost, and the fallback limits.
105
+ Member resolution is conservative but forgiving: a member resolves directly
106
+ from `/v1/models` when its prefixed row carries data; otherwise it joins the
107
+ prefixed servings of the same model name (exact suffix first, then a
108
+ normalized last-segment match, which covers multi-segment prefixes such as
109
+ `cp/cline-pass/<model>`). Combos whose members resolve to nothing at all get
110
+ safe defaults: text-only, reasoning-capable, `0` cost, and the fallback limits.
79
111
 
80
112
  ## Refresh semantics
81
113
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-inferhub",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "Pi extension for the InferHub inference provider",
5
5
  "license": "MIT",
6
6
  "keywords": [
@@ -23,9 +23,15 @@
23
23
  "typebox": "*"
24
24
  },
25
25
  "peerDependenciesMeta": {
26
- "@earendil-works/pi-ai": { "optional": false },
27
- "@earendil-works/pi-coding-agent": { "optional": false },
28
- "typebox": { "optional": false }
26
+ "@earendil-works/pi-ai": {
27
+ "optional": false
28
+ },
29
+ "@earendil-works/pi-coding-agent": {
30
+ "optional": false
31
+ },
32
+ "typebox": {
33
+ "optional": false
34
+ }
29
35
  },
30
36
  "devDependencies": {
31
37
  "@earendil-works/pi-ai": "^1.0.4",
@@ -34,5 +40,17 @@
34
40
  "typebox": "^1.3.36",
35
41
  "typescript": "^6.0.3",
36
42
  "vitest": "^5.0.3"
43
+ },
44
+ "files": [
45
+ "src",
46
+ "README.md",
47
+ "LICENSE"
48
+ ],
49
+ "engines": {
50
+ "node": ">=22"
51
+ },
52
+ "repository": {
53
+ "type": "git",
54
+ "url": "git+https://github.com/damnmersault/pi-inferhub.git"
37
55
  }
38
56
  }
package/src/catalog.ts CHANGED
@@ -31,6 +31,8 @@ export const CatalogModelSchema = Type.Object({
31
31
  id: Type.String(),
32
32
  owned_by: Type.Optional(Type.String()),
33
33
  modality: Type.Optional(Type.String()),
34
+ output_modality: Type.Optional(Type.String()),
35
+ upstream_label: Type.Optional(Type.String()),
34
36
  input_token_limit: Type.Optional(Type.Number()),
35
37
  max_output_tokens: Type.Optional(Type.Number()),
36
38
  reasoning_levels: Type.Optional(Type.Array(Type.String())),
@@ -60,17 +62,25 @@ const ModelsEnvelope = Type.Object({ data: Type.Array(CatalogModelSchema) });
60
62
  const CombosValidator = Compile(CombosEnvelope);
61
63
  const ModelsValidator = Compile(ModelsEnvelope);
62
64
 
65
+ export interface FetchCatalogOptions extends FetchJsonOptions {
66
+ /** Also publish direct-selectable upstream models from /v1/models. */
67
+ publishUpstream?: boolean;
68
+ }
69
+
63
70
  /** GET /combos + GET /v1/models (independent), then map combos to pi models. */
64
71
  export async function fetchComboCatalog(
65
72
  token: string,
66
73
  signal?: AbortSignal,
67
- opts?: FetchJsonOptions,
74
+ opts?: FetchCatalogOptions,
68
75
  ): Promise<Model<"openai-completions">[]> {
69
76
  const [combos, models] = await Promise.all([
70
77
  fetchJson<Combo[]>(`${MGMT_BASE_URL}/combos`, token, CombosValidator, signal, opts),
71
78
  fetchJson<Static<typeof ModelsEnvelope>>(`${INFERENCE_BASE_URL}/models`, token, ModelsValidator, signal, opts),
72
79
  ]);
73
- return combosToModels(combos, models.data);
80
+ const comboModels = combosToModels(combos, models.data);
81
+ // Combos are the product: they stay first. The upstream catalog only widens
82
+ // the list when the operator opts in.
83
+ return opts?.publishUpstream ? [...comboModels, ...catalogToModels(models.data)] : comboModels;
74
84
  }
75
85
 
76
86
  /** Pure combo → Model mapping; output order follows `combos` order. */
@@ -90,6 +100,12 @@ function comboToModel(
90
100
  // Alias rows (`owned_by: "alias"`, no data) and alias-kind members resolve
91
101
  // through prefixed sibling servings of the same model name; min-of over the
92
102
  // siblings bounds the combo exactly like directly-joined members.
103
+ //
104
+ // Live shape 2026-10-18: combos reference multi-segment prefixes
105
+ // (cp/cline-pass/<model>, cp/zai/<model>) that have no exact serving, while
106
+ // the same model is served under another prefix (zai/<model>,
107
+ // cmc/moonshotai/<model>). Exact suffix matches win; the normalized
108
+ // last-segment name is the recovery path, never a wider net.
93
109
  const resolvable: CatalogModel[] = [];
94
110
  for (const member of combo.members) {
95
111
  const name = member.kind === "model" ? member.model : member.alias;
@@ -98,27 +114,73 @@ function comboToModel(
98
114
  resolvable.push(entry);
99
115
  continue;
100
116
  }
101
- for (const sibling of catalogModels) {
102
- if (sibling.owned_by === "alias") continue;
103
- if (sibling.id.endsWith(`/${name}`)) resolvable.push(sibling);
117
+ const exact = catalogModels.filter(
118
+ (sibling) => sibling.owned_by !== "alias" && sibling.id.endsWith(`/${name}`),
119
+ );
120
+ if (exact.length > 0) {
121
+ resolvable.push(...exact);
122
+ continue;
123
+ }
124
+ const normalized = normalizeModelName(name);
125
+ if (normalized.length > 0) {
126
+ resolvable.push(
127
+ ...catalogModels.filter(
128
+ (sibling) => sibling.owned_by !== "alias" && normalizeModelName(sibling.id) === normalized,
129
+ ),
130
+ );
104
131
  }
105
132
  }
106
133
 
107
- // Alias-only / all-missed-join combos reason optimistically: a false
108
- // negative hides thinking levels, a false positive shows an inert knob.
134
+ // Optimistic by design: a false negative hides thinking levels (the worse
135
+ // error); a false positive shows an inert knob. Applies to every combo,
136
+ // including level-less resolved members (live-verified 2026-10-18 on
137
+ // combo/deepseek-v4-pro: no advertised levels, honors reasoning_effort).
138
+ const name = combo.name.trim().length > 0 ? combo.name : combo.slug;
139
+ return buildModel(`combo/${combo.slug}`, name, resolvable);
140
+ }
141
+
142
+ /**
143
+ * Direct-selectable upstream models from `/v1/models`, in catalog order.
144
+ * Skipped: alias rows (information-free duplicates of a prefixed serving),
145
+ * image-output rows (not chat models), and blank ids (live shape `cc/`).
146
+ */
147
+ export function catalogToModels(catalogModels: CatalogModel[]): Model<"openai-completions">[] {
148
+ return catalogModels
149
+ .filter(
150
+ (row) =>
151
+ row.owned_by !== "alias" &&
152
+ // AGENTS.md product rule: never surface image/video generation or
153
+ // `free/` tier models, even when the upstream catalog is published.
154
+ row.output_modality !== "image" &&
155
+ row.output_modality !== "video" &&
156
+ !row.id.startsWith("free/") &&
157
+ // Live shape `cc/`: prefix with an empty model segment — not selectable.
158
+ (row.id.split("/").pop() ?? "").trim().length > 0,
159
+ )
160
+ .map((row) => {
161
+ const label = row.upstream_label?.trim();
162
+ return buildModel(row.id, label && label.length > 0 ? label : row.id, [row]);
163
+ });
164
+ }
165
+
166
+ /** Shared pi-model construction for combo and direct-catalog entries. */
167
+ function buildModel(
168
+ id: string,
169
+ name: string,
170
+ sources: CatalogModel[],
171
+ ): Model<"openai-completions"> {
109
172
  const advertised = new Set<string>();
110
- for (const model of resolvable) {
173
+ for (const model of sources) {
111
174
  for (const level of model.reasoning_levels ?? []) advertised.add(level);
112
175
  }
113
- const reasoning = resolvable.length === 0 || advertised.size > 0;
114
176
 
115
- const image = resolvable.length > 0 && resolvable.every((model) => (model.modality ?? "").includes("image"));
177
+ const image = sources.length > 0 && sources.every((model) => (model.modality ?? "").includes("image"));
116
178
 
117
- const priced = resolvable.flatMap((model) => (model.pricing ? [model.pricing] : []));
179
+ const priced = sources.flatMap((model) => (model.pricing ? [model.pricing] : []));
118
180
 
119
181
  const model: Model<"openai-completions"> = {
120
- id: `combo/${combo.slug}`,
121
- name: combo.name.trim().length > 0 ? combo.name : combo.slug,
182
+ id,
183
+ name,
122
184
  api: "openai-completions",
123
185
  provider: PROVIDER_NAME,
124
186
  baseUrl: INFERENCE_BASE_URL,
@@ -129,17 +191,27 @@ function comboToModel(
129
191
  cacheRead: 0,
130
192
  cacheWrite: 0,
131
193
  },
132
- reasoning,
133
- contextWindow: minOf(resolvable.map((m) => m.input_token_limit)) ?? CONTEXT_WINDOW_FALLBACK,
134
- maxTokens: minOf(resolvable.map((m) => m.max_output_tokens)) ?? MAX_TOKENS_FALLBACK,
194
+ reasoning: true,
195
+ contextWindow: minPositiveOf(sources.map((source) => source.input_token_limit)) ?? CONTEXT_WINDOW_FALLBACK,
196
+ maxTokens: minPositiveOf(sources.map((source) => source.max_output_tokens)) ?? MAX_TOKENS_FALLBACK,
197
+ // InferHub's [OI]-compatible surface ignores max_completion_tokens (verified
198
+ // live 2026-10-18: a 20-token cap sent that way produced 107 tokens; the
199
+ // same cap as max_tokens stopped at 20). pi-ai auto-detects per-provider
200
+ // and inferhub is not on its list, so pin the field explicitly.
201
+ compat: { maxTokensField: "max_tokens" },
135
202
  };
136
- // thinkingLevelMap: built only from an advertised union; an empty union
137
- // (alias-only combos) omits the map so pi defaults keep thinking levels
138
- // selectable — marking them all `null` would hide the knob entirely.
139
- if (reasoning && advertised.size > 0) model.thinkingLevelMap = buildThinkingLevelMap(advertised);
203
+ // thinkingLevelMap: built only from an advertised union; a level-less union
204
+ // omits the map so pi defaults keep thinking levels selectable — marking
205
+ // them all `null` would hide the knob entirely.
206
+ if (advertised.size > 0) model.thinkingLevelMap = buildThinkingLevelMap(advertised);
140
207
  return model;
141
208
  }
142
209
 
210
+ /** Last path segment, lowercased, alphanumerics only: "cp/cline-pass/Kimi-K3" → "kimik3". */
211
+ function normalizeModelName(id: string): string {
212
+ return (id.split("/").pop() ?? id).toLowerCase().replace(/[^a-z0-9]/g, "");
213
+ }
214
+
143
215
  /** Each pi level maps to itself when the combo advertises it, else `null`. */
144
216
  function buildThinkingLevelMap(advertised: ReadonlySet<string>): ThinkingLevelMap {
145
217
  const map: ThinkingLevelMap = { off: "off" };
@@ -153,3 +225,9 @@ function minOf(values: Array<number | undefined>): number | undefined {
153
225
  const present = values.filter((value): value is number => value !== undefined);
154
226
  return present.length > 0 ? Math.min(...present) : undefined;
155
227
  }
228
+
229
+ /** Limits: 0 (or non-finite) is missing data, never a real ceiling. Prices keep 0. */
230
+ function minPositiveOf(values: Array<number | undefined>): number | undefined {
231
+ const present = values.filter((value): value is number => value !== undefined && Number.isFinite(value) && value > 0);
232
+ return present.length > 0 ? Math.min(...present) : undefined;
233
+ }
package/src/config.ts ADDED
@@ -0,0 +1,75 @@
1
+ import { readFileSync } from "node:fs";
2
+ import { homedir } from "node:os";
3
+ import { join } from "node:path";
4
+
5
+ /** Canonical config file for this package, under pi's agent dir. */
6
+ export const CONFIG_FILE_NAME = "pi-inferhub.json";
7
+
8
+ /** Env override for `publishUpstream`; wins over the file when set. */
9
+ export const PUBLISH_UPSTREAM_ENV = "INFERHUB_PUBLISH_UPSTREAM";
10
+
11
+ export interface InferhubConfig {
12
+ /**
13
+ * Also publish every chat model from `/v1/models` as a direct-selectable
14
+ * `prefix/model` entry, alongside the `combo/<slug>` models. Combos stay
15
+ * first; this only widens the catalog. Default: false.
16
+ */
17
+ publishUpstream: boolean;
18
+ /** Human-readable problems found while loading; never fatal. */
19
+ issues: string[];
20
+ }
21
+
22
+ /** pi's agent dir: `PI_AGENT_DIR` when set, else `~/.pi/agent`. */
23
+ export function defaultAgentDir(): string {
24
+ return process.env.PI_AGENT_DIR ?? join(homedir(), ".pi", "agent");
25
+ }
26
+
27
+ function parseEnvBoolean(raw: string): boolean | undefined {
28
+ const value = raw.trim().toLowerCase();
29
+ if (["1", "true", "yes", "on"].includes(value)) return true;
30
+ if (["0", "false", "no", "off"].includes(value)) return false;
31
+ return undefined;
32
+ }
33
+
34
+ /**
35
+ * Load `<agentDir>/pi-inferhub.json`. Missing file = defaults; malformed file =
36
+ * defaults + issue; unknown keys are ignored, wrong-typed values are reported
37
+ * and leave the default in place. Never throws for file content problems.
38
+ */
39
+ export function loadInferhubConfig(agentDir: string = defaultAgentDir()): InferhubConfig {
40
+ const issues: string[] = [];
41
+ let raw: unknown;
42
+ try {
43
+ raw = JSON.parse(readFileSync(join(agentDir, CONFIG_FILE_NAME), "utf8"));
44
+ } catch (error) {
45
+ const code = (error as NodeJS.ErrnoException)?.code;
46
+ if (code === "ENOENT") return { publishUpstream: false, issues };
47
+ const message = error instanceof Error ? error.message : String(error);
48
+ issues.push(`${CONFIG_FILE_NAME} is not valid JSON (${message}); using defaults`);
49
+ return { publishUpstream: false, issues };
50
+ }
51
+
52
+ if (typeof raw !== "object" || raw === null || Array.isArray(raw)) {
53
+ issues.push(`${CONFIG_FILE_NAME} must contain a JSON object; using defaults`);
54
+ return { publishUpstream: false, issues };
55
+ }
56
+
57
+ let publishUpstream = false;
58
+ for (const [key, value] of Object.entries(raw as Record<string, unknown>)) {
59
+ if (key !== "publishUpstream") continue;
60
+ if (typeof value === "boolean") publishUpstream = value;
61
+ else issues.push(`publishUpstream must be a boolean (got ${typeof value}); using false`);
62
+ }
63
+
64
+ const envRaw = process.env[PUBLISH_UPSTREAM_ENV];
65
+ if (envRaw !== undefined) {
66
+ const envValue = parseEnvBoolean(envRaw);
67
+ if (envValue === undefined) {
68
+ issues.push(`${PUBLISH_UPSTREAM_ENV} must be a boolean-like value; ignoring`);
69
+ } else {
70
+ publishUpstream = envValue;
71
+ }
72
+ }
73
+
74
+ return { publishUpstream, issues };
75
+ }
package/src/index.ts CHANGED
@@ -4,6 +4,7 @@ import type { ExtensionAPI, ExtensionCommandContext } from "@earendil-works/pi-c
4
4
  import { openAICompletionsApi } from "@earendil-works/pi-ai/compat";
5
5
  import { inferhubApiKeyAuth, loginInferhub } from "./auth.js";
6
6
  import { fetchComboCatalog } from "./catalog.js";
7
+ import { type InferhubConfig, loadInferhubConfig } from "./config.js";
7
8
  import { INFERENCE_BASE_URL, PROVIDER_DISPLAY_NAME, PROVIDER_NAME } from "./inferhub.js";
8
9
 
9
10
  /**
@@ -17,7 +18,13 @@ export interface InferhubRegistrationPi {
17
18
  registerCommand(name: string, options: { description?: string; handler: (args: string, ctx: ExtensionCommandContext) => Promise<void> }): void;
18
19
  }
19
20
 
20
- function inferhubProvider(): Provider<"openai-completions"> {
21
+ export interface InferhubRegistrationOptions {
22
+ /** Config loader seam; production reads `<agentDir>/pi-inferhub.json`. */
23
+ loadConfig?: () => InferhubConfig;
24
+ }
25
+
26
+ function inferhubProvider(options: InferhubRegistrationOptions = {}): Provider<"openai-completions"> {
27
+ const loadConfig = options.loadConfig ?? (() => loadInferhubConfig());
21
28
  return createProvider({
22
29
  id: PROVIDER_NAME,
23
30
  name: PROVIDER_DISPLAY_NAME,
@@ -36,15 +43,15 @@ function inferhubProvider(): Provider<"openai-completions"> {
36
43
  const credential = context.credential;
37
44
  const token = credential?.type === "api_key" ? credential.key : undefined;
38
45
  if (!token) return [];
39
- return fetchComboCatalog(token, context.signal);
46
+ return fetchComboCatalog(token, context.signal, { publishUpstream: loadConfig().publishUpstream });
40
47
  },
41
48
  api: openAICompletionsApi(),
42
49
  });
43
50
  }
44
51
 
45
52
  /** Register the InferHub provider and its companion command. */
46
- export function registerInferhubProvider(pi: InferhubRegistrationPi): void {
47
- pi.registerProvider(inferhubProvider());
53
+ export function registerInferhubProvider(pi: InferhubRegistrationPi, options?: InferhubRegistrationOptions): void {
54
+ pi.registerProvider(inferhubProvider(options));
48
55
 
49
56
  // pi's registerCommand takes a single command name; `/login inferhub` is
50
57
  // provided by pi's auth system through the apiKey login wired above. This
package/AGENTS.md DELETED
@@ -1,146 +0,0 @@
1
- # pi-inferhub
2
-
3
- pi provider extension exposing the user's InferHub (https://inferhub.dev)
4
- **combos** as selectable chat models. Combos are the product: the extension
5
- never surfaces `prefix/model` upstream ids, bare aliases, `free/` tier
6
- models, or images/video generation. Combos are `combo/<slug>` models routed
7
- through InferHub's OpenAI-compatible inference surface; all failover,
8
- serialization, and selection happens server-side.
9
-
10
- Spec (binding design authority): `docs/superpowers/specs/2026-10-06-inferhub-provider-design.md`
11
- Plan: `docs/superpowers/plans/2026-10-06-inferhub-provider.md`
12
-
13
- ## Architecture
14
-
15
- Four modules, acyclic, one-directional:
16
-
17
- ```
18
- src/inferhub.ts constants only — PROVIDER_NAME="inferhub", PROVIDER_DISPLAY_NAME="InferHub",
19
- INFERENCE_BASE_URL="https://api.inferhub.dev/v1", MGMT_BASE_URL="https://inferhub.dev/api"
20
- src/http.ts ← transport only — fetchJson (Bearer GET + typebox validation), InferhubHttpError{status,code,message}
21
- src/auth.ts ← loginInferhub (prompt → validate GET /me → store), inferhubApiKeyAuth (envApiKeyAuth)
22
- src/catalog.ts ← fetchComboCatalog (Promise.all of the two GETs), pure combosToModels for tests
23
- src/index.ts ← default extension factory: createProvider registration + inferhub-login signpost command
24
- ```
25
-
26
- No custom `streamSimple`. Everything streams through pi-ai's
27
- `openAICompletionsApi()` (imported from `@earendil-works/pi-ai/compat` —
28
- pi's extension loader does not virtualize lazy API subpaths). pi-ai owns
29
- message conversion, tool handling, usage, cancellation, retries. Adding a
30
- custom stream is a design-level decision, never a drive-by fix for an
31
- upstream quirk.
32
-
33
- ## Wire contract (live-pinned 2026-10-07, verify before trusting)
34
-
35
- - Auth: `Authorization: Bearer sk-airo-…` on BOTH surfaces.
36
- - `GET MGMT_BASE_URL/combos` → **bare JSON array** of combos (id/name/slug/
37
- members[]; member kinds `model` with `model: "prefix/upstream_id"` and
38
- `alias` with bare name, no catalog linkage).
39
- - `GET INFERENCE_BASE_URL/models` → `{object:"list", data:[...]}`; items
40
- carry `id, modality ("text" | "text,image"), input_token_limit,
41
- max_output_tokens, reasoning_levels, pricing.official_in/official_out`.
42
- Combos do NOT appear here — that is why refresh fetches /combos
43
- separately. `/v1/models` requires auth (401 unauthenticated).
44
- - `GET MGMT_BASE_URL/me` → any JSON object; login validation endpoint.
45
- - Management API rate limit 30 req/min/account; inference 500 req/min/account.
46
- - Errors: `{"error":{"code","message"}}`; suspended/banned → 403
47
- `account_blocked` with the ban reason in `message`.
48
- - Cost: inference responses report `usage.cost` USD (OpenRouter convention),
49
- $0.00001 minimum on billed requests. All management-surface money is USDC
50
- decimal strings.
51
-
52
- ## Invariants (breaking any of these is a reverted commit)
53
-
54
- 1. **Reserved-key play-safe**: combo-level metadata (maxInputPerMtok/
55
- maxOutputPerMtok caps, selection mode, filters, sticky, members) never
56
- rides on the pi-ai `Model` object. Mapped models carry exactly pi-ai
57
- `Model<"openai-completions">` fields.
58
- 2. **Cost honesty**: `cost.input/output` = per-direction MIN of resolvable
59
- members' `official` rates (conservative upper bound — InferHub bills
60
- discounts below official). `cost.cacheRead/cacheWrite` = 0 always —
61
- InferHub's /v1/models pricing carries no cache fields; never fabricate.
62
- 3. **Fallback constants**: `CONTEXT_WINDOW_FALLBACK = 262_144`,
63
- `MAX_TOKENS_FALLBACK = 131_072` — used when no resolvable member supplies
64
- the field (last resort after the alias sibling join; see 5). pi-ai's
65
- `Model` type requires both as numbers; never guess a model's ceiling
66
- other than via these constants.
67
- 4. **Image rule**: `input: ["text","image"]` only when ≥1 resolvable member
68
- AND every resolvable member's `modality` includes `image`. Understating
69
- input is acceptable; overstating sends requests the upstream rejects.
70
- 5. **Alias-member fallback**: bare alias rows (`owned_by: "alias"`, no
71
- data — observed live 2026-10-15) and alias-kind members first resolve
72
- through prefixed sibling servings of the same model name (`<prefix>/<name>`
73
- rows in the /v1/models catalog, min-of applies). Only when no sibling
74
- carries data either → `input:["text"]`, `reasoning:true`, zero cost,
75
- fallback limits. `reasoning:true` is deliberate: a false negative hides
76
- thinking levels (the worse error); a false positive shows an inert knob.
77
- 6. **thinkingLevelMap**: built from the union of resolvable members'
78
- `reasoning_levels`; each pi level (`minimal|low|medium|high|xhigh|max`)
79
- maps to itself when advertised, else `null`; `off:"off"`. Set only when
80
- `reasoning` is true. Alias-only/level-less combos OMIT the map entirely —
81
- an all-null map would hide every selectable level.
82
- 7. **No key leakage**: the API key appears ONLY in the Authorization header
83
- and pi's credential store (`auth.json` via pi's own store). Never log,
84
- print, or persist it anywhere else. Tests inject fetch mocks; a test
85
- asserting the header is fine, echoing the key in fixtures is not.
86
- 8. **Unconfigured = zero models, no network**: `fetchModels` returns `[]`
87
- and fetches nothing when auth resolves no key. pi-ai passes BOTH
88
- stored-credential and env-resolved keys as `context.credential`
89
- (verified in pi-ai 1.0.4 `dist/models.js` `resolveRefreshCredential`) —
90
- do not add a second key-resolution path.
91
- 9. **No cross-run catalog persistence**: refresh is always live. A stale
92
- combo snapshot silently serving the wrong routing is worse than an
93
- explicit refresh.
94
-
95
- ## Commands
96
-
97
- ```
98
- npx vitest run # full suite (18 tests, <2s)
99
- npx tsc --noEmit # typecheck
100
- npm pack --dry-run # inspect package contents before publishing
101
- ```
102
-
103
- pi packs the extension from `package.json` `pi.extensions:
104
- ["./src/index.ts"]`. Local dev: `pi -e <repo path>`. Published install:
105
- `npm:pi-inferhub` in `~/.pi/agent/settings.json` packages (NOT yet
106
- published npm:pi-inferhub (v0.1.1).
107
-
108
- ## Environment gotchas (learned the hard way)
109
-
110
- - **Pi packages declare host modules as peerDependencies, never
111
- dependencies**: `@earendil-works/pi-ai`, `@earendil-works/pi-coding-agent`,
112
- `typebox` (and pi-tui if used) go in `peerDependencies` with `"*"` ranges;
113
- keep `dependencies` empty. Regular dependencies make pi's package installer
114
- warn about duplicate host-module copies — extensions must bind to the
115
- copies the pi host already loads. Mirror the same modules in
116
- `devDependencies` (with real version pins) for the repo's own
117
- tests/typecheck.
118
-
119
-
120
- - **TypeScript 6.0.3**: tsconfig MUST carry `types: ["node"]` — TS 6 dropped
121
- automatic `@types/*` inclusion; without it `node:module`/`fetch`/`AbortSignal`
122
- don't resolve and `tsc --noEmit` fails.
123
- - **typebox 1.3.36**: compiled validators are typed `Validator<TProperties,
124
- TSchema>` — pass the validator, not the schema, to `fetchJson`. Validation
125
- error detail lives on `instancePath` (there is no `path` field). Extra
126
- payload keys pass through by default; do NOT add
127
- `additionalProperties: false` to wire schemas.
128
- - **pi-ai `AuthEvent`** is a union; non-info variants carry no `message`.
129
- Anything typing an interaction callback as `{type: string; message:
130
- string}` breaks under contravariance — use `{type: string; message?:
131
- string}`.
132
- - **Live pinning of wire shapes is allowed and encouraged** when a shape is
133
- undocumented: run a free listing GET, pin the parsed envelope in a typebox
134
- validator, and leave a dated comment. Drift surfaces as
135
- `code: "schema_mismatch"` with `instancePath` — fail-loud by design.
136
-
137
- ## Testing conventions
138
-
139
- - Typebox schemas live at module scope (`Compile(Type.Object({...}))`).
140
- - Tabs, ESM with `.js` import specifiers, single-responsibility modules.
141
- - Tests verify real behavior through public seams (`combosToModels` pure
142
- function; real `createProvider` refresh path in test/index.test.ts with
143
- injected `fetchImpl`). No seams into internals.
144
- - Deleting a brief-verbatim test line, or "fixing" a failing test by editing
145
- the test to fit the code rather than the contract, is not allowed without
146
- naming the contract being violated.