fa-model-catalog 0.0.0-stage → 0.1.1

package/docs/catalog.md

@@ -0,0 +1,27 @@
+# Model catalog
+
+The catalog combines model prices, billing units, and capabilities.
+An invalid revision never replaces the active one, so applications continue receiving the last validated snapshot.
+
+The format is defined by the [schema and validator](../src/catalog-schemas.js); initial entries are available in the
+[example catalog](../catalog.example.yaml).
+The server settings specify the working file, which must remain separate from source files replaced during updates.
+
+An entry can contain a model description or a string naming another entry exactly.
+A string reference is an alternative name for the same pricing record; YAML aliases are not allowed.
+An unknown target or reference cycle invalidates the entire revision.
+
+Lookup first checks the full name. If it is missing, lookup removes the prefix through the first @ and checks the rest.
+Slashes and trailing dates are not removed automatically; such names require explicit entries.
+Responses preserve the requested, matched, and canonical names to explain the selected price.
+
+Prices and supported modes are described separately.
+Matching rates do not imply mode support, and an unknown rate differs from a confirmed zero.
+Billing units are explicit; minutes, characters, images, and video seconds cannot be treated as text usage.
+
+The server loads the catalog and access settings before opening its port.
+File changes apply as a fully validated snapshot; failures and recovery are logged.
+The content version is independent of property order, and each batch response uses a single version.
+
+Insufficient evidence is stored alongside the model description.
+It does not prevent access to the original values or a conditional calculation, but explains the result's limitations.

package/docs/client.md

@@ -0,0 +1,49 @@
+# Client and request cost
+
+The client caches prices in process memory and reuses them for calculations.
+Each calculation returns quantities, rates, components, and a tariff snapshot so stored usage can be explained.
+
+Parameters and results are defined in the [client types](../src/types/client.d.ts) and [calculation types](../src/types/pricing.d.ts).
+A connection example is available in the [main description](../README.md).
+Calculation inputs can also be obtained separately as a [tariff snapshot](../src/pricing.js).
+
+The model resolution endpoint is `POST /v1/models/resolve` on the configured HTTPS origin.
+Set `baseUrl` to the origin, for example `https://catalog.example.com`, without `/v1/models/resolve`.
+The package client appends that path automatically; passing the full endpoint as `baseUrl` repeats the path.
+For direct HTTP requests, send `Authorization: Bearer <token>` and `Content-Type: application/json`:
+
+```http
+POST /v1/models/resolve HTTP/1.1
+Host: catalog.example.com
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{"models":["gpt-6-sol"]}
+```
+
+Startup loads the initial model set; requesting a new name retrieves its entry immediately.
+Concurrent duplicate requests are combined, and missing entries are cached until the next refresh.
+A forced refresh fetches all accumulated names again, including previously unknown names.
+
+Scheduled refreshes run around 1 a.m. and 1 p.m. in an explicitly configured time zone.
+Each process independently selects a random offset within twenty minutes for each window.
+Missed polls do not accumulate while the process sleeps; stopping the client cancels timers and requests.
+Scheduling is implemented in [time selection](../src/schedule.js), and failure handling in the [client](../src/client.js).
+
+A network failure preserves the previous entry with a staleness reason and the last successful retrieval time.
+If no data has been received, the client returns an unavailable state rather than reporting the model as missing.
+Retries after temporary failures are limited; access denial and invalid requests do not trigger automatic retries.
+For a corporate certificate, enable the Node.js system certificate store with `node --use-system-ca` if necessary;
+certificate verification does not need to be disabled.
+
+For text usage, ordinary input equals total input minus cache reads and writes.
+Reasoning tokens already included in output are not added again.
+Inconsistent quantities remain visible with an explanation; independent known components remain in the result.
+
+An unknown total has no numeric substitute; the known subtotal is returned separately.
+Calculations using stale or insufficiently supported prices remain numeric and include the reasons.
+Applications handle rounding for storage or display; catalog updates do not change historical snapshots.
+
+Audio and other non-text usage must use the corresponding unit.
+Quantity billing uses the rate for the specified unit; image matrices use the selected quality and size.
+A missing rate or mismatched unit never becomes a zero price.

package/docs/development.md

@@ -0,0 +1,32 @@
+# Development setup
+
+Install the Node.js version specified in `package.json`. Run `npm ci`, then `npm run quality` from the repository root.
+To describe servers, copy `servers.example.md` to `servers.local.md` and fill in local values.
+The private file is excluded from Git and is read by the agent only for operations that require connections.
+Service configuration and startup are described in the [operations guide](operations.md).
+
+## Checks
+
+| Command                           | Purpose                                           |
+| --------------------------------- | ------------------------------------------------- |
+| `npm test` or `npm run test:code` | Unit checks without external systems.             |
+| `npm run test:integration`        | Server and client checks over a local connection. |
+| `npm run lint`                    | Source analysis using `.oxlintrc.json`.           |
+| `npm run lint:fix`                | Automatic fixes and formatting.                   |
+| `npm run format:check`            | Formatting checks without changing files.         |
+| `npm run docs:check`              | Relative documentation link checks.               |
+| `npm run quality`                 | All checks above without automatic fixes.         |
+
+Review automatic fixes and rerun `npm run quality`.
+Formatting checks include Markdown, JSON, YAML, and type declarations.
+
+## Project settings
+
+Development rules are in the root `AGENTS.md`. `CLAUDE.md` imports only `AGENTS.md`.
+Code analysis uses `.oxlintrc.json`.
+The service runs JavaScript directly, without a build step or a TypeScript compiler.
+Public `.d.ts` declarations remain available for applications that use type checking.
+The service reads YAML settings and does not load `.env`.
+Use the deployment skill's `update` command to update an existing installation.
+Keep addresses, credentials, ports, certificate paths, and installation paths in private configuration.
+Public examples must use placeholders or reserved example domains.

package/docs/operations.md

@@ -0,0 +1,58 @@
+# Startup and operations
+
+The service endpoint, host, ports, and certificate settings are defined by each deployment.
+Clients call `POST /v1/models/resolve`, with a bearer access key and a JSON body such as
+`{"models":["gpt-6-sol"]}`. The package client takes only the HTTPS origin in `baseUrl` and adds this path.
+The deployment setup uses a loopback application listener and an existing nginx HTTPS site.
+Its managed snippet adds only `/v1/models/resolve` and `/model-catalog/health`; other routes remain owned by the site.
+The service requires neither databases nor calls to model providers.
+
+For deployment from a source checkout, use the project deployment skill with your own repository and server settings.
+Connection and application settings follow the [configuration example](../.agents/skills/deploy/config/config.example.yaml).
+The working catalog is `catalog.yaml` in the server project root and is excluded from Git.
+The example is `catalog.example.yaml` in the project root. Code updates preserve the working catalog.
+Service settings and keys are stored in `/etc/fa-model-catalog`.
+
+For a local run:
+
+```bash
+npm ci
+npm run catalog:check -- --file /path/to/catalog.yaml
+npm start -- --config /path/to/server.yaml
+```
+
+The server settings specify paths to the working catalog and access file.
+All consumers share one key; its sole local source is the private configuration file
+(.agents/skills/deploy/config/config.local.yaml), at access.clients[0].keys[0].token.
+Keys are created and validated by [fa-web-access](https://github.com/Bazilio-san/fa-web-access);
+never place a real key in an example or log.
+
+Manage the server from the repository root:
+
+```bash
+node .agents/skills/deploy/scripts/remote.js status
+node .agents/skills/deploy/scripts/remote.js check
+node .agents/skills/deploy/scripts/remote.js update
+node .agents/skills/deploy/scripts/remote.js verify
+node .agents/skills/deploy/scripts/remote.js push-catalog
+node .agents/skills/deploy/scripts/remote.js push-access
+```
+
+Catalog and access replacement commands validate their inputs before transfer and replace the working file atomically.
+Routine code updates preserve the catalog, keys, and settings.
+The complete command list is in the repository's deployment skill, which is not included in the installed client.
+
+Server code is installed only from Git. The wrapper selects the clean local Git commit and requires it to exist
+in the configured remote branch. Commit and push reviewed code after local quality checks before deployment.
+The wrapper refreshes the root `update.js` from that commit, then uses it for updates and forced restarts.
+The updater resets tracked files to the selected commit, preserves ignored settings and untracked data,
+installs production dependencies when needed, and checks settings, catalog, access, systemd, and readiness.
+It does not compile code or run database migrations. Failed updates leave a pending marker for the next retry.
+Use `--config <path>` to select another private deployment file. No code is uploaded from the working directory.
+
+Readiness is checked through `/health` on loopback and `/model-catalog/health` publicly.
+Model retrieval requires a key; request handling is described in the [server](../src/server.js).
+After updates, check the service, readiness, authenticated price retrieval, access denial without a key, and recent errors.
+
+Catalog data does not confirm current provider prices without source verification.
+Entry limitations remain until the data is updated with confirmed sources.

package/package.json

@@ -1,6 +1,64 @@
 {
   "name": "fa-model-catalog",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.1",
+  "description": "Shared model catalog, pricing service, and client with cost calculation",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/Bazilio-san/fa-model-catalog.git"
+  },
+  "bin": {
+    "fa-model-catalog": "src/cli/server.js"
+  },
+  "files": [
+    "src/*.js",
+    "src/types",
+    "src/cli",
+    "docs",
+    "catalog.example.yaml",
+    ".agents/skills/deploy/config/*.example.yaml",
+    "README.md",
+    "LICENSE"
+  ],
+  "type": "module",
+  "exports": {
+    "./client": {
+      "types": "./src/types/client.d.ts",
+      "import": "./src/client.js"
+    },
+    "./pricing": {
+      "types": "./src/types/pricing.d.ts",
+      "import": "./src/pricing.js"
+    },
+    "./server": {
+      "types": "./src/types/server.d.ts",
+      "import": "./src/server.js"
+    }
+  },
+  "scripts": {
+    "test": "npm run test:code",
+    "start": "node src/cli/server.js",
+    "catalog:check": "node src/cli/check.js",
+    "test:code": "node --test \"tests/unit/*.test.js\"",
+    "test:integration": "node --test \"tests/integration/*.test.js\"",
+    "lint": "oxlint src tests scripts .agents/skills update.js",
+    "lint:fix": "oxlint --fix src tests scripts .agents/skills update.js && npm run format",
+    "format": "oxfmt .",
+    "format:check": "oxfmt --check .",
+    "docs:check": "node scripts/checks-docs.js",
+    "quality": "npm run test:code && npm run test:integration && npm run lint && npm run format:check && npm run docs:check",
+    "prepublishOnly": "npm run quality"
+  },
+  "dependencies": {
+    "ajv": "8.20.0",
+    "fa-web-access": "0.1.2",
+    "yaml": "2.9.0"
+  },
+  "devDependencies": {
+    "oxfmt": "0.61.0",
+    "oxlint": "1.76.0"
+  },
+  "engines": {
+    "node": ">=22.12"
+  }
+}

package/src/catalog-schemas.js

@@ -0,0 +1,165 @@
+import Ajv from 'ajv';
+import { parseDocument } from 'yaml';
+import { createHash } from 'node:crypto';
+
+export const priceFields = [
+  'inp',
+  'inpC',
+  'inpW',
+  'out',
+  'inpF',
+  'inpCF',
+  'inpWF',
+  'outF',
+  'inpB',
+  'outB',
+  'inpFast',
+  'outFast',
+];
+const amount = { type: ['number', 'null'], minimum: 0 };
+const stringArray = { type: 'array', items: { type: 'string', minLength: 1 }, uniqueItems: true };
+const matrix = { type: 'object', minProperties: 1, additionalProperties: amount };
+export const modelSchema = {
+  type: 'object',
+  additionalProperties: false,
+  required: ['billing', 'modes', 'capabilities', 'evidence'],
+  properties: {
+    ...Object.fromEntries(priceFields.map((k) => [k, amount])),
+    kT: amount,
+    mot: amount,
+    ui: { type: 'boolean' },
+    dimensions: { type: 'array', items: { type: 'integer', minimum: 1 }, uniqueItems: true },
+    dimensionsAllowed: { type: 'boolean' },
+    low: matrix,
+    medium: matrix,
+    high: matrix,
+    modes: { type: 'array', minItems: 1, uniqueItems: true, items: { enum: ['standard', 'flex', 'batch', 'fast'] } },
+    billing: {
+      type: 'object',
+      additionalProperties: false,
+      required: ['kind', 'unit', 'per'],
+      properties: {
+        kind: { enum: ['tokens', 'quantity', 'matrix'] },
+        unit: { enum: ['tokens', 'characters', 'minutes', 'seconds', 'images'] },
+        per: { type: 'number', exclusiveMinimum: 0 },
+      },
+    },
+    capabilities: {
+      type: 'object',
+      additionalProperties: false,
+      required: ['promptCacheKey', 'explicitPromptCaching', 'reasoningEfforts'],
+      properties: {
+        promptCacheKey: { type: 'boolean' },
+        explicitPromptCaching: { type: 'boolean' },
+        reasoningEfforts: stringArray,
+      },
+    },
+    evidence: {
+      type: 'object',
+      additionalProperties: false,
+      required: ['status', 'reason', 'sources'],
+      properties: {
+        status: { enum: ['confirmed', 'insufficient'] },
+        reason: { type: ['string', 'null'] },
+        sources: stringArray,
+      },
+    },
+  },
+};
+const schema = {
+  type: 'object',
+  additionalProperties: false,
+  required: ['schemaVersion', 'currency', 'models'],
+  properties: {
+    schemaVersion: { const: 1 },
+    currency: { const: 'USD' },
+    models: {
+      type: 'object',
+      minProperties: 1,
+      propertyNames: { minLength: 1, maxLength: 512 },
+      additionalProperties: { anyOf: [{ type: 'string', minLength: 1 }, modelSchema] },
+    },
+  },
+};
+const ajv = new Ajv({ allErrors: true, strictNumbers: true, allowUnionTypes: true });
+const validate = ajv.compile(schema);
+export const validateModel = ajv.compile(modelSchema);
+
+export function freeze(value) {
+  if (value && typeof value === 'object' && !Object.isFrozen(value)) {
+    for (const child of Object.values(value)) {
+      freeze(child);
+    }
+    Object.freeze(value);
+  }
+  return value;
+}
+
+function sorted(value) {
+  if (Array.isArray(value)) {
+    return value.map(sorted);
+  }
+  if (value && typeof value === 'object') {
+    return Object.fromEntries(
+      Object.keys(value)
+        .sort()
+        .map((k) => [k, sorted(value[k])]),
+    );
+  }
+  return value;
+}
+
+export function parseCatalog(text) {
+  const doc = parseDocument(text, { uniqueKeys: true });
+  if (doc.errors.length) {
+    throw new Error('The catalog file contains a YAML error or a duplicate key.');
+  }
+  const data = doc.toJS({ maxAliasCount: 0 });
+  if (!validate(data)) {
+    throw new Error(`Invalid catalog structure: ${ajv.errorsText(validate.errors)}.`);
+  }
+  const resolved = Object.create(null);
+  for (const key of Object.keys(data.models)) {
+    const seen = new Set();
+    let target = key;
+    while (typeof data.models[target] === 'string') {
+      if (seen.has(target)) {
+        throw new Error(`Cyclic model reference: "${key}".`);
+      }
+      seen.add(target);
+      target = data.models[target];
+      if (!Object.hasOwn(data.models, target)) {
+        throw new Error(`Reference target not found for model "${key}".`);
+      }
+    }
+    const model = data.models[target];
+    if (model.billing.kind === 'tokens' && model.billing.unit !== 'tokens') {
+      throw new Error('Invalid token billing unit.');
+    }
+    if (model.billing.kind === 'matrix' && model.billing.unit !== 'images') {
+      throw new Error('Invalid image matrix billing unit.');
+    }
+    if (model.evidence.status === 'insufficient' && !model.evidence.reason) {
+      throw new Error('A reason for insufficient evidence is required.');
+    }
+    for (const mode of model.modes) {
+      const fields = {
+        standard: ['inp', 'out'],
+        flex: ['inpF', 'outF'],
+        batch: ['inpB', 'outB'],
+        fast: ['inpFast', 'outFast'],
+      }[mode];
+      if (model.billing.kind !== 'matrix' && fields.some((f) => !Object.hasOwn(model, f))) {
+        throw new Error(`Missing rates for model "${target}".`);
+      }
+    }
+    resolved[key] = { canonicalName: target, model };
+  }
+  return freeze({
+    ...data,
+    resolved,
+    catalogVersion: createHash('sha256')
+      .update(JSON.stringify(sorted(data)))
+      .digest('hex'),
+  });
+}

package/src/catalog.js

@@ -0,0 +1,108 @@
+import { readFile } from 'node:fs/promises';
+import { parseCatalog } from './catalog-schemas.js';
+
+export function resolveModels(snapshot, names) {
+  return names.map((requestedName) => {
+    let matchedName = requestedName;
+    if (!Object.hasOwn(snapshot.resolved, matchedName) && matchedName.includes('@')) {
+      matchedName = matchedName.slice(matchedName.indexOf('@') + 1);
+    }
+    const found = Object.hasOwn(snapshot.resolved, matchedName) ? snapshot.resolved[matchedName] : null;
+    return {
+      requestedName,
+      matchedName: found ? matchedName : null,
+      canonicalName: found?.canonicalName ?? null,
+      status: found ? 'found' : 'not_found',
+      model: found?.model ?? null,
+      catalogVersion: snapshot.catalogVersion,
+    };
+  });
+}
+
+export function createCatalogStore({ file, reloadIntervalMs = 5000, onEvent = () => {} }) {
+  let snapshot;
+  let timer;
+  let stopped = false;
+  let pending;
+  let error = null;
+  const emit = (event) => {
+    try {
+      onEvent(event);
+    } catch {
+      /* Observer cannot change state. */
+    }
+  };
+  const reload = () => {
+    if (stopped) {
+      return Promise.reject(new Error('The catalog has been stopped.'));
+    }
+    if (pending) {
+      return pending;
+    }
+    pending = (async () => {
+      try {
+        const text = await readFile(file, 'utf8');
+        if (Buffer.byteLength(text) > 8 * 1024 * 1024) {
+          throw new Error('The catalog exceeds 8 MiB.');
+        }
+        const next = parseCatalog(text);
+        if (stopped) {
+          return false;
+        }
+        const changed = snapshot?.catalogVersion !== next.catalogVersion;
+        if (changed || error) {
+          emit({
+            event: 'catalog_applied',
+            catalogVersion: next.catalogVersion,
+            count: Object.keys(next.models).length,
+          });
+        }
+        snapshot = next;
+        error = null;
+        return changed;
+      } catch (e) {
+        if (!stopped) {
+          error = 'Could not read or validate the catalog.';
+          emit({ event: 'catalog_reload_failed', reason: error });
+        }
+        throw e;
+      } finally {
+        pending = null;
+      }
+    })();
+    return pending;
+  };
+  const schedule = () => {
+    if (stopped) {
+      return;
+    }
+    timer = setTimeout(async () => {
+      try {
+        await reload();
+      } catch {
+        /* Keep last good snapshot. */
+      }
+      schedule();
+    }, reloadIntervalMs);
+    timer.unref();
+  };
+  return {
+    async load() {
+      await reload();
+      if (!timer) {
+        schedule();
+      }
+    },
+    reload,
+    get snapshot() {
+      return snapshot;
+    },
+    get error() {
+      return error;
+    },
+    stop() {
+      stopped = true;
+      clearTimeout(timer);
+    },
+  };
+}

package/src/cli/check.js

@@ -0,0 +1,12 @@
+#!/usr/bin/env node
+import { readFile } from 'node:fs/promises';
+import { parseArgs } from 'node:util';
+import { parseCatalog } from '../catalog-schemas.js';
+try {
+  const { values } = parseArgs({ options: { file: { type: 'string', default: 'catalog.example.yaml' } } });
+  const data = parseCatalog(await readFile(values.file, 'utf8'));
+  console.log(`Catalog validated: ${Object.keys(data.models).length} names; version ${data.catalogVersion}.`);
+} catch (e) {
+  console.error(e.message);
+  process.exitCode = 1;
+}

package/src/cli/server.js

@@ -0,0 +1,49 @@
+#!/usr/bin/env node
+import { readFile } from 'node:fs/promises';
+import { resolve, dirname } from 'node:path';
+import { parseArgs } from 'node:util';
+import { parseDocument } from 'yaml';
+import { createCatalogServer } from '../server.js';
+try {
+  const { values } = parseArgs({
+    options: { config: { type: 'string', default: '.agents/skills/deploy/config/config.local.yaml' } },
+  });
+  const file = resolve(values.config);
+  const doc = parseDocument(await readFile(file, 'utf8'), { uniqueKeys: true });
+  if (doc.errors.length) {
+    throw new Error();
+  }
+  const document = doc.toJS({ maxAliasCount: 0 });
+  const config = document.server ?? document;
+  const allowed = ['catalogFile', 'accessFile', 'host', 'port', 'reloadIntervalMs'];
+  if (
+    !config ||
+    Object.keys(config).some((k) => !allowed.includes(k)) ||
+    !Number.isInteger(config.port) ||
+    config.port < 1 ||
+    config.port > 65535 ||
+    typeof config.host !== 'string'
+  ) {
+    throw new Error();
+  }
+  const app = createCatalogServer({
+    ...config,
+    catalogFile: resolve(dirname(file), config.catalogFile),
+    accessFile: resolve(dirname(file), config.accessFile),
+    onEvent: (e) => console.log(JSON.stringify(e)),
+  });
+  await app.listen(config);
+  console.log(`Service ready; port ${config.port}.`);
+  let closing = false;
+  for (const signal of ['SIGINT', 'SIGTERM']) {
+    process.on(signal, async () => {
+      if (!closing) {
+        closing = true;
+        await app.close();
+      }
+    });
+  }
+} catch {
+  console.error('Could not start the service: check the settings, catalog, and access file.');
+  process.exitCode = 1;
+}