llm-models 1.0.0 → 1.2.2

package/README.md

@@ -2,7 +2,7 @@
 
 Fetch latest LLM models from [OpenRouter](https://openrouter.ai) and [models.dev](https://models.dev) APIs.
 
-Validates API responses with [Zod](https://zod.dev) schemas to detect breaking changes.
+Validates API responses with [Zod](https://zod.dev) schemas to detect breaking changes. Provides smart wrapper functions for model discovery, comparison, cost estimation, recommendations, statistics, and provider info. Features local caching, MCP server integration, multiple output formats, and a fluent query builder API.
 
 ## Installation
 
@@ -32,60 +32,315 @@ Download the latest binary from [GitHub Releases](https://github.com/maxgfr/llm-
 
 ## CLI Usage
 
-### OpenRouter
+### Smart Commands
+
+#### Find models
+
+Discover models across both APIs with smart filtering:
+
+```bash
+# Find cheapest reasoning models
+llm-models find -C reasoning --sort cost_input -n 10
+
+# Find models accepting images, sorted by cost
+llm-models find -m image --sort cost_input
+
+# Find OpenAI models with large context
+llm-models find -p openai --min-context 200000
+
+# Filter by family
+llm-models find -f gpt --sort cost_input
+
+# Filter by status
+llm-models find --status active -C reasoning
+
+# Search by name
+llm-models find -s "claude" --sort cost_input
+
+# Sort by cost-effectiveness (context per dollar)
+llm-models find --sort value --desc -n 10
+
+# Count matching models
+llm-models find -C reasoning --count
+
+# Output IDs only (for piping)
+llm-models find -p openai --ids-only
+
+# Output as CSV or Markdown
+llm-models find -p anthropic --format csv
+llm-models find -p anthropic --format markdown
+
+# Output as JSON
+llm-models find -p google --json
+```
+
+#### Compare models
+
+Side-by-side model comparison with context, output limit, costs, knowledge cutoff, and capabilities:
+
+```bash
+llm-models compare openai/gpt-4o anthropic/claude-sonnet-4 google/gemini-2.0-flash
+```
+
+#### Model info
+
+Detailed information about a single model (description, costs, capabilities, parameters, tokenizer):
+
+```bash
+llm-models info openai/gpt-4o
+```
+
+#### Cost estimation
+
+Estimate costs for specific token volumes:
+
+```bash
+# Basic cost estimation
+llm-models cost openai/gpt-4o anthropic/claude-sonnet-4 -i 1M -o 100K
+
+# With daily/monthly projections
+llm-models cost openai/gpt-4o -i 1M -o 100K --daily 1000
+
+# Using a workload profile
+llm-models cost openai/gpt-4o -i 1K -o 1K -P chatbot --daily 5000
+```
+
+#### Cheapest models
+
+Find the cheapest models matching your criteria:
+
+```bash
+llm-models cheapest -C reasoning --min-context 100000
+```
+
+#### Recommendations
+
+Get model recommendations for a specific use case:
+
+```bash
+# List available use cases
+llm-models recommend --list
+
+# Get recommendations
+llm-models recommend reasoning
+llm-models recommend code-gen --max-cost 5
+llm-models recommend vision -n 5
+```
+
+Available use cases: `code-gen`, `vision`, `cheap-chatbot`, `reasoning`, `long-context`, `open-source`, `audio`, `tool-use`.
+
+#### Statistics
+
+Get aggregate statistics about the LLM model landscape:
+
+```bash
+llm-models stats
+llm-models stats --json
+```
+
+Shows: total models/providers, top providers, capabilities distribution, modality distribution, cost percentiles, context length stats, newest models.
+
+#### Diff
+
+Show changes since last cached fetch:
 
 ```bash
-# List all models
-llm-models openrouter
+llm-models diff
+```
 
-# Filter by provider
-llm-models openrouter --provider google
+Shows: new models added, models removed, price changes, status changes.
 
-# Search by name or ID
-llm-models openrouter --search "claude"
+#### Provider info
+
+Get provider details (SDK package, env vars, API endpoint, models):
+
+```bash
+# List all providers
+llm-models provider --list
 
-# Show model count
-llm-models openrouter --count
+# Get details for a specific provider
+llm-models provider openai
 ```
 
-### models.dev
+### Raw API Commands
+
+#### OpenRouter
+
+```bash
+llm-models openrouter                     # All models (JSON)
+llm-models openrouter --provider google   # Filter by provider
+llm-models openrouter --search "claude"   # Search
+llm-models openrouter --count             # Count only
+```
+
+#### models.dev
 
 ```bash
-# List all providers and models
-llm-models models-dev
+llm-models models-dev                     # All providers (JSON)
+llm-models models-dev --provider openai   # Filter by provider
+llm-models models-dev --count             # Count only
+```
 
-# Filter by provider
-llm-models models-dev --provider openai
+### Utility Commands
 
-# Search models
-llm-models models-dev --search "gpt-4"
+#### Cache management
 
-# Show count
-llm-models models-dev --count
+```bash
+llm-models cache              # Show cache info
+llm-models cache clear        # Clear cache
+```
+
+API responses are cached for 1 hour in `~/.cache/llm-models/`. Use `--no-cache` to bypass.
+
+#### Configuration
+
+```bash
+llm-models config             # Show current config
+llm-models config init        # Create default config file
+llm-models config path        # Show config file path
+llm-models config profiles    # List workload profiles
 ```
 
-### Options
+Config file locations: `.llm-modelsrc` (project) or `~/.config/llm-models/config.json` (global).
+
+#### Shell completions
+
+```bash
+# Bash
+llm-models completion bash >> ~/.bashrc
+
+# Zsh
+llm-models completion zsh >> ~/.zshrc
+
+# Fish
+llm-models completion fish > ~/.config/fish/completions/llm-models.fish
+```
+
+#### MCP Server
+
+Start an MCP (Model Context Protocol) server for AI agent integration:
+
+```bash
+llm-models mcp
+```
+
+Exposes 8 tools: `find_models`, `compare_models`, `estimate_cost`, `cheapest_models`, `get_provider`, `list_providers`, `get_stats`, `recommend_models`.
+
+### Global Options
 
 | Option | Description |
 |--------|-------------|
-| `-p, --provider <id>` | Filter by provider prefix/ID |
-| `-s, --search <term>` | Search models by name or ID |
-| `-c, --count` | Show count only |
+| `--no-cache` | Disable cache for this request |
+| `--quiet` | Suppress informational messages |
+| `--verbose` | Show debug info (timing, cache status) |
 | `-V, --version` | Show version |
 | `-h, --help` | Show help |
 
+### Find Options
+
+| Option | Description |
+|--------|-------------|
+| `-p, --provider <id>` | Filter by provider |
+| `-C, --capability <cap>` | Filter: reasoning, tool_call, structured_output, open_weights, attachment |
+| `-m, --modality <mod>` | Filter by input modality (image, audio, video, pdf) |
+| `--max-cost <n>` | Max input cost ($/million tokens) |
+| `--max-cost-output <n>` | Max output cost ($/million tokens) |
+| `--min-context <n>` | Min context window |
+| `-s, --search <term>` | Search by name or ID |
+| `--status <status>` | Filter: active, beta, deprecated |
+| `-f, --family <name>` | Filter by model family (e.g. gpt, claude, gemini) |
+| `--sort <field>` | Sort: cost_input, cost_output, context_length, release_date, name, knowledge_cutoff, value |
+| `--desc` | Sort descending |
+| `-n, --limit <n>` | Max results (default: 20) |
+| `-c, --count` | Show model count only |
+| `--ids-only` | Output model IDs only, one per line |
+| `--json` | Raw JSON output |
+| `--format <fmt>` | Output format: table, json, csv, markdown |
+
+### Cost Options
+
+| Option | Description |
+|--------|-------------|
+| `-i, --input <n>` | Input tokens (supports K/M suffix) |
+| `-o, --output <n>` | Output tokens (supports K/M suffix) |
+| `--daily <n>` | Number of daily requests for cost projection |
+| `--monthly <n>` | Number of monthly requests for cost projection |
+| `-P, --profile <name>` | Use workload profile (chatbot, code-gen, rag, summarization, translation) |
+
 ## Library Usage
 
+```typescript
+import {
+  fetchUnifiedModels,
+  findModels,
+  compareModels,
+  getProvider,
+  listProviders,
+  estimateCost,
+  cheapestModels,
+  getStats,
+  recommendModels,
+  listUseCases,
+  diffModels,
+  query,
+  setCacheEnabled,
+  clearCache,
+} from "llm-models";
+
+// Get all models (unified from both APIs)
+const models = await fetchUnifiedModels();
+
+// Find cheapest reasoning models
+const cheap = await findModels({
+  filter: { capability: "reasoning" },
+  sort: "cost_input",
+  limit: 5,
+});
+
+// Fluent query builder
+const results = await query()
+  .provider("openai")
+  .capability("reasoning")
+  .maxCost(5)
+  .sortBy("cost_input")
+  .limit(10)
+  .execute();
+
+// Compare models side by side
+const comparison = await compareModels(["openai/gpt-4o", "anthropic/claude-sonnet-4"]);
+
+// Get provider info (SDK, env vars, docs, models)
+const openai = await getProvider("openai");
+console.log(openai.npm); // "@ai-sdk/openai"
+
+// Estimate costs
+const costs = await estimateCost({
+  modelIds: ["openai/gpt-4o"],
+  inputTokens: 1_000_000,
+  outputTokens: 100_000,
+});
+
+// Get recommendations for a use case
+const recs = await recommendModels("code-gen", { maxCost: 5, limit: 5 });
+
+// Get market statistics
+const stats = await getStats();
+
+// Detect changes since last fetch
+const diff = await diffModels();
+
+// Cache control
+setCacheEnabled(false); // Disable caching
+clearCache();           // Clear cache directory
+```
+
+### Raw API Access
+
 ```typescript
 import { fetchOpenRouterModels, fetchModelsDevModels } from "llm-models";
 
-// Fetch from OpenRouter
 const openrouter = await fetchOpenRouterModels();
-console.log(openrouter.data.length, "models");
-
-// Fetch from models.dev
 const modelsDev = await fetchModelsDevModels();
-console.log(Object.keys(modelsDev).length, "providers");
 ```
 
 ### Zod Schemas
@@ -93,12 +348,7 @@ console.log(Object.keys(modelsDev).length, "providers");
 All schemas are exported for custom validation:
 
 ```typescript
-import { OpenRouterResponseSchema, ModelsDevResponseSchema } from "llm-models";
-
-const result = OpenRouterResponseSchema.safeParse(myData);
-if (!result.success) {
-  console.error(result.error.issues);
-}
+import { UnifiedModelSchema, OpenRouterResponseSchema, ModelsDevResponseSchema } from "llm-models";
 ```
 
 ### Types
@@ -106,7 +356,29 @@ if (!result.success) {
 All types are inferred from Zod schemas:
 
 ```typescript
-import type { OpenRouterModel, ModelsDevModel, ModelsDevProvider } from "llm-models";
+import type {
+  UnifiedModel,
+  NormalizedCost,
+  Capabilities,
+  ProviderInfo,
+  ModelFilter,
+  ModelSortField,
+  CostEstimate,
+  ModelComparison,
+} from "llm-models";
+```
+
+## GitHub Action
+
+Use llm-models in CI/CD to check model costs and status:
+
+```yaml
+- uses: maxgfr/llm-models@v1
+  with:
+    command: cost
+    args: "openai/gpt-4o anthropic/claude-sonnet-4 -i 1M -o 100K"
+    max-budget: "10"
+    fail-on-deprecated: "true"
 ```
 
 ## Data Sources

package/build/cache.d.ts

@@ -0,0 +1,13 @@
+export declare function setCacheEnabled(enabled: boolean): void;
+export declare function isCacheEnabled(): boolean;
+export declare function readCache<T>(key: string, ttlMs?: number): T | null;
+export declare function writeCache<T>(key: string, data: T): void;
+export declare function readSnapshot<T>(key: string): {
+    timestamp: number;
+    data: T;
+} | null;
+export declare function clearCache(): void;
+export declare function getCacheInfo(): {
+    files: number;
+    sizeBytes: number;
+};

package/build/colors.d.ts

@@ -0,0 +1,10 @@
+export declare const bold: (text: string) => string;
+export declare const dim: (text: string) => string;
+export declare const green: (text: string) => string;
+export declare const yellow: (text: string) => string;
+export declare const red: (text: string) => string;
+export declare const cyan: (text: string) => string;
+export declare const gray: (text: string) => string;
+export declare function colorizeCost(cost: number | undefined | null, formatted: string): string;
+export declare function stripAnsi(str: string): number;
+export declare function isColorEnabled(): boolean;

package/build/completions.d.ts

@@ -0,0 +1,3 @@
+export declare function generateBashCompletion(): string;
+export declare function generateZshCompletion(): string;
+export declare function generateFishCompletion(): string;

package/build/config.d.ts

@@ -0,0 +1,22 @@
+export interface WorkloadProfile {
+    input: string;
+    output: string;
+}
+export interface LlmModelsConfig {
+    cache?: {
+        enabled?: boolean;
+        ttl?: number;
+    };
+    defaults?: {
+        format?: "table" | "json" | "csv" | "markdown";
+        limit?: number;
+        sort?: string;
+        desc?: boolean;
+    };
+    profiles?: Record<string, WorkloadProfile>;
+}
+export declare function loadConfig(): LlmModelsConfig;
+export declare function getProfile(name: string): WorkloadProfile | undefined;
+export declare function listProfiles(): Record<string, WorkloadProfile>;
+export declare function initConfig(): void;
+export declare function getConfigPath(): string;

package/build/format.d.ts

@@ -0,0 +1,11 @@
+export declare function formatNumber(n: number): string;
+export declare function formatCost(cost: number | undefined | null): string;
+export declare function formatCostRaw(cost: number | undefined | null): string;
+export declare function formatContext(n: number): string;
+export declare function parseTokenCount(input: string): number;
+export declare function formatCapabilities(caps: Record<string, boolean | undefined>): string;
+export declare function printTable(headers: string[], rows: string[][]): void;
+export type OutputFormat = "table" | "json" | "csv" | "markdown";
+export declare function formatAsCSV(headers: string[], rows: string[][]): string;
+export declare function formatAsMarkdown(headers: string[], rows: string[][]): string;
+export declare function outputFormatted(format: OutputFormat, headers: string[], rows: string[][], jsonData?: unknown): void;

package/build/functions/compare.d.ts

@@ -0,0 +1,2 @@
+import type { ModelComparison } from "../types";
+export declare function compareModels(modelIds: string[]): Promise<ModelComparison>;

package/build/functions/cost.d.ts

@@ -0,0 +1,11 @@
+import type { Capabilities, CostEstimate, UnifiedModel } from "../types";
+export declare function estimateCost(options: {
+    modelIds: string[];
+    inputTokens: number;
+    outputTokens: number;
+}): Promise<CostEstimate[]>;
+export declare function cheapestModels(options?: {
+    minContext?: number;
+    capability?: keyof Capabilities;
+    limit?: number;
+}): Promise<UnifiedModel[]>;

package/build/functions/diff.d.ts

@@ -0,0 +1,18 @@
+import type { UnifiedModel } from "../types";
+export interface ModelChange {
+    model_id: string;
+    field: string;
+    old_value: string | number | null;
+    new_value: string | number | null;
+}
+export interface ModelDiff {
+    timestamp_previous: number | null;
+    timestamp_current: number;
+    added: UnifiedModel[];
+    removed: string[];
+    price_changes: ModelChange[];
+    status_changes: ModelChange[];
+    total_before: number;
+    total_after: number;
+}
+export declare function diffModels(): Promise<ModelDiff>;

package/build/functions/index.d.ts

@@ -0,0 +1,9 @@
+export { compareModels } from "./compare";
+export { cheapestModels, estimateCost } from "./cost";
+export { diffModels } from "./diff";
+export { fetchUnifiedModels } from "./normalize";
+export { getProvider, listProviders } from "./provider";
+export { QueryBuilder, query } from "./query";
+export { listUseCases, recommendModels } from "./recommend";
+export { filterModels, findModels, sortModels } from "./search";
+export { getStats } from "./stats";

package/build/functions/normalize.d.ts

@@ -0,0 +1,5 @@
+import type { ModelsDevModel, OpenRouterModel, UnifiedModel } from "../types";
+export declare function openRouterPriceToPerMillion(perToken: string): number;
+export declare function normalizeOpenRouterModel(model: OpenRouterModel): UnifiedModel;
+export declare function normalizeModelsDevModel(model: ModelsDevModel, providerId: string): UnifiedModel;
+export declare function fetchUnifiedModels(): Promise<UnifiedModel[]>;

package/build/functions/provider.d.ts

@@ -0,0 +1,3 @@
+import type { ProviderInfo } from "../types";
+export declare function getProvider(providerId: string): Promise<ProviderInfo>;
+export declare function listProviders(): Promise<ProviderInfo[]>;

package/build/functions/query.d.ts

@@ -0,0 +1,20 @@
+import type { Capabilities, ModelSortField, UnifiedModel } from "../types";
+export declare class QueryBuilder {
+    private filter;
+    private sortField?;
+    private sortDesc;
+    private maxResults?;
+    provider(id: string): this;
+    capability(cap: keyof Capabilities): this;
+    modality(mod: string): this;
+    maxCost(input: number): this;
+    maxCostOutput(output: number): this;
+    minContext(ctx: number): this;
+    search(term: string): this;
+    status(s: "active" | "deprecated" | "beta"): this;
+    family(f: string): this;
+    sortBy(field: ModelSortField, descending?: boolean): this;
+    limit(n: number): this;
+    execute(): Promise<UnifiedModel[]>;
+}
+export declare function query(): QueryBuilder;

package/build/functions/recommend.d.ts

@@ -0,0 +1,16 @@
+import type { ModelFilter, ModelSortField, UnifiedModel } from "../types";
+export interface UseCasePreset {
+    description: string;
+    filter: ModelFilter;
+    sort: ModelSortField;
+    descending?: boolean;
+}
+export declare function listUseCases(): Record<string, UseCasePreset>;
+export declare function recommendModels(useCase: string, options?: {
+    limit?: number;
+    maxCost?: number;
+    minContext?: number;
+}): Promise<{
+    preset: UseCasePreset;
+    models: UnifiedModel[];
+}>;

package/build/functions/search.d.ts

@@ -0,0 +1,9 @@
+import type { ModelFilter, ModelSortField, UnifiedModel } from "../types";
+export declare function filterModels(models: UnifiedModel[], filter: ModelFilter): UnifiedModel[];
+export declare function sortModels(models: UnifiedModel[], field: ModelSortField, descending?: boolean): UnifiedModel[];
+export declare function findModels(options: {
+    filter?: ModelFilter;
+    sort?: ModelSortField;
+    descending?: boolean;
+    limit?: number;
+}): Promise<UnifiedModel[]>;

package/build/functions/stats.d.ts

@@ -0,0 +1,38 @@
+export interface ModelStats {
+    total_models: number;
+    total_providers: number;
+    by_provider: {
+        provider: string;
+        count: number;
+    }[];
+    by_capability: Record<string, number>;
+    by_modality: Record<string, number>;
+    cost: {
+        input: {
+            min: number;
+            max: number;
+            median: number;
+            p25: number;
+            p75: number;
+            p90: number;
+        };
+        output: {
+            min: number;
+            max: number;
+            median: number;
+            p25: number;
+            p75: number;
+            p90: number;
+        };
+    } | null;
+    newest_models: {
+        id: string;
+        release_date: string;
+    }[];
+    context_length: {
+        min: number;
+        max: number;
+        median: number;
+    };
+}
+export declare function getStats(): Promise<ModelStats>;

package/build/index.d.ts

@@ -1,6 +1,10 @@
 #!/usr/bin/env node
-export { fetchOpenRouterModels } from "./clients/openrouter";
+export { clearCache, setCacheEnabled } from "./cache";
 export { fetchModelsDevModels } from "./clients/models-dev";
-export { OpenRouterModelSchema, OpenRouterResponseSchema, OpenRouterArchitectureSchema, OpenRouterPricingSchema, OpenRouterTopProviderSchema, OpenRouterDefaultParametersSchema, OpenRouterLinksSchema, } from "./schemas/openrouter";
-export { ModelsDevModelSchema, ModelsDevProviderSchema, ModelsDevResponseSchema, ModelsDevCostSchema, ModelsDevLimitSchema, ModelsDevModalitiesSchema, } from "./schemas/models-dev";
-export type { OpenRouterModel, OpenRouterResponse, ModelsDevModel, ModelsDevProvider, ModelsDevResponse, } from "./types";
+export { fetchOpenRouterModels } from "./clients/openrouter";
+export { getProfile, listProfiles, loadConfig } from "./config";
+export { cheapestModels, compareModels, diffModels, estimateCost, fetchUnifiedModels, filterModels, findModels, getProvider, getStats, listProviders, listUseCases, QueryBuilder, query, recommendModels, sortModels, } from "./functions/index";
+export { CapabilitiesSchema, CostEstimateSchema, ModelComparisonSchema, ModelFilterSchema, ModelSortFieldSchema, NormalizedCostSchema, ProviderInfoSchema, UnifiedModelSchema, } from "./schemas/functions";
+export { ModelsDevCostSchema, ModelsDevLimitSchema, ModelsDevModalitiesSchema, ModelsDevModelSchema, ModelsDevProviderSchema, ModelsDevResponseSchema, } from "./schemas/models-dev";
+export { OpenRouterArchitectureSchema, OpenRouterDefaultParametersSchema, OpenRouterLinksSchema, OpenRouterModelSchema, OpenRouterPricingSchema, OpenRouterResponseSchema, OpenRouterTopProviderSchema, } from "./schemas/openrouter";
+export type { Capabilities, CostEstimate, ModelComparison, ModelFilter, ModelSortField, ModelsDevModel, ModelsDevProvider, ModelsDevResponse, NormalizedCost, OpenRouterModel, OpenRouterResponse, ProviderInfo, UnifiedModel, } from "./types";