noosphere 0.8.0 → 0.9.1

package/README.md

@@ -7,6 +7,8 @@ One import. Every model. Every modality.
 ## Features
 
 - **7 modalities** — LLM, image, video, TTS, STT, music, and embeddings
+- **OpenAI media** — GPT-Image-1/1.5, DALL-E 2/3, Sora 2/Pro (video), TTS-1/HD, Whisper — all auto-fetched from `OPENAI_API_KEY`
+- **Google media** — Imagen 4.0 (image), Veo 2/3/3.1 (video), Gemini TTS — all auto-fetched from `GEMINI_API_KEY`
 - **Always up-to-date models** — Dynamic auto-fetch from ALL provider APIs at runtime (OpenAI, Anthropic, Google, Groq, Mistral, xAI, Cerebras, OpenRouter)
 - **Dynamic descriptions** — Model descriptions fetched from source (Ollama library, HuggingFace READMEs, CivitAI API) — no hardcoded strings
 - **Modality-filtered sync** — `syncModels('llm')` only fetches LLM providers, avoiding unnecessary requests
@@ -14,6 +16,8 @@ One import. Every model. Every modality.
 - **30+ HuggingFace tasks** — LLM, image, TTS, translation, summarization, classification, and more
 - **Local-first architecture** — Auto-detects Ollama, ComfyUI, Whisper, AudioCraft, Piper, and Kokoro on your machine
 - **Org-aware logos** — HuggingFace models show the real org logo (Meta, Google, NVIDIA) instead of generic HF logo
+- **Pre-request token counting** — Count tokens before sending, for ALL providers (OpenAI/Groq/Ollama via tiktoken, Google/Anthropic via API)
+- **Full pi-ai access** — Agent loop with tool calling, preprocessor (compaction hook), `calculateCost`, direct stream/complete APIs — all re-exported
 - **Agentic capabilities** — Tool use, function calling, reasoning/thinking, vision, and agent loops via Pi-AI
 - **Failover & retry** — Automatic retries with exponential backoff and cross-provider failover
 - **Usage tracking** — Real-time cost, latency, and token tracking across all providers
@@ -38,13 +42,29 @@ const response = await ai.chat({
 });
 console.log(response.content);
 
-// Generate an image
+// Generate an image with GPT-Image-1 (OpenAI) — just needs OPENAI_API_KEY
 const image = await ai.image({
   prompt: 'A sunset over mountains',
+  provider: 'openai-media',
+});
+// image.buffer contains the PNG data
+
+// Generate an image with Imagen 4.0 (Google) — just needs GEMINI_API_KEY
+const googleImage = await ai.image({
+  prompt: 'A sunset over mountains',
+  provider: 'google-media',
+});
+// googleImage.buffer contains the PNG data
+
+// Generate an image with DALL-E 3
+const dalle = await ai.image({
+  prompt: 'A sunset over mountains',
+  provider: 'openai-media',
+  model: 'dall-e-3',
   width: 1024,
   height: 1024,
 });
-console.log(image.url);
+console.log(dalle.url);
 
 // Generate a video
 const video = await ai.video({
@@ -53,7 +73,7 @@ const video = await ai.video({
 });
 console.log(video.url);
 
-// Text-to-speech
+// Text-to-speech with OpenAI TTS — just needs OPENAI_API_KEY
 const audio = await ai.speak({
   text: 'Welcome to Noosphere',
   voice: 'alloy',
@@ -368,6 +388,8 @@ await ai.uninstallModel('deepseek-r1:14b');
 | Provider | Modality | Models | Source | Auto-Detect |
 |---|---|---|---|---|
 | **pi-ai** | LLM | 482 | OpenAI, Anthropic, Google, Groq, Mistral, xAI, OpenRouter, Cerebras | API keys |
+| **openai-media** | image, video, tts, stt | 12 | GPT-Image-1/1.5, DALL-E 2/3, Sora 2/Pro, TTS-1/HD, Whisper | `OPENAI_API_KEY` |
+| **google-media** | image, video, tts | 10 | Imagen 4.0, Veo 2/3/3.1, Gemini TTS (Flash/Pro) | `GEMINI_API_KEY` |
 | **ollama** | LLM, embedding | 70 | 38 installed + 32 from Ollama web catalog | `localhost:11434` |
 | **hf-local** | image, video, tts, stt, music | 220 | HuggingFace catalog (FLUX, SDXL, Wan2.2, Whisper, MusicGen) | Always (no API key) |
 | **huggingface** | LLM, image, tts | dynamic | HuggingFace Inference API | `HUGGINGFACE_TOKEN` |
@@ -400,11 +422,11 @@ await ai.syncModels();
 
 | Modality | Providers Synced |
 |---|---|
-| `llm` | pi-ai, ollama, openai-compat, huggingface (cloud, needs API key) |
-| `image` | hf-local, comfyui, fal, huggingface (cloud) |
-| `video` | hf-local, comfyui, fal |
-| `tts` | hf-local (speech models), fal, piper, kokoro, huggingface (cloud) |
-| `stt` | hf-local, whisper-local |
+| `llm` | pi-ai, ollama, openai-compat, huggingface (cloud) |
+| `image` | **openai-media** (GPT-Image-1, DALL-E), **google-media** (Imagen 4.0), hf-local, comfyui, fal, huggingface (cloud) |
+| `video` | **openai-media** (Sora 2/Pro), **google-media** (Veo 2/3/3.1), hf-local, comfyui, fal |
+| `tts` | **openai-media** (TTS-1, TTS-1-HD), **google-media** (Gemini TTS), hf-local, fal, piper, kokoro, huggingface (cloud) |
+| `stt` | **openai-media** (Whisper), hf-local, whisper-local |
 | `music` | hf-local (MusicGen, AudioLDM, etc.), audiocraft |
 | `embedding` | ollama |
 
@@ -609,6 +631,102 @@ Noosphere auto-detects all local runtimes on startup:
 
 ---
 
+## Pre-Request Token Counting
+
+Count tokens **before** sending a request to any provider. Know the cost upfront.
+
+```typescript
+// Via Noosphere instance (auto-routes by model)
+const result = await ai.countTokens({
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'Explain quantum computing.' },
+  ],
+  model: 'gpt-4o',
+});
+console.log(result.tokens);   // 26
+console.log(result.method);   // "tiktoken" (instant, local)
+console.log(result.provider); // "openai"
+
+// Google — exact count via API
+const google = await ai.countTokens({
+  messages: [{ role: 'user', content: 'Hello!' }],
+  model: 'gemini-2.5-flash',
+});
+console.log(google.tokens);   // 3
+console.log(google.method);   // "api" (exact)
+```
+
+**Token counting by provider:**
+
+| Provider | Method | Speed | Accuracy |
+|---|---|---|---|
+| **OpenAI** (GPT-4o, o1, o3, o4, GPT-5) | tiktoken (local) | Instant | Exact |
+| **Google** (Gemini) | `/countTokens` API | ~200ms | Exact |
+| **Anthropic** (Claude) | `/messages/count_tokens` API | ~200ms | Exact |
+| **Groq** (Llama, Mixtral, Gemma) | tiktoken (local) | Instant | Exact |
+| **Cerebras** (Llama) | tiktoken (local) | Instant | Exact |
+| **Mistral** (Mistral, Mixtral, Codestral) | tiktoken (local) | Instant | Close approx |
+| **xAI** (Grok) | tiktoken (local) | Instant | Close approx |
+| **OpenRouter** (all models) | tiktoken (local) | Instant | Close approx |
+| **Ollama** (all local models) | tiktoken (local) | Instant | Close approx |
+
+You can also use standalone functions without a Noosphere instance:
+
+```typescript
+import {
+  countTokensOpenAI, countTokensGoogle, countTokensAnthropic,
+  countTokensGroq, countTokensMistral, countTokensXai,
+  countTokensCerebras, countTokensOpenRouter, countTokensOllama,
+} from 'noosphere';
+
+// Local (instant, no API key needed)
+const tokens = countTokensOpenAI(messages, 'gpt-4o');       // 26
+const groq   = countTokensGroq(messages, 'llama-3.3-70b');  // 26
+const ollama = countTokensOllama(messages, 'qwen3:8b');     // 26
+
+// API-based (exact, needs key)
+const google = await countTokensGoogle(messages, GEMINI_KEY, 'gemini-2.5-flash');     // 16
+const claude = await countTokensAnthropic(messages, ANTHROPIC_KEY, 'claude-sonnet-4-20250514'); // exact
+```
+
+---
+
+## Agent Loop & pi-ai Access
+
+Noosphere re-exports the full [pi-ai](https://github.com/nicholasgriffintn/pi-ai) library for direct access to agent loops, tool calling, cost calculation, and streaming APIs.
+
+```typescript
+import {
+  agentLoop, calculateCost,
+  piStream, piComplete, piStreamSimple, piCompleteSimple,
+  setApiKey, getApiKey, getPiModel, getPiModels, getPiProviders,
+} from 'noosphere';
+
+// Agent loop with tool calling and preprocessor (compaction hook)
+import type { AgentLoopConfig, AgentContext, AgentTool } from 'noosphere';
+
+const config: AgentLoopConfig = {
+  model: getPiModel('openai', 'gpt-4o'),
+  // Preprocessor runs before each LLM call — use for context compaction
+  preprocessor: async (messages) => {
+    // Truncate old messages, summarize, etc.
+    if (messages.length > 50) {
+      return messages.slice(-20); // keep last 20
+    }
+    return messages;
+  },
+};
+
+// Calculate cost before sending
+const model = getPiModel('openai', 'gpt-4o');
+const usage = { input: 1000, output: 500, cacheRead: 0, cacheWrite: 0 };
+const cost = calculateCost(model, usage);
+console.log(cost.total); // $0.00625
+```
+
+---
+
 ## Configuration
 
 API keys are resolved from the constructor config or environment variables (config takes priority):