nexus-ai-pro 0.2.0 → 0.6.0

package/EXPLANATION.md

@@ -0,0 +1,389 @@
+# nexus-ai-pro Explanation
+
+`nexus-ai-pro` is a TypeScript library for building AI features without rebuilding the same provider, routing, security, cache, context, and tool plumbing in every app.
+
+For a quick start, read [README.md](./README.md). For testing and operational checks, read [NEXUS.md](./NEXUS.md).
+
+## Plain-English Summary
+
+Most AI apps begin with one provider call. Then the app grows:
+
+- users want streaming
+- teams want fallback models
+- prompts need guardrails
+- long chats exceed context windows
+- model costs need planning
+- tools need safe execution loops
+- documents need RAG
+- production needs metrics, retries, health checks, and test fixtures
+
+`nexus-ai-pro` puts those pieces behind one main class: `NexusAI`.
+
+You can use it lightly:
+
+```ts
+const ai = new NexusAI({
+  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
+  routing: { mode: 'direct' },
+  defaultModel: 'openai/fast',
+  security: 'off',
+  tokenOptimizer: { enabled: false },
+});
+```
+
+Or you can enable the production layers you need:
+
+```ts
+const ai = new NexusAI({
+  providers: {
+    openai: { apiKey: process.env.OPENAI_API_KEY! },
+    anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },
+    google: { apiKey: process.env.GOOGLE_API_KEY! },
+  },
+  routing: { mode: 'auto', strategy: 'quality' },
+  security: 'standard',
+  contextWindow: {
+    strategy: 'last-messages-with-summary',
+    lastMessages: 10,
+    summary: { model: 'openai/fast', maxTokens: 600 },
+  },
+  cache: { enabled: true, strategy: 'hybrid' },
+  metrics: { enabled: true },
+});
+```
+
+## Request Lifecycle
+
+A normal `ai.complete(...)` request moves through these steps:
+
+```txt
+request
+-> pipeline hooks/custom steps
+-> response-format prompt shaping
+-> context-window compaction
+-> token optimization
+-> input security
+-> routing
+-> cost budget check
+-> cache lookup
+-> provider call with retry/failover
+-> output security
+-> response-format validation
+-> cache write
+-> final response
+```
+
+Most layers are optional. The user chooses the tradeoff between latency, safety, cost, and observability.
+
+## Main Concepts
+
+### Providers
+
+Provider adapters normalize request and response shapes.
+
+Supported adapters:
+
+- OpenAI
+- Anthropic
+- Google Gemini
+- Ollama
+- OpenRouter
+- Groq
+- Mistral
+- Cohere
+
+Provider SDKs are optional peer dependencies. Install only the SDKs your app uses.
+
+### Routing
+
+Routing chooses a model and provider.
+
+Modes:
+
+- `direct`
+- `auto`
+- `rules`
+- `hybrid`
+
+Strategies:
+
+- `quality`
+- `cost`
+- `speed`
+- `privacy`
+
+Routing can also use:
+
+- candidate model lists
+- allow and deny patterns
+- required capabilities
+- weighted preferences
+- provider health scores
+
+### Context Windows
+
+Context-window management compacts long conversations before the prompt goes to the model.
+
+It solves the "10+ prompts and growing chat history" problem without forcing every app to write its own memory logic.
+
+Strategies:
+
+- keep the last N messages
+- keep recent messages up to a token budget
+- summarize older messages and keep recent messages
+- automatically compact only when thresholds are exceeded
+
+Example:
+
+```ts
+contextWindow: {
+  strategy: 'last-messages-with-summary',
+  lastMessages: 8,
+  summary: {
+    model: 'openai/fast',
+    maxTokens: 500,
+    fallbackToLocal: true,
+  },
+}
+```
+
+Summaries can be:
+
+- local
+- generated by a configured model
+- custom through a user-provided summarizer function
+
+Metadata is exposed at:
+
+- `response.meta.contextWindow`
+- `stream done.meta.contextWindow`
+- `ai.plan(...).contextWindow`
+
+### Token Optimizer
+
+The token optimizer is separate from context windows.
+
+Context windows decide which conversation history to keep. Token optimization then estimates, densifies, truncates, warns, blocks, or allows the final request.
+
+Budget actions:
+
+- `error`
+- `truncate`
+- `densify`
+- `allow`
+
+### Security
+
+Security runs before provider calls and after provider responses.
+
+Input protections include:
+
+- schema validation
+- prompt-injection detection
+- optional prompt-injection neutralization
+- PII detection
+- secret detection
+- URL checks
+- tool allowlists
+- max input length
+
+Output protections include:
+
+- PII redaction
+- secret redaction
+- max output length
+- moderation term checks
+- data leakage checks
+- grounding overlap checks
+
+Security levels:
+
+- `off`
+- `basic`
+- `standard`
+- `strict`
+- `paranoid`
+
+These are guardrails, not a complete security boundary. Apps still need authorization, least-privilege tools, provider-side moderation where appropriate, and human review for high-risk workflows.
+
+### Streaming
+
+`ai.stream(...)` normalizes provider streaming into one shape:
+
+- `text`
+- `tool_call`
+- `done`
+- `error`
+
+Streaming failover can retry before any chunks have been emitted. After partial output is emitted, the stream reports an error instead of silently switching models mid-answer.
+
+### RAG and Verification
+
+RAG helpers attach retrieved context to a request.
+
+Verification helpers can check generated claims against supplied context and optionally repair unsupported answers.
+
+Main helpers:
+
+- `withRagContext(...)`
+- `MemoryVectorStore`
+- `completeVerified(...)`
+- `withKnowledgeGraphContext(...)`
+- `completeConsistent(...)`
+
+The default vector store can run without an external embedding service, but production semantic search should use a real embedding provider.
+
+### Tools and Agents
+
+Tools are declared with `tool(...)`.
+
+`ai.agent(...)` can:
+
+- choose tools
+- execute tool calls
+- inject tool results back into the conversation
+- stop at a max iteration count
+- call step callbacks
+
+The agent loop is intentionally small. Complex agent products should still keep their own authorization and business rules outside model output.
+
+### Cache, Jobs, and Evals
+
+Cache:
+
+- exact memory cache
+- semantic cache
+- hybrid exact/semantic strategy
+- Redis and SQLite adapter wrappers
+
+Jobs:
+
+- concurrency-limited batch completion
+- in-memory queue
+- Redis and BullMQ adapter wrappers
+
+Evals:
+
+- built-in eval runner
+- quality metrics
+- operational metrics
+- RAG metrics
+- safety heuristics
+
+## Voice Integrations
+
+Voice is now an optional provider-neutral layer. It is deliberately split from the main runtime so text-only users do not pay for voice code.
+
+Import choices:
+
+- `nexus-ai-pro/voice` gives `VoiceManager`, types, and errors.
+- `nexus-ai-pro/voice/openai` gives the OpenAI voice adapter.
+- Custom providers can be registered without importing hosted provider adapters.
+
+The basic operations are:
+
+- `ai.transcribe(...)` for speech-to-text
+- `ai.speak(...)` for text-to-speech
+- `ai.voice(...)` for the full voice pipeline
+- `registerVoiceProvider(...)` for private, local, or hosted services
+
+The full voice turn is:
+
+```txt
+audio/transcript
+-> transcribe if needed
+-> append transcript to completion messages
+-> normal Nexus complete pipeline
+-> speak if requested
+```
+
+Because the completion step calls `ai.complete(...)`, it reuses routing, security, context-window compaction, token optimization, cache, retries, metrics, and tracing.
+
+Realtime sessions are still future work. That should be added as a separate `VoiceSession` abstraction instead of forcing realtime state into normal completion streams.
+
+## Choosing a Setup
+
+For a small app:
+
+```ts
+const ai = new NexusAI({
+  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
+  routing: { mode: 'direct' },
+  defaultModel: 'openai/fast',
+  security: 'standard',
+});
+```
+
+For a latency-sensitive route:
+
+```ts
+const ai = new NexusAI({
+  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
+  routing: { mode: 'direct' },
+  defaultModel: 'openai/fast',
+  security: 'off',
+  tokenOptimizer: { enabled: false },
+  cache: { enabled: false },
+  pipeline: { trace: false, includeTraceInResponse: false },
+});
+```
+
+For a production assistant:
+
+```ts
+const ai = new NexusAI({
+  providers: {
+    openai: { apiKey: process.env.OPENAI_API_KEY! },
+    anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },
+  },
+  routing: {
+    mode: 'auto',
+    strategy: 'quality',
+    fallback: { onError: ['openai/fast'] },
+  },
+  security: 'strict',
+  contextWindow: {
+    strategy: 'auto',
+    lastMessages: 12,
+    maxInputTokens: 12000,
+    summary: { model: 'openai/fast', maxTokens: 600 },
+  },
+  retry: { enabled: true, maxRetries: 2 },
+  health: { enabled: true },
+  metrics: { enabled: true },
+});
+```
+
+## Important Files
+
+- `src/core/nexus.ts` - main runtime
+- `src/providers/*` - provider adapters
+- `src/router/*` - routing and failover
+- `src/security/*` - guardrails
+- `src/context/*` - context-window management
+- `src/optimizer/*` - token and cost optimization
+- `src/hallucination/*` - RAG, verification, vector search, knowledge graph context
+- `src/agent/*` - tools and agent loop
+- `src/cache/*` - exact and semantic cache
+- `src/jobs/*` - batch and queue support
+- `src/evals/*` - eval runner and metrics
+- `src/workflow/*` - reusable workflow templates
+- `src/types/*` - public TypeScript types
+- `tests/*` - unit, smoke, API, type, and conformance tests
+
+## Known Limitations
+
+- Voice is documented as a roadmap, not a finished runtime.
+- Audio/video preprocessing is limited.
+- Provider modality support differs and may require preprocessing.
+- Built-in model metadata is a routing convenience, not a pricing contract.
+- Process-memory cache and queues are not enough for distributed production systems.
+- Default hash embeddings are useful for tests and demos, not strong semantic search.
+- NLI verification needs a specialized verifier for high-confidence entailment.
+- Guardrails reduce risk but do not replace application authorization or review.
+
+## How To Read The Docs
+
+- Start with `README.md` if you are using the package.
+- Read this file if you want the design explained.
+- Read `NEXUS.md` if you are testing, publishing, or maintaining the package.