@memoryintelligence/sdk 2.0.0

package/README.md

@@ -0,0 +1,691 @@
+## Memory Intelligence JavaScript / TypeScript SDK
+
+Official JS/TS SDK for building web apps, browser extensions, integrations, and Node.js services powered by Memory Intelligence.
+
+The JavaScript / TypeScript SDK enables developers to interact with the Memory Intelligence Operating System from:
+	•	Web applications
+	•	Browser extensions
+	•	Node.js servers
+	•	Edge runtimes (Cloudflare Workers, Vercel, Bun, Deno)
+	•	React/Next.js frontends
+	•	Electron/desktop applications
+
+It provides a fully typed, modern, secure client for working with UMOs, semantic search, hybrid retrieval, timeline APIs, agents, collections, and the meaning graph.
+
+⸻
+
+## ⚠️ SECURITY WARNING
+
+**🔐 Backend/Server-Side Use Only**
+
+This SDK is designed for **backend and server-side use only**. 
+
+**❌ DO NOT use this SDK in client-side browser code** - your API key will be exposed to anyone who opens the developer console or inspects network requests.
+
+**✅ Correct Usage:**
+- Next.js API routes
+- Node.js servers
+- Express/Fastify backends
+- Serverless functions (Vercel, Netlify, AWS Lambda)
+- Cloudflare Workers (with secret environment variables)
+
+**❌ Incorrect Usage:**
+- React components running in the browser
+- Vue/Svelte client-side code
+- Static HTML pages
+- Browser extensions (content scripts)
+
+📖 **Read the security guide:** [docs/SECURITY.md](./docs/SECURITY.md)
+
+⸻
+
+1. Features
+
+TypeScript-First Architecture
+	•	Full type safety
+	•	Autocompletion for all API models
+	•	Tightly aligned with MemoryOS schema
+	•	SDK-generated types remain stable across versions
+
+Browser + Node Support
+	•	Works in React, Next.js, Vue, Svelte
+	•	Works in Cloudflare Workers, Node, Bun, Deno
+	•	Zero heavy dependencies
+	•	ES modules + CommonJS builds
+
+Performance & Reliability
+	•	Built-in request caching
+	•	Automatic retries
+	•	Adaptive backoff
+	•	AbortController support
+	•	Connection reuse for Node environments
+
+Meaning & Privacy Enforcement
+	•	UMO validation helpers
+	•	governance-aware field masking
+	•	ULID identity enforcement
+	•	provenance hashing utilities
+	•	strict prevention of raw external ID leakage
+
+Developer Ergonomics
+	•	Async/await everywhere
+	•	Tree-shakeable modules
+	•	Optional React hooks package
+**Step 1: Environment Variables**
+
+Create a `.env` file (never commit this!):
+
+```bash
+MI_API_KEY=mi_sk_test_your_secret_key_here
+```
+
+Add `.env` to your `.gitignore`:
+
+```bash
+# .gitignore
+.env
+.env.local
+.env.*.local
+```
+
+**Step 2: Initialize Client (Server-Side Only)**
+
+```typescript
+// ✅ CORRECT: Next.js API Route (server-side)
+// app/api/process/route.ts
+import { MemoryClient } from '@memoryintelligence/sdk';
+
+const client = new MemoryClient({
+  apiKey: process.env.MI_API_KEY!  // ✅ Safe - runs on server
+});
+
+export async function POST(request: Request) {
+  const { content } = await request.json();
+  const umo = await client.umo.process(content);
+  return Response.json({ umo });
+}
+```
+
+```typescript
+// ❌ WRONG: React Component (client-side)
+// components/MyComponent.tsx
+import { MemoryClient } from '@memoryintelligence/sdk';
+
+function MyComponent() {
+  const client = new MemoryClient({
+    apiKey: 'mi_sk_xxx'  // ❌ EXPOSED! Never do this!
+  });
+  // ...
+}
+```
+
+**See `.env.example` for all configuration options.**
+npm install @memoryintelligence/sdk
+
+Yarn
+
+yarn add @memoryintelligence/sdk
+
+PNPM
+
+pnpm add @memoryintelligence/sdk
+
+
+⸻
+
+3. Setup
+
+import { MemoryClient } from "@memory-intelligence/sdk";
+
+const client = new MemoryClient({
+  apiKey: process.env.MI_API_KEY,
+  apiUrl: "https://api.memory-intelligence.com"
+});
+
+You may also use MI_API_KEY and MI_API_URL environment variables globally.
+
+⸻
+
+4. Creating Memories
+
+Simple Text Memory
+
+const umo = await client.memories.create({
+  content: "Design meeting notes about Q2 strategy."
+});
+
+With Tags + Context
+
+const umo = await client.memories.create({
+  content: "Interview with prospective client.",
+  tags: ["interview", "client"],
+  context: { department: "sales" }
+});
+
+With Provenance Hash
+
+import { computeHash } from "@memory-intelligence/sdk";
+
+const umo = await client.memories.create({
+  content: "Draft press release.",
+  provenanceHash: computeHash("Draft press release.")
+});
+
+
+⸻
+
+5. Searching Memories
+
+Semantic Search
+
+const results = await client.search.semantic("project planning");
+
+Hybrid Search
+
+const results = await client.search.hybrid({
+  query: "customer onboarding",
+  limit: 20
+});
+
+Entity-Based Search
+
+const results = await client.search.byEntity({
+  entity: "OpenAI",
+  limit: 10
+});
+
+Graph Neighborhood
+
+const neighbors = await client.graph.neighbors({
+  umoId: "01HK..."
+});
+
+
+⸻
+
+6. Timeline Queries
+
+Time Window Retrieval
+
+const window = await client.timeline.range({
+  start: "2025-01-01",
+  end: "2025-02-02"
+});
+
+Significance-Based Resurfacing
+
+const resurfaced = await client.timeline.resurface({ limit: 12 });
+
+
+⸻
+
+7. Agents API
+
+Summarize a Memory
+
+const summary = await client.agents.summarize({ umoId: "01HK..." });
+
+Generate Insights
+
+const insights = await client.agents.insights({
+  collectionId: "col_001"
+});
+
+Narrative Generation
+
+const narrative = await client.agents.narrative({
+  umoIds: ["01A..", "01B.."]
+});
+
+
+⸻
+
+8. Collections
+
+const col = await client.collections.create({
+  name: "Research Articles"
+});
+
+await client.collections.add({
+  collectionId: col.id,
+  umoId: "01HG..."
+});
+
+Retrieve:
+
+const items = await client.collections.get({ collectionId: col.id });
+
+
+⸻
+
+9. Response Types & Safety
+
+All responses are strongly typed:
+
+type UMO = Awaited<ReturnType<typeof client.memories.get>>;
+
+SDK includes types for:
+	•	UMO
+	•	SearchResult
+	•	TimelineWindow
+	•	GraphNode
+	•	ClusterSummary
+	•	AgentSummary
+	•	GovernanceError
+	•	ProvenanceRecord
+
+⸻
+
+10. Error Handling
+
+import {
+  AuthenticationError,
+  ValidationError,
+  GovernanceError,
+  RateLimitError,
+  ServerError
+} from "@memory-intelligence/sdk";
+
+try {
+  await client.memories.create({ content: null });
+} catch (err) {
+  if (err instanceof ValidationError) {
+    console.error("Invalid input:", err.message);
+  }
+}
+
+
+⸻
+
+11. Request Caching
+
+Automatic caching for safe GET operations:
+
+const results = await client.search.semantic("meaning", { cache: true });
+
+Custom caches may be injected:
+
+const client = new MemoryClient({
+  apiKey: "...",
+  cacheProvider: customCache
+});
+
+
+⸻
+
+12. Abort Controllers
+
+const controller = new AbortController();
+
+const promise = client.search.semantic("long query", {
+  signal: controller.signal
+});
+
+// cancel
+controller.abort();
+
+
+⸻
+
+13. Browser Compatibility
+
+CDN Build
+
+<script src="https://cdn.memory-intelligence.com/sdk.js"></script>
+<script>
+  const client = new MemoryIntelligence.MemoryClient({
+    apiKey: "YOUR_KEY"
+  });
+
+  client.search.semantic("example").then(console.log);
+</script>
+
+Supports:
+	•	Chrome
+	•	Firefox
+	•	Safari
+	•	Edge
+	•	WebExtensions (browser extensions)
+
+⸻
+
+14. React Hooks (Optional Package)
+
+Separate package: @memoryintelligence/react
+
+npm install @memoryintelligence/react
+
+**Provider Setup:**
+
+```typescript
+import { MemoryClientProvider } from '@memoryintelligence/react';
+import { MemoryClient } from '@memoryintelligence/sdk';
+
+const client = new MemoryClient({ apiKey: 'your-key' });
+
+function App() {
+  return (
+    <MemoryClientProvider client={client}>
+      <YourApp />
+    </MemoryClientProvider>
+  );
+}
+```
+
+**Available Hooks:**
+
+```typescript
+import {
+  useMemoryClient,
+  useDriftSubscription,
+  useComputeProgress,
+  useMemorySearch,
+  useMemoryProcess,
+} from '@memoryintelligence/react';
+
+// Search memories
+function SearchBox() {
+  const { search, data, loading, error } = useMemorySearch();
+  
+  return (
+    <input onChange={e => search(e.target.value)} />
+  );
+}
+
+// Process content
+function CaptureForm() {
+  const { process, data, loading } = useMemoryProcess();
+  
+  const handleSubmit = async (content: string) => {
+    await process({ content, type: 'text/plain' });
+  };
+}
+
+// Monitor drift updates
+function DriftMonitor() {
+  const { updates, connected } = useDriftSubscription({
+    entityUlids: ['01ARZ3ND...']
+  });
+  
+  return <div>{updates.length} drift updates</div>;
+}
+
+// Track compute progress
+function ProgressTracker({ operationId }: { operationId: string }) {
+  const { progress, result, isComplete } = useComputeProgress(operationId);
+  
+  return <div>{progress?.percentage}% complete</div>;
+}
+```
+
+⸻
+
+15. Streaming Utilities (Phase 4)
+
+For processing large datasets efficiently:
+
+**Batch Processing:**
+
+```typescript
+import { MemoryClient, UMOStreamProcessor } from '@memoryintelligence/sdk';
+
+const client = new MemoryClient({ apiKey: 'your-key' });
+const processor = new UMOStreamProcessor(client);
+
+// Process 1000 items in batches
+const items = Array.from({ length: 1000 }, (_, i) => ({
+  content: `Memory ${i}`,
+  type: 'text/plain' as const,
+}));
+
+for await (const umo of processor.processBatch({
+  items,
+  config: {
+    chunkSize: 10,      // Process 10 at a time
+    concurrency: 3,      // Max 3 concurrent requests
+    onProgress: (done, total) => {
+      console.log(`${done}/${total} processed`);
+    },
+  },
+})) {
+  // Each UMO is available as it's processed
+  console.log('Processed:', umo.umo_id);
+}
+```
+
+**Batch Search:**
+
+```typescript
+// Execute multiple searches
+for await (const result of processor.searchBatch({
+  queries: ['AI', 'machine learning', 'neural networks'],
+  searchOptions: { limit: 5 },
+})) {
+  console.log(`${result.query}: ${result.results.length} results`);
+}
+```
+
+**Paginated Search:**
+
+```typescript
+import { SearchResultStream } from '@memoryintelligence/sdk';
+
+const stream = new SearchResultStream(client);
+
+for await (const page of stream.paginate('artificial intelligence', {
+  pageSize: 20,
+  maxResults: 100,
+})) {
+  console.log(`Page: ${page.length} results`);
+}
+```
+
+**Progress Tracking:**
+
+```typescript
+import { ProgressTracker } from '@memoryintelligence/sdk';
+
+const tracker = new ProgressTracker(1000, (progress) => {
+  console.log(`${progress.percentage.toFixed(1)}% - ETA: ${progress.eta}s`);
+});
+
+// Update as you process
+tracker.update(10); // Processed 10 more items
+```
+
+See `examples/streaming.ts` for more patterns.
+
+⸻
+
+16. React Native Support (Phase 4)
+
+The SDK works in React Native with proper polyfills.
+
+**Installation:**
+
+```bash
+npm install @memoryintelligence/sdk
+npm install react-native-get-random-values whatwg-fetch abortcontroller-polyfill text-encoding
+```
+
+**Setup (App.tsx):**
+
+```typescript
+// Import polyfills FIRST!
+import 'react-native-get-random-values';
+import 'whatwg-fetch';
+import 'abortcontroller-polyfill/dist/polyfill-patch-fetch';
+import 'text-encoding';
+
+import {
+  checkReactNativeCompatibility,
+  createReactNativeClient,
+} from '@memoryintelligence/sdk';
+
+// Check compatibility
+const compat = checkReactNativeCompatibility();
+if (!compat.compatible) {
+  console.warn('Missing:', compat.missing);
+  console.warn('Install:', compat.polyfillsAvailable);
+}
+
+// Create client
+const client = createReactNativeClient('your-api-key');
+
+// Use normally
+const result = await client.umo.process({
+  content: 'React Native memory',
+  type: 'text/plain',
+});
+```
+
+**Compatibility Utilities:**
+
+```typescript
+import { ReactNativeUtils } from '@memoryintelligence/sdk';
+
+// Check if running in RN
+if (ReactNativeUtils.isReactNative()) {
+  console.log('Running in React Native');
+}
+
+// Get required polyfills
+const polyfills = ReactNativeUtils.getRecommendedPolyfills();
+console.log('Install:', polyfills);
+
+// Get install command
+const cmd = ReactNativeUtils.getInstallCommand();
+// => "npm install react-native-get-random-values whatwg-fetch ..."
+```
+
+See `examples/react-native.tsx` for a complete app example.
+
+⸻
+
+17. Governance & Identity Protections
+
+Automatically enforced:
+	•	external IDs → ULIDs
+	•	forbidden fields stripped before request
+	•	provenance hashing
+	•	p50/p95 audit logging hooks
+	•	masking of governed content
+
+JS SDK ensures meaning-safe ingestion & retrieval.
+
+⸻
+
+16. Node.js Service Examples
+
+Batch Ingestion
+
+await client.bulk.ingest([
+  { content: "Note 1" },
+  { content: "Note 2" }
+]);
+
+ETL + Export
+
+const dump = await client.bulk.export({
+  since: "2025-01-01",
+  includeProvenance: true
+});
+
+
+⸻
+
+17. E2E Testing (Phase 4)
+
+Run tests against the live API to verify end-to-end functionality.
+
+**Setup:**
+
+```bash
+# Set your test API key
+export MI_TEST_API_KEY=mi_sk_test_xxx
+
+# Run E2E tests
+npm run test:e2e
+```
+
+**Testing Features:**
+- ✅ UMO processing and retrieval
+- ✅ Semantic search
+- ✅ UMO matching and similarity
+- ✅ Explainability
+- ✅ Provenance verification
+- ✅ WebSocket subscriptions (drift, compute)
+- ✅ Rate limiting
+- ✅ Error handling
+
+E2E tests automatically clean up created test data!
+
+**Skip E2E Tests:**
+
+```bash
+# Run only unit tests (default)
+npm test
+
+# Explicitly run unit tests
+npm run test:unit
+```
+
+See `docs/E2E_TESTING.md` for complete guide.
+
+⸻
+
+18. Edge Runtime Support
+
+Compatible with:
+	•	Cloudflare Workers
+	•	Vercel Edge Functions
+	•	Bun
+	•	Deno
+
+export default {
+  async fetch(req, env) {
+    const client = new MemoryClient({ apiKey: env.KEY });
+    return Response.json(await client.search.semantic("edge test"));
+  }
+};
+
+
+⸻
+
+19. Roadmap
+
+**Completed (Phase 4):**
+	•	✅ Streaming search results and batch processing
+	•	✅ E2E testing against live API
+	•	✅ React Native compatibility layer
+	•	✅ WebSocket subscriptions (drift, compute)
+	•	✅ React hooks package
+
+**Upcoming Enhancements:**
+	•	WebAssembly-based local summarizer
+	•	Browser-side meaning compression helpers
+	•	Built-in provenance visualizer
+	•	UMO schema validator for frontend forms
+	•	Offline-first mode with sync
+	•	GraphQL API support
+	•	Vue and Svelte framework packages
+
+⸻
+
+20. Summary
+
+The JavaScript/TypeScript SDK is the official, high-performance, meaning-first client for interacting with Memory Intelligence from any JavaScript environment.
+
+It provides:
+	•	typed UMO creation
+	•	meaning-aware search
+	•	timeline access
+	•	graph exploration
+	•	collections management
+	•	agent execution
+	•	governance compliance
+	•	production-grade reliability
+
+It is the foundation for building MemorySpace-like apps, browser extensions, integrations, dashboards, and enterprise systems in the JavaScript ecosystem.
+
+⸻