@railtownai/railengine 0.0.3 → 0.0.5

package/README.md

@@ -1,570 +1,208 @@
-# @railtownai/railengine
-
-JavaScript/TypeScript SDK for Railtown AI Rail Engine - Retrieval package. This package provides a simple interface for reading and searching data from Rail Engine.
-
-## Installation
-
-```bash
-npm install @railtownai/railengine
-```
-
-## Requirements
-
-- Node.js 20+
-- TypeScript 5.0+ (optional, but recommended)
-
-## Quick Start
-
-### Basic Usage
-
-```typescript
-import { RailEngine } from "@railtownai/railengine";
-
-// Initialize client
-const client = new RailEngine({
-  pat: "your-pat-token",
-  engineId: "your-engine-id"
-});
-
-// Search vector store
-const results = client.searchVectorStore("engine-id", "VectorStore1", "search query");
-for await (const result of results) {
-  console.log(result);
-}
-
-// Get storage document by EventId
-const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
-console.log(document);
-```
-
-### Using Environment Variables
-
-```typescript
-import { RailEngine } from "@railtownai/railengine";
-
-// ENGINE_PAT and ENGINE_ID are read from environment variables automatically
-const client = new RailEngine();
-
-// Use client.engineId if you need it
-const results = client.searchVectorStore(client.engineId, "VectorStore1", "query");
-for await (const result of results) {
-  console.log(result);
-}
-```
-
-## Authentication Configuration
-
-The retrieval package uses PAT (Personal Access Token) for authentication.
-
-### Environment Variables
-
-- `ENGINE_PAT` - Personal Access Token (required)
-- `ENGINE_ID` - Engine ID (optional, but required if not provided via constructor)
-- `RAILTOWN_API_URL` - Base API URL (optional, defaults to `https://cndr.railtown.ai/api`)
-
-### Setting Environment Variables
-
-#### Windows (PowerShell)
-
-```powershell
-$env:ENGINE_PAT="<your-pat-token>"
-$env:ENGINE_ID="<your-engine-id>"
-$env:RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional
-```
-
-#### Windows (Command Prompt)
-
-```cmd
-set ENGINE_PAT=<your-pat-token>
-set ENGINE_ID=<your-engine-id>
-set RAILTOWN_API_URL=https://cndr.railtown.ai/api
-```
-
-#### macOS/Linux
-
-```bash
-export ENGINE_PAT="<your-pat-token>"
-export ENGINE_ID="<your-engine-id>"
-export RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional
-```
-
-#### .env file
-
-Create a `.env` file in your project root:
-
-```
-ENGINE_PAT=<your-pat-token>
-ENGINE_ID=<your-engine-id>
-RAILTOWN_API_URL=https://cndr.railtown.ai/api
-```
-
-Then use a package like `dotenv` to load it:
-
-```typescript
-import "dotenv/config";
-import { RailEngine } from "@railtownai/railengine";
-
-const client = new RailEngine(); // Reads from environment variables
-```
-
-### Configuration Priority
-
-1. Constructor parameters (highest priority)
-2. Environment variables
-3. Default values (where applicable)
-
-**Note**: If `ENGINE_ID` is set in environment variables, you don't need to specify it in client initialization. If `ENGINE_ID` is NOT set in environment variables, you MUST specify it in client initialization.
-
-## Zod Schema Support
-
-You can provide a Zod schema during client initialization to automatically validate retrieved data:
-
-```typescript
-import { z } from "zod";
-import { RailEngine } from "@railtownai/railengine";
-
-const FoodDiaryItemSchema = z.object({
-  food_name: z.string(),
-  calories: z.number(),
-  carbs: z.number(),
-  proteins: z.number(),
-  fats: z.number()
-});
-
-type FoodDiaryItem = z.infer<typeof FoodDiaryItemSchema>;
-
-// Initialize with schema
-const client = new RailEngine({
-  pat: "your-pat-token",
-  engineId: "your-engine-id",
-  schema: FoodDiaryItemSchema
-});
-
-// Results automatically validated against FoodDiaryItemSchema
-const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
-for await (const item of results) {
-  // item is FoodDiaryItem
-  console.log(item.food_name, item.calories);
-}
-```
-
-### Overriding Schema Per Call
-
-You can override the default schema on individual method calls:
-
-```typescript
-const FoodSummarySchema = z.object({
-  name: z.string(),
-  cal: z.number()
-});
-
-const results = client.searchVectorStore("engine-id", "VectorStore1", "apple", {
-  schema: FoodSummarySchema // Override default schema
-});
-```
-
-## Client-Side Filtering
-
-All retrieval methods support optional client-side filtering via `filterFn`:
-
-```typescript
-// Filter results by CustomerKey pattern
-const results = client.searchVectorStore("engine-id", "VectorStore1", "query", {
-  filterFn: (item) => {
-    const customerKey = (item as { CustomerKey?: string }).CustomerKey || "";
-    return customerKey.startsWith("doc-");
-  }
-});
-
-for await (const result of results) {
-  console.log(result);
-}
-```
-
-### Filtering with Pagination
-
-Filtering works seamlessly with pagination - filters are applied across all pages automatically:
-
-```typescript
-// List documents and filter by version
-const documents = client.listStorageDocuments("engine-id", {
-  filterFn: (doc) => ((doc as { Version?: number }).Version || 0) > 1
-});
-
-for await (const doc of documents) {
-  console.log(doc);
-}
-```
-
-## API Methods
-
-### Embeddings API
-
-#### searchVectorStore
-
-Search a vector store using semantic similarity.
-
-```typescript
-const results = client.searchVectorStore(
-  engineId: string,
-  vectorStore: string,
-  query: string,
-  options?: {
-    filterFn?: (item: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): AsyncIterable<T | Record<string, unknown>>
-```
-
-**Example:**
-
-```typescript
-const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
-for await (const result of results) {
-  console.log(result);
-}
-```
-
-### Storage API
-
-#### getStorageDocumentByEventId
-
-Get a single storage document by EventId.
-
-```typescript
-const document = await client.getStorageDocumentByEventId(
-  engineId: string,
-  eventId: string,
-  options?: {
-    filterFn?: (document: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): Promise<T | Record<string, unknown> | null>
-```
-
-**Example:**
-
-```typescript
-const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
-if (document) {
-  console.log(document);
-}
-```
-
-#### getStorageDocumentByCustomerKey
-
-Get storage documents by CustomerKey with automatic pagination.
-
-```typescript
-const documents = client.getStorageDocumentByCustomerKey(
-  engineId: string,
-  customerKey: string,
-  options?: {
-    pageNumber?: number;  // Default: 1
-    pageSize?: number;    // Default: 25, max: 100
-    filterFn?: (item: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): AsyncIterable<T | Record<string, unknown>>
-```
-
-**Example:**
-
-```typescript
-const documents = client.getStorageDocumentByCustomerKey("engine-id", "doc-123", {
-  pageSize: 50
-});
-
-for await (const doc of documents) {
-  console.log(doc);
-}
-```
-
-#### queryStorageByJsonPath
-
-Query storage documents using JSONPath.
-
-```typescript
-const documents = client.queryStorageByJsonPath(
-  engineId: string,
-  jsonPathQuery: string,
-  options?: {
-    filterFn?: (item: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): AsyncIterable<T | Record<string, unknown>>
-```
-
-**Example:**
-
-```typescript
-const documents = client.queryStorageByJsonPath("engine-id", "$.status:active");
-for await (const doc of documents) {
-  console.log(doc);
-}
-```
-
-#### listStorageDocuments
-
-List storage documents with automatic pagination.
-
-```typescript
-const documents = client.listStorageDocuments(
-  engineId: string,
-  options?: {
-    customerKey?: string;
-    pageNumber?: number;  // Default: 1
-    pageSize?: number;    // Default: 100, max: 100
-    filterFn?: (item: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): AsyncIterable<T | Record<string, unknown>>
-```
-
-**Example:**
-
-```typescript
-// List all documents
-const documents = client.listStorageDocuments("engine-id");
-
-// List documents filtered by customerKey
-const filtered = client.listStorageDocuments("engine-id", {
-  customerKey: "doc-123"
-});
-
-for await (const doc of documents) {
-  console.log(doc);
-}
-```
-
-### Indexing API
-
-#### searchIndex
-
-Search index using Azure Search.
-
-```typescript
-const results = client.searchIndex(
-  projectId: string,
-  engineId: string,
-  query: Record<string, unknown>,
-  options?: {
-    filterFn?: (item: T | Record<string, unknown>) => boolean;
-    schema?: z.ZodSchema<unknown>;
-  }
-): AsyncIterable<T | Record<string, unknown>>
-```
-
-**Example:**
-
-```typescript
-const results = client.searchIndex("project-id", "engine-id", {
-  search: "example query"
-});
-
-for await (const result of results) {
-  console.log(result);
-}
-```
-
-## Pagination
-
-Pagination is handled automatically by the SDK. Methods that return multiple results use async iterables that handle pagination internally:
-
-```typescript
-// Automatically handles pagination - you don't need to manage pages manually
-const documents = client.listStorageDocuments("engine-id", {
-  pageSize: 100
-});
-
-// Iterate through all pages automatically
-for await (const document of documents) {
-  console.log(document);
-}
-```
-
-Filtering is applied across all pages automatically:
-
-```typescript
-// Filter is applied to all pages, not just the first page
-const documents = client.listStorageDocuments("engine-id", {
-  filterFn: (doc) => {
-    const customerKey = (doc as { CustomerKey?: string }).CustomerKey || "";
-    return customerKey.startsWith("doc-");
-  }
-});
-
-for await (const doc of documents) {
-  // All documents from all pages that match the filter
-  console.log(doc);
-}
-```
-
-## Complete Examples
-
-### Example: Building a Simple Agent
-
-```typescript
-import { RailEngine } from "@railtownai/railengine";
-import { z } from "zod";
-
-const DocumentSchema = z.object({
-  id: z.string(),
-  title: z.string(),
-  content: z.string(),
-  tags: z.array(z.string())
-});
-
-const client = new RailEngine({
-  pat: process.env.ENGINE_PAT!,
-  engineId: process.env.ENGINE_ID!,
-  schema: DocumentSchema
-});
-
-async function searchDocuments(query: string) {
-  // Search vector store
-  const vectorResults = client.searchVectorStore(client.engineId, "VectorStore1", query, {
-    filterFn: (doc) => {
-      const tags = (doc as { tags?: string[] }).tags || [];
-      return tags.includes("public");
-    }
-  });
-
-  const results: unknown[] = [];
-  for await (const result of vectorResults) {
-    results.push(result);
-  }
-
-  return results;
-}
-
-// Usage
-const docs = await searchDocuments("machine learning");
-console.log(`Found ${docs.length} documents`);
-```
-
-### Example: Retrieving and Filtering Documents
-
-```typescript
-import { RailEngine } from "@railtownai/railengine";
-
-const client = new RailEngine({
-  pat: process.env.ENGINE_PAT!,
-  engineId: process.env.ENGINE_ID!
-});
-
-// Get document by EventId
-const doc = await client.getStorageDocumentByEventId("engine-id", "event-123");
-if (doc) {
-  console.log("Found document:", doc);
-}
-
-// Get documents by CustomerKey with filtering
-const customerDocs = client.getStorageDocumentByCustomerKey("engine-id", "customer-123", {
-  filterFn: (doc) => {
-    const version = (doc as { Version?: number }).Version || 0;
-    const status = (doc as { Status?: string }).Status;
-    return version > 1 && status === "active";
-  }
-});
-
-for await (const doc of customerDocs) {
-  console.log("Customer document:", doc);
-}
-
-// Query using JSONPath
-const activeDocs = client.queryStorageByJsonPath("engine-id", "$.status:active", {
-  filterFn: (doc) => {
-    const dateCreated = (doc as { DateCreated?: string }).DateCreated;
-    return dateCreated ? new Date(dateCreated) > new Date("2024-01-01") : false;
-  }
-});
-
-for await (const doc of activeDocs) {
-  console.log("Active document:", doc);
-}
-```
-
-## API Reference
-
-### RailEngine
-
-Main client class for retrieving data from Rail Engine.
-
-#### Constructor
-
-```typescript
-new RailEngine(options?: RailEngineOptions<T>)
-```
-
-**Options:**
-
-- `pat?: string` - Personal Access Token. If not provided, reads from `ENGINE_PAT` environment variable.
-- `engineId?: string` - Engine ID. If not provided, reads from `ENGINE_ID` environment variable.
-- `apiUrl?: string` - Base API URL. If not provided, reads from `RAILTOWN_API_URL` environment variable or defaults to production.
-- `schema?: z.ZodSchema<T>` - Optional Zod schema for validating retrieved data.
-- `timeout?: number` - Request timeout in milliseconds. Defaults to 30000 (30 seconds).
-
-**Example:**
-
-```typescript
-const client = new RailEngine({
-  pat: "your-pat-token",
-  engineId: "your-engine-id",
-  schema: MySchema,
-  timeout: 60000
-});
-```
-
-#### Properties
-
-- `pat: string` - Personal Access Token for authentication
-- `engineId: string` - Engine identifier
-- `apiUrl: string` - Base API URL (normalized)
-
-#### Methods
-
-All methods are documented above in the API Methods section.
-
-## Error Handling
-
-The SDK uses graceful error handling for read operations:
-
-- **Read operations** return `null` or empty iterables on errors (graceful degradation)
-- **Configuration errors** (missing PAT, engineId) throw exceptions
-
-The SDK uses custom exception classes for different error scenarios:
-
-- `RailtownError` - Base error class
-- `RailtownUnauthorizedError` - Authentication failed (401)
-- `RailtownNotFoundError` - Resource not found (404)
-- `RailtownBadRequestError` - Bad request (400)
-- `RailtownConflictError` - Conflict (409)
-- `RailtownServerError` - Server error (500+)
-
-**Example:**
-
-```typescript
-import { RailEngine, RailtownUnauthorizedError, RailtownBadRequestError } from "@railtownai/railengine";
-
-try {
-  const client = new RailEngine({
-    pat: "invalid-pat",
-    engineId: "engine-id"
-  });
-} catch (error) {
-  if (error instanceof RailtownBadRequestError) {
-    console.error("Invalid configuration:", error.message);
-  }
-}
-
-// Read operations return null/empty on error
-const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
-if (document === null) {
-  console.log("Document not found or error occurred");
-}
-```
-
-## License
-
-See [LICENSE](../../LICENSE) file for details.
+# @railtownai/railengine
+
+JavaScript/TypeScript SDK for [Railtown AI Rail Engine](https://railengine.ai/) - Retrieval package. This package provides a simple interface for reading and searching data from Rail Engine.
+
+## Installation
+
+```bash
+npm install @railtownai/railengine
+```
+
+## Requirements
+
+- Node.js 20+
+- TypeScript 5.0+ (optional, but recommended)
+
+## Quick Start
+
+### Basic Usage
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+
+// ENGINE_PAT and ENGINE_ID are read from environment variables automatically
+
+// Initialize client
+const client = new RailEngine();
+
+// Get storage document by EventId
+const document = await client.getStorageDocumentByEventId({
+  eventId: "event-id"
+});
+console.log(document);
+```
+
+### Using Environment Variables
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+
+const client = new RailEngine();
+
+// Use client.engineId if you need it
+const results = client.searchVectorStore(client.engineId, "VectorStore1", "query");
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+## Zod Schema Support
+
+You can provide a Zod schema during client initialization to automatically validate retrieved data:
+
+```typescript
+import { z } from "zod";
+import { RailEngine } from "@railtownai/railengine";
+
+const FoodDiaryItemSchema = z.object({
+  food_name: z.string(),
+  calories: z.number(),
+  carbs: z.number(),
+  proteins: z.number(),
+  fats: z.number()
+});
+
+type FoodDiaryItem = z.infer<typeof FoodDiaryItemSchema>;
+
+// Initialize with schema
+const client = new RailEngine({
+  schema: FoodDiaryItemSchema
+});
+
+// Results automatically validated against FoodDiaryItemSchema
+const results = client.searchVectorStore({
+  vectorStore: "VectorStore1",
+  query: "apple"
+});
+for await (const item of results) {
+  // item is FoodDiaryItem
+  console.log(item.food_name, item.calories);
+}
+```
+
+## API Methods
+
+### Embeddings API
+
+#### searchVectorStore
+
+Search a vector store using semantic similarity.
+
+**Example:**
+
+```typescript
+const results = client.searchVectorStore({
+  engineId: "engine-id", // Optional, defaults to process.env.ENGINE_ID
+  vectorStore: "VectorStore1",
+  query: "apple"
+});
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+### Storage API
+
+#### getStorageDocumentByEventId
+
+Get a single storage document by EventId.
+
+**Example:**
+
+```typescript
+const document = await client.getStorageDocumentByEventId({
+  eventId: "event-id",
+  engineId: "engine-id" // Optional, defaults to process.env.ENGINE_ID
+});
+if (document) {
+  console.log(document);
+}
+```
+
+#### getStorageDocumentByCustomerKey
+
+Get storage documents by CustomerKey with automatic pagination.
+
+**Example:**
+
+```typescript
+const documents = client.getStorageDocumentByCustomerKey({
+  customerKey: "doc-123",
+  engineId: "engine-id", // Optional, defaults to process.env.ENGINE_ID
+  pageSize: 50
+});
+
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+#### queryStorageByJsonPath
+
+Query storage documents using JSONPath.
+
+```typescript
+const documents = client.queryStorageByJsonPath(
+  engineId: string,
+  jsonPathQuery: string,
+  options?: {
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+const documents = await client.queryStorageByJsonPath({
+  engineId: "engine-id",
+  jsonPathQuery: "$.status:active"
+});
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+#### listStorageDocuments
+
+List storage documents with automatic pagination.
+
+**Example:**
+
+```typescript
+// List all documents
+const documents = await client.listStorageDocuments({
+  engineId: "engine-id"
+});
+
+// List documents filtered by customerKey
+const filtered = await client.listStorageDocuments({
+  engineId: "engine-id",
+  customerKey: "doc-123"
+});
+
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+### Indexing API
+
+#### searchIndex
+
+Search Railengine Index.
+
+**Example:**
+
+```typescript
+const results = client.searchIndex("engine-id", { search: "example query" });
+
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+## License
+
+MIT.