@railtownai/railengine 0.0.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Railtown AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,570 @@
+# @railtownai/railengine
+
+JavaScript/TypeScript SDK for Railtown AI Rail Engine - Retrieval package. This package provides a simple interface for reading and searching data from Rail Engine.
+
+## Installation
+
+```bash
+npm install @railtownai/railengine
+```
+
+## Requirements
+
+- Node.js 20+
+- TypeScript 5.0+ (optional, but recommended)
+
+## Quick Start
+
+### Basic Usage
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+
+// Initialize client
+const client = new RailEngine({
+  pat: "your-pat-token",
+  engineId: "your-engine-id"
+});
+
+// Search vector store
+const results = client.searchVectorStore("engine-id", "VectorStore1", "search query");
+for await (const result of results) {
+  console.log(result);
+}
+
+// Get storage document by EventId
+const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
+console.log(document);
+```
+
+### Using Environment Variables
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+
+// ENGINE_PAT and ENGINE_ID are read from environment variables automatically
+const client = new RailEngine();
+
+// Use client.engineId if you need it
+const results = client.searchVectorStore(client.engineId, "VectorStore1", "query");
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+## Authentication Configuration
+
+The retrieval package uses PAT (Personal Access Token) for authentication.
+
+### Environment Variables
+
+- `ENGINE_PAT` - Personal Access Token (required)
+- `ENGINE_ID` - Engine ID (optional, but required if not provided via constructor)
+- `RAILTOWN_API_URL` - Base API URL (optional, defaults to `https://cndr.railtown.ai/api`)
+
+### Setting Environment Variables
+
+#### Windows (PowerShell)
+
+```powershell
+$env:ENGINE_PAT="<your-pat-token>"
+$env:ENGINE_ID="<your-engine-id>"
+$env:RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional
+```
+
+#### Windows (Command Prompt)
+
+```cmd
+set ENGINE_PAT=<your-pat-token>
+set ENGINE_ID=<your-engine-id>
+set RAILTOWN_API_URL=https://cndr.railtown.ai/api
+```
+
+#### macOS/Linux
+
+```bash
+export ENGINE_PAT="<your-pat-token>"
+export ENGINE_ID="<your-engine-id>"
+export RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional
+```
+
+#### .env file
+
+Create a `.env` file in your project root:
+
+```
+ENGINE_PAT=<your-pat-token>
+ENGINE_ID=<your-engine-id>
+RAILTOWN_API_URL=https://cndr.railtown.ai/api
+```
+
+Then use a package like `dotenv` to load it:
+
+```typescript
+import "dotenv/config";
+import { RailEngine } from "@railtownai/railengine";
+
+const client = new RailEngine(); // Reads from environment variables
+```
+
+### Configuration Priority
+
+1. Constructor parameters (highest priority)
+2. Environment variables
+3. Default values (where applicable)
+
+**Note**: If `ENGINE_ID` is set in environment variables, you don't need to specify it in client initialization. If `ENGINE_ID` is NOT set in environment variables, you MUST specify it in client initialization.
+
+## Zod Schema Support
+
+You can provide a Zod schema during client initialization to automatically validate retrieved data:
+
+```typescript
+import { z } from "zod";
+import { RailEngine } from "@railtownai/railengine";
+
+const FoodDiaryItemSchema = z.object({
+  food_name: z.string(),
+  calories: z.number(),
+  carbs: z.number(),
+  proteins: z.number(),
+  fats: z.number()
+});
+
+type FoodDiaryItem = z.infer<typeof FoodDiaryItemSchema>;
+
+// Initialize with schema
+const client = new RailEngine({
+  pat: "your-pat-token",
+  engineId: "your-engine-id",
+  schema: FoodDiaryItemSchema
+});
+
+// Results automatically validated against FoodDiaryItemSchema
+const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
+for await (const item of results) {
+  // item is FoodDiaryItem
+  console.log(item.food_name, item.calories);
+}
+```
+
+### Overriding Schema Per Call
+
+You can override the default schema on individual method calls:
+
+```typescript
+const FoodSummarySchema = z.object({
+  name: z.string(),
+  cal: z.number()
+});
+
+const results = client.searchVectorStore("engine-id", "VectorStore1", "apple", {
+  schema: FoodSummarySchema // Override default schema
+});
+```
+
+## Client-Side Filtering
+
+All retrieval methods support optional client-side filtering via `filterFn`:
+
+```typescript
+// Filter results by CustomerKey pattern
+const results = client.searchVectorStore("engine-id", "VectorStore1", "query", {
+  filterFn: (item) => {
+    const customerKey = (item as { CustomerKey?: string }).CustomerKey || "";
+    return customerKey.startsWith("doc-");
+  }
+});
+
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+### Filtering with Pagination
+
+Filtering works seamlessly with pagination - filters are applied across all pages automatically:
+
+```typescript
+// List documents and filter by version
+const documents = client.listStorageDocuments("engine-id", {
+  filterFn: (doc) => ((doc as { Version?: number }).Version || 0) > 1
+});
+
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+## API Methods
+
+### Embeddings API
+
+#### searchVectorStore
+
+Search a vector store using semantic similarity.
+
+```typescript
+const results = client.searchVectorStore(
+  engineId: string,
+  vectorStore: string,
+  query: string,
+  options?: {
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+### Storage API
+
+#### getStorageDocumentByEventId
+
+Get a single storage document by EventId.
+
+```typescript
+const document = await client.getStorageDocumentByEventId(
+  engineId: string,
+  eventId: string,
+  options?: {
+    filterFn?: (document: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): Promise<T | Record<string, unknown> | null>
+```
+
+**Example:**
+
+```typescript
+const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
+if (document) {
+  console.log(document);
+}
+```
+
+#### getStorageDocumentByCustomerKey
+
+Get storage documents by CustomerKey with automatic pagination.
+
+```typescript
+const documents = client.getStorageDocumentByCustomerKey(
+  engineId: string,
+  customerKey: string,
+  options?: {
+    pageNumber?: number;  // Default: 1
+    pageSize?: number;    // Default: 25, max: 100
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+const documents = client.getStorageDocumentByCustomerKey("engine-id", "doc-123", {
+  pageSize: 50
+});
+
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+#### queryStorageByJsonPath
+
+Query storage documents using JSONPath.
+
+```typescript
+const documents = client.queryStorageByJsonPath(
+  engineId: string,
+  jsonPathQuery: string,
+  options?: {
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+const documents = client.queryStorageByJsonPath("engine-id", "$.status:active");
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+#### listStorageDocuments
+
+List storage documents with automatic pagination.
+
+```typescript
+const documents = client.listStorageDocuments(
+  engineId: string,
+  options?: {
+    customerKey?: string;
+    pageNumber?: number;  // Default: 1
+    pageSize?: number;    // Default: 100, max: 100
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+// List all documents
+const documents = client.listStorageDocuments("engine-id");
+
+// List documents filtered by customerKey
+const filtered = client.listStorageDocuments("engine-id", {
+  customerKey: "doc-123"
+});
+
+for await (const doc of documents) {
+  console.log(doc);
+}
+```
+
+### Indexing API
+
+#### searchIndex
+
+Search index using Azure Search.
+
+```typescript
+const results = client.searchIndex(
+  projectId: string,
+  engineId: string,
+  query: Record<string, unknown>,
+  options?: {
+    filterFn?: (item: T | Record<string, unknown>) => boolean;
+    schema?: z.ZodSchema<unknown>;
+  }
+): AsyncIterable<T | Record<string, unknown>>
+```
+
+**Example:**
+
+```typescript
+const results = client.searchIndex("project-id", "engine-id", {
+  search: "example query"
+});
+
+for await (const result of results) {
+  console.log(result);
+}
+```
+
+## Pagination
+
+Pagination is handled automatically by the SDK. Methods that return multiple results use async iterables that handle pagination internally:
+
+```typescript
+// Automatically handles pagination - you don't need to manage pages manually
+const documents = client.listStorageDocuments("engine-id", {
+  pageSize: 100
+});
+
+// Iterate through all pages automatically
+for await (const document of documents) {
+  console.log(document);
+}
+```
+
+Filtering is applied across all pages automatically:
+
+```typescript
+// Filter is applied to all pages, not just the first page
+const documents = client.listStorageDocuments("engine-id", {
+  filterFn: (doc) => {
+    const customerKey = (doc as { CustomerKey?: string }).CustomerKey || "";
+    return customerKey.startsWith("doc-");
+  }
+});
+
+for await (const doc of documents) {
+  // All documents from all pages that match the filter
+  console.log(doc);
+}
+```
+
+## Complete Examples
+
+### Example: Building a Simple Agent
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+import { z } from "zod";
+
+const DocumentSchema = z.object({
+  id: z.string(),
+  title: z.string(),
+  content: z.string(),
+  tags: z.array(z.string())
+});
+
+const client = new RailEngine({
+  pat: process.env.ENGINE_PAT!,
+  engineId: process.env.ENGINE_ID!,
+  schema: DocumentSchema
+});
+
+async function searchDocuments(query: string) {
+  // Search vector store
+  const vectorResults = client.searchVectorStore(client.engineId, "VectorStore1", query, {
+    filterFn: (doc) => {
+      const tags = (doc as { tags?: string[] }).tags || [];
+      return tags.includes("public");
+    }
+  });
+
+  const results: unknown[] = [];
+  for await (const result of vectorResults) {
+    results.push(result);
+  }
+
+  return results;
+}
+
+// Usage
+const docs = await searchDocuments("machine learning");
+console.log(`Found ${docs.length} documents`);
+```
+
+### Example: Retrieving and Filtering Documents
+
+```typescript
+import { RailEngine } from "@railtownai/railengine";
+
+const client = new RailEngine({
+  pat: process.env.ENGINE_PAT!,
+  engineId: process.env.ENGINE_ID!
+});
+
+// Get document by EventId
+const doc = await client.getStorageDocumentByEventId("engine-id", "event-123");
+if (doc) {
+  console.log("Found document:", doc);
+}
+
+// Get documents by CustomerKey with filtering
+const customerDocs = client.getStorageDocumentByCustomerKey("engine-id", "customer-123", {
+  filterFn: (doc) => {
+    const version = (doc as { Version?: number }).Version || 0;
+    const status = (doc as { Status?: string }).Status;
+    return version > 1 && status === "active";
+  }
+});
+
+for await (const doc of customerDocs) {
+  console.log("Customer document:", doc);
+}
+
+// Query using JSONPath
+const activeDocs = client.queryStorageByJsonPath("engine-id", "$.status:active", {
+  filterFn: (doc) => {
+    const dateCreated = (doc as { DateCreated?: string }).DateCreated;
+    return dateCreated ? new Date(dateCreated) > new Date("2024-01-01") : false;
+  }
+});
+
+for await (const doc of activeDocs) {
+  console.log("Active document:", doc);
+}
+```
+
+## API Reference
+
+### RailEngine
+
+Main client class for retrieving data from Rail Engine.
+
+#### Constructor
+
+```typescript
+new RailEngine(options?: RailEngineOptions<T>)
+```
+
+**Options:**
+
+- `pat?: string` - Personal Access Token. If not provided, reads from `ENGINE_PAT` environment variable.
+- `engineId?: string` - Engine ID. If not provided, reads from `ENGINE_ID` environment variable.
+- `apiUrl?: string` - Base API URL. If not provided, reads from `RAILTOWN_API_URL` environment variable or defaults to production.
+- `schema?: z.ZodSchema<T>` - Optional Zod schema for validating retrieved data.
+- `timeout?: number` - Request timeout in milliseconds. Defaults to 30000 (30 seconds).
+
+**Example:**
+
+```typescript
+const client = new RailEngine({
+  pat: "your-pat-token",
+  engineId: "your-engine-id",
+  schema: MySchema,
+  timeout: 60000
+});
+```
+
+#### Properties
+
+- `pat: string` - Personal Access Token for authentication
+- `engineId: string` - Engine identifier
+- `apiUrl: string` - Base API URL (normalized)
+
+#### Methods
+
+All methods are documented above in the API Methods section.
+
+## Error Handling
+
+The SDK uses graceful error handling for read operations:
+
+- **Read operations** return `null` or empty iterables on errors (graceful degradation)
+- **Configuration errors** (missing PAT, engineId) throw exceptions
+
+The SDK uses custom exception classes for different error scenarios:
+
+- `RailtownError` - Base error class
+- `RailtownUnauthorizedError` - Authentication failed (401)
+- `RailtownNotFoundError` - Resource not found (404)
+- `RailtownBadRequestError` - Bad request (400)
+- `RailtownConflictError` - Conflict (409)
+- `RailtownServerError` - Server error (500+)
+
+**Example:**
+
+```typescript
+import { RailEngine, RailtownUnauthorizedError, RailtownBadRequestError } from "@railtownai/railengine";
+
+try {
+  const client = new RailEngine({
+    pat: "invalid-pat",
+    engineId: "engine-id"
+  });
+} catch (error) {
+  if (error instanceof RailtownBadRequestError) {
+    console.error("Invalid configuration:", error.message);
+  }
+}
+
+// Read operations return null/empty on error
+const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
+if (document === null) {
+  console.log("Document not found or error occurred");
+}
+```
+
+## License
+
+See [LICENSE](../../LICENSE) file for details.