lua-cli 3.5.0-alpha.3 → 3.5.0-beta.1

package/docs/api/Data.md

@@ -0,0 +1,446 @@
+# Data API
+
+The Data API provides custom data storage with vector search capabilities. Store any JSON data and search it semantically using AI embeddings.
+
+## Import
+
+```typescript
+import { Data } from 'lua-cli';
+```
+
+## Overview
+
+The Data API is perfect for:
+- Custom data collections (movies, recipes, documents)
+- Vector/semantic search
+- User-specific data storage
+- Knowledge bases
+- Content management
+
+## Methods
+
+### `Data.get(collectionName, filter?, page?, limit?)`
+
+Retrieves entries from a collection with optional filtering and pagination.
+
+**Parameters:**
+- `collectionName` (string) - Name of the collection
+- `filter` (object, optional) - MongoDB-style filter criteria
+- `page` (number, optional) - Page number (default: 1)
+- `limit` (number, optional) - Max entries per page (default: 10)
+
+**Returns:** `Promise<GetCustomDataResponse>`
+
+```typescript
+{
+  data: Entry[];
+  pagination: {
+    currentPage: number;
+    totalPages: number;
+    totalCount: number;
+    limit: number;
+    hasNextPage: boolean;
+    hasPrevPage: boolean;
+    nextPage: number | null;
+    prevPage: number | null;
+  };
+}
+```
+
+**Example:**
+```typescript
+// Get all (first page)
+const result = await Data.get('movies');
+
+for (const movie of result.data) {
+  console.log(movie.title);
+  console.log(movie.year);
+}
+
+// With pagination
+const page2 = await Data.get('movies', {}, 2, 20);
+
+// With filter
+const sciFi = await Data.get('movies', { genre: 'Sci-Fi' });
+```
+
+### `Data.search(collectionName, query, limit?, scoreThreshold?)`
+
+Performs semantic search on a collection using AI embeddings.
+
+**Parameters:**
+- `collectionName` (string) - Collection to search
+- `query` (string) - Search query (uses AI embeddings)
+- `limit` (number, optional) - Max results (default: 10)
+- `scoreThreshold` (number, optional) - Minimum similarity score 0-1 (default: 0.6)
+
+**Returns:** `Promise<DataEntryInstance[]>`
+
+Each result includes a `score` property indicating similarity (0-1).
+
+**Example:**
+```typescript
+const results = await Data.search('movies', 'sci-fi action thriller', 5, 0.7);
+
+// Results include similarity scores
+for (const movie of results) {
+  console.log(`${movie.title} (score: ${movie.score})`);
+}
+```
+
+### `Data.create(collectionName, data, searchText?)`
+
+Creates a new entry in a collection.
+
+**Parameters:**
+- `collectionName` (string) - Collection name
+- `data` (object) - Data to store
+- `searchText` (string, optional) - Text to use for vector search indexing
+
+**Returns:** `Promise<DataEntryInstance>`
+
+**Example:**
+```typescript
+const movie = await Data.create('movies', {
+  title: 'Inception',
+  year: 2010,
+  director: 'Christopher Nolan',
+  genre: ['Sci-Fi', 'Thriller']
+}, 'Inception 2010 sci-fi thriller Christopher Nolan dream heist');
+
+console.log('Created:', movie.id);
+```
+
+### `Data.update(collectionName, entryId, data, searchText?)`
+
+Updates an existing entry.
+
+**Parameters:**
+- `collectionName` (string) - Collection name
+- `entryId` (string) - Entry ID to update
+- `data` (object) - Updated data
+- `searchText` (string, optional) - Updated search text
+
+**Returns:** `Promise<UpdateCustomDataResponse>`
+
+**Example:**
+```typescript
+const result = await Data.update(
+  'movies',
+  'entry_123',
+  { rating: 9.5, views: 1000 },
+  'Inception highly rated popular'
+);
+
+if (result.status === 'success') {
+  console.log(result.message); // "Custom data entry updated"
+}
+```
+
+### `Data.delete(collectionName, entryId)`
+
+Deletes an entry from a collection.
+
+**Parameters:**
+- `collectionName` (string) - Collection name
+- `entryId` (string) - Entry ID to delete
+
+**Returns:** `Promise<DeleteCustomDataResponse>`
+
+**Example:**
+```typescript
+const result = await Data.delete('movies', 'entry_123');
+
+if (result.status === 'success') {
+  console.log(result.message); // "Custom data entry deleted"
+}
+```
+
+## DataEntryInstance
+
+Data entries have convenient methods:
+
+### `entry.update(data, searchText?)`
+
+Updates this entry.
+
+**Example:**
+```typescript
+const movies = await Data.get('movies');
+const movie = movies[0];
+
+await movie.update({
+  rating: 9.0,
+  views: movie.views + 1
+});
+```
+
+### `entry.delete()`
+
+Deletes this entry.
+
+**Example:**
+```typescript
+const movies = await Data.get('movies');
+const oldMovie = movies.find(m => m.year < 1950);
+
+if (oldMovie) {
+  await oldMovie.delete();
+}
+```
+
+### `entry.save(searchText?)`
+
+Saves modifications to this entry.
+
+**Example:**
+```typescript
+const movies = await Data.get('movies');
+const movie = movies[0];
+
+// Modify properties
+movie.data.views = (movie.data.views || 0) + 1;
+movie.data.lastViewed = new Date().toISOString();
+
+// Save changes
+await movie.save('Updated view count');
+```
+
+### Property Access
+
+```typescript
+const movie = await Data.get('movies');
+
+// Direct access (via Proxy)
+console.log(movie[0].title);
+console.log(movie[0].year);
+
+// Or via .data
+console.log(movie[0].data.title);
+console.log(movie[0].data.year);
+
+// Special properties
+console.log(movie[0].id);
+console.log(movie[0].collectionName);
+console.log(movie[0].score); // Only present in search results
+```
+
+## Complete Examples
+
+### Movie Database Tool
+
+```typescript
+import { LuaTool, Data } from 'lua-cli/skill';
+import { z } from 'zod';
+
+export default class MovieDatabaseTool implements LuaTool {
+  name = 'manage_movies';
+  description = 'Manage a movie database';
+  
+  inputSchema = z.object({
+    action: z.enum(['add', 'search', 'update', 'delete']),
+    title: z.string().optional(),
+    year: z.number().optional(),
+    director: z.string().optional(),
+    genre: z.array(z.string()).optional(),
+    rating: z.number().min(0).max(10).optional(),
+    movieId: z.string().optional(),
+    query: z.string().optional()
+  });
+  
+  async execute(input: z.infer<typeof this.inputSchema>) {
+    switch (input.action) {
+      case 'add':
+        if (!input.title || !input.year) {
+          return { success: false, error: 'Title and year required' };
+        }
+        
+        const searchText = `${input.title} ${input.year} ${input.director || ''} ${input.genre?.join(' ') || ''}`;
+        
+        const movie = await Data.create('movies', {
+          title: input.title,
+          year: input.year,
+          director: input.director,
+          genre: input.genre || [],
+          rating: input.rating || 0,
+          createdAt: new Date().toISOString()
+        }, searchText);
+        
+        return {
+          success: true,
+          message: `Added "${input.title}"`,
+          movieId: movie.id
+        };
+        
+      case 'search':
+        if (!input.query) {
+          return { success: false, error: 'Query required' };
+        }
+        
+        const results = await Data.search('movies', input.query, 10);
+        
+        return {
+          success: true,
+          count: results.data.length,
+          movies: results.data.map(r => ({
+            id: r.id,
+            title: r.data.title,
+            year: r.data.year,
+            director: r.data.director,
+            relevance: r.score
+          }))
+        };
+        
+      case 'update':
+        if (!input.movieId) {
+          return { success: false, error: 'Movie ID required' };
+        }
+        
+        const updates: any = {};
+        if (input.rating !== undefined) updates.rating = input.rating;
+        if (input.director) updates.director = input.director;
+        
+        const updateResult = await Data.update('movies', input.movieId, updates);
+        
+        return {
+          success: updateResult.status === 'success',
+          message: updateResult.message
+        };
+        
+      case 'delete':
+        if (!input.movieId) {
+          return { success: false, error: 'Movie ID required' };
+        }
+        
+        const deleteResult = await Data.delete('movies', input.movieId);
+        
+        return {
+          success: deleteResult.status === 'success',
+          message: deleteResult.message
+        };
+    }
+  }
+}
+```
+
+### Knowledge Base Tool
+
+```typescript
+import { LuaTool, Data } from 'lua-cli/skill';
+import { z } from 'zod';
+
+export default class KnowledgeBaseTool implements LuaTool {
+  name = 'search_knowledge_base';
+  description = 'Search company knowledge base for information';
+  
+  inputSchema = z.object({
+    question: z.string().describe('User question')
+  });
+  
+  async execute(input: z.infer<typeof this.inputSchema>) {
+    // Semantic search finds relevant articles
+    const results = await Data.search('knowledge-base', input.question, 3);
+    
+    if (results.data.length === 0) {
+      return {
+        success: true,
+        found: false,
+        message: 'No relevant articles found'
+      };
+    }
+    
+    // Return top results
+    return {
+      success: true,
+      found: true,
+      articles: results.data.map(r => ({
+        title: r.data.title,
+        content: r.data.content,
+        relevance: Math.round(r.score * 100),
+        url: r.data.url
+      }))
+    };
+  }
+}
+```
+
+### Recipe Manager
+
+```typescript
+import { LuaTool, Data } from 'lua-cli/skill';
+import { z } from 'zod';
+
+export default class RecipeTool implements LuaTool {
+  name = 'find_recipe';
+  description = 'Find recipes based on ingredients or cuisine';
+  
+  inputSchema = z.object({
+    searchTerm: z.string().describe('Ingredients, cuisine, or dish name')
+  });
+  
+  async execute(input: z.infer<typeof this.inputSchema>) {
+    // Search recipes
+    const results = await Data.search('recipes', input.searchTerm, 5);
+    
+    return {
+      success: true,
+      count: results.data.length,
+      recipes: results.data.map(r => ({
+        name: r.data.name,
+        cuisine: r.data.cuisine,
+        ingredients: r.data.ingredients,
+        instructions: r.data.instructions,
+        prepTime: r.data.prepTime,
+        difficulty: r.data.difficulty,
+        matchScore: Math.round(r.score * 100)
+      }))
+    };
+  }
+}
+```
+
+## Vector Search
+
+The Data API uses AI embeddings for semantic search:
+
+```typescript
+// These queries will find similar results:
+await Data.search('movies', 'science fiction space adventure');
+await Data.search('movies', 'futuristic cosmic exploration');
+await Data.search('movies', 'Star Wars type movies');
+
+// All might return: Star Wars, Interstellar, The Martian, etc.
+```
+
+**Tips:**
+- Use descriptive search text when creating entries
+- Search queries can be natural language
+- Results include similarity scores (0-1)
+- More specific queries return better results
+
+## Best Practices
+
+### ✅ DO
+- Use descriptive collection names (`user-notes`, not `notes`)
+- Provide searchText for better semantic search
+- Include all relevant keywords in searchText
+- Structure your data consistently
+- Use meaningful property names
+
+### ❌ DON'T
+- Don't store large binary data (use URLs instead)
+- Don't use Data API for relational data (use Products/Orders)
+- Don't skip searchText if you plan to search
+- Don't store sensitive unencrypted data
+
+## Related APIs
+
+- [User API](./User.md) - User-specific data
+- [Products API](./Products.md) - Product catalog
+- [DataEntryInstance](./instances/DataEntryInstance.md) - Entry methods
+
+## See Also
+
+- [Custom Data Tool Example](../../template/src/skills/tools/CustomDataTool.ts)
+- [Vector Search Guide](../guides/VectorSearch.md)
+