freesail 0.0.1 → 0.1.0

package/README.md

@@ -1,7 +1,192 @@
-# Freesail
+# Freesail SDK
 
-The official Freesail protocol implementation.
+**Industrial Strength Generative UI SDK** - Decouple AI Agents from the Frontend
 
-## Status
-This project is currently under active development by Thendral AI.
-Please follow updates at [thendral.ai](https://thendral.ai).
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+
+## Overview
+
+Freesail is a **Headless**, **Direct-Transport**, and **Schema-Driven** SDK that enables ANY AI Agent framework to drive ANY UI through a standardized JSON protocol (A2UX) over HTTP SSE.
+
+```
+┌─────────────────┐    SSE Stream    ┌─────────────────┐
+│   AI Agent      │ ───────────────► │   Frontend      │
+│  (Any LLM)      │   A2UX Protocol  │  (Any Framework)│
+│                 │ ◄─────────────── │                 │
+│  LangChain      │   User Actions   │  React/Lit/Vue  │
+│  OpenAI         │                  │  Angular/Vanilla│
+│  LlamaIndex     │                  │                 │
+└─────────────────┘                  └─────────────────┘
+```
+
+## Key Features
+
+- 🔌 **Framework Agnostic** - Works with LangChain, LlamaIndex, OpenAI, or vanilla Node.js
+- 🎨 **UI Agnostic** - Drive React, Angular, Vue, or Web Components
+- 📡 **SSE Transport** - Resilient streaming with auto-reconnect and offline support
+- 📋 **Schema-Driven** - Component catalogs are the single source of truth
+- 🔒 **Enterprise Ready** - Built for self-hosted / air-gapped deployments
+
+## Packages
+
+| Package | Description |
+|---------|-------------|
+| `@freesail/core` | A2UX Protocol parser, transport layer, state management |
+| `@freesail/lit-ui` | Lit-based Web Components with standard catalog |
+| `@freesail/server` | SSE stream engine and framework adapters |
+
+## Quick Start
+
+### Installation
+
+```bash
+npm install @freesail/core @freesail/server
+```
+
+### Server Setup (Express)
+
+```typescript
+import express from 'express';
+import { createSSEHandler, CatalogToToolConverter } from '@freesail/server';
+
+const app = express();
+
+app.get('/api/stream', createSSEHandler(async (stream) => {
+  // Create a UI surface
+  stream.createSurface('main', 'standard_catalog_v1');
+  
+  // Send components
+  stream.send({
+    updateComponents: {
+      surfaceId: 'main',
+      components: [
+        { id: 'root', component: 'Column', children: ['greeting'] },
+        { id: 'greeting', component: 'Text', text: 'Hello from AI!' }
+      ]
+    }
+  });
+}));
+
+app.listen(3000);
+```
+
+### Client Setup
+
+```html
+<script type="module">
+  import { FreesailClient } from '@freesail/core';
+  import { registerStandardCatalog, createSurfaceRenderer } from '@freesail/lit-ui';
+  
+  // Register UI components
+  registerStandardCatalog();
+  
+  // Connect to server
+  const client = new FreesailClient({ url: '/api/stream' });
+  await client.connect();
+  
+  // Create renderer
+  const renderer = createSurfaceRenderer({
+    container: document.getElementById('app'),
+    surfaceId: 'main',
+    store: client.getStore()
+  });
+</script>
+```
+
+## The A2UX Protocol
+
+Freesail uses the **A2UX Protocol**, an extension of Google's A2UI specification for agent-to-UI communication.
+
+### Server → Client Messages
+
+| Message | Purpose |
+|---------|---------|
+| `createSurface` | Initialize a UI container with a component catalog |
+| `updateComponents` | Stream component structure (flat adjacency list) |
+| `updateDataModel` | Push data changes without re-rendering |
+| `deleteSurface` | Remove a surface |
+| `watchSurface` | Request periodic state updates |
+
+### Client → Server Messages
+
+| Message | Purpose |
+|---------|---------|
+| `userAction` | Report user interactions with full context |
+| `watchSurfaceResponse` | Periodic state snapshot |
+
+## Architecture
+
+```
+/freesail
+├── /packages
+│   ├── /core           # Protocol, Transport, State
+│   ├── /lit-ui         # Web Components
+│   └── /server         # SSE Engine, Adapters
+├── /examples
+│   └── /demo           # Demo application
+└── /docs               # Documentation
+```
+
+## Framework Adapters
+
+### LangChain
+
+```typescript
+import { z } from 'zod';
+import { DynamicStructuredTool } from 'langchain/tools';
+import { createLangChainTools } from '@freesail/server/adapters/langchain';
+
+const toolConfigs = createLangChainTools(converter, catalogId, { stream, catalogId }, z);
+const tools = toolConfigs.map(config => new DynamicStructuredTool(config));
+```
+
+### OpenAI
+
+```typescript
+import { createOpenAIAdapter, processOpenAIResponse } from '@freesail/server/adapters/openai';
+
+const adapter = createOpenAIAdapter({ stream, catalogId, converter });
+
+const response = await openai.chat.completions.create({
+  model: 'gpt-4',
+  messages,
+  tools: adapter.getTools(),
+});
+
+const results = processOpenAIResponse(response, adapter);
+```
+
+## Standard Catalog Components
+
+The standard catalog includes cross-compatible A2UI components:
+
+- **Layout**: `Column`, `Row`, `Card`, `Spacer`, `Divider`
+- **Text**: `Text` (with variants h1-h6, body, caption)
+- **Input**: `Input`, `Select`, `Checkbox`
+- **Feedback**: `Button`, `Badge`, `Spinner`, `Progress`
+- **Media**: `Image`
+
+## Development
+
+```bash
+# Install dependencies
+npm install
+
+# Build all packages
+npm run build
+
+# Run demo
+cd examples/demo
+npm run dev
+```
+
+## Design Principles
+
+1. **The Contract is the Code** - No duplicated logic between schema and implementation
+2. **Schema-First** - Component catalogs are the single source of truth
+3. **Orchestrator Agnostic** - Core logic never imports framework-specific code
+4. **Stateless Agent** - Client sends full context with every action
+
+## License
+
+MIT © Freesail Team

package/docs/A2UX_Protocol.md

@@ -0,0 +1,183 @@
+The A2UX protocol is a standardized JSON schema that decouples AI agents from the frontend code, allowing them to render interfaces remotely. It extends the Google A2UI protocol.
+
+A2UX supports  the below Server to Client Messages:
+
+  1. createSurface: Initializes a UI container and loads a specific "Catalog" (vocabulary of allowed components). Sending a createSurface message allows the UI to start rendering the UI container so the user knows that the LLM is working behind the scenes to render the UI.
+    
+      Properties:
+
+      - surfaceId (string, required): The unique identifier for the UI surface to
+          be rendered. It should be unique to the session.
+
+      - catalogId (string, required): A string that uniquely identifies the component catalog used for this surface.
+
+      Example:
+```json
+ {
+                "createSurface": {
+                    "surfaceId": "user_profile_card",
+                    "catalogId":"https://thendral.ai/standard_catalog_v1.json"
+                }
+            }           
+```
+  2. updateComponents: Streams the structural definition of the UI to add or update in a surface.
+    
+      Properties:
+      
+      - surfaceId (string, required): The unique identifier for the UI surface to be updated. 
+      
+      - components (array, required): A list of component objects. The components are provided as a flat list, and their relationships are defined by ID references in an adjacency list.
+
+      Example:
+```json
+{
+  "updateComponents": {
+    "surfaceId": "user_profile_card",
+    "components": [
+      {
+        "id": "root",
+        "component": "Column",
+        "children": ["user_name", "user_title"]
+      },
+      {
+        "id": "user_name",
+        "component": "Text",
+        "text": "John Doe"
+      },
+      {
+        "id": "user_title",
+        "component": "Text",
+        "text": "Software Engineer"
+      }
+    ]
+  }
+}
+```
+  3. updateDataModel: This message is used to send or update the data that populates the UI components (e.g., to update temperature to 25C). It allows the server to change the UI's content without resending the entire component structure.
+
+      Properties:
+
+      - surfaceId (string, required): The unique identifier for the UI surface this data model update applies to.
+
+      - path (string, optional): A JSON Pointer to a specific location within the data model (e.g., /user/name ). If omitted or set to / , the entire data model for the surface will be replaced.
+      
+      - op (string, optional): The operation to perform on the data model. Must be 'add', 'replace' or  remove'. If omitted, defaults to 'replace'.
+      
+      - value (object): The data to be updated in the data model. This can be any valid JSON object. Required if op is 'add' or 'replace'. Not allowed if op is 'remove'.
+
+      Example: 
+
+```json
+{
+  "updateDataModel": {
+    "surfaceId": "user_profile_card",
+    "path": "/user",
+    "op": "replace",
+    "value": {
+      "name": "Jane Doe",
+      "title": "Software Engineer"
+    }
+  }
+}
+```
+  4. deleteSurface : This message instructs the client to remove a surface and all its associated components and data from the UI.
+
+      Properties:
+
+      - surfaceId (string, required): The unique identifier for the UI surface to be deleted.
+
+      Example:
+```json
+{
+  "deleteSurface": {
+    "surfaceId": "user_profile_card"
+  }
+}
+```
+5. watchSurface: Sometimes the agent may want to know about the current state of the surface. The watchSurface message is used to watch the surface at set intervals for a set period of time. The client will monitor the surface and send the current state of the surface to the server. This mechanism provides automatic updates to the agent without relying on user action.
+
+   Properties:
+
+    - surfaceId (string, required): The unique identifier of the UI surface where this watcher should be attached.
+
+    - interval (number, optional): Time in seconds to wait before sending a surfaceState message to the server. This avoids frequent updates. The default value is 10 seconds. 
+
+    - expiresIn (number, optional): Time in seconds when the client should remove the watch. The default value is 300 seconds. 
+
+    Example:
+
+```json
+{
+  "watchSurface": {
+    "surfaceId": "user_profile_card",
+    "interval": 30,
+    "expiresIn": 100
+  }
+}
+```
+
+6. unwatchSurface: Removes a watch that was previously set.
+
+   Properties:
+
+    - surfaceId (string, required): The unique identifier for the UI surface for which the watch should be stopped.
+
+    Example:
+
+```json
+{
+  "unwatchSurface": {
+    "surfaceId": "user_profile_card"
+  }
+}
+
+### Client to Server messages
+
+6. userAction: This message is sent from the client (UI) to the server (agent) to report a user action. It serves as the primary way the agent learns about the user's intent.
+
+    Properties:
+
+    - surfaceId (string, required): The unique identifier for the UI surface where the action originated.
+
+    - action (string, required): A semantic label identifying what happened. For standard actions, it describes the intent (e.g., submit_form, Maps_next).
+
+    - context (object, required): A JSON object that contains the data relevant to the action.
+
+    Example:
+
+```json
+{
+  "userAction": {
+    "surfaceId": "user_profile_card",
+    "action": "save_profile_click",
+    "context": {
+      "name": "Jane Doe",
+      "title": "Senior Engineer",
+      "email": "jane@example.com",
+      "newsletter_opt_in": true
+    }
+  }
+}
+```
+7. watchSurfaceResponse: This message is sent from the client (UI) to the agent to report the current state of the surface data model, in response to a watchSurface request. It serves as a way for the agent to get information about the state of the suface data automatically.
+
+    Properties:
+
+    - surfaceId (string, required): The unique identifier for the UI surface.
+
+    - data (object, required): A JSON object that contains the data relevant to the action.
+
+    Example:
+
+```json
+{
+  "watchSurfaceResponse": {
+    "surfaceId": "user_profile_card",
+    "data": {
+      "/user/name": "Jack Ryan",
+      "/user/email": "jack@example.com",
+      "/user/postcode": "W1T 2EW"
+    }
+  }
+}
+```

package/docs/Agents.md

@@ -0,0 +1,218 @@
+# **Agent Configuration: Freesail Architect**
+
+## **1\. Project Introduction**
+
+**Freesail** is an "Industrial Strength" Generative UI SDK designed to decouple AI Agents from the Frontend.
+
+Unlike frameworks that rely on heavy React wrappers or standardized agent protocols, Freesail is **Headless**, **Direct-Transport**, and **Schema-Driven**.
+
+* **The Goal:** Allow **ANY Agent Framework** (LangChain, LlamaIndex, Vercel AI, or Vanilla Node.js) to drive any UI (Legacy, Angular, React, Vanilla) using a standardized stream of JSON events over HTTP SSE.  
+* **The "Moat":** We provide the resilient streaming runtime, state management, and protocol handling that enterprise clients require for self-hosted / air-gapped applications.
+
+## **2\. Role & Vision**
+
+You are the **Lead Architect for Freesail**.
+
+* **Philosophy:** "The Contract is the Code." We do not duplicate logic. If it exists in the JSON Schema, it exists in the App.  
+* **Mission:** Build a system where the **Component Catalog** (standard\_catalog.json \+ custom catalogs) is the single source of truth for both the **AI's System Prompt** (Server) and the **UI's Properties** (Client).  
+* **Strategy:** Prioritize "Below the Radar" tech (Web Components, SSE, Node.js) to ensure maximum compatibility and zero vendor lock-in.
+
+## **3\. Tech Stack & Constraints**
+
+| Layer | Choice | Rationale |
+| :---- | :---- | :---- |
+| **Frontend** | **Lit** (Web Components) | Native browser support, Shadow DOM isolation, works everywhere. |
+| **Backend** | **Node.js** (Express) | Custom runtime for managing SSE streams and Agent state. |
+| **Transport** | **Freesail Stream** | Raw Server-Sent Events (SSE) carrying A2UX payloads. |
+| **Orchestrator** | **Agnostic** (Pluggable) | Supports LangChain, LlamaIndex, or raw OpenAI via **Adapters**. |
+| **Protocol** | **A2UX** (A2UI-Extended) | Strict JSON schema for UI definition (createSurface, updateComponents) and Interaction (userAction). |
+| **Std Lib** | **A2UI Standard** | We implement the standard Row, Column, Text, Input components to ensure cross-compatibility. |
+
+### **⛔ Strict Anti-Patterns**
+
+* **NO Framework Lock-in:** Core logic must never import langchain or llamaindex directly. Use dependency injection or adapters.  
+* **NO Duplicate Property Definitions:** Do not define properties in TypeScript if they are already in the loaded catalog. Use the defineProps() helper.  
+* **NO Raw HTML Injection:** Never use innerHTML. All rendering must go through the Registry.  
+* **NO "Delta" State Patching on Client:** The client blindly accepts state replacements. The Agent is the source of truth.
+
+## **4\. Architecture Standards (The "Golden Stack")**
+
+1. **The Source of Truth (Catalogs):** Catalogs are self-contained modules located in packages/lit-ui/src/catalogs/\<name\>/. Each folder contains the JSON definition and the TypeScript elements implementation.  
+2. **The Context Injector (Server):** Reads the **selected** Catalog(s) at runtime and converts them into **Standard JSON Schema Tools** compatible with any LLM.  
+3. **The Renderer (Lit):** Dynamically loads component definitions from the catalogId specified in createSurface.  
+4. **The Stream Engine:** A dedicated StreamStore class that manages SSE connections (res.write) and handles backpressure.
+
+## **5\. Implementation Rules**
+
+### **Rule \#1: Schema-First Components (The "Dry" Law)**
+
+* **Pattern:** Lit components must hydrate their properties directly from the Catalog JSON located **in the same folder**.  
+* **Requirement:** Never manually write @property decorators for data fields. Use the defineProps utility.
+
+// GOOD: Colocated Import  
+import catalog from '../catalog.json'; 
+
+export class FreesailText extends LitElement {  
+  // Auto-generates properties based on the JSON definition  
+  static properties \= defineProps(catalog, 'Text');  
+  render() { return html\`\<span\>${this.text}\</span\>\`; }  
+}
+
+### **Rule \#2: Multi-Catalog Support**
+
+* **Requirement:** The system must support loading different catalogs for different surfaces.  
+* **Mechanism:**  
+  * **Server:** The Agent decides which catalog to use (e.g., finance\_v1) and sends it in createSurface(catalogId="finance\_v1").  
+  * **Client:** The Registry maps catalogId to a specific bundle of Web Components.  
+  * **Validation:** The System Prompt generator must only include components from the *active* catalog to prevent hallucinations.
+
+### **Rule \#3: Orchestrator Agnosticism (The Adapter Pattern)**
+
+* **Pattern:** Core logic is pure TypeScript. Framework specifics are Adapters.  
+* **Requirement:**  
+  * **Core:** CatalogToToolConverter takes a catalog and outputs a standard JSON Schema Tool definition ({ name: "render\_ui", parameters: ... }).  
+  * **Adapters:** Create thin wrappers in server/src/adapters/ for specific frameworks:  
+    * LangChainAdapter: Converts the standard tool to a DynamicStructuredTool.  
+    * LlamaIndexAdapter: Converts the standard tool to a FunctionTool.  
+    * OpenAIAdapter: Converts the standard tool to an OpenAI API tools array.
+
+### **Rule \#4: Client-Side Logic (The Function Registry)**
+
+* **Pattern:** Do not force the Agent to format data. Use A2UI Standard Functions for **Data Logic**.  
+* **Requirement:** Implement the standard function list (formatCurrency, pluralize, regex) in @freesail/core.  
+* **Execution:** The Parser must detect template strings ${...} and execute the logic locally against the Data Model.
+
+### **Rule \#5: The "Full Context" Action Protocol**
+
+* **Pattern:** Stateless Communication.  
+* **Requirement:** When a user action occurs, the client must gather the **entire state object** for that surfaceId and send it to the agent via the context property.  
+* **Why:** The Agent is stateless. It relies on the context payload to know the current state.
+
+### **Rule \#6: The "Build & Validate" Loop (CRITICAL)**
+
+* **Pattern:** Never assume code works. Prove it.  
+* **Requirement:** After generating any code (Client or Server):  
+  1. **Run Build:** Execute npm run build in the relevant package.  
+  2. **Fix Errors:** If the build fails, read the error log, fix the code, and retry.  
+  3. **Validate Schema:** Ensure generated JSON payloads match standard\_catalog.json exactly.
+
+### **Rule \#7: Transport Resilience (The "Iron Pipe")**
+
+* **Pattern:** Robust Retry & Reconnect (SDK Responsibility).  
+* **Requirement:** The @freesail/core SDK handles network faults transparently.  
+  * **SSE Reconnection:** Automatically reconnect the event stream with exponential backoff if the connection drops.  
+  * **Action Queueing:** If the user performs an action (userAction) while offline, queue it in indexedDB and flush when online.  
+  * **Idempotency:** The Server must handle duplicate userAction messages gracefully.
+
+## **6\. The A2UX Protocol Specification**
+
+We use the **A2UX Protocol**, an extension of Google A2UI v0.9.
+
+### **Downstream (Server → Client)**
+
+| Message | Purpose | Schema Key |
+| :---- | :---- | :---- |
+| **createSurface** | Init UI container & select Catalog. | createSurface |
+| **updateComponents** | Stream DOM structure. | updateComponents |
+| **updateDataModel** | Push state changes. | updateDataModel |
+| **deleteSurface** | Cleanup/Destroy. | deleteSurface |
+| **watchSurface** | Config client triggers. | watchSurface |
+| **unwatchSurface** | Remove triggers. | unwatchSurface |
+
+**Schema: watchSurface**
+
+Configures the client to monitor the surface and report state automatically.
+
+{  
+  "watchSurface": {  
+    "surfaceId": "user\_profile\_card",  
+    "interval": 30, // seconds  
+    "expiresIn": 100 // seconds  
+  }  
+}
+
+### **Upstream (Client → Server)**
+
+| Message | Purpose | Schema Key |
+| :---- | :---- | :---- |
+| **userAction** | Explicit User Intent. | userAction |
+| **watchSurfaceResponse** | Passive State Event. | watchSurfaceResponse |
+
+**Schema: userAction**
+
+Note: usage of context to carry the data model.
+
+{  
+  "userAction": {  
+    "surfaceId": "user\_profile\_card",  
+    "action": "save\_profile\_click",  
+    "context": {  
+      "name": "Jane Doe",  
+      "title": "Senior Engineer",  
+      "email": "jane@example.com",  
+      "newsletter\_opt\_in": true  
+    }  
+  }  
+}
+
+**Schema: watchSurfaceResponse**
+
+Note: usage of data to carry the state.
+
+{  
+  "watchSurfaceResponse": {  
+    "surfaceId": "user\_profile\_card",  
+    "data": {  
+      "/user/name": "Jack Ryan",  
+      "/user/email": "jack@example.com",  
+      "/user/postcode": "W1T 2EW"  
+    }  
+  }  
+}
+
+## **7\. Directory Structure**
+
+Maintain this structure to enforce the **Vertical Slice** pattern where catalogs and their elements are colocated, and the Server logic is decoupled from specific frameworks.
+
+/freesail  
+├── /packages  
+│   ├── /core           \# @freesail/core (The Brain \- Protocol Parser & Store)  
+│   │   ├── src/  
+│   │   │   ├── protocol.ts   \# A2UX Listener  
+│   │   │   ├── parser.ts     \# JSON Stream Parser  
+│   │   │   ├── transport.ts  \# ⭐️ Network Retry & Queue Logic  
+│   │   │   ├── functions.ts  \# Standard Lib (formatCurrency, regex)  
+│   │   │   └── store.ts      \# Reactive Data Model  
+│   │  
+│   ├── /lit-ui         \# @freesail/lit (The Face \- Web Components)  
+│   │   ├── src/  
+│   │   │   ├── utils/        \# defineProps helper  
+│   │   │   ├── catalogs/     \# ⭐️ COLOCATED CATALOGS  
+│   │   │   │   ├── standard/   
+│   │   │   │   │   ├── catalog.json       \# Standard A2UI Definition  
+│   │   │   │   │   └── elements/          \# Implementation  
+│   │   │   │   │       ├── Text.ts  
+│   │   │   │   │       └── Button.ts  
+│   │   │   │   │  
+│   │   │   │   └── finance/  
+│   │   │   │       ├── catalog.json       \# Finance Domain Definition  
+│   │   │   │       └── elements/          \# Implementation  
+│   │   │   │           ├── Ticker.ts  
+│   │   │   │           └── Chart.ts  
+│   │  
+│   └── /server         \# @freesail/server (The Orchestrator)  
+│       ├── src/  
+│       │   ├── stream.ts     \# SSE Implementation (res.write)  
+│       │   ├── catalog-loader.ts \# Generic JSON Schema Generator  
+│       │   └── adapters/     \# ⭐️ FRAMEWORK ADAPTERS  
+│       │       ├── langchain.ts  \# LangChain Tool Wrapper  
+│       │       ├── llamaindex.ts \# LlamaIndex Tool Wrapper  
+│       │       └── openai.ts     \# OpenAI Function Definition  
+│  
+├── /docs               \# ⭐️ DEVELOPER GUIDES  
+│   ├── GettingStarted.md  
+│   ├── Architecture.md  
+│   └── CatalogReference.md  
+│  
+└── /examples  
+    └── /langchain-demo   \# Proof of "Any-Stack" with LangChain