@ixo/matrix 1.1.0

package/README.md

@@ -0,0 +1,396 @@
+# @ixo/matrix
+
+## Overview
+
+The `@ixo/matrix` package is a robust wrapper around the Matrix.org client SDK, specifically designed for the ixo-oracles ecosystem. It provides a secure, type-safe interface for managing Matrix communications with end-to-end encryption.
+
+### Key Features
+
+- 🔐 End-to-end encrypted room management
+- 💬 Secure messaging with threading support
+- 🔄 Type-safe state management
+- 🔑 Cross-signing and cryptographic key management
+- 🏗️ Oracle-specific room naming and access control
+- 📦 Local storage for crypto and general data
+- ⚡ Singleton pattern for efficient resource management
+
+## Table of Contents
+
+1. [Getting Started](#getting-started)
+   - [Installation](#installation)
+   - [Configuration](#configuration)
+   - [Quick Start](#quick-start)
+2. [Core Concepts](#core-concepts)
+   - [Room Management](#room-management)
+   - [Messaging](#messaging)
+   - [State Management](#state-management)
+   - [Client Operations and User Tokens](#client-operations-and-user-tokens)
+3. [API Reference](#api-reference)
+4. [Development](#development)
+
+## Getting Started
+
+### Installation
+
+```bash
+# Install using pnpm (recommended)
+pnpm install @ixo/matrix
+
+# Or using npm
+npm install @ixo/matrix
+
+# Or using yarn
+yarn add @ixo/matrix
+```
+
+### Configuration
+
+The package requires several environment variables for proper operation:
+
+#### Matrix Server
+
+```bash
+MATRIX_BASE_URL=https://your-matrix-server.com    # Your Matrix homeserver URL
+```
+
+#### Oracle Admin Credentials
+
+```bash
+MATRIX_ORACLE_ADMIN_ACCESS_TOKEN=your_token       # Admin access token
+MATRIX_ORACLE_ADMIN_USER_ID=@admin:your.server    # Admin user ID
+MATRIX_ORACLE_ADMIN_PASSWORD=your_password        # Password for cross-signing
+```
+
+#### Security
+
+```bash
+MATRIX_RECOVERY_PHRASE=your_recovery_phrase       # For secret storage & key backup
+```
+
+#### Storage Paths (Optional)
+
+All paths are created in the `matrix-local-storage` folder:
+
+```bash
+MATRIX_CRYPTO_STORE_PATH=./matrix-crypto-store              # Crypto storage
+MATRIX_STORE_PATH=./matrix-store                           # General storage
+MATRIX_SECRET_STORAGE_KEYS_PATH=./matrix-secret-storage    # Secret keys
+```
+
+### Environment Variables Explained
+
+#### Cross-Signing and Security Variables
+
+The package uses Matrix's cross-signing feature for enhanced security. This requires specific environment variables:
+
+- `MATRIX_ORACLE_ADMIN_PASSWORD`: Required for authenticating cross-signing key uploads. When setting up cross-signing, the package needs to upload device signing keys to the server. This operation requires authentication using the admin password.
+
+- `MATRIX_RECOVERY_PHRASE`: Used for:
+  - Creating and managing secret storage
+  - Generating recovery keys
+  - Restoring key backups
+  - Accessing encrypted data across devices
+
+#### Prerequisites
+
+- Access to a Matrix homeserver
+- fs write permissions
+
+### Quick Start
+
+```typescript
+import { MatrixManager } from '@ixo/matrix';
+
+async function main() {
+  // 1. Get the singleton instance
+  const manager = MatrixManager.getInstance();
+
+  // 2. Initialize the manager
+  await manager.init();
+
+  // 3. Create and join a room
+  const roomId = await manager.createRoomAndJoin({
+    did: 'did:ixo:123',
+    oracleName: 'myOracle',
+    userAccessToken: 'user_access_token',
+  });
+
+  // 4. Send a message
+  await manager.sendMessage({
+    roomId,
+    message: 'Hello World',
+  });
+}
+```
+
+## Core Concepts
+
+### Room Management
+
+Rooms in the ixo-matrix ecosystem are secure, encrypted spaces for communication. Each room has:
+
+- Unique DID-based identification
+- Oracle-specific naming convention
+- Controlled access permissions
+- End-to-end encryption
+
+#### Room Creation Process
+
+1. **Room Naming**
+
+```typescript
+// Rooms are named using a deterministic hash of DID and oracle name
+const roomName = MatrixManager.generateRoomNameFromDidAndOracle(
+  'did:ixo:123',
+  'myOracle',
+); // Results in format: 'ixo-{md5hash}'
+
+// Room aliases are generated by replacing spaces with underscores
+const roomAlias = MatrixManager.generateRoomAliasFromName(roomName);
+```
+
+1. **Room Configuration**
+   Each room is created with specific security settings:
+
+- Private visibility (not publicly listed)
+- End-to-end encryption enabled (using m.megolm.v1.aes-sha2)
+- Guest access forbidden
+- Shared history visibility
+- Admin power levels for specific operations:
+  - Kick: 9999
+  - Ban: 9999
+  - Invite: 9999
+  - Redact: 9999
+
+1. **Creating and Joining**
+
+```typescript
+// Create a room with all security features enabled
+const roomId = await manager.createRoomAndJoin({
+  did: 'did:ixo:123',
+  oracleName: 'myOracle',
+  userAccessToken: 'user_token',
+});
+```
+
+1. **Access Control**
+
+```typescript
+// Check if a user has access to a room
+const hasAccess = await manager.checkIsUserInRoom({
+  roomId,
+  userAccessToken: 'user_token',
+});
+
+// Get room by ID
+const room = manager.getRoom(roomId);
+```
+
+1. **Room Lifecycle**
+
+- Rooms are created with encryption enabled by default
+- Admin user is automatically given highest power level
+- Initial state includes encryption, guest access, and history visibility settings
+- Room can be found using DID and oracle name:
+
+```typescript
+// Get room ID from DID and oracle name
+const existingRoomId = await manager.getRoomId({
+  did: 'did:ixo:123',
+  oracleName: 'myOracle',
+});
+```
+
+### Client Operations and User Tokens
+
+The package uses two types of tokens for different operations:
+
+#### 1. Admin Token (`MATRIX_ORACLE_ADMIN_ACCESS_TOKEN`)
+
+Used for:
+
+- Sending messages in rooms
+- Managing room state
+- Setting up encryption
+- Cross-signing operations
+- Getting room information
+- Managing room power levels
+- Creating rooms
+- Inviting users to rooms
+
+#### 2. User Token (Provided per operation)
+
+Used only for:
+
+- joining rooms
+- Verifying room membership
+
+Example usage:
+
+```typescript
+// Operations using Admin Token (automatically handled)
+await manager.sendMessage({
+  roomId,
+  message: 'Hello World',
+});
+
+// Operations requiring User Token
+// the user token is used only for joining the room -- the rest of the operations are handled by the admin token
+await manager.createRoomAndJoin({
+  did: 'did:ixo:123',
+  oracleName: 'myOracle',
+  userAccessToken: 'user_token', // User token required
+});
+
+// Check if the user is in the room using the user token
+await manager.checkIsUserInRoom({
+  roomId,
+  userAccessToken: 'user_token', // User token required
+});
+```
+
+Note: User tokens are temporary and automatically cleaned up after use. All other operations use the admin token internally.
+
+### Messaging
+
+The package supports various message types and threading:
+All the messages will be sent from the Oracle Admin client.
+
+```typescript
+// Regular message
+await manager.sendMessage({
+  roomId,
+  message: 'Hello World',
+});
+
+// Threaded reply
+await manager.sendMessage({
+  roomId,
+  message: 'Reply',
+  threadId: 'original_message_id',
+});
+
+// Oracle admin message
+await manager.sendMessage({
+  roomId,
+  message: 'Admin notification',
+  isOracleAdmin: true,
+});
+```
+
+### State Management
+
+Type-safe state management with validation:
+
+```typescript
+interface ProjectState {
+  status: string;
+  lastUpdate: number;
+}
+
+// Set state
+await manager.stateManager.setState<ProjectState>({
+  roomId,
+  stateKey: 'oracle_project',
+  data: {
+    status: 'active',
+    lastUpdate: Date.now(),
+  },
+});
+
+// Get state
+const state = await manager.stateManager.getState<ProjectState>(
+  roomId,
+  'oracle_project',
+);
+
+// Update state
+await manager.stateManager.updateState<ProjectState>({
+  roomId,
+  stateKey: 'oracle_project',
+  data: {
+    status: 'completed',
+  },
+});
+```
+
+### LangChain Graph Checkpointing
+
+The package provides a Matrix-based checkpointer implementation for LangChain graphs, allowing you to persist graph state in Matrix rooms:
+
+```typescript
+import { MatrixCheckpointSaver } from '@ixo/matrix';
+import { StateGraph } from '@langchain/langgraph';
+
+// Create your graph
+const workflow = new StateGraph(graphState)
+  .addNode('myNode', (state) => {
+    // Your node logic
+  })
+  .addEdge(START, 'myNode')
+  .addEdge('myNode', END);
+
+// Compile the graph with Matrix checkpointing
+const graph = workflow.compile({
+  checkpointer: new MatrixCheckpointSaver('your-graph-name'),
+});
+```
+
+## API Reference
+
+### MatrixManager
+
+The main interface for Matrix operations:
+
+- `getInstance()`: Get singleton instance
+- `init()`: Initialize the manager
+- `createRoomAndJoin()`: Create and join a room
+- `sendMessage()`: Send a message
+- `stop()`: Cleanup resources
+
+### MatrixStateManager
+
+Handles state management:
+
+- `setState<T>()`: Set typed state
+- `getState<T>()`: Get typed state
+- `updateState<T>()`: Update existing state
+- `listStateEvents<T>()`: List all state events
+
+## Development
+
+### Testing
+
+```bash
+# Run unit tests
+pnpm test
+
+# Run integration tests
+pnpm test:e2e
+
+# Run with coverage
+pnpm test:coverage
+```
+
+### Error Handling
+
+```typescript
+try {
+  await manager.sendMessage({
+    roomId,
+    message: 'Hello',
+  });
+} catch (error) {
+  if (error instanceof MatrixError) {
+    // Handle Matrix-specific errors
+    console.error('Matrix error:', error.errcode);
+  } else {
+    // Handle other errors
+    console.error('General error:', error);
+  }
+}
+```
+
+## License
+
+Internal package - All rights reserved.

package/dist/src/checkpointer/checkpointer.d.ts

@@ -0,0 +1,56 @@
+import 'dotenv/config';
+import type { RunnableConfig } from '@langchain/core/runnables';
+import { BaseCheckpointSaver, type Checkpoint, type CheckpointListOptions, type CheckpointMetadata, type CheckpointTuple, type PendingWrite, type SerializerProtocol } from '@langchain/langgraph-checkpoint';
+import type { IGraphStateWithRequiredFields } from './types.js';
+interface CacheMetrics {
+    hits: number;
+    misses: number;
+    indexRebuilds: number;
+    filteredEvents: number;
+    duplicateEvents: number;
+}
+export declare class MatrixCheckpointSaver<_GraphState extends IGraphStateWithRequiredFields = IGraphStateWithRequiredFields> extends BaseCheckpointSaver {
+    private stateManager;
+    private checkpointCache;
+    private writesCache;
+    private latestCache;
+    private indexCache;
+    private metrics;
+    private readonly cacheTTL;
+    private readonly cacheMax;
+    private readonly maxCheckpointSizeBytes;
+    constructor(serde?: SerializerProtocol);
+    private sanitizeOracleDid;
+    private getCheckpointKey;
+    private getWritesKey;
+    private getLatestCheckpointKey;
+    private getThreadMapKey;
+    private getCacheWritesKey;
+    private getCacheLatestKey;
+    private getCacheCheckpointKey;
+    private getThreadMap;
+    private updateThreadMap;
+    private storeCheckpoint;
+    private storeWrites;
+    private getStoredCheckpoint;
+    private isValidCheckpoint;
+    private getStoredWrites;
+    private migratePendingSends;
+    getTuple(config: RunnableConfig): Promise<CheckpointTuple | undefined>;
+    list(config: RunnableConfig, options?: CheckpointListOptions): AsyncGenerator<CheckpointTuple>;
+    put(config: RunnableConfig, checkpoint: Checkpoint, metadata: CheckpointMetadata): Promise<RunnableConfig>;
+    putWrites(config: RunnableConfig, writes: PendingWrite[], taskId: string): Promise<void>;
+    deleteThread(threadId: string): Promise<void>;
+    deleteThreadWithContext(roomId: string, oracleDid: string, threadId: string, checkpointNs?: string): Promise<void>;
+    rebuildThreadMap(roomId: string, oracleDid: string): Promise<void>;
+    end(): Promise<void>;
+    getCacheMetrics(): CacheMetrics;
+    getFilteringMetrics(): {
+        filteredEvents: number;
+        duplicateEvents: number;
+        indexRebuilds: number;
+        avgEventsPerRebuild: number;
+    };
+}
+export {};
+//# sourceMappingURL=checkpointer.d.ts.map

package/dist/src/checkpointer/checkpointer.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"checkpointer.d.ts","sourceRoot":"","sources":["../../../src/checkpointer/checkpointer.ts"],"names":[],"mappings":"AAAA,OAAO,eAAe,CAAC;AAEvB,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,2BAA2B,CAAC;AAChE,OAAO,EACL,mBAAmB,EACnB,KAAK,UAAU,EACf,KAAK,qBAAqB,EAC1B,KAAK,kBAAkB,EAEvB,KAAK,eAAe,EACpB,KAAK,YAAY,EACjB,KAAK,kBAAkB,EAKxB,MAAM,iCAAiC,CAAC;AAKzC,OAAO,KAAK,EAAE,6BAA6B,EAAE,MAAM,YAAY,CAAC;AA6FhE,UAAU,YAAY;IACpB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,aAAa,EAAE,MAAM,CAAC;IACtB,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,EAAE,MAAM,CAAC;CACzB;AAED,qBAAa,qBAAqB,CAChC,WAAW,SACT,6BAA6B,GAAG,6BAA6B,CAC/D,SAAQ,mBAAmB;IAC3B,OAAO,CAAC,YAAY,CAAsB;IAG1C,OAAO,CAAC,eAAe,CAA6B;IACpD,OAAO,CAAC,WAAW,CAA0B;IAC7C,OAAO,CAAC,WAAW,CAGhB;IACH,OAAO,CAAC,UAAU,CAAgB;IAGlC,OAAO,CAAC,OAAO,CAMb;IAGF,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,sBAAsB,CAAS;gBAEpC,KAAK,CAAC,EAAE,kBAAkB;IAyBtC,OAAO,CAAC,iBAAiB;IAKzB,OAAO,CAAC,gBAAgB;IAUxB,OAAO,CAAC,YAAY;IAUpB,OAAO,CAAC,sBAAsB;IAU9B,OAAO,CAAC,eAAe;IAMvB,OAAO,CAAC,iBAAiB;IAUzB,OAAO,CAAC,iBAAiB;IASzB,OAAO,CAAC,qBAAqB;YAWf,YAAY;YAqCZ,eAAe;YAwBf,eAAe;YAsEf,WAAW;YAwEX,mBAAmB;IA6EjC,OAAO,CAAC,iBAAiB;YAcX,eAAe;YAqEf,mBAAmB;IA2C3B,QAAQ,CAAC,MAAM,EAAE,cAAc,GAAG,OAAO,CAAC,eAAe,GAAG,SAAS,CAAC;IAyHrE,IAAI,CACT,MAAM,EAAE,cAAc,EACtB,OAAO,CAAC,EAAE,qBAAqB,GAC9B,cAAc,CAAC,eAAe,CAAC;IAyJ5B,GAAG,CACP,MAAM,EAAE,cAAc,EACtB,UAAU,EAAE,UAAU,EACtB,QAAQ,EAAE,kBAAkB,GAC3B,OAAO,CAAC,cAAc,CAAC;IAqDpB,SAAS,CACb,MAAM,EAAE,cAAc,EACtB,MAAM,EAAE,YAAY,EAAE,EACtB,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,IAAI,CAAC;IA2DH,YAAY,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAoB7C,uBAAuB,CAClC,MAAM,EAAE,MAAM,EACd,SAAS,EAAE,MAAM,EACjB,QAAQ,EAAE,MAAM,EAChB,YAAY,GAAE,MAAW,GACxB,OAAO,CAAC,IAAI,CAAC;IAwGH,gBAAgB,CAC3B,MAAM,EAAE,MAAM,EACd,SAAS,EAAE,MAAM,GAChB,OAAO,CAAC,IAAI,CAAC;IA6DH,GAAG,IAAI,OAAO,CAAC,IAAI,CAAC;IAM1B,eAAe,IAAI,YAAY;IAK/B,mBAAmB,IAAI;QAC5B,cAAc,EAAE,MAAM,CAAC;QACvB,eAAe,EAAE,MAAM,CAAC;QACxB,aAAa,EAAE,MAAM,CAAC;QACtB,mBAAmB,EAAE,MAAM,CAAC;KAC7B;CAaF"}