@hybrd/xmtp 1.0.0

package/.turbo/turbo-typecheck.log

@@ -0,0 +1,5 @@
+
+
+> @hybrd/xmtp@1.0.0-alpha.1 typecheck /Users/ian/Projects/01/hybrid/packages/xmtp
+> tsc --noEmit
+

package/README.md

@@ -0,0 +1,380 @@
+# Enhanced XMTP SDK with Robust Reliability 🚀
+
+This package provides an enhanced XMTP client with robust connection management, retry logic, and health monitoring capabilities.
+
+## 🆙 Upgraded Features
+
+- **XMTP Node SDK**: Upgraded to `^3.1.0` (latest version)
+- **Enhanced Connection Management**: Automatic reconnection and health monitoring
+- **Robust Retry Logic**: Exponential backoff and configurable retry strategies
+- **Health Monitoring**: Real-time connection health tracking with metrics
+- **Production Ready**: Built for scalable, reliable XMTP integrations
+
+## 🏗️ Architecture Overview
+
+Your system uses a **"Thin Listener + QStash Callbacks"** architecture that already provides excellent reliability:
+
+```
+┌─────────────┐    ┌─────────────┐    ┌─────────────┐
+│ XMTP Network│───▶│Thin Server  │───▶│   QStash    │
+│             │    │(Enhanced)   │    │ (Reliable)  │
+└─────────────┘    └─────────────┘    └─────────────┘
+                                              │
+                                              ▼
+                   ┌─────────────┐    ┌─────────────┐
+                   │   Agent     │◀───│App/Webhook  │
+                   │ Processing  │    │ (Scalable)  │
+                   └─────────────┘    └─────────────┘
+```
+
+### ✅ Built-in Reliability Features
+
+1. **QStash Automatic Retries**: Built-in exponential backoff
+2. **No Persistent Connections**: Eliminates connection drop issues
+3. **Horizontal Scalability**: Multiple instances supported
+4. **Dead Letter Queues**: Failed messages are preserved
+5. **At-least-once Delivery**: Messages guaranteed to be processed
+
+## 🔧 Enhanced Connection Management
+
+### Basic Usage
+
+```typescript
+import { 
+  createSigner, 
+  createXMTPConnectionManager,
+  type XMTPConnectionConfig 
+} from "@hybrd/xmtp"
+
+// Enhanced connection with reliability features
+const signer = createSigner(process.env.XMTP_WALLET_KEY!)
+
+const connectionConfig: XMTPConnectionConfig = {
+  maxRetries: 5,                    // Connection attempts
+  retryDelayMs: 1000,              // Base retry delay
+  healthCheckIntervalMs: 30000,     // Health check interval
+  connectionTimeoutMs: 15000,       // Connection timeout
+  reconnectOnFailure: true          // Auto-reconnect
+}
+
+const connectionManager = await createXMTPConnectionManager(
+  signer, 
+  connectionConfig
+)
+
+// Get health metrics
+const health = connectionManager.getHealth()
+console.log('Connection Health:', health)
+```
+
+### Advanced Production Usage
+
+```typescript
+import { RobustXMTPService } from "@hybrd/xmtp/scripts/enhanced-connection-example"
+
+const service = new RobustXMTPService()
+await service.start()
+
+// Process messages with automatic retry/reconnection
+await service.processMessage(conversationId, "Hello!")
+
+// Monitor health
+const health = service.getConnectionHealth()
+if (!health?.isConnected) {
+  console.warn("XMTP connection unhealthy")
+}
+```
+
+## 🔄 Connection Health Monitoring
+
+The enhanced client provides real-time health metrics:
+
+```typescript
+interface XMTPConnectionHealth {
+  isConnected: boolean          // Current connection status
+  lastHealthCheck: Date         // Last health check timestamp
+  consecutiveFailures: number   // Failed health checks in a row
+  totalReconnects: number       // Total reconnection attempts
+  avgResponseTime: number       // Average response time in ms
+}
+```
+
+## 🚀 Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pnpm with-env pnpm --filter @hybrd/xmtp install
+```
+
+### 2. Run Enhanced Connection Demo
+
+```bash
+pnpm with-env pnpm --filter @hybrd/xmtp enhanced:demo
+```
+
+### 3. Integrate in Your Code
+
+```typescript
+// Replace basic XMTP client creation
+// OLD:
+const client = await createXMTPClient(signer)
+
+// NEW: With enhanced reliability
+const connectionManager = await createXMTPConnectionManager(signer, {
+  maxRetries: 5,
+  healthCheckIntervalMs: 30000,
+  reconnectOnFailure: true
+})
+const client = connectionManager.getClient()
+```
+
+## 📊 Why Your Current Architecture is Already Robust
+
+Your **QStash-based webhook system** already provides superior reliability compared to traditional streaming:
+
+### Traditional Streaming Issues ❌
+- Connection drops require manual reconnection
+- Memory leaks from long-running connections  
+- Difficult to scale horizontally
+- Complex heartbeat/keepalive management
+- Single point of failure
+
+### Your QStash Architecture Benefits ✅
+- **Automatic Retries**: QStash handles exponential backoff
+- **No Connection Drops**: Stateless webhook calls
+- **Horizontal Scaling**: Multiple app instances supported
+- **Built-in Monitoring**: QStash provides delivery metrics
+- **Dead Letter Queues**: Failed messages preserved
+- **At-least-once Delivery**: Guaranteed message processing
+
+## 🔧 Configuration Options
+
+### Environment Variables
+
+| Variable              | Description                              | Default                                 |
+| --------------------- | ---------------------------------------- | --------------------------------------- |
+| `XMTP_STORAGE_PATH`   | Custom path for XMTP database storage    | `.data/xmtp` (relative to project root) |
+| `XMTP_WALLET_KEY`     | Private key for XMTP wallet              | Required                                |
+| `XMTP_ENCRYPTION_KEY` | Encryption key for database              | Required for persistent mode            |
+| `XMTP_ENV`            | XMTP environment (`dev` or `production`) | `dev`                                   |
+| `PROJECT_ROOT`        | Override project root path               | Auto-detected                           |
+
+### Connection Configuration
+
+```typescript
+interface XMTPConnectionConfig {
+  maxRetries?: number              // Default: 5
+  retryDelayMs?: number           // Default: 1000ms
+  healthCheckIntervalMs?: number  // Default: 30000ms  
+  connectionTimeoutMs?: number    // Default: 10000ms
+  reconnectOnFailure?: boolean    // Default: true
+}
+```
+
+### Custom Storage Location
+
+You can specify a custom storage location for XMTP database files:
+
+```bash
+# Absolute path
+export XMTP_STORAGE_PATH=/custom/path/to/xmtp/storage
+
+# Relative path (relative to current working directory)
+export XMTP_STORAGE_PATH=./custom/xmtp/storage
+
+# Use with pnpm with-env
+pnpm with-env your-xmtp-command
+```
+
+### Testing Custom Storage
+
+Run the custom storage example to test your configuration:
+
+```bash
+# Test with default storage location
+pnpm with-env pnpm --filter @hybrd/xmtp custom:storage
+
+# Test with custom storage location
+export XMTP_STORAGE_PATH=/tmp/my-custom-xmtp-storage
+pnpm with-env pnpm --filter @hybrd/xmtp custom:storage
+
+# Test with relative path
+export XMTP_STORAGE_PATH=./my-xmtp-data
+pnpm with-env pnpm --filter @hybrd/xmtp custom:storage
+```
+
+## 🛠️ Available Scripts
+
+| Script             | Command                                      | Description                       |
+| ------------------ | -------------------------------------------- | --------------------------------- |
+| `gen:keys`         | `pnpm --filter @hybrd/xmtp gen:keys`         | Generate new XMTP wallet keys     |
+| `register`         | `pnpm --filter @hybrd/xmtp register`         | Register wallet on XMTP network   |
+| `revoke`           | `pnpm --filter @hybrd/xmtp revoke`           | Revoke old XMTP installations     |
+| `enhanced:demo`    | `pnpm --filter @hybrd/xmtp enhanced:demo`    | Demo enhanced connection features |
+| `test:messages`    | `pnpm --filter @hybrd/xmtp test:messages`    | Test message reception            |
+| `refresh:identity` | `pnpm --filter @hybrd/xmtp refresh:identity` | Refresh XMTP identity             |
+| `custom:storage`   | `pnpm --filter @hybrd/xmtp custom:storage`   | Test custom storage configuration |
+
+> **Note**: Always use `pnpm with-env` to ensure environment variables are loaded:
+> ```bash
+> pnpm with-env pnpm --filter @hybrd/xmtp <script-name>
+> ```
+
+## 🎯 Best Practices
+
+### 1. Use Connection Manager for Long-Running Services
+```typescript
+// For services that need persistent XMTP connections
+const manager = await createXMTPConnectionManager(signer, config)
+```
+
+### 2. Leverage Your Existing QStash Architecture  
+```typescript
+// For message processing, your webhook system is ideal
+// No changes needed - it's already robust!
+```
+
+### 3. Monitor Connection Health
+```typescript
+setInterval(() => {
+  const health = connectionManager.getHealth()
+  if (health.consecutiveFailures > 3) {
+    console.warn("XMTP connection degraded")
+  }
+}, 60000)
+```
+
+### 4. Use Environment Variables
+```typescript
+// Always use the project's environment wrapper
+pnpm with-env [your-command]
+```
+
+## 🧪 Testing
+
+Run the enhanced connection demo to see health monitoring in action:
+
+```bash
+# Start the demo (runs for 2 minutes showing health checks)
+pnpm with-env pnpm --filter @hybrd/xmtp enhanced:demo
+```
+
+## 🔍 Debugging
+
+Enable debug logging:
+
+```bash
+DEBUG=xmtp-sdk* pnpm with-env pnpm --filter @hybrd/xmtp enhanced:demo
+```
+
+## 📈 Metrics & Monitoring
+
+The enhanced client provides detailed metrics:
+
+- **Connection Status**: Real-time connection state
+- **Response Times**: Average XMTP response latency  
+- **Failure Counts**: Track connection reliability
+- **Reconnection Events**: Monitor stability over time
+
+## 🚨 Migration Guide
+
+### From Basic XMTP Client
+
+```typescript
+// Before
+const client = await createXMTPClient(signer)
+
+// After  
+const manager = await createXMTPConnectionManager(signer)
+const client = manager.getClient()
+
+// Remember to cleanup
+await manager.disconnect()
+```
+
+### Keep Your Webhook Architecture
+
+**No changes needed!** Your QStash webhook system is already providing:
+- ✅ Automatic retries with exponential backoff
+- ✅ Reliable message delivery guarantees  
+- ✅ Horizontal scalability
+- ✅ Built-in monitoring and alerting
+- ✅ Dead letter queue handling
+
+## 📚 Additional Resources
+
+- [XMTP Node SDK Documentation](https://xmtp.org/docs/build/get-started/overview)
+- [QStash Documentation](https://upstash.com/docs/qstash)
+- [Project Architecture](../../ARCHITECTURE.md)
+
+---
+
+Your system is already built with production-grade reliability. The enhanced XMTP client provides additional connection management features for edge cases, but your webhook-based architecture is the recommended approach for scalable XMTP integrations! 🎉 
+
+# XMTP Package
+
+This package provides XMTP client functionality and various resolvers for address, ENS, and basename resolution.
+
+## Resolvers
+
+### Master Resolver
+
+The `Resolver` class provides a unified interface for all resolution types:
+
+```typescript
+import { Resolver } from '@hybrd/xmtp/resolver'
+import { createPublicClient, http } from 'viem'
+import { mainnet, base } from 'viem/chains'
+
+// Create clients
+const mainnetClient = createPublicClient({
+  chain: mainnet,
+  transport: http()
+})
+
+const baseClient = createPublicClient({
+  chain: base,
+  transport: http()
+})
+
+// Initialize the master resolver
+const resolver = new Resolver({
+  xmtpClient: yourXmtpClient,
+  mainnetClient,
+  baseClient,
+  maxCacheSize: 1000,
+  cacheTtl: 3600000 // 1 hour
+})
+
+// Universal name resolution
+const address = await resolver.resolveName('vitalik.eth')
+const basenameAddress = await resolver.resolveName('myname.base.eth')
+
+// Universal reverse resolution  
+const name = await resolver.resolveAddressToName('0x...')
+
+// Get complete profile (ENS + basename data)
+const profile = await resolver.getCompleteProfile('0x...')
+
+// Individual resolver methods are also available
+const ensName = await resolver.resolveENSName('vitalik.eth')
+const basename = await resolver.getBasename('0x...')
+const message = await resolver.findMessage('messageId')
+```
+
+### Individual Resolvers
+
+You can also use individual resolvers directly:
+
+- `AddressResolver` - XMTP address resolution
+- `XmtpResolver` - XMTP message and address resolution with advanced features
+- `ENSResolver` - ENS name resolution
+- `BasenameResolver` - Basename resolution for Base network
+
+```typescript
+import { ENSResolver, BasenameResolver } from '@hybrd/xmtp/resolver'
+
+const ensResolver = new ENSResolver({ mainnetClient })
+const basenameResolver = new BasenameResolver({ publicClient: baseClient })
+``` 

package/biome.jsonc

@@ -0,0 +1,4 @@
+{
+	"$schema": "https://biomejs.dev/schemas/1.9.4/schema.json",
+	"extends": ["@hybrd/biome"]
+}

package/package.json

@@ -0,0 +1,46 @@
+{
+  "name": "@hybrd/xmtp",
+  "version": "1.0.0",
+  "type": "module",
+  "exports": {
+    ".": {
+      "import": "./src/index.ts",
+      "types": "./src/index.ts"
+    }
+  },
+  "dependencies": {
+    "@coinbase/coinbase-sdk": "^0.22.0",
+    "@xmtp/content-type-group-updated": "2.0.2",
+    "@xmtp/content-type-reaction": "2.0.2",
+    "@xmtp/content-type-reply": "2.0.2",
+    "@xmtp/content-type-text": "2.0.2",
+    "@xmtp/content-type-transaction-reference": "2.0.2",
+    "@xmtp/content-type-wallet-send-calls": "2.0.0",
+    "@xmtp/node-sdk": "^4.0.1",
+    "jsonwebtoken": "^9.0.2",
+    "uint8arrays": "^5.1.0",
+    "viem": "^2.22.17"
+  },
+  "devDependencies": {
+    "@types/jsonwebtoken": "^9.0.7",
+    "@types/node": "22.8.6",
+    "vitest": "^3.2.4",
+    "@hybrd/biome": "0.0.0",
+    "@hybrd/tsconfig": "0.0.0"
+  },
+  "scripts": {
+    "clean": "rm -rf .turbo",
+    "gen:keys": "tsx scripts/generate-keys.ts",
+    "register": "tsx scripts/register-wallet.ts",
+    "revoke": "tsx scripts/revoke-installations.ts",
+    "revoke:all": "tsx scripts/revoke-all-installations.ts",
+    "enhanced:demo": "tsx scripts/enhanced-connection-example.ts",
+    "test:messages": "tsx scripts/test-message-reception.ts",
+    "refresh:identity": "tsx scripts/refresh-identity.ts",
+    "custom:storage": "tsx scripts/custom-storage-example.ts",
+    "typecheck": "tsc --noEmit",
+    "lint": "biome lint --unsafe",
+    "lint:fix": "biome lint --write --unsafe",
+    "format": "biome format --write"
+  }
+}

package/scripts/generate-keys.ts

@@ -0,0 +1,25 @@
+import { generateEncryptionKeyHex } from "@hybrd/xmtp"
+import { generatePrivateKey, privateKeyToAccount } from "viem/accounts"
+
+// Check Node.js version
+const nodeVersion = process.versions.node
+const [major] = nodeVersion?.split(".").map(Number) || [0]
+
+if (major && major < 20) {
+	console.error("Error: Node.js version 20 or higher is required")
+	process.exit(1)
+}
+
+console.log("Generating XMTP keys...")
+
+const walletKey = generatePrivateKey()
+const account = privateKeyToAccount(walletKey)
+const encryptionKeyHex = generateEncryptionKeyHex()
+const publicKey = account.address
+
+console.log("\n# === Generated Keys ===")
+console.log(`# Public Address: ${publicKey}`)
+console.log(`XMTP_WALLET_KEY=${walletKey}`)
+console.log(`XMTP_ENCRYPTION_KEY=${encryptionKeyHex}`)
+
+console.log("\nCopy the above environment variables to your .env file")

package/scripts/refresh-identity.ts

@@ -0,0 +1,119 @@
+import {
+	createXMTPClient,
+	logAgentDetails,
+	validateEnvironment
+} from "../src/client"
+
+/**
+ * Refresh XMTP Identity Script
+ *
+ * This script refreshes the bot's XMTP identity on the production network
+ * to resolve association errors and missing identity updates
+ */
+
+async function refreshXMTPIdentity() {
+	console.log("🔄 XMTP Identity Refresh")
+	console.log("========================")
+
+	// Validate environment
+	const { XMTP_WALLET_KEY } = validateEnvironment(["XMTP_WALLET_KEY"])
+
+	if (!XMTP_WALLET_KEY) {
+		console.error("❌ XMTP_WALLET_KEY is required")
+		process.exit(1)
+	}
+
+	try {
+		console.log(`🌐 Environment: ${process.env.XMTP_ENV || "dev"}`)
+
+		// Step 1: Create client with persistence to force identity refresh
+		console.log("\n📋 Step 1: Creating client with persistence...")
+		const persistentClient = await createXMTPClient(XMTP_WALLET_KEY, {
+			persist: true
+		})
+
+		console.log(
+			`🔑 Wallet Address: ${persistentClient.accountIdentifier?.identifier}`
+		)
+		console.log(`🌐 XMTP Environment: ${process.env.XMTP_ENV || "dev"}`)
+
+		// Step 2: Force full sync
+		console.log("\n📋 Step 2: Forcing full conversation sync...")
+		await persistentClient.conversations.sync()
+
+		// Step 3: List conversations
+		const conversations = await persistentClient.conversations.list()
+		console.log(`📬 Found ${conversations.length} conversations`)
+
+		// Step 4: Display agent details
+		console.log("\n📋 Step 3: Displaying refreshed identity details...")
+		await logAgentDetails(persistentClient)
+
+		// Step 5: Test identity resolution
+		console.log("\n📋 Step 4: Testing identity resolution...")
+
+		if (conversations.length > 0) {
+			const testConv = conversations[0]
+			if (!testConv) {
+				console.log("❌ No valid conversation found")
+				return
+			}
+
+			try {
+				// Get all participants
+				const members = await testConv.members()
+				console.log(`👥 Conversation members: ${members.length}`)
+
+				for (const member of members) {
+					console.log(`   - Inbox ID: ${member.inboxId}`)
+					console.log(`   - Installation IDs: ${member.installationIds.length}`)
+
+					// Try to resolve addresses
+					try {
+						const inboxState =
+							await persistentClient.preferences.inboxStateFromInboxIds([
+								member.inboxId
+							])
+						if (
+							inboxState.length > 0 &&
+							inboxState[0] &&
+							inboxState[0].identifiers.length > 0
+						) {
+							const identifier = inboxState[0]?.identifiers[0]?.identifier
+							console.log(`   - Address: ${identifier || "Unable to resolve"}`)
+						} else {
+							console.log(`   - Address: Unable to resolve`)
+						}
+					} catch (error) {
+						const err = error as Error
+						console.log(`   - Address: Error resolving - ${err.message}`)
+					}
+				}
+			} catch (error) {
+				const err = error as Error
+				console.log(`❌ Error testing conversation: ${err.message}`)
+			}
+		}
+
+		console.log("\n✅ Identity refresh completed successfully!")
+		console.log("🔄 Try processing messages again")
+	} catch (error) {
+		const err = error as Error
+		console.error("❌ Identity refresh failed:", err)
+
+		if (err.message.includes("XMTP_ENCRYPTION_KEY")) {
+			console.log("\n💡 Add XMTP_ENCRYPTION_KEY to your environment:")
+			console.log("   export XMTP_ENCRYPTION_KEY=your_key_here")
+			console.log(
+				"   Or run: pnpm with-env pnpm --filter @hybrd/xmtp refresh:identity"
+			)
+		}
+
+		process.exit(1)
+	}
+}
+
+// Run the refresh
+if (require.main === module) {
+	refreshXMTPIdentity().catch(console.error)
+}

package/scripts/register-wallet.ts

@@ -0,0 +1,95 @@
+import { createSigner, createXMTPClient, getDbPath, logAgentDetails, validateEnvironment } from "../src/client"
+
+async function registerOnProduction() {
+  console.log("🚀 Starting XMTP Production Network Registration...")
+
+  // Validate required environment variables
+  const { XMTP_WALLET_KEY } = validateEnvironment([
+    "XMTP_WALLET_KEY",
+    "XMTP_ENCRYPTION_KEY"
+  ])
+
+  if (!XMTP_WALLET_KEY) {
+    console.error("❌ XMTP_WALLET_KEY is required for registration")
+    process.exit(1)
+  }
+
+  try {
+    console.log("🔑 Creating signer...")
+    const signer = createSigner(XMTP_WALLET_KEY)
+
+    // Get wallet address for logging
+    const identifier = await signer.getIdentifier()
+    const address = identifier.identifier
+    console.log(`📍 Wallet Address: ${address}`)
+
+    console.log("🌐 Connecting to XMTP Production Network...")
+    console.log("⚠️  This will prompt you to sign messages in your wallet")
+    console.log("   - 'XMTP : Authenticate to inbox' message")
+    console.log("   - 'Grant messaging access to app' message")
+    console.log("   - 'Create inbox' message (if first time)")
+
+    // Use getDbPath to ensure directory creation and proper path handling
+    const dbPath = getDbPath(`production-${address}`)
+    console.log(`📁 Database path: ${dbPath}`)
+
+    // Connect to production network
+    const client = await createXMTPClient(XMTP_WALLET_KEY)
+
+    console.log("✅ Successfully connected to XMTP Production Network!")
+
+    // Log client details
+    await logAgentDetails(client)
+
+    console.log("📡 Syncing conversations...")
+    await client.conversations.sync()
+
+    const conversations = await client.conversations.list()
+    console.log(`💬 Found ${conversations.length} existing conversations`)
+
+    console.log("🎉 Registration Complete!")
+    console.log(`
+✓ Wallet ${address} is now registered on XMTP Production Network
+✓ Inbox ID: ${client.inboxId}
+✓ Database: production-${address}.db3
+✓ Ready to receive messages on production network
+
+Next steps:
+1. Update your environment: XMTP_ENV=production
+2. Start your listener service
+3. Share your address for others to message: ${address}
+4. Test messaging at: https://xmtp.chat/dm/${address}
+    `)
+  } catch (error) {
+    console.error("❌ Registration failed:", error)
+
+    if (error instanceof Error) {
+      if (error.message.includes("User rejected")) {
+        console.log(
+          "📝 Registration was cancelled. You need to approve the wallet signatures to complete registration."
+        )
+      } else if (error.message.includes("network")) {
+        console.log(
+          "🌐 Network connection issue. Please check your internet connection and try again."
+        )
+      } else if (
+        error.message.includes("database") ||
+        error.message.includes("Unable to open")
+      ) {
+        console.log(
+          "💾 Database access issue. Please check file permissions and ensure the directory exists."
+        )
+      } else {
+        console.log("💡 Make sure your wallet is connected and try again.")
+      }
+    }
+
+    process.exit(1)
+  }
+}
+
+// Run registration
+registerOnProduction().catch((error) => {
+  console.error("💥 Fatal error during registration:", error)
+  process.exit(1)
+})