@dignetwork/chia-block-listener 0.1.9 → 0.1.12

package/README.md

@@ -6,10 +6,15 @@ A high-performance Chia blockchain listener for Node.js, built with Rust and NAP
 
 - **Real-time Block Monitoring**: Listen for new blocks as they're produced on the Chia network
 - **Peer Management**: Connect to multiple Chia full nodes simultaneously
+- **Automatic Failover**: Intelligent peer failover with automatic retry across multiple peers
+- **Enhanced Error Handling**: Automatic disconnection of peers that refuse blocks or have protocol errors
 - **Efficient Parsing**: Fast extraction of coin spends, additions, and removals from blocks
 - **Event-Driven Architecture**: TypeScript-friendly event system with full type safety
 - **Transaction Analysis**: Parse CLVM puzzles and solutions from coin spends
-- **Historical Block Access**: Retrieve blocks by height or ranges
+- **Historical Block Access**: Retrieve blocks by height or ranges with automatic load balancing
+- **Connection Pool**: ChiaPeerPool provides automatic load balancing and rate limiting for historical queries
+- **Peak Height Tracking**: Monitor blockchain sync progress across all connected peers
+- **DNS Peer Discovery**: Automatic peer discovery using Chia network DNS introducers
 - **Cross-platform Support**: Works on Windows, macOS, and Linux (x64 and ARM64)
 - **TypeScript Support**: Complete TypeScript definitions with IntelliSense
 
@@ -33,11 +38,11 @@ const listener = new ChiaBlockListener()
 // Listen for block events
 listener.on('blockReceived', (block) => {
   console.log(`New block received: ${block.height}`)
-  console.log(`Header hash: ${block.header_hash}`)
+  console.log(`Header hash: ${block.headerHash}`)
   console.log(`Timestamp: ${new Date(block.timestamp * 1000)}`)
-  console.log(`Coin additions: ${block.coin_additions.length}`)
-  console.log(`Coin removals: ${block.coin_removals.length}`)
-  console.log(`Coin spends: ${block.coin_spends.length}`)
+  console.log(`Coin additions: ${block.coinAdditions.length}`)
+  console.log(`Coin removals: ${block.coinRemovals.length}`)
+  console.log(`Coin spends: ${block.coinSpends.length}`)
 })
 
 // Listen for peer connection events
@@ -64,6 +69,30 @@ process.on('SIGINT', () => {
 })
 ```
 
+## Recent Enhancements 🚀
+
+This library includes several powerful enhancements for production use:
+
+### Automatic Failover System
+- **Intelligent Retry Logic**: Automatically tries up to 3 different peers when block requests fail
+- **Smart Peer Selection**: Chooses the best available peer for each request based on response times
+- **Transparent Operation**: Failover happens automatically without user intervention
+
+### Enhanced Error Handling
+- **Protocol Error Detection**: Automatically detects and disconnects peers that refuse blocks or send invalid data
+- **Connection Health Monitoring**: Tracks peer connection health and removes problematic peers
+- **Error Classification**: Distinguishes between temporary issues and permanent peer problems
+
+### Performance Optimizations
+- **Aggressive Rate Limiting**: 50ms rate limiting per peer for maximum throughput
+- **Persistent Connections**: Reuses WebSocket connections for optimal performance
+- **Parallel Processing**: Efficient handling of multiple concurrent block requests
+
+### Developer Experience
+- **camelCase API**: All JavaScript methods use proper camelCase naming conventions
+- **Comprehensive Events**: Detailed event system for monitoring peer lifecycle and errors
+- **TypeScript Support**: Full type safety with complete TypeScript definitions
+
 ## API Reference
 
 ### ChiaBlockListener Class
@@ -127,6 +156,80 @@ Retrieves a range of blocks from a connected peer.
 
 **Returns:** An array of `BlockReceivedEvent` objects
 
+### ChiaPeerPool Class
+
+The `ChiaPeerPool` provides a managed pool of peer connections for retrieving historical blocks with automatic load balancing and intelligent failover across multiple peers. When a peer fails to provide a block or experiences protocol errors, the pool automatically tries alternative peers and removes problematic peers from the pool.
+
+#### Constructor
+
+```javascript
+const pool = new ChiaPeerPool()
+```
+
+Creates a new peer pool instance with built-in rate limiting (500ms per peer).
+
+#### Methods
+
+##### `addPeer(host, port, networkId): Promise<string>`
+
+Adds a peer to the connection pool.
+
+**Parameters:**
+- `host` (string): The hostname or IP address of the Chia node
+- `port` (number): The port number (typically 8444 for mainnet)
+- `networkId` (string): The network identifier ('mainnet', 'testnet', etc.)
+
+**Returns:** A Promise that resolves to a unique peer ID string
+
+##### `getBlockByHeight(height): Promise<BlockReceivedEvent>`
+
+Retrieves a specific block by height using automatic peer selection and load balancing.
+
+**Parameters:**
+- `height` (number): The block height to retrieve
+
+**Returns:** A Promise that resolves to a `BlockReceivedEvent` object
+
+##### `removePeer(peerId): Promise<boolean>`
+
+Removes a peer from the pool.
+
+**Parameters:**
+- `peerId` (string): The peer ID to remove
+
+**Returns:** A Promise that resolves to `true` if the peer was removed, `false` otherwise
+
+##### `shutdown(): Promise<void>`
+
+Shuts down the pool and disconnects all peers.
+
+##### `getConnectedPeers(): Promise<string[]>`
+
+Gets the list of currently connected peer IDs.
+
+**Returns:** Array of peer ID strings (format: "host:port")
+
+##### `getPeakHeight(): Promise<number | null>`
+
+Gets the highest blockchain peak height seen across all connected peers.
+
+**Returns:** The highest peak height as a number, or null if no peaks have been received yet
+
+##### `on(event, callback): void`
+
+Registers an event handler for pool events.
+
+**Parameters:**
+- `event` (string): The event name ('peerConnected' or 'peerDisconnected')
+- `callback` (function): The event handler function
+
+##### `off(event, callback): void`
+
+Removes an event handler.
+
+**Parameters:**
+- `event` (string): The event name to stop listening for
+
 ### Events
 
 The `ChiaBlockListener` emits the following events:
@@ -149,6 +252,28 @@ Fired when a peer connection is lost.
 
 **Callback:** `(event: PeerDisconnectedEvent) => void`
 
+### ChiaPeerPool Events
+
+The `ChiaPeerPool` emits the following events:
+
+#### `peerConnected`
+
+Fired when a peer is successfully added to the pool.
+
+**Callback:** `(event: PeerConnectedEvent) => void`
+
+#### `peerDisconnected`
+
+Fired when a peer is removed from the pool or disconnects.
+
+**Callback:** `(event: PeerDisconnectedEvent) => void`
+
+#### `newPeakHeight`
+
+Fired when a new highest blockchain peak is discovered.
+
+**Callback:** `(event: NewPeakHeightEvent) => void`
+
 ### Event Data Types
 
 #### `BlockReceivedEvent`
@@ -190,6 +315,16 @@ interface PeerDisconnectedEvent {
 }
 ```
 
+#### `NewPeakHeightEvent`
+
+```typescript
+interface NewPeakHeightEvent {
+  oldPeak: number | null  // Previous highest peak (null if first peak)
+  newPeak: number        // New highest peak height
+  peerId: string         // Peer that discovered this peak
+}
+```
+
 #### `CoinRecord`
 
 ```typescript
@@ -211,14 +346,219 @@ interface CoinSpend {
 }
 ```
 
+## ChiaPeerPool Usage
+
+The `ChiaPeerPool` is designed for efficiently retrieving historical blocks with automatic load balancing and intelligent failover across multiple peers. When a peer fails to provide a block or experiences protocol errors, the pool automatically tries alternative peers and removes problematic peers from the pool.
+
+### Basic Usage
+
+```javascript
+const { ChiaPeerPool, initTracing } = require('@dignetwork/chia-block-listener')
+
+async function main() {
+  // Initialize tracing
+  initTracing()
+  
+  // Create a peer pool
+  const pool = new ChiaPeerPool()
+  
+  // Listen for pool events
+  pool.on('peerConnected', (event) => {
+    console.log(`Peer connected to pool: ${event.peerId}`)
+  })
+  
+  pool.on('peerDisconnected', (event) => {
+    console.log(`Peer disconnected from pool: ${event.peerId}`)
+  })
+  
+  pool.on('newPeakHeight', (event) => {
+    console.log(`New blockchain peak detected!`)
+    console.log(`  Previous: ${event.oldPeak || 'None'}`)
+    console.log(`  New: ${event.newPeak}`)
+    console.log(`  Discovered by: ${event.peerId}`)
+  })
+  
+  // Add multiple peers
+  await pool.addPeer('node1.chia.net', 8444, 'mainnet')
+  await pool.addPeer('node2.chia.net', 8444, 'mainnet')
+  await pool.addPeer('node3.chia.net', 8444, 'mainnet')
+  
+  // Fetch blocks with automatic load balancing
+  const block1 = await pool.getBlockByHeight(5000000)
+  const block2 = await pool.getBlockByHeight(5000001)
+  const block3 = await pool.getBlockByHeight(5000002)
+  
+  console.log(`Block ${block1.height}: ${block1.coinSpends.length} spends`)
+  console.log(`Block ${block2.height}: ${block2.coinSpends.length} spends`)
+  console.log(`Block ${block3.height}: ${block3.coinSpends.length} spends`)
+  
+  // Shutdown the pool
+  await pool.shutdown()
+}
+
+main().catch(console.error)
+```
+
+### Advanced Pool Features
+
+#### Rate Limiting
+
+The pool automatically enforces a 50ms rate limit per peer for maximum performance while preventing node overload:
+
+```javascript
+// Rapid requests are automatically queued and distributed
+const promises = []
+for (let i = 5000000; i < 5000100; i++) {
+  promises.push(pool.getBlockByHeight(i))
+}
+
+// All requests will be processed efficiently across all peers
+// with automatic load balancing and rate limiting
+const blocks = await Promise.all(promises)
+console.log(`Retrieved ${blocks.length} blocks`)
+```
+
+#### Automatic Failover and Error Handling
+
+The pool provides robust error handling with automatic failover:
+
+```javascript
+// The pool automatically handles various error scenarios:
+
+// 1. Connection failures - automatically tries other peers
+try {
+  const block = await pool.getBlockByHeight(5000000)
+  console.log(`Retrieved block ${block.height}`)
+} catch (error) {
+  // If all peers fail, you'll get an error after all retry attempts
+  console.error('All peers failed:', error.message)
+}
+
+// 2. Protocol errors - peers that refuse blocks are automatically disconnected
+pool.on('peerDisconnected', (event) => {
+  console.log(`Peer ${event.peerId} disconnected: ${event.reason}`)
+  // Reasons include: "Block request rejected", "Protocol error", "Connection timeout"
+})
+
+// 3. Automatic peer cleanup - problematic peers are removed from the pool
+console.log('Active peers before:', await pool.getConnectedPeers())
+await pool.getBlockByHeight(5000000) // May trigger peer removal
+console.log('Active peers after:', await pool.getConnectedPeers())
+
+// 4. Multiple retry attempts - tries up to 3 different peers per request
+// This happens automatically and transparently
+const block = await pool.getBlockByHeight(5000000) // Will try multiple peers if needed
+```
+
+**Error Types Handled Automatically:**
+- **Connection Errors**: Timeouts, network failures, WebSocket errors
+- **Protocol Errors**: Block rejections, parsing failures, handshake failures
+- **Peer Misbehavior**: Unexpected responses, invalid data formats
+
+#### Dynamic Peer Management
+
+```javascript
+// Monitor pool health
+const peers = await pool.getConnectedPeers()
+console.log(`Active peers in pool: ${peers.length}`)
+
+// Remove underperforming peers
+if (slowPeer) {
+  await pool.removePeer(slowPeer)
+  console.log('Removed slow peer from pool')
+}
+
+// Add new peers dynamically
+if (peers.length < 3) {
+  await pool.addPeer('backup-node.chia.net', 8444, 'mainnet')
+}
+```
+
+#### Error Handling
+
+```javascript
+try {
+  const block = await pool.getBlockByHeight(5000000)
+  console.log(`Retrieved block ${block.height}`)
+} catch (error) {
+  console.error('Failed to retrieve block:', error)
+  
+  // The pool will automatically try other peers
+  // You can also add more peers if needed
+  const peers = await pool.getConnectedPeers()
+  if (peers.length === 0) {
+    console.log('No peers available, adding new ones...')
+    await pool.addPeer('node1.chia.net', 8444, 'mainnet')
+  }
+}
+```
+
+#### Peak Height Tracking
+
+```javascript
+// Monitor blockchain sync progress
+const pool = new ChiaPeerPool()
+
+// Track peak changes
+let currentPeak = null
+pool.on('newPeakHeight', (event) => {
+  currentPeak = event.newPeak
+  const progress = event.oldPeak 
+    ? `+${event.newPeak - event.oldPeak} blocks` 
+    : 'Initial peak'
+  console.log(`Peak update: ${event.newPeak} (${progress})`)
+})
+
+// Add peers
+await pool.addPeer('node1.chia.net', 8444, 'mainnet')
+await pool.addPeer('node2.chia.net', 8444, 'mainnet')
+
+// Check current peak
+const peak = await pool.getPeakHeight()
+console.log(`Current highest peak: ${peak || 'None yet'}`)
+
+// Fetch some blocks to trigger peak updates
+await pool.getBlockByHeight(5000000)
+await pool.getBlockByHeight(5100000)
+await pool.getBlockByHeight(5200000)
+
+// Monitor sync status
+setInterval(async () => {
+  const peak = await pool.getPeakHeight()
+  if (peak) {
+    const estimatedCurrent = 5200000 + Math.floor((Date.now() / 1000 - 1700000000) / 18.75)
+    const syncPercentage = (peak / estimatedCurrent * 100).toFixed(2)
+    console.log(`Sync status: ${syncPercentage}% (peak: ${peak})`)
+  }
+}, 60000) // Check every minute
+```
+
+### When to Use ChiaPeerPool vs ChiaBlockListener
+
+- **Use ChiaPeerPool when:**
+  - You need to fetch historical blocks
+  - You want automatic load balancing across multiple peers
+  - You're making many block requests and need rate limiting
+  - You don't need real-time block notifications
+
+- **Use ChiaBlockListener when:**
+  - You need real-time notifications of new blocks
+  - You want to monitor the blockchain as it grows
+  - You need to track specific addresses or puzzle hashes in real-time
+  - You're building applications that react to blockchain events
+
+Both classes can be used together in the same application for different purposes.
+
 ## TypeScript Usage
 
 ```typescript
 import { 
   ChiaBlockListener, 
+  ChiaPeerPool,
   BlockReceivedEvent, 
   PeerConnectedEvent, 
   PeerDisconnectedEvent,
+  NewPeakHeightEvent,
   CoinRecord,
   CoinSpend,
   initTracing,
@@ -236,14 +576,14 @@ listener.on('blockReceived', (block: BlockReceivedEvent) => {
   console.log(`Block ${block.height} from peer ${block.peerId}`)
   
   // Process coin additions
-  block.coin_additions.forEach((coin: CoinRecord) => {
+  block.coinAdditions.forEach((coin: CoinRecord) => {
     console.log(`New coin: ${coin.amount} mojos`)
   })
   
   // Process coin spends
-  block.coin_spends.forEach((spend: CoinSpend) => {
+  block.coinSpends.forEach((spend: CoinSpend) => {
     console.log(`Spend: ${spend.coin.amount} mojos`)
-    console.log(`Puzzle: ${spend.puzzle_reveal}`)
+    console.log(`Puzzle: ${spend.puzzleReveal}`)
     console.log(`Solution: ${spend.solution}`)
   })
 })
@@ -267,7 +607,7 @@ const testnetPeer = listener.addPeer('testnet-node.chia.net', 58444, 'testnet')
 async function getHistoricalBlocks() {
   try {
     const block = listener.getBlockByHeight(mainnetPeer, 1000000)
-    console.log(`Block 1000000 hash: ${block.header_hash}`)
+    console.log(`Block 1000000 hash: ${block.headerHash}`)
     
     const blocks = listener.getBlocksRange(mainnetPeer, 1000000, 1000010)
     console.log(`Retrieved ${blocks.length} blocks`)
@@ -279,6 +619,29 @@ async function getHistoricalBlocks() {
 // Get event type constants
 const eventTypes = getEventTypes()
 console.log('Available events:', eventTypes)
+
+// TypeScript support for ChiaPeerPool
+const pool = new ChiaPeerPool()
+
+// Type-safe event handling
+pool.on('peerConnected', (event: PeerConnectedEvent) => {
+  console.log(`Pool peer connected: ${event.peerId}`)
+})
+
+pool.on('newPeakHeight', (event: NewPeakHeightEvent) => {
+  console.log(`New peak: ${event.oldPeak} → ${event.newPeak}`)
+})
+
+// Async/await with proper typing
+async function fetchHistoricalData() {
+  const block: BlockReceivedEvent = await pool.getBlockByHeight(5000000)
+  const peers: string[] = await pool.getConnectedPeers()
+  const peak: number | null = await pool.getPeakHeight()
+  
+  console.log(`Block ${block.height} has ${block.coinSpends.length} spends`)
+  console.log(`Pool has ${peers.length} active peers`)
+  console.log(`Current peak: ${peak || 'No peak yet'}`)
+}
 ```
 
 ## Advanced Usage
@@ -290,8 +653,8 @@ console.log('Available events:', eventTypes)
 listener.on('blockReceived', (block) => {
   const targetPuzzleHash = '0x1234...' // Your puzzle hash
   
-  block.coin_spends.forEach((spend) => {
-    if (spend.coin.puzzle_hash === targetPuzzleHash) {
+  block.coinSpends.forEach((spend) => {
+    if (spend.coin.puzzleHash === targetPuzzleHash) {
       console.log('Found spend for our puzzle!')
       console.log('Amount:', spend.coin.amount)
       console.log('Solution:', spend.solution)

package/crate/chia-generator-parser/src/parser.rs

@@ -1,9 +1,6 @@
 use crate::{
     error::{GeneratorParserError, Result},
-    types::{
-        BlockHeightInfo, CoinInfo, CoinSpendInfo, GeneratorAnalysis, GeneratorBlockInfo,
-        ParsedBlock, ParsedGenerator,
-    },
+    types::{BlockHeightInfo, CoinInfo, CoinSpendInfo, GeneratorBlockInfo, ParsedBlock},
 };
 use chia_bls::Signature;
 use chia_consensus::{
@@ -14,7 +11,7 @@ use chia_consensus::{
     run_block_generator::{run_block_generator2, setup_generator_args},
     validation_error::{atom, first, next, rest, ErrorCode},
 };
-use chia_protocol::{Bytes32, FullBlock};
+use chia_protocol::FullBlock;
 use chia_traits::streamable::Streamable;
 use clvm_utils::tree_hash;
 use clvmr::{
@@ -63,7 +60,7 @@ impl BlockParser {
             .map(|g| g.len() as u32);
 
         // Extract generator info
-        let generator_info = block
+        let _generator_info = block
             .transactions_generator
             .as_ref()
             .map(|gen| GeneratorBlockInfo {
@@ -101,7 +98,6 @@ impl BlockParser {
             coin_creations,
             has_transactions_generator,
             generator_size,
-            generator_info,
         })
     }
 
@@ -373,15 +369,15 @@ impl BlockParser {
         // Get created coins from conditions
         let created_coins = self.extract_created_coins(spend_index, spend_bundle_conditions);
 
-        Some(CoinSpendInfo {
-            coin: coin_info,
-            puzzle_reveal,
-            solution: solution_bytes,
-            real_data: true,
-            parsing_method: "clvm_execution".to_string(),
-            offset: 0,
+        Some(CoinSpendInfo::new(
+            coin_info,
+            hex::encode(puzzle_reveal),
+            hex::encode(solution_bytes),
+            true,
+            "From transaction generator".to_string(),
+            0,
             created_coins,
-        })
+        ))
     }
 
     /// Extract parent coin info from a coin spend node
@@ -464,42 +460,62 @@ impl BlockParser {
         })
     }
 
+    /*
     /// Parse generator from hex string
     pub fn parse_generator_from_hex(&self, generator_hex: &str) -> Result<ParsedGenerator> {
         let generator_bytes =
-            hex::decode(generator_hex).map_err(|e| GeneratorParserError::HexDecodingError(e))?;
+            hex::decode(generator_hex).map_err(|e| ParseError::HexDecodingError(e))?;
         self.parse_generator_from_bytes(&generator_bytes)
     }
 
     /// Parse generator from bytes
     pub fn parse_generator_from_bytes(&self, generator_bytes: &[u8]) -> Result<ParsedGenerator> {
-        let analysis = self.analyze_generator(generator_bytes)?;
-
+        // Create a dummy GeneratorBlockInfo for now
         Ok(ParsedGenerator {
-            block_info: GeneratorBlockInfo {
-                prev_header_hash: Bytes32::default(),
-                transactions_generator: Some(generator_bytes.to_vec()),
-                transactions_generator_ref_list: Vec::new(),
-            },
+            block_info: GeneratorBlockInfo::new(
+                [0u8; 32].into(),
+                Some(generator_bytes.to_vec()),
+                vec![],
+            ),
             generator_hex: Some(hex::encode(generator_bytes)),
-            analysis,
+            analysis: self.analyze_generator(generator_bytes)?,
         })
     }
 
     /// Analyze generator bytecode
     pub fn analyze_generator(&self, generator_bytes: &[u8]) -> Result<GeneratorAnalysis> {
         let size_bytes = generator_bytes.len();
-        let is_empty = size_bytes == 0;
-
-        // Check for CLVM patterns
-        let contains_clvm_patterns = generator_bytes
-            .windows(2)
-            .any(|w| matches!(w, [0xff, _] | [_, 0xff]));
-
-        // Check for coin patterns (CREATE_COIN opcode)
-        let contains_coin_patterns = generator_bytes.windows(1).any(|w| w[0] == 0x33);
+        let is_empty = generator_bytes.is_empty();
+
+        // Check for common CLVM patterns
+        let contains_clvm_patterns = generator_bytes.windows(2).any(|w| {
+            w == [0x01, 0x00] || // pair
+            w == [0x02, 0x00] || // cons
+            w == [0x03, 0x00] || // first
+            w == [0x04, 0x00]    // rest
+        });
+
+        // Check for coin patterns (32-byte sequences)
+        let contains_coin_patterns = generator_bytes.len() >= 32;
+
+        // Calculate simple entropy
+        let mut byte_counts = [0u64; 256];
+        for &byte in generator_bytes {
+            byte_counts[byte as usize] += 1;
+        }
 
-        let entropy = self.calculate_entropy(generator_bytes);
+        let total = generator_bytes.len() as f64;
+        let entropy = if total > 0.0 {
+            byte_counts.iter()
+                .filter(|&&count| count > 0)
+                .map(|&count| {
+                    let p = count as f64 / total;
+                    -p * p.log2()
+                })
+                .sum()
+        } else {
+            0.0
+        };
 
         Ok(GeneratorAnalysis {
             size_bytes,
@@ -509,8 +525,10 @@ impl BlockParser {
             entropy,
         })
     }
+    */
 
     /// Calculate Shannon entropy of data
+    #[allow(dead_code)]
     fn calculate_entropy(&self, data: &[u8]) -> f64 {
         if data.is_empty() {
             return 0.0;