holosphere 1.1.5 → 1.1.7

package/FEDERATION.md

@@ -0,0 +1,213 @@
+# HoloSphere Federation
+
+HoloSphere's federation system allows different holons (spaces) to share and access data from each other. Federation creates a relationship between spaces that enables data propagation and cross-space access.
+
+## Key Concepts
+
+- **Federation Relationship**: A connection between two spaces that allows data to flow between them.
+- **Soul References**: Lightweight references that point to data in its original location (single source of truth).
+- **Notification Flow**: Data notifications flow from source spaces to target spaces that are in the notify list.
+- **Source-Target Relationship**: Each federation sets up a source space (federation list) and a target space (notify list).
+
+## Federation Data Flow
+
+The HoloSphere federation system works with a clear source-target relationship:
+
+1. **Federation List**: When space A federates with space B, space A adds B to its federation list.
+2. **Notify List**: Space B adds space A to its notify list.
+3. **Data Flow**: When space A changes, space B gets notified (but not vice versa unless bidirectional).
+
+## Creating Federation
+
+Create federation relationships between spaces:
+
+```javascript
+// Create federation between space1 and space2
+await holoSphere.federate('space1', 'space2');
+
+// This sets up:
+// - space1.federation includes space2
+// - space2.notify includes space1
+```
+
+The bidirectional parameter is largely unused in the current implementation since the federation system naturally sets up the correct notification flow. The default relationship allows space2 to be notified of changes in space1.
+
+## Storing and Propagating Data
+
+Data must be explicitly propagated to federated spaces:
+
+```javascript
+const data = {
+  id: 'item1',
+  title: 'Federation Example',
+  value: 42
+};
+
+// Store data in space1
+await holoSphere.put('space1', 'items', data);
+
+// Propagate to federated spaces
+await holoSphere.propagate('space1', 'items', data);
+```
+
+You can also enable automatic propagation in the `put()` method:
+
+```javascript
+// Store data and automatically propagate
+await holoSphere.put('space1', 'items', data, null, {
+  autoPropagate: true
+});
+```
+
+## Accessing Federated Data
+
+### Direct Retrieval
+
+You can access data directly from any space:
+
+```javascript
+// Retrieve data from space2 (will resolve reference if it's a reference)
+const data = await holoSphere.get('space2', 'items', 'item1', null, {
+  resolveReferences: true  // Default is true
+});
+```
+
+### Aggregate Federated Data
+
+Use `getFederated()` to get data from multiple federated spaces:
+
+```javascript
+// Get combined data from the local space and all its federated spaces
+const federatedData = await holoSphere.getFederated('space2', 'items', {
+  resolveReferences: true,  // Default: true
+  idField: 'id'             // Field to use as the unique identifier
+});
+```
+
+## Soul References
+
+HoloSphere uses a simplified reference system based on soul paths:
+
+1. A reference contains only an `id` and a `soul` property
+2. The soul path is in the format: `appname/holon/lens/key`
+3. When resolving a reference, HoloSphere follows the soul path to retrieve the original data
+
+By default, federation propagation uses references instead of duplicating data. This can be controlled:
+
+```javascript
+// Propagate with full data copy instead of references
+await holoSphere.propagate('space1', 'items', data, {
+  useReferences: false
+});
+```
+
+## Removing Federation
+
+```javascript
+// Remove federation relationship
+await holoSphere.unfederate('space1', 'space2');
+```
+
+## Complete Example
+
+Here's a complete example showing the proper way to set up and use federation:
+
+```javascript
+import HoloSphere from './holosphere.js';
+
+async function federationExample() {
+  const holoSphere = new HoloSphere('example-app');
+  
+  try {
+    const space1 = 'public-space1';
+    const space2 = 'public-space2';
+    
+    // Step 1: Create federation relationship
+    await holoSphere.federate(space1, space2);
+    
+    // Step 2: Verify federation is set up properly
+    const fedInfo1 = await holoSphere.getFederation(space1);
+    const fedInfo2 = await holoSphere.getFederation(space2);
+    
+    console.log(`Federation info for ${space1}:`, fedInfo1);
+    // Should include: federation: ['space2']
+    
+    console.log(`Federation info for ${space2}:`, fedInfo2);
+    // Should include: notify: ['space1']
+    
+    // Step 3: Store data in space1
+    const item = { 
+      id: 'item1', 
+      title: 'Federation Test', 
+      value: 42 
+    };
+    
+    await holoSphere.put(space1, 'items', item);
+    
+    // Step 4: Propagate data to federated spaces
+    await holoSphere.propagate(space1, 'items', item);
+    
+    // Allow time for propagation
+    await new Promise(resolve => setTimeout(resolve, 1000));
+    
+    // Step 5: Access data from space2 (resolves reference)
+    const itemFromSpace2 = await holoSphere.get(space2, 'items', 'item1');
+    console.log('Item from space2:', itemFromSpace2);
+    
+    // Step 6: Update item in space1
+    const updatedItem = {
+      ...item,
+      value: 100,
+      updated: true
+    };
+    
+    await holoSphere.put(space1, 'items', updatedItem);
+    
+    // Since we're using soul references, the update is immediately visible
+    // through the reference without needing to propagate again
+    
+    // Verify update is visible through the reference
+    const updatedItemFromSpace2 = await holoSphere.get(space2, 'items', 'item1');
+    console.log('Updated item from space2:', updatedItemFromSpace2);
+    
+    // Step 7: Clean up
+    await holoSphere.unfederate(space1, space2);
+  } finally {
+    // Always close the HoloSphere instance
+    await holoSphere.close();
+  }
+}
+
+federationExample().catch(console.error);
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Federation Relationship**: Make sure to check both the federation list and notify list to understand data flow.
+
+2. **Data Propagation**: If data isn't appearing in federated spaces, check:
+   - The federation relationship was created correctly
+   - The data was propagated explicitly or `autoPropagate` was set to `true`
+   - The notify list includes the target space
+
+3. **Reference Resolution**: If you're getting reference objects instead of the actual data:
+   - Make sure `resolveReferences` is set to `true` (it's the default)
+   - Check that the original data still exists at the referenced location
+
+4. **Timing Issues**: Data propagation is asynchronous. Add small delays (500-1000ms) between operations to allow propagation to complete.
+
+### Best Practices
+
+1. **Verify Federation Structure**: After creating a federation, check both spaces to ensure:
+   - Source space has the target in its federation list
+   - Target space has the source in its notify list
+
+2. **Explicit Propagation**: Unless you're using `autoPropagate`, always call `propagate()` explicitly after storing data that should be shared.
+
+3. **Choose the Right Propagation Method**:
+   - Use `useReferences: true` (default) to keep a single source of truth
+   - Use `useReferences: false` only when you need independent copies
+
+4. **Cleanup**: Always close the HoloSphere instance when done to prevent resource leaks. 

package/README.md

@@ -329,3 +329,143 @@ Data in HoloSphere is organized by:
 
 GPL-3.0-or-later
 
+# HoloSphere Federation
+
+HoloSphere provides a federation system that allows spaces to share data and messages across different holons. This document outlines the federation functionality and how to use it.
+
+## Core Federation Features
+
+### Space Federation
+- Create relationships between spaces with clear source-target connections
+- Use soul references to maintain a single source of truth
+- Control data propagation between federated spaces
+- Manage notification settings for each space
+
+### Message Federation
+- Track messages across federated spaces
+- Maintain message relationships between original and federated copies
+- Update messages consistently across all federated spaces
+
+## API Reference
+
+### Space Federation
+
+#### `federate(spaceId1, spaceId2, password1, password2, bidirectional)`
+Creates a federation relationship between two spaces.
+
+```javascript
+await holosphere.federate('space1', 'space2', 'pass1', 'pass2');
+```
+
+This sets up:
+- space1.federation includes space2
+- space2.notify includes space1
+
+Parameters:
+- `spaceId1`: First space ID (source space)
+- `spaceId2`: Second space ID (target space)
+- `password1`: Optional password for first space
+- `password2`: Optional password for second space
+- `bidirectional`: Whether to set up bidirectional notifications (default: true, but generally not needed)
+
+### Data Propagation
+
+#### `propagate(holon, lens, data, options)`
+Propagates data to federated spaces.
+
+```javascript
+await holosphere.propagate('space1', 'items', data, {
+  useReferences: true,         // Default: uses soul references 
+  targetSpaces: ['space2']     // Optional: specific targets
+});
+```
+
+Parameters:
+- `holon`: The holon identifier
+- `lens`: The lens identifier
+- `data`: The data to propagate
+- `options`: Propagation options
+
+Alternatively, you can use auto-propagation:
+
+```javascript
+await holosphere.put('space1', 'items', data, null, {
+  autoPropagate: true  // Enable auto-propagation
+});
+```
+
+### Message Federation
+
+#### `federateMessage(originalChatId, messageId, federatedChatId, federatedMessageId, type)`
+Tracks a federated message across different chats.
+
+```javascript
+await holosphere.federateMessage('chat1', 'msg1', 'chat2', 'msg2', 'quest');
+```
+
+#### `getFederatedMessages(originalChatId, messageId)`
+Gets all federated messages for a given original message.
+
+```javascript
+const messages = await holosphere.getFederatedMessages('chat1', 'msg1');
+```
+
+#### `updateFederatedMessages(originalChatId, messageId, updateCallback)`
+Updates a message across all federated chats.
+
+```javascript
+await holosphere.updateFederatedMessages('chat1', 'msg1', async (chatId, messageId) => {
+    // Update message in this chat
+});
+```
+
+## Usage Example
+
+```javascript
+// Create federation between spaces
+await holosphere.federate('space1', 'space2');
+
+// Store data in space1
+const data = { id: 'item1', value: 42 };
+await holosphere.put('space1', 'items', data);
+
+// Propagate to federated spaces
+await holosphere.propagate('space1', 'items', data);
+
+// Retrieve data from space2 (will resolve the reference)
+const item = await holosphere.get('space2', 'items', 'item1');
+
+// Track a federated message
+await holosphere.federateMessage('chat1', 'msg1', 'chat2', 'msg2', 'quest');
+
+// Update message across all federated chats
+await holosphere.updateFederatedMessages('chat1', 'msg1', async (chatId, messageId) => {
+    await updateMessageInChat(chatId, messageId);
+});
+```
+
+## Soul References
+
+When using the default `useReferences: true` with propagation:
+
+1. Only a lightweight reference is stored in the federated space
+2. The reference contains the original item's ID and soul path
+3. When accessed, the reference is automatically resolved to the original data
+4. Changes to the original data are immediately visible through references
+
+This maintains a single source of truth while keeping storage efficient.
+
+## Federation Structure
+
+The federation system uses two key arrays to manage relationships:
+
+```javascript
+{
+    id: string,
+    name: string,
+    federation: string[],  // Source spaces this holon federates with
+    notify: string[],      // Target spaces to notify of changes
+    timestamp: number
+}
+```
+

package/babel.config.js

@@ -0,0 +1,5 @@
+module.exports = {
+    presets: [
+        ['@babel/preset-env', {targets: {node: 'current'}}],
+    ],
+}; 

package/examples/README-environmental.md

@@ -0,0 +1,158 @@
+# HoloSphere Environmental Data Examples
+
+This directory contains examples demonstrating how to use HoloSphere for storing, federating, and querying environmental data across different geographical scales.
+
+## Overview
+
+The examples show how to:
+
+1. Store environmental data from various APIs into hierarchical holons
+2. Create federation relationships between holons at different scales
+3. Use soul references to efficiently share data between holons
+4. Query environmental data, including resolving federation references
+5. Compute aggregated metrics across holons
+
+## Files
+
+- `environmentalData.js` - Shows how to fetch, transform and store environmental data in HoloSphere
+- `queryEnvironmentalData.js` - Demonstrates how to query the stored data across federated holons
+
+## Environmental Data Structure
+
+The examples organize environmental data into a hierarchical structure:
+
+- **City level (H3 resolution 7)**: Air quality data 
+- **Region level (H3 resolution 5)**: Carbon sequestration and soil data
+- **Country level (H3 resolution 3)**: Biodiversity data
+
+Federation relationships are established between these levels, allowing data to flow from smaller to larger scales while maintaining a single source of truth through soul references.
+
+## Data Schemas
+
+The examples define several schemas for environmental data:
+
+### Air Quality Schema
+```json
+{
+  "id": "string",
+  "holon": "string",
+  "timestamp": "number",
+  "coordinates": {
+    "lat": "number",
+    "lon": "number"
+  },
+  "airQualityIndex": "number",
+  "pollutants": {
+    "co": "number",
+    "no2": "number",
+    "o3": "number",
+    "pm2_5": "number",
+    "pm10": "number",
+    "so2": "number"
+  }
+}
+```
+
+### Carbon Sequestration Schema
+```json
+{
+  "id": "string",
+  "holon": "string",
+  "timestamp": "number",
+  "coordinates": {
+    "lat": "number",
+    "lon": "number"
+  },
+  "carbonSequestration": "number",
+  "temperature": "number",
+  "precipitation": "number"
+}
+```
+
+### Soil Carbon Schema
+```json
+{
+  "id": "string",
+  "holon": "string",
+  "timestamp": "number",
+  "coordinates": {
+    "lat": "number",
+    "lon": "number"
+  },
+  "soilCarbon": "number",
+  "soilType": "string",
+  "soilDepth": "string"
+}
+```
+
+### Biodiversity Schema
+```json
+{
+  "id": "string",
+  "holon": "string",
+  "timestamp": "number",
+  "countryCode": "string",
+  "biodiversityCount": "number",
+  "species": [
+    {
+      "scientificName": "string",
+      "count": "number"
+    }
+  ]
+}
+```
+
+## Running the Examples
+
+1. Make sure you have all required API keys in your `.env` file:
+   - `OPENWEATHER_API_KEY` - For air quality data
+   - `NOAA_API_KEY` - For climate data
+
+2. Run the data storage example:
+   ```
+   node examples/environmentalData.js
+   ```
+
+3. Run the query example:
+   ```
+   node examples/queryEnvironmentalData.js
+   ```
+
+## Federation in Practice
+
+These examples demonstrate several key federation concepts:
+
+1. **Hierarchical Data Organization**: Environmental data is organized by geographical scale, with relationships between scales.
+
+2. **Soul References**: When propagating data from city to region level, soul references are used instead of duplicating data.
+
+3. **Automatic Reference Resolution**: When querying data, references are automatically resolved to their original source.
+
+4. **Cross-Scale Aggregation**: The compute function is used to aggregate metrics from different scales.
+
+## Example Output
+
+Running the query example should produce output similar to:
+
+```
+Querying environmental data for location (41.9, 12.5)
+  - City: 8773237a267ffff
+  - Region: 852e3ffffffffffffff
+  - Country: 832ffffffffffffffff
+
+=== CITY LEVEL: Air Quality Data ===
+Latest Air Quality Index: 2
+Pollutant levels:
+  - co: 201.94
+  - no2: 0.78
+  - o3: 68.64
+  - pm2_5: 0.5
+  - pm10: 0.82
+  - so2: 0.52
+
+=== REGION LEVEL: Soil Carbon Data ===
+Soil Carbon: 78.4 g/kg
+Soil Type: Clay Loam
+Soil Depth: 0-30cm
+
+...