@genobank/biofs 2.5.0 → 2.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +2 -0
- package/LICENSE +37 -0
- package/README.md +2 -1
- package/bin/biofs.js +2 -1
- package/contracts/ClaraJobNFT.sol +2 -0
- package/dist/commands/link-clara.d.ts.map +1 -1
- package/dist/commands/link-clara.js +9 -3
- package/dist/commands/link-clara.js.map +1 -1
- package/dist/commands/tokenize-biosample.d.ts.map +1 -1
- package/dist/commands/tokenize-biosample.js +9 -3
- package/dist/commands/tokenize-biosample.js.map +1 -1
- package/dist/index.js +1 -1
- package/docs/AI_AGENT_MCP_MANUAL.md +2 -0
- package/docs/USER_MANUAL.md +2 -0
- package/docs/technical-details.html +555 -0
- package/lib/bionfs-client.js +2 -0
- package/lib/consent-manager.js +2 -0
- package/lib/consent-prompt.js +2 -0
- package/package.json +3 -3
- package/presentation/biofs-x402-kite-presentation.html +2 -0
- package/scripts/build_annotator_dictionary.py +1 -0
- package/scripts/deploy-clara-nft.js +2 -0
- package/test-complete-flow.sh +1 -1
- package/test.vcf +1 -0
- package/test_23andme.txt +1 -0
- package/ADMIN_REGISTER_LAB.md +0 -252
- package/ANNOTATOR_RECOMMENDATIONS_API.md +0 -490
- package/BIOFS_QUICK_START.md +0 -200
- package/BIOFS_TOKENIZE_COMPLETE.md +0 -541
- package/BIOFS_V1.2.7_PUBLISHED.md +0 -317
- package/BIOFS_V1.2.9_FIXES.md +0 -306
- package/BIOFS_V1.3.0_DIRECT_GRANT.md +0 -381
- package/BIOFS_V1.3.1_GRANTED_FILES.md +0 -328
- package/BIONFS_DEPLOYMENT_GUIDE.md +0 -408
- package/BIONFS_MOUNT_INTEGRATION_PLAN.md +0 -415
- package/BIONFS_PROTOCOL_SPECIFICATION.md +0 -688
- package/BIONFS_SEQUENTIAS_INTEGRATION.md +0 -457
- package/BIONFT_NFS_PROTOCOL_ARCHITECTURE.md +0 -583
- package/BIOOS_CLI_IMPLEMENTATION_COMPLETE.md +0 -422
- package/BIOOS_JOB_MANAGEMENT.md +0 -681
- package/BUGFIX_V2.1.3.md +0 -72
- package/CHANGELOG_V2.1.0.md +0 -386
- package/CHANGELOG_V2.2.0.md +0 -284
- package/ENHANCEMENT_DOCUMENTATION.md +0 -218
- package/GDPR_BIONFS_CONSENT_ARCHITECTURE.md +0 -798
- package/GDPR_BIONFS_IMPLEMENTATION_STATUS.md +0 -462
- package/INSTALL_V1.2.7_ON_MAC.md +0 -306
- package/LABNFT_BIOFS_INTEGRATION.md +0 -396
- package/OPENCRAVAT_ANNOTATORS_REFERENCE.md +0 -340
- package/OPENCRAVAT_ANNOTATOR_INTELLIGENCE_COMPLETE.md +0 -479
- package/QUICK_START_V2.1.0.md +0 -116
- package/RELEASE_SUMMARY_V1.2.7.md +0 -479
- package/RELEASE_SUMMARY_V1.3.0.md +0 -324
- package/SEQUENTIAS_PROTOCOL_ARCHITECTURE.md +0 -880
- package/SEQUENTIA_MIGRATION_GUIDE.md +0 -337
- package/SESSION_SUMMARY_DIRECT_GRANT.md +0 -321
- package/ULTRATHINK_BIONFS_COMPLETE.md +0 -458
- package/WEB3FUSE_LICENSE_TOKEN_ANALYSIS.md +0 -494
- package/WHERE_PERMISSIONS_LIVE.md +0 -165
- package/artifacts/@openzeppelin/contracts/access/Ownable.sol/Ownable.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/access/Ownable.sol/Ownable.json +0 -63
- package/artifacts/@openzeppelin/contracts/interfaces/IERC4906.sol/IERC4906.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/interfaces/IERC4906.sol/IERC4906.json +0 -328
- package/artifacts/@openzeppelin/contracts/token/ERC721/ERC721.sol/ERC721.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/token/ERC721/ERC721.sol/ERC721.json +0 -357
- package/artifacts/@openzeppelin/contracts/token/ERC721/IERC721.sol/IERC721.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/token/ERC721/IERC721.sol/IERC721.json +0 -296
- package/artifacts/@openzeppelin/contracts/token/ERC721/IERC721Receiver.sol/IERC721Receiver.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/token/ERC721/IERC721Receiver.sol/IERC721Receiver.json +0 -45
- package/artifacts/@openzeppelin/contracts/token/ERC721/extensions/ERC721URIStorage.sol/ERC721URIStorage.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/token/ERC721/extensions/ERC721URIStorage.sol/ERC721URIStorage.json +0 -373
- package/artifacts/@openzeppelin/contracts/token/ERC721/extensions/IERC721Metadata.sol/IERC721Metadata.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/token/ERC721/extensions/IERC721Metadata.sol/IERC721Metadata.json +0 -341
- package/artifacts/@openzeppelin/contracts/utils/Address.sol/Address.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/Address.sol/Address.json +0 -10
- package/artifacts/@openzeppelin/contracts/utils/Context.sol/Context.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/Context.sol/Context.json +0 -10
- package/artifacts/@openzeppelin/contracts/utils/Strings.sol/Strings.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/Strings.sol/Strings.json +0 -10
- package/artifacts/@openzeppelin/contracts/utils/introspection/ERC165.sol/ERC165.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/introspection/ERC165.sol/ERC165.json +0 -30
- package/artifacts/@openzeppelin/contracts/utils/introspection/IERC165.sol/IERC165.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/introspection/IERC165.sol/IERC165.json +0 -30
- package/artifacts/@openzeppelin/contracts/utils/math/Math.sol/Math.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/math/Math.sol/Math.json +0 -10
- package/artifacts/@openzeppelin/contracts/utils/math/SignedMath.sol/SignedMath.dbg.json +0 -4
- package/artifacts/@openzeppelin/contracts/utils/math/SignedMath.sol/SignedMath.json +0 -10
- package/artifacts/build-info/1adfbc50ae3e1821260f858a9c689a37.json +0 -1
- package/artifacts/contracts/ClaraJobNFT.sol/ClaraJobNFT.dbg.json +0 -4
- package/artifacts/contracts/ClaraJobNFT.sol/ClaraJobNFT.json +0 -667
- package/cache/solidity-files-cache.json +0 -628
- package/check_clara2.js +0 -70
- package/check_clara3.js +0 -65
- package/config.json +0 -21
- package/hardhat.config.js +0 -25
- package/jest.config.js +0 -27
- package/mint_and_link_all.js +0 -128
- package/opencravat_annotators_dictionary.json +0 -2356
- package/package.json.working.v1.9.2.20251105_023952 +0 -48
- package/test_v2_mint.js +0 -101
|
@@ -1,880 +0,0 @@
|
|
|
1
|
-
# 🌍 Sequentias Protocol Architecture
|
|
2
|
-
## Decentralized Genomics Network with Global Node Orchestration
|
|
3
|
-
|
|
4
|
-
**Vision**: Every GenoBank API/microservice can be replicated as a node in the Sequentias Network, enabling truly decentralized genomics with patient-owned data AND distributed compute.
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
## 🎯 The Real Problem We're Solving
|
|
9
|
-
|
|
10
|
-
### Current Architecture (Centralized):
|
|
11
|
-
```
|
|
12
|
-
User → api.genobank.app (Single Linux Server) → S3
|
|
13
|
-
↑
|
|
14
|
-
All jobs execute HERE
|
|
15
|
-
(CPU/GPU bottleneck)
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
### Sequentias Protocol Architecture (Decentralized):
|
|
19
|
-
```
|
|
20
|
-
User → Sequentias Network → Optimal Node
|
|
21
|
-
↓
|
|
22
|
-
Global Registry:
|
|
23
|
-
- Mexico Node (CPU, low latency for Mexico users)
|
|
24
|
-
- California Node (GPU, for AlphaGenome jobs)
|
|
25
|
-
- Europe Node (GDPR compliance, EU data)
|
|
26
|
-
- Brazil Node (cached replicas for SA users)
|
|
27
|
-
|
|
28
|
-
Each Node = Full GenoBank API + Bioinformatics + Story Protocol
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
---
|
|
32
|
-
|
|
33
|
-
## 🏗️ Three-Layer Architecture: FUSE + BioNFS + Sequentias
|
|
34
|
-
|
|
35
|
-
### Layer 1: FUSE (Client Experience Layer)
|
|
36
|
-
**Purpose**: User sees distributed files as if they're local
|
|
37
|
-
|
|
38
|
-
```bash
|
|
39
|
-
# User experience (same worldwide)
|
|
40
|
-
biofs mount ~/genomics
|
|
41
|
-
ls ~/genomics/
|
|
42
|
-
# Files appear local, but are actually distributed across global nodes!
|
|
43
|
-
|
|
44
|
-
biofs analyze sample.vcf --pipeline alphagenome
|
|
45
|
-
# Job automatically routes to optimal node (GPU required? → California)
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
**Implementation**: `/home/ubuntu/web3fuse/` (existing codebase)
|
|
49
|
-
- Mounts distributed filesystem locally
|
|
50
|
-
- Translates file operations to BioNFS protocol calls
|
|
51
|
-
- Handles session management (24-hour Web3 sessions)
|
|
52
|
-
|
|
53
|
-
### Layer 2: BioNFS Protocol (Network Communication Layer)
|
|
54
|
-
**Purpose**: Standardized protocol for node-to-node communication
|
|
55
|
-
|
|
56
|
-
```
|
|
57
|
-
Node-to-Node Communication:
|
|
58
|
-
1. Node Discovery (find available nodes)
|
|
59
|
-
2. Job Submission (send work to optimal node)
|
|
60
|
-
3. Data Replication (cache files at edge)
|
|
61
|
-
4. Result Streaming (return output to client)
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
**Implementation**: New BioNFS server (FastAPI + gRPC)
|
|
65
|
-
- HTTP/2 for client connections
|
|
66
|
-
- gRPC for node-to-node communication
|
|
67
|
-
- WebSocket for real-time job updates
|
|
68
|
-
|
|
69
|
-
### Layer 3: Sequentias Network (Orchestration Layer)
|
|
70
|
-
**Purpose**: Smart routing, resource allocation, fault tolerance
|
|
71
|
-
|
|
72
|
-
```javascript
|
|
73
|
-
// Sequentias Protocol decides:
|
|
74
|
-
const optimalNode = sequentias.findNode({
|
|
75
|
-
userLocation: "Mexico City",
|
|
76
|
-
jobType: "alphagenome", // Needs GPU
|
|
77
|
-
dataLocation: "California S3",
|
|
78
|
-
constraints: {
|
|
79
|
-
maxLatency: 100, // ms
|
|
80
|
-
maxCost: 10, // USD
|
|
81
|
-
requireGPU: true
|
|
82
|
-
}
|
|
83
|
-
});
|
|
84
|
-
|
|
85
|
-
// Result: Routes to Austin, Texas node
|
|
86
|
-
// Why? Middle ground: Has GPU, close to both user and data
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
---
|
|
90
|
-
|
|
91
|
-
## 🌐 Sequentias Node Architecture
|
|
92
|
-
|
|
93
|
-
### What is a Sequentias Node?
|
|
94
|
-
|
|
95
|
-
Each node is a **full replica** of current GenoBank infrastructure:
|
|
96
|
-
|
|
97
|
-
```
|
|
98
|
-
Sequentias Node (Mexico City):
|
|
99
|
-
├── BioNFS Server (network protocol)
|
|
100
|
-
├── GenoBank APIs
|
|
101
|
-
│ ├── api_bioip (tokenization)
|
|
102
|
-
│ ├── api_vcf_annotator (OpenCRAVAT)
|
|
103
|
-
│ ├── api_alphagenome (variant scoring)
|
|
104
|
-
│ ├── api_somos_dao (ancestry)
|
|
105
|
-
│ ├── api_clara (GPU variant calling)
|
|
106
|
-
│ └── api_trio (family analysis)
|
|
107
|
-
├── Bioinformatics Tools
|
|
108
|
-
│ ├── DeepVariant (GPU)
|
|
109
|
-
│ ├── OpenCRAVAT (annotation)
|
|
110
|
-
│ ├── bcftools, samtools, etc.
|
|
111
|
-
├── AI Models
|
|
112
|
-
│ ├── AlphaGenome (local inference)
|
|
113
|
-
│ ├── Claude API integration
|
|
114
|
-
│ └── Gemini API integration
|
|
115
|
-
├── Vault Storage
|
|
116
|
-
│ ├── Local cache (hot data)
|
|
117
|
-
│ ├── S3 connection (cold data)
|
|
118
|
-
│ └── IPFS pinning (NFT metadata)
|
|
119
|
-
└── Story Protocol Client
|
|
120
|
-
├── License verification
|
|
121
|
-
├── IP registration
|
|
122
|
-
└── PIL enforcement
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
### Node Capabilities Matrix:
|
|
126
|
-
|
|
127
|
-
| Node Type | CPU | GPU | Storage | Use Cases |
|
|
128
|
-
|-----------|-----|-----|---------|-----------|
|
|
129
|
-
| **Basic** | ✅ | ❌ | 1TB | VCF annotation, ancestry analysis |
|
|
130
|
-
| **GPU** | ✅ | ✅ | 5TB | AlphaGenome, DeepVariant, Clara |
|
|
131
|
-
| **Storage** | ✅ | ❌ | 50TB | Data replication, edge caching |
|
|
132
|
-
| **AI** | ✅ | ✅ | 10TB | Claude analysis, AlphaGenome |
|
|
133
|
-
|
|
134
|
-
---
|
|
135
|
-
|
|
136
|
-
## 🚀 The Hybrid Answer: BOTH FUSE + NFS
|
|
137
|
-
|
|
138
|
-
### Why FUSE Alone Won't Work:
|
|
139
|
-
```
|
|
140
|
-
❌ FUSE is client-side only
|
|
141
|
-
- Can't route jobs to different nodes
|
|
142
|
-
- No node-to-node communication
|
|
143
|
-
- Each client needs full implementation
|
|
144
|
-
|
|
145
|
-
❌ No network protocol
|
|
146
|
-
- Nodes can't discover each other
|
|
147
|
-
- Can't distribute workload
|
|
148
|
-
- No fault tolerance (one node down = job fails)
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
### Why NFS-like Protocol Alone Won't Work:
|
|
152
|
-
```
|
|
153
|
-
❌ Requires mounting on every client
|
|
154
|
-
- User must configure NFS mounts
|
|
155
|
-
- Complex setup (not "just works")
|
|
156
|
-
- Doesn't feel like local files
|
|
157
|
-
|
|
158
|
-
❌ Not suitable for CLI tools
|
|
159
|
-
- bcftools, samtools expect local files
|
|
160
|
-
- Performance overhead for small operations
|
|
161
|
-
- Doesn't integrate with existing workflows
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
### ✅ The Hybrid Solution: FUSE + BioNFS
|
|
165
|
-
|
|
166
|
-
```
|
|
167
|
-
┌─────────────────────────────────────────────────────┐
|
|
168
|
-
│ CLIENT SIDE (User's Computer) │
|
|
169
|
-
│ │
|
|
170
|
-
│ ┌──────────────────────────────────────────────┐ │
|
|
171
|
-
│ │ FUSE Layer (web3fuse) │ │
|
|
172
|
-
│ │ - Mounts: /mnt/genomics/ │ │
|
|
173
|
-
│ │ - User sees files as local │ │
|
|
174
|
-
│ │ - Works with bcftools, samtools, IGV │ │
|
|
175
|
-
│ └──────────────┬───────────────────────────────┘ │
|
|
176
|
-
│ │ │
|
|
177
|
-
│ ┌──────────────▼───────────────────────────────┐ │
|
|
178
|
-
│ │ BioNFS Client Library │ │
|
|
179
|
-
│ │ - Protocol implementation │ │
|
|
180
|
-
│ │ - Session management │ │
|
|
181
|
-
│ │ - Node discovery │ │
|
|
182
|
-
│ └──────────────┬───────────────────────────────┘ │
|
|
183
|
-
└─────────────────┼────────────────────────────────────┘
|
|
184
|
-
│
|
|
185
|
-
BioNFS Protocol
|
|
186
|
-
(HTTP/2 + gRPC + WebSocket)
|
|
187
|
-
│
|
|
188
|
-
┌─────────────────▼────────────────────────────────────┐
|
|
189
|
-
│ SEQUENTIAS NETWORK (Global Distributed Nodes) │
|
|
190
|
-
│ │
|
|
191
|
-
│ ┌──────────────┐ ┌──────────────┐ ┌────────────┐ │
|
|
192
|
-
│ │ Mexico Node │ │ California │ │ Europe │ │
|
|
193
|
-
│ │ (Basic) │ │ (GPU) │ │ (GDPR) │ │
|
|
194
|
-
│ │ │ │ │ │ │ │
|
|
195
|
-
│ │ BioNFS Server│ │ BioNFS Server│ │ BioNFS │ │
|
|
196
|
-
│ │ Annotation │ │ AlphaGenome │ │ Server │ │
|
|
197
|
-
│ │ Ancestry │ │ DeepVariant │ │ Storage │ │
|
|
198
|
-
│ └──────────────┘ └──────────────┘ └────────────┘ │
|
|
199
|
-
│ │
|
|
200
|
-
│ ┌─────────────────────────────────────────────────┐ │
|
|
201
|
-
│ │ Sequentias Orchestrator (Smart Routing) │ │
|
|
202
|
-
│ │ - Find optimal node based on: │ │
|
|
203
|
-
│ │ • User location (latency) │ │
|
|
204
|
-
│ │ • Job requirements (CPU/GPU) │ │
|
|
205
|
-
│ │ • Data location (bandwidth) │ │
|
|
206
|
-
│ │ • Cost constraints (budget) │ │
|
|
207
|
-
│ │ • Node availability (load balancing) │ │
|
|
208
|
-
│ └─────────────────────────────────────────────────┘ │
|
|
209
|
-
└────────────────────────────────────────────────────────┘
|
|
210
|
-
```
|
|
211
|
-
|
|
212
|
-
---
|
|
213
|
-
|
|
214
|
-
## 📋 Implementation Roadmap
|
|
215
|
-
|
|
216
|
-
### Week 1-2: FUSE Client Enhancement
|
|
217
|
-
**Update `/home/ubuntu/web3fuse/`**
|
|
218
|
-
|
|
219
|
-
```c
|
|
220
|
-
// web3fuse with Sequentias support
|
|
221
|
-
static int bionfs_fuse_open(const char *path, struct fuse_file_info *fi) {
|
|
222
|
-
// 1. Parse IP Asset ID from path
|
|
223
|
-
char *ip_id = parse_ip_from_path(path);
|
|
224
|
-
|
|
225
|
-
// 2. Query Sequentias registry for optimal node
|
|
226
|
-
sequentias_node_t *node = sequentias_find_optimal_node(
|
|
227
|
-
current_user_location,
|
|
228
|
-
ip_id,
|
|
229
|
-
/* requires_gpu */ false
|
|
230
|
-
);
|
|
231
|
-
|
|
232
|
-
// 3. Connect to node via BioNFS protocol
|
|
233
|
-
bionfs_connection_t *conn = bionfs_connect(node->endpoint);
|
|
234
|
-
|
|
235
|
-
// 4. Authenticate with session
|
|
236
|
-
bionfs_auth(conn, user_session);
|
|
237
|
-
|
|
238
|
-
// 5. Open file on remote node
|
|
239
|
-
bionfs_file_t *file = bionfs_remote_open(conn, ip_id);
|
|
240
|
-
fi->fh = (uint64_t)file;
|
|
241
|
-
|
|
242
|
-
return 0;
|
|
243
|
-
}
|
|
244
|
-
|
|
245
|
-
static int bionfs_fuse_read(const char *path, char *buf, size_t size, off_t offset,
|
|
246
|
-
struct fuse_file_info *fi) {
|
|
247
|
-
bionfs_file_t *file = (bionfs_file_t *)fi->fh;
|
|
248
|
-
|
|
249
|
-
// Stream from remote node (Mexico, California, wherever optimal)
|
|
250
|
-
return bionfs_remote_read(file->connection, file->handle, buf, size, offset);
|
|
251
|
-
}
|
|
252
|
-
```
|
|
253
|
-
|
|
254
|
-
### Week 3-4: BioNFS Protocol Server
|
|
255
|
-
**New: `/home/ubuntu/sequentias_node/bionfs_server/`**
|
|
256
|
-
|
|
257
|
-
```python
|
|
258
|
-
from fastapi import FastAPI, WebSocket
|
|
259
|
-
from grpc import aio
|
|
260
|
-
import sequentias_pb2 # Protocol Buffers
|
|
261
|
-
|
|
262
|
-
app = FastAPI()
|
|
263
|
-
|
|
264
|
-
class BioNFSServer:
|
|
265
|
-
def __init__(self):
|
|
266
|
-
self.node_id = os.environ['SEQUENTIAS_NODE_ID']
|
|
267
|
-
self.location = os.environ['NODE_LOCATION'] # "Mexico City"
|
|
268
|
-
self.has_gpu = torch.cuda.is_available()
|
|
269
|
-
self.registry = SequentiasRegistry()
|
|
270
|
-
|
|
271
|
-
async def register_with_network(self):
|
|
272
|
-
"""Register this node with Sequentias Network"""
|
|
273
|
-
await self.registry.register_node({
|
|
274
|
-
"node_id": self.node_id,
|
|
275
|
-
"location": self.location,
|
|
276
|
-
"endpoint": "https://mexico.sequentias.io",
|
|
277
|
-
"capabilities": {
|
|
278
|
-
"cpu_cores": 64,
|
|
279
|
-
"gpu": "NVIDIA A100" if self.has_gpu else None,
|
|
280
|
-
"storage_tb": 10,
|
|
281
|
-
"services": [
|
|
282
|
-
"vcf_annotation",
|
|
283
|
-
"ancestry_analysis",
|
|
284
|
-
"alphagenome" if self.has_gpu else None
|
|
285
|
-
]
|
|
286
|
-
},
|
|
287
|
-
"operator": os.environ['NODE_OPERATOR_WALLET']
|
|
288
|
-
})
|
|
289
|
-
|
|
290
|
-
@app.post("/bionfs/v1/execute_job")
|
|
291
|
-
async def execute_job(job_spec: JobSpec):
|
|
292
|
-
"""Execute job on this node"""
|
|
293
|
-
# 1. Verify session
|
|
294
|
-
session = verify_session(job_spec.session_token)
|
|
295
|
-
|
|
296
|
-
# 2. Check if file is cached locally
|
|
297
|
-
if not self.cache.has(job_spec.ip_id):
|
|
298
|
-
# Fetch from primary vault
|
|
299
|
-
await self.replicate_file(job_spec.ip_id)
|
|
300
|
-
|
|
301
|
-
# 3. Execute pipeline
|
|
302
|
-
if job_spec.pipeline == "alphagenome":
|
|
303
|
-
if not self.has_gpu:
|
|
304
|
-
return {"error": "This node has no GPU, routing error!"}
|
|
305
|
-
result = await run_alphagenome(job_spec.ip_id)
|
|
306
|
-
elif job_spec.pipeline == "rare_coding":
|
|
307
|
-
result = await run_opencravat(job_spec.ip_id, "rare_coding")
|
|
308
|
-
|
|
309
|
-
# 4. Replicate result to user's preferred vault
|
|
310
|
-
await self.replicate_result(result, session.wallet)
|
|
311
|
-
|
|
312
|
-
# 5. Return presigned URL for download
|
|
313
|
-
return {
|
|
314
|
-
"status": "success",
|
|
315
|
-
"result_ip_id": result.ip_id,
|
|
316
|
-
"download_url": self.generate_download_url(result.ip_id)
|
|
317
|
-
}
|
|
318
|
-
|
|
319
|
-
# gRPC service for node-to-node communication
|
|
320
|
-
class NodeCommunication(sequentias_pb2_grpc.NodeServicer):
|
|
321
|
-
async def ReplicateFile(self, request, context):
|
|
322
|
-
"""Another node requesting file replication"""
|
|
323
|
-
# Transfer file to requesting node
|
|
324
|
-
file_data = await self.vault.get_file(request.ip_id)
|
|
325
|
-
return sequentias_pb2.FileData(
|
|
326
|
-
ip_id=request.ip_id,
|
|
327
|
-
data=file_data,
|
|
328
|
-
checksum=hashlib.sha256(file_data).hexdigest()
|
|
329
|
-
)
|
|
330
|
-
```
|
|
331
|
-
|
|
332
|
-
### Week 5-6: Sequentias Registry (Smart Contract)
|
|
333
|
-
**Deploy on Story Protocol**
|
|
334
|
-
|
|
335
|
-
```solidity
|
|
336
|
-
// SPDX-License-Identifier: MIT
|
|
337
|
-
pragma solidity ^0.8.20;
|
|
338
|
-
|
|
339
|
-
contract SequentiasRegistry {
|
|
340
|
-
struct Node {
|
|
341
|
-
address operator;
|
|
342
|
-
string location; // "Mexico City", "Palo Alto", "Berlin"
|
|
343
|
-
string endpoint; // "https://mexico.sequentias.io"
|
|
344
|
-
bool hasGPU;
|
|
345
|
-
uint256 cpuCores;
|
|
346
|
-
uint256 storageTB;
|
|
347
|
-
uint256 uptime; // Percentage (9500 = 95.00%)
|
|
348
|
-
uint256 jobsCompleted;
|
|
349
|
-
bool active;
|
|
350
|
-
}
|
|
351
|
-
|
|
352
|
-
struct Job {
|
|
353
|
-
bytes32 jobId;
|
|
354
|
-
address requester;
|
|
355
|
-
address executingNode;
|
|
356
|
-
string pipeline; // "alphagenome", "rare_coding"
|
|
357
|
-
bytes32 ipAssetId;
|
|
358
|
-
uint256 startTime;
|
|
359
|
-
uint256 endTime;
|
|
360
|
-
JobStatus status;
|
|
361
|
-
}
|
|
362
|
-
|
|
363
|
-
enum JobStatus { Pending, Running, Completed, Failed }
|
|
364
|
-
|
|
365
|
-
mapping(address => Node) public nodes;
|
|
366
|
-
mapping(bytes32 => Job) public jobs;
|
|
367
|
-
|
|
368
|
-
event NodeRegistered(address indexed operator, string location);
|
|
369
|
-
event JobRouted(bytes32 indexed jobId, address indexed node);
|
|
370
|
-
event JobCompleted(bytes32 indexed jobId, uint256 executionTime);
|
|
371
|
-
|
|
372
|
-
function registerNode(
|
|
373
|
-
string memory location,
|
|
374
|
-
string memory endpoint,
|
|
375
|
-
bool hasGPU,
|
|
376
|
-
uint256 cpuCores,
|
|
377
|
-
uint256 storageTB
|
|
378
|
-
) external {
|
|
379
|
-
nodes[msg.sender] = Node({
|
|
380
|
-
operator: msg.sender,
|
|
381
|
-
location: location,
|
|
382
|
-
endpoint: endpoint,
|
|
383
|
-
hasGPU: hasGPU,
|
|
384
|
-
cpuCores: cpuCores,
|
|
385
|
-
storageTB: storageTB,
|
|
386
|
-
uptime: 10000, // Start at 100%
|
|
387
|
-
jobsCompleted: 0,
|
|
388
|
-
active: true
|
|
389
|
-
});
|
|
390
|
-
|
|
391
|
-
emit NodeRegistered(msg.sender, location);
|
|
392
|
-
}
|
|
393
|
-
|
|
394
|
-
function findOptimalNode(
|
|
395
|
-
string memory userLocation,
|
|
396
|
-
bool requiresGPU,
|
|
397
|
-
string memory dataLocation
|
|
398
|
-
) external view returns (address) {
|
|
399
|
-
// Scoring algorithm:
|
|
400
|
-
// 1. Location proximity (30%)
|
|
401
|
-
// 2. Resource availability (30%)
|
|
402
|
-
// 3. Uptime/reliability (20%)
|
|
403
|
-
// 4. Data proximity (20%)
|
|
404
|
-
|
|
405
|
-
address bestNode;
|
|
406
|
-
uint256 bestScore = 0;
|
|
407
|
-
|
|
408
|
-
for (uint i = 0; i < getAllNodes().length; i++) {
|
|
409
|
-
address nodeAddr = getAllNodes()[i];
|
|
410
|
-
Node memory node = nodes[nodeAddr];
|
|
411
|
-
|
|
412
|
-
if (!node.active) continue;
|
|
413
|
-
if (requiresGPU && !node.hasGPU) continue;
|
|
414
|
-
|
|
415
|
-
uint256 score = calculateNodeScore(
|
|
416
|
-
node,
|
|
417
|
-
userLocation,
|
|
418
|
-
dataLocation
|
|
419
|
-
);
|
|
420
|
-
|
|
421
|
-
if (score > bestScore) {
|
|
422
|
-
bestScore = score;
|
|
423
|
-
bestNode = nodeAddr;
|
|
424
|
-
}
|
|
425
|
-
}
|
|
426
|
-
|
|
427
|
-
return bestNode;
|
|
428
|
-
}
|
|
429
|
-
|
|
430
|
-
function submitJob(
|
|
431
|
-
bytes32 jobId,
|
|
432
|
-
address nodeOperator,
|
|
433
|
-
string memory pipeline,
|
|
434
|
-
bytes32 ipAssetId
|
|
435
|
-
) external {
|
|
436
|
-
jobs[jobId] = Job({
|
|
437
|
-
jobId: jobId,
|
|
438
|
-
requester: msg.sender,
|
|
439
|
-
executingNode: nodeOperator,
|
|
440
|
-
pipeline: pipeline,
|
|
441
|
-
ipAssetId: ipAssetId,
|
|
442
|
-
startTime: block.timestamp,
|
|
443
|
-
endTime: 0,
|
|
444
|
-
status: JobStatus.Running
|
|
445
|
-
});
|
|
446
|
-
|
|
447
|
-
emit JobRouted(jobId, nodeOperator);
|
|
448
|
-
}
|
|
449
|
-
|
|
450
|
-
function completeJob(bytes32 jobId) external {
|
|
451
|
-
Job storage job = jobs[jobId];
|
|
452
|
-
require(job.executingNode == nodes[msg.sender].operator, "Not executor");
|
|
453
|
-
|
|
454
|
-
job.endTime = block.timestamp;
|
|
455
|
-
job.status = JobStatus.Completed;
|
|
456
|
-
|
|
457
|
-
// Update node stats
|
|
458
|
-
nodes[msg.sender].jobsCompleted++;
|
|
459
|
-
|
|
460
|
-
emit JobCompleted(jobId, job.endTime - job.startTime);
|
|
461
|
-
}
|
|
462
|
-
}
|
|
463
|
-
```
|
|
464
|
-
|
|
465
|
-
### Week 7-8: Orchestration Logic
|
|
466
|
-
**New: `/home/ubuntu/sequentias_node/orchestrator/`**
|
|
467
|
-
|
|
468
|
-
```python
|
|
469
|
-
from web3 import Web3
|
|
470
|
-
from geopy.distance import geodesic
|
|
471
|
-
|
|
472
|
-
class SequentiasOrchestrator:
|
|
473
|
-
def __init__(self):
|
|
474
|
-
self.w3 = Web3(Web3.HTTPProvider(os.environ['STORY_RPC_URL']))
|
|
475
|
-
self.registry = self.w3.eth.contract(
|
|
476
|
-
address=SEQUENTIAS_REGISTRY_ADDRESS,
|
|
477
|
-
abi=SEQUENTIAS_REGISTRY_ABI
|
|
478
|
-
)
|
|
479
|
-
|
|
480
|
-
async def route_job(self, job_spec: JobSpec) -> Node:
|
|
481
|
-
"""
|
|
482
|
-
Find optimal node for job execution
|
|
483
|
-
|
|
484
|
-
Considers:
|
|
485
|
-
1. User location (minimize latency)
|
|
486
|
-
2. Data location (minimize bandwidth)
|
|
487
|
-
3. Resource requirements (CPU/GPU)
|
|
488
|
-
4. Cost constraints (budget)
|
|
489
|
-
5. Node reputation (uptime, success rate)
|
|
490
|
-
"""
|
|
491
|
-
|
|
492
|
-
# Get user's location (from IP or explicitly provided)
|
|
493
|
-
user_location = await self.get_user_location(job_spec.user_ip)
|
|
494
|
-
|
|
495
|
-
# Get data location (where is the IP Asset stored?)
|
|
496
|
-
data_location = await self.get_data_location(job_spec.ip_id)
|
|
497
|
-
|
|
498
|
-
# Query registry for available nodes
|
|
499
|
-
all_nodes = await self.get_all_active_nodes()
|
|
500
|
-
|
|
501
|
-
# Score each node
|
|
502
|
-
scored_nodes = []
|
|
503
|
-
for node in all_nodes:
|
|
504
|
-
score = await self.calculate_node_score(
|
|
505
|
-
node=node,
|
|
506
|
-
user_location=user_location,
|
|
507
|
-
data_location=data_location,
|
|
508
|
-
requires_gpu=job_spec.pipeline in ["alphagenome", "deepvariant"],
|
|
509
|
-
budget=job_spec.max_cost
|
|
510
|
-
)
|
|
511
|
-
scored_nodes.append((score, node))
|
|
512
|
-
|
|
513
|
-
# Sort by score (highest first)
|
|
514
|
-
scored_nodes.sort(reverse=True, key=lambda x: x[0])
|
|
515
|
-
|
|
516
|
-
# Return best node
|
|
517
|
-
return scored_nodes[0][1]
|
|
518
|
-
|
|
519
|
-
async def calculate_node_score(
|
|
520
|
-
self,
|
|
521
|
-
node: Node,
|
|
522
|
-
user_location: Location,
|
|
523
|
-
data_location: Location,
|
|
524
|
-
requires_gpu: bool,
|
|
525
|
-
budget: float
|
|
526
|
-
) -> float:
|
|
527
|
-
"""
|
|
528
|
-
Scoring algorithm:
|
|
529
|
-
- Latency score (30%): Distance from user
|
|
530
|
-
- Bandwidth score (20%): Distance from data
|
|
531
|
-
- Resource score (30%): Has required resources?
|
|
532
|
-
- Reliability score (10%): Uptime percentage
|
|
533
|
-
- Cost score (10%): Within budget?
|
|
534
|
-
"""
|
|
535
|
-
|
|
536
|
-
# Latency score (closer to user = better)
|
|
537
|
-
user_distance = geodesic(
|
|
538
|
-
(user_location.lat, user_location.lon),
|
|
539
|
-
(node.location.lat, node.location.lon)
|
|
540
|
-
).kilometers
|
|
541
|
-
latency_score = max(0, 100 - (user_distance / 100)) # 0-100
|
|
542
|
-
|
|
543
|
-
# Bandwidth score (closer to data = better)
|
|
544
|
-
data_distance = geodesic(
|
|
545
|
-
(data_location.lat, data_location.lon),
|
|
546
|
-
(node.location.lat, node.location.lon)
|
|
547
|
-
).kilometers
|
|
548
|
-
bandwidth_score = max(0, 100 - (data_distance / 100))
|
|
549
|
-
|
|
550
|
-
# Resource score
|
|
551
|
-
if requires_gpu and not node.has_gpu:
|
|
552
|
-
return 0 # Disqualify immediately
|
|
553
|
-
resource_score = 100 if node.has_gpu or not requires_gpu else 0
|
|
554
|
-
|
|
555
|
-
# Reliability score
|
|
556
|
-
reliability_score = node.uptime / 100 # Convert to 0-100
|
|
557
|
-
|
|
558
|
-
# Cost score
|
|
559
|
-
estimated_cost = await self.estimate_job_cost(node, requires_gpu)
|
|
560
|
-
cost_score = 100 if estimated_cost <= budget else 0
|
|
561
|
-
|
|
562
|
-
# Weighted average
|
|
563
|
-
total_score = (
|
|
564
|
-
latency_score * 0.3 +
|
|
565
|
-
bandwidth_score * 0.2 +
|
|
566
|
-
resource_score * 0.3 +
|
|
567
|
-
reliability_score * 0.1 +
|
|
568
|
-
cost_score * 0.1
|
|
569
|
-
)
|
|
570
|
-
|
|
571
|
-
return total_score
|
|
572
|
-
|
|
573
|
-
async def execute_job_on_node(self, node: Node, job_spec: JobSpec):
|
|
574
|
-
"""Submit job to selected node via BioNFS protocol"""
|
|
575
|
-
|
|
576
|
-
async with aiohttp.ClientSession() as session:
|
|
577
|
-
# Submit job to node
|
|
578
|
-
response = await session.post(
|
|
579
|
-
f"{node.endpoint}/bionfs/v1/execute_job",
|
|
580
|
-
json={
|
|
581
|
-
"session_token": job_spec.session_token,
|
|
582
|
-
"ip_id": job_spec.ip_id,
|
|
583
|
-
"pipeline": job_spec.pipeline,
|
|
584
|
-
"parameters": job_spec.parameters
|
|
585
|
-
}
|
|
586
|
-
)
|
|
587
|
-
|
|
588
|
-
result = await response.json()
|
|
589
|
-
|
|
590
|
-
# Register job completion on blockchain
|
|
591
|
-
tx = self.registry.functions.completeJob(
|
|
592
|
-
jobId=job_spec.job_id
|
|
593
|
-
).build_transaction({
|
|
594
|
-
'from': node.operator,
|
|
595
|
-
'nonce': self.w3.eth.get_transaction_count(node.operator)
|
|
596
|
-
})
|
|
597
|
-
|
|
598
|
-
# Node operator signs and submits
|
|
599
|
-
# ...
|
|
600
|
-
|
|
601
|
-
return result
|
|
602
|
-
```
|
|
603
|
-
|
|
604
|
-
---
|
|
605
|
-
|
|
606
|
-
## 🌍 Real-World Usage Scenarios
|
|
607
|
-
|
|
608
|
-
### Scenario 1: Researcher in Mexico Analyzes Local VCF
|
|
609
|
-
```bash
|
|
610
|
-
# User in Mexico City
|
|
611
|
-
biofs login
|
|
612
|
-
biofs mount ~/genomics
|
|
613
|
-
|
|
614
|
-
# File is in California S3
|
|
615
|
-
ls ~/genomics/granted/
|
|
616
|
-
# 0xCCe1.../father.vcf (152 MB, California vault)
|
|
617
|
-
|
|
618
|
-
# Analyze with rare_coding pipeline
|
|
619
|
-
biofs analyze ~/genomics/granted/0xCCe1.../father.vcf --pipeline rare_coding
|
|
620
|
-
|
|
621
|
-
# Behind the scenes:
|
|
622
|
-
# 1. Sequentias finds optimal node:
|
|
623
|
-
# - Mexico City node (50ms latency, no GPU needed)
|
|
624
|
-
# 2. Mexico node caches file from California (one-time transfer)
|
|
625
|
-
# 3. Job executes on Mexico node
|
|
626
|
-
# 4. Results streamed back to user
|
|
627
|
-
# Total time: 30 seconds (vs 5 minutes if run in California)
|
|
628
|
-
```
|
|
629
|
-
|
|
630
|
-
### Scenario 2: Researcher in Palo Alto Needs GPU
|
|
631
|
-
```bash
|
|
632
|
-
# User in Palo Alto, California
|
|
633
|
-
biofs login
|
|
634
|
-
biofs mount ~/genomics
|
|
635
|
-
|
|
636
|
-
# Same file (in California S3)
|
|
637
|
-
# Analyze with AlphaGenome (needs GPU)
|
|
638
|
-
biofs analyze ~/genomics/granted/0xCCe1.../father.vcf --pipeline alphagenome
|
|
639
|
-
|
|
640
|
-
# Behind the scenes:
|
|
641
|
-
# 1. Sequentias finds optimal node:
|
|
642
|
-
# - San Francisco GPU node (30 miles away, has A100)
|
|
643
|
-
# 2. SF node already has file cached (same region)
|
|
644
|
-
# 3. Job executes on GPU
|
|
645
|
-
# 4. Results in <10 minutes (GPU accelerated)
|
|
646
|
-
```
|
|
647
|
-
|
|
648
|
-
### Scenario 3: Patient in Brazil Downloads Own Data
|
|
649
|
-
```bash
|
|
650
|
-
# Patient in São Paulo (owns data, no license needed)
|
|
651
|
-
biofs login # Authenticate as owner
|
|
652
|
-
biofs mount ~/my-genomics
|
|
653
|
-
|
|
654
|
-
# Their files appear
|
|
655
|
-
ls ~/my-genomics/
|
|
656
|
-
# my_exome.vcf (owned file)
|
|
657
|
-
|
|
658
|
-
# Download to local
|
|
659
|
-
biofs download ~/my-genomics/my_exome.vcf ./local_copy.vcf
|
|
660
|
-
|
|
661
|
-
# Behind the scenes:
|
|
662
|
-
# 1. Sequentias checks: Brazil node has cached copy?
|
|
663
|
-
# - No → Replicate from California vault to Brazil node
|
|
664
|
-
# - Yes → Stream from Brazil node
|
|
665
|
-
# 2. Future access: Instant (cached in Brazil)
|
|
666
|
-
# 3. Latency: 20ms (vs 200ms from California)
|
|
667
|
-
```
|
|
668
|
-
|
|
669
|
-
### Scenario 4: GPU Jobs with Data in Different Regions
|
|
670
|
-
```bash
|
|
671
|
-
# User in Tokyo needs GPU for DeepVariant
|
|
672
|
-
# Data is in EU (GDPR-compliant storage)
|
|
673
|
-
|
|
674
|
-
biofs analyze eu_patient_data.fastq --pipeline deepvariant
|
|
675
|
-
|
|
676
|
-
# Behind the scenes:
|
|
677
|
-
# 1. Sequentias scoring:
|
|
678
|
-
# - Tokyo GPU node: High latency to EU data (disqualified)
|
|
679
|
-
# - Singapore GPU node: Medium latency to EU + Tokyo (optimal)
|
|
680
|
-
# - EU GPU node: High latency to Tokyo user (suboptimal)
|
|
681
|
-
# 2. Routes to Singapore node (middle ground)
|
|
682
|
-
# 3. Singapore caches EU data
|
|
683
|
-
# 4. Executes with GPU
|
|
684
|
-
# 5. Results stream to Tokyo (low latency)
|
|
685
|
-
```
|
|
686
|
-
|
|
687
|
-
---
|
|
688
|
-
|
|
689
|
-
## 🔐 Security & Licensing in Distributed Network
|
|
690
|
-
|
|
691
|
-
### License Token Verification Across Nodes
|
|
692
|
-
|
|
693
|
-
```python
|
|
694
|
-
# Every node verifies licenses independently
|
|
695
|
-
def verify_license_on_node(ip_id: str, wallet: str) -> bool:
|
|
696
|
-
"""
|
|
697
|
-
Each node queries Story Protocol directly
|
|
698
|
-
No central authority needed!
|
|
699
|
-
"""
|
|
700
|
-
|
|
701
|
-
# Query local MongoDB cache first (fast)
|
|
702
|
-
local_license = license_cache.find_one({
|
|
703
|
-
"ip_id": ip_id,
|
|
704
|
-
"receiver": wallet,
|
|
705
|
-
"status": "active"
|
|
706
|
-
})
|
|
707
|
-
|
|
708
|
-
if local_license:
|
|
709
|
-
# Verify cache is fresh (< 1 hour old)
|
|
710
|
-
if local_license['cached_at'] > time.time() - 3600:
|
|
711
|
-
return True
|
|
712
|
-
|
|
713
|
-
# If no cache or stale, verify on blockchain
|
|
714
|
-
license_token_contract = story_protocol.get_contract("LicenseToken")
|
|
715
|
-
owner = license_token_contract.functions.ownerOf(local_license['license_token_id']).call()
|
|
716
|
-
|
|
717
|
-
if owner.lower() == wallet.lower():
|
|
718
|
-
# Update cache
|
|
719
|
-
license_cache.update_one(
|
|
720
|
-
{"_id": local_license['_id']},
|
|
721
|
-
{"$set": {"cached_at": time.time()}}
|
|
722
|
-
)
|
|
723
|
-
return True
|
|
724
|
-
|
|
725
|
-
return False
|
|
726
|
-
```
|
|
727
|
-
|
|
728
|
-
### Data Replication with License Enforcement
|
|
729
|
-
|
|
730
|
-
```python
|
|
731
|
-
# Node can only replicate files it's authorized to serve
|
|
732
|
-
async def replicate_file(source_node: str, ip_id: str, requester_wallet: str):
|
|
733
|
-
"""
|
|
734
|
-
Replicate file from source node to this node
|
|
735
|
-
License verification at both ends!
|
|
736
|
-
"""
|
|
737
|
-
|
|
738
|
-
# 1. This node verifies requester has license
|
|
739
|
-
if not verify_license_on_node(ip_id, requester_wallet):
|
|
740
|
-
raise PermissionError("Requester lacks license for this IP Asset")
|
|
741
|
-
|
|
742
|
-
# 2. Request file from source node
|
|
743
|
-
async with grpc.aio.insecure_channel(source_node) as channel:
|
|
744
|
-
stub = sequentias_pb2_grpc.NodeStub(channel)
|
|
745
|
-
|
|
746
|
-
# Source node ALSO verifies (defense in depth)
|
|
747
|
-
response = await stub.ReplicateFile(
|
|
748
|
-
sequentias_pb2.ReplicationRequest(
|
|
749
|
-
ip_id=ip_id,
|
|
750
|
-
requester_node=THIS_NODE_ID,
|
|
751
|
-
requester_wallet=requester_wallet
|
|
752
|
-
)
|
|
753
|
-
)
|
|
754
|
-
|
|
755
|
-
# 3. Verify checksum
|
|
756
|
-
if hashlib.sha256(response.data).hexdigest() != response.checksum:
|
|
757
|
-
raise IntegrityError("File corrupted during transfer")
|
|
758
|
-
|
|
759
|
-
# 4. Store locally with license metadata
|
|
760
|
-
await local_vault.store(
|
|
761
|
-
ip_id=ip_id,
|
|
762
|
-
data=response.data,
|
|
763
|
-
license_verified_for=requester_wallet,
|
|
764
|
-
replicated_from=source_node,
|
|
765
|
-
replicated_at=time.time()
|
|
766
|
-
)
|
|
767
|
-
```
|
|
768
|
-
|
|
769
|
-
---
|
|
770
|
-
|
|
771
|
-
## 📊 Performance Comparison
|
|
772
|
-
|
|
773
|
-
### Current Centralized Architecture:
|
|
774
|
-
```
|
|
775
|
-
Researcher in Mexico → California Server (200ms latency)
|
|
776
|
-
↓
|
|
777
|
-
Download 150GB WGS file (30 minutes on 1 Gbps)
|
|
778
|
-
↓
|
|
779
|
-
Analyze locally (2 hours)
|
|
780
|
-
↓
|
|
781
|
-
Total: 2.5 hours
|
|
782
|
-
```
|
|
783
|
-
|
|
784
|
-
### Sequentias Distributed Architecture:
|
|
785
|
-
```
|
|
786
|
-
Researcher in Mexico → Sequentias Network
|
|
787
|
-
↓
|
|
788
|
-
Routes to Mexico City node (20ms latency)
|
|
789
|
-
↓
|
|
790
|
-
Mexico node caches file from California (one-time, 30 min)
|
|
791
|
-
↓
|
|
792
|
-
Analyze on Mexico node (2 hours, but starts immediately)
|
|
793
|
-
↓
|
|
794
|
-
Results stream back (instant)
|
|
795
|
-
↓
|
|
796
|
-
Total first run: 2.5 hours (same as centralized)
|
|
797
|
-
Total subsequent runs: 2 hours (30min saved, file already cached!)
|
|
798
|
-
|
|
799
|
-
Next researcher in Mexico using same file: 2 hours (no download!)
|
|
800
|
-
```
|
|
801
|
-
|
|
802
|
-
### GPU Jobs:
|
|
803
|
-
```
|
|
804
|
-
AlphaGenome job from Tokyo on EU data:
|
|
805
|
-
|
|
806
|
-
Centralized:
|
|
807
|
-
Tokyo → California server → Fetch EU data → GPU analyze → Return
|
|
808
|
-
Total: 3 hours (200ms latency + data transfer + compute)
|
|
809
|
-
|
|
810
|
-
Sequentias:
|
|
811
|
-
Tokyo → Singapore node → Fetch EU data → GPU analyze → Return
|
|
812
|
-
Total: 2 hours (50ms latency + optimized data transfer + same compute)
|
|
813
|
-
Savings: 33% faster!
|
|
814
|
-
```
|
|
815
|
-
|
|
816
|
-
---
|
|
817
|
-
|
|
818
|
-
## 🎯 THE ANSWER: Build BOTH FUSE + BioNFS
|
|
819
|
-
|
|
820
|
-
### FUSE (Client Layer):
|
|
821
|
-
✅ **Purpose**: User experience - mount files locally
|
|
822
|
-
✅ **Location**: `/home/ubuntu/web3fuse/` (enhance existing)
|
|
823
|
-
✅ **Implementation**: C library with Sequentias integration
|
|
824
|
-
✅ **User sees**: `~/genomics/0xCCe1.../file.vcf` (appears local)
|
|
825
|
-
✅ **Actually**: File streamed from optimal Sequentias node
|
|
826
|
-
|
|
827
|
-
### BioNFS Protocol (Network Layer):
|
|
828
|
-
✅ **Purpose**: Node-to-node communication
|
|
829
|
-
✅ **Location**: `/home/ubuntu/sequentias_node/bionfs_server/` (new)
|
|
830
|
-
✅ **Implementation**: FastAPI + gRPC + WebSocket
|
|
831
|
-
✅ **Handles**: Job routing, data replication, streaming
|
|
832
|
-
|
|
833
|
-
### Sequentias Registry (Orchestration Layer):
|
|
834
|
-
✅ **Purpose**: Smart routing & resource allocation
|
|
835
|
-
✅ **Location**: Story Protocol smart contract (on-chain)
|
|
836
|
-
✅ **Implementation**: Solidity contract + Python orchestrator
|
|
837
|
-
✅ **Decides**: Which node executes which job
|
|
838
|
-
|
|
839
|
-
---
|
|
840
|
-
|
|
841
|
-
## 🚀 Implementation Priority
|
|
842
|
-
|
|
843
|
-
### Month 1: Foundation
|
|
844
|
-
- ✅ BioNFS protocol specification (done)
|
|
845
|
-
- ✅ Basic FastAPI server
|
|
846
|
-
- ✅ FUSE client with network support
|
|
847
|
-
- ✅ Simple node registration
|
|
848
|
-
|
|
849
|
-
### Month 2: Distribution
|
|
850
|
-
- ✅ Sequentias smart contract
|
|
851
|
-
- ✅ Orchestration logic (node scoring)
|
|
852
|
-
- ✅ Data replication protocol
|
|
853
|
-
- ✅ 3 test nodes (Mexico, California, EU)
|
|
854
|
-
|
|
855
|
-
### Month 3: Production
|
|
856
|
-
- ✅ Full node capabilities (all APIs)
|
|
857
|
-
- ✅ GPU node setup (Clara, AlphaGenome)
|
|
858
|
-
- ✅ Fault tolerance (retry logic)
|
|
859
|
-
- ✅ 10+ production nodes globally
|
|
860
|
-
|
|
861
|
-
### Month 4: Optimization
|
|
862
|
-
- ✅ Edge caching optimization
|
|
863
|
-
- ✅ Cost minimization
|
|
864
|
-
- ✅ Performance benchmarks
|
|
865
|
-
- ✅ Open node registration (anyone can run a node!)
|
|
866
|
-
|
|
867
|
-
---
|
|
868
|
-
|
|
869
|
-
## 💡 Why This is Revolutionary
|
|
870
|
-
|
|
871
|
-
1. **True Decentralization**: Not just data, but compute too
|
|
872
|
-
2. **Patient Sovereignty**: Data follows patient, not server location
|
|
873
|
-
3. **Cost Optimization**: Jobs route to cheapest available node
|
|
874
|
-
4. **Fault Tolerance**: Node down? Automatically retry elsewhere
|
|
875
|
-
5. **Global Scale**: Add nodes anywhere in the world
|
|
876
|
-
6. **Open Network**: Anyone can run a Sequentias node
|
|
877
|
-
7. **Standards-Based**: Built on IETF NFS v4 + Story Protocol
|
|
878
|
-
|
|
879
|
-
This is not just GenoBank CLI. **This is the decentralized genomics network of the future.** 🌍🧬
|
|
880
|
-
|