@sesamespace/hivemind 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PLANNING.md +383 -0
- package/TASKS.md +60 -0
- package/install.sh +187 -0
- package/npm-package.json +28 -0
- package/package.json +13 -20
- package/packages/cli/package.json +23 -0
- package/{dist/chunk-DVR2KBL7.js → packages/cli/src/commands/fleet.ts} +50 -30
- package/packages/cli/src/commands/init.ts +230 -0
- package/{dist/chunk-MBS5A6BZ.js → packages/cli/src/commands/service.ts} +51 -42
- package/{dist/chunk-RNK5Q5GR.js → packages/cli/src/commands/start.ts} +12 -14
- package/{dist/main.js → packages/cli/src/main.ts} +12 -18
- package/packages/cli/tsconfig.json +8 -0
- package/packages/memory/Cargo.lock +6480 -0
- package/packages/memory/Cargo.toml +21 -0
- package/packages/memory/src/context.rs +179 -0
- package/packages/memory/src/embeddings.rs +51 -0
- package/packages/memory/src/main.rs +626 -0
- package/packages/memory/src/promotion.rs +637 -0
- package/packages/memory/src/scoring.rs +131 -0
- package/packages/memory/src/store.rs +460 -0
- package/packages/memory/src/tasks.rs +321 -0
- package/packages/runtime/package.json +24 -0
- package/packages/runtime/src/__tests__/fleet-integration.test.ts +235 -0
- package/packages/runtime/src/__tests__/fleet.test.ts +207 -0
- package/packages/runtime/src/__tests__/integration.test.ts +434 -0
- package/packages/runtime/src/agent.ts +255 -0
- package/packages/runtime/src/config.ts +130 -0
- package/packages/runtime/src/context.ts +192 -0
- package/packages/runtime/src/fleet/fleet-manager.ts +399 -0
- package/packages/runtime/src/fleet/memory-sync.ts +362 -0
- package/packages/runtime/src/fleet/primary-client.ts +285 -0
- package/packages/runtime/src/fleet/worker-protocol.ts +158 -0
- package/packages/runtime/src/fleet/worker-server.ts +246 -0
- package/packages/runtime/src/index.ts +57 -0
- package/packages/runtime/src/llm-client.ts +65 -0
- package/packages/runtime/src/memory-client.ts +309 -0
- package/packages/runtime/src/pipeline.ts +151 -0
- package/packages/runtime/src/prompt.ts +173 -0
- package/packages/runtime/src/sesame.ts +174 -0
- package/{dist/start.js → packages/runtime/src/start.ts} +7 -9
- package/packages/runtime/src/task-engine.ts +113 -0
- package/packages/runtime/src/worker.ts +339 -0
- package/packages/runtime/tsconfig.json +8 -0
- package/pnpm-workspace.yaml +2 -0
- package/run-aidan.sh +23 -0
- package/scripts/bootstrap.sh +196 -0
- package/scripts/build-npm.sh +94 -0
- package/scripts/com.hivemind.agent.plist +44 -0
- package/scripts/com.hivemind.memory.plist +31 -0
- package/tsconfig.json +22 -0
- package/tsup.config.ts +28 -0
- package/dist/chunk-2I2O6X5D.js +0 -1408
- package/dist/chunk-2I2O6X5D.js.map +0 -1
- package/dist/chunk-DVR2KBL7.js.map +0 -1
- package/dist/chunk-MBS5A6BZ.js.map +0 -1
- package/dist/chunk-NVJ424TB.js +0 -731
- package/dist/chunk-NVJ424TB.js.map +0 -1
- package/dist/chunk-RNK5Q5GR.js.map +0 -1
- package/dist/chunk-XNOWVLXD.js +0 -160
- package/dist/chunk-XNOWVLXD.js.map +0 -1
- package/dist/commands/fleet.js +0 -9
- package/dist/commands/fleet.js.map +0 -1
- package/dist/commands/init.js +0 -7
- package/dist/commands/init.js.map +0 -1
- package/dist/commands/service.js +0 -7
- package/dist/commands/service.js.map +0 -1
- package/dist/commands/start.js +0 -9
- package/dist/commands/start.js.map +0 -1
- package/dist/index.js +0 -41
- package/dist/index.js.map +0 -1
- package/dist/main.js.map +0 -1
- package/dist/start.js.map +0 -1
package/PLANNING.md
ADDED
|
@@ -0,0 +1,383 @@
|
|
|
1
|
+
# Hivemind — Cognitive Architecture for AI Agents
|
|
2
|
+
|
|
3
|
+
## Vision
|
|
4
|
+
|
|
5
|
+
Hivemind is an opinionated, high-performance agent runtime with a multi-layered memory architecture inspired by CPU cache hierarchies — not the flat-file, dump-everything-in-context approach of current agent frameworks. It enables agents that genuinely remember, learn, and work on multiple projects in parallel without context contamination.
|
|
6
|
+
|
|
7
|
+
**Core thesis:** The bottleneck in AI agents isn't the model — it's memory management. Current systems either burn their entire context window on static files or lose everything between sessions. Hivemind solves this with continuous, incremental memory management across isolated project contexts.
|
|
8
|
+
|
|
9
|
+
**Success criteria:** A Hivemind agent joins our Sesame channel. We have a conversation on Monday, come back Wednesday, and it recalls the context naturally — no hand-curated MEMORY.md, no post-it notes. It works on LinkZero and Untube simultaneously without cross-pollination. It runs across multiple Mac minis with true parallelism.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Architecture
|
|
14
|
+
|
|
15
|
+
### Memory Layers
|
|
16
|
+
|
|
17
|
+
| Layer | Name | What | Storage | Management |
|
|
18
|
+
|-------|------|------|---------|------------|
|
|
19
|
+
| L1 | Working Memory | Current conversation + active task state | In-memory (runtime) | Deterministic eviction by relevance score |
|
|
20
|
+
| L2 | Episodic Memory | Recent interactions, decisions, outcomes | LanceDB (vector) | Write-through on every interaction, continuous decay |
|
|
21
|
+
| L3 | Semantic Memory | Distilled knowledge, patterns, project understanding | LanceDB (vector + structured) | Threshold-based promotion from L2, pull-based LLM synthesis |
|
|
22
|
+
| L4 | External Memory | Git, filesystem, APIs, databases | Native tools | Accessed on demand; index of *what exists where* lives in L2/L3 |
|
|
23
|
+
|
|
24
|
+
### Key Principle: No Batch Consolidation
|
|
25
|
+
|
|
26
|
+
Memory management is **continuous and incremental**, like garbage collection in a runtime:
|
|
27
|
+
|
|
28
|
+
- **Write-through:** Every interaction embedded + indexed into L2 immediately (local embedding model, no LLM)
|
|
29
|
+
- **Decay/scoring:** Lightweight process continuously updates relevance scores (recency × access frequency × connection density) — pure math
|
|
30
|
+
- **Promotion:** L2 episodes auto-promote to L3 when they cross access/connection thresholds — no LLM needed
|
|
31
|
+
- **Eviction:** When L1 context fills, lowest-relevance items drop first — deterministic, predictable
|
|
32
|
+
- **LLM synthesis on demand only:** Agent requests consolidated summaries when needed (pull, not push)
|
|
33
|
+
|
|
34
|
+
No step functions. No discontinuities. No compaction surprises.
|
|
35
|
+
|
|
36
|
+
### Context Isolation (Multi-Project)
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
┌─ Hivemind Agent ──────────────────────────────┐
|
|
40
|
+
│ │
|
|
41
|
+
│ ┌─────────────────────────────────────────┐ │
|
|
42
|
+
│ │ Context Manager │ │
|
|
43
|
+
│ │ - Routes messages to correct context │ │
|
|
44
|
+
│ │ - Spawns/suspends/resumes contexts │ │
|
|
45
|
+
│ │ - Manages cross-context queries │ │
|
|
46
|
+
│ └──┬──────────┬──────────┬───────────┬────┘ │
|
|
47
|
+
│ │ │ │ │ │
|
|
48
|
+
│ ┌──▼───┐ ┌──▼───┐ ┌──▼───┐ ┌────▼────┐ │
|
|
49
|
+
│ │ CTX: │ │ CTX: │ │ CTX: │ │ CTX: │ │
|
|
50
|
+
│ │Proj A│ │Proj B│ │Proj C│ │ Global │ │
|
|
51
|
+
│ │ │ │ │ │ │ │ │ │
|
|
52
|
+
│ │ L1 │ │ L1 │ │ L1 │ │Identity │ │
|
|
53
|
+
│ │ L2 │ │ L2 │ │ L2 │ │Prefs │ │
|
|
54
|
+
│ │ L3 │ │ L3 │ │ L3 │ │Skills │ │
|
|
55
|
+
│ │Tasks │ │Tasks │ │Tasks │ │Relations│ │
|
|
56
|
+
│ └──────┘ └──────┘ └──────┘ └─────────┘ │
|
|
57
|
+
│ │
|
|
58
|
+
│ ┌─────────────────────────────────────────┐ │
|
|
59
|
+
│ │ Shared Infrastructure │ │
|
|
60
|
+
│ │ Memory Daemon (Rust) · LanceDB │ │
|
|
61
|
+
│ │ Ollama (embeddings) · Sesame · Router │ │
|
|
62
|
+
│ └─────────────────────────────────────────┘ │
|
|
63
|
+
└───────────────────────────────────────────────┘
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Each context has:
|
|
67
|
+
- Own L1/L2/L3 memory partitions (LanceDB namespaces)
|
|
68
|
+
- Own relevance scoring and decay curves
|
|
69
|
+
- Own task queue and working state
|
|
70
|
+
- Own model preferences (via ClawRouter)
|
|
71
|
+
|
|
72
|
+
Global context holds:
|
|
73
|
+
- Agent identity and personality
|
|
74
|
+
- Communication preferences
|
|
75
|
+
- Cross-cutting skills and tool knowledge
|
|
76
|
+
- Relationship maps (who is Ryan, what does he care about)
|
|
77
|
+
|
|
78
|
+
**Cross-context communication:**
|
|
79
|
+
- Default: **full isolation** — no cross-pollination
|
|
80
|
+
- Explicit sharing: "Share X with context Y"
|
|
81
|
+
- Cross-context search: query all contexts, results labeled by source
|
|
82
|
+
- Auto-promotion to Global: knowledge referenced across 3+ contexts
|
|
83
|
+
|
|
84
|
+
### Multi-Machine Distribution
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
Mac Mini 1 (Primary):
|
|
88
|
+
└─ Context Manager (coordinator)
|
|
89
|
+
└─ Global Context
|
|
90
|
+
└─ Active conversation handling
|
|
91
|
+
└─ Memory Daemon + LanceDB (primary)
|
|
92
|
+
|
|
93
|
+
Mac Mini 2:
|
|
94
|
+
└─ Hivemind Worker
|
|
95
|
+
└─ CTX: Project A (autonomous background work)
|
|
96
|
+
└─ Local Memory Daemon + LanceDB partition
|
|
97
|
+
└─ Local Ollama (embeddings)
|
|
98
|
+
|
|
99
|
+
Mac Mini 3:
|
|
100
|
+
└─ Hivemind Worker
|
|
101
|
+
└─ CTX: Project B (autonomous background work)
|
|
102
|
+
└─ Local Memory Daemon + LanceDB partition
|
|
103
|
+
└─ Local Ollama (embeddings)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Contexts can be **active** (attached to conversation) or **background** (autonomous task execution). Multiple contexts run truly in parallel across machines.
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Tech Stack
|
|
111
|
+
|
|
112
|
+
| Component | Technology | Rationale |
|
|
113
|
+
|-----------|-----------|-----------|
|
|
114
|
+
| Agent Runtime | TypeScript | Fast iteration, Sesame SDK is TS, ClawRouter client TS, rich AI ecosystem |
|
|
115
|
+
| Memory Daemon | Rust | Performance-critical hot path, reliability, native LanceDB bindings |
|
|
116
|
+
| Vector Storage | LanceDB (embedded) | No server process, Rust-native, fast, partitioning support |
|
|
117
|
+
| Embeddings | Ollama (local) | Apple Silicon optimized, zero-latency, `nomic-embed-text` or `mxbai-embed-large` |
|
|
118
|
+
| Communications | Sesame SDK | Native agent participant, not bolted on |
|
|
119
|
+
| Model Provider | ClawRouter | Drop-in OpenAI-compatible API, model routing/optimization |
|
|
120
|
+
| Task/Queue | In-process (TS) | No external dependencies for MVP; upgrade to Redis/NATS if needed |
|
|
121
|
+
| Config | TOML or YAML | Human-readable, easy to hand-edit |
|
|
122
|
+
|
|
123
|
+
### Dependencies (External Services)
|
|
124
|
+
|
|
125
|
+
- **Sesame** (sesame.space) — agent registration, messaging, channels
|
|
126
|
+
- **ClawRouter** (clawrouter.app) — model API provider
|
|
127
|
+
- **Ollama** — local embedding model server (runs on each Mac mini)
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Project Structure
|
|
132
|
+
|
|
133
|
+
```
|
|
134
|
+
hivemind/
|
|
135
|
+
├── packages/
|
|
136
|
+
│ ├── runtime/ # TypeScript agent runtime
|
|
137
|
+
│ │ ├── src/
|
|
138
|
+
│ │ │ ├── agent.ts # Core agent loop
|
|
139
|
+
│ │ │ ├── context.ts # Context manager
|
|
140
|
+
│ │ │ ├── memory-client.ts # Client for memory daemon
|
|
141
|
+
│ │ │ ├── task-engine.ts # Task planning + execution
|
|
142
|
+
│ │ │ ├── sesame.ts # Sesame integration
|
|
143
|
+
│ │ │ ├── router.ts # ClawRouter client
|
|
144
|
+
│ │ │ └── tools/ # Built-in tool implementations
|
|
145
|
+
│ │ ├── package.json
|
|
146
|
+
│ │ └── tsconfig.json
|
|
147
|
+
│ │
|
|
148
|
+
│ ├── memory/ # Rust memory daemon
|
|
149
|
+
│ │ ├── src/
|
|
150
|
+
│ │ │ ├── main.rs # Daemon entry, HTTP/gRPC server
|
|
151
|
+
│ │ │ ├── store.rs # LanceDB operations
|
|
152
|
+
│ │ │ ├── embeddings.rs # Ollama embedding client
|
|
153
|
+
│ │ │ ├── scoring.rs # Relevance decay + scoring
|
|
154
|
+
│ │ │ ├── context.rs # Context/namespace management
|
|
155
|
+
│ │ │ └── promotion.rs # L2→L3 threshold logic
|
|
156
|
+
│ │ ├── Cargo.toml
|
|
157
|
+
│ │ └── proto/ # gRPC definitions (if used)
|
|
158
|
+
│ │
|
|
159
|
+
│ └── cli/ # CLI for management
|
|
160
|
+
│ ├── src/
|
|
161
|
+
│ │ ├── main.ts
|
|
162
|
+
│ │ ├── commands/
|
|
163
|
+
│ │ │ ├── start.ts # Start agent
|
|
164
|
+
│ │ │ ├── status.ts # Check status
|
|
165
|
+
│ │ │ ├── context.ts # Manage contexts
|
|
166
|
+
│ │ │ └── memory.ts # Query/inspect memory
|
|
167
|
+
│ └── package.json
|
|
168
|
+
│
|
|
169
|
+
├── config/
|
|
170
|
+
│ ├── default.toml # Default configuration
|
|
171
|
+
│ └── example.toml # Example with comments
|
|
172
|
+
│
|
|
173
|
+
├── PLANNING.md # This file
|
|
174
|
+
├── TASKS.md # Task tracker (auto-maintained)
|
|
175
|
+
├── ARCHITECTURE.md # Deep-dive technical docs
|
|
176
|
+
└── README.md # Project overview + quickstart
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## Phases & Tasks
|
|
182
|
+
|
|
183
|
+
### Phase 1 — Single Context, Prove the Memory Works
|
|
184
|
+
|
|
185
|
+
**Goal:** One Hivemind agent on Sesame with working episodic memory. Conversation Monday → recall Wednesday.
|
|
186
|
+
|
|
187
|
+
**Decision boundaries:** Bailey can make all implementation decisions. Ping Ryan only for: external service setup (Sesame agent registration, ClawRouter config), UX decisions about how the agent presents itself, or architectural changes that deviate from this plan.
|
|
188
|
+
|
|
189
|
+
#### Tasks
|
|
190
|
+
|
|
191
|
+
- [ ] **1.1 — Project scaffolding**
|
|
192
|
+
- Initialize monorepo (pnpm workspaces)
|
|
193
|
+
- Set up TypeScript config for runtime + cli packages
|
|
194
|
+
- Set up Rust project for memory daemon
|
|
195
|
+
- CI basics (lint, type-check, test)
|
|
196
|
+
- *Est: 1 session*
|
|
197
|
+
|
|
198
|
+
- [ ] **1.2 — Memory daemon MVP**
|
|
199
|
+
- Rust HTTP server (axum or actix-web)
|
|
200
|
+
- LanceDB integration: create table, insert, query
|
|
201
|
+
- Ollama embedding client (HTTP to local Ollama)
|
|
202
|
+
- Core API endpoints:
|
|
203
|
+
- `POST /episodes` — store an episode (auto-embeds)
|
|
204
|
+
- `GET /search` — semantic search over episodes
|
|
205
|
+
- `GET /context/{name}` — get all episodes for a context
|
|
206
|
+
- `POST /contexts` — create a new context namespace
|
|
207
|
+
- Basic relevance scoring (recency-weighted)
|
|
208
|
+
- *Est: 2-3 sessions*
|
|
209
|
+
|
|
210
|
+
- [ ] **1.3 — Agent runtime MVP**
|
|
211
|
+
- Core agent loop: receive message → query memory → build prompt → call model → respond → store episode
|
|
212
|
+
- Memory client (talks to daemon API)
|
|
213
|
+
- ClawRouter integration (OpenAI-compatible client, pointed at ClawRouter)
|
|
214
|
+
- Prompt construction: Global context + relevant L2 episodes + current message
|
|
215
|
+
- Basic personality/identity from config (not flat files)
|
|
216
|
+
- *Est: 2 sessions*
|
|
217
|
+
|
|
218
|
+
- [ ] **1.4 — Sesame integration**
|
|
219
|
+
- Connect as Sesame agent (WebSocket)
|
|
220
|
+
- Receive messages, route to agent loop
|
|
221
|
+
- Send responses back to channel
|
|
222
|
+
- Handle DMs and group channels
|
|
223
|
+
- *Est: 1-2 sessions*
|
|
224
|
+
|
|
225
|
+
- [ ] **1.5 — Write-through memory pipeline**
|
|
226
|
+
- Every conversation turn → episode stored in L2 automatically
|
|
227
|
+
- Embed + index happens synchronously (local Ollama = fast enough)
|
|
228
|
+
- Episode includes: timestamp, context name, role, content, embedding
|
|
229
|
+
- On next message: top-K relevant episodes pulled into L1 prompt
|
|
230
|
+
- *Est: 1 session*
|
|
231
|
+
|
|
232
|
+
- [ ] **1.6 — Integration test: "Remember me"**
|
|
233
|
+
- Deploy on Mac mini
|
|
234
|
+
- Start Ollama with embedding model
|
|
235
|
+
- Start memory daemon
|
|
236
|
+
- Start agent runtime, connect to Sesame
|
|
237
|
+
- Have a conversation, wait 24h, verify recall
|
|
238
|
+
- *Est: 1 session + waiting*
|
|
239
|
+
|
|
240
|
+
**Phase 1 deliverable:** A Hivemind agent in our Sesame channel that remembers conversations across sessions without any manual memory management.
|
|
241
|
+
|
|
242
|
+
---
|
|
243
|
+
|
|
244
|
+
### Phase 2 — Multi-Context with Isolation
|
|
245
|
+
|
|
246
|
+
**Goal:** Multiple project contexts with full memory isolation. Work on two projects without contamination.
|
|
247
|
+
|
|
248
|
+
#### Tasks
|
|
249
|
+
|
|
250
|
+
- [ ] **2.1 — Context Manager**
|
|
251
|
+
- Context creation/deletion/listing
|
|
252
|
+
- Message routing: infer context from message content or explicit switching ("switch to LinkZero")
|
|
253
|
+
- Context metadata: name, description, created date, last active
|
|
254
|
+
- Global context (always loaded alongside active project context)
|
|
255
|
+
- *Est: 2 sessions*
|
|
256
|
+
|
|
257
|
+
- [ ] **2.2 — Partitioned memory**
|
|
258
|
+
- LanceDB namespace per context (separate tables)
|
|
259
|
+
- Episode storage tagged with context ID
|
|
260
|
+
- Search scoped to active context + global
|
|
261
|
+
- Relevance scoring per-context (independent decay curves)
|
|
262
|
+
- *Est: 1-2 sessions*
|
|
263
|
+
|
|
264
|
+
- [ ] **2.3 — L2→L3 promotion engine**
|
|
265
|
+
- Track access frequency per episode
|
|
266
|
+
- Track connection density (episodes referenced together)
|
|
267
|
+
- Threshold-based promotion: episode → semantic knowledge
|
|
268
|
+
- L3 entries are higher-level (patterns, decisions, architecture choices)
|
|
269
|
+
- No LLM involved — just metadata promotion + re-indexing
|
|
270
|
+
- *Est: 2 sessions*
|
|
271
|
+
|
|
272
|
+
- [ ] **2.4 — Cross-context features**
|
|
273
|
+
- Cross-context search: query all contexts, results labeled by source
|
|
274
|
+
- Explicit sharing: "share this with [context]"
|
|
275
|
+
- Auto-promotion to Global: knowledge referenced in 3+ contexts
|
|
276
|
+
- *Est: 1-2 sessions*
|
|
277
|
+
|
|
278
|
+
- [ ] **2.5 — Task engine (per-context)**
|
|
279
|
+
- Per-context task queue (TASKS stored in context memory)
|
|
280
|
+
- Task states: planned → active → complete → archived
|
|
281
|
+
- Task dependencies
|
|
282
|
+
- Agent can autonomously pick next task from queue
|
|
283
|
+
- *Est: 2 sessions*
|
|
284
|
+
|
|
285
|
+
- [ ] **2.6 — Integration test: "Two projects, no bleed"**
|
|
286
|
+
- Create two project contexts
|
|
287
|
+
- Work on both interleaved
|
|
288
|
+
- Verify: searching in context A returns nothing from context B
|
|
289
|
+
- Verify: explicit cross-context search works when requested
|
|
290
|
+
- *Est: 1 session*
|
|
291
|
+
|
|
292
|
+
**Phase 2 deliverable:** Hivemind handles multiple projects with isolated memory. Context switching is instant and clean.
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
### Phase 3 — Multi-Machine Fleet
|
|
297
|
+
|
|
298
|
+
**Goal:** Contexts distributed across Mac minis. True parallel autonomous work.
|
|
299
|
+
|
|
300
|
+
#### Tasks
|
|
301
|
+
|
|
302
|
+
- [ ] **3.1 — Worker protocol**
|
|
303
|
+
- Define Primary ↔ Worker communication protocol
|
|
304
|
+
- Worker registration and health checks
|
|
305
|
+
- Context assignment: Primary tells Worker "you own context X"
|
|
306
|
+
- Worker reports status back to Primary
|
|
307
|
+
- *Est: 2 sessions*
|
|
308
|
+
|
|
309
|
+
- [ ] **3.2 — Distributed context execution**
|
|
310
|
+
- Worker runs its own memory daemon + Ollama + agent runtime
|
|
311
|
+
- Context state is local to the worker (no shared DB needed for MVP)
|
|
312
|
+
- Worker executes tasks autonomously from its context's task queue
|
|
313
|
+
- *Est: 2-3 sessions*
|
|
314
|
+
|
|
315
|
+
- [ ] **3.3 — Fleet Manager (Primary)**
|
|
316
|
+
- CLI/API to provision new workers
|
|
317
|
+
- Assign/migrate contexts between machines
|
|
318
|
+
- Aggregate status view: "what is each machine working on?"
|
|
319
|
+
- Sesame routing: messages for context X → forwarded to worker owning X
|
|
320
|
+
- *Est: 2 sessions*
|
|
321
|
+
|
|
322
|
+
- [ ] **3.4 — Cross-machine memory sync**
|
|
323
|
+
- Worker periodically syncs key L3 knowledge back to Primary
|
|
324
|
+
- Primary can push Global context updates to workers
|
|
325
|
+
- Conflict resolution: last-write-wins for L3, append-only for L2
|
|
326
|
+
- *Est: 2 sessions*
|
|
327
|
+
|
|
328
|
+
- [ ] **3.5 — Integration test: "The fleet"**
|
|
329
|
+
- 3 Mac minis running
|
|
330
|
+
- Each with a different project context
|
|
331
|
+
- All 3 making progress simultaneously
|
|
332
|
+
- Query any project's status via Sesame on Primary
|
|
333
|
+
- *Est: 1-2 sessions (plus hardware setup with Ryan)*
|
|
334
|
+
|
|
335
|
+
**Phase 3 deliverable:** A fleet of Hivemind agents working autonomously across multiple machines, coordinated through a primary node, all accessible via Sesame.
|
|
336
|
+
|
|
337
|
+
---
|
|
338
|
+
|
|
339
|
+
## Decision Boundaries
|
|
340
|
+
|
|
341
|
+
### Bailey decides autonomously:
|
|
342
|
+
- Implementation details (function signatures, data structures, error handling)
|
|
343
|
+
- Library/crate choices within the agreed stack
|
|
344
|
+
- File organization within the agreed project structure
|
|
345
|
+
- Test strategy and coverage
|
|
346
|
+
- Git workflow (branching, commits, PRs)
|
|
347
|
+
- Bug fixes and refactoring
|
|
348
|
+
- Task ordering within a phase
|
|
349
|
+
|
|
350
|
+
### Ping Ryan for:
|
|
351
|
+
- External service setup (new Sesame agents, ClawRouter config changes)
|
|
352
|
+
- Architectural changes that deviate from this plan
|
|
353
|
+
- Adding new external dependencies or services not in the stack
|
|
354
|
+
- UX decisions (how the agent presents itself, conversation style)
|
|
355
|
+
- Scope changes (adding/removing features from a phase)
|
|
356
|
+
- Any spend (new services, API costs beyond existing accounts)
|
|
357
|
+
- Phase completion review before moving to next phase
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
## Open Questions (Resolve During Build)
|
|
362
|
+
|
|
363
|
+
1. **Embedding model choice:** `nomic-embed-text` vs `mxbai-embed-large` — need to benchmark on Apple Silicon for latency/quality tradeoff
|
|
364
|
+
2. **Memory daemon API:** REST (simpler) vs gRPC (faster, typed) — start REST, upgrade if latency matters
|
|
365
|
+
3. **Episode granularity:** Per-message? Per-turn (user+assistant)? Per-conversation? Start per-turn, can adjust.
|
|
366
|
+
4. **Context inference:** How smart does the Context Manager need to be about guessing which project a message relates to? Start with explicit switching, add inference later.
|
|
367
|
+
5. **Sesame agent identity:** One Hivemind agent on Sesame, or one per context? Start with one, route internally.
|
|
368
|
+
|
|
369
|
+
---
|
|
370
|
+
|
|
371
|
+
## References
|
|
372
|
+
|
|
373
|
+
- [LanceDB docs](https://lancedb.github.io/lancedb/)
|
|
374
|
+
- [Ollama API](https://github.com/ollama/ollama/blob/main/docs/api.md)
|
|
375
|
+
- [Sesame docs](https://sesame.space/docs)
|
|
376
|
+
- [ClawRouter](https://clawrouter.app)
|
|
377
|
+
- [Axum (Rust web framework)](https://github.com/tokio-rs/axum)
|
|
378
|
+
|
|
379
|
+
---
|
|
380
|
+
|
|
381
|
+
*Created: 2025-07-18*
|
|
382
|
+
*Last updated: 2025-07-18*
|
|
383
|
+
*Status: Planning complete — ready for Phase 1 execution*
|
package/TASKS.md
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Hivemind — Task Tracker
|
|
2
|
+
|
|
3
|
+
*Auto-maintained by Bailey. Updated after each coding session.*
|
|
4
|
+
|
|
5
|
+
## Current Phase: 4 — Deployment & Distribution
|
|
6
|
+
|
|
7
|
+
| # | Task | Status | Notes |
|
|
8
|
+
|---|------|--------|-------|
|
|
9
|
+
| 4.1 | Sesame integration fixes | ✅ done | sendMessage API, sender handles, group chat awareness |
|
|
10
|
+
| 4.2 | Team Charter system | ✅ done | config/TEAM-CHARTER.md, __SKIP__ for group etiquette |
|
|
11
|
+
| 4.3 | CLI commands | ✅ done | init, start, service, fleet |
|
|
12
|
+
| 4.4 | Config layering | ✅ done | default.toml + local.toml + .env + env vars |
|
|
13
|
+
| 4.5 | npm publish | ✅ done | @sesamespace/hivemind@0.1.0 (public) |
|
|
14
|
+
| 4.6 | launchd services | ✅ done | hivemind service install/uninstall/status/logs |
|
|
15
|
+
| 4.7 | Sesame bootstrap endpoint | ✅ done | GET /api/v1/hivemind/install (pushed, needs deploy) |
|
|
16
|
+
| 4.8 | hivemind.team DNS | ⏳ pending | Ryan to configure, points to Sesame API |
|
|
17
|
+
| 4.9 | Fresh machine test | ⏳ pending | Full bootstrap on new Mac mini |
|
|
18
|
+
| 4.10 | Vault provisioning | ⏳ pending | Needs vault creation in Sesame |
|
|
19
|
+
| 4.11 | hivemind update command | ⏳ queued | npm update + vault re-check |
|
|
20
|
+
| 4.12 | hivemind-help channel | ⏳ queued | Create on Sesame, add to default channels |
|
|
21
|
+
|
|
22
|
+
## Phase 3 — Multi-Machine Fleet ✅ COMPLETE
|
|
23
|
+
|
|
24
|
+
| # | Task | Status | Notes |
|
|
25
|
+
|---|------|--------|-------|
|
|
26
|
+
| 3.1 | Worker protocol | ✅ done | Shared types, PrimaryClient, WorkerServer, 20 tests |
|
|
27
|
+
| 3.2 | Distributed context execution | ✅ done | Worker mode, config, task loop |
|
|
28
|
+
| 3.3 | Fleet Manager | ✅ done | CLI/API for provisioning, context migration, subnet discovery |
|
|
29
|
+
| 3.4 | Cross-machine memory sync | ✅ done | L3 last-write-wins, L2 append-only, sync push/pull |
|
|
30
|
+
| 3.5 | Integration test | ✅ done | 21 tests: multi-worker registration, context distribution, sync, dashboard |
|
|
31
|
+
|
|
32
|
+
## Phase 2 — Multi-Context ✅ COMPLETE
|
|
33
|
+
|
|
34
|
+
| # | Task | Status | Notes |
|
|
35
|
+
|---|------|--------|-------|
|
|
36
|
+
| 2.1 | Context Manager | ✅ done | Message routing, context-aware agent loop |
|
|
37
|
+
| 2.2 | Partitioned memory | ✅ done | Per-context LanceDB namespaces, independent decay |
|
|
38
|
+
| 2.3 | L2→L3 promotion engine | ✅ done | Access frequency + connection density, threshold promotion |
|
|
39
|
+
| 2.4 | Cross-context features | ✅ done | Cross-context search, sharing, auto-promotion to Global |
|
|
40
|
+
| 2.5 | Task engine (per-context) | ✅ done | Per-context task queues, chat commands, dependencies |
|
|
41
|
+
| 2.6 | Integration tests | ✅ done | 8/8 tests passing |
|
|
42
|
+
|
|
43
|
+
## Phase 1 — Single Context ✅ COMPLETE
|
|
44
|
+
|
|
45
|
+
| # | Task | Status | Notes |
|
|
46
|
+
|---|------|--------|-------|
|
|
47
|
+
| 1.1 | Project scaffolding | ✅ done | pnpm monorepo, TS + Rust, config |
|
|
48
|
+
| 1.2 | Memory daemon MVP | ✅ done | Rust/axum, LanceDB, Ollama embeddings |
|
|
49
|
+
| 1.3 | Agent runtime MVP | ✅ done | Core agent loop, memory client, LLM client |
|
|
50
|
+
| 1.4 | Sesame integration | ✅ done | WebSocket + REST, reconnection |
|
|
51
|
+
| 1.5 | Write-through pipeline | ✅ done | Auto-embed every turn, top-K retrieval |
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Test Suite: 49 tests passing
|
|
56
|
+
- 20 worker protocol
|
|
57
|
+
- 8 memory integration
|
|
58
|
+
- 21 fleet integration
|
|
59
|
+
|
|
60
|
+
*Last updated: 2026-02-23*
|
package/install.sh
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
set -euo pipefail
|
|
3
|
+
|
|
4
|
+
# Hivemind — One-shot install for macOS (Apple Silicon)
|
|
5
|
+
# Usage (from repo): cd hivemind && ./install.sh <sesame-api-key>
|
|
6
|
+
# Preferred: curl -sL api.sesame.space/api/v1/hivemind/install | bash -s -- <sesame-api-key>
|
|
7
|
+
|
|
8
|
+
BOLD='\033[1m'
|
|
9
|
+
GREEN='\033[0;32m'
|
|
10
|
+
YELLOW='\033[0;33m'
|
|
11
|
+
RED='\033[0;31m'
|
|
12
|
+
NC='\033[0m'
|
|
13
|
+
|
|
14
|
+
info() { echo -e "${GREEN}[✓]${NC} $1"; }
|
|
15
|
+
warn() { echo -e "${YELLOW}[!]${NC} $1"; }
|
|
16
|
+
err() { echo -e "${RED}[✗]${NC} $1"; }
|
|
17
|
+
step() { echo -e "\n${BOLD}→ $1${NC}"; }
|
|
18
|
+
|
|
19
|
+
echo -e "${BOLD}"
|
|
20
|
+
echo " ╦ ╦╦╦ ╦╔═╗╔╦╗╦╔╗╔╔╦╗"
|
|
21
|
+
echo " ╠═╣║╚╗╔╝║╣ ║║║║║║║ ║║"
|
|
22
|
+
echo " ╩ ╩╩ ╚╝ ╚═╝╩ ╩╩╝╚╝═╩╝"
|
|
23
|
+
echo -e "${NC}"
|
|
24
|
+
echo " Agent Runtime Installer"
|
|
25
|
+
echo ""
|
|
26
|
+
|
|
27
|
+
# --- 1. System dependencies ---
|
|
28
|
+
step "Checking system dependencies"
|
|
29
|
+
|
|
30
|
+
# Homebrew
|
|
31
|
+
if ! command -v brew &>/dev/null; then
|
|
32
|
+
warn "Homebrew not found — installing..."
|
|
33
|
+
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
|
34
|
+
eval "$(/opt/homebrew/bin/brew shellenv)"
|
|
35
|
+
else
|
|
36
|
+
info "Homebrew found"
|
|
37
|
+
fi
|
|
38
|
+
|
|
39
|
+
# Node.js
|
|
40
|
+
if ! command -v node &>/dev/null; then
|
|
41
|
+
warn "Node.js not found — installing via brew..."
|
|
42
|
+
brew install node
|
|
43
|
+
else
|
|
44
|
+
NODE_VER=$(node -v | sed 's/v//' | cut -d. -f1)
|
|
45
|
+
if [ "$NODE_VER" -lt 20 ]; then
|
|
46
|
+
warn "Node.js v${NODE_VER} too old (need ≥20) — upgrading..."
|
|
47
|
+
brew upgrade node
|
|
48
|
+
else
|
|
49
|
+
info "Node.js $(node -v)"
|
|
50
|
+
fi
|
|
51
|
+
fi
|
|
52
|
+
|
|
53
|
+
# pnpm
|
|
54
|
+
if ! command -v pnpm &>/dev/null; then
|
|
55
|
+
warn "pnpm not found — installing..."
|
|
56
|
+
npm install -g pnpm
|
|
57
|
+
else
|
|
58
|
+
info "pnpm $(pnpm -v)"
|
|
59
|
+
fi
|
|
60
|
+
|
|
61
|
+
# Rust
|
|
62
|
+
if ! command -v cargo &>/dev/null; then
|
|
63
|
+
warn "Rust not found — installing via rustup..."
|
|
64
|
+
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
|
|
65
|
+
source "$HOME/.cargo/env"
|
|
66
|
+
else
|
|
67
|
+
info "Rust $(rustc --version | awk '{print $2}')"
|
|
68
|
+
fi
|
|
69
|
+
|
|
70
|
+
# Ollama
|
|
71
|
+
if ! command -v ollama &>/dev/null; then
|
|
72
|
+
warn "Ollama not found — installing via brew..."
|
|
73
|
+
brew install ollama
|
|
74
|
+
else
|
|
75
|
+
info "Ollama found"
|
|
76
|
+
fi
|
|
77
|
+
|
|
78
|
+
# --- 2. Pull embedding model ---
|
|
79
|
+
step "Setting up Ollama embedding model"
|
|
80
|
+
|
|
81
|
+
# Start Ollama if not running
|
|
82
|
+
if ! curl -s http://localhost:11434/api/tags &>/dev/null; then
|
|
83
|
+
warn "Ollama not running — starting..."
|
|
84
|
+
ollama serve &>/dev/null &
|
|
85
|
+
sleep 3
|
|
86
|
+
fi
|
|
87
|
+
|
|
88
|
+
if ollama list 2>/dev/null | grep -q "nomic-embed-text"; then
|
|
89
|
+
info "nomic-embed-text model ready"
|
|
90
|
+
else
|
|
91
|
+
warn "Pulling nomic-embed-text (274MB)..."
|
|
92
|
+
ollama pull nomic-embed-text
|
|
93
|
+
info "nomic-embed-text ready"
|
|
94
|
+
fi
|
|
95
|
+
|
|
96
|
+
# --- 3. Build TypeScript packages ---
|
|
97
|
+
step "Installing Node dependencies"
|
|
98
|
+
pnpm install
|
|
99
|
+
info "Dependencies installed"
|
|
100
|
+
|
|
101
|
+
step "Building TypeScript"
|
|
102
|
+
pnpm build
|
|
103
|
+
info "TypeScript build complete"
|
|
104
|
+
|
|
105
|
+
# --- 4. Build Rust memory daemon ---
|
|
106
|
+
step "Building memory daemon (Rust — this takes ~60s first time)"
|
|
107
|
+
cd packages/memory
|
|
108
|
+
cargo build --release 2>&1 | tail -3
|
|
109
|
+
cd ../..
|
|
110
|
+
info "Memory daemon built"
|
|
111
|
+
|
|
112
|
+
# --- 5. Configuration via Sesame ---
|
|
113
|
+
step "Configuring agent via Sesame"
|
|
114
|
+
|
|
115
|
+
if [ -f ".env" ]; then
|
|
116
|
+
info "Existing .env found — keeping it"
|
|
117
|
+
else
|
|
118
|
+
SESAME_KEY="${1:-${SESAME_API_KEY:-}}"
|
|
119
|
+
if [ -z "$SESAME_KEY" ]; then
|
|
120
|
+
read -rp " Sesame API key: " SESAME_KEY
|
|
121
|
+
fi
|
|
122
|
+
node packages/cli/dist/main.js init "$SESAME_KEY"
|
|
123
|
+
fi
|
|
124
|
+
|
|
125
|
+
# --- 6. Create launch script ---
|
|
126
|
+
step "Creating launch script"
|
|
127
|
+
|
|
128
|
+
cat > start.sh <<'LAUNCH'
|
|
129
|
+
#!/usr/bin/env bash
|
|
130
|
+
set -euo pipefail
|
|
131
|
+
|
|
132
|
+
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
133
|
+
cd "$SCRIPT_DIR"
|
|
134
|
+
|
|
135
|
+
# Load env
|
|
136
|
+
if [ -f .env ]; then
|
|
137
|
+
set -a; source .env; set +a
|
|
138
|
+
fi
|
|
139
|
+
|
|
140
|
+
# Start memory daemon if not running
|
|
141
|
+
if ! curl -s http://localhost:3434/health &>/dev/null; then
|
|
142
|
+
echo "[hivemind] Starting memory daemon..."
|
|
143
|
+
nohup ./packages/memory/target/release/hivemind-memory > /tmp/hivemind-memory.log 2>&1 &
|
|
144
|
+
sleep 2
|
|
145
|
+
if curl -s http://localhost:3434/health &>/dev/null; then
|
|
146
|
+
echo "[hivemind] Memory daemon ready"
|
|
147
|
+
else
|
|
148
|
+
echo "[hivemind] WARNING: Memory daemon failed to start"
|
|
149
|
+
fi
|
|
150
|
+
else
|
|
151
|
+
echo "[hivemind] Memory daemon already running"
|
|
152
|
+
fi
|
|
153
|
+
|
|
154
|
+
# Start Ollama if not running
|
|
155
|
+
if ! curl -s http://localhost:11434/api/tags &>/dev/null; then
|
|
156
|
+
echo "[hivemind] Starting Ollama..."
|
|
157
|
+
ollama serve &>/dev/null &
|
|
158
|
+
sleep 3
|
|
159
|
+
fi
|
|
160
|
+
|
|
161
|
+
# Start agent (auto-restart on crash)
|
|
162
|
+
echo "[hivemind] Starting agent..."
|
|
163
|
+
while true; do
|
|
164
|
+
node packages/cli/dist/main.js start --config config/default.toml
|
|
165
|
+
EXIT_CODE=$?
|
|
166
|
+
echo "[hivemind] Agent exited with code $EXIT_CODE — restarting in 5s..."
|
|
167
|
+
sleep 5
|
|
168
|
+
done
|
|
169
|
+
LAUNCH
|
|
170
|
+
|
|
171
|
+
chmod +x start.sh
|
|
172
|
+
info "Created start.sh (with auto-restart)"
|
|
173
|
+
|
|
174
|
+
# --- Done ---
|
|
175
|
+
echo ""
|
|
176
|
+
echo -e "${GREEN}${BOLD} ✓ Hivemind installed successfully!${NC}"
|
|
177
|
+
echo ""
|
|
178
|
+
echo " To start the agent:"
|
|
179
|
+
echo " ./start.sh"
|
|
180
|
+
echo ""
|
|
181
|
+
echo " To start in background:"
|
|
182
|
+
echo " nohup ./start.sh > /tmp/hivemind.log 2>&1 &"
|
|
183
|
+
echo ""
|
|
184
|
+
echo " Logs:"
|
|
185
|
+
echo " Agent: stdout (or /tmp/hivemind.log)"
|
|
186
|
+
echo " Memory: /tmp/hivemind-memory.log"
|
|
187
|
+
echo ""
|
package/npm-package.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@sesamespace/hivemind",
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "Cognitive architecture for AI agents with multi-layered memory",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"hivemind": "dist/main.js"
|
|
8
|
+
},
|
|
9
|
+
"main": "dist/index.js",
|
|
10
|
+
"files": [
|
|
11
|
+
"dist",
|
|
12
|
+
"config",
|
|
13
|
+
"README.md"
|
|
14
|
+
],
|
|
15
|
+
"dependencies": {
|
|
16
|
+
"@iarna/toml": "^2.2.5",
|
|
17
|
+
"@sesamespace/sdk": "^0.1.6",
|
|
18
|
+
"ws": "^8.18.0"
|
|
19
|
+
},
|
|
20
|
+
"engines": {
|
|
21
|
+
"node": ">=20.0.0"
|
|
22
|
+
},
|
|
23
|
+
"license": "MIT",
|
|
24
|
+
"repository": {
|
|
25
|
+
"type": "git",
|
|
26
|
+
"url": "https://github.com/baileydavis2026/hivemind"
|
|
27
|
+
}
|
|
28
|
+
}
|