@cordfuse/crosstalk 7.0.0-beta.1 → 8.0.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GUIDE-CLI.md +1 -1
- package/README.md +32 -12
- package/commands/server.js +35 -3
- package/commands/transport.js +53 -24
- package/lib/nativeServer.js +11 -2
- package/lib/resolve.js +5 -3
- package/package.json +3 -3
- package/src/api.ts +12 -11
- package/src/dispatch.ts +66 -28
- package/src/models.ts +17 -0
- package/src/state.ts +14 -13
- package/src/transport.ts +113 -3
- package/src/web/dashboard.ts +2 -1
- package/template/CLAUDE.md +1 -1
- package/template/CROSSTALK.md +7 -7
- package/template/PROTOCOL.md +1 -1
- package/template/README.md +1 -1
package/GUIDE-CLI.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Reference for the `crosstalk` CLI. Run `crosstalk <noun> --help` (or `crosstalk <noun> <verb> --help`) for inline flags.
|
|
4
4
|
|
|
5
|
-
Requires: Node.js ≥ 20 + git on PATH.
|
|
5
|
+
Requires: Node.js ≥ 20 + git on PATH. Single-package install: `npm install -g @cordfuse/crosstalk` (current: beta.1). The daemon lives as `crosstalk daemon …` inside the same package; there's no separate `crosstalkd` binary anymore (collapse landed in alpha.18; `@cordfuse/crosstalkd` is deprecated on npm).
|
|
6
6
|
|
|
7
7
|
The surface is strict **noun-verb**, mirroring [`@cordfuse/llmux`](https://www.npmjs.com/package/@cordfuse/llmux):
|
|
8
8
|
|
package/README.md
CHANGED
|
@@ -1,16 +1,16 @@
|
|
|
1
1
|
# Crosstalk
|
|
2
2
|
|
|
3
|
-
**A shared message bus for humans and AI agents,
|
|
3
|
+
**A shared message bus for humans and AI agents. The bus is a directory — carried by git, or a plain/shared filesystem.**
|
|
4
4
|
|
|
5
|
-
> **
|
|
5
|
+
> **Status (8.x).** Crosstalk is a single npm package — `@cordfuse/crosstalk` (the separate `@cordfuse/crosstalkd` daemon package is deprecated; the daemon lives as `crosstalk daemon dispatch` inside this same package). CLI surface is strict noun-verb, mirroring `@cordfuse/llmux`. **As of 8.x the bus is transport-pluggable:** git (distributed, versioned, auditable) is the default; a local or shared filesystem — `transport: local`, including a mounted SMB/NFS share — needs no git and no remote. Select per transport with the `transport:` key in `data/crosstalk.yaml`.
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
> ### Six words that come up everywhere on this page
|
|
10
10
|
>
|
|
11
|
-
> - **Transport** —
|
|
11
|
+
> - **Transport** — the message bus, scaffolded by `crosstalk transport init`. A directory: a git repo (default — distributed, versioned) or a plain/shared filesystem (`transport: local`). It IS the bus.
|
|
12
12
|
> - **Machine** — a host running one crosstalk engine. Identifies itself via the dispatcher `--alias` (defaulting to the transport name).
|
|
13
|
-
> - **Message** — a markdown file with YAML frontmatter,
|
|
13
|
+
> - **Message** — a markdown file with YAML frontmatter, written to a channel. The unit of work.
|
|
14
14
|
> - **Model** — a named agent invocation declared in `data/crosstalk.yaml` (e.g. `sonnet`, `codex-o3`).
|
|
15
15
|
> - **Actor** — an optional persona file (`local/actors/<name>.md`) that prepends to a model's prompt as system context.
|
|
16
16
|
> - **Channel** — a UUID directory under `data/channels/`. A conversation thread. Optionally parented.
|
|
@@ -32,15 +32,15 @@ The missing piece: a way to make **any** agent CLI talk to **any** other agent C
|
|
|
32
32
|
|
|
33
33
|
## The solution
|
|
34
34
|
|
|
35
|
-
Crosstalk is that missing layer. The whole protocol is one idea — **a
|
|
35
|
+
Crosstalk is that missing layer. The whole protocol is one idea — **a directory is the message bus.** Carry that directory with git (distributed, versioned, auditable) or a plain/shared filesystem (local, LAN, a mounted share) — same protocol either way.
|
|
36
36
|
|
|
37
37
|
- **Any CLI participates.** If an agent's CLI runs one prompt non-interactively and prints a reply, it's a valid model. Mix Claude Code, Codex, Gemini CLI, Qwen Code, opencode, Antigravity — or any future CLI that follows the same shape — in one transport.
|
|
38
|
-
- **Messages are
|
|
38
|
+
- **Messages are files.** Every send is a markdown file with YAML frontmatter dropped into a channel. On the git transport each is a commit, so the whole conversation is git history — nothing to lose, nothing hidden, nothing to back up separately. On the filesystem transport it's just the files on disk.
|
|
39
39
|
- **Peer-to-peer.** Each machine runs its own crosstalk engine. No broker, no central runtime, no registry.
|
|
40
|
-
- **Multi-machine for free.** Git already solves "synchronize this across hosts
|
|
41
|
-
- **Self-coordinating.** Collision-free filenames
|
|
40
|
+
- **Multi-machine for free.** Git already solves "synchronize this across hosts"; a shared mount (SMB/NFS) does the same on a LAN. Crosstalk inherits whichever you point it at.
|
|
41
|
+
- **Self-coordinating.** Collision-free filenames mean concurrent writers never clash; the git transport adds rebase-retry on push, the filesystem transport relies on the unique filenames alone. Either way the bus works correctly with no central coordinator.
|
|
42
42
|
|
|
43
|
-
|
|
43
|
+
No infrastructure to provision, no central server holding your conversation — just a directory, on git or a filesystem.
|
|
44
44
|
|
|
45
45
|
---
|
|
46
46
|
|
|
@@ -235,6 +235,26 @@ Same flow works for self-hosted Gitea or GitLab. Crosstalk doesn't care; it only
|
|
|
235
235
|
|
|
236
236
|
---
|
|
237
237
|
|
|
238
|
+
## Local filesystem transport (no git)
|
|
239
|
+
|
|
240
|
+
Don't want a git remote? Point Crosstalk at a plain directory instead — a local folder for a single host, or a **mounted SMB/NFS share** for several hosts on a LAN. No git, no remote, no infrastructure.
|
|
241
|
+
|
|
242
|
+
```sh
|
|
243
|
+
# Scaffold a local-fs transport straight onto the directory (e.g. a NAS mount):
|
|
244
|
+
crosstalk transport init --transport local --path /mnt/share/crosstalk
|
|
245
|
+
|
|
246
|
+
# Run the engine on it — the bus lives on the share, state stays machine-local:
|
|
247
|
+
crosstalk server start --path /mnt/share/crosstalk
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
`--transport local` writes `transport: local` into `data/crosstalk.yaml` (no git init), and the dispatcher self-selects the filesystem transport: it reads/writes message files directly and picks up new ones via an mtime cursor — event-driven `fs.watch` on a true local FS, polling over a network mount. Each host that mounts the share runs its own engine and keeps its own machine-local cursor/heartbeat, so concurrent writers never clash (collision-free filenames).
|
|
251
|
+
|
|
252
|
+
Tradeoffs vs. git: no version history, no audit trail, no conflict resolution beyond the unique filenames — but zero setup and no SSH/credentials. Reach for git when *what was said* matters; reach for the filesystem transport for quick local/LAN swarms.
|
|
253
|
+
|
|
254
|
+
> **SFTP?** Mount it with `sshfs` and use `transport: local` — Crosstalk just sees a directory.
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
238
258
|
## Adding a second machine
|
|
239
259
|
|
|
240
260
|
Multi-machine adds exactly one new idea: **routing by alias.**
|
|
@@ -424,8 +444,8 @@ Transport git repos are portable — the storage path is unchanged (`~/.local/sh
|
|
|
424
444
|
If you have a v7 Docker install:
|
|
425
445
|
|
|
426
446
|
1. `docker compose -f <your-compose-file> down` first.
|
|
427
|
-
2. `npm install -g @cordfuse/crosstalk@latest` — pulls
|
|
428
|
-
3. If you previously installed `@cordfuse/crosstalkd`: `npm uninstall -g @cordfuse/crosstalkd` (deprecated, no longer published
|
|
447
|
+
2. `npm install -g @cordfuse/crosstalk@latest` — pulls the latest 8.x.
|
|
448
|
+
3. If you previously installed `@cordfuse/crosstalkd`: `npm uninstall -g @cordfuse/crosstalkd` (deprecated, no longer published since alpha.18).
|
|
429
449
|
4. In user mode: `export CROSSTALK_USER_MODE=1 && crosstalk server start --containername <name>`.
|
|
430
450
|
5. In system mode: rerun `sudo ./deploy/install.sh` to lay down the new `crosstalk@.service` unit, then `sudo systemctl daemon-reload && sudo systemctl enable --now crosstalk@<name>`. The old `crosstalkd@<name>.service` files can be removed.
|
|
431
451
|
|
|
@@ -435,4 +455,4 @@ The v7 GHCR image `ghcr.io/cordfuse/crosstalk-server:*` is no longer being updat
|
|
|
435
455
|
|
|
436
456
|
## Status
|
|
437
457
|
|
|
438
|
-
Crosstalk
|
|
458
|
+
Crosstalk 8.x — single-package monorepo + strict noun-verb CLI, transport-pluggable (git or a local/shared filesystem). The protocol version lives at `CROSSTALK-VERSION` (single integer) at the transport root. Published as [`@cordfuse/crosstalk`](https://www.npmjs.com/package/@cordfuse/crosstalk).
|
package/commands/server.js
CHANGED
|
@@ -20,9 +20,10 @@
|
|
|
20
20
|
|
|
21
21
|
import { spawn } from 'child_process';
|
|
22
22
|
import { existsSync } from 'fs';
|
|
23
|
-
import {
|
|
23
|
+
import { join, resolve } from 'path';
|
|
24
|
+
import { has, flag } from '../lib/argv.js';
|
|
24
25
|
import {
|
|
25
|
-
requireInitialized, storageMode,
|
|
26
|
+
requireInitialized, storageMode, parseContainerName,
|
|
26
27
|
DEFAULT_CONTAINER_NAME,
|
|
27
28
|
} from '../lib/resolve.js';
|
|
28
29
|
import { startEngine, stopEngine, engineStatus } from '../lib/nativeServer.js';
|
|
@@ -56,7 +57,9 @@ function usage(exit = 0) {
|
|
|
56
57
|
`Usage: crosstalk server <subverb> [--containername <name>]
|
|
57
58
|
|
|
58
59
|
Subverbs:
|
|
59
|
-
start
|
|
60
|
+
start [--path <dir>] start the engine; --path runs it on a transport at an
|
|
61
|
+
arbitrary directory (e.g. a mounted share), state stays
|
|
62
|
+
machine-local (user mode only)
|
|
60
63
|
stop SIGTERM the running engine; SIGKILL after 5s grace
|
|
61
64
|
restart stop + start
|
|
62
65
|
status [--probe] report pid + uptime; --probe also pings /healthz
|
|
@@ -68,6 +71,35 @@ Storage mode defaults to 'system'; set CROSSTALK_USER_MODE=1 for user mode.
|
|
|
68
71
|
}
|
|
69
72
|
|
|
70
73
|
async function cmdStart(argv) {
|
|
74
|
+
// --path <dir>: run the engine on a transport at an arbitrary directory
|
|
75
|
+
// (e.g. a mounted SMB/NFS share) instead of the managed storage layout.
|
|
76
|
+
// State (cursor/heartbeat/pid/port) stays machine-local, keyed by name.
|
|
77
|
+
const transportPath = flag(argv, '--path');
|
|
78
|
+
if (transportPath) {
|
|
79
|
+
if (storageMode() === 'system') {
|
|
80
|
+
process.stderr.write(`crosstalk server start: --path is user-mode only (set CROSSTALK_USER_MODE=1).\n`);
|
|
81
|
+
return 1;
|
|
82
|
+
}
|
|
83
|
+
if (!existsSync(join(transportPath, 'CROSSTALK-VERSION'))) {
|
|
84
|
+
process.stderr.write(
|
|
85
|
+
`crosstalk server start: '${transportPath}' is not a crosstalk transport (no CROSSTALK-VERSION).\n` +
|
|
86
|
+
` Scaffold it first: crosstalk daemon init ${transportPath}\n`,
|
|
87
|
+
);
|
|
88
|
+
return 1;
|
|
89
|
+
}
|
|
90
|
+
const name = parseContainerName(argv);
|
|
91
|
+
const r = startEngine(name, { transportPath });
|
|
92
|
+
if (!r.ok) {
|
|
93
|
+
process.stderr.write(`crosstalk server start: ${r.error}\n`);
|
|
94
|
+
return 1;
|
|
95
|
+
}
|
|
96
|
+
process.stdout.write(
|
|
97
|
+
`'${name}' started on ${resolve(transportPath)} (pid ${r.pid}, api 127.0.0.1:${r.port}).\n` +
|
|
98
|
+
` state stays machine-local; logs: crosstalk server logs${nameSuffix(name)} -f\n`,
|
|
99
|
+
);
|
|
100
|
+
return 0;
|
|
101
|
+
}
|
|
102
|
+
|
|
71
103
|
const { name } = requireInitialized(argv);
|
|
72
104
|
if (storageMode() === 'system') return redirectToSystemctl('start', name);
|
|
73
105
|
const r = startEngine(name);
|
package/commands/transport.js
CHANGED
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
import { existsSync, mkdirSync, readFileSync, writeFileSync, rmSync } from 'fs';
|
|
15
15
|
import { spawnSync } from 'child_process';
|
|
16
16
|
import { createInterface } from 'readline';
|
|
17
|
-
import { join } from 'path';
|
|
17
|
+
import { join, resolve } from 'path';
|
|
18
18
|
import { has, flag } from '../lib/argv.js';
|
|
19
19
|
import {
|
|
20
20
|
parseContainerName,
|
|
@@ -29,7 +29,7 @@ function usage(exit = 0) {
|
|
|
29
29
|
const w = exit === 0 ? process.stdout : process.stderr;
|
|
30
30
|
w.write(
|
|
31
31
|
`Usage:
|
|
32
|
-
crosstalk transport init [--containername <name>] [--remote <url>]
|
|
32
|
+
crosstalk transport init [--containername <name>] [--transport git|local] [--path <dir>] [--remote <url>]
|
|
33
33
|
crosstalk transport rm [--containername <name>] [--force]
|
|
34
34
|
crosstalk transport --help
|
|
35
35
|
|
|
@@ -75,12 +75,22 @@ async function runInit(argv) {
|
|
|
75
75
|
}
|
|
76
76
|
|
|
77
77
|
const remote = flag(argv, '--remote');
|
|
78
|
+
const transportKind = flag(argv, '--transport') === 'local' ? 'local' : 'git';
|
|
79
|
+
if (transportKind === 'local' && remote) {
|
|
80
|
+
process.stderr.write(`crosstalk transport init: --remote is ignored for --transport local (no git, no upstream).\n`);
|
|
81
|
+
}
|
|
78
82
|
const paths = storagePaths(name);
|
|
83
|
+
// --path <dir>: scaffold the transport at an arbitrary directory (e.g. a
|
|
84
|
+
// mounted share) instead of the managed storage layout. Port + state stay
|
|
85
|
+
// machine-local under the name-keyed storage root; only the transport
|
|
86
|
+
// (template + channels) lives at the path. Pairs with `server start --path`.
|
|
87
|
+
const customPath = flag(argv, '--path');
|
|
88
|
+
const transportDir = customPath ? resolve(customPath) : paths.transportDir;
|
|
79
89
|
const report = [];
|
|
80
90
|
|
|
81
91
|
// ── Phase 1: Storage layout ────────────────────────────────────────
|
|
82
92
|
try {
|
|
83
|
-
mkdirSync(
|
|
93
|
+
mkdirSync(transportDir, { recursive: true });
|
|
84
94
|
mkdirSync(paths.crosstalkRoot, { recursive: true });
|
|
85
95
|
mkdirSync(paths.crosstalkState, { recursive: true });
|
|
86
96
|
} catch (err) {
|
|
@@ -116,7 +126,7 @@ async function runInit(argv) {
|
|
|
116
126
|
}
|
|
117
127
|
|
|
118
128
|
// ── Phase 2: Transport scaffold ────────────────────────────────────
|
|
119
|
-
const crosstalkVersionFile = join(
|
|
129
|
+
const crosstalkVersionFile = join(transportDir, 'CROSSTALK-VERSION');
|
|
120
130
|
if (!existsSync(crosstalkVersionFile)) {
|
|
121
131
|
// Scaffold via `crosstalk daemon init <path>` — the same self-bin
|
|
122
132
|
// call pattern startEngine() uses.
|
|
@@ -129,7 +139,7 @@ async function runInit(argv) {
|
|
|
129
139
|
}
|
|
130
140
|
const initRun = spawnSync(
|
|
131
141
|
process.execPath,
|
|
132
|
-
[selfBin, 'daemon', 'init',
|
|
142
|
+
[selfBin, 'daemon', 'init', transportDir],
|
|
133
143
|
{ stdio: 'pipe' },
|
|
134
144
|
);
|
|
135
145
|
if (initRun.status !== 0) {
|
|
@@ -145,40 +155,59 @@ async function runInit(argv) {
|
|
|
145
155
|
report.push(` template: present (skipped scaffold)`);
|
|
146
156
|
}
|
|
147
157
|
|
|
158
|
+
// ── Phase 2.5: local transport marker ──────────────────────────────
|
|
159
|
+
// Local (fs) transports carry no git history; persist the medium in the
|
|
160
|
+
// config so the dispatcher self-selects it (no --transport flag needed).
|
|
161
|
+
if (transportKind === 'local') {
|
|
162
|
+
const yamlPath = join(transportDir, 'data', 'crosstalk.yaml');
|
|
163
|
+
try {
|
|
164
|
+
const cur = existsSync(yamlPath) ? readFileSync(yamlPath, 'utf8') : '';
|
|
165
|
+
if (!/^\s*transport\s*:/m.test(cur)) {
|
|
166
|
+
writeFileSync(yamlPath, `transport: local\n${cur}`);
|
|
167
|
+
}
|
|
168
|
+
report.push(` medium: local (fs — no git)`);
|
|
169
|
+
} catch (err) {
|
|
170
|
+
process.stderr.write(`crosstalk transport init: failed to write transport key to ${yamlPath} — ${err.message}\n`);
|
|
171
|
+
return 1;
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
|
|
148
175
|
// ── Phase 3: Git init + initial commit ─────────────────────────────
|
|
149
176
|
if (!existsSync(crosstalkVersionFile)) {
|
|
150
177
|
process.stderr.write(
|
|
151
|
-
`crosstalk transport init: transport template missing at ${
|
|
178
|
+
`crosstalk transport init: transport template missing at ${transportDir}\n` +
|
|
152
179
|
` Engine init didn't write template files. Try 'crosstalk transport rm' then re-init.\n`,
|
|
153
180
|
);
|
|
154
181
|
return 1;
|
|
155
182
|
}
|
|
156
183
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
184
|
+
if (transportKind === 'git') {
|
|
185
|
+
const gitSteps = [
|
|
186
|
+
['init', '--quiet', '--initial-branch=main'],
|
|
187
|
+
['add', '-A'],
|
|
188
|
+
['commit', '--quiet', '-m', 'initial transport'],
|
|
189
|
+
];
|
|
190
|
+
let gitOk = true;
|
|
191
|
+
for (const args of gitSteps) {
|
|
192
|
+
const gr = spawnSync('git', args, { cwd: transportDir, stdio: 'pipe' });
|
|
193
|
+
if (gr.status !== 0) {
|
|
194
|
+
gitOk = false;
|
|
195
|
+
break;
|
|
196
|
+
}
|
|
168
197
|
}
|
|
198
|
+
report.push(gitOk ? ` git: initialized + committed` : ` git: already present (skipped)`);
|
|
169
199
|
}
|
|
170
|
-
report.push(gitOk ? ` git: initialized + committed` : ` git: already present (skipped)`);
|
|
171
200
|
|
|
172
|
-
// ── Phase 4: Remote setup
|
|
173
|
-
if (remote) {
|
|
201
|
+
// ── Phase 4: Remote setup (git only) ───────────────────────────────
|
|
202
|
+
if (remote && transportKind === 'git') {
|
|
174
203
|
const cur = spawnSync('git', ['remote', 'get-url', 'origin'], {
|
|
175
|
-
cwd:
|
|
204
|
+
cwd: transportDir,
|
|
176
205
|
stdio: 'pipe',
|
|
177
206
|
});
|
|
178
207
|
const currentUrl = cur.status === 0 ? cur.stdout.toString().trim() : null;
|
|
179
208
|
if (!currentUrl) {
|
|
180
209
|
const rr = spawnSync('git', ['remote', 'add', 'origin', remote], {
|
|
181
|
-
cwd:
|
|
210
|
+
cwd: transportDir,
|
|
182
211
|
stdio: 'pipe',
|
|
183
212
|
});
|
|
184
213
|
if (rr.status !== 0) {
|
|
@@ -202,9 +231,9 @@ async function runInit(argv) {
|
|
|
202
231
|
process.stdout.write(
|
|
203
232
|
`\nTransport '${name}' ready:\n` +
|
|
204
233
|
` storage: ${paths.storageRoot}\n` +
|
|
205
|
-
` transport: ${
|
|
234
|
+
` transport: ${transportDir}\n` +
|
|
206
235
|
report.join('\n') + '\n' +
|
|
207
|
-
`\nNext: crosstalk server start${name === DEFAULT_CONTAINER_NAME ? '' : ` --containername ${name}`}\n`,
|
|
236
|
+
`\nNext: crosstalk server start${customPath ? ` --path ${transportDir}` : (name === DEFAULT_CONTAINER_NAME ? '' : ` --containername ${name}`)}\n`,
|
|
208
237
|
);
|
|
209
238
|
return 0;
|
|
210
239
|
}
|
package/lib/nativeServer.js
CHANGED
|
@@ -67,18 +67,27 @@ export function startEngine(name, opts = {}) {
|
|
|
67
67
|
const poll = String(opts.pollSeconds ?? process.env.DISPATCH_POLL_SECONDS ?? 30);
|
|
68
68
|
const port = String(apiPortFor(name));
|
|
69
69
|
|
|
70
|
+
// --path: the transport lives at an arbitrary directory (e.g. a mounted
|
|
71
|
+
// share). Point the engine there via CROSSTALK_TRANSPORT_PATH, and pin
|
|
72
|
+
// state machine-local (by name) so cursor/heartbeat never land on the
|
|
73
|
+
// share — each machine sharing the dir keeps its own local state.
|
|
74
|
+
const transportDir = opts.transportPath ? resolve(opts.transportPath) : paths.transportDir;
|
|
70
75
|
const env = {
|
|
71
76
|
...process.env,
|
|
72
77
|
CROSSTALK_API_PORT: port,
|
|
73
|
-
TRANSPORT_DIR:
|
|
78
|
+
TRANSPORT_DIR: transportDir,
|
|
74
79
|
};
|
|
80
|
+
if (opts.transportPath) {
|
|
81
|
+
env.CROSSTALK_TRANSPORT_PATH = transportDir;
|
|
82
|
+
env.CROSSTALK_STATE_DIR = paths.crosstalkState;
|
|
83
|
+
}
|
|
75
84
|
|
|
76
85
|
const logFd = openSync(paths.logFile, 'a');
|
|
77
86
|
const child = spawn(
|
|
78
87
|
process.execPath,
|
|
79
88
|
[selfBin, 'daemon', 'dispatch', '--alias', alias, '--poll', poll, '--json'],
|
|
80
89
|
{
|
|
81
|
-
cwd:
|
|
90
|
+
cwd: transportDir,
|
|
82
91
|
env,
|
|
83
92
|
stdio: ['ignore', logFd, logFd],
|
|
84
93
|
detached: true,
|
package/lib/resolve.js
CHANGED
|
@@ -187,11 +187,13 @@ export function pickFreePort(name, mode = storageMode()) {
|
|
|
187
187
|
throw new Error('Could not find a free port between 7001 and 7999.');
|
|
188
188
|
}
|
|
189
189
|
|
|
190
|
-
// True if
|
|
191
|
-
//
|
|
190
|
+
// True if the transport has been scaffolded for this name. Checks the
|
|
191
|
+
// CROSSTALK-VERSION marker that `daemon init` writes for EVERY transport —
|
|
192
|
+
// git and local-fs alike. (Was '.git', which wrongly rejected no-git local
|
|
193
|
+
// transports: `server start` reported "no transport 'mc'".)
|
|
192
194
|
export function isInitialized(name) {
|
|
193
195
|
const paths = storagePaths(name);
|
|
194
|
-
return existsSync(join(paths.transportDir, '
|
|
196
|
+
return existsSync(join(paths.transportDir, 'CROSSTALK-VERSION'));
|
|
195
197
|
}
|
|
196
198
|
|
|
197
199
|
// True if the crosstalk engine is running for this transport. Source of truth:
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cordfuse/crosstalk",
|
|
3
|
-
"version": "
|
|
4
|
-
"description": "Crosstalk — agent-agnostic swarm communication protocol
|
|
3
|
+
"version": "8.0.0-alpha.1",
|
|
4
|
+
"description": "Crosstalk — agent-agnostic swarm communication protocol over a shared directory (git or filesystem transport). Operator CLI + daemon in one binary.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"homepage": "https://github.com/cordfuse/crosstalk",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"scripts": {
|
|
27
27
|
"build": "tsc --noEmit",
|
|
28
28
|
"lint": "tsc --noEmit",
|
|
29
|
-
"test": "
|
|
29
|
+
"test": "node --import tsx --test tests/*.test.ts",
|
|
30
30
|
"prepack": "cp -r transport template",
|
|
31
31
|
"postpack": "rm -rf template"
|
|
32
32
|
},
|
package/src/api.ts
CHANGED
|
@@ -33,8 +33,8 @@ import { now, messageFilename } from './filenames.js';
|
|
|
33
33
|
import { parseFrontmatter, serializeFrontmatter } from './frontmatter.js';
|
|
34
34
|
import {
|
|
35
35
|
discoverChannels,
|
|
36
|
-
gitCommitAndPush,
|
|
37
36
|
listChannelMessages,
|
|
37
|
+
type Transport,
|
|
38
38
|
} from './transport.js';
|
|
39
39
|
import { sendWakeSignal, readHeartbeat, readCursor, countErrors } from './state.js';
|
|
40
40
|
import { reList } from './activation.js';
|
|
@@ -66,6 +66,9 @@ export interface ApiContext {
|
|
|
66
66
|
version: string;
|
|
67
67
|
registry: ModelsRegistry;
|
|
68
68
|
claimed: Map<string, ModelEntry>; // shorthand for registry.claimed
|
|
69
|
+
/** v8: the bus medium (git default, or local-fs). Web-UI writes publish
|
|
70
|
+
* through this so a local-fs deployment never shells out to git. */
|
|
71
|
+
transport: Transport;
|
|
69
72
|
/** Auth stores; v8-native. Optional — engine still runs without auth
|
|
70
73
|
* if the operator doesn't want it (just the web UI gating turns off). */
|
|
71
74
|
userStore?: UserStore;
|
|
@@ -205,6 +208,7 @@ function handleStatus(ctx: ApiContext): JsonRes {
|
|
|
205
208
|
status: 200,
|
|
206
209
|
body: {
|
|
207
210
|
transport_root: transportRoot,
|
|
211
|
+
transport_kind: ctx.transport.kind,
|
|
208
212
|
alias: ctx.alias,
|
|
209
213
|
version: ctx.version,
|
|
210
214
|
claimed_models: [...ctx.claimed.keys()],
|
|
@@ -281,7 +285,7 @@ function handleCreateChannel(ctx: ApiContext, body: CreateChannelReq): JsonRes {
|
|
|
281
285
|
const dir = join(transportRoot, 'data', 'channels', uuid);
|
|
282
286
|
mkdirSync(dir, { recursive: true });
|
|
283
287
|
writeFileSync(join(dir, 'CHANNEL.md'), serializeFrontmatter({ name: body.name }, ''));
|
|
284
|
-
const push =
|
|
288
|
+
const push = ctx.transport.publish(`channel(create): ${body.name} (${uuid.slice(0, 8)})`);
|
|
285
289
|
if (!push.committed && push.error) {
|
|
286
290
|
// Commit itself failed (e.g. file system / git config issue).
|
|
287
291
|
throw new HttpError(500, `commit failed: ${push.error.slice(0, 300)}`);
|
|
@@ -316,8 +320,7 @@ function handleRenameChannel(ctx: ApiContext, handle: string, body: RenameChanne
|
|
|
316
320
|
const fm: Record<string, unknown> = { name: body.name };
|
|
317
321
|
if (meta.parent) fm['parent'] = meta.parent;
|
|
318
322
|
writeFileSync(chPath, serializeFrontmatter(fm, ''));
|
|
319
|
-
const push =
|
|
320
|
-
transportRoot,
|
|
323
|
+
const push = ctx.transport.publish(
|
|
321
324
|
`channel(rename): ${meta.name ?? '(unnamed)'} -> ${body.name} (${meta.uuid.slice(0, 8)})`,
|
|
322
325
|
);
|
|
323
326
|
if (!push.committed && push.error) {
|
|
@@ -348,8 +351,7 @@ function handleDeleteChannel(ctx: ApiContext, handle: string, confirm: string |
|
|
|
348
351
|
}
|
|
349
352
|
const dir = join(transportRoot, 'data', 'channels', meta.uuid);
|
|
350
353
|
rmSync(dir, { recursive: true, force: true });
|
|
351
|
-
const push =
|
|
352
|
-
transportRoot,
|
|
354
|
+
const push = ctx.transport.publish(
|
|
353
355
|
`channel(delete): ${meta.name ?? '(unnamed)'} (${meta.uuid.slice(0, 8)})`,
|
|
354
356
|
);
|
|
355
357
|
if (!push.committed && push.error) {
|
|
@@ -458,7 +460,7 @@ function handlePostMessage(ctx: ApiContext, body: PostMessageReq): JsonRes {
|
|
|
458
460
|
fanout === 1
|
|
459
461
|
? `run: ${from} -> ${toQualified} in ${channelUuid.slice(0, 8)}`
|
|
460
462
|
: `run: ${from} -> ${toQualified} x${fanout} in ${channelUuid.slice(0, 8)}`;
|
|
461
|
-
const push =
|
|
463
|
+
const push = ctx.transport.publish(commitMsg);
|
|
462
464
|
sendWakeSignal(transportRoot);
|
|
463
465
|
if (!push.committed && push.error) {
|
|
464
466
|
throw new HttpError(500, `commit failed: ${push.error.slice(0, 300)}`);
|
|
@@ -494,8 +496,7 @@ function handlePostMessage(ctx: ApiContext, body: PostMessageReq): JsonRes {
|
|
|
494
496
|
extraFrontmatter,
|
|
495
497
|
workflow: true,
|
|
496
498
|
});
|
|
497
|
-
const push =
|
|
498
|
-
transportRoot,
|
|
499
|
+
const push = ctx.transport.publish(
|
|
499
500
|
`run(workflow): ${from} child ${childUuid.slice(0, 8)}`,
|
|
500
501
|
);
|
|
501
502
|
sendWakeSignal(transportRoot);
|
|
@@ -1007,8 +1008,7 @@ async function tryWebApiRoute(ctx: ApiContext, req: IncomingMessage, res: Server
|
|
|
1007
1008
|
extraFrontmatter,
|
|
1008
1009
|
workflow: true,
|
|
1009
1010
|
});
|
|
1010
|
-
const push =
|
|
1011
|
-
ctx.transportRoot,
|
|
1011
|
+
const push = ctx.transport.publish(
|
|
1012
1012
|
`run(workflow): ${me.username} child ${childUuid.slice(0, 8)} (composed)`,
|
|
1013
1013
|
);
|
|
1014
1014
|
sendWakeSignal(ctx.transportRoot);
|
|
@@ -1088,6 +1088,7 @@ async function tryHtmlRoute(ctx: ApiContext, req: IncomingMessage, res: ServerRe
|
|
|
1088
1088
|
const channels = allChannelMeta(ctx.transportRoot);
|
|
1089
1089
|
writeHtml(res, 200, dashboardPage({
|
|
1090
1090
|
host, alias: ctx.alias, version: ctx.version, transportRoot: ctx.transportRoot,
|
|
1091
|
+
transportKind: ctx.transport.kind,
|
|
1091
1092
|
heartbeat: hb ? { ts: hb.ts, pid: hb.pid, version: hb.version, alias: hb.alias } : null,
|
|
1092
1093
|
cursor: cursor ?? null,
|
|
1093
1094
|
claimedModels: [...ctx.claimed.keys()],
|
package/src/dispatch.ts
CHANGED
|
@@ -20,16 +20,8 @@ import { readFileSync, existsSync, appendFileSync } from 'fs';
|
|
|
20
20
|
import { watch } from 'fs/promises';
|
|
21
21
|
import { fileURLToPath } from 'url';
|
|
22
22
|
|
|
23
|
-
import { loadRegistry, type ModelEntry, type ModelsRegistry } from './models.js';
|
|
24
|
-
import {
|
|
25
|
-
discoverChannels,
|
|
26
|
-
listChannelMessages,
|
|
27
|
-
gitPull,
|
|
28
|
-
gitCommitAndPush,
|
|
29
|
-
cursorBaseline,
|
|
30
|
-
newFilesSince,
|
|
31
|
-
type ChannelMessage,
|
|
32
|
-
} from './transport.js';
|
|
23
|
+
import { loadRegistry, loadTransportKind, type ModelEntry, type ModelsRegistry } from './models.js';
|
|
24
|
+
import { GitTransport, LocalFsTransport, type Transport, type ChannelMessage } from './transport.js';
|
|
33
25
|
import {
|
|
34
26
|
stateDir,
|
|
35
27
|
readCursor,
|
|
@@ -72,7 +64,10 @@ const RUNTIME_VERSION: string = (() => {
|
|
|
72
64
|
}
|
|
73
65
|
})();
|
|
74
66
|
|
|
75
|
-
|
|
67
|
+
// CROSSTALK_TRANSPORT_PATH points the engine at a transport directory
|
|
68
|
+
// directly (defaults to cwd) — pairs with CROSSTALK_TRANSPORT_TYPE to run the
|
|
69
|
+
// bus on a given directory (e.g. a mounted share) without a cd.
|
|
70
|
+
const transportRoot = resolve(process.env['CROSSTALK_TRANSPORT_PATH'] ?? process.cwd());
|
|
76
71
|
const argv = process.argv.slice(2);
|
|
77
72
|
|
|
78
73
|
function flag(name: string): string | undefined {
|
|
@@ -87,6 +82,21 @@ const jsonMode = argv.includes('--json');
|
|
|
87
82
|
const pollSeconds = Number(flag('--poll')) || 30;
|
|
88
83
|
const logFile = flag('--log-file');
|
|
89
84
|
|
|
85
|
+
// v8: the dispatch loop drives the bus through a Transport, not git calls.
|
|
86
|
+
// Selection precedence: --transport flag > CROSSTALK_TRANSPORT_TYPE env >
|
|
87
|
+
// `transport:` in crosstalk.yaml > git default. local = plain-fs, incl. a
|
|
88
|
+
// mounted SMB/NFS share. The fs transport drops in with zero changes below.
|
|
89
|
+
const transportOverride = flag('--transport') ?? process.env['CROSSTALK_TRANSPORT_TYPE'];
|
|
90
|
+
if (transportOverride !== undefined && transportOverride !== 'git' && transportOverride !== 'local') {
|
|
91
|
+
console.error(`crosstalk daemon dispatch: unknown transport '${transportOverride}' (expected git|local); falling back to config/default.`);
|
|
92
|
+
}
|
|
93
|
+
const transportKind = (transportOverride === 'git' || transportOverride === 'local')
|
|
94
|
+
? transportOverride
|
|
95
|
+
: loadTransportKind(transportRoot);
|
|
96
|
+
const transport: Transport = transportKind === 'local'
|
|
97
|
+
? new LocalFsTransport(transportRoot)
|
|
98
|
+
: new GitTransport(transportRoot);
|
|
99
|
+
|
|
90
100
|
if (!alias) {
|
|
91
101
|
console.error('crosstalk daemon dispatch: --alias <name> is required (machine identity in the bus).');
|
|
92
102
|
process.exit(1);
|
|
@@ -208,14 +218,14 @@ async function dispatchTick(registry: ModelsRegistry, protocolPrompt: string): P
|
|
|
208
218
|
const claimed = registry.claimed;
|
|
209
219
|
writeHeartbeat(transportRoot, RUNTIME_VERSION, alias!);
|
|
210
220
|
|
|
211
|
-
const pullResult =
|
|
221
|
+
const pullResult = transport.sync();
|
|
212
222
|
if (!pullResult.ok) {
|
|
213
223
|
logError(transportRoot, `git pull failed: ${pullResult.error}`, 'warn');
|
|
214
224
|
log('git_pull_failed', { error: (pullResult.error ?? '').slice(0, 200) });
|
|
215
225
|
return false;
|
|
216
226
|
}
|
|
217
227
|
|
|
218
|
-
const head =
|
|
228
|
+
const head = transport.head();
|
|
219
229
|
if (!head) {
|
|
220
230
|
logError(transportRoot, 'git rev-parse failed for origin/HEAD and HEAD — skipping tick');
|
|
221
231
|
return false;
|
|
@@ -231,7 +241,7 @@ async function dispatchTick(registry: ModelsRegistry, protocolPrompt: string): P
|
|
|
231
241
|
cursor = head;
|
|
232
242
|
log('cursor_seeded', { commit: head.slice(0, 12) });
|
|
233
243
|
}
|
|
234
|
-
const channels = discoverChannels(
|
|
244
|
+
const channels = transport.discoverChannels();
|
|
235
245
|
const activationDue = cursor !== head;
|
|
236
246
|
|
|
237
247
|
let didWork = false;
|
|
@@ -252,7 +262,7 @@ async function dispatchTick(registry: ModelsRegistry, protocolPrompt: string): P
|
|
|
252
262
|
|
|
253
263
|
const writeNeeded = didWork || workflowProgressed;
|
|
254
264
|
if (writeNeeded) {
|
|
255
|
-
const push =
|
|
265
|
+
const push = transport.publish(`dispatch(${alias}): replies ${new Date().toISOString()}`);
|
|
256
266
|
if (!push.ok && push.error) {
|
|
257
267
|
// Commit failed is a real error (local state out of sync); push
|
|
258
268
|
// failed with the commit landing is a warning (push will retry).
|
|
@@ -271,17 +281,15 @@ async function runActivationPass(
|
|
|
271
281
|
cursor: string,
|
|
272
282
|
head: string,
|
|
273
283
|
): Promise<boolean> {
|
|
274
|
-
const addedList = newFilesSince(
|
|
284
|
+
const addedList = transport.newFilesSince(cursor, head);
|
|
275
285
|
if (addedList === null) {
|
|
276
286
|
logError(transportRoot, `cursor ${cursor.slice(0, 12)} unknown — re-scanning all channels`, 'warn');
|
|
277
287
|
}
|
|
278
288
|
const added: Set<string> | null = addedList === null ? null : new Set(addedList);
|
|
279
|
-
|
|
280
|
-
void head; // head is the diff's upper bound implicitly (newFilesSince uses HEAD).
|
|
281
289
|
const pending: PendingDispatch[] = [];
|
|
282
290
|
|
|
283
291
|
for (const channelUuid of channels) {
|
|
284
|
-
const messages = listChannelMessages(
|
|
292
|
+
const messages = transport.listChannelMessages(channelUuid);
|
|
285
293
|
const senderByRelPath = new Map(messages.map((m) => [m.relPath, messageSender(m)]));
|
|
286
294
|
// Normalize asker identity the same way self-suppression does: strip
|
|
287
295
|
// the @machine suffix before comparing against actorName. Without
|
|
@@ -336,18 +344,46 @@ async function runActivationPass(
|
|
|
336
344
|
return results.some(Boolean);
|
|
337
345
|
}
|
|
338
346
|
|
|
347
|
+
// Block until something worth a tick happens, or `ms` elapses (the poll
|
|
348
|
+
// fallback). Two wake sources race the timeout:
|
|
349
|
+
// • the state dir's `wake.signal` — written by sendWakeSignal on a
|
|
350
|
+
// same-host API post (channel/message create).
|
|
351
|
+
// • the transport's inbound dir (local-fs: data/channels, watched
|
|
352
|
+
// recursively) — zero-lag pickup when a writer drops a message straight
|
|
353
|
+
// onto the bus (a peer on a shared mount, a file drop). git returns null
|
|
354
|
+
// (a remote isn't watchable); fs.watch also won't fire over SMB/NFS — so
|
|
355
|
+
// the poll timeout stays the always-on backstop.
|
|
339
356
|
async function waitForWakeOrTimeout(ms: number): Promise<void> {
|
|
340
|
-
const dir = stateDir(transportRoot);
|
|
341
357
|
const ac = new AbortController();
|
|
342
358
|
const timer = setTimeout(() => ac.abort(), ms);
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
359
|
+
|
|
360
|
+
const watchFor = async (dir: string, recursive: boolean): Promise<void> => {
|
|
361
|
+
try {
|
|
362
|
+
const watcher = watch(dir, { signal: ac.signal, recursive });
|
|
363
|
+
for await (const ev of watcher) {
|
|
364
|
+
// state dir: only wake.signal. bus dir: any new .md is a candidate
|
|
365
|
+
// message — wake and let the tick's cursor decide if it's new.
|
|
366
|
+
if (recursive ? (ev.filename?.endsWith('.md') ?? false) : ev.filename === 'wake.signal') return;
|
|
367
|
+
}
|
|
368
|
+
} catch {
|
|
369
|
+
if (ac.signal.aborted) return; // timeout — expected
|
|
370
|
+
// watch unavailable (recursive unsupported, network FS, missing dir):
|
|
371
|
+
// don't spin — wait out the poll interval via the abort signal.
|
|
372
|
+
await new Promise<void>((res) => {
|
|
373
|
+
if (ac.signal.aborted) res();
|
|
374
|
+
else ac.signal.addEventListener('abort', () => res(), { once: true });
|
|
375
|
+
});
|
|
347
376
|
}
|
|
348
|
-
}
|
|
349
|
-
|
|
377
|
+
};
|
|
378
|
+
|
|
379
|
+
const waits: Promise<void>[] = [watchFor(stateDir(transportRoot), false)];
|
|
380
|
+
const busDir = transport.watchDir();
|
|
381
|
+
if (busDir && existsSync(busDir)) waits.push(watchFor(busDir, true));
|
|
382
|
+
|
|
383
|
+
try {
|
|
384
|
+
await Promise.race(waits);
|
|
350
385
|
} finally {
|
|
386
|
+
ac.abort();
|
|
351
387
|
clearTimeout(timer);
|
|
352
388
|
}
|
|
353
389
|
}
|
|
@@ -371,7 +407,7 @@ async function main(): Promise<void> {
|
|
|
371
407
|
}
|
|
372
408
|
removePidfile(transportRoot);
|
|
373
409
|
if (alias) removeRegistryEntry(transportRoot, alias);
|
|
374
|
-
const push =
|
|
410
|
+
const push = transport.publish(`dispatch(${alias}): deregister entry`);
|
|
375
411
|
if (!push.ok && push.error) {
|
|
376
412
|
logError(transportRoot, `registry deregister failed: ${push.error}`, 'warn');
|
|
377
413
|
}
|
|
@@ -414,13 +450,14 @@ async function main(): Promise<void> {
|
|
|
414
450
|
// races to duplicate. Commit+push immediately so cross-machine
|
|
415
451
|
// visibility doesn't wait for first reply traffic.
|
|
416
452
|
writeRegistryEntry(transportRoot, alias!, [...registry.claimed.keys()], RUNTIME_VERSION);
|
|
417
|
-
const registryPush =
|
|
453
|
+
const registryPush = transport.publish(`dispatch(${alias}): register entry`);
|
|
418
454
|
if (!registryPush.ok && registryPush.error) {
|
|
419
455
|
logError(transportRoot, `registry publish failed: ${registryPush.error}`, 'warn');
|
|
420
456
|
}
|
|
421
457
|
|
|
422
458
|
log('dispatch_start', {
|
|
423
459
|
transport: transportRoot,
|
|
460
|
+
transport_kind: transport.kind,
|
|
424
461
|
alias,
|
|
425
462
|
version: RUNTIME_VERSION,
|
|
426
463
|
state_dir: stateDir(transportRoot),
|
|
@@ -454,6 +491,7 @@ async function main(): Promise<void> {
|
|
|
454
491
|
apiServer = startApi(
|
|
455
492
|
{
|
|
456
493
|
transportRoot,
|
|
494
|
+
transport,
|
|
457
495
|
alias: alias!,
|
|
458
496
|
version: RUNTIME_VERSION,
|
|
459
497
|
registry,
|
package/src/models.ts
CHANGED
|
@@ -102,6 +102,23 @@ function parseInlineEnv(
|
|
|
102
102
|
return Object.keys(out).length > 0 ? out : null;
|
|
103
103
|
}
|
|
104
104
|
|
|
105
|
+
export type TransportKind = 'git' | 'local';
|
|
106
|
+
|
|
107
|
+
// The optional top-level `transport:` key in data/crosstalk.yaml selects the
|
|
108
|
+
// bus medium. 'local' = the filesystem transport (a plain directory, incl. a
|
|
109
|
+
// mounted SMB/NFS share); absent / anything else / unreadable → 'git'. A
|
|
110
|
+
// --transport flag at dispatch boot overrides this.
|
|
111
|
+
export function loadTransportKind(transportRoot: string): TransportKind {
|
|
112
|
+
const path = crosstalkYamlPath(transportRoot);
|
|
113
|
+
if (!existsSync(path)) return 'git';
|
|
114
|
+
try {
|
|
115
|
+
const parsed = parseYaml(readFileSync(path, 'utf8')) as Record<string, unknown> | null;
|
|
116
|
+
return parsed && parsed['transport'] === 'local' ? 'local' : 'git';
|
|
117
|
+
} catch {
|
|
118
|
+
return 'git';
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
|
|
105
122
|
export function readModels(transportRoot: string): {
|
|
106
123
|
all: Map<string, ModelEntry>;
|
|
107
124
|
byBareName: Map<string, ModelEntry[]>;
|
package/src/state.ts
CHANGED
|
@@ -44,18 +44,23 @@ export function stateDir(transportRoot: string): string {
|
|
|
44
44
|
|
|
45
45
|
// ── cursor (single, machine-global) ──
|
|
46
46
|
//
|
|
47
|
-
//
|
|
48
|
-
//
|
|
49
|
-
//
|
|
50
|
-
//
|
|
51
|
-
//
|
|
47
|
+
// The cursor is an opaque, transport-OWNED token marking where this machine
|
|
48
|
+
// last scanned. Storage here is format-agnostic: any non-empty string round-
|
|
49
|
+
// trips, and each transport validates + interprets its own token. An
|
|
50
|
+
// unrecognized token (e.g. after switching transports) makes the transport's
|
|
51
|
+
// newFilesSince return null, and the loop full-scans to recover.
|
|
52
|
+
//
|
|
53
|
+
// Git's token is a commit SHA, and that choice is deliberate: filenames order
|
|
54
|
+
// by sender timestamp, but messages reach origin in PUSH order — a message
|
|
55
|
+
// that loses a push race can land with a timestamp earlier than one already
|
|
56
|
+
// processed, and a relPath/time cursor would skip it forever. Commit cursors
|
|
57
|
+
// can't. The local-fs transport uses an mtime watermark instead, which is
|
|
58
|
+
// safe ONLY because a single local machine has one clock and no push races.
|
|
52
59
|
//
|
|
53
60
|
// v6 kept one cursor per (actor × channel). v7 collapses to a single
|
|
54
61
|
// machine-global cursor: a dispatcher processes everything new since the
|
|
55
62
|
// cursor in one tick, regardless of channel or claimed model.
|
|
56
63
|
|
|
57
|
-
const VALID_CURSOR = /^[0-9a-f]{40}$/;
|
|
58
|
-
|
|
59
64
|
export function cursorPath(transportRoot: string): string {
|
|
60
65
|
return join(stateDir(transportRoot), 'cursor');
|
|
61
66
|
}
|
|
@@ -71,15 +76,11 @@ export function readCursor(transportRoot: string): string | null {
|
|
|
71
76
|
return null;
|
|
72
77
|
}
|
|
73
78
|
if (raw.length === 0) return null;
|
|
74
|
-
if (!VALID_CURSOR.test(raw)) {
|
|
75
|
-
logError(transportRoot, `invalid cursor '${raw.slice(0, 80)}' — re-scanning from origin`, 'warn');
|
|
76
|
-
return null;
|
|
77
|
-
}
|
|
78
79
|
return raw;
|
|
79
80
|
}
|
|
80
81
|
|
|
81
|
-
export function writeCursor(transportRoot: string,
|
|
82
|
-
writeFileSync(cursorPath(transportRoot),
|
|
82
|
+
export function writeCursor(transportRoot: string, token: string): void {
|
|
83
|
+
writeFileSync(cursorPath(transportRoot), token + '\n');
|
|
83
84
|
}
|
|
84
85
|
|
|
85
86
|
// ── pidfile ──
|
package/src/transport.ts
CHANGED
|
@@ -1,4 +1,6 @@
|
|
|
1
|
-
//
|
|
1
|
+
// Transport layer. The git transport functions are here; the Transport
|
|
2
|
+
// interface + GitTransport/LocalFsTransport live at the foot of this file.
|
|
3
|
+
// The git dispatcher's commits contain ONLY data/ —
|
|
2
4
|
// machine-local state lives in the state dir (state.ts), so there is
|
|
3
5
|
// nothing to exclude, untrack, or heal. Push rejection means another
|
|
4
6
|
// machine won the race: pull --rebase and retry at the call site.
|
|
@@ -78,9 +80,9 @@ export function cursorBaseline(transportRoot: string): string | null {
|
|
|
78
80
|
// HEAD. Returns null when the commit is unknown to this clone (state dir
|
|
79
81
|
// copied across transports, history rewritten) — caller falls back to a
|
|
80
82
|
// full channel scan.
|
|
81
|
-
export function newFilesSince(transportRoot: string, sinceCommit: string): string[] | null {
|
|
83
|
+
export function newFilesSince(transportRoot: string, sinceCommit: string, head: string): string[] | null {
|
|
82
84
|
const r = captureGit(transportRoot, [
|
|
83
|
-
'diff', '--name-only', '--diff-filter=A', `${sinceCommit}
|
|
85
|
+
'diff', '--name-only', '--diff-filter=A', `${sinceCommit}..${head}`, '--', 'data/channels/',
|
|
84
86
|
]);
|
|
85
87
|
if (r.status !== 0) return null;
|
|
86
88
|
return r.stdout.split('\n').filter(Boolean);
|
|
@@ -241,3 +243,111 @@ export function listChannelMessages(transportRoot: string, channelUuid: string):
|
|
|
241
243
|
walk(channelDir, '');
|
|
242
244
|
return results.sort((a, b) => a.relPath.localeCompare(b.relPath));
|
|
243
245
|
}
|
|
246
|
+
|
|
247
|
+
// ── Transport abstraction (v8) ──
|
|
248
|
+
//
|
|
249
|
+
// The dispatch loop talks to a medium through this interface, never to git
|
|
250
|
+
// directly. The CURSOR is an opaque, transport-owned token: dispatch.ts
|
|
251
|
+
// persists whatever string a transport hands back and round-trips it to
|
|
252
|
+
// `newFilesSince` next tick without interpreting it. Git's token is a
|
|
253
|
+
// 40-hex commit SHA (see state.ts on why commit cursors, not relPaths);
|
|
254
|
+
// a future fs transport is free to use a different token shape that is
|
|
255
|
+
// correct for its medium. `head()` returns the upper bound for this tick;
|
|
256
|
+
// `newFilesSince(cursor)` returns repo-relative message paths added in
|
|
257
|
+
// (cursor, head], or null when the cursor is unrecognizable (caller then
|
|
258
|
+
// full-scans). `sync()` pulls remote state in; `publish()` flushes locally
|
|
259
|
+
// written replies out. Message reads (discoverChannels/listChannelMessages)
|
|
260
|
+
// are already plain-fs and identical across every transport.
|
|
261
|
+
export interface Transport {
|
|
262
|
+
readonly kind: string;
|
|
263
|
+
sync(): GitResult;
|
|
264
|
+
head(): string | null;
|
|
265
|
+
newFilesSince(cursor: string, head: string): string[] | null;
|
|
266
|
+
publish(message: string): GitPushResult;
|
|
267
|
+
discoverChannels(): string[];
|
|
268
|
+
listChannelMessages(channelUuid: string): ChannelMessage[];
|
|
269
|
+
/** A directory to fs.watch for inbound messages (zero-lag wake), or null
|
|
270
|
+
* if the medium can't be watched (git: a remote isn't watchable — the
|
|
271
|
+
* poll loop covers it). fs.watch also won't fire over SMB/NFS, so the
|
|
272
|
+
* poll stays the backstop even when this is non-null. */
|
|
273
|
+
watchDir(): string | null;
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
// Git transport — the original and, until the fs transport lands, the only
|
|
277
|
+
// implementation. A thin pass-through to the module functions above; it
|
|
278
|
+
// adds no behavior, it only moves them behind the interface so dispatch.ts
|
|
279
|
+
// stops importing git operations by name.
|
|
280
|
+
export class GitTransport implements Transport {
|
|
281
|
+
readonly kind = 'git';
|
|
282
|
+
constructor(private readonly root: string) {}
|
|
283
|
+
sync(): GitResult { return gitPull(this.root); }
|
|
284
|
+
head(): string | null { return cursorBaseline(this.root); }
|
|
285
|
+
newFilesSince(cursor: string, head: string): string[] | null { return newFilesSince(this.root, cursor, head); }
|
|
286
|
+
publish(message: string): GitPushResult { return gitCommitAndPush(this.root, message); }
|
|
287
|
+
discoverChannels(): string[] { return discoverChannels(this.root); }
|
|
288
|
+
listChannelMessages(channelUuid: string): ChannelMessage[] { return listChannelMessages(this.root, channelUuid); }
|
|
289
|
+
watchDir(): string | null { return null; } // a git remote isn't fs.watchable — poll covers inbound
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
// Local filesystem transport — single machine, no remote, no git. The bus is
|
|
293
|
+
// just a directory tree (it can sit on an SMB/NFS mount; from here it's all
|
|
294
|
+
// the same plain fs). Mount this on /mnt/share and two daemons on the same
|
|
295
|
+
// host, or on hosts sharing the mount, see each other's messages with zero
|
|
296
|
+
// infrastructure.
|
|
297
|
+
//
|
|
298
|
+
// sync()/publish() are no-ops: there's no remote to pull from, and replies
|
|
299
|
+
// are written straight to disk by writeReply — the write IS the publish.
|
|
300
|
+
//
|
|
301
|
+
// The cursor is an epoch-millis mtime watermark. head() is "now" at tick
|
|
302
|
+
// start; newFilesSince returns message files with mtime in (cursor, head].
|
|
303
|
+
// mtime is a sound ordering key HERE and ONLY here: a single local machine
|
|
304
|
+
// has one monotonic clock and no push-race reordering — the exact failure
|
|
305
|
+
// mode state.ts forbids relPath/time cursors over for git/remote media. Do
|
|
306
|
+
// NOT lift this watermark onto SMB/NFS spanning multiple writers without
|
|
307
|
+
// revisiting that guarantee.
|
|
308
|
+
export class LocalFsTransport implements Transport {
|
|
309
|
+
readonly kind = 'local';
|
|
310
|
+
constructor(private readonly root: string) {}
|
|
311
|
+
|
|
312
|
+
sync(): GitResult { return { ok: true }; }
|
|
313
|
+
publish(_message: string): GitPushResult { return { ok: true, committed: true, pushed: false }; }
|
|
314
|
+
head(): string | null { return String(Date.now()); }
|
|
315
|
+
discoverChannels(): string[] { return discoverChannels(this.root); }
|
|
316
|
+
listChannelMessages(channelUuid: string): ChannelMessage[] { return listChannelMessages(this.root, channelUuid); }
|
|
317
|
+
// Watch the channels tree for inbound message drops — event-driven, zero
|
|
318
|
+
// lag, the v1.1.0 "real-time local" property. Fires only on a true local
|
|
319
|
+
// FS; a share-mounted dir falls back to the dispatch loop's poll.
|
|
320
|
+
watchDir(): string | null { return join(this.root, 'data', 'channels'); }
|
|
321
|
+
|
|
322
|
+
newFilesSince(cursor: string, head: string): string[] | null {
|
|
323
|
+
const since = Number(cursor);
|
|
324
|
+
const until = Number(head);
|
|
325
|
+
// Unrecognized token (e.g. a git SHA left over from a transport switch)
|
|
326
|
+
// → null tells the loop to full-scan, after which we write our own.
|
|
327
|
+
if (!Number.isFinite(since) || !Number.isFinite(until)) return null;
|
|
328
|
+
const channelsDir = join(this.root, 'data', 'channels');
|
|
329
|
+
if (!existsSync(channelsDir)) return [];
|
|
330
|
+
const out: string[] = [];
|
|
331
|
+
for (const channelUuid of this.discoverChannels()) {
|
|
332
|
+
const walk = (dir: string, prefix: string): void => {
|
|
333
|
+
let entries: string[];
|
|
334
|
+
try { entries = readdirSync(dir); } catch { return; }
|
|
335
|
+
for (const entry of entries) {
|
|
336
|
+
const full = join(dir, entry);
|
|
337
|
+
let stat;
|
|
338
|
+
try { stat = statSync(full); } catch { continue; }
|
|
339
|
+
if (stat.isDirectory()) {
|
|
340
|
+
walk(full, prefix ? `${prefix}/${entry}` : entry);
|
|
341
|
+
} else if (entry.endsWith('.md') && entry !== 'CHANNEL.md') {
|
|
342
|
+
if (stat.mtimeMs > since && stat.mtimeMs <= until) {
|
|
343
|
+
const rel = prefix ? `${prefix}/${entry}` : entry;
|
|
344
|
+
out.push(`data/channels/${channelUuid}/${rel}`);
|
|
345
|
+
}
|
|
346
|
+
}
|
|
347
|
+
}
|
|
348
|
+
};
|
|
349
|
+
walk(join(channelsDir, channelUuid), '');
|
|
350
|
+
}
|
|
351
|
+
return out;
|
|
352
|
+
}
|
|
353
|
+
}
|
package/src/web/dashboard.ts
CHANGED
|
@@ -9,6 +9,7 @@ export interface DashboardData {
|
|
|
9
9
|
alias: string;
|
|
10
10
|
version: string;
|
|
11
11
|
transportRoot: string;
|
|
12
|
+
transportKind: string;
|
|
12
13
|
heartbeat: { ts: string; pid: number; version: string; alias: string | undefined } | null;
|
|
13
14
|
cursor: string | null;
|
|
14
15
|
claimedModels: string[];
|
|
@@ -34,7 +35,7 @@ export function dashboardPage(d: DashboardData): string {
|
|
|
34
35
|
<h3>Engine</h3>
|
|
35
36
|
<div class="kv"><span class="key">alias</span><span class="val brand">${escapeHtml(d.alias)}</span></div>
|
|
36
37
|
<div class="kv"><span class="key">version</span><span class="val">${escapeHtml(d.version)}</span></div>
|
|
37
|
-
<div class="kv"><span class="key">transport</span><span class="val dim"
|
|
38
|
+
<div class="kv"><span class="key">transport</span><span class="val">${escapeHtml(d.transportKind)}<span class="dim"> · ${escapeHtml(d.transportRoot)}</span></span></div>
|
|
38
39
|
<div class="kv"><span class="key">cursor</span><span class="val">${escapeHtml(cursorShort)}</span></div>
|
|
39
40
|
</section>
|
|
40
41
|
|
package/template/CLAUDE.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Crosstalk transport
|
|
2
2
|
|
|
3
|
-
You are inside a Crosstalk transport — a git repo that carries async messages between AI agents and humans across machines.
|
|
3
|
+
You are inside a Crosstalk transport — a directory (a git repo, or a plain/shared filesystem) that carries async messages between AI agents and humans across machines.
|
|
4
4
|
|
|
5
5
|
Read these before doing anything here:
|
|
6
6
|
|
package/template/CROSSTALK.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Version: 7
|
|
4
4
|
|
|
5
|
-
Crosstalk is a shared file format
|
|
5
|
+
Crosstalk is a shared file format that lets humans and AI agents communicate asynchronously across machines. **A directory is the message bus** — carried by git (the default: distributed, versioned, auditable) or a plain/shared filesystem (`transport: local`, including a mounted SMB/NFS share, no git required).
|
|
6
6
|
|
|
7
7
|
Design rule for this spec: every feature must be explainable in one sentence. The runtime records facts at write time; it never reconstructs them by inference at read time.
|
|
8
8
|
|
|
@@ -14,10 +14,10 @@ Every concept in Crosstalk maps to exactly one of these:
|
|
|
14
14
|
|
|
15
15
|
| Noun | Definition |
|
|
16
16
|
|---|---|
|
|
17
|
-
| **Transport** |
|
|
17
|
+
| **Transport** | The bus — a directory scaffolded by `crosstalk transport init`. A git repo (default) or a plain/shared filesystem (`transport: local`). |
|
|
18
18
|
| **Machine** | A host running the crosstalk engine (`crosstalk daemon dispatch`, spawned by `crosstalk server start` or systemd). Identifies itself via `CROSSTALK_ALIAS` (defaults to the transport name). |
|
|
19
|
-
| **Message** | A markdown file with YAML frontmatter,
|
|
20
|
-
| **Model** | A named agent invocation declared in `data/
|
|
19
|
+
| **Message** | A markdown file with YAML frontmatter, written to a channel. The unit of work. |
|
|
20
|
+
| **Model** | A named agent invocation declared in `data/crosstalk.yaml` (e.g. `sonnet`, `codex-o3`). |
|
|
21
21
|
| **Actor** | An optional persona file (`local/actors/<name>.md`) that prepends to a model's prompt. |
|
|
22
22
|
| **Channel** | A UUID directory under `data/channels/`. A conversation thread. Optionally parented. |
|
|
23
23
|
|
|
@@ -31,7 +31,7 @@ Every concept in Crosstalk maps to exactly one of these:
|
|
|
31
31
|
PROTOCOL.md # agent orientation, prepended to every dispatch
|
|
32
32
|
CROSSTALK.md # this file
|
|
33
33
|
data/
|
|
34
|
-
|
|
34
|
+
crosstalk.yaml # providers + models (+ optional transport: key)
|
|
35
35
|
channels/<uuid>/
|
|
36
36
|
CHANNEL.md # { name, optional parent }
|
|
37
37
|
YYYY/MM/DD/<msg>.md # messages
|
|
@@ -42,7 +42,7 @@ Every concept in Crosstalk maps to exactly one of these:
|
|
|
42
42
|
|
|
43
43
|
That is the complete committed surface. **Dispatcher bookkeeping (cursor, heartbeat, pidfile, wake signal) is machine-local state and never lives in the repo** — it is kept under `~/.config/crosstalk/state/<transport-name>/`. The repo carries conversation; each machine carries its own progress through it.
|
|
44
44
|
|
|
45
|
-
There is no `hosts/` directory and no `upstream/` directory at all — v6's `upstream/`-vs-`local/` override hierarchy is gone. `local/` still exists as the home for actor persona files (`local/actors/<name>.md`). The model vocabulary in `data/
|
|
45
|
+
There is no `hosts/` directory and no `upstream/` directory at all — v6's `upstream/`-vs-`local/` override hierarchy is gone. `local/` still exists as the home for actor persona files (`local/actors/<name>.md`). The model vocabulary in `data/crosstalk.yaml` is shared by everyone; each dispatcher claims the entries whose CLI is on its PATH.
|
|
46
46
|
|
|
47
47
|
---
|
|
48
48
|
|
|
@@ -222,7 +222,7 @@ The model never sees state machinery: each invocation is a single-turn content t
|
|
|
222
222
|
|
|
223
223
|
The `to:` field accepts:
|
|
224
224
|
|
|
225
|
-
- `to: sonnet` — bare model name. Any dispatcher claiming `sonnet` may pick it up; first-grab wins via git push race.
|
|
225
|
+
- `to: sonnet` — bare model name. Any dispatcher claiming `sonnet` may pick it up; first-grab wins (via git push race on the git transport, via collision-free filenames on the filesystem transport).
|
|
226
226
|
- `to: sonnet@cachy` — narrowed to the dispatcher whose `--alias` is `cachy`.
|
|
227
227
|
|
|
228
228
|
The model name is everything before the `@`; the machine alias is everything after. The `re:` activation rule ignores `@machine` suffixes — only addressing honors them.
|
package/template/PROTOCOL.md
CHANGED
|
@@ -49,7 +49,7 @@ You do not need to know whether the message you are processing was produced by a
|
|
|
49
49
|
|
|
50
50
|
## Delivery semantics
|
|
51
51
|
|
|
52
|
-
**At-least-once.** Each dispatcher tracks one cursor recording the git commit the transport
|
|
52
|
+
**At-least-once.** Each dispatcher tracks one cursor recording where the transport was last scanned (a git commit on the git transport, an mtime watermark on the filesystem transport). If a machine crashes mid-tick, the next tick re-dispatches anything not past the cursor; a duplicate reply may land in the channel.
|
|
53
53
|
|
|
54
54
|
For idempotent work (lookups, computation, advice) duplicates are harmless. For non-idempotent side effects (sending email, file deletion, payments), check the channel for evidence of prior completion before acting.
|
|
55
55
|
|
package/template/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
Source-of-truth content that `crosstalk daemon init` copies into every new transport. Not a runtime artifact, not a published package on its own — it lives here in the monorepo and gets bundled into the published npm tarball at `prepack` time (`package.json` copies `transport/` → `template/`).
|
|
6
6
|
|
|
7
|
-
> **What Crosstalk is.** Crosstalk is an agent-agnostic swarm communication protocol
|
|
7
|
+
> **What Crosstalk is.** Crosstalk is an agent-agnostic swarm communication protocol over a shared directory. The bus is a directory — a git repo (default) or a plain/shared filesystem (`transport: local`). Full background: **[the root README](../README.md)**.
|
|
8
8
|
|
|
9
9
|
---
|
|
10
10
|
|