@dtudury/streamo 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +10 -0
- package/LICENSE +661 -0
- package/README.md +194 -0
- package/ROADMAP.md +111 -0
- package/bin/streamo.js +238 -0
- package/jsconfig.json +9 -0
- package/package.json +26 -0
- package/public/apps/chat/index.html +61 -0
- package/public/apps/chat/main.js +144 -0
- package/public/apps/styles/proto.css +71 -0
- package/public/index.html +109 -0
- package/public/streamo/Addressifier.js +212 -0
- package/public/streamo/CodecRegistry.js +195 -0
- package/public/streamo/ContentMap.js +79 -0
- package/public/streamo/DESIGN.md +61 -0
- package/public/streamo/Repo.js +176 -0
- package/public/streamo/Repo.test.js +82 -0
- package/public/streamo/RepoRegistry.js +91 -0
- package/public/streamo/RepoRegistry.test.js +87 -0
- package/public/streamo/Signature.js +15 -0
- package/public/streamo/Signer.js +91 -0
- package/public/streamo/Streamo.js +392 -0
- package/public/streamo/Streamo.test.js +205 -0
- package/public/streamo/archiveSync.js +62 -0
- package/public/streamo/chat-cli.js +122 -0
- package/public/streamo/chat-server.js +60 -0
- package/public/streamo/codecs.js +400 -0
- package/public/streamo/fileSync.js +238 -0
- package/public/streamo/h.js +202 -0
- package/public/streamo/h.mount.test.js +67 -0
- package/public/streamo/h.test.js +121 -0
- package/public/streamo/mount.js +248 -0
- package/public/streamo/originSync.js +60 -0
- package/public/streamo/outletSync.js +105 -0
- package/public/streamo/registrySync.js +333 -0
- package/public/streamo/registrySync.test.js +373 -0
- package/public/streamo/s3Sync.js +99 -0
- package/public/streamo/stateFileSync.js +17 -0
- package/public/streamo/sync.test.js +98 -0
- package/public/streamo/utils/NestedSet.js +41 -0
- package/public/streamo/utils/Recaller.js +77 -0
- package/public/streamo/utils/mockDOM.js +113 -0
- package/public/streamo/utils/nextTick.js +22 -0
- package/public/streamo/utils/noble-secp256k1.js +602 -0
- package/public/streamo/utils/testing.js +90 -0
- package/public/streamo/utils.js +57 -0
- package/public/streamo/webSync.js +118 -0
- package/scripts/serve.js +15 -0
- package/smoke.test.js +132 -0
package/README.md
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
# streamo
|
|
2
|
+
|
|
3
|
+
> every device is an equal author
|
|
4
|
+
|
|
5
|
+
Streamo is a content-addressed, cryptographically signed, peer-to-peer sync library. There is no central server — your keypair is your identity, your commit log is the source of truth, and every connected peer sees the same history.
|
|
6
|
+
|
|
7
|
+
## core ideas
|
|
8
|
+
|
|
9
|
+
- **Content-addressed** — data is identified by what it is, not where it lives. The same value always lands at the same address; deduplication and diffing are free.
|
|
10
|
+
- **Signed** — every write is authenticated with a secp256k1 keypair derived from your username and password. Peers reject unsigned or mis-signed data.
|
|
11
|
+
- **Append-only** — history is never rewritten. Every commit is permanent and verifiable.
|
|
12
|
+
- **P2P sync** — repos replicate over WebSocket. The server is just another peer; disconnect it and the data is still yours.
|
|
13
|
+
|
|
14
|
+
## install
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npm install streamo
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Or run the CLI directly:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npx streamo --help
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## cli
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
streamo \
|
|
30
|
+
--name my-notes \
|
|
31
|
+
--username alice \
|
|
32
|
+
--files ./notes \
|
|
33
|
+
--web 8080
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Opens `notes/` for editing, syncs every save to all connected peers, and serves a browser UI at `http://localhost:8080`. All options can come from a `.env` file:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
streamo --env-file .env
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
| env var | flag | description |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| `STREAMO_NAME` | `--name` | dataset name |
|
|
45
|
+
| `STREAMO_USERNAME` | `--username` | signing identity |
|
|
46
|
+
| `STREAMO_PASSWORD` | `--password` | signing password |
|
|
47
|
+
| `STREAMO_DATA_DIR` | `--data-dir` | archive directory (default `.streamo`) |
|
|
48
|
+
| `STREAMO_FILES` | `--files` | mirror local files |
|
|
49
|
+
| `STREAMO_WEB` | `--web` | HTTP + WebSocket server port |
|
|
50
|
+
| `STREAMO_OUTLET` | `--outlet` | accept inbound peer connections |
|
|
51
|
+
| `STREAMO_ORIGIN` | `--origin` | connect to a remote outlet |
|
|
52
|
+
| `STREAMO_S3_BUCKET` | `--s3-bucket` | S3 bucket for replication |
|
|
53
|
+
|
|
54
|
+
## javascript api
|
|
55
|
+
|
|
56
|
+
### Streamo — reactive append-only store
|
|
57
|
+
|
|
58
|
+
```js
|
|
59
|
+
import { Streamo } from 'streamo/public/streamo/Streamo.js'
|
|
60
|
+
import { Recaller } from 'streamo/public/streamo/utils/Recaller.js'
|
|
61
|
+
|
|
62
|
+
const store = new Streamo()
|
|
63
|
+
store.set({ name: 'alice', score: 42 })
|
|
64
|
+
store.get('name') // 'alice'
|
|
65
|
+
store.get('score') // 42
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Values are encoded with a self-describing codec (strings, numbers, dates, booleans, arrays, objects, `Uint8Array`). Same value → same bytes → same address; dedup is automatic.
|
|
69
|
+
|
|
70
|
+
### Repo — signed commit log
|
|
71
|
+
|
|
72
|
+
`Repo` wraps a `Streamo` so every `set()` becomes a commit — message, date, data address, and parent pointer. The raw commit log is what syncs over the wire.
|
|
73
|
+
|
|
74
|
+
```js
|
|
75
|
+
import { Repo } from 'streamo/public/streamo/Repo.js'
|
|
76
|
+
|
|
77
|
+
const repo = new Repo()
|
|
78
|
+
repo.set({ name: 'alice', messages: [] })
|
|
79
|
+
repo.get('name') // 'alice'
|
|
80
|
+
repo.lastCommit // { message: '', date: Date, dataAddress: n, parent: n|undefined }
|
|
81
|
+
[...repo.history()] // newest-first iterator over commits
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### Signer — deterministic identity
|
|
85
|
+
|
|
86
|
+
```js
|
|
87
|
+
import { Signer } from 'streamo/public/streamo/Signer.js'
|
|
88
|
+
import { bytesToHex } from 'streamo/public/streamo/utils.js'
|
|
89
|
+
|
|
90
|
+
const signer = new Signer('alice', 'my-password')
|
|
91
|
+
const { publicKey } = await signer.keysFor('my-dataset')
|
|
92
|
+
const publicKeyHex = bytesToHex(publicKey) // stable identity for this (user, dataset) pair
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Keys are derived with PBKDF2 so the same username + password always produces the same keypair. No key files to manage.
|
|
96
|
+
|
|
97
|
+
### RepoRegistry — multi-repo store
|
|
98
|
+
|
|
99
|
+
```js
|
|
100
|
+
import { RepoRegistry } from 'streamo/public/streamo/RepoRegistry.js'
|
|
101
|
+
import { archiveSync } from 'streamo/public/streamo/archiveSync.js'
|
|
102
|
+
|
|
103
|
+
const registry = new RepoRegistry(async key => {
|
|
104
|
+
const repo = new Repo()
|
|
105
|
+
await archiveSync(repo, '.streamo', key) // persist to disk
|
|
106
|
+
return repo
|
|
107
|
+
})
|
|
108
|
+
|
|
109
|
+
const repo = await registry.open(publicKeyHex)
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### registrySync — peer sync over WebSocket
|
|
113
|
+
|
|
114
|
+
```js
|
|
115
|
+
import { registrySync } from 'streamo/public/streamo/registrySync.js'
|
|
116
|
+
|
|
117
|
+
const session = await registrySync(registry, 'localhost', 8080, {
|
|
118
|
+
// only sync repos you care about
|
|
119
|
+
filter: key => key === rootKey,
|
|
120
|
+
|
|
121
|
+
// follow links embedded in repo data (content-driven discovery)
|
|
122
|
+
follow: (keyHex, repo, subscribe) => {
|
|
123
|
+
for (const memberKey of repo.get('members') ?? []) subscribe(memberKey)
|
|
124
|
+
},
|
|
125
|
+
|
|
126
|
+
// react to peer announcements
|
|
127
|
+
onAnnounce: key => session.subscribe(key)
|
|
128
|
+
})
|
|
129
|
+
|
|
130
|
+
session.interest(rootKey) // receive announcements for this topic
|
|
131
|
+
session.announce(myKey, rootKey) // tell interested peers about your repo
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### h + mount — reactive UI
|
|
135
|
+
|
|
136
|
+
```js
|
|
137
|
+
import { h } from 'streamo/public/streamo/h.js'
|
|
138
|
+
import { mount } from 'streamo/public/streamo/mount.js'
|
|
139
|
+
import { Recaller } from 'streamo/public/streamo/utils/Recaller.js'
|
|
140
|
+
|
|
141
|
+
const recaller = new Recaller('app')
|
|
142
|
+
|
|
143
|
+
mount(h`
|
|
144
|
+
<div class="card">
|
|
145
|
+
<h2>${() => repo.get('name')}</h2>
|
|
146
|
+
<p>${() => repo.get('bio')}</p>
|
|
147
|
+
</div>
|
|
148
|
+
`, document.body, recaller)
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Functions interpolated as `${() => ...}` are reactive cells — they re-run automatically whenever the data they read changes. No virtual DOM diffing; only the exact DOM nodes bound to changed data update.
|
|
152
|
+
|
|
153
|
+
## sync backends
|
|
154
|
+
|
|
155
|
+
| module | what it does |
|
|
156
|
+
|---|---|
|
|
157
|
+
| `archiveSync` | persist chunks to numbered binary files under `.streamo/archive/` |
|
|
158
|
+
| `fileSync` | mirror a repo's value to/from the local filesystem (respects `.gitignore`) |
|
|
159
|
+
| `outletSync` | WebSocket server — accepts inbound peer connections |
|
|
160
|
+
| `originSync` | WebSocket client — connects to a remote outlet |
|
|
161
|
+
| `webSync` | HTTP + WebSocket server with browser-ready assets |
|
|
162
|
+
| `s3Sync` | replicate chunks to S3-compatible object storage |
|
|
163
|
+
| `stateFileSync` | write repo state as JSON on every change |
|
|
164
|
+
|
|
165
|
+
## chat example
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
# start the server
|
|
169
|
+
node public/streamo/chat-server.js 8080
|
|
170
|
+
|
|
171
|
+
# join from the browser
|
|
172
|
+
open http://localhost:8080
|
|
173
|
+
|
|
174
|
+
# join from the terminal
|
|
175
|
+
node public/streamo/chat-cli.js alice secret localhost 8080
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Each participant owns their own message stream. The server holds only a root repo listing members; it has no special authority over anyone's data.
|
|
179
|
+
|
|
180
|
+
## tests
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
node --test # all tests
|
|
184
|
+
node --test public/streamo/Repo.test.js # single file
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
## roadmap
|
|
188
|
+
|
|
189
|
+
See [ROADMAP.md](./ROADMAP.md) for what's been built, what's next, and what we're
|
|
190
|
+
aiming at for 1.0.
|
|
191
|
+
|
|
192
|
+
## license
|
|
193
|
+
|
|
194
|
+
AGPL-3.0-only
|
package/ROADMAP.md
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# streamo roadmap
|
|
2
|
+
|
|
3
|
+
This is a living document — updated with every meaningful change to give a clear
|
|
4
|
+
picture of where the project is and where it's headed.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## where we are (0.1.0)
|
|
9
|
+
|
|
10
|
+
The foundation is solid and working. Here's what's in:
|
|
11
|
+
|
|
12
|
+
**Core data layer**
|
|
13
|
+
- `Streamo` — reactive, content-addressed, append-only byte store with a
|
|
14
|
+
self-describing codec. Same value always encodes to the same bytes; dedup and
|
|
15
|
+
diffing are free.
|
|
16
|
+
- `Repo` — every write is a signed commit. Message, date, data address, parent.
|
|
17
|
+
The full history is always there.
|
|
18
|
+
- `Signer` — deterministic secp256k1 keypairs from username + password via PBKDF2.
|
|
19
|
+
No key files to manage; same credentials always produce the same identity.
|
|
20
|
+
- `Recaller` — fine-grained reactive dependency tracker. Watchers re-run only when
|
|
21
|
+
the exact paths they accessed are mutated. Efficient and precise.
|
|
22
|
+
|
|
23
|
+
**Sync layer**
|
|
24
|
+
- `registrySync` — bidirectional multi-repo sync over a single WebSocket. Catalog,
|
|
25
|
+
subscribe, and content-driven discovery via `follow`. Works in both Node and the
|
|
26
|
+
browser.
|
|
27
|
+
- `outletSync` / `originSync` — server and client sides of a peer connection.
|
|
28
|
+
- `archiveSync` — persists chunks to binary files on disk. Repos survive restarts.
|
|
29
|
+
- `fileSync` — mirrors a repo's value to/from the local filesystem.
|
|
30
|
+
- `s3Sync` — replicates chunks to S3-compatible object storage.
|
|
31
|
+
- Ephemeral messaging layer — `interest` / `announce` for peer discovery without
|
|
32
|
+
any persistence.
|
|
33
|
+
|
|
34
|
+
**UI layer**
|
|
35
|
+
- `h` — tagged template literal parser. Turns `h\`<div class=${cls}>...\`` into a
|
|
36
|
+
virtual tree of `HElement` / `HText` / slot nodes.
|
|
37
|
+
- `mount` — reactive DOM renderer. Slots that are functions re-run automatically
|
|
38
|
+
when the data they read changes. No virtual DOM diffing — only the exact nodes
|
|
39
|
+
bound to mutated paths update.
|
|
40
|
+
|
|
41
|
+
**Apps**
|
|
42
|
+
- Chat — full p2p messaging app. Each participant owns their own message stream;
|
|
43
|
+
the server is just a relay and holds no special authority. Runs in the browser
|
|
44
|
+
and from the terminal (`chat-cli.js`).
|
|
45
|
+
- Homepage at `public/index.html`.
|
|
46
|
+
- `npm run serve` — static file server for `public/`.
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## what's next
|
|
51
|
+
|
|
52
|
+
### fix watcher leaks in `mount` ← start here
|
|
53
|
+
When a reactive slot re-renders, child watchers registered inside it are never
|
|
54
|
+
cleaned up. Every re-render accumulates more. This is a correctness bug and the
|
|
55
|
+
thing most worth fixing before building more UI on top of `mount`.
|
|
56
|
+
|
|
57
|
+
### component support in `h`
|
|
58
|
+
Functions as tags: `h\`<${Card} title="hi"/>\``. This is the difference between
|
|
59
|
+
a templating tool and a UI framework. Once watcher cleanup exists, components
|
|
60
|
+
become straightforward — a component is just a function that returns nodes and
|
|
61
|
+
cleans up after itself.
|
|
62
|
+
|
|
63
|
+
### SVG namespace
|
|
64
|
+
`mount` hardcodes the XHTML namespace. `h\`<svg><path/></svg>\`` won't render
|
|
65
|
+
correctly until `mount` auto-detects SVG elements and switches namespaces.
|
|
66
|
+
|
|
67
|
+
### `class` as array or object
|
|
68
|
+
`class=${['btn', isActive && 'active']}` is such a common pattern that not
|
|
69
|
+
supporting it is a daily papercut. Easy win.
|
|
70
|
+
|
|
71
|
+
### chat persistence
|
|
72
|
+
Right now the chat server is in-memory — restart it and history is gone. Wiring
|
|
73
|
+
`archiveSync` into `chat-server.js` is a small change with a big quality-of-life
|
|
74
|
+
improvement.
|
|
75
|
+
|
|
76
|
+
### chat signing
|
|
77
|
+
Messages aren't cryptographically verified yet. Anyone who knows a participant's
|
|
78
|
+
public key hex could theoretically spoof them. Wiring `repo.sign()` after each
|
|
79
|
+
`set()` closes this.
|
|
80
|
+
|
|
81
|
+
### presence indicators
|
|
82
|
+
Who's currently online? The `interest` / `announce` layer is ephemeral by design,
|
|
83
|
+
so presence is a heartbeat + timeout — announce yourself periodically, time out
|
|
84
|
+
peers you haven't heard from.
|
|
85
|
+
|
|
86
|
+
### rebuild the browser app
|
|
87
|
+
The old repository-browser app was left behind during the migration because its
|
|
88
|
+
imports broke. Rebuilding it with `h` / `mount` would be the first substantial
|
|
89
|
+
real-world test of the UI layer.
|
|
90
|
+
|
|
91
|
+
### fix dead links on the homepage
|
|
92
|
+
`public/index.html` links to the browser and components apps that no longer exist.
|
|
93
|
+
Either rebuild them or remove the links.
|
|
94
|
+
|
|
95
|
+
### CLAUDE.md
|
|
96
|
+
Add a `CLAUDE.md` for the streamo repo so future Claude sessions have the full
|
|
97
|
+
context.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## toward 1.0
|
|
102
|
+
|
|
103
|
+
The three things blocking a stable `1.0` claim:
|
|
104
|
+
|
|
105
|
+
1. **Watcher leak fixed** — `mount` must be correct before anyone builds on it
|
|
106
|
+
2. **Components** — without them, `h` / `mount` is too limited for real apps
|
|
107
|
+
3. **Chat signing** — the whole point of the project is cryptographic authorship;
|
|
108
|
+
the flagship app should demonstrate it
|
|
109
|
+
|
|
110
|
+
Keyed list reconciliation and refs are quality-of-life improvements that can come
|
|
111
|
+
after 1.0.
|
package/bin/streamo.js
ADDED
|
@@ -0,0 +1,238 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
import { readFileSync } from 'fs'
|
|
4
|
+
import { dirname } from 'path'
|
|
5
|
+
import { Option, program } from 'commander'
|
|
6
|
+
import { config } from 'dotenv'
|
|
7
|
+
import { question, questionNewPassword } from 'readline-sync'
|
|
8
|
+
import { start as startRepl } from 'repl'
|
|
9
|
+
import { Signer } from '../public/streamo/Signer.js'
|
|
10
|
+
import { Repo } from '../public/streamo/Repo.js'
|
|
11
|
+
import { RepoRegistry } from '../public/streamo/RepoRegistry.js'
|
|
12
|
+
import { archiveSync } from '../public/streamo/archiveSync.js'
|
|
13
|
+
import { fileSync } from '../public/streamo/fileSync.js'
|
|
14
|
+
import { outletSync } from '../public/streamo/outletSync.js'
|
|
15
|
+
import { originSync } from '../public/streamo/originSync.js'
|
|
16
|
+
import { webSync } from '../public/streamo/webSync.js'
|
|
17
|
+
import { s3Sync } from '../public/streamo/s3Sync.js'
|
|
18
|
+
import { stateFileSync } from '../public/streamo/stateFileSync.js'
|
|
19
|
+
|
|
20
|
+
const { version } = JSON.parse(readFileSync(new URL('../package.json', import.meta.url)))
|
|
21
|
+
|
|
22
|
+
program
|
|
23
|
+
.name('streamo')
|
|
24
|
+
.description('streamo CLI')
|
|
25
|
+
.version(version)
|
|
26
|
+
|
|
27
|
+
.addOption(
|
|
28
|
+
new Option('--env-file <path>', 'path to .env file')
|
|
29
|
+
)
|
|
30
|
+
.addOption(
|
|
31
|
+
new Option('--name <string>', 'name for this dataset')
|
|
32
|
+
.env('STREAMO_NAME')
|
|
33
|
+
)
|
|
34
|
+
.addOption(
|
|
35
|
+
new Option('--username <string>', 'username for signing')
|
|
36
|
+
.env('STREAMO_USERNAME')
|
|
37
|
+
)
|
|
38
|
+
.addOption(
|
|
39
|
+
new Option('--password <string>', 'password for signing')
|
|
40
|
+
.env('STREAMO_PASSWORD')
|
|
41
|
+
)
|
|
42
|
+
.addOption(
|
|
43
|
+
new Option('--data-dir <path>', 'directory for archive files')
|
|
44
|
+
.env('STREAMO_DATA_DIR')
|
|
45
|
+
.default('.streamo')
|
|
46
|
+
)
|
|
47
|
+
.addOption(
|
|
48
|
+
new Option('--files [path]', 'mirror local files to/from streamo (defaults to current directory)')
|
|
49
|
+
.env('STREAMO_FILES')
|
|
50
|
+
.preset('.')
|
|
51
|
+
)
|
|
52
|
+
.addOption(
|
|
53
|
+
new Option('--state-file <path>', 'write streamo state as JSON to this file on every change')
|
|
54
|
+
.env('STREAMO_STATE_FILE')
|
|
55
|
+
)
|
|
56
|
+
.addOption(
|
|
57
|
+
new Option('--s3-bucket <name>', 'S3 bucket name')
|
|
58
|
+
.env('STREAMO_S3_BUCKET')
|
|
59
|
+
)
|
|
60
|
+
.addOption(
|
|
61
|
+
new Option('--s3-endpoint <url>', 'S3-compatible endpoint (omit for AWS)')
|
|
62
|
+
.env('STREAMO_S3_ENDPOINT')
|
|
63
|
+
)
|
|
64
|
+
.addOption(
|
|
65
|
+
new Option('--s3-region <region>', 'S3 region')
|
|
66
|
+
.env('STREAMO_S3_REGION')
|
|
67
|
+
)
|
|
68
|
+
.addOption(
|
|
69
|
+
new Option('--s3-access-key-id <id>', 'S3 access key ID')
|
|
70
|
+
.env('STREAMO_S3_ACCESS_KEY_ID')
|
|
71
|
+
)
|
|
72
|
+
.addOption(
|
|
73
|
+
new Option('--s3-secret-access-key <key>', 'S3 secret access key')
|
|
74
|
+
.env('STREAMO_S3_SECRET_ACCESS_KEY')
|
|
75
|
+
)
|
|
76
|
+
.addOption(
|
|
77
|
+
new Option('--web [port]', 'start HTTP + WebSocket server for browsers and peers')
|
|
78
|
+
.env('STREAMO_WEB')
|
|
79
|
+
.preset('8080')
|
|
80
|
+
)
|
|
81
|
+
.addOption(
|
|
82
|
+
new Option('--outlet [port]', 'accept inbound WebSocket peer connections')
|
|
83
|
+
.env('STREAMO_OUTLET')
|
|
84
|
+
.preset('1024')
|
|
85
|
+
)
|
|
86
|
+
.addOption(
|
|
87
|
+
new Option('--origin <host:port>', 'connect to a remote outlet')
|
|
88
|
+
.env('STREAMO_ORIGIN')
|
|
89
|
+
)
|
|
90
|
+
.addOption(
|
|
91
|
+
new Option('--interactive', 'start a REPL with streamo, signer, and helpers as globals')
|
|
92
|
+
.env('STREAMO_INTERACTIVE')
|
|
93
|
+
)
|
|
94
|
+
.addOption(
|
|
95
|
+
new Option('--key-iterations <number>', 'PBKDF2 iterations for key derivation (lower = faster startup, less secure)')
|
|
96
|
+
.env('STREAMO_KEY_ITERATIONS')
|
|
97
|
+
.default(100000)
|
|
98
|
+
.argParser(Number)
|
|
99
|
+
)
|
|
100
|
+
.addOption(
|
|
101
|
+
new Option('--verbose', 'enable verbose logging')
|
|
102
|
+
.env('STREAMO_VERBOSE')
|
|
103
|
+
)
|
|
104
|
+
|
|
105
|
+
.parse()
|
|
106
|
+
|
|
107
|
+
const options = program.opts()
|
|
108
|
+
|
|
109
|
+
if (options.envFile) {
|
|
110
|
+
config({ path: options.envFile })
|
|
111
|
+
program.parse()
|
|
112
|
+
Object.assign(options, program.opts())
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
options.name ||= question('Name: ')
|
|
116
|
+
options.username ||= question('Username: ')
|
|
117
|
+
const password = options.password || questionNewPassword('Password [ATTENTION!: Backspace won\'t work here]: ', { min: 4, max: 999 })
|
|
118
|
+
|
|
119
|
+
const signer = new Signer(options.username, password, options.keyIterations)
|
|
120
|
+
const { publicKey } = await signer.keysFor(options.name)
|
|
121
|
+
const publicKeyHex = Array.from(publicKey).map(b => b.toString(16).padStart(2, '0')).join('')
|
|
122
|
+
|
|
123
|
+
const name = options.name
|
|
124
|
+
const username = options.username
|
|
125
|
+
const appPath = options.envFile
|
|
126
|
+
? '/' + dirname(options.envFile).replace(/^public\//, '') + '/'
|
|
127
|
+
: '/'
|
|
128
|
+
const webUrl = options.web ? `http://localhost:${+options.web}${appPath}` : null
|
|
129
|
+
const rows = [
|
|
130
|
+
['NAME', name],
|
|
131
|
+
['USERNAME', username],
|
|
132
|
+
['PUBLIC KEY', publicKeyHex],
|
|
133
|
+
...(webUrl ? [['URL', webUrl]] : []),
|
|
134
|
+
]
|
|
135
|
+
const maxLength = Math.max(...rows.map(([, v]) => v.length))
|
|
136
|
+
const pad = (v) => v + ' '.repeat(maxLength - v.length)
|
|
137
|
+
const div = '─'.repeat(maxLength)
|
|
138
|
+
const label = (l) => l.padStart(16)
|
|
139
|
+
console.log(`\x1b[35m
|
|
140
|
+
╭${'─'.repeat(maxLength + 23)}╮
|
|
141
|
+
╞══════════════════╤══${'═'.repeat(maxLength)}══╡
|
|
142
|
+
${rows.map(([l, v], i) => [
|
|
143
|
+
` │ ${label(l + ':')} │ \x1b[0m${pad(v)}\x1b[35m │`,
|
|
144
|
+
i < rows.length - 1 ? ` ├──────────────────┼──${div}──┤` : null
|
|
145
|
+
].filter(Boolean).join('\n')).join('\n')}
|
|
146
|
+
╰──────────────────┴──${'━'.repeat(maxLength)}──╯\x1b[0m`)
|
|
147
|
+
|
|
148
|
+
const dataDir = options.dataDir
|
|
149
|
+
const registry = new RepoRegistry(async key => {
|
|
150
|
+
const repo = new Repo()
|
|
151
|
+
await archiveSync(repo, dataDir, key)
|
|
152
|
+
return repo
|
|
153
|
+
})
|
|
154
|
+
const streamo = await registry.open(publicKeyHex)
|
|
155
|
+
|
|
156
|
+
if (options.files) {
|
|
157
|
+
const folder = typeof options.files === 'string' ? options.files : '.'
|
|
158
|
+
await fileSync(streamo, folder, options.dataDir)
|
|
159
|
+
console.log(`\x1b[32mmirroring files: ${folder}\x1b[0m`)
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
if (options.stateFile) {
|
|
163
|
+
stateFileSync(streamo, options.stateFile)
|
|
164
|
+
console.log(`\x1b[32mstate file: ${options.stateFile}\x1b[0m`)
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
if (options.s3Bucket) {
|
|
168
|
+
await s3Sync(streamo, publicKeyHex, {
|
|
169
|
+
bucket: options.s3Bucket,
|
|
170
|
+
endpoint: options.s3Endpoint,
|
|
171
|
+
region: options.s3Region,
|
|
172
|
+
accessKeyId: options.s3AccessKeyId,
|
|
173
|
+
secretAccessKey: options.s3SecretAccessKey
|
|
174
|
+
})
|
|
175
|
+
console.log(`\x1b[32ms3: syncing to bucket ${options.s3Bucket}\x1b[0m`)
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
if (options.web) {
|
|
179
|
+
await webSync(registry, publicKeyHex, +options.web, name, options.keyIterations)
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
if (options.outlet) {
|
|
183
|
+
const port = +options.outlet
|
|
184
|
+
outletSync(registry, port)
|
|
185
|
+
console.log(`\x1b[32moutlet: listening on port ${port}\x1b[0m`)
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
if (options.origin) {
|
|
189
|
+
const [host, port] = options.origin.split(':')
|
|
190
|
+
await originSync(streamo, publicKeyHex, host, +port)
|
|
191
|
+
console.log(`\x1b[32morigin: connected to ${options.origin}\x1b[0m`)
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
if (options.verbose) {
|
|
195
|
+
console.log(`archive: ${options.dataDir}/${publicKeyHex}.bin (${streamo.byteLength} bytes loaded)`)
|
|
196
|
+
console.log({ options })
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
if (options.interactive) {
|
|
200
|
+
const get = (...args) => streamo.get(...args)
|
|
201
|
+
const set = (...args) => streamo.set(...args)
|
|
202
|
+
const ls = () => [...registry].map(([k, s]) => ({ key: k.slice(0, 8) + '…', bytes: s.byteLength }))
|
|
203
|
+
const connect = (hostPort) => {
|
|
204
|
+
const [host, port] = hostPort.split(':')
|
|
205
|
+
return originSync(streamo, publicKeyHex, host, +port)
|
|
206
|
+
}
|
|
207
|
+
|
|
208
|
+
Object.assign(globalThis, {
|
|
209
|
+
// identity
|
|
210
|
+
name, username, publicKeyHex, signer,
|
|
211
|
+
// data
|
|
212
|
+
streamo, registry,
|
|
213
|
+
// shorthands
|
|
214
|
+
get, set, ls,
|
|
215
|
+
// networking
|
|
216
|
+
connect, originSync, outletSync,
|
|
217
|
+
// sync modules
|
|
218
|
+
archiveSync, fileSync, s3Sync,
|
|
219
|
+
// class
|
|
220
|
+
Repo, RepoRegistry,
|
|
221
|
+
})
|
|
222
|
+
|
|
223
|
+
console.log(`\x1b[36m
|
|
224
|
+
get(...path) streamo.get() — read a value by path
|
|
225
|
+
set(value) streamo.set() — write a value
|
|
226
|
+
ls() list all open streamos in the registry
|
|
227
|
+
connect('host:port') connect this streamo to a remote outlet
|
|
228
|
+
streamo / registry the live streamo and registry instances
|
|
229
|
+
signer sign / verify data
|
|
230
|
+
originSync(s,k,h,p) attach any streamo as an origin
|
|
231
|
+
outletSync(reg,port) start a new outlet server\x1b[0m`)
|
|
232
|
+
|
|
233
|
+
const replServer = startRepl({ breakEvalOnSigint: true })
|
|
234
|
+
replServer.setupHistory('.node_repl_history', err => {
|
|
235
|
+
if (err) console.error(err)
|
|
236
|
+
})
|
|
237
|
+
replServer.on('exit', process.exit)
|
|
238
|
+
}
|
package/jsconfig.json
ADDED
package/package.json
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@dtudury/streamo",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "content-addressed p2p sync",
|
|
5
|
+
"repository": "git@github.com:dtudury/streamo.git",
|
|
6
|
+
"author": "David Tudury <david.tudury@gmail.com>",
|
|
7
|
+
"license": "AGPL-3.0-only",
|
|
8
|
+
"bin": {
|
|
9
|
+
"streamo": "./bin/streamo.js"
|
|
10
|
+
},
|
|
11
|
+
"type": "module",
|
|
12
|
+
"dependencies": {
|
|
13
|
+
"@aws-sdk/client-s3": "^3.958.0",
|
|
14
|
+
"@gerhobbelt/gitignore-parser": "^0.2.0-9",
|
|
15
|
+
"@parcel/watcher": "^2.5.1",
|
|
16
|
+
"commander": "^14.0.2",
|
|
17
|
+
"dotenv": "^17.2.3",
|
|
18
|
+
"express": "^5.2.1",
|
|
19
|
+
"mkcert": "^3.2.0",
|
|
20
|
+
"readline-sync": "^1.4.10",
|
|
21
|
+
"ws": "^8.18.3"
|
|
22
|
+
},
|
|
23
|
+
"scripts": {
|
|
24
|
+
"serve": "node scripts/serve.js"
|
|
25
|
+
}
|
|
26
|
+
}
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="UTF-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
6
|
+
<title>streamo chat</title>
|
|
7
|
+
<style>
|
|
8
|
+
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0 }
|
|
9
|
+
:root { font-family: system-ui, sans-serif; font-size: 15px; --bg: #f5f5f5; --surface: #fff; --accent: #0070f3; --border: #ddd }
|
|
10
|
+
body { background: var(--bg); height: 100dvh; display: flex; align-items: center; justify-content: center }
|
|
11
|
+
|
|
12
|
+
/* Login */
|
|
13
|
+
#login { background: var(--surface); border: 1px solid var(--border); border-radius: 10px; padding: 2rem; width: min(360px, 90vw); display: flex; flex-direction: column; gap: .75rem }
|
|
14
|
+
#login h1 { font-size: 1.2rem; font-weight: 600 }
|
|
15
|
+
#login input { border: 1px solid var(--border); border-radius: 6px; padding: .5rem .75rem; font-size: 1rem; width: 100% }
|
|
16
|
+
#login button { background: var(--accent); color: #fff; border: none; border-radius: 6px; padding: .6rem; font-size: 1rem; cursor: pointer }
|
|
17
|
+
#login button:hover { opacity: .85 }
|
|
18
|
+
#status { font-size: .8rem; color: #666; min-height: 1.2em }
|
|
19
|
+
|
|
20
|
+
/* Chat */
|
|
21
|
+
#chat { display: none; flex-direction: column; width: min(600px, 100vw); height: 100dvh; background: var(--surface) }
|
|
22
|
+
#chat-header { padding: .75rem 1rem; border-bottom: 1px solid var(--border); font-weight: 600; font-size: .95rem; display: flex; gap: .5rem; align-items: center }
|
|
23
|
+
#chat-header span { font-size: .75rem; font-weight: 400; color: #888 }
|
|
24
|
+
#messages { flex: 1; overflow-y: auto; padding: 1rem; display: flex; flex-direction: column; gap: .5rem }
|
|
25
|
+
.msg { max-width: 75%; padding: .5rem .75rem; border-radius: 10px; line-height: 1.4; word-break: break-word }
|
|
26
|
+
.msg.mine { align-self: flex-end; background: var(--accent); color: #fff; border-bottom-right-radius: 3px }
|
|
27
|
+
.msg.theirs { align-self: flex-start; background: #f0f0f0; border-bottom-left-radius: 3px }
|
|
28
|
+
.msg .sender { font-size: .7rem; font-weight: 600; margin-bottom: .2rem; opacity: .7 }
|
|
29
|
+
.msg .text { font-size: .95rem }
|
|
30
|
+
.msg .time { font-size: .65rem; opacity: .5; margin-top: .2rem; text-align: right }
|
|
31
|
+
#input-row { display: flex; gap: .5rem; padding: .75rem; border-top: 1px solid var(--border) }
|
|
32
|
+
#msg-input { flex: 1; border: 1px solid var(--border); border-radius: 20px; padding: .5rem 1rem; font-size: .95rem; outline: none }
|
|
33
|
+
#msg-input:focus { border-color: var(--accent) }
|
|
34
|
+
#send-btn { background: var(--accent); color: #fff; border: none; border-radius: 50%; width: 38px; height: 38px; cursor: pointer; font-size: 1.1rem; flex-shrink: 0 }
|
|
35
|
+
#send-btn:hover { opacity: .85 }
|
|
36
|
+
</style>
|
|
37
|
+
</head>
|
|
38
|
+
<body>
|
|
39
|
+
|
|
40
|
+
<div id="login">
|
|
41
|
+
<h1>streamo chat</h1>
|
|
42
|
+
<input id="username" placeholder="username" autocomplete="username">
|
|
43
|
+
<input id="password" type="password" placeholder="password" autocomplete="current-password">
|
|
44
|
+
<button id="join-btn">join</button>
|
|
45
|
+
<div id="status"></div>
|
|
46
|
+
</div>
|
|
47
|
+
|
|
48
|
+
<div id="chat">
|
|
49
|
+
<div id="chat-header">
|
|
50
|
+
streamo chat <span id="my-name"></span>
|
|
51
|
+
</div>
|
|
52
|
+
<div id="messages"></div>
|
|
53
|
+
<div id="input-row">
|
|
54
|
+
<input id="msg-input" placeholder="message…" autocomplete="off">
|
|
55
|
+
<button id="send-btn">↑</button>
|
|
56
|
+
</div>
|
|
57
|
+
</div>
|
|
58
|
+
|
|
59
|
+
<script type="module" src="./main.js"></script>
|
|
60
|
+
</body>
|
|
61
|
+
</html>
|