weed-cli 0.1.0__tar.gz

weed_cli-0.1.0/PKG-INFO

@@ -0,0 +1,870 @@
+Metadata-Version: 2.4
+Name: weed-cli
+Version: 0.1.0
+Summary: Censorship-resistant video PoC — discovery/hosting/download over signed relay events, a real Kademlia DHT, or a TLS-capable NAT-traversal tunnel
+License: MIT
+Keywords: p2p,video,censorship-resistant,discovery,dht,kademlia,nat-traversal
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Communications :: File Sharing
+Classifier: Topic :: Internet
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: btcvm>=1.3.7
+Requires-Dist: cryptography>=41.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Provides-Extra: dht
+Requires-Dist: kademlia>=2.2; extra == "dht"
+Provides-Extra: qr
+Requires-Dist: qrcode>=7.4; extra == "qr"
+Provides-Extra: viz
+Requires-Dist: matplotlib>=3.8; extra == "viz"
+
+# Censorship-resistant video platform — PoC notes
+
+Brainstormed in #all-pdx 2026-08-22: a YouTube replacement indexed on Bitcoin,
+distributed over BitTorrent-style magnet links. This tracks what got built and
+what was actually learned, not just the idea.
+
+## The core design problem
+
+Storage and delivery over BitTorrent-plus-a-chain-timestamp is the easy part —
+solved plumbing. The two things that actually kill projects like this:
+
+- **Incentives.** LBRY minted a token and got sued as an unregistered security.
+  BitTube minted a token and died to the standard watch-to-earn Ponzi spiral.
+  PeerTube minted nothing and stayed permanently niche — no incentive layer at
+  all, purely volunteer-hosted. Anchoring to *existing* BTC (no new token) and
+  paying for service directly over Lightning avoids both failure modes at once.
+- **Discovery.** A single global index or "most popular" list is exactly as
+  seizable as YouTube's own trending page. The answer that doesn't reintroduce
+  a chokepoint: no canonical list at all — gossiped signals (payments, in this
+  case), any number of independent, replaceable indexer apps computing their
+  own view, Nostr-style. Not built yet, just designed.
+
+## What's built
+
+### `poc_challenge_auction.py` — possession-gated reverse auction
+
+`merkle_root`/`merkle_proof`/`verify_proof` are imported from the real,
+[published](https://pypi.org/project/btcvm/) `btcvm` package (`pip install
+btcvm`) — same functions `ott verify-chunk` runs locally. Used to be
+vendored copies (kept this repo dependency-free before btcvm was on PyPI);
+now that it's real and installable, this repo depends on it properly
+instead. `ott` is the storage/archive layer — what you have and can prove
+you have; this repo is the distribution/incentive layer built on top of
+it. Deliberately kept as separate packages rather than merged: `ott` stays
+the stable, already-published tool with real users, this stays free to be
+a rougher-edged PoC without dragging Lightning/Docker into `ott`'s
+dependency surface.
+
+In-process simulation, five peers, one 8-chunk file.
+
+**Part 1 — chunk-index challenge + auction.** A naive price-only auction picks
+the cheapest bidder regardless of whether they can deliver — and does, in the
+run captured here (a peer holding zero chunks wins on price alone). Gating bid
+eligibility on passing a random chunk-index + Merkle-proof challenge fixes it:
+a peer with nothing gets caught every round; a peer that tampers its response
+~70% of the time gets caught most rounds and wins once by pure luck — real
+evidence that a single spot-check isn't airtight, only a statistical one is.
+A legitimately partial holder (only half the chunks) correctly sits out
+rounds for chunks it doesn't have, without being flagged dishonest.
+
+**Part 2 — nonce-salted challenge + timing bound.** Knowing a file's public
+SHA256 gives an attacker nothing — SHA256 preimage resistance means you can't
+derive `hash(chunk||nonce)` from `hash(chunk)` alone, so a peer with only the
+published hash can't even attempt a response. A peer that *does* have the
+real bytes but fetches them from someone else in real time (relaying) answers
+with a cryptographically **correct** hash every time — the nonce alone can't
+catch that. Only an added timing bound can, because the relay hop costs real
+latency a local holder never pays.
+
+### `poc_network_challenge.py` — the same mechanism over real sockets
+
+Takes the timing-bound claim off paper: real TCP, real OS subprocesses, real
+`time.perf_counter()`, loopback (127.0.0.1). Three modes:
+
+- `holder <port>` — actually stores chunks, answers directly
+- `relay <port> <holder_host> <holder_port>` — stores nothing, chains a
+  second real TCP connection to the holder to fetch real bytes before
+  answering
+- `verify-remote <h_host> <h_port> <r_host> <r_port> [b_host] [b_port]` —
+  client mode: connects to already-running holder/relay (e.g. other
+  containers) and runs the separation analysis against them, with an
+  optional second honest holder as a same-vs-same jitter baseline
+- (no args) — local convenience: single-shot challenge rounds on loopback, narrated
+- `stats` — local convenience: spawns holder+relay as subprocesses on
+  loopback, bulk collection (80 real samples per role) + bootstrap analysis
+  of how many repeated challenges it takes for the *session mean* to
+  reliably separate holder from relay
+
+**Real finding, not the expected one:** on loopback, single-shot timing does
+*not* reliably separate a holder from a relay — the honest holder's worst
+recorded round was slower than the relay's best. Averaging repeated
+challenges does separate them (typically somewhere in the k=3–20 range across
+several runs — see below); trust a session average, not any one round.
+
+RunPod was down the first time this was tried, so real WAN latency got
+measured against public hosts as a substitute (1.1.1.1, 8.8.8.8,
+api.github.com: 5-30ms real TCP-connect RTT) — suggestive, not conclusive.
+**Update: ran it for real once a RunPod box came back up**, tunneled over a
+real SSH connection (`ssh -L`, since the pod only exposes its SSH port, not
+arbitrary TCP) — local honest holder vs. a local relay that secretly fetches
+from that real remote box for every challenge:
+
+```
+holder: mean 0.254ms  min 0.186ms  max 1.250ms
+relay:  mean 432.648ms  min 395.583ms  max 638.218ms
+
+session size k   worst honest mean   best cheater mean  separated?
+             1             1.250ms           395.583ms  YES
+```
+
+~1700x gap, separates cleanly at k=1 — single-shot is all you need once real
+geographic distance is involved. Confirms the substitute-host hypothesis:
+real WAN distance makes this *easy*; the hard case this PoC actually
+stress-tests is two peers that are genuinely close together, which is
+exactly when a nearby relay is hardest to catch on timing
+alone.
+
+![session-size separation chart](poc_challenge_separation.png)
+
+Chart from `viz_challenge_separation.py` — regenerates real measurements each
+run rather than plotting a fixed snapshot. **The exact crossover k is not a
+fixed constant** — it moved between k=3, k=8, and k=20 across different runs
+of this same script on the same machine, purely from real system jitter. That
+instability is itself the finding: don't hardcode a specific k, measure it
+live and adapt, and prefer statistical separation over any fixed threshold.
+
+### `docker-compose.yml` — the same test over real container networking
+
+Four services: `holder1` and `holder2` (two independent honest peers),
+`relay` (holds nothing, relays from `holder1` over the compose network),
+`verifier` (runs the same repeated-challenge analysis against all three,
+using DNS service names instead of loopback).
+
+```bash
+docker compose up --build --abort-on-container-exit verifier
+```
+
+Real run, over podman's docker-compose shim, actual separate containers on
+the compose bridge network:
+
+```
+holder: mean 0.246ms  min 0.188ms  max 0.953ms
+relay:  mean 0.483ms  min 0.414ms  max 1.134ms
+holder2 (2nd honest holder, same-vs-same jitter baseline): mean 0.236ms  min 0.209ms  max 0.311ms
+
+session size k   worst honest mean   best cheater mean  separated?
+             1             0.953ms             0.414ms  no
+             2             0.626ms             0.414ms  no
+             3             0.712ms             0.421ms  no
+             5             0.541ms             0.432ms  no
+             8             0.430ms             0.437ms  YES
+            12             0.425ms             0.445ms  YES
+            20             0.346ms             0.448ms  YES
+            30             0.314ms             0.451ms  YES
+```
+
+Same shape as loopback (single-shot doesn't separate, k≈8 does), plus one
+useful sanity check the loopback version can't give: holder2's baseline mean
+(0.236ms) sits right next to holder1's (0.246ms) — two equally honest,
+unrelated containers naturally land close together, while the relay
+(0.483ms, roughly double) is a real structural gap, not just inter-container
+noise.
+
+### `poc_reputation.py` — persistent local reputation + signed portable attestations
+
+Two mechanisms, both real (Ed25519 via the `cryptography` package, actual
+signing and verification, not simulated):
+
+1. **Local reputation store** (`ReputationStore`, persisted to JSON) — a
+   client's own record of direct experience with a peer (passes/fails/avg
+   latency), so a known-good peer doesn't need to re-earn trust from zero on
+   every interaction.
+2. **Signed attestations** — a client signs its own verification outcome for
+   a peer and hands the signed blob to another client, who didn't do the
+   verification but can check the signature and decide how much to trust it.
+   Same shape as PGP's Web of Trust, applied to possession-verification
+   outcomes instead of key identity — including PGP's actual historical
+   weak point: the crypto is the easy part, "how much do I trust this
+   signer" is the unsolved UX problem, not a technical one.
+3. **Revocation** — a signer can kill their own earlier vouch (`sign_revocation`,
+   keyed to the attestation's content-hash `attestation_id`). Only accepted
+   if the revocation's signer matches the original attestation's signer;
+   the revoked attestation stays on record rather than being deleted, so
+   "X vouched for Y, then revoked it" stays an honest, auditable fact
+   instead of quietly disappearing.
+
+Demonstrated for real in one run: a fresh client with zero direct history
+bootstraps a trust score for an unknown peer purely from another client's
+signed vouch; a vouch from a signer you don't trust at all is cryptographically
+valid but contributes zero weight; mutating a signed payload after the fact
+(`passes: 8 → 800`) is caught by signature verification; a 90-day-old
+attestation is worth 0.125x a fresh one under a 30-day trust half-life;
+alice revoking her own vouch drops bob's trust score for that peer without
+bob ever re-verifying it himself; mallory forging a revocation of *alice's*
+vouch (valid signature, wrong signer) is correctly rejected; a revocation
+referencing an attestation nobody's ever seen is rejected too.
+
+### `lightning_settle.py` + `lightning/` — real Lightning HTLC settlement
+
+Replaces `poc_challenge_auction.py`'s mock "settlement" print with a real
+one: two real LND nodes (Lightning Labs' production node software) on
+regtest, real bitcoind backing them, a real funded channel between them.
+`poc_challenge_auction.py --lightning` settles every auction round's winner
+with a genuine BOLT11 invoice + HTLC — not simulated, and not just trusting
+LND's own "SUCCEEDED" status: `lightning_settle.py` independently re-hashes
+the revealed preimage and checks it against the invoice's payment_hash
+locally before calling it settled.
+
+Real run, 5 winning rounds, real preimages each verified against their own
+payment hash:
+
+```
+WINNER: bob      9 sat   preimage 4ac71143706b...  payment_hash d1be1130c553...
+WINNER: bob      5 sat   preimage eea88d802d1a...  payment_hash 8acabea4ddf7...
+WINNER: bob      6 sat   preimage 76f1ade0f9be...  payment_hash f9c9bda28be0...
+WINNER: bob     10 sat   preimage 96e696c1cde9...  payment_hash f8eeb66d1878...
+WINNER: mallory  1 sat   preimage 508385841cb3...  payment_hash 1c2872b6018f...
+```
+
+Bob's cumulative channel balance after the run matched the sum of every
+settled payment exactly, checked directly against LND rather than assumed.
+Full setup steps in `lightning/README.md` — real bitcoind + LND takes a
+one-time channel-funding setup regtest can't skip (mine to coinbase
+maturity, open a channel, mine confirmations) before it's usable.
+
+### `poc_real_archive_challenge.py` — real `.ott` archive, real video, real scale
+
+Every other PoC file here used `os.urandom` fake chunks (8 of them).
+This one points the same mechanism at a real 217MB video, archived with the
+real `ott` CLI at a real 64KB chunk size:
+
+```
+real archive: real_video.mp4, 217,831,234 bytes, 3324 real chunks x 65536 bytes
+recomputed Merkle root matches ott's own commit: True
+```
+
+The thing this was actually checking — proof size at real scale:
+
+```
+chunk     0: 12 steps, 396B raw, 1176B as JSON
+chunk  1662: 12 steps, 396B raw, 1168B as JSON
+chunk  3323: 12 steps, 396B raw, 1167B as JSON
+```
+
+12 proof steps at 3324 real chunks vs. 3 steps at the toy 8-chunk scale —
+exactly log2(N), not linear, confirmed with real numbers instead of just
+trusting the math. Even a 2-hour movie at these settings (~10GB, ~163,840
+64KB chunks) would only need ~17 steps, still under 1KB. Then ran the same
+nonce-salted-challenge logic from `poc_challenge_auction.py` Part 2 against
+real bytes read straight off disk at real offsets — all 5 real rounds
+checked out: hash matches ott's own committed leaf, Merkle proof verifies,
+nonce response is internally consistent.
+
+`real_archive/real_video.mp4` isn't committed to this repo (208MB, and it's
+not this repo's to redistribute) — `real_archive/.ott/`'s metadata is
+tracked, so the chunk list and commitment are there for inspection even
+without the video itself. Reproduce with any file:
+
+```bash
+cd real_archive
+python3 /path/to/btcvm/ott.py init
+# edit .ott/config, set "chunk_size" to whatever you want (65536 used here)
+python3 /path/to/btcvm/ott.py add your_video.mp4
+python3 /path/to/btcvm/ott.py commit
+cd ..
+python3 poc_real_archive_challenge.py
+```
+
+### `discovery_relay.py` + `poc_discovery.py` — discovery, no canonical index
+
+The last unsolved piece from the original brainstorm, actually built: no
+single "trending" list, no server whose seizure kills discoverability.
+Three independent relay processes (`discovery_relay.py` — real stdlib
+`http.server`, no deps), each deliberately dumb: verifies a posted event's
+signature (a relay won't store garbage) but has zero opinion on content
+quality, zero ranking logic. A creator (carol) publishes a real event
+pointing at the real video's real Merkle root from item 6. Viewers like it
+and subscribe to each other, spread across the three relays — nobody posts
+to all three, on purpose.
+
+Two clients, `bob` (subscribes to dan + erin) and `mallory` (subscribes to
+frank only), each query all three relays, verify every event's signature
+themselves (never trust a relay's word for it), and compute their own
+ranking from their own subscribe graph — subscriptions *are* the trust
+graph, not a separate feature, same insight from the Slack thread now
+actually running as code:
+
+```
+same 27 gossiped events, both clients saw all 23 likes (3 honest + 20 sybil),
+but scored the content differently — 2.0 (bob) vs 1.0 (mallory) — because
+ranking runs on each client's own trust graph, not vote count.
+```
+
+A 20-identity sybil swarm likes the same content — every signature is
+real and individually valid, a relay has no basis to reject any of them —
+and moves neither client's score, because neither bob nor mallory
+subscribes to any of the sybils. Sybil resistance from the trust graph,
+not from relay-side moderation.
+
+Then relay:9101 — the one carol's publish event and dan's like both
+happened to live on — gets killed outright. Real result, not a clean win:
+the content stays discoverable and rankable (erin's like survived on a
+different relay), but the human-readable title and dan's like are gone for
+good, since neither was posted anywhere else. Redundancy has to be
+deliberate — post to more than one relay — it isn't automatic just because
+relays are plural. Same limitation a real Nostr relay dying would have.
+
+```bash
+python3 poc_discovery.py
+```
+
+### `node.py` — the integration piece: host, discover, download, for real
+
+Everything above is a demo of one mechanism at a time. `node.py` (via
+`weed.py host/discover/download/like/subscribe/whoami`) is the actual
+integration: a real node that hosts a real archived file over the real
+wire protocol from `poc_network_challenge.py` (extended with `INFO` and
+`LEAVES` so a downloader can learn the archive's shape first), announces
+itself on a real relay, and — new, not just wired from existing pieces —
+actually downloads a file from a peer and reassembles it on disk, which
+nothing before this verified chunk-by-chunk *and* wrote a real file.
+
+A persistent identity now lives at `~/.weed_identity.key` — every other
+script tonight generated a fresh Ed25519 keypair per run, which is fine for
+a demo but means nobody could ever accumulate reputation or be subscribed
+to across invocations. A real node needs a stable pubkey.
+
+Real end-to-end run: hosted the real 217MB video, discovered it from a
+separate process, downloaded it to a new path, and diffed the result
+against the original with `cmp` (not just checking the tool's own claim of
+success) — byte-for-byte identical, matching SHA256 on both sides, 3324
+chunks downloaded and verified in 1.3s.
+
+Caught a real bug doing this, not a clean pass on the first try: `ott`
+records a video's `sha256` manifest field as the **Merkle root** over its
+chunk hashes (`digest = merkle_root(chunks)` in `ott.py`'s `cmd_add`), not
+a linear whole-file hash — my first version streamed a plain
+`hashlib.sha256()` over the received bytes and compared that, which does
+not and structurally cannot equal a Merkle root. Every individual chunk
+was verifying correctly the whole time; only the final whole-file check
+was comparing the wrong thing. Fixed by recomputing the Merkle root over
+the received leaves and checking it against the host's advertised
+`sha256` — done *before* downloading any chunk, not after, so a host lying
+about its own archive gets caught immediately instead of after wasting
+bandwidth on it.
+
+```bash
+# terminal 1
+python3 discovery_relay.py 9101
+
+# terminal 2 — host the video from item 6
+python3 weed.py host real_archive --port 9201 --relay http://127.0.0.1:9101
+
+# terminal 3
+python3 weed.py whoami
+python3 weed.py discover --relay http://127.0.0.1:9101
+python3 weed.py download <content_hash_prefix> --relay http://127.0.0.1:9101 --out downloaded.mp4
+python3 weed.py like <content_hash> --relay http://127.0.0.1:9101
+python3 weed.py subscribe <target_pubkey> --relay http://127.0.0.1:9101
+```
+
+`--advertise-host` on `host` matters if you're not on localhost — no NAT
+traversal here, it just tells the relay what address to hand out, real
+reachability is on you. Same point-to-point-known-address limitation
+named earlier in this README, now visible as an actual CLI flag instead of
+just a caveat in prose.
+
+### `download` now runs the actual stack, not just a direct fetch
+
+Until this point, `download` trusted whichever host `discover` found
+first, for free, with no possession check and no reputation. That was the
+real gap flagged after the last round of shell bug-fixes: the auction
+(`poc_challenge_auction.py`), the reputation/trust-graph layer
+(`poc_reputation.py`), and Lightning settlement (`lightning_settle.py`)
+were all built and validated standalone, but none of them were reachable
+from a real download. Now they are:
+
+1. **Resolve every host** claiming to have the content (`discover` already
+   deduped by event, not by content — `download_with_auction` groups by
+   content_hash so a second host publishing the same file actually gets
+   considered, not silently dropped).
+2. **Possession-challenge each one** — sample-FETCH `k` random chunks
+   (default 3) and verify against the already Merkle-root-checked LEAVES.
+   Scoped deliberately to poc_challenge_auction.py's **Part 1** mechanism
+   (chunk-index challenge), not Part 2's nonce/timing relay-detection —
+   that one needs ground-truth bytes the verifier already trusts, which a
+   first-time downloader doesn't have until *after* this same sampling
+   step. Noted here rather than silently narrowed.
+3. **Auction survivors** by local reputation first, then price — same
+   "challenge gates the auction" shape as the original PoC's naive-vs-
+   gated comparison. A host with no history starts at 0.0, same as
+   everyone; reputation only pulls ahead of price once there's real
+   experience behind it (verified — see below).
+4. **Pay the winner** over a real Lightning HTLC if `--lightning` is given
+   and the price is nonzero — same `lightning_settle.py` regtest demo path
+   as before. Honest limitation, not glossed over: this settles with the
+   fixed alice/bob demo nodes, not a general "pay this specific host's own
+   Lightning node" protocol — that would need hosts to serve their own
+   real BOLT11 invoices, which isn't built.
+5. **Download and record** — same chunk-verified `download()` as before,
+   then the outcome (pass/fail, latency) gets written to
+   `~/.weed_reputation.json` via `ReputationStore.record_direct`, so the
+   next auction for this host starts from real history instead of 0.0.
+
+Real test, two independent hosts (separate identities, separate `HOME`s so
+podman's rootless state didn't collide) serving the same real video —
+one free, one priced at 500 sat:
+
+```
+found 2 candidate host(s) for 7f2477c7ea675004...
+  + 127.0.0.1:9202: possession verified (3/3 chunks), price=0 sat, reputation=0.00, avg_latency=3.4ms
+  + 127.0.0.1:9203: possession verified (3/3 chunks), price=500 sat, reputation=0.00, avg_latency=0.7ms
+selected 127.0.0.1:9202 — price 0 sat, reputation 0.00, 3.4ms avg
+```
+
+Ties on reputation, cheapest wins — correct. Then manually seeded
+contrasting reputations (hostA bad, hostB good) to prove reputation
+actually *overrides* price rather than the selection just always
+defaulting to cheapest:
+
+```
++ 127.0.0.1:9202: ... price=0 sat, reputation=0.10 ...
++ 127.0.0.1:9203: ... price=500 sat, reputation=1.00 ...
+selected 127.0.0.1:9203 — price 500 sat, reputation 1.00, 0.4ms avg
+```
+
+Picked the pricier, more trusted host. Then killed the free host outright
+and re-ran with `--lightning`: real HTLC settled (500 sat, preimage
+independently re-verified against the invoice's own `r_hash` via
+`lncli listinvoices`, not just trusting the printed claim), download
+proceeded, byte-identical against the source via `cmp`.
+
+```bash
+python3 weed.py download <content_hash> --relay http://127.0.0.1:9101 \
+    --rounds 5 --lightning --out downloaded.mp4
+```
+
+`host --price N` sets what a host charges (sats, default free — `PRICE` is
+a new wire-protocol verb, backward compatible: a host that doesn't
+implement it just gets treated as free by an older/newer client either way).
+
+### Transitive trust — real attestations flow through the subscribe graph
+
+The gap named right after the auction landed: `select_host` was only ever
+passed *your own* direct history, never other people's signed vouches, so
+a host you'd genuinely never dealt with always scored a flat 0.0 no matter
+who else had already verified it. `poc_reputation.py`'s attestation/
+revocation machinery could support exactly this — it just was never fed
+anything, since attestations were never gossiped through a relay the way
+publish/like/subscribe already were.
+
+Considered adopting real PGP for this (Ryan asked) — decided against it:
+PGP's actual trust-level/path-counting *idea* (marginal vs full trust,
+computed transitively) is worth borrowing, but the OpenPGP *format* is
+built for signing emails, not cheaply gossiping dozens of small JSON
+events, and its classic path to real interoperability — public keyservers
+— reintroduces exactly the single-point-of-failure problem discovery.py's
+relay design exists to avoid. Built the trust-level idea on the Ed25519
+signing already in place instead of adopting the standard.
+
+`build_trust_graph()`: real BFS outward from your own pubkey through real
+signed `subscribe` events pulled from a relay (a subscribe *is* a trust
+edge — same insight from the original discovery-layer design, now actually
+computed transitively instead of 1-hop-only). Trust decays per hop
+(default 0.5×) — a friend counts fully, a friend-of-a-friend counts less.
+Takes the shortest path to each reachable pubkey, not the sum across every
+path — summing would let a sybil ring inflate a target's trust just by
+adding more low-value paths to it.
+
+`download_with_auction` now also pulls real `attestation` events from
+relays (new event type — no relay code changes needed, `discovery_relay.py`
+already verifies and filters by type generically), verifies each one, and
+feeds them into `select_host` alongside the trust graph. And after every
+successful download it publishes its own outcome as a real signed
+attestation, not just recording it locally — so the next person who trusts
+*you*, even transitively, benefits without ever dealing with that host
+first.
+
+Real end-to-end proof, not just the math: identity A downloaded from a
+host directly (0.00 reputation, no history, same as before this change),
+which auto-published a real attestation. A second identity, ROOT, who had
+**never talked to that host**, subscribed to A (one real signed edge), then
+ran `download`:
+
+```
+trust graph: 1 pubkey(s) reachable within 3 hop(s) of your own subscribes
+pulled 1/1 real attestation(s) from relays (others' vouches, weighted by your trust in whoever signed them)
+  + 127.0.0.1:9204: possession verified (3/3 chunks), price=0 sat, reputation=1.00 (1 attestation(s), weighted by signer trust + age), avg_latency=0.4ms
+```
+
+Would've been a flat `reputation=0.00` before this change — ROOT had zero
+direct history with the host. Instead it inherited A's real, already-
+verified experience through one real hop of trust. Byte-identical download
+confirmed via `cmp`, same as every other download in this repo.
+
+Separately verified the decay math itself against a real 3-hop chain
+(root→A→B→C) plus a disconnected stranger: `A=0.5, B=0.25, C=0.125`,
+stranger absent from the graph entirely, `max_hops` correctly bounding how
+far it searches — exact, not approximate.
+
+### `tunnel_relay.py` + persistent sessions in `node.py` — NAT traversal
+
+`host --advertise-host`'s own help text used to say it outright: "no NAT
+traversal here." Real gap — almost nobody has a directly reachable
+inbound port. Fixed with a relay-mediated tunnel rather than real
+STUN/ICE hole-punching: works behind *any* NAT including CGNAT (both
+sides only ever make outbound connections, so there's nothing for a
+firewall to block), at the honest cost of relay bandwidth/latency and
+someone having to run `tunnel_relay.py` somewhere reachable — same
+operational shape as already running a discovery relay, not a new kind of
+problem.
+
+Rendezvous protocol (hand-rolled to match this repo's own line-based wire
+protocol rather than pulling in an external tunnel tool like `bore`):
+a NAT'd host opens one persistent outbound `REGISTER <token>` control
+connection; a downloader connects and sends `CONNECT <token>`; the relay
+asks the host to dial back (`NEWSTREAM <stream_id>` on the control
+channel, `DATA <stream_id>` as the reply) and, once paired, does nothing
+but shovel raw bytes between the two sockets — same "dumb relay, no
+opinion on the payload" design `discovery_relay.py` already uses, just
+for bytes instead of signed JSON events. `token` is the archive's
+`content_hash`; the relay has no idea what it means, same as everywhere
+else in this repo that keys off it.
+
+Had to fix a real prerequisite bug first, not just add the relay: every
+wire-protocol command (`INFO`, `LEAVES`, one `FETCH` per chunk — 3324 of
+them for the real archive) used to open a brand-new TCP connection.
+Cheap directly, fatal through a relay — every single chunk would pay a
+full rendezvous round-trip before any bytes moved. Fixed by giving both
+sides a persistent session (`HostConnection` client-side, `serve_session`
+host-side, looping over many commands per connection instead of one) —
+a net win for direct connections too, not just a tunnel workaround.
+
+Second real bug, caught while verifying this against real sockets, not
+just in review: Python's `socketserver.ThreadingMixIn` closes a
+connection's socket the instant its handler function *returns* — a
+handler that spawns a pipe thread and returns immediately gets its own
+socket killed out from under that thread mid-transfer. Fixed by having
+each handler thread block for the tunneled session's full lifetime
+(`_Pairing`'s `ready`/`done` events in `tunnel_relay.py`) instead of
+firing off detached threads and returning early. First attempt failed
+with `json.decoder.JSONDecodeError: Expecting value` — an empty response,
+not a corrupted one, which is exactly what a socket closed mid-read looks
+like.
+
+Real end-to-end proof: hosted the real 217MB/3324-chunk archive,
+advertised at a deliberately unreachable address (`10.255.255.1`, not
+localhost) so there was no possibility of a direct connection carrying
+the download, `--tunnel 127.0.0.1:9199`. Downloaded entirely through the
+tunnel, `cmp`-verified byte-identical against the source, 3324 chunks in
+1.4s — comparable to a direct download, not a meaningfully slower path.
+
+```bash
+# terminal 1
+python3 discovery_relay.py 9101
+# terminal 2
+python3 tunnel_relay.py 9199
+# terminal 3 — no reachable --advertise-host at all
+python3 weed.py host real_archive --port 9201 --tunnel 127.0.0.1:9199 \
+    --relay http://127.0.0.1:9101 --advertise-host 10.255.255.1
+# terminal 4
+python3 weed.py download <content_hash> --relay http://127.0.0.1:9101 --out downloaded.mp4
+```
+
+Additive, backward-compatible protocol change, same shape as the
+optional `PRICE` wire verb: `publish()` gained an optional `tunnel`
+field (`'relay_host:relay_port'` or absent); a candidate without it is
+just connected to directly, exactly as before this existed.
+
+**TLS**, for tunnel relays that terminate it at the edge instead of
+speaking it themselves (deployed one on Fly with `handlers = ["tls"]` —
+`fly.tunnel-relay.toml` + `Dockerfile.tunnel-relay` in this repo): prefix
+the address with `tls://` — `--tunnel tls://tunnel.example.com:9199`.
+`tunnel_relay.py` itself never changes; edge termination decrypts before
+the bytes ever reach it, so only the two ends that actually cross the
+public internet (the host's `REGISTER`/`DATA` connections, a
+downloader's `CONNECT`) wrap the socket in
+`ssl.create_default_context()` — real CA validation by default, not a
+weakened check. Verified against a real self-signed-cert TLS-terminating
+proxy built specifically to test this (mimicking exactly what Fly's edge
+does): confirmed the client correctly *rejects* an untrusted cert before
+trusting it, then a full host→tunnel→download round-trip over the
+encrypted path, byte-identical result.
+
+**Heartbeat**, for the same idle-connection problem real deployments
+actually hit: the `REGISTER` control connection sends nothing between
+registering and the first real download, sometimes for a long time.
+Fly's own edge (confirmed from real production logs, not a guess) resets
+TCP connections idle more than a few minutes, which silently
+unregistered the host with no error until the next download failed.
+Fixed with a small periodic `PING` on the control connection —
+`tunnel_relay.py` needed zero changes, its `REGISTER` loop already
+discards anything it receives that isn't relevant. Proved it against a
+real idle-enforcing test server (3-second idle limit): without the
+heartbeat, killed at exactly 3.0s; with it, survived 8 full seconds with
+no kill event at all.
+
+### `dht.py` — real Kademlia DHT discovery, no relay required
+
+Every other discovery path in this repo needs a relay URL, told to you
+out of band — real, and a real limitation, until now: `dht.py` answers
+"how do two nodes find each other with no shared server" using an actual
+Kademlia DHT, via the real `kademlia` PyPI library rather than
+reimplementing node-IDs/k-buckets/RPC routing from scratch. Scoped
+honestly: this covers `announce(content_hash, host_addr)` /
+`lookup(content_hash)` — not the richer signed-event system
+(publish/like/subscribe/attestation/trust-graph) `discovery_relay.py`
+already handles, since Kademlia's plain key→value store isn't a natural
+fit for an append-only event log. That richer system staying on relays
+is a deliberate scope boundary, not an oversight.
+
+Multiple announcers of the same content are merged, not overwritten —
+`kademlia`'s `set()` is single-value-per-key, so a naive announce would
+silently drop everyone else's listing; `_announce()` fetches, merges,
+re-announcing from the same host again correctly doesn't duplicate.
+
+Proved with three separate, escalating real tests: three chained nodes
+(C only ever bootstrapped through B, never spoke to A directly) — content
+announced on A was found from C, real Kademlia routing, not a shared-
+memory illusion. Two different hosts announcing the same content — both
+preserved. And the strongest one: a node announced content, then that
+node's *entire process exited* — a completely independent fourth process,
+knowing only the original bootstrap node, still found what was announced.
+Real value replication surviving the announcer going offline, which is
+the actual point of a DHT over a relay.
+
+```bash
+python3 dht.py 8468                          # first node, new swarm
+python3 dht.py 8469 127.0.0.1:8468           # second node, joins the first
+```
+
+Or from the shell: `dht start [port] [bootstrap_host:port]`,
+`dht announce <content_hash> <host:port> [title]`, `dht lookup <content_hash>`.
+
+### `web_ui.py` — local web UI
+
+Hosting/discovering/downloading/liking/subscribing all required
+memorizing `weed.py`'s CLI flags — real friction for anyone who isn't
+already comfortable with argparse. `web_ui.py` is a small stdlib
+`http.server`/`ThreadingHTTPServer` JSON API — same tool
+`discovery_relay.py` already uses, no new dependency, still just the
+three packages in `requirements.txt` — wrapping the exact same
+`node.py` functions the CLI calls, no reimplemented protocol logic. The
+frontend (`web/`) is a single static page, vanilla JS, no build step —
+matches the rest of this repo's no-toolchain style.
+
+Binds `127.0.0.1` by default on purpose: a local control surface, not
+something meant to face the internet, and there's no auth built — same
+"reachability is on you" honesty `--advertise-host`'s docs already apply
+elsewhere. `--bind` widens it at your own risk.
+
+Endpoints, all thin wrappers: `GET /api/whoami`, `GET /api/discover`,
+`POST /api/host` (backgrounds `run_host_server`/`run_host_tunnel` the
+same way `shell.py`'s `do_host` already does) + `GET /api/hosts`,
+`POST /api/download` + `GET /api/download/<job_id>` for polling progress
+(`download()` gained an optional `on_progress(idx, n_chunks)` callback,
+default no-op, so CLI output is unaffected), `POST /api/like`,
+`POST /api/subscribe`, `GET /api/reputation/<pubkey>`,
+`GET /api/stream/<job_id>`, `GET /api/qr?data=...`, `GET /api/lan-url`.
+
+Real end-to-end proof, not just the API responding: hosted a file
+through the **Host** form, confirmed a second terminal's `weed discover`
+actually saw it (proves the API called the real `node.publish`, not a
+mock); downloaded through the **Downloads** form with a live-polling
+progress bar, `cmp`-verified byte-identical; liked and subscribed
+through the UI, confirmed the real signed events landed on the relay by
+querying it directly.
+
+**Streaming** (`/api/stream/<job_id>`) — the actual gap behind "play
+media on my phone": the UI could already trigger a download, but the
+bytes only ever landed on this server's disk, never reached the browser.
+Real HTTP range support (`Accept-Ranges`, `206 Partial Content`, `416`
+for out-of-range), so a `<video>` tag can seek instead of downloading
+blind. Verified with real range requests against a real completed
+download — full fetch, a mid-file range, and an open-ended range, all
+byte-exact against the source; error paths (out-of-range, unknown job)
+checked too.
+
+**QR codes** (`/api/qr`, terminal QR on startup) — same `qrcode` package
+`ott`'s own `ott qr` already uses. `--bind 0.0.0.0` auto-detects the real
+LAN IP (a UDP-route trick, no packet actually sent) instead of printing
+the useless literal `0.0.0.0`, so the printed/scanned URL is one a phone
+can actually reach — and it's computed server-side and exposed via
+`/api/lan-url` specifically because the browser's own
+`location.origin` lies the instant you load the page via `localhost`
+instead of the LAN address; a real bug this caused (the header's "open
+on phone" QR pointing at `127.0.0.1`) was reproduced and fixed by having
+the client fetch the server's own answer instead of trusting
+`location.origin`.
+
+```bash
+python3 weed.py serve                # alias for `web`, positional args: serve [bind] [port]
+python3 weed.py serve 0.0.0.0 8080    # reachable from your phone; prints a scan-to-open QR
+# or, from the shell: `serve [bind] [port]`
+```
+
+### `shell.py` — interactive, tab-completing, same pattern as `ott`'s shell
+
+`python3 weed.py` with no arguments (or `weed.py shell`) drops into an
+interactive shell — same `cmd.Cmd` + readline pattern as `ott`'s own shell,
+same conventions: short aliases (`w`/`h`/`r`/`disc`/`dl`/`l`/`sub`), `help`
+or `?` for commands, `Ctrl-D` or `q` to exit, tab completes.
+
+`relay` runs a real discovery relay in the background too, same pattern as
+`host` — the whole flow (relay, host, discover, download, like, subscribe)
+runs from one shell session, no second terminal required. Discovered
+running it that way for real (`host` with no relay running produces
+"unreachable, skipped, nothing found," `discover` alone can't conjure a
+relay that isn't there — real friction that surfaced from actually using
+it, not a hypothetical). `relay` also sets itself as the session's default
+relay, so `discover`/`download`/`like`/`subscribe` don't need `--relay`
+repeated every time. `host` also takes `--tunnel [tls://]RELAY_HOST:PORT`
+now, same as the CLI — see `tunnel_relay.py` above. `serve [bind] [port]`
+runs the web UI in the background without a second terminal — see
+`web_ui.py` above. `dht start/announce/lookup` runs a real Kademlia node
+in the background — see `dht.py` above. Every background command
+(`host`, `relay`, `serve`, `dht start`) goes through a shared `_bg()`
+wrapper now: a real failure inside one of those threads used to dump a
+raw Python traceback into the middle of the prompt (an uncaught
+exception in a background thread was never covered by `onecmd()`'s own
+try/except, which only wraps the synchronous part of a command) — caught
+live from a real `--tunnel ~/share` typo, fixed, reproduced the same
+crash again afterward to confirm it now prints a clean `✗ ...` line
+instead and the shell stays fully usable.
+
+Completion resolves against real state, not a fixed list — same idea as
+`ott`'s completions (which complete against the real archive). `download`
+and `like` tab-complete against content hashes actually seen in the last
+`discover`; `subscribe` completes against pubkeys actually seen:
+
+```
+weed> relay
+  relay running on port 9101 in the background — set as your default relay
+weed> host real_archive --relay http://127.0.0.1:9101
+  hosting real_video.mp4 on port 9201 in the background — shell still usable
+weed> discover
+  'real_video.mp4'   hash=7f2477c7ea675004...  host=127.0.0.1:9201  by=409a15dcfc59...
+weed> download 7f24<TAB>
+7f2477c7ea675004ad5dbab6dc7c44327c724b880cc389807df1965b77966acc
+weed> download 7f2477c7ea675004ad5dbab6dc7c44327c724b880cc389807df1965b77966acc
+3324 chunks downloaded and verified in 1.4s
+```
+
+`host` runs the server in a background thread instead of blocking the
+shell — genuinely new, not copied from `ott`, since nothing in `ott`
+blocks forever the way a hosting server does. Ran this exact sequence for
+real (scripted, not just described): host → discover → download →
+`cmp`-verified byte-identical → `like`, all in one shell session, download
+still finishing in 1.4s with the server running in the background thread
+the whole time.
+
+## Running it
+
+`./weed.py --help` (or `weed.py lightning --help` for the nested ones) is
+the friendliest entry point — it's a thin argparse wrapper over the
+Makefile, same targets, real subcommands and `--help` text instead of
+needing to remember `make` target names. `make help` lists the same
+targets directly. Or run any command below on its own:
+
+```bash
+python3 poc_challenge_auction.py          # in-process, Parts 1 + 2, narrated
+python3 poc_network_challenge.py          # real sockets, single-shot rounds, loopback
+python3 poc_network_challenge.py stats    # real sockets, repeated-challenge separation, loopback
+python3 poc_reputation.py                 # real Ed25519 signing/verification demo
+python3 viz_challenge_separation.py       # regenerates the chart above from fresh data
+docker compose up --build --abort-on-container-exit verifier   # same test, real containers
+cd lightning && docker compose up -d && cd ..   # real bitcoind + 2 LND nodes (see lightning/README.md for setup)
+python3 poc_challenge_auction.py --lightning    # same auction, real HTLC settlement
+python3 poc_real_archive_challenge.py           # same challenge mechanism, real 3324-chunk video
+python3 poc_discovery.py                        # 3 real relays, personalized ranking, sybil test
+python3 tunnel_relay.py 9199                    # NAT-traversal relay — see host --tunnel below
+python3 weed.py serve --bind 0.0.0.0 --port 8080  # local web UI, reachable from your phone
+python3 dht.py 8468                             # real Kademlia DHT node — see dht.py above
+```
+
+`pip install -r requirements.txt` gets everything (`btcvm`, `cryptography`,
+`matplotlib`, `kademlia`). Broken down: `poc_challenge_auction.py` (and
+`poc_real_archive_challenge.py`, which imports from it) needs `btcvm`;
+`poc_reputation.py` and `poc_discovery.py` (which imports from it) need
+`cryptography`; `viz_challenge_separation.py` needs `matplotlib`;
+`dht.py` needs `kademlia`. `poc_network_challenge.py` and
+`discovery_relay.py` are pure stdlib, no install needed. `qrcode` is
+optional (`pip install qrcode`) — `web_ui.py`'s QR endpoints degrade to a
+plain URL, printed instead of rendered, if it's missing. Docker/Compose
+needed for the container-network test and for `--lightning` (real
+bitcoind + LND, see `lightning/README.md`).
+
+Or, packaged: `pip install -e .` (see `pyproject.toml`) installs a real
+`weed` command on your `PATH` instead of `python3 weed.py`.
+
+## Next steps
+
+1. ~~Nonce-salted challenge + timing bound~~ — done, `poc_challenge_auction.py` Part 2
+2. ~~Real network round-trip instead of in-process~~ — done, `poc_network_challenge.py`
+3. ~~Local reputation + signed portable attestations~~ — done, `poc_reputation.py`
+4. ~~Real WAN calibration against an actual second machine~~ — done, real
+   RunPod box over an SSH tunnel: ~1700x gap, separates at k=1
+5. ~~Real Lightning HTLC settlement~~ — done, `lightning_settle.py` +
+   `lightning/` (real bitcoind + 2 LND nodes, real BOLT11 invoices, real
+   preimage reveal independently re-verified). Regtest, not public testnet —
+   same reasoning as #4: real protocol code, skip the wait on chain
+   sync/faucets.
+6. ~~Point the mechanism at a real `.ott` archive~~ — done,
+   `poc_real_archive_challenge.py`: real 217MB video, 3324 real chunks,
+   12-step proofs (~400B), confirmed O(log N) not linear.
+7. ~~Attestation revocation~~ — done, `poc_reputation.py`: signer-only
+   revocation keyed to `attestation_id`, forged revocation from a different
+   signer correctly rejected, revoked attestation kept on record not deleted.
+8. ~~Discovery layer~~ — done, `discovery_relay.py` + `poc_discovery.py`:
+   3 independent dumb relays, personalized client-side ranking from each
+   client's own subscribe graph, sybil-resistant (20 fake identities move
+   neither client's score), real relay-death test (content survives,
+   anything posted only to the dead relay doesn't — redundancy isn't free).
+9. ~~NAT traversal~~ — done, `tunnel_relay.py`: relay-mediated rendezvous
+   (not real STUN/ICE hole-punching), real 217MB/3324-chunk archive
+   downloaded end-to-end through the tunnel with the host advertised at
+   an unreachable address, byte-identical. Required a persistent-session
+   refactor of `node.py`'s wire protocol first — see above.
+10. ~~Local UI~~ — done, `web_ui.py` + `web/`: stdlib JSON API wrapping
+    the same `node.py` functions the CLI calls, static vanilla-JS
+    frontend, no new dependency. Host/discover/download/like/subscribe
+    all verified working from the browser, not just the API responding.
+11. ~~Real P2P/DHT discovery~~ — done, `dht.py`: real Kademlia via the
+    `kademlia` library, not reimplemented. Content survived the
+    announcing node's process exiting entirely, found by a fourth,
+    independent process that only knew the original bootstrap node — real
+    value replication, not a two-party memory trick. Scoped honestly:
+    covers host-discovery only, not the richer event system.
+12. ~~TLS for tunnel relays~~ — done, `tls://` prefix on `--tunnel`,
+    verified against a real self-signed-cert TLS-terminating proxy built
+    specifically to test it, full host→tunnel→download round-trip over
+    the encrypted path.
+13. ~~Play media from the web UI~~ — done, `/api/stream` with real HTTP
+    range support, verified byte-exact against full/mid-file/open-ended
+    range requests.
+14. ~~Multi-file hosting~~ — done, `host <dir>` (no `--file`) serves every
+    archived file over one port, downloader `SELECT`s by content hash.
+    Fixes the original bug: a 45-file archive silently collapsed to
+    whichever file happened to be last in the manifest.
+
+Every item on the original roadmap is now built and verified against real
+output, not just designed — and `node.py` (below) wires host/discover/
+download/like/subscribe into one real tool instead of six disconnected
+demos. What's left is scaling and hardening this, not proving the
+mechanisms work — see each section above for the honest edges that are
+still real constraints even though the core ideas held up: loopback
+timing separation isn't airtight without averaging, relay death loses
+non-redundant data, RunPod flakiness, regtest-only Lightning, the DHT
+covers host-discovery but not the richer signed-event system, the tunnel
+relay (even with TLS) is still a single point of failure/bandwidth cost
+with no redundancy story the way discovery relays have, and the web UI
+has no auth, local-only by design.