socketduct 0.1.15

package/docs/security.md

@@ -0,0 +1,258 @@
+# Security Model
+
+Socketduct must not become an open proxy.
+
+## Authentication
+
+Authentication is required before any session operation:
+
+- `session.create`
+- `session.resume`
+- stream attachment
+- admin/user management
+
+Initial implementation can use local users with password hashes.
+
+Preferred password storage:
+
+1. Argon2id
+2. scrypt
+3. bcrypt
+
+Never store plaintext passwords.
+
+## Authorization
+
+Every session belongs to one user.
+
+A user may only:
+
+- create sessions within their allowed destination policy
+- resume their own sessions
+- close their own sessions
+- inspect their own session status
+
+Admin operations should be separate from normal tunnel operations.
+
+## Destination policy
+
+Default should be deny-by-default for generic proxying.
+
+Protocol v3 authenticates the physical connection before accepting any OPEN,
+DATA or resume operation. Authentication is performed once per physical
+generation, but authorization is deliberately not cached across targets: every
+virtual stream OPEN passes its exact `{host, port}` and authenticated principal
+through the deny-by-default target policy. A denied/open-failed stream is reset
+without affecting authorized siblings. Stream ID zero cannot carry application
+data.
+
+The current MVP relay is not a generic open proxy: every target is supplied in `session.create`, validated by an injected authorization function such as `authorizeTarget({principal, target})`, and stored on the session. Stream data is rejected until a `session.create` or `session.resume` message authenticates successfully. Broader proxy modes must keep this deny-by-default shape with explicit per-user allowlists before accepting arbitrary destinations.
+
+Recommended first policy shape:
+
+```json
+{
+  "allow": [
+    { "host": "10.0.0.15", "port": 22 },
+    { "host": "example.com", "port": 443 }
+  ]
+}
+```
+
+Later:
+
+- CIDR allowlists
+- DNS suffix allowlists
+- per-user policies
+- max bandwidth/session limits
+
+## Tokens
+
+After username/password login, the server may issue a short-lived reconnect token.
+
+Token properties:
+
+- random opaque value
+- bound to user id
+- optionally bound to session id
+- expiration time
+- revocable
+
+Do not log token values.
+
+## Spool files
+
+Spool files may contain application traffic and must be treated as sensitive.
+
+Requirements:
+
+- owner-only file permissions (`0600`) where supported
+- predictable directory but unpredictable filenames or session-owned paths
+- no world-readable temp files
+- cleanup on close/expiry
+- safe startup cleanup of abandoned files
+
+Optional later hardening:
+
+- encrypt spool files at rest
+- per-session spool keys
+- authenticated checksums
+
+## Logging
+
+The optional persistent JSONL logger uses a closed event/field allowlist. It
+records process/role identity, lifecycle state, bounded counters, validated
+gateway IDs, and normalized error codes/classes. Unknown event names are
+dropped and unknown error codes become `UNKNOWN`.
+
+It never records protocol passwords, tokens, authentication frames, headers,
+certificates or keys, environment values, session/stream names or identifiers,
+spool paths/contents, raw application payload, or arbitrary objects. Error
+diagnostics retain bounded error names, messages, and stacks so operational
+failures can be investigated while credential-shaped values are redacted. Each
+JSON line is capped at 1024 bytes. Logger I/O and custom-writer failures are
+isolated from network behavior and produce only one bounded stderr
+classification. See [Persistent structured logging](logging.md) for
+configuration and rotation.
+
+V3 status is payload-free and credential-free: physical counts/generation,
+virtual stream state counts, active local targets, bounded queued/spool bytes,
+resume outcomes, stable reset-reason counters and aggregate stream-open latency.
+It never includes bearer tokens, passwords, target credentials, certificate/key
+contents, stream names, payloads or spool contents.
+
+Multi-gateway transport-set status follows the same boundary. It exposes only
+configured gateway IDs, lifecycle/counter totals, assignment counts and bounded
+byte counts. It never exposes relay or target addresses, bearer tokens, TLS
+material, stream identifiers/names, payloads or spool contents. Gateway IDs are
+restricted to short URL-safe values; opaque per-gateway connection identities
+and spool subdirectories are derived with SHA-256 from the stable base identity
+and gateway ID, preventing path traversal and cross-gateway spool collisions.
+
+Transport-set lifecycle events include only normalized error codes and classes;
+they never include arbitrary member error names, messages, or stacks.
+
+Each gateway entry still authenticates its own v3 physical connection before
+any target stream opens, and every target is authorized by its selected relay.
+The set does not add unauthenticated fallback, anonymous members, or migration
+to a sibling after an uncertain write. Shared process byte reservations prevent
+multiple gateway members from multiplying the intended memory/spool ceiling.
+Durable spool reservations remain charged until truncate/remove succeeds; a
+cleanup failure is counted and closes the affected member without exposing its
+stream identity, path or contents. Recovery requires the original configured
+gateway ID before startup, preventing durable state from being probed or moved
+through a sibling path.
+
+Native reverse mode adds two independent authorization decisions. The hub
+binds each configured gateway ID to an authenticated principal and authorizes
+every client/gateway/target route. The selected home gateway then checks the
+exact target against its own local deny-by-default allowlist. A gateway ID
+cannot be claimed by another stable connection identity, stale generations are
+rejected, and a newer generation from the same identity fences the old owner.
+Reverse transports require verified TLS and negotiated zstd. There is no
+plaintext or certificate-verification opt-out. Docker port 2375 is permitted
+only as a private `docker-server:2375` target and is never host-published.
+
+Reverse registration can reconnect; established Docker streams cannot resume
+or spool across loss. They fail with an explicit uncertain-operation error so
+the caller must reconcile before retrying. This prevents privileged Docker API
+writes from being applied twice.
+
+## Network exposure
+
+Defaults should be conservative:
+
+- edge local proxy listens on `127.0.0.1` unless configured otherwise
+- relay tunnel endpoint listens on configured public interface
+- admin API listens on `127.0.0.1` unless explicitly exposed
+- require TLS or equivalent transport security before real use
+
+Current implementation notes:
+
+- `startTcpEdge` and `startTcpRelay` default to `127.0.0.1`.
+- `startTcpRelay` has no static upstream target; callers must provide target authorization.
+- `startTcpEdge` includes the requested target in `session.create`.
+- The MVP protocol is framed TCP; protect it with TLS (configured via `SOCKETDUCT_TLS_CERT_FILE`/`SOCKETDUCT_TLS_KEY_FILE`) before exposing it on untrusted networks.
+- Tokens are opaque and short-lived in memory, but should still be treated as bearer credentials.
+- Spool chunks contain application payload and are written with owner-only file permissions where the platform supports them.
+
+### TLS
+
+For production deployment, configure TLS on the relay listener:
+
+- Set `SOCKETDUCT_TLS_CERT_FILE` and `SOCKETDUCT_TLS_KEY_FILE` as an atomic pair pointing to PEM-encoded certificate and private key files. Both must be set or neither.
+- Optionally set `SOCKETDUCT_TLS_CLIENT_CA_FILE` to a PEM-encoded client CA certificate to enable mutual TLS (mTLS). When set, clients must present a certificate signed by this CA.
+- Certificate material is read at runtime with `readFileSync` and never logged. The relay fails fast on missing or unreadable certificate files.
+- The relay enforces `rejectUnauthorized: true` for all TLS connections by default.
+- When mTLS is enabled (`SOCKETDUCT_TLS_CLIENT_CA_FILE`), both `requestCert: true` and `rejectUnauthorized: true` are set.
+
+### Insecure public opt-out
+
+Binding to a non-loopback address (any interface other than `127.0.0.0/8`, `::1`, or `localhost`) without TLS is refused at startup unless you explicitly opt out:
+
+- Set `SOCKETDUCT_ALLOW_INSECURE_PUBLIC=1` or `SOCKETDUCT_ALLOW_INSECURE_PUBLIC=true`.
+- Any other value is rejected with a configuration error.
+- Loopback plaintext binds remain compatible without this flag for local development and testing.
+
+### Client-side TLS
+
+The raw client (`connectSocketduct`, `SocketductSocket`, `SocketductHttpAgent`, `createSocketductConnectionFactory`) accepts an optional `tls` configuration for the edge-to-relay connection with:
+
+- `caFile`: path to a PEM-encoded CA certificate to verify the relay server certificate.
+- `certFile` and `keyFile`: an atomic pair of PEM-encoded client certificate and private key for mTLS. Both must be set or neither. Used when the relay requires client certificates.
+- `servername`: SNI hostname for certificate identity validation. Defaults to `relayHost` by Node.js but can be overridden when the certificate CN/SAN differs from the connection host.
+- `rejectUnauthorized`: defaults to `true`. Set to `false` only for testing.
+
+`SocketductHttpsAgent` uses the clearer name `relayTls` for those relay settings and a separate `targetTls` object for target `ca`, client `cert`/`key`, `rejectUnauthorized`, and certificate `servername`. It never logs certificate or key material. Target verification defaults to strict, client certificate/key must be supplied atomically, and IP identities are verified without sending an IP literal as SNI.
+
+Initial target retry does not weaken authorization: the requested destination is authenticated and allowlisted once before the session enters `creating`, and every attempt uses that stored target. No stream frames are accepted before readiness. After `session.created`, target failure and uncertain application writes are terminal to prevent duplicated privileged operations.
+
+### Migration and rollback
+
+Protocol v3 migration is explicit because v2 and v3 use different first bytes.
+Set `SOCKETDUCT_PROTOCOL_VERSION=3` on the relay and use
+`MultiplexedSocketductConnection` clients together. Keep the old v2 listener or
+deployment available during a staged migration. Rollback consists of selecting
+version `2` (the runtime default) and switching clients back to
+`connectSocketduct`/the legacy agents. Never attempt an unauthenticated framing
+fallback on the same connection.
+
+To migrate an existing plaintext deployment to TLS:
+
+1. Generate a TLS certificate and key pair for the relay hostname.
+2. Place the certificate and key files where the relay container can read them (e.g., mount into `/certs`).
+3. Set `SOCKETDUCT_TLS_CERT_FILE` and `SOCKETDUCT_TLS_KEY_FILE` in `.env`.
+4. Mount the certificate directory in `docker-compose.yml` (`SOCKETDUCT_TLS_CERT_DIR` or the default `./certs`).
+5. Update edges to use TLS by providing the server CA certificate via `tls.caFile`.
+6. Set the verified digest-pinned `SOCKETDUCT_IMAGE` and restart the relay with
+   `npm run gateway:deploy` (or add `-- --docker-host` when using the private
+   Docker socket shim).
+7. Verify edges connect successfully with TLS.
+8. Once all edges are on TLS, optionally enable mTLS by setting `SOCKETDUCT_TLS_CLIENT_CA_FILE`.
+
+To roll back TLS: remove the TLS env vars from `.env`. If the relay binds to a non-loopback address (`0.0.0.0`), you must set `SOCKETDUCT_ALLOW_INSECURE_PUBLIC=1` to start the relay without TLS on a public interface. For a loopback-only relay, plaintext works without the opt-out flag. Re-apply the previously verified image digest with the deployment command and restart edges without TLS options. Image rollback itself never uses a tag: change `SOCKETDUCT_IMAGE` back to the prior digest and re-apply.
+
+### Threats to explicitly test
+
+- unauthenticated session create is rejected
+- wrong password is rejected
+- user A cannot resume user B's session
+- unknown identifier is rejected
+- replaying an old spool file does not duplicate already-acked data
+- spool path traversal is impossible
+- destination not in allowlist is rejected
+- non-loopback public bind without TLS or explicit opt-out is rejected at startup
+- SOCKETDUCT_TLS_CERT_FILE without SOCKETDUCT_TLS_KEY_FILE (or vice versa) fails at config parse
+- wrong CA on the client side is rejected during TLS handshake
+- mTLS with a missing or untrusted client certificate is rejected
+- plaintext clients can still connect to a plaintext relay
+- TLS reconnection/replay preserves spooled data across TLS session resumption
+- unauthenticated connections are terminated after the configurable handshake timeout
+- connections exceeding the per-connection input byte budget before authentication are disconnected
+- an initially unavailable authorized target is retried only before `session.created`, with abort/close returning all resources to baseline
+- target HTTPS rejects a wrong CA or hostname and target mTLS rejects a missing/untrusted client certificate
+- a slow/partial frame (fragmented uint32, incomplete header, incomplete payload) does not accumulate unbounded memory
+- reverse registration rejects wrong principals, duplicate IDs, and stale generations
+- reverse clients reject unknown/unavailable gateway IDs and wrong TLS CAs
+- both hub route policy and the home-local allowlist deny unauthorized Docker targets
+- gateway loss cleans every stream and target without replay, then a fresh stream works after re-registration

package/docs/session-lifecycle.md

@@ -0,0 +1,127 @@
+# Session Lifecycle
+
+## Protocol-v3 connection and stream lifecycle
+
+V3 separates the physical state (`connecting -> authenticated -> ready ->
+interrupted -> resuming -> ready`) from each virtual stream (`opening -> open ->
+half-closed -> closed`, with terminal `reset`). One transport interruption moves
+all active streams into the same connection-wide resume attempt; it does not
+spawn per-stream reconnects.
+
+## Reverse-gateway lifecycle
+
+An outbound gateway moves through `connecting -> authenticated -> registering
+-> ready`. Transport loss destroys every local target stream, advances the
+gateway generation, and reconnects with bounded exponential delay. Selected
+Server3 client members leave readiness when that gateway disappears and rejoin
+only after it registers again.
+
+Reverse stream loss is terminal even when registration later recovers. No
+established Docker bytes are persisted or replayed because their outcome may
+already be committed at dockerd. The application receives
+`GATEWAY_DISCONNECTED`, reconciles the operation, and opens a new stream.
+
+The bounded, paginated resume manifest carries stable identifiers and both
+directional acknowledgement positions. The relay returns an independent result
+per stream. A stream omitted because it was closed while disconnected is
+cleaned up at the relay. A target lost during interruption returns a stream-local
+resume reset and a typed session-loss error, while retained siblings continue.
+Only bytes above the peer acknowledgement are replayed.
+
+## States
+
+```text
+creating -> active -> interrupted -> resuming -> active
+                          |             |
+                          v             v
+                       expired        closed
+```
+
+Additional terminal state:
+
+```text
+failed
+```
+
+## Create
+
+1. Client connects to relay.
+2. Client authenticates.
+3. Client requests `session.create` with a human-readable `name` and requested `target` `{host, port}`.
+4. Server validates the target through the authenticated user's allowlist policy.
+5. Server creates or accepts a stable opaque `identifier`.
+6. Server associates the session with the authenticated user and target metadata.
+7. The authenticated session remains `creating` while the relay opens its target-side socket.
+8. An initial refusal or per-attempt connect timeout produces no client error or readiness event. The relay retries after approximately 250ms, doubling to a configurable 5s cap while the creating tunnel remains open. Each connect attempt has a separately configurable 10s default bound.
+9. Once target TCP connect succeeds, the relay transitions the session to `active` and returns `session.created` exactly once.
+10. The client emits `connect`; only then can stream data reach the target.
+
+The authentication deadline is cleared after successful authentication and authorization, before target acquisition. A tunnel close or client abort before `session.created` immediately deletes the unannounced session and cancels its target attempt and retry timer; it is not resumable.
+
+This retry is deliberately narrow. It is safe because no application bytes have been accepted by the relay before `session.created`. Once a target has connected and the session has been announced, target close, uncertain writes, overflow, authorization failures, and protocol failures are terminal. Socketduct never reconnects the established target or automatically replays an application operation.
+
+## Normal active streaming
+
+- Every data frame has a monotonic sequence.
+- Receiver sends acknowledgements.
+- Sender keeps unacknowledged chunks in memory and/or spool.
+- Acked chunks can be discarded.
+
+## Interruption
+
+When the edge loses the WAN tunnel:
+
+1. Edge keeps the app-facing socket open if configured limits allow.
+2. Edge marks the session `interrupted`.
+3. Edge writes session metadata and unacknowledged chunks to a temporary spool file.
+4. Edge reconnects in the background.
+5. If local buffers fill, edge applies backpressure to the app-facing socket.
+6. If max interruption time or spool size is exceeded, edge closes the app-facing socket and marks the session `failed` or `expired`.
+
+## Resume
+
+1. Edge reconnects to relay.
+2. Edge authenticates using token or username/password.
+3. Edge sends `session.resume` with the existing `identifier`.
+4. Relay verifies user ownership and session state.
+5. Relay reattaches the new tunnel socket to the existing session and retained target socket when possible.
+6. Relay answers with last known sequence/ack positions.
+7. Edge reads the temporary spool file.
+8. Edge sends spool metadata and unacknowledged frames from the requested sequence.
+9. Relay deduplicates already-seen frames.
+10. Streaming continues.
+11. Edge truncates/deletes acked spool data.
+
+## Clean close
+
+1. Either side sends `session.close`.
+2. When the relay processes a client-requested logical close it answers with
+   `{"type":"session.close","direction":"target-to-client","acknowledged":true}`.
+   A target-originated terminal close omits `acknowledged`; this distinction
+   prevents an already-in-flight terminal notification from being mistaken for
+   confirmation that a later client close reached the relay.
+3. Both sides flush pending stream acknowledgements.
+4. Target/app sockets close.
+5. Spool files are deleted.
+6. Session metadata is marked `closed`.
+
+## Expiry
+
+A session expires when:
+
+- interruption exceeds configured duration
+- spool file exceeds configured age
+- target-side connection closes and cannot be held
+- target hold/resume limits are exceeded
+- user/admin deletes it
+- authentication/token policy invalidates it
+
+## Initial defaults to consider
+
+- Max interruption hold time: 60 seconds.
+- Max spool size per session: 64 MiB.
+- Max sessions per user: 32.
+- Initial target acquisition: 250ms -> 500ms -> 1s -> 2s -> 4s -> 5s, capped.
+- Session identifier: UUIDv7 or 128-bit random URL-safe string.
+
+These are starting points, not final decisions.