@rivetkit/engine 1.0.0 → 2.2.1-pr.4600.b74ff3b

package/contrib-docs/RUNNER_LIFECYCLE.md

@@ -1,172 +0,0 @@
-# Runner Lifecycle
-
-## Connection
-
-```mermaid
-sequenceDiagram
-	participant R as Runner
-	participant P as Pegboard
-	participant RW as Runner Workflow
-
-	note over R,RW: Phase 1: WebSocket Connection
-
-	R->>P: WebSocket open
-	R->>P: ToServerInit (name, version, totalSlots, lastCommandIdx)
-	note over R: start ping interval (3s)
-	note over R: start command ack interval (5min)
-
-	P->>RW: Forward (ToServerInit)
-	note over RW: ProcessInit activity
-	note over RW: load state, process prepopulated actors
-
-	note over R,RW: Phase 2: Initialize Runner
-
-	RW->>P: ToClientInit (runnerId, lastEventIdx, metadata)
-	P->>R: ToClientInit
-
-	note over R: store runnerId
-	note over R: store runnerLostThreshold from metadata
-
-	note over R,RW: Phase 3: Resend Pending State
-
-	note over R: processUnsentKvRequests
-	note over R: resendUnacknowledgedEvents
-	note over R: tunnel.resendBufferedEvents
-
-	note over R,RW: Phase 4: Send Missed Commands
-
-	RW->>P: ToClientCommands (missed commands)
-	P->>R: ToClientCommands
-	note over R: handleCommands
-
-	note over R,RW: Phase 5: Complete Connection
-
-	note over RW: InsertDb activity
-	note over RW: write runner to database
-	note over RW: update allocation indexes
-
-	note over R: config.onConnected callback
-```
-
-## Reconnect
-
-```mermaid
-sequenceDiagram
-	participant R as Runner
-	participant P as Pegboard
-	participant RW as Runner Workflow
-
-	note over R,RW: Phase 1: Detect Disconnection
-
-	alt WebSocket error/close
-		P--xR: connection lost
-		note over R: start runner lost timeout (if threshold configured)
-		note over R: schedule reconnect with backoff
-		note over R: config.onDisconnected callback
-	end
-
-	note over R,RW: Phase 2: Reconnect
-
-	note over R: calculate backoff delay
-	note over R: increment reconnectAttempt counter
-
-	R->>P: WebSocket open (reconnect)
-	R->>P: ToServerInit (lastCommandIdx preserved)
-
-	note over R: clear reconnect timeout
-	note over R: clear runner lost timeout
-	note over R: reset reconnectAttempt = 0
-
-	P->>RW: Forward (ToServerInit)
-	RW->>P: ToClientInit (lastEventIdx)
-	P->>R: ToClientInit
-
-	note over R,RW: Phase 3: Resynchronize
-
-	note over R: if runnerId changed, clear event history
-
-	note over R: processUnsentKvRequests
-	note over R: resendUnacknowledgedEvents (from lastEventIdx)
-	note over R: tunnel.resendBufferedEvents
-
-	alt missed commands exist
-		RW->>P: ToClientCommands (missed commands)
-		P->>R: ToClientCommands
-		note over R: handleCommands
-	end
-
-	note over R: config.onConnected callback
-```
-
-## Shutdown
-
-```mermaid
-sequenceDiagram
-	participant R as Runner
-	participant P as Pegboard
-	participant RW as Runner Workflow
-	participant A as Actors
-
-	note over R,RW: Phase 1: Initiate Shutdown
-
-	alt graceful shutdown
-		R->>P: ToServerStopping
-		P->>RW: Forward (ToServerStopping)
-	else forced stop
-		RW->>RW: receive Stop signal
-	end
-
-	note over R,RW: Phase 2: Drain Runner
-
-	note over RW: handle_stopping
-	note over RW: set state.draining = true
-	note over RW: ClearDb activity (update_state = Draining)
-	note over RW: remove from allocation indexes
-	note over RW: set drain_ts, expired_ts
-
-	note over RW: FetchRemainingActors activity
-	loop for each actor
-		RW->>A: GoingAway signal
-		note over A: actor workflows begin stopping
-	end
-
-	note over R,RW: Phase 3: Wait for Actors
-
-	note over R: waitForActorsToStop (max 120s)
-	loop check every 100ms
-		alt all actors stopped
-			note over R: continue shutdown
-		else websocket closed
-			note over R: force continue shutdown
-		else timeout reached
-			note over R: force continue shutdown
-		end
-	end
-
-	note over R,RW: Phase 4: Close WebSocket
-
-	note over R: send ToServerStopping (if not sent)
-	R->>P: WebSocket close (code=1000, reason=pegboard.runner_shutdown)
-	note over R: clear ping interval
-	note over R: clear ack interval
-	note over R: tunnel.shutdown
-
-	note over R: config.onShutdown callback
-
-	note over R,RW: Phase 5: Complete Workflow
-
-	note over RW: workflow exits drain loop after runner_lost_threshold
-
-	note over RW: ClearDb activity (update_state = Stopped)
-	note over RW: remove from active indexes
-	note over RW: set stop_ts
-
-	note over RW: FetchRemainingActors activity
-	loop for each remaining actor
-		RW->>A: Lost signal
-		note over A: reschedule actors if needed
-	end
-
-	RW->>P: ToClientClose
-	note over RW: workflow complete
-```

package/contrib-docs/SDKS.md

@@ -1,9 +0,0 @@
-# Generating SDKs
-
-To make sure the openapi spec is up to date, run `cargo check -p rivet-dump-openapi`
-
-Use `./scripts/fern/gen.sh` to generate the SDKs, which will be outputted under `sdks/api`
-
-# Endpoints
-
-A current list of API endpoints can be found at `packages/core/api-public/src/router.rs` or in the `out/openapi.json` file. 

package/contrib-docs/TEST_DEPENDENCIES.md

@@ -1,43 +0,0 @@
-# Test Dependencies
-
-The `rivet-test-deps` package provides isolated test environments with configurable database and pub/sub backends.
-
-## Configuration
-
-Configure backends via environment variables:
-
-- `RIVET_TEST_DATABASE`: Choose database backend
-  - `foundationdb` - Runs FoundationDB in Docker
-  - `postgres` - PostgreSQL in Docker
-  - `filesystem` - RocksDB with temp directory (default)
-
-- `RIVET_TEST_PUBSUB`: Choose pub/sub backend
-  - `nats` - Runs NATS in Docker
-  - `postgres_notify` - PostgreSQL in Docker
-  - `memory` - In-memory channels (default)
-
-- `RUST_LOG`: Enable debug logs to see container lifecycle details
-	- For example, `RUST_LOG=debug` will enable more verbose logs
-
-## How It Works
-
-1. **Port Selection**: Automatically picks unused ports for all services
-2. **Container Management**: 
-   - Starts Docker containers for services that require them
-   - Each test gets unique container names using UUIDs
-   - Containers are automatically cleaned up on drop
-3. **Config Generation**: Creates a complete Rivet config with the selected backends
-4. **Isolation**: Each test instance gets isolated resources (unique channels, temp dirs, containers)
-
-## Usage
-
-```rust
-let deps = TestDeps::new().await?;
-// Use deps.pools() and deps.config() for your tests
-// Containers are cleaned up automatically when deps is dropped
-```
-
-For multi-datacenter tests:
-```rust
-let deps = TestDeps::new_multi(&[1, 2, 3]).await?;
-```

package/contrib-docs/design-choicse/EMBEDDED_KV.md

@@ -1,80 +0,0 @@
-# Embedded KV
-
-## Goals
-
-- Embedded (no need for multi-process)
-- Similarity to FDB in transacations
-- Supports high write throughput
-
-## Decision
-
-Chose RocksDB because:
-
-- It's stable & boring
-- Supports high write throughput
-
-Nice to haves:
-
-- Better compression on disk
-
-## Evaluation
-
-### RocksDB
-
-Uses LSM trees. Requires backgound compaction. Has a lot of overhead to work like a "real" database.
-
-Pros:
-
-- Supports a WAL for logging & buffering writes
-
-Cons:
-
-- Significantly more complicated
-- Overhead of background threads managing compaction
-- Does not provide native MVCC + conflict ranges, we must re-implement this ourselves
-
-### LMDB
-
-Uses B+ tree for storage. mmaps database.
-
-**The critical downside of LMDB is that performance suffers under write-heavy workoads from (a) single writer locks and (b) uses mmap instead of buffering writes.**
-
-Pros:
-
-- Simple implementation
-- Very little overhead
-
-Cons:
-
-- No compaction
-- Single writer lock
-- Transaction commits each have an fsync, while RocksDB can optimize with a WAL + batching writes
-- Uses pages with mmap which is slower than using a WAL + batching writes
-- Does not work well with small writes since it's copied at the page level (Rivet is almost all small writes)
-
-### sled, redb, fjall
-
-All three differe significantly, but the primary reason for disqualifying these is maturity. Sled claims to be "beta" and not mature yet, while redb and fjall claim to have a stable disk format.
-
-**This is not a place we need to innovate, so it makes more sense to go with an established embedded KV.**
-
-Pros:
-
-- Rust-native
-	- Less potential issues with building & linking
-	- Theoretically faster compilation than RocksDB
-
-### SQLite
-
-This would be the simplest & most straightforward option in terms of support. FDB's pre-7.0 storage engine is based on the SQLite's B-tree implementation.
-
-**SQLite provides overhead we don't need when options like RocksDB or LMDB exist.**
-
-Pros:
-
-- Widely supported, no issues with building it
-- We already have a deep experience with SQLite
-
-## Resources
-
-- https://github.com/marvin-j97/rust-storage-bench

package/contrib-docs/operate/TRACING_RECONFIGURE.md

@@ -1,78 +0,0 @@
-# Dynamic Tracing Configuration
-
-Dynamically reconfigure log levels and OpenTelemetry sampling for all running services without restart.
-
-## Log Filter Configuration
-
-Control which log messages are displayed by setting filter directives (similar to `RUST_LOG`).
-
-**Set log filter to debug**
-
-```bash
-rivet-engine tracing config -f debug
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"filter":"debug"}'
-```
-
-**Debug a specific package**
-
-```bash
-rivet-engine tracing config -f "debug,rivet_api_peer=trace"
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"filter":"debug,rivet_api_peer=trace"}'
-```
-
-**Reset log filter to defaults**
-
-```bash
-rivet-engine tracing config -f ""
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"filter":null}'
-```
-
-## OpenTelemetry Sampler Ratio
-
-Control what percentage of traces are sampled and sent to the OpenTelemetry collector.
-
-**Set sampler ratio to 10%**
-
-```bash
-rivet-engine tracing config -s 0.1
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"sampler_ratio":0.1}'
-```
-
-**Set sampler ratio to 100% (capture all traces)**
-
-```bash
-rivet-engine tracing config -s 1.0
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"sampler_ratio":1.0}'
-```
-
-**Reset sampler ratio to default**
-
-```bash
-rivet-engine tracing config -s 0.001
-
-# Or via HTTP API:
-curl -X PUT http://localhost:6421/debug/tracing/config \
-  -H "Content-Type: application/json" \
-  -d '{"sampler_ratio":null}'
-```
-

package/docker/dev/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-host/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: 127.0.0.1:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc/datacenters/dc-a/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-a:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc/datacenters/dc-b/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-b:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc/datacenters/dc-c/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-c:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc-multinode/datacenters/dc-a/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-a:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc-multinode/datacenters/dc-b/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-b:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server

package/docker/dev-multidc-multinode/datacenters/dc-c/otel-collector-client/config.yaml

@@ -1,39 +0,0 @@
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: 0.0.0.0:4317
-      http:
-        endpoint: 0.0.0.0:4318
-processors:
-  batch:
-    timeout: 5s
-    send_batch_size: 10000
-exporters:
-  otlp/server:
-    endpoint: otel-collector-server-dc-c:4317
-    tls:
-      insecure: true
-service:
-  pipelines:
-    logs:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    traces:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server
-    metrics:
-      receivers:
-        - otlp
-      processors:
-        - batch
-      exporters:
-        - otlp/server