@aria_asi/cli 0.2.40 → 0.2.41

package/skills/deno-kv-raft-pubsub/reference/pubsub-api.md

@@ -0,0 +1,376 @@
+# PubSub API Contract
+
+The client-facing API for the Raft-backed PubSub system. All writes go through the Raft leader; reads can be served by any node (all have identical committed state).
+
+---
+
+## 1. Transport
+
+- **HTTP/1.1** for request-response endpoints
+- **WebSocket** for streaming subscriptions
+- Default port: **8080** (configurable)
+- Content-Type: `application/json`
+
+---
+
+## 2. Leader Discovery
+
+Any node can handle discovery. Clients should cache the leader address.
+
+### GET /cluster
+
+Returns the current cluster state.
+
+**Response (200):**
+```json
+{
+  "node_id": "node-1",
+  "leader_id": "node-2",
+  "leader_address": "node-2:8080",
+  "peers": ["node-1:8080", "node-2:8080", "node-3:8080"],
+  "raft_term": 5,
+  "commit_index": 1042
+}
+```
+
+**Redirect behavior:** If a client sends a write to a follower, the follower returns:
+
+**Response (307 Temporary Redirect):**
+```json
+{
+  "error": "not_leader",
+  "leader_id": "node-2",
+  "leader_address": "node-2:8080"
+}
+```
+With HTTP status 307 and `Location: http://node-2:8080/...`
+
+---
+
+## 3. Topic Management
+
+### POST /topics/:name
+
+Create a new topic. Idempotent — success if topic already exists.
+
+**Request:**
+```json
+{
+  "metadata": {
+    "description": "Order events",
+    "retention_hours": 24
+  }
+}
+```
+
+**Response (201 Created):**
+```json
+{
+  "topic": "orders",
+  "created": true,
+  "raft_index": 42
+}
+```
+
+**Response (200 OK) — topic already exists:**
+```json
+{
+  "topic": "orders",
+  "created": false,
+  "raft_index": 0
+}
+```
+
+### GET /topics
+
+List all active topics.
+
+**Response (200):**
+```json
+{
+  "topics": [
+    {"name": "orders", "message_count": 150, "created_at": 1714000000000},
+    {"name": "notifications", "message_count": 89, "created_at": 1714000001000}
+  ]
+}
+```
+
+### GET /topics/:name
+
+Get topic metadata and current sequence number.
+
+**Response (200):**
+```json
+{
+  "topic": "orders",
+  "message_count": 150,
+  "current_sequence": 150,
+  "created_at": 1714000000000,
+  "metadata": {
+    "description": "Order events"
+  }
+}
+```
+
+### DELETE /topics/:name
+
+Delete a topic. All messages are retained but no new publishes are accepted.
+
+**Response (200):**
+```json
+{
+  "topic": "orders",
+  "deleted": true,
+  "raft_index": 57
+}
+```
+
+---
+
+## 4. Publish
+
+### POST /topics/:name/messages
+
+Publish a message to a topic. Must be sent to the leader.
+
+**Request:**
+```json
+{
+  "id": "msg-abc-123",
+  "payload": {
+    "order_id": "ORD-456",
+    "status": "shipped"
+  },
+  "timestamp": 1714000000000
+}
+```
+
+- `id`: Unique message identifier. Client-generated (UUID recommended). Used for deduplication.
+- `payload`: Any JSON-serializable value.
+- `timestamp`: Client-provided timestamp in milliseconds since epoch.
+
+**Response (201 Created):**
+```json
+{
+  "id": "msg-abc-123",
+  "topic": "orders",
+  "sequence": 151,
+  "raft_index": 1043,
+  "committed": true
+}
+```
+
+**Response (409 Conflict) — duplicate ID:**
+```json
+{
+  "error": "duplicate_message_id",
+  "id": "msg-abc-123",
+  "existing_sequence": 42
+}
+```
+
+**Error Handling:**
+- `404`: Topic does not exist
+- `307`: Not the leader (redirect)
+- `503`: Leader election in progress (no leader)
+- `409`: Duplicate message ID
+
+### Ack Mode (Optional)
+
+Add `?ack=committed` to wait for commit before responding. Default behavior.
+
+Add `?ack=appended` to return immediately after leader appends (before commit). Faster but message may not be committed yet.
+
+**Response (202 Accepted) — `ack=appended`:**
+```json
+{
+  "id": "msg-abc-123",
+  "topic": "orders",
+  "sequence": 151,
+  "raft_index": 1043,
+  "committed": false
+}
+```
+
+---
+
+## 5. Subscribe
+
+### WebSocket /topics/:name/subscribe
+
+Establish a persistent, bidirectional subscription.
+
+**Connection:** WebSocket upgrade on GET.
+
+**Query parameters:**
+- `cursor` (optional): Sequence number to start from. Omit for latest.
+- `subscriber_id` (required): Unique identifier for this subscriber.
+
+Example: `ws://node-1:8080/topics/orders/subscribe?subscriber_id=sub-xyz&cursor=100`
+
+**Server → Client Messages:**
+
+```json
+{
+  "type": "message",
+  "id": "msg-abc-123",
+  "topic": "orders",
+  "sequence": 151,
+  "payload": {
+    "order_id": "ORD-456",
+    "status": "shipped"
+  },
+  "timestamp": 1714000000000
+}
+```
+
+```json
+{
+  "type": "control",
+  "subtype": "subscribed",
+  "topic": "orders",
+  "cursor": 150,
+  "subscriber_id": "sub-xyz"
+}
+```
+
+```json
+{
+  "type": "control",
+  "subtype": "topic_deleted",
+  "topic": "orders"
+}
+```
+
+```json
+{
+  "type": "control",
+  "subtype": "leader_change",
+  "new_leader": "node-2:8080",
+  "reason": "The current leader stepped down. Reconnect to new leader for publishes. Subscription continues."
+}
+```
+
+**Client → Server Messages:**
+
+```json
+{
+  "type": "ack",
+  "sequence": 151
+}
+```
+
+**Reconnection:**
+If the connection drops, the client reconnects with `cursor` set to the last acknowledged sequence. The server replays any messages after that cursor.
+
+**At-Least-Once Delivery:**
+Messages are delivered at least once. Clients must deduplicate using the `id` field. To get exactly-once, clients track the highest `id` received and ignore duplicates.
+
+---
+
+## 6. Error Responses
+
+All errors follow this format:
+
+```json
+{
+  "error": "error_code",
+  "message": "Human-readable description",
+  "details": {}
+}
+```
+
+### Error Codes
+
+| Code | HTTP Status | Meaning |
+|------|-------------|---------|
+| `not_leader` | 307 | Request sent to follower; redirect to leader |
+| `no_leader` | 503 | No leader currently elected; retry |
+| `topic_not_found` | 404 | Topic does not exist |
+| `topic_already_exists` | 409 | Create called on existing topic (non-idempotent mode) |
+| `duplicate_message_id` | 409 | Message with this ID already published |
+| `invalid_cursor` | 400 | Requested cursor is beyond current sequence |
+| `raft_timeout` | 504 | Command did not commit within timeout |
+| `not_committed` | 500 | Log entry appended but failed to commit |
+
+---
+
+## 7. Delivery Semantics
+
+| Mode | Guarantee | How |
+|------|-----------|-----|
+| **Publish (ack=committed)** | Message is durable and will be delivered | Waits for Raft commit |
+| **Publish (ack=appended)** | Message is appended but not yet committed | Leader crash may lose message |
+| **Subscribe** | At-least-once delivery | Client acks and cursor tracking |
+| **Subscribe + Dedup** | Exactly-once (client-side) | Track message IDs, ignore duplicates |
+
+---
+
+## 8. Performance Bounds
+
+- **Publish latency (p50):** ~10-20ms (1 RTT for consensus in same region)
+- **Publish latency (p99):** ~50-100ms
+- **Subscribe delivery latency:** ~5-10ms after commit (kv.watch poll interval)
+- **Max message size:** 1MB (Deno KV value limit)
+- **Max topics:** Unlimited (KV key space)
+- **Max subscribers per topic:** Bounded by Deno's watch stream limit (~100 per KV connection)
+
+---
+
+## 9. Client Library Pseudocode
+
+```typescript
+// Publishing
+const response = await fetch(`http://${leader}/topics/orders/messages`, {
+  method: 'POST',
+  headers: { 'Content-Type': 'application/json' },
+  body: JSON.stringify({
+    id: crypto.randomUUID(),
+    payload: { data: "hello" },
+    timestamp: Date.now()
+  })
+});
+if (response.status === 307) {
+  // Redirect to leader
+  leader = response.headers.get('Location');
+  // retry...
+}
+
+// Subscribing
+const ws = new WebSocket(
+  `ws://any-node:8080/topics/orders/subscribe?subscriber_id=my-app&cursor=${lastSeen}`
+);
+ws.onmessage = (event) => {
+  const msg = JSON.parse(event.data);
+  if (msg.type === 'message') {
+    if (!seenIds.has(msg.id)) {
+      handleMessage(msg.payload);
+      seenIds.add(msg.id);
+      ws.send(JSON.stringify({ type: 'ack', sequence: msg.sequence }));
+    }
+  }
+};
+```
+
+---
+
+## 10. Internal API (inter-node)
+
+### WebSocket RPC messages (not exposed to clients)
+
+**RequestVote:**
+```json
+{"type": "request_vote", "term": 5, "candidate_id": "node-1", "last_log_index": 42, "last_log_term": 5}
+```
+
+**AppendEntries:**
+```json
+{"type": "append_entries", "term": 5, "leader_id": "node-1", "prev_log_index": 41, "prev_log_term": 5, "entries": [...], "leader_commit": 40}
+```
+
+**InstallSnapshot:**
+```json
+{"type": "install_snapshot", "term": 5, "leader_id": "node-1", "last_included_index": 1000, "last_included_term": 5, "data": "...", "done": true}
+```
+
+These RPCs are handled by the Raft core, not by the PubSub API layer.