@zewstid/zeye-capture 0.2.0

package/README.md

@@ -0,0 +1,194 @@
+# @zewstid/zeye-capture
+
+Local-first capture and continuous sync for Zeye Engineering Memory. Metadata
+capture is the default. Complete provider-visible prompts, responses and tool
+records are captured only after explicit Zeye consent and local secret
+redaction. Hidden model reasoning is never captured.
+
+## What exists now
+
+- `FileOutbox`: a JSON-backed durable outbox with atomic state replacement.
+- Idempotent delivery records keyed by `adapter + sessionId + sourceEventId`,
+  including immutable repository/branch/HEAD facts observed at enqueue time.
+- Stable content hashes: replays with the same source and content collapse;
+  changed content from the same provider event is reported as a conflict.
+- Source checkpoints for JSONL/backfill cursors.
+- A required `RedactionBoundary`, executed before a payload reaches disk.
+- Delivery attempt and acknowledgement state, allowing a future HTTP transport
+  to replay safely after an offline period.
+- Claude Code hook and Codex JSONL adapters for lifecycle metadata and,
+  when explicitly consented, provider-visible messages and tool records.
+- A one-command background service that continuously imports and retries on
+  macOS (launchd), Linux (systemd user service), and Windows (Task Scheduler).
+- `CaptureHttpTransport`, which turns one durable local delivery into one
+  authenticated, idempotent `POST /engineering-memory/ingest` batch.
+- A production-oriented local CLI: Claude Code hook install/removal, configured
+  Codex JSONL sync, safe Git identity discovery, durable outbox locking,
+  explicit heartbeat, and non-secret status diagnostics.
+
+## Security and limits
+
+- Transcript content is locally redacted before either the durable outbox or
+  network boundary. The API envelope-encrypts each segment with a separate data
+  key and keeps ciphertext apart from metadata.
+- The one-time enrollment token is kept in an owner-only local service config,
+  never in the service definition, command line, repository, or outbox. Native
+  keychain integration remains future hardening.
+- VS Code/GitHub Copilot currently has a safe companion-event boundary; a
+  packaged extension is not included in this release.
+- “Complete” means all provider-visible records emitted by supported adapters.
+  It never means hidden chain-of-thought or provider-private state.
+
+## Usage
+
+```ts
+import { FileOutbox, type RedactionBoundary } from '@zewstid/zeye-capture'
+
+const redactor: RedactionBoundary = {
+  redact(event) {
+    return { payload: event.payload, redactionCount: 0 }
+  },
+}
+
+const outbox = new FileOutbox('/private/path/zeye-capture', redactor)
+const result = await outbox.enqueue({
+  adapter: 'claude-code',
+  sessionId: 'provider-session-id',
+  sourceEventId: 'provider-event-id',
+  occurredAt: new Date().toISOString(),
+  type: 'tool.completed',
+  payload: { tool: 'Bash', exitCode: 0 },
+})
+```
+
+The included transport calls `markAttempt()` for unsuccessful uploads and
+`acknowledge(deliveryId, receiptId)` only after a server receipt is durably
+returned. It never regenerates a delivery ID or mutates an already queued
+event.
+
+## Provider adapters
+
+```ts
+import {
+  FileOutbox, metadataRedactor, fromClaudeCodeHook,
+  fromCodexJsonl, CaptureHttpTransport,
+} from '@zewstid/zeye-capture'
+
+const outbox = new FileOutbox('/private/path/zeye-capture', metadataRedactor)
+
+// Claude Code hook stdin JSON → local durable evidence.
+const claudeEvent = fromClaudeCodeHook(hookPayload)
+if (claudeEvent) await outbox.enqueue(claudeEvent)
+
+// One Codex JSONL record → local durable evidence.
+const codexEvent = fromCodexJsonl(JSON.parse(jsonlLine), 'rollout.jsonl')
+if (codexEvent) await outbox.enqueue(codexEvent)
+
+// A VS Code companion extension sends a deliberately metadata-only event to
+// `zeye-capture vscode-copilot-event` over stdin. Its event shape is also
+// available to hosts through fromVSCodeCopilotEvent().
+
+// A host application supplies the enrollment token at runtime; it is not
+// written into the outbox. The server still has to enable
+// engineering-memory.session-ledger for the organization.
+const transport = new CaptureHttpTransport(outbox, enrollmentConfiguration)
+await transport.flush()
+```
+
+The adapters intentionally treat unknown provider record shapes as
+`capture.warning`. That keeps capture health honest while provider formats
+change, instead of inventing a complete record.
+
+The VS Code adapter is an integration boundary for a companion extension; it
+does not claim to scrape Copilot chats or private extension storage. A future
+packaged VSIX can emit these events using the same contract without changing
+the durable outbox or server API.
+
+## Local host workflow
+
+Build the package first, then make the executable available on your `PATH` (or
+use the built `dist/src/cli.js` through Node). Run all commands from the Git
+repository being captured, or pass `--repo /absolute/path`.
+
+```bash
+# Add Zeye-owned groups to the repository-local, untracked Claude settings.
+zeye-capture install claude
+
+# Register a known Codex rollout/session JSONL source. This does not guess a
+# provider-private location and does not start a background process.
+zeye-capture install codex --jsonl /absolute/path/to/rollout.jsonl
+
+# Import configured Codex sources, upload any durable outbox records, and send
+# a heartbeat when runtime enrollment is available.
+zeye-capture sync
+
+# View safe local diagnostics: queue counts, lock state, Git identity, and
+# whether upload variables are complete. It never prints the machine token.
+zeye-capture status
+zeye-capture heartbeat
+
+# With ZEYE_CAPTURE_* values present, install and immediately start continuous
+# sync. The service survives logout/restart according to the host OS policy.
+zeye-capture service install --repo /path/to/repo
+
+# Stop continuous sync and remove both its definition and private enrollment.
+zeye-capture service uninstall --repo /path/to/repo
+
+# Remove only hook groups/sources created for Zeye.
+zeye-capture uninstall claude
+zeye-capture uninstall codex --jsonl /absolute/path/to/rollout.jsonl
+```
+
+`install claude` defaults to `.claude/settings.local.json` and refuses to
+modify that file if Git tracks it. It preserves other hook groups. The host
+configuration is stored under `$XDG_CONFIG_HOME/zeye-capture/host-v1.json` (or
+`~/.config/...`) with owner-only permissions where the OS supports POSIX modes.
+The default outbox is under `$XDG_STATE_HOME/zeye-capture/` (or
+`~/.local/state/...`) and is also owner-restricted. `ZEYE_CAPTURE_OUTBOX_DIR`,
+when set, is a private base directory: Zeye creates a distinct hashed child for
+each repository so records cannot cross repository boundaries. Neither location
+stores the upload token.
+
+Interactive setup supplies these variables once. `service install` persists
+them in an owner-only local service enrollment so no scheduler is configured
+by hand:
+
+```text
+ZEYE_CAPTURE_ENDPOINT=https://api.example/engineering-memory/ingest
+ZEYE_CAPTURE_TOKEN=zem_...                 # runtime only; never written by CLI
+ZEYE_CAPTURE_ORGANIZATION_ID=...
+ZEYE_CAPTURE_INSTALLATION_ID=...
+ZEYE_CAPTURE_ACTOR_JSON={...}
+ZEYE_CAPTURE_REPOSITORY_ID=...             # optional internal Zeye binding
+ZEYE_CAPTURE_PROVIDER_REPOSITORY_ID=...    # optional provider numeric ID
+ZEYE_CAPTURE_CONTENT_CONSENT_ID=...        # required only for transcripts
+ZEYE_CAPTURE_TRANSCRIPT_CONSENT=yes        # explicit local opt-in
+```
+
+`ZEYE_CAPTURE_ENDPOINT` must be an HTTPS URL ending in
+`/engineering-memory/ingest`. HTTP is accepted only for an explicit loopback
+development collector. Without both content-consent variables, capture remains
+metadata-only even if the installation is transcript-capable.
+
+Without all required values the host remains useful offline: it captures into
+the local outbox and reports `uploadConfigured: false`. Git discovery supplies
+the remote's canonical `owner/repository` name, branch, and HEAD SHA when
+available; the raw remote URL is not stored or printed. A configured internal
+repository ID enables the server to bind those facts to existing Zeye evidence.
+
+For continuous capture, run `zeye-capture service install --repo /path/to/repo`
+once. The CLI
+uses an owner-only single-process lock and waits briefly for an overlapping
+hook/run before failing, so normal bursts are queued rather than dropped. A
+`sync` exits non-zero if an input source or upload fails, while retaining its
+outbox records for retry. Hook calls use the same lock.
+
+## Build and test
+
+```bash
+npm -w @zewstid/zeye-capture run build
+npm -w @zewstid/zeye-capture test
+```
+
+The root workspace already includes `packages/*`, so no workspace manifest
+change is required. The package uses only Node built-ins at runtime.

package/dist/src/canonical.d.ts

@@ -0,0 +1,3 @@
+export declare function canonicalJson(value: unknown): string;
+export declare function sha256(value: unknown): string;
+export declare function newDeliveryId(): string;

package/dist/src/canonical.js

@@ -0,0 +1,41 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.canonicalJson = canonicalJson;
+exports.sha256 = sha256;
+exports.newDeliveryId = newDeliveryId;
+const node_crypto_1 = require("node:crypto");
+function normalize(value) {
+    if (value === null || typeof value === 'boolean' || typeof value === 'string')
+        return value;
+    if (typeof value === 'number') {
+        if (!Number.isFinite(value))
+            throw new TypeError('Capture payload cannot contain non-finite numbers');
+        return value;
+    }
+    if (typeof value === 'bigint' || typeof value === 'function' || typeof value === 'symbol' || value === undefined) {
+        throw new TypeError(`Capture payload contains unsupported value type: ${typeof value}`);
+    }
+    if (Array.isArray(value))
+        return value.map(normalize);
+    if (value instanceof Date)
+        return value.toISOString();
+    if (typeof value === 'object') {
+        const object = value;
+        return Object.keys(object)
+            .sort()
+            .reduce((result, key) => {
+            result[key] = normalize(object[key]);
+            return result;
+        }, {});
+    }
+    throw new TypeError('Capture payload is not serializable');
+}
+function canonicalJson(value) {
+    return JSON.stringify(normalize(value));
+}
+function sha256(value) {
+    return (0, node_crypto_1.createHash)('sha256').update(canonicalJson(value), 'utf8').digest('hex');
+}
+function newDeliveryId() {
+    return (0, node_crypto_1.randomUUID)();
+}

package/dist/src/cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};