@unifyplane/logsdk 1.0.0

package/.github/copilot-instructions.md

@@ -0,0 +1,48 @@
+# Copilot Instructions — LogSDK
+
+Purpose: Help AI agents work productively in this repository by capturing architecture, workflows, and project-specific conventions that are enforced by code and tests.
+
+## Overview
+- System: Execution logging SDK producing immutable, canonical "step" records, delivered to sinks with authoritative failure handling.
+- Entrypoint: `initLogSDK(config)` returns an API with `step(message)` and optional `flush()` in [src/index.ts](src/index.ts).
+- Runtime: Node.js (TypeScript → ESM); tests run via Vitest.
+
+## Architecture & Data Flow
+- Context capsule: Initialize once via `initContext()`; exposes `context_hash`/`context_version` for records. See [src/core/context.ts](src/core/context.ts).
+- IDs & sequencing: `createIdGenerator()` provides monotonic `sequence` and `record_id`. See [src/core/ids.ts](src/core/ids.ts) and [src/core/clock.ts](src/core/clock.ts).
+- Record builder: `buildStepRecord()` validates inputs, sets dual clocks, canonicalizes payload, computes `record_hash` (sha256), and deep-freezes the result. See [src/core/record_builder.ts](src/core/record_builder.ts).
+- Fanout & spool safety: `fanout()` emits to sinks, requires at least one authoritative sink, writes emergency spool if authoritative emit fails; returns `OK | DEGRADED | FAILED`. See [src/core/fanout.ts](src/core/fanout.ts) and [src/core/spool.ts](src/core/spool.ts).
+- Canonical schema & guards: `assertStepRecord()` and `assertApiSurface()` enforce schema and API discipline. See [src/validate/schema_guard.ts](src/validate/schema_guard.ts) and [src/validate/api_surface_guard.ts](src/validate/api_surface_guard.ts).
+- Sinks: NDJSON file sinks (authoritative/observability) and stdout sink. See [src/sinks/file_ndjson.ts](src/sinks/file_ndjson.ts), [src/sinks/file_ndjson_sink.ts](src/sinks/file_ndjson_sink.ts), [src/sinks/stdout_sink.ts](src/sinks/stdout_sink.ts).
+- Types: Canonical `StepRecord` fields and sink interfaces. See [src/core/types.ts](src/core/types.ts) and [src/sinks/sink_types.ts](src/sinks/sink_types.ts).
+
+## Usage Pattern (Examples)
+- Construct sinks: `authoritativeNdjsonSink('...')` for governance-critical emission; `fileNdjsonSink('...')` for observability.
+- Initialize:
+  - `const log = initLogSDK({ context: { /* object */ }, system, sinks });`
+  - `system` must include `institution, system_name, system_type, environment, system_version` (optional `instance_id`).
+- Emit & flush:
+  - `await log.step('...');` → produces one immutable record; sequence increments per instance.
+  - `await log.flush?.();` → asks all sinks to flush (if implemented).
+
+## Project-Specific Conventions
+- Message constraints: 512-char max; must be a simple string; reject JSON-shaped or base64-like blobs. See [src/core/message_constraints.ts](src/core/message_constraints.ts).
+- API surface: Only `step(message)` and optional `flush()`; `child()` is reserved and must declare `allowedScopes` if present; forbidden names include `log`, `emit`, `metric`, `span`, etc. See [src/validate/api_surface_guard.ts](src/validate/api_surface_guard.ts).
+- Authoritative sink requirement: At least one sink with `sinkClass: 'authoritative'`; otherwise `NO_AUTHORITATIVE_SINK`. See [src/validate/schema_guard.ts](src/validate/schema_guard.ts).
+- Emergency spool path: Controlled by `LOGSDK_EMERGENCY_SPOOL_PATH`; default is OS temp dir `logsdk-emergency-spool.ndjson`. See [src/core/spool.ts](src/core/spool.ts).
+- Immutability: Records are deep-frozen before emission and validated as frozen. Never mutate records in sinks.
+- Integrity: `record_hash` is computed over an ordered, normalized JSON; do not change field ordering, hashing, or algorithm.
+
+## Developer Workflows
+- Quick tests: `npm test` (Vitest).
+- Full verify pipeline (strict): `npm run verify` → runs repo structure validator, Vitest (`--reporter=json`), normalizes results, generates traceability evidence.
+- Build: `npm run build` → runs `verify` then TypeScript compile via `tsc`.
+- Tooling scripts: See [tools/test_results/run-tests-then-prebuild.js](tools/test_results/run-tests-then-prebuild.js), [tools/test_results/normalize-test-results.js](tools/test_results/normalize-test-results.js), and [tools/test_results/generate-test-traceability.js](tools/test_results/generate-test-traceability.js).
+
+## Implementation Notes for Agents
+- Prefer `authoritativeNdjsonSink()` for governance-grade emission; add `fileNdjsonSink()` or `createStdoutSink()` for observability.
+- Do not embed per-step context; context is initialized once and referenced via hash/version.
+- When adding fields to `StepRecord`, update schema guards and hash field order consistently; otherwise tests may fail.
+- Use `NonComplianceError` codes for violations; do not throw generic errors for governance checks. See [src/validate/noncompliance.ts](src/validate/noncompliance.ts).
+
+If any section is unclear or misses a repo-specific nuance, tell us what you need clarified and we’ll refine these instructions.

package/README.md

@@ -0,0 +1,8 @@
+# LogSDK
+
+System Type: pipeline
+
+## Runtime Scope
+
+LogSDK core is implemented for the Node.js/CommonJS runtime and depends on the Node standard library (fs, crypto, perf_hooks, etc.). Other runtimes (Python, JVM, browsers) must provide a compatible core that enforces the same canonical contracts, and thin adapters should bridge them into the canonical API surface described in docs/LogSDKFuntionalSpec.md.
+

package/contracts/specs/LogSDKFuntionalSpec.md

@@ -0,0 +1,394 @@
+
+# LogSDK Functional Specification (v1.1)
+
+## Table of Contents
+- [1. Overview](#section-1-overview)
+- [2. Initialization & Configuration](#section-2-initialization-and-configuration)
+- [3. Public API Surface (Closed)](#section-3-public-api-surface-closed)
+- [4. Context Handling (Non-Negotiable)](#section-4-context-handling-non-negotiable)
+- [5. Step Record Construction](#section-5-step-record-construction)
+- [6. Canonical Serialization & Hashing (Normative)](#section-6-canonical-serialization-and-hashing-normative)
+- [7. Sequencing & Identity](#section-7-sequencing-and-identity)
+- [8. Message Constraints (Strict)](#section-8-message-constraints-strict)
+- [9. Sink Model & Fan-Out Semantics](#section-9-sink-model-and-fan-out-semantics)
+- [10. Authoritative Failure Semantics](#section-10-authoritative-failure-semantics)
+- [11. Flush Behavior](#section-11-flush-behavior)
+- [12. Validation & Non-Compliance](#section-12-validation-and-non-compliance)
+- [13. Explicit Constraints & Non-Goals](#section-13-explicit-constraints-and-non-goals)
+- [14. Known Gaps (As Implemented)](#section-14-known-gaps-as-implemented)
+- [15. Final Lock Statement](#section-15-final-lock-statement)
+- [16. Runtime Scope](#section-16-runtime-scope)
+- [Appendix A. Revision Safety Notes (Non-Normative)](#appendix-a-revision-safety-notes-non-normative)
+
+<a id="section-1-overview"></a>
+## 1. Overview
+
+LogSDK provides a **minimal, governance-aligned logging facility** that emits **immutable execution step records** to configured sinks.
+
+It enforces:
+
+* strict message constraints
+* immutable, single-capture context referencing
+* deterministic sequencing and fan-out
+* canonical schema validation at emission
+* cryptographic integrity via canonical hashing
+
+LogSDK produces **execution evidence only**.
+It does not interpret meaning, enforce authority, or perform analytics.
+
+---
+
+<a id="section-2-initialization-and-configuration"></a>
+## 2. Initialization & Configuration
+
+Initialization occurs **once per process or execution boundary**.
+
+### 2.1 Initialization Inputs
+
+LogSDK initialization accepts:
+
+* **context**
+
+  * A single plain object (no arrays, no prototypes)
+  * Captured exactly once
+  * Normalized and deeply frozen
+  * Never embedded in records
+  * Referenced only by `context_hash` and `context_version`
+
+* **system**
+
+  * Ownership metadata identifying the emitting system
+  * Must be consistent with declared system classification
+
+* **sinks**
+
+  * An ordered list of sink entries
+  * Each sink declares a `class`:
+
+    * `authoritative` — evidence-bearing
+    * `observability` — best-effort visibility
+  * Each sink implements:
+
+    * `emit(record)`
+    * optional `flush()`
+
+### 2.2 Default Sink Support
+
+The SDK may provide a file-based **NDJSON sink** as a default `authoritative` sink.
+
+NDJSON sinks must:
+
+* append records atomically
+* detect and reject partial writes
+* never mutate records
+
+---
+
+<a id="section-3-public-api-surface-closed"></a>
+## 3. Public API Surface (Closed)
+
+LogSDK intentionally exposes a **minimal, closed API**.
+
+### 3.1 Canonical Methods
+
+* **`step(message: string): Promise<void>`**
+
+  * Validates message constraints
+  * Constructs a canonical step record
+  * Applies canonical serialization
+  * Computes integrity hashes
+  * Deep-freezes the record
+  * Validates record against schema
+  * Emits deterministically to all sinks
+  * Throws a **non-compliance error** if authoritative emission fails
+
+* **`flush(): Promise<void>`** *(optional)*
+
+  * Invokes `flush()` on sinks that implement it
+
+### 3.2 Forbidden API Patterns
+
+The SDK must not expose:
+
+* per-step context injection
+* severity levels, metrics, or counters
+* arbitrary metadata setters
+* hooks or interceptors that mutate records
+* payload-carrying APIs
+
+The **message string is the only per-step input**.
+
+---
+
+<a id="section-4-context-handling-non-negotiable"></a>
+## 4. Context Handling (Non-Negotiable)
+
+* Context is captured **once** at initialization
+* Context must be a plain object
+* Context is normalized and deeply frozen
+* Context is referenced only via:
+
+  * `context_hash`
+  * `context_version`
+
+Per-step context mutation or augmentation is **explicitly forbidden**.
+
+---
+
+<a id="section-5-step-record-construction"></a>
+## 5. Step Record Construction
+
+Each invocation of `step()` produces **exactly one immutable step record**.
+
+A canonical step record includes:
+
+* SDK identity and version
+* sequence number (monotonic)
+* dual time:
+
+  * wall-clock timestamp (UTC)
+  * monotonic timestamp
+* system ownership metadata
+* optional execution correlation fields
+* optional source location metadata
+* bounded message string
+* optional message code
+* context references (hash + version)
+* optional evidence references
+* integrity fields:
+
+  * `record_hash`
+  * `hash_algorithm`
+
+Records are:
+
+* constructed once
+* deep-frozen prior to emission
+* identical across all sinks
+
+---
+
+<a id="section-6-canonical-serialization-and-hashing-normative"></a>
+## 6. Canonical Serialization & Hashing (Normative)
+
+### 6.1 Canonical Serialization
+
+Before hashing, records and context **must be serialized canonically** using a deterministic JSON representation.
+
+Normative requirements:
+
+* UTF-8 encoding
+* stable key ordering
+* normalized numbers and strings
+* no insignificant whitespace
+* no runtime-dependent ordering
+
+An RFC-8785–compatible JSON Canonicalization Scheme is recommended.
+
+### 6.2 Hash Scope
+
+* `context_hash` is computed over the canonical serialized context object
+* `record_hash` is computed over the canonical serialized record **excluding**:
+
+  * `record_hash`
+  * sink-specific transport metadata
+
+Hash algorithm must be explicitly declared via `hash_algorithm`.
+
+---
+
+<a id="section-7-sequencing-and-identity"></a>
+## 7. Sequencing & Identity
+
+* Sequence is a **non-negative, monotonically increasing integer**
+* Scoped to a single SDK instance
+* Never resets during process lifetime
+
+Record identifiers:
+
+* are unique per process and SDK instance
+* incorporate monotonic time and sequence
+* may include an optional sanitized instance tag
+
+These guarantees support deterministic replay and auditability.
+
+---
+
+<a id="section-8-message-constraints-strict"></a>
+## 8. Message Constraints (Strict)
+
+Message input must:
+
+* be a non-empty string
+* respect a hard maximum length
+
+Rejected inputs include:
+
+* objects or arrays
+* embedded JSON
+* base64-like payloads
+* attempts to smuggle structured data
+* non-string values
+
+Messages describe **what step occurred**, not **what data flowed**.
+
+---
+
+<a id="section-9-sink-model-and-fan-out-semantics"></a>
+## 9. Sink Model & Fan-Out Semantics
+
+### 9.1 Sink Classes
+
+* **Authoritative sinks**
+
+  * Evidence-bearing
+  * Failure is system-significant
+  * Failure raises non-compliance error
+
+* **Observability sinks**
+
+  * Best-effort only
+  * Failure does not block execution
+
+### 9.2 Fan-Out Rules
+
+* A single frozen record is dispatched
+* Dispatch is **sequential and deterministic**
+* Sink order is configuration-declared
+* No concurrent fan-out
+* No per-sink mutation or enrichment
+
+---
+
+<a id="section-10-authoritative-failure-semantics"></a>
+## 10. Authoritative Failure Semantics
+
+If an authoritative sink fails during emission:
+
+* The step is considered **non-compliant**
+* A standardized non-compliance error is raised
+* Observability sinks may still receive the record
+* No retries are implied unless explicitly implemented by the sink
+
+The SDK does **not** silently degrade evidence guarantees.
+
+---
+
+<a id="section-11-flush-behavior"></a>
+## 11. Flush Behavior
+
+If provided:
+
+* `flush()` drains buffered records
+* finalizes sink resources
+* behavior is sink-specific
+
+Flush is:
+
+* explicit
+* never automatic
+* never invoked implicitly by `step()`
+
+---
+
+<a id="section-12-validation-and-non-compliance"></a>
+## 12. Validation & Non-Compliance
+
+LogSDK enforces compliance via:
+
+* **API guard**
+
+  * only `step()` and optional `flush()`
+
+* **Schema guard**
+
+  * required fields
+  * correct types
+  * immutability
+  * timestamp validity
+
+* **Message guard**
+
+  * bounded strings only
+  * no embedded context or payloads
+
+* **Emission guard**
+
+  * authoritative sink failure surfaces error
+
+Violations emit standardized **non-compliance codes**.
+
+Non-compliance is classified as a **project defect**, not a governance failure.
+
+---
+
+<a id="section-13-explicit-constraints-and-non-goals"></a>
+## 13. Explicit Constraints & Non-Goals
+
+LogSDK intentionally does **not** provide:
+
+* severity levels
+* metrics or counters
+* payload embedding
+* per-sink enrichment
+* concurrent fan-out
+* dynamic context mutation
+
+It emits **factual execution step evidence only**.
+
+---
+
+<a id="section-14-known-gaps-as-implemented"></a>
+## 14. Known Gaps (As Implemented)
+
+The following are explicitly out of scope for v1.x:
+
+* emergency spooling or degraded evidence modes
+* concurrent sink dispatch
+* automatic correlation or surface population
+* recovery semantics beyond error signaling
+
+These gaps are acknowledged and frozen.
+
+---
+
+<a id="section-15-final-lock-statement"></a>
+## 15. Final Lock Statement
+
+This specification defines the **canonical enforcement layer** for UnifyPlane logging.
+
+Any change to:
+
+* canonical serialization rules
+* hashing scope
+* context immutability
+* message constraints
+* authoritative sink semantics
+
+requires a **versioned specification update**, not incremental evolution.
+
+--- 
+
+<a id="section-16-runtime-scope"></a>
+## 16. Runtime Scope
+
+LogSDK core is implemented for the Node.js/CommonJS runtime and relies on the Node standard library (`fs`, `crypto`, `perf_hooks`, etc.). Any other runtime (Python, JVM, embedded, browser) that wants to reuse the canonical contracts must provide its own core implementation that mirrors the structure, determinism, and evidence guarantees described above, exposing only a thin adapter surface.
+
+<a id="appendix-a-revision-safety-notes-non-normative"></a>
+## Appendix A. Revision Safety Notes (Non-Normative)
+
+### Why this revision is safe
+
+* No new APIs
+* No new responsibilities
+* No governance leakage
+* Determinism and auditability are now explicit
+* Suitable for long-running pipelines *and* high-integrity processes
+
+If you want next, the **correct follow-on artifacts** are:
+
+* a **formal JSON Schema** for the step record
+* a **non-compliance code registry**
+* a **migration delta (v1 → v1.1)**
+
+Tell me which one you want to generate next.

package/contracts/specs/fanout-semantics.v1.md

@@ -0,0 +1,244 @@
+# 🧩 LogSDK Core — Fan-Out Semantics & Failure Rules (v1.0)
+
+## Table of Contents
+
+- [0. Purpose (Non-Negotiable)](#section-0-purpose-non-negotiable)
+- [1. Emit-Once Rule (Canonical)](#section-1-emit-once-rule-canonical)
+- [2. Sink Ordering Rule (Deterministic)](#section-2-sink-ordering-rule-deterministic)
+- [3. Delivery Mode (Default & Allowed Modes)](#section-3-delivery-mode-default-and-allowed-modes)
+- [4. Acknowledgement Semantics](#section-4-acknowledgement-semantics)
+- [5. Failure Classification (Locked)](#section-5-failure-classification-locked)
+- [6. Core Failure Handling (Default Policy)](#section-6-core-failure-handling-default-policy)
+- [7. Emergency Spool Semantics (If Enabled)](#section-7-emergency-spool-semantics-if-enabled)
+- [8. Backpressure & Rate Limits (Bounded Behavior)](#section-8-backpressure-and-rate-limits-bounded-behavior)
+- [9. Flush Semantics (If Exposed)](#section-9-flush-semantics-if-exposed)
+- [10. Compliance Outcomes (Deterministic)](#section-10-compliance-outcomes-deterministic)
+- [11. Freeze Statement](#section-11-freeze-statement)
+
+<a id="section-0-purpose-non-negotiable"></a>
+## 0. Purpose (Non-Negotiable)
+
+Fan-out exists to allow:
+
+* **Authoritative evidence capture** (must succeed)
+* **Observability projection** (may fail without impact)
+
+while ensuring the system emits **one canonical Step Record**.
+
+---
+
+<a id="section-1-emit-once-rule-canonical"></a>
+## 1. Emit-Once Rule (Canonical)
+
+For each `step(message)` call:
+
+1. Core constructs **exactly one** Step Record
+2. Core freezes it (immutability)
+3. Core computes integrity fields (record hash)
+4. Core submits the same frozen record to **all configured sinks**
+
+There is **no per-sink mutation** and **no per-sink context**.
+
+---
+
+<a id="section-2-sink-ordering-rule-deterministic"></a>
+## 2. Sink Ordering Rule (Deterministic)
+
+Fan-out order must be deterministic:
+
+### 2.1 Declared Order
+
+Sinks are invoked in the **declared configuration order**.
+
+This enables:
+
+* predictable I/O patterns
+* reproducible behavior in tests
+* stable incident diagnosis
+
+### 2.2 Recommended Ordering (Standard)
+
+Configuration should typically be:
+
+1. **Authoritative sinks first**
+2. **Observability sinks after**
+
+But the core rule is “declared order,” not “type order.”
+
+---
+
+<a id="section-3-delivery-mode-default-and-allowed-modes"></a>
+## 3. Delivery Mode (Default & Allowed Modes)
+
+### 3.1 Default Mode: Sequential Dispatch
+
+Core sends the record to sinks **sequentially** in order.
+
+Reason: deterministic error semantics.
+
+### 3.2 Allowed Future Mode: Concurrent Dispatch (Explicit Only)
+
+Concurrent dispatch is permitted only if:
+
+* ordering of *sink acknowledgements* does not affect correctness
+* failure classification remains deterministic
+* authoritative sink guarantees remain enforced
+
+If concurrent mode exists, it must be **explicitly configured** and must preserve all rules in this document.
+
+---
+
+<a id="section-4-acknowledgement-semantics"></a>
+## 4. Acknowledgement Semantics
+
+Each sink invocation results in one of:
+
+* **ACK**: sink confirms record accepted/persisted
+* **NACK**: sink rejects record (explicit failure)
+* **TIMEOUT**: sink did not respond in bounded time
+
+Core must treat TIMEOUT as failure.
+
+---
+
+<a id="section-5-failure-classification-locked"></a>
+## 5. Failure Classification (Locked)
+
+Each sink is configured with a **sink_class**:
+
+### 5.1 `authoritative`
+
+Meaning:
+
+* required for governance evidence continuity
+* failure must be treated as system-significant
+
+### 5.2 `observability`
+
+Meaning:
+
+* best-effort visibility only
+* failure must not block execution by default
+
+> Sink class is a configuration property, not inferred dynamically.
+
+---
+
+<a id="section-6-core-failure-handling-default-policy"></a>
+## 6. Core Failure Handling (Default Policy)
+
+### 6.1 Authoritative Sink Failure
+
+If **any authoritative sink** fails for a record:
+
+Core must produce a **local failure signal** in one of two canonical forms:
+
+**A) Fail-Closed (Strict)**
+
+* `step()` returns failure status to caller boundary (platform code)
+* caller may decide to abort the request/job
+
+**B) Fail-Open With Local Spool (Resilient)**
+
+* record is written to a **local emergency spool**
+* core reports “degraded evidence capture”
+* execution may continue
+
+**Constraint:**
+Fail-open is permitted **only** if emergency spool exists and is itself local/owned.
+
+The default policy should be **fail-open with spool** for production services, and **fail-closed** for tests/CI.
+
+---
+
+### 6.2 Observability Sink Failure
+
+If an observability sink fails:
+
+* core must continue dispatching to remaining sinks
+* core must not mark the overall step as failed
+* core may optionally increment an internal counter (not emitted as logs)
+
+Observability failures are never evidence failures.
+
+---
+
+<a id="section-7-emergency-spool-semantics-if-enabled"></a>
+## 7. Emergency Spool Semantics (If Enabled)
+
+Emergency spool is the only allowed “fail-open” mechanism.
+
+Rules:
+
+* append-only
+* local filesystem (or equivalent local owned store)
+* bounded rotation
+* replay-capable (a later process can re-emit)
+
+Spool must never:
+
+* enrich records
+* re-order within a single execution context
+
+It is a safety net, not a datastore.
+
+---
+
+<a id="section-8-backpressure-and-rate-limits-bounded-behavior"></a>
+## 8. Backpressure & Rate Limits (Bounded Behavior)
+
+LogSDK must not allow unbounded memory growth.
+
+Two mechanisms are permitted:
+
+### 8.1 Bounded Buffer
+
+* If sink is slow, core buffers up to a fixed limit
+* When full, apply the failure policy (authoritative vs observability)
+
+### 8.2 Drop Policy (Observability Only)
+
+Drops are allowed **only** for observability sinks when buffers overflow.
+
+Drops are forbidden for authoritative sinks unless emergency spool is active and successful.
+
+---
+
+<a id="section-9-flush-semantics-if-exposed"></a>
+## 9. Flush Semantics (If Exposed)
+
+If `flush()` exists:
+
+* it must attempt to drain buffers to all sinks
+* it must return a status indicating whether authoritative sinks are fully flushed
+* it must not block indefinitely (bounded by timeouts)
+
+Flush is an operational tool, not a correctness requirement.
+
+---
+
+<a id="section-10-compliance-outcomes-deterministic"></a>
+## 10. Compliance Outcomes (Deterministic)
+
+For each `step()` call, the core must be able to classify the outcome:
+
+* **OK**: all authoritative sinks ACK’d
+* **DEGRADED**: authoritative sink failed but record spooled
+* **FAILED**: authoritative sink failed and not spooled
+
+Observability sink failures do not change these outcomes.
+
+---
+
+<a id="section-11-freeze-statement"></a>
+## 11. Freeze Statement
+
+This **Fan-Out Semantics & Failure Rules v1.0** is now locked:
+
+* deterministic order (declared)
+* authoritative vs observability classes
+* fail-open allowed only with spool
+* bounded buffers
+* no unbounded memory
+* no per-sink mutation
+