domainforge 0.13.0

package/docs/semantic-pack-signing.md

@@ -0,0 +1,234 @@
+# Semantic Pack Signing and Verification
+
+Semantic packs use Ed25519 detached signatures to provide tamper evidence and authenticity verification. This document describes the signing algorithm, payload format, CLI commands, and how the LSP and CI handle signature states.
+
+## Signing Algorithm
+
+DomainForge uses **Ed25519** (Edwards-curve Digital Signature Algorithm) for pack signing. Ed25519 provides:
+
+- 128-bit security level.
+- Deterministic signatures (no randomness in signature generation).
+- Compact signatures (64 bytes).
+- Fast verification.
+
+The implementation uses the `ed25519-dalek` crate with PKCS#8 PEM key encoding.
+
+## Signature Payload Format
+
+The signing payload is a UTF-8 byte string with the following format:
+
+```
+DOMAINFORGE_SEMANTIC_PACK_V0_3\n<content_hash>\n<pack_id>\n<schema_version>\n
+```
+
+Where:
+
+- `DOMAINFORGE_SEMANTIC_PACK_V0_3` is a fixed domain separation string.
+- `<content_hash>` is the pack content hash (SHA-256) with signature fields excluded.
+- `<pack_id>` is the pack's unique identifier (e.g., `acme/logistics/1.1.0`).
+- `<schema_version>` is the pack schema version (e.g., `0.3`).
+
+The trailing newline ensures the payload is unambiguous when concatenated.
+
+## Pack Content Hash
+
+The pack content hash is computed over the canonical JSON representation of the pack with all signature fields cleared:
+
+```rust
+// Fields cleared before hashing:
+pack.trust.signature = None;
+pack.trust.signature_state = Unsigned;
+```
+
+This means:
+
+- Signing a pack does not change its content hash.
+- The content hash is deterministic regardless of signature state.
+- Two packs with identical semantic content but different signatures have the same content hash.
+
+The canonicalization process sorts all keys, arrays, and sub-objects deterministically to ensure reproducible hashes across builds.
+
+## CLI Commands
+
+### Signing a Pack
+
+```bash
+domainforge pack sign <pack-path> --key <private-key.pem>
+```
+
+Options:
+
+| Option        | Description                                   |
+|---------------|-----------------------------------------------|
+| `<pack-path>` | Path to the unsigned pack JSON file.          |
+| `--key`       | Path to the Ed25519 private key (PEM format). |
+| `--out`       | (Optional) Output path for the signed pack.   |
+
+When `--out` is omitted, the signed pack is written to stdout. To overwrite the input file in-place, use `--out <pack-path>` (back up the original first).
+
+The command reads the pack, computes the content hash, signs the payload, and writes the signed pack with the `trust` fields updated:
+
+```json
+{
+  "trust": {
+    "approval_state": "approved",
+    "signature_state": "signed",
+    "signed_by": "pack-filename",
+    "signature_alg": "ed25519",
+    "signature": "<base64-encoded-signature>"
+  }
+}
+```
+
+Example:
+
+```bash
+domainforge pack sign packs/acme-logistics-1.1.0.json \
+  --key keys/acme-private.pem \
+  --out packs/acme-logistics-1.1.0-signed.json
+```
+
+### Verifying a Pack Signature
+
+```bash
+domainforge pack verify <pack-path> --key <public-key.pem>
+```
+
+Options:
+
+| Option        | Description                                   |
+|---------------|-----------------------------------------------|
+| `<pack-path>` | Path to the signed pack JSON file.            |
+| `--key`       | Path to the Ed25519 public key (PEM format).  |
+
+The command:
+
+1. Reads the pack.
+2. Computes the content hash (excluding signature fields).
+3. Reconstructs the signing payload.
+4. Decodes the base64 signature from the pack.
+5. Verifies the Ed25519 signature against the public key.
+
+Example:
+
+```bash
+domainforge pack verify packs/acme-logistics-1.1.0-signed.json \
+  --key keys/acme-public.pem
+```
+
+## Exit Codes
+
+| Code | Meaning                                 |
+|------|-----------------------------------------|
+| 0    | Signature is valid.                     |
+| 3    | Pack is unsigned (no signature present).|
+| 4    | Signature is invalid or has been tampered with. |
+
+The `domainforge pack validate` command uses these same codes when `--require-signature` is set:
+
+```bash
+# Exit 3 if unsigned
+domainforge pack validate --pack packs/unsigned.json --require-signature models/*.sea
+
+# Exit 4 if signature is invalid
+domainforge pack validate --pack packs/tampered.json --require-signature models/*.sea
+```
+
+## LSP Behavior
+
+The DomainForge LSP adjusts its behavior based on the pack's signature state:
+
+| Signature State    | LSP Behavior                                                            |
+|--------------------|-------------------------------------------------------------------------|
+| `unsigned`         | Loads the pack in **warn mode**. All features are available but a diagnostic warns that the pack is unsigned. |
+| `signed`           | Full functionality with no warnings.                                    |
+| `invalid_signature`| Pack features are **disabled**. The LSP reports a critical error and does not use the pack for resolution. |
+
+In practice, unsigned packs are common during development. Teams should sign packs before merging to main branches.
+
+## CI Strict Behavior
+
+When `--mode strict --require-signature` is set:
+
+- Unsigned packs cause the validation to fail with exit code 3.
+- Invalid signatures cause the validation to fail with exit code 4.
+- The `--expected-hash` flag provides additional tamper protection by pinning the expected content hash.
+
+```bash
+domainforge pack validate \
+  --pack packs/acme-logistics-1.1.0-signed.json \
+  --mode strict \
+  --require-signature \
+  --expected-hash sha256:abc123def456... \
+  models/**/*.sea
+```
+
+If the pack's content hash does not match `--expected-hash`, the command exits with code 4 regardless of signature validity.
+
+## Generating Ed25519 Keys
+
+Use `openssl` to generate Ed25519 key pairs:
+
+### Generate a Private Key
+
+```bash
+openssl genpkey -algorithm Ed25519 -out acme-private.pem
+```
+
+### Extract the Public Key
+
+```bash
+openssl pkey -in acme-private.pem -pubout -out acme-public.pem
+```
+
+### Verify Key Format
+
+Private keys should be in PKCS#8 PEM format. Do not commit real private keys; generate them locally at runtime or in CI.
+
+```
+<PKCS#8 PEM-encoded Ed25519 private key generated locally>
+```
+
+Public keys should be in SPKI PEM format:
+
+```
+<SPKI PEM-encoded Ed25519 public key corresponding to that generated key>
+```
+
+## Expected-Hash Pinning
+
+For environments where defense-in-depth is required, the `--expected-hash` flag pins the expected content hash of the pack. This provides tamper protection even if the signing key is compromised:
+
+```bash
+EXPECTED_HASH=$(domainforge pack inspect --pack packs/acme-logistics-1.1.0.json --format json | jq -r '.content_hash')
+
+# Store the hash in CI configuration
+echo "PACK_HASH=$EXPECTED_HASH" >> ci.env
+
+# Later, in CI:
+domainforge pack validate \
+  --pack packs/acme-logistics-1.1.0-signed.json \
+  --require-signature \
+  --expected-hash "$PACK_HASH" \
+  models/**/*.sea
+```
+
+The content hash is computed with signature fields excluded, so it remains stable across signing and verification operations.
+
+## Signature Verification Errors
+
+The verifier reports specific error types:
+
+| Error                        | Cause                                                    |
+|------------------------------|----------------------------------------------------------|
+| `MissingSignature`          | The pack has no `signature` field in `trust`.            |
+| `InvalidSignatureFormat`    | The base64 signature cannot be decoded to 64 bytes.      |
+| `InvalidSignature`          | The Ed25519 signature does not verify against the public key and payload. |
+| `KeyParseError`             | The public key PEM is malformed or not Ed25519.          |
+| `Base64Error`               | The signature base64 encoding is invalid.                |
+
+## See Also
+
+- [Semantic Packs](semantic-packs.md) for the overall pack system.
+- [Semantic Pack Review Process](semantic-pack-review.md) for the review gate that precedes signing.
+- [Semantic Diagnostic Codes](diagnostics.md) for `pack_unsigned` and `pack_signature_invalid` codes.

package/docs/semantic-packs.md

@@ -0,0 +1,284 @@
+# Semantic Packs
+
+A **semantic pack** is a deterministic, review-gated, signed JSON artifact that defines an organization's approved vocabulary of business concepts, relations, metrics, dimensions, units, aliases, and mapping rules.
+
+Semantic packs act as a contract between the people who define business terminology and the tools that consume it. Every concept in a pack carries a human-reviewed definition, an owner, and a cryptographic hash of its semantic content. Downstream consumers---the DomainForge LSP, CI validators, and the SEA-Forge pipeline---resolve .sea source terms against the pack and emit structured diagnostics when terms are unknown, ambiguous, deprecated, or rejected.
+
+## What a Semantic Pack Is
+
+A semantic pack captures:
+
+- **Concepts**: the core nouns of your domain---entities, resources, roles, flows, policies, metrics, dimensions, and units.
+- **Relations**: typed predicates between concepts (e.g., `processes`, `owned_by`).
+- **Metrics and dimensions**: observability definitions with unit and dimension bindings.
+- **Aliases**: alternative names mapped to canonical concepts, each with a status (`approved`, `deprecated`, `ambiguous`, `blocked`).
+- **Mapping rules**: cross-domain or cross-system term mappings with confidence scores.
+
+Each pack is a self-contained JSON document with a stable content hash, a meaning fingerprint, and a review manifest hash. Two builds from identical inputs produce bit-for-bit identical output.
+
+## What a Semantic Pack Is Not
+
+- Not an embedding model or vector similarity index.
+- Not a customer-specific LSP binary---the LSP loads packs dynamically.
+- Not an ontology governance workflow UI---packs are the artifact that governance tooling produces.
+
+## Pack Authority States
+
+Every pack carries a `trust` object with two orthogonal dimensions: **approval state** and **signature state**.
+
+| Approval State      | Signature State | What It Powers                                      |
+|---------------------|-----------------|-----------------------------------------------------|
+| `candidate`         | `unsigned`      | Local development, IDE exploration                  |
+| `approved`          | `unsigned`      | CI warn mode, LSP with warnings, pre-release checks |
+| `approved`          | `signed`        | CI strict mode, SEA-Forge production pipeline       |
+| `rejected`          | any             | Nothing---rejected packs block downstream use       |
+
+An approved-but-unsigned pack allows teams to iterate on definitions while still enforcing review coverage. Moving to `signed` is the final gate before production use.
+
+## Three-Valued Semantic Truth
+
+DomainForge uses three-valued logic for concept resolution:
+
+| Truth Value | Meaning                                                        |
+|-------------|----------------------------------------------------------------|
+| `valid`     | The term resolved to an active, reviewed concept.              |
+| `invalid`   | The term resolved to a rejected concept or a blocked alias.    |
+| `unknown`   | The term could not be resolved, or resolved to a proposed concept, or is ambiguous. |
+
+`unknown` stays distinct from `invalid` even when mapped to an error severity in strict mode. An `unknown` result means "we lack enough information to confirm or deny"---the tooling should encourage the user to add a definition or resolve the ambiguity rather than treat it as a hard rejection.
+
+This separation lets the LSP offer "add to pack" quick-fix suggestions for `unknown` terms while still surfacing `invalid` terms as genuine errors.
+
+## Pack Lifecycle
+
+```
+candidate --> approved_unsigned --> approved_signed --> retired/revoked
+                  |                                      ^
+                  +---- rejected ------------------------+
+```
+
+1. **Candidate**: Built from .sea source files. No review required. Suitable for local exploration.
+2. **Approved (unsigned)**: All active concepts have matching review records with `approve` or `minor_amendment_no_semantic_change` decisions. Definition hashes are verified. Meaning version has been bumped if the fingerprint changed.
+3. **Approved (signed)**: An Ed25519 detached signature covers the pack content hash. The signing payload includes the content hash, pack ID, and schema version.
+4. **Retired/Revoked**: Listed in `replaces_pack_ids` of a successor pack. Downstream tools should treat retired packs as deprecated and redirect to the replacement.
+
+A pack never transitions backward. Once approved, it cannot return to candidate. Once signed, the signature can only be invalidated (tampered) or superseded by a new signed version.
+
+## Compatibility and Versioning
+
+Each pack carries several versioning fields:
+
+| Field                  | Purpose                                                                 |
+|------------------------|-------------------------------------------------------------------------|
+| `schema_version`       | The pack JSON schema version. Currently `0.3`.                          |
+| `pack_version`         | Semver for the pack artifact itself (e.g., `1.2.0`).                    |
+| `meaning_version`      | Semver for the semantic content. Must be bumped when `meaning_fingerprint` changes. |
+| `meaning_fingerprint`  | SHA-256 hash of all concept definition records. Changes when any concept definition, status, or decision_ref changes. |
+| `replaces_pack_ids`    | List of pack IDs that this pack supersedes.                             |
+
+The build system enforces that `meaning_version` increases whenever `meaning_fingerprint` changes. A pack that changes meaning without bumping the version fails the build with `meaning_version_not_bumped`.
+
+## CLI Examples
+
+### Building a Pack
+
+Build a candidate pack from .sea source files:
+
+```bash
+domainforge pack build \
+  --source "models/**/*.sea" \
+  --org acme \
+  --domain logistics \
+  --version 1.0.0 \
+  --meaning-version 1.0.0 \
+  --approval candidate \
+  --out packs/acme-logistics-1.0.0.json
+```
+
+Build an approved pack with review records:
+
+```bash
+domainforge pack build \
+  --source "models/**/*.sea" \
+  --org acme \
+  --domain logistics \
+  --version 1.1.0 \
+  --meaning-version 1.1.0 \
+  --approval approved \
+  --review reviews/logistics-review.jsonl \
+  --previous-pack packs/acme-logistics-1.0.0.json \
+  --out packs/acme-logistics-1.1.0.json
+```
+
+For the first approved pack in a domain, use `--allow-first-approved-version`:
+
+```bash
+domainforge pack build \
+  --source "models/**/*.sea" \
+  --org acme \
+  --domain logistics \
+  --version 1.0.0 \
+  --meaning-version 1.0.0 \
+  --approval approved \
+  --review reviews/logistics-review.jsonl \
+  --allow-first-approved-version \
+  --out packs/acme-logistics-1.0.0.json
+```
+
+### Validating Against a Pack
+
+Validate .sea files against a pack in warn mode:
+
+```bash
+domainforge pack validate \
+  --pack packs/acme-logistics-1.1.0.json \
+  --mode warn \
+  models/payment.sea models/shipping.sea
+```
+
+Validate in strict mode with signature required:
+
+```bash
+domainforge pack validate \
+  --pack packs/acme-logistics-1.1.0.json \
+  --mode strict \
+  --require-signature \
+  --expected-hash sha256:abc123... \
+  models/**/*.sea
+```
+
+### Inspecting a Pack
+
+```bash
+domainforge pack inspect --pack packs/acme-logistics-1.1.0.json
+
+# JSON output for tooling
+domainforge pack inspect --pack packs/acme-logistics-1.1.0.json --format json
+```
+
+### Diffing Two Packs
+
+```bash
+domainforge pack diff \
+  --old packs/acme-logistics-1.0.0.json \
+  --new packs/acme-logistics-1.1.0.json
+
+# JSON output
+domainforge pack diff \
+  --old packs/acme-logistics-1.0.0.json \
+  --new packs/acme-logistics-1.1.0.json \
+  --format json
+```
+
+Diff classifications:
+
+| Classification         | Meaning                                                |
+|------------------------|--------------------------------------------------------|
+| `additive`             | New concept added.                                     |
+| `definitional_change`  | Definition hash changed (non-breaking).                |
+| `deprecating`          | Concept status changed to deprecated.                  |
+| `breaking`             | Active concept removed, or meaning changed without version bump. |
+| `governance_critical`  | Status transition that is not deprecated or rejected.  |
+| `signature_only`       | Only the signature changed, semantic content is identical. |
+
+### Signing and Verifying
+
+```bash
+# Sign a pack
+domainforge pack sign packs/acme-logistics-1.1.0.json --key keys/acme-private.pem --out packs/acme-logistics-1.1.0-signed.json
+
+# Verify a pack signature
+domainforge pack verify packs/acme-logistics-1.1.0-signed.json --key keys/acme-public.pem
+```
+
+## Workspace Configuration
+
+Add a `.domainforge/config.json` to your project root to configure pack resolution:
+
+```json
+{
+  "semantic_packs": [
+    {
+      "path": "packs/acme-logistics-1.1.0.json",
+      "priority": 0
+    },
+    {
+      "path": "packs/acme-shared-2.0.0.json",
+      "priority": 1
+    }
+  ],
+  "validation_mode": "warn",
+  "deprecated_policy": "warn",
+  "require_signed_packs": false
+}
+```
+
+The LSP reads this configuration on startup and resolves terms against the configured packs in priority order.
+
+## CI Strict Mode
+
+For CI pipelines that enforce semantic correctness:
+
+```bash
+#!/bin/bash
+set -euo pipefail
+
+# Build and sign the pack
+domainforge pack build \
+  --source "models/**/*.sea" \
+  --org "$ORG" \
+  --domain "$DOMAIN" \
+  --version "$VERSION" \
+  --meaning-version "$MEANING_VERSION" \
+  --approval approved \
+  --review "reviews/review.jsonl" \
+  --previous-pack "packs/previous.json" \
+  --out "packs/current-unsigned.json"
+
+domainforge pack sign "packs/current-unsigned.json" \
+  --key "$SIGNING_KEY_PATH" \
+  --out "packs/current.json"
+
+# Validate all source files in strict mode
+domainforge pack validate \
+  --pack "packs/current.json" \
+  --mode strict \
+  --require-signature \
+  --expected-hash "$EXPECTED_HASH" \
+  models/**/*.sea
+```
+
+Exit codes for pack validation:
+
+| Code | Meaning                          |
+|------|----------------------------------|
+| 0    | Validation passed.               |
+| 1    | Semantic validation failed.      |
+| 2    | Parse error in input file.       |
+| 3    | Pack unsigned when required.     |
+| 4    | Signature verification failed.   |
+
+## Consumer Contract
+
+Downstream tools that consume semantic packs must follow these rules:
+
+1. **Never modify a pack in place.** Treat packs as immutable artifacts. To change vocabulary, build a new pack version.
+2. **Always verify the signature before trusting content.** Unsigned packs are acceptable in development; signed packs are required in production.
+3. **Respect the three-valued truth.** Map `unknown` to actionable suggestions, not hard errors, unless in strict mode.
+4. **Check `meaning_version` before `pack_version`.** A meaning change without a version bump is a bug, not a feature.
+5. **Use `normalize_lookup_key` for term matching.** The normalization algorithm (NFC, case-fold, whitespace collapse) is the canonical equality function for lookup keys.
+6. **Honor `replaces_pack_ids`.** When loading multiple packs, redirect references to superseded packs to the replacement.
+
+### Consumers
+
+- **Context Kernel**: Uses packs to build context windows for LLM-assisted architecture modeling.
+- **SWE_SEED**: Uses packs to validate that generated code references approved business concepts.
+- **SEA-Forge Pipeline**: Uses signed packs as the authoritative vocabulary for production code generation.
+- **DomainForge LSP**: Loads packs at workspace initialization and provides real-time diagnostics.
+
+## See Also
+
+- [Semantic Pack Review Process](semantic-pack-review.md)
+- [Semantic Pack Signing and Verification](semantic-pack-signing.md)
+- [Semantic Diagnostic Codes](diagnostics.md)
+- [Semantic Modeling Concepts](explanations/semantic-modeling-concepts.md)

package/docs/specs/ADR-001-sea-dsl-semantic-source-of-truth.md

@@ -0,0 +1,33 @@
+# ADR-001: Treat SEA-DSL as the Semantic Source of Truth
+
+**Status:** Accepted  
+**Date:** 2025-12-14  
+**Deciders:** DomainForge Architecture Team
+
+## Context
+
+SEA-DSL models enterprise meaning using constructs such as `Entity`, `Resource`, `Flow`, `Policy`, `Metric`, and `ConceptChange`. There is a need to generate schemas, code, and runtime contracts (DDD/Hex, Protobuf, JSON, XML) without collapsing the DSL into a schema-first or implementation-first language.
+
+## Decision
+
+SEA-DSL SHALL remain a **semantic language only**.
+
+All schemas, code artifacts, and runtime contracts SHALL be produced via **projection engines** from the semantic graph.
+
+## Consequences
+
+### Positive
+
+- SEA-DSL remains stable, expressive, and technology-agnostic.
+- Multiple projections (Protobuf, DDD manifests, GraphQL, etc.) can coexist.
+- Schema evolution is governed by semantic evolution, not implementation drift.
+
+### Negative
+
+- Requires additional tooling (projection engines) to produce usable artifacts.
+- Learning curve for users who expect schema-first approaches.
+
+## Related
+
+- [ADR-002: Introduce Projection as a First-Class DSL Construct](./ADR-002-projection-first-class-construct.md)
+- [ADR-003: Add Protobuf as a Projection Target](./ADR-003-protobuf-projection-target.md)

package/docs/specs/ADR-002-projection-first-class-construct.md

@@ -0,0 +1,50 @@
+# ADR-002: Introduce Projection as a First-Class DSL Construct
+
+**Status:** Accepted  
+**Date:** 2025-12-14  
+**Deciders:** DomainForge Architecture Team
+
+## Context
+
+Enterprises need explicit, governable control over _which_ parts of the semantic model are projected _where_, _for whom_, and _under what compatibility guarantees_.
+
+## Decision
+
+Introduce a top-level `Projection` construct in SEA-DSL that:
+
+- Declares **scope** (namespace, tags, profiles)
+- Declares **target format** (protobuf, ddd-manifest, etc.)
+- Declares **governance metadata** (stability, compatibility, audience)
+
+Projection constructs SHALL NOT contain format-specific details.
+
+### Example Syntax
+
+```sea
+Projection PaymentsProto {
+  source: "com.example.finance.payments"
+  format: protobuf
+  compatibility: backward
+  stability: stable
+  audience: ["agents", "services"]
+}
+```
+
+## Consequences
+
+### Positive
+
+- Projections are auditable and declarative.
+- Compatibility rules can be enforced centrally.
+- Multiple consumers (agents, services, pipelines) can rely on explicit contracts.
+
+### Negative
+
+- Adds complexity to the DSL grammar.
+- Requires governance tooling to validate projection declarations.
+
+## Related
+
+- [ADR-001: Treat SEA-DSL as the Semantic Source of Truth](./ADR-001-sea-dsl-semantic-source-of-truth.md)
+- [ADR-003: Add Protobuf as a Projection Target](./ADR-003-protobuf-projection-target.md)
+- [ADR-004: Enforce Projection Compatibility Semantics](./ADR-004-projection-compatibility-semantics.md)

package/docs/specs/ADR-003-protobuf-projection-target.md

@@ -0,0 +1,51 @@
+# ADR-003: Add Protobuf as a Projection Target (Not a Semantic Format)
+
+**Status:** Accepted  
+**Date:** 2025-12-14  
+**Deciders:** DomainForge Architecture Team
+
+## Context
+
+SEA requires high-performance, versioned, cross-language contracts for agents, services, governance events, and telemetry.
+
+## Decision
+
+Add **Protobuf** as a supported projection target.
+
+Protobuf SHALL be generated **only** from the semantic graph via a projection engine.
+
+Protobuf SHALL NOT:
+
+- Replace SEA-DSL
+- Encode policy logic
+- Encode semantic evolution rules
+
+## Rationale
+
+Protobuf provides:
+
+- Efficient binary serialization
+- Strong cross-language support
+- Built-in versioning primitives
+- Industry-standard tooling
+
+By treating Protobuf as a projection target rather than a source format, we maintain SEA-DSL's semantic integrity while gaining Protobuf's runtime benefits.
+
+## Consequences
+
+### Positive
+
+- Agents receive stable, deterministic contracts.
+- Runtime governance and telemetry become efficient and auditable.
+- SEA avoids becoming data-centric or schema-centric.
+
+### Negative
+
+- Requires building and maintaining a Protobuf projection engine.
+- Some Protobuf features (services, streaming) may require additional mapping work.
+
+## Related
+
+- [ADR-001: Treat SEA-DSL as the Semantic Source of Truth](./ADR-001-sea-dsl-semantic-source-of-truth.md)
+- [ADR-002: Introduce Projection as a First-Class DSL Construct](./ADR-002-projection-first-class-construct.md)
+- [ADR-004: Enforce Projection Compatibility Semantics](./ADR-004-projection-compatibility-semantics.md)