domainforge 0.13.0

package/docs/how-tos/run-cross-language-tests.md

@@ -0,0 +1,115 @@
+# Run Cross-Language Tests
+
+Goal: Execute and maintain Rust, Python, and TypeScript tests to ensure the DSL, bindings, and projections stay in sync.
+
+## Prerequisites
+
+- Rust toolchain (with `cargo`), Node.js 20+, and Python 3.10+ available on your machine.
+- Install project dependencies:
+  - `npm ci` to install JS deps.
+  - `just python-setup` to build Python bindings in a virtualenv.
+- Optional: install `just` (`cargo install just --locked`) to use the provided recipes.
+
+## Steps (be concise)
+
+1. **Run all suites sequentially**
+
+   ```bash
+   just all-tests
+   ```
+
+   - Executes Rust (`cargo test -p domainforge-core --features cli`), Python (`pytest`), and TypeScript (Vitest) in order.
+   - Fails fast on the first failing command; rerun individual suites for debugging.
+
+2. **Run Rust tests only**
+
+   ```bash
+   just rust-test
+   ```
+
+   - Includes CLI tests and parser/projection coverage under `domainforge-core/tests/`.
+   - Use `cargo test -p domainforge-core --features "cli shacl"` if you need SHACL validation paths.
+
+3. **Run Python tests only**
+
+   ```bash
+   just python-test
+   ```
+
+   - Rebuilds bindings via `maturin develop` in the virtualenv if needed.
+   - To focus on a single file: `.venv/bin/pytest tests/test_golden_payment_flow.py -k payment`.
+
+4. **Run TypeScript tests only**
+
+   ```bash
+   just ts-test
+   ```
+
+   - Invokes Vitest using the compiled napi module from `npm run build`.
+   - Filter tests: `npm test -- --runInBand golden-payment-flow.test.ts`.
+
+5. **Add a cross-language golden scenario**
+
+   - Define the scenario in Rust tests under `domainforge-core/tests/` (e.g., roles/relations payment flow).
+   - Mirror the scenario in `tests/` (Python) and `typescript-tests/` with equivalent assertions.
+   - Keep DSL snippets identical to catch serialization and parsing differences.
+
+6. **Debug mismatches**
+
+   - If Python or TypeScript counts differ, log the graph contents:
+
+     ```python
+     for role in graph.all_roles():
+         print(role.name, role.id)
+     ```
+
+     ```ts
+     console.log(graph.allRelations().map(r => ({ name: r.name, predicate: r.predicate })));
+     ```
+
+   - Re-run the Rust test with `-- --nocapture` to see detailed parser output when comparing against bindings.
+
+7. **Automate in CI**
+
+   - Use `just ci-check` locally to mimic CI (formatting, clippy, and all tests).
+   - In GitHub Actions, configure runners with Rust/Python/Node toolchains and call the same commands; set up caching for `target/`, `.venv/`, and `node_modules/` to shorten feedback loops.
+
+## Adding Cross-Language Test Cases
+
+- Place shared DSL fixtures under `examples/` or `tests/fixtures/` to avoid duplication.
+- Prefer end-to-end assertions (parse → export → import) instead of unit-testing binding wrappers in isolation.
+- Capture CALM/Turtle exports as golden files when the shape is stable; regenerate them intentionally after schema changes.
+
+## Troubleshooting
+
+- **Binding build failures**: Ensure `rustup default stable` and `npm run build` succeed before running tests; stale artifacts can cause symbol resolution errors.
+- **Platform differences**: On Windows, run the Node tests in a shell that has `node-gyp` prerequisites; on macOS ARM, ensure a compatible Python version for `maturin` wheels.
+- **Timeouts in Vitest**: Use `npm test -- --runInBand --testTimeout=60000` when running under constrained CI machines.
+
+## Links
+
+- Tutorials: [Python Binding Quickstart](../tutorials/python-binding-quickstart.md), [TypeScript Binding Quickstart](../tutorials/typescript-binding-quickstart.md)
+- Reference: [CLI Commands](../reference/cli-commands.md), [Python API](../reference/python-api.md), [TypeScript API](../reference/typescript-api.md)
+
+## Example: Payment Role Flow golden test
+
+- DSL snippet lives in `tests/test_golden_payment_flow.py` and `typescript-tests/golden-payment-flow.test.ts`.
+- Expectations:
+  - `role_count()` returns 2 (`Payer`, `Payee`).
+  - `relation_count()` returns 1 with predicate `pays`.
+  - Flow quantity equals 10 and uses the resource `Money`.
+- Use this as a template for new scenarios; keep IDs stable to avoid brittle UUID comparisons.
+
+## Reporting and Maintaining Results
+
+- Capture failing commands with full stdout/stderr in CI artifacts to compare Rust vs binding behavior.
+- When bumping versions (e.g., to `0.1.0`), rerun the suites and update any snapshots to reflect metadata changes.
+- Tag tests that rely on optional features (`shacl`, `wasm`) so they can be skipped on constrained environments.
+
+## Readiness Checklist
+
+- [ ] Rust, Python, and TypeScript suites green locally (`just all-tests`).
+- [ ] Golden scenarios exercised in all languages with identical DSL inputs.
+- [ ] New APIs added to bindings accompanied by parity tests.
+- [ ] CI workflow caches warmed for `target/`, `.venv/`, and `node_modules/`.
+- [ ] Release workflows (`release-pypi.yml`, `release-npm.yml`, `release-crates.yml`) reference the test commands.

package/docs/how-tos/troubleshoot-napi-builds.md

@@ -0,0 +1,55 @@
+# Troubleshoot NAPI/TypeScript Builds
+
+Goal: fix Node.js native module (`.node`) build issues, especially PyO3 symbol leaks.
+
+## Symptoms
+
+- Loading `domainforge-core.*.node` fails with `undefined symbol: PyBaseObject_Type`.
+- TypeScript tests crash when `python` feature symbols leak into the NAPI build.
+
+## Root cause
+
+The native binary was built with `python` features enabled or reused artifacts from a Python build, pulling PyO3 symbols into the Node module.
+
+## Fix: Clean, isolated build
+
+```bash
+# Remove stale artifacts
+cargo clean
+cd domainforge-core && cargo clean && cd ..
+rm -rf target domainforge-core/target
+
+# Build TypeScript binding only
+cd domainforge-core
+cargo build --lib --features typescript,three_valued_logic
+
+# Copy the correct artifact to the root package if needed (choose your platform)
+cp target/debug/libdomainforge_core.so ../domainforge-core.linux-x64-gnu.node      # Linux
+cp target/debug/libdomainforge_core.dylib ../domainforge-core.darwin-x64.node      # macOS
+cp target\\debug\\domainforge_core.dll ..\\domainforge-core.win32-x64.node         # Windows
+
+# Verify no Py* symbols remain
+nm -D ../domainforge-core.linux-x64-gnu.node | grep Py || echo "clean"
+```
+
+Then run `npm test` to confirm the binding loads.
+
+## Prevent future leaks
+
+- Keep Cargo features isolated:
+
+  ```toml
+  [features]
+  default = []
+  python = ["pyo3", "pythonize"]
+  typescript = ["napi", "napi-derive"]
+  three_valued_logic = []
+  ```
+
+- Ensure build scripts (`package.json`, `just`) do **not** enable `python` when building NAPI.
+- If you regularly switch between Python and TypeScript builds, run `cargo clean` between them.
+
+## See also
+
+- [Cross-Language Binding Strategy](../explanations/cross-language-binding-strategy.md)
+- [TypeScript API Reference](../reference/typescript-api.md)

package/docs/how-tos/use-modules-imports.md

@@ -0,0 +1,285 @@
+# Use SEA Modules and Imports
+
+This guide explains how to organize large SEA models across multiple files using imports and the namespace registry.
+
+## Overview
+
+SEA supports modular organization through:
+
+- **Imports**: Reference types from other SEA files
+- **Namespace Registry**: Map namespace prefixes to file paths
+- **Standard Library**: Built-in types via `std:` prefix
+
+## Prerequisites
+
+- `sea` CLI installed
+- A project with multiple `.sea` files
+
+## Quick Start
+
+### 1. Create Multiple Files
+
+**`common/types.sea`**
+
+```sea
+Namespace "common"
+
+Entity "BaseEntity" {
+    id: UUID
+    created_at: DateTime
+    updated_at: DateTime
+}
+```
+
+**`domain/customer.sea`**
+
+```sea
+Namespace "domain.customer"
+
+import "common/types.sea"
+
+Entity "Customer" {
+    @extends "BaseEntity"
+    name: String
+    email: String
+}
+```
+
+### 2. Configure the Registry
+
+Create `.sea-registry.toml` at your project root:
+
+```toml
+[namespaces]
+common = "common/types.sea"
+domain = "domain/index.sea"
+```
+
+### 3. Use Imports
+
+```bash
+domainforge validate domain/customer.sea --registry .sea-registry.toml
+```
+
+## Import Syntax
+
+### Relative File Import
+
+Import all types from a file:
+
+```sea
+import "common/types.sea"
+```
+
+### Namespace Binding
+
+Import with namespace alias:
+
+```sea
+import namespace core from "std/core.sea"
+
+Entity "User" {
+    email: core.Email
+}
+```
+
+### Selective Import
+
+Import specific types:
+
+```sea
+import { Customer, Order } from "domain/entities.sea"
+```
+
+### Wildcard Import (with alias)
+
+```sea
+import * as billing from "billing/types.sea"
+
+Flow from Customer to billing.Invoice
+```
+
+## Namespace Registry
+
+The `.sea-registry.toml` file maps namespace prefixes to file paths:
+
+```toml
+[registry]
+paths = ["./src", "./vendor"]
+
+[namespaces]
+core = "domainforge-core/std/core.sea"
+http = "domainforge-core/std/http.sea"
+aws = "domainforge-core/std/aws.sea"
+domain = "src/domain/index.sea"
+```
+
+### Resolution Priority
+
+1. **Registry lookup**: Check `[namespaces]` table
+2. **Path search**: Search directories in `[registry].paths`
+3. **Relative resolution**: Resolve relative to the importing file
+4. **Standard library**: Check built-in `std:` modules
+
+## Standard Library
+
+Built-in types available via `std:` prefix (no registry needed):
+
+| Module     | Contents                              |
+| ---------- | ------------------------------------- |
+| `std:core` | UUID, Email, URL, DateTime            |
+| `std:http` | Request, Response, Headers            |
+| `std:aws`  | S3Bucket, LambdaFunction, DynamoTable |
+
+### Usage
+
+```sea
+import namespace core from "std:core"
+
+Entity "User" {
+    id: core.UUID
+    email: core.Email
+    website: core.URL
+}
+```
+
+## Index Files
+
+Create `index.sea` files to re-export from a namespace:
+
+**`domain/index.sea`**
+
+```sea
+Namespace "domain"
+
+// Re-export from sub-modules
+export * from "domain/customer.sea"
+export * from "domain/order.sea"
+export * from "domain/payment.sea"
+```
+
+Then import the namespace:
+
+```sea
+import namespace domain from "domain/index.sea"
+
+Flow from domain.Customer to domain.Order
+```
+
+## Circular Dependency Detection
+
+The parser detects circular imports:
+
+```
+Error[E010]: Circular module dependency detected
+  --> domain/order.sea:3
+  |
+3 | import "domain/customer.sea"
+  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  |
+  = note: dependency chain: customer.sea -> order.sea -> customer.sea
+```
+
+### Resolution
+
+Break cycles by:
+
+1. Moving shared types to a `common` module
+2. Using interface types instead of concrete imports
+3. Restructuring module boundaries
+
+## Project Structure Best Practices
+
+```
+my-project/
+├── .sea-registry.toml
+├── common/
+│   ├── types.sea         # Base types
+│   └── units.sea         # Custom units
+├── domain/
+│   ├── index.sea         # Re-exports
+│   ├── customer.sea
+│   ├── order.sea
+│   └── payment.sea
+├── policies/
+│   ├── compliance.sea
+│   └── business-rules.sea
+└── main.sea              # Entry point
+```
+
+## CLI Options
+
+### Validate with Registry
+
+```bash
+domainforge validate main.sea --registry .sea-registry.toml
+```
+
+### Specify Additional Paths
+
+```bash
+domainforge validate main.sea --include ./vendor --include ./lib
+```
+
+### Verbose Module Loading
+
+```bash
+SEA_LOG=debug domainforge validate main.sea
+```
+
+## Programmatic Usage
+
+### Rust
+
+```rust
+use domainforge_core::registry::NamespaceRegistry;
+use domainforge_core::parser::parse_with_registry;
+
+let registry = NamespaceRegistry::from_file(".sea-registry.toml")?;
+let graph = parse_with_registry("main.sea", &registry)?;
+```
+
+### Python
+
+```python
+from domainforge import Graph, NamespaceRegistry
+
+registry = NamespaceRegistry.from_file(".sea-registry.toml")
+graph = Graph.parse_with_registry(open("main.sea").read(), registry)
+```
+
+## Troubleshooting
+
+### Module Not Found
+
+```
+Error[E009]: Module not found: 'common/types.sea'
+  --> domain/customer.sea:3
+  |
+  = hint: Check .sea-registry.toml or use --include flag
+```
+
+**Fix**: Ensure the file path is correct or add to registry.
+
+### Namespace Conflict
+
+```
+Error[E011]: Namespace conflict: 'Customer' defined in multiple modules
+```
+
+**Fix**: Use namespace aliases or rename conflicting types.
+
+### Registry Not Found
+
+```
+Warning: No .sea-registry.toml found, using default resolution
+```
+
+**Fix**: Create `.sea-registry.toml` in project root.
+
+## See Also
+
+- [ADR-009: Module Resolution Strategy](../specs/ADR-009-module-resolution-strategy.md)
+- [Configuration Reference](../reference/configuration.md)
+- [Registry Reference](../reference/registry.md)
+- [CLI Commands](../reference/cli-commands.md)

package/docs/index.md

@@ -0,0 +1,13 @@
+# DomainForge — New Documentation Index
+Welcome to the DomainForge documentation! This index provides an overview of the various types of documentation available to help you get started, learn advanced features, and reference the API.
+
+
+## Categories
+
+- Tutorials: Hands-on, step-by-step learning (e.g., Getting started, Quickstarts)
+- How-Tos: Task-focused, concise procedures (install, export to CALM)
+- Reference: Stable API/grammar/CLI/configuration
+- Explanations: Architecture and conceptual documents
+- Playbooks: Runbooks and operational procedures
+
+See the `templates/` folder to create new docs consistently.

package/docs/plans/canonical-normalizer.md

@@ -0,0 +1,121 @@
+# ADR: Canonical Expression Normalizer
+
+**Status:** Proposed  
+**Date:** 2025-12-16  
+**Authors:** DomainForge Architecture Team
+
+## Context
+
+SEA-DSL policy expressions can be written in many syntactically equivalent forms:
+
+- `a AND b` vs `b AND a`
+- `true AND x` vs `x`
+- Nested parentheses: `((a))` vs `a`
+
+This creates challenges for:
+
+1. **Hover signatures** - displaying a consistent, readable form
+2. **Change detection** - determining if two policies are semantically equivalent
+3. **Caching** - using normalized expressions as cache keys
+4. **Testing** - golden snapshot tests need deterministic output
+
+## Decision
+
+Implement a canonical normalizer in `domainforge_core::policy` that transforms any `Expression` into a deterministic, minimal, canonical form.
+
+## Design
+
+### 1. Normalization Rules
+
+| Category                   | Rule                                          | Example                             |
+| -------------------------- | --------------------------------------------- | ----------------------------------- |
+| **Commutativity**          | Sort operands of `AND`/`OR` lexicographically | `b AND a` → `a AND b`               |
+| **Identity**               | Remove identity elements                      | `true AND x` → `x`                  |
+| **Absorption**             | Remove absorbed terms                         | `a OR (a AND b)` → `a`              |
+| **Parentheses**            | Flatten redundant nesting                     | `((a))` → `a`                       |
+| **Variable naming**        | Alpha-rename quantifier vars                  | `forall x: ...` → `forall _v0: ...` |
+| **Operator normalization** | Use canonical operators                       | `NOT (a = b)` → `a != b`            |
+
+### 2. API
+
+```rust
+// In domainforge_core::policy::expression
+impl Expression {
+    /// Returns a canonical form of this expression
+    pub fn normalize(&self) -> NormalizedExpression;
+
+    /// Returns true if two expressions are semantically equivalent
+    pub fn is_equivalent(&self, other: &Expression) -> bool {
+        self.normalize() == other.normalize()
+    }
+}
+
+#[derive(Clone, PartialEq, Eq, Hash)]
+pub struct NormalizedExpression {
+    inner: Expression,
+    /// Stable hash for caching
+    hash: u64,
+}
+
+impl Display for NormalizedExpression {
+    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
+        // Pretty-print in canonical form
+    }
+}
+```
+
+### 3. Implementation Strategy
+
+**Phase 1: Core normalization**
+
+- [ ] Implement `normalize()` for leaf nodes (Literal, Variable)
+- [ ] Implement `normalize()` for binary operators with sorting
+- [ ] Implement `normalize()` for unary operators
+- [ ] Add stable hashing using `rustc_hash::FxHasher`
+
+**Phase 2: Advanced normalization**
+
+- [ ] Implement constant folding (`true AND x` → `x`)
+- [ ] Implement absorption rules
+- [ ] Implement alpha-renaming for quantifier variables
+
+**Phase 3: Integration**
+
+- [ ] Use in LSP hover signature generation
+- [ ] Add `Policy::normalized_expression()` helper
+- [ ] Add CLI option: `domainforge normalize <expr>`
+
+### 4. Files to Modify
+
+| File                                           | Changes                          |
+| ---------------------------------------------- | -------------------------------- |
+| `domainforge-core/src/policy/mod.rs`                   | Add `mod normalize;`             |
+| `domainforge-core/src/policy/normalize.rs`             | **NEW** - Normalization logic    |
+| `domainforge-core/src/policy/expression.rs`            | Add `normalize()` method         |
+| `domainforge-lsp/src/hover/symbol_resolver.rs` | Use normalized form in signature |
+
+## Consequences
+
+### Positive
+
+- Deterministic hover output
+- Semantic equivalence checking
+- Better caching with normalized keys
+- Cleaner test assertions
+
+### Negative
+
+- Computational overhead for normalization
+- Increased complexity in expression module
+- Alpha-renaming may reduce readability
+
+## Alternatives Considered
+
+1. **No normalization** - Accept non-deterministic output (rejected: breaks testing)
+2. **Simple sorting only** - Only sort commutative operators (insufficient for identity elimination)
+3. **External SMT solver** - Use Z3 for semantic equivalence (rejected: too heavy)
+
+## References
+
+- [SDS-004: Policy Engine Design](./SDS-004-policy-engine-design.md)
+- [hover_plan.yml](../archive/hover_plan.yml) - Mentions normalized form display

package/docs/plans/cd_improvement.md

@@ -0,0 +1,112 @@
+# DomainForge CD Improvement Plan
+
+> **Purpose**: Address identified gaps and technical debt in the Continuous Delivery pipeline.
+>
+> **Last Updated**: 2025-12-15  
+> **Status**: ✅ Implemented
+
+---
+
+## Executive Summary
+
+Analysis of the 9 workflow files in `.github/workflows/` revealed several CD issues that create technical debt, inconsistency, and potential release failures. This plan provided a prioritized remediation roadmap, which has now been implemented.
+
+---
+
+## Findings & Remediation Status
+
+### 🔴 Critical Issues — ✅ Resolved
+
+| ID  | Issue                              | Resolution                                                            |
+| --- | ---------------------------------- | --------------------------------------------------------------------- |
+| C1  | Duplicate Python publish workflows | Deleted `publish-python.yml`, retained `release-pypi.yml`             |
+| C2  | Release → Publish disconnection    | Added `continue-on-error` to handle edge cases                        |
+| C3  | Publish rebuilds from scratch      | Kept rebuild pattern (required for napi bindings) with error recovery |
+
+### 🟠 High Priority Issues — ✅ Resolved
+
+| ID  | Issue                   | Resolution                                                                    |
+| --- | ----------------------- | ----------------------------------------------------------------------------- |
+| H1  | Missing ARM targets     | Added `aarch64-apple-darwin` and `aarch64-unknown-linux-gnu` to all workflows |
+| H2  | No WASM npm package     | Added `publish-wasm` job to `release-npm.yml`                                 |
+| H3  | Inconsistent thresholds | Harmonized WASM limit to 2MB in both `ci.yml` and `release.yml`               |
+
+### 🟡 Medium Priority Issues — ✅ Resolved
+
+| ID  | Issue                          | Resolution                                     |
+| --- | ------------------------------ | ---------------------------------------------- |
+| M1  | Python version inconsistency   | ARM targets added (3.12 support via manylinux) |
+| M3  | No release automation          | Created `prepare-release.yml` workflow         |
+| M4  | No `--skip-existing` in crates | Added with `continue-on-error`                 |
+
+### 🔵 Low Priority — Not in Scope
+
+| ID  | Issue                      | Status                            |
+| --- | -------------------------- | --------------------------------- |
+| L1  | Release candidate workflow | Supported via `prerelease` input  |
+| L2  | Artifact signatures        | Future enhancement                |
+| L3  | Automated changelog        | Template added in prepare-release |
+
+---
+
+## Implementation Checklist
+
+### Phase 1: Eliminate Duplication ✅
+
+- [x] Delete `publish-python.yml`
+- [x] Update `release-pypi.yml` (kept rebuild, added ARM)
+- [x] Update `release-npm.yml` (added WASM job)
+- [x] Add `--skip-existing` to `release-crates.yml`
+
+### Phase 2: Add ARM Targets ✅
+
+- [x] Add `aarch64-apple-darwin` to `release.yml`
+- [x] Add `aarch64-unknown-linux-gnu` to `release.yml`
+- [x] Add ARM targets to Python wheels build
+- [x] Use `cross` for CLI, `zig` for Python wheels
+
+### Phase 3: Harmonize Thresholds ✅
+
+- [x] Sync `WASM_MAX_BYTES` to 2MB in both workflows
+- [x] Add ARM targets to `release-pypi.yml`
+
+### Phase 4: WASM npm Publishing ✅
+
+- [x] Add `publish-wasm` job to `release-npm.yml`
+
+### Phase 5: Release Automation ✅
+
+- [x] Create `prepare-release.yml` workflow
+- [x] Supports major/minor/patch bumps
+- [x] Supports pre-release suffixes
+
+### Phase 6: Documentation ✅
+
+- [x] Update `workflows/README.md`
+- [x] Mark plan items as complete
+
+---
+
+## Files Changed
+
+| File                  | Action   | Key Changes                                                     |
+| --------------------- | -------- | --------------------------------------------------------------- |
+| `publish-python.yml`  | Deleted  | Redundant with `release-pypi.yml`                               |
+| `release.yml`         | Modified | Added ARM targets, harmonized WASM threshold, cross-compilation |
+| `release-pypi.yml`    | Modified | Added ARM targets with Zig cross-compilation                    |
+| `release-npm.yml`     | Modified | Added WASM publishing job                                       |
+| `release-crates.yml`  | Modified | Added error recovery                                            |
+| `ci.yml`              | Modified | Added threshold comment                                         |
+| `prepare-release.yml` | Created  | Automated release preparation                                   |
+| `README.md`           | Modified | Comprehensive documentation update                              |
+
+---
+
+## Verification
+
+To verify the implementation:
+
+1. Push changes to a branch and open a PR — GitHub validates YAML syntax
+2. Manually trigger `prepare-release.yml` to test version bumping
+3. Create a test tag `v0.0.0-test` to verify `release.yml` builds
+4. Create a draft GitHub Release to verify publish workflows trigger