domainforge 0.13.0

package/docs/playbooks/adding-new-primitive.md

@@ -0,0 +1,68 @@
+# Adding a New Primitive
+
+This playbook outlines the steps required to add a new primitive (e.g., `Zone`, `Cluster`) to the DomainForge language.
+
+**Warning**: This is a cross-cutting change that affects Core, Grammar, and all Bindings.
+
+## Checklist
+
+### 1. Rust Core (`domainforge-core`)
+
+- [ ] **Grammar**: Update `domainforge-core/grammar/sea.pest`. Add the new keyword and syntax rules.
+- [ ] **AST**: Update `domainforge-core/src/parser/ast.rs` to include the new node type.
+- [ ] **Primitive Struct**: Create `domainforge-core/src/primitives/new_primitive.rs`. Implement `Debug`, `Clone`, `Serialize`.
+- [ ] **Module Exports**: Add the new module to `domainforge-core/src/primitives/mod.rs` and export it (`pub mod new_primitive;` and `pub use` if needed).
+- [ ] **Graph**: Update `domainforge-core/src/graph/mod.rs`. Add an `IndexMap` for the new primitive, expose it with `pub mod`/`pub use`, and wire the new collection into Graph.
+- [ ] **Parser Logic**: Update `domainforge-core/src/parser/mod.rs` to transform the AST node into the Primitive struct, add the module to the `mod` list, and export it.
+
+### 2. Bindings
+
+- [ ] **Python**: Update `domainforge-core/src/python/lib.rs` and `domainforge-core/src/python/primitives.rs` to add the `#[pyclass]` wrapper and register it in `lib.rs`.
+- [ ] **TypeScript**: Update `domainforge-core/src/typescript/primitives.rs` and `domainforge-core/src/typescript/index.ts` to add the napi-compatible struct and export it.
+- [ ] **WASM**: Update `domainforge-core/src/wasm/primitives.rs` and `domainforge-core/src/wasm/lib.rs` if the primitive needs specific exposure.
+
+### 3. Testing
+
+- [ ] **Rust Tests**: Add a test case in `domainforge-core/tests/parser_tests.rs`.
+- [ ] **Python Tests**: Add a test in `tests/test_primitives.py`.
+- [ ] **TypeScript Tests**: Add a test in `typescript-tests/primitives.test.ts`.
+
+### 4. Documentation
+
+- [ ] Update `semantic-modeling-concepts.md`.
+- [ ] Update `sea.pest` comments.
+
+## Example: Adding "Zone"
+
+1. **Grammar**:
+
+   ```pest
+   zone_decl = { "zone" ~ identifier ~ "{" ~ (property)* ~ "}" }
+   ```
+
+2. **Struct**:
+
+   ```rust
+   #[derive(Debug, Clone, Serialize, Deserialize)]
+   pub struct Zone {
+       pub id: ConceptId,
+       pub properties: IndexMap<String, Value>,
+   }
+   ```
+
+3. **Graph**:
+
+   ```rust
+      pub struct Graph {
+         pub zones: IndexMap<ConceptId, Zone>,
+         // ... other primitive collections (entities, resources, flows, roles, relations, instances)
+      }
+   ```
+
+## Verification
+
+Run the full suite to ensure no regressions:
+
+```bash
+just all-tests
+```

package/docs/playbooks/debugging-parser-failures.md

@@ -0,0 +1,42 @@
+# Debugging Parser Failures
+
+When the parser fails to process a `.sea` file, follow these steps to diagnose the issue.
+
+## 1. Read the Error Message
+
+Pest provides detailed error locations.
+```text
+Error:   --> 3:12
+  |
+3 | entity Foo { type = "service"
+  |            ^--- expected "}"
+```
+*Interpretation*: Missing closing brace.
+
+## 2. Common Syntax Mistakes
+
+- **Missing Quotes**: Strings must be double-quoted. `type = service` is wrong; `type = "service"` is right.
+- **Trailing Commas**: SEA does not use commas between properties. Use newlines.
+- **Keywords**: Ensure you aren't using a reserved keyword as an identifier.
+
+## 3. Using the Pest Debugger
+
+If you are modifying the grammar, you can use either the web-based playground or the command-line debugger depending on your workflow:
+
+- Web playground (fast, visual):
+  1. Go to [pest.rs](https://pest.rs/).
+  2. Paste the content of `domainforge-core/grammar/sea.pest` into the grammar box.
+  3. Paste your failing input into the input box.
+  4. Watch the rule matching visualization to see where it diverges.
+
+- CLI debugger (`pest_debugger`) — useful for iterative debugging locally:
+  1. Install the CLI via cargo: `cargo install pest_debugger` or via `cargo install --git https://github.com/pest-parser/pest` if you prefer the latest.
+  2. Run the CLI against your grammar and failing input: `pest_debugger --grammar domainforge-core/grammar/sea.pest --input test.repro.sea`.
+  3. Use the interactive prompt to step rules and inspect matches.
+
+## 4. Reporting Grammar Bugs
+
+If valid syntax is rejected:
+1. Create a minimal reproduction file (`repro.sea`).
+2. Open an issue with the file content and the error output.
+3. Tag with `area/parser`.

package/docs/playbooks/local-release-preparation.md

@@ -0,0 +1,139 @@
+# Local Release Preparation
+
+This playbook describes how to prepare a release locally using the just recipes, allowing you to preview changes, generate replay scripts, and test the release process before triggering the GitHub Actions workflow.
+
+## Available Recipes
+
+| Recipe                         | Purpose                                                         |
+| ------------------------------ | --------------------------------------------------------------- |
+| `just release-preview [bump]`  | Preview all changes + generate replay script (no modifications) |
+| `just prepare-release [bump]`  | Apply version bump to all files                                 |
+| `just changelog-entry VERSION` | Add a changelog entry for a specific version                    |
+| `just test-changelog-logic`    | Test changelog insertion logic (restores original)              |
+
+Where `[bump]` is one of: `patch`, `minor`, `major`
+
+## Prerequisites
+
+- `jq` installed (for JSON processing)
+- `just` command runner
+- Clean git working directory (recommended)
+
+## Workflow
+
+### 1. Create a Release Branch
+
+Always create a dedicated branch for the release. This allows you to group the changelog updates, version bumps, and any other release-related changes (like updating this playbook!) into a single PR.
+
+```bash
+# From main
+git checkout -b release/x.y.z main
+# Example
+git checkout -b release/0.7.0 main
+```
+
+### 2. Preview Changes (Recommended)
+
+```bash
+just release-preview minor
+```
+
+This will:
+
+- Calculate the new version based on bump type
+- Show unified diffs for all affected files:
+  - `domainforge-core/Cargo.toml`
+  - `pyproject.toml`
+  - `package.json`
+  - `CHANGELOG.md`
+- Generate an executable replay script (e.g., `release-0.5.0.sh`)
+
+**No files are modified** - this is purely a preview.
+
+### 3. Apply Changes
+
+You have two options:
+
+**Option A: Run the generated script**
+
+```bash
+./release-0.7.0.sh
+```
+
+**Option B: Use the just recipe directly**
+
+```bash
+just prepare-release minor
+```
+
+Both approaches produce identical results.
+
+### 4. Review and Commit
+
+```bash
+# Review the changes
+git diff
+
+# Edit CHANGELOG.md with actual release notes
+vim CHANGELOG.md
+
+# Commit
+git add -A
+git commit -m "chore: bump version to 0.5.0"
+```
+
+### 5. Continue with Release
+
+After preparing locally, you can either:
+
+1. **Push and let CI handle it**: Push the release branch, merge to `main`, create a tag
+2. **Use the GitHub workflow**: Trigger `prepare-release.yml` which will create a PR
+
+## Pre-release Versions
+
+To create a pre-release (e.g., `0.5.0-beta.1`):
+
+```bash
+# Preview
+just release-preview minor beta.1
+
+# Apply
+just prepare-release minor beta.1
+```
+
+## Debugging
+
+### Test Changelog Logic Only
+
+```bash
+just test-changelog-logic
+```
+
+This adds a test entry to CHANGELOG.md, displays the result, then restores the original file.
+
+### Verbose Diff
+
+The `release-preview` recipe shows diffs for all files. If you need to inspect a specific file more closely:
+
+```bash
+just release-preview patch 2>&1 | grep -A 50 "Cargo.toml"
+```
+
+## Integration with GitHub Actions
+
+The local recipes mirror the logic in `.github/workflows/prepare-release.yml`:
+
+| GitHub Action Step    | Local Equivalent                                      |
+| --------------------- | ----------------------------------------------------- |
+| Calculate new version | Same algorithm in `release-preview`/`prepare-release` |
+| Bump Cargo.toml       | `sed` replacement                                     |
+| Sync pyproject.toml   | `sed` replacement                                     |
+| Sync package.json     | `jq` replacement                                      |
+| Update CHANGELOG.md   | `just changelog-entry`                                |
+
+This ensures you can validate the release logic locally before triggering the workflow.
+
+## See Also
+
+- [Releasing Beta](./releasing-beta.md) - Full release process
+- [Secret Management](./secret-management.md) - Managing release tokens

package/docs/playbooks/migrating-schema-versions.md

@@ -0,0 +1,43 @@
+# Migrating Schema Versions
+
+As DomainForge evolves, the `.sea` file format may change. This guide explains how to handle migrations.
+
+## Detection
+
+The CLI will warn if you are using deprecated syntax.
+
+```text
+Warning: 'interface' keyword is deprecated. Use 'flow' instead.
+```
+
+## Breaking Changes Strategy
+
+### 1. Automated Migration (Future)
+
+We plan to add `sea-cli migrate <file>` to automatically update syntax.
+
+### 2. Manual Migration
+
+For now, manual updates are required.
+
+**Example: Renaming `connection` to `flow`**
+*Old:*
+```sea
+connection "c1" from "A" to "B"
+```
+*New:*
+```sea
+flow "Data" from "A" to "B"
+```
+
+## Backward Compatibility
+
+We strive to support the previous MAJOR version's syntax for one release cycle.
+
+- **v0.4**: Introduces `flow`, deprecates `connection`. Both work.
+- **v0.5**: Removes `connection`.
+
+## Best Practices
+
+- Pin the CLI version in your CI pipeline to avoid unexpected breakages.
+- Read the `CHANGELOG.md` before upgrading.

package/docs/playbooks/onboarding-contributors.md

@@ -0,0 +1,64 @@
+# Onboarding Contributors
+
+Welcome to the DomainForge team! This guide will get you set up and ready to contribute.
+
+## 1. Development Environment
+
+You need:
+
+- **Rust**: Latest stable (`rustup update`)
+- **Python**: 3.8+
+- **Node.js**: 16+
+- **Just**: Command runner (`cargo install just`)
+
+## 2. Initial Setup
+
+Run the setup script to install dependencies for all languages.
+
+```bash
+just setup
+```
+
+## 3. Codebase Tour
+
+- **`domainforge-core/`**: The brain. Start here.
+  - `grammar/`: The syntax definition.
+  - `primitives/`: The data structures.
+- **`tests/`**: Integration tests.
+- **`docs/`**: You are here.
+
+## 4. Running Tests
+
+Before submitting a PR, ensure everything passes.
+
+```bash
+just all-tests
+```
+
+This runs:
+
+1. `cargo test` (Core)
+2. `pytest` (Python bindings)
+3. `vitest` (TypeScript bindings)
+
+## 5. Your First Issue
+
+Look for issues labeled `good first issue`.
+
+- **Documentation**: Fix typos or add examples.
+- **Parser**: Add a small syntax feature.
+- **CLI**: Improve output formatting.
+
+## 6. PR Process
+
+1. Fork the repo.
+2. Create a feature branch.
+3. Make changes.
+4. Add tests! (We love tests).
+5. Run `cargo fmt` and `cargo clippy`.
+6. Submit PR.
+
+## See Also
+
+- [Architecture Overview](../explanations/architecture-overview.md)
+- [Developer Commands](../../../justfile)

package/docs/playbooks/releasing-beta.md

@@ -0,0 +1,86 @@
+# Releasing Beta
+
+This playbook describes the process for releasing a new beta version of DomainForge.
+
+## Prerequisites
+
+- Clean git working directory.
+- Passing CI (`just all-tests`).
+- Write access to crates.io, PyPI, and NPM.
+
+## Steps
+
+1. **Version Bump**:
+
+> **Recommended**: Use `just release-preview minor` to preview changes first, then `just prepare-release minor` to apply them. See [Local Release Preparation](./local-release-preparation.md) for details.
+
+> **Automated (GitHub)**: Use the `prepare-release.yml` workflow (Actions → Prepare Release → Run workflow) to bump all versions and create a PR automatically.
+
+If bumping manually, update version in all files (they must stay in sync):
+
+- `domainforge-core/Cargo.toml` (source of truth)
+- `pyproject.toml`
+- `package.json`
+
+> Note: WASM `pkg/package.json` is auto-generated from `domainforge-core/Cargo.toml` during build.
+
+1. **Changelog**:
+
+Update `CHANGELOG.md` with new features and breaking changes.
+
+1. **Build & Test**:
+
+```bash
+just setup
+just all-tests
+```
+
+1. **Git Tag**:
+
+```bash
+git tag -a v0.x.0 -m "Release v0.x.0"
+git push origin v0.x.0
+```
+
+1. **Publish Rust Core**:
+
+```bash
+cargo publish -p domainforge-core
+```
+
+1. **Publish Python**:
+
+```bash
+maturin publish
+```
+
+1. **Publish TypeScript**:
+
+```bash
+npm publish
+```
+
+1. **GitHub Release**:
+
+Draft a new release on GitHub using the changelog notes.
+
+If a critical bug is found immediately:
+
+- **Rust**: Yank the crate (`cargo yank --vers <version>`).
+
+- **NPM**: Deprecate the package version and suggest replacement:
+
+```bash
+npm deprecate @godspeedai/domainforge@0.x.0 "Critical security bug; use @godspeedai/domainforge@0.x.1"
+```
+
+- **PyPI**: Yank/unyank through standard tooling (e.g., `twine` or `maturin publish --yank`) or publish a hotfix patch:
+
+```bash
+# Publish a hotfix
+git checkout -b hotfix/0.x.1
+bumpversion patch
+git push origin hotfix/0.x.1
+# Create release and upload via twine/maturin
+maturin publish --skip-existing
+```

package/docs/playbooks/secret-management.md

@@ -0,0 +1,64 @@
+# Secret Management with SOPS + age
+
+Scope: securely manage project secrets under `secrets/` using SOPS and age; keep encrypted files in git and avoid plaintext leaks.
+
+## Preconditions
+
+- age key pair available (or generate one).
+- `sops` installed locally/CI.
+- `secrets/secrets.template.yaml` present as a placeholder.
+
+## Steps
+
+1. **Create editable copy**
+
+   ```bash
+   cp secrets/secrets.template.yaml secrets/secrets.yaml
+   # fill in real values
+   ```
+
+2. **Encrypt in place**
+
+   ```bash
+   # replace AGE_PUBLIC_KEY with your age public key
+   sops --encrypt --age "AGE_PUBLIC_KEY" --in-place secrets/secrets.yaml
+   ```
+
+   - Verify encryption (`head -n 5 secrets/secrets.yaml` shows a `sops:` section).
+
+3. **Commit only encrypted files**
+
+   ```bash
+   git add secrets/secrets.yaml .sops.yaml
+   git commit -m "Add encrypted secrets"
+   ```
+
+4. **Decrypt when needed (local only)**
+
+   ```bash
+   sops --decrypt secrets/secrets.yaml > /tmp/decrypted-secrets.yaml
+   # or edit inline
+   sops secrets/secrets.yaml
+   ```
+
+5. **Rotate or add recipients**
+
+   ```bash
+   sops --add-recipients "AGE_PUBLIC_KEY" --in-place secrets/secrets.yaml
+   # remove a key by re-encrypting without it
+   ```
+
+6. **Use in CI**
+
+   - Provide `SOPS_AGE_KEY` (private key) as a CI secret.
+   - Decrypt with `sops --decrypt secrets/secrets.yaml > /tmp/secrets.yaml` before publishing packages.
+
+## Security notes
+
+- Never commit plaintext secrets or private age keys.
+- Keep `.sops.yaml` aligned with repository paths; use per-environment keys when possible.
+
+## See also
+
+- [Configuration](../reference/configuration.md) for registry/environment flags
+- [.sops.yaml](../../.sops.yaml) for creation rules