semantifold 0.7.0 → 0.8.0

package/README.md

@@ -2,7 +2,7 @@
 
 Task037 adds the first concrete standard-library portability slice: exact parser-proved Ruby `require "socket"`, genuine `TCPSocket.new(host, port)`, receiver `gets`/`close`, and one-string `puts`/`Kernel.puts` compile through executable Ruby facades and canonical v1 contracts to protected PHP providers using native `fsockopen`, `fgets`, `feof`, `fclose`, and stdout. Task036's neutral versioned facade foundation was delivered through PR #46 and merge `9aba48f9d48bcef4fd4c9275aec14bfcc4f0904c`; Task035's canonical contract/provider hub was delivered through PR #45 and merge `a2aad9d4932bfc6087adc4ff8bca43678c92413c`. Task034 supplies explicit compiler-authorized host effects, linear owned resources/references, typed host failures, exactly-once lowering, and path-complete explicit close. Task033 adds bounded nominal reference classes, Task032 adds strict-Boolean condition-controlled loops, and Task021 adds a [browser-oriented WebAssembly target](docs/webassembly.md).
 
-Semantifold is an early language-neutral semantic code toolkit. The source tree supports a deliberately small fifteen-language baseline: PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, Kotlin/JVM, strictly annotated Python, canonical C#, bounded Go, C17, C++20, Rust edition 2021, strict Swift, strict Dart VM/native, and strict Zig 0.15.2 source share one typed semantic representation. Browser WebAssembly is an additional target-only ID, not a sixteenth source language. The published baseline is `semantifold@0.6.0` / annotated `v0.6.0` at exact `gitHead` `f76e6d64b388d65959e77f747ae2d60a999d61f5`; Task 028 merged in PR #51 and Task 031 merged in PR #52. The Task 030 source tree adds the bounded Android-qualified Flutter application target while retaining explicit iOS/macOS deferral.
+Semantifold is an early language-neutral semantic code toolkit. The source tree supports a deliberately small fifteen-language baseline: PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, Kotlin/JVM, strictly annotated Python, canonical C#, bounded Go, C17, C++20, Rust edition 2021, strict Swift, strict Dart VM/native, and strict Zig 0.15.2 source share one typed semantic representation. Browser WebAssembly is an additional target-only ID, not a sixteenth source language. The published baseline is `semantifold@0.7.0` / annotated `v0.7.0` at exact `gitHead` `8cd70c5a6c7de98df2d4a183d6555ad4f3d5ba5a`; it includes Task 028 Android, Task 030's bounded Android-qualified Flutter application target, and Task 031 Zig while retaining explicit iOS/macOS deferral. The planned [watch/build/check pipeline](docs/watch-build-pipeline.md) starts from this exact baseline; no product watch CLI exists yet.
 
 ## Current API
 
@@ -41,7 +41,7 @@ const original = originalPositionFor(artifact.mapping, {offset: artifact.code.in
 
 Generated text output always uses LF. Exact original content, including LF, CRLF, lone CR, and astral UTF-16 coordinates, remains in the source registry and `sourcesContent`. Optional inline/external `sourceMappingURL` directives apply only to JavaScript and TypeScript; PHP, Ruby, Java, Kotlin, Python, C#, Go, C, CPP, Rust, Swift, Dart, and Zig use the same rich/v3 model without foreign comments. Wasm instead uses rich byte ranges, a line-zero/byte-column Source Map v3 sidecar, and a relative `sourceMappingURL` custom section. See [source provenance and mappings](docs/source-maps.md) for schemas, lookups, composition, diagnostics, directives, and compatibility.
 
-`supportedLanguages` is the immutable registry-derived list `php`, `ruby`, `javascript`, `typescript`, `java`, `kotlin`, `python`, `csharp`, `go`, `c`, `cpp`, `rust`, `swift`, `dart`, and `zig`. `languageCapabilities` additionally includes target-only `wasm`, `ios`, and `android` records and exposes all eighteen IDs' independent frontend, text-backend, binary-backend, application-backend, and interoperability roles together with `generalFunctionsAndCalls`, `immutableCollections`, `optionalValues`, `orderedListIteration`, `orderedMapIteration`, `conditionControlledLoops`, `referenceClasses`, `effectfulCapabilitiesAndResources`, `closedRecords`, `typedErrors`, and `typeParametersAndGenerics` feature flags, artifact multiplicity, round-trip, mapping, acceptance-stage, and toolchain declarations. Unknown IDs use `UNSUPPORTED_LANGUAGE`; a registered ID missing a requested role uses `UNSUPPORTED_ROLE`; an existing backend that cannot represent a semantic module uses `UNSUPPORTED_CAPABILITY`.
+`supportedLanguages` is the immutable registry-derived list `php`, `ruby`, `javascript`, `typescript`, `java`, `kotlin`, `python`, `csharp`, `go`, `c`, `cpp`, `rust`, `swift`, `dart`, and `zig`. `languageCapabilities` additionally includes target-only `wasm`, `ios`, `android`, and `flutter` records and exposes all nineteen IDs' independent frontend, text-backend, binary-backend, application-backend, and interoperability roles together with `generalFunctionsAndCalls`, `immutableCollections`, `optionalValues`, `orderedListIteration`, `orderedMapIteration`, `conditionControlledLoops`, `referenceClasses`, `effectfulCapabilitiesAndResources`, `closedRecords`, `typedErrors`, and `typeParametersAndGenerics` feature flags, artifact multiplicity, round-trip, mapping, acceptance-stage, and toolchain declarations. Unknown IDs use `UNSUPPORTED_LANGUAGE`; a registered ID missing a requested role uses `UNSUPPORTED_ROLE`; an existing backend that cannot represent a semantic module uses `UNSUPPORTED_CAPABILITY`.
 
 `generateArtifactSet` is the additive single-module artifact API. Original text targets return an entry artifact with the exact legacy bytes and rich/v3 mappings; an external JavaScript-family map directive also returns the referenced serialized mapping sidecar. C# returns mapped `Program.cs` then synthetic-provenance `Semantifold.csproj`. Go returns exactly two ordered artifacts: fixed synthetic-provenance `go.mod` first, then mapped `main.go`; the fixed module is `example.com/semantifold/generated` with `go 1.26.0` and creates no `go.sum`. C returns mapped `program.c` followed by synthetic `semantifold_runtime.h`. Rust returns synthetic `Cargo.toml`, synthetic Cargo-generated `Cargo.lock`, then mapped `src/main.rs`, with fixed edition 2021 and Rust/Cargo 1.98.1. Dart returns synthetic `pubspec.yaml`, synthetic deterministic `pubspec.lock`, then mapped sole-entry `bin/program.dart`. Zig returns synthetic `build.zig` then mapped sole-entry `src/main.zig`; Zig 0.15.2 needs no `build.zig.zon` for this dependency-free native-host project. Wasm binary/application generation returns `program.wasm`, `program.wasm.map`, `semantifold-loader.mjs`, then sole entry `index.html`, plus frozen `semantifold.browser.v1` ABI metadata. Because C#, Go, C, Rust, Dart, Zig, and Wasm are deliberately multi-artifact, `generate()` and `generateArtifact()` reject them with `UNSUPPORTED_ROLE`. Go accepts only an omitted filename or exact `main.go`, C accepts only omitted or exact `program.c`, Rust and Zig only omitted or exact `src/main.rs` and `src/main.zig` respectively, Dart only omitted or exact `bin/program.dart`, and Wasm only omitted or exact `program.wasm`; these multi-artifact backends reject unsupported mapping options rather than ignoring them. `createGeneratedArtifactSet` validates multi-file text, binary, mixed loader/binary, and application shapes transactionally: paths are ordered, unique, safe relative POSIX paths with no file/directory prefix collisions; content is explicitly text or bytes; media type, role, generated ownership, and provenance are required; exactly one artifact has role `entry`; and optional metadata is detached deeply frozen JSON. Binary content is held privately and returned as a fresh detached byte view on every read, so caller mutation cannot change validated bytes or their provenance. Generated helper files do not create semantic modules and are not Task 010 projects.
 
@@ -132,6 +132,7 @@ The execution specs require PHP CLI, Ruby, Python 3, TypeScript, a Java JDK, Kot
 
 - [Goals](docs/goals.md)
 - [Architecture](docs/architecture.md)
+- [Watch, build, and target-check pipeline](docs/watch-build-pipeline.md)
 - [Coding standards](docs/coding-standards.md)
 - [Language support](docs/language-support.md)
 - [Rust source and Cargo target](docs/rust.md)

package/docs/watch-build-pipeline.md

@@ -0,0 +1,154 @@
+# Watch, build, and target-check pipeline
+
+## Status and verified baseline
+
+This document is a roadmap contract, not an implemented CLI contract.
+
+The inspected baseline is `semantifold@0.7.0`, annotated tag `v0.7.0`, commit `8cd70c5a6c7de98df2d4a183d6555ad4f3d5ba5a`. At that revision:
+
+- JavaScript/JSDoc is a registered source frontend and Java is a registered text backend.
+- `parse`, `parseProgram`, `generateArtifactSet`, and `generateProgramArtifactSet` are public, deterministic, in-memory APIs.
+- `discoverCanonicalToolchain` finds the configured `javac` and `java` executables.
+- `runAcceptanceStages` materializes an artifact set in a temporary directory and runs caller-supplied exact argument arrays with bounded process execution.
+- A bounded direct proof parsed the 2,648-byte JSDoc fixture `spec/fixtures/compatibility/program.js`, generated `Main.java`, compiled it with `javac 25.0.4`, executed it with Java 25.0.4, and observed the canonical output `compat\nbranch-ok!\npresent\nabsent\n7\n2\n3\n5\n4\n`.
+- `package.json` has no `bin` entry. The public API does not persist generated artifacts, load a project manifest, derive compiler arguments from a target, or watch source files.
+- The only repository file-change watcher is a test helper; there is no product watch coordinator, incremental dependency graph, event coalescing policy, or long-lived compiler owner.
+
+The current system therefore already proves **JavaScript/JSDoc → semantic IR → Java → `javac`** as a one-shot composition. The roadmap below turns that composition into a safe project workflow without moving filesystem or process lifecycle concerns into parsers or backends.
+
+## Product outcome
+
+A project can declare explicit source modules, one or more generated targets, one publisher-owned project publication root, target-relative generated/build projections, and whether each target is checked with its real toolchain. It can then run:
+
+- a planned one-shot `semantifold build`; or
+- a planned long-lived `semantifold watch`.
+
+For the first end-to-end slice, changing a JSDoc-typed JavaScript source regenerates Java and compiles the staged candidate with `javac`. A successful cycle publishes one coherent generated-source/compiler-output generation through one active-generation pointer switch. A parse, semantic, generation, tool-discovery, compiler, or publication failure reports a located diagnostic, keeps the process alive in watch mode, and leaves the previous successful generation usable. A later valid edit recovers without restarting the watcher.
+
+The architecture is target-neutral. Every current text backend eventually supplies one target-owned check plan. Every current source frontend participates through the existing parser/project APIs; the watcher never contains language syntax logic.
+
+## Architecture boundaries
+
+### Project manifest and loader
+
+A versioned, strict manifest declares ordered source modules and target entries. Source language, module identity, entry identity, target language/role, one project publication root, and each target's generated/check subpaths inside an immutable project generation are explicit. These target paths are logical projections, not independently promoted filesystem roots. Version 1 uses explicit file paths rather than introducing a glob language or package-manager resolution.
+
+The loader owns path normalization and rejects unknown fields, duplicate module/target identities, ambiguous target roles, source/publication overlap, publication roots outside the project root, colliding or nested target projections, and symlink/path traversal across the owned publication root. It returns an immutable project request and never parses language syntax.
+
+### Snapshot builder
+
+Each cycle reads a complete content snapshot of every declared source and the project manifest. File events are hints only. Before parsing, the coordinator re-reads and hashes the declared graph in stable order; it never assumes an event filename is complete or authoritative.
+
+A snapshot is immutable for the cycle. Edits arriving during parse/generation/check mark the watcher dirty and cause one immediate follow-up cycle after the current owned work closes. Cycles never overlap.
+
+### Existing semantic compiler core
+
+Frontends, semantic validation, linking, and backends remain pure with respect to the project filesystem. They consume the immutable snapshot and return complete `GeneratedArtifactSet` values. No frontend or backend starts watchers, writes files, spawns a compiler, or reads undeclared project files.
+
+### Transactional artifact publication
+
+A publisher stages one complete project candidate as a new immutable generation beneath the publisher-owned publication root. Every target's generated artifacts, mappings, and optional compiler outputs live in generation-relative subtrees on the same filesystem. The publisher validates paths, mappings, hashes, collisions, ownership, and manifests before touching committed state.
+
+When checking is enabled, target checks run against those staged generation subtrees. Only a fully generated and successfully checked project candidate may be published. The generation manifest is the explicit versioned authority for files the publisher created; it never authorizes mutation outside the publication root.
+
+Publication commits exactly one small active-generation pointer by writing and syncing a sibling temporary pointer, atomically replacing the prior pointer on the same filesystem, and syncing the parent directory before reporting success. A host filesystem that cannot provide atomic same-directory regular-file replacement is rejected before publication; copying, truncating, or rewriting the live pointer in place is never a fallback. Readers resolve that pointer once and consume all generated/build paths from the referenced immutable generation. Configured generated-source and compiler-output names are logical projections returned from that resolved generation; they are never independent directories, symlinks, or manifests promoted one after another. The same rule covers every target in a multi-target cycle.
+
+An interruption before the pointer replacement leaves the previous generation active; an interruption after it leaves either the old or new complete pointer durably recoverable, never a composite state. A recovery journal may finish cleanup of unpublished candidates and temporary pointer files, but it is not used to claim atomicity across multiple filesystem operations. Publication never removes a committed generation, because a reader may already hold its resolved snapshot; committed-generation retention/garbage collection is a separate explicit policy. A failed candidate is removed without changing the active pointer.
+
+### Target check-plan registry
+
+The existing language registry remains authoritative for target identity, roles, declared acceptance stages, and required toolchain IDs. A planned target-check capability derives exact stage requests from a validated staged artifact set and its generation-scoped target build subtree.
+
+The generic runner owns process lifecycle, locale/timezone normalization, output capture, timeout/cancellation, and close observation. A target check plan owns filenames, exact argv, toolchain IDs, offline/cache policy, and which stages constitute a non-executing developer check. The watcher does not switch on language IDs.
+
+The initial check plan is Java/`javac`. Later tasks add current text targets in two cohorts:
+
+- interpreted/managed: PHP, Ruby, JavaScript, TypeScript, Kotlin/JVM, Python, and C# (Java is supplied by the initial slice);
+- native/project: Go, C, C++, Rust, Swift, Dart, and Zig.
+
+A watch check validates or compiles but does not execute generated application code by default. Restore/download behavior is never an implicit per-edit action; targets with project metadata use pinned, offline, isolated state and rerun prerequisite stages only when their declared inputs change.
+
+### Watch coordinator
+
+One lifecycle-aware class owns filesystem subscriptions, content snapshots, cycle state, the active checker child, reporting, and shutdown. Its state machine is:
+
+```text
+starting -> building -> ready
+                 |       |
+                 v       v
+               failed <- rebuilding
+                 |       |
+                 +-- valid edit --+
+
+any live state -- SIGINT/SIGTERM --> stopping --> stopped
+```
+
+`failed` means the last candidate failed; it does not discard the last successful output or terminate watch mode. A watcher event during `building`/`rebuilding` sets one dirty generation marker. The coordinator starts exactly one follow-up after the owned child closes. It never starts a retry while prior compiler work is still live.
+
+Native filesystem events require a bounded reconciliation strategy because Node and TypeScript both document platform-dependent watcher behavior. Version 1 watches the explicit manifest/source files and relevant parent directories, re-scans the declared graph after each hint, supports a documented polling fallback, and suppresses no-op cycles by content hash. The publication root and every generated/build projection are rejected as source roots and are never watched.
+
+### Reporting contract
+
+Human output is concise and stable. A machine-readable mode emits one JSON record per state transition with at least project identity, monotonically increasing cycle number, source snapshot hash, target ID/role, state, timing, changed paths, tool identity, exit status, and a structured diagnostic when present.
+
+Exactly one terminal cycle record is emitted per cycle. Process exit is non-zero for failed one-shot builds, but ordinary watch-cycle failures keep the watcher process alive. Startup/configuration failure exits. Signal-driven shutdown forwards cancellation to an active owned child, waits for `close`, removes subscriptions and staging state, and exits once.
+
+## Generation and failure semantics
+
+1. Initial watch startup performs a real build; it does not wait for the first edit.
+2. Event bursts coalesce into one snapshot rebuild after a bounded quiet period.
+3. Content hashes, not timestamps alone, decide whether a cycle is a no-op.
+4. Parse/generation failure never starts a target checker and never publishes.
+5. Tool discovery/check failure never publishes the staged generated source or build products.
+6. Publication failure before the active-pointer replacement leaves the previous generation active; failure after that replacement reports the new generation as committed and limits recovery to cleanup.
+7. A valid edit after failure performs a fresh complete cycle and may publish normally.
+8. Deleted or renamed declared sources are ordinary invalid snapshots until the manifest is updated; stale owned outputs are not silently treated as current.
+9. Multiple target entries are staged from one source snapshot beneath one immutable project generation. The manifest defines whether all targets must pass before the single active pointer is switched; version 1 defaults to one all-or-nothing project generation.
+10. No shell command strings, compiler-specific branches in the watcher, implicit network access, or arbitrary user hooks are introduced.
+
+## Cross-language scope
+
+“Works for all languages” means:
+
+- every registered frontend can be an explicitly declared watched source when the selected semantic features are supported;
+- every current text backend has an explicit real-toolchain check plan or a fail-loud reason it cannot be checked;
+- the same watcher/materializer/runner code handles every source/target pairing;
+- acceptance uses a bounded matrix: all frontends and all text targets are covered, with the JavaScript/JSDoc-to-Java path plus representative cross-family paths, not a quadratic promise that every semantic feature works for every pair.
+
+Binary and application targets use the same generation watcher only after explicit target policy is defined. Browser Wasm can validate/instantiate in its own lane. Android and Flutter checks are heavier platform builds and must be opt-in rather than an automatic keystroke default. iOS remains generation-only until the separately deferred Apple/Xcode acceptance work is authorized; this roadmap does not reopen Tasks 026 or 027.
+
+## Test strategy
+
+- Unit contracts cover strict manifest validation, snapshot hashing, event coalescing, dirty-during-build behavior, no-op suppression, generation manifests, single-pointer publication, recovery journals, reader snapshot resolution, and exact check-plan construction.
+- Real temporary-directory tests cover create/change/delete/rename behavior and ensure generated roots do not self-trigger.
+- Real-child lifecycle tests cover spawn failure, non-zero compiler close, late output, signal forwarding, shutdown during an active check, and no leaked child/staging directory.
+- Real `javac` acceptance proves the initial JavaScript/JSDoc-to-Java cycle, compiler failure reporting at the check boundary, an unchanged last-good active pointer, recovery after a valid edit, and source/class execution from one resolved committed generation.
+- The terminal text-target matrix invokes every declared real checker/compiler in CI and fails rather than skips when a configured tool is unavailable.
+- Focused local tests remain small; full language/toolchain matrices run only in the existing CI lanes.
+
+## Dependency order
+
+```text
+015 artifact/toolchain foundation
+        |
+        038 transactional publication
+        |
+010 ----+---- 039 project manifest + one-shot build CLI
+                   |
+                   040 generic check plans + Java/javac
+                   |                    |
+             041 managed cohort    043 watcher coordinator
+             042 native cohort          |
+                   |                    044 JS/JSDoc -> Java watch slice
+                   +--------------------+
+                              |
+                  045 terminal all-text-language matrix
+                              |
+                  046 later binary/application policies
+```
+
+Tasks 038–045 are the selected text-language delivery chain. Task 045 is the single terminal acceptance task for that chain. Task 046 is a non-blocking later extension.
+
+## Non-goals
+
+This roadmap does not implement language-server/editor protocols, incremental semantic-IR invalidation, daemon IPC, remote/distributed builds, arbitrary compiler flags, shell hooks, package installation, source discovery by glob, runtime execution on every edit, hot reload into a running application, production deployment, signing, store submission, or Apple/Xcode acceptance. It does not change semantic language support or claim all-pairs feature compatibility.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "semantifold",
-  "version": "0.7.0",
+  "version": "0.8.0",
   "description": "Language-neutral semantic code analysis, transformation, generation, and execution toolkit",
   "type": "module",
   "main": "build/index.js",