@calcit/procs 0.13.51 → 0.13.53

package/editing-history/20260827-1739-remove-legacy-macro-schemas.md

@@ -0,0 +1,38 @@
+# Remove legacy macro schemas
+
+## Context
+
+Issue #480 ends the migration window for runtime-shaped macro schemas. A
+`defmacro` now has one persisted and executable schema representation: the
+phase-aware `Macro` contract.
+
+## Changes
+
+- Removed the legacy macro compatibility state, provenance metadata, staged
+  warning environment switch, metrics/cache exceptions, and coverage-report
+  fields.
+- Made Snapshot loading reject a `defmacro` with a `Fn`, `Dynamic`, or other
+  non-`Macro` schema before compilation. Errors include the Snapshot
+  definition path and the final compatible Calcit version, 0.13.51.
+- Made schema editing reject legacy `:kind :macro` function maps, preventing
+  current tools from emitting a schema the loader will reject.
+- Made preprocessing and runtime macro evaluation use only strict parameter,
+  expansion, and capability checks.
+- Migrated the JS integration fixture and direct temporary-Snapshot test helper
+  to explicit strict macro contracts.
+- Updated migration, static-analysis, macro-contract, and agent documentation.
+
+## Validation
+
+- `cargo fmt --check`
+- `cargo clippy -- -D warnings`
+- `cargo test -- --test-threads=1`
+- `yarn compile`
+- `yarn check-agent-interface`
+- latest Respo and Recollect `--check-only` plus
+  `analyze check-types --deps --summary-only --format json` with the local
+  release binary
+
+`yarn check-all` was also run after the agent-interface lock cleared; its
+component gates are covered by the full Rust, integration-fixture, and release
+checks above.

package/editing-history/20260827-2250-ffi-blocking-protocol-v1.md

@@ -0,0 +1,52 @@
+# C-safe blocking FFI protocol v1
+
+## Goal
+
+Remove Rust closures, trait objects, `Vec<Edn>`, `Result`, and `FnOnce` from
+the supported `&blocking-dylib-edn-fn` boundary while preserving methods that
+must own and call back on the Calcit host thread.
+
+## Protocol decisions
+
+- use a distinct `<method>_calcit_ffi_blocking_v1` symbol so blocking and
+  asynchronous starts cannot be confused;
+- reuse `calcit_ffi_async_version() -> 1`, the generation task registry,
+  OneShot lifecycle, serial event sequence, task descriptor, status codes, and
+  Cirru EDN request encoding;
+- invoke the Calcit callback directly only on the task's owner thread instead
+  of enqueueing work that cannot drain while the dylib owns that thread;
+- let modules finish explicitly exactly once, with method return providing the
+  implicit finish when no explicit finish occurred;
+- keep callback result bytes host-owned and indexed by exact pointer/length;
+  modules release them through the task-bound host `free_buffer` function;
+- report foreign-thread access, callback failures, forged/duplicate frees, and
+  leaked host buffers even if the module ignores the returned status;
+- probe the migrated method before allocating runtime task state, preserving
+  the build-ID-guarded per-method Rust ABI fallback for unmigrated modules;
+- make legacy blocking task release exactly once on explicit finish or method
+  return, fixing the previous pending-task leak on error paths.
+
+## Tests
+
+- C-layout host table and versioned method name;
+- inline callback execution and host-owned EDN result;
+- exact buffer metadata and duplicate-free rejection;
+- owner-thread enforcement;
+- explicit finish exactly once and callback rejection after finish.
+
+## Native verification
+
+An optimized standalone C dylib was loaded by the debug Calcit host and
+verified both explicit and implicit finish. Separate methods confirmed that a
+foreign pthread callback, an unfreed callback buffer, and a Calcit callback
+error all fail deterministically and release the pending task.
+
+## Full verification
+
+- `cargo fmt -- --check`
+- `cargo clippy --all-targets -- -D warnings`
+- `cargo test` (864 Rust tests)
+- `yarn compile`
+- `yarn check-agent-interface` (17/17)
+- `yarn check-all` (Calcit core 221/221, JS, IR, WASM and runtime benchmarks)
+- `bash scripts/check-docs-md.sh` (57 files, 325 blocks)

package/editing-history/20260827-2257-release-0.13.52.md

@@ -0,0 +1,17 @@
+# Release 0.13.52
+
+Release the C-safe asynchronous and blocking FFI runtime completed after
+0.13.51.
+
+Highlights:
+
+- bounded C-safe async task/event queue and host-thread callback drain;
+- native callback-v1 with per-method guarded fallback;
+- cancellable Server tasks and exactly-once response capabilities;
+- C-safe blocking callback-v1 with owner-thread dispatch, explicit/implicit
+  finish, and host-owned callback buffer tracking;
+- removal of the remaining supported Rust closure/container boundary for
+  migrated blocking methods.
+
+The release provides the stable host version required for the Phase 6 module
+migration under issue #482.

package/editing-history/202608271831-ffi-async-protocol-foundation.md

@@ -0,0 +1,28 @@
+# Async FFI protocol foundation
+
+## Summary
+
+- Defined a versioned, C-safe async task descriptor for one-shot, stream,
+  server, and response handles without changing the existing synchronous FFI.
+- Added a thread-safe generational handle registry with ordered event
+  sequences, explicit close/finish/release transitions, and host shutdown.
+- Retired exhausted generation slots so stale handles cannot become valid
+  again after counter wraparound.
+- Documented the transport-neutral lifecycle intended for timers, watchers,
+  HTTP/WebSocket servers, and a future WASM adapter.
+
+## Compatibility constraint
+
+Native libraries must enqueue events for the Calcit host instead of invoking
+Calcit callbacks from foreign threads. The protocol models stable integer
+handles and byte payloads; a WASM adapter may map them to imports/exports
+without reproducing native pointers or function pointers.
+
+## Validation
+
+- `cargo fmt --check`
+- `cargo clippy -- -D warnings`
+- `cargo test -- --test-threads=1`
+- `yarn compile`
+- `yarn check-agent-interface`
+- `yarn check-all`

package/editing-history/202608271928-ffi-host-event-queue.md

@@ -0,0 +1,29 @@
+# Async FFI host event queue
+
+## Summary
+
+- Added a C-layout event descriptor for repeating emit and terminal
+  complete/fail events.
+- Added a bounded multi-producer queue that only its creating host thread may
+  wait on or drain into the Calcit runtime.
+- Kept sequence reservation atomic with queue admission, made terminal events
+  exactly once, and limited coalescing to explicitly opted-in stream events.
+- Returned callback and lifecycle failures as structured drain diagnostics;
+  callback failure finishes the task and purges its remaining queued events.
+- Retained producer-thread and queue-delay metadata for low-payload FFI traces.
+
+## Compatibility constraint
+
+Existing Rust callback and blocking dylib calls remain unchanged. This queue
+is the scheduling foundation for the next C host-function-table PR; native
+threads and a future WASM polling adapter share descriptors and lifecycle
+semantics without sharing pointers or callback layouts.
+
+## Validation
+
+- `cargo fmt --all`
+- `cargo clippy --all-targets -- -D warnings`
+- `cargo test -- --test-threads=1`
+- `yarn compile`
+- `yarn check-agent-interface`
+- `yarn check-all`

package/editing-history/202608271947-preserve-ffi-drain-report.md

@@ -0,0 +1,15 @@
+# Preserve async FFI drain accounting
+
+## Summary
+
+- Kept malformed event-kind failures inside the structured drain report rather
+  than returning after a batch had already been removed from the queue.
+- Recorded queue-purge failures alongside callback and lifecycle failures.
+- Separated discarded batch events from events purged before drain, preserving
+  the invariant that every dequeued event is delivered or discarded.
+
+## Validation
+
+- `cargo fmt --all`
+- `cargo clippy --all-targets -- -D warnings`
+- `cargo test ffi_async::tests -- --nocapture`

package/editing-history/202608272013-native-ffi-callback-v1.md

@@ -0,0 +1,31 @@
+# Native C-safe callback FFI v1
+
+## Goal
+
+Move `&call-dylib-edn-fn` onto the versioned asynchronous C ABI without
+breaking modules that still expose the guarded Rust callback ABI.
+
+## Changes
+
+- added the versioned native async start symbol and process-lifetime C host
+  function table;
+- copied and validated foreign byte payloads before placing them in the
+  bounded host queue;
+- decoded `Emit` arguments on the CLI host thread, required explicit `&unit`
+  completion, and surfaced structured `Fail` diagnostics;
+- reclaimed handles and tracking state on terminal delivery, callback failure,
+  and start failure;
+- kept per-method fallback to the build-identity-guarded Rust ABI while making
+  an advertised version mismatch a hard error;
+- added ABI, payload, foreign-producer, host-drain, lifecycle, and failure
+  regression coverage;
+- documented symbols, ownership, payload rules, statuses, and rollout.
+
+## Verification
+
+- `cargo fmt --all`
+- `cargo clippy --all-targets -- -D warnings`
+- `cargo test -- --test-threads=1`
+- `yarn compile`
+- `yarn check-agent-interface`
+- `yarn check-all`

package/editing-history/202608272133-ffi-response-server-v1.md

@@ -0,0 +1,45 @@
+# Async FFI response and server protocol v1
+
+## Goal
+
+Extend native callback-v1 with cancellable Server tasks and exactly-once
+response capabilities without exposing generation handles as lossy Calcit
+numbers or allowing foreign threads to enter the runtime.
+
+## Changes
+
+- appended C-safe task configuration and response registration functions to
+  the versioned host table;
+- allowed modules to declare OneShot, Stream, or Server semantics before the
+  first event and install a Server cancellation hook;
+- returned opaque task capabilities from callback-v1 methods that install a
+  cancel hook, preserved `&unit` for non-cancellable methods, and added
+  `&ffi-task-cancel`;
+- issued opaque AnyRef response capabilities, appended them to request callback
+  arguments, and added `&ffi-response-resolve` / `&ffi-response-reject`;
+- enforced response owner, kind, active lifecycle, `REQUIRES_RESPONSE`, timeout,
+  and non-coalescing queue rules;
+- rejected and released unresolved responses on timeout or owner completion,
+  while startup rollback discards handles without entering untransferred module
+  context;
+- added registry configuration/snapshot, response queue invariants,
+  exactly-once/stale reuse, timeout, cross-server, and foreign producer tests;
+- documented the expanded C ABI, Calcit capability API, ownership, timeout,
+  cancellation, backpressure, and WASM representation boundary.
+
+## Runtime verification
+
+A standalone C dylib configured a cancellable Server, opened a response,
+published a request from a pthread, received the Calcit response on the host
+thread, enqueued explicit `&unit` completion, and released with pending task
+count zero. A second idle Server fixture verified `&ffi-task-cancel` and its
+terminal acknowledgement path.
+
+## Verification
+
+- `cargo fmt --all`
+- `cargo clippy --all-targets -- -D warnings`
+- `cargo test -- --test-threads=1`
+- `yarn compile`
+- `yarn check-agent-interface`
+- `yarn check-all`

package/editing-history/202608272158-ffi-response-review-fixes.md

@@ -0,0 +1,33 @@
+# Async FFI response review fixes
+
+## 中文
+
+- 将每个 host table 的 context 绑定到对应 task handle，阻止不同任务或模块意外复用 host context；
+- 在调用 dylib resolver 前原子 claim response，保证并发 resolve/reject 只会进入模块一次，模块返回错误时也会释放 capability；
+- 使用 ordered deadline index、response lookup 与 owner index 代替每轮 drain 和 task 结束时的全 registry clone/scan；
+- 每个任务最多允许 1024 个并发未完成 response，避免只 open 不 enqueue 导致无界增长；
+- response 在队列等待期间过期时只 reject/skip 当前 request，不再把整个长期 Server 标记失败并清空后续事件；
+- 修正文档中的 `REQUIRES_RESPONSE` 术语，并补充 host table 生命周期、并发与超时语义。
+
+## English
+
+- bound each host-table context to its task handle so another task or module cannot accidentally reuse it;
+- atomically claim a response before entering the dylib resolver, ensuring concurrent resolve/reject attempts invoke the module at most once and release the capability even when the module reports an error;
+- replaced full-registry clones/scans during drains and task completion with an ordered deadline index, response lookup, and owner index;
+- capped concurrent outstanding responses at 1024 per task to prevent unbounded open-without-enqueue growth;
+- reject and skip a request that expires in the queue without failing the long-lived Server or purging later events;
+- aligned documentation with `REQUIRES_RESPONSE` terminology and clarified host-table lifetime, concurrency, and timeout semantics.
+
+## Validation
+
+- `cargo fmt --all`;
+- `cargo clippy --all-targets -- -D warnings`;
+- `cargo test -- --test-threads=1` (855 Rust tests passed);
+- `yarn compile`;
+- `yarn check-agent-interface` (17/17);
+- `yarn check-all` (Calcit core 221/221 plus JS, IR, WASM, and benchmark checks);
+- rebuilt the standalone C dylib and verified pthread Server request → Calcit
+  callback → response resolver → explicit `&unit` completion with no runtime
+  errors;
+- verified an idle Server task capability can still be cancelled and reaches
+  explicit `&unit` completion with no runtime errors.

package/editing-history/20260828-0312-c-safe-opaque-resource-v1.md

@@ -0,0 +1,8 @@
+# C-safe opaque resource v1
+
+- 在同步 buffer v1 上增加保留的 `CalcitFfiResourceV1` token，以 little-endian `u64` handle 和 generation 表示模块资源。
+- 宿主将 token 水合为自动生命周期的 `AnyRef`，固定创建资源的 dylib，并在最后一个引用释放时 exactly-once 调用模块 C ABI release。
+- 调用前递归校验资源归属；拒绝 wrong-module、普通 AnyRef、伪造 token 和畸形 token。
+- 覆盖嵌套资源、并发 clone/drop、错误边界和协议格式测试，并为 `--trace-ffi` 增加资源创建与释放事件。
+- 新增中英双语协议文档，并从 FFI bindings 与升级手册链接。
+

package/editing-history/20260828-0327-intern-ffi-resource-leases.md

@@ -0,0 +1,7 @@
+# Intern FFI resource leases
+
+- 根据 review 修复重复 resource token 产生独立 lease、提前释放和重复释放的问题。
+- 宿主按 `(module, handle, generation)` 保存 weak intern entry；同一响应及跨响应 alias 复用同一 lease。
+- 最后一个强引用析构后 exactly-once release，并清理失效的 weak entry。
+- 增加同响应重复 token 与跨响应重复 token 的回归测试，并同步协议文档。
+

package/editing-history/20260828-0331-release-0.13.53.md

@@ -0,0 +1,5 @@
+# Release 0.13.53
+
+- 发布 C-safe opaque resource protocol v1。
+- 包含宿主资源归属校验、dylib pin、weak lease interning、自动 exactly-once release 与 trace 事件。
+- 为 `calcit-regex` 的 Rust ABI/AnyRef 边界迁移提供正式版本基线。

package/lib/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@calcit/procs",
-    "version": "0.13.51",
+    "version": "0.13.53",
     "main": "./lib/calcit.procs.mjs",
     "devDependencies": {
         "@types/node": "^25.7.0",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@calcit/procs",
-  "version": "0.13.51",
+  "version": "0.13.53",
   "main": "./lib/calcit.procs.mjs",
   "devDependencies": {
     "@types/node": "^25.7.0",