@cratis/pi 7.113.0 → 7.114.0

package/package/corpus/harnesses/copilot/instructions/orleans.instructions.md

@@ -11,7 +11,7 @@ paths:
 > **⚠️ APPLIES ONLY TO PROJECTS USING MICROSOFT ORLEANS**
 > If your project does not reference `Microsoft.Orleans` or any Orleans packages, **ignore this entire file**. These rules are irrelevant outside of Orleans contexts.
 
-In the Cratis organization, Orleans is the Chronicle kernel's actor runtime (`Source/Kernel/`). The kernel conventions below are verified against `Cratis/Chronicle` **v19.50.0**. Shared hosting and jobs come from `Cratis/Orleans` **v1.16.3**; use its setup for other services that co-host a silo.
+In the Cratis organization, Orleans is the Chronicle kernel's actor runtime (`Source/Kernel/`). The kernel conventions below are verified against `Cratis/Chronicle` **v19.50.0**. Shared hosting and jobs come from `Cratis/Orleans` **v1.30.0**; use its setup for other services that co-host a silo.
 
 ## General
 
@@ -50,7 +50,7 @@ framework/service guidance; they do not impose application slice architecture.
 
 For service authors, `Cratis.Orleans` supplies hosting, grain serializers, and the
 shared job system without requiring Chronicle. With `Cratis.Orleans` and
-`Cratis.Orleans.Storage.MongoDB` 1.16.3, this host uses durable MongoDB membership
+`Cratis.Orleans.Storage.MongoDB` 1.30.0, this host uses durable MongoDB membership
 and jobs storage (a local MongoDB server is required):
 
 ```csharp
@@ -81,6 +81,14 @@ app.Run();
 - Register jobs storage separately: `AddCratisOrleansMongoDBJobsStorage`,
   `AddCratisOrleansSqlJobsStorage` (`Cratis.Orleans.Setup`), or
   `AddCratisOrleansInMemoryJobsStorage` (`Cratis.Orleans.Storage`).
+- Use the local `IJobs` service to start/stop/delete typed jobs during rolling
+  upgrades; generic manager calls can land on a release that does not know the
+  type. Provide singleton `IJobsManagerKeys` on every silo for all host-owned
+  scope/namespace pairs; process-local discovery is not a recovery catalog.
+- Controller and worker placement are separate opt-ins through `AddJobPlacement`
+  and `AddJobStepPlacement`. Enable them only after every silo supports the
+  strategies and register them identically. Worker throttling is not admission;
+  see **cratis-orleans-jobs** for drain-before-replacement and metadata limits.
 - Give grain implementations a unique `[GrainType("unique-name")]`. Orleans'
   default type name derives from the class name alone; a collision across
   assemblies prevents silo startup.

package/package/corpus/harnesses/copilot/instructions/vertical-slices.instructions.md

@@ -258,7 +258,7 @@ public record AuthorRegistered(AuthorName Name);
 
 ## Read Models and Queries
 
-**[contract]** `[ReadModel]` records expose eligible public and internal static methods as queries; internal visibility does not hide an endpoint. Dependencies are plain method parameters (no `[FromServices]`). For application code, start with model-bound mappings, use fluent when the definition needs structure, and choose reducer code when transitions read better as code. For step-by-step creation, invoke the **cratis-chronicle-read-model** skill.
+**[contract]** `[ReadModel]` records expose eligible public static methods as queries. Requires Arc 22.121.0 for non-public methods to be excluded even with `[Query]` (ARC0029 warns); older versions also expose eligible internal statics. Keep helpers outside the read model when supporting those versions. Dependencies are plain method parameters (no `[FromServices]`). For application code, start with model-bound mappings, use fluent when the definition needs structure, and choose reducer code when transitions read better as code. For step-by-step creation, invoke the **cratis-chronicle-read-model** skill.
 
 Use `[ReadModel(RequireQueryAttribute = true)]` with `[Query]` on intended
 endpoints when folds exist (ARC0026/ARCCHR0019). `[Query]` alone does not hide

package/package/corpus/hooks/scripts/cratis-store-mutations.json

@@ -61,6 +61,7 @@
         { "command": "read-models occurrences", "reason": "lists a read model's replay history" },
         { "command": "read-models snapshots", "reason": "reads snapshots of one read model instance" },
         { "command": "recommendations list", "reason": "lists active recommendations" },
+        { "command": "streams closed list", "reason": "lists closed stream scopes and their closure metadata without changing them" },
         { "command": "subscriptions list", "reason": "lists event store subscriptions" },
         { "command": "users list", "reason": "lists registered users" }
     ],
@@ -68,13 +69,16 @@
         { "command": "observers replay", "effect": "replays an observer from the beginning of the event log, resetting all state it has built" },
         { "command": "observers replay-partition", "effect": "replays one partition of an observer from the beginning, resetting that partition's state" },
         { "command": "observers retry-partition", "effect": "retries a failed partition from the sequence number where it failed" },
+        { "command": "observers retry-failed-partitions", "effect": "starts bounded-batch recovery of every failed partition of an observer; acceptance does not mean recovery finished" },
         { "command": "observers clear-quarantine", "effect": "clears observer-level quarantine only, not a partition's quarantine" },
         { "command": "observers clear-partition-quarantine", "effect": "clears a failed partition's quarantine and resets its retry budget; retries the handler unless --no-retry is given" },
         { "command": "observers remove", "effect": "deletes an observer's definition, state, handled counts, failed partitions and projection definition across every namespace of the event store; leaves read model data" },
         { "command": "jobs stop", "effect": "stops a running background job" },
         { "command": "jobs resume", "effect": "resumes a stopped background job" },
+        { "command": "jobs delete", "effect": "requests irreversible deletion of a job and its retained state; acceptance does not confirm completed removal" },
         { "command": "recommendations perform", "effect": "executes a server recommendation, such as a schema or replay maintenance action" },
         { "command": "recommendations ignore", "effect": "marks a recommendation as ignored so it no longer appears" },
+        { "command": "streams closed reopen", "effect": "reopens one exact manual stream closure and records the authenticated actor and repair reason in a StreamScopeReopened system event; event-driven closures are refused" },
         { "command": "subscriptions add", "effect": "adds an event store subscription to a target event store" },
         { "command": "subscriptions remove", "effect": "removes an event store subscription from a target event store" },
         { "command": "users add", "effect": "adds a user to the server's user store" },

package/package/corpus/product-sync.json

@@ -63,9 +63,9 @@
         }
       ],
       "synced": {
-        "version": "22.117.0",
-        "date": "2026-10-17",
-        "by": "Cratis/AI#773"
+        "version": "22.139.0",
+        "date": "2026-10-25",
+        "by": "Cratis/AI#820"
       },
       "covers": {
         "skills": [
@@ -99,7 +99,7 @@
           "vertical-slices.md"
         ]
       },
-      "notes": "Reviewed releases 22.100.1-22.117.0 at tags: occurrence command receipts and retry classification, tenant/purpose storage resolution, explicit-offset binding and authorization of malformed bodies, camelCase validation members, conflict event details, hub fallback authorization and snapshot guards, and React query-key isolation/resubscription. Existing read-protection and generated-value guidance retained. Query discovery includes internal statics; public-only discovery in Arc#3368 is unreleased. Screenplay-generator deltas classified for the owning lane, not changed here. No new runtime probe."
+      "notes": "Reviewed all 53 releases 22.117.1-22.139.0 at tagged sources. Corrected public-only query discovery, single observations, pre-transport occurrence append refusal, assignable pipeline responses, concept binding/annotations, protected validator construction and key metadata; added live query/calendar, upsert, scenario, authentication and proxy-generation guidance. Existing response validation, React lifetimes, tenant routing and storage resolution guidance retained after reviewing fixes; internal/dependency changes need no extra consumer action. Screenplay/Scene/Stage deltas belong to their owning lanes. No live runtime probe; prior floor established in Cratis/AI#773."
     },
     {
       "id": "ai",
@@ -125,9 +125,9 @@
         }
       ],
       "synced": {
-        "version": "3.75.0",
-        "date": "2026-10-17",
-        "by": "Cratis/AI#752"
+        "version": "3.100.0",
+        "date": "2026-10-25",
+        "by": "Cratis/AI#820"
       },
       "covers": {
         "skills": [
@@ -158,7 +158,7 @@
         ],
         "rules": []
       },
-      "notes": "Render bundle source-checked at v3.74.0 (release notes v3.63.0-v3.74.0 and Documentation/reference/screenplay.md): Screenplay 4.201.0 and Stage 4.135.0; CLI rendering admits ESM v1-v26 (v17/v19 from 3.63.0, v18 from 3.65.0, v20 from 3.66.0, v21-v26 from 3.74.0); Stage refuses unsupported constructs with typed STAGE-ESM codes (v22 STAGE-ESM-016, v26 date comparisons STAGE-ESM-046) and CLI-RENDER-004 blocks v27 and later. No render probe: the installed CLI is 3.65.0. Prior at v3.62.0: Screenplay 4.159.0 and Stage 4.99.0, ESM v1-v16. PATH render uses persisted root identities read-only, rejects root/name conflicts, checks destination event ids at every scope, and reports baseline warnings. CLI baseline admit/check and open-question validation are available. Other CLI operations retain prior evidence; no new render/browser probe."
+      "notes": "Reviewed all 35 releases 3.76.0-3.100.0 at tagged sources: refusal-aware job actions, observer-wide retry, exact manual stream repair, retained-job deletion, bounded namespace metadata comparison, release notes/cached templates, Pi adapters/profile MCP and isolated AI conformance/install-only checks. Updated CLI operations and terminal Workbench guidance. Multi-service context/orchestration unchanged. Reviewed bundled render/run changes through Stage 4.196.0 and Screenplay 4.240.2, ESM v29/v30 admission, reducer migration, executable target validation, model-root resolution and sample packs; Screenplay/Stage/Prologue skill edits reserved for their owning lanes. No live store, render, harness or browser probe. Prior floor established in Cratis/AI#752."
     },
     {
       "id": "screenplay",
@@ -229,9 +229,9 @@
       "repository": "Cratis/Orleans",
       "packages": [],
       "synced": {
-        "version": "1.16.3",
-        "date": "2026-10-12",
-        "by": "Cratis/AI#620"
+        "version": "1.30.0",
+        "date": "2026-10-25",
+        "by": "Cratis/AI#820"
       },
       "covers": {
         "skills": [
@@ -243,7 +243,7 @@
         ]
       },
       "coverage": "tracked",
-      "notes": "First synced in Cratis/AI#620 at 1.16.3."
+      "notes": "Reviewed all 33 releases 1.17.0-1.30.0 at tagged sources: local IJobs and recovery keys, activation-safe callbacks and drain/admission, bounded retries/test runs, durable plans and initialization hooks, finalized-stop/outcome storage and independent placement/metadata throttle; corrected jobs and Orleans hosting guidance. Internal fixes require no extra consumer action; no live silo probe. Prior floor established in Cratis/AI#620."
     },
     {
       "id": "keldakraft",

package/package/corpus/rules/orleans.md

@@ -10,7 +10,7 @@ paths:
 > **⚠️ APPLIES ONLY TO PROJECTS USING MICROSOFT ORLEANS**
 > If your project does not reference `Microsoft.Orleans` or any Orleans packages, **ignore this entire file**. These rules are irrelevant outside of Orleans contexts.
 
-In the Cratis organization, Orleans is the Chronicle kernel's actor runtime (`Source/Kernel/`). The kernel conventions below are verified against `Cratis/Chronicle` **v19.50.0**. Shared hosting and jobs come from `Cratis/Orleans` **v1.16.3**; use its setup for other services that co-host a silo.
+In the Cratis organization, Orleans is the Chronicle kernel's actor runtime (`Source/Kernel/`). The kernel conventions below are verified against `Cratis/Chronicle` **v19.50.0**. Shared hosting and jobs come from `Cratis/Orleans` **v1.30.0**; use its setup for other services that co-host a silo.
 
 ## General
 
@@ -49,7 +49,7 @@ framework/service guidance; they do not impose application slice architecture.
 
 For service authors, `Cratis.Orleans` supplies hosting, grain serializers, and the
 shared job system without requiring Chronicle. With `Cratis.Orleans` and
-`Cratis.Orleans.Storage.MongoDB` 1.16.3, this host uses durable MongoDB membership
+`Cratis.Orleans.Storage.MongoDB` 1.30.0, this host uses durable MongoDB membership
 and jobs storage (a local MongoDB server is required):
 
 ```csharp
@@ -80,6 +80,14 @@ app.Run();
 - Register jobs storage separately: `AddCratisOrleansMongoDBJobsStorage`,
   `AddCratisOrleansSqlJobsStorage` (`Cratis.Orleans.Setup`), or
   `AddCratisOrleansInMemoryJobsStorage` (`Cratis.Orleans.Storage`).
+- Use the local `IJobs` service to start/stop/delete typed jobs during rolling
+  upgrades; generic manager calls can land on a release that does not know the
+  type. Provide singleton `IJobsManagerKeys` on every silo for all host-owned
+  scope/namespace pairs; process-local discovery is not a recovery catalog.
+- Controller and worker placement are separate opt-ins through `AddJobPlacement`
+  and `AddJobStepPlacement`. Enable them only after every silo supports the
+  strategies and register them identically. Worker throttling is not admission;
+  see **cratis-orleans-jobs** for drain-before-replacement and metadata limits.
 - Give grain implementations a unique `[GrainType("unique-name")]`. Orleans'
   default type name derives from the class name alone; a collision across
   assemblies prevents silo startup.

package/package/corpus/rules/vertical-slices.md

@@ -257,7 +257,7 @@ public record AuthorRegistered(AuthorName Name);
 
 ## Read Models and Queries
 
-**[contract]** `[ReadModel]` records expose eligible public and internal static methods as queries; internal visibility does not hide an endpoint. Dependencies are plain method parameters (no `[FromServices]`). For application code, start with model-bound mappings, use fluent when the definition needs structure, and choose reducer code when transitions read better as code. For step-by-step creation, invoke the **cratis-chronicle-read-model** skill.
+**[contract]** `[ReadModel]` records expose eligible public static methods as queries. Requires Arc 22.121.0 for non-public methods to be excluded even with `[Query]` (ARC0029 warns); older versions also expose eligible internal statics. Keep helpers outside the read model when supporting those versions. Dependencies are plain method parameters (no `[FromServices]`). For application code, start with model-bound mappings, use fluent when the definition needs structure, and choose reducer code when transitions read better as code. For step-by-step creation, invoke the **cratis-chronicle-read-model** skill.
 
 Use `[ReadModel(RequireQueryAttribute = true)]` with `[Query]` on intended
 endpoints when folds exist (ARC0026/ARCCHR0019). `[Query]` alone does not hide

package/package/corpus/skills/cratis-application-slice-specifications/references/query-and-decision-scenarios.md

@@ -1,5 +1,11 @@
 # Query and decision scenarios
 
+## Contents
+
+- Query pipeline
+- Guarded decisions
+- Sources
+
 <!-- Copyright (c) Cratis. All rights reserved. -->
 <!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
 
@@ -52,6 +58,13 @@ not a caller-supplied tenant filter. Compare whole returned items when the contr
 value; otherwise assert each named value. A query backed by a database still needs that provider's
 integration setup: scenario pins do not replace arbitrary stores.
 
+`QueryScenario` owns and disposes its provider. Dispose the scenario (prefer
+`DisposeAsync` for async-owned resources) as well as individual observations.
+Register dependencies in `Services` before the first execution. Scenario
+execution runs the query through Arc's hosted query pipeline and establishes context
+before `Provide`/`Handle` on the command path. A direct static query call does not
+cover those pipeline hooks.
+
 ## Guarded decisions
 
 Create `new CommandScenario<TCommand>()`. It selects Chronicle's in-process decision
@@ -87,6 +100,12 @@ read-model dictionary is the only backing are **legacy-mode** warnings, not a
 reason to fake protected decision readers. Keep non-key enumeration/provider
 boundaries explicit in either mode; decision mode is not a MongoDB substitute.
 
+The default Chronicle-backed `CommandScenario` discovers constraints
+automatically. Assert real constraint refusals instead of disabling them. Do not
+subclass or substitute Chronicle's `EventScenario`; use its public
+seeding/assertion surface or explicit `UseDecisionReads` mode, not a synthetic
+EventScenario facade.
+
 ## Sources
 
 - [Query scenarios](https://github.com/Cratis/Arc/blob/main/Documentation/backend/csharp/testing/query-scenario.md)

package/package/corpus/skills/cratis-arc-authentication-authorization-and-identity/SKILL.md

@@ -179,6 +179,14 @@ filters, and system-execution.
 `Task<AuthenticationResult> HandleAuthentication(IHttpRequestContext context)`.
 Implement it and it is discovered — there is **no** `IArcBuilder` extension and
 **no** options type for authentication.
+Requires Arc 22.139.0 for handlers to opt out of anonymous endpoints through
+`AppliesToAnonymousEndpoints => false`. The default is `true`: existing handlers
+keep running and may establish an identity even when no credential is required.
+Opt out when a handler only rejects credentials and contributes no identity on
+those endpoints. A handler that still runs can inspect the endpoint's anonymous
+metadata and return `AuthenticationResult.Anonymous` instead of rejecting a
+credential the endpoint never required. Specify anonymous and protected requests;
+anonymous access does not bypass other authorization boundaries.
 
 Arc ships `MicrosoftIdentityPlatformAuthenticationHandler` in `Arc.Core`, which
 reads the forwarded principal. See

package/package/corpus/skills/cratis-arc-command/references/proxy-generation.md

@@ -16,7 +16,7 @@ Reference the build package from the project that contains the commands,
 queries and read models, and set an output path:
 
 ```xml
-<PackageReference Include="Cratis.Arc.ProxyGenerator.Build" Version="22.75.0" />
+<PackageReference Include="Cratis.Arc.ProxyGenerator.Build" Version="22.139.0" />
 
 <PropertyGroup>
   <CratisProxiesOutputPath>$(MSBuildThisFileDirectory)../<Web>/src/api</CratisProxiesOutputPath>
@@ -47,6 +47,8 @@ switches without a declared default are off when absent.
 | --- | --- | --- |
 | `CratisProxiesOutputPath` | empty | Where to write. Empty disables generation entirely |
 | `CratisProxiesInputAssembly` | empty | Overrides the assembly to read |
+| `CratisProxiesIncremental` | `true` | Requires Arc 22.138.0; reuse unchanged owned output; set `false` to regenerate every build |
+| `CratisProxiesFileHeader` | project setting | Requires Arc 22.123.0; otherwise uses `.editorconfig` `file_header_template`, or no header; `none` disables it |
 | `CratisProxiesSegmentsToSkip` | empty | Leading namespace segments to drop from the output path |
 | `CratisProxiesLibraryMode` | `false` | Generates for a library rather than an application |
 | `CratisProxiesSkipOutputDeletion` | **`true`** | Already the default — the generator does not wipe the output folder |
@@ -70,6 +72,13 @@ are optional. Sources:
 `Tools/ProxyGenerator.Build/build/Cratis.Arc.ProxyGenerator.Build.targets` under
 `Source/DotNET`.
 
+Incremental generation fingerprints assemblies, generator/configuration, validation metadata
+and owned output. A changed or missing generated file forces regeneration, as
+does a previous warning so diagnostics remain visible. It does not make a
+hand-edited generated file an input. Configure `file_header_template` explicitly
+if you previously relied on the built-in Cratis MIT header; it is no longer
+inserted without configuration.
+
 Item groups the target also forwards: `AssemblyToPackageMapping`,
 `ExcludeType`, `ExcludeNamespace`, `NamespaceRoot`, and `TypeToTsType`.
 

package/package/corpus/skills/cratis-arc-command-execution/SKILL.md

@@ -101,8 +101,9 @@ if (result.IsSuccess)
 }
 ```
 
-⚠️ `Execute<TResult>` throws `InvalidCastException` when the handler returned a
-response of a different type. When the handler returns no response, or the
+`Execute<TResult>` accepts an assignable response, such as an array requested
+as `IEnumerable<T>` or a concrete value requested as an interface/base type.
+⚠️ An incompatible non-null response still throws `InvalidCastException`. When the handler returns no response, or the
 command failed for any reason, `Response` is `default` — a failed result never
 carries a response, because the pipeline clears it once the execution scopes
 have completed.

package/package/corpus/skills/cratis-arc-command-execution/references/occurrence-commands.md

@@ -61,12 +61,13 @@ key where that service supports it; otherwise an external effect can repeat.
 Return events from the command, or use transactional appends. Do not complete a
 stream or commit an aggregate early inside the executor's command.
 
-An immediate append can commit **before** the executor detects it and reports
-`OccurrenceCommandAppendedImmediately`. No receipt is recorded for that failed
-attempt, so retry can duplicate the escaped append. The executor does not
-implicitly provide Chronicle's `RefuseImmediateAppends()` guard. Read
-**cratis-chronicle-client-dotnet**'s immediate-append refusal reference when
-building a custom boundary that needs prevention before transport.
+Requires Arc 22.118.0 for the keyed executor to refuse immediate appends
+**before transport**, with neither appended events nor an effect receipt. Arc
+22.117.x detected `OccurrenceCommandAppendedImmediately` only after an append
+could escape; upgrade rather than treating that detection as retry safety.
+Use `.Transactional` or returned events, and assert that a forbidden direct
+append writes nothing. For a custom non-executor boundary, read
+**cratis-chronicle-client-dotnet**'s immediate-append refusal reference.
 
 ## Verify
 

package/package/corpus/skills/cratis-arc-command-validation/SKILL.md

@@ -142,6 +142,11 @@ public class TransferFundsValidator : CommandValidator<TransferFunds>
 - A validator that itself throws does not become a 500 — Arc catches it and
   emits a validation error with reason `ValidatorFailed`, returning 400.
 
+Read [protected validator construction](references/protected-validator-construction.md)
+for plain read-model constructor inputs under `On`, the separate untrimmed-JIT
+provider opt-in for decision tokens outside `On`, and generated composite-key
+metadata. Do not conflate metadata coverage with validator certification.
+
 ## Step 4 — Data the handler needs: `Provide()`
 
 `Provide()` runs after authorization and validation and before `Handle()`. Its

package/package/corpus/skills/cratis-arc-command-validation/references/input-binding.md

@@ -15,6 +15,14 @@ with `DateTime command property '<member>' requires an explicit offset or Z.`
 `DateTimeOffset` retains the input offset; `DateTime` becomes UTC. The attribute
 is in `Cratis.Arc.Commands`.
 
+Requires Arc 22.120.0 for concepts over `DateTime` or `DateTimeOffset` with
+Arc's default JSON options. Concepts over `DateOnly` are refused as a
+configuration error: calendar dates have no offset. The marker affects
+JSON-bound properties, including nested collection elements, not query strings
+or route scalars. Use reflection-based JSON metadata for these properties;
+System.Text.Json source-generated metadata does not support derived
+`JsonConverterAttribute` markers.
+
 ## Malformed bodies still authorize
 
 Requires Arc 22.110.0 for authorization-before-binding-failure handling on
@@ -31,6 +39,15 @@ policies can run; built-in role, claim, name and authenticated-user requirements
 are already resource-free. Do not mark a requirement resource-free if it reads
 the command.
 
+## Annotations on concepts
+
+Requires Arc 22.121.2 for data annotations on command properties and model-bound
+query parameters to validate a concept's wrapped value: `[Required]` rejects an
+empty wrapped string, `[Range]` checks the primitive, and `[StringLength]` does
+not throw on the wrapper. This uses generated validation metadata; types without
+that metadata (including external assemblies) keep the previous behavior. Do
+not assume annotations replace a concept's reusable validator.
+
 ## Validation members are field paths
 
 Requires Arc 22.115.0 for **all** `ValidationResult.Members` paths to be

package/package/corpus/skills/cratis-arc-command-validation/references/protected-validator-construction.md

@@ -0,0 +1,94 @@
+<!-- Copyright (c) Cratis. All rights reserved. -->
+<!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
+
+# Protected validator construction and key metadata
+
+## Plain read models under On
+
+Requires Arc 22.106.0 for plain Chronicle read-model dependencies in
+`CommandValidator<T>` constructors under read protection `On`. Those reads share
+the command's fold cache and append guard with `Provide`/`Handle`. Let convention
+discovery register a transient validator. Scoped/singleton/factory/instance
+registrations are refused; a plain transient self-binding is also allowed.
+Ineligible construction fails closed; never fall back to unguarded reads. Use
+`[LookUpBy(nameof(Command.Property))]` on each constructor read for multiple
+keys. Another source/stream can instead be read explicitly in `Provide`.
+
+## Explicit decision tokens outside On
+
+Requires Arc 22.126.0 for the opt-in construction of validators taking
+`DecisionRead<T>` or command-aware `IDecisionReads` on `[ProtectedDecision]`
+commands outside `On`. This is **untrimmed JIT only**, not trimmed JIT or
+NativeAOT, and not a switch making every DI container trusted.
+
+Install `ProtectedDecisionServiceProviderFactory` from
+`Cratis.Arc.DependencyInjection` as the host's actual Microsoft DI factory
+**after** `AddCratisArc`, passing the desired `ServiceProviderOptions` explicitly.
+Without them both `ValidateScopes` and `ValidateOnBuild` default to false; host
+Development settings are not inherited. `BuildProtectedDecisionServiceProvider`
+is for independently owned service collections, not a second root inside the
+running host. Execute in a native child scope (or the pipeline's scope-owning
+overload), never directly on that root. Default `CommandScenario` does not opt in.
+
+Register Chronicle through Arc's `WithChronicle`. Do not replace readers or
+re-register tokens. Arc records origins at root construction; copied descriptors,
+a wrapper provider, custom container or later collection mutation cannot acquire
+that evidence.
+
+Every validator/collaborator needs exactly one public constructor and a fully
+inspectable transient graph. Arc constructs fresh instances for the invocation.
+It refuses scoped/singleton validators or collaborators, application validator
+bindings, factory/instance dependencies, cycles, keyed/open generic/deferred or
+optional dependencies, provider/scope accessors and collections (including token
+collections). Plain models and raw `IReadModels`, `IEventStore` or `IEventLog`
+are refused on this path: use tokens or choose `On`.
+
+Requires Arc 22.126.0 for
+`services.AddProtectedDecisionValidator<TValidator>(configure)`. Use it instead
+of an application DI factory when adding rules. The callback runs once on the
+actual fresh instance before validation; thrown callbacks fail the command and
+callback denials are not bypassed. It can replace one Arc convention binding,
+not an explicit application registration. Extra explicit bindings refuse the
+protected path. Ordinary commands still use normal constructor resolution.
+
+Arc checks rule/component identities before invocation and refuses changed or
+cross-scope/reused instances. Construction and cleanup failures block before
+commit even with allowed-severity overrides. Arc disposes owned validators and
+collaborators in reverse order; DI owns read leaves. This is not a sandbox for
+trusted constructor/callback code, mutable captures, static reads or nested-child
+rule changes. `Validate` stays advisory and creates a separate invocation; only
+Execute enrolls its decision for append-time checking. Under `Off`, choose the
+blocking severity needed for business denials.
+
+## Generated key metadata is not validator certification
+
+`CommandReadModelKeys` registers command property/model-key metadata.
+Requires Arc 22.125.0 for generated composite-key descriptors and
+`ProtectedReadModelKeyAccessors`, which registers exact-type composite members.
+These optimize matching; they do not authorize a read or certify a validator's
+dependency graph.
+
+Known concrete composite keys receive generated complete descriptors. For an
+external/runtime key, register `ProtectedReadModelKeyAccessors.Register<T>` with
+every public readable non-indexed instance property, including inherited ones,
+before commands run. Generated metadata uses `DeclareKeyTypes` for startup
+coverage checks; that is not a replacement for registering member accessors.
+Duplicate registration fails. Do not omit a member or treat a
+base descriptor as covering a derived runtime type. Missing declared metadata
+fails with `ProtectedReadModelKeyAccessorMissing`.
+
+Requires Arc 22.134.0 for `ReadProtection:RequireGeneratedMetadata` (false by
+default). Enabling it rejects missing command/model metadata instead of using
+reflection. `ProtectedReadModelKeyOptions.Strict` is also false by default;
+strict composite matching or a restricted runtime requires exact accessors.
+Untrimmed JIT can use the compatibility reflection path when strict mode is off.
+ARC0030 is the composite coverage warning. Do not suppress it or promise
+NativeAOT support for the unrelated dynamic validator opt-in.
+
+## Specify the boundary
+
+Cover a fresh allowed/denied validator, callback-added denial and exception,
+refused DI graphs and cleanup failure, and repeated validation/execution without
+reusing an instance. For keys, cover inherited/flattened members, exact derived
+types, missing metadata under strict settings and generated versus compatibility
+matching. An advisory Validate success is not proof that Execute commits.

package/package/corpus/skills/cratis-arc-ef-core-migration/SKILL.md

@@ -187,6 +187,24 @@ Mark the corresponding property `[Json]`.
 `migrationBuilder.GetDatabaseType()` is available when a migration genuinely has
 to branch per provider.
 
+## Tracked and native upserts
+
+Requires Arc 22.135.0 for `DbSet<TEntity>.UpsertAsync(item, cancellationToken)`.
+On supported SQL Server, PostgreSQL and SQLite mappings it executes an immediate,
+parameterized native upsert matching the complete primary key, including composite
+keys and value converters. Native key matching and writing are atomic. It uses
+the context's connection/current transaction, bypasses tracking and does not
+refresh generated values or already tracked instances. It adds no retry policy.
+
+Use application-assigned keys and single-table scalar mappings. Unsupported
+providers or mappings (including unset generated keys, SQL Server identity
+columns, navigations, owned entities, shadow properties, inheritance and multiple
+tables) fall back to tracked persistence and require `SaveChangesAsync`. The
+existing `Upsert(item)` always uses tracked read-then-write persistence and also
+requires saving; that path is not an atomic database upsert. Choose it explicitly
+when deferred persistence is required. Do not rely on fallback for concurrency
+protection or assume the native path flushes pending tracked changes.
+
 ## Observing an EF Core read model
 
 `DbSet<T>` gains `Observe(filter?, configure?)`, `ObserveSingle(filter?, configure?)`

package/package/corpus/skills/cratis-arc-query-paging/SKILL.md

@@ -56,10 +56,12 @@ Sources: `Arc.Core/Queries/QueryStringQueryRequestReader.cs`,
 
 ## Which return shapes page
 
-A query is an eligible public or internal static method on the `[ReadModel]`
-type, discovered by its return type. Internal visibility does not hide an
-endpoint; keep helpers outside that type. Read **cratis-chronicle-read-model**'s
-query reference for explicit `[Query]` discovery and older-version limits.
+A query is an eligible public static method on the `[ReadModel]` type,
+discovered by its return type.
+Requires Arc 22.121.0 for non-public statics to be
+excluded even with `[Query]` (ARC0029 warns); older Arc exposes internal methods
+as endpoints too. Read **cratis-chronicle-read-model**'s query reference for
+explicit `[Query]` discovery and older-version limits.
 
 | Return | Paged? | How |
 | --- | --- | --- |
@@ -174,6 +176,11 @@ For composed MongoDB/EF observations, pass `ignoreQueryContext: true` to
 auxiliary sources so only the primary consumes client paging/sorting and writes
 totals. Single-instance observations emit `default` when absent, deleted or
 outside the filter; guard `hasData`/`isReady` in React.
+Requires Arc 22.117.1 for
+`ObserveSingle`/`ObserveById` never to page or count; their retained
+`ignoreQueryContext` argument has no effect. Use collection results for paging.
+Read [live query processing](references/live-query-processing.md) for shared
+MongoDB watches, emitted-value interception and calendar-bound answers.
 Source: [MongoDB observations](https://github.com/Cratis/Arc/blob/main/Documentation/backend/csharp/mongodb/observing-collections.md).
 
 ## Consume it from the frontend

package/package/corpus/skills/cratis-arc-query-paging/references/live-query-processing.md

@@ -0,0 +1,71 @@
+<!-- Copyright (c) Cratis. All rights reserved. -->
+<!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
+
+# Live query processing
+
+## Shared MongoDB watches
+
+Requires Arc 22.136.0. MongoDB `Observe()` uses a shared database-level watch keyed by deployment and
+database rather than opening a collection watch for every observer. Each
+observation keeps its filter, paging and tenant context.
+Requires Arc 22.137.0 for observations of collections created by the caller's
+own `MongoClient` to receive changes. The watcher derives the deployment key
+from the database client instead of requiring prior registration. Joined observations coalesce refresh notifications into one pending
+refresh and re-read every joined source. Snapshot read failures receive a
+bounded retry burst; exhaustion waits for a later notification rather than
+spinning or buffering unbounded work.
+
+Watch reconnects resume from the token when possible and request a new snapshot
+when the token cannot resume. The watch reopens before asking for snapshots so
+writes during the read are still observed. Disposing one observer does not stop
+the shared watch; disposing its subject or losing the last subscriber stops that
+observation's refresh work and prevents a late emission. Shared watches do not
+replace each subscription's cancellation and disposal.
+
+## Intercept the emitted model, not its wrapper
+
+Implement `Cratis.Arc.Queries.IInterceptReadModel<T>` with
+`Task<T> Intercept(T readModel)`. Return a copy when transforming data that another
+subscriber could share. A non-abstract open generic `IInterceptReadModel<T>`
+implementation must target exactly its own type parameter; closed
+implementations also work.
+
+Interception runs for supported direct/hub streaming emissions and observable
+HTTP snapshot/wait results in the requesting subscriber's scope and identity. Interceptors run before emission guards; a thrown interceptor
+fails the result rather than exposing the untransformed value. Ordinary model
+queries and Arc-wrapped MVC GETs also intercept results; `[AspNetResult]` opts
+out of that result processing. Interceptors are transformations, **not access
+control**. Filter rows/fields and authorize independently on every exposed path,
+and specify snapshot, SSE/WebSocket and hub paths separately.
+
+## Date-sensitive answers without data changes
+
+Requires Arc 22.138.0 for `Cratis.Arc.Calendar`. Inject `IApplicationCalendar` and
+pass an instant to `Read`; it does not read the clock. Zone precedence is an
+operation-declared zone, then the first non-null `ICanProvideTenantCalendar`
+answer, then `Cratis:Arc:Calendar:TimeZone` (default `UTC`). Headers and query
+arguments cannot choose it. Configured and tenant-provider ids must be usable
+IANA ids: unknown, Windows-only and nonzero fixed-offset ids fail with
+`TimeZoneIsNotAnIanaZone`,
+never a silent UTC fallback. Containers need time-zone data (`tzdata`).
+
+A live "due today" answer needs candidates observed **without a clock-dependent
+store filter**, or tomorrow's members will never enter the source. Use
+`IClockBoundaries.Begin(tenant, operationZone, token)` to obtain a
+`ClockBoundaryContext`, then `candidates.ReevaluateAtBoundaries(context,
+(candidates, reading) => new ClockEvaluation<TAnswer>(answer, nextBoundary))`.
+`LocalDateChanges.NextLocalDateChange(reading.Zone, reading.Instant)` handles
+midnight, skipped dates and backwards date transitions. `reading.Today.AddDays`
+steps calendar days; subtracting elapsed hours from the instant is different
+across daylight-saving changes.
+
+Every next boundary must be strictly after the evaluated instant or null;
+otherwise the stream fails with `ClockBoundaryIsNotInTheFuture`. The first result
+always emits; equal subsequent answers do not (collections compare in order).
+Evaluation is serialized in the context's tenant. The zone is fixed at Begin;
+application changes require restart and tenant-zone changes affect new
+subscriptions. The operator connects on first subscription and disposes timer
+and candidates on last unsubscribe, cancellation, error or completion. It does
+not accept `OnNext`. Use the registered `TimeProvider` and a `FakeTimeProvider`
+with `Advance` to specify boundary changes without sleeps. This is not a
+continuously ticking `now` or an advanceable clock in query scenarios.

package/package/corpus/skills/cratis-chronicle-cli-operations/SKILL.md

@@ -60,6 +60,10 @@ Two things worth knowing before running it:
   connection guidance in a repository-owned project context file; never patch
   packaged or generated bytes.
 
+Read [project tooling](references/project-tooling.md) for embedded release notes,
+cached templates, managed MCP registration and install-only versus harness AI
+conformance checks. Updating instructions does not run those behavior checks.
+
 ## Reach the right server
 
 The connection string resolves in this exact order:
@@ -180,6 +184,10 @@ needs Chronicle 19.29.0 or later, resets the retry budget and retries unless `--
   is given; handler side effects may repeat. `retry-partition` exits non-zero
   when quarantined or not failed. `jobs resume` resumes Stopped jobs, not Failed.
 
+Read [recovery outcomes and namespace comparison](references/recovery-and-comparison.md)
+for refusal-aware job actions, retained-job deletion, observer-wide failed-partition
+recovery, exact manual stream reopening and bounded metadata comparison.
+
 ## Before you change anything
 
 Default to read-only inspection. A request to diagnose a live store does not
@@ -188,12 +196,13 @@ authorize replay, retry, removal, quarantine clearing, or another mutation.
 These commands mutate the running store:
 
 - `chronicle observers replay`, `replay-partition`, `retry-partition`,
-  `clear-quarantine`
+  `retry-failed-partitions`, `clear-quarantine`
 - `chronicle observers remove` — destructive across every namespace of the event
   store: definition, state, handled counts, failures and projection definition;
   read model data stays. Refused while running or subscribed, no force flag.
 - `chronicle observers clear-partition-quarantine` — may retry the handler
-- `chronicle jobs stop`, `chronicle jobs resume`
+- `chronicle jobs stop`, `chronicle jobs resume`, `chronicle jobs delete`
+- `chronicle streams closed reopen` — exact manual scope repair with a reason
 - `chronicle recommendations perform`, `chronicle recommendations ignore`
 - `chronicle subscriptions add`, `chronicle subscriptions remove`
 - `chronicle users add`, `chronicle users remove`

package/package/corpus/skills/cratis-chronicle-cli-operations/references/command-effects.md

@@ -20,6 +20,7 @@ non-read-only commands.
 | `workbench` | destructive | false (its actions have their own dialogs) |
 | `applications add` | mutating | false |
 | `applications remove` | destructive | true |
+| `jobs delete` | destructive | true |
 | `jobs resume` | mutating | true |
 | `jobs stop` | mutating | true |
 | `observers clear-partition-quarantine` | mutating | true |
@@ -27,14 +28,20 @@ non-read-only commands.
 | `observers remove` | destructive | true |
 | `observers replay` | destructive | true |
 | `observers replay-partition` | destructive | true |
+| `observers retry-failed-partitions` | mutating | true |
 | `observers retry-partition` | mutating | true |
 | `recommendations ignore` | mutating | true |
 | `recommendations perform` | destructive | true |
+| `streams closed reopen` | mutating | true |
 | `subscriptions add` | mutating | false |
 | `subscriptions remove` | destructive | true |
 | `users add` | mutating | false |
 | `users remove` | destructive | true |
 
+See [recovery outcomes](recovery-and-comparison.md) for CLI/server floors and
+acceptance versus completed work. `jobs delete` exits nonzero on declined/unavailable
+confirmation; success means deletion requested, not completed removal.
+
 The remaining Chronicle commands are cataloged read-only, including
 `report-error`; the guard deliberately blocks that command because it opens an
 external browser issue form. `chronicle version` is not a command in this

package/package/corpus/skills/cratis-chronicle-cli-operations/references/project-tooling.md

@@ -0,0 +1,91 @@
+<!-- Copyright (c) Cratis. All rights reserved. -->
+<!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
+
+# CLI updates and project tooling
+
+This complements live-store operations; it does not grant permission to install
+servers, change a repository or run quota-consuming agent sessions. Inspect the
+installed version with `cratis --version` and its catalog before using a flag.
+
+## Inspect the upgrade, not a remembered bundle
+
+Requires CLI 3.87.0 for offline embedded release notes:
+
+```bash
+cratis release-notes --since <version> -o json
+cratis release-notes --version <version>
+```
+
+`cratis update` displays the skipped releases after its update results. Requires
+3.89.0 for `cratis new` to use the template set cached at `~/.cratis/templates`:
+first use downloads from NuGet; offline use can reuse a complete cached set.
+`cratis new cratis --language csharp --version <version>` selects a version for
+one run rather than replacing the installed choice. `cratis update` refreshes
+compatible templates without downgrading or failing the CLI update when refresh
+fails; `--no-templates` skips it. CLI and template versions are separate. Do not
+silently enable scaffold scripts; preserve the template's `--allow-scripts`
+choice. Multi-service orchestration still needs explicit per-store settings.
+
+For `cratis render` / `run`, read **cratis-screenplay-render-and-gap-fill** and
+**cratis-stage-rendering-and-sandbox**. Their bundled renderer determines support;
+a CLI upgrade can change admission, generated output and the default container
+without changing the model. Never infer support from the ESM version alone or
+edit generated output to make an older CLI accept it.
+
+## Managed AI installation and MCP
+
+Requires CLI 3.77.1 for `.pi/agents` to link to Pi-native corpus adapters when
+available, falling back to canonical agents only for older corpora. Do not
+hand-edit generated adapters to restore missing tools; update the CLI/corpus.
+
+Requires 3.90.0 for profile-selected servers in the corpus's `mcp-servers.json` to
+be registered by `cratis ai install` / `update` in Claude Code, Copilot, Cursor,
+Codex and OpenCode. Installation registers configuration; it does not start the
+servers. Deselecting/uninstalling removes only unchanged owned entries. Foreign
+or modified entries remain conflicts, even with force. Resolve a conflict by
+intentionally renaming/removing your entry or disabling that server with
+`mcpServers.<id>.enabled = false` in the project configuration; do not overwrite
+it speculatively. Pi supports only Screenplay through its native extension and
+reports other servers unsupported. An invalid selected descriptor is reported
+and skipped; successful guidance installation does not prove MCP availability.
+Installing a server does not authorize its tools or commands.
+
+## Deterministic AI checks versus harness behavior
+
+Requires CLI 3.98.0 for opt-in `cratis ai test`. Each fixture × harness uses a
+fresh temporary workspace with the fixture's selected profiles/languages; the
+invoking repository is unchanged. It is **not** a security sandbox: fixture
+commands and harnesses still have host tools/authentication. Trust the source,
+request the intended session/budget, and keep transcripts off unless needed.
+The default fixture root is `Evaluations/conformance`; `--fixtures` overrides
+it, `--fixture` selects names, and `--harnesses claude,codex` narrows adapters.
+`--output json` is this command's structured format, not proof of a passing run.
+
+Requires 3.100.0 for install-only CI without a harness, credentials or model quota:
+
+```bash
+cratis ai test --source <corpus> --install-only --output json
+```
+
+This runs install-phase checks for declared harness ids (or the configured ids
+when a fixture declares none), narrowed by `--harnesses`. Result-phase checks
+stay `not_run`; no install checks means skipped, not behavioral conformance.
+Normal mode rejects missing/modified installations unless corpus-source mode is
+selected. `--source`/`CRATIS_AI_SOURCE` selects a trusted local corpus.
+
+Fixtures require a non-optional result check even in install-only mode. File
+checks choose one path/glob; an empty glob match fails. Install checks cannot be
+commands. A result command uses `run: ["executable", "argument"]` with no implicit
+shell and cannot also declare file path/glob/pattern fields. Invalid check shapes
+are errors. Paths and links cannot escape the workspace.
+
+A timeout kills the process tree; output is capped. Requested `--timeout` and
+`--max-budget-usd` can lower, not raise, fixture limits. Claude enforces a spending
+limit; Codex reports that bound `not_enforced`. Missing harnesses/credentials are
+skipped with a reason, never passed. Exit 0 can mean all skipped or no selected
+runs: inspect counts and `no fixtures ran successfully`. Exit 1 means fixture,
+harness or check failures/errors; exit 2 means usage/install errors. Installed
+file hashes, observed tool reads and check results are separate evidence levels;
+unknown reads remain unknown. `--keep-transcripts` retains best-effort-redacted
+temporary output; inspect it for secrets before sharing. Passing install checks
+is not proof that an agent read or obeyed the guidance.

package/package/corpus/skills/cratis-chronicle-cli-operations/references/recovery-and-comparison.md

@@ -0,0 +1,68 @@
+<!-- Copyright (c) Cratis. All rights reserved. -->
+<!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
+
+# Recovery outcomes and namespace comparison
+
+Check the installed `cratis llm-context` and server version before using these
+commands. Confirm server, event store, namespace and exact targets. A prompt or
+`--yes` is not authority to mutate a live store.
+
+## Jobs: acceptance is not completed removal
+
+Requires CLI 3.79.0 for `chronicle jobs stop` / `resume` and the terminal
+Workbench to report kernel refusals as failures, with reasons in text/JSON and a
+nonzero exit. Do not parse an old CLI's success message as proof of progress.
+Inspect the job again; resume can be refused for missing, concluded, unpersisted
+or otherwise unresumable work. Do not repeat it without understanding the reason.
+
+Requires CLI 3.99.0 for `cratis chronicle jobs delete <JOB_ID>`. It is destructive
+and irreversible: retained job state is removed. Before confirmation it names
+the store, namespace and job id. An invalid GUID, declined/unavailable prompt,
+server refusal, authorization failure or server error is nonzero. `--yes` skips
+only the local prompt; the server still authorizes and decides. Exit zero and
+"deletion requested" are **not** proof of completed removal. Capture the relevant
+pre-state first and inspect again after an authorized deletion. This is not a
+terminal Workbench key or permission to delete every retained job.
+
+## Retry all failed partitions
+
+Requires CLI 3.79.0 and Chronicle 19.80.0 for
+`cratis chronicle observers retry-failed-partitions <OBSERVER_ID>`. It starts
+server-owned bounded-batch recovery and reports how many partitions were failed;
+it does not wait for all handlers to succeed. Inspect failures first, fix their
+cause, authorize the observer-wide target, then re-read state after the request.
+Do not mistake this for a client loop of one-partition requests or a guarantee
+that quarantines were cleared.
+
+## Inspect or repair a closed scope
+
+Requires CLI 3.80.0 and Chronicle 19.39.0 for
+`cratis chronicle streams closed list`. It is read-only and lists exact scope
+dimensions, origin, owner constraint, sequence number and closing time. Filter by
+event source id/type and event stream type/id; do not select a repair from a
+partial dimension match.
+
+`cratis chronicle streams closed reopen --reason <text>` is a confirmed,
+mutating repair of **one exact** mistaken manual close. Supply the scope flags
+shown by the installed help/catalog and the reason. The kernel records actor
+and reason in `StreamScopeReopened` and refuses a closure owned by a closing
+event. `--yes` skips confirmation, not that ownership check. Never turn this into
+a way around a modeled closure or reopen broad scopes speculatively.
+
+## Compare a model with live metadata
+
+Requires CLI 3.85.0 for
+`cratis screenplay conform-namespace <model> --event-store <store> --namespace <ns>`.
+This is a read-only comparison, not model authoring, rendering or state repair.
+It compares registered event types, producing commands/reactors and observers
+with the model, scanning metadata up to the tail captured at start. The default
+`--max-events` is `100000`; `--from`/`--to` bound a range. A partial scan stays
+partial; without `--fail-on-drift` a drift report need not fail the command.
+
+Requires 3.89.1 for authorization failures to exit 4 (`authentication_error`),
+Ctrl+C between pages to exit 8 (`inconclusive`), and event attribution only when
+the causation is a command. Chronicle's metadata API still transmits event source
+ids and subjects; the CLI drops them without persisting them. Do not claim they
+never cross the connection. It does not request payloads or command values.
+Redact the report before sharing it, and do not treat an incomplete or cancelled
+comparison as evidence of conformance.

package/package/corpus/skills/cratis-chronicle-read-model/references/queries.md

@@ -32,9 +32,11 @@ Chronicle extensions — Chronicle declares no `Observe` on `IMongoCollection<T>
 
 ## Discovery rules
 
-Arc discovers eligible **public and internal static** methods on `[ReadModel]`
-types. Internal visibility is not an endpoint privacy boundary. Keep helpers
-outside the read model, regardless of their visibility. Use
+Arc discovers eligible **public static** methods on `[ReadModel]` types.
+Requires Arc 22.121.0: non-public methods are excluded even with `[Query]`, and
+ARC0029 warns on a non-public query-shaped method. A public query on an internal
+read-model type still works. On older Arc, internal statics are exposed too;
+keep helpers outside the read model when supporting those versions. Use
 `[ReadModel(RequireQueryAttribute = true)]` and `[Query]` on intended endpoints
 when the type also has folds (`Create`/`Reduce`); `[Query]` alone does not hide
 siblings. Requires Arc 22.27.1 for exclusion of private, protected, special-name

package/package/corpus/skills/cratis-cli-terminal-workbench/SKILL.md

@@ -138,7 +138,11 @@ Several views bind a key to an operation that changes the running server:
 | Jobs | `S` / `U` | stop / resume the job |
 | Recommendations | `A` / `I` | perform / ignore the recommendation |
 
-Each has a bulk form that applies to every checked row.
+Each has a bulk form that applies to every checked row. Requires CLI 3.79.0 for
+job stop/resume to display kernel refusals as failures rather than success.
+Inspect the reason and re-read the job; acknowledgement is not completed work.
+Retained-job deletion is a separate CLI command (3.99.0), not a Workbench key;
+read **cratis-chronicle-cli-operations** before requesting it.
 
 Every one of these opens a centered confirmation modal that states the action
 cannot be undone. **Only an explicit `Y` (or the Confirm button) confirms;

package/package/corpus/skills/cratis-orleans-jobs/SKILL.md

@@ -11,32 +11,22 @@ license: MIT
 
 For services and kernel contributors using Orleans. This is library-consumer guidance, not the application vertical-slice architecture. For live Chronicle operations use **cratis-chronicle-cli-operations** instead. Co-hosting conventions are in [orleans.md](../../rules/orleans.md).
 
-## Verified product sources
-
-All Orleans claims below come from `Cratis/Orleans` source.
-
-| Packages | Sources |
-| --- | --- |
-| `Cratis.Orleans`, `Cratis.Orleans.Storage` | `Source/Orleans/Jobs/{Job.cs,Job.Steps.cs,Job.Stages.cs,JobStep.cs,IJobsManager.cs,JobsOptions.cs}`; `Source/Orleans.Storage/Jobs/*` |
-| `Cratis.Orleans.Storage.MongoDB`, `Cratis.Orleans.Storage.Sql` | `Source/Orleans.Storage.MongoDB/{MongoDBJobsStorageOptions.cs,DatabaseNames.cs}`; `Source/Orleans.Storage.Sql/{SqlJobsStorage.cs,SqlJobsStorageOptions.cs,Jobs/Migrations/v1_0_0.cs}` |
-| `Cratis.Orleans.Chronicle` | `Source/Orleans.Chronicle/StateMachines/EventSourcedStateMachine.cs`; [state-machine reference](references/state-machines.md) |
-
-The API snippets cite their exact files and lines below. Product docs `Documentation/concepts/{jobs,storage-scopes,state-machines,event-sourced-state-machines}.md` supply the task vocabulary; source resolves contradictions.
-
 ## 1. Choose a job, not a detached task
 
-A job plans recoverable work; each step is a grain and does the work. Use it for work that must outlive a request. It is **at-least-once**, not exactly-once: a step interrupted before a durable checkpoint can perform the same work again. Protect external effects with receiver-owned idempotency/deduplication; a checkpoint does not atomically commit an external write (`JobStep.cs`, `JobsOptions.cs`).
+A job plans recoverable work; each step is a grain and does the work. Use it for work that must outlive a request. It is **at-least-once**, not exactly-once: a step interrupted before a durable checkpoint can perform the same work again. Protect external effects with receiver-owned idempotency/deduplication; a checkpoint does not atomically commit an external write.
+
+- Job: derive `Job<TRequest, TJobState>`; the request is a class implementing `IJobRequest`, the state derives `JobState`. Implement `PrepareSteps(TRequest)` returning `Task<IImmutableList<JobStepDetails>>`.
+- Step: derive `JobStep<TRequest, TResult, TState>`; state derives `JobStepState`. Its constructor receives `[PersistentState(nameof(JobStepState), WellKnownGrainStorageProviders.JobSteps)] IPersistentState<TState>`, `IJobStepThrottle`, and `ILogger<JobStep<TRequest, TResult, TState>>`.
+- Implement `PrepareStep(TRequest)` → `Task<Result<PrepareJobStepError>>`; `InitializeState(TRequest)` → `ValueTask`, called after preparation succeeds; and `PerformStep(TState, CancellationToken)` → `Task<Catch<JobStepResult>>`. `TResult` is the step's result type; the work method returns the framework's `JobStepResult` envelope, not `TResult` directly.
+- Give the concrete job and step unique `[GrainType("product-specific-name")]` names. Use serializable request/state/result types. Do not reuse a class-derived grain name across assemblies.
 
-- Job: derive `Job<TRequest, TJobState>`; the request is a class implementing `IJobRequest`, the state derives `JobState` (`Job.cs`). Implement `PrepareSteps(TRequest)` returning `Task<IImmutableList<JobStepDetails>>` (`Job.Steps.cs`).
-- Step: derive `JobStep<TRequest, TResult, TState>`; state derives `JobStepState`. Its constructor receives `[PersistentState(nameof(JobStepState), WellKnownGrainStorageProviders.JobSteps)] IPersistentState<TState>`, `IJobStepThrottle`, and `ILogger<JobStep<TRequest, TResult, TState>>` (`JobStep.cs`).
-- Implement `PrepareStep(TRequest)` → `Task<Result<PrepareJobStepError>>`; `InitializeState(TRequest)` → `ValueTask`, called after preparation succeeds; and `PerformStep(TState, CancellationToken)` → `Task<Catch<JobStepResult>>` (`JobStep.cs`). `TResult` is the step's result type; the work method returns the framework's `JobStepResult` envelope, not `TResult` directly.
-- Give the concrete job and step unique `[GrainType("product-specific-name")]` names. Use serializable request/state/result types. Do not reuse a class-derived grain name across assemblies (`Jobs/JobsManager.cs`, `Documentation/how-to/host-the-silo.md`, and `orleans.md`).
+`PerformStep` runs **outside the activation context** with a deep-copied state snapshot. Mutating that snapshot does not persist it; do not access/write grain `State` directly from background work. A specification host such as Orleans.TestKit must register `DeepCopier<TState>`.
 
-`PerformStep` runs **outside the activation context** with a state snapshot obtained through the grain reference. Mutating that snapshot does not persist it; do not access/write grain `State` directly from background work (`JobStep.cs`). Keep progress/state coordination on grain calls. Requires Cratis.Orleans 1.16.2 for framework-owned stop reporting; there, do not report cancellation again from a `PerformWork` override (`Documentation/concepts/jobs.md`).
+Requires Cratis.Orleans 1.19.0 for `TryInvokeOnActivation` to coordinate state/checkpoints on the current activation, including during deactivation. Call it only from the run's background work. Keep callbacks local: capture copied inputs, call remote systems outside the callback, then commit in a second callback. Never call another grain, nest and await another callback, or start unawaited work inside it. Inspect `ActivationCallbackRejected` (`NoRun`, `StaleRun`, `Closed`, `Cancelled`); exceptions propagate. The optional `queueCancellationToken` cancels waiting callbacks only; started callbacks finish. Stop/deactivation waits for started callbacks and work to settle; `JobsOptions.StepStopTimeout` defaults to 10 seconds and `JobStepError.StopTimedOut` is not proof of quiescence. Requires 1.16.2 for framework-owned stop reporting: do not report cancellation again from a `PerformWork` override.
 
 ## 2. Plan dependencies as stages
 
-Default steps all use `JobStepStage.First`. Prefer independent fan-out. Use stages only when correctness requires earlier work to finish (`Job.Stages.cs`).
+Default steps all use `JobStepStage.First`. Prefer independent fan-out. Use stages only when correctness requires earlier work to finish.
 
 Planning excerpt inside an existing `Job<RecoveryRequest, RecoveryState>`. It assumes `RecoveryRequest : IJobRequest`, `RecoveryState : JobState`, and the two existing grain interfaces `IRepairIndex` and `IRebuildCatalog`, each deriving a typed `IJobStep<,,>`:
 
@@ -52,64 +42,62 @@ protected override Task<IImmutableList<JobStepDetails>> PrepareSteps(RecoveryReq
     ]);
 ```
 
-Import `System.Collections.Immutable` and `Cratis.Orleans.Jobs` in that job file. `CreateStep<TStep>` takes the **grain interface**, not the implementation (`Job.Steps.cs`). Its stage overload accepts `JobStepStage`, with implicit integer conversion (`Source/Orleans.Storage/Jobs/Stages/JobStepStage.cs`).
+Import `System.Collections.Immutable` and `Cratis.Orleans.Jobs` in that job file. `CreateStep<TStep>` takes the **grain interface**, not the implementation. Its stage overload accepts `JobStepStage`, with implicit integer conversion.
 
-Every step in a stage reaches an outcome before later stages start; steps within a stage run in parallel. All steps are prepared up front, so later-stage preparation cannot read the effects of earlier-stage execution. Read such dependencies in `PerformStep` (`Job.Stages.cs`).
+Every step in a stage reaches an outcome before later stages start; steps within a stage run in parallel. All steps are prepared up front, so later-stage preparation cannot read the effects of earlier-stage execution. Read such dependencies in `PerformStep`.
 
-A failed stage stops the pipeline by default: later steps become `Unreachable`, and the job completes with failures. A best-effort stage may override `GetFailureBehaviorFor(JobStepStage)` to return `JobStageFailureBehavior.ContinueWithNextStage` (`Job.Stages.cs`, namespace `Cratis.Orleans.Jobs.Stages`). Do not silently continue when a later stage relies on the failed output.
+A failed stage stops the pipeline by default: later steps become `Unreachable`, and the job completes with failures. A best-effort stage may override `GetFailureBehaviorFor(JobStepStage)` to return `JobStageFailureBehavior.ContinueWithNextStage` (namespace `Cratis.Orleans.Jobs.Stages`). Do not silently continue when a later stage relies on the failed output.
 
-## 3. Start and resume through the scoped manager
+Requires 1.25.0 for durable step plans: preparation requests must round-trip through the host's JSON options. Invalid plans return `StartJobError.InvalidStepPlan`. Recovery reuses identities and requests, prepares only unsaved steps, and never calls `PrepareSteps` again. Put retry-safe initialization needed before start **and** resume in `OnBeforeDispatchingJobSteps(JobStepDispatchContext)`, after the respective start/resume hook. `InitializationMayBeIncomplete` flags recovery from preparing/starting, not exactly-once initialization. A failed hook dispatches nothing; recovery never replays the start hook. A legacy preparing/starting job without a saved plan returns `CannotResumeJobError.MissingStepPlan` rather than inventing identities.
 
-`IGrainFactory.GetJobsManager(scope, namespace)` resolves the manager (`JobsManagerExtensions.cs`). `IJobsManager.Start<TJob,TRequest>(request)` returns `Result<JobId, StartJobError>`; inspect its error before telling a caller that a job started (`IJobsManager.cs`). Start success is not completed work; observe the job's status/outcomes.
+## 3. Start locally and recover every scope
 
-`IJobsManager.Resume(JobId)` returns **`Task<bool>`**. False means nothing took the job; do not wait for a completion callback as though resumption succeeded. True can mean resumed, already running, or finalized as completed (`IJobsManager.cs`). `Rehydrate()` completes after loading/preparing interrupted jobs, **before** its bounded background drain has resumed them all.
+Requires 1.18.0 for the singleton local service `Cratis.Orleans.Jobs.IJobs`, registered by `AddCratisOrleans`. Prefer `Start<TJob,TRequest>(new JobKey(scope, namespace), request)` (or its explicit job-id overload), `Stop(key, jobId)` and `Delete(key, jobId)` during rolling upgrades. Generic `IJobsManager.Start`/`GetJobsOfType` send types through a manager that may run an older release and cannot decode them.
 
-`StartStepsInBackground` defaults to false. Overriding it to true makes Start return once the job is durable, moving step preparation/start to the grain's next turn. Later startup errors belong to the job, not the Start response (`Job.cs`).
+Inspect `Result<JobId, StartJobError>` before telling a caller that work started. `FailedStartingSomeJobSteps` means remaining work is still running, not that nothing started; `AllJobStepsFailedStarting` means no step started. Start success is not completion. Match `StopJobResult`/`DeleteJobResult`: accepted, `NotFound`, `UnknownJobType`, or `Refused` with the job's error. Unknown types require a release that knows the type; exceptions propagate. Deletion acceptance is not completed removal. Third-party storage should override `IJobStorage.GetJobType` with a metadata-only read; its fallback reads the request too.
 
-## 4. Know what storage retains
+`IGrainFactory.GetJobsManager(scope, namespace)` remains available. `IJobsManager.Resume(JobId)` returns **`Task<bool>`**; false means nothing took the job: do not wait for a completion callback as though resumption succeeded. True may also mean already running or finalized. `IJobsManager.Stop` returns `Result<StopJobError>`; preserve a persistence refusal. `Rehydrate()` returns after loading/preparing interrupted jobs, before its bounded background drain resumes them all. `StartStepsInBackground = true` moves step preparation/start to the next turn after durability; later failures belong to the job, not Start.
 
-Successful jobs clear themselves by default; override `KeepAfterCompleted` to retain job state. Jobs completed with failures default to `KeepAfterCompletedWithFailures = true`, but retention cleanup still applies (`Job.cs`, `JobsOptions.cs`).
+Requires 1.17.0 for automatic recovery on silo activation/member death. Provide singleton `IJobsManagerKeys` implementations on **every silo**, enumerating all host-owned scope/namespace pairs through `GetKeys(CancellationToken)`. The default remembers only keys used in this process: it is not a durable catalog. Providers run outside request scope; never inject a current tenant/user or scoped database. Requires 1.23.3 (1.24.2 for provider construction failures) for automatic recovery retries of failed providers/managers, from 1 second doubling to 60 seconds until success or silo shutdown. There is no periodic sweep by default. `AddCratisOrleans` registers recovery; separately configured silos need `AddCratisOrleansJobsRecovery()` (`Cratis.Orleans.Hosting`).
 
-**Keeping a job does not keep successful step state.** The step provider deletes successful/removing state, moves failed state to failed-step storage, and saves stopped/scheduled/running/unreachable state (`JobStepGrainStorageProvider.cs`). SQL keeps failed step rows with a failure status rather than a separate collection (`Source/Orleans.Storage.Sql/Jobs/JobStepStorage.cs`, `MoveToFailed`). Copy results the job must retain through `OnStepCompletedOrStopped(JobStepId, JobStepResult)` into job state (`Job.Steps.cs`); do not read deleted successful step state later.
+Read [lifecycle and placement](references/lifecycle-and-placement.md) for drain/admission, bounded attempts, placement, custom storage compatibility and step test runs.
 
-Deliver external callbacks to a method on the **job's grain interface**, update job state there and persist it with `WriteState()`. Preserve the returned `Catch` failure; do not acknowledge input whose write failed (`Job.State.cs`, `WriteState`; `Documentation/concepts/jobs.md`). Step state is not a callback inbox: a concurrent callback can race the final write, and successful completion deletes it. A running step's snapshot will not see later job/step writes automatically.
+## 4. Know what storage retains
 
-## 5. Register storage and budget concurrency
+Successful jobs clear themselves by default; override `KeepAfterCompleted` to retain state. Jobs completed with failures default to `KeepAfterCompletedWithFailures = true`, but retention cleanup still applies.
+
+Keeping a job does not keep successful step state: successful **steps delete their state**. Failed step state is retained. A durable callback receipt, deduplication set or result must live on the **job** (and be persisted), not only on a successful step. Deliver external callbacks to a method on the job's grain interface and preserve `WriteState()`'s `Catch` failure: do not acknowledge a failed write. Step state is not a callback inbox, and a running snapshot does not see later writes automatically.
 
-`IJobsStorage.GetFor(scope, namespace)` returns `JobsStorage`. Scope and namespace are opaque engine coordinates carried in grain keys; the host/provider owns their mapping (`Source/Orleans.Storage/Jobs/IJobsStorage.cs`, `Job.cs`). They are storage boundaries, not authorization checks.
+## 5. Register storage and budget concurrency
 
-Choose one storage registration alongside `AddCratisOrleans`:
+`IJobsStorage.GetFor(scope, namespace)` returns `JobsStorage`. Scope and namespace are opaque host-owned storage coordinates, not authorization checks. Choose one storage registration alongside `AddCratisOrleans`:
 
 | Registration | Package / namespace | Use |
 | --- | --- | --- |
-| `AddCratisOrleansMongoDBJobsStorage` | `.Storage.MongoDB` / `Cratis.Orleans.Setup` | Durable MongoDB; database-name resolver can override the default `{scope}+jobs[+{namespace}]` |
-| `AddCratisOrleansSqlJobsStorage` | `.Storage.Sql` / `Cratis.Orleans.Setup` | Durable EF Core; `OptionsResolver(scope, namespace)` selects typed `DbContextOptions<JobsDbContext>` |
-| `AddCratisOrleansInMemoryJobsStorage` | `.Storage` / `Cratis.Orleans.Storage` | Specs/development only; process restart loses state |
-
-Evidence: each package's `*JobsStorageServiceCollectionExtensions.cs`; MongoDB `DatabaseNames.cs` and `MongoDBJobsStorageOptions.cs`; SQL `SqlJobsStorageOptions.cs`; in-memory `InMemory/InMemoryJobsStorage.cs`.
-
-### SQL schema creation
+| `AddCratisOrleansMongoDBJobsStorage` | `.Storage.MongoDB` / `Cratis.Orleans.Setup` | Durable MongoDB; default database `{scope}+jobs[+{namespace}]`, overridable |
+| `AddCratisOrleansSqlJobsStorage` | `.Storage.Sql` / `Cratis.Orleans.Setup` | Durable EF Core; `OptionsResolver(scope, namespace)` selects `DbContextOptions<JobsDbContext>` |
+| `AddCratisOrleansInMemoryJobsStorage` | `.Storage` / `Cratis.Orleans.Storage` | Specs/development; restart loses state |
 
-The product's SQL setup docs say to create the schema yourself. **the provider applies migrations automatically**: `SqlJobsStorage.GetFor` calls `EnsureSchema`, which calls `dbContext.Database.Migrate()` before constructing storage (`Source/Orleans.Storage.Sql/SqlJobsStorage.cs`). The migration is `Jobs/Migrations/v1_0_0.cs`; do not substitute `EnsureCreated` or hand-create those tables. The connection needs permissions to apply the migration, and scope-to-database mapping must be deliberate. Storage is cached per scope/namespace; `IJobsStorage.Reset()` clears the cache and the next resolution migrates again (`SqlJobsStorage.cs`). This resolves the documentation conflict from source, not from a release-note inference.
+The SQL provider calls `dbContext.Database.Migrate()` automatically before constructing storage. Do not substitute `EnsureCreated` or hand-create tables; grant migration permissions and choose scope-to-database mapping deliberately. `IJobsStorage.Reset()` clears the cache; the next resolution migrates again. There is no MySQL/MariaDB jobs provider.
 
-Configure `JobsOptions` through the host's options registration. Its properties are **init-only**: supply a constructed instance via the options mechanism rather than assigning them in a `Configure(options => options.Property = …)` callback (`JobsOptions.cs`).
+`JobsOptions` properties are **init-only**: supply a constructed instance through the host's options mechanism, not assignment in `Configure(options => …)`.
 
 | Option | Default | Purpose |
 | --- | --- | --- |
-| `MaxParallelSteps` | `max(1, ProcessorCount - 1)` when unset | Actual execution throttle |
-| `MaxConcurrentStepStarts` | `16` | Startup/resume calls per job, not the execution limit |
+| `MaxParallelSteps` | `max(1, ProcessorCount - 1)` when unset | Execution throttle per silo |
+| `MaxConcurrentStepStarts` | `16` | Startup/resume calls per job, not execution limit |
 | `MaxConcurrentRehydration` | `8` | Jobs resumed concurrently by a manager |
 | `MaxConcurrentCleanup` | `4` | Dead-job deletions |
-| `StepCheckpointBatchInterval` | `100` | Batches between durable progress checkpoints |
-| `StepCheckpointFlushInterval` | `5 seconds` | Timed checkpoint flush; nonpositive disables it |
-| `DeadJobThreshold`, `CleanupCadence` | `1 hour` each | Preparation-with-no-steps scavenging |
-| `TerminalJobRetention` | `7 days` | Terminal history retention; `Timeout.InfiniteTimeSpan` retains indefinitely |
+| `StepCheckpointBatchInterval` | `100` | Batches between durable checkpoints |
+| `StepCheckpointFlushInterval` | `5 seconds` | Timed flush; nonpositive disables it |
+| `DeadJobThreshold`, `CleanupCadence` | `1 hour` each | Preparation-with-no-steps scavenging; saved plans are excluded |
+| `TerminalJobRetention` | `7 days` | Terminal history; `Timeout.InfiniteTimeSpan` retains indefinitely |
 | `MaxTerminalJobsPerCleanup` | `100` | Bounded oldest-first cleanup page |
 
-Source: `JobsOptions.cs`. The three concurrency options for starts/rehydration/cleanup treat values below one as one. Retention does not delete preparing/running/stopped/removing jobs. Larger checkpoint intervals mean more post-crash redelivery, not higher delivery guarantees.
+Concurrency values below one become one. Retention does not delete preparing/running/stopped/removing jobs. Larger checkpoint intervals increase post-crash redelivery, not delivery guarantees.
 
 ## State machines and verification
 
-Use [state-machines.md](references/state-machines.md) for `StateMachine<TStoredState>` and `EventSourcedStateMachine<TModel>`: jobs are recoverable work, state machines are lifecycle behavior. Neither makes external effects exactly-once.
+Use [state-machines.md](references/state-machines.md) for `StateMachine<TStoredState>` and `EventSourcedStateMachine<TModel>`. Neither makes external effects exactly-once.
 
-Verify a new job with specs for planning, idempotent redelivery, stop/resume (including false), failed barriers/Unreachable, retention and persistence failures. Exercise the selected durable provider for restart recovery and SQL migration permissions. A build alone cannot establish crash recovery. Do not operate a live store merely to validate a code example.
+Verify planning, invalid/missing saved plans, idempotent redelivery, stop/resume/refusal, failed barriers/Unreachable, retention and persistence failures. Exercise durable storage restart recovery and SQL migration permissions. A build alone cannot establish crash recovery. Do not operate a live store merely to validate an example.

package/package/corpus/skills/cratis-orleans-jobs/references/lifecycle-and-placement.md

@@ -0,0 +1,52 @@
+<!-- Copyright (c) Cratis. All rights reserved. -->
+<!-- Licensed under the MIT license. See LICENSE file in the project root for full license information. -->
+
+# Lifecycle, recovery and placement
+
+## Stop, drain and admission
+
+Requires Cratis.Orleans 1.20.0 for `IJobs.StopAndDrain(key, jobId, timeout)` and the grain protocols `IJobDrain` / `IJobStepDrain`. Unlike ordinary stop acceptance, `Drained` means no step code can run, final outcomes/checkpoints are durable and completion/stop hooks finished. Other outcomes are `TimedOut` (call again), `Completed`, `NotFound` and `UnknownJobType`. `TimeSpan.Zero` polls. Add the method to hand-written `IJobs` implementations. Enable only after every silo supports it. A pending drain survives reactivation and refuses resume; a timeout is not proof of quiescence. Hooks can repeat or be missed after a crash, so keep them idempotent rather than collecting results only there.
+
+Requires 1.21.0 for owner admission: implement `IJobAdmission.MayRun(JobAdmissionRequest, CancellationToken)` and register with `siloBuilder.AddJobAdmission<T>()` on every silo that hosts those job types. Every admission must allow; none means allow. It runs before start, every resume and each later stage. Refusals return `StartJobError.NotAdmitted` / `CannotResumeJobError.NotAdmitted`; manager resume returns false. A refused running job is stopped/drained, and a refused later stage stops the job. Exceptions/timeouts are failures, not refusals; a later stage retries an unanswered admission on a timer.
+
+`MayRun` runs in the job's turn: never call the job, its steps, or a non-interleaving grain that could be waiting on this start/resume. To change what existing work may do, the owner calls `StopAndDrain` **before** changing the admission answer. Older silos do not ask, so admission guarantees require every silo to run the supporting version. Admission does not by itself serialize arbitrary host-owned mutual-exclusion rules.
+
+## Bounded delivery and retries
+
+Requires 1.23.0 for these opt-in execution controls. Defaults preserve unbounded dispatch, one execution attempt and no lease timeout:
+
+| Option | Default / meaning |
+| --- | --- |
+| `MaxActiveStepsPerJob` | `null` (all); a positive limit fills a dispatch window within a stage |
+| `MaxStepAttempts` | `1`, including first attempt; failed work can retry when increased |
+| `StepRetryBackoff`, `MaxStepRetryBackoff` | `5 seconds`, `5 minutes`; capped exponential delay |
+| `JobWatchdogInterval` | `1 minute`; probes stalled delivery/progress |
+| `StepLeaseTimeout` | `TimeSpan.Zero` disables heartbeat lease enforcement |
+| `StepDeliveryMaxAttempts` | `10` |
+| `StepDeliveryRetryDelay`, `MaxStepDeliveryRetryDelay` | `1 second`, `30 seconds` |
+
+The watchdog restarts a leased step only after its owner silo left the cluster **and** its lease expired. Steps saved by older releases get one lease period. `Job.ShouldRetryStep` decides which failures retry, and `Job.MaxActiveSteps` can override the window. Persist optional `Attempt` and `Lease` on custom step state; old records default to attempt 1 and no lease. `IJobStepStorage.CountByStatus(jobId)` has a default implementation. A step persists its terminal state before retrying outcome delivery; exhausted delivery does not turn that completed step into a new failed execution. The watchdog reconciles stored outcomes if reports never arrive. Keep work idempotent: leases and delivery deduplication do not make external effects exactly-once.
+
+`IJobsManager.RetryFailedSteps(jobId)` is opt-in through `Job.CanRetryFailedSteps` and runs failed steps of a job completed with failures again. Ordinary resume is unchanged. It is not a general rerun of a successful job.
+
+## Step test runs
+
+Requires 1.22.0 for `step.BeginTestRun()` and `JobStepTestRun<TState>.Perform(state)` in a specification host. `Perform` runs `PerformStep` off activation with the supplied state and returns its `Catch<JobStepResult>`; `TryInvokeOnActivation` callbacks run serially on a stand-in scheduler. `IsOnActivation` tests that state changes happen in callbacks. `Stop()` cancels work while admission remains open; `CloseAdmission()` refuses callbacks. A run performs once and records/delivers nothing to a job. Cover `JobStepAlreadyHasALiveRun` and `JobStepTestRunWasSuperseded` misuse. This does not isolate external effects: use safe substitutes, not a live job/store.
+
+## Custom storage and rolling upgrades
+
+Persist the complete `JobState`, including `StopFinalized`, `StoppedStepsCounted` and `CompletedStepsCounted`. Missing lists read as empty for old records; do not project these fields away when storing state. The completion list tracks outcomes in executions begun by resume/retry, not jobs never resumed, and warns at 100,000 entries. Cross-version code that does not persist the list can lose duplicate protection, so keep mixed-version windows short.
+
+A completed stop sets `StopFinalized` after all required notifications have succeeded. Unfinalized stops are retried; finalized stops are excluded from recovery through `IJobStorage.GetJobsExcludingFinalizedStops`. Custom providers can override its default in-memory filtering to filter in storage. MongoDB and SQL filter there; MongoDB adds `status_stopFinalized`. SQLite, PostgreSQL and SQL Server jobs storage are supported; MySQL/MariaDB is not. Removing jobs likewise retries unacknowledged notifications; acceptance of removal is not quiescence or immediate physical deletion.
+
+## Independent controller and worker placement
+
+Requires 1.30.0 for `Cratis.Orleans.Hosting` extensions on `ISiloBuilder`:
+
+- `AddJobPlacement()` or `AddJobPlacement<TDirector>()` opts controllers into `JobsPlacementStrategy`.
+- `AddJobStepPlacement()` or `AddJobStepPlacement<TDirector>()` opts workers into `JobStepsPlacementStrategy`, including recovered steps.
+- Directors implement Orleans `IPlacementDirector`. Host directors decide eligibility; throw `NoEligibleJobWorker` when no silo is eligible. Orleans does not retry that exception; the job's failure handling applies.
+- Without an opt-in, controller placement remains resource-optimized and workers use the cluster default. The built-in directors preserve those defaults when an opt-in has no custom director.
+- **Wait until every silo runs the supporting version**, then register each strategy identically on every silo. Do not introduce the strategy during a mixed-version rollout.
+
+`UseJobStepParallelismFromSiloMetadata()` independently sizes each silo's throttle from the positive integer `cratis.jobs.max-parallel-steps` metadata value (or the supplied metadata key). Missing, nonnumeric or nonpositive values fall back to `JobsOptions.MaxParallelSteps` and its processor-count default. Publish with Orleans `UseSiloMetadata`. This is a startup value, not live resizing, authorization or admission. Keep controller placement, worker eligibility, execution throttling and job admission as separate choices.

package/package/corpus/skills/cratis-orleans-jobs/references/state-machines.md

@@ -44,7 +44,9 @@ public class Finished : State<ProcessState>;
 
 Split types into their house-style files in the consumer; the block collects declarations to show the API. Do not copy the `processes` provider name into Chronicle's kernel; use its registered constants.
 
-A refused transition is a **no-op**, not an exception or a successful change. An unknown target throws `UnknownStateTypeInStateMachine`. The machine runs leaving hooks, `OnLeave`, entering hooks, `OnEnter`, then persistence (`StateMachine.cs`). Transitions requested while entering are scheduled; transitions while leaving throw `TransitioningDuringOnLeaveIsNotSupported`.
+A refused transition is a **no-op**, not an exception or a successful change. An unknown target throws `UnknownStateTypeInStateMachine`. The machine runs leaving hooks, `OnLeave`, entering hooks, `OnEnter`, then persistence (`StateMachine.cs`).
+
+Requires 1.22.3 for transitions admitted during entry to queue FIFO rather than be dropped. Their request tasks complete on admission, **before** the queued transition executes. If another state has been entered, the target is rechecked and silently skipped when refused; a failed drain discards pending targets and fails its owning task. Transitions while leaving still throw `TransitioningDuringOnLeaveIsNotSupported`.
 
 When stored state derives `StateMachineState`, `CurrentState` stores the state's full type name; activation resolves it, enters it again, and can repeat enter effects. Renames need a stored-value migration; an unknown recorded name throws `UnknownCurrentState` (`StateMachine.cs`). A self-transition with the **same state instance** skips persistence: return a new stored-state instance when a self-transition must write.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cratis/pi",
-  "version": "7.113.0",
+  "version": "7.114.0",
   "description": "Configuration-aware Cratis AI integration for Pi",
   "type": "module",
   "files": [