@klok-integration/cli 0.6.224 → 0.6.245

package/assets/agents/AGENTS.integration.md

@@ -12,6 +12,8 @@ needs. Read `klok docs read klok-framework-klok-quarkus-extension` before writin
 ./gradlew updateKlokLibsToml         # refresh gradle/klok-libs.versions.toml from the dependency catalog
 ./gradlew quarkusDev                 # run locally against the configured broker
 ./gradlew initDocsGeneration         # route facts to build/klokRoutes/docs.json
+./gradlew klokTopics                 # the broker topics the service needs, to build/klok-topics/topics.json
+./gradlew klokTopicCheck --report    # asks klok.dev whether each topic exists and is granted (KLOK_INTEGRATION_CONTEXT_TOKEN)
 ```
 
 ### Rules
@@ -25,6 +27,10 @@ needs. Read `klok docs read klok-framework-klok-quarkus-extension` before writin
   matches it, and both projects' `klok-docs/` write it down. A topic has at most one purpose; a reader is its
   subscription (its route id), never a suffix. Never consume a shared topic and discard the other flows' messages.
   The `klok-route` skill has the details.
+* The broker creates no topics. After adding or changing a route, a purpose or an override, run
+  `./gradlew klokTopics`, resolve its warnings, and make sure every topic it lists exists: request a missing one (or
+  a grant) in klok.dev, or add it to the tenant's Pulsar file in a merge request, and list it as a manual step. The
+  service fails at startup with `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED` and the fix when one is missing.
 * Errors: throw `Klok4XXException` for caller errors (no retry) and `Klok5XXException` or plain exceptions for
   retryable failures; the framework retries, republishes to `_DLQ` and emits the failure event. Do not add
   `onException` blocks unless the route owns the failure (`failureHandled()`).

package/assets/agents/AGENTS.workspace.md

@@ -338,9 +338,17 @@ access, Keycloak broker clients or a service that serves several tenants.
 * **Who applies it:** Ansible (`klok/environment/klok-environment-setup`, `keycloak.yml` and `pulsar.yml`) applies
   what those files declare: today the platform and tenant klok's services, plus any entries a person adds there for
   another tenant's service (a playbook per tenant: klok/environment/klok-environment-setup#35). Broker access (a
-  tenant, a namespace, a topic created in advance, a grant) changes only through a merge request to those Pulsar
-  files, applied by Ansible run by a person. klok.dev provisions no broker clients or grants yet
-  (klok/application/frontend-klok-dev#51). A grant made only in the broker's admin API is drift.
+  tenant, a namespace, a topic, a grant) changes only through a merge request to those Pulsar files. klok.dev merges
+  what needs no approval (the generic topic of every datamodel) and what an owner approved in klok.dev, and applies
+  topics and grants from the files; Ansible, run by a person, applies the same files and alone creates namespaces and
+  revokes grants. klok.dev provisions no broker clients yet (klok/application/frontend-klok-dev#51). A topic or grant
+  made only in the broker's admin API is drift.
+* **Topics are provisioned, never created by the broker.** A topic exists when its tenant's Pulsar file lists it and
+  the file is applied; its `_DLQ`, and the `_Response` of a `_Request` topic, come with it. `./gradlew klokTopics`
+  lists what a project needs, and a missing topic or grant fails the deploy (`klokTopicCheck`) and the service's start
+  with `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED` and the fix. Follow the message: request the topic or access in
+  klok.dev, or add the line it names in a merge request, and hand that to a person. Never ask for automatic topic
+  creation and never create a topic through the broker's admin API.
 
 ## Documentation is part of every change
 

package/assets/agents/knowledge/platform-rules.md

@@ -93,6 +93,25 @@ documentation or the dependency catalog can give you.
   for reading across tenants.
 * Request/reply: `ReqRes_<Request>_<Reply>[_<purpose>]_Request` and `_Response`. Dead letters: `<topic>_DLQ`.
 * One datamodel per topic. A breaking change is a new major version and therefore a new topic.
+* Topics are provisioned from Git; the broker never creates one by itself. A topic exists when its owning tenant's
+  Pulsar file in `klok/environment/klok-environment-setup`
+  (`ansible-master/playbooks/group_vars/all/pulsar_tenant_<tenant>.yml`) lists it and the file has been applied. A
+  producer or consumer on any other topic is refused. Subscriptions are still created by their consumer.
+* Companions come with every topic and are never requested or listed on their own: `<topic>_DLQ` for every topic, and
+  for a `_Request` topic its `_Response` (with that topic's `_DLQ`).
+* klok.dev declares the generic topic `<Model>` of every datamodel, and the ODL's change event topics. Every other
+  topic (a purpose, a request/reply pair) and every grant is requested in klok.dev and approved by the owner (a
+  TenantAdmin of the tenant or a DomainAdmin of the namespace), or added to the Pulsar file in a merge request.
+* `./gradlew klokTopics` lists the topics a project needs (`build/klok-topics/topics.json`, with a warning for every
+  topic the scan cannot resolve) and `./gradlew klokTopicCheck` asks klok.dev whether each exists and is granted to
+  the service; in CI it blocks the deploy. A service checks the same at startup and does not start without them.
+* A missing topic or grant is `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED`, never retried. The message names the topic,
+  the role, the operation, who approves, the klok.dev request link (`https://klok.dev/monitoring/broker/request?...`)
+  and the line to add by hand. Follow it: hand the request link or the merge request to a person. Never create a topic
+  or a grant through the broker's admin API, never ask for automatic topic creation, and never turn the startup check
+  (`klok.broker.verify-topics`) off in a deployed service.
+* A route that consumes another tenant's topic cannot write its `_DLQ`; where such dead letters go is open
+  (klok/framework/klok-common#11).
 * Every broker message is a CloudEvent in binary content mode: the attributes are message properties, the payload
   is the typed datamodel. `ce-source` is `urn:klok:<tenant>:<solutionId>:<routeId>`; `ce-type` says what happened
   (default `dev.klok.<tenant>.<domain>.<Model>.published`).
@@ -135,9 +154,10 @@ whether a model may travel on a topic; it decides how widely the model is shared
   (`persistent://<tenant>[_<env>]/common/<Model>`), the service is granted `consume` on exactly those topics, and
   what the platform writes back lives in klok's namespace, one topic per tenant
   (`persistent://klok[_<env>]/<domain>/<Model>_<tenant>`).
-* Broker access (a tenant, a namespace, a topic created in advance, a grant) changes only through a merge request to
-  the Pulsar variables in `klok/environment/klok-environment-setup`, applied by Ansible run by a person. A grant made
-  only in the broker's admin API is drift.
+* Broker access (a tenant, a namespace, a topic, a grant) changes only in Git, in the owning tenant's Pulsar file in
+  `klok/environment/klok-environment-setup`. klok.dev applies topics and grants from the file (merging what needs no
+  approval and what an owner approved); Ansible, run by a person, applies the same files and alone creates namespaces
+  and revokes. A topic or grant made only in the broker's admin API is drift.
 * One Keycloak client per running deployment, never shared and never used by a person. Broker rights are called
   `publish` and `consume`; *ingress* and *egress* say what a service is and are never permissions.
 

package/assets/agents/knowledge/retired-patterns.txt

@@ -27,6 +27,13 @@ topic-dlq # dead letters are <topic>_DLQ
 (?i)last purpose segment # a topic has one optional purpose set by the publisher; a reader is its subscription (route id)
 (?i)_<reader> # a topic has one optional purpose set by the publisher; a reader is its subscription (route id)
 (?i)\breader (as|after) the (last )?purpose # a topic has one optional purpose set by the publisher; a reader is its subscription
+# Topics are provisioned from Git and never created by the broker (2026-10-08): a topic exists when its tenant's
+# Pulsar file lists it and the file is applied; its _DLQ and a _Request topic's _Response come with it.
+(?i)\btopics? (is|are) (auto-?)?created (automatically )?(by|on|with|at) (the |its )?first (producer|publisher|consumer|publish|send|use|message) # topics are provisioned from Git, never created on first use
+(?i)created automatically by (the|its) first (producer|publisher|consumer) # topics are provisioned from Git, never created on first use
+(?i)\btopics? (appears?|comes? into existence) (with|on|at) (the |its )?first (producer|publisher|consumer|publish|use|message) # topics are provisioned from Git, never created on first use
+(?i)\ba topic created in advance # every topic is listed in its tenant's Pulsar file before it is used; "in advance" is the only way
+(?i)\b(enable|turn on|switch on) (automatic|auto-?)[ -]?topic[ -]creation # the Klok broker runs with allowAutoTopicCreation=false; request the topic in klok.dev
 #
 # Framework classes that do not exist in the Quarkus extension 2.0 and later.
 KlokSoapApi # removed in 2.0; SOAP is a KlokCustomRoute with camel-cxf

package/assets/agents/knowledge/roles/integration-developer.md

@@ -206,6 +206,14 @@ blindly. `klok-mock-*` headers are for test and staging and are stripped in prod
    authentication: alert and dead letter), the tests.
 5. **Code and review.** Check the transform, the dead letter path, the CloudEvent attributes, logging through the
    framework (never `System.out`), and the service's broker client and grants.
+6. **Topics.** The broker never creates a topic. A datamodel's generic topic `<Model>` and every companion (`_DLQ`,
+   the `_Response` of a `_Request`) come by themselves; a purpose topic, a request/reply pair and every grant outside
+   the service's own namespaces are requested. Run `./gradlew klokTopics`, then have each missing topic or grant
+   requested in klok.dev (`/monitoring/broker/request`) or proposed as a merge request to the tenant's Pulsar file,
+   approved by the namespace's Domain Admin or the Tenant Admin. When `klokTopicCheck`, the pipeline's
+   `klok_topic_check` or the startup report names a `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED`, follow the message:
+   it holds the request link, the approver and the line to add. Never suggest automatic topic creation. The whole
+   way is the user guide page *Topics On The Klok Broker* (`klok docs read klok-user-guide`).
 
 ## Documentation, tests and delivery
 

package/assets/agents/skills/klok-release/SKILL.md

@@ -17,21 +17,26 @@ roles and secrets, catalog update) before acting.
    dependency catalog; datamodels publish the generated artifact and update the catalog; integrations build and
    push an image and update the deployment manifest in `klok/environment/ipaas/ipaas-service-deploy-manifests`,
    which ArgoCD applies; documentation projects publish to https://docs.klok.dev.
-3. Check the pipeline after every push:
+3. An integration's deploy waits for its broker topics: the job `klok_topic_check` (`./gradlew klokTopicCheck`) asks
+   klok.dev whether every topic the service needs exists and is granted, blocks the manifest update on `main` when
+   one is not, and only warns in a merge request pipeline. Before a merge, run `./gradlew klokTopics` and hand every
+   missing topic or grant over as a request in klok.dev (the link in the message), approved by its owner; the broker
+   never creates a topic, so there is nothing to wait out and no automatic creation to ask for.
+4. Check the pipeline after every push:
    `glab api --hostname git.klok.dev "projects/<id>/pipelines?per_page=3"`, then
    `projects/<id>/pipelines/<pid>/jobs` and `projects/<id>/jobs/<jid>/trace`. Fix a red pipeline before the next task.
-4. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
+5. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
    catalog update of one before merging the next: the catalog's commit for that release, not the merge, which only
    starts the pipeline that produces it:
    `glab api --hostname git.klok.dev "projects/klok%2Fframework%2Fklok-gradle-dependency-catalog/repository/commits?ref_name=main&per_page=10" | jq -r '.[].title'`.
    A library that needs no change of its own is not released by hand: after each catalog update the job runs
    `klok catalog release-dependents`, which starts the pipeline of every framework library built against an older
    version of what was released (its log says which; `--dry-run --library <artifact>` shows it by hand).
-5. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
+6. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
    release waiting on Maven Central or npm can take half an hour, and `main` pipelines queue behind each other or run
    in parallel, so a long wait is normal. Check the current time with `date -u` before judging how long anything has run. Report the
    job id, its run time, the last log lines with timestamps and what waits behind it, propose what to do, and wait.
-6. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
+7. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
    every merge request with its link, whether the order matters and why, and for each later step the release
    result to wait for and the command that shows it.
 
@@ -42,7 +47,8 @@ project by hand — `glab api POST /projects`, or cloning the template and swapp
 fallback under *Git är sanningen*.
 
 Neither path provisions a pipeline schedule yet, nor for a deployed service the manifest, its Vault objects, its
-Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Name
+Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Its
+topics beyond the generic topic of each datamodel are requested in klok.dev and wait for an owner's approval. Name
 those human steps up front instead of letting them surface
 as a red pipeline. Vault CI login is not one of them: it needs nothing per project.
 
@@ -63,7 +69,9 @@ as a red pipeline. Vault CI login is not one of them: it needs nothing per proje
   hand before the first deploy, with the Kubernetes service account; a Vault Kubernetes role `<name>-role` (with
   `audience=https://kubernetes.default.svc`) and its policy `<name>-policy` reading
   `secret/data/<tenant>/integration/<name>/env`; the env secret itself (broker credentials, service specific
-  keys); and the service's own Keycloak client with its broker grants.
+  keys); the service's own Keycloak client with its broker grants; and every topic `./gradlew klokTopics` lists, in
+  its owning tenant's Pulsar file and applied (requested in klok.dev, or a merge request to
+  `klok/environment/klok-environment-setup`). A topic's `_DLQ` and a `_Request` topic's `_Response` come with it.
 * Documentation projects need the tenant's docs upload client: Keycloak client `klok-docs-upload-<tenant>` and its
   id and secret in `secret/<tenant>/common/repo/cicd-environment-variables`.
 
@@ -84,3 +92,8 @@ field, and let a person run it.
 | `already deployed` from JReleaser | Maven Central already has that version; a `main` build is a new timestamp, do not retry the same job |
 | `sed: can't read klok/integration/<name>/<name>-crds.yaml` | no deployment manifest for the service yet |
 | `cannot run ssh` | the job needs `openssh-client`; the templates install it before pushing over SSH |
+| `klok_topic_check` fails with `TOPIC_MISSING` | a topic the service needs is not in its tenant's Pulsar file: request it with the link in the output, an owner approves |
+| `klok_topic_check` fails with `TOPIC_ACCESS_DENIED` | the service's broker client holds no grant on the topic: request `consume` (or `publish` in its own tenant) with the link |
+| `klok_topic_check` fails with `NOT_APPLIED` | the entry is merged but not on the broker yet: the next *Ensure* run in klok.dev applies it |
+| `klok_topic_check` says to configure `KLOK_DEV_URL` or `KLOK_INTEGRATION_CONTEXT_TOKEN` | the CI environment object lacks them; a person adds them in Vault |
+| a pod restarts with `broker topics are missing or not granted; this service cannot start` | the startup check found the same problems; follow each entry of the report, then restart the deployment |

package/assets/agents/skills/klok-route/SKILL.md

@@ -20,7 +20,8 @@ motivates.
 2. **Name it.** `<TENANTSHORT><NNN><LEG><seq>_<Description>`, one class per route, the solution number from the
    portal registry. Sub routes are `S`, timers `T`.
 3. **Bind the datamodels** as type parameters; take them from the catalog bundle, never hand written DTOs.
-4. **Choose the topic purpose** of every topic the route publishes or consumes (next section).
+4. **Choose the topic purpose** of every topic the route publishes or consumes (*Topic purpose*), and make sure every
+   topic exists (*Topics on the broker*): the broker never creates one.
 5. **Configure** through `routeConfig("key", default)` / `requiredRouteConfig("key")` and document every key in
    `application.properties` as `<RouteId>.<key>`. Reserved keys: `enabled`, `logevents.*`, `retry.*`, `dlq.*`,
    `topic.*`, `subscription.*`, `claimcheck.*`.
@@ -89,10 +90,42 @@ the purpose.
 The user guide page *Topic Purpose: One Datamodel, Several Flows* (`klok docs read klok-user-guide`)
 explains the reasons and the naming in full.
 
+## Topics on the broker
+
+The broker creates no topics. A topic exists when its owning tenant's Pulsar file in
+`klok/environment/klok-environment-setup` (`ansible-master/playbooks/group_vars/all/pulsar_tenant_<tenant>.yml`)
+lists it and the file has been applied; its `_DLQ`, and for a `_Request` topic its `_Response`, come with it.
+Subscriptions are created by their consumer. klok.dev declares the generic topic `<Model>` of every datamodel; a
+purpose topic, a request/reply pair and a grant on another domain's or tenant's topic are requested.
+
+1. **List.** After adding or changing a route, a purpose, a subscription or an override, run `./gradlew klokTopics`
+   and read `build/klok-topics/topics.json` (not the console). Every entry in `warnings` is work: make the purpose,
+   subscription or override something the scan can read (configuration or a constant), or name the topic in the
+   merge request yourself.
+2. **Check.** With `KLOK_INTEGRATION_CONTEXT_TOKEN` set, run `./gradlew klokTopicCheck --report` and read
+   `build/klok-topics/topic-check.json`. `TOPIC_MISSING`: the topic is not in Git. `TOPIC_ACCESS_DENIED`: the service's
+   broker client (`service-account-<klok.broker.auth.client-id>`) holds no grant. `NOT_APPLIED`: in Git, not yet on
+   the broker; nothing to request.
+3. **Add what is missing, through the request.** For each problem, hand the person the request link from the message
+   (`https://klok.dev/monitoring/broker/request?topic=…&access=…`) and who approves it (a TenantAdmin of the owning
+   tenant or a DomainAdmin of its namespace), or, when the task asks for it, open the merge request that adds the
+   exact line the message names to the owning tenant's Pulsar file. List the topic itself, never its `_DLQ` or
+   `_Response`. Both are manual steps in your hand-over guide; the deploy waits for them.
+4. **When a service does not start,** its log has one report from `KlokTopicCheck` with an entry per missing topic or
+   grant, each ending with the request link and the line to add by hand. Do what each entry says; never work around
+   it.
+
+`klok.broker.verify-topics=false` is only for a local broker that creates topics itself, never in a deployed service.
+A route that consumes another tenant's topic cannot write that topic's `_DLQ`; say so in the design, and set
+`<RouteId>.failure.unresolved` deliberately (open question klok/framework/klok-common#11).
+
 ## Things that are always wrong
 
 * A route id that does not start with the solution id, or two routes with the same id.
 * Publishing to a hand written topic string; use the resolver and the datamodel.
+* Asking for automatic topic creation, creating a topic or a grant through the broker's admin API, or retrying a
+  `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED` failure as if the broker were down; the topic is requested and approved
+  through Git.
 * Consuming a shared topic and discarding the messages of other flows (a filter on a payload field, an ODL lookup
   per message) where the producer could publish the flow with a topic purpose.
 * A purpose with `_`, a version or an environment in it, a second suffix after the purpose, a tenant in it that the

package/assets/agents/skills/klok-solution/SKILL.md

@@ -74,8 +74,9 @@ shared platform service) in `klok/integration-solutions`.
 * When the document and the code disagree, the code on `main` is what is built: correct the document. When the code
   differs from what was agreed, file an issue in the owning project and name it in the document's open questions.
 * The deployed landscape counts too: a leg that is on `main` but not deployed, or deployed from somewhere the
-  document does not name, is written down. Deployment manifests are in `klok/environment/ipaas-se-deploy-manifests`,
-  hosts and stateful services in the Ansible inventories of `klok/environment/klok-environment-setup`.
+  document does not name, is written down. Deployment manifests are in
+  `klok/environment/ipaas/ipaas-service-deploy-manifests`, hosts and stateful services in the Ansible inventories of
+  `klok/environment/klok-environment-setup`.
 
 ## Things that are always wrong