@klok-integration/cli 0.6.235 → 0.7.248

package/assets/agents/skills/klok-docs/SKILL.md

@@ -10,12 +10,16 @@ project exposes its whole content as a single markdown file, generated by znai (
 serves it to a browser session and to a GitLab personal access token, which is what klok-cli sends:
 
 ```
+klok docs list                       # the projects you may read: doc id, title, description
 klok docs read <doc-id>              # the markdown on stdout, one status line on stderr
 klok docs read <doc-id> --json       # {docId, url, source, status, etag, fetchedAt, bytes, content}
 klok docs read <doc-id> --out x.md   # to a file
 klok docs id [tenant/group/.../repo] # the doc id of a repository, this one by default
 ```
 
+`klok docs list` reads the portal's index, so it is where to find the doc id of a project you do not know yet. A
+docs server without the index answers 404; derive the doc id with `klok docs id` instead.
+
 Read it; it is the authoritative version of the repository's `klok-docs/` plus the generated route, datamodel and
 configuration pages. Do not answer from memory when a page exists. The answer is cached under
 `~/.cache/klok/docs/` with its ETag, so reading a page twice costs one conditional request; without network the
@@ -64,7 +68,8 @@ question is about what a project does.
 
 1. Run `klok agents update --check` once at the start of the task; on exit 5 run `klok agents update`, so the
    skills you read are the catalogs' current ones.
-2. Identify the repositories involved and derive their doc ids (`klok docs id <path>`).
+2. Identify the repositories involved and find their doc ids: `klok docs list`, or `klok docs id <path>` for a
+   repository you know.
 3. `klok docs read <doc-id>` for each. Handle 404, 401 and 403 as the table says.
 4. Read the strategy documentation first when the task touches design or naming, and the *Onboarding a tenant*
    runbook in `klok-framework-development-guide` when it touches onboarding; read the framework documentation

package/assets/agents/skills/klok-release/SKILL.md

@@ -17,21 +17,26 @@ roles and secrets, catalog update) before acting.
    dependency catalog; datamodels publish the generated artifact and update the catalog; integrations build and
    push an image and update the deployment manifest in `klok/environment/ipaas/ipaas-service-deploy-manifests`,
    which ArgoCD applies; documentation projects publish to https://docs.klok.dev.
-3. Check the pipeline after every push:
+3. An integration's deploy waits for its broker topics: the job `klok_topic_check` (`./gradlew klokTopicCheck`) asks
+   klok.dev whether every topic the service needs exists and is granted, blocks the manifest update on `main` when
+   one is not, and only warns in a merge request pipeline. Before a merge, run `./gradlew klokTopics` and hand every
+   missing topic or grant over as a request in klok.dev (the link in the message), approved by its owner; the broker
+   never creates a topic, so there is nothing to wait out and no automatic creation to ask for.
+4. Check the pipeline after every push:
    `glab api --hostname git.klok.dev "projects/<id>/pipelines?per_page=3"`, then
    `projects/<id>/pipelines/<pid>/jobs` and `projects/<id>/jobs/<jid>/trace`. Fix a red pipeline before the next task.
-4. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
+5. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
    catalog update of one before merging the next: the catalog's commit for that release, not the merge, which only
    starts the pipeline that produces it:
    `glab api --hostname git.klok.dev "projects/klok%2Fframework%2Fklok-gradle-dependency-catalog/repository/commits?ref_name=main&per_page=10" | jq -r '.[].title'`.
    A library that needs no change of its own is not released by hand: after each catalog update the job runs
    `klok catalog release-dependents`, which starts the pipeline of every framework library built against an older
    version of what was released (its log says which; `--dry-run --library <artifact>` shows it by hand).
-5. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
+6. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
    release waiting on Maven Central or npm can take half an hour, and `main` pipelines queue behind each other or run
    in parallel, so a long wait is normal. Check the current time with `date -u` before judging how long anything has run. Report the
    job id, its run time, the last log lines with timestamps and what waits behind it, propose what to do, and wait.
-6. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
+7. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
    every merge request with its link, whether the order matters and why, and for each later step the release
    result to wait for and the command that shows it.
 
@@ -42,7 +47,8 @@ project by hand — `glab api POST /projects`, or cloning the template and swapp
 fallback under *Git är sanningen*.
 
 Neither path provisions a pipeline schedule yet, nor for a deployed service the manifest, its Vault objects, its
-Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Name
+Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Its
+topics beyond the generic topic of each datamodel are requested in klok.dev and wait for an owner's approval. Name
 those human steps up front instead of letting them surface
 as a red pipeline. Vault CI login is not one of them: it needs nothing per project.
 
@@ -63,7 +69,9 @@ as a red pipeline. Vault CI login is not one of them: it needs nothing per proje
   hand before the first deploy, with the Kubernetes service account; a Vault Kubernetes role `<name>-role` (with
   `audience=https://kubernetes.default.svc`) and its policy `<name>-policy` reading
   `secret/data/<tenant>/integration/<name>/env`; the env secret itself (broker credentials, service specific
-  keys); and the service's own Keycloak client with its broker grants.
+  keys); the service's own Keycloak client with its broker grants; and every topic `./gradlew klokTopics` lists, in
+  its owning tenant's Pulsar file and applied (requested in klok.dev, or a merge request to
+  `klok/environment/klok-environment-setup`). A topic's `_DLQ` and a `_Request` topic's `_Response` come with it.
 * Documentation projects need the tenant's docs upload client: Keycloak client `klok-docs-upload-<tenant>` and its
   id and secret in `secret/<tenant>/common/repo/cicd-environment-variables`.
 
@@ -84,3 +92,8 @@ field, and let a person run it.
 | `already deployed` from JReleaser | Maven Central already has that version; a `main` build is a new timestamp, do not retry the same job |
 | `sed: can't read klok/integration/<name>/<name>-crds.yaml` | no deployment manifest for the service yet |
 | `cannot run ssh` | the job needs `openssh-client`; the templates install it before pushing over SSH |
+| `klok_topic_check` fails with `TOPIC_MISSING` | a topic the service needs is not in its tenant's Pulsar file: request it with the link in the output, an owner approves |
+| `klok_topic_check` fails with `TOPIC_ACCESS_DENIED` | the service's broker client holds no grant on the topic: request `consume` (or `publish` in its own tenant) with the link |
+| `klok_topic_check` fails with `NOT_APPLIED` | the entry is merged but not on the broker yet: the next *Ensure* run in klok.dev applies it |
+| `klok_topic_check` says to configure `KLOK_DEV_URL` or `KLOK_INTEGRATION_CONTEXT_TOKEN` | the CI environment object lacks them; a person adds them in Vault |
+| a pod restarts with `broker topics are missing or not granted; this service cannot start` | the startup check found the same problems; follow each entry of the report, then restart the deployment |

package/assets/agents/skills/klok-route/SKILL.md

@@ -20,7 +20,8 @@ motivates.
 2. **Name it.** `<TENANTSHORT><NNN><LEG><seq>_<Description>`, one class per route, the solution number from the
    portal registry. Sub routes are `S`, timers `T`.
 3. **Bind the datamodels** as type parameters; take them from the catalog bundle, never hand written DTOs.
-4. **Choose the topic purpose** of every topic the route publishes or consumes (next section).
+4. **Choose the topic purpose** of every topic the route publishes or consumes (*Topic purpose*), and make sure every
+   topic exists (*Topics on the broker*): the broker never creates one.
 5. **Configure** through `routeConfig("key", default)` / `requiredRouteConfig("key")` and document every key in
    `application.properties` as `<RouteId>.<key>`. Reserved keys: `enabled`, `logevents.*`, `retry.*`, `dlq.*`,
    `topic.*`, `subscription.*`, `claimcheck.*`.
@@ -89,10 +90,42 @@ the purpose.
 The user guide page *Topic Purpose: One Datamodel, Several Flows* (`klok docs read klok-user-guide`)
 explains the reasons and the naming in full.
 
+## Topics on the broker
+
+The broker creates no topics. A topic exists when its owning tenant's Pulsar file in
+`klok/environment/klok-environment-setup` (`ansible-master/playbooks/group_vars/all/pulsar_tenant_<tenant>.yml`)
+lists it and the file has been applied; its `_DLQ`, and for a `_Request` topic its `_Response`, come with it.
+Subscriptions are created by their consumer. klok.dev declares the generic topic `<Model>` of every datamodel; a
+purpose topic, a request/reply pair and a grant on another domain's or tenant's topic are requested.
+
+1. **List.** After adding or changing a route, a purpose, a subscription or an override, run `./gradlew klokTopics`
+   and read `build/klok-topics/topics.json` (not the console). Every entry in `warnings` is work: make the purpose,
+   subscription or override something the scan can read (configuration or a constant), or name the topic in the
+   merge request yourself.
+2. **Check.** With `KLOK_INTEGRATION_CONTEXT_TOKEN` set, run `./gradlew klokTopicCheck --report` and read
+   `build/klok-topics/topic-check.json`. `TOPIC_MISSING`: the topic is not in Git. `TOPIC_ACCESS_DENIED`: the service's
+   broker client (`service-account-<klok.broker.auth.client-id>`) holds no grant. `NOT_APPLIED`: in Git, not yet on
+   the broker; nothing to request.
+3. **Add what is missing, through the request.** For each problem, hand the person the request link from the message
+   (`https://klok.dev/monitoring/broker/request?topic=…&access=…`) and who approves it (a TenantAdmin of the owning
+   tenant or a DomainAdmin of its namespace), or, when the task asks for it, open the merge request that adds the
+   exact line the message names to the owning tenant's Pulsar file. List the topic itself, never its `_DLQ` or
+   `_Response`. Both are manual steps in your hand-over guide; the deploy waits for them.
+4. **When a service does not start,** its log has one report from `KlokTopicCheck` with an entry per missing topic or
+   grant, each ending with the request link and the line to add by hand. Do what each entry says; never work around
+   it.
+
+`klok.broker.verify-topics=false` is only for a local broker that creates topics itself, never in a deployed service.
+A route that consumes another tenant's topic cannot write that topic's `_DLQ`; say so in the design, and set
+`<RouteId>.failure.unresolved` deliberately (open question klok/framework/klok-common#11).
+
 ## Things that are always wrong
 
 * A route id that does not start with the solution id, or two routes with the same id.
 * Publishing to a hand written topic string; use the resolver and the datamodel.
+* Asking for automatic topic creation, creating a topic or a grant through the broker's admin API, or retrying a
+  `TOPIC_MISSING` or `TOPIC_ACCESS_DENIED` failure as if the broker were down; the topic is requested and approved
+  through Git.
 * Consuming a shared topic and discarding the messages of other flows (a filter on a payload field, an ODL lookup
   per message) where the producer could publish the flow with a topic purpose.
 * A purpose with `_`, a version or an environment in it, a second suffix after the purpose, a tenant in it that the

package/assets/agents/skills/klok-solution/SKILL.md

@@ -74,8 +74,9 @@ shared platform service) in `klok/integration-solutions`.
 * When the document and the code disagree, the code on `main` is what is built: correct the document. When the code
   differs from what was agreed, file an issue in the owning project and name it in the document's open questions.
 * The deployed landscape counts too: a leg that is on `main` but not deployed, or deployed from somewhere the
-  document does not name, is written down. Deployment manifests are in `klok/environment/ipaas-se-deploy-manifests`,
-  hosts and stateful services in the Ansible inventories of `klok/environment/klok-environment-setup`.
+  document does not name, is written down. Deployment manifests are in
+  `klok/environment/ipaas/ipaas-service-deploy-manifests`, hosts and stateful services in the Ansible inventories of
+  `klok/environment/klok-environment-setup`.
 
 ## Things that are always wrong
 

package/assets/agents/skills/klok-worktree/SKILL.md

@@ -35,7 +35,8 @@ basename, never put the worktree inside `.claude`, inside the canonical checkout
    work on. A worktree under `<root>/worktrees` does not sit below those directories, so no tool loads them for
    you. Keep the existing project rules; this skill does not replace them.
 3. **Inspect, do not touch.** `git -C <canonical> status --short`, `git -C <canonical> worktree list --porcelain`
-   and the canonical branch. Never `stash`, `reset`, `clean`, `checkout` or `switch` there. Uncommitted changes in
+   and the canonical branch. Never `stash`, `reset`, `clean`, `checkout` or `switch` there; the only update of a
+   canonical checkout is `klok sync pull`, which fast-forwards it or reports why it cannot. Uncommitted changes in
    the canonical checkout are not a blocker; the worktree starts from committed history. If the task depends on
    those changes, say so and ask for a handoff instead of moving them.
 4. **Choose the base.** Fetch the remote and resolve its default branch from the remote HEAD