@ccoalm/ccl-skills 0.1.0

package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-service-connectivity/references/multi-env-routing.md

@@ -0,0 +1,192 @@
+# Multi-Environment Routing (Lane / Canary / Shadow)
+
+How a request labelled "I belong to env X" reliably reaches X-labelled instances, across every hop including queues.
+
+## Lane is the single primitive
+
+One concept, many uses:
+
+| Use | Lane value example |
+|---|---|
+| Long-lived environment | `prod`, `staging`, `pre`, `test` |
+| Per-developer iteration | `dev-alice`, `dev-bob` |
+| Per-integration-test run | `it-<yyyymmdd>-<n>` |
+| Canary slice | `prod-canary-1pct` |
+| Shadow traffic | `prod-shadow` |
+| Stress/load test traffic | `stress` (often via separate `stress_tag` flag, not lane) |
+
+Treat them all as lanes for routing purposes. The mesh and SDK do not need to distinguish "dev" from "canary"; they only match labels.
+
+## End-to-end lane chain
+
+```
+Client (browser/app/external caller)
+   │ sets <lane-header> in request (or default lane assumed)
+   ▼
+Ingress gateway
+   │ reads <lane-header>; if missing, sets default ("prod")
+   │ forwards
+   ▼
+Service A's mesh sidecar
+   │ VirtualService: match <lane-header> → route to subset where instance tag lane=<value>
+   ▼
+Service A app code
+   │ framework server middleware extracts lane → ctx
+   │ business logic runs in lane-aware mode if needed (rare; usually transparent)
+   ▼
+Service A outbound RPC
+   │ framework client middleware reads lane from ctx → sets RPC base.Tags.lane
+   │ framework client middleware sets <lane-header> on outbound HTTP if used
+   ▼
+Service B's mesh sidecar
+   │ VirtualService routes lane to lane=B-subset
+   ▼
+... and so on
+```
+
+Lose lane anywhere → request leaks to wrong env. The most common leak points:
+
+- **Queue boundary**: producer writes a message without lane attribute; consumer doesn't know which lane to use → defaults to prod and processes test traffic.
+- **Async task**: a goroutine / asyncio task spawned without copying lane from parent ctx.
+- **Caching layer**: cache keyed without lane → test reads prod cache or vice versa.
+- **Cron job**: no inbound request, no lane → defaults to prod (often the right default; document it).
+
+## VirtualService pattern
+
+```yaml
+apiVersion: networking.istio.io/v1
+kind: VirtualService
+metadata:
+  name: <callee>
+spec:
+  hosts: ["<callee-service>"]
+  http:
+    - match:
+        - headers:
+            <lane-header>:
+              exact: test
+      route:
+        - destination:
+            host: <callee-service>
+            subset: lane-test
+    - match:
+        - headers:
+            <lane-header>:
+              exact: dev-alice
+      route:
+        - destination:
+            host: <callee-service>
+            subset: lane-dev-alice
+    - route:                                # default: prod
+        - destination:
+            host: <callee-service>
+            subset: lane-prod
+```
+
+Pair with `DestinationRule` declaring subsets by tag:
+
+```yaml
+kind: DestinationRule
+spec:
+  host: <callee-service>
+  subsets:
+    - name: lane-prod
+      labels: { lane: prod }
+    - name: lane-test
+      labels: { lane: test }
+    - name: lane-dev-alice
+      labels: { lane: dev-alice }
+```
+
+Generate VirtualService + DestinationRule programmatically per lane; manual YAML for dozens of lanes is unmaintainable.
+
+## Queue propagation
+
+Producer:
+```
+msg.headers["<lane-header>"] = appctx.GetLane(ctx)
+msg.headers["<log-id-header>"] = appctx.GetLogId(ctx)
+msg.headers["traceparent"]    = otel_inject(ctx)
+```
+
+Consumer:
+```
+ctx = appctx.ContextWithLane(ctx, msg.headers["<lane-header>"])
+ctx = appctx.ContextWithLogId(ctx, msg.headers["<log-id-header>"])
+ctx = otel_extract(ctx, msg.headers["traceparent"])
+```
+
+If the queue technology has no headers (rare), use a JSON envelope:
+```
+{ "_meta": { "lane": "...", "log_id": "...", "traceparent": "..." },
+  "payload": {...} }
+```
+
+## Async / goroutine propagation
+
+Whenever you fork work from a request:
+
+```go
+// Wrong: lane is lost
+go doSideEffect()
+
+// Right
+go doSideEffect(appctx.Clone(ctx))   // platform helper that copies lane, log-id, trace
+```
+
+Provide a platform helper (`appctx.Clone` or equivalent) that copies platform ctx keys but cancels independently. Forbid raw `context.Background()` in handler-spawned goroutines.
+
+## Fallback rules
+
+What happens when a callee has zero instances in the requested lane?
+
+| Strategy | When to use |
+|---|---|
+| **Strict** (fail with 503) | Lane is mandatory for the use case (canary safety) |
+| **Fall back to prod** | Long-running dev lanes that only customize a few services |
+| **Fall back to a named default lane** | Pre-prod hierarchies (e.g. `dev → pre → prod`) |
+| **Round-robin across all lanes** | Bad idea; don't do this |
+
+Document the fallback per environment. "Silent fall through" creates the worst class of bug: works in dev, breaks in prod.
+
+## Canary as a lane
+
+Canary deploys are just a small-percentage lane:
+
+```yaml
+- match:
+    - headers:
+        <lane-header>:
+          exact: prod
+  route:
+    - destination: { host: <svc>, subset: lane-prod-canary }
+      weight: 1
+    - destination: { host: <svc>, subset: lane-prod-stable }
+      weight: 99
+```
+
+This means the release skill (`platform-release-engineering`) shifts traffic by editing VirtualService weights, not by deploying new pods. The connectivity layer provides the primitive; release owns the policy.
+
+## Shadow traffic
+
+Shadow = send a copy of prod traffic to a non-prod target without affecting the user response. Istio supports `mirror`:
+
+```yaml
+- route:
+    - destination: { host: <svc>, subset: lane-prod }
+  mirror:
+    host: <svc>
+    subset: lane-shadow
+  mirrorPercentage:
+    value: 10.0
+```
+
+Shadow targets MUST tag responses as discarded; metrics from shadow must not contaminate prod SLIs.
+
+## Verification
+
+1. Start a service with `lane=test`. Register to SD with that tag. Verify with `nacos-cli list <svc>` (or registry equivalent) — instance tag includes `lane=test`.
+2. Send a request with `<lane-header>: test` from an external client. Inspect mesh access log `upstream_host` — confirm it resolves to a `lane=test` pod IP.
+3. Send a request with no lane header. Inspect — confirm it routes to the default lane.
+4. Spawn an async task from a handler. Confirm the spawned task's logs carry the same lane as the originating request.
+5. Send a message to a queue. Consumer logs should show the same lane in its handler ctx.

package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-service-connectivity/references/protobuf-http-contract-signals.md

@@ -0,0 +1,64 @@
+# Protobuf HTTP Contract Signals
+
+Use this when deciding whether an HTTP surface is routine JSON/OpenAPI or protobuf-backed HTTP.
+
+## Authoritative Record
+
+The wire-format decision must be owner-authored or owner-approved and locatable in at least one of:
+
+- shared IDL / IDLGen contract metadata
+- generated client or server package metadata
+- backend architecture document
+- backend repo contract
+- MR description that records the architecture decision and carries an owner sign-off or approval reference distinct from the change author's own assertion
+
+API docs can support the decision, but do not replace the authoritative record when protobuf may be in scope. Verbal owner guidance counts only after it is captured in a quotable, locatable record. Agent-authored notes can cite owner records, but cannot substitute for owner confirmation.
+
+Owner approval must be checkable from an artifact outside the diff under review. Acceptable signals include an explicit owner sign-off line in an external approval system, approval reference, contract-owner field from an existing contract-owner registry, IDL/IDLGen owner metadata from a prior commit or published artifact, generated package provenance, or a linked approval record. Author-written MR text alone is unconfirmed; in-diff sign-off text never proves ownership or approval.
+
+Classify owner approval with this decision list:
+
+- Existing surface with pre-existing owner record, and implementation author matches that recorded owner: `owner-confirmed`; self-authored MR prose is not the evidence, the pre-existing owner record is.
+- Existing surface with pre-existing owner record, and implementation author is not that owner: require a checkable approval from the recorded owner or owner delegate; missing approval is `pending-contract-owner`.
+- Existing surface with owner metadata or generated provenance added or regenerated only by the same diff: `pending-contract-owner`; same-diff metadata cannot establish ownership.
+- Genuinely new surface: the same diff may introduce the first contract-owner record only when it is paired with a checkable external approval reference or a verified gate record outside the diff. If the approver is distinct from the implementation author, record `owner-confirmed`; if the implementation author is the new recorded owner and no distinct approver exists, record `pending-external-approval-check` until a checkable external gate or human owner process proves owner self-approval is accepted for first introduction.
+- If a required approval reference is missing: `pending-contract-owner`.
+- If only the external identity or self-approval gate is inaccessible to the agent: `pending-external-approval-check`; this blocks the wire-format / owner-confirmation claim, not unrelated implementation work.
+- Default: unresolved owner state is non-pass; do not report it as owner-confirmed.
+
+`Unclear wire format` means no authoritative record is locatable for a surface that is in this wire-format gate.
+
+A surface enters the gate when any of these are true:
+
+- the new or modified surface has a protobuf signal, cross-language/shared-IDL requirement, binary transport, or protobuf-generated artifact boundary
+- the diff changes contract source, protobuf or binary payload content-type/media type, protobuf response serialization, protobuf/binary marshaling config, wire format, protobuf-generated client, or protobuf-generated schema
+
+A surface does not enter the gate when any of these are true:
+
+- existing unchanged JSON/OpenAPI or protobuf-backed surfaces are only touched by unrelated work, and the diff shows no contract source, protobuf or binary payload content-type/media type, protobuf response serialization, protobuf/binary marshaling config, wire format, protobuf-generated client, or protobuf-generated schema change
+- routine JSON/OpenAPI services, public API surfaces, backend integrations, request/response shape, endpoint, status/error, JSON serialization, or OpenAPI-generated client/schema changes have no protobuf signal, cross-language/shared-IDL requirement, binary transport, protobuf/binary content-type/media-type change, protobuf response serialization change, protobuf/binary marshaling config change, or protobuf-generated artifact change
+
+A standing protobuf signal on an unchanged surface does not pull unrelated work into this gate.
+
+Out-of-scope classification needs evidence. Cite the specific diff lines or unchanged generated artifacts that prove the changed slice did not alter any in-scope trigger above. If that proof is unavailable, classify as pending owner/wire-format evidence instead of out of scope.
+
+## Protobuf Signal
+
+Treat protobuf as in scope when any of these are present. This signal list is not a standalone approval trigger; apply the gate above to distinguish changed surfaces from unrelated work on unchanged surfaces.
+
+- IDL / IDLGen metadata mentions the HTTP surface
+- protobuf generated client, server, schema, or package is imported or published for the surface
+- protobuf generation config or generation command covers the surface
+- request or response uses a protobuf content type or documented binary SDK path
+- existing client code serializes or parses protobuf-generated message types for the surface
+- backend owner guidance captured in an architecture doc, repo contract, or MR description says the surface uses protobuf as contract source or wire format; MR-description guidance counts only when it carries the same checkable owner signal required by Authoritative Record
+
+The signal list is intentionally shared across Go, Python, web, mini-program, and testing skills. Do not redefine it per stack.
+
+## Decision Rules
+
+- New services, public API surfaces, and backend integrations require a contract-definition record before implementation. They require positive contract-owner confirmation of wire format only when there is a protobuf signal, cross-language/shared-IDL requirement, binary transport, or protobuf-generated artifact boundary. Routine JSON/OpenAPI services, public surfaces, and endpoints may use the contract-definition record when none of those triggers exists. JSON-only choices that do need owner confirmation require a checkable owner signal; an agent-authored inline note is not enough.
+- For modified HTTP surfaces, first prove whether the diff changes contract source, protobuf or binary payload content-type/media type, protobuf response serialization, protobuf/binary marshaling config, wire format, protobuf-generated client, or protobuf-generated schema. If yes, record the wire format before implementation; missing records route to the contract owner and block approval.
+- Existing unchanged JSON/OpenAPI or protobuf-backed surfaces preserve current behavior for unrelated work when the diff proves no contract or wire-behavior change. Missing records in that case do not block the unrelated change, but the work must not claim backend contract conformance.
+- If the client stack cannot reach the backend record or owner, an assumed-wire-format note never unblocks a wire-format change. Any change touching contract source, protobuf or binary payload content-type/media type, protobuf response serialization, protobuf/binary marshaling config, wire format, protobuf-generated client, or protobuf-generated schema stays blocked until the contract owner confirms the wire format.
+- Never change an existing wire format because documentation is missing. Preserve current client behavior, add the missing record, then require an explicit consumer-migration decision before changing the format.

package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-service-connectivity/references/retry-timeout-circuit-breaker.md

@@ -0,0 +1,124 @@
+# Retry, Timeout, Circuit Breaker
+
+Where each policy lives, how they compose, and how to keep them from killing each other.
+
+## Layered policy
+
+For a single call A → B, four policies can fire:
+
+```
+App business deadline             ← ctx.Deadline; "this user can wait at most 800ms"
+   ⊇
+Framework client per-call budget   ← SDK option; "give this call 500ms"
+   ⊇
+Mesh transport timeout (ceiling)   ← DestinationRule; "any call to B caps at 2s"
+   ⊇
+TCP/HTTP2 connect + idle timeouts  ← Envoy defaults; rarely tuned
+```
+
+Each outer policy must be ≥ the next inner one. Inversions cause "request succeeded internally but caller already returned timeout" — a hard-to-debug class.
+
+## Timeout budget rule
+
+For a chain A → B → C:
+
+```
+A.ctx.deadline = D
+A's client-to-B budget    ≤ D - margin                   (margin: 50-100ms for serialization)
+B's internal work         ≤ (B-budget - C-budget)
+B's client-to-C budget    ≤ B-budget - margin
+```
+
+A framework helper should compute "remaining ctx deadline" and pass `min(ctx.Deadline, my-budget)` to outbound calls.
+
+## Retry placement
+
+| Layer | Retries WHAT | When |
+|---|---|---|
+| Mesh (DestinationRule) | network errors, gRPC `UNAVAILABLE`, certain 5xx | Always-safe-to-retry transports |
+| Framework client | idempotent business RPCs | Per-method opt-in |
+| App handler | nothing | App level retry usually wrong |
+| App business logic | high-level workflows | Saga / orchestration patterns, not "I'll retry the call once" |
+
+Double retry = real bad. If mesh retries 2x and framework retries 2x, you get 4x amplification on every flapping backend.
+
+**Rule**: when enabling framework-level retry, disable mesh retry for that callee (DestinationRule `retries.attempts: 0`).
+
+## Idempotency awareness
+
+Framework client retries MUST consider idempotency:
+
+```
+RPC method declares "idempotent: true" in IDL or annotation
+   ↓
+SDK retry middleware reads this; only retries idempotent methods
+   ↓
+Non-idempotent retry happens only if the network error proves the request didn't reach the server
+(connect refused, TLS handshake failure — yes; "request sent, no response" — no)
+```
+
+Don't trust HTTP method (POST can be idempotent; GET can have side effects). Trust the declaration.
+
+## Circuit breaker / outlier detection
+
+Mesh outlier-detection (Envoy):
+
+```yaml
+trafficPolicy:
+  outlierDetection:
+    consecutive5xxErrors: 5         # 5 consecutive 5xx → eject this pod
+    interval: 10s                   # check every 10s
+    baseEjectionTime: 30s           # eject for 30s minimum
+    maxEjectionPercent: 50          # never eject more than 50% of pool
+```
+
+This is the right place for circuit breaking. Per-host, automatic, observable via Envoy stats.
+
+Framework SDK circuit breakers (e.g. hystrix-style) are a fallback for environments without mesh, or for business-logic-driven breaking (e.g. "this dependency's error rate hit 10%, switch to degraded mode").
+
+Don't run mesh outlier-detection AND SDK circuit breaker simultaneously without understanding the interaction.
+
+## Cascading cancel
+
+When a caller's ctx is cancelled (deadline, client disconnect, user back-button), the cancel MUST propagate to in-flight downstream calls. Frameworks should support this natively; verify by spawning a long-running downstream call and cancelling the parent — both should terminate.
+
+If a service swallows ctx cancel, downstream load amplifies during user disconnects (every abandoned tab continues hammering the DB).
+
+## Hedging
+
+Hedging = send a second request after a timeout T, return whichever responds first. Useful for latency-sensitive read APIs.
+
+Risks:
+- Doubles load if T is too short.
+- Not safe for non-idempotent calls.
+- Mesh-level hedging is preferred (e.g. Envoy `retry_priority` patterns); SDK hedging is workable but harder to tune.
+
+Default: off. Enable per-method after measuring p99 latency distribution.
+
+## Common mistakes
+
+- Setting framework client timeout > mesh timeout → client never sees mesh-side errors, retries the same call, mesh times out anyway. Align them.
+- Retrying on 4xx → client errors don't fix themselves; you just amplify load.
+- Retrying with exponential backoff while caller deadline is 200ms → backoff exceeds deadline, retry never fires, you wasted code.
+- Mesh retry + SDK retry both on, both default 3 → 9 actual attempts per logical call.
+- Circuit breaker tripped but no metric → debugging blind.
+
+## Tuning starting points
+
+| Policy | Default |
+|---|---|
+| Mesh request timeout | 5s (HTTP), 10s (RPC); per-callee override |
+| Mesh retry attempts | 2 |
+| Mesh retry per-try timeout | 1s |
+| Mesh outlier detection | 5 consecutive 5xx, 30s eject |
+| Framework client per-call budget | 500ms or shrunk from ctx deadline |
+| Framework client retry | OFF by default; opt-in per idempotent method |
+
+Tune from observed p99 + error rate, not vibes.
+
+## Verification
+
+- Trigger downstream 5xx storm → mesh outlier-detection metrics show ejection events; client-side error rate spikes then recovers.
+- Force a connect refusal → mesh retry attempts visible in mesh metric; final caller sees one error.
+- Set per-call timeout below mesh ceiling → confirm caller sees its own deadline, not mesh's.
+- Cancel a request mid-flight → confirm downstream stops within one network round-trip.