@ccoalm/ccl-skills 0.6.2 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/assets/marketplace/plugins/ccl-skills/hooks/hooks.json +11 -0
- package/dist/assets/marketplace/plugins/ccl-skills/hooks/remind-unverified-cli-flag.sh +309 -0
- package/dist/assets/marketplace/plugins/ccl-skills/hooks/test_remind_unverified_cli_flag.sh +483 -0
- package/dist/assets/marketplace/plugins/ccl-skills/packages/opencode-plugin/ccl-skills.ts +5 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/app-cross-platform-dev/SKILL.md +10 -8
- package/dist/assets/marketplace/plugins/ccl-skills/skills/app-cross-platform-dev/references/mobile-quality-release.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/SKILL.md +16 -17
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/references/client-routing.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/references/staged-review-contract.md +195 -7
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/references/timeout-auth-and-capabilities.md +3 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/claude_review.sh +13 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/codex_review.sh +9 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/kimi_review.sh +9 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/normalize_review_timeout.sh +22 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/opencode_review.sh +9 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/review_gate.py +1540 -129
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/test_claude_review_probe.sh +8 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/test_review_client_compat.py +76 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/test_review_gate.sh +1858 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/test_update_review_plan_intent.sh +789 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/code-review/scripts/update_review_plan_intent.py +513 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-architecture/SKILL.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-architecture/references/architecture-playbook.md +2 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-architecture/references/data-platform-architecture.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-architecture/references/event-driven-architecture.md +14 -11
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-architecture/references/multi-tenant-isolation.md +2 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/go-microservice-dev/SKILL.md +5 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/llm-inference-integration/SKILL.md +2 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/miniapp-product-dev/SKILL.md +13 -11
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/SKILL.md +64 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/agents/openai.yaml +4 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/references/async-lifecycle-and-performance.md +72 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/references/runtime-and-project-contract.md +58 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/references/source-map.md +41 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/nodejs-service-dev/references/verification-diagnostics-and-security.md +63 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-observability/SKILL.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-observability/references/sli-slo-design.md +25 -9
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-observability/references/source-register.md +1 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-release-engineering/SKILL.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-release-engineering/references/promotion-gate-and-review.md +16 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-release-engineering/references/secret-and-config-management.md +7 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/platform-service-connectivity/references/retry-timeout-circuit-breaker.md +11 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-rd-workflow/SKILL.md +8 -10
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-rd-workflow/references/design-routing-and-readiness.md +10 -14
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-rd-workflow/references/verify-developer-experience.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/SKILL.md +135 -86
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/behavioral-aesthetic-logic.md +66 -80
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/delivery-contract.md +275 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/design-execution-checklist.md +88 -214
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/design-impl-naming-and-versioning.md +2 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/design-intake-and-acceptance.md +10 -8
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/design-system-source-of-truth.md +4 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/external-ui-ux-quality-benchmarks.md +112 -95
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/frontend-code-evidence-map.md +30 -21
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/interaction-design-patterns.md +22 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/layout-recipes-and-screenshot-acceptance.md +20 -17
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/multi-project-token-consistency.md +7 -9
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/multi-stack-strategy.md +14 -10
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/operational-processing-workflows.md +2 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/platform-mobile-patterns.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/product-lifecycle-acceptance-and-iteration.md +9 -6
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/product-surface-patterns.md +3 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/source-map.md +37 -10
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/tokens-and-components.md +7 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/ui-ux-audit.md +8 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/ui-ux-design-development.md +16 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/product-ui-ux-design/references/visual-craft.md +4 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/SKILL.md +5 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/architecture-playbook.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/audit-history-architecture.md +31 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/data-platform-architecture.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/event-driven-architecture.md +7 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/multi-tenant-isolation.md +2 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/notification-architecture.md +28 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/packaging-runtime-readiness.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/replay-comparison-architecture.md +28 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/references/workflow-state-architecture.md +39 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/SKILL.md +10 -7
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/ai-service-wiring-patterns.md +8 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/audit-history-patterns.md +29 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/background-job-patterns.md +16 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/batch-and-artifact-patterns.md +25 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/notification-patterns.md +40 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/public-api-security-patterns.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/replay-comparison-patterns.md +30 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/state-machine-task-patterns.md +48 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-dev/references/testing-and-quality-patterns.md +10 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/release-coordination/SKILL.md +2 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/SKILL.md +4 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/coverage-exhaustion-traps.md +45 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/dual-track-review-gate.md +142 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/external-practice-controls.md +21 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/extraction-quickstart.md +11 -9
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/firing-point-placement.md +8 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/parallel-stack-references-pattern.md +5 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/r0-leakage-audit.md +102 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/source-register.md +69 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/source-to-skill-extraction.md +10 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/uiux-judgment-extraction.md +6 -6
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/references/validation-and-landing.md +4 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/check-ccl-skills.sh +93 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/check-parallel-stack-parity.sh +119 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/extraction_review_gate.sh +22 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/impact-chain-gate.rb +49 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/obligation-ledger.py +2748 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/register-firing-path-resolution.rb +20 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/shared_git_surface_gate.py +1142 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_check_ccl_parallel_stack_parity.sh +183 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_check_ccl_regressions.sh +19 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_check_ccl_skill_catalog.sh +41 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_ci_checkout_ref_binding.sh +120 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_entrypoint_domain_scan_terms.sh +82 -8
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_extraction_review_gate.sh +336 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_impact_chain_self_adjudication.sh +82 -10
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_obligation_ledger.sh +1416 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_obligation_ledger_repo_audit.sh +57 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_register_firing_path_wiring.sh +141 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_routing_pointer_integrity.sh +3 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_shared_git_surface_gate.sh +1696 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_uiux_delivery_contract.sh +2117 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_uiux_loading_budget.sh +316 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_validate_extraction_review_state.sh +1176 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/test_validate_skill_cross_refs.sh +31 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/validate-skill.sh +9 -4
- package/dist/assets/marketplace/plugins/ccl-skills/skills/skill-extraction-workflow/scripts/validate_extraction_review_state.py +980 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/terminal-cli-dev/SKILL.md +9 -6
- package/dist/assets/marketplace/plugins/ccl-skills/skills/testing-strategy/SKILL.md +11 -11
- package/dist/assets/marketplace/plugins/ccl-skills/skills/testing-strategy/references/client-runtime-test-matrices.md +10 -2
- package/dist/assets/marketplace/plugins/ccl-skills/skills/testing-strategy/references/fitness-functions.md +16 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/testing-strategy/references/scenario-testing.md +1 -1
- package/dist/assets/marketplace/plugins/ccl-skills/skills/testing-strategy/references/test-code-authoring-patterns.md +16 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/tighten-doc/SKILL.md +5 -3
- package/dist/assets/marketplace/plugins/ccl-skills/skills/tighten-doc/references/delivery-face-closeout.md +16 -6
- package/dist/assets/marketplace/plugins/ccl-skills/skills/tighten-doc/references/self-benchmark-baseline.md +37 -0
- package/dist/assets/marketplace/plugins/ccl-skills/skills/web-react-dev/SKILL.md +7 -5
- package/dist/assets/marketplace/plugins/ccl-skills/skills/web-react-dev/references/complex-workspace-patterns.md +1 -1
- package/dist/assets/release.json +275 -105
- package/package.json +1 -1
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# UI/UX Design And Development
|
|
2
2
|
|
|
3
|
-
Use this reference when the task is to design a UI/UX surface and implement it in
|
|
3
|
+
Use this reference when the task is to design a UI/UX surface and implement it in client code. It is a primitive catalog and translation guide; `delivery-contract.md` is authoritative for the design brief, testing selection, producer/client returns, evidence semantics, and design verdict.
|
|
4
4
|
|
|
5
5
|
This reference is for new-product UI/UX execution. Do not copy education-domain workflows, wording, assets, product assumptions, or information architecture from source artifacts.
|
|
6
6
|
|
|
@@ -13,6 +13,8 @@ Before coding, define:
|
|
|
13
13
|
- **Density**: consumer relaxed, productive compact, or hybrid.
|
|
14
14
|
- **Source pattern**: which Figma/code pattern is being reused and what domain details are discarded.
|
|
15
15
|
- **State set**: use the canonical taxonomy in `product-surface-patterns.md`, then add implementation-specific loading, retry, cancellation, permission, long-content, and responsive behavior.
|
|
16
|
+
- **Client owner set**: follow every affected rendered layer in `delivery-contract.md`. React web → `web-react-dev`; Vue/Svelte/static/vendor/other web → its installed web-content owner or fail-closed project-convention lookup; native mobile/host → `app-cross-platform-dev`; mini-app → `miniapp-product-dev`; terminal/CLI/TUI → `terminal-cli-dev`; Electron/desktop/TV shell → its installed owner or the same project-convention lookup. Composite hosts keep separate content and shell members.
|
|
17
|
+
- **Producer owner**: every changed or claim-bearing backend, config, content, or inference source that supplies a rendered value or behavior; record its exact artifact/version identity before client execution.
|
|
16
18
|
|
|
17
19
|
Do not start from a decorative layout. Start from the user job, interaction loop, and required states.
|
|
18
20
|
|
|
@@ -25,8 +27,8 @@ Do not start from a decorative layout. Start from the user job, interaction loop
|
|
|
25
27
|
5. **Implement state model**: represent loading/error/empty/partial/success/permission explicitly in data and UI.
|
|
26
28
|
6. **Implement responsive behavior**: mobile safe area and keyboard behavior; web secondary panel collapse and min/max widths.
|
|
27
29
|
7. **Implement feedback**: inline validation, toast/message, alert/notice, drawer/sheet, modal/dialog, result page.
|
|
28
|
-
8. **
|
|
29
|
-
9. **
|
|
30
|
+
8. **Return execution evidence**: have every changed or claim-bearing producer and every affected client return its own immutable record, then have the test owner bind its definition and execution records to the exact producer/client versions exercised. Keep commands, artifacts, criterion results, dimensions, coverage boundary, and gaps in their owning records as required by `delivery-contract.md`; a screenshot proves only its captured state.
|
|
31
|
+
9. **Record the design verdict**: run the relevant checks in `ui-ux-audit.md`, evaluate every criterion, and bind the design record plus `candidate`, `accepted`, `rejected`, or `pending` to the complete design/test/producer/client candidate-binding set.
|
|
30
32
|
|
|
31
33
|
## Mobile Frontend Patterns
|
|
32
34
|
|
|
@@ -134,6 +136,8 @@ Avoid silent catches and console-only errors for user-triggered actions.
|
|
|
134
136
|
|
|
135
137
|
## Responsive Acceptance
|
|
136
138
|
|
|
139
|
+
Treat the Mobile and Web checks below as stack-specific examples. Build the complete affected client-owner set from `delivery-contract.md`; add mini-app host/device, ordinary CLI or terminal/TUI, Electron/desktop/TV shell, other-Web, and composite-host evidence whenever those layers are affected.
|
|
140
|
+
|
|
137
141
|
Mobile:
|
|
138
142
|
|
|
139
143
|
- Safe top/bottom areas are respected.
|
|
@@ -150,6 +154,12 @@ Web:
|
|
|
150
154
|
- Sidebar collapsed mode remains discoverable.
|
|
151
155
|
- Tables/lists preserve row identity and selected/filter state.
|
|
152
156
|
|
|
157
|
+
Other affected clients:
|
|
158
|
+
|
|
159
|
+
- Mini-app evidence covers the shipped host/tool, supported device class, safe area, permissions/capabilities, package/platform constraints, and embedded web-view bridge when present.
|
|
160
|
+
- Ordinary CLI or terminal/TUI evidence covers command/help/default/exit/recovery semantics, TTY and non-TTY/plain modes as applicable, width/capability/color fallback, and interactive lifecycle only where used.
|
|
161
|
+
- Electron/desktop/TV and other-Web evidence comes from the actual content owner plus shell owner or fail-closed project convention, including supported sizes/scaling, input/focus, bridge, and content-shell integration.
|
|
162
|
+
|
|
153
163
|
Screenshot acceptance:
|
|
154
164
|
|
|
155
165
|
- First viewport shows the primary workflow, not a decorative banner or empty dead area.
|
|
@@ -159,13 +169,13 @@ Screenshot acceptance:
|
|
|
159
169
|
|
|
160
170
|
## Development Review Checklist
|
|
161
171
|
|
|
162
|
-
|
|
172
|
+
Use this list as client-side criteria before returning the client record. It cannot by itself finish the slice; completion requires bound design and test records, every changed producer and affected client return, Test Phase 1 sufficiency, and an allowed design verdict under `delivery-contract.md`.
|
|
163
173
|
|
|
164
174
|
- The code uses local primitives and tokens before custom markup/styles.
|
|
165
175
|
- The flow maps to `discover -> inspect -> act -> confirm -> return`.
|
|
166
176
|
- Canonical states from `product-surface-patterns.md` are implemented, not just documented.
|
|
167
177
|
- Feedback strength follows `interaction-design-patterns.md`.
|
|
168
|
-
- Mobile safe-area/keyboard and
|
|
178
|
+
- Every affected client's adaptation contract is covered; Mobile safe-area/keyboard and Web responsive behavior are examples, not the closed set.
|
|
169
179
|
- Global feedback providers, async wrappers, and route/workspace state are mounted at the shell level when multiple feature pages rely on them.
|
|
170
180
|
- Long-running uploads, imports, downloads, generation, or review jobs remain visible after route changes and have retry/fail/complete states.
|
|
171
181
|
- Designed states have a code owner: shell/provider state, route state, feature state, server task state, or local draft state. Do not leave a designed state as static markup with no data transition.
|
|
@@ -174,3 +184,4 @@ Before finishing UI/UX implementation, verify:
|
|
|
174
184
|
- Visual polish passes `visual-craft.md`.
|
|
175
185
|
- Screenshot acceptance passes `layout-recipes-and-screenshot-acceptance.md`.
|
|
176
186
|
- UI/UX review passes `ui-ux-audit.md`.
|
|
187
|
+
- The client return names the exact producer member/version exercised and contributes its immutable member to the complete design/test/producer/client binding set.
|
|
@@ -102,10 +102,12 @@ If a pattern appears in a Figma source, preserve it only when it has a clear pro
|
|
|
102
102
|
|
|
103
103
|
## Acceptance Questions
|
|
104
104
|
|
|
105
|
-
|
|
105
|
+
These questions contribute visual-craft criteria; they cannot mark a runtime slice ready or complete. Evaluate them on every affected rendered layer and bind the resulting evidence through the complete design/test/producer/client set and design verdict in `delivery-contract.md`.
|
|
106
|
+
|
|
107
|
+
Before calling client visual work polished, ask:
|
|
106
108
|
|
|
107
109
|
- Does the screen have a clear product-level visual point of view?
|
|
108
110
|
- Does it avoid generic AI frontend patterns?
|
|
109
111
|
- Does the visual direction support the target product loop instead of distracting from it?
|
|
110
112
|
- Are typography, color, spacing, radius, motion, and background choices tied to existing tokens or an explicit product reason?
|
|
111
|
-
- Does the
|
|
113
|
+
- Does the surface remain readable, accessible, and performant across the supported sizes, host modes, input/capability modes, and adaptation matrix of every affected client—not only Mobile and desktop Web?
|
package/dist/assets/marketplace/plugins/ccl-skills/skills/python-service-architecture/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: python-service-architecture
|
|
3
|
-
description: Python 后端架构 / FastAPI 项目结构 / Celery worker 拆分 / Python 微服务边界 / 服务分层重构 / Python 服务重构 → design or review Python backend, service, worker, package, API contract, data ownership, reliability, async/job, and runtime boundaries. Prefer this for architecture/boundary decisions; use python-service-dev for implementation work and localized refactor (某文件/某类); multi-stage / cross-module refactor delivery re-enters product-rd-workflow.
|
|
3
|
+
description: Python 后端架构 / FastAPI 项目结构 / Celery worker 拆分 / Python 微服务边界 / 多租户隔离怎么设计 / 消费 Kafka·消息队列与事件驱动架构 / 数据平台(分库分表·读写分离·备份恢复) / 服务分层重构 / Python 服务重构 → design or review Python backend, service, worker, package, API contract, data ownership, reliability, async/job, and runtime boundaries. Prefer this for architecture/boundary decisions; use python-service-dev for implementation work and localized refactor (某文件/某类); multi-stage / cross-module refactor delivery re-enters product-rd-workflow.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Python Service Architecture
|
|
@@ -145,6 +145,10 @@ Before changing architecture guidance, contracts, service boundaries, diagrams,
|
|
|
145
145
|
- For SQLAlchemy/Django ORM, migrations, transactions, and data ownership, read `references/data-modeling-and-migrations.md`.
|
|
146
146
|
- For asyncio, blocking work, GIL, and concurrency design, read `references/async-execution-model.md`.
|
|
147
147
|
- For Celery/RQ/arq, scheduled jobs, worker leases, and batch/restart policy, read `references/background-jobs-and-scheduling.md`.
|
|
148
|
+
- For durable workflow/task state contracts — state enums, terminal states, transition ownership, duplicate handling, retry/recovery policy — read `references/workflow-state-architecture.md`.
|
|
149
|
+
- For audit logs, operation records, and change-tracking boundaries, read `references/audit-history-architecture.md`.
|
|
150
|
+
- For notification delivery, operator alerting, outbound webhooks, and realtime channel boundaries, read `references/notification-architecture.md`.
|
|
151
|
+
- For replay, shadow-traffic, and response-comparison system design, read `references/replay-comparison-architecture.md`.
|
|
148
152
|
- For event-driven architecture concerns — delivery semantics taxonomy, producer-side patterns, transactional outbox/inbox, idempotency design, partition-key ordering, schema evolution, retry/DLQ/replay strategy, fanout patterns, saga vs choreography, end-to-end "exactly-once" illusion, and Python-specific implementation glue (asyncio consumer + bounded queue, SQLAlchemy outbox poller with `SKIP LOCKED`, exception hierarchy, sync-vs-async consumer choice, graceful-shutdown order) — read `references/event-driven-architecture.md`. Stack-agnostic core sections mirror the sibling `go-microservice-architecture/references/event-driven-architecture.md`; maintainers updating those sections must update both files in the same change.
|
|
149
153
|
- For multi-tenant SaaS isolation concerns — isolation-tier decision tree (RLS / schema-per-tenant / DB-per-tenant / region-per-tenant), tenant context as a first-class value, tenant-aware data access with DB-engine enforcement, per-tenant quota and rate limit at every layer, tenant-aware observability with cardinality management, per-tenant lifecycle (provision / suspend / export / delete / retention / archive), per-tenant rollout and feature flags, cross-tenant capability gating, compliance / residency / sovereignty, migration between tiers, and Python-specific implementation glue (tenant on `contextvars.ContextVar`, FastAPI dependency + Starlette middleware, SQLAlchemy RLS session variables, connection pool reset via pool reset event, cache key helper, async task spawning with `copy_context`, async message consumer pattern, outbox tenant propagation) — read `references/multi-tenant-isolation.md`. Stack-agnostic core sections mirror the sibling `go-microservice-architecture/references/multi-tenant-isolation.md`; maintainers updating those sections must update both files in the same change.
|
|
150
154
|
- For data-platform architecture concerns — DB engine choice axis (single-instance OLTP / sharding middleware like Vitess / distributed SQL like TiDB / managed cloud DB), HA topology and failover model, read scaling and replica routing with staleness budget, sharding and resharding strategy, cross-region replication and data residency, backup with tested recovery (RPO/RTO + restore drill), cluster lifecycle (provision / scale / decommission), capacity planning (storage / IOPS / connections / latency / replica lag), fleet-wide schema-migration coordination, connection-pool and proxy topology (PgBouncer / ProxySQL / Vitess gateway), cost and efficiency, and Python-specific implementation glue (sync-vs-async driver choice, SQLAlchemy 2.x async with asyncpg/asyncmy, Alembic migrations, PgBouncer prepared-statement caveat, health-check FastAPI dependency, connection-storm mitigation) — read `references/data-platform-architecture.md`. Stack-agnostic core sections mirror the sibling `go-microservice-architecture/references/data-platform-architecture.md`; maintainers updating those sections must update both files in the same change.
|
|
@@ -26,7 +26,7 @@ Use this as the first reference for Python backend, Python microservice, AI-serv
|
|
|
26
26
|
|
|
27
27
|
## Layering Depth (Apply In Moderation)
|
|
28
28
|
|
|
29
|
-
- The transport / application / domain / infrastructure split above is the **Ports-and-Adapters / Hexagonal** idea in moderation — `infrastructure` modules are the adapters, the application-service layer hides them behind plain function / class boundaries. Useful when: an external dependency has multiple real implementations (S3 + GCS, OpenAI + local-inference + vendor-N), a domain rule is stable enough that the test fake is reusable across years, or a regulated boundary requires a single audit point. Counter-indicated when: there is one real implementation that will not change, the "adapter" is a thin pass-through with no behavior, or the layering would force every DTO through 3 mappings. Do not introduce an interface per class out of habit — the Java-style "every service has an interface, DTO mirrors entity, entity mirrors row" pattern is over-engineering in Python and produces churn without testability or substitution gains.
|
|
29
|
+
- The transport / application / domain / infrastructure split above is the **[Ports-and-Adapters / Hexagonal](https://alistair.cockburn.us/hexagonal-architecture/)** idea (Alistair Cockburn; borrowed scope: the ports/adapters placement idea only, not the full pattern vocabulary) in moderation — `infrastructure` modules are the adapters, the application-service layer hides them behind plain function / class boundaries. Useful when: an external dependency has multiple real implementations (S3 + GCS, OpenAI + local-inference + vendor-N), a domain rule is stable enough that the test fake is reusable across years, or a regulated boundary requires a single audit point. Counter-indicated when: there is one real implementation that will not change, the "adapter" is a thin pass-through with no behavior, or the layering would force every DTO through 3 mappings. Do not introduce an interface per class out of habit — the Java-style "every service has an interface, DTO mirrors entity, entity mirrors row" pattern is over-engineering in Python and produces churn without testability or substitution gains.
|
|
30
30
|
- **Functional core, imperative shell**: prefer pure functions for calculation / rule / state-transition logic; let FastAPI handlers, SQLAlchemy sessions, Celery tasks, and external clients be the I/O shell that calls them. The pure core is easy to unit-test without fixtures; the shell is small enough to integration-test directly. Conflating them — domain rules sprinkled inside ORM event listeners, or business calculation inside a Celery task — is the recurring source of "we cannot test this without a real database / queue / network." Distinct and **not** optional: domain invariants must not be hidden inside route/FastAPI handlers, repositories, external-client wrappers, SQLAlchemy/ORM event listeners or hooks, or Celery task/worker callbacks — anywhere outside the domain/service layer. That is the layering rule, independent of whether you adopt the pure-core style. Sibling: `go-microservice-architecture/references/architecture-playbook.md` ("Functional Core, Imperative Shell") carries the same principle for Go; keep the two in sync.
|
|
31
31
|
- **Shared foundation/utility packages need stricter tiering than a single service**, scaled to blast radius — a widely-imported cross-service `common` library earns it; a tiny single-consumer helper does not. Such a module inherits one import cycle or one heavyweight coupling into every consumer, and a leaf utility cannot be reused once it transitively drags in unrelated packages. Document an explicit linear package-tier order in the module itself (illustrative: `generated value-types -> constants -> generic pure utils -> logging/metrics -> framework/business adapters`); forbid earlier tiers importing later tiers (one-directional, acyclic). Default to sibling independence so each leaf stays independently importable; when one same-tier package genuinely needs another, extract the shared piece down a tier rather than copy-pasting or adding a micro-tier. Generated pure contracts may be imported anywhere; generated clients carry transport deps and belong in the adapter tier only; all generated code is regenerate-only. Enforce with `import-linter` `layers` + `independence` contracts (a `src/` layout aids packaging isolation but does not enforce tiering) rather than review memory; a new cross-tier or sibling import is an architecture-review item.
|
|
32
32
|
- **How a layer boundary is enforced is itself an architecture decision**, with a strength ladder: physical package boundary (a separate distribution package whose violation is an import or packaging error), `import-linter` contracts in CI (above), then review convention — in decreasing strength; prefer mechanisms where a violation is a CI error, not a review comment. For a core where a frozen contract must coexist with continuous evolution (a gateway data plane, a billing domain), prefer the physical boundary, and add a deterministic digest/conformance anchor as machine proof that evolution has not touched the frozen surface. Record why the chosen strength is enough (cost versus strength); a weaker tier is a documented tradeoff, not a default. Sibling: `go-microservice-architecture/references/architecture-playbook.md` ("Dependency Direction") carries the same rule for Go; keep the two in sync.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Audit And History Architecture
|
|
2
|
+
|
|
3
|
+
Use this when designing audit logs, operation records, resource history, or change tracking. Implementation mechanics live in `python-service-dev/references/audit-history-patterns.md`.
|
|
4
|
+
|
|
5
|
+
Sibling note: `go-microservice-architecture/references/audit-history-architecture.md` carries the Go rendering; adapted per stack, kept in sync by review (not under the parallel-stack parity gate).
|
|
6
|
+
|
|
7
|
+
## Audit Boundary
|
|
8
|
+
|
|
9
|
+
- Decide which operations require durable audit and which can use best-effort activity logs.
|
|
10
|
+
- Define actor identity, service identity, resource scope, resource id, operation name, result, timestamp, trace id, and canonical error fields.
|
|
11
|
+
- Keep end user, client application, service caller, and operator identity separate.
|
|
12
|
+
- Use stable operation names rather than function names that change during refactors.
|
|
13
|
+
- Define retention, privacy, redaction, query authorization, export, and deletion policy up front (multi-tenant deletion modes: `multi-tenant-isolation.md`).
|
|
14
|
+
|
|
15
|
+
## Data Shape
|
|
16
|
+
|
|
17
|
+
- Store selective before/after summaries or field-level diffs when needed; never raw secrets, tokens, passwords, signatures, or unrelated payloads.
|
|
18
|
+
- Serialize request parameters and result data through redaction helpers.
|
|
19
|
+
- Prefer append-only audit records; corrections are new records unless a legal deletion policy requires removal.
|
|
20
|
+
- For event-derived history, define lag, rebuild, reconciliation, and source event retention.
|
|
21
|
+
|
|
22
|
+
## Write Path
|
|
23
|
+
|
|
24
|
+
- Critical audit belongs in the same transaction or outbox as the state change when correctness depends on it.
|
|
25
|
+
- Best-effort audit is acceptable only when explicitly non-critical and observable through logs or metrics on write failure.
|
|
26
|
+
- Async audit pipelines need bounded queues, retry policy, backpressure or an explicit drop policy, and shutdown flush; the full detached side-path posture is canonical in `python-service-dev/references/async-and-worker-patterns.md`.
|
|
27
|
+
|
|
28
|
+
## Query Path
|
|
29
|
+
|
|
30
|
+
- Query APIs need resource-scope authorization, pagination, time-range filters, and redaction on read.
|
|
31
|
+
- High-volume audit stores need partitioning or retention windows before launch.
|
|
@@ -4,7 +4,7 @@ Use when designing the data-platform substrate of a service or service-fleet: DB
|
|
|
4
4
|
|
|
5
5
|
This complements `data-modeling-and-migrations.md` (which owns schema, index, transaction, outbox, and per-service migration concerns): this file owns the **substrate** that schema and queries sit on. Load both when designing a new data-bound service or auditing an existing one.
|
|
6
6
|
|
|
7
|
-
> **Sibling sync.** A parallel `go-microservice-architecture/references/data-platform-architecture.md` mirrors **all non-stack-specific sections** of this file. Only the *Python-specific implementation patterns* section diverges by stack. The mirrored sections stay free of three categories of stack-specific token: DB-engine-specific syntax, runtime/concurrency-mechanic names, and library/framework API names. The concrete token list and grep command live in the *Mirrored-section grep gate* subsection at the end of this file's stack-glue.
|
|
7
|
+
> **Sibling sync.** A parallel `go-microservice-architecture/references/data-platform-architecture.md` mirrors **all non-stack-specific sections** of this file. Only the *Python-specific implementation patterns* section diverges by stack. The mirrored sections stay free of three categories of stack-specific token: DB-engine-specific syntax, runtime/concurrency-mechanic names, and library/framework API names. The concrete token list and grep command live in the *Mirrored-section grep gate* subsection at the end of this file's stack-glue. Tree-specific routing references are written inline for both trees so the mirrored bytes stay identical; cross-file parity is machine-checked by `skill-extraction-workflow/scripts/check-parallel-stack-parity.sh` (wired into `check-ccl-skills.sh`), which diffs the mirrored regions byte-for-byte (no normalization) and blocks on any divergence.
|
|
8
8
|
|
|
9
9
|
> **Sanitization boundary.** Vendor names (PostgreSQL, MySQL, Vitess, TiDB, CockroachDB, Aurora, Cloud Spanner, AlloyDB, Cloud SQL, DynamoDB, RDS Proxy, PgBouncer, ProxySQL, S3, Glacier, gp3, io2, etc.) below are illustrative; concrete topology choices, region names, cluster identifiers, and capacity numbers live only in the maintainer's private alias map. The sanitization audience list is positive (external / client / regulator / SOC / procurement / internal-compliance / sales-engineering / partner draft / forwardable-internal); sanitize before any document leaves the implementation team's approved audience.
|
|
10
10
|
>
|
|
@@ -6,7 +6,7 @@ This complements `async-execution-model.md` (concurrency model choice), `backgro
|
|
|
6
6
|
|
|
7
7
|
> **Conforms to the parallel-stack references pattern.** This file follows the layout documented in `skill-extraction-workflow/references/parallel-stack-references-pattern.md`: mirrored stack-agnostic core (when-applies through operations checklist), stack-specific implementation patterns section, and the embedded `### Mirrored-section grep gate` at the end of the stack-glue. The sibling `go-microservice-architecture/references/event-driven-architecture.md` mirrors the same structure. Either this file or `multi-tenant-isolation.md` may be used as a template for new parallel-stack extractions; multi-tenant additionally demonstrates the `## Topic-extension backlog` H2 for topic-wider-than-loop cases.
|
|
8
8
|
|
|
9
|
-
> **Sibling sync.** A parallel `go-microservice-architecture/references/event-driven-architecture.md` mirrors **all non-stack-specific sections** of this file (when-applies/not-applies, delivery semantics, event vs command vs query, idempotency, outbox, ordering, schema evolution, retry/DLQ/replay, backpressure, fanout, saga, end-to-end exactly-once, anti-patterns, operations checklist). Only the *Python-specific implementation patterns* section diverges by stack. Maintainers updating any mirrored section here must update the sibling in the same change to prevent drift.
|
|
9
|
+
> **Sibling sync.** A parallel `go-microservice-architecture/references/event-driven-architecture.md` mirrors **all non-stack-specific sections** of this file (when-applies/not-applies, delivery semantics, event vs command vs query, idempotency, outbox, ordering, schema evolution, retry/DLQ/replay, backpressure, fanout, saga, end-to-end exactly-once, anti-patterns, operations checklist). Only the *Python-specific implementation patterns* section diverges by stack. Maintainers updating any mirrored section here must update the sibling in the same change to prevent drift. Tree-specific routing references are written inline for both trees so the mirrored bytes stay identical; cross-file parity is machine-checked by `skill-extraction-workflow/scripts/check-parallel-stack-parity.sh` (wired into `check-ccl-skills.sh`), which diffs the mirrored regions byte-for-byte (no normalization) and blocks on any divergence.
|
|
10
10
|
|
|
11
11
|
> **Sanitization boundary.** The named brokers (Kafka, Pulsar, RabbitMQ, NATS JetStream, Redis Streams) and libraries below are concrete examples for **internal** implementation guidance, scoped to the implementation team's approved audience. Before this file (or excerpts) is copied into any document leaving that audience — external / client-facing materials, customer-specific deliverables, regulator or auditor evidence packages, SOC / compliance reports, procurement responses, or partner architecture appendices — replace the named choices with generic categories (`the broker`, `a partitioned log`, `a confirm-mode AMQP queue`) unless the vendor selection is already approved for disclosure to that specific audience.
|
|
12
12
|
|
|
@@ -20,8 +20,8 @@ Apply when the service:
|
|
|
20
20
|
|
|
21
21
|
Skip when the service:
|
|
22
22
|
- only does in-process pub-sub or fire-and-forget logging,
|
|
23
|
-
- uses synchronous HTTP/RPC with no durable async boundary (use `api-contract-and-schema.md` instead),
|
|
24
|
-
- uses a job queue purely for in-tenant background work where loss is acceptable (use `background-jobs-and-scheduling.md` or `batch-and-pipeline-architecture.md`).
|
|
23
|
+
- uses synchronous HTTP/RPC with no durable async boundary (use `api-contract-and-schema.md` on the Python tree / `protobuf-contract-architecture.md` on the Go tree instead),
|
|
24
|
+
- uses a job queue purely for in-tenant background work where loss is acceptable (use `background-jobs-and-scheduling.md` or `batch-and-pipeline-architecture.md` on the Python tree / `notification-architecture.md` or `bulk-workflow-architecture.md` on the Go tree).
|
|
25
25
|
|
|
26
26
|
## Delivery semantics taxonomy
|
|
27
27
|
|
|
@@ -139,7 +139,7 @@ Consumer lag, broker queue depth, and producer rate are the three backpressure s
|
|
|
139
139
|
- *Ordered partitioned log* (Kafka, Pulsar key-shared, NATS JetStream ordered consumer): **one serial work lane per assigned partition/key**. Feed each partition into its own bounded work-queue + single worker, or process messages serially within the partition's poll loop. Feeding multiple ordered partitions into a single shared work-queue + worker pool loses per-partition ordering, and a slow message on one partition can starve cold partitions or let later offsets overtake earlier ones. The bounded-queue-plus-pool shape is correct for unordered work queues; not for ordered partitioned logs.
|
|
140
140
|
- *Rebalance handling for partition-assigned consumers* — when a Kafka / Pulsar key-shared / similar consumer group rebalances and a partition is revoked, "one lane per partition" is unsafe without explicit rebalance discipline. The revoked owner must (1) stop fetching from the partition immediately, (2) drain or cancel its in-flight lane (await handler completion to a bounded deadline, or cancel with an explicit `partial-failure` disposition), (3) commit or abort offsets according to the handler outcome (commit only completed offsets; do not commit `last poll` blindly), and (4) be fenced so it cannot still publish a side effect after the new owner has started — typically by tagging each in-flight message with the assignment epoch and refusing side effects whose epoch is stale. Without fencing, the new owner and the old owner can process the same business key concurrently; per-partition ordering at steady state is meaningless if the rebalance window allows concurrent processing.
|
|
141
141
|
- Avoid unbounded worker-per-message fanout in all cases (the stack-glue section names the specific anti-pattern API).
|
|
142
|
-
- **Producer-side** — when the broker buffer fills (Kafka producer queue, RabbitMQ unconfirmed-publishes limit, NATS slow-consumer warning), block the producer's caller with a bounded wait or shed load at the producer entry point. Never block forever; surface a typed
|
|
142
|
+
- **Producer-side** — when the broker buffer fills (Kafka producer queue, RabbitMQ unconfirmed-publishes limit, NATS slow-consumer warning), block the producer's caller with a bounded wait or shed load at the producer entry point. Never block forever; surface a typed error after a bounded wait so upstream can backpressure further.
|
|
143
143
|
- **Cross-service** — a slow consumer is an upstream producer's problem to know about. Consumer lag must be exposed as a metric and alerted; producers cannot fix what they cannot see.
|
|
144
144
|
|
|
145
145
|
## Fanout patterns
|
|
@@ -187,6 +187,8 @@ Stack-agnostic recipe; document each clause for every event-driven boundary that
|
|
|
187
187
|
|
|
188
188
|
If any clause is missing, the boundary is at-least-once with duplicates. Tell consumers honestly.
|
|
189
189
|
|
|
190
|
+
External grounding (adopted in part): this recipe is an instance of the end-to-end argument — [Saltzer, Reed & Clark, *End-to-End Arguments in System Design*, ACM TOCS 2(4), 1984](https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf) — a function that "can completely and correctly be implemented only with the knowledge and help of the application standing at the endpoints of the communication system" cannot be delegated to the communication layer, and broker-level transactional features are that paper's "incomplete version … useful as a performance enhancement", never the end-to-end guarantee. Borrowed scope: the placement argument only; the five-clause recipe and the atomic-domain boundary are this skill's own operational criteria.
|
|
191
|
+
|
|
190
192
|
## Anti-patterns
|
|
191
193
|
|
|
192
194
|
- **Post-commit publish (durable cross-process)** — publishing the event after the DB transaction commits, without an outbox, when consumers are in another process. A crash between commit and publish silently drops the event. In-process, same-instance, rebuildable post-commit hooks are not this anti-pattern.
|
|
@@ -226,6 +228,7 @@ These are stack-localized recipes that implement the stack-agnostic patterns abo
|
|
|
226
228
|
- *Ordered partitioned log*: one `asyncio.Task` per assigned partition that processes serially, or a per-partition bounded `asyncio.Queue` with a single worker task. A shared `asyncio.Event` triggers shutdown; never feed multiple ordered partitions into a shared worker pool.
|
|
227
229
|
- Avoid `asyncio.create_task(handle(msg))` inside a `for msg in consumer` loop — unbounded fanout will OOM under load and discards ordering.
|
|
228
230
|
- **Sync vs async consumers** — if the handler is CPU-bound or calls a sync DB driver (psycopg2, sync SQLAlchemy), use a thread pool or process pool; do not block the event loop. For mixed workloads, route async-friendly handlers to the asyncio worker pool and CPU/sync handlers to a `ProcessPoolExecutor` or to Celery/RQ.
|
|
231
|
+
- **Event payload freezing** — the typed-model layer for Python event payloads is the service's Pydantic/typed model: freezing means serializing to immutable bytes at the enqueue/publish boundary (`model_dump_json()` or an explicit serializer writing into the outbox row) and binding those bytes to the envelope (`event_type`, `event_version`) — never enqueue a mutable model instance that later code can mutate before the poller publishes; the schema version alone does not freeze the instance.
|
|
229
232
|
- **Context propagation** — extract correlation id, trace context, lane/env from message headers into `contextvars` at the consumer boundary. OpenTelemetry's `aiokafka`/`pika` instrumentations restore the trace context automatically; verify they are wired in `observability-and-ops.md`'s OTel/startup section, not in `async-execution-model.md`.
|
|
230
233
|
- **Outbox poller with SQLAlchemy** — an asyncio task that runs a **claim → commit → publish → mark-sent** loop, not "publish inside the DB tx" (a broker call inside a SQLAlchemy session held open for the broker round-trip violates the short-transaction rule in `data-modeling-and-migrations.md`):
|
|
231
234
|
1. **Claim tx (short)**: `BEGIN; SELECT … FROM outbox WHERE sent_at IS NULL AND (processing_until IS NULL OR processing_until < NOW()) ORDER BY id LIMIT N FOR UPDATE SKIP LOCKED; UPDATE outbox SET processing_until = NOW() + lease, owner = :owner WHERE id IN (…); COMMIT;` — the row is now leased to this poller; session closes immediately.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Use when designing the tenant-isolation architecture of a SaaS service: how tenants are kept apart at the data, compute, network, identity, observability, lifecycle, and compliance layers; how queries, jobs, caches, and external calls carry tenant context safely; how a tenant's data can be exported or deleted on demand; and how shared services keep cross-tenant aggregation auditable.
|
|
4
4
|
|
|
5
|
-
> **Sibling sync.** A parallel `go-microservice-architecture/references/multi-tenant-isolation.md` mirrors **all non-stack-specific sections** of this file. Only the *Python-specific implementation patterns* section diverges by stack. Maintainers updating any mirrored section here must update the sibling in the same change. The mirrored sections stay free of three categories of stack-specific token: DB-engine-specific syntax, runtime/concurrency-mechanic names, and library / framework API names. The concrete token list and grep command live in the *Mirrored-section grep gate* subsection at the end of this file's stack-glue section. Before commit, run that grep against the file's mirrored sections; zero hits required. The same gate lives in the Go sibling. Routing
|
|
5
|
+
> **Sibling sync.** A parallel `go-microservice-architecture/references/multi-tenant-isolation.md` mirrors **all non-stack-specific sections** of this file. Only the *Python-specific implementation patterns* section diverges by stack. Maintainers updating any mirrored section here must update the sibling in the same change. The mirrored sections stay free of three categories of stack-specific token: DB-engine-specific syntax, runtime/concurrency-mechanic names, and library / framework API names. The concrete token list and grep command live in the *Mirrored-section grep gate* subsection at the end of this file's stack-glue section. Before commit, run that grep against the file's mirrored sections; zero hits required. The same gate lives in the Go sibling. Routing text that differs per tree is written inline for both trees (`x.md` on the Python tree / `y.md` on the Go tree), so the mirrored bytes stay identical. Cross-file parity is machine-checked by `skill-extraction-workflow/scripts/check-parallel-stack-parity.sh` (wired into `check-ccl-skills.sh`): it diffs the mirrored regions as a byte-identical region (no normalization; tree-specific routing references are written inline for both trees), and blocks on any divergence.
|
|
6
6
|
|
|
7
7
|
> **Sanitization boundary.** Tenant identifiers, customer names, lane / region names, regulator labels, and quota numbers below are illustrative; concrete values live only in the private alias map. The list of audiences that require sanitization is **positive** (these audiences require it unless explicitly approved otherwise): external / client-facing materials, customer-specific deliverables, regulator or auditor evidence, SOC / compliance reports, procurement responses, internal compliance reviews, sales-engineering or security-questionnaire appendices, partner architecture drafts, and any document that could be forwarded to any of those. "Internal" by itself is not safety; internal documents are routinely forwarded.
|
|
8
8
|
|
|
@@ -16,7 +16,7 @@ Apply when:
|
|
|
16
16
|
|
|
17
17
|
Skip when:
|
|
18
18
|
- the service is single-tenant by deployment (per-customer dedicated stack with no shared layer); route to `platform-release-engineering/SKILL.md` and to this file's *Compliance, residency, sovereignty* section for residency commitments,
|
|
19
|
-
- the service is internal-only with a single owning team (employees of one org are not "tenants" for this purpose); identity/permission boundaries still apply but route to `web-framework-boundaries.md` or `api-contract-and-schema.md
|
|
19
|
+
- the service is internal-only with a single owning team (employees of one org are not "tenants" for this purpose); identity/permission boundaries still apply but route to `web-framework-boundaries.md` or `api-contract-and-schema.md` on the Python tree / `api-security-boundaries.md` on the Go tree,
|
|
20
20
|
- tenant-equivalent isolation is owned entirely by a platform layer above the service (e.g., per-tenant namespace owned by the platform); route to `platform-service-connectivity/SKILL.md` for the platform contract.
|
|
21
21
|
|
|
22
22
|
## Tenant isolation tiers (decision tree)
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Notification Architecture
|
|
2
|
+
|
|
3
|
+
Use this when designing notification delivery, operator alerting, outbound webhooks, or realtime client channels. Implementation mechanics live in `python-service-dev/references/notification-patterns.md`.
|
|
4
|
+
|
|
5
|
+
Sibling note: `go-microservice-architecture/references/notification-architecture.md` carries the Go rendering; adapted per stack, kept in sync by review (not under the parallel-stack parity gate).
|
|
6
|
+
|
|
7
|
+
## Boundary
|
|
8
|
+
|
|
9
|
+
- Classify notifications as user-facing, operator-facing, integration callbacks, or internal alerts.
|
|
10
|
+
- Define whether each notification is critical, retryable, idempotent, and auditable.
|
|
11
|
+
- Keep notification templates and delivery endpoints in config or a template store, not inline code.
|
|
12
|
+
- Recipient and endpoint selection is scoped by resource ownership and environment.
|
|
13
|
+
- Notification payloads use safe summaries, not raw request bodies or secrets.
|
|
14
|
+
|
|
15
|
+
## Delivery Policy
|
|
16
|
+
|
|
17
|
+
- Critical notifications need a durable outbox, retry, idempotency key, and terminal delivery state.
|
|
18
|
+
- Best-effort alerts may use async delivery, but failures must be observable.
|
|
19
|
+
- Delivery clients need timeout, status-code validation, response body size limit, and rate limit.
|
|
20
|
+
- Configurable delivery endpoints are an SSRF trust boundary: architecture names the scheme/port policy, private/link-local/metadata blocking, redirect policy, and whether deliveries route through a constrained egress proxy.
|
|
21
|
+
- Retries use bounded backoff and stop on permanent errors.
|
|
22
|
+
- Duplicate delivery must be acceptable to receivers or prevented with stable dedupe keys.
|
|
23
|
+
|
|
24
|
+
## Observability
|
|
25
|
+
|
|
26
|
+
- Track sent, failed, retried, dropped, and suppressed counts by notification type.
|
|
27
|
+
- Include trace/log id and config/template version in delivery logs.
|
|
28
|
+
- Alert storms need grouping, throttling, and suppression policy.
|
|
@@ -16,7 +16,7 @@ Use this for pyproject, uv/poetry/pip, lockfiles, tooling, containers, and deplo
|
|
|
16
16
|
- Define entrypoint: `uvicorn`/`gunicorn`, framework command, worker command, scheduler command, or CLI.
|
|
17
17
|
- Define process model: worker count, async event loop, thread/process workers, memory limits, graceful shutdown, and health probes.
|
|
18
18
|
- Release readiness includes migrations, startup validation, smoke tests, canary, rollback, and observability checks.
|
|
19
|
-
- **ASGI server choice has expanded beyond `uvicorn` / `gunicorn+uvicorn` / `hypercorn`** — `granian` (emmett-framework, Rust-based) is the current credible high-throughput alternative for ASGI services, supporting ASGI/3, RSGI, WSGI, HTTP/1, HTTP/2, TLS, WebSockets (HTTP/3 planned per the project README's "eventually 3" roadmap note — verified not shipped as of
|
|
19
|
+
- **ASGI server choice has expanded beyond `uvicorn` / `gunicorn+uvicorn` / `hypercorn`** — `granian` (emmett-framework, Rust-based) is the current credible high-throughput alternative for ASGI services, supporting ASGI/3, RSGI, WSGI, HTTP/1, HTTP/2, TLS, WebSockets (HTTP/3 planned per the project README's "eventually 3" roadmap note — re-verified not shipped as of Aug 2026, granian 2.8.x). Per the Granian project's own `benchmarks/vs.md` and third-party load-test repos (e.g., `piccolo-orm/asgi_server_performance`, `synodriver/asgi-server-benchmark`), ASGI echo on 10KB payload typically lands granian > uvicorn-httptools > hypercorn by roughly the ratios 58k / 51k / 8k RPS in April-2026-era runs; file-serving gap is wider (granian ~47k vs uvicorn ~18k via `pathsend`). Treat the absolute numbers as benchmark-snapshot-specific; re-run against your workload before basing a switch on them. Architecture impact: when serving throughput is the binding constraint, granian can buy headroom without rewriting the **plain ASGI path**. **"Without rewriting" caveats**: granian's worker / process model differs from `gunicorn+uvicorn` fork-based workers (Rust runtime + Python interpreters with different lifecycle hooks); ASGI lifespan events, contextvars propagation across worker boundaries, custom signal handlers, prometheus/metrics exporters tied to uvicorn internals, and ASGI middleware that depends on uvicorn-specific behavior all need smoke-testing on granian before a switch. If the team plans to adopt granian's bespoke RSGI protocol for max performance (rather than ASGI), application code that uses ASGI-specific middleware, ASGI scope manipulation, or third-party ASGI libraries WILL need rewriting — RSGI is a different protocol, not a faster ASGI. Trade-offs: smaller operational maturity, fewer community recipes, Rust-runtime-on-the-side observability differs from a pure-Python server. **Choose uvicorn** for ecosystem maturity, broad reference material, and known operational patterns; **choose granian** when (a) profiled benchmarks on your workload show uvicorn saturation, (b) the team has Rust-toolchain debugging capacity, (c) the deployment story can absorb a less-common runtime. Hypercorn remains the choice when HTTP/2 + ASGI under pure-Python ops matters more than peak throughput.
|
|
20
20
|
- **Python runtime version baseline (2025-2026)**: Python 3.13 (released October 2024) ships **experimental** free-threaded build per PEP 703 — GIL-disabled, ~40% single-threaded perf hit per python.org "What's New in 3.13" notes, used at the team's risk for parallel-CPU workloads. Python 3.14 (released 7 October 2025) advances free-threading to **supported (Phase II of PEP 703)** per PEP 779 — meaning the free-threaded build is a first-class supported configuration, NOT that it is the default Python build or the default production choice. Per the python.org free-threading howto, the single-threaded penalty narrowed to ~5-10% (specializing adaptive interpreter re-enabled thread-safely); PEP 803 defines the `abi3t` stable ABI for free-threaded C extensions. Architecture impact: for services where parallel CPU work matters (in-process ML inference fan-out, heavy parsing, parallel compression), 3.14 free-threading is the first version where adoption is **a defensible experiment for a selected service**, not yet a defensible default. Pre-flight burn-in required before any production switch: (a) C-extension readiness — all extensions in the service's dependency tree must declare free-threading support; many popular extensions (numpy, pandas, lxml, psycopg native bits, asyncpg native bits, pillow, cryptography) were still mid-migration at 2026-Q1, verify per-version; mixing GIL-only and free-threading-aware extensions in one process is unsupported; (b) GC and runtime behavior under sustained threading load differs from GIL build — measure tail latency, memory residency, and CPU efficiency on the actual workload; (c) debugger / profiler ergonomics (gdb, pdb, py-spy, scalene, prometheus exporters) may have rough edges on free-threaded builds; (d) library-level thread-safety: code paths that were "implicitly safe because of the GIL" can race in free-threaded mode (singletons built at import time, module-level mutable caches, third-party libraries that rely on GIL-protected dict mutation). For pure I/O-bound services, stay on stock GIL build — the 5-10% overhead is pure cost. For services on 3.13 or older, treat free-threading as opt-in research, not default.
|
|
21
21
|
|
|
22
22
|
## Topic-extension backlog
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Replay Comparison Architecture
|
|
2
|
+
|
|
3
|
+
Use this when designing replay, shadow-traffic, response-comparison, or migration-verification systems. Implementation mechanics live in `python-service-dev/references/replay-comparison-patterns.md`.
|
|
4
|
+
|
|
5
|
+
Sibling note: `go-microservice-architecture/references/replay-comparison-architecture.md` carries the Go rendering; adapted per stack, kept in sync by review (not under the parallel-stack parity gate).
|
|
6
|
+
|
|
7
|
+
## Execution Model
|
|
8
|
+
|
|
9
|
+
- Separate capture, replay, comparison, storage, and reporting.
|
|
10
|
+
- Replays need durable job state, concurrency limit, timeout, delay policy, rate limit, target environment or lane, and cancellation policy.
|
|
11
|
+
- Captured input is redacted and bounded by size before storage.
|
|
12
|
+
- Replay requests preserve only approved headers and metadata; never replay credentials blindly.
|
|
13
|
+
- Shadow execution must not commit side effects unless the target is explicitly isolated (environment, lane, rollback, or dry-run).
|
|
14
|
+
|
|
15
|
+
## Comparison Policy
|
|
16
|
+
|
|
17
|
+
- Define comparator selection by method, content type, schema, or route.
|
|
18
|
+
- Generic JSON comparison supports ignored fields, custom field comparators, numeric tolerance, null handling, array handling, and type-mismatch reporting.
|
|
19
|
+
- Diff output includes field path, original value summary, replay value summary, diff type, score, and ignored status.
|
|
20
|
+
- Comparison thresholds are config-driven and versioned.
|
|
21
|
+
- Positive, negative, and neutral diffs are classified only when the product has a defensible definition.
|
|
22
|
+
|
|
23
|
+
## Retention And Reporting
|
|
24
|
+
|
|
25
|
+
- Store aggregate job counts and paginated diff details separately.
|
|
26
|
+
- Define retention for captured requests, replay responses, and diff artifacts.
|
|
27
|
+
- Expose summary metrics: total, processed, success, failed, diff count, similarity, p95 replay latency, and error categories.
|
|
28
|
+
- Treat replay as a confidence signal, not an automatic release approval, unless acceptance gates are explicit.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Workflow State Architecture
|
|
2
|
+
|
|
3
|
+
Use this when designing durable task, async workflow, import/export, backfill, or scheduled processing state. Implementation mechanics live in `python-service-dev/references/state-machine-task-patterns.md`.
|
|
4
|
+
|
|
5
|
+
Sibling note: `go-microservice-architecture/references/workflow-state-architecture.md` carries the Go rendering; adapted per stack, kept in sync by review (not under the parallel-stack parity gate).
|
|
6
|
+
|
|
7
|
+
## State Contract
|
|
8
|
+
|
|
9
|
+
- Define state enum, terminal states, allowed transitions, retry policy, and ownership before handlers are written.
|
|
10
|
+
- Keep allowed transitions in one table, diagram, or policy function so handlers do not invent their own rules.
|
|
11
|
+
- Separate state, progress, result pointer, error metadata, retry metadata, and audit metadata.
|
|
12
|
+
- Persist the task record before starting expensive work, background execution, or external artifact generation so process failure leaves an inspectable, repairable state.
|
|
13
|
+
- Define duplicate handling for every entrypoint: create, start, process, fail, succeed, cancel, and retry.
|
|
14
|
+
- Transitions that affect durable truth need compare-and-update, a transaction, or a lock.
|
|
15
|
+
- Completion events are published after the durable state update and are idempotent.
|
|
16
|
+
|
|
17
|
+
## Processing Semantics
|
|
18
|
+
|
|
19
|
+
- Create is idempotent when the caller supplies an idempotency key or natural unique key.
|
|
20
|
+
- Start claims ownership before expensive side effects; process re-checks state after ownership is acquired.
|
|
21
|
+
- Success persists result data before publishing completion.
|
|
22
|
+
- Failure records canonical error code, safe message, retryable flag, retry count, and trace/log id.
|
|
23
|
+
- Cancellation defines whether in-flight work is interrupted, allowed to finish, or marked for later stop.
|
|
24
|
+
|
|
25
|
+
## Retry And Recovery
|
|
26
|
+
|
|
27
|
+
- Retry threshold and backoff belong in policy/config, not inline literals.
|
|
28
|
+
- Terminal-state recovery is explicit; ordinary retries must not move successful or cancelled tasks.
|
|
29
|
+
- Delayed events and queue messages re-check current state before acting.
|
|
30
|
+
- Process restart leaves enough durable state to resume, retry, or safely skip.
|
|
31
|
+
- Background workers and scheduled cleaners need exception recovery and a bounded cursor/batch model; long-lived loops need a stoppable schedule or an explicit process-lifetime owner (`async-execution-model.md`, `background-jobs-and-scheduling.md`).
|
|
32
|
+
|
|
33
|
+
## Acceptance Checks
|
|
34
|
+
|
|
35
|
+
- Illegal transitions are rejected or no-op according to a documented policy.
|
|
36
|
+
- Duplicate messages, delayed messages, and concurrent processors produce one durable outcome.
|
|
37
|
+
- Every terminal state includes enough result or error context for support and reconciliation.
|
|
38
|
+
- Expensive background work cannot start without an inspectable durable task record, or carries an explicitly documented non-durable rationale.
|
|
39
|
+
- Worker, scheduler, and cleaner paths have exception recovery, bounded scan/batch behavior, and a visible recovery/repair result.
|
|
@@ -19,7 +19,10 @@ Use this for implementation of Python backend products, services, microservices,
|
|
|
19
19
|
- Use `go-microservice-dev` for Go services. Do not load Go implementation rules for Python work unless the task is explicitly cross-language contract or generated-client integration.
|
|
20
20
|
- Use codebase-specific skills only when the task is explicitly about an existing repository.
|
|
21
21
|
- For money, billing, quota, permission, tenant/user data isolation, high-impact AI, repeated writes, async finality, or incident-explanation risk, apply `product-rd-workflow` high-risk resilience gates and route test-layer design through `testing-strategy`.
|
|
22
|
-
- When a change
|
|
22
|
+
- When a change can alter what a client renders or which state, action, or decision path it offers—including strings/templates/config/flags and API/event/schema fields, enums, status/progress, permission/capability signals, defaults, or result shapes—load `../product-ui-ux-design/references/delivery-contract.md`, create the applicable full or lightweight record in that contract, and follow its canonical consumer-universe classification, design/test/client handoffs, and terminal-status rules.
|
|
23
|
+
- This Python owner returns only its `producer_record` delta: immutable binding, build/schema/config artifact identity, exact command/environment, and API/event/log/output observation.
|
|
24
|
+
|
|
25
|
+
- For a standalone Python CLI, this skill owns Python parser/library implementation mechanics. Any change to a user-facing command tree, subcommand, flag/default/action path, help/output/exit behavior, confirmation, progress, or recovery path also loads `terminal-cli-dev`, which owns the terminal contract and its UI/UX/testing handoff. Only internal parser refactors proven to preserve all user-visible semantics may skip that owner.
|
|
23
26
|
|
|
24
27
|
## Generalization Discipline
|
|
25
28
|
|
|
@@ -75,14 +78,10 @@ Repo-local agent contracts (`AGENTS.md` at the repo root and in source directori
|
|
|
75
78
|
|
|
76
79
|
5. Verify at the right scope.
|
|
77
80
|
- Run focused pytest tests for changed packages.
|
|
81
|
+
- When writing the test code itself (structure, naming, smells, fixtures, behavior-vs-state, coverage, isolation, parameterization), pick the matching § from the decision table in `testing-strategy/references/test-code-authoring-patterns.md`; enable the per-stack lint executors for its machine-decidable smells (conditional logic / sleep / assertion-free tests) per `testing-strategy/references/fitness-functions.md` §4.1.4 (e.g. Ruff `TID251` banning `time.sleep`).
|
|
78
82
|
- Run async tests with the repo's configured `pytest-asyncio` mode.
|
|
79
83
|
- Run integration tests only when required services and credentials are available.
|
|
80
|
-
- **TC traceability
|
|
81
|
-
- **废弃级联:业务代码是否仍在用** — grep 只找出"测试函数引用了什么 import"是第一步;判断"该 import 是否还有其他 caller"才能定生死。Python 顺序:
|
|
82
|
-
1. 看测试体导入的模块:`grep -E "^(from |import )" tests/test_<x>.py`
|
|
83
|
-
2. 对每个产品模块(非 stdlib / 非测试 helper),找全仓库 caller:`grep -rEn "from <pkg>\.<mod>|import <pkg>\.<mod>" --include='*.py' --exclude-dir=tests`
|
|
84
|
-
3. 零产品 caller → 同 commit 删该模块 + 测试;有产品 caller → 测试目标仍在用,不删测试(若 TC 已废弃但代码活,先确认产品决策)
|
|
85
|
-
4. 边界:动态 import(`importlib.import_module("...")`)grep 抓不到;含 reflection 的代码人工确认;DI/插件注册(`@register` 装饰器)的产品代码需查注册表而非 import
|
|
84
|
+
- **TC traceability and the deprecation cascade** are mandatory when the repo tracks test cases in Bitable: before adding a test, check existing TC coverage; before deleting one, run the caller-liveness sequence. Mechanics (marker registration, coverage grep, the four-step 废弃级联, dynamic-import boundaries) live in `references/testing-and-quality-patterns.md` (TC Traceability And Deprecation Cascade).
|
|
86
85
|
- Run ruff, mypy/pyright, formatting, and codegen/migration checks when the repo uses them.
|
|
87
86
|
- Keep fast tests deterministic; isolate live infrastructure, long sleeps, generated files, and external credentials behind markers.
|
|
88
87
|
|
|
@@ -135,6 +134,10 @@ Repo-local agent contracts (`AGENTS.md` at the repo root and in source directori
|
|
|
135
134
|
- For asyncio, blocking work isolation, concurrency limits, and cancellation, read `references/async-and-worker-patterns.md`.
|
|
136
135
|
- For Redis, cache, locks, idempotency, counters, and rate limits, read `references/redis-cache-lock-patterns.md`.
|
|
137
136
|
- For Celery/RQ/arq, queues, scheduled tasks, and job execution, read `references/background-job-patterns.md`.
|
|
137
|
+
- For durable task state machines, status transitions, leases, terminal states, and scheduled repair jobs, read `references/state-machine-task-patterns.md`.
|
|
138
|
+
- For audit logs, operation records, resource history, and change tracking, read `references/audit-history-patterns.md`.
|
|
139
|
+
- For notification delivery, operator alerts, outbound webhooks, and realtime client channels, read `references/notification-patterns.md`.
|
|
140
|
+
- For replay jobs, shadow execution, response comparison, and migration verification, read `references/replay-comparison-patterns.md`.
|
|
138
141
|
- For external HTTP clients, SDKs, generated clients, service discovery, and dependency adapters, read `references/dependency-client-patterns.md`.
|
|
139
142
|
- For errors, exception mapping, response envelopes, and validation errors, read `references/error-handling-patterns.md`.
|
|
140
143
|
- For logs, metrics, traces, health checks, and instrumentation, read `references/observability-implementation-patterns.md`.
|
|
@@ -11,6 +11,14 @@ Use this for Python implementation around LLM/RAG/inference calls after `llm-inf
|
|
|
11
11
|
- For CPU/GPU-heavy local inference, isolate concurrency and memory limits.
|
|
12
12
|
- Use fakes for provider tests and mark live provider tests explicitly.
|
|
13
13
|
|
|
14
|
+
## Streaming And Session Mechanics
|
|
15
|
+
|
|
16
|
+
- Reject a new turn while a prior turn for the same session/conversation is in-flight: check-and-claim the session at the boundary (lock or idempotency marker) and return a typed busy error; do not silently interleave two generations into one conversation state.
|
|
17
|
+
- Persist partial state on a timer during long streams (partial transcript, token counts) so a crash mid-stream can resume or at least account for cost; the cadence is a product decision, the mechanism belongs here.
|
|
18
|
+
- Close stream readers deterministically on every exit path — client disconnect, deadline, terminal error — via async context managers or cancellation handlers, or provider connections leak until pool exhaustion.
|
|
19
|
+
- Record token/cost usage after completion, or on terminal failure with the partial count, never only at request start; tie the usage record to the same request/session id the logs carry.
|
|
20
|
+
- Mark terminal vs recoverable stream states explicitly (completed / cancelled / provider-error / resumable); a consumer that cannot distinguish them retries unresumable streams.
|
|
21
|
+
|
|
14
22
|
## Do Not
|
|
15
23
|
|
|
16
24
|
- Encode prompt policy, retrieval strategy, evaluation rubric, or model routing here; route those decisions to `llm-inference-integration`.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Audit And History Patterns
|
|
2
|
+
|
|
3
|
+
Use this when implementing audit logs, operation records, resource history, or change tracking.
|
|
4
|
+
|
|
5
|
+
Sibling note: `go-microservice-dev/references/audit-history-patterns.md` carries the Go rendering; adapted per stack, kept in sync by review (not under the parallel-stack parity gate).
|
|
6
|
+
|
|
7
|
+
## Audit Record Shape
|
|
8
|
+
|
|
9
|
+
- Capture actor, service identity, resource scope, resource id, operation name, operation type, request id, trace/log id, timestamp, result, and canonical error code.
|
|
10
|
+
- Store safe before/after summaries or field-level diffs when needed.
|
|
11
|
+
- Never store raw secrets, tokens, passwords, signatures, or credentials.
|
|
12
|
+
- Serialize params and data through redaction helpers; do not dump whole request or response objects.
|
|
13
|
+
- Use stable operation names, not function names that change during refactors.
|
|
14
|
+
|
|
15
|
+
## Write Path
|
|
16
|
+
|
|
17
|
+
- Audit writes for critical operations belong in the same transaction or outbox as the state change when correctness depends on them (`sqlalchemy-and-migrations-patterns.md` Outbox section).
|
|
18
|
+
- Best-effort audit is acceptable only when explicitly non-critical; catch and log write failures with a failure metric — audit write errors must be observable even when they do not fail the main request.
|
|
19
|
+
- Async audit writers follow the detached side-path posture in `async-and-worker-patterns.md` (whitelisted correlation only, accept boundary at durable commit, per-attempt reservation for non-rollbackable external effects, bounded queue, serialized accept-vs-drain shutdown, loss policy matched to data class). For billing/ledger/usage records with no tested reconstruction source, use a same-transaction outbox or backpressure instead of drop — that posture is canonical there; do not restate it here.
|
|
20
|
+
|
|
21
|
+
## Query Path
|
|
22
|
+
|
|
23
|
+
- Audit query APIs need resource-scope authorization, pagination, time-range filters, and redaction on read.
|
|
24
|
+
- Default ordering is deterministic — usually newest first with a stable tie-breaker.
|
|
25
|
+
- Large audit tables need partitioning, retention, or archive strategy before high-volume launch.
|
|
26
|
+
|
|
27
|
+
## Tests
|
|
28
|
+
|
|
29
|
+
- Test redaction, critical-write rollback behavior, best-effort write failure visibility, exception recovery, pagination, scope filtering, and time-range boundaries.
|
|
@@ -11,6 +11,22 @@ Use this for Celery, RQ, arq, APScheduler, queue consumers, and scheduled tasks.
|
|
|
11
11
|
- Store status for user-visible jobs.
|
|
12
12
|
- Protect singleton jobs with locks or scheduler guarantees.
|
|
13
13
|
|
|
14
|
+
## Tool-Specific Caveats
|
|
15
|
+
|
|
16
|
+
Defaults shift across major versions; verify against the installed tool version's docs before relying on any default named here.
|
|
17
|
+
|
|
18
|
+
- Celery ack semantics: the default early ack loses a task on worker crash; `acks_late=True` moves the ack to after completion, so a crash mid-task causes redelivery — pair it with idempotent task bodies, and decide `task_reject_on_worker_lost` deliberately rather than by default.
|
|
19
|
+
- Celery visibility timeout (Redis/SQS-style brokers): must exceed the longest task runtime plus retry backoff, or the broker redelivers a still-running task and it executes concurrently with itself.
|
|
20
|
+
- Celery prefetch and worker lifecycle: set `worker_prefetch_multiplier=1` for long tasks (default prefetch head-of-line blocks the queue behind one slow task); use `worker_max_tasks_per_child` to recycle leaky workers.
|
|
21
|
+
- Celery beat is a single point of scheduling: run exactly one beat instance, or guard schedule dispatch with a distributed lock; two beats double-fire every schedule.
|
|
22
|
+
- RQ: a queued job silently expires when its `ttl` passes before a worker picks it up, and `job_timeout` kills execution past the budget; failed jobs land in the failed registry and requeueing is an explicit operation, not automatic.
|
|
23
|
+
- arq: async-native — `max_tries` bounds retries, `job_timeout` bounds execution, `defer_by`/`defer_until` schedule, and results live in Redis only for the `keep_result` TTL; treat result reads after that window as misses, not errors.
|
|
24
|
+
|
|
25
|
+
## Transactional Enqueue Boundary
|
|
26
|
+
|
|
27
|
+
- Enqueueing from inside an open DB transaction is a dual write: the broker publish does not roll back with the transaction. Either enqueue through an outbox row committed with the business write (see `python-service-architecture/references/event-driven-architecture.md` and the outbox section of `sqlalchemy-and-migrations-patterns.md`), or enqueue after commit and accept the crash window between commit and enqueue with a documented reconciliation path.
|
|
28
|
+
- The inverse ordering — enqueue first, then commit — hands the worker a job for state that may never commit; workers must re-read durable state, not trust the enqueue payload as proof the write happened.
|
|
29
|
+
|
|
14
30
|
## Request Boundary
|
|
15
31
|
|
|
16
32
|
- Do not hide long work behind a synchronous request unless the timeout budget proves it is safe.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Batch And Artifact Patterns
|
|
2
2
|
|
|
3
|
-
Use this for import/export scripts, backfills, reports, generated files, CSV/XLSX/PDF artifacts, and repair tools.
|
|
3
|
+
Use this for import/export scripts, backfills, reports, generated files, CSV/XLSX/PDF artifacts, and repair tools. For durable job status transitions, duplicate delivery, retry, and cancellation, `state-machine-task-patterns.md` is the canonical guide; this file covers row/file handling, execution, artifacts, and reports.
|
|
4
4
|
|
|
5
5
|
## Implementation
|
|
6
6
|
|
|
@@ -11,3 +11,27 @@ Use this for import/export scripts, backfills, reports, generated files, CSV/XLS
|
|
|
11
11
|
- For object or artifact migration, record success, error, skipped, and conflict rows in replayable output so reruns can resume or audit decisions without re-discovering every item.
|
|
12
12
|
- Make output artifact paths, object storage keys, retention, and download permissions explicit.
|
|
13
13
|
- Test parsing, validation, edge rows, and retry/resume behavior.
|
|
14
|
+
|
|
15
|
+
## Import Pipeline
|
|
16
|
+
|
|
17
|
+
- Represent each input row as a typed row object with row number, source name (sheet/tab/file part), raw values, normalized values, and an error list.
|
|
18
|
+
- Validate file type, size, sheet/partition count, header shape, start row, and column-count bounds before row parsing; normalize cells at the boundary (trim, pad missing optional columns, parse accepted list separators, reject unsupported encodings).
|
|
19
|
+
- Static row validation accumulates row errors; ordinary row errors do not fail the whole file. Cross-row validation detects duplicates and conflicts, then annotates every affected row.
|
|
20
|
+
- Enrichment resolves external names/codes to IDs through typed lookup caches; mapping misses become row errors unless the workflow is explicitly fail-fast.
|
|
21
|
+
- Commit only valid rows, chunk large batches, and preserve per-row outcome. Persist the job record (status, progress, counts, error-file pointer, actor, source file, idempotency key) per `state-machine-task-patterns.md`.
|
|
22
|
+
|
|
23
|
+
## Error Reports And Export
|
|
24
|
+
|
|
25
|
+
- Error reports preserve original row order with source row, source name, and failure reasons; generate with streaming writers; upload to object storage with bounded retention and store only the object key or signed reference. Represent an empty report explicitly rather than failing generation. No secrets or sensitive raw payloads in reports.
|
|
26
|
+
- Exports use cursor or id-window pagination (never offset for large datasets), streaming writers, and the same authorization/resource-scope filters as API reads; long exports run as jobs with progress and a downloadable artifact status.
|
|
27
|
+
|
|
28
|
+
## Cursor-Based Data Migration
|
|
29
|
+
|
|
30
|
+
- Read source rows by monotonic cursor or id window; discover min/max before splitting work and persist chunk boundaries so failed chunks are replayable.
|
|
31
|
+
- Insert into the target with idempotent create/upsert semantics before deleting from the source; delete only the cursor window that was successfully written, and record a replay/reconciliation path when source and target do not share one transaction boundary.
|
|
32
|
+
- Emit per-worker progress (current cursor, chunk range, migrated count, speed, terminal error); progress is best-effort — final migration state must be durable or replayable, and a closed progress channel is not proof all chunks succeeded.
|
|
33
|
+
|
|
34
|
+
## Tests
|
|
35
|
+
|
|
36
|
+
- Test empty file, hidden/empty sheets, malformed headers, short/long rows, duplicate rows, mapping misses, partial success, error-report generation, slice idempotency, cancellation, and retry after crash.
|
|
37
|
+
- For cursor migrations, test empty range, cursor boundary inclusiveness, idempotent insert, delete-after-insert ordering, worker error propagation, and replaying a partial chunk.
|