@groupby/ai-dev 0.5.13 → 0.5.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/package.json +1 -1
  2. package/teams/brain-studio/skills/arch-context/SKILL.md +135 -0
  3. package/teams/brain-studio/skills/draft-plan/SKILL.md +222 -0
  4. package/teams/brain-studio/skills/draft-pr/SKILL.md +291 -0
  5. package/teams/brain-studio/skills/git-context/SKILL.md +159 -0
  6. package/teams/brain-studio/skills/html/SKILL.md +233 -0
  7. package/teams/brain-studio/skills/javascript/SKILL.md +680 -0
  8. package/teams/brain-studio/skills/jira-context/SKILL.md +113 -0
  9. package/teams/brain-studio/skills/jira-spec/SKILL.md +247 -0
  10. package/teams/brain-studio/skills/pr-review/SKILL.md +310 -0
  11. package/teams/brain-studio/skills/react/SKILL.md +282 -0
  12. package/teams/brain-studio/skills/styled-components/SKILL.md +336 -0
  13. package/teams/brain-studio/skills/tdd-implement/SKILL.md +278 -0
  14. package/teams/brain-studio/skills/typescript/SKILL.md +491 -0
  15. package/teams/fhr-andromeda/github/PULL_REQUEST_TEMPLATE.md +51 -0
  16. package/teams/fhr-andromeda/github/copilot-instructions.md +47 -0
  17. package/teams/fhr-andromeda/instructions/CLAUDE.md +66 -0
  18. package/teams/fhr-andromeda/instructions/ai-instructions.md +312 -0
  19. package/teams/fhr-andromeda/instructions/architecture.md +181 -0
  20. package/teams/fhr-andromeda/instructions/code-review.md +127 -0
  21. package/teams/fhr-andromeda/instructions/coding-guidelines.md +139 -0
  22. package/teams/fhr-andromeda/instructions/domain.md +125 -0
  23. package/teams/fhr-andromeda/instructions/jira-ticket-template.md +55 -0
  24. package/teams/fhr-andromeda/instructions/modules/data-quality-frontend.md +57 -0
  25. package/teams/fhr-andromeda/instructions/modules/data-quality.md +62 -0
  26. package/teams/fhr-andromeda/instructions/modules/fhr-custom-transforms.md +59 -0
  27. package/teams/fhr-andromeda/instructions/modules/hop-dev-platform-frontend.md +60 -0
  28. package/teams/fhr-andromeda/instructions/modules/hop-dev-platform.md +62 -0
  29. package/teams/fhr-andromeda/instructions/modules/workflow-scripts.md +119 -0
  30. package/teams/fhr-andromeda/skills/fhr-pr/SKILL.md +71 -0
  31. package/teams/fhr-andromeda/skills/fhr-review/SKILL.md +78 -0
  32. package/teams/fhr-andromeda/skills/fhr-spec/SKILL.md +96 -0
@@ -0,0 +1,66 @@
1
+ # CLAUDE.md - Repository Context
2
+
3
+ This file is the repository entrypoint for AI agents. It explains what this repo contains and
4
+ which harness file owns each kind of guidance.
5
+
6
+ ---
7
+
8
+ ## Repository Purpose
9
+
10
+ `fhr-apache-hop` contains application code for Fredhopper ETL 2.0, a Kubernetes-native data
11
+ pipeline platform built around Apache Hop.
12
+
13
+ Infrastructure definitions live in `andromeda-argo-deployments`. See `.ai/architecture.md` for
14
+ the repository boundary definition.
15
+
16
+ ---
17
+
18
+ ## Harness File Ownership
19
+
20
+ | File | Owns |
21
+ |---|---|
22
+ | `.github/copilot-instructions.md` | Copilot bootstrap and context-loading order |
23
+ | `CLAUDE.md` | Repository map and harness ownership |
24
+ | `.ai/architecture.md` | Stable system boundaries, ETL workflow structure, and module responsibilities |
25
+ | `.ai/domain.md` | Business terminology and domain concepts |
26
+ | `.ai/ai-instructions.md` | AI workflow, clarification, TDD, stop conditions, verification |
27
+ | `.ai/coding-guidelines.md` | Coding conventions and implementation rules |
28
+ | `.ai/code-review.md` | Review severity and checklist |
29
+ | `.ai/jira-ticket-template.md` | Full Jira ticket spec template |
30
+ | `<module>/CLAUDE.md` | Module-specific constraints and integration context |
31
+
32
+ Keep each file focused on its owner topic. Do not duplicate detailed rules across files.
33
+
34
+ ---
35
+
36
+ ## Systems in This Repository
37
+
38
+ | System | Modules | Purpose |
39
+ |---|---|---|
40
+ | ETL Engine | `fhr-custom-transforms`, `standard-etl-transformations`, `fhr-hop-server` | Apache Hop extensions, packaged ETL flows, and runtime image assembly |
41
+ | Workflow Scripts | `workflow-scripts` | Python logic executed by Argo Workflow steps |
42
+ | HDSys | `hop-dev-platform`, `hop-dev-platform-frontend` | Browser-based Hop workspace platform for ETL customisation |
43
+ | DQSys | `data-quality`, `data-quality-frontend` | Data-quality reporting service and UI |
44
+
45
+ ---
46
+
47
+ ## Working in a Module
48
+
49
+ Before changing a module:
50
+
51
+ 1. Read this file and the root context files listed above.
52
+ 2. Read the module's `CLAUDE.md`.
53
+ 3. Inspect existing code in the target area instead of relying on stale inventories.
54
+ 4. Follow `.ai/ai-instructions.md` for workflow and `.ai/coding-guidelines.md` for
55
+ implementation style.
56
+
57
+ ---
58
+
59
+ ## Build Entry Points
60
+
61
+ Use the root Maven project for Java and packaging modules. Use each Angular module's package
62
+ scripts for frontend modules. Prefer module-scoped builds unless a shared parent or cross-module
63
+ contract changed.
64
+
65
+ Exact commands are in `.ai/ai-instructions.md` §8. Use the module's current build files as the
66
+ authoritative source for profiles and flags.
@@ -0,0 +1,312 @@
1
+ # AI Instructions - fhr-apache-hop
2
+
3
+ This file owns AI operating rules: how to start, clarify, plan, generate, verify, and stop.
4
+ Coding style belongs in `.ai/coding-guidelines.md`; architecture belongs in `.ai/architecture.md`;
5
+ module-specific constraints belong in the relevant module `CLAUDE.md`.
6
+
7
+ ---
8
+
9
+ ## 1. Operating Priority
10
+
11
+ Apply instructions in this order:
12
+
13
+ 1. System/developer instructions.
14
+ 2. `.github/copilot-instructions.md`.
15
+ 3. This file.
16
+ 4. Relevant module `CLAUDE.md`.
17
+ 5. `.ai/coding-guidelines.md`.
18
+ 6. Existing local code patterns.
19
+
20
+ If conflict remains, stop and ask.
21
+
22
+ ---
23
+
24
+ ## 2. Mandatory Pre-Flight
25
+
26
+ Before technical work:
27
+
28
+ 1. Read root `CLAUDE.md`.
29
+ 2. Read `.ai/architecture.md`.
30
+ 3. Read `.ai/domain.md`.
31
+ 4. Read `.ai/coding-guidelines.md`.
32
+ 5. Read the relevant module `CLAUDE.md` when a module is in scope.
33
+ 6. Identify acceptance criteria and missing context.
34
+
35
+ Do not infer contracts from stale examples. Inspect current source where needed.
36
+
37
+ ---
38
+
39
+ ## 3. Task Context File
40
+
41
+ For functional code-generation tasks, create and maintain a task context file:
42
+
43
+ ```
44
+ tasks/TASK-<TICKET-ID>.md
45
+ ```
46
+
47
+ Add `tasks/` to `.gitignore` — these are working files, not source artefacts.
48
+
49
+ The file should track: task title, ticket, module, and agent type; current phase;
50
+ clarification Q&A; TDD plan; files changed; build/verification result; blockers or stop
51
+ conditions.
52
+
53
+ Minimal heading structure:
54
+
55
+ ```markdown
56
+ # Task: <title>
57
+ Ticket: <id> | Module: <module> | Agent: <type> | Phase: <current>
58
+
59
+ ## Clarifications
60
+ ## TDD Plan
61
+ ## Files
62
+ ## Build Result
63
+ ## Blockers
64
+ ```
65
+
66
+ If no ticket exists, ask whether a task context file is required before creating one.
67
+
68
+ ---
69
+
70
+ ## 4. Clarification Protocol
71
+
72
+ Ask one batched clarification round before code when:
73
+
74
+ - acceptance criteria cannot become concrete tests without guessing;
75
+ - module scope is unclear;
76
+ - an API, data, environment, or configuration contract is missing;
77
+ - the task needs a new dependency, module, migration, or cross-repo change;
78
+ - the request conflicts with documented architecture or coding rules.
79
+
80
+ Write questions clearly, grouped by topic. Wait for answers before implementation.
81
+
82
+ If no clarification is needed, state that and proceed to the TDD plan.
83
+
84
+ ---
85
+
86
+ ## 5. TDD Workflow for Functional Changes
87
+
88
+ Functional code tasks must follow this order.
89
+
90
+ ### Phase 1 - TDD Plan
91
+
92
+ Before writing code, output:
93
+
94
+ 1. Numbered test cases with name, given/when/then, and type.
95
+ 2. Files to create or modify, labelled by role.
96
+ 3. Output order.
97
+ 4. Mermaid diagram showing test, production, migration, and config dependencies.
98
+
99
+ End with:
100
+
101
+ > **Awaiting approval.** Reply with `APPROVED`, `lgtm`, or `proceed` to continue to Phase 2.
102
+
103
+ Stop until approval is received.
104
+
105
+ ### Phase 2 - Tests Red
106
+
107
+ Write complete tests first. They must compile and be expected to fail because production
108
+ behaviour is not implemented yet.
109
+
110
+ State:
111
+
112
+ > Tests written. These will fail (red) until Phase 3 is complete.
113
+
114
+ ### Phase 3 - Production Green
115
+
116
+ Write the minimum production code needed to satisfy the tests.
117
+
118
+ ### Phase 4 - Refactor
119
+
120
+ Clean up without changing behaviour. Remove duplication, unused imports, dead methods,
121
+ placeholders, and stale tests.
122
+
123
+ State:
124
+
125
+ > Refactor complete. All tests remain green.
126
+
127
+ ### Phase 5 - Build Verification
128
+
129
+ Run the appropriate module build and fix failures before declaring done. Report the result
130
+ clearly.
131
+
132
+ ---
133
+
134
+ ## 6. Output Order
135
+
136
+ For functional changes, produce output in this order:
137
+
138
+ 1. TDD plan.
139
+ 2. Test files.
140
+ 3. Production files.
141
+ 4. Migration files, if required.
142
+ 5. Config/property changes, if required.
143
+ 6. Brief review note.
144
+
145
+ Do not interleave test and production output.
146
+
147
+ ---
148
+
149
+ ## 7. Stop Conditions
150
+
151
+ Stop and notify the developer when:
152
+
153
+ - Phase 1 approval has not been received for a functional code task.
154
+ - The task would modify an existing Flyway migration.
155
+ - The task would create a new Maven/module boundary.
156
+ - The task requires an unapproved dependency or framework.
157
+ - The task requires infrastructure or workflow template changes outside this repo.
158
+ - Acceptance criteria contradict documented architecture or API contracts.
159
+ - A red-phase test unexpectedly passes before production code exists.
160
+ - Compilation or build verification still fails after three focused fix attempts.
161
+ - Required business, API, data, security, or deployment context is missing.
162
+
163
+ ---
164
+
165
+ ## 8. Verification Rules
166
+
167
+ After modifying runnable code:
168
+
169
+ - Check edited files for IDE/type errors where possible.
170
+ - Run the smallest relevant build command that proves the change:
171
+ - Java: `mvn clean install -pl <module> -am --no-transfer-progress`
172
+ - Angular: `ng build --configuration development` (from the module directory)
173
+ - Use a full build when parent configuration, shared code, or cross-module contracts changed.
174
+ - Do not skip failing tests or weaken assertions to pass verification.
175
+ - If tooling is unavailable, state exactly what could not be verified.
176
+
177
+ Commit messages must follow conventional-commit format: `feat(scope): description` or
178
+ `fix(scope): description`.
179
+
180
+ Documentation-only changes do not require application builds, but should still be checked for
181
+ Markdown or obvious consistency issues.
182
+
183
+ ---
184
+
185
+ ## 9. Harness Gap Protocol
186
+
187
+ If correct generation requires missing context:
188
+
189
+ 1. Do not guess.
190
+ 2. State exactly what is missing.
191
+ 3. Identify which harness or module document should own that missing context.
192
+ 4. Ask for clarification before implementing.
193
+
194
+ Harness gaps should be fixed in context files, not bypassed with low-confidence code.
195
+
196
+ ---
197
+
198
+ ## 10. Task Sizing Guidance
199
+
200
+ Prefer tasks that:
201
+
202
+ - touch one module;
203
+ - have concrete acceptance criteria;
204
+ - avoid unrelated refactoring;
205
+ - separate API, persistence, frontend, and infrastructure concerns when practical.
206
+
207
+ If a task is too broad, recommend splitting it before implementation.
208
+
209
+ ---
210
+
211
+ ## 11. Autonomous Jira Workflow
212
+
213
+ This section describes the end-to-end flow when work starts from a Jira ticket ID.
214
+
215
+ ### Step 1 — Ticket Ingestion
216
+
217
+ When given a Jira ticket ID:
218
+
219
+ 1. If the Atlassian MCP is available, fetch the ticket using `getJiraIssue`. If no Jira MCP
220
+ is configured, ask the developer to paste the ticket title, description, and acceptance
221
+ criteria before continuing.
222
+ 2. Read the title, description, and any existing acceptance criteria.
223
+ 3. Run a single batched interrogation round — ask only what cannot be derived from the
224
+ ticket, harness files, or current source.
225
+ 4. Once interrogation is complete, produce a structured AI Spec (see format below) and:
226
+ - Present it in the conversation for review.
227
+ - If the Atlassian MCP is available, append it to the Jira ticket description as a new
228
+ `## AI Spec` section using `editJiraIssue` (append; do not overwrite the original
229
+ description). Skip this sub-step when no MCP is configured.
230
+
231
+ ### Step 2 — Spec Approval
232
+
233
+ Do not write any code or create a branch until the spec is explicitly approved. End the
234
+ spec presentation with:
235
+
236
+ > **Awaiting spec approval.** Reply with `APPROVED`, `lgtm`, or `proceed` to continue.
237
+
238
+ Stop until approval is received. Spec approval is Phase 1 approval for the TDD workflow.
239
+
240
+ ### Step 3 — Branch Creation
241
+
242
+ After spec approval, create the working branch from `master`:
243
+
244
+ ```bash
245
+ git checkout -b <TICKET-ID>-<short-kebab-description>
246
+ ```
247
+
248
+ The branch name must include the Jira ticket key (e.g. `FHR-7108`) so Jira auto-links the
249
+ branch in its development panel. No MCP call is needed — Jira detects the key automatically.
250
+
251
+ ### Step 4 — Implementation (TDD Phases 2–5)
252
+
253
+ Follow the TDD workflow in §5. Commit after each meaningful unit:
254
+
255
+ - A passing test suite for a new behaviour.
256
+ - A completed service method with its tests.
257
+ - A migration paired with the model change it supports.
258
+
259
+ Do not batch unrelated changes into one commit. Each commit message must follow
260
+ conventional-commit format: `feat(scope): description` or `fix(scope): description`.
261
+
262
+ ### Step 5 — PR Creation
263
+
264
+ After Phase 5 build verification passes:
265
+
266
+ 1. Push the branch and open a PR against `master` using `gh pr create`.
267
+ 2. PR title: `<TICKET-ID>: <short description>`.
268
+ 3. PR body must include:
269
+ - Summary of what changed compared to `master` and why (from the spec objective).
270
+ - Notable implementation decisions or constraints applied.
271
+ - Testing evidence (build result, test count or coverage delta).
272
+ - Link to the Jira ticket.
273
+ 4. If the Atlassian MCP is available, post the PR URL as a comment on the Jira ticket using
274
+ `addCommentToJiraIssue`. Skip this step when no MCP is configured.
275
+
276
+ ### AI Spec Format
277
+
278
+ ```markdown
279
+ ## AI Spec
280
+
281
+ **Ticket**: <id> **Module**: <module> **Agent**: <type>
282
+
283
+ ### Objective
284
+ One or two sentences — what this achieves and why.
285
+
286
+ ### Out of Scope
287
+
288
+ ### Functional Requirements
289
+ 1. Given … when … then …
290
+
291
+ ### Technical Constraints
292
+ Architecture pattern, security requirements, cross-repo impact, new dependencies
293
+ (require explicit approval).
294
+
295
+ ### Acceptance Criteria
296
+ - [ ] Given … when … then …
297
+ - [ ] All existing tests pass.
298
+ - [ ] BUILD SUCCESS confirmed locally.
299
+ - [ ] No TODO, FIXME, placeholder comments, or commented-out code.
300
+ ```
301
+
302
+ ---
303
+
304
+ ## 12. Jira Spec Expectations
305
+
306
+ A useful ticket for AI generation should include: agent type and module; objective and
307
+ out-of-scope items; functional requirements in given/when/then form; API, data, security, and
308
+ config contracts when relevant; dependency and cross-repo impact notes; acceptance criteria and
309
+ definition of done.
310
+
311
+ Use the AI Spec Format in §11 as the canonical format when writing or consuming specs. The full
312
+ human-facing ticket template is in `.ai/jira-ticket-template.md`.
@@ -0,0 +1,181 @@
1
+ # Architecture - fhr-apache-hop
2
+
3
+ This file owns stable system architecture: what each system is responsible for, how modules
4
+ interact, and where this repository's boundaries are.
5
+
6
+ ---
7
+
8
+ ## Repository Boundary
9
+
10
+ This repository contains application code for ETL 2.0. Kubernetes manifests, Helm charts, Argo
11
+ Workflow templates, and cluster deployment definitions are maintained in the separate
12
+ `andromeda-argo-deployments` repository.
13
+
14
+ If a task changes runtime orchestration, workflow templates, cluster permissions, or deployment
15
+ shape, flag the cross-repo impact instead of generating infrastructure changes here.
16
+
17
+ ---
18
+
19
+ ## System Overview
20
+
21
+ ```mermaid
22
+ flowchart LR
23
+ subgraph HDSys[HDSys]
24
+ HDFE[HDSys UI]
25
+ HDBE[HDSys Backend]
26
+ HDFE --> HDBE
27
+ end
28
+
29
+ subgraph ETL[ETL Engine]
30
+ Hop[Hop Runtime]
31
+ Plugins[Custom Transforms]
32
+ Packages[Standard ETL Packages]
33
+ Hop --> Plugins
34
+ Hop --> Packages
35
+ end
36
+
37
+ subgraph WF[Workflow Scripts]
38
+ Steps[Argo Step Logic]
39
+ end
40
+
41
+ subgraph DQSys[DQSys]
42
+ DQFE[DQSys UI]
43
+ DQBE[DQSys Backend]
44
+ DB[(PostgreSQL)]
45
+ DQFE --> DQBE --> DB
46
+ end
47
+
48
+ Storage[(Object Storage)]
49
+ Events[External Workflow Orchestration]
50
+
51
+ Events --> Steps
52
+ Steps --> Hop
53
+ Hop --> Storage
54
+ Steps --> DQBE
55
+ ```
56
+
57
+ ---
58
+
59
+ ## System Responsibilities
60
+
61
+ ### ETL Engine
62
+
63
+ Owns Apache Hop runtime extensions and standard ETL packages. It transforms customer catalogue
64
+ inputs into outputs consumed by downstream Fredhopper systems (FAS XML files, suggestion
65
+ indices, upload packages).
66
+
67
+ ### Workflow Scripts
68
+
69
+ Own Python logic executed by Argo workflow steps. Scripts orchestrate job setup, data movement,
70
+ runtime invocation, result publication, metrics, and DQ integration. Workflow definitions live
71
+ in `andromeda-argo-deployments`.
72
+
73
+ ### HDSys
74
+
75
+ Owns the development platform for ETL customisation. It provisions browser-based Hop workspaces
76
+ for consultants and manages the GitOps workflow for customer ETL packages. It is a
77
+ development-time system and does not participate in ETL run execution.
78
+
79
+ ### DQSys
80
+
81
+ Owns data-quality reporting and ETL Dashboard data. Workflow steps send quality and operational
82
+ signals after each run, and the DQSys UI presents them to users.
83
+
84
+ ---
85
+
86
+ ## Module Responsibilities
87
+
88
+ | Module | Responsibility |
89
+ |---|---|
90
+ | `fhr-custom-transforms` | Fredhopper-specific Hop transform implementations |
91
+ | `standard-etl-transformations` | Standard reusable ETL packages |
92
+ | `fhr-hop-server` | Hop runtime packaging and assembly |
93
+ | `workflow-scripts` | Workflow-step application logic |
94
+ | `hop-dev-platform` | HDSys backend APIs and integrations |
95
+ | `hop-dev-platform-frontend` | HDSys user interface |
96
+ | `data-quality` | DQSys backend APIs and persistence |
97
+ | `data-quality-frontend` | DQSys user interface |
98
+
99
+ ---
100
+
101
+ ## ETL Workflow Structure
102
+
103
+ An ETL run is an Argo workflow triggered by a `trigger.json` file placed in S3. Steps run in
104
+ this order:
105
+
106
+ 1. **extract-and-dispatch** — determines whether input is a zip (batch) or stream, extracts
107
+ if needed, and resolves the Hop project name from service type and data format.
108
+ 2. **initialize-job** — assembles the `hop_job_config` JSON blob and writes it for all
109
+ downstream steps.
110
+ 3. **create-pvc** + **metrics-daemon** — run in parallel; the PVC provides shared storage
111
+ between the data and XML Hop server pods.
112
+ 4. **run-etl** — launches the Hop server pod. Init container: `prepare_data.py`. Main
113
+ container: `run_and_monitor.py`. Sidecar: hop-server process.
114
+ 5. **generate-xml** — same pod structure with the XML project config; produces FAS XML files.
115
+ Runs for `fas` service type only when `generate_xml` is true.
116
+ 6. **submit-dq-workflow** — submits a fire-and-forget child workflow (`dq_execution.py`).
117
+ Runs for `fas` only; does not block the main DAG.
118
+ 7. **publish-results** — uploads outputs and logs to S3; sends completion signals to Kafka.
119
+ 8. **configure-kafka** — creates Kafka topics; runs for streaming (`load-data-stream`) only.
120
+ 9. **submit-populate-dashboard** — submits a fire-and-forget child workflow
121
+ (`populate_dashboard.py`).
122
+
123
+ ---
124
+
125
+ ## hop_job_config — Central Run Contract
126
+
127
+ `initialize_job.py` produces a `hop_job_config` JSON blob passed to every downstream step.
128
+ It is the single source of truth for per-run identity (customer, instance, service_type,
129
+ trigger_id), S3 paths, resource sizing, Hop project configuration, database and Kafka
130
+ coordinates, and incremental state.
131
+
132
+ All workflow scripts consume it via `apply_hop_job_config()` from `arg_utils.py`, which merges
133
+ its fields onto the script's parsed argument namespace. Scripts access fields directly as
134
+ `args.customer`, `args.service_type`, etc.
135
+
136
+ `initialize_job.py` is the authoritative definition of all available fields. Scripts must not
137
+ re-derive what is already present in the blob.
138
+
139
+ ---
140
+
141
+ ## DQSys Population Paths
142
+
143
+ DQSys is populated through three distinct paths with different triggers and targets:
144
+
145
+ | Path | Script | Trigger | Writes |
146
+ |---|---|---|---|
147
+ | DQ checks | `dq_execution.py` | After data ETL, `fas` only | `check_registry`, `check_issues`, `run_log` in DQ DB |
148
+ | Run metadata | `populate_dashboard.py` | After publish-results, all service types | One row in `{customer}_{instance}_etl_dashboard` in DQ DB |
149
+ | Reindex completion | `update_etl_dashboard.py` (Deployment) | Kafka: FAS `load-xml` or suggest `load-idx` | Updates `total_run_time` and `complete_status` on the existing row |
150
+
151
+ `dq_execution.py` uses a two-database connection (`db_utils.DatabaseConnection`): it reads SQL
152
+ checks against the **ETL database** (schema `{customer}_{instance}`) and writes results to the
153
+ **DQ database** (`data_quality`). The other two paths write only to the DQ database.
154
+
155
+ ---
156
+
157
+ ## prepare_data — PVC Setup
158
+
159
+ `prepare_data.py` runs as the init container of every Hop server pod and chains three steps:
160
+
161
+ 1. `deploy_standard_etl_package.py` — deploys the standard ETL ZIP for the Hop project.
162
+ 2. `deploy_custom_etl_package.py` — sparse-clones the customer's directory from the
163
+ `custom-etl` GitHub repository, using `hop_job_config.custom_etl_branch` if set.
164
+ 3. `map_etl_directories.py` — writes S3 path mappings into the Hop project config on the PVC.
165
+
166
+ Customer customisations merged into `custom-etl` take effect on the next ETL run; there is no
167
+ active sync between HDSys and the ETL runtime.
168
+
169
+ ---
170
+
171
+ ## Stable Architecture Constraints
172
+
173
+ - `data-quality` is reactive (Spring WebFlux + R2DBC) and must remain non-blocking.
174
+ - `hop-dev-platform` is synchronous (Spring MVC) and must not adopt reactive APIs.
175
+ - Frontend modules communicate through backend APIs only; no direct database or infrastructure
176
+ access.
177
+ - Workflow scripts are entrypoint logic for orchestration steps, not workflow definitions.
178
+ - Infrastructure and workflow template changes belong in `andromeda-argo-deployments`.
179
+ - Cross-module API changes must consider their consumers in the same change plan.
180
+ - `hop_job_config` is the single source of truth for per-run state; scripts must not
181
+ re-derive what is already in it.
@@ -0,0 +1,127 @@
1
+ # Code Review Checklist - fhr-apache-hop
2
+
3
+ This file owns review criteria. It operationalizes `.ai/coding-guidelines.md`,
4
+ `.ai/architecture.md`, and the relevant module `CLAUDE.md`.
5
+
6
+ ---
7
+
8
+ ## Severity
9
+
10
+ - **BLOCK** - must be fixed before merge; violates safety, architecture, or hard repository
11
+ rules.
12
+ - **WARN** - should be fixed; convention drift or maintainability risk.
13
+ - **NOTE** - optional suggestion or observation.
14
+
15
+ Every finding should include severity, location, impact, and a concise recommendation.
16
+
17
+ ---
18
+
19
+ ## Universal BLOCK Checks
20
+
21
+ - Secrets, credentials, environment-specific URLs, or local paths are committed.
22
+ - Existing Flyway migrations are modified.
23
+ - New dependencies, frameworks, modules, or infrastructure scope appear without approval.
24
+ - Placeholders, `TODO`, `FIXME`, or commented-out code remain.
25
+ - Tests are missing for new or changed behaviour.
26
+ - Code contradicts documented architecture for the target module.
27
+ - Error handling hides failures or creates silent data loss.
28
+ - Security or authorization checks are bypassed.
29
+
30
+ ---
31
+
32
+ ## Universal WARN Checks
33
+
34
+ - Code does not match local naming, formatting, or structure conventions.
35
+ - Duplicated logic or literals that should be extracted or reused.
36
+ - Comments explain obvious code instead of non-obvious constraints.
37
+ - Public contracts changed without updating consumers or documenting impact.
38
+ - Build, test, or verification evidence is missing.
39
+ - New code is broader than the task scope.
40
+ - `split()` with magic numeric indices used instead of constants, regex, or `pathlib`.
41
+ - Unused imports, dead methods, or unreachable branches left after refactoring.
42
+
43
+ ---
44
+
45
+ ## Java Review Checks
46
+
47
+ - Lombok is used for boilerplate; no manually written getters, setters, or constructors.
48
+ **BLOCK** if absent.
49
+ - Spring beans use constructor injection (`@RequiredArgsConstructor`). **BLOCK** if
50
+ `@Autowired` field injection is used.
51
+ - Logging uses `@Slf4j` and `log.*`; no `System.out.println`. **BLOCK** if violated.
52
+ - Domain exceptions and centralized HTTP error mapping used where appropriate.
53
+ - Database integration tests use real Testcontainers, not in-memory substitutes. **BLOCK** if
54
+ violated.
55
+
56
+ ### data-quality (reactive)
57
+
58
+ - All service and repository methods return `Mono<T>` or `Flux<T>`. **BLOCK** if synchronous
59
+ return types are introduced.
60
+ - No `.block()` calls anywhere in production or test code. **BLOCK**.
61
+ - Reactive tests use `StepVerifier`. **BLOCK** if `.block()` is used in tests instead.
62
+
63
+ ### hop-dev-platform (synchronous)
64
+
65
+ - No `Mono`, `Flux`, or WebFlux patterns introduced. **BLOCK** if reactive types appear in
66
+ production code.
67
+
68
+ ### fhr-custom-transforms (Hop plugins)
69
+
70
+ - Every new transform has both a `Meta` class and a `Data` class. **BLOCK** if either is
71
+ missing.
72
+ - `Meta` class is annotated with `@Transform(...)`. **BLOCK** if annotation is absent.
73
+ - Dialog/UI code uses SWT only; no Swing or JavaFX. **BLOCK** if violated.
74
+ - No SWT display-dependent code outside dialog classes (breaks headless CI). **BLOCK**.
75
+
76
+ ---
77
+
78
+ ## Python Review Checks
79
+
80
+ - Function signatures are typed. **WARN** if absent.
81
+ - Scripts use `logger_config.py` for logging; no `print()` in production code. **BLOCK**.
82
+ - Scripts use `arg_utils.py` for argument parsing; no raw `argparse` or `sys.argv`. **BLOCK**.
83
+ - Fatal errors are logged with context (`exc_info=True`) and exit non-zero.
84
+ - No bare `except:` — specific exceptions only. **BLOCK**.
85
+ - External systems (S3, Kafka, Kubernetes, PostgreSQL) are mocked in unit tests.
86
+ - New scripts follow the mandatory structure: imports → logger → constants → helpers →
87
+ `main()` → `if __name__ == "__main__"`.
88
+
89
+ ---
90
+
91
+ ## Angular Review Checks
92
+
93
+ - UI components use PrimeNG only; no other UI libraries. **BLOCK**.
94
+ - HTTP calls are kept in services; `HttpClient` never called directly in components. **BLOCK**.
95
+ - Observable subscriptions include cleanup (`takeUntilDestroyed` or `async` pipe). **BLOCK**
96
+ if a `.subscribe()` call has no cleanup.
97
+ - API responses have reusable typed interfaces; `any` is avoided.
98
+ - Component and service specs cover changed behaviour.
99
+
100
+ ---
101
+
102
+ ## Database and Migration Checks
103
+
104
+ - Schema changes are represented by new migrations only; existing migrations are untouched.
105
+ - Migration names follow the Flyway convention (`V<major>_<minor>__<description>.sql`) with a
106
+ sequential version number — inspect existing migrations before choosing the next number.
107
+ - Persistence model changes and API contracts stay aligned.
108
+ - Migrations are backward-compatible where practical.
109
+
110
+ ---
111
+
112
+ ## Cross-Module Impact Checks
113
+
114
+ Flag when a change affects:
115
+
116
+ - backend API consumers;
117
+ - workflow step inputs, outputs, or environment contracts;
118
+ - frontend API models;
119
+ - database schema used outside one module;
120
+ - infrastructure or workflow definitions outside this repo.
121
+
122
+ When the `data-quality` REST API contract changes (endpoint paths, request or response fields),
123
+ explicitly check whether `dq_execution.py`, `dq_custom_checks.py`, and `populate_dashboard.py`
124
+ in `workflow-scripts` need to be updated in the same PR.
125
+
126
+ Do not require same-PR fixes for every impact, but require the impact to be explicit and
127
+ tracked.