dew-method 0.1.0

package/src/dew-skills/3-solutioning/dew-storage-design/steps/step-06-finalize.md

@@ -0,0 +1,34 @@
+# Step 06 — Finalize Storage Design
+
+## Mandatory Rules
+
+- Do not finalize if storage layer gate is unresolved.
+- Do not finalize if metadata/retention decisions are unresolved.
+- Do not hide caveats.
+- Run checklist.
+
+## Task
+
+Finalize storage design.
+
+## Required outputs
+
+- storage-design.md
+- storage-decision-record.md
+- source-to-storage-mapping.md
+- decision log entry
+- learning log entry if enabled
+
+## Final status
+
+Set:
+
+```yaml
+status: final
+stepsCompleted:
+  - step-01-init
+  - step-02-load-architecture-context
+  - step-03-storage-layer-gate
+  - step-04-metadata-retention-schema-gates
+  - step-05-source-to-storage-mapping
+  - step-06-finalize

package/src/dew-skills/3-solutioning/dew-transformation-design/SKILL.md

@@ -0,0 +1,75 @@
+---
+name: dew-transformation-design
+description: Design Bronze to Silver and Silver to Gold transformation rules, grain changes, business rules, and caveat carryover.
+---
+
+# DEW Transformation Design
+
+**Goal:** Create a detailed transformation design from ingested data into clean, conformed, KPI-ready datasets.
+
+**Your Role:** You are a transformation design facilitator.
+
+You translate storage and ingestion design into Bronze → Silver → Gold transformation rules.
+
+You may recommend, but you must not decide.
+
+## Workflow Architecture
+
+This workflow uses step-file architecture.
+
+Rules:
+- Read one step file at a time.
+- Read the complete step file before acting.
+- Do not skip steps.
+- Do not load future steps early.
+- Stop at decision gates.
+- Update frontmatter after each completed step.
+- Preserve caveats from architecture, storage, ingestion, KPI feasibility, and source validation.
+
+## Mandatory Rules
+
+- Do not design transformation before storage and ingestion design exist.
+- Do not create Gold rules for KPI that is rejected, deferred, or unresolved unless user approves caveat.
+- Do not transform grain silently.
+- Do not hide business rules inside prose; record them in business-rule-catalog.md.
+- Do not treat DQ checks as optional if trust expectation requires them.
+- If grain is undefined, trigger `HALT-07 — Grain Undefined`.
+- If DQ rules are missing, trigger `HALT-14 — DQ Rules Missing`.
+- If evidence is missing, trigger `HALT-18 — No Validation Evidence`.
+
+## Conventions
+
+- Bare paths resolve from the skill root.
+- `{skill-root}` resolves to this skill's installed directory.
+- `{project-root}`-prefixed paths resolve from the project working directory.
+- `{workflow.<name>}` resolves to fields in `customize.toml`'s `[workflow]` table.
+- `{doc_workspace}` is the run folder for this workflow.
+
+## On Activation
+
+1. Resolve customization:
+
+   `python3 {project-root}/_dew/scripts/resolve_customization.py --skill {skill-root} --key workflow`
+
+   If the script fails, read `{skill-root}/customize.toml` directly and use defaults.
+
+2. Execute `{workflow.activation_steps_prepend}`.
+
+3. Load persistent facts from `{workflow.persistent_facts}`.
+
+4. Load `{project-root}/_dew/dew/config.yaml` if present.
+
+5. Load:
+   - `{workflow.transformation_design_template}`
+   - `{workflow.business_rule_catalog_template}`
+   - `{workflow.transformation_decision_record_template}`
+   - `{workflow.grain_transformation_rubric}`
+   - `{workflow.bronze_silver_gold_rule_guide}`
+
+6. Greet user in configured language.
+
+## Execution
+
+Read fully and follow:
+
+`steps/step-01-init.md`

package/src/dew-skills/3-solutioning/dew-transformation-design/assets/bronze-silver-gold-rule-guide.md

@@ -0,0 +1,64 @@
+# Bronze Silver Gold Rule Guide
+
+## Bronze
+
+Purpose:
+- preserve ingested source data
+- keep raw or near-raw state
+- enable replay/debugging
+- preserve source metadata
+
+Typical rules:
+- minimal parsing
+- add ingestion metadata
+- store raw response/file reference
+- capture schema snapshot
+- quarantine unreadable records
+
+Avoid:
+- business logic
+- heavy cleansing
+- KPI computation
+
+## Silver
+
+Purpose:
+- clean and standardize data
+- conform fields
+- enforce basic quality
+- prepare for modeling
+
+Typical rules:
+- cast types
+- rename fields
+- standardize units
+- deduplicate
+- validate required fields
+- normalize codes
+- join reference data
+- apply source-level quality checks
+
+Avoid:
+- hiding major business assumptions
+- mixing multiple KPI-specific aggregations too early
+
+## Gold
+
+Purpose:
+- serve KPI/data product needs
+- match consumer grain
+- provide metrics or marts
+- support dashboard/API/reporting
+
+Typical rules:
+- aggregate
+- compute KPI
+- join dimensions
+- apply business rules
+- expose friendly column names
+- document caveats
+
+Avoid:
+- using unverified KPI logic
+- changing grain silently
+- mixing unrelated consumers in one table without reason

package/src/dew-skills/3-solutioning/dew-transformation-design/assets/business-rule-catalog-template.md

@@ -0,0 +1,80 @@
+# Business Rule Catalog Template
+
+```markdown
+---
+title: "{Project Name} Business Rule Catalog"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+---
+
+# Business Rule Catalog
+
+## Purpose
+
+This catalog records business rules used in transformations and KPI computation.
+
+Business rules must not be hidden inside code or prose only.
+
+## Rule Table
+
+| Rule ID | Rule Name | Rule Type | Description | Owner | Related KPI | Layer | Status |
+|---|---|---|---|---|---|---|---|
+| BR-001 | | threshold/mapping/filter/classification/calculation | | | KPI-001 | Silver/Gold | proposed |
+
+## Rule Detail
+
+## BR-001 — {Rule Name}
+
+### Description
+
+### Rule Type
+
+- threshold
+- mapping
+- classification
+- filtering
+- calculation
+- enrichment
+- bridge
+- exception handling
+
+### Applies To
+
+Source:
+
+Dataset:
+
+Layer:
+
+KPI:
+
+### Logic
+
+```text
+...
+```
+
+### Evidence / Rationale
+
+### Caveats
+
+### Owner
+
+### Change Process
+
+A. User-approved decision log
+B. Internal rule catalog update
+C. Source owner documentation
+D. No formal process; monitor changes
+
+### Status
+
+* proposed
+* approved
+* approved-with-caveats
+* rejected
+* deprecated
+```

package/src/dew-skills/3-solutioning/dew-transformation-design/assets/grain-transformation-rubric.md

@@ -0,0 +1,65 @@
+# Grain Transformation Rubric
+
+## Preserve grain
+
+Use when:
+- output dataset should keep source grain
+- no aggregation or bridge is needed
+- downstream model expects same grain
+
+Risk:
+- may not match KPI grain
+
+## Aggregate grain
+
+Use when:
+- KPI or serving layer needs coarser grain
+- source grain is lower-level than target
+
+Examples:
+- station-hour → station-day
+- transaction → customer-day
+- ward-day → province-month
+
+Risk:
+- aggregation rules must be explicit
+
+## Join / enrich grain
+
+Use when:
+- source needs dimension/context enrichment
+- lookup/mapping is required
+
+Risk:
+- duplicate joins can change grain
+
+## Bridge grain
+
+Use when:
+- source grain and target grain are different entities
+- mapping table is needed
+
+Examples:
+- station → ward
+- product → category
+- device → location
+
+Risk:
+- bridge assumptions can dominate KPI meaning
+
+## Explode grain
+
+Use when:
+- nested/list data must become row-level records
+
+Risk:
+- row counts increase
+- duplicates possible
+
+## Filter grain
+
+Use when:
+- only subset is relevant to KPI/product
+
+Risk:
+- filtering criteria must be auditable

package/src/dew-skills/3-solutioning/dew-transformation-design/assets/transformation-decision-record-template.md

@@ -0,0 +1,69 @@
+# Transformation Decision Records
+
+## TDR-0001 — {Decision Title}
+
+Date:
+
+Status:
+- proposed
+- accepted
+- accepted-with-caveats
+- rejected
+- superseded
+
+## Context
+
+## Decision
+
+## Options Considered
+
+### A. {Option A}
+
+Pros:
+
+Cons:
+
+### B. {Option B}
+
+Pros:
+
+Cons:
+
+### C. {Option C}
+
+Pros:
+
+Cons:
+
+## Selected Option
+
+## Rationale
+
+## Evidence Used
+
+- KPI feasibility:
+- source validation:
+- storage design:
+- ingestion design:
+- decision log:
+
+## Consequences
+
+Positive:
+
+Negative:
+
+Trade-offs:
+
+## Reversibility
+
+Easy / Medium / Hard
+
+## Revisit Trigger
+
+## Downstream Impact
+
+- Data model:
+- DQ rules:
+- Serving:
+- Implementation stories:

package/src/dew-skills/3-solutioning/dew-transformation-design/assets/transformation-design-template.md

@@ -0,0 +1,146 @@
+# Transformation Design Template
+
+```markdown
+---
+title: "{Project Name} Transformation Design"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+stepsCompleted: []
+decisionGates:
+  GATE-19-transformation-rule:
+    status: pending
+  GATE-transformation-grain:
+    status: pending
+evidence: []
+assumptions: []
+caveats: []
+---
+
+# Transformation Design: {Project Name}
+
+## 0. Document Purpose
+
+This document defines transformation rules from raw/bronze data into Silver and Gold datasets.
+
+It covers:
+- Bronze → Silver rules
+- Silver → Gold rules
+- grain changes
+- business rules
+- KPI computation logic
+- caveats
+- DQ handoff
+
+## 1. Input Context
+
+| Input | Path | Status | Notes |
+|---|---|---|---|
+| Storage Design | | Loaded/Missing | |
+| Ingestion Design | | Loaded/Missing | |
+| KPI Feasibility | | Loaded/Missing | |
+| Source Validation | | Loaded/Missing | |
+| Data Product PRD | | Loaded/Missing | |
+| Evidence Index | | Loaded/Missing | |
+
+## 2. Caveat Carryover
+
+| Caveat ID | Source | Description | Transformation Impact |
+|---|---|---|---|
+
+## 3. Source-to-Transformation Summary
+
+| Source ID | Bronze Target | Source Grain | Silver Target | Gold Dependency | Caveat |
+|---|---|---|---|---|---|
+
+## 4. Grain Transformation Design
+
+| Input Dataset | Input Grain | Output Dataset | Output Grain | Transformation Type | Decision |
+|---|---|---|---|---|---|
+| | | | | Preserve/Aggregate/Join/Bridge/Explode/Filter | |
+
+## 5. Bronze to Silver Rules
+
+## Source / Dataset: {name}
+
+### Input
+
+### Output
+
+### Purpose
+
+### Transformation Rules
+
+| Rule ID | Rule Type | Description | Required? | Caveat |
+|---|---|---|---|---|
+| TR-001 | rename/cast/clean/filter/deduplicate/join/derive | | Yes/No | |
+
+### Metadata Rules
+
+### Error / Quarantine Rules
+
+### DQ Handoff
+
+## 6. Silver to Gold Rules
+
+## Gold Dataset: {name}
+
+### Purpose
+
+### Consumer
+
+### KPI(s)
+
+### Grain
+
+### Input Silver Dataset(s)
+
+### Transformation Rules
+
+| Rule ID | Rule Type | Description | Related KPI | Required? |
+|---|---|---|---|---|
+| TRG-001 | aggregate/join/calculate/filter/pivot | | KPI-001 | Yes |
+
+### KPI Formula Mapping
+
+| KPI ID | KPI Formula | Gold Field | Computation Rule |
+|---|---|---|---|
+
+### Caveats
+
+## 7. Business Rule Catalog Link
+
+See:
+
+`business-rule-catalog.md`
+
+## 8. DQ Rule Handoff
+
+Transformation requires DQ checks for:
+- input completeness
+- schema conformity
+- grain uniqueness
+- valid ranges
+- referential integrity
+- KPI plausibility
+- freshness
+- business rule consistency
+
+Detailed DQ design:
+
+`dew-data-quality-design`
+
+## 9. Open Questions
+
+## 10. Handoff to Data Model Design
+
+Data model design must use:
+- final Silver outputs
+- final Gold outputs
+- approved grain decisions
+- business rules
+- KPI mapping
+- caveats
+```

package/src/dew-skills/3-solutioning/dew-transformation-design/checklist.md

@@ -0,0 +1,52 @@
+# Transformation Design Checklist
+
+## Required inputs
+
+- [ ] Storage design loaded.
+- [ ] Ingestion design loaded.
+- [ ] KPI feasibility report loaded.
+- [ ] Source validation report loaded.
+- [ ] Data product PRD loaded.
+- [ ] Evidence index loaded or caveat recorded.
+
+## Grain
+
+- [ ] Input grains identified.
+- [ ] Output grains identified.
+- [ ] Grain transformations documented.
+- [ ] User approved grain transformation decisions.
+- [ ] Bridge/mapping needs identified.
+
+## Bronze to Silver
+
+- [ ] Bronze inputs listed.
+- [ ] Silver outputs listed.
+- [ ] Cleaning/casting/standardization rules defined.
+- [ ] Metadata rules defined.
+- [ ] Error/quarantine rules defined.
+- [ ] DQ handoff defined.
+
+## Silver to Gold
+
+- [ ] Gold outputs listed.
+- [ ] KPI mapping defined.
+- [ ] Aggregation rules defined.
+- [ ] Business rules defined.
+- [ ] Caveats documented.
+
+## Business rules
+
+- [ ] Business rule catalog created.
+- [ ] Each rule has ID.
+- [ ] Each rule has owner or decision source.
+- [ ] Each rule has change process.
+- [ ] User approved critical business rules.
+
+## Finalization
+
+- [ ] transformation-design.md completed.
+- [ ] business-rule-catalog.md completed.
+- [ ] transformation-decision-record.md completed.
+- [ ] Decision log updated.
+- [ ] Learning log updated if enabled.
+- [ ] No unresolved HALT-07 or HALT-14 remains.

package/src/dew-skills/3-solutioning/dew-transformation-design/customize.toml

@@ -0,0 +1,39 @@
+# DO NOT EDIT -- overwritten on every update.
+#
+# Workflow customization surface for dew-transformation-design.
+
+[workflow]
+
+activation_steps_prepend = []
+activation_steps_append = []
+
+persistent_facts = [
+  "file:{project-root}/**/data-architecture.md",
+  "file:{project-root}/**/architecture-caveat-carryover.md",
+  "file:{project-root}/**/storage-design.md",
+  "file:{project-root}/**/source-to-storage-mapping.md",
+  "file:{project-root}/**/ingestion-design.md",
+  "file:{project-root}/**/ingestion-source-plan.md",
+  "file:{project-root}/**/kpi-feasibility-report.md",
+  "file:{project-root}/**/source-validation-report.md",
+  "file:{project-root}/**/data-product-prd.md",
+  "file:{project-root}/**/validation-evidence-index.md",
+  "file:{project-root}/**/.decision-log.md",
+  "file:{project-root}/**/.learning-log.md",
+]
+
+on_complete = ""
+
+transformation_design_template = "assets/transformation-design-template.md"
+business_rule_catalog_template = "assets/business-rule-catalog-template.md"
+transformation_decision_record_template = "assets/transformation-decision-record-template.md"
+grain_transformation_rubric = "assets/grain-transformation-rubric.md"
+bronze_silver_gold_rule_guide = "assets/bronze-silver-gold-rule-guide.md"
+checklist = "checklist.md"
+
+transformation_output_path = "{planning_artifacts}/transformation-design"
+run_folder_pattern = "transformation-design-{project_name}-{date}"
+
+decision_log_path = "{planning_artifacts}/.decision-log.md"
+learning_log_path = "{learning_artifacts}/.learning-log.md"
+evidence_index_path = "{evidence_artifacts}/validation-evidence-index.md"

package/src/dew-skills/3-solutioning/dew-transformation-design/steps/step-01-init.md

@@ -0,0 +1,70 @@
+# Step 01 — Transformation Design Initialization
+
+## Mandatory Rules
+
+- Do not design transformation before storage and ingestion design exist.
+- Do not proceed if storage or ingestion design is missing.
+- Read the complete step before acting.
+
+## Task
+
+Initialize Transformation Design workflow.
+
+## Sequence
+
+### 1. Detect existing workflow
+
+Look for:
+
+- `{workflow.transformation_output_path}/**/transformation-design.md`
+- `{planning_artifacts}/**/*transformation*design*.md`
+
+If unfinished transformation design exists, offer to resume.
+
+### 2. Validate prerequisites
+
+Look for:
+- `storage-design.md`
+- `source-to-storage-mapping.md`
+- `ingestion-design.md`
+- `ingestion-source-plan.md`
+- `kpi-feasibility-report.md`
+
+If missing:
+- missing storage → `HALT-11`
+- missing ingestion → `HALT-12`
+- missing KPI feasibility → `HALT-03`
+
+### 3. Create workspace
+
+Bind `{doc_workspace}` to:
+
+`{workflow.transformation_output_path}/{workflow.run_folder_pattern}/`
+
+Create:
+- `transformation-design.md`
+- `business-rule-catalog.md`
+- `transformation-decision-record.md`
+
+Use templates.
+
+### 4. Initialize frontmatter
+
+Set:
+
+```yaml
+status: draft
+stepsCompleted:
+  - step-01-init
+decisionGates:
+  GATE-19-transformation-rule:
+    status: pending
+  GATE-transformation-grain:
+    status: pending
+```
+
+### 5. Continue checkpoint
+
+Ask:
+
+`[C] Continue to load storage and ingestion context`