dew-method 0.1.0

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-03-lifecycle-architecture.md

@@ -0,0 +1,91 @@
+
+# Step 03 — Lifecycle Architecture
+
+## Mandatory Rules
+
+- Use data engineering lifecycle structure.
+- Do not over-design detailed storage/ingestion/transformation here.
+- Architecture-level decisions only.
+- Detailed designs belong to later skills.
+
+## Task
+
+Create lifecycle-level architecture.
+
+## Lifecycle sections
+
+### 1. Generation / Source Systems
+
+For each P1/P2 source, summarize:
+- source type
+- role
+- access method
+- grain
+- freshness
+- schema stability
+- validation status
+- caveats
+
+### 2. Storage
+
+Define high-level intent:
+- raw/bronze purpose
+- silver purpose
+- gold purpose
+- metadata expectations
+- evidence retention
+
+Do not finalize detailed storage layout yet.
+
+### 3. Ingestion
+
+Define high-level intent:
+- batch/streaming direction
+- snapshot/incremental direction
+- replay/backfill expectation
+- schema drift direction
+- failure handling expectation
+
+Do not finalize detailed ingestion spec yet.
+
+### 4. Transformation
+
+Define high-level intent:
+- KPI computation layer
+- grain transformation
+- bridge/mapping needs
+- conformance logic
+- DQ responsibility
+
+### 5. Serving
+
+Define high-level intent:
+- serving mode
+- consumer
+- freshness
+- trust statement
+- limitations
+
+### 6. Undercurrents
+
+Summarize:
+- security
+- privacy
+- data management
+- DataOps
+- orchestration
+- software engineering
+- observability
+- cost
+
+## Update documents
+
+Update `data-architecture.md`.
+
+## Update frontmatter
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-validated-context
+  - step-03-lifecycle-architecture

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-04-architecture-pattern-gate.md

@@ -0,0 +1,72 @@
+# Step 04 — Architecture Pattern Gate
+
+## Mandatory Rules
+
+- Stop at this gate.
+- Recommend, but do not decide.
+- User must approve the architecture pattern.
+- Explain trade-offs.
+- Record decision in ADR and decision log.
+
+## Task
+
+Run GATE-13 — Architecture Pattern Gate.
+
+## Pattern options
+
+A. Local-first  
+B. Warehouse-first  
+C. Lakehouse  
+D. Hybrid local-first + cloud/lakehouse  
+E. Streaming-first  
+F. Application-integrated  
+
+## Evaluate using
+
+- project type
+- user skill level
+- trust expectation
+- source types
+- data volume
+- freshness requirement
+- KPI complexity
+- serving mode
+- governance/security needs
+- cost/complexity
+- local feedback speed
+- deployment target
+
+## Required comparison table
+
+| Pattern | Pros | Cons | Fit | Risk | When to choose |
+|---|---|---|---|---|---|
+
+## Recommendation
+
+Recommend one pattern.
+
+Label it clearly as recommendation.
+
+## User decision
+
+Ask user to choose A/B/C/D/E/F.
+
+Do not continue until user chooses.
+
+## Record
+
+Update:
+- `data-architecture.md`
+- `architecture-decision-record.md`
+- `.decision-log.md`
+- `.learning-log.md` if learning mode is enabled
+
+Update frontmatter:
+
+```yaml
+decisionGates:
+  GATE-13-architecture-pattern:
+    status: approved / approved-with-caveats
+architecturePattern:
+  selected: "{selected pattern}"
+  status: approved

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-05-technology-decision-gate.md

@@ -0,0 +1,79 @@
+# Step 05 — Technology Decision Gate
+
+## Mandatory Rules
+
+- Stop at this gate.
+- Do not choose technology silently.
+- Do not choose tools because they are trendy.
+- Every technology decision must include alternatives, rationale, risk, reversibility, and revisit trigger.
+
+## Task
+
+Run GATE-14 — Technology Choice Gate.
+
+## Decision areas
+
+Evaluate at architecture level:
+
+1. Local development environment
+2. Storage direction
+3. Compute engine
+4. Orchestration direction
+5. Transformation framework
+6. Data quality/testing direction
+7. Serving direction
+8. Documentation/catalog direction
+
+## Required technology decision table
+
+| Decision Area | Option A | Option B | Option C | Recommended | Rationale |
+|---|---|---|---|---|---|
+| Local development | | | | | |
+| Storage | | | | | |
+| Compute | | | | | |
+| Orchestration | | | | | |
+| Transformation | | | | | |
+| Quality/testing | | | | | |
+| Serving | | | | | |
+| Documentation/catalog | | | | | |
+
+## Minimum decision quality
+
+For each selected direction, record:
+- selected option
+- alternatives considered
+- why selected
+- why alternatives rejected
+- evidence used
+- risk
+- reversibility
+- revisit trigger
+
+## User decision
+
+Ask user to approve technology direction:
+
+A. Approve recommended stack direction  
+B. Approve with caveats  
+C. Revise one or more technology choices  
+D. Defer technology decision to detailed design  
+E. Return to architecture pattern gate  
+
+Do not continue until user chooses.
+
+## Record
+
+Update:
+- `data-architecture.md`
+- `architecture-decision-record.md`
+- `.decision-log.md`
+- `.learning-log.md` if learning mode is enabled
+
+Update frontmatter:
+
+```yaml
+decisionGates:
+  GATE-14-technology-choice:
+    status: approved / approved-with-caveats / deferred
+technologyDecisions:
+  status: approved / approved-with-caveats / deferred

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-06-risk-caveat-and-reversibility.md

@@ -0,0 +1,76 @@
+# Step 06 — Risk, Caveat, and Reversibility Review
+
+## Mandatory Rules
+
+- Do not hide Requirement Gate caveats.
+- Do not convert caveats into solved problems.
+- Every hard-to-reverse decision needs a revisit trigger.
+- Every high-risk caveat needs owner or later skill responsibility.
+
+## Task
+
+Review risks, caveats, and reversibility.
+
+## Caveat carryover
+
+For each caveat from Requirement Gate:
+
+| Caveat ID | Area | Description | Architecture Impact | Later Skill Responsible |
+|---|---|---|---|---|
+
+Assign later skill:
+- dew-storage-design
+- dew-ingestion-design
+- dew-transformation-design
+- dew-serving-design
+- dew-governance-security-design
+- dew-create-data-story
+
+## Architecture risks
+
+Record:
+
+| Risk ID | Risk | Severity | Mitigation | Owner / Later Skill |
+|---|---|---|---|---|
+
+Risk areas:
+- source access instability
+- source schema drift
+- grain mismatch
+- freshness mismatch
+- DQ uncertainty
+- cost/complexity
+- operational burden
+- privacy/security
+- platform lock-in
+- local/cloud drift
+
+## Reversibility
+
+Record:
+
+| Decision | Reversibility | Why | Revisit Trigger |
+|---|---|---|---|
+
+Classify:
+- Easy
+- Medium
+- Hard
+
+## Update documents
+
+Update:
+- `data-architecture.md`
+- `architecture-caveat-carryover.md`
+- `architecture-decision-record.md`
+
+Update frontmatter:
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-validated-context
+  - step-03-lifecycle-architecture
+  - step-04-architecture-pattern-gate
+  - step-05-technology-decision-gate
+  - step-06-risk-caveat-and-reversibility

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-07-dataops-security-governance-undercurrents.md

@@ -0,0 +1,89 @@
+# Step 07 — DataOps, Security, and Governance Undercurrents
+
+## Mandatory Rules
+
+- Architecture must include undercurrents.
+- Do not leave security/privacy as afterthoughts.
+- Do not design full governance here; capture architecture-level responsibilities and defer details.
+
+## Task
+
+Review FDE-style undercurrents at architecture level.
+
+## Undercurrent review
+
+### Security
+
+Record:
+- data sensitivity
+- access control direction
+- secrets handling
+- network/auth assumptions
+- encryption direction if relevant
+
+### Privacy
+
+Record:
+- PII/sensitive data status
+- privacy posture
+- minimization expectation
+- retention caveats
+
+### Data Management
+
+Record:
+- metadata expectations
+- lineage expectations
+- ownership
+- catalog/documentation needs
+
+### DataOps
+
+Record:
+- orchestration direction
+- CI/CD direction
+- environment strategy
+- deployment path
+
+### Observability
+
+Record:
+- logging
+- monitoring
+- alerting
+- data quality checks
+- freshness checks
+
+### Software Engineering
+
+Record:
+- repo structure direction
+- testing approach
+- code review expectation
+- modularity
+- configuration management
+
+### Cost / FinOps
+
+Record:
+- cost risks
+- local-first cost advantage if applicable
+- cloud cost triggers
+- scaling thresholds
+
+## Update document
+
+Update:
+- `data-architecture.md`
+
+Update frontmatter:
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-validated-context
+  - step-03-lifecycle-architecture
+  - step-04-architecture-pattern-gate
+  - step-05-technology-decision-gate
+  - step-06-risk-caveat-and-reversibility
+  - step-07-dataops-security-governance-undercurrents

package/src/dew-skills/3-solutioning/dew-data-architecture/steps/step-08-finalize.md

@@ -0,0 +1,44 @@
+# Step 08 — Finalize Data Architecture
+
+## Mandatory Rules
+
+- Do not finalize if architecture pattern gate is unresolved.
+- Do not finalize if technology decision gate is unresolved unless explicitly deferred by user.
+- Do not hide caveats.
+- Do not mark detailed storage/ingestion/transformation/serving complete here.
+- Run checklist before finalizing.
+
+## Task
+
+Finalize data architecture.
+
+## Run checklist
+
+Use:
+
+`{workflow.checklist}`
+
+## Required final outputs
+
+- `data-architecture.md`
+- `architecture-decision-record.md`
+- `architecture-caveat-carryover.md`
+- decision log entries
+- learning log entries if enabled
+
+## Final status
+
+Set:
+
+```yaml
+status: final
+updated: {date}
+stepsCompleted:
+  - step-01-init
+  - step-02-load-validated-context
+  - step-03-lifecycle-architecture
+  - step-04-architecture-pattern-gate
+  - step-05-technology-decision-gate
+  - step-06-risk-caveat-and-reversibility
+  - step-07-dataops-security-governance-undercurrents
+  - step-08-finalize

package/src/dew-skills/3-solutioning/dew-data-model-design/SKILL.md

@@ -0,0 +1,47 @@
+---
+name: dew-data-model-design
+description: Design Silver and Gold data models with explicit grain, KPI mapping, bridge tables, dimensions, facts, and history handling.
+---
+
+# DEW Data Model Design
+
+**Goal:** Create data model specifications for Silver and Gold datasets.
+
+**Your Role:** You are a data modeling facilitator.
+
+You translate transformation rules and KPI requirements into explicit data models.
+
+You may recommend, but you must not decide.
+
+## Mandatory Rules
+
+- Do not design models before transformation design exists.
+- Do not define a model without explicit grain.
+- Do not create Gold model for unresolved KPI unless caveated.
+- Do not silently choose dimension/fact/bridge design.
+- If model grain is unclear, trigger `HALT-07 — Grain Undefined`.
+- If DQ rules are missing for critical models, trigger `HALT-14 — DQ Rules Missing`.
+
+## Workflow Architecture
+
+This workflow uses step-file architecture.
+
+## On Activation
+
+1. Resolve customization.
+2. Execute `{workflow.activation_steps_prepend}`.
+3. Load persistent facts.
+4. Load config if present.
+5. Load:
+   - `{workflow.data_model_spec_template}`
+   - `{workflow.kpi_to_model_mapping_template}`
+   - `{workflow.model_grain_rubric}`
+   - `{workflow.dimensional_modeling_rubric}`
+   - `{workflow.history_scd_rubric}`
+6. Greet user in configured language.
+
+## Execution
+
+Read fully and follow:
+
+`steps/step-01-init.md`

package/src/dew-skills/3-solutioning/dew-data-model-design/assets/data-model-spec-template.md

@@ -0,0 +1,152 @@
+# Data Model Spec Template
+
+```markdown
+---
+title: "{Project Name} Data Model Spec"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+stepsCompleted: []
+decisionGates:
+  GATE-24-modeling-grain:
+    status: pending
+  GATE-history-scd:
+    status: pending
+evidence: []
+assumptions: []
+caveats: []
+---
+
+# Data Model Spec: {Project Name}
+
+## 0. Document Purpose
+
+This document defines Silver and Gold model specifications.
+
+It includes:
+- model grain
+- fields
+- keys
+- dimensions
+- facts
+- bridge tables
+- history/SCD handling
+- KPI mapping
+- caveats
+
+## 1. Input Context
+
+| Input | Path | Status | Notes |
+|---|---|---|---|
+| Transformation Design | | Loaded/Missing | |
+| Business Rule Catalog | | Loaded/Missing | |
+| KPI Feasibility | | Loaded/Missing | |
+| Data Product PRD | | Loaded/Missing | |
+| Source Validation | | Loaded/Missing | |
+
+## 2. Model Summary
+
+| Model | Layer | Type | Grain | Consumer | Status |
+|---|---|---|---|---|---|
+| silver_x | Silver | Conformed table | | | proposed |
+| gold_y | Gold | Fact/Mart | | | proposed |
+
+## 3. Silver Models
+
+## Model: silver_{name}
+
+### Purpose
+
+### Grain
+
+One row represents:
+
+### Input Dataset(s)
+
+### Primary Key / Natural Key
+
+### Fields
+
+| Field | Type | Description | Source | Required? | DQ Rule |
+|---|---|---|---|---|---|
+| | | | | Yes/No | |
+
+### Business Rules Applied
+
+### Caveats
+
+## 4. Gold Models
+
+## Model: gold_{name}
+
+### Purpose
+
+### Consumer
+
+### KPI(s)
+
+### Grain
+
+One row represents:
+
+### Model Type
+
+- fact table
+- dimension table
+- aggregate mart
+- wide reporting table
+- bridge table
+- snapshot table
+
+### Input Dataset(s)
+
+### Keys
+
+### Fields
+
+| Field | Type | Description | Source/Rule | Related KPI | Required? |
+|---|---|---|---|---|---|
+| | | | | | |
+
+### KPI Mapping
+
+| KPI ID | KPI Name | Gold Field | Formula Rule | Feasibility Status |
+|---|---|---|---|---|
+
+### Caveats
+
+## 5. Dimensions
+
+| Dimension | Grain | Key | Purpose | Source |
+|---|---|---|---|---|
+
+## 6. Facts / Metrics
+
+| Fact / Metric | Grain | Additive? | Formula | Related KPI |
+|---|---|---|---|---|
+
+## 7. Bridge Tables
+
+| Bridge | Left Grain | Right Grain | Purpose | Caveat |
+|---|---|---|---|---|
+
+## 8. History / SCD Handling
+
+| Entity | History Needed? | SCD Type | Reason | Caveat |
+|---|---|---|---|---|
+
+## 9. Open Questions
+
+## 10. Handoff to DQ Design
+
+DQ design must create rules for:
+- primary keys
+- required fields
+- grain uniqueness
+- accepted ranges
+- referential integrity
+- KPI plausibility
+- bridge table consistency
+```

package/src/dew-skills/3-solutioning/dew-data-model-design/assets/dimensional-modeling-rubric.md

@@ -0,0 +1,48 @@
+# Dimensional Modeling Rubric
+
+## Fact table
+
+Use when:
+- model represents measurements/events
+- metrics are calculated
+- dimensions provide context
+
+## Dimension table
+
+Use when:
+- model represents descriptive entity attributes
+- attributes are used to filter/group facts
+
+## Aggregate mart
+
+Use when:
+- serving layer needs precomputed KPI/summary
+- dashboard/report performance matters
+- grain is consumer-oriented
+
+## Wide reporting table
+
+Use when:
+- dashboard/report needs simple consumption
+- project scope is small
+- semantic layer is lightweight
+
+Risk:
+- duplication
+- harder reuse
+
+## Bridge table
+
+Use when:
+- two grains/entities must be mapped
+- many-to-many relation exists
+- source grain differs from target grain
+
+Risk:
+- bridge assumptions affect KPI meaning
+
+## Snapshot table
+
+Use when:
+- point-in-time state matters
+- history tracking is needed