dew-method 0.1.0

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-01-init.md

@@ -0,0 +1,46 @@
+# Ingestion Design Checklist
+
+## Required inputs
+
+- [ ] Data architecture loaded.
+- [ ] Storage design loaded.
+- [ ] Source-to-storage mapping loaded.
+- [ ] Source validation report loaded.
+- [ ] Source contract test results loaded or caveat recorded.
+- [ ] Evidence index loaded or caveat recorded.
+
+## Ingestion pattern
+
+- [ ] Ingestion pattern selected by source.
+- [ ] Batch vs streaming decision made.
+- [ ] Snapshot vs incremental decision made.
+- [ ] Frequency mapped to freshness requirement.
+- [ ] User approved key ingestion decisions.
+
+## Source-specific plans
+
+- [ ] Each P1 source has ingestion source plan.
+- [ ] Raw/Bronze target specified.
+- [ ] Metadata requirements specified.
+- [ ] Schema snapshot target specified.
+- [ ] Quarantine/error handling specified.
+- [ ] Idempotency strategy specified.
+- [ ] Replay/backfill strategy specified.
+
+## Schema and failure handling
+
+- [ ] Schema evolution strategy selected.
+- [ ] Failure handling strategy defined.
+- [ ] Retry/backoff behavior defined if relevant.
+- [ ] Partial load behavior defined.
+- [ ] Evidence/logging requirements defined.
+
+## Finalization
+
+- [ ] ingestion-design.md completed.
+- [ ] ingestion-decision-record.md completed.
+- [ ] ingestion-source-plan.md completed.
+- [ ] Decision log updated.
+- [ ] Learning log updated if learning mode is enabled.
+- [ ] No unresolved HALT-12 remains.
+- [ ] No unresolved HALT-13 remains.

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-02-load-source-and-storage-context.md

@@ -0,0 +1,46 @@
+# Step 02 — Load Source and Storage Context
+
+## Mandatory Rules
+
+- Load source validation and storage mapping before choosing ingestion patterns.
+- Do not invent source access behavior.
+- Do not invent storage targets.
+
+## Task
+
+Extract ingestion-relevant context.
+
+## Extract from source validation
+
+For each source:
+- source ID
+- source type
+- access method
+- validation status
+- source grain
+- freshness
+- schema stability
+- DQ risk
+- caveats
+
+## Extract from storage design
+
+For each source:
+- raw/bronze target
+- schema snapshot target
+- quarantine target
+- metadata requirements
+- retention rule
+
+## Update documents
+
+Update:
+- ingestion-design.md
+- ingestion-source-plan.md
+
+## Update frontmatter
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-source-and-storage-context

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-03-ingestion-pattern-gate.md

@@ -0,0 +1,48 @@
+# Step 03 — Ingestion Pattern Gate
+
+## Mandatory Rules
+
+- Stop at this gate.
+- Recommend, but do not decide.
+- User must approve ingestion pattern for P1 sources.
+
+## Task
+
+Run GATE-16 — Ingestion Pattern Gate.
+
+## Options
+
+A. Pull API  
+B. File download / file drop  
+C. Database query/export  
+D. CDC  
+E. Stream/event ingestion  
+F. Manual upload  
+
+## Required table
+
+| Source ID | Source Type | Access Method | Recommended Pattern | Rationale | Risk |
+|---|---|---|---|---|---|
+
+## User decision
+
+Ask user to approve pattern by source:
+
+A/B/C/D/E/F, or revise.
+
+Do not continue until user chooses.
+
+## Record
+
+Update:
+- ingestion-design.md
+- ingestion-decision-record.md
+- .decision-log.md
+- .learning-log.md if enabled
+
+Update frontmatter:
+
+```yaml
+decisionGates:
+  GATE-16-ingestion-pattern:
+    status: approved / approved-with-caveats

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-04-snapshot-incremental-and-frequency-gates.md

@@ -0,0 +1,57 @@
+# Step 04 — Snapshot, Incremental, and Frequency Gates
+
+## Mandatory Rules
+
+- Stop for user decisions.
+- Snapshot/incremental choice must be source-specific.
+- Ingestion frequency must be tied to freshness requirement.
+- Do not assume streaming unless freshness requires it.
+
+## Gate 1 — Batch vs Streaming
+
+Options:
+
+A. Batch  
+B. Micro-batch  
+C. Streaming  
+D. Event-driven  
+E. Manual/on-demand  
+
+Ask user to choose.
+
+## Gate 2 — Snapshot vs Incremental
+
+Options:
+
+A. Full snapshot  
+B. Incremental by timestamp  
+C. Incremental by cursor/token  
+D. Append-only events  
+E. Manual versioned files  
+F. Hybrid  
+
+Ask user to choose per source if needed.
+
+## Gate 3 — Frequency
+
+For each source:
+
+| Source ID | Required Freshness | Observed Freshness | Recommended Frequency | User Decision |
+|---|---|---|---|---|
+
+## Record
+
+Update:
+- ingestion-design.md
+- ingestion-decision-record.md
+- ingestion-source-plan.md
+- .decision-log.md
+
+Update frontmatter:
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-source-and-storage-context
+  - step-03-ingestion-pattern-gate
+  - step-04-snapshot-incremental-and-frequency-gates

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-05-schema-evolution-failure-replay-design.md

@@ -0,0 +1,67 @@
+# Step 05 — Schema Evolution, Failure Handling, Idempotency, Replay
+
+## Mandatory Rules
+
+- Schema evolution must be explicit.
+- Failure handling must be explicit.
+- Idempotency must be explicit.
+- Replay/backfill must be explicit.
+- If schema evolution is undefined, trigger HALT-13.
+
+## Schema Evolution Gate
+
+Options:
+
+A. Strict schema contract  
+B. Schema snapshot + drift detection  
+C. Flexible rescued fields  
+D. Manual review on change  
+E. Source-specific strategy  
+
+Ask user to choose.
+
+## Failure handling
+
+Define:
+- retry
+- backoff
+- partial load behavior
+- quarantine/dead-letter
+- manual intervention
+- alerting/logging
+
+## Idempotency
+
+Define:
+- idempotency key
+- duplicate detection
+- rerun behavior
+- run ID expectations
+
+## Replay / Backfill
+
+Define:
+- whether raw data is retained
+- whether source can be re-pulled
+- backfill method
+- replay limitations
+
+## Update
+
+Update:
+- ingestion-design.md
+- ingestion-decision-record.md
+- ingestion-source-plan.md
+
+Update frontmatter:
+
+```yaml
+stepsCompleted:
+  - step-01-init
+  - step-02-load-source-and-storage-context
+  - step-03-ingestion-pattern-gate
+  - step-04-snapshot-incremental-and-frequency-gates
+  - step-05-schema-evolution-failure-replay-design
+decisionGates:
+  GATE-11-schema-evolution:
+    status: approved / approved-with-caveats

package/src/dew-skills/3-solutioning/dew-ingestion-design/steps/step-06-finalize.md

@@ -0,0 +1,34 @@
+# Step 06 — Finalize Ingestion Design
+
+## Mandatory Rules
+
+- Do not finalize if ingestion pattern is unresolved.
+- Do not finalize if schema evolution is unresolved.
+- Do not finalize if idempotency/replay/failure handling are missing.
+- Run checklist.
+
+## Task
+
+Finalize ingestion design.
+
+## Required outputs
+
+- ingestion-design.md
+- ingestion-decision-record.md
+- ingestion-source-plan.md
+- decision log entries
+- learning log entries if enabled
+
+## Final status
+
+Set:
+
+```yaml
+status: final
+stepsCompleted:
+  - step-01-init
+  - step-02-load-source-and-storage-context
+  - step-03-ingestion-pattern-gate
+  - step-04-snapshot-incremental-and-frequency-gates
+  - step-05-schema-evolution-failure-replay-design
+  - step-06-finalize

package/src/dew-skills/3-solutioning/dew-serving-design/SKILL.md

@@ -0,0 +1,74 @@
+---
+name: dew-serving-design
+description: Design how Gold models, metrics, and data products are served to consumers with trust, caveats, contracts, and feedback loop.
+---
+
+# DEW Serving Design
+
+**Goal:** Define how the data product is served to consumers.
+
+**Your Role:** You are a data product serving facilitator.
+
+You translate Gold models, KPI definitions, DQ rules, and consumer needs into a clear serving specification.
+
+You may recommend, but you must not decide.
+
+## Workflow Architecture
+
+This workflow uses step-file architecture.
+
+Rules:
+- Read one step file at a time.
+- Read the complete step file before acting.
+- Do not skip steps.
+- Do not load future steps early.
+- Stop at decision gates.
+- Update frontmatter after each completed step.
+- Preserve caveats from KPI, source, transformation, modeling, and DQ.
+
+## Mandatory Rules
+
+- Do not design serving before Gold/data model and DQ design exist.
+- Do not expose a metric without formula, grain, caveats, and trust statement.
+- Do not hide limitations from the consumer-facing data product.
+- Do not choose serving mode silently.
+- If serving consumer is unclear, trigger `HALT-15 — Serving Consumer Unclear`.
+- If DQ rules are missing, trigger `HALT-14 — DQ Rules Missing`.
+- If privacy/security is unreviewed, trigger `HALT-16 — Security Privacy Unreviewed`.
+
+## Conventions
+
+- Bare paths resolve from the skill root.
+- `{skill-root}` resolves to this skill's installed directory.
+- `{project-root}`-prefixed paths resolve from the project working directory.
+- `{workflow.<name>}` resolves to fields in `customize.toml`'s `[workflow]` table.
+- `{doc_workspace}` is the run folder for this workflow.
+
+## On Activation
+
+1. Resolve customization:
+
+   `python3 {project-root}/_dew/scripts/resolve_customization.py --skill {skill-root} --key workflow`
+
+   If the script fails, read `{skill-root}/customize.toml` directly and use defaults.
+
+2. Execute `{workflow.activation_steps_prepend}`.
+
+3. Load persistent facts from `{workflow.persistent_facts}`.
+
+4. Load `{project-root}/_dew/dew/config.yaml` if present.
+
+5. Load:
+   - `{workflow.serving_spec_template}`
+   - `{workflow.metric_contract_template}`
+   - `{workflow.trust_limitations_template}`
+   - `{workflow.serving_mode_rubric}`
+   - `{workflow.self_service_rubric}`
+
+6. Greet user in configured language.
+
+## Execution
+
+Read fully and follow:
+
+`steps/step-01-init.md`

package/src/dew-skills/3-solutioning/dew-serving-design/assets/metric-serving-contract-template.md

@@ -0,0 +1,67 @@
+# Metric Serving Contract Template
+
+```markdown
+---
+title: "{Project Name} Metric Serving Contract"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+---
+
+# Metric Serving Contract
+
+## Purpose
+
+This contract defines how metrics are exposed to consumers.
+
+## Metric Table
+
+| KPI ID | Metric Name | Formula | Grain | Gold Model | Serving Asset | Caveat |
+|---|---|---|---|---|---|---|
+| KPI-001 | | | | gold_x | dashboard/API/mart | |
+
+## Metric Detail
+
+## KPI-001 — {Metric Name}
+
+### Business Question
+
+### Decision Supported
+
+### Consumer
+
+### Formula
+
+### Grain
+
+One row / observation represents:
+
+### Gold Model
+
+### Gold Field
+
+### Dimensions
+
+### Filters
+
+### Freshness
+
+### DQ Rules
+
+### Caveats
+
+### Trust Level
+
+### Display Name / API Name
+
+### Owner
+
+### Change Process
+
+A. Decision log update  
+B. Business rule catalog update  
+C. Metric contract versioning  
+D. Manual approval only  
+```

package/src/dew-skills/3-solutioning/dew-serving-design/assets/self-service-rubric.md

@@ -0,0 +1,28 @@
+# Self-Service Rubric
+
+## No self-service; curated outputs only
+
+Use when:
+- trust is limited
+- consumers should not derive arbitrary metrics
+- MVP scope is narrow
+
+## Limited self-service with documented metrics
+
+Use when:
+- analysts can filter/slice approved metrics
+- formulas are documented
+- dimensions are controlled
+
+## Full semantic/metric layer
+
+Use when:
+- multiple teams consume metrics
+- metric consistency matters
+- reusable definitions are required
+
+## API-first contract
+
+Use when:
+- applications consume data
+- versioning and schema contracts matter

package/src/dew-skills/3-solutioning/dew-serving-design/assets/serving-data-product-spec-template.md

@@ -0,0 +1,139 @@
+# Serving Data Product Spec Template
+
+```markdown
+---
+title: "{Project Name} Serving Data Product Spec"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+stepsCompleted: []
+decisionGates:
+  GATE-20-serving-mode:
+    status: pending
+  GATE-serving-trust-limitations:
+    status: pending
+evidence: []
+assumptions: []
+caveats: []
+---
+
+# Serving Data Product Spec: {Project Name}
+
+## 0. Document Purpose
+
+This document defines how the data product is served to users or downstream systems.
+
+It covers:
+- serving mode
+- data consumer experience
+- metric contracts
+- trust statement
+- known limitations
+- freshness
+- access expectations
+- feedback loop
+
+## 1. Input Context
+
+| Input | Path | Status | Notes |
+|---|---|---|---|
+| Data Product PRD | | Loaded/Missing | |
+| Data Model Spec | | Loaded/Missing | |
+| KPI to Model Mapping | | Loaded/Missing | |
+| DQ Rules | | Loaded/Missing | |
+| Business Rule Catalog | | Loaded/Missing | |
+| Evidence Index | | Loaded/Missing | |
+
+## 2. Data Product Summary
+
+### Product Name
+
+### Product Type
+
+### Consumers
+
+| Consumer ID | Consumer | Role | Decision Supported | Serving Need |
+|---|---|---|---|---|
+
+### Decisions Supported
+
+## 3. Serving Mode
+
+Choose one or more:
+
+A. Dashboard  
+B. Gold data mart  
+C. API  
+D. Notebook/report  
+E. Web app feature  
+F. ML feature table  
+G. Reverse ETL output  
+
+Selected:
+
+## 4. Served Assets
+
+| Asset ID | Asset Name | Type | Source Model | Consumer | Status |
+|---|---|---|---|---|---|
+| SERV-001 | | dashboard/mart/API/report | gold_x | | proposed |
+
+## 5. Metric Serving Contracts
+
+See:
+
+`metric-serving-contract.md`
+
+## 6. Freshness and Availability
+
+| Asset | Expected Freshness | Source Freshness | Ingestion Frequency | Caveat |
+|---|---|---|---|---|
+
+## 7. Trust and Limitations Statement
+
+See:
+
+`trust-and-limitations-statement.md`
+
+## 8. Consumer Experience
+
+### User Journey
+
+### Filters / Dimensions
+
+### Drill-downs
+
+### Alerts / Recommendations
+
+### Export / API Access
+
+### Documentation Shown to User
+
+## 9. Self-Service Policy
+
+A. No self-service; curated outputs only  
+B. Limited self-service with documented metrics  
+C. Full semantic/metric layer  
+D. API-first contract  
+
+Selected:
+
+## 10. Feedback Loop
+
+How consumers report:
+- metric issue
+- missing data
+- freshness problem
+- wrong business logic
+- new KPI request
+
+## 11. Handoff to Governance/Security
+
+Governance must review:
+- access control
+- privacy posture
+- metric exposure
+- documentation ownership
+- auditability
+```

package/src/dew-skills/3-solutioning/dew-serving-design/assets/serving-mode-rubric.md

@@ -0,0 +1,70 @@
+# Serving Mode Rubric
+
+## Dashboard
+
+Use when:
+- users need visual monitoring
+- metrics are consumed by analysts/business users
+- interaction/filtering is important
+
+Risk:
+- users may over-trust metrics if limitations are hidden
+
+## Gold data mart
+
+Use when:
+- analysts need queryable curated tables
+- BI or downstream analysis consumes data
+- semantic flexibility matters
+
+Risk:
+- requires documentation and metric contract
+
+## API
+
+Use when:
+- application or external system needs programmatic access
+- stable contract matters
+- latency/availability requirements exist
+
+Risk:
+- versioning and access control become important
+
+## Notebook/report
+
+Use when:
+- analysis is exploratory
+- output is periodic or narrative
+- trust level is lower
+
+Risk:
+- weak operationalization
+
+## Web app feature
+
+Use when:
+- data product is embedded in end-user workflow
+- recommendations/advisory are shown in app
+- user experience matters
+
+Risk:
+- stronger product, security, and explanation requirements
+
+## ML feature table
+
+Use when:
+- output feeds ML models
+- point-in-time correctness matters
+- training/serving consistency matters
+
+Risk:
+- feature leakage and historical correctness
+
+## Reverse ETL
+
+Use when:
+- metrics/actions are sent back to operational systems
+- downstream actions depend on data product
+
+Risk:
+- errors can affect operational users directly