dew-method 0.1.0

package/src/dew-skills/3-solutioning/dew-dataops-design/assets/dataops-runbook-template.md

@@ -0,0 +1,124 @@
+# DataOps Runbook Template
+
+```markdown
+---
+title: "{Project Name} DataOps Runbook"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "3-solutioning"
+stepsCompleted: []
+decisionGates:
+  GATE-22-operational-readiness:
+    status: pending
+evidence: []
+assumptions: []
+caveats: []
+---
+
+# DataOps Runbook: {Project Name}
+
+## 0. Document Purpose
+
+This runbook defines how the data product is operated.
+
+It covers:
+- operational readiness
+- run schedule
+- monitoring
+- alerting
+- incident handling
+- replay/backfill
+- change process
+- environments
+- CI/CD handoff
+
+## 1. Input Context
+
+| Input | Path | Status |
+|---|---|---|
+| Storage Design | | |
+| Ingestion Design | | |
+| Transformation Design | | |
+| Data Model Spec | | |
+| DQ Rules | | |
+| Serving Spec | | |
+| Governance Review | | |
+
+## 2. Operational Mode
+
+Choose one:
+
+A. Manual / learning mode  
+B. Scheduled internal pipeline  
+C. Monitored internal data product  
+D. SLA-backed production pipeline  
+
+Selected:
+
+## 3. Run Schedule
+
+| Pipeline / Asset | Frequency | Trigger | Owner | Notes |
+|---|---|---|---|---|
+
+## 4. Operational Dependencies
+
+| Dependency | Type | Failure Impact | Mitigation |
+|---|---|---|---|
+
+## 5. Monitoring Summary
+
+See:
+
+`observability-plan.md`
+
+## 6. Incident Handling
+
+### Incident Types
+
+- source access failure
+- schema drift
+- DQ failure
+- freshness miss
+- transformation failure
+- serving failure
+- security/access incident
+
+### Incident Process
+
+## 7. Replay / Backfill
+
+## 8. Change Process
+
+Change types:
+- source schema change
+- business rule change
+- KPI definition change
+- DQ rule change
+- serving contract change
+- access policy change
+
+## 9. Environment Strategy
+
+- local
+- dev
+- test
+- prod
+- cloud workspace
+- notebook/runtime
+- CI runner
+
+## 10. CI/CD Handoff
+
+## 11. Open Operational Risks
+
+## 12. Handoff to Epics and Stories
+
+Implementation stories must include:
+- operational evidence
+- DQ evidence
+- logs
+- runbook updates
+- incident/backfill notes
+```

package/src/dew-skills/3-solutioning/dew-dataops-design/assets/incident-backfill-rubric.md

@@ -0,0 +1,38 @@
+# Incident and Backfill Rubric
+
+## Incident process must define
+
+- how incident is detected
+- who is notified
+- how severity is classified
+- how data serving is paused or caveated
+- how fix is verified
+- how incident is documented
+
+## Backfill process must define
+
+- source of truth for historical data
+- raw data availability
+- date range selection
+- idempotency behavior
+- downstream recomputation
+- DQ validation after backfill
+- serving refresh after backfill
+
+## Severity
+
+### Critical
+
+Data product is misleading, unsafe, or unavailable for primary consumer.
+
+### High
+
+Core KPI/source/DQ/freshness issue affects decisions.
+
+### Medium
+
+Partial issue with caveated impact.
+
+### Low
+
+Minor documentation or non-critical issue.

package/src/dew-skills/3-solutioning/dew-dataops-design/assets/monitoring-alerting-rubric.md

@@ -0,0 +1,44 @@
+# Monitoring and Alerting Rubric
+
+## Minimal monitoring
+
+Use when:
+- learning/portfolio project
+- manual runs
+- low trust expectation
+
+Monitor:
+- run success/failure
+- basic row counts
+- obvious errors
+
+## Internal monitoring
+
+Use when:
+- internal decision-support
+- scheduled pipeline
+- DQ matters
+
+Monitor:
+- source access
+- schema drift
+- freshness
+- row counts
+- critical DQ rules
+- output availability
+
+## Production monitoring
+
+Use when:
+- trusted shared or production-grade product
+- multiple consumers
+- SLA/contract exists
+
+Monitor:
+- pipeline runtime
+- latency
+- freshness SLA
+- DQ SLA
+- incident alerts
+- access/security events
+- cost/resource usage

package/src/dew-skills/3-solutioning/dew-dataops-design/assets/observability-plan-template.md

@@ -0,0 +1,45 @@
+# Observability Plan Template
+
+```markdown
+# Observability Plan
+
+## Purpose
+
+Define what the project must monitor.
+
+## Monitoring Table
+
+| Monitor ID | Area | Signal | Threshold | Severity | Action |
+|---|---|---|---|---|---|
+| MON-001 | Source access | API call success | fail | High | alert/retry |
+| MON-002 | Freshness | latest data date | > expected lag | High | warn/fail |
+| MON-003 | DQ | critical DQ fail | > 0 | Critical | fail pipeline |
+| MON-004 | Row count | row count anomaly | threshold | Medium | warn |
+| MON-005 | Schema | schema drift | changed | High | block/review |
+
+## Log Requirements
+
+Each run should log:
+- run ID
+- start/end time
+- source
+- row counts
+- schema version
+- DQ result
+- errors
+- output paths
+
+## Alerting
+
+| Alert | Trigger | Recipient | Action |
+|---|---|---|---|
+
+## Evidence Outputs
+
+| Evidence | Path | Produced By |
+|---|---|---|
+| ingestion run log | | |
+| DQ result | | |
+| freshness report | | |
+| schema drift report | | |
+```

package/src/dew-skills/3-solutioning/dew-dataops-design/assets/operational-readiness-report-template.md

@@ -0,0 +1,37 @@
+# Operational Readiness Report Template
+
+```markdown
+# Operational Readiness Report
+
+## Summary
+
+Operational readiness status:
+
+- ready
+- ready with caveats
+- not ready
+- manual only
+
+## Readiness Matrix
+
+| Area | Status | Evidence / Design | Caveat |
+|---|---|---|---|
+| Schedule | | | |
+| Monitoring | | | |
+| Alerting | | | |
+| Incident process | | | |
+| Replay/backfill | | | |
+| Change process | | | |
+| Access control | | | |
+| Documentation | | | |
+| Owner | | | |
+
+## Final Decision
+
+A. Ready for implementation stories  
+B. Ready with operational caveats  
+C. Not ready; return to design  
+D. Manual/learning mode only  
+
+User decision:
+```

package/src/dew-skills/3-solutioning/dew-dataops-design/checklist.md

@@ -0,0 +1,21 @@
+# DataOps Design Checklist
+
+- [ ] Storage design loaded.
+- [ ] Ingestion design loaded.
+- [ ] Transformation design loaded.
+- [ ] Data model spec loaded.
+- [ ] DQ rules loaded.
+- [ ] Serving design loaded.
+- [ ] Governance/security review loaded.
+- [ ] Operational mode selected by user.
+- [ ] Run schedule defined.
+- [ ] Monitoring plan defined.
+- [ ] Alerting plan defined.
+- [ ] Incident process defined.
+- [ ] Replay/backfill process defined.
+- [ ] Change process defined.
+- [ ] Environment strategy defined.
+- [ ] CI/CD handoff defined.
+- [ ] Operational readiness report created.
+- [ ] Decision log updated.
+- [ ] Learning log updated if enabled.

package/src/dew-skills/3-solutioning/dew-dataops-design/customize.toml

@@ -0,0 +1,39 @@
+# DO NOT EDIT -- overwritten on every update.
+#
+# Workflow customization surface for dew-dataops-design.
+
+[workflow]
+
+activation_steps_prepend = []
+activation_steps_append = []
+
+persistent_facts = [
+  "file:{project-root}/**/storage-design.md",
+  "file:{project-root}/**/ingestion-design.md",
+  "file:{project-root}/**/transformation-design.md",
+  "file:{project-root}/**/data-model-spec.md",
+  "file:{project-root}/**/data-quality-rules.md",
+  "file:{project-root}/**/serving-data-product-spec.md",
+  "file:{project-root}/**/governance-security-review.md",
+  "file:{project-root}/**/access-control-matrix.md",
+  "file:{project-root}/**/validation-evidence-index.md",
+  "file:{project-root}/**/.decision-log.md",
+  "file:{project-root}/**/.learning-log.md",
+]
+
+on_complete = ""
+
+dataops_runbook_template = "assets/dataops-runbook-template.md"
+observability_plan_template = "assets/observability-plan-template.md"
+operational_readiness_report_template = "assets/operational-readiness-report-template.md"
+monitoring_alerting_rubric = "assets/monitoring-alerting-rubric.md"
+incident_backfill_rubric = "assets/incident-backfill-rubric.md"
+ci_cd_environment_rubric = "assets/ci-cd-environment-rubric.md"
+checklist = "checklist.md"
+
+dataops_output_path = "{planning_artifacts}/dataops-design"
+run_folder_pattern = "dataops-design-{project_name}-{date}"
+
+decision_log_path = "{planning_artifacts}/.decision-log.md"
+learning_log_path = "{learning_artifacts}/.learning-log.md"
+evidence_index_path = "{evidence_artifacts}/validation-evidence-index.md"

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-01-init.md

@@ -0,0 +1,62 @@
+# Step 01 — DataOps Design Initialization
+
+## Mandatory Rules
+
+- Do not design DataOps before serving and governance design exist.
+- Do not mark implementation ready before operational readiness gate.
+- Read the complete step before acting.
+
+## Task
+
+Initialize DataOps Design workflow.
+
+## Sequence
+
+### 1. Detect existing workflow
+
+Look for:
+- `{workflow.dataops_output_path}/**/dataops-runbook.md`
+- `{planning_artifacts}/**/*dataops*.md`
+- `{planning_artifacts}/**/*runbook*.md`
+
+If unfinished workflow exists, offer to resume.
+
+### 2. Validate prerequisites
+
+Look for:
+- `serving-data-product-spec.md`
+- `governance-security-review.md`
+- `data-quality-rules.md`
+- `ingestion-design.md`
+
+If missing:
+- missing serving → return to `dew-serving-design`
+- missing governance → return to `dew-governance-security-design`
+- missing DQ → `HALT-14`
+- missing ingestion → `HALT-12`
+
+### 3. Create workspace
+
+Bind `{doc_workspace}` to:
+
+`{workflow.dataops_output_path}/{workflow.run_folder_pattern}/`
+
+Create:
+- `dataops-runbook.md`
+- `observability-plan.md`
+- `operational-readiness-report.md`
+
+Use templates.
+
+### 4. Initialize frontmatter
+
+Set:
+
+```yaml
+status: draft
+stepsCompleted:
+  - step-01-init
+decisionGates:
+  GATE-22-operational-readiness:
+    status: pending
+```

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-02-load-pipeline-serving-governance-context.md

@@ -0,0 +1,52 @@
+# Step 02 — Load Pipeline, Serving, and Governance Context
+
+## Mandatory Rules
+
+- Load all operationally relevant designs before readiness decision.
+- Do not invent owners, schedules, or monitoring that are not supported by design.
+
+## Task
+
+Extract DataOps context.
+
+## Extract
+
+From ingestion:
+- source plans
+- frequency
+- schema evolution
+- failure handling
+- replay/backfill
+
+From transformation/model:
+- model dependencies
+- Gold outputs
+- KPI calculations
+
+From DQ:
+- critical rules
+- evidence requirements
+- DQ gate policy
+
+From serving:
+- serving assets
+- freshness expectation
+- trust limitations
+
+From governance:
+- access controls
+- privacy posture
+- ownership/audit requirements
+
+## Update
+
+Update:
+- dataops-runbook.md
+- observability-plan.md
+- operational-readiness-report.md
+
+## Continue checkpoint
+
+Ask:
+
+`[C] Continue to operational readiness gate`

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-03-operational-readiness-gate.md

@@ -0,0 +1,47 @@
+# Step 03 — Operational Readiness Gate
+
+## Mandatory Rules
+
+- Stop at this gate.
+- User must approve operational mode.
+- Do not mark implementation ready without operational decision.
+
+## Options
+
+A. Manual / learning mode  
+B. Scheduled internal pipeline  
+C. Monitored internal data product  
+D. SLA-backed production pipeline  
+
+## Required table
+
+| Mode | Fit | Pros | Cons | Required Monitoring | Risk |
+|---|---|---|---|---|---|
+
+## User decision
+
+Ask user to choose A/B/C/D.
+
+Do not continue until user chooses.
+
+## Record
+
+Update:
+- dataops-runbook.md
+- operational-readiness-report.md
+- .decision-log.md
+- .learning-log.md if enabled
+
+Update frontmatter:
+
+```yaml
+decisionGates:
+  GATE-22-operational-readiness:
+    status: approved / approved-with-caveats
+```
+
+## Continue checkpoint
+
+Ask:
+
+`[C] Continue to observability monitoring alerting`

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-04-observability-monitoring-alerting.md

@@ -0,0 +1,42 @@
+# Step 04 — Observability, Monitoring, and Alerting
+
+## Mandatory Rules
+
+- Monitoring must match operational mode and trust expectation.
+- Critical DQ rules require observable evidence.
+- Freshness must be monitored if it is part of the trust statement.
+
+## Task
+
+Define observability plan.
+
+## Required monitors
+
+Consider:
+- source access success
+- ingestion row count
+- schema drift
+- freshness
+- critical DQ failures
+- transformation success
+- Gold output availability
+- serving asset refresh
+- access/security events
+- cost/resource usage if relevant
+
+## Required table
+
+| Monitor ID | Area | Signal | Threshold | Severity | Action |
+|---|---|---|---|---|---|
+
+## Update
+
+Update:
+- observability-plan.md
+- dataops-runbook.md
+
+## Continue checkpoint
+
+Ask:
+
+`[C] Continue to incident backfill and change process`

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-05-incident-backfill-and-change-process.md

@@ -0,0 +1,56 @@
+# Step 05 — Incident, Backfill, and Change Process
+
+## Mandatory Rules
+
+- Incidents must have owner/action path or caveat.
+- Backfill must respect idempotency and raw retention.
+- Change process must cover schema, KPI, business rules, DQ, serving contract.
+
+## Task
+
+Define operational processes.
+
+## Incident process
+
+Define:
+- detection
+- severity
+- owner
+- response
+- communication
+- serving caveat/update
+- resolution evidence
+
+## Backfill process
+
+Define:
+- source of truth
+- raw availability
+- date range selection
+- rerun behavior
+- downstream recomputation
+- DQ validation
+- serving refresh
+
+## Change process
+
+Define for:
+- source schema change
+- source freshness/access change
+- KPI formula change
+- business rule change
+- DQ rule change
+- serving contract change
+- access policy change
+
+## Update
+
+Update:
+- dataops-runbook.md
+- operational-readiness-report.md
+
+## Continue checkpoint
+
+Ask:
+
+`[C] Continue to CI CD environments and handoff`

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-06-ci-cd-environments-and-handoff.md

@@ -0,0 +1,55 @@
+# Step 06 — CI/CD, Environments, and Handoff
+
+## Mandatory Rules
+
+- Environment strategy must be explicit.
+- Implementation stories must inherit operational evidence requirements.
+- Do not over-design production CI/CD for learning mode unless user chooses it.
+
+## Task
+
+Define environment and implementation handoff.
+
+## Environment strategy
+
+Options:
+A. Local/manual only  
+B. Local + lightweight CI  
+C. Dev/test/prod environments  
+D. Cloud workspace deployment pipeline  
+E. Hybrid local-first + cloud target  
+
+Ask user if current operational mode requires a choice.
+
+## CI/CD handoff
+
+Define:
+- what tests must run
+- source contract tests
+- DQ tests
+- lint/format if relevant
+- deployment steps
+- evidence generated by CI
+- manual approvals
+
+## Implementation handoff
+
+Stories must include:
+- run logs
+- DQ evidence
+- freshness evidence
+- schema drift evidence
+- incident/backfill notes
+- runbook updates
+
+## Update
+
+Update:
+- dataops-runbook.md
+- operational-readiness-report.md
+
+## Continue checkpoint
+
+Ask:
+
+`[C] Continue to finalize DataOps design`

package/src/dew-skills/3-solutioning/dew-dataops-design/steps/step-07-finalize.md

@@ -0,0 +1,37 @@
+# Step 07 — Finalize DataOps Design
+
+## Mandatory Rules
+
+- Do not finalize if operational readiness gate is unresolved.
+- Do not finalize if critical monitoring is missing for selected operational mode.
+- Run checklist.
+
+## Required outputs
+
+- dataops-runbook.md
+- observability-plan.md
+- operational-readiness-report.md
+- decision log entries
+- learning log entries if enabled
+
+## Final status
+
+Set:
+
+```yaml
+status: final
+stepsCompleted:
+  - step-01-init
+  - step-02-load-pipeline-serving-governance-context
+  - step-03-operational-readiness-gate
+  - step-04-observability-monitoring-alerting
+  - step-05-incident-backfill-and-change-process
+  - step-06-ci-cd-environments-and-handoff
+  - step-07-finalize
+```
+
+## Next workflow
+
+Recommend:
+
+`dew-create-epics-and-stories`