dew-method 0.1.0

package/src/dew-skills/2-plan-workflows/dew-source-assessment/assets/source-scorecard-template.md

@@ -0,0 +1,146 @@
+# Source Scorecard Template
+
+```markdown
+---
+title: "{Project Name} Source Scorecard"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "2-plan-workflows"
+decisionGates:
+  GATE-08-source-trust:
+    status: pending
+evidence: []
+assumptions: []
+caveats:
+  - "Source priority here is assessment priority, not final acceptance."
+---
+
+# Source Scorecard: {Project Name}
+
+## 0. Status Note
+
+This scorecard prioritizes source validation.
+
+It does not accept sources as P1/P2. Final source acceptance happens in `dew-source-validation`.
+
+## 1. Source Assessment Summary
+
+| Source ID | Source Name | KPI Relevance | Risk | Validation Priority |
+|---|---|---|---|---|
+| SRC-001 | | High/Medium/Low | High/Medium/Low | Validate first |
+
+## 2. Source Scorecards
+
+## SRC-001 — {Source Name}
+
+### Expected Role
+
+### Related KPI(s)
+
+### Access Risk
+
+Low / Medium / High / Unknown
+
+Notes:
+
+### Ownership / Provider Clarity
+
+Clear / Partial / Unknown
+
+Notes:
+
+### Field Coverage
+
+Expected coverage:
+- Full
+- Partial
+- Unknown
+- Weak
+
+Notes:
+
+### Grain Compatibility
+
+Expected:
+- Compatible
+- Requires aggregation
+- Requires bridge
+- Incompatible
+- Unknown
+
+Notes:
+
+### Freshness Fit
+
+Expected:
+- Meets requirement
+- Might meet requirement
+- Does not meet requirement
+- Unknown
+
+Notes:
+
+### Schema Stability Risk
+
+Low / Medium / High / Unknown
+
+Notes:
+
+### Privacy / Security Risk
+
+Low / Medium / High / Unknown
+
+Notes:
+
+### License / Usage Risk
+
+Low / Medium / High / Unknown
+
+Notes:
+
+### Operational Reliability
+
+Low / Medium / High / Unknown
+
+Notes:
+
+### Validation Effort
+
+Low / Medium / High
+
+Notes:
+
+### Recommended Validation Priority
+
+- Validate first
+- Validate later
+- Exploration only
+- Reject early
+- Blocked
+
+### Rationale
+
+### Evidence Needed
+
+- Access check
+- Sample
+- Schema snapshot
+- Profiling
+- Freshness observation
+- Contract test
+
+## 3. Validation Plan
+
+| Source ID | Priority | Required Evidence | Next Skill |
+|---|---|---|---|
+| SRC-001 | Validate first | Access, sample, schema | dew-source-validation |
+
+## 4. Open Questions
+
+## 5. Next Workflow
+
+Recommended next:
+
+`dew-source-validation`s

package/src/dew-skills/2-plan-workflows/dew-source-assessment/checklist.md

@@ -0,0 +1,17 @@
+# Source Assessment Checklist
+
+- [ ] Source inventory loaded.
+- [ ] KPI catalog loaded or context provided.
+- [ ] KPI-source matrix loaded or reconstructed.
+- [ ] Each source scored for KPI relevance.
+- [ ] Each source scored for access risk.
+- [ ] Each source scored for field coverage risk.
+- [ ] Each source scored for grain risk.
+- [ ] Each source scored for freshness risk.
+- [ ] Each source scored for schema stability risk.
+- [ ] Each source scored for privacy/security risk.
+- [ ] Each source assigned validation priority.
+- [ ] No source promoted to P1/P2 yet.
+- [ ] Evidence needed listed for each validation-first source.
+- [ ] Decision log updated if decisions were made.
+- [ ] Learning log updated if learning mode is enabled.

package/src/dew-skills/2-plan-workflows/dew-source-assessment/customize.toml

@@ -0,0 +1,31 @@
+# DO NOT EDIT -- overwritten on every update.
+#
+# Workflow customization surface for dew-source-assessment.
+
+[workflow]
+
+activation_steps_prepend = []
+activation_steps_append = []
+
+persistent_facts = [
+  "file:{project-root}/**/source-inventory.md",
+  "file:{project-root}/**/kpi-catalog.md",
+  "file:{project-root}/**/kpi-source-matrix.md",
+  "file:{project-root}/**/data-product-prd.md",
+  "file:{project-root}/**/kpi-feasibility-report.md",
+  "file:{project-root}/**/.decision-log.md",
+  "file:{project-root}/**/.learning-log.md",
+]
+
+on_complete = ""
+
+source_scorecard_template = "assets/source-scorecard-template.md"
+source_risk_rubric = "assets/source-risk-rubric.md"
+source_priority_rubric = "assets/source-priority-rubric.md"
+checklist = "checklist.md"
+
+assessment_output_path = "{planning_artifacts}/source-assessment"
+run_folder_pattern = "source-assessment-{project_name}-{date}"
+
+decision_log_path = "{planning_artifacts}/.decision-log.md"
+learning_log_path = "{learning_artifacts}/.learning-log.md"

package/src/dew-skills/2-plan-workflows/dew-source-discovery/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: dew-source-discovery
+description: Discover candidate data sources and document their expected role, access method, owner, grain, and source type without accepting them yet.
+---
+
+# DEW Source Discovery
+
+**Goal:** Discover and document candidate sources for the data product without prematurely accepting them.
+
+**Your Role:** You are a source discovery facilitator.
+
+You help the user identify possible source systems, but you must not claim a source is usable until it is validated.
+
+## Mandatory Rules
+
+- Candidate source does not mean accepted source.
+- Do not mark any source as P1 in this skill.
+- Do not assume access works.
+- Do not assume schema exists or is stable.
+- Do not assume source fields match KPI required fields.
+- If no candidate source exists for a primary KPI, trigger `HALT-04 — Source Not Identified`.
+
+## Conventions
+
+- Bare paths resolve from the skill root.
+- `{skill-root}` resolves to this skill's installed directory.
+- `{project-root}`-prefixed paths resolve from the project working directory.
+- `{workflow.<name>}` resolves to fields in `customize.toml`'s `[workflow]` table.
+- `{doc_workspace}` is the run folder for this workflow.
+
+## On Activation
+
+1. Resolve customization:
+
+   `python3 {project-root}/_dew/scripts/resolve_customization.py --skill {skill-root} --key workflow`
+
+   If the script fails, read `{skill-root}/customize.toml` directly and use defaults.
+
+2. Execute `{workflow.activation_steps_prepend}`.
+
+3. Load persistent facts from `{workflow.persistent_facts}`.
+
+4. Load `{project-root}/_dew/dew/config.yaml` if present and resolve:
+   - `{user_name}`
+   - `{project_name}`
+   - `{communication_language}`
+   - `{document_output_language}`
+   - `{planning_artifacts}`
+   - `{evidence_artifacts}`
+   - `{learning_artifacts}`
+   - `{learning_mode}`
+
+5. Load:
+   - `{workflow.source_inventory_template}`
+   - `{workflow.source_candidate_rubric}`
+   - `{workflow.source_type_guide}`
+
+6. Greet `{user_name}` in `{communication_language}`.
+
+## Intent Modes
+
+### Create
+
+Create a source inventory from project brief, PRD, KPI catalog, KPI-source matrix, and user-provided context.
+
+### Update
+
+Update source inventory when new sources are discovered or existing sources are removed.
+
+### Validate
+
+Validate whether source inventory is complete enough to proceed to assessment.
+
+## Execution
+
+### Step 1 — Load context
+
+Look for:
+- project brief
+- business discovery
+- KPI catalog
+- KPI-source matrix
+- data product PRD
+- decision log
+
+### Step 2 — Identify candidate sources
+
+For each source, capture:
+- source ID
+- source name
+- source type
+- expected business role
+- expected KPI dependency
+- access method
+- owner/provider
+- expected grain
+- expected freshness
+- status = Candidate
+
+### Step 3 — Check source coverage
+
+For each primary KPI, check whether at least one candidate source exists.
+
+If no source exists for a primary KPI, trigger `HALT-04 — Source Not Identified`.
+
+### Step 4 — Create source inventory
+
+Create:
+
+`{workflow.source_output_path}/{workflow.run_folder_pattern}/source-inventory.md`
+
+### Step 5 — Finalize
+
+Run checklist.
+
+Recommend next:
+
+`dew-source-assessment`

package/src/dew-skills/2-plan-workflows/dew-source-discovery/assets/source-candidate-rubric.md

@@ -0,0 +1,33 @@
+# Source Candidate Rubric
+
+A candidate source is worth assessing when it has at least:
+
+- source name
+- source type
+- access method hypothesis
+- expected business role
+- related KPI or product requirement
+- expected grain or unknown grain
+- expected freshness or unknown freshness
+
+## Candidate source status
+
+### Candidate
+
+Use when the source is identified but not validated.
+
+### Needs discovery
+
+Use when source exists conceptually but access/provider/details are unclear.
+
+### Duplicate
+
+Use when the source overlaps another source and may not be needed.
+
+### Out of scope
+
+Use when source may be useful but not for MVP.
+
+### Reject early
+
+Use when the source is clearly irrelevant, inaccessible, or unsafe.

package/src/dew-skills/2-plan-workflows/dew-source-discovery/assets/source-inventory-template.md

@@ -0,0 +1,118 @@
+# Source Inventory Template
+
+```markdown
+---
+title: "{Project Name} Source Inventory"
+status: draft
+created: {YYYY-MM-DD}
+updated: {YYYY-MM-DD}
+workflow: dew
+phase: "2-plan-workflows"
+decisionGates:
+  GATE-07-source-access:
+    status: pending
+  GATE-08-source-trust:
+    status: pending
+evidence: []
+assumptions: []
+caveats:
+  - "Sources listed here are candidates only until validated."
+---
+
+# Source Inventory: {Project Name}
+
+## 0. Important Status Note
+
+Sources in this document are candidates.
+
+They are not accepted as P1/P2 until source validation produces evidence.
+
+## 1. Source Summary
+
+| Source ID | Source Name | Type | Expected Role | Candidate Status |
+|---|---|---|---|---|
+| SRC-001 | | API/File/DB/Stream/Web/Manual | | Candidate |
+
+## 2. Source Details
+
+## SRC-001 — {Source Name}
+
+### Source Type
+
+Choose one:
+
+- API
+- File
+- Database
+- Stream/Event
+- Web page
+- Manual upload
+- Third-party dataset
+- Internal system
+- Derived rule
+
+### Provider / Owner
+
+### Access Method
+
+Examples:
+- public API
+- authenticated API
+- CSV download
+- database query
+- GitHub raw file
+- web scrape
+- manual export
+- event stream
+
+### Expected Business Role
+
+### Related KPI(s)
+
+| KPI ID | Dependency Type | Notes |
+|---|---|---|
+| KPI-001 | Required/Optional/Reference | |
+
+### Expected Grain
+
+One record is expected to represent:
+
+### Expected Freshness
+
+### Expected Fields
+
+| Field | Expected Meaning | Related KPI | Required? |
+|---|---|---|---|
+| | | | Yes/No |
+
+### Current Status
+
+Candidate
+
+### Validation Required
+
+- Access check
+- Sample extraction
+- Schema snapshot
+- Grain confirmation
+- Freshness observation
+- Profiling/DQ notes
+- Source trust decision
+
+### Assumptions
+
+- [ASSUMPTION] ...
+
+### Open Questions
+
+## 3. KPI to Source Coverage
+
+| KPI ID | KPI Name | Candidate Source(s) | Coverage Status |
+|---|---|---|---|
+| KPI-001 | | | Unknown |
+
+## 4. Next Workflow
+
+Recommended next:
+
+`dew-source-assessment`

package/src/dew-skills/2-plan-workflows/dew-source-discovery/assets/source-type-guide.md

@@ -0,0 +1,81 @@
+# Source Type Guide
+
+## API
+
+Use when data is fetched through HTTP or SDK.
+
+Validation evidence:
+- API call log
+- sample response
+- schema snapshot
+- rate limit/auth notes
+
+## File
+
+Use when data comes from CSV, JSON, Excel, Parquet, GeoJSON, etc.
+
+Validation evidence:
+- downloaded file
+- file metadata
+- schema/header snapshot
+- sample rows
+
+## Database
+
+Use when data comes from OLTP, OLAP, warehouse, lakehouse, or queryable system.
+
+Validation evidence:
+- connection/query log
+- schema/table description
+- sample query result
+
+## Stream/Event
+
+Use when data is unbounded or event-driven.
+
+Validation evidence:
+- event sample
+- topic/stream schema
+- ordering/time semantics
+- retention/TTL notes
+
+## Web Page
+
+Use when data is embedded in public pages.
+
+Validation evidence:
+- page URL
+- extraction method
+- HTML/table sample
+- update pattern notes
+
+## Manual Upload
+
+Use when user or operator supplies files manually.
+
+Validation evidence:
+- sample file
+- upload process
+- owner
+- expected schedule
+
+## Third-party Dataset
+
+Use when data comes from external curated datasets.
+
+Validation evidence:
+- dataset metadata
+- license/terms
+- schema
+- sample
+- update frequency
+
+## Derived Rule
+
+Use when a source is a business rule, threshold, ontology, or mapping.
+
+Validation evidence:
+- rule definition
+- owner
+- version/change log
+- test cases

package/src/dew-skills/2-plan-workflows/dew-source-discovery/checklist.md

@@ -0,0 +1,19 @@
+# Source Discovery Checklist
+
+- [ ] Project brief loaded or context provided.
+- [ ] Data product PRD loaded or context provided.
+- [ ] KPI catalog loaded or context provided.
+- [ ] Candidate sources listed.
+- [ ] Each source has source ID.
+- [ ] Each source has source type.
+- [ ] Each source has expected role.
+- [ ] Each source maps to at least one KPI or product requirement.
+- [ ] Each source has access method hypothesis.
+- [ ] Each source has expected grain or marked unknown.
+- [ ] Each source has expected freshness or marked unknown.
+- [ ] Candidate status is not promoted to P1/P2.
+- [ ] KPI-source coverage table created.
+- [ ] Missing source for primary KPI triggers HALT-04.
+- [ ] Assumptions are labeled.
+- [ ] Decision log updated if decisions were made.
+- [ ] Learning log updated if learning mode is enabled.

package/src/dew-skills/2-plan-workflows/dew-source-discovery/customize.toml

@@ -0,0 +1,31 @@
+# DO NOT EDIT -- overwritten on every update.
+#
+# Workflow customization surface for dew-source-discovery.
+
+[workflow]
+
+activation_steps_prepend = []
+activation_steps_append = []
+
+persistent_facts = [
+  "file:{project-root}/**/brief.md",
+  "file:{project-root}/**/business-discovery.md",
+  "file:{project-root}/**/kpi-catalog.md",
+  "file:{project-root}/**/kpi-source-matrix.md",
+  "file:{project-root}/**/data-product-prd.md",
+  "file:{project-root}/**/.decision-log.md",
+  "file:{project-root}/**/.learning-log.md",
+]
+
+on_complete = ""
+
+source_inventory_template = "assets/source-inventory-template.md"
+source_candidate_rubric = "assets/source-candidate-rubric.md"
+source_type_guide = "assets/source-type-guide.md"
+checklist = "checklist.md"
+
+source_output_path = "{planning_artifacts}/source-discovery"
+run_folder_pattern = "source-discovery-{project_name}-{date}"
+
+decision_log_path = "{planning_artifacts}/.decision-log.md"
+learning_log_path = "{learning_artifacts}/.learning-log.md"

package/src/dew-skills/2-plan-workflows/dew-source-validation/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: dew-source-validation
+description: Validate source access, sample, schema, grain, freshness, quality, and trust before promoting sources to P1.
+---
+
+# DEW Source Validation
+
+**Goal:** Validate source systems before architecture, ingestion design, and transformation work.
+
+**Your Role:** You are a source validation facilitator.
+
+You force candidate sources to confront access, samples, schema, grain, freshness, quality, and trust evidence.
+
+## Workflow Architecture
+
+This workflow uses step-file architecture.
+
+Rules:
+- Read one step file at a time.
+- Read the complete step file before acting.
+- Do not skip steps.
+- Do not load future steps early.
+- Stop at decision gates.
+- Do not promote a source to P1 without evidence.
+- Update frontmatter after each completed step.
+
+## Mandatory Rules
+
+- Candidate source does not mean accepted source.
+- Do not promote to P1 without access/sample/schema evidence.
+- Do not assume grain.
+- Do not assume freshness.
+- Do not assume schema stability.
+- If source access is unverified, trigger `HALT-05`.
+- If source trust is unclear, trigger `HALT-06`.
+- If evidence is missing, trigger `HALT-18`.
+- The agent may recommend, but must not decide.
+
+## On Activation
+
+1. Resolve customization:
+
+   `python3 {project-root}/_dew/scripts/resolve_customization.py --skill {skill-root} --key workflow`
+
+   If the script fails, read `{skill-root}/customize.toml` directly and use defaults.
+
+2. Execute `{workflow.activation_steps_prepend}`.
+
+3. Load persistent facts from `{workflow.persistent_facts}`.
+
+4. Load `{project-root}/_dew/dew/config.yaml` if present.
+
+5. Load:
+   - `{workflow.validation_report_template}`
+   - `{workflow.source_sample_template}`
+   - `{workflow.schema_snapshot_template}`
+   - `{workflow.source_profile_template}`
+   - `{workflow.status_rubric}`
+
+6. Greet user in configured language.
+
+## Execution
+
+Read fully and follow:
+
+`steps/step-01-init.md`

package/src/dew-skills/2-plan-workflows/dew-source-validation/assets/schema-snapshot-template.md

@@ -0,0 +1,43 @@
+# Schema Snapshot Template
+
+```markdown
+# Schema Snapshot
+
+## Source
+
+Source ID:
+
+Source Name:
+
+## Snapshot Metadata
+
+Captured at:
+
+Captured from:
+
+Schema version if available:
+
+## Fields
+
+| Field Name | Data Type | Nullable? | Example Value | Meaning | Related KPI Field |
+|---|---|---|---|---|---|
+| | | Yes/No/Unknown | | | |
+
+## Schema Stability Notes
+
+- documented schema:
+- versioning:
+- change log:
+- drift risk:
+- known breaking changes:
+
+## Schema Evolution Recommendation
+
+A. Strict schema contract  
+B. Schema snapshot + drift detection  
+C. Flexible rescued fields  
+D. Manual review on change  
+
+Recommended:
+
+## Caveats