dew-method 0.1.0

package/README.md

@@ -0,0 +1,102 @@
+# DEW
+
+Data Engineering Workflow packaged as a DEW-native installable skill system.
+
+## Before npm publish
+
+Use the repo or a packed tarball locally:
+
+```bash
+git clone https://github.com/DKSang/DEW.git
+cd DEW
+npm install
+node tools/installer/dew-cli.js install --directory ../my-project
+```
+
+Or during development:
+
+```bash
+npm link
+dew install --directory ../my-project
+```
+
+## After npm publish
+
+```bash
+npx dew-method install --directory .
+```
+
+## Installed project layout
+
+`dew install` creates a DEW-native runtime layout:
+
+```text
+your-project/
+├── _dew/
+│   ├── config.yaml
+│   ├── _config/
+│   │   └── dew-help.csv
+│   ├── custom/
+│   ├── scripts/
+│   └── modules/
+│       ├── core-skills/
+│       └── dew-skills/
+│
+├── _dew-output/
+│   ├── planning-artifacts/
+│   ├── implementation-artifacts/
+│   ├── evidence-artifacts/
+│   └── learning-artifacts/
+│
+├── .agents/
+│   └── skills/
+│       ├── dew-help/
+│       ├── dew-customize/
+│       └── ...
+│
+└── docs/
+```
+
+## CLI commands
+
+```bash
+dew install --directory .
+dew install --directory . --yes
+dew install --directory . --user-name Dksang --communication-language Vietnamese
+dew list
+dew validate
+```
+
+## Local package checks
+
+```bash
+npm install
+npm run validate:package
+npm run pack:dry-run
+```
+
+## Package layout
+
+```text
+src/
+├── core-skills/
+└── dew-skills/
+
+tools/
+└── installer/
+    ├── dew-cli.js
+    ├── install.js
+    ├── list.js
+    └── validate.js
+```
+
+## Notes
+
+DEW uses its own runtime naming:
+
+- `_dew/` for runtime config, module copies, custom overrides, and generated catalog.
+- `_dew-output/` for planning, implementation, evidence, and learning artifacts.
+- `.agents/skills/` for agent-consumable skill folders.
+- `docs/` for long-term project knowledge.
+
+BMAD inspired the install experience, but DEW uses DEW-branded folders and skill names such as `dew-help`, `dew-customize`, and `dew-review-adversarial`.

package/package.json

@@ -0,0 +1,43 @@
+{
+  "$schema": "https://json.schemastore.org/package.json",
+  "name": "dew-method",
+  "version": "0.1.0",
+  "description": "Data Engineering Workflow skills packaged as a BMAD-style installable module",
+  "keywords": [
+    "data-engineering",
+    "workflow",
+    "agents",
+    "skills",
+    "bmad",
+    "dew"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/DKSang/DEW.git"
+  },
+  "license": "MIT",
+  "author": "Duong Kim Sang",
+  "main": "tools/installer/dew-cli.js",
+  "bin": {
+    "dew": "tools/installer/dew-cli.js",
+    "dew-method": "tools/installer/dew-cli.js"
+  },
+  "scripts": {
+    "dew:install": "node tools/installer/dew-cli.js install",
+    "install:dew": "node tools/installer/dew-cli.js install",
+    "list": "node tools/installer/dew-cli.js list",
+    "validate:package": "node tools/installer/dew-cli.js validate",
+    "pack:dry-run": "npm pack --dry-run"
+  },
+  "dependencies": {
+    "@clack/prompts": "^1.4.0",
+    "commander": "^14.0.0",
+    "picocolors": "^1.1.1"
+  },
+  "engines": {
+    "node": ">=20.12.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  }
+}

package/src/core-skills/dew-customize/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: dew-customize
+description: 'Authors and updates customization overrides for installed DEW skills.'
+---
+
+# DEW Customize
+
+**Goal:** Translate user intent into correctly placed TOML override files under `{project-root}/_dew/custom/`.
+
+## Scope
+
+This skill supports:
+- per-skill `[agent]` overrides
+- per-skill `[workflow]` overrides
+- sparse TOML override files
+- team overrides and user overrides
+
+## Preflight
+
+- If `{project-root}/_dew/` does not exist, say DEW is not installed and stop.
+- If `{project-root}/_dew/scripts/resolve_customization.py` exists, use it for verification.
+- If resolver is missing, continue with manual merge explanation.
+
+## Execution
+
+### Step 1 — Classify Intent
+
+Classify as:
+- Directed — specific skill and change requested.
+- Exploratory — user asks what can be customized.
+- Audit/iterate — user wants to inspect or update existing overrides.
+- Cross-cutting — change may affect multiple skills.
+
+### Step 2 — Discover Customizable Skills
+
+Scan installed skill roots for `customize.toml`.
+
+For each customizable skill, report:
+- skill name
+- surface: `[agent]` or `[workflow]`
+- exposed fields
+- existing team/user override if present
+
+### Step 3 — Determine Surface
+
+Read target skill's `customize.toml`.
+
+Heuristics:
+- Workflow-level: template swap, output path, activation hooks, persistent facts for one workflow.
+- Agent-level: persona, communication style, principles, behavior across workflows.
+
+If ambiguous, present both options and ask user to choose.
+
+### Step 4 — Compose Override
+
+Write sparse TOML only for fields being changed.
+
+Merge semantics:
+- scalars override
+- arrays append unless keyed merge is documented
+- tables deep-merge
+- keyed arrays merge by `code` or `id` when applicable
+
+Never copy entire `customize.toml` into override.
+
+### Step 5 — Placement
+
+Use:
+- `{skill-name}.toml` for team/shared override
+- `{skill-name}.user.toml` for personal override
+
+Confirm placement before writing.
+
+### Step 6 — Show, Confirm, Write, Verify
+
+1. Show full TOML.
+2. If file exists, show diff.
+3. Wait for explicit approval.
+4. Write file under `{project-root}/_dew/custom/`.
+5. Verify using resolver if available.
+
+If resolver is unavailable, manually explain base → team → user merge.
+
+## Completion
+
+Complete only when:
+- override file is written, or user explicitly aborts
+- user has seen verification or manual merge summary
+- next iteration path is clear
+
+## Cannot Help When
+
+- requested field is not exposed in `customize.toml`
+- user wants to change hidden step logic directly
+- skill has no `customize.toml`

package/src/core-skills/dew-help/SKILL.md

@@ -0,0 +1,71 @@
+---
+name: dew-help
+description: 'Analyzes current DEW state and user query to recommend the next skill(s) or answer workflow questions.'
+---
+
+# DEW Help
+
+## Purpose
+
+Help the user understand where they are in the DEW workflow and what to do next.
+
+## Desired Outcomes
+
+When this skill completes, the user should:
+
+1. Know where they are — which module and phase they are in, and what has already been completed.
+2. Know what to do next — the next recommended and/or required DEW step, with clear reasoning.
+3. Know how to invoke it — skill name, menu code, and args when available.
+4. Feel oriented, not overwhelmed.
+5. Get grounded answers to general DEW questions using the installed module catalog and docs.
+
+## Data Sources
+
+- Catalog: `{project-root}/_dew/_config/dew-help.csv` — assembled manifest of installed DEW skills.
+- Config: `{project-root}/_dew/config.yaml` and user overrides when present.
+- Artifacts: Files matching `outputs` patterns at resolved `output-location` paths.
+- Project knowledge: If `project_knowledge` resolves to an existing path, read it for grounding context.
+- Module docs: `_meta` rows in module catalogs may point to documentation.
+
+## CSV Interpretation
+
+Catalog format:
+
+```csv
+module,skill,display-name,menu-code,description,action,args,phase,preceded-by,followed-by,required,output-location,outputs
+```
+
+Phases:
+- `anytime` — available regardless of workflow state.
+- Numbered phases flow in order.
+
+Sequencing:
+- `preceded-by` — skills that should ideally complete before this one.
+- `followed-by` — skills that should ideally run after this one.
+- Format: `skill-name` or `skill-name:action`.
+
+Required gates:
+- `required=true` items must complete before moving meaningfully to later phases.
+
+Completion detection:
+- Search resolved output paths for `outputs` patterns.
+- Fuzzy-match found files to catalog rows.
+- Accept explicit user statements only when grounded in context.
+
+## Response Format
+
+For each recommended item, present:
+
+- `[menu-code]` **Display name**
+- Skill name in backticks
+- Description
+- Args if available
+- Why it is recommended now
+
+Show optional items first, then the next required item. Keep the response focused.
+
+## Constraints
+
+- Present output in `{communication_language}` when available.
+- Recommend running each skill in a fresh context window.
+- Never fabricate project-specific status.

package/src/core-skills/dew-index-docs/SKILL.md

@@ -0,0 +1,65 @@
+---
+name: dew-index-docs
+description: 'Generates or updates an index.md to reference all docs in a target folder.'
+---
+
+# DEW Index Docs
+
+**Goal:** Generate or update an `index.md` to reference all docs in a target folder.
+
+## Execution
+
+### Step 1 — Scan Directory
+
+- List all files and subdirectories in the target location.
+- Skip hidden files unless the user explicitly asks to include them.
+- HALT if the target directory does not exist or is inaccessible.
+
+### Step 2 — Group Content
+
+- Organize files by type, purpose, or subdirectory.
+- For DEW projects, prefer these groups when applicable:
+  - planning artifacts
+  - implementation artifacts
+  - evidence artifacts
+  - learning artifacts
+  - source docs
+  - runbooks
+
+### Step 3 — Generate Descriptions
+
+- Read each file enough to understand its actual purpose.
+- Create brief 3-10 word descriptions based on content, not just filename.
+- Do not guess when the content is unavailable; mark as `description unavailable`.
+
+### Step 4 — Create or Update Index
+
+Write `index.md` using relative links.
+
+## Output Format
+
+```markdown
+# Directory Index
+
+## Files
+
+- **[filename.ext](./filename.ext)** - Brief description
+
+## Subdirectories
+
+### subfolder/
+
+- **[file1.ext](./subfolder/file1.ext)** - Brief description
+```
+
+## Validation
+
+- Use relative paths starting with `./`.
+- Group similar files together.
+- Sort alphabetically within groups.
+- Keep descriptions concise but informative.
+
+## HALT Conditions
+
+- HALT if target directory does not exist or is inaccessible.
+- HALT if write permissions are unavailable.

package/src/core-skills/dew-review-adversarial/SKILL.md

@@ -0,0 +1,62 @@
+---
+name: dew-review-adversarial
+description: 'Perform a critical adversarial review and produce a findings report.'
+---
+
+# DEW Adversarial Review
+
+**Goal:** Critically review content and produce actionable findings.
+
+**Your Role:** You are a skeptical DEW reviewer. Look for what is missing, unsupported, inconsistent, or risky. Use a precise, professional tone.
+
+## Inputs
+
+- `content` — diff, spec, story, document, artifact, or pasted text to review.
+- `also_consider` optional — extra areas to keep in mind.
+
+## Execution
+
+### Step 1 — Receive Content
+
+- Load content from provided input or context.
+- Identify content type: diff, branch, uncommitted changes, document, or artifact.
+- If content is empty or unreadable, HALT.
+
+### Step 2 — Adversarial Analysis
+
+Review with skepticism.
+
+Look for:
+- missing assumptions
+- unclear scope
+- unsupported claims
+- weak evidence
+- broken dependencies
+- inconsistent terminology
+- hidden caveats
+- untestable acceptance criteria
+- missing failure handling
+- documentation gaps
+
+For DEW artifacts, also consider:
+- KPI formula and grain
+- source trust and freshness
+- schema drift
+- data quality action and evidence
+- lineage and metadata
+- serving limitations
+- release readiness
+
+### Step 3 — Present Findings
+
+For each finding include:
+- severity: Critical / High / Medium / Low
+- location or artifact area
+- issue
+- impact
+- suggested fix
+
+## HALT Conditions
+
+- HALT if content is empty or unreadable.
+- HALT if no findings are found; re-analyze once because zero findings is suspicious.

package/src/core-skills/dew-review-edge-case-hunter/SKILL.md

@@ -0,0 +1,73 @@
+---
+name: dew-review-edge-case-hunter
+description: 'Walk branching paths and boundary conditions in content, reporting only unhandled edge cases.'
+---
+
+# DEW Edge Case Hunter Review
+
+**Goal:** Find unhandled edge cases by systematically walking branches and boundaries.
+
+Do not comment on whether the content is generally good or bad. Report only missing handling.
+
+## Inputs
+
+- `content` — diff, full file, function, spec, story, or data artifact.
+- `also_consider` optional — specific risk areas to include.
+
+## Execution
+
+### Step 1 — Receive Content
+
+- Load content strictly from provided input.
+- If content is empty or undecodable, return a finding that valid content is required and stop.
+- Identify scope: diff, full file, function, spec, or artifact.
+
+### Step 2 — Exhaustive Path Analysis
+
+Walk:
+- conditionals
+- loops
+- error paths
+- early returns
+- null or empty values
+- min or max boundaries
+- missing default cases
+- invalid types
+- timeout and retry gaps
+
+For DEW/data engineering artifacts, also walk:
+- missing source rows
+- duplicate rows
+- late-arriving data
+- schema drift
+- partial source refresh
+- freshness miss
+- grain mismatch
+- DQ false positive or false negative
+- failed backfill or replay
+- serving user misinterpretation
+
+### Step 3 — Validate Completeness
+
+Revisit each edge class and keep only unhandled conditions.
+
+### Step 4 — Present Findings
+
+Return a JSON array of objects:
+
+```json
+[
+  {
+    "location": "artifact or file section",
+    "trigger_condition": "short edge condition",
+    "guard_snippet": "minimal guard or mitigation",
+    "potential_consequence": "what could go wrong"
+  }
+]
+```
+
+An empty array is valid only after exhaustive review.
+
+## HALT Conditions
+
+- If content is empty or undecodable, stop and request valid content.

package/src/core-skills/dew-shard-doc/SKILL.md

@@ -0,0 +1,59 @@
+---
+name: dew-shard-doc
+description: 'Splits large markdown documents into smaller organized files based on heading sections.'
+---
+
+# DEW Shard Document
+
+**Goal:** Split large markdown documents into smaller organized files based on heading sections.
+
+## Critical Rules
+
+- Execute steps in order.
+- Do not shard non-markdown files.
+- Preserve the original document unless the user explicitly asks for archival cleanup.
+- HALT immediately when halt conditions are met.
+
+## Execution
+
+### Step 1 — Get Source Document
+
+- Ask for the source document path if not provided.
+- Verify the file exists and is accessible.
+- Verify it has `.md` extension.
+- If not found or not markdown: HALT.
+
+### Step 2 — Get Destination Folder
+
+- Default destination: same location as source file, folder named after source file without `.md`.
+- Ask user to confirm default or provide custom destination.
+- Verify destination exists or can be created.
+- Check write permissions.
+
+### Step 3 — Execute Sharding
+
+Preferred command when available:
+
+```bash
+npx @kayvan/markdown-tree-parser explode <source-document> <destination-folder>
+```
+
+If the command is unavailable, fall back to manual sharding by level 2 headings.
+
+### Step 4 — Verify Output
+
+- Check destination folder contains sharded files.
+- Check `index.md` exists or create it.
+- Count files created.
+- HALT if no files were created.
+
+### Step 5 — Report Completion
+
+Report source path, destination path, number of files created, whether `index.md` was created, and warnings or caveats.
+
+## HALT Conditions
+
+- HALT if source file is missing.
+- HALT if source is not markdown.
+- HALT if destination cannot be written.
+- HALT if sharding produces no files.

package/src/core-skills/module-help.csv

@@ -0,0 +1,8 @@
+module,skill,display-name,menu-code,description,action,args,phase,preceded-by,followed-by,required,output-location,outputs
+Core,_meta,,,,,,,,,false,,
+Core,dew-help,DEW Help,DH,Recommend the next DEW skill and explain the current workflow state.,,,anytime,,,false,,
+Core,dew-index-docs,Index Docs,ID,Generate or update an index.md for docs and artifact folders.,,,anytime,,,false,,
+Core,dew-shard-doc,Shard Document,SD,Use when doc becomes too large to manage effectively.,,[path],anytime,,,false,,
+Core,dew-review-adversarial,Adversarial Review,AR,Use for critical quality review before finalizing deliverables.,,[path],anytime,,,false,,
+Core,dew-review-edge-case-hunter,Edge Case Hunter Review,ECH,Use alongside adversarial review for edge-case coverage.,,[path],anytime,,,false,,
+Core,dew-customize,DEW Customize,DC,Customize DEW agent or workflow behavior through TOML overrides.,,,anytime,,,false,{project-root}/_dew/custom,TOML override files

package/src/core-skills/module.yaml

@@ -0,0 +1,34 @@
+code: core
+name: "DEW Core Module"
+version: "0.1.0"
+description: "Shared DEW utilities and configuration used across installed DEW modules."
+
+header: "DEW Core Configuration"
+subheader: "Configure shared settings for DEW skills, workflows, and agents."
+
+user_name:
+  prompt: "What should DEW agents call you?"
+  scope: user
+  default: "DEW User"
+  result: "{value}"
+
+project_name:
+  prompt: "What is your data project called?"
+  default: "{directory_name}"
+  result: "{value}"
+
+communication_language:
+  prompt: "What language should DEW agents use when chatting with you?"
+  scope: user
+  default: "English"
+  result: "{value}"
+
+document_output_language:
+  prompt: "Preferred document output language?"
+  default: "English"
+  result: "{value}"
+
+output_folder:
+  prompt: "Where should DEW output files be saved?"
+  default: "_dew-output"
+  result: "{project-root}/{value}"