deepwork 0.5.1__py3-none-any.whl → 0.7.0a1__py3-none-any.whl

deepwork/standard_jobs/deepwork_jobs/steps/fix_jobs.md

@@ -0,0 +1,207 @@
+# Fix Job Definitions
+
+## Objective
+
+Update all job.yml files and step instructions in `.deepwork/jobs/` to the current DeepWork format. This step migrates deprecated fields, removes references to deleted steps, and ensures all jobs are compatible with the MCP-based workflow system.
+
+## Task
+
+Audit and repair all job definitions, migrating from legacy formats to current specifications.
+
+### Step 1: Inventory All Jobs
+
+List all jobs in the project:
+
+```bash
+ls -la .deepwork/jobs/
+```
+
+For each job directory, you'll need to check and potentially fix the `job.yml` file.
+
+### Step 1.5: Process Jobs in Parallel
+
+**For each job** (except `deepwork_jobs` which should be updated via `deepwork install`), kick off a sub-agent to audit and repair that job's `job.yml` file. The sub-agent should:
+
+1. Read the job's `job.yml` file
+2. Check for and fix all issues described in Steps 2-6 below
+3. Validate the YAML is still valid after changes
+4. Report what was changed
+
+**Run sub-agents in parallel** - one for each job to speed up the process.
+
+**Example prompt for sub-agent:**
+```
+Audit and repair the job at `.deepwork/jobs/[job_name]/job.yml`:
+1. Remove any `exposed: true` fields from steps
+2. Migrate `stop_hooks` to `hooks.after_agent` format
+3. Remove references to deleted steps (like `review_job_spec`)
+4. Fix orphaned steps by adding them to workflows
+5. Bump version and add changelog entry if changes were made
+6. Validate YAML syntax
+
+Report what changes were made.
+```
+
+### Step 2: Remove `exposed` Field
+
+The `exposed` field on steps no longer has any effect in MCP-based DeepWork. Steps are now only accessible through workflows.
+
+**Find and remove:**
+```yaml
+steps:
+  - id: some_step
+    exposed: true  # REMOVE THIS LINE
+```
+
+If a step was `exposed: true` and is not in any workflow, it should either:
+1. Be added to a workflow, OR
+2. Be removed from the job entirely
+
+### Step 3: Migrate `stop_hooks` to `hooks.after_agent`
+
+The `stop_hooks` field is deprecated. Migrate to the new `hooks` structure:
+
+**Before (deprecated):**
+```yaml
+steps:
+  - id: my_step
+    stop_hooks:
+      - prompt: "Verify the output meets quality standards"
+```
+
+**After (current format):**
+```yaml
+steps:
+  - id: my_step
+    hooks:
+      after_agent:
+        - prompt: "Verify the output meets quality standards"
+```
+
+### Step 4: Remove References to Deleted Steps
+
+Check for references to steps that no longer exist in the standard jobs:
+
+**Steps that have been removed:**
+- `review_job_spec` - Was removed from `deepwork_jobs` in v1.0.1
+
+**What to fix:**
+- Remove from workflow `steps` arrays
+- Update `from_step` references in inputs
+- Update `dependencies` arrays
+
+**Example fix:**
+```yaml
+# Before
+workflows:
+  - name: new_job
+    steps:
+      - define
+      - review_job_spec  # REMOVE
+      - implement
+
+steps:
+  - id: implement
+    inputs:
+      - file: job.yml
+        from_step: review_job_spec  # CHANGE TO: define
+    dependencies:
+      - review_job_spec  # CHANGE TO: define
+```
+
+### Step 5: Fix Orphaned Steps
+
+Steps not included in any workflow cannot be invoked via the MCP interface.
+
+**How to handle orphaned steps depends on whether the job has ANY workflows defined:**
+
+#### Case A: Job has NO workflows defined
+
+If the job has no `workflows:` section at all (or it's empty), create a **single workflow with the same name as the job** containing all steps in their defined order:
+
+```yaml
+# For a job named "my_job" with steps: step_a, step_b, step_c
+workflows:
+  - name: my_job  # Same name as the job
+    summary: "Runs the complete my_job workflow"
+    steps:
+      - step_a
+      - step_b
+      - step_c
+```
+
+This preserves the original intent of the job as a sequential workflow.
+
+#### Case B: Job has SOME workflows defined
+
+If the job already has one or more workflows defined, but some steps are not included in any of them, create a **separate single-step workflow for each orphaned step** with the same name as the step:
+
+```yaml
+# Existing workflows stay as-is, add new ones for orphans
+workflows:
+  - name: existing_workflow
+    summary: "..."
+    steps: [...]
+
+  # Add for each orphaned step:
+  - name: orphaned_step_name  # Same name as the step
+    summary: "Runs the orphaned_step_name step"
+    steps:
+      - orphaned_step_name
+```
+
+This ensures all steps remain accessible via the MCP interface while preserving the existing workflow structure.
+
+### Step 6: Update Version Numbers
+
+If you made significant changes to a job, bump its version number:
+
+```yaml
+# Bump patch version for minor fixes
+version: "1.0.0"  ->  version: "1.0.1"
+
+# Add changelog entry
+changelog:
+  - version: "1.0.1"
+    changes: "Migrated to current DeepWork format; removed deprecated fields"
+```
+
+## Common Issues and Fixes
+
+### Issue: Step references non-existent step in `from_step`
+```
+Error: Step 'implement' has file input from 'review_job_spec' but 'review_job_spec' is not in dependencies
+```
+**Fix:** Update `from_step` to reference a step that still exists.
+
+### Issue: Workflow references non-existent step
+```
+Error: Workflow 'new_job' references non-existent step 'review_job_spec'
+```
+**Fix:** Remove the step from the workflow's `steps` array.
+
+### Issue: Orphaned step warning
+```
+Warning: Job 'my_job' has steps not included in any workflow: standalone_step
+```
+**Fix:**
+- If the job has NO workflows: Create one workflow named `my_job` with all steps in order
+- If the job has SOME workflows: Add a `standalone_step` workflow containing just that step
+
+## Jobs to Check
+
+For each job in `.deepwork/jobs/`, check:
+
+| Check | What to Look For |
+|-------|------------------|
+| `exposed` field | Remove from all steps |
+| `stop_hooks` | Migrate to `hooks.after_agent` |
+| Workflow steps | Remove references to deleted steps |
+| Dependencies | Update to valid step IDs |
+| File inputs | Update `from_step` references |
+| Version | Bump if changes were made |
+
+## Important Notes
+
+1. **Preserve custom logic** - When migrating hooks, preserve the prompt content
+2. **Test after changes** - Validate YAML syntax after each job fix to catch errors early

deepwork/standard_jobs/deepwork_jobs/steps/fix_settings.md

@@ -0,0 +1,177 @@
+# Fix Settings Files
+
+## Objective
+
+Clean up `.claude/settings.json` and related configuration files, removing legacy artifacts from prior DeepWork versions. This step ensures the Claude Code settings are free of deprecated permissions, duplicate hooks, and hardcoded paths.
+
+## Task
+
+Audit and repair the `.claude/settings.json` file, removing gunk accumulated from older DeepWork implementations.
+
+### Step 1: Create Backup
+
+Before making any changes, create a backup:
+
+```bash
+cp .claude/settings.json .claude/settings.json.backup
+```
+
+### Step 2: Inventory DeepWork Jobs
+
+First, get the list of jobs that exist in `.deepwork/jobs/`:
+
+```bash
+ls .deepwork/jobs/
+```
+
+Note these job names - you will use them to identify which `Skill(...)` entries to remove.
+
+### Step 3: Remove DeepWork Skill Permissions
+
+Look for and **remove** `Skill(...)` permission entries that match DeepWork jobs. Only remove entries where the skill name matches a job in `.deepwork/jobs/`.
+
+**What to look for:**
+```json
+"permissions": {
+  "allow": [
+    "Skill(deepwork_jobs)",           // Remove if 'deepwork_jobs' is in .deepwork/jobs/
+    "Skill(deepwork_jobs.define)",    // Remove - matches job_name.step pattern
+    "Skill(competitive_research)",    // Remove if 'competitive_research' is in .deepwork/jobs/
+    "Skill(my_custom_skill)",         // KEEP - not a DeepWork job
+    ...
+  ]
+}
+```
+
+**IMPORTANT:** Only remove skills that:
+- Exactly match a job name in `.deepwork/jobs/` (e.g., `Skill(job_name)`)
+- Match the pattern `job_name.step_name` where `job_name` is in `.deepwork/jobs/`
+
+**DO NOT remove** skills that don't match DeepWork jobs - the user may have created these manually for other purposes.
+
+### Step 4: Remove Duplicate Hooks
+
+Check for duplicate hook entries in the `hooks` section. Prior versions sometimes added the same hook multiple times.
+
+**Example of duplicates to consolidate:**
+```json
+"hooks": {
+  "UserPromptSubmit": [
+    {
+      "matcher": "",
+      "hooks": [{ "type": "command", "command": "some_command" }]
+    },
+    {
+      "matcher": "",
+      "hooks": [{ "type": "command", "command": "some_command" }]  // DUPLICATE
+    }
+  ]
+}
+```
+
+Keep only one instance of each unique hook.
+
+### Step 5: Remove Hardcoded User Paths
+
+Search for and remove any hardcoded paths that reference specific user directories:
+
+**Patterns to find and remove:**
+- `/Users/username/.local/pipx/venvs/deepwork/bin/python`
+- `/home/username/.local/...`
+- Any path containing a specific username
+
+These should either be removed or replaced with relative paths.
+
+### Step 6: Remove DeepWork Rules Hooks (Fully Deprecated)
+
+DeepWork Rules have been completely removed from the system. Remove ALL hooks related to rules:
+
+**Hooks to remove entirely:**
+- Any hook with command `deepwork hook rules_check`
+- Any hook with command containing `rules_check`
+- Any hook referencing `.deepwork/jobs/deepwork_rules/hooks/`
+- Any hook referencing `.deepwork/rules/`
+
+**Also remove these permissions if present:**
+- `Skill(deepwork_rules)`
+- `Skill(deepwork_rules.define)`
+- `Bash(rm -rf .deepwork/tmp/rules/queue/*.json)`
+
+### Step 7: Remove Other Deprecated Commands
+
+Remove hooks referencing other deprecated DeepWork commands:
+
+**Commands to remove:**
+- `deepwork hook *` - The entire hook subcommand is deprecated
+- References to any `.deepwork/jobs/*/hooks/` scripts
+
+### Step 8: Clean Up Empty Sections
+
+If after cleanup any sections are empty, consider removing them:
+
+```json
+// Remove if empty:
+"hooks": {
+  "Stop": []  // Remove this empty array
+}
+```
+
+### Step 9: Validate JSON
+
+After all edits, ensure the file is valid JSON:
+
+```bash
+python -c "import json; json.load(open('.claude/settings.json'))"
+```
+
+If there are syntax errors, fix them before proceeding.
+
+## Example Before/After
+
+### Before (with gunk):
+```json
+{
+  "hooks": {
+    "UserPromptSubmit": [
+      { "matcher": "", "hooks": [{ "type": "command", "command": ".deepwork/jobs/deepwork_rules/hooks/user_prompt_submit.sh" }] },
+      { "matcher": "", "hooks": [{ "type": "command", "command": ".deepwork/jobs/deepwork_rules/hooks/user_prompt_submit.sh" }] }
+    ],
+    "Stop": [
+      { "matcher": "", "hooks": [{ "type": "command", "command": "deepwork hook rules_check" }] }
+    ],
+    "SubagentStop": [
+      { "matcher": "", "hooks": [{ "type": "command", "command": "/Users/tyler/.local/pipx/venvs/deepwork/bin/python -m deepwork.hooks.rules_check" }] }
+    ]
+  },
+  "permissions": {
+    "allow": [
+      "Skill(competitive_research)",
+      "Skill(competitive_research.discover_competitors)",
+      "Skill(deepwork_jobs)",
+      "Skill(deepwork_jobs.define)",
+      "Read(./.deepwork/**)",
+      "WebSearch"
+    ]
+  }
+}
+```
+
+### After (cleaned):
+```json
+{
+  "hooks": {},
+  "permissions": {
+    "allow": [
+      "Read(./.deepwork/**)",
+      "WebSearch"
+    ]
+  }
+}
+```
+
+## Important Notes
+
+1. **Don't remove non-DeepWork permissions** - Keep permissions like `WebSearch`, `Read(...)`, `Bash(...)` that aren't related to old DeepWork skills
+2. **Preserve `make_new_job.sh`** - Keep any `Bash(...)` permission referencing `make_new_job.sh` (e.g., `Bash(.deepwork/jobs/deepwork_jobs/scripts/make_new_job.sh *)`) - this is a current DeepWork script
+3. **Be conservative** - If unsure whether something is legacy, ask the user
+4. **Document changes** - Note what was removed for the final summary

deepwork/standard_jobs/deepwork_jobs/steps/implement.md

@@ -2,37 +2,16 @@
 
 ## Objective
 
-Generate the DeepWork job directory structure and instruction files for each step based on the validated `job.yml` specification from the review_job_spec step.
+Generate step instruction files for each step based on the `job.yml` specification from the define step.
 
 ## Task
 
-Read the `job.yml` specification file and create all the necessary files to make the job functional, including directory structure and step instruction files. Then sync the commands to make them available.
+Read the `job.yml` specification file created by the define step and generate comprehensive instruction files for each step. The define step has already created the job directory structure.
 
-### Step 1: Create Directory Structure Using Script
-
-Run the `make_new_job.sh` script to create the standard directory structure:
-
-```bash
-.deepwork/jobs/deepwork_jobs/make_new_job.sh [job_name]
-```
-
-This creates:
-- `.deepwork/jobs/[job_name]/` - Main job directory
-- `.deepwork/jobs/[job_name]/steps/` - Step instruction files
-- `.deepwork/jobs/[job_name]/hooks/` - Custom validation scripts (with .gitkeep)
-- `.deepwork/jobs/[job_name]/templates/` - Example file formats (with .gitkeep)
-- `.deepwork/jobs/[job_name]/AGENTS.md` - Job management guidance
-
-**Note**: If the directory already exists (e.g., job.yml was created by define step), you can skip this step or manually create the additional directories:
-```bash
-mkdir -p .deepwork/jobs/[job_name]/hooks .deepwork/jobs/[job_name]/templates
-touch .deepwork/jobs/[job_name]/hooks/.gitkeep .deepwork/jobs/[job_name]/templates/.gitkeep
-```
-
-### Step 2: Read and Validate the Specification
+### Step 1: Read and Validate the Specification
 
 1. **Locate the job.yml file**
-   - Read `.deepwork/jobs/[job_name]/job.yml` from the review_job_spec step
+   - Read `.deepwork/jobs/[job_name]/job.yml` from the define step
    - Parse the YAML content
 
 2. **Validate the specification**
@@ -46,7 +25,7 @@ touch .deepwork/jobs/[job_name]/hooks/.gitkeep .deepwork/jobs/[job_name]/templat
    - List of all steps with their details
    - Understand the workflow structure
 
-### Step 3: Generate Step Instruction Files
+### Step 2: Generate Step Instruction Files
 
 For each step in the job.yml, create a comprehensive instruction file at `.deepwork/jobs/[job_name]/steps/[step_id].md`.
 
@@ -71,11 +50,11 @@ For each step in the job.yml, create a comprehensive instruction file at `.deepw
 6. **Align with stop hooks** - If the step has `stop_hooks` defined, ensure the quality criteria in the instruction file match the validation criteria in the hooks
 7. **Ask structured questions** - When a step has user inputs, the instructions MUST explicitly tell the agent to "ask structured questions" using the AskUserQuestion tool to gather that information. Never use generic phrasing like "ask the user" - always use "ask structured questions"
 
-### Handling Stop Hooks
+### Handling Quality Hooks
 
-If a step in the job.yml has `stop_hooks` defined, the generated instruction file should:
+If a step in the job.yml has `hooks.after_agent` defined, the generated instruction file should:
 
-1. **Mirror the quality criteria** - The "Quality Criteria" section should match what the stop hooks will validate
+1. **Mirror the quality criteria** - The "Quality Criteria" section should match what the hooks will validate
 2. **Be explicit about success** - Help the agent understand when the step is truly complete
 3. **Include the promise pattern** - Mention that `<promise>✓ Quality Criteria Met</promise>` should be included when criteria are met
 
@@ -83,12 +62,13 @@ If a step in the job.yml has `stop_hooks` defined, the generated instruction fil
 ```yaml
 - id: research_competitors
   name: "Research Competitors"
-  stop_hooks:
-    - prompt: |
-        Verify the research meets criteria:
-        1. Each competitor has at least 3 data points
-        2. Sources are cited
-        3. Information is current (within last year)
+  hooks:
+    after_agent:
+      - prompt: |
+          Verify the research meets criteria:
+          1. Each competitor has at least 3 data points
+          2. Sources are cited
+          3. Information is current (within last year)
 ```
 
 **The instruction file should include:**
@@ -109,86 +89,11 @@ Step instructions can include additional `.md` files in the `steps/` directory f
 
 See `.deepwork/jobs/deepwork_jobs/steps/supplemental_file_references.md` for detailed documentation and examples.
 
-### Step 4: Verify job.yml Location
-
-Verify that `job.yml` is in the correct location at `.deepwork/jobs/[job_name]/job.yml`. The define and review_job_spec steps should have created and validated it. If for some reason it's not there, you may need to create or move it.
-
-### Step 5: Sync Skills
-
-Run `deepwork sync` to generate the skills for this job:
-
-```bash
-deepwork sync
-```
-
-This will:
-- Parse the job definition
-- Generate skills for each step
-- Make the skills available in `.claude/skills/` (or appropriate platform directory)
-
-### Step 6: Relay Reload Instructions
-
-After running `deepwork sync`, look at the "To use the new skills" section in the output. **Relay these exact reload instructions to the user** so they know how to pick up the new skills. Don't just reference the sync output - tell them directly what they need to do (e.g., "Type 'exit' then run 'claude --resume'" for Claude Code, or "Run '/memory refresh'" for Gemini CLI).
-
-### Step 7: Consider Rules for the New Job
-
-After implementing the job, consider whether there are **rules** that would help enforce quality or consistency when working with this job's domain.
-
-**What are rules?**
-
-Rules are automated guardrails stored as markdown files in `.deepwork/rules/` that trigger when certain files change during an AI session. They help ensure:
-- Documentation stays in sync with code
-- Team guidelines are followed
-- Architectural decisions are respected
-- Quality standards are maintained
+### Step 3: Verify Files
 
-**When to suggest rules:**
-
-Think about the job you just implemented and ask:
-- Does this job produce outputs that other files depend on?
-- Are there documentation files that should be updated when this job's outputs change?
-- Are there quality checks or reviews that should happen when certain files in this domain change?
-- Could changes to the job's output files impact other parts of the project?
-
-**Examples of rules that might make sense:**
-
-| Job Type | Potential Rule |
-|----------|----------------|
-| API Design | "Update API docs when endpoint definitions change" |
-| Database Schema | "Review migrations when schema files change" |
-| Competitive Research | "Update strategy docs when competitor analysis changes" |
-| Feature Development | "Update changelog when feature files change" |
-| Configuration Management | "Update install guide when config files change" |
-
-**How to offer rule creation:**
-
-If you identify one or more rules that would benefit the user, explain:
-1. **What the rule would do** - What triggers it and what action it prompts
-2. **Why it would help** - How it prevents common mistakes or keeps things in sync
-3. **What files it would watch** - The trigger patterns
-
-Then ask the user:
-
-> "Would you like me to create this rule for you? I can run `/deepwork_rules.define` to set it up."
-
-If the user agrees, invoke the `/deepwork_rules.define` command to guide them through creating the rule.
-
-**Example dialogue:**
-
-```
-Based on the competitive_research job you just created, I noticed that when
-competitor analysis files change, it would be helpful to remind you to update
-your strategy documentation.
-
-I'd suggest a rule like:
-- **Name**: "Update strategy when competitor analysis changes"
-- **Trigger**: `**/positioning_report.md`
-- **Action**: Prompt to review and update `docs/strategy.md`
-
-Would you like me to create this rule? I can run `/deepwork_rules.define` to set it up.
-```
-
-**Note:** Not every job needs rules. Only suggest them when they would genuinely help maintain consistency or quality. Don't force rules where they don't make sense.
+Verify that all files are in their correct locations:
+- `job.yml` at `.deepwork/jobs/[job_name]/job.yml` (created by define step)
+- Step instruction files at `.deepwork/jobs/[job_name]/steps/[step_id].md`
 
 ## Example Implementation
 
@@ -205,34 +110,13 @@ For a complete worked example showing a job.yml and corresponding step instructi
 5. **Use context** - The job description provides valuable context for each step
 6. **Be specific** - Tailor instructions to the specific step, not generic advice
 
-## Validation Before Sync
-
-Before running `deepwork sync`, verify:
-- All directories exist
-- `job.yml` is in place
-- All step instruction files exist (one per step)
-- No file system errors
-
 ## Completion Checklist
 
 Before marking this step complete, ensure:
-- [ ] job.yml validated and copied to job directory
+- [ ] job.yml validated and in job directory
 - [ ] All step instruction files created
 - [ ] Each instruction file is complete and actionable
-- [ ] `deepwork sync` executed successfully
-- [ ] Skills generated in platform directory
-- [ ] User informed to follow reload instructions from `deepwork sync`
-- [ ] Considered whether rules would benefit this job (Step 7)
-- [ ] If rules suggested, offered to run `/deepwork_rules.define`
 
-## Quality Criteria
+## Note: Workflow Availability
 
-- Job directory structure is correct
-- All instruction files are complete (not stubs)
-- Instructions are specific and actionable
-- Output examples are provided in each instruction file
-- Quality criteria defined for each step
-- Steps with user inputs explicitly use "ask structured questions" phrasing
-- Sync completed successfully
-- Skills available for use
-- Thoughtfully considered relevant rules for the job domain
+Once the job.yml and step instruction files are created, the workflow is immediately available through the DeepWork MCP server. The MCP server reads job definitions directly from `.deepwork/jobs/` - no separate sync or installation step is required.