deepwork 0.1.0__py3-none-any.whl → 0.1.1__py3-none-any.whl

deepwork/standard_jobs/deepwork_jobs/steps/learn.md

@@ -0,0 +1,346 @@
+# Learn from Job Execution
+
+## Objective
+
+Think deeply about this task. Reflect on the current conversation to identify learnings from DeepWork job executions, improve job instructions with generalizable insights, and capture bespoke (run-specific) learnings in AGENTS.md files in the appropriate working folder.
+
+## Task
+
+Analyze the conversation history to extract learnings and improvements, then apply them appropriately:
+- **Generalizable learnings** → Update job instruction files
+- **Bespoke learnings** (specific to this run) → Add to AGENTS.md in working folder
+
+### Step 1: Analyze Conversation for Job Executions
+
+1. **Scan the conversation** for DeepWork slash commands that were run
+   - Look for patterns like `/job_name.step_id`
+   - Identify which jobs and steps were executed
+   - Note the order of execution
+
+2. **Identify the working folder**
+   - Should be clear from conversation history where work was done
+   - If unclear, run `git diff` to see where changes were made on the branch
+
+3. **If no job was specified**, ask the user:
+   - "Which DeepWork job would you like me to learn from?"
+   - List available jobs from `.deepwork/jobs/`
+
+### Step 2: Identify Points of Confusion and Inefficiency
+
+Review the conversation for:
+
+1. **Confusion signals**
+   - Questions the agent asked that shouldn't have been necessary
+   - Misunderstandings about what a step required
+   - Incorrect outputs that needed correction
+   - Ambiguous instructions that led to wrong interpretations
+
+2. **Inefficiency signals**
+   - Extra steps or iterations that were needed
+   - Information that had to be repeated
+   - Context that was missing from instructions
+   - Dependencies that weren't clear
+
+3. **Error patterns**
+   - Failed validations and why they failed
+   - Quality criteria that were misunderstood
+   - Edge cases that weren't handled
+
+4. **Success patterns**
+   - What worked particularly well
+   - Efficient approaches worth preserving
+   - Good examples that could be added to instructions
+
+### Step 3: Classify Learnings
+
+For each learning identified, determine if it is:
+
+**Generalizable** (should improve instructions):
+- Would help ANY future run of this job
+- Addresses unclear or missing guidance
+- Fixes incorrect assumptions in instructions
+- Adds helpful examples or context
+- Examples:
+  - "Step instructions should mention that X format is required"
+  - "Quality criteria should include checking for Y"
+  - "Add example of correct output format"
+
+**Bespoke** (should go in AGENTS.md):
+- Specific to THIS project/codebase/run
+- Depends on local conventions or structure
+- References specific files or paths
+- Would not apply to other uses of this job
+- Examples:
+  - "In this codebase, API endpoints are in `src/api/`"
+  - "This project uses camelCase for function names"
+  - "The main config file is at `config/settings.yml`"
+
+### Step 4: Update Job Instructions (Generalizable Learnings)
+
+For each generalizable learning:
+
+1. **Locate the instruction file**
+   - Path: `.deepwork/jobs/[job_name]/steps/[step_id].md`
+
+2. **Make targeted improvements**
+   - Add missing context or clarification
+   - Include helpful examples
+   - Clarify ambiguous instructions
+   - Update quality criteria if needed
+
+3. **Keep instructions concise**
+   - Avoid redundancy - don't repeat the same guidance in multiple places
+   - Be direct - remove verbose explanations that don't add value
+   - Prefer bullet points over paragraphs where appropriate
+
+4. **Preserve instruction structure**
+   - Keep existing sections (Objective, Task, Process, Output Format, Quality Criteria)
+   - Add to appropriate sections rather than restructuring
+   - Maintain consistency with other steps
+
+5. **Track changes for changelog**
+   - Note what was changed and why
+   - Prepare changelog entry for job.yml
+
+### Step 4b: Extract Shared Content into Referenced Files
+
+Review all instruction files for the job and identify content that:
+- Appears in multiple step instructions (duplicated)
+- Is lengthy and could be extracted for clarity
+- Would benefit from being maintained in one place
+
+**Extract to shared files:**
+
+1. **Create shared files** in `.deepwork/jobs/[job_name]/steps/shared/`
+   - `conventions.md` - Coding/formatting conventions used across steps
+   - `examples.md` - Common examples referenced by multiple steps
+   - `schemas.md` - Data structures or formats used throughout
+
+2. **Reference from instructions** using markdown includes or explicit references:
+   ```markdown
+   ## Conventions
+
+   Follow the conventions defined in `shared/conventions.md`.
+   ```
+
+3. **Benefits of extraction:**
+   - Single source of truth - update once, applies everywhere
+   - Shorter instruction files - easier to read and maintain
+   - Consistent guidance across steps
+
+### Step 5: Create/Update AGENTS.md (Bespoke Learnings)
+
+The AGENTS.md file captures project-specific knowledge that helps future agent runs.
+
+1. **Determine the correct location**
+   - Place AGENTS.md in the working folder where job outputs live
+   - This ensures the knowledge is available when working in that context
+   - If uncertain, place at the project root
+
+2. **Use file references where possible**
+   - Instead of duplicating information, reference source files
+   - This keeps AGENTS.md in sync as the codebase evolves
+   - Pattern: "See `path/to/file.ext` for [description]"
+
+3. **AGENTS.md structure**:
+
+```markdown
+# Project Context for [Job Name]
+
+## Codebase Structure
+
+<!-- Reference files rather than duplicating content -->
+- Project structure: See `README.md` for overview
+- API documentation: See `docs/api.md`
+- Configuration: See `config/README.md`
+
+## Conventions
+
+### Naming Conventions
+- [Convention]: See example in `path/to/example.ext:LINE`
+
+### File Organization
+- [Pattern]: Reference `path/to/pattern/`
+
+## Job-Specific Context
+
+### [Job Name]
+
+#### [Step Name]
+- [Learning]: Reference `relevant/file.ext`
+- [Context]: [Brief explanation with file reference]
+
+## Known Issues and Workarounds
+
+- [Issue]: [Workaround with file reference if applicable]
+
+## Last Updated
+- Date: [YYYY-MM-DD]
+- From conversation about: [Brief description]
+```
+
+4. **Writing entries**
+   - Be concise but specific
+   - Always prefer file references over inline content
+   - Use line numbers when referencing specific code: `file.ext:42`
+   - Group related learnings together
+
+### Step 6: Update Job Version and Changelog
+
+If instruction files were modified:
+
+1. **Bump version in job.yml**
+   - Patch version (0.0.x) for instruction improvements
+   - Minor version (0.x.0) if quality criteria changed
+
+2. **Add changelog entry**
+   ```yaml
+   - version: "[new_version]"
+     changes: "Improved [step] instructions based on execution learnings: [brief description]"
+   ```
+
+### Step 7: Sync and Summarize
+
+1. **Run deepwork sync** (if instructions were modified)
+   ```bash
+   deepwork sync
+   ```
+
+2. **Create learning_summary.md** in the working folder:
+   ```markdown
+   # Learning Summary
+
+   ## Job Analyzed
+   - Job: [job_name]
+   - Steps executed: [list of steps]
+
+   ## Generalizable Improvements Made
+   - [Step]: [What was improved]
+
+   ## Bespoke Learnings Captured
+   - Location: [path to AGENTS.md]
+   - Entries added: [list of entries]
+
+   ## Files Modified
+   - [List of files changed]
+
+   ## Recommendations
+   - [Any additional suggestions]
+   ```
+
+3. **If commands were regenerated**, look at the "To use the new commands" section in the `deepwork sync` output and **relay these exact reload instructions to the user** (e.g., "Type 'exit' then run 'claude --resume'" for Claude Code)
+
+## File Reference Patterns
+
+When adding entries to AGENTS.md, prefer these patterns:
+
+**Good (references)**:
+```markdown
+- API endpoints follow REST conventions. See `src/api/routes.ts` for examples.
+- Error handling pattern: See `src/utils/errors.ts:15-30`
+- Configuration schema: Defined in `config/schema.json`
+```
+
+**Avoid (duplicating)**:
+```markdown
+- API endpoints should return JSON with this format: { status: ..., data: ... }
+- Error handling should use try/catch with custom error classes
+- Configuration requires these fields: name, version, ...
+```
+
+## Quality Criteria
+
+- Conversation has been analyzed for job executions
+- Points of confusion and inefficiency are identified
+- Learnings are correctly classified (generalizable vs bespoke)
+- Job instructions updated for generalizable improvements
+- Instructions are concise - no redundancy or unnecessary verbosity
+- Shared/lengthy content extracted into referenced files where appropriate
+- AGENTS.md created/updated with bespoke learnings
+- File references used instead of duplicating content
+- AGENTS.md is in the correct working folder
+- learning_summary.md documents all changes
+- When all criteria are met, include `<promise>✓ Quality Criteria Met</promise>`
+
+## Example Dialog
+
+```
+User: /deepwork_jobs.learn
+
+Claude: I'll analyze this conversation for learnings from DeepWork job executions.
+
+**Conversation Analysis**
+
+I found the following job executions:
+1. `/competitive_research.identify_competitors` - Completed
+2. `/competitive_research.research_competitors` - Completed with 2 iterations
+3. `/competitive_research.comparative_analysis` - Completed
+
+**Identified Issues**
+
+1. **Confusion in step 2**: The instructions didn't specify that competitor research should include pricing information. This caused an extra iteration.
+
+2. **Missing context**: The agent had to ask about the industry focus, which should have been carried forward from step 1.
+
+**Classification**
+
+- Generalizable: Add pricing to research requirements in instructions
+- Bespoke: This project focuses on B2B SaaS in the HR tech space
+
+**Actions Taken**
+
+1. Updated `.deepwork/jobs/competitive_research/steps/research_competitors.md`:
+   - Added "Include pricing information (tiers, models, approximate ranges)" to research requirements
+   - Added note about carrying forward industry context from previous step
+
+2. Created `./competitive_research_output/AGENTS.md`:
+   ```markdown
+   # Project Context for Competitive Research
+
+   ## Job-Specific Context
+
+   ### competitive_research
+
+   #### Industry Focus
+   - Domain: B2B SaaS, HR Technology sector
+   - See `identify_competitors_output.md` for market definition
+
+   #### Competitors Identified
+   - Reference: `competitor_list.md` for full details
+
+   ## Last Updated
+   - Date: 2024-01-15
+   - From conversation about: Initial competitive analysis run
+   ```
+
+3. Updated job.yml version to 1.0.1 with changelog entry
+
+4. Ran `deepwork sync`
+
+**Summary**
+
+Created `learning_summary.md` documenting all changes. To get the updated commands, type 'exit' then run 'claude --resume'.
+```
+
+## Handling Edge Cases
+
+**No job executions found**:
+- Ask user which job they'd like to analyze
+- Or offer to review available jobs
+
+**Multiple jobs executed**:
+- Analyze each job separately
+- Create separate AGENTS.md entries or files as appropriate
+
+**AGENTS.md already exists**:
+- Read existing content
+- Append new learnings to appropriate sections
+- Update "Last Updated" section
+
+**No issues found**:
+- Document what worked well
+- Consider if any successful patterns should be added to instructions as examples
+
+**Sensitive information**:
+- Never include secrets, credentials, or PII in AGENTS.md
+- Reference config files instead of including values

deepwork/standard_jobs/deepwork_jobs/steps/supplemental_file_references.md

@@ -0,0 +1,40 @@
+# Supplementary Reference Files
+
+Step instructions can include additional `.md` files in the `steps/` directory. These supplementary files are useful for:
+
+- Providing detailed examples or templates that would clutter the main instruction file
+- Sharing common reference material across multiple steps
+- Including technical specifications, API documentation, or style guides
+
+## How to Use
+
+1. Place additional `.md` files in the `steps/` directory alongside the main step instruction files
+2. Reference them in your step instructions using the **full path from the project root**
+
+## Example
+
+If you have a job called `my_job` and want to include an API specification template:
+
+1. Create the file at `.deepwork/jobs/my_job/steps/api_spec.md`
+2. Reference it in your step instructions like this:
+
+```markdown
+Use the template in `.deepwork/jobs/my_job/steps/api_spec.md` to structure your API endpoints.
+```
+
+## Path Format
+
+Always use the full relative path from the project root:
+
+```
+.deepwork/jobs/[job_name]/steps/[filename].md
+```
+
+For example:
+- `.deepwork/jobs/competitive_research/steps/competitor_template.md`
+- `.deepwork/jobs/api_design/steps/endpoint_schema.md`
+- `.deepwork/jobs/onboarding/steps/checklist_template.md`
+
+## Benefits
+
+Using supplementary files keeps your main step instructions focused and readable while allowing you to provide detailed reference material when needed. The AI agent can read these files during execution to get additional context.

deepwork/standard_jobs/deepwork_policy/hooks/{capture_work_tree.sh → capture_prompt_work_tree.sh}

@@ -1,9 +1,10 @@
 #!/bin/bash
-# capture_work_tree.sh - Captures the current git work tree state
+# capture_prompt_work_tree.sh - Captures the git work tree state at prompt submission
 #
 # This script creates a snapshot of the current git state by recording
 # all files that have been modified, added, or deleted. This baseline
-# is used later to detect what changed during an agent session.
+# is used for policies with compare_to: prompt to detect what changed
+# during an agent response (between user prompts).
 
 set -e
 

deepwork/standard_jobs/deepwork_policy/hooks/policy_stop_hook.sh

@@ -2,16 +2,13 @@
 # policy_stop_hook.sh - Evaluates policies when the agent stops
 #
 # This script is called as a Claude Code Stop hook. It:
-# 1. Gets the list of files changed during the session
-# 2. Evaluates policies from .deepwork.policy.yml
+# 1. Evaluates policies from .deepwork.policy.yml
+# 2. Computes changed files based on each policy's compare_to setting
 # 3. Checks for <promise> tags in the conversation transcript
 # 4. Returns JSON to block stop if policies need attention
-# 5. Resets the work tree baseline for the next iteration
 
 set -e
 
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
 # Check if policy file exists
 if [ ! -f .deepwork.policy.yml ]; then
     # No policies defined, nothing to do
@@ -31,16 +28,6 @@ if [ -n "${HOOK_INPUT}" ]; then
     TRANSCRIPT_PATH=$(echo "${HOOK_INPUT}" | jq -r '.transcript_path // empty' 2>/dev/null || echo "")
 fi
 
-# Get changed files
-changed_files=$("${SCRIPT_DIR}/get_changed_files.sh" 2>/dev/null || echo "")
-
-# If no files changed, nothing to evaluate
-if [ -z "${changed_files}" ]; then
-    # Reset baseline for next iteration
-    "${SCRIPT_DIR}/capture_work_tree.sh" 2>/dev/null || true
-    exit 0
-fi
-
 # Extract conversation text from the JSONL transcript
 # The transcript is JSONL format - each line is a JSON object
 # We need to extract the text content from assistant messages
@@ -57,16 +44,13 @@ fi
 # Call the Python evaluator
 # The Python module handles:
 # - Parsing the policy file
+# - Computing changed files based on each policy's compare_to setting
 # - Matching changed files against triggers/safety patterns
 # - Checking for promise tags in the conversation context
 # - Generating appropriate JSON output
 result=$(echo "${conversation_context}" | python -m deepwork.hooks.evaluate_policies \
     --policy-file .deepwork.policy.yml \
-    --changed-files "${changed_files}" \
     2>/dev/null || echo '{}')
 
-# Reset the work tree baseline for the next iteration
-"${SCRIPT_DIR}/capture_work_tree.sh" 2>/dev/null || true
-
 # Output the result (JSON for Claude Code hooks)
 echo "${result}"

deepwork/standard_jobs/deepwork_policy/hooks/user_prompt_submit.sh

@@ -1,17 +1,16 @@
 #!/bin/bash
 # user_prompt_submit.sh - Runs on every user prompt submission
 #
-# This script captures the work tree baseline if it doesn't exist yet.
-# This ensures we have a baseline to compare against when evaluating policies.
+# This script captures the work tree state at each prompt submission.
+# This baseline is used for policies with compare_to: prompt to detect
+# what changed during an agent response.
 
 set -e
 
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 
-# Only capture if no baseline exists yet (first prompt of session)
-if [ ! -f .deepwork/.last_work_tree ]; then
-    "${SCRIPT_DIR}/capture_work_tree.sh"
-fi
+# Capture work tree state at each prompt for compare_to: prompt policies
+"${SCRIPT_DIR}/capture_prompt_work_tree.sh"
 
 # Exit successfully - don't block the prompt
 exit 0

deepwork/standard_jobs/deepwork_policy/steps/define.md

@@ -56,6 +56,22 @@ If there are files that, when also changed, mean the policy shouldn't fire:
   - Trigger: `src/auth/**/*`
   - Safety: `SECURITY.md`, `docs/security_review.md`
 
+### Step 3b: Choose the Comparison Mode (Optional)
+
+The `compare_to` field controls what baseline is used when detecting "changed files":
+
+**Options:**
+- `base` (default) - Compares to the base of the current branch (merge-base with main/master). This is the most common choice for feature branches, as it shows all changes made on the branch.
+- `default_tip` - Compares to the current tip of the default branch (main/master). Useful when you want to see the difference from what's currently in production.
+- `prompt` - Compares to the state at the start of each prompt. Useful for policies that should only fire based on changes made during a single agent response.
+
+**When to use each:**
+- **base**: Best for most policies. "Did this branch change config files?" → trigger docs review
+- **default_tip**: For policies about what's different from production/main
+- **prompt**: For policies that should only consider very recent changes within the current session
+
+Most policies should use the default (`base`) and don't need to specify `compare_to`.
+
 ### Step 4: Write the Instructions
 
 Create clear, actionable instructions for what the agent should do when the policy fires.
@@ -86,6 +102,7 @@ Create or update `.deepwork.policy.yml` in the project root.
 - name: "[Friendly name for the policy]"
   trigger: "[glob pattern]"  # or array: ["pattern1", "pattern2"]
   safety: "[glob pattern]"   # optional, or array
+  compare_to: "base"         # optional: "base" (default), "default_tip", or "prompt"
   instructions: |
     [Multi-line instructions for the agent...]
 ```
@@ -95,6 +112,7 @@ Create or update `.deepwork.policy.yml` in the project root.
 - name: "[Friendly name for the policy]"
   trigger: "[glob pattern]"
   safety: "[glob pattern]"
+  compare_to: "base"         # optional
   instructions_file: "path/to/instructions.md"
 ```
 
@@ -166,7 +184,10 @@ Create or update this file at the project root with the new policy entry.
 ## Context
 
 Policies are evaluated automatically when you finish working on a task. The system:
-1. Tracks which files you changed during the session
+1. Determines which files have changed based on each policy's `compare_to` setting:
+   - `base` (default): Files changed since the branch diverged from main/master
+   - `default_tip`: Files different from the current main/master branch
+   - `prompt`: Files changed since the last prompt submission
 2. Checks if any changes match policy trigger patterns
 3. Skips policies where safety patterns also matched
 4. Prompts you with instructions for any triggered policies