deepwork 0.3.1__py3-none-any.whl → 0.4.0__py3-none-any.whl

deepwork/standard_jobs/deepwork_jobs/doc_specs/job_spec.md

@@ -0,0 +1,190 @@
+---
+name: "DeepWork Job Specification"
+description: "YAML specification file that defines a multi-step workflow job for AI agents"
+path_patterns:
+  - ".deepwork/jobs/*/job.yml"
+target_audience: "AI agents executing jobs and developers defining workflows"
+frequency: "Created once per job, updated as workflow evolves"
+quality_criteria:
+  - name: Valid Identifier
+    description: "Job name must be lowercase with underscores, no spaces or special characters (e.g., `competitive_research`, `monthly_report`)"
+  - name: Semantic Version
+    description: "Version must follow semantic versioning format X.Y.Z (e.g., `1.0.0`, `2.1.3`)"
+  - name: Concise Summary
+    description: "Summary must be under 200 characters and clearly describe what the job accomplishes"
+  - name: Rich Description
+    description: "Description must be multi-line and explain: the problem solved, the process, expected outcomes, and target users"
+  - name: Changelog Present
+    description: "Must include a changelog array with at least the initial version entry. Changelog should only include one entry per branch at most"
+  - name: Complete Steps
+    description: "Each step must have: id (lowercase_underscores), name, description, instructions_file, outputs (at least one), and dependencies array"
+  - name: Valid Dependencies
+    description: "Dependencies must reference existing step IDs with no circular references"
+  - name: Input Consistency
+    description: "File inputs with `from_step` must reference a step that is in the dependencies array"
+  - name: Output Paths
+    description: "Outputs must be valid filenames or paths within the main repo (not in dot-directories). Use specific, descriptive paths that lend themselves to glob patterns, e.g., `competitive_research/competitors_list.md` or `competitive_research/[competitor_name]/research.md`. Avoid generic names like `output.md`."
+  - name: Concise Instructions
+    description: "The content of the file, particularly the description, must not have excessively redundant information. It should be concise and to the point given that extra tokens will confuse the AI."
+---
+
+# DeepWork Job Specification: [job_name]
+
+A `job.yml` file defines a complete multi-step workflow that AI agents can execute. Each job breaks down a complex task into reviewable steps with clear inputs and outputs.
+
+## Required Fields
+
+### Top-Level Metadata
+
+```yaml
+name: job_name                    # lowercase, underscores only
+version: "1.0.0"                  # semantic versioning
+summary: "Brief description"      # max 200 characters
+description: |                    # detailed multi-line explanation
+  [Explain what this workflow does, why it exists,
+  what outputs it produces, and who should use it]
+```
+
+### Changelog
+
+```yaml
+changelog:
+  - version: "1.0.0"
+    changes: "Initial job creation"
+  - version: "1.1.0"
+    changes: "Added quality validation hooks"
+```
+
+### Steps Array
+
+```yaml
+steps:
+  - id: step_id                   # unique, lowercase_underscores
+    name: "Human Readable Name"
+    description: "What this step accomplishes"
+    instructions_file: steps/step_id.md
+    inputs:
+      # User-provided inputs:
+      - name: param_name
+        description: "What the user provides"
+      # File inputs from previous steps:
+      - file: output.md
+        from_step: previous_step_id
+    outputs:
+      - competitive_research/competitors_list.md           # descriptive path
+      - competitive_research/[competitor_name]/research.md # parameterized path
+      # With doc spec reference:
+      - file: competitive_research/final_report.md
+        doc_spec: .deepwork/doc_specs/report_type.md
+    dependencies:
+      - previous_step_id          # steps that must complete first
+```
+
+## Optional Fields
+
+### Exposed Steps
+
+```yaml
+steps:
+  - id: learn
+    exposed: true                 # Makes step available without running dependencies
+```
+
+### Quality Hooks
+
+```yaml
+steps:
+  - id: step_id
+    hooks:
+      after_agent:
+        # Inline prompt for quality validation:
+        - prompt: |
+            Verify the output meets criteria:
+            1. [Criterion 1]
+            2. [Criterion 2]
+            If ALL criteria are met, include `<promise>...</promise>`.
+        # External prompt file:
+        - prompt_file: hooks/quality_check.md
+        # Script for programmatic validation:
+        - script: hooks/run_tests.sh
+```
+
+### Stop Hooks (Legacy)
+
+```yaml
+steps:
+  - id: step_id
+    stop_hooks:
+      - prompt: "Validation prompt..."
+      - prompt_file: hooks/check.md
+      - script: hooks/validate.sh
+```
+
+## Validation Rules
+
+1. **No circular dependencies**: Step A cannot depend on Step B if Step B depends on Step A
+2. **File inputs require dependencies**: If a step uses `from_step: X`, then X must be in its dependencies
+3. **Unique step IDs**: No two steps can have the same id
+4. **Valid file paths**: Output paths must not contain invalid characters and should be in the main repo (not dot-directories)
+5. **Instructions files exist**: Each `instructions_file` path should have a corresponding file created
+
+## Example: Complete Job Specification
+
+```yaml
+name: competitive_research
+version: "1.0.0"
+summary: "Systematic competitive analysis workflow"
+description: |
+  A comprehensive workflow for analyzing competitors in your market segment.
+  Helps product teams understand the competitive landscape through systematic
+  identification, research, comparison, and positioning recommendations.
+
+  Produces:
+  - Vetted competitor list
+  - Research notes per competitor
+  - Comparison matrix
+  - Strategic positioning report
+
+changelog:
+  - version: "1.0.0"
+    changes: "Initial job creation"
+
+steps:
+  - id: identify_competitors
+    name: "Identify Competitors"
+    description: "Identify 5-7 key competitors in the target market"
+    instructions_file: steps/identify_competitors.md
+    inputs:
+      - name: market_segment
+        description: "The market segment to analyze"
+      - name: product_category
+        description: "The product category"
+    outputs:
+      - competitive_research/competitors_list.md
+    dependencies: []
+
+  - id: research_competitors
+    name: "Research Competitors"
+    description: "Deep dive research on each identified competitor"
+    instructions_file: steps/research_competitors.md
+    inputs:
+      - file: competitive_research/competitors_list.md
+        from_step: identify_competitors
+    outputs:
+      - competitive_research/[competitor_name]/research.md
+    dependencies:
+      - identify_competitors
+
+  - id: positioning_report
+    name: "Positioning Report"
+    description: "Strategic positioning recommendations"
+    instructions_file: steps/positioning_report.md
+    inputs:
+      - file: competitive_research/[competitor_name]/research.md
+        from_step: research_competitors
+    outputs:
+      - file: competitive_research/positioning_report.md
+        doc_spec: .deepwork/doc_specs/positioning_report.md
+    dependencies:
+      - research_competitors
+```

deepwork/standard_jobs/deepwork_jobs/job.yml

@@ -1,6 +1,6 @@
 name: deepwork_jobs
-version: "0.5.0"
-summary: "DeepWork job management commands"
+version: "0.9.0"
+summary: "Creates and manages multi-step AI workflows. Use when defining, implementing, or improving DeepWork jobs."
 description: |
   Core commands for managing DeepWork jobs. These commands help you define new multi-step
   workflows and learn from running them.
@@ -24,21 +24,34 @@ changelog:
     changes: "Removed implementation_summary and learning_summary outputs; simplified step outputs"
   - version: "0.5.0"
     changes: "Standardized on 'ask structured questions' phrasing for user input; Updated quality criteria hooks to verify phrase usage; Added guidance in implement.md to use phrase in generated instructions"
+  - version: "0.6.0"
+    changes: "Added doc spec support; define.md now detects document-oriented workflows and guides doc spec creation; learn.md now identifies and applies doc spec-related improvements"
+  - version: "0.7.0"
+    changes: "Added job.yml doc spec; define step now outputs job.yml with doc_spec reference for quality validation"
+  - version: "0.8.0"
+    changes: "Added review_job_spec step between define and implement for doc spec-based quality validation using sub-agent review"
+  - version: "0.9.0"
+    changes: "Improved skill descriptions with third-person voice and 'Use when...' triggers for better discoverability"
 
 steps:
   - id: define
     name: "Define Job Specification"
-    description: "Create the job.yml specification file by understanding workflow requirements"
+    description: "Creates a job.yml specification by gathering workflow requirements through structured questions. Use when starting a new multi-step workflow."
     instructions_file: steps/define.md
     inputs:
       - name: job_purpose
         description: "What complex task or workflow are you trying to accomplish?"
     outputs:
-      - job.yml
+      - file: job.yml
+        doc_spec: .deepwork/doc_specs/job_spec.md
     dependencies: []
     quality_criteria:
       - "**User Understanding**: Did the agent fully understand the user's workflow by asking structured questions?"
       - "**Structured Questions Used**: Did the agent ask structured questions (using the AskUserQuestion tool) to gather user input?"
+      - "**Document Detection**: For document-oriented workflows, did the agent detect patterns and offer doc spec creation?"
+      - "**doc spec Created (if applicable)**: If a doc spec was needed, was it created in `.deepwork/doc_specs/[doc_spec_name].md` with proper quality criteria?"
+      - "**doc spec References**: Are document outputs properly linked to their doc specs using `{file, doc_spec}` format?"
+      - "**Valid Against doc spec**: Does the job.yml conform to the job.yml doc spec quality criteria (valid identifier, semantic version, concise summary, rich description, complete steps, valid dependencies)?"
       - "**Clear Inputs/Outputs**: Does every step have clearly defined inputs and outputs?"
       - "**Logical Dependencies**: Do step dependencies make sense and avoid circular references?"
       - "**Concise Summary**: Is the summary under 200 characters and descriptive?"
@@ -46,17 +59,35 @@ steps:
       - "**Valid Schema**: Does the job.yml follow the required schema (name, version, summary, steps)?"
       - "**File Created**: Has the job.yml file been created in `.deepwork/jobs/[job_name]/job.yml`?"
 
+  - id: review_job_spec
+    name: "Review Job Specification"
+    description: "Reviews job.yml against quality criteria using a sub-agent for unbiased validation. Use after defining a job specification."
+    instructions_file: steps/review_job_spec.md
+    inputs:
+      - file: job.yml
+        from_step: define
+    outputs:
+      - file: job.yml
+        doc_spec: .deepwork/doc_specs/job_spec.md
+    dependencies:
+      - define
+    quality_criteria:
+      - "**Sub-Agent Used**: Was a sub-agent spawned to provide unbiased review?"
+      - "**All doc spec Criteria Evaluated**: Did the sub-agent assess all 9 quality criteria?"
+      - "**Findings Addressed**: Were all failed criteria addressed by the main agent?"
+      - "**Validation Loop Complete**: Did the review-fix cycle continue until all criteria passed?"
+
   - id: implement
     name: "Implement Job Steps"
-    description: "Generate instruction files for each step based on the job.yml specification"
+    description: "Generates step instruction files and syncs slash commands from the job.yml specification. Use after job spec review passes."
     instructions_file: steps/implement.md
     inputs:
       - file: job.yml
-        from_step: define
+        from_step: review_job_spec
     outputs:
       - steps/
     dependencies:
-      - define
+      - review_job_spec
     quality_criteria:
       - "**Directory Structure**: Is `.deepwork/jobs/[job_name]/` created correctly?"
       - "**Complete Instructions**: Are ALL step instruction files complete (not stubs or placeholders)?"
@@ -70,7 +101,7 @@ steps:
 
   - id: learn
     name: "Learn from Job Execution"
-    description: "Reflect on conversation to improve job instructions and capture learnings"
+    description: "Analyzes conversation history to improve job instructions and capture learnings. Use after running a job to refine it."
     instructions_file: steps/learn.md
     exposed: true
     inputs:
@@ -85,6 +116,8 @@ steps:
       - "**Instructions Improved**: Were job instructions updated to address identified issues?"
       - "**Instructions Concise**: Are instructions free of redundancy and unnecessary verbosity?"
       - "**Shared Content Extracted**: Is lengthy/duplicated content extracted into referenced files?"
+      - "**doc spec Reviewed (if applicable)**: For jobs with doc spec outputs, were doc spec-related learnings identified?"
+      - "**doc spec Updated (if applicable)**: Were doc spec files updated with improved quality criteria or structure?"
       - "**Bespoke Learnings Captured**: Were run-specific learnings added to AGENTS.md?"
       - "**File References Used**: Do AGENTS.md entries reference other files where appropriate?"
       - "**Working Folder Correct**: Is AGENTS.md in the correct working folder for the job?"

deepwork/standard_jobs/deepwork_jobs/steps/define.md

@@ -31,6 +31,55 @@ Start by asking structured questions to understand what the user wants to accomp
    - What are the distinct stages from start to finish?
    - Are there any dependencies between phases?
 
+### Step 1.5: Detect Document-Oriented Workflows
+
+**Check for document-focused patterns** in the user's description:
+- Keywords: "report", "summary", "document", "create", "monthly", "quarterly", "for stakeholders", "for leadership"
+- Final deliverable is a specific document (e.g., "AWS spending report", "competitive analysis", "sprint summary")
+- Recurring documents with consistent structure
+
+**If a document-oriented workflow is detected:**
+
+1. Inform the user: "This workflow produces a specific document type. I recommend defining a doc spec first to ensure consistent quality."
+
+2. Ask structured questions to understand if they want to:
+   - Create a doc spec for this document
+   - Use an existing doc spec (if any exist in `.deepwork/doc_specs/`)
+   - Skip doc spec and proceed with simple outputs
+
+### Step 1.6: Define the Doc Spec (if needed)
+
+When creating a doc spec, gather the following information:
+
+1. **Document Identity**
+   - What is the document called? (e.g., "Monthly AWS Spending Report")
+   - Brief description of its purpose
+   - Where should these documents be stored? (path patterns like `finance/aws-reports/*.md`)
+
+2. **Audience and Context**
+   - Who reads this document? (target audience)
+   - How often is it produced? (frequency)
+
+3. **Quality Criteria** (3-5 criteria, each with name and description)
+   Examples for a spending report:
+   - **Visualization**: Must include charts showing spend breakdown by service
+   - **Variance Analysis**: Must compare current month against previous with percentages
+   - **Action Items**: Must include recommended cost optimization actions
+
+4. **Document Structure**
+   - What sections should it have?
+   - Any required elements (tables, charts, summaries)?
+
+### Step 1.7: Create the doc spec File (if needed)
+
+Create the doc spec file at `.deepwork/doc_specs/[doc_spec_name].md`:
+
+**Template reference**: See `.deepwork/jobs/deepwork_jobs/templates/doc_spec.md.template` for the standard structure.
+
+**Complete example**: See `.deepwork/jobs/deepwork_jobs/templates/doc_spec.md.example` for a fully worked example.
+
+After creating the doc spec, proceed to Step 2 with the doc spec reference for the final step's output.
+
 ### Step 2: Define Each Step
 
 For each major phase they mentioned, ask structured questions to gather details:
@@ -53,6 +102,9 @@ For each major phase they mentioned, ask structured questions to gather details:
    - Should outputs be organized in subdirectories? (e.g., `reports/`, `data/`, `drafts/`)
    - Will other steps need this output?
 
+   **Important**: Output paths should always be within the main repository directory structure, not in dot-directories like `.deepwork/`. Dot-directories are for configuration and job definitions, not for job outputs. Use paths like `research/competitors/report.md` rather than `.deepwork/outputs/report.md`.
+   - **Does this output have a doc spec?** If a doc spec was created in Step 1.6/1.7, reference it for the appropriate output
+
 4. **Step Dependencies**
    - Which previous steps must complete before this one?
    - Are there any ordering constraints?
@@ -64,6 +116,18 @@ For each major phase they mentioned, ask structured questions to gather details:
 
 **Note**: You're gathering this information to understand what instructions will be needed, but you won't create the instruction files yet - that happens in the `implement` step.
 
+#### Doc Spec-Aware Output Format
+
+When a step produces a document with a doc spec reference, use this format in job.yml:
+
+```yaml
+outputs:
+  - file: reports/monthly_spending.md
+    doc_spec: .deepwork/doc_specs/monthly_aws_report.md
+```
+
+The doc spec's quality criteria will automatically be included in the generated skill, ensuring consistent document quality.
+
 ### Capability Considerations
 
 When defining steps, identify any that require specialized tools:
@@ -158,6 +222,8 @@ This creates:
 
 (Where `[job_name]` is the name of the NEW job you're creating, e.g., `competitive_research`)
 
+**Doc Spec**: See `.deepwork/doc_specs/job_spec.md` for the complete specification with quality criteria.
+
 **Template reference**: See `.deepwork/jobs/deepwork_jobs/templates/job.yml.template` for the standard structure.
 
 **Complete example**: See `.deepwork/jobs/deepwork_jobs/templates/job.yml.example` for a fully worked example.
@@ -277,7 +343,7 @@ Claude: Great! Creating the job.yml specification now...
 - .deepwork/jobs/competitive_research/job.yml
 
 **Next step:**
-Run `/deepwork_jobs.implement` to generate the instruction files for each step based on this specification.
+Run `/deepwork_jobs.review_job_spec` to validate the specification against quality criteria.
 ```
 
 ## Important Guidelines
@@ -317,7 +383,7 @@ The complete YAML specification file (example shown in Step 5 above).
 After creating the file:
 1. Inform the user that the specification is complete
 2. Recommend that they review the job.yml file
-3. Tell them to run `/deepwork_jobs.implement` next
+3. Tell them to run `/deepwork_jobs.review_job_spec` next
 
 ## Quality Criteria
 

deepwork/standard_jobs/deepwork_jobs/steps/implement.md

@@ -2,7 +2,7 @@
 
 ## Objective
 
-Generate the DeepWork job directory structure and instruction files for each step based on the `job.yml` specification created in the previous step.
+Generate the DeepWork job directory structure and instruction files for each step based on the validated `job.yml` specification from the review_job_spec step.
 
 ## Task
 
@@ -32,7 +32,7 @@ touch .deepwork/jobs/[job_name]/hooks/.gitkeep .deepwork/jobs/[job_name]/templat
 ### Step 2: Read and Validate the Specification
 
 1. **Locate the job.yml file**
-   - Read `.deepwork/jobs/[job_name]/job.yml` from the define step
+   - Read `.deepwork/jobs/[job_name]/job.yml` from the review_job_spec step
    - Parse the YAML content
 
 2. **Validate the specification**
@@ -111,7 +111,7 @@ See `.deepwork/jobs/deepwork_jobs/steps/supplemental_file_references.md` for det
 
 ### Step 4: Verify job.yml Location
 
-Verify that `job.yml` is in the correct location at `.deepwork/jobs/[job_name]/job.yml`. The define step should have created it there. If for some reason it's not there, you may need to create or move it.
+Verify that `job.yml` is in the correct location at `.deepwork/jobs/[job_name]/job.yml`. The define and review_job_spec steps should have created and validated it. If for some reason it's not there, you may need to create or move it.
 
 ### Step 5: Sync Skills
 

deepwork/standard_jobs/deepwork_jobs/steps/learn.md

@@ -2,13 +2,13 @@
 
 ## Objective
 
-Think deeply about this task. Reflect on the current conversation to identify learnings from DeepWork job executions, improve job instructions with generalizable insights, and capture bespoke (run-specific) learnings in AGENTS.md files in the appropriate working folder.
+Think deeply about this task. Reflect on the current conversation to identify learnings from DeepWork job executions, improve job instructions with generalizable insights, and capture bespoke (run-specific) learnings in AGENTS.md files in the deepest common folder that would contain all work on the topic in the future.
 
 ## Task
 
 Analyze the conversation history to extract learnings and improvements, then apply them appropriately:
 - **Generalizable learnings** → Update job instruction files
-- **Bespoke learnings** (specific to this run) → Add to AGENTS.md in working folder
+- **Bespoke learnings** (specific to this run) → Add to AGENTS.md in the deepest common folder for the topic
 
 ### Step 1: Analyze Conversation for Job Executions
 
@@ -17,7 +17,8 @@ Analyze the conversation history to extract learnings and improvements, then app
    - Identify which jobs and steps were executed
    - Note the order of execution
 
-2. **Identify the working folder**
+2. **Identify the target folder**
+   - This should be the deepest common folder that would contain all work on the topic in the future
    - Should be clear from conversation history where work was done
    - If unclear, run `git diff` to see where changes were made on the branch
 
@@ -65,6 +66,15 @@ For each learning identified, determine if it is:
   - "Quality criteria should include checking for Y"
   - "Add example of correct output format"
 
+**doc spec-Related** (should improve doc spec files):
+- Improvements to document quality criteria
+- Changes to document structure or format
+- Updated audience or frequency information
+- Examples:
+  - "The report should include a summary table"
+  - "Quality criterion 'Visualization' needs clearer requirements"
+  - "Documents need a section for action items"
+
 **Bespoke** (should go in AGENTS.md):
 - Specific to THIS project/codebase/run
 - Depends on local conventions or structure
@@ -75,6 +85,30 @@ For each learning identified, determine if it is:
   - "This project uses camelCase for function names"
   - "The main config file is at `config/settings.yml`"
 
+### Step 3.5: Identify doc spec-Related Learnings
+
+Review the conversation for doc spec-related improvements:
+
+1. **Quality Criteria Changes**
+   - Were any quality criteria unclear or insufficient?
+   - Did the agent repeatedly fail certain criteria?
+   - Are there new criteria that should be added?
+
+2. **Document Structure Changes**
+   - Did the user request different sections?
+   - Were parts of the document format confusing?
+   - Should the example document be updated?
+
+3. **Metadata Updates**
+   - Has the target audience changed?
+   - Should frequency or path patterns be updated?
+
+**Signals for doc spec improvements:**
+- User asked for changes to document format
+- Repeated validation failures on specific criteria
+- Feedback about missing sections or information
+- Changes to how documents are organized/stored
+
 ### Step 4: Update Job Instructions (Generalizable Learnings)
 
 For each generalizable learning:
@@ -128,12 +162,47 @@ Review all instruction files for the job and identify content that:
    - Shorter instruction files - easier to read and maintain
    - Consistent guidance across steps
 
+### Step 4.5: Update doc spec Files (doc spec-Related Learnings)
+
+If doc spec-related learnings were identified:
+
+1. **Locate the doc spec file**
+   - Find doc spec references in job.yml outputs (look for `doc_spec: .deepwork/doc_specs/[doc_spec_name].md`)
+   - doc spec files are at `.deepwork/doc_specs/[doc_spec_name].md`
+
+2. **Update quality_criteria array**
+   - Add new criteria with name and description
+   - Modify existing criteria descriptions for clarity
+   - Remove criteria that are no longer relevant
+
+3. **Update example document**
+   - Modify the markdown body to reflect structure changes
+   - Ensure the example matches updated criteria
+
+4. **Update metadata as needed**
+   - target_audience: If audience has changed
+   - frequency: If production cadence has changed
+   - path_patterns: If storage location has changed
+
+**Example doc spec update:**
+```yaml
+# Before
+quality_criteria:
+  - name: Visualization
+    description: Include charts
+
+# After
+quality_criteria:
+  - name: Visualization
+    description: Include Mermaid.js charts showing spend breakdown by service and month-over-month trend
+```
+
 ### Step 5: Create/Update AGENTS.md (Bespoke Learnings)
 
 The AGENTS.md file captures project-specific knowledge that helps future agent runs.
 
 1. **Determine the correct location**
-   - Place AGENTS.md in the working folder where job outputs live
+   - Place AGENTS.md in the deepest common folder that would contain all work on the topic in the future
    - This ensures the knowledge is available when working in that context
    - If uncertain, place at the project root
 
@@ -201,7 +270,7 @@ When adding entries to AGENTS.md, prefer these patterns:
 - Shared/lengthy content extracted into referenced files where appropriate
 - AGENTS.md created/updated with bespoke learnings
 - File references used instead of duplicating content
-- AGENTS.md is in the correct working folder
+- AGENTS.md is in the correct folder (the deepest common folder for the topic)
 - When all criteria are met, include `<promise>✓ Quality Criteria Met</promise>`
 
 ## Example Dialog