@bugzy-ai/bugzy 1.9.5 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +273 -273
- package/dist/cli/index.cjs +20 -20
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.js +19 -19
- package/dist/cli/index.js.map +1 -1
- package/dist/index.cjs +17 -17
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +17 -17
- package/dist/index.js.map +1 -1
- package/dist/subagents/index.cjs +1 -1
- package/dist/subagents/index.cjs.map +1 -1
- package/dist/subagents/index.js +1 -1
- package/dist/subagents/index.js.map +1 -1
- package/dist/subagents/metadata.cjs +1 -1
- package/dist/subagents/metadata.cjs.map +1 -1
- package/dist/subagents/metadata.js +1 -1
- package/dist/subagents/metadata.js.map +1 -1
- package/dist/tasks/index.cjs.map +1 -1
- package/dist/tasks/index.js.map +1 -1
- package/package.json +1 -1
- package/templates/init/.bugzy/runtime/knowledge-base.md +61 -61
- package/templates/init/.bugzy/runtime/knowledge-maintenance-guide.md +97 -97
- package/templates/init/.bugzy/runtime/project-context.md +35 -35
- package/templates/init/.bugzy/runtime/subagent-memory-guide.md +87 -87
- package/templates/init/.bugzy/runtime/templates/test-plan-template.md +50 -50
- package/templates/init/.bugzy/runtime/templates/test-result-schema.md +498 -498
- package/templates/init/.bugzy/runtime/test-execution-strategy.md +535 -535
- package/templates/init/.bugzy/runtime/testing-best-practices.md +724 -724
- package/templates/init/.env.testdata +18 -18
- package/templates/init/.gitignore-template +24 -24
- package/templates/init/AGENTS.md +155 -155
- package/templates/init/CLAUDE.md +157 -157
- package/templates/init/test-runs/README.md +45 -45
- package/templates/playwright/BasePage.template.ts +190 -190
- package/templates/playwright/auth.setup.template.ts +89 -89
- package/templates/playwright/dataGenerators.helper.template.ts +148 -148
- package/templates/playwright/dateUtils.helper.template.ts +96 -96
- package/templates/playwright/pages.fixture.template.ts +50 -50
- package/templates/playwright/playwright.config.template.ts +97 -97
- package/templates/playwright/reporters/bugzy-reporter.ts +454 -454
- package/dist/templates/init/.bugzy/runtime/knowledge-base.md +0 -61
- package/dist/templates/init/.bugzy/runtime/knowledge-maintenance-guide.md +0 -97
- package/dist/templates/init/.bugzy/runtime/project-context.md +0 -35
- package/dist/templates/init/.bugzy/runtime/subagent-memory-guide.md +0 -87
- package/dist/templates/init/.bugzy/runtime/templates/test-plan-template.md +0 -50
- package/dist/templates/init/.bugzy/runtime/templates/test-result-schema.md +0 -498
- package/dist/templates/init/.bugzy/runtime/test-execution-strategy.md +0 -535
- package/dist/templates/init/.bugzy/runtime/testing-best-practices.md +0 -632
- package/dist/templates/init/.gitignore-template +0 -25
|
@@ -1,18 +1,18 @@
|
|
|
1
|
-
# Non-Secret Test Data
|
|
2
|
-
# This file contains actual test values (non-sensitive) and IS committed to version control
|
|
3
|
-
# This file will be populated by /onboard-testing command with discovered test data
|
|
4
|
-
#
|
|
5
|
-
# IMPORTANT: Passwords and API keys go in .env (not here!)
|
|
6
|
-
# This file only contains non-sensitive data like URLs and email addresses.
|
|
7
|
-
|
|
8
|
-
# Application Under Test
|
|
9
|
-
TEST_BASE_URL=https://example.com
|
|
10
|
-
|
|
11
|
-
# Test User Identifiers (non-secret - emails can be committed)
|
|
12
|
-
# TEST_OWNER_EMAIL=owner@test.example.com
|
|
13
|
-
# TEST_ADMIN_EMAIL=admin@test.example.com
|
|
14
|
-
|
|
15
|
-
# NOTE: The corresponding passwords/secrets should be added to .env:
|
|
16
|
-
# TEST_OWNER_PASSWORD=<add to .env>
|
|
17
|
-
# TEST_ADMIN_PASSWORD=<add to .env>
|
|
18
|
-
# TEST_API_KEY=<add to .env>
|
|
1
|
+
# Non-Secret Test Data
|
|
2
|
+
# This file contains actual test values (non-sensitive) and IS committed to version control
|
|
3
|
+
# This file will be populated by /onboard-testing command with discovered test data
|
|
4
|
+
#
|
|
5
|
+
# IMPORTANT: Passwords and API keys go in .env (not here!)
|
|
6
|
+
# This file only contains non-sensitive data like URLs and email addresses.
|
|
7
|
+
|
|
8
|
+
# Application Under Test
|
|
9
|
+
TEST_BASE_URL=https://example.com
|
|
10
|
+
|
|
11
|
+
# Test User Identifiers (non-secret - emails can be committed)
|
|
12
|
+
# TEST_OWNER_EMAIL=owner@test.example.com
|
|
13
|
+
# TEST_ADMIN_EMAIL=admin@test.example.com
|
|
14
|
+
|
|
15
|
+
# NOTE: The corresponding passwords/secrets should be added to .env:
|
|
16
|
+
# TEST_OWNER_PASSWORD=<add to .env>
|
|
17
|
+
# TEST_ADMIN_PASSWORD=<add to .env>
|
|
18
|
+
# TEST_API_KEY=<add to .env>
|
|
@@ -1,25 +1,25 @@
|
|
|
1
|
-
# Environment files (keep .env.testdata tracked, .env.example managed externally)
|
|
2
|
-
.env
|
|
3
|
-
.env.local
|
|
4
|
-
|
|
5
|
-
# Logs and temporary files
|
|
6
|
-
logs/
|
|
7
|
-
tmp/
|
|
8
|
-
|
|
9
|
-
# Playwright MCP cache
|
|
10
|
-
.playwright-mcp/
|
|
11
|
-
|
|
12
|
-
# Playwright test results
|
|
13
|
-
test-results/
|
|
14
|
-
playwright-report/
|
|
15
|
-
playwright/.cache/
|
|
16
|
-
tests/.auth/
|
|
17
|
-
|
|
18
|
-
# Node modules if using any Node.js tooling
|
|
19
|
-
node_modules/
|
|
20
|
-
.DS_Store
|
|
21
|
-
|
|
22
|
-
# Test result media files
|
|
23
|
-
**/*.webm
|
|
24
|
-
**/*.zip
|
|
1
|
+
# Environment files (keep .env.testdata tracked, .env.example managed externally)
|
|
2
|
+
.env
|
|
3
|
+
.env.local
|
|
4
|
+
|
|
5
|
+
# Logs and temporary files
|
|
6
|
+
logs/
|
|
7
|
+
tmp/
|
|
8
|
+
|
|
9
|
+
# Playwright MCP cache
|
|
10
|
+
.playwright-mcp/
|
|
11
|
+
|
|
12
|
+
# Playwright test results
|
|
13
|
+
test-results/
|
|
14
|
+
playwright-report/
|
|
15
|
+
playwright/.cache/
|
|
16
|
+
tests/.auth/
|
|
17
|
+
|
|
18
|
+
# Node modules if using any Node.js tooling
|
|
19
|
+
node_modules/
|
|
20
|
+
.DS_Store
|
|
21
|
+
|
|
22
|
+
# Test result media files
|
|
23
|
+
**/*.webm
|
|
24
|
+
**/*.zip
|
|
25
25
|
**/*.png
|
package/templates/init/AGENTS.md
CHANGED
|
@@ -1,155 +1,155 @@
|
|
|
1
|
-
# Project Memory
|
|
2
|
-
|
|
3
|
-
## Project Overview
|
|
4
|
-
**Type**: Test Management System
|
|
5
|
-
**Purpose**: Autonomous QA testing with AI-based test generation and execution
|
|
6
|
-
|
|
7
|
-
## SECURITY NOTICE
|
|
8
|
-
|
|
9
|
-
**CRITICAL: Never Read the .env File**
|
|
10
|
-
|
|
11
|
-
All agents, subagents, and slash commands must follow this security policy:
|
|
12
|
-
|
|
13
|
-
- **NEVER read the `.env` file** - It contains secrets, credentials, and sensitive data
|
|
14
|
-
- **ALWAYS use `.env.testdata`** instead - It contains only variable names, never actual values
|
|
15
|
-
- **Reference variables by name only** - e.g., TEST_BASE_URL, TEST_USER_EMAIL (never read the actual values)
|
|
16
|
-
|
|
17
|
-
When you need to know what environment variables are available:
|
|
18
|
-
1. Read `.env.testdata` to see variable names and structure
|
|
19
|
-
2. Reference variables by name in documentation and instructions
|
|
20
|
-
3. Trust that users will configure their own `.env` file from the example
|
|
21
|
-
4. Never attempt to access actual credential values
|
|
22
|
-
|
|
23
|
-
This policy applies to all agents, commands, and any code execution.
|
|
24
|
-
|
|
25
|
-
## Agent System
|
|
26
|
-
|
|
27
|
-
### Active Agents
|
|
28
|
-
<!-- AUTO-GENERATED: This section is populated during project creation -->
|
|
29
|
-
<!-- Configured agents will be listed here based on project setup -->
|
|
30
|
-
|
|
31
|
-
Agents configured for this project are stored in the agents directory (configured during setup). These are specialized sub-agents that handle specific domains. The active agents and their integrations will be configured during project setup.
|
|
32
|
-
|
|
33
|
-
### Agent Memory Index
|
|
34
|
-
<!-- AUTO-GENERATED: Agent memory files are created during project setup -->
|
|
35
|
-
|
|
36
|
-
Specialized memory files for domain experts will be created in `.bugzy/runtime/memory/` based on configured agents.
|
|
37
|
-
|
|
38
|
-
## Project Structure
|
|
39
|
-
|
|
40
|
-
### Core Components
|
|
41
|
-
- **Test Plan**: `test-plan.md` - Generated via `/generate-test-plan` command using template
|
|
42
|
-
- **Test Cases**: `./test-cases/` - Individual test case files generated via `/generate-test-cases`
|
|
43
|
-
- **Exploration Reports**: `./exploration-reports/` - Structured reports from application exploration sessions
|
|
44
|
-
- **Knowledge Base**: `.bugzy/runtime/knowledge-base.md` - Curated knowledge about the project (maintained using `.bugzy/runtime/knowledge-maintenance-guide.md`)
|
|
45
|
-
- **Issue Tracking**: Managed by issue-tracker agent in your configured project management system
|
|
46
|
-
- **Runtime**: `.bugzy/runtime/` - Project-specific runtime files:
|
|
47
|
-
- `memory/` - Agent memory and knowledge base files
|
|
48
|
-
- `templates/` - Document templates for test plan generation
|
|
49
|
-
- `knowledge-base.md` - Accumulated insights
|
|
50
|
-
- `project-context.md` - **CRITICAL**: Project SDLC, team information, QA workflow, and testing guidelines
|
|
51
|
-
|
|
52
|
-
### Available Commands
|
|
53
|
-
1. `/generate-test-plan` - Generate comprehensive test plan from product documentation using the documentation-researcher agent
|
|
54
|
-
2. `/explore-application` - Systematically explore the application to discover UI elements, workflows, and behaviors, updating test plan with findings
|
|
55
|
-
3. `/generate-test-cases` - Create E2E browser test cases (exploratory, functional, regression, smoke) based on documentation and test plan
|
|
56
|
-
4. `/run-tests` - Execute test cases using the test-runner agent
|
|
57
|
-
5. `/handle-message` - Process team responses and manage ongoing conversations with the product team using the team-communicator agent
|
|
58
|
-
|
|
59
|
-
### Git Workflow
|
|
60
|
-
|
|
61
|
-
**Git operations are now handled automatically by the execution environment.**
|
|
62
|
-
|
|
63
|
-
Agents and commands should NOT perform git operations (commit, push). Instead:
|
|
64
|
-
|
|
65
|
-
1. **Focus on core work**: Agents generate files and execute tests
|
|
66
|
-
2. **External service handles git**: After task completion, the Cloud Run environment:
|
|
67
|
-
- Commits all changes with standardized messages
|
|
68
|
-
- Pushes to the remote repository
|
|
69
|
-
- Handles authentication and errors
|
|
70
|
-
|
|
71
|
-
**Files Automatically Committed**:
|
|
72
|
-
- `test-plan.md` - Test planning documents
|
|
73
|
-
- `.env.testdata` - Environment variable templates
|
|
74
|
-
- `./test-cases/*.md` - Individual test case files
|
|
75
|
-
- `./exploration-reports/*.md` - Application exploration findings
|
|
76
|
-
- `.bugzy/runtime/knowledge-base.md` - Accumulated testing insights
|
|
77
|
-
- `.bugzy/runtime/memory/*.md` - Agent memory and knowledge base
|
|
78
|
-
- `AGENTS.md` - Updated project context (when modified)
|
|
79
|
-
- `.bugzy/runtime/project-context.md` - Project-specific runtime context
|
|
80
|
-
|
|
81
|
-
**Git-Ignored Files** (NOT committed):
|
|
82
|
-
- `.env` - Local environment variables with secrets
|
|
83
|
-
- `logs/` - Temporary log files
|
|
84
|
-
- `tmp/` - Temporary files
|
|
85
|
-
- `.playwright-mcp/` - Playwright videos (uploaded to GCS separately)
|
|
86
|
-
- `node_modules/` - Node.js dependencies
|
|
87
|
-
|
|
88
|
-
### Workflow
|
|
89
|
-
|
|
90
|
-
#### Testing Workflow
|
|
91
|
-
1. Generate test plan: `/generate-test-plan` command leverages the documentation-researcher agent to gather requirements
|
|
92
|
-
2. Explore application: `/explore-application --focus [area] --depth [shallow|deep]` discovers actual UI elements and behaviors
|
|
93
|
-
3. Create test cases: `/generate-test-cases --type [type] --focus [feature]` uses discovered documentation and exploration findings
|
|
94
|
-
4. Execute tests: `/run-tests [test-id|type|tag|all]` runs test cases via test-runner agent
|
|
95
|
-
5. Continuous refinement through exploration and test execution
|
|
96
|
-
|
|
97
|
-
### Testing Lifecycle Phases
|
|
98
|
-
1. **Initial Test Plan Creation** - From product documentation
|
|
99
|
-
2. **Application Exploration** - Systematic discovery via `/explore-application`
|
|
100
|
-
3. **Test Case Generation** - Stored as individual files with actual UI elements
|
|
101
|
-
4. **Test Execution** - Automated runs via `/run-tests`
|
|
102
|
-
5. **Continuous Refinement** - Through exploration and test execution
|
|
103
|
-
6. **Regression Testing** - Scheduled or triggered execution
|
|
104
|
-
|
|
105
|
-
## Key Project Insights
|
|
106
|
-
<!-- Critical discoveries and patterns that apply across all domains -->
|
|
107
|
-
|
|
108
|
-
## Cross-Domain Knowledge
|
|
109
|
-
|
|
110
|
-
### Project Context Usage
|
|
111
|
-
The `.bugzy/runtime/project-context.md` file contains critical project information that MUST be referenced by:
|
|
112
|
-
- All agents during initialization to understand project specifics
|
|
113
|
-
- All commands before processing to align with project requirements
|
|
114
|
-
- Any custom implementations to maintain consistency
|
|
115
|
-
|
|
116
|
-
Key information includes: QA workflow, story status management, bug reporting guidelines, testing environment details, and SDLC methodology.
|
|
117
|
-
|
|
118
|
-
### Test Execution Standards
|
|
119
|
-
|
|
120
|
-
**Automated Test Execution** (via `/run-tests` command):
|
|
121
|
-
- Test runs are stored in hierarchical structure: `./test-runs/YYYYMMDD-HHMMSS/TC-XXX/exec-N/`
|
|
122
|
-
- Each test run session creates:
|
|
123
|
-
- `execution-id.txt`: Contains BUGZY_EXECUTION_ID for session tracking
|
|
124
|
-
- `manifest.json`: Overall run metadata with all test cases and executions
|
|
125
|
-
- Per test case folders (`TC-001-login/`, etc.) containing execution attempts
|
|
126
|
-
- Each execution attempt (`exec-1/`, `exec-2/`, `exec-3/`) contains:
|
|
127
|
-
- `result.json`: Playwright test result format with status, duration, errors, attachments
|
|
128
|
-
- `steps.json`: Step-by-step execution with video timestamps (if test uses `test.step()`)
|
|
129
|
-
- `video.webm`: Video recording (copied from Playwright's temp location)
|
|
130
|
-
- `trace.zip`: Trace file (only for failures)
|
|
131
|
-
- `screenshots/`: Screenshots directory (only for failures)
|
|
132
|
-
- Videos are recorded for ALL tests, traces/screenshots only for failures
|
|
133
|
-
- Custom Bugzy reporter handles all artifact organization automatically
|
|
134
|
-
- External service uploads videos from execution folders to GCS
|
|
135
|
-
|
|
136
|
-
**Step-Level Tracking** (Automated Tests):
|
|
137
|
-
- Tests using `test.step()` API generate steps.json files with video timestamps
|
|
138
|
-
- Each step includes timestamp and videoTimeSeconds for easy video navigation
|
|
139
|
-
- High-level steps recommended (3-7 per test) for manageable navigation
|
|
140
|
-
- Steps appear in both result.json (Playwright format) and steps.json (user-friendly format)
|
|
141
|
-
- Tests without `test.step()` still work but won't have steps.json
|
|
142
|
-
|
|
143
|
-
**Manual Test Execution** (via test-runner agent):
|
|
144
|
-
- Manual test cases use separate format: `./test-runs/YYYYMMDD-HHMMSS/TC-XXX/`
|
|
145
|
-
- Each test run generates:
|
|
146
|
-
- `summary.json`: Structured test result with video filename reference
|
|
147
|
-
- `steps.json`: Step-by-step execution with timestamps and video synchronization
|
|
148
|
-
- Video recording via Playwright MCP --save-video flag
|
|
149
|
-
- Videos remain in `.playwright-mcp/` folder - external service uploads to GCS
|
|
150
|
-
|
|
151
|
-
**General Rules**:
|
|
152
|
-
- NO test-related files should be created in project root
|
|
153
|
-
- DO NOT copy, move, or delete video files manually
|
|
154
|
-
- DO NOT CREATE ANY SUMMARY, REPORT, OR ADDITIONAL FILES unless explicitly requested
|
|
155
|
-
<!-- Additional cross-domain information below -->
|
|
1
|
+
# Project Memory
|
|
2
|
+
|
|
3
|
+
## Project Overview
|
|
4
|
+
**Type**: Test Management System
|
|
5
|
+
**Purpose**: Autonomous QA testing with AI-based test generation and execution
|
|
6
|
+
|
|
7
|
+
## SECURITY NOTICE
|
|
8
|
+
|
|
9
|
+
**CRITICAL: Never Read the .env File**
|
|
10
|
+
|
|
11
|
+
All agents, subagents, and slash commands must follow this security policy:
|
|
12
|
+
|
|
13
|
+
- **NEVER read the `.env` file** - It contains secrets, credentials, and sensitive data
|
|
14
|
+
- **ALWAYS use `.env.testdata`** instead - It contains only variable names, never actual values
|
|
15
|
+
- **Reference variables by name only** - e.g., TEST_BASE_URL, TEST_USER_EMAIL (never read the actual values)
|
|
16
|
+
|
|
17
|
+
When you need to know what environment variables are available:
|
|
18
|
+
1. Read `.env.testdata` to see variable names and structure
|
|
19
|
+
2. Reference variables by name in documentation and instructions
|
|
20
|
+
3. Trust that users will configure their own `.env` file from the example
|
|
21
|
+
4. Never attempt to access actual credential values
|
|
22
|
+
|
|
23
|
+
This policy applies to all agents, commands, and any code execution.
|
|
24
|
+
|
|
25
|
+
## Agent System
|
|
26
|
+
|
|
27
|
+
### Active Agents
|
|
28
|
+
<!-- AUTO-GENERATED: This section is populated during project creation -->
|
|
29
|
+
<!-- Configured agents will be listed here based on project setup -->
|
|
30
|
+
|
|
31
|
+
Agents configured for this project are stored in the agents directory (configured during setup). These are specialized sub-agents that handle specific domains. The active agents and their integrations will be configured during project setup.
|
|
32
|
+
|
|
33
|
+
### Agent Memory Index
|
|
34
|
+
<!-- AUTO-GENERATED: Agent memory files are created during project setup -->
|
|
35
|
+
|
|
36
|
+
Specialized memory files for domain experts will be created in `.bugzy/runtime/memory/` based on configured agents.
|
|
37
|
+
|
|
38
|
+
## Project Structure
|
|
39
|
+
|
|
40
|
+
### Core Components
|
|
41
|
+
- **Test Plan**: `test-plan.md` - Generated via `/generate-test-plan` command using template
|
|
42
|
+
- **Test Cases**: `./test-cases/` - Individual test case files generated via `/generate-test-cases`
|
|
43
|
+
- **Exploration Reports**: `./exploration-reports/` - Structured reports from application exploration sessions
|
|
44
|
+
- **Knowledge Base**: `.bugzy/runtime/knowledge-base.md` - Curated knowledge about the project (maintained using `.bugzy/runtime/knowledge-maintenance-guide.md`)
|
|
45
|
+
- **Issue Tracking**: Managed by issue-tracker agent in your configured project management system
|
|
46
|
+
- **Runtime**: `.bugzy/runtime/` - Project-specific runtime files:
|
|
47
|
+
- `memory/` - Agent memory and knowledge base files
|
|
48
|
+
- `templates/` - Document templates for test plan generation
|
|
49
|
+
- `knowledge-base.md` - Accumulated insights
|
|
50
|
+
- `project-context.md` - **CRITICAL**: Project SDLC, team information, QA workflow, and testing guidelines
|
|
51
|
+
|
|
52
|
+
### Available Commands
|
|
53
|
+
1. `/generate-test-plan` - Generate comprehensive test plan from product documentation using the documentation-researcher agent
|
|
54
|
+
2. `/explore-application` - Systematically explore the application to discover UI elements, workflows, and behaviors, updating test plan with findings
|
|
55
|
+
3. `/generate-test-cases` - Create E2E browser test cases (exploratory, functional, regression, smoke) based on documentation and test plan
|
|
56
|
+
4. `/run-tests` - Execute test cases using the test-runner agent
|
|
57
|
+
5. `/handle-message` - Process team responses and manage ongoing conversations with the product team using the team-communicator agent
|
|
58
|
+
|
|
59
|
+
### Git Workflow
|
|
60
|
+
|
|
61
|
+
**Git operations are now handled automatically by the execution environment.**
|
|
62
|
+
|
|
63
|
+
Agents and commands should NOT perform git operations (commit, push). Instead:
|
|
64
|
+
|
|
65
|
+
1. **Focus on core work**: Agents generate files and execute tests
|
|
66
|
+
2. **External service handles git**: After task completion, the Cloud Run environment:
|
|
67
|
+
- Commits all changes with standardized messages
|
|
68
|
+
- Pushes to the remote repository
|
|
69
|
+
- Handles authentication and errors
|
|
70
|
+
|
|
71
|
+
**Files Automatically Committed**:
|
|
72
|
+
- `test-plan.md` - Test planning documents
|
|
73
|
+
- `.env.testdata` - Environment variable templates
|
|
74
|
+
- `./test-cases/*.md` - Individual test case files
|
|
75
|
+
- `./exploration-reports/*.md` - Application exploration findings
|
|
76
|
+
- `.bugzy/runtime/knowledge-base.md` - Accumulated testing insights
|
|
77
|
+
- `.bugzy/runtime/memory/*.md` - Agent memory and knowledge base
|
|
78
|
+
- `AGENTS.md` - Updated project context (when modified)
|
|
79
|
+
- `.bugzy/runtime/project-context.md` - Project-specific runtime context
|
|
80
|
+
|
|
81
|
+
**Git-Ignored Files** (NOT committed):
|
|
82
|
+
- `.env` - Local environment variables with secrets
|
|
83
|
+
- `logs/` - Temporary log files
|
|
84
|
+
- `tmp/` - Temporary files
|
|
85
|
+
- `.playwright-mcp/` - Playwright videos (uploaded to GCS separately)
|
|
86
|
+
- `node_modules/` - Node.js dependencies
|
|
87
|
+
|
|
88
|
+
### Workflow
|
|
89
|
+
|
|
90
|
+
#### Testing Workflow
|
|
91
|
+
1. Generate test plan: `/generate-test-plan` command leverages the documentation-researcher agent to gather requirements
|
|
92
|
+
2. Explore application: `/explore-application --focus [area] --depth [shallow|deep]` discovers actual UI elements and behaviors
|
|
93
|
+
3. Create test cases: `/generate-test-cases --type [type] --focus [feature]` uses discovered documentation and exploration findings
|
|
94
|
+
4. Execute tests: `/run-tests [test-id|type|tag|all]` runs test cases via test-runner agent
|
|
95
|
+
5. Continuous refinement through exploration and test execution
|
|
96
|
+
|
|
97
|
+
### Testing Lifecycle Phases
|
|
98
|
+
1. **Initial Test Plan Creation** - From product documentation
|
|
99
|
+
2. **Application Exploration** - Systematic discovery via `/explore-application`
|
|
100
|
+
3. **Test Case Generation** - Stored as individual files with actual UI elements
|
|
101
|
+
4. **Test Execution** - Automated runs via `/run-tests`
|
|
102
|
+
5. **Continuous Refinement** - Through exploration and test execution
|
|
103
|
+
6. **Regression Testing** - Scheduled or triggered execution
|
|
104
|
+
|
|
105
|
+
## Key Project Insights
|
|
106
|
+
<!-- Critical discoveries and patterns that apply across all domains -->
|
|
107
|
+
|
|
108
|
+
## Cross-Domain Knowledge
|
|
109
|
+
|
|
110
|
+
### Project Context Usage
|
|
111
|
+
The `.bugzy/runtime/project-context.md` file contains critical project information that MUST be referenced by:
|
|
112
|
+
- All agents during initialization to understand project specifics
|
|
113
|
+
- All commands before processing to align with project requirements
|
|
114
|
+
- Any custom implementations to maintain consistency
|
|
115
|
+
|
|
116
|
+
Key information includes: QA workflow, story status management, bug reporting guidelines, testing environment details, and SDLC methodology.
|
|
117
|
+
|
|
118
|
+
### Test Execution Standards
|
|
119
|
+
|
|
120
|
+
**Automated Test Execution** (via `/run-tests` command):
|
|
121
|
+
- Test runs are stored in hierarchical structure: `./test-runs/YYYYMMDD-HHMMSS/TC-XXX/exec-N/`
|
|
122
|
+
- Each test run session creates:
|
|
123
|
+
- `execution-id.txt`: Contains BUGZY_EXECUTION_ID for session tracking
|
|
124
|
+
- `manifest.json`: Overall run metadata with all test cases and executions
|
|
125
|
+
- Per test case folders (`TC-001-login/`, etc.) containing execution attempts
|
|
126
|
+
- Each execution attempt (`exec-1/`, `exec-2/`, `exec-3/`) contains:
|
|
127
|
+
- `result.json`: Playwright test result format with status, duration, errors, attachments
|
|
128
|
+
- `steps.json`: Step-by-step execution with video timestamps (if test uses `test.step()`)
|
|
129
|
+
- `video.webm`: Video recording (copied from Playwright's temp location)
|
|
130
|
+
- `trace.zip`: Trace file (only for failures)
|
|
131
|
+
- `screenshots/`: Screenshots directory (only for failures)
|
|
132
|
+
- Videos are recorded for ALL tests, traces/screenshots only for failures
|
|
133
|
+
- Custom Bugzy reporter handles all artifact organization automatically
|
|
134
|
+
- External service uploads videos from execution folders to GCS
|
|
135
|
+
|
|
136
|
+
**Step-Level Tracking** (Automated Tests):
|
|
137
|
+
- Tests using `test.step()` API generate steps.json files with video timestamps
|
|
138
|
+
- Each step includes timestamp and videoTimeSeconds for easy video navigation
|
|
139
|
+
- High-level steps recommended (3-7 per test) for manageable navigation
|
|
140
|
+
- Steps appear in both result.json (Playwright format) and steps.json (user-friendly format)
|
|
141
|
+
- Tests without `test.step()` still work but won't have steps.json
|
|
142
|
+
|
|
143
|
+
**Manual Test Execution** (via test-runner agent):
|
|
144
|
+
- Manual test cases use separate format: `./test-runs/YYYYMMDD-HHMMSS/TC-XXX/`
|
|
145
|
+
- Each test run generates:
|
|
146
|
+
- `summary.json`: Structured test result with video filename reference
|
|
147
|
+
- `steps.json`: Step-by-step execution with timestamps and video synchronization
|
|
148
|
+
- Video recording via Playwright MCP --save-video flag
|
|
149
|
+
- Videos remain in `.playwright-mcp/` folder - external service uploads to GCS
|
|
150
|
+
|
|
151
|
+
**General Rules**:
|
|
152
|
+
- NO test-related files should be created in project root
|
|
153
|
+
- DO NOT copy, move, or delete video files manually
|
|
154
|
+
- DO NOT CREATE ANY SUMMARY, REPORT, OR ADDITIONAL FILES unless explicitly requested
|
|
155
|
+
<!-- Additional cross-domain information below -->
|