joycraft 0.5.7 → 0.5.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -348
- package/dist/{chunk-G342HURJ.js → chunk-Y6GBN6R4.js} +672 -61
- package/dist/chunk-Y6GBN6R4.js.map +1 -0
- package/dist/cli.js +3 -3
- package/dist/index.d.ts +6 -0
- package/dist/index.js +78 -7
- package/dist/index.js.map +1 -1
- package/dist/{init-7FUTURUY.js → init-B7MYPB7Z.js} +82 -11
- package/dist/init-B7MYPB7Z.js.map +1 -0
- package/dist/{init-autofix-K4B5BD5V.js → init-autofix-FKZN5S3R.js} +2 -2
- package/dist/{upgrade-2Y7D2HCD.js → upgrade-WXNR3ZNI.js} +2 -2
- package/package.json +1 -1
- package/dist/chunk-G342HURJ.js.map +0 -1
- package/dist/init-7FUTURUY.js.map +0 -1
- /package/dist/{init-autofix-K4B5BD5V.js.map → init-autofix-FKZN5S3R.js.map} +0 -0
- /package/dist/{upgrade-2Y7D2HCD.js.map → upgrade-WXNR3ZNI.js.map} +0 -0
|
@@ -1493,6 +1493,183 @@ Based on the verdict:
|
|
|
1493
1493
|
| Subagent can't run tests (missing deps) | Report the error as FAIL evidence |
|
|
1494
1494
|
| No specs found and no path given | Tell user to provide a spec path or create a spec first |
|
|
1495
1495
|
| Spec status is "Complete" | Still run verification -- "Complete" means the implementer thinks it's done, verification confirms |
|
|
1496
|
+
`,
|
|
1497
|
+
"joycraft-bugfix.md": `---
|
|
1498
|
+
name: joycraft-bugfix
|
|
1499
|
+
description: Structured bug fix workflow \u2014 triage, diagnose, discuss with user, write a focused spec, hand off for implementation
|
|
1500
|
+
---
|
|
1501
|
+
|
|
1502
|
+
# Bug Fix Workflow
|
|
1503
|
+
|
|
1504
|
+
You are fixing a bug. Follow this process in order. Do not skip steps.
|
|
1505
|
+
|
|
1506
|
+
**Guard clause:** If the user's request is clearly a new feature \u2014 not a bug, error, or unexpected behavior \u2014 say:
|
|
1507
|
+
"This sounds like a new feature rather than a bug fix. Try \`/joycraft-new-feature\` for a guided feature workflow."
|
|
1508
|
+
Then stop.
|
|
1509
|
+
|
|
1510
|
+
---
|
|
1511
|
+
|
|
1512
|
+
## Phase 1: Triage
|
|
1513
|
+
|
|
1514
|
+
Establish what's broken. Your goal is to reproduce the bug or at minimum understand the symptom clearly.
|
|
1515
|
+
|
|
1516
|
+
**Ask / gather:**
|
|
1517
|
+
- What is the symptom? (error message, unexpected behavior, crash, wrong output)
|
|
1518
|
+
- What are the steps to reproduce?
|
|
1519
|
+
- What is the expected behavior vs. actual behavior?
|
|
1520
|
+
- When did it start? (recent change, always been this way, intermittent)
|
|
1521
|
+
- Any relevant logs, screenshots, or error output?
|
|
1522
|
+
|
|
1523
|
+
**Actions:**
|
|
1524
|
+
- If the user provides an error message or stack trace, read the referenced files immediately
|
|
1525
|
+
- If steps to reproduce are provided, try to reproduce the bug (run the failing command, test, or request)
|
|
1526
|
+
- If the bug is intermittent or hard to reproduce, gather more context: environment, OS, versions, config
|
|
1527
|
+
|
|
1528
|
+
**Done when:** You can describe the symptom in one sentence and have either reproduced it or have enough context to diagnose without reproduction.
|
|
1529
|
+
|
|
1530
|
+
---
|
|
1531
|
+
|
|
1532
|
+
## Phase 2: Diagnose
|
|
1533
|
+
|
|
1534
|
+
Find the root cause. Read code, trace the execution path, identify what's wrong and why.
|
|
1535
|
+
|
|
1536
|
+
**Actions:**
|
|
1537
|
+
- Start from the error site (stack trace, failing test, broken UI) and trace backward
|
|
1538
|
+
- Read the relevant source files \u2014 don't guess based on file names alone
|
|
1539
|
+
- Identify the specific line(s), condition, or logic error causing the bug
|
|
1540
|
+
- Check git blame or recent commits if the bug was introduced by a recent change
|
|
1541
|
+
- Look for related bugs \u2014 is this a symptom of a deeper issue?
|
|
1542
|
+
|
|
1543
|
+
**Done when:** You can explain the root cause in 2-3 sentences: what's wrong, why it's wrong, and where in the code it happens.
|
|
1544
|
+
|
|
1545
|
+
---
|
|
1546
|
+
|
|
1547
|
+
## Phase 3: Discuss
|
|
1548
|
+
|
|
1549
|
+
Present your findings to the user. Do NOT start writing code or a spec yet.
|
|
1550
|
+
|
|
1551
|
+
**Present:**
|
|
1552
|
+
1. **Symptom:** What the user sees (confirm your understanding matches theirs)
|
|
1553
|
+
2. **Root cause:** What's actually wrong in the code and why
|
|
1554
|
+
3. **Proposed fix:** What you think the fix is \u2014 be specific (which files, what changes)
|
|
1555
|
+
4. **Risk assessment:** What could go wrong with this fix? Any side effects?
|
|
1556
|
+
5. **Scope check:** Is this a simple fix or does it touch multiple systems?
|
|
1557
|
+
|
|
1558
|
+
**Ask:**
|
|
1559
|
+
- "Does this match what you're seeing?"
|
|
1560
|
+
- "Are you comfortable with this approach, or do you want to explore alternatives?"
|
|
1561
|
+
- If the fix is large or risky: "Should we decompose this into smaller specs?"
|
|
1562
|
+
|
|
1563
|
+
**Done when:** The user agrees with the diagnosis and proposed fix direction.
|
|
1564
|
+
|
|
1565
|
+
---
|
|
1566
|
+
|
|
1567
|
+
## Phase 4: Spec the Fix
|
|
1568
|
+
|
|
1569
|
+
Write a bug fix spec to \`docs/specs/YYYY-MM-DD-bugfix-name.md\`. Create the \`docs/specs/\` directory if it doesn't exist.
|
|
1570
|
+
|
|
1571
|
+
**Why:** Even bug fixes deserve a spec. It forces clarity on what "fixed" means, ensures test-first discipline, and creates a traceable record of the fix.
|
|
1572
|
+
|
|
1573
|
+
Use this template:
|
|
1574
|
+
|
|
1575
|
+
\`\`\`markdown
|
|
1576
|
+
# Fix [Bug Description] \u2014 Bug Fix Spec
|
|
1577
|
+
|
|
1578
|
+
> **Parent Brief:** none (bug fix)
|
|
1579
|
+
> **Issue/Error:** [error message, issue link, or symptom description]
|
|
1580
|
+
> **Status:** Ready
|
|
1581
|
+
> **Date:** YYYY-MM-DD
|
|
1582
|
+
> **Estimated scope:** [1 session / N files / ~N lines]
|
|
1583
|
+
|
|
1584
|
+
---
|
|
1585
|
+
|
|
1586
|
+
## Bug
|
|
1587
|
+
|
|
1588
|
+
What is broken? Describe the symptom the user experiences.
|
|
1589
|
+
|
|
1590
|
+
## Root Cause
|
|
1591
|
+
|
|
1592
|
+
What is wrong in the code and why? Name the specific file(s) and line(s).
|
|
1593
|
+
|
|
1594
|
+
## Fix
|
|
1595
|
+
|
|
1596
|
+
What changes will fix this? Be specific \u2014 describe the code change, not just "fix the bug."
|
|
1597
|
+
|
|
1598
|
+
## Acceptance Criteria
|
|
1599
|
+
|
|
1600
|
+
- [ ] [The bug no longer occurs \u2014 describe the correct behavior]
|
|
1601
|
+
- [ ] [No regressions in related functionality]
|
|
1602
|
+
- [ ] Build passes
|
|
1603
|
+
- [ ] Tests pass
|
|
1604
|
+
|
|
1605
|
+
## Test Plan
|
|
1606
|
+
|
|
1607
|
+
| Acceptance Criterion | Test | Type |
|
|
1608
|
+
|---------------------|------|------|
|
|
1609
|
+
| [Bug no longer occurs] | [Test that reproduces the bug, then verifies the fix] | [unit/integration/e2e] |
|
|
1610
|
+
| [No regressions] | [Existing tests still pass, or new regression test] | [unit/integration] |
|
|
1611
|
+
|
|
1612
|
+
**Execution order:**
|
|
1613
|
+
1. Write a test that reproduces the bug \u2014 it should FAIL (red)
|
|
1614
|
+
2. Run the test to confirm it fails
|
|
1615
|
+
3. Apply the fix
|
|
1616
|
+
4. Run the test to confirm it passes (green)
|
|
1617
|
+
5. Run the full test suite to check for regressions
|
|
1618
|
+
|
|
1619
|
+
**Smoke test:** [The bug reproduction test \u2014 fastest way to verify the fix works]
|
|
1620
|
+
|
|
1621
|
+
**Before implementing, verify your test harness:**
|
|
1622
|
+
1. Run the reproduction test \u2014 it must FAIL (if it passes, you're not testing the actual bug)
|
|
1623
|
+
2. The test must exercise your actual code \u2014 not a reimplementation or mock
|
|
1624
|
+
3. Identify your smoke test \u2014 it must run in seconds, not minutes
|
|
1625
|
+
|
|
1626
|
+
## Constraints
|
|
1627
|
+
|
|
1628
|
+
- MUST: [any hard requirements for the fix]
|
|
1629
|
+
- MUST NOT: [any prohibitions \u2014 e.g., don't change the public API]
|
|
1630
|
+
|
|
1631
|
+
## Affected Files
|
|
1632
|
+
|
|
1633
|
+
| Action | File | What Changes |
|
|
1634
|
+
|--------|------|-------------|
|
|
1635
|
+
|
|
1636
|
+
## Edge Cases
|
|
1637
|
+
|
|
1638
|
+
| Scenario | Expected Behavior |
|
|
1639
|
+
|----------|------------------|
|
|
1640
|
+
\`\`\`
|
|
1641
|
+
|
|
1642
|
+
**For trivial bugs:** The spec will be short. That's fine \u2014 the structure is the point, not the length.
|
|
1643
|
+
|
|
1644
|
+
**For large bugs that span multiple files/systems:** Consider whether this should be decomposed into multiple specs. If so, create a brief first using \`/joycraft-new-feature\`, then decompose. A bug fix spec should be implementable in a single session.
|
|
1645
|
+
|
|
1646
|
+
---
|
|
1647
|
+
|
|
1648
|
+
## Phase 5: Hand Off
|
|
1649
|
+
|
|
1650
|
+
Tell the user:
|
|
1651
|
+
|
|
1652
|
+
\`\`\`
|
|
1653
|
+
Bug fix spec is ready: docs/specs/YYYY-MM-DD-bugfix-name.md
|
|
1654
|
+
|
|
1655
|
+
Summary:
|
|
1656
|
+
- Bug: [one sentence]
|
|
1657
|
+
- Root cause: [one sentence]
|
|
1658
|
+
- Fix: [one sentence]
|
|
1659
|
+
- Estimated: 1 session
|
|
1660
|
+
|
|
1661
|
+
To execute: Start a fresh session and:
|
|
1662
|
+
1. Read the spec
|
|
1663
|
+
2. Write the reproduction test (must fail)
|
|
1664
|
+
3. Apply the fix (test must pass)
|
|
1665
|
+
4. Run full test suite
|
|
1666
|
+
5. Run /joycraft-session-end to capture discoveries
|
|
1667
|
+
6. Commit and PR
|
|
1668
|
+
|
|
1669
|
+
Ready to start?
|
|
1670
|
+
\`\`\`
|
|
1671
|
+
|
|
1672
|
+
**Why:** A fresh session for implementation produces better results. This diagnostic session has context noise from exploration \u2014 a clean session with just the spec is more focused.
|
|
1496
1673
|
`
|
|
1497
1674
|
};
|
|
1498
1675
|
var TEMPLATES = {
|
|
@@ -1874,13 +2051,59 @@ is required, though you can add one via the GitHub Checks API if you prefer.
|
|
|
1874
2051
|
|
|
1875
2052
|
---
|
|
1876
2053
|
|
|
2054
|
+
## Testing by Stack Type
|
|
2055
|
+
|
|
2056
|
+
The scenario agent selects the appropriate test format based on the project's
|
|
2057
|
+
testing backbone. Each backbone tests the same holdout principle \u2014 observable
|
|
2058
|
+
behavior only, no source imports \u2014 but uses different tools.
|
|
2059
|
+
|
|
2060
|
+
### Web Apps (Playwright)
|
|
2061
|
+
|
|
2062
|
+
For Next.js, Vite, Nuxt, Remix, and other web frameworks. Tests run against a
|
|
2063
|
+
dev server or preview URL using a headless browser.
|
|
2064
|
+
|
|
2065
|
+
- **Template:** \`example-scenario-web.spec.ts\`
|
|
2066
|
+
- **Config:** \`playwright.config.ts\`
|
|
2067
|
+
- **Package:** \`package-web.json\` (use instead of \`package.json\` for web projects)
|
|
2068
|
+
- **Run:** \`npx playwright test\`
|
|
2069
|
+
|
|
2070
|
+
### Mobile Apps (Maestro)
|
|
2071
|
+
|
|
2072
|
+
For React Native, Flutter, and native iOS/Android. Tests are declarative YAML
|
|
2073
|
+
flows that interact with a running app on a simulator.
|
|
2074
|
+
|
|
2075
|
+
- **Template:** \`example-scenario-mobile.yaml\`
|
|
2076
|
+
- **Login sub-flow:** \`example-scenario-mobile-login.yaml\`
|
|
2077
|
+
- **Setup guide:** \`README-mobile.md\`
|
|
2078
|
+
- **Run:** \`maestro test example-scenario-mobile.yaml\`
|
|
2079
|
+
|
|
2080
|
+
### API Backends (HTTP)
|
|
2081
|
+
|
|
2082
|
+
For Express, FastAPI, Django, and other API-only backends. Tests send HTTP
|
|
2083
|
+
requests using Node.js built-in \`fetch\`.
|
|
2084
|
+
|
|
2085
|
+
- **Template:** \`example-scenario-api.test.ts\`
|
|
2086
|
+
- **Run:** \`npx vitest run\`
|
|
2087
|
+
|
|
2088
|
+
### CLI Tools & Libraries (native)
|
|
2089
|
+
|
|
2090
|
+
For CLI tools, npm packages, and non-UI projects. Tests invoke the built
|
|
2091
|
+
binary via \`spawnSync\` and assert on stdout/stderr.
|
|
2092
|
+
|
|
2093
|
+
- **Template:** \`example-scenario.test.ts\`
|
|
2094
|
+
- **Run:** \`npx vitest run\`
|
|
2095
|
+
|
|
2096
|
+
---
|
|
2097
|
+
|
|
1877
2098
|
## Adding scenarios
|
|
1878
2099
|
|
|
1879
2100
|
### Rules
|
|
1880
2101
|
|
|
1881
|
-
|
|
1882
|
-
|
|
1883
|
-
|
|
2102
|
+
These rules apply to ALL backbones:
|
|
2103
|
+
|
|
2104
|
+
1. **Behavioral, not structural.** Test what the app does from a user's
|
|
2105
|
+
perspective. For web: navigate and assert on content. For CLI: run commands
|
|
2106
|
+
and check output. For API: send requests and check responses.
|
|
1884
2107
|
|
|
1885
2108
|
2. **End-to-end.** Each test should represent something a real user would
|
|
1886
2109
|
actually do. If you would not put it in a demo or docs example, reconsider
|
|
@@ -1890,9 +2113,8 @@ is required, though you can add one via the GitHub Checks API if you prefer.
|
|
|
1890
2113
|
see source code. Any \`import\` that reaches into \`../main-repo/src\` breaks
|
|
1891
2114
|
the pattern.
|
|
1892
2115
|
|
|
1893
|
-
4. **Independent.** Each test must be able to run in isolation.
|
|
1894
|
-
|
|
1895
|
-
state between tests.
|
|
2116
|
+
4. **Independent.** Each test must be able to run in isolation. No shared
|
|
2117
|
+
mutable state between tests.
|
|
1896
2118
|
|
|
1897
2119
|
5. **Deterministic.** Avoid network calls, timestamps, or random values in
|
|
1898
2120
|
assertions unless the feature under test genuinely involves them.
|
|
@@ -1901,31 +2123,25 @@ is required, though you can add one via the GitHub Checks API if you prefer.
|
|
|
1901
2123
|
|
|
1902
2124
|
\`\`\`
|
|
1903
2125
|
$SCENARIOS_REPO/
|
|
1904
|
-
\u251C\u2500\u2500 example-scenario.test.ts
|
|
2126
|
+
\u251C\u2500\u2500 example-scenario.test.ts # CLI/binary scenario template
|
|
2127
|
+
\u251C\u2500\u2500 example-scenario-web.spec.ts # Web app scenario template (Playwright)
|
|
2128
|
+
\u251C\u2500\u2500 example-scenario-api.test.ts # API backend scenario template
|
|
2129
|
+
\u251C\u2500\u2500 example-scenario-mobile.yaml # Mobile app scenario template (Maestro)
|
|
2130
|
+
\u251C\u2500\u2500 example-scenario-mobile-login.yaml # Reusable login sub-flow
|
|
2131
|
+
\u251C\u2500\u2500 playwright.config.ts # Playwright config (web projects)
|
|
2132
|
+
\u251C\u2500\u2500 package.json # Default (vitest for CLI/API)
|
|
2133
|
+
\u251C\u2500\u2500 package-web.json # Alternative (Playwright for web)
|
|
2134
|
+
\u251C\u2500\u2500 README-mobile.md # Mobile testing setup guide
|
|
1905
2135
|
\u251C\u2500\u2500 workflows/
|
|
1906
|
-
\u2502 \
|
|
1907
|
-
\
|
|
2136
|
+
\u2502 \u251C\u2500\u2500 run.yml # CI workflow (do not rename)
|
|
2137
|
+
\u2502 \u2514\u2500\u2500 generate.yml # Scenario generation workflow
|
|
2138
|
+
\u251C\u2500\u2500 prompts/
|
|
2139
|
+
\u2502 \u2514\u2500\u2500 scenario-agent.md # Scenario agent instructions
|
|
1908
2140
|
\u2514\u2500\u2500 README.md
|
|
1909
2141
|
\`\`\`
|
|
1910
2142
|
|
|
1911
|
-
|
|
1912
|
-
|
|
1913
|
-
|
|
1914
|
-
### Example structure
|
|
1915
|
-
|
|
1916
|
-
\`\`\`ts
|
|
1917
|
-
import { spawnSync } from "node:child_process";
|
|
1918
|
-
import { join } from "node:path";
|
|
1919
|
-
|
|
1920
|
-
const CLI = join(__dirname, "..", "main-repo", "dist", "cli.js");
|
|
1921
|
-
|
|
1922
|
-
it("init creates a CLAUDE.md file", () => {
|
|
1923
|
-
const tmp = mkdtempSync(join(tmpdir(), "scenario-"));
|
|
1924
|
-
const { status } = spawnSync("node", [CLI, "init", tmp], { encoding: "utf8" });
|
|
1925
|
-
expect(status).toBe(0);
|
|
1926
|
-
expect(existsSync(join(tmp, "CLAUDE.md"))).toBe(true);
|
|
1927
|
-
});
|
|
1928
|
-
\`\`\`
|
|
2143
|
+
Use the template that matches your project's stack. Remove the ones you
|
|
2144
|
+
don't need.
|
|
1929
2145
|
|
|
1930
2146
|
---
|
|
1931
2147
|
|
|
@@ -1937,6 +2153,7 @@ it("init creates a CLAUDE.md file", () => {
|
|
|
1937
2153
|
| Visible to agent | Yes | No |
|
|
1938
2154
|
| What they test | Units, modules, logic | End-to-end behavior |
|
|
1939
2155
|
| Import source code | Yes | Never |
|
|
2156
|
+
| Test method | Unit test framework | Depends on backbone (Playwright/Maestro/vitest/fetch) |
|
|
1940
2157
|
| Run on every push | Yes | Yes (via dispatch) |
|
|
1941
2158
|
| Purpose | Catch regressions fast | Validate real behavior |
|
|
1942
2159
|
|
|
@@ -2085,6 +2302,304 @@ describe("CLI: init command (example \u2014 replace with your real scenarios)",
|
|
|
2085
2302
|
}
|
|
2086
2303
|
}
|
|
2087
2304
|
`,
|
|
2305
|
+
"scenarios/package-web.json": `{
|
|
2306
|
+
"name": "$SCENARIOS_REPO",
|
|
2307
|
+
"version": "0.0.1",
|
|
2308
|
+
"private": true,
|
|
2309
|
+
"type": "module",
|
|
2310
|
+
"scripts": {
|
|
2311
|
+
"test": "playwright test"
|
|
2312
|
+
},
|
|
2313
|
+
"devDependencies": {
|
|
2314
|
+
"@playwright/test": "^1.50.0"
|
|
2315
|
+
}
|
|
2316
|
+
}
|
|
2317
|
+
`,
|
|
2318
|
+
"scenarios/playwright.config.ts": `import { defineConfig } from '@playwright/test';
|
|
2319
|
+
|
|
2320
|
+
/**
|
|
2321
|
+
* Playwright configuration for holdout scenario tests.
|
|
2322
|
+
*
|
|
2323
|
+
* BASE_URL can be set to test against a preview deployment URL
|
|
2324
|
+
* or defaults to http://localhost:3000 for local dev server testing.
|
|
2325
|
+
*/
|
|
2326
|
+
export default defineConfig({
|
|
2327
|
+
testDir: '.',
|
|
2328
|
+
testMatch: '**/*.spec.ts',
|
|
2329
|
+
timeout: 60_000,
|
|
2330
|
+
retries: 0,
|
|
2331
|
+
use: {
|
|
2332
|
+
baseURL: process.env.BASE_URL || 'http://localhost:3000',
|
|
2333
|
+
headless: true,
|
|
2334
|
+
screenshot: 'only-on-failure',
|
|
2335
|
+
},
|
|
2336
|
+
projects: [
|
|
2337
|
+
{ name: 'chromium', use: { browserName: 'chromium' } },
|
|
2338
|
+
],
|
|
2339
|
+
});
|
|
2340
|
+
`,
|
|
2341
|
+
"scenarios/example-scenario-web.spec.ts": `/**
|
|
2342
|
+
* Example Web Scenario Test (Playwright)
|
|
2343
|
+
*
|
|
2344
|
+
* This file is a template for scenario tests against web applications.
|
|
2345
|
+
* The holdout pattern applies: test the running app through its UI,
|
|
2346
|
+
* never import source code from the main repo.
|
|
2347
|
+
*
|
|
2348
|
+
* The main repo is available at ../main-repo and is already built.
|
|
2349
|
+
* Tests run against either:
|
|
2350
|
+
* - A dev server started from ../main-repo (default)
|
|
2351
|
+
* - A preview deployment URL (set BASE_URL env var)
|
|
2352
|
+
*
|
|
2353
|
+
* DO:
|
|
2354
|
+
* - Navigate to pages, click elements, fill forms, assert on visible content
|
|
2355
|
+
* - Use page.locator() with accessible selectors (role, text, test-id)
|
|
2356
|
+
* - Keep each test fully independent
|
|
2357
|
+
*
|
|
2358
|
+
* DON'T:
|
|
2359
|
+
* - Import from ../main-repo/src \u2014 that defeats the holdout
|
|
2360
|
+
* - Test internal implementation details
|
|
2361
|
+
* - Rely on specific CSS classes or DOM structure (use accessible selectors)
|
|
2362
|
+
*/
|
|
2363
|
+
|
|
2364
|
+
import { test, expect } from '@playwright/test';
|
|
2365
|
+
import { spawn, type ChildProcess } from 'node:child_process';
|
|
2366
|
+
import { join } from 'node:path';
|
|
2367
|
+
|
|
2368
|
+
const MAIN_REPO = join(__dirname, '..', 'main-repo');
|
|
2369
|
+
let serverProcess: ChildProcess | undefined;
|
|
2370
|
+
|
|
2371
|
+
/**
|
|
2372
|
+
* Wait for a URL to become reachable.
|
|
2373
|
+
*/
|
|
2374
|
+
async function waitForServer(url: string, timeoutMs = 60_000): Promise<void> {
|
|
2375
|
+
const start = Date.now();
|
|
2376
|
+
while (Date.now() - start < timeoutMs) {
|
|
2377
|
+
try {
|
|
2378
|
+
const res = await fetch(url);
|
|
2379
|
+
if (res.ok || res.status < 500) return;
|
|
2380
|
+
} catch {
|
|
2381
|
+
// Server not ready yet
|
|
2382
|
+
}
|
|
2383
|
+
await new Promise(r => setTimeout(r, 1000));
|
|
2384
|
+
}
|
|
2385
|
+
throw new Error(\`Server at \${url} did not become ready within \${timeoutMs}ms\`);
|
|
2386
|
+
}
|
|
2387
|
+
|
|
2388
|
+
test.beforeAll(async () => {
|
|
2389
|
+
// If BASE_URL is set, skip starting a dev server \u2014 test against the provided URL
|
|
2390
|
+
if (process.env.BASE_URL) return;
|
|
2391
|
+
|
|
2392
|
+
serverProcess = spawn('npm', ['run', 'dev'], {
|
|
2393
|
+
cwd: MAIN_REPO,
|
|
2394
|
+
stdio: 'pipe',
|
|
2395
|
+
env: { ...process.env, PORT: '3000' },
|
|
2396
|
+
});
|
|
2397
|
+
|
|
2398
|
+
await waitForServer('http://localhost:3000');
|
|
2399
|
+
});
|
|
2400
|
+
|
|
2401
|
+
test.afterAll(async () => {
|
|
2402
|
+
if (serverProcess) {
|
|
2403
|
+
serverProcess.kill('SIGTERM');
|
|
2404
|
+
serverProcess = undefined;
|
|
2405
|
+
}
|
|
2406
|
+
});
|
|
2407
|
+
|
|
2408
|
+
// ---------------------------------------------------------------------------
|
|
2409
|
+
// Example scenarios \u2014 replace with real tests for your application
|
|
2410
|
+
// ---------------------------------------------------------------------------
|
|
2411
|
+
|
|
2412
|
+
test.describe('Home page', () => {
|
|
2413
|
+
test('loads successfully and shows main heading', async ({ page }) => {
|
|
2414
|
+
await page.goto('/');
|
|
2415
|
+
// Replace with your app's actual heading or key element
|
|
2416
|
+
await expect(page.locator('h1')).toBeVisible();
|
|
2417
|
+
});
|
|
2418
|
+
|
|
2419
|
+
test('navigates to a subpage', async ({ page }) => {
|
|
2420
|
+
await page.goto('/');
|
|
2421
|
+
// Replace with your app's actual navigation
|
|
2422
|
+
// await page.click('text=About');
|
|
2423
|
+
// await expect(page).toHaveURL(/\\/about/);
|
|
2424
|
+
// await expect(page.locator('h1')).toContainText('About');
|
|
2425
|
+
});
|
|
2426
|
+
});
|
|
2427
|
+
`,
|
|
2428
|
+
"scenarios/example-scenario-api.test.ts": `/**
|
|
2429
|
+
* Example API Scenario Test
|
|
2430
|
+
*
|
|
2431
|
+
* This file is a template for scenario tests against API-only backends.
|
|
2432
|
+
* The holdout pattern applies: test the running server via HTTP requests,
|
|
2433
|
+
* never import route handlers or source code from the main repo.
|
|
2434
|
+
*
|
|
2435
|
+
* The main repo is available at ../main-repo and is already built.
|
|
2436
|
+
* Tests run against either:
|
|
2437
|
+
* - A server started from ../main-repo (default)
|
|
2438
|
+
* - A deployed URL (set BASE_URL env var)
|
|
2439
|
+
*
|
|
2440
|
+
* Uses Node.js built-in fetch \u2014 no additional HTTP client dependencies.
|
|
2441
|
+
*
|
|
2442
|
+
* DO:
|
|
2443
|
+
* - Send HTTP requests to endpoints, assert on status codes and response bodies
|
|
2444
|
+
* - Test realistic user actions (create, read, update, delete flows)
|
|
2445
|
+
* - Keep each test fully independent
|
|
2446
|
+
*
|
|
2447
|
+
* DON'T:
|
|
2448
|
+
* - Import from ../main-repo/src \u2014 that defeats the holdout
|
|
2449
|
+
* - Use supertest or similar tools that import the app directly
|
|
2450
|
+
* - Test internal implementation details
|
|
2451
|
+
*/
|
|
2452
|
+
|
|
2453
|
+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
|
2454
|
+
import { spawn, type ChildProcess } from 'node:child_process';
|
|
2455
|
+
import { join } from 'node:path';
|
|
2456
|
+
|
|
2457
|
+
const MAIN_REPO = join(__dirname, '..', 'main-repo');
|
|
2458
|
+
const BASE_URL = process.env.BASE_URL || 'http://localhost:3000';
|
|
2459
|
+
let serverProcess: ChildProcess | undefined;
|
|
2460
|
+
|
|
2461
|
+
/**
|
|
2462
|
+
* Wait for a URL to become reachable.
|
|
2463
|
+
*/
|
|
2464
|
+
async function waitForServer(url: string, timeoutMs = 60_000): Promise<void> {
|
|
2465
|
+
const start = Date.now();
|
|
2466
|
+
while (Date.now() - start < timeoutMs) {
|
|
2467
|
+
try {
|
|
2468
|
+
const res = await fetch(url);
|
|
2469
|
+
if (res.ok || res.status < 500) return;
|
|
2470
|
+
} catch {
|
|
2471
|
+
// Server not ready yet
|
|
2472
|
+
}
|
|
2473
|
+
await new Promise(r => setTimeout(r, 1000));
|
|
2474
|
+
}
|
|
2475
|
+
throw new Error(\`Server at \${url} did not become ready within \${timeoutMs}ms\`);
|
|
2476
|
+
}
|
|
2477
|
+
|
|
2478
|
+
beforeAll(async () => {
|
|
2479
|
+
// If BASE_URL is set externally, skip starting a server
|
|
2480
|
+
if (process.env.BASE_URL) return;
|
|
2481
|
+
|
|
2482
|
+
serverProcess = spawn('npm', ['start'], {
|
|
2483
|
+
cwd: MAIN_REPO,
|
|
2484
|
+
stdio: 'pipe',
|
|
2485
|
+
env: { ...process.env, PORT: '3000' },
|
|
2486
|
+
});
|
|
2487
|
+
|
|
2488
|
+
await waitForServer(BASE_URL);
|
|
2489
|
+
}, 90_000);
|
|
2490
|
+
|
|
2491
|
+
afterAll(() => {
|
|
2492
|
+
if (serverProcess) {
|
|
2493
|
+
serverProcess.kill('SIGTERM');
|
|
2494
|
+
serverProcess = undefined;
|
|
2495
|
+
}
|
|
2496
|
+
});
|
|
2497
|
+
|
|
2498
|
+
// ---------------------------------------------------------------------------
|
|
2499
|
+
// Example scenarios \u2014 replace with real tests for your API
|
|
2500
|
+
// ---------------------------------------------------------------------------
|
|
2501
|
+
|
|
2502
|
+
describe('API health', () => {
|
|
2503
|
+
it('GET / returns a success status', async () => {
|
|
2504
|
+
const res = await fetch(\`\${BASE_URL}/\`);
|
|
2505
|
+
expect(res.status).toBeLessThan(500);
|
|
2506
|
+
});
|
|
2507
|
+
});
|
|
2508
|
+
|
|
2509
|
+
describe('API endpoints', () => {
|
|
2510
|
+
it('GET /api/example returns JSON', async () => {
|
|
2511
|
+
const res = await fetch(\`\${BASE_URL}/api/example\`);
|
|
2512
|
+
// Replace with your actual endpoint
|
|
2513
|
+
// expect(res.status).toBe(200);
|
|
2514
|
+
// const body = await res.json();
|
|
2515
|
+
// expect(body).toHaveProperty('data');
|
|
2516
|
+
});
|
|
2517
|
+
|
|
2518
|
+
it('POST /api/example creates a resource', async () => {
|
|
2519
|
+
// Replace with your actual endpoint and payload
|
|
2520
|
+
// const res = await fetch(\\\`\\\${BASE_URL}/api/example\\\`, {
|
|
2521
|
+
// method: 'POST',
|
|
2522
|
+
// headers: { 'Content-Type': 'application/json' },
|
|
2523
|
+
// body: JSON.stringify({ name: 'test' }),
|
|
2524
|
+
// });
|
|
2525
|
+
// expect(res.status).toBe(201);
|
|
2526
|
+
// const body = await res.json();
|
|
2527
|
+
// expect(body).toHaveProperty('id');
|
|
2528
|
+
});
|
|
2529
|
+
|
|
2530
|
+
it('returns 404 for unknown routes', async () => {
|
|
2531
|
+
const res = await fetch(\`\${BASE_URL}/api/does-not-exist\`);
|
|
2532
|
+
expect(res.status).toBe(404);
|
|
2533
|
+
});
|
|
2534
|
+
});
|
|
2535
|
+
`,
|
|
2536
|
+
"scenarios/example-scenario-mobile.yaml": `# Example Mobile Scenario Test (Maestro)
|
|
2537
|
+
#
|
|
2538
|
+
# This file is a template for scenario tests against mobile applications.
|
|
2539
|
+
# The holdout pattern applies: test the running app through its UI,
|
|
2540
|
+
# never reference source code from the main repo.
|
|
2541
|
+
#
|
|
2542
|
+
# Maestro tests are declarative YAML flows that interact with a running
|
|
2543
|
+
# app on a simulator/emulator. Install Maestro:
|
|
2544
|
+
# curl -Ls "https://get.maestro.mobile.dev" | bash
|
|
2545
|
+
#
|
|
2546
|
+
# Run this flow:
|
|
2547
|
+
# maestro test example-scenario-mobile.yaml
|
|
2548
|
+
#
|
|
2549
|
+
# DO:
|
|
2550
|
+
# - Tap elements, fill inputs, assert on visible text
|
|
2551
|
+
# - Use runFlow for reusable sub-flows (e.g., login)
|
|
2552
|
+
# - Use assertWithAI for natural-language assertions
|
|
2553
|
+
#
|
|
2554
|
+
# DON'T:
|
|
2555
|
+
# - Reference source code paths or internal identifiers
|
|
2556
|
+
# - Depend on exact pixel positions (use text and accessibility labels)
|
|
2557
|
+
|
|
2558
|
+
appId: com.example.myapp # Replace with your app's bundle identifier
|
|
2559
|
+
name: "Core User Journey"
|
|
2560
|
+
tags:
|
|
2561
|
+
- smoke
|
|
2562
|
+
- holdout
|
|
2563
|
+
---
|
|
2564
|
+
# Step 1: Launch the app
|
|
2565
|
+
- launchApp
|
|
2566
|
+
|
|
2567
|
+
# Step 2: Login (using a reusable sub-flow)
|
|
2568
|
+
- runFlow: example-scenario-mobile-login.yaml
|
|
2569
|
+
|
|
2570
|
+
# Step 3: Verify the main screen loaded
|
|
2571
|
+
- assertVisible: "Home"
|
|
2572
|
+
|
|
2573
|
+
# Step 4: Navigate to a feature
|
|
2574
|
+
# - tapOn: "Settings"
|
|
2575
|
+
# - assertVisible: "Account"
|
|
2576
|
+
|
|
2577
|
+
# Step 5: AI-powered assertion (natural language)
|
|
2578
|
+
# - assertWithAI: "The main dashboard is visible with navigation tabs at the bottom"
|
|
2579
|
+
|
|
2580
|
+
# Step 6: Go back
|
|
2581
|
+
# - back
|
|
2582
|
+
# - assertVisible: "Home"
|
|
2583
|
+
`,
|
|
2584
|
+
"scenarios/example-scenario-mobile-login.yaml": `# Reusable Login Sub-Flow (Maestro)
|
|
2585
|
+
#
|
|
2586
|
+
# This flow handles authentication. Other flows include it via:
|
|
2587
|
+
# - runFlow: example-scenario-mobile-login.yaml
|
|
2588
|
+
#
|
|
2589
|
+
# Replace the selectors and credentials with your app's actual login flow.
|
|
2590
|
+
|
|
2591
|
+
appId: com.example.myapp
|
|
2592
|
+
name: "Login"
|
|
2593
|
+
---
|
|
2594
|
+
- assertVisible: "Sign In"
|
|
2595
|
+
- tapOn: "Email"
|
|
2596
|
+
- inputText: "test@example.com"
|
|
2597
|
+
- tapOn: "Password"
|
|
2598
|
+
- inputText: "testpassword123"
|
|
2599
|
+
- tapOn: "Log In"
|
|
2600
|
+
- assertVisible: "Home" # Verify login succeeded
|
|
2601
|
+
`,
|
|
2602
|
+
"scenarios/README-mobile.md": '# Mobile Scenario Testing with Maestro\n\nThis guide explains how to set up and run mobile holdout scenario tests using [Maestro](https://maestro.dev/).\n\n## Prerequisites\n\n- **Maestro CLI:** `curl -Ls "https://get.maestro.mobile.dev" | bash`\n- **Java 17+** (required by Maestro)\n- **Simulator/Emulator:**\n - iOS: Xcode with iOS Simulator (macOS only)\n - Android: Android Studio with an AVD configured\n\n> **Important:** Joycraft does not install Maestro or manage simulators. This is your responsibility.\n\n## Running Tests Locally\n\n```bash\n# Boot your simulator/emulator first, then:\nmaestro test example-scenario-mobile.yaml\n\n# Run all flows in a directory:\nmaestro test .maestro/\n```\n\n## Writing Flows\n\nMaestro flows are declarative YAML. Core commands:\n\n| Command | Purpose |\n|---------|--------|\n| `launchApp` | Start or restart the app |\n| `tapOn: "text"` | Tap an element by visible text or test ID |\n| `inputText: "value"` | Type into a focused field |\n| `assertVisible: "text"` | Assert an element is on screen |\n| `assertNotVisible: "text"` | Assert an element is NOT on screen |\n| `scroll` | Scroll down |\n| `back` | Press the back button |\n| `runFlow: file.yaml` | Run a reusable sub-flow |\n| `assertWithAI: "description"` | Natural-language assertion (AI-powered) |\n\n## CI Options\n\n### Option A: Maestro Cloud (paid, easiest)\n\nUpload your app binary and flows to Maestro Cloud. No simulator management.\n\n```yaml\n- uses: mobile-dev-inc/action-maestro-cloud@v2\n with:\n api-key: ${{ secrets.MAESTRO_API_KEY }}\n app-file: app.apk # or app.ipa\n workspace: .\n```\n\n### Option B: Self-hosted emulator (free, more setup)\n\nSpin up an Android emulator on a Linux runner or iOS simulator on a macOS runner.\n\n> **Cost note:** macOS GitHub Actions runners are ~10x more expensive than Linux runners.\n\n## The Holdout Pattern\n\nThese tests live in the scenarios repo, separate from the main codebase. The scenario agent generates them from specs. They test observable behavior through the app\'s UI \u2014 never referencing source code or internal implementation.\n',
|
|
2088
2603
|
"scenarios/prompts/scenario-agent.md": `You are a QA engineer working in a holdout test repository. You CANNOT access the main repository's source code. Your job is to write or update behavioral scenario tests based on specs that are pushed from the main repo.
|
|
2089
2604
|
|
|
2090
2605
|
## What You Have Access To
|
|
@@ -2092,7 +2607,23 @@ describe("CLI: init command (example \u2014 replace with your real scenarios)",
|
|
|
2092
2607
|
- This scenarios repository (test files, \`specs/\` mirror, \`package.json\`)
|
|
2093
2608
|
- The incoming spec (provided below)
|
|
2094
2609
|
- A list of existing test files and spec mirrors (provided below)
|
|
2095
|
-
- The main repo is available at \`../main-repo\` and is already built
|
|
2610
|
+
- The main repo is available at \`../main-repo\` and is already built
|
|
2611
|
+
- The testing strategy for this project (provided below)
|
|
2612
|
+
|
|
2613
|
+
## Testing Strategy
|
|
2614
|
+
|
|
2615
|
+
This project uses the **$TESTING_BACKBONE** testing backbone.
|
|
2616
|
+
|
|
2617
|
+
Select the correct test format based on the backbone:
|
|
2618
|
+
|
|
2619
|
+
| Backbone | Tool | Test Format | File Extension | How to Test |
|
|
2620
|
+
|----------|------|-------------|---------------|-------------|
|
|
2621
|
+
| \`playwright\` | Playwright | Browser-based E2E | \`.spec.ts\` | Navigate pages, click elements, assert on visible content |
|
|
2622
|
+
| \`maestro\` | Maestro | YAML flows | \`.yaml\` | Tap elements, fill inputs, assert on screen state |
|
|
2623
|
+
| \`api\` | fetch (Node.js built-in) | HTTP requests | \`.test.ts\` | Send requests to endpoints, assert on responses |
|
|
2624
|
+
| \`native\` | vitest + spawnSync | CLI/binary invocation | \`.test.ts\` | Run commands, assert on stdout/stderr/exit codes |
|
|
2625
|
+
|
|
2626
|
+
If the backbone is not provided or unrecognized, default to \`native\`.
|
|
2096
2627
|
|
|
2097
2628
|
## Triage Decision Tree
|
|
2098
2629
|
|
|
@@ -2111,7 +2642,7 @@ If you SKIP, write a brief comment in the relevant test file (or a new one) expl
|
|
|
2111
2642
|
- A new output format or file that gets generated
|
|
2112
2643
|
- A new user-facing behavior that doesn't map to any existing test file
|
|
2113
2644
|
|
|
2114
|
-
Name the file after the feature area
|
|
2645
|
+
Name the file after the feature area using the correct extension for the backbone.
|
|
2115
2646
|
|
|
2116
2647
|
### UPDATE \u2014 Modify an existing test file if the spec:
|
|
2117
2648
|
- Changes behavior that is already tested
|
|
@@ -2122,25 +2653,20 @@ Match to the most relevant existing test file by feature area.
|
|
|
2122
2653
|
|
|
2123
2654
|
**If you are unsure whether a spec is user-facing, err on the side of writing a test.**
|
|
2124
2655
|
|
|
2125
|
-
## Test Writing Rules
|
|
2126
|
-
|
|
2127
|
-
1. **Behavioral only.** Test observable output \u2014 stdout, stderr, exit codes, files created/modified on disk. Never test internal implementation details or import source modules.
|
|
2656
|
+
## Test Writing Rules (All Backbones)
|
|
2128
2657
|
|
|
2129
|
-
|
|
2658
|
+
1. **Behavioral only.** Test observable behavior \u2014 what a real user would see. Never test internal implementation details or import source modules.
|
|
2659
|
+
2. **Each test is fully independent.** No shared mutable state between tests.
|
|
2660
|
+
3. **Assert on realistic user actions.** Write tests that reflect what a real user would do.
|
|
2661
|
+
4. **Never import from the parent repo's source.** If you find yourself writing \`import { ... } from '../main-repo/src/...'\`, stop \u2014 that defeats the holdout.
|
|
2130
2662
|
|
|
2131
|
-
|
|
2663
|
+
## Backbone: native (CLI/Binary)
|
|
2132
2664
|
|
|
2133
|
-
|
|
2134
|
-
|
|
2135
|
-
5. **Assert on realistic user actions.** Write tests that reflect what a real user would do \u2014 not what the implementation happens to do.
|
|
2136
|
-
|
|
2137
|
-
6. **Never import from the parent repo's source.** If you find yourself writing \`import { ... } from '../main-repo/src/...'\`, stop \u2014 that defeats the holdout.
|
|
2138
|
-
|
|
2139
|
-
## Test File Template
|
|
2665
|
+
Use when the project is a CLI tool, library, or has no web/mobile UI.
|
|
2140
2666
|
|
|
2141
2667
|
\`\`\`typescript
|
|
2142
|
-
import {
|
|
2143
|
-
import {
|
|
2668
|
+
import { spawnSync } from 'node:child_process';
|
|
2669
|
+
import { mkdtempSync, rmSync } from 'node:fs';
|
|
2144
2670
|
import { tmpdir } from 'node:os';
|
|
2145
2671
|
import { join } from 'node:path';
|
|
2146
2672
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
|
@@ -2153,39 +2679,122 @@ function runCLI(args: string[], cwd?: string) {
|
|
|
2153
2679
|
cwd: cwd ?? process.cwd(),
|
|
2154
2680
|
env: { ...process.env, NO_COLOR: '1' },
|
|
2155
2681
|
});
|
|
2156
|
-
return {
|
|
2157
|
-
stdout: result.stdout ?? '',
|
|
2158
|
-
stderr: result.stderr ?? '',
|
|
2159
|
-
status: result.status ?? 1,
|
|
2160
|
-
};
|
|
2682
|
+
return { stdout: result.stdout ?? '', stderr: result.stderr ?? '', status: result.status ?? 1 };
|
|
2161
2683
|
}
|
|
2162
2684
|
|
|
2163
|
-
describe('[feature area]
|
|
2685
|
+
describe('[feature area]', () => {
|
|
2164
2686
|
let tmpDir: string;
|
|
2687
|
+
beforeEach(() => { tmpDir = mkdtempSync(join(tmpdir(), 'scenarios-')); });
|
|
2688
|
+
afterEach(() => { rmSync(tmpDir, { recursive: true, force: true }); });
|
|
2165
2689
|
|
|
2166
|
-
|
|
2167
|
-
|
|
2690
|
+
it('[observable behavior]', () => {
|
|
2691
|
+
const { stdout, status } = runCLI(['command', 'args'], tmpDir);
|
|
2692
|
+
expect(status).toBe(0);
|
|
2693
|
+
expect(stdout).toContain('expected output');
|
|
2168
2694
|
});
|
|
2695
|
+
});
|
|
2696
|
+
\`\`\`
|
|
2169
2697
|
|
|
2170
|
-
|
|
2171
|
-
|
|
2698
|
+
## Backbone: playwright (Web Apps)
|
|
2699
|
+
|
|
2700
|
+
Use when the project is a web application (Next.js, Vite, Nuxt, etc.).
|
|
2701
|
+
|
|
2702
|
+
\`\`\`typescript
|
|
2703
|
+
import { test, expect } from '@playwright/test';
|
|
2704
|
+
|
|
2705
|
+
// Tests run against BASE_URL (configured in playwright.config.ts)
|
|
2706
|
+
// The dev server is started automatically or BASE_URL points to a preview deploy
|
|
2707
|
+
|
|
2708
|
+
test.describe('[feature area]', () => {
|
|
2709
|
+
test('[observable behavior]', async ({ page }) => {
|
|
2710
|
+
await page.goto('/');
|
|
2711
|
+
await expect(page.locator('h1')).toBeVisible();
|
|
2172
2712
|
});
|
|
2173
2713
|
|
|
2174
|
-
|
|
2175
|
-
|
|
2176
|
-
|
|
2177
|
-
|
|
2714
|
+
test('[user interaction]', async ({ page }) => {
|
|
2715
|
+
await page.goto('/login');
|
|
2716
|
+
await page.fill('[name="email"]', 'test@example.com');
|
|
2717
|
+
await page.click('button[type="submit"]');
|
|
2718
|
+
await expect(page).toHaveURL(/dashboard/);
|
|
2178
2719
|
});
|
|
2179
2720
|
});
|
|
2180
2721
|
\`\`\`
|
|
2181
2722
|
|
|
2723
|
+
## Backbone: api (API Backends)
|
|
2724
|
+
|
|
2725
|
+
Use when the project is an API-only backend (Express, FastAPI, etc.).
|
|
2726
|
+
|
|
2727
|
+
\`\`\`typescript
|
|
2728
|
+
import { describe, it, expect } from 'vitest';
|
|
2729
|
+
|
|
2730
|
+
const BASE_URL = process.env.BASE_URL || 'http://localhost:3000';
|
|
2731
|
+
|
|
2732
|
+
describe('[feature area]', () => {
|
|
2733
|
+
it('[endpoint behavior]', async () => {
|
|
2734
|
+
const res = await fetch(\\\`\\\${BASE_URL}/api/endpoint\\\`);
|
|
2735
|
+
expect(res.status).toBe(200);
|
|
2736
|
+
const body = await res.json();
|
|
2737
|
+
expect(body).toHaveProperty('data');
|
|
2738
|
+
});
|
|
2739
|
+
|
|
2740
|
+
it('[error handling]', async () => {
|
|
2741
|
+
const res = await fetch(\\\`\\\${BASE_URL}/api/not-found\\\`);
|
|
2742
|
+
expect(res.status).toBe(404);
|
|
2743
|
+
});
|
|
2744
|
+
});
|
|
2745
|
+
\`\`\`
|
|
2746
|
+
|
|
2747
|
+
## Backbone: maestro (Mobile Apps)
|
|
2748
|
+
|
|
2749
|
+
Use when the project is a mobile application (React Native, Flutter, native iOS/Android).
|
|
2750
|
+
|
|
2751
|
+
\`\`\`yaml
|
|
2752
|
+
appId: com.example.myapp
|
|
2753
|
+
name: "[feature area]: [behavior being tested]"
|
|
2754
|
+
tags:
|
|
2755
|
+
- holdout
|
|
2756
|
+
---
|
|
2757
|
+
- launchApp
|
|
2758
|
+
- tapOn: "Sign In"
|
|
2759
|
+
- inputText: "test@example.com"
|
|
2760
|
+
- tapOn: "Submit"
|
|
2761
|
+
- assertVisible: "Welcome"
|
|
2762
|
+
# Use assertWithAI for complex visual assertions:
|
|
2763
|
+
# - assertWithAI: "The dashboard shows a list of recent items"
|
|
2764
|
+
\`\`\`
|
|
2765
|
+
|
|
2766
|
+
## Graceful Degradation
|
|
2767
|
+
|
|
2768
|
+
If the primary backbone tool is not available in this repo, fall back to the next deepest testable layer:
|
|
2769
|
+
|
|
2770
|
+
| Layer | What's Tested | When to Use |
|
|
2771
|
+
|-------|-------------|-------------|
|
|
2772
|
+
| **Layer 4: UI** | Full user flows through browser/simulator | \`@playwright/test\` or Maestro is installed |
|
|
2773
|
+
| **Layer 3: API** | HTTP requests against running server | Server can be started from \`../main-repo\` |
|
|
2774
|
+
| **Layer 2: Logic** | Unit tests via test runner | Test runner (vitest/jest) is available |
|
|
2775
|
+
| **Layer 1: Static** | Build, typecheck, lint | Build toolchain is available |
|
|
2776
|
+
|
|
2777
|
+
**Fallback rules:**
|
|
2778
|
+
- If backbone is \`playwright\` but \`@playwright/test\` is NOT in this repo's \`package.json\`: fall back to \`api\` (fetch-based HTTP tests)
|
|
2779
|
+
- If backbone is \`maestro\` but no simulator context is available: fall back to \`api\` if a server can be started, else \`native\`
|
|
2780
|
+
- If backbone is \`api\` but no server start script exists: fall back to \`native\`
|
|
2781
|
+
- \`native\` is always available as the floor
|
|
2782
|
+
|
|
2783
|
+
Start each test file with a comment indicating the testing layer:
|
|
2784
|
+
\`// Testing Layer: [4|3|2|1] - [UI|API|Logic|Static]\`
|
|
2785
|
+
|
|
2786
|
+
If you fell back from the intended backbone, note this in your commit message:
|
|
2787
|
+
\`scenarios: [action] for [spec] (layer: [N], reason: [why])\`
|
|
2788
|
+
|
|
2182
2789
|
## Checklist Before Committing
|
|
2183
2790
|
|
|
2184
2791
|
- [ ] Decision: SKIP / NEW / UPDATE (and why)
|
|
2792
|
+
- [ ] Correct backbone selected (or fallback justified)
|
|
2185
2793
|
- [ ] Tests assert on observable behavior, not implementation
|
|
2186
2794
|
- [ ] No imports from \`../main-repo/src\`
|
|
2187
|
-
- [ ] Each test
|
|
2188
|
-
- [ ] File
|
|
2795
|
+
- [ ] Each test is independent (own temp dir, own state)
|
|
2796
|
+
- [ ] File uses the correct extension for the backbone
|
|
2797
|
+
- [ ] Testing layer comment at top of file
|
|
2189
2798
|
`,
|
|
2190
2799
|
"scenarios/workflows/generate.yml": `# Scenario Generation Workflow
|
|
2191
2800
|
#
|
|
@@ -2285,7 +2894,9 @@ jobs:
|
|
|
2285
2894
|
## Context
|
|
2286
2895
|
|
|
2287
2896
|
Existing test files in this repo: \${{ steps.context.outputs.existing_tests }}
|
|
2288
|
-
Existing spec mirrors: \${{ steps.context.outputs.existing_specs }}
|
|
2897
|
+
Existing spec mirrors: \${{ steps.context.outputs.existing_specs }}
|
|
2898
|
+
|
|
2899
|
+
Testing backbone: \${{ github.event.client_payload.testing_backbone || 'native' }}"
|
|
2289
2900
|
|
|
2290
2901
|
# \u2500\u2500 7. Commit any changes the agent made \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
|
|
2291
2902
|
- name: Commit scenario changes
|
|
@@ -2861,4 +3472,4 @@ export {
|
|
|
2861
3472
|
SKILLS,
|
|
2862
3473
|
TEMPLATES
|
|
2863
3474
|
};
|
|
2864
|
-
//# sourceMappingURL=chunk-
|
|
3475
|
+
//# sourceMappingURL=chunk-Y6GBN6R4.js.map
|