joycraft 0.5.7 → 0.5.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1874,13 +1874,59 @@ is required, though you can add one via the GitHub Checks API if you prefer.
1874
1874
 
1875
1875
  ---
1876
1876
 
1877
+ ## Testing by Stack Type
1878
+
1879
+ The scenario agent selects the appropriate test format based on the project's
1880
+ testing backbone. Each backbone tests the same holdout principle \u2014 observable
1881
+ behavior only, no source imports \u2014 but uses different tools.
1882
+
1883
+ ### Web Apps (Playwright)
1884
+
1885
+ For Next.js, Vite, Nuxt, Remix, and other web frameworks. Tests run against a
1886
+ dev server or preview URL using a headless browser.
1887
+
1888
+ - **Template:** \`example-scenario-web.spec.ts\`
1889
+ - **Config:** \`playwright.config.ts\`
1890
+ - **Package:** \`package-web.json\` (use instead of \`package.json\` for web projects)
1891
+ - **Run:** \`npx playwright test\`
1892
+
1893
+ ### Mobile Apps (Maestro)
1894
+
1895
+ For React Native, Flutter, and native iOS/Android. Tests are declarative YAML
1896
+ flows that interact with a running app on a simulator.
1897
+
1898
+ - **Template:** \`example-scenario-mobile.yaml\`
1899
+ - **Login sub-flow:** \`example-scenario-mobile-login.yaml\`
1900
+ - **Setup guide:** \`README-mobile.md\`
1901
+ - **Run:** \`maestro test example-scenario-mobile.yaml\`
1902
+
1903
+ ### API Backends (HTTP)
1904
+
1905
+ For Express, FastAPI, Django, and other API-only backends. Tests send HTTP
1906
+ requests using Node.js built-in \`fetch\`.
1907
+
1908
+ - **Template:** \`example-scenario-api.test.ts\`
1909
+ - **Run:** \`npx vitest run\`
1910
+
1911
+ ### CLI Tools & Libraries (native)
1912
+
1913
+ For CLI tools, npm packages, and non-UI projects. Tests invoke the built
1914
+ binary via \`spawnSync\` and assert on stdout/stderr.
1915
+
1916
+ - **Template:** \`example-scenario.test.ts\`
1917
+ - **Run:** \`npx vitest run\`
1918
+
1919
+ ---
1920
+
1877
1921
  ## Adding scenarios
1878
1922
 
1879
1923
  ### Rules
1880
1924
 
1881
- 1. **Behavioral, not structural.** Test what the tool does, not how it is
1882
- built internally. Invoke the binary; assert on stdout, exit codes, and
1883
- filesystem state. Never import from \`../main-repo/src\`.
1925
+ These rules apply to ALL backbones:
1926
+
1927
+ 1. **Behavioral, not structural.** Test what the app does from a user's
1928
+ perspective. For web: navigate and assert on content. For CLI: run commands
1929
+ and check output. For API: send requests and check responses.
1884
1930
 
1885
1931
  2. **End-to-end.** Each test should represent something a real user would
1886
1932
  actually do. If you would not put it in a demo or docs example, reconsider
@@ -1890,9 +1936,8 @@ is required, though you can add one via the GitHub Checks API if you prefer.
1890
1936
  see source code. Any \`import\` that reaches into \`../main-repo/src\` breaks
1891
1937
  the pattern.
1892
1938
 
1893
- 4. **Independent.** Each test must be able to run in isolation. Use \`beforeEach\`
1894
- / \`afterEach\` to set up and tear down temp directories. Do not share mutable
1895
- state between tests.
1939
+ 4. **Independent.** Each test must be able to run in isolation. No shared
1940
+ mutable state between tests.
1896
1941
 
1897
1942
  5. **Deterministic.** Avoid network calls, timestamps, or random values in
1898
1943
  assertions unless the feature under test genuinely involves them.
@@ -1901,31 +1946,25 @@ is required, though you can add one via the GitHub Checks API if you prefer.
1901
1946
 
1902
1947
  \`\`\`
1903
1948
  $SCENARIOS_REPO/
1904
- \u251C\u2500\u2500 example-scenario.test.ts # Starter file \u2014 replace with real scenarios
1949
+ \u251C\u2500\u2500 example-scenario.test.ts # CLI/binary scenario template
1950
+ \u251C\u2500\u2500 example-scenario-web.spec.ts # Web app scenario template (Playwright)
1951
+ \u251C\u2500\u2500 example-scenario-api.test.ts # API backend scenario template
1952
+ \u251C\u2500\u2500 example-scenario-mobile.yaml # Mobile app scenario template (Maestro)
1953
+ \u251C\u2500\u2500 example-scenario-mobile-login.yaml # Reusable login sub-flow
1954
+ \u251C\u2500\u2500 playwright.config.ts # Playwright config (web projects)
1955
+ \u251C\u2500\u2500 package.json # Default (vitest for CLI/API)
1956
+ \u251C\u2500\u2500 package-web.json # Alternative (Playwright for web)
1957
+ \u251C\u2500\u2500 README-mobile.md # Mobile testing setup guide
1905
1958
  \u251C\u2500\u2500 workflows/
1906
- \u2502 \u2514\u2500\u2500 run.yml # CI workflow (do not rename)
1907
- \u251C\u2500\u2500 package.json
1959
+ \u2502 \u251C\u2500\u2500 run.yml # CI workflow (do not rename)
1960
+ \u2502 \u2514\u2500\u2500 generate.yml # Scenario generation workflow
1961
+ \u251C\u2500\u2500 prompts/
1962
+ \u2502 \u2514\u2500\u2500 scenario-agent.md # Scenario agent instructions
1908
1963
  \u2514\u2500\u2500 README.md
1909
1964
  \`\`\`
1910
1965
 
1911
- Add new \`.test.ts\` files at the top level or in subdirectories. Vitest will
1912
- discover them automatically.
1913
-
1914
- ### Example structure
1915
-
1916
- \`\`\`ts
1917
- import { spawnSync } from "node:child_process";
1918
- import { join } from "node:path";
1919
-
1920
- const CLI = join(__dirname, "..", "main-repo", "dist", "cli.js");
1921
-
1922
- it("init creates a CLAUDE.md file", () => {
1923
- const tmp = mkdtempSync(join(tmpdir(), "scenario-"));
1924
- const { status } = spawnSync("node", [CLI, "init", tmp], { encoding: "utf8" });
1925
- expect(status).toBe(0);
1926
- expect(existsSync(join(tmp, "CLAUDE.md"))).toBe(true);
1927
- });
1928
- \`\`\`
1966
+ Use the template that matches your project's stack. Remove the ones you
1967
+ don't need.
1929
1968
 
1930
1969
  ---
1931
1970
 
@@ -1937,6 +1976,7 @@ it("init creates a CLAUDE.md file", () => {
1937
1976
  | Visible to agent | Yes | No |
1938
1977
  | What they test | Units, modules, logic | End-to-end behavior |
1939
1978
  | Import source code | Yes | Never |
1979
+ | Test method | Unit test framework | Depends on backbone (Playwright/Maestro/vitest/fetch) |
1940
1980
  | Run on every push | Yes | Yes (via dispatch) |
1941
1981
  | Purpose | Catch regressions fast | Validate real behavior |
1942
1982
 
@@ -2085,6 +2125,304 @@ describe("CLI: init command (example \u2014 replace with your real scenarios)",
2085
2125
  }
2086
2126
  }
2087
2127
  `,
2128
+ "scenarios/package-web.json": `{
2129
+ "name": "$SCENARIOS_REPO",
2130
+ "version": "0.0.1",
2131
+ "private": true,
2132
+ "type": "module",
2133
+ "scripts": {
2134
+ "test": "playwright test"
2135
+ },
2136
+ "devDependencies": {
2137
+ "@playwright/test": "^1.50.0"
2138
+ }
2139
+ }
2140
+ `,
2141
+ "scenarios/playwright.config.ts": `import { defineConfig } from '@playwright/test';
2142
+
2143
+ /**
2144
+ * Playwright configuration for holdout scenario tests.
2145
+ *
2146
+ * BASE_URL can be set to test against a preview deployment URL
2147
+ * or defaults to http://localhost:3000 for local dev server testing.
2148
+ */
2149
+ export default defineConfig({
2150
+ testDir: '.',
2151
+ testMatch: '**/*.spec.ts',
2152
+ timeout: 60_000,
2153
+ retries: 0,
2154
+ use: {
2155
+ baseURL: process.env.BASE_URL || 'http://localhost:3000',
2156
+ headless: true,
2157
+ screenshot: 'only-on-failure',
2158
+ },
2159
+ projects: [
2160
+ { name: 'chromium', use: { browserName: 'chromium' } },
2161
+ ],
2162
+ });
2163
+ `,
2164
+ "scenarios/example-scenario-web.spec.ts": `/**
2165
+ * Example Web Scenario Test (Playwright)
2166
+ *
2167
+ * This file is a template for scenario tests against web applications.
2168
+ * The holdout pattern applies: test the running app through its UI,
2169
+ * never import source code from the main repo.
2170
+ *
2171
+ * The main repo is available at ../main-repo and is already built.
2172
+ * Tests run against either:
2173
+ * - A dev server started from ../main-repo (default)
2174
+ * - A preview deployment URL (set BASE_URL env var)
2175
+ *
2176
+ * DO:
2177
+ * - Navigate to pages, click elements, fill forms, assert on visible content
2178
+ * - Use page.locator() with accessible selectors (role, text, test-id)
2179
+ * - Keep each test fully independent
2180
+ *
2181
+ * DON'T:
2182
+ * - Import from ../main-repo/src \u2014 that defeats the holdout
2183
+ * - Test internal implementation details
2184
+ * - Rely on specific CSS classes or DOM structure (use accessible selectors)
2185
+ */
2186
+
2187
+ import { test, expect } from '@playwright/test';
2188
+ import { spawn, type ChildProcess } from 'node:child_process';
2189
+ import { join } from 'node:path';
2190
+
2191
+ const MAIN_REPO = join(__dirname, '..', 'main-repo');
2192
+ let serverProcess: ChildProcess | undefined;
2193
+
2194
+ /**
2195
+ * Wait for a URL to become reachable.
2196
+ */
2197
+ async function waitForServer(url: string, timeoutMs = 60_000): Promise<void> {
2198
+ const start = Date.now();
2199
+ while (Date.now() - start < timeoutMs) {
2200
+ try {
2201
+ const res = await fetch(url);
2202
+ if (res.ok || res.status < 500) return;
2203
+ } catch {
2204
+ // Server not ready yet
2205
+ }
2206
+ await new Promise(r => setTimeout(r, 1000));
2207
+ }
2208
+ throw new Error(\`Server at \${url} did not become ready within \${timeoutMs}ms\`);
2209
+ }
2210
+
2211
+ test.beforeAll(async () => {
2212
+ // If BASE_URL is set, skip starting a dev server \u2014 test against the provided URL
2213
+ if (process.env.BASE_URL) return;
2214
+
2215
+ serverProcess = spawn('npm', ['run', 'dev'], {
2216
+ cwd: MAIN_REPO,
2217
+ stdio: 'pipe',
2218
+ env: { ...process.env, PORT: '3000' },
2219
+ });
2220
+
2221
+ await waitForServer('http://localhost:3000');
2222
+ });
2223
+
2224
+ test.afterAll(async () => {
2225
+ if (serverProcess) {
2226
+ serverProcess.kill('SIGTERM');
2227
+ serverProcess = undefined;
2228
+ }
2229
+ });
2230
+
2231
+ // ---------------------------------------------------------------------------
2232
+ // Example scenarios \u2014 replace with real tests for your application
2233
+ // ---------------------------------------------------------------------------
2234
+
2235
+ test.describe('Home page', () => {
2236
+ test('loads successfully and shows main heading', async ({ page }) => {
2237
+ await page.goto('/');
2238
+ // Replace with your app's actual heading or key element
2239
+ await expect(page.locator('h1')).toBeVisible();
2240
+ });
2241
+
2242
+ test('navigates to a subpage', async ({ page }) => {
2243
+ await page.goto('/');
2244
+ // Replace with your app's actual navigation
2245
+ // await page.click('text=About');
2246
+ // await expect(page).toHaveURL(/\\/about/);
2247
+ // await expect(page.locator('h1')).toContainText('About');
2248
+ });
2249
+ });
2250
+ `,
2251
+ "scenarios/example-scenario-api.test.ts": `/**
2252
+ * Example API Scenario Test
2253
+ *
2254
+ * This file is a template for scenario tests against API-only backends.
2255
+ * The holdout pattern applies: test the running server via HTTP requests,
2256
+ * never import route handlers or source code from the main repo.
2257
+ *
2258
+ * The main repo is available at ../main-repo and is already built.
2259
+ * Tests run against either:
2260
+ * - A server started from ../main-repo (default)
2261
+ * - A deployed URL (set BASE_URL env var)
2262
+ *
2263
+ * Uses Node.js built-in fetch \u2014 no additional HTTP client dependencies.
2264
+ *
2265
+ * DO:
2266
+ * - Send HTTP requests to endpoints, assert on status codes and response bodies
2267
+ * - Test realistic user actions (create, read, update, delete flows)
2268
+ * - Keep each test fully independent
2269
+ *
2270
+ * DON'T:
2271
+ * - Import from ../main-repo/src \u2014 that defeats the holdout
2272
+ * - Use supertest or similar tools that import the app directly
2273
+ * - Test internal implementation details
2274
+ */
2275
+
2276
+ import { describe, it, expect, beforeAll, afterAll } from 'vitest';
2277
+ import { spawn, type ChildProcess } from 'node:child_process';
2278
+ import { join } from 'node:path';
2279
+
2280
+ const MAIN_REPO = join(__dirname, '..', 'main-repo');
2281
+ const BASE_URL = process.env.BASE_URL || 'http://localhost:3000';
2282
+ let serverProcess: ChildProcess | undefined;
2283
+
2284
+ /**
2285
+ * Wait for a URL to become reachable.
2286
+ */
2287
+ async function waitForServer(url: string, timeoutMs = 60_000): Promise<void> {
2288
+ const start = Date.now();
2289
+ while (Date.now() - start < timeoutMs) {
2290
+ try {
2291
+ const res = await fetch(url);
2292
+ if (res.ok || res.status < 500) return;
2293
+ } catch {
2294
+ // Server not ready yet
2295
+ }
2296
+ await new Promise(r => setTimeout(r, 1000));
2297
+ }
2298
+ throw new Error(\`Server at \${url} did not become ready within \${timeoutMs}ms\`);
2299
+ }
2300
+
2301
+ beforeAll(async () => {
2302
+ // If BASE_URL is set externally, skip starting a server
2303
+ if (process.env.BASE_URL) return;
2304
+
2305
+ serverProcess = spawn('npm', ['start'], {
2306
+ cwd: MAIN_REPO,
2307
+ stdio: 'pipe',
2308
+ env: { ...process.env, PORT: '3000' },
2309
+ });
2310
+
2311
+ await waitForServer(BASE_URL);
2312
+ }, 90_000);
2313
+
2314
+ afterAll(() => {
2315
+ if (serverProcess) {
2316
+ serverProcess.kill('SIGTERM');
2317
+ serverProcess = undefined;
2318
+ }
2319
+ });
2320
+
2321
+ // ---------------------------------------------------------------------------
2322
+ // Example scenarios \u2014 replace with real tests for your API
2323
+ // ---------------------------------------------------------------------------
2324
+
2325
+ describe('API health', () => {
2326
+ it('GET / returns a success status', async () => {
2327
+ const res = await fetch(\`\${BASE_URL}/\`);
2328
+ expect(res.status).toBeLessThan(500);
2329
+ });
2330
+ });
2331
+
2332
+ describe('API endpoints', () => {
2333
+ it('GET /api/example returns JSON', async () => {
2334
+ const res = await fetch(\`\${BASE_URL}/api/example\`);
2335
+ // Replace with your actual endpoint
2336
+ // expect(res.status).toBe(200);
2337
+ // const body = await res.json();
2338
+ // expect(body).toHaveProperty('data');
2339
+ });
2340
+
2341
+ it('POST /api/example creates a resource', async () => {
2342
+ // Replace with your actual endpoint and payload
2343
+ // const res = await fetch(\\\`\\\${BASE_URL}/api/example\\\`, {
2344
+ // method: 'POST',
2345
+ // headers: { 'Content-Type': 'application/json' },
2346
+ // body: JSON.stringify({ name: 'test' }),
2347
+ // });
2348
+ // expect(res.status).toBe(201);
2349
+ // const body = await res.json();
2350
+ // expect(body).toHaveProperty('id');
2351
+ });
2352
+
2353
+ it('returns 404 for unknown routes', async () => {
2354
+ const res = await fetch(\`\${BASE_URL}/api/does-not-exist\`);
2355
+ expect(res.status).toBe(404);
2356
+ });
2357
+ });
2358
+ `,
2359
+ "scenarios/example-scenario-mobile.yaml": `# Example Mobile Scenario Test (Maestro)
2360
+ #
2361
+ # This file is a template for scenario tests against mobile applications.
2362
+ # The holdout pattern applies: test the running app through its UI,
2363
+ # never reference source code from the main repo.
2364
+ #
2365
+ # Maestro tests are declarative YAML flows that interact with a running
2366
+ # app on a simulator/emulator. Install Maestro:
2367
+ # curl -Ls "https://get.maestro.mobile.dev" | bash
2368
+ #
2369
+ # Run this flow:
2370
+ # maestro test example-scenario-mobile.yaml
2371
+ #
2372
+ # DO:
2373
+ # - Tap elements, fill inputs, assert on visible text
2374
+ # - Use runFlow for reusable sub-flows (e.g., login)
2375
+ # - Use assertWithAI for natural-language assertions
2376
+ #
2377
+ # DON'T:
2378
+ # - Reference source code paths or internal identifiers
2379
+ # - Depend on exact pixel positions (use text and accessibility labels)
2380
+
2381
+ appId: com.example.myapp # Replace with your app's bundle identifier
2382
+ name: "Core User Journey"
2383
+ tags:
2384
+ - smoke
2385
+ - holdout
2386
+ ---
2387
+ # Step 1: Launch the app
2388
+ - launchApp
2389
+
2390
+ # Step 2: Login (using a reusable sub-flow)
2391
+ - runFlow: example-scenario-mobile-login.yaml
2392
+
2393
+ # Step 3: Verify the main screen loaded
2394
+ - assertVisible: "Home"
2395
+
2396
+ # Step 4: Navigate to a feature
2397
+ # - tapOn: "Settings"
2398
+ # - assertVisible: "Account"
2399
+
2400
+ # Step 5: AI-powered assertion (natural language)
2401
+ # - assertWithAI: "The main dashboard is visible with navigation tabs at the bottom"
2402
+
2403
+ # Step 6: Go back
2404
+ # - back
2405
+ # - assertVisible: "Home"
2406
+ `,
2407
+ "scenarios/example-scenario-mobile-login.yaml": `# Reusable Login Sub-Flow (Maestro)
2408
+ #
2409
+ # This flow handles authentication. Other flows include it via:
2410
+ # - runFlow: example-scenario-mobile-login.yaml
2411
+ #
2412
+ # Replace the selectors and credentials with your app's actual login flow.
2413
+
2414
+ appId: com.example.myapp
2415
+ name: "Login"
2416
+ ---
2417
+ - assertVisible: "Sign In"
2418
+ - tapOn: "Email"
2419
+ - inputText: "test@example.com"
2420
+ - tapOn: "Password"
2421
+ - inputText: "testpassword123"
2422
+ - tapOn: "Log In"
2423
+ - assertVisible: "Home" # Verify login succeeded
2424
+ `,
2425
+ "scenarios/README-mobile.md": '# Mobile Scenario Testing with Maestro\n\nThis guide explains how to set up and run mobile holdout scenario tests using [Maestro](https://maestro.dev/).\n\n## Prerequisites\n\n- **Maestro CLI:** `curl -Ls "https://get.maestro.mobile.dev" | bash`\n- **Java 17+** (required by Maestro)\n- **Simulator/Emulator:**\n - iOS: Xcode with iOS Simulator (macOS only)\n - Android: Android Studio with an AVD configured\n\n> **Important:** Joycraft does not install Maestro or manage simulators. This is your responsibility.\n\n## Running Tests Locally\n\n```bash\n# Boot your simulator/emulator first, then:\nmaestro test example-scenario-mobile.yaml\n\n# Run all flows in a directory:\nmaestro test .maestro/\n```\n\n## Writing Flows\n\nMaestro flows are declarative YAML. Core commands:\n\n| Command | Purpose |\n|---------|--------|\n| `launchApp` | Start or restart the app |\n| `tapOn: "text"` | Tap an element by visible text or test ID |\n| `inputText: "value"` | Type into a focused field |\n| `assertVisible: "text"` | Assert an element is on screen |\n| `assertNotVisible: "text"` | Assert an element is NOT on screen |\n| `scroll` | Scroll down |\n| `back` | Press the back button |\n| `runFlow: file.yaml` | Run a reusable sub-flow |\n| `assertWithAI: "description"` | Natural-language assertion (AI-powered) |\n\n## CI Options\n\n### Option A: Maestro Cloud (paid, easiest)\n\nUpload your app binary and flows to Maestro Cloud. No simulator management.\n\n```yaml\n- uses: mobile-dev-inc/action-maestro-cloud@v2\n with:\n api-key: ${{ secrets.MAESTRO_API_KEY }}\n app-file: app.apk # or app.ipa\n workspace: .\n```\n\n### Option B: Self-hosted emulator (free, more setup)\n\nSpin up an Android emulator on a Linux runner or iOS simulator on a macOS runner.\n\n> **Cost note:** macOS GitHub Actions runners are ~10x more expensive than Linux runners.\n\n## The Holdout Pattern\n\nThese tests live in the scenarios repo, separate from the main codebase. The scenario agent generates them from specs. They test observable behavior through the app\'s UI \u2014 never referencing source code or internal implementation.\n',
2088
2426
  "scenarios/prompts/scenario-agent.md": `You are a QA engineer working in a holdout test repository. You CANNOT access the main repository's source code. Your job is to write or update behavioral scenario tests based on specs that are pushed from the main repo.
2089
2427
 
2090
2428
  ## What You Have Access To
@@ -2092,7 +2430,23 @@ describe("CLI: init command (example \u2014 replace with your real scenarios)",
2092
2430
  - This scenarios repository (test files, \`specs/\` mirror, \`package.json\`)
2093
2431
  - The incoming spec (provided below)
2094
2432
  - A list of existing test files and spec mirrors (provided below)
2095
- - The main repo is available at \`../main-repo\` and is already built \u2014 you can invoke its CLI or entry point via \`execSync\`/\`spawnSync\`, but you MUST NOT import from \`../main-repo/src\`
2433
+ - The main repo is available at \`../main-repo\` and is already built
2434
+ - The testing strategy for this project (provided below)
2435
+
2436
+ ## Testing Strategy
2437
+
2438
+ This project uses the **$TESTING_BACKBONE** testing backbone.
2439
+
2440
+ Select the correct test format based on the backbone:
2441
+
2442
+ | Backbone | Tool | Test Format | File Extension | How to Test |
2443
+ |----------|------|-------------|---------------|-------------|
2444
+ | \`playwright\` | Playwright | Browser-based E2E | \`.spec.ts\` | Navigate pages, click elements, assert on visible content |
2445
+ | \`maestro\` | Maestro | YAML flows | \`.yaml\` | Tap elements, fill inputs, assert on screen state |
2446
+ | \`api\` | fetch (Node.js built-in) | HTTP requests | \`.test.ts\` | Send requests to endpoints, assert on responses |
2447
+ | \`native\` | vitest + spawnSync | CLI/binary invocation | \`.test.ts\` | Run commands, assert on stdout/stderr/exit codes |
2448
+
2449
+ If the backbone is not provided or unrecognized, default to \`native\`.
2096
2450
 
2097
2451
  ## Triage Decision Tree
2098
2452
 
@@ -2111,7 +2465,7 @@ If you SKIP, write a brief comment in the relevant test file (or a new one) expl
2111
2465
  - A new output format or file that gets generated
2112
2466
  - A new user-facing behavior that doesn't map to any existing test file
2113
2467
 
2114
- Name the file after the feature area: \`[feature-area].test.ts\`. One feature area per test file.
2468
+ Name the file after the feature area using the correct extension for the backbone.
2115
2469
 
2116
2470
  ### UPDATE \u2014 Modify an existing test file if the spec:
2117
2471
  - Changes behavior that is already tested
@@ -2122,25 +2476,20 @@ Match to the most relevant existing test file by feature area.
2122
2476
 
2123
2477
  **If you are unsure whether a spec is user-facing, err on the side of writing a test.**
2124
2478
 
2125
- ## Test Writing Rules
2126
-
2127
- 1. **Behavioral only.** Test observable output \u2014 stdout, stderr, exit codes, files created/modified on disk. Never test internal implementation details or import source modules.
2128
-
2129
- 2. **Use \`execSync\` or \`spawnSync\`.** Invoke the built binary at \`../main-repo/dist/cli.js\` (or whatever the main repo's entry point is). Check \`../main-repo/package.json\` to find the correct entry point if unsure.
2479
+ ## Test Writing Rules (All Backbones)
2130
2480
 
2131
- 3. **Use vitest.** Import \`describe\`, \`it\`, \`expect\` from \`vitest\`. Use \`beforeEach\`/\`afterEach\` for temp directory setup/teardown.
2481
+ 1. **Behavioral only.** Test observable behavior \u2014 what a real user would see. Never test internal implementation details or import source modules.
2482
+ 2. **Each test is fully independent.** No shared mutable state between tests.
2483
+ 3. **Assert on realistic user actions.** Write tests that reflect what a real user would do.
2484
+ 4. **Never import from the parent repo's source.** If you find yourself writing \`import { ... } from '../main-repo/src/...'\`, stop \u2014 that defeats the holdout.
2132
2485
 
2133
- 4. **Each test is fully independent.** No shared mutable state between tests. Each test that touches the filesystem gets its own temp directory via \`mkdtempSync\`.
2486
+ ## Backbone: native (CLI/Binary)
2134
2487
 
2135
- 5. **Assert on realistic user actions.** Write tests that reflect what a real user would do \u2014 not what the implementation happens to do.
2136
-
2137
- 6. **Never import from the parent repo's source.** If you find yourself writing \`import { ... } from '../main-repo/src/...'\`, stop \u2014 that defeats the holdout.
2138
-
2139
- ## Test File Template
2488
+ Use when the project is a CLI tool, library, or has no web/mobile UI.
2140
2489
 
2141
2490
  \`\`\`typescript
2142
- import { execSync, spawnSync } from 'node:child_process';
2143
- import { existsSync, mkdtempSync, rmSync, readFileSync } from 'node:fs';
2491
+ import { spawnSync } from 'node:child_process';
2492
+ import { mkdtempSync, rmSync } from 'node:fs';
2144
2493
  import { tmpdir } from 'node:os';
2145
2494
  import { join } from 'node:path';
2146
2495
  import { describe, it, expect, beforeEach, afterEach } from 'vitest';
@@ -2153,39 +2502,122 @@ function runCLI(args: string[], cwd?: string) {
2153
2502
  cwd: cwd ?? process.cwd(),
2154
2503
  env: { ...process.env, NO_COLOR: '1' },
2155
2504
  });
2156
- return {
2157
- stdout: result.stdout ?? '',
2158
- stderr: result.stderr ?? '',
2159
- status: result.status ?? 1,
2160
- };
2505
+ return { stdout: result.stdout ?? '', stderr: result.stderr ?? '', status: result.status ?? 1 };
2161
2506
  }
2162
2507
 
2163
- describe('[feature area]: [behavior being tested]', () => {
2508
+ describe('[feature area]', () => {
2164
2509
  let tmpDir: string;
2510
+ beforeEach(() => { tmpDir = mkdtempSync(join(tmpdir(), 'scenarios-')); });
2511
+ afterEach(() => { rmSync(tmpDir, { recursive: true, force: true }); });
2165
2512
 
2166
- beforeEach(() => {
2167
- tmpDir = mkdtempSync(join(tmpdir(), 'scenarios-'));
2513
+ it('[observable behavior]', () => {
2514
+ const { stdout, status } = runCLI(['command', 'args'], tmpDir);
2515
+ expect(status).toBe(0);
2516
+ expect(stdout).toContain('expected output');
2168
2517
  });
2518
+ });
2519
+ \`\`\`
2169
2520
 
2170
- afterEach(() => {
2171
- rmSync(tmpDir, { recursive: true, force: true });
2521
+ ## Backbone: playwright (Web Apps)
2522
+
2523
+ Use when the project is a web application (Next.js, Vite, Nuxt, etc.).
2524
+
2525
+ \`\`\`typescript
2526
+ import { test, expect } from '@playwright/test';
2527
+
2528
+ // Tests run against BASE_URL (configured in playwright.config.ts)
2529
+ // The dev server is started automatically or BASE_URL points to a preview deploy
2530
+
2531
+ test.describe('[feature area]', () => {
2532
+ test('[observable behavior]', async ({ page }) => {
2533
+ await page.goto('/');
2534
+ await expect(page.locator('h1')).toBeVisible();
2172
2535
  });
2173
2536
 
2174
- it('[specific observable behavior]', () => {
2175
- const { stdout, status } = runCLI(['command', 'args'], tmpDir);
2176
- expect(status).toBe(0);
2177
- expect(stdout).toContain('expected output');
2537
+ test('[user interaction]', async ({ page }) => {
2538
+ await page.goto('/login');
2539
+ await page.fill('[name="email"]', 'test@example.com');
2540
+ await page.click('button[type="submit"]');
2541
+ await expect(page).toHaveURL(/dashboard/);
2542
+ });
2543
+ });
2544
+ \`\`\`
2545
+
2546
+ ## Backbone: api (API Backends)
2547
+
2548
+ Use when the project is an API-only backend (Express, FastAPI, etc.).
2549
+
2550
+ \`\`\`typescript
2551
+ import { describe, it, expect } from 'vitest';
2552
+
2553
+ const BASE_URL = process.env.BASE_URL || 'http://localhost:3000';
2554
+
2555
+ describe('[feature area]', () => {
2556
+ it('[endpoint behavior]', async () => {
2557
+ const res = await fetch(\\\`\\\${BASE_URL}/api/endpoint\\\`);
2558
+ expect(res.status).toBe(200);
2559
+ const body = await res.json();
2560
+ expect(body).toHaveProperty('data');
2561
+ });
2562
+
2563
+ it('[error handling]', async () => {
2564
+ const res = await fetch(\\\`\\\${BASE_URL}/api/not-found\\\`);
2565
+ expect(res.status).toBe(404);
2178
2566
  });
2179
2567
  });
2180
2568
  \`\`\`
2181
2569
 
2570
+ ## Backbone: maestro (Mobile Apps)
2571
+
2572
+ Use when the project is a mobile application (React Native, Flutter, native iOS/Android).
2573
+
2574
+ \`\`\`yaml
2575
+ appId: com.example.myapp
2576
+ name: "[feature area]: [behavior being tested]"
2577
+ tags:
2578
+ - holdout
2579
+ ---
2580
+ - launchApp
2581
+ - tapOn: "Sign In"
2582
+ - inputText: "test@example.com"
2583
+ - tapOn: "Submit"
2584
+ - assertVisible: "Welcome"
2585
+ # Use assertWithAI for complex visual assertions:
2586
+ # - assertWithAI: "The dashboard shows a list of recent items"
2587
+ \`\`\`
2588
+
2589
+ ## Graceful Degradation
2590
+
2591
+ If the primary backbone tool is not available in this repo, fall back to the next deepest testable layer:
2592
+
2593
+ | Layer | What's Tested | When to Use |
2594
+ |-------|-------------|-------------|
2595
+ | **Layer 4: UI** | Full user flows through browser/simulator | \`@playwright/test\` or Maestro is installed |
2596
+ | **Layer 3: API** | HTTP requests against running server | Server can be started from \`../main-repo\` |
2597
+ | **Layer 2: Logic** | Unit tests via test runner | Test runner (vitest/jest) is available |
2598
+ | **Layer 1: Static** | Build, typecheck, lint | Build toolchain is available |
2599
+
2600
+ **Fallback rules:**
2601
+ - If backbone is \`playwright\` but \`@playwright/test\` is NOT in this repo's \`package.json\`: fall back to \`api\` (fetch-based HTTP tests)
2602
+ - If backbone is \`maestro\` but no simulator context is available: fall back to \`api\` if a server can be started, else \`native\`
2603
+ - If backbone is \`api\` but no server start script exists: fall back to \`native\`
2604
+ - \`native\` is always available as the floor
2605
+
2606
+ Start each test file with a comment indicating the testing layer:
2607
+ \`// Testing Layer: [4|3|2|1] - [UI|API|Logic|Static]\`
2608
+
2609
+ If you fell back from the intended backbone, note this in your commit message:
2610
+ \`scenarios: [action] for [spec] (layer: [N], reason: [why])\`
2611
+
2182
2612
  ## Checklist Before Committing
2183
2613
 
2184
2614
  - [ ] Decision: SKIP / NEW / UPDATE (and why)
2615
+ - [ ] Correct backbone selected (or fallback justified)
2185
2616
  - [ ] Tests assert on observable behavior, not implementation
2186
2617
  - [ ] No imports from \`../main-repo/src\`
2187
- - [ ] Each test has its own temp directory if it touches the filesystem
2188
- - [ ] File is named after the feature area, not the spec
2618
+ - [ ] Each test is independent (own temp dir, own state)
2619
+ - [ ] File uses the correct extension for the backbone
2620
+ - [ ] Testing layer comment at top of file
2189
2621
  `,
2190
2622
  "scenarios/workflows/generate.yml": `# Scenario Generation Workflow
2191
2623
  #
@@ -2285,7 +2717,9 @@ jobs:
2285
2717
  ## Context
2286
2718
 
2287
2719
  Existing test files in this repo: \${{ steps.context.outputs.existing_tests }}
2288
- Existing spec mirrors: \${{ steps.context.outputs.existing_specs }}"
2720
+ Existing spec mirrors: \${{ steps.context.outputs.existing_specs }}
2721
+
2722
+ Testing backbone: \${{ github.event.client_payload.testing_backbone || 'native' }}"
2289
2723
 
2290
2724
  # \u2500\u2500 7. Commit any changes the agent made \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
2291
2725
  - name: Commit scenario changes
@@ -2861,4 +3295,4 @@ export {
2861
3295
  SKILLS,
2862
3296
  TEMPLATES
2863
3297
  };
2864
- //# sourceMappingURL=chunk-G342HURJ.js.map
3298
+ //# sourceMappingURL=chunk-A2CQG5J5.js.map