@sous-io/sous 0.1.1 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +115 -35
- package/bin/run.js +10 -1
- package/docs/markdown/README.md +27 -0
- package/docs/markdown/_sidebar.md +18 -0
- package/docs/markdown/commands.md +308 -0
- package/docs/markdown/config-discovery.md +74 -0
- package/docs/markdown/config-inspection.md +69 -0
- package/docs/markdown/config-layers.md +92 -0
- package/docs/markdown/config-variables.md +79 -0
- package/docs/markdown/configuration.md +71 -0
- package/docs/markdown/design-principles.md +59 -0
- package/docs/markdown/repositories-authoring.md +409 -0
- package/docs/markdown/repositories-consuming.md +580 -0
- package/docs/markdown/repositories-file-formats.md +1084 -0
- package/docs/markdown/repositories-variables.md +387 -0
- package/docs/markdown/repositories.md +303 -0
- package/docs/markdown/skill-categories.md +58 -0
- package/package.json +72 -8
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/SKILL.tpl.md +20 -20
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/examples/about-something.md +2 -2
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/examples/do-something.md +1 -1
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/references/advanced-patterns.md +6 -6
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/references/commands.md +5 -5
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/references/frontmatter.md +3 -3
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-liquid-templates/SKILL.tpl.md +40 -25
- package/recipes/core/sous-skills/skills/about-sous/SKILL.tpl.md +70 -0
- package/recipes/core/sous-skills/skills/about-sous-configuration/SKILL.tpl.md +75 -0
- package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/create-skill/SKILL.tpl.md +8 -9
- package/recipes/core/sous-skills/sous.recipe.yaml +45 -0
- package/sous.config.schema.json +337 -0
- package/src/base-command.ts +220 -67
- package/src/commands/build.ts +150 -73
- package/src/commands/clear.ts +23 -15
- package/src/commands/compile.ts +74 -16
- package/src/commands/config/get.ts +110 -0
- package/src/commands/config/show.ts +32 -0
- package/src/commands/config/validate.ts +53 -0
- package/src/commands/help.ts +46 -0
- package/src/commands/launch.ts +36 -14
- package/src/commands/lock/rebuild.ts +241 -0
- package/src/commands/lock/show.ts +115 -0
- package/src/commands/namespace/list.ts +117 -0
- package/src/commands/namespace/show.ts +110 -0
- package/src/commands/prune.ts +3 -11
- package/src/commands/recipe/list.ts +95 -0
- package/src/commands/recipe/show.ts +301 -0
- package/src/commands/repo/add.ts +145 -0
- package/src/commands/repo/gc.ts +172 -0
- package/src/commands/repo/init.ts +136 -0
- package/src/commands/repo/link.ts +500 -0
- package/src/commands/repo/list.ts +179 -0
- package/src/commands/repo/release.ts +625 -0
- package/src/commands/repo/remove.ts +193 -0
- package/src/commands/repo/search.ts +189 -0
- package/src/commands/repo/submit.ts +133 -0
- package/src/commands/repo/unlink.ts +147 -0
- package/src/commands/subscription/add.ts +285 -0
- package/src/commands/subscription/list.ts +129 -0
- package/src/commands/subscription/remove.ts +181 -0
- package/src/commands/vars/ask.ts +374 -0
- package/src/commands/vars/index.ts +79 -0
- package/src/commands/vars/list.ts +67 -0
- package/src/commands/vars/show.ts +77 -0
- package/src/config-command.ts +30 -0
- package/src/lib/build-service.ts +206 -54
- package/src/lib/config-discovery.ts +220 -27
- package/src/lib/config-inspect.ts +145 -0
- package/src/lib/config-kernel.mjs +377 -0
- package/src/lib/config-schema.ts +361 -0
- package/src/lib/env-file.ts +328 -0
- package/src/lib/env-local.ts +18 -1
- package/src/lib/errors.ts +32 -0
- package/src/lib/include-resolver.ts +108 -15
- package/src/lib/interactive.ts +165 -0
- package/src/lib/markdown-compiler.ts +118 -37
- package/src/lib/package-info.ts +25 -0
- package/src/lib/pid-service.ts +32 -21
- package/src/lib/refs/find.ts +589 -0
- package/src/lib/refs/index.ts +12 -0
- package/src/lib/refs/pick.ts +147 -0
- package/src/lib/refs/scopes.ts +61 -0
- package/src/lib/repos/catalog-display.ts +116 -0
- package/src/lib/repos/catalog-inputs.ts +160 -0
- package/src/lib/repos/catalog.ts +722 -0
- package/src/lib/repos/core-recipe.ts +105 -0
- package/src/lib/repos/defaults.ts +175 -0
- package/src/lib/repos/formats/common.ts +389 -0
- package/src/lib/repos/formats/index-file.ts +215 -0
- package/src/lib/repos/formats/links-map.ts +96 -0
- package/src/lib/repos/formats/lockfile.ts +167 -0
- package/src/lib/repos/formats/patterns.ts +57 -0
- package/src/lib/repos/formats/recipe-manifest.ts +395 -0
- package/src/lib/repos/formats/repo-manifest.ts +88 -0
- package/src/lib/repos/formats/store-entry.ts +84 -0
- package/src/lib/repos/freshness.ts +208 -0
- package/src/lib/repos/git-clone.ts +312 -0
- package/src/lib/repos/identity.ts +89 -0
- package/src/lib/repos/index.ts +58 -0
- package/src/lib/repos/links.ts +353 -0
- package/src/lib/repos/load-manifest.ts +236 -0
- package/src/lib/repos/lock-service.ts +453 -0
- package/src/lib/repos/locked-namespace-resolver.ts +90 -0
- package/src/lib/repos/locked-recipes.ts +254 -0
- package/src/lib/repos/managed-layer.ts +422 -0
- package/src/lib/repos/namespace-resolver.ts +370 -0
- package/src/lib/repos/providers/base.ts +206 -0
- package/src/lib/repos/providers/git.ts +233 -0
- package/src/lib/repos/providers/github.ts +294 -0
- package/src/lib/repos/providers/gitlab.ts +263 -0
- package/src/lib/repos/providers/http.ts +102 -0
- package/src/lib/repos/providers/index-cache.ts +382 -0
- package/src/lib/repos/providers/index.ts +106 -0
- package/src/lib/repos/providers/local.ts +391 -0
- package/src/lib/repos/providers/provider.ts +401 -0
- package/src/lib/repos/recipe-config-layers.ts +287 -0
- package/src/lib/repos/recipe-targets.ts +223 -0
- package/src/lib/repos/ref-search.ts +46 -0
- package/src/lib/repos/ref.ts +513 -0
- package/src/lib/repos/reference-report.ts +122 -0
- package/src/lib/repos/release/bump.ts +161 -0
- package/src/lib/repos/release/git-state.ts +305 -0
- package/src/lib/repos/release/index-builder.ts +635 -0
- package/src/lib/repos/release/index.ts +16 -0
- package/src/lib/repos/release/plan.ts +512 -0
- package/src/lib/repos/release/submit-service.ts +496 -0
- package/src/lib/repos/release/tags.ts +243 -0
- package/src/lib/repos/release/validate.ts +463 -0
- package/src/lib/repos/resolver.ts +789 -0
- package/src/lib/repos/scaffold/index.ts +238 -0
- package/src/lib/repos/scaffold/templates.ts +415 -0
- package/src/lib/repos/seed.ts +414 -0
- package/src/lib/repos/store/contract.ts +64 -0
- package/src/lib/repos/store/hash.ts +114 -0
- package/src/lib/repos/store/recipe-store.ts +599 -0
- package/src/lib/repos/store/settings.ts +58 -0
- package/src/lib/repos/subscription-service.ts +2678 -0
- package/src/lib/repos/trust.ts +447 -0
- package/src/lib/settings.ts +546 -189
- package/src/lib/sous-home.ts +104 -0
- package/src/lib/state.ts +52 -20
- package/src/lib/vars/ask.ts +1152 -0
- package/src/lib/vars/definition-source.ts +252 -0
- package/src/lib/vars/display.ts +233 -0
- package/src/lib/vars/index.ts +18 -0
- package/src/lib/vars/ladder.ts +282 -0
- package/src/lib/vars/mappings.ts +265 -0
- package/src/lib/vars/names.ts +94 -0
- package/src/lib/vars/preanswers.ts +395 -0
- package/src/lib/vars/question-plan.ts +218 -0
- package/src/lib/vars/report.ts +228 -0
- package/src/lib/vars/safe-regex.ts +235 -0
- package/src/lib/vars/validate.ts +312 -0
- package/src/lib/watch-loop.ts +148 -0
- package/src/templating/init-liquid-engine.ts +58 -16
- package/src/utils/choice-prompt.ts +143 -0
- package/src/utils/command-errors.ts +186 -0
- package/src/utils/command-help.ts +45 -0
- package/src/utils/confirm-prompt.ts +110 -0
- package/src/utils/flags.ts +153 -0
- package/src/utils/formatting.ts +540 -55
- package/src/utils/prompts.ts +35 -1
- package/src/utils/sous-directory.ts +245 -0
- package/src/utils/table.ts +603 -0
- package/src/utils/value-prompt.ts +119 -0
- package/shared-prompts/_partials/resume-task.md +0 -51
- package/shared-prompts/_partials/sub-agent-delegation.md +0 -32
- package/shared-prompts/_partials/update-task-file.md +0 -52
- package/shared-prompts/memories/automated-browser-tasks/INDEX.tpl.md +0 -52
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/SKILL.tpl.md +0 -102
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/examples/auth-failure-handling.mjs +0 -81
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/examples/chained-workflow.mjs +0 -126
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/examples/simple-fetch.mjs +0 -92
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/references/architecture.md +0 -61
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/references/auth-and-sessions.md +0 -65
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/references/ctx-api.md +0 -96
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/references/installation.md +0 -104
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/references/script-conventions.md +0 -243
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/chrome-state.mjs +0 -148
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/debug.mjs +0 -383
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/debug.spec.mjs +0 -267
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/eslint.config.mjs +0 -56
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/harness.mjs +0 -169
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/keyring.mjs +0 -59
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/logger.mjs +0 -25
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/params.mjs +0 -140
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/run.mjs +0 -140
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/settings.tpl.mjs +0 -1
- package/shared-prompts/skills/automated-browser-tasks/about-automated-browser-tasks/scripts/utils.mjs +0 -185
- package/shared-prompts/skills/automated-browser-tasks/create-automated-browser-task/SKILL.tpl.md +0 -52
- package/shared-prompts/skills/automated-browser-tasks/running-automated-browser-tasks/SKILL.tpl.md +0 -59
- package/shared-prompts/skills/automated-browser-tasks/update-automated-browser-task/SKILL.tpl.md +0 -47
- package/shared-prompts/skills/control-flow/approve/SKILL.tpl.md +0 -26
- package/shared-prompts/skills/control-flow/opine/SKILL.tpl.md +0 -58
- package/shared-prompts/skills/control-flow/repeat/SKILL.tpl.md +0 -27
- package/shared-prompts/skills/control-flow/research/SKILL.tpl.md +0 -34
- package/shared-prompts/skills/sous-skills/about-sous/SKILL.tpl.md +0 -51
- package/shared-prompts/skills/task-files/about-task-files/SKILL.tpl.md +0 -122
- package/shared-prompts/skills/task-files/continue-task-in-new-branch/SKILL.tpl.md +0 -80
- package/shared-prompts/skills/task-files/go/SKILL.tpl.md +0 -14
- package/shared-prompts/skills/task-files/resume-task/SKILL.tpl.md +0 -13
- package/shared-prompts/skills/task-files/start-task/SKILL.tpl.md +0 -93
- package/shared-prompts/skills/task-files/update/SKILL.tpl.md +0 -14
- package/shared-prompts/skills/task-files/update-task-file/SKILL.tpl.md +0 -13
- /package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-agent-skills/references/substitutions.md +0 -0
- /package/{shared-prompts/skills/sous-skills → recipes/core/sous-skills/skills}/about-liquid-templates/references/liquid-filters.md +0 -0
|
@@ -1,92 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* simple-fetch
|
|
3
|
-
*
|
|
4
|
-
* The minimal shape of an automation script: navigate to one page, confirm we
|
|
5
|
-
* are authenticated, and extract a piece of text. Use this as the starting
|
|
6
|
-
* template for single-page data extraction.
|
|
7
|
-
*/
|
|
8
|
-
|
|
9
|
-
export const meta = {
|
|
10
|
-
name: 'simple-fetch',
|
|
11
|
-
description: 'Navigate to a single page and extract its <h1> (demonstration script).',
|
|
12
|
-
params: {
|
|
13
|
-
url: {
|
|
14
|
-
required: true,
|
|
15
|
-
description:
|
|
16
|
-
'The absolute URL to load. Must include the scheme (https://). The page ' +
|
|
17
|
-
'is loaded as the authenticated user via injected Chrome cookies.',
|
|
18
|
-
validate: /^https?:\/\/.+/,
|
|
19
|
-
invalidMessage: 'url must be an absolute http(s) URL, e.g. "https://example.com".',
|
|
20
|
-
},
|
|
21
|
-
selector: {
|
|
22
|
-
required: false,
|
|
23
|
-
default: 'h1',
|
|
24
|
-
description:
|
|
25
|
-
'CSS selector for the element whose text to extract. Defaults to the ' +
|
|
26
|
-
'first <h1>. The longest matching element\'s text is returned.',
|
|
27
|
-
},
|
|
28
|
-
outputFile: {
|
|
29
|
-
required: false,
|
|
30
|
-
description:
|
|
31
|
-
'Optional absolute path. When set, the runner writes the extracted text ' +
|
|
32
|
-
'here instead of printing the result object.',
|
|
33
|
-
validate: /^\//,
|
|
34
|
-
invalidMessage: 'outputFile must be an absolute path (starting with "/").',
|
|
35
|
-
},
|
|
36
|
-
},
|
|
37
|
-
};
|
|
38
|
-
|
|
39
|
-
/**
|
|
40
|
-
* Entry point. Loads the page, verifies auth, and extracts the target text.
|
|
41
|
-
*
|
|
42
|
-
* @param {object} ctx - The harness context.
|
|
43
|
-
* @returns {Promise<object>} `{ found, url, content, outputFile }`.
|
|
44
|
-
*/
|
|
45
|
-
export async function execute(ctx) {
|
|
46
|
-
const { page, params } = ctx;
|
|
47
|
-
const { url, selector, outputFile } = params;
|
|
48
|
-
|
|
49
|
-
await loadPage(ctx, url, selector);
|
|
50
|
-
const content = await extractText(ctx, selector);
|
|
51
|
-
|
|
52
|
-
return { found: content !== null, url: page.url(), content, outputFile };
|
|
53
|
-
}
|
|
54
|
-
|
|
55
|
-
/**
|
|
56
|
-
* Load a URL and wait for the target element to render. Uses `domcontentloaded`
|
|
57
|
-
* (NOT `networkidle`, which is not a readiness signal) and then waits for the
|
|
58
|
-
* specific thing we need — the target selector. If that times out, we check auth:
|
|
59
|
-
* a login redirect (resolved late by many apps) surfaces as a clear AuthError,
|
|
60
|
-
* otherwise the render timeout is re-thrown.
|
|
61
|
-
*
|
|
62
|
-
* @param {object} ctx - The harness context.
|
|
63
|
-
* @param {string} url - Absolute URL to load.
|
|
64
|
-
* @param {string} selector - The element we expect to render (our success signal).
|
|
65
|
-
* @returns {Promise<void>}
|
|
66
|
-
*/
|
|
67
|
-
async function loadPage(ctx, url, selector) {
|
|
68
|
-
const { page, logger, timeout, checkAuth } = ctx;
|
|
69
|
-
const log = logger.child('navigate');
|
|
70
|
-
log.info(`Loading ${url}`);
|
|
71
|
-
await page.goto(url, { waitUntil: 'domcontentloaded', timeout });
|
|
72
|
-
try {
|
|
73
|
-
await page.locator(selector).first().waitFor({ state: 'visible', timeout });
|
|
74
|
-
} catch (renderTimeout) {
|
|
75
|
-
await checkAuth(); // throws AuthError if the settled URL is a login page
|
|
76
|
-
throw renderTimeout;
|
|
77
|
-
}
|
|
78
|
-
}
|
|
79
|
-
|
|
80
|
-
/**
|
|
81
|
-
* Extract the longest text matching a selector.
|
|
82
|
-
*
|
|
83
|
-
* @param {object} ctx - The harness context.
|
|
84
|
-
* @param {string} selector - CSS selector to read.
|
|
85
|
-
* @returns {Promise<string|null>} The text, or null if nothing matched.
|
|
86
|
-
*/
|
|
87
|
-
async function extractText(ctx, selector) {
|
|
88
|
-
const { utils, logger } = ctx;
|
|
89
|
-
const text = await utils.extractLongestText([selector], { minLength: 1 });
|
|
90
|
-
logger.child('extract').info(text ? `Got ${text.length} chars` : 'No match');
|
|
91
|
-
return text;
|
|
92
|
-
}
|
|
@@ -1,61 +0,0 @@
|
|
|
1
|
-
# Architecture
|
|
2
|
-
|
|
3
|
-
The system has six framework modules (in this skill's `scripts/`) plus the
|
|
4
|
-
user's own scripts (in the project's configured scripts directory).
|
|
5
|
-
|
|
6
|
-
## Components
|
|
7
|
-
|
|
8
|
-
1. **`keyring.mjs`** — Reads Chrome's Safe Storage password from the OS keyring
|
|
9
|
-
(GNOME keyring) via the D-Bus Secret Service API. Pure JS (`dbus-next`), no
|
|
10
|
-
native deps, no Python. Exports `getChromeSafeStoragePassword()`.
|
|
11
|
-
|
|
12
|
-
2. **`chrome-state.mjs`** — Reads cookies from Chrome's `Cookies` SQLite DB
|
|
13
|
-
(read-only; Chrome stays open), decrypts them with a PBKDF2 key derived from
|
|
14
|
-
the keyring password, and builds a Playwright `storageState` object. Exports
|
|
15
|
-
`extractCookies()`, `buildStorageState()`, `listProfiles()`.
|
|
16
|
-
|
|
17
|
-
3. **`logger.mjs`** — Plain prefixed-line logger. `createLogger(prefix)` returns
|
|
18
|
-
`{ info, warn, error, child }`; `child(section)` extends the prefix.
|
|
19
|
-
|
|
20
|
-
4. **`utils.mjs`** — `createUtils(page, logger)` builds the `ctx.utils` toolkit,
|
|
21
|
-
bound to the live page (modal dismissal, waits, text extraction, screenshot,
|
|
22
|
-
retry, …). See `ctx-api.md`.
|
|
23
|
-
|
|
24
|
-
5. **`params.mjs`** — `resolveParams(meta, explicit, settings)` resolves params
|
|
25
|
-
in priority order and validates them, throwing `ParamError` on failure.
|
|
26
|
-
|
|
27
|
-
6. **`harness.mjs`** — `runScript(script, params, options)`. Resolves/validates
|
|
28
|
-
params, extracts cookies, launches headless Chromium with the injected
|
|
29
|
-
storageState, builds `ctx`, runs `execute(ctx)` under an overall timeout, and
|
|
30
|
-
returns a result envelope. Defines `AuthError` and `ctx.checkAuth()`.
|
|
31
|
-
|
|
32
|
-
7. **`run.mjs`** — CLI entry point. Parses `--key=value` args into harness
|
|
33
|
-
options vs. script params, loads compiled `settings.mjs`, imports the target
|
|
34
|
-
script by absolute path, prints resolved params, runs, reports, writes any
|
|
35
|
-
`outputFile`.
|
|
36
|
-
|
|
37
|
-
## Data Flow
|
|
38
|
-
|
|
39
|
-
```
|
|
40
|
-
run.mjs
|
|
41
|
-
├─ parse args → { params, options }
|
|
42
|
-
├─ load settings.mjs → ctx.settings
|
|
43
|
-
├─ import <script>.mjs
|
|
44
|
-
└─ runScript(script, params, { ...options, settings })
|
|
45
|
-
├─ resolveParams(meta, params, settings) ← validation gate
|
|
46
|
-
├─ chrome-state.buildStorageState()
|
|
47
|
-
│ └─ keyring.getChromeSafeStoragePassword()
|
|
48
|
-
├─ chromium.launch({ headless }) + newContext({ storageState })
|
|
49
|
-
├─ build ctx { page, params, settings, logger, utils, checkAuth, runChild }
|
|
50
|
-
└─ script.execute(ctx) → result
|
|
51
|
-
└─ report result, write outputFile
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
## Result Envelope
|
|
55
|
-
|
|
56
|
-
`runScript` always returns `{ success, ... }`:
|
|
57
|
-
|
|
58
|
-
- success → `{ success: true, data: <script return value> }`
|
|
59
|
-
- param failure → `{ success: false, error: 'params', message }`
|
|
60
|
-
- auth failure → `{ success: false, error: 'auth', message, url, indicators }`
|
|
61
|
-
- other failure → `{ success: false, error: 'script', message, stack }`
|
|
@@ -1,65 +0,0 @@
|
|
|
1
|
-
# Auth & Sessions
|
|
2
|
-
|
|
3
|
-
Scripts run as the logged-in user without any manual login step, and without
|
|
4
|
-
closing or controlling the user's running Chrome.
|
|
5
|
-
|
|
6
|
-
## How auth works
|
|
7
|
-
|
|
8
|
-
1. The harness reads the user's Chrome `Cookies` SQLite DB **read-only** — Chrome
|
|
9
|
-
can stay open; there is no lock conflict.
|
|
10
|
-
2. Cookie values are encrypted. The harness fetches Chrome's "Safe Storage"
|
|
11
|
-
password from the OS keyring (GNOME keyring) over the D-Bus Secret Service
|
|
12
|
-
API, derives an AES key (PBKDF2, salt `saltysalt`, 1 iteration, SHA1, 16
|
|
13
|
-
bytes), and decrypts each cookie.
|
|
14
|
-
- **v10**: AES-128-CBC, IV = 16 spaces.
|
|
15
|
-
- **v11**: AES-128-CBC, IV embedded in bytes 3–18; strip a 16-byte random
|
|
16
|
-
prefix from the decrypted plaintext.
|
|
17
|
-
3. Decrypted cookies become a Playwright `storageState`, injected into a fresh
|
|
18
|
-
headless context. The browser is now authenticated as the user.
|
|
19
|
-
|
|
20
|
-
This is all local. Nothing leaves the machine. No Python; pure JS via
|
|
21
|
-
`dbus-next`, `better-sqlite3`, and Node's `crypto`.
|
|
22
|
-
|
|
23
|
-
## Choosing the profile
|
|
24
|
-
|
|
25
|
-
The Chrome profile defaults to `Default`. Override per run with
|
|
26
|
-
`--profileName="Profile 1"`, or set a project default in settings
|
|
27
|
-
(`chromeProfile`) so it flows in via `ctx.settings`.
|
|
28
|
-
|
|
29
|
-
## Detecting auth failures
|
|
30
|
-
|
|
31
|
-
Cookies expire; SSO sessions lapse. After each navigation a script should call
|
|
32
|
-
`await ctx.checkAuth()`. It inspects the current URL for login indicators
|
|
33
|
-
(`/login`, `/sso`, `multipass`, `accounts.google.com`, etc.) and throws an
|
|
34
|
-
`AuthError` if found.
|
|
35
|
-
|
|
36
|
-
The harness catches `AuthError` and returns:
|
|
37
|
-
|
|
38
|
-
```
|
|
39
|
-
{ success: false, error: 'auth', message, url, indicators }
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
The `message` is actionable and meant to reach the user verbatim. A sub-agent
|
|
43
|
-
returns it to the orchestrator, which relays it:
|
|
44
|
-
|
|
45
|
-
> Authentication required... Log in to the target site in your Chrome browser
|
|
46
|
-
> (profile: "Default"), then retry this script.
|
|
47
|
-
|
|
48
|
-
## The retry contract
|
|
49
|
-
|
|
50
|
-
There is no mid-script recovery. On an auth failure:
|
|
51
|
-
|
|
52
|
-
1. The message reaches the user, who logs into the site in their Chrome profile.
|
|
53
|
-
2. Re-run the **same** command. The harness re-reads the now-valid cookies.
|
|
54
|
-
|
|
55
|
-
A sub-agent cannot wait for a login, so it stops at step 1 and reports; the
|
|
56
|
-
orchestrator relays the message and dispatches the re-run.
|
|
57
|
-
|
|
58
|
-
Never attempt to script the login itself, and never weaken `checkAuth` to get
|
|
59
|
-
past a login wall.
|
|
60
|
-
|
|
61
|
-
## Scope cookie extraction
|
|
62
|
-
|
|
63
|
-
Extraction can be limited to domain substrings for speed/privacy. The harness
|
|
64
|
-
`domains` option (and `extractCookies(profile, domains)`) filters
|
|
65
|
-
`host_key LIKE '%domain%'`. Leave null to extract all cookies.
|
|
@@ -1,96 +0,0 @@
|
|
|
1
|
-
# The `ctx` API
|
|
2
|
-
|
|
3
|
-
Every script's `execute(ctx)` receives a single context object. Destructure what
|
|
4
|
-
you need at the top of each function.
|
|
5
|
-
|
|
6
|
-
## Core members
|
|
7
|
-
|
|
8
|
-
| Member | Type | Description |
|
|
9
|
-
|--------|------|-------------|
|
|
10
|
-
| `ctx.page` | `Page` | Playwright page. Primary interaction surface. |
|
|
11
|
-
| `ctx.context` | `BrowserContext` | The browser context (cookies already injected). |
|
|
12
|
-
| `ctx.browser` | `Browser` | The Chromium instance. |
|
|
13
|
-
| `ctx.params` | object | Fully resolved + validated params. |
|
|
14
|
-
| `ctx.settings` | object | Compiled project settings (from `settings.mjs`). |
|
|
15
|
-
| `ctx.timeout` | number | Overall timeout (ms); use for `goto`/wait calls. |
|
|
16
|
-
| `ctx.logger` | Logger | Prefixed logger (see below). |
|
|
17
|
-
| `ctx.utils` | object | Generic action helpers (see below). |
|
|
18
|
-
| `ctx.debug` | object | Exploration & debugging tools (see below). |
|
|
19
|
-
| `ctx.checkAuth()` | fn | Throws `AuthError` if the current URL looks like a login page. Call after every navigation. |
|
|
20
|
-
| `ctx.runChild(script, params)` | fn | Run another script module in the same browser context; returns its result. Child gets its own resolved params, logger, and utils. |
|
|
21
|
-
|
|
22
|
-
## `ctx.logger`
|
|
23
|
-
|
|
24
|
-
Plain prefixed lines. `info` → stdout; `warn`/`error` → stderr, tagged.
|
|
25
|
-
|
|
26
|
-
```js
|
|
27
|
-
const log = ctx.logger.child('navigate'); // prefix: [script-name:navigate]
|
|
28
|
-
log.info('Loading repo'); // [script-name:navigate] Loading repo
|
|
29
|
-
log.warn('modal not found, using Escape'); // [script-name:navigate] WARN ...
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
## `ctx.utils`
|
|
33
|
-
|
|
34
|
-
Built per-run, bound to the live `page` and the script's logger. All are async
|
|
35
|
-
unless noted.
|
|
36
|
-
|
|
37
|
-
| Helper | Signature | Description |
|
|
38
|
-
|--------|-----------|-------------|
|
|
39
|
-
| `autoDismiss` | `(triggerLocator, dismiss)` | **Preferred** modal handling: register a `page.addLocatorHandler` so `dismiss` runs whenever `triggerLocator` blocks an action. Timing-independent — no sleeps, no races. Use for any modal/banner that may appear asynchronously. |
|
|
40
|
-
| `dismissModals` | `(selectors, { timeout? })` | One-shot best-effort dismissal of modals KNOWN to be present already. Clicks the first of each candidate selector that currently exists; no fixed delays. For async modals, use `autoDismiss`. |
|
|
41
|
-
| `clickByText` | `(text, { force?, timeout? })` | Wait for and click the first element matching `text` (RegExp or string). `force` defaults **false** (auto-waits for actionability). |
|
|
42
|
-
| `waitForText` | `(text, { timeout? })` | Wait for text to appear; does not click. |
|
|
43
|
-
| `extractLongestText` | `(selectors, { minLength? })` | Longest `textContent` across candidate selectors meeting `minLength`, or null. |
|
|
44
|
-
| `extractTextByPattern` | `(pattern, { source?, group? })` | First regex match from page body (or supplied `source`); returns trimmed match or null. |
|
|
45
|
-
| `screenshot` | `(path, { fullPage? })` | Save a debug screenshot. Failures are logged, never thrown. |
|
|
46
|
-
| `scrollIntoView` | `(selector)` | Scroll matching element into view; returns boolean found. |
|
|
47
|
-
| `retry` | `(fn, { attempts?, delay?, label? })` | Retry an async fn; throws the last error if exhausted. |
|
|
48
|
-
|
|
49
|
-
### On `force` clicks
|
|
50
|
-
|
|
51
|
-
Non-forced clicks are the resilient default: Playwright auto-waits for the element
|
|
52
|
-
to be visible, stable, and not covered before clicking — which naturally waits out
|
|
53
|
-
overlays and transitions. `force: true` bypasses those checks and is fire-and-forget
|
|
54
|
-
(it can "succeed" while landing on nothing), so reserve it for the rare element a
|
|
55
|
-
component library (e.g. Blueprint.js) wrongly reports as disabled — and verify the
|
|
56
|
-
outcome with a follow-up wait. There is no `waitForSPA` helper: do not wait on
|
|
57
|
-
`networkidle` as a readiness signal (see `script-conventions.md` → Waits); wait for
|
|
58
|
-
the specific element/text you need instead.
|
|
59
|
-
|
|
60
|
-
## `ctx.debug`
|
|
61
|
-
|
|
62
|
-
Tools for **observing the real page** so you build waits and selectors from facts,
|
|
63
|
-
not guesses. Conventions require observing a page before automating it
|
|
64
|
-
(`script-conventions.md` → "observe the real page"); these are how. Built per-run,
|
|
65
|
-
bound to the live `page`/`logger`. All are async and none throw (a debug aid must
|
|
66
|
-
never break a run). Artifacts are written under `ctx.debug.dir` (a unique per-run
|
|
67
|
-
directory under the OS temp dir, created lazily on first write).
|
|
68
|
-
|
|
69
|
-
| Tool | Signature | Description |
|
|
70
|
-
|------|-----------|-------------|
|
|
71
|
-
| `screenshot` | `(name?, { fullPage? })` | Save a PNG; returns its path (or null). |
|
|
72
|
-
| `html` | `(name?)` | Save the current page HTML; returns its path. |
|
|
73
|
-
| `dump` | `(name?)` | Full snapshot — URL, title, visible text, screenshot, HTML — to the debug dir; returns `{ dir, url, title, files, textPreview }`. The go-to "what does this page look like right now?" tool. |
|
|
74
|
-
| `describe` | `(selector, { limit? })` | What a CSS selector matches: `{ count, elements[] }` with tag/role/id/classes/text/visibility/href. Fastest way to check a selector is right. |
|
|
75
|
-
| `count` | `(selector)` | Number of matches. |
|
|
76
|
-
| `clickables` | `({ limit? })` | List visible interactive elements (anchors, buttons, role buttons/tabs/links, onclick). Use when the obvious `<a>`/button doesn't exist and the real control is a click-handled div. |
|
|
77
|
-
| `findText` | `(pattern, { limit? })` | Where does this text live? For each match, the owning element AND its clickable ancestor — i.e. *what to click* to act on that text. Matches text on an element's own text nodes (works even when the element also has icon/element children). |
|
|
78
|
-
| `watch` | `(fn, { samples?, intervalMs?, label? })` | Sample a browser-side metric over time and log how it evolves. Answers "WHEN is this ready?" — e.g. `watch(() => document.body.innerText.length)` exposes that `networkidle` fired on an empty shell. Returns `[{ t, value }]`. |
|
|
79
|
-
| `dir` | getter | Absolute path to this run's artifact directory. |
|
|
80
|
-
|
|
81
|
-
### Auto-capture on failure
|
|
82
|
-
|
|
83
|
-
When a script throws an unexpected error, the harness automatically runs
|
|
84
|
-
`debug.dump('failure')` before closing the browser and reports the directory in the
|
|
85
|
-
result (`result.debugDir`, printed by the runner). So a failed run leaves a
|
|
86
|
-
screenshot, HTML, and text snapshot of the exact failing state with no extra code.
|
|
87
|
-
|
|
88
|
-
### The intended workflow
|
|
89
|
-
|
|
90
|
-
1. Stuck on a selector or timing? `await ctx.debug.dump()` and look at the files.
|
|
91
|
-
2. Selector returning nothing? `describe(it)` / `clickables()` / `findText(text)`
|
|
92
|
-
to discover the real element (and its clickable ancestor).
|
|
93
|
-
3. Wait firing too early/never? `watch(() => <metric>)` to find the moment the
|
|
94
|
-
page is genuinely ready, then wait on that concrete signal.
|
|
95
|
-
4. Remove debug calls (or leave a couple of cheap ones) once the script is solid;
|
|
96
|
-
`dump`/`screenshot`/`html` write files, so don't leave those in hot loops.
|
|
@@ -1,104 +0,0 @@
|
|
|
1
|
-
# Installation
|
|
2
|
-
|
|
3
|
-
## Runtime & platform
|
|
4
|
-
|
|
5
|
-
- Node.js ≥ 22.
|
|
6
|
-
- Linux with a GNOME-keyring-compatible Secret Service (the user's Chrome must
|
|
7
|
-
have stored its Safe Storage key there — true after Chrome has run once on a
|
|
8
|
-
desktop session with an unlocked keyring).
|
|
9
|
-
- Google Chrome installed with at least one profile the user has logged into.
|
|
10
|
-
|
|
11
|
-
macOS (Keychain) and Windows (DPAPI) are not yet supported by `keyring.mjs`.
|
|
12
|
-
|
|
13
|
-
## Dependencies
|
|
14
|
-
|
|
15
|
-
The runner and harness import these at runtime; the framework does not bundle
|
|
16
|
-
them. Install them **at the consuming project's root** — NOT globally.
|
|
17
|
-
|
|
18
|
-
Why not global: the scripts use ESM `import 'playwright'`. ESM resolves bare
|
|
19
|
-
imports by walking *up* the directory tree from the importing file looking for a
|
|
20
|
-
`node_modules`. The compiled runner lives at
|
|
21
|
-
`<projectRoot>/.claude/skills/about-automated-browser-tasks/scripts/run.mjs`, so a
|
|
22
|
-
`node_modules` at `<projectRoot>` is found by walking up; a global npm install is
|
|
23
|
-
never on that resolution path.
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
cd <projectRoot> # the repo root that contains .claude/skills/
|
|
27
|
-
npm install playwright better-sqlite3 dbus-next
|
|
28
|
-
npx playwright install chromium
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
Add a `package.json` at `<projectRoot>` if none exists (`{"type":"module","private":true}`)
|
|
32
|
-
and gitignore `node_modules/`.
|
|
33
|
-
|
|
34
|
-
- `playwright` — headless browser automation.
|
|
35
|
-
- `better-sqlite3` — reads Chrome's `Cookies` SQLite DB.
|
|
36
|
-
- `dbus-next` — pure-JS D-Bus client for the keyring (no Python, no native build).
|
|
37
|
-
|
|
38
|
-
Tested with: Playwright 1.61, better-sqlite3 12.x, Node 22, Chrome cookie format
|
|
39
|
-
v11, Ubuntu 22.04.
|
|
40
|
-
|
|
41
|
-
## Project wiring (via sous)
|
|
42
|
-
|
|
43
|
-
A downstream project compiles this bundle into its skills directory and compiles
|
|
44
|
-
`settings.tpl.mjs` → `settings.mjs` (sibling of `run.mjs`) so scripts get
|
|
45
|
-
`ctx.settings`. Example compilation targets:
|
|
46
|
-
|
|
47
|
-
```js
|
|
48
|
-
compilation: {
|
|
49
|
-
targets: [
|
|
50
|
-
{
|
|
51
|
-
// The skills (SKILL.tpl.md, references, examples, scripts) → skills dir
|
|
52
|
-
entryGlob: "${sousRootPath}/shared-prompts/skills/automated-browser-tasks/**/*",
|
|
53
|
-
outputs: [{ destinationDir: "${projectRoot}/.claude/skills" }],
|
|
54
|
-
},
|
|
55
|
-
],
|
|
56
|
-
}
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
Define project values (`chromeProfile`, base URLs, resource IDs, …) in `_vars`. The
|
|
60
|
-
`{% exportScalarVarsJs %}` tag in `settings.tpl.mjs` emits all in-scope scalars
|
|
61
|
-
as the runtime settings module — no per-key wiring needed. Be sure to define
|
|
62
|
-
`browserAutomationScriptsDir` (the absolute path to the project's task scripts)
|
|
63
|
-
in `_vars` — both the runtime and the task manifest below rely on it.
|
|
64
|
-
|
|
65
|
-
## Task manifest in core memory
|
|
66
|
-
|
|
67
|
-
So the agent always knows which browser tasks exist (without relying on a skill
|
|
68
|
-
trigger firing), render the shared memory partial into the project's memory source
|
|
69
|
-
tree, then `@include` it from a core-memory file. It renders a live list of every
|
|
70
|
-
task script via `{% getFiles … import="meta" %}`, reading each script's `meta`.
|
|
71
|
-
|
|
72
|
-
`@include` does NOT substitute variables, so you cannot `@`-include the shared
|
|
73
|
-
`INDEX.tpl.md` by an absolute `${...}` path. Instead, add a compilation target that
|
|
74
|
-
renders it into your memory tree (exactly how `runtimeContext` emits
|
|
75
|
-
`session-context.md`):
|
|
76
|
-
|
|
77
|
-
```js
|
|
78
|
-
// A target that renders the shared manifest into the project's memory source.
|
|
79
|
-
const browserTaskManifest = {
|
|
80
|
-
entryPoint: "${sousRootPath}/shared-prompts/memories/automated-browser-tasks/INDEX.tpl.md",
|
|
81
|
-
outputs: [
|
|
82
|
-
{ destinationFile: "${memoryRoot}/tools/automated-browser-tasks.md" },
|
|
83
|
-
],
|
|
84
|
-
};
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Order this target BEFORE the memories target that composes core memory. Then pull
|
|
88
|
-
it into a memory file (e.g. `tools/README.md`) with a plain relative Sous include:
|
|
89
|
-
put an `@`-prefixed line containing just the rendered filename
|
|
90
|
-
(`automated-browser-tasks.md`) on its own line in that file.
|
|
91
|
-
|
|
92
|
-
The manifest auto-rebuilds on every `xcv build`, so newly created tasks appear
|
|
93
|
-
automatically. It requires `browserAutomationScriptsDir` to be in scope (the
|
|
94
|
-
absolute path to the task scripts).
|
|
95
|
-
|
|
96
|
-
## Verifying
|
|
97
|
-
|
|
98
|
-
Run any example script by absolute path:
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
node <scriptsDir>/run.mjs <scriptsDir>/../examples/simple-fetch.mjs --url=https://example.com
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
A clean run prints extracted cookie counts, a browser-ready line, and the result.
|
|
@@ -1,243 +0,0 @@
|
|
|
1
|
-
# Script Conventions
|
|
2
|
-
|
|
3
|
-
These rules are non-negotiable. A reviewer (or linter) should be able to reject a
|
|
4
|
-
script that violates them.
|
|
5
|
-
|
|
6
|
-
## Quality bar: bulletproof or it doesn't ship
|
|
7
|
-
|
|
8
|
-
A flaky script is a broken script. "Works most of the time" is failure. Write for
|
|
9
|
-
100% reliability across many consecutive and parallel runs from the first draft —
|
|
10
|
-
do not ship something that "usually works" and plan to harden later.
|
|
11
|
-
|
|
12
|
-
The single greatest source of flakiness is **guessing about timing instead of
|
|
13
|
-
waiting for facts**. Every wait must key off a concrete, observable condition that
|
|
14
|
-
*proves* the thing you need is ready. Spend the extra time to find that signal.
|
|
15
|
-
Arbitrary delays (`waitForTimeout`) are the enemy — see [Waits](#waits); they are
|
|
16
|
-
effectively banned.
|
|
17
|
-
|
|
18
|
-
Before writing a single wait or selector, **observe the real page.** Do not assume
|
|
19
|
-
DOM structure. Build your waits from what you actually see — selectors invented
|
|
20
|
-
from imagination are how you get a script that passes once and fails in CI.
|
|
21
|
-
|
|
22
|
-
`ctx.debug` exists for exactly this (full surface in `ctx-api.md`):
|
|
23
|
-
|
|
24
|
-
- `ctx.debug.dump()` — snapshot URL, title, text, screenshot, and HTML to disk.
|
|
25
|
-
- `ctx.debug.describe(selector)` / `ctx.debug.clickables()` — see whether a
|
|
26
|
-
selector matches and what the real interactive elements are (often a
|
|
27
|
-
click-handled `div`, not the `<a>`/`<button>` you assumed).
|
|
28
|
-
- `ctx.debug.findText(text)` — locate text and the clickable ancestor to target.
|
|
29
|
-
- `ctx.debug.watch(() => metric)` — sample a metric over time to find the *moment*
|
|
30
|
-
the page is genuinely ready (this is how you discover that `networkidle` fired
|
|
31
|
-
on an empty shell), then wait on that concrete signal.
|
|
32
|
-
|
|
33
|
-
On an unexpected throw, the harness auto-captures a failure snapshot
|
|
34
|
-
(`result.debugDir`) — check it first when a run fails. Remove file-writing debug
|
|
35
|
-
calls (`dump`/`screenshot`/`html`) once the script is solid; keep them out of hot
|
|
36
|
-
loops.
|
|
37
|
-
|
|
38
|
-
## Structure
|
|
39
|
-
|
|
40
|
-
`execute(ctx)` is a thin orchestrator that reads like a table of contents. All
|
|
41
|
-
real work lives in small, named step functions defined below `execute` in the
|
|
42
|
-
same file.
|
|
43
|
-
|
|
44
|
-
- One discrete action per step function (navigate, dismiss, extract, parse…).
|
|
45
|
-
- ≤ 30 lines per function; 10 or fewer is ideal.
|
|
46
|
-
- Module-level functions, not class methods.
|
|
47
|
-
- Generic patterns → `ctx.utils`. Site-specific patterns → step functions (which
|
|
48
|
-
a project may later factor into shared libs it imports).
|
|
49
|
-
|
|
50
|
-
## Doc-blocks
|
|
51
|
-
|
|
52
|
-
EVERY function — `execute` included — has a proper JSDoc block: a description
|
|
53
|
-
line plus `@param` for every argument and `@returns`. Use
|
|
54
|
-
`@returns {Promise<void>}` for functions that return nothing. Single-line
|
|
55
|
-
`/** … */` comments are NOT sufficient.
|
|
56
|
-
|
|
57
|
-
## Destructuring
|
|
58
|
-
|
|
59
|
-
Each function destructures the members it needs off `ctx` (and off `params`) at
|
|
60
|
-
the top of its body, so the body never repeats `ctx.`/`params.` prefixes:
|
|
61
|
-
|
|
62
|
-
```js
|
|
63
|
-
async function navigateToRepo(ctx, baseUrl, repoId) {
|
|
64
|
-
const { page, logger, timeout, checkAuth } = ctx;
|
|
65
|
-
...
|
|
66
|
-
}
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
## Params (`meta.params`)
|
|
70
|
-
|
|
71
|
-
The framework resolves and validates params before `execute` runs. Scripts never
|
|
72
|
-
validate their own params. Resolution priority (low → high):
|
|
73
|
-
`ctx.settings` < `meta.params[x].default` < explicit (CLI) params.
|
|
74
|
-
|
|
75
|
-
Each param spec:
|
|
76
|
-
|
|
77
|
-
| Field | Type | Meaning |
|
|
78
|
-
|-------|------|---------|
|
|
79
|
-
| `required` | boolean | Error if nothing resolves. |
|
|
80
|
-
| `default` | any | Fallback value. |
|
|
81
|
-
| `description` | string | Be genuinely descriptive: what it is, where to find it, how it's used, consequence of omitting. Shown in listings and errors. |
|
|
82
|
-
| `validate` | `RegExp` \| `Function` | See below. |
|
|
83
|
-
| `invalidMessage` | string | Error for a failing RegExp, or a `validate` fn returning `false`. |
|
|
84
|
-
|
|
85
|
-
`validate`:
|
|
86
|
-
- **RegExp** — resolved value (as string) must match.
|
|
87
|
-
- **Function** `(value, resolvedParams) => true | false | string` — `true` valid;
|
|
88
|
-
a returned `string` is used as the error; `false` falls back to `invalidMessage`.
|
|
89
|
-
The function gets all resolved params, enabling cross-param checks.
|
|
90
|
-
|
|
91
|
-
All failures across params are collected into one `ParamError`.
|
|
92
|
-
|
|
93
|
-
## Logging
|
|
94
|
-
|
|
95
|
-
Use `ctx.logger`, never `console.log`. Create a child per section:
|
|
96
|
-
`const log = ctx.logger.child('navigate')`. Output is
|
|
97
|
-
`[script-name:section] message`. Levels: `info`, `warn`, `error`.
|
|
98
|
-
|
|
99
|
-
## Return shape
|
|
100
|
-
|
|
101
|
-
Return a plain object. Common keys:
|
|
102
|
-
|
|
103
|
-
- `found: boolean` — whether the target content was located.
|
|
104
|
-
- `content: string` — extracted content (when found).
|
|
105
|
-
- `outputFile: string` — path for the runner to write `content` to.
|
|
106
|
-
- a URL key (e.g. `buildUrl`) — where content was found.
|
|
107
|
-
- `message: string` — human-readable explanation, especially on failure.
|
|
108
|
-
|
|
109
|
-
On `found: false`, include diagnostics (`pageTextPreview`, `message`).
|
|
110
|
-
|
|
111
|
-
## Verify every action
|
|
112
|
-
|
|
113
|
-
Do not assume an action took effect — prove it. After every navigation or click
|
|
114
|
-
that changes state, wait for a signal that confirms the *intended outcome*:
|
|
115
|
-
|
|
116
|
-
- After a navigation: `await page.waitForURL(/expected-path/)`, or wait for an
|
|
117
|
-
element that only exists on the destination.
|
|
118
|
-
- After a click that should open a view: wait for that view's content, not just
|
|
119
|
-
for the click to return.
|
|
120
|
-
- After triggering content load: wait for the content to be present AND non-empty
|
|
121
|
-
(e.g. a `<pre>` whose text length exceeds a threshold), not merely attached.
|
|
122
|
-
|
|
123
|
-
A click with `{ force: true }` is fire-and-forget: it bypasses Playwright's
|
|
124
|
-
actionability checks (visible, stable, not covered) and reports success even when
|
|
125
|
-
it lands on nothing. Prefer a plain click — Playwright then auto-waits for the
|
|
126
|
-
element to be actionable, which naturally waits out overlays and transitions.
|
|
127
|
-
Reserve `force` for the rare element a component library wrongly reports as
|
|
128
|
-
disabled, and even then verify the outcome afterward.
|
|
129
|
-
|
|
130
|
-
## Error handling
|
|
131
|
-
|
|
132
|
-
Throw on unexpected failures; the harness catches and reports. Never
|
|
133
|
-
catch-and-continue to paper over a problem. Auth failures come from
|
|
134
|
-
`ctx.checkAuth()`; page-interaction failures (missing element, timeout) should
|
|
135
|
-
propagate naturally. Fix root causes, not symptoms.
|
|
136
|
-
|
|
137
|
-
**Auth resolves late in SPAs.** A single-page app often loads its shell, *then*
|
|
138
|
-
decides client-side that the session is invalid and redirects to a login page a
|
|
139
|
-
beat later. So:
|
|
140
|
-
- Do NOT call `ctx.checkAuth()` immediately after `goto` — the redirect may not
|
|
141
|
-
have happened yet (false pass) and the URL may not have settled.
|
|
142
|
-
- Do NOT race the success signal against the login URL — a valid session can
|
|
143
|
-
*transiently* touch a login-ish URL before bouncing back (false fail).
|
|
144
|
-
- DO wait for your success signal (the authenticated view's element). Only if that
|
|
145
|
-
times out, *then* call `ctx.checkAuth()` — by then the URL has settled, so a
|
|
146
|
-
login page is a real `AuthError` and anything else is a genuine render timeout.
|
|
147
|
-
|
|
148
|
-
## Naming
|
|
149
|
-
|
|
150
|
-
- Files: `verb-noun-qualifier.mjs` (e.g. `get-repo-ci-error.mjs`).
|
|
151
|
-
- Step functions: `verbNoun` camelCase (`navigateToRepo`, `dismissModals`).
|
|
152
|
-
- Log sections: short, lowercase, no spaces (`navigate`, `dismiss`, `extract`).
|
|
153
|
-
|
|
154
|
-
## Waits
|
|
155
|
-
|
|
156
|
-
Wait for **specific, verifiable things** — never for time. This is the rule that
|
|
157
|
-
makes scripts bulletproof.
|
|
158
|
-
|
|
159
|
-
### `waitForTimeout` is effectively banned
|
|
160
|
-
|
|
161
|
-
A fixed sleep is a bet that something will be ready by then. The bet loses
|
|
162
|
-
intermittently — that is precisely what flakiness *is*. Exhaust every avenue for a
|
|
163
|
-
condition-based wait before even considering a sleep:
|
|
164
|
-
|
|
165
|
-
1. Wait for an element/state that proves readiness (`locator.waitFor`,
|
|
166
|
-
`page.waitForURL`, `expect(locator).toBeVisible()`).
|
|
167
|
-
2. Wait for a content predicate via `page.waitForFunction(() => …)` when readiness
|
|
168
|
-
is "the data populated", not just "an element exists".
|
|
169
|
-
3. Wait for a network response (`page.waitForResponse`) when the DOM gives no
|
|
170
|
-
signal but a known request does.
|
|
171
|
-
4. Install a handler for interrupting UI (`page.addLocatorHandler`, below) instead
|
|
172
|
-
of sleeping to "let a modal pass".
|
|
173
|
-
|
|
174
|
-
Only if ALL of these are genuinely impossible may you fall back to
|
|
175
|
-
`page.waitForTimeout` — and then you must (a) keep it short, (b) write a comment
|
|
176
|
-
explaining what DOM-observable signal you searched for and why none exists, and
|
|
177
|
-
(c) feel bad about it. Treat each one as a defect to be removed later. A script
|
|
178
|
-
should aim for **zero** `waitForTimeout` calls.
|
|
179
|
-
|
|
180
|
-
### `networkidle` is NOT a readiness signal
|
|
181
|
-
|
|
182
|
-
`waitForLoadState('networkidle')` means "the network went quiet", which in a
|
|
183
|
-
modern SPA happens long before — or long after — the content you want renders.
|
|
184
|
-
A large SPA commonly hits network idle while the DOM is still an empty ~600-char
|
|
185
|
-
shell, with every real value still to be fetched and rendered client-side. Never
|
|
186
|
-
treat `networkidle` as "the page is ready". Wait for the *specific element or
|
|
187
|
-
text* you need instead. Use `domcontentloaded` for the initial `goto`, then a
|
|
188
|
-
concrete element wait.
|
|
189
|
-
|
|
190
|
-
### Pick a signal that proves the exact thing you need
|
|
191
|
-
|
|
192
|
-
- "Tab bar loaded" → wait for a specific named tab to be visible.
|
|
193
|
-
- "List rendered" → wait for a row's distinguishing text (e.g. a commit hash
|
|
194
|
-
pattern), not a generic container that exists while empty.
|
|
195
|
-
- "Log loaded" → wait for the log element AND a length/content predicate, so an
|
|
196
|
-
empty placeholder doesn't satisfy the wait.
|
|
197
|
-
|
|
198
|
-
### Virtualized lists/grids → set a tall viewport, don't scroll-accumulate
|
|
199
|
-
|
|
200
|
-
A virtualized list or grid renders only the rows within the scroll viewport (a
|
|
201
|
-
31-row table may put only ~17 rows in the DOM). A single DOM sweep then
|
|
202
|
-
silently returns a partial set. The cheap, robust fix is to enlarge the viewport
|
|
203
|
-
BEFORE navigating, so the grid materializes every row at once:
|
|
204
|
-
|
|
205
|
-
```js
|
|
206
|
-
await page.setViewportSize({ width: 1600, height: 20000 }); // then goto()
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
This overrides the harness's default 1920×1080 per-page and needs no harness change.
|
|
210
|
-
Prefer it over a scroll-accumulate loop: far less code, no timing loop. Then **verify
|
|
211
|
-
completeness** — extract the count the UI advertises (e.g. a "Properties 31" header
|
|
212
|
-
badge) and assert the extracted row count equals it, so a clipped read fails loud
|
|
213
|
-
instead of returning a silent subset. A tall viewport is not universal: a virtualizer
|
|
214
|
-
bounded by its own container's fixed CSS height can still clip regardless of window
|
|
215
|
-
size — the assertion is what catches that, and scroll-accumulate is the fallback.
|
|
216
|
-
|
|
217
|
-
### Unpredictable interrupting UI → `addLocatorHandler`, not sleeps
|
|
218
|
-
|
|
219
|
-
Modals/banners that appear at an unpredictable moment (welcome dialogs, "what's
|
|
220
|
-
new", cookie prompts) are a classic flake source: dismiss-then-continue races the
|
|
221
|
-
modal's appearance. Register a handler once; Playwright auto-runs it whenever that
|
|
222
|
-
element would block an action — fully timing-independent:
|
|
223
|
-
|
|
224
|
-
```js
|
|
225
|
-
await page.addLocatorHandler(
|
|
226
|
-
page.getByRole('dialog').filter({ has: page.getByRole('button', { name: 'Close' }) }),
|
|
227
|
-
async (dialog) => { await dialog.getByRole('button', { name: 'Close' }).click(); }
|
|
228
|
-
);
|
|
229
|
-
```
|
|
230
|
-
|
|
231
|
-
### Timeouts
|
|
232
|
-
|
|
233
|
-
Pass `ctx.timeout` to waits rather than hardcoding numbers, so a slow environment
|
|
234
|
-
can be accommodated centrally. A generous timeout on a *correct* condition is
|
|
235
|
-
fine — it only ever waits as long as it must, then proceeds the instant the
|
|
236
|
-
condition holds. That is the opposite of a fixed sleep.
|
|
237
|
-
|
|
238
|
-
### Prefer robust locators
|
|
239
|
-
|
|
240
|
-
Favor role/text/label locators (`getByRole`, `getByText`, `getByLabel`) and stable
|
|
241
|
-
attributes (`data-testid`) over brittle CSS/class chains — component-library class
|
|
242
|
-
names (`bp6-…`) change between versions. When the only distinguishing feature is
|
|
243
|
-
visible text, a text/regex locator is more durable than a guessed class.
|