@lowdefy/docs-content 6.1.0 → 7.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -0
- package/content/actions/downloadcsv.md +1 -1
- package/content/actions/emailotpsend.md +58 -0
- package/content/actions/emailotpverify.md +79 -0
- package/content/actions/getlocalstorage.md +43 -0
- package/content/actions/link.md +47 -1
- package/content/actions/login.md +25 -5
- package/content/actions/logout.md +7 -5
- package/content/actions/magiclinkverify.md +117 -0
- package/content/actions/pdfmake.md +2 -2
- package/content/actions/posthogalias.md +29 -0
- package/content/actions/posthogcapture.md +49 -0
- package/content/actions/posthogcapturepageleave.md +38 -0
- package/content/actions/posthogfeatureflag.md +50 -0
- package/content/actions/posthoggroup.md +37 -0
- package/content/actions/posthogidentify.md +54 -0
- package/content/actions/posthoginit.md +96 -0
- package/content/actions/posthogoptin.md +29 -0
- package/content/actions/posthogoptout.md +29 -0
- package/content/actions/posthogpageview.md +37 -0
- package/content/actions/posthogreloadfeatureflags.md +47 -0
- package/content/actions/posthogreset.md +28 -0
- package/content/actions/posthogsetpersonproperties.md +36 -0
- package/content/actions/removelocalstorage.md +28 -0
- package/content/actions/resetvalidation.md +7 -3
- package/content/actions/setdarkmode.md +3 -3
- package/content/actions/setglobal.md +2 -0
- package/content/actions/setlocale.md +1 -1
- package/content/actions/setlocalstorage.md +34 -0
- package/content/actions/validate.md +7 -4
- package/content/adapter-reference/mongodbauthadapter.md +46 -0
- package/content/agents/agent-endpoint-tools.md +24 -0
- package/content/agents/agent-properties.md +13 -3
- package/content/agents/agent-server-hooks.md +1 -1
- package/content/agents/agentchat.md +43 -7
- package/content/agents/agentconversations.md +42 -8
- package/content/agents/agents-introduction.md +107 -0
- package/content/callback-reference/auth0logoutcallback.md +54 -0
- package/content/concepts/ai-agent-docs.md +455 -106
- package/content/concepts/blocks.md +150 -0
- package/content/concepts/connections-and-requests.md +402 -0
- package/content/concepts/custom-html.md +42 -0
- package/content/concepts/custom-styling.md +50 -5
- package/content/concepts/dynamic-page-content.md +309 -0
- package/content/concepts/events-and-actions.md +154 -2
- package/content/concepts/four-primitives.md +31 -0
- package/content/concepts/html-attributes.md +293 -0
- package/content/concepts/i18n.md +8 -3
- package/content/concepts/keyboard-shortcuts.md +2 -2
- package/content/concepts/layout-overview.md +1 -1
- package/content/concepts/logger.md +99 -0
- package/content/concepts/lowdefy-api.md +1319 -0
- package/content/concepts/lowdefy-schema.md +136 -3
- package/content/concepts/menus.md +8 -7
- package/content/concepts/module-authoring.md +76 -5
- package/content/concepts/modules.md +3 -1
- package/content/concepts/notifications.md +180 -0
- package/content/concepts/operators.md +7 -0
- package/content/concepts/page-and-app-state.md +252 -0
- package/content/concepts/page-paths.md +209 -0
- package/content/concepts/theming.md +582 -0
- package/content/connections/aigateway.md +375 -0
- package/content/connections/anthropic.md +169 -2
- package/content/connections/awss3.md +178 -2
- package/content/connections/axioshttp.md +181 -0
- package/content/connections/google.md +169 -2
- package/content/connections/mongodb.md +1014 -3
- package/content/connections/openai-connection.md +169 -2
- package/content/connections/sendgridmail.md +2 -0
- package/content/connections/smtp.md +3 -1
- package/content/connections/treg.md +329 -0
- package/content/container-blocks/affix.md +47 -7
- package/content/container-blocks/alert.md +76 -24
- package/content/container-blocks/badge.md +100 -24
- package/content/container-blocks/box.md +9 -9
- package/content/container-blocks/card.md +132 -27
- package/content/container-blocks/carousel.md +60 -7
- package/content/container-blocks/collapse.md +153 -9
- package/content/container-blocks/configprovider.md +35 -13
- package/content/container-blocks/confirmmodal.md +69 -23
- package/content/container-blocks/content.md +62 -98
- package/content/container-blocks/descriptions.md +45 -4
- package/content/container-blocks/drawer.md +139 -28
- package/content/container-blocks/dropdownmenu.md +119 -31
- package/content/container-blocks/flex.md +40 -62
- package/content/container-blocks/footer.md +15 -11
- package/content/container-blocks/googlemapsheatmap.md +1 -1
- package/content/container-blocks/googlemapsscript.md +2 -2
- package/content/container-blocks/header.md +71 -36
- package/content/container-blocks/label.md +34 -7
- package/content/container-blocks/layout.md +52 -34
- package/content/container-blocks/masonry.md +17 -71
- package/content/container-blocks/mobilemenu.md +131 -88
- package/content/container-blocks/modal.md +161 -26
- package/content/container-blocks/pageheadermenu.md +138 -31
- package/content/container-blocks/pagesidebarlayout.md +230 -27
- package/content/container-blocks/pagesidermenu.md +153 -43
- package/content/container-blocks/popover.md +34 -31
- package/content/container-blocks/result.md +32 -21
- package/content/container-blocks/sider.md +85 -122
- package/content/container-blocks/span.md +7 -7
- package/content/container-blocks/spinner.md +3 -3
- package/content/container-blocks/splitter.md +150 -68
- package/content/container-blocks/tabs.md +64 -12
- package/content/container-blocks/tooltip.md +22 -46
- package/content/container-blocks/watermark.md +22 -4
- package/content/controls/parallel_for.md +21 -2
- package/content/controls/reject.md +24 -3
- package/content/controls/return.md +17 -1
- package/content/controls/throw.md +1 -0
- package/content/controls/try.md +4 -2
- package/content/controls/while.md +46 -0
- package/content/deployment/deployment-environments.md +10 -7
- package/content/deployment/docker.md +22 -7
- package/content/deployment/node-server.md +4 -0
- package/content/deployment/vercel.md +17 -6
- package/content/display-blocks/aggrid.md +36 -19
- package/content/display-blocks/anchor.md +16 -15
- package/content/display-blocks/avatar.md +57 -37
- package/content/display-blocks/breadcrumb.md +70 -21
- package/content/display-blocks/button.md +93 -19
- package/content/display-blocks/calendar.md +12 -0
- package/content/display-blocks/clickablehtml.md +239 -0
- package/content/display-blocks/dangeroushtml.md +1 -1
- package/content/display-blocks/difflist.md +1 -1
- package/content/display-blocks/difftimeline.md +1 -1
- package/content/display-blocks/divider.md +64 -3
- package/content/display-blocks/dropdownbutton.md +81 -30
- package/content/display-blocks/echart.md +35 -1
- package/content/display-blocks/eventlog.md +363 -0
- package/content/display-blocks/floatbutton.md +129 -42
- package/content/display-blocks/html.md +61 -1
- package/content/display-blocks/icon.md +250 -67
- package/content/display-blocks/img.md +2 -2
- package/content/display-blocks/menu.md +146 -73
- package/content/display-blocks/message.md +38 -124
- package/content/display-blocks/notification.md +94 -222
- package/content/display-blocks/paragraph.md +59 -9
- package/content/display-blocks/pollingtimer.md +263 -0
- package/content/display-blocks/progress.md +96 -17
- package/content/display-blocks/qrcode.md +19 -4
- package/content/display-blocks/s3download.md +1 -1
- package/content/display-blocks/search.md +19 -9
- package/content/display-blocks/skeletonbutton.md +1 -0
- package/content/display-blocks/skeletoninput.md +1 -0
- package/content/display-blocks/statistic.md +75 -17
- package/content/display-blocks/steps.md +58 -25
- package/content/display-blocks/table.md +2527 -0
- package/content/display-blocks/tablelight.md +621 -0
- package/content/display-blocks/tag.md +102 -17
- package/content/display-blocks/title.md +50 -11
- package/content/display-blocks/tour.md +33 -23
- package/content/input-blocks/autocomplete.md +81 -9
- package/content/input-blocks/buttonselector.md +45 -8
- package/content/input-blocks/captcha.md +40 -0
- package/content/input-blocks/checkboxselector.md +97 -7
- package/content/input-blocks/checkboxswitch.md +33 -3
- package/content/input-blocks/colorselector.md +63 -14
- package/content/input-blocks/daterangeselector.md +165 -45
- package/content/input-blocks/dateselector.md +139 -41
- package/content/input-blocks/datetimeselector.md +158 -49
- package/content/input-blocks/inputgallery.md +3 -3
- package/content/input-blocks/intentinput.md +207 -0
- package/content/input-blocks/monthselector.md +120 -52
- package/content/input-blocks/multipleselector.md +150 -39
- package/content/input-blocks/numberinput.md +98 -12
- package/content/input-blocks/pagination.md +70 -14
- package/content/input-blocks/paragraphinput.md +51 -37
- package/content/input-blocks/passwordinput.md +60 -6
- package/content/input-blocks/phonenumberinput.md +59 -26
- package/content/input-blocks/placesautocomplete.md +319 -0
- package/content/input-blocks/radioselector.md +83 -7
- package/content/input-blocks/ratingslider.md +57 -26
- package/content/input-blocks/s3uploadbutton.md +24 -24
- package/content/input-blocks/s3uploaddragger.md +3 -3
- package/content/input-blocks/s3uploadphoto.md +5 -5
- package/content/input-blocks/segmentedselector.md +72 -32
- package/content/input-blocks/selector.md +109 -21
- package/content/input-blocks/switch.md +65 -17
- package/content/input-blocks/tableinput.md +445 -0
- package/content/input-blocks/tagmultipleselector.md +114 -0
- package/content/input-blocks/tagselector.md +114 -0
- package/content/input-blocks/textarea.md +32 -3
- package/content/input-blocks/textinput.md +70 -29
- package/content/input-blocks/tiptapinput.md +1 -1
- package/content/input-blocks/tiptapmentioninput.md +2 -2
- package/content/input-blocks/titleinput.md +39 -12
- package/content/input-blocks/treeinput.md +49 -25
- package/content/input-blocks/treemultipleselector.md +127 -10
- package/content/input-blocks/treeselector.md +97 -9
- package/content/input-blocks/upload.md +64 -15
- package/content/input-blocks/uploaddragger.md +17 -1
- package/content/input-blocks/uploadphoto.md +18 -1
- package/content/input-blocks/weekselector.md +120 -55
- package/content/learn/introduction.md +27 -0
- package/content/learn/quickstart.md +53 -0
- package/content/list-blocks/controlledlist.md +24 -15
- package/content/list-blocks/list.md +11 -11
- package/content/list-blocks/masonrylist.md +7 -34
- package/content/migration/auth-upgrade.md +337 -0
- package/content/migration/upgrade-guide.md +1 -0
- package/content/migration/v3-to-v4.md +417 -0
- package/content/migration/v6-to-v7.md +620 -0
- package/content/operators/_agent.md +155 -0
- package/content/operators/_app.md +2 -1
- package/content/operators/_array.md +122 -1
- package/content/operators/_build.md +1 -0
- package/content/operators/_credential.md +57 -0
- package/content/operators/_cron.md +292 -0
- package/content/operators/_date.md +2 -0
- package/content/operators/_error.md +24 -1
- package/content/operators/_get.md +6 -6
- package/content/operators/_hmac.md +78 -0
- package/content/operators/_js.md +2 -1
- package/content/operators/_location.md +1 -1
- package/content/operators/_media.md +13 -0
- package/content/operators/_module.md +56 -2
- package/content/operators/_operator.md +17 -3
- package/content/operators/_path_params.md +82 -0
- package/content/operators/_ref.md +258 -0
- package/content/operators/_request.md +1 -1
- package/content/operators/_request_details.md +6 -0
- package/content/operators/_type.md +14 -3
- package/content/operators/_url_query.md +2 -0
- package/content/operators/_user.md +3 -3
- package/content/plugins/plugins-actions.md +10 -0
- package/content/plugins/plugins-connections.md +25 -0
- package/content/plugins/plugins-dev.md +66 -0
- package/content/plugins/plugins-icon-sets.md +267 -0
- package/content/plugins/plugins-introduction.md +27 -1
- package/content/plugins/plugins.md +19 -0
- package/content/plugins/posthog.md +359 -0
- package/content/provider-reference/auth-js-providers.md +107 -0
- package/content/provider-reference/openidconnectprovider.md +44 -0
- package/content/recipes/recipes.md +16 -0
- package/content/reference/actions-reference.md +31 -0
- package/content/reference/blocks-reference.md +35 -0
- package/content/reference/cli.md +403 -0
- package/content/reference/connections-reference.md +23 -0
- package/content/reference/operators-reference.md +47 -0
- package/content/reference/reference.md +16 -0
- package/content/reference/schema-reference.md +12 -0
- package/content/testing/config-tests.md +677 -0
- package/content/testing/e2e-configuration.md +31 -1
- package/content/testing/e2e-getting-started.md +1 -1
- package/content/testing/e2e-introduction.md +1 -1
- package/content/testing/e2e-writing-tests.md +12 -1
- package/content/testing/explore-pull-request.md +136 -0
- package/content/testing/journey-data-sets.md +326 -0
- package/content/tutorial/tutorial-actions-operators.md +2 -2
- package/content/tutorial/tutorial-add-blocks.md +2 -2
- package/content/tutorial/tutorial-create-page.md +48 -2
- package/content/tutorial/tutorial-display-data-page.md +3 -3
- package/content/tutorial/tutorial-requests-api.md +155 -0
- package/content/tutorial/tutorial-requests-sql.md +153 -0
- package/content/tutorial/tutorial-start.md +66 -1
- package/content/user-authentication/auth-configuration.md +267 -0
- package/content/user-authentication/auth-providers.md +24 -0
- package/content/user-authentication/auth-steps.md +141 -0
- package/content/user-authentication/login-and-logout.md +160 -8
- package/content/user-authentication/mcp-oauth.md +424 -0
- package/content/user-authentication/organizations.md +246 -0
- package/content/user-authentication/protected-pages-apis.md +97 -0
- package/content/user-authentication/roles.md +49 -0
- package/content/user-authentication/two-factor.md +229 -0
- package/content/user-authentication/user-object.md +58 -0
- package/content/user-authentication/users-introduction.md +8 -0
- package/content/websockets/websocket-publish.md +105 -0
- package/content/websockets/websocket-subscriptions.md +136 -0
- package/content/websockets/websockets-introduction.md +101 -0
- package/hazards.json +44 -0
- package/index.json +3691 -14
- package/package.json +11 -1
- package/scripts/collectImportedFiles.js +80 -0
- package/scripts/extractAgentDocs.js +41 -30
- package/scripts/hashDocSources.js +38 -0
- package/scripts/isRedirectPage.js +26 -0
- package/scripts/recordSourceHashes.js +100 -0
- package/scripts/resolvePageSections.js +42 -0
- package/scripts/sectionKind.js +27 -0
- package/content/concepts/cli.md +0 -223
- package/content/other/introduction.md +0 -39
- package/content/other/next-auth-providers.md +0 -3
- /package/content/{other → actions}/updatesession.md +0 -0
|
@@ -0,0 +1,677 @@
|
|
|
1
|
+
# Config Tests
|
|
2
|
+
|
|
3
|
+
Config tests let you test a Lowdefy app in the same language you build it in. A **journey** is a YAML file that names a page, a user to act as, and a list of steps — click this block, fill that input, wait for a request, expect this state — and `lowdefy test` runs every journey in your app and reports which passed.
|
|
4
|
+
|
|
5
|
+
Journeys need no JavaScript, no Playwright setup and no separate test build. The runner boots your app's development server headless, drives each journey through the dev server's [journey route](/ai-agent-docs) (the same one the `lowdefy_run_journey` MCP tool uses), and prints a pass/fail line per journey. Because the format is declarative, an AI coding agent can write a journey for a page it just changed, run it, and read the failing step.
|
|
6
|
+
|
|
7
|
+
> Config tests run against the **development server**. For tests against a production build, or tests that need custom browser code, use the Playwright based [e2e testing](/e2e-introduction) toolkit instead.
|
|
8
|
+
|
|
9
|
+
## Layout
|
|
10
|
+
|
|
11
|
+
Journeys live in `tests/journeys/` inside your config directory, one `.yaml` file per journey or a top-level list of journeys per file. Group them in sub-folders as the suite grows; `lowdefy test` reads every sub-folder except those whose name starts with `_`, such as `_candidates/`, which hold journeys that are not part of the suite yet:
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
my-app/
|
|
15
|
+
├── lowdefy.yaml
|
|
16
|
+
├── pages/
|
|
17
|
+
└── tests/
|
|
18
|
+
└── journeys/
|
|
19
|
+
├── controls.yaml
|
|
20
|
+
├── sign-up.yaml
|
|
21
|
+
├── review/
|
|
22
|
+
│ ├── approve.yaml
|
|
23
|
+
│ └── reject.yaml
|
|
24
|
+
└── _candidates/
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Files run in path order, and journeys run one at a time — each journey opens its own browser context, but they all share your app's database, so parallel journeys would interfere with one another.
|
|
28
|
+
|
|
29
|
+
## A journey
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
# tests/journeys/controls.yaml
|
|
33
|
+
- name: member creates a control
|
|
34
|
+
pageId: controls
|
|
35
|
+
user: { roles: [admin] }
|
|
36
|
+
urlQuery: { status: open }
|
|
37
|
+
steps:
|
|
38
|
+
- click: new_control
|
|
39
|
+
- fill: { blockId: title, value: Access reviews }
|
|
40
|
+
- click: submit
|
|
41
|
+
- wait: { request: get_controls }
|
|
42
|
+
- expect: { state: { path: controls.0.title, equals: Access reviews } }
|
|
43
|
+
|
|
44
|
+
- name: guest sees the empty state
|
|
45
|
+
pageId: controls
|
|
46
|
+
steps:
|
|
47
|
+
- expect: { visible: empty_state }
|
|
48
|
+
|
|
49
|
+
- name: member opens a control from its own page
|
|
50
|
+
pageId: control
|
|
51
|
+
pathParams: { control_id: c-access-reviews }
|
|
52
|
+
user: { roles: [admin] }
|
|
53
|
+
steps:
|
|
54
|
+
- expect: { text: { blockId: title, contains: Access reviews } }
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
A page whose [`path`](/page-paths) has placeholders, such as `controls/{control_id}`, has one instance per set of values. A journey names the instance it opens with `pathParams`, and the runner builds the URL from the page's path. It stops before opening a browser if a placeholder has no value.
|
|
58
|
+
|
|
59
|
+
| Field | Required | Description |
|
|
60
|
+
| ------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
61
|
+
| `name` | Yes | A short name, at most 100 characters. `--filter` matches against it, and it is printed in the results. |
|
|
62
|
+
| `description` | No | What the journey proves, its goal, in at most 60 words. It is for people and agents reading the suite; `lowdefy test` does not print it. |
|
|
63
|
+
| `pageId` | Yes | The page to open. |
|
|
64
|
+
| `user` | No | The user to act as, as an inline user object such as `{ sub: u1, roles: [admin] }`. Leave it out to run as the default roleless headless user, or as `auth.dev.mockUser` when one is active; a `user` given here wins over the mock user. On a journey with `data`, the name of one of the data set's users, such as `member`, or a list of them, such as `[admin, member]`, to [run the journey once as each](/journey-data-sets#one-journey-several-users). |
|
|
65
|
+
| `data` | No | A [data set](/journey-data-sets) name. The journey runs against a fresh in-memory MongoDB database of its own, loaded with the data set, so it may write freely. |
|
|
66
|
+
| `urlQuery` | No | An object appended to the page URL as a query string, for pages that read `_url_query`. |
|
|
67
|
+
| `pathParams` | No | An object of strings, one per placeholder of the page's [`path`](/page-paths), for pages that read `_path_params`. Leave it out for a page whose path has no placeholders. |
|
|
68
|
+
| `tags` | No | A list of tags naming the sections of the suite the journey belongs to, such as `[smoke, review]`. `lowdefy test --tag smoke` runs the journeys tagged `smoke`. Tags are lowercase letters, digits, `-` and `_`. |
|
|
69
|
+
| `steps` | Yes | At least one step. Each step is an object with exactly one key from the step grammar below. |
|
|
70
|
+
|
|
71
|
+
`user: none` injects no user at all, so the journey signs in through the app's own auth — see [Testing sign-up and sign-in](#testing-sign-up-and-sign-in). It is refused on a data set journey while auth is configured, and on any journey while a dev mock user is active, since every request would then act as the mock user.
|
|
72
|
+
|
|
73
|
+
`timeout` sets how long each step may wait, in milliseconds (a whole number from 1 to 60000, default 5000). Raise it on a slow machine rather than adding `wait: { ms }` steps.
|
|
74
|
+
|
|
75
|
+
### Timeouts
|
|
76
|
+
|
|
77
|
+
The journey's `timeout` (the step timeout) bounds every wait a step makes for something to happen, and a step moves on as soon as it has:
|
|
78
|
+
|
|
79
|
+
| Wait | Bound |
|
|
80
|
+
| -------------------------------------------------------------------------------------------------- | ------------------------------------------ |
|
|
81
|
+
| A control becoming actionable for `click`, `open`, `fill`, `select`, and the popup an `open` shows | The step timeout. |
|
|
82
|
+
| `expect` (`state`, `visible`, `text`, `url`, `title`) matching | The step timeout. |
|
|
83
|
+
| `wait: { request }` and `wait: { state }` | The step timeout. |
|
|
84
|
+
| `back` loading the previous page | The step timeout. |
|
|
85
|
+
| An email arriving for `email` or `fill.fromEmail` | The step timeout. |
|
|
86
|
+
| Opening a page: the journey's page, `goto`, the first `as` for a name | The step timeout, but at least 15 seconds. |
|
|
87
|
+
| Settling after an interaction (`click`, `fill`, `select`, `press`, `back`) | The step timeout, but at most 5 seconds. |
|
|
88
|
+
| `wait: { ms }` | Exactly `ms`; the timeout does not apply. |
|
|
89
|
+
|
|
90
|
+
The settle after an interaction lets the page's own events and requests finish before the next step. It never fails a step: a page still busy after 5 seconds (an event that ends in a resend cooldown, for example) moves on, and the next step waits for what it needs itself. So raising `timeout` makes a slow journey pass without making a passing one slower.
|
|
91
|
+
|
|
92
|
+
## Steps
|
|
93
|
+
|
|
94
|
+
Blocks are addressed by their `blockId`. A step that does not complete within the step timeout (5 seconds, or the journey's [`timeout`](#timeouts)) fails the journey. An `expect` step waits, up to that timeout, for what it checks to become true, so a value a click leads to can arrive a moment later.
|
|
95
|
+
|
|
96
|
+
| Step | Meaning |
|
|
97
|
+
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
98
|
+
| `click: target` | Click the block, or the control a [target](#targets) narrows to. |
|
|
99
|
+
| `click: { ...target, count: 2 }` | Click 2 (or 3) times in quick succession, as a person's double click, before the runner waits for the page to settle. `count` defaults to 1. |
|
|
100
|
+
| `open: target` | Open an input's dropdown or picker popup (Selector, MultipleSelector, AutoComplete, DateSelector, Cascader...) and wait for it to show, so a following `screenshot` or `select` sees the options. A block with no popup is clicked; one whose click opens nothing fails the step. |
|
|
101
|
+
| `fill: { blockId, value }` | Type `value` into the input inside the block (or the grid cell a target names). A block that renders its input in a portal, such as a `Search` modal once opened, is reached through the same `blockId`. A `ColorSelector` takes a 6 or 8 digit hex colour (`#1677ff`): its picker opens, the hex is typed and the picker closes. Its value reads back as lowercase hex without an `ff` alpha, so `#1677FFFF` reads `#1677ff`. A block with neither a text input nor a colour picker fails, naming its type. |
|
|
102
|
+
| `fill: { blockId, fromEmail }` | Type text read from an [email](#emails) instead of a fixed value, such as a one-time sign-in code. The actor stays on the page. |
|
|
103
|
+
| `select: { blockId, value }` | Open the selector block (or grid cell) and choose the option whose text is `value`. A radio, button or segmented selector's option is clicked by its label; a checkbox option is ticked, and left ticked if it already is. |
|
|
104
|
+
| `press: Enter` | Press a key or chord. `Mod` in a chord (`Mod+k`) resolves to Cmd on macOS and Ctrl elsewhere. |
|
|
105
|
+
| `back: true` | Go back one page, like the browser's Back button. Fails when the journey has not navigated from an earlier page. |
|
|
106
|
+
| `goto: pageId` | Load a page the way a typed URL does; `{ pageId, pathParams, urlQuery }` adds the values of a [page path](/page-paths)'s placeholders and a query string. A protected page may redirect (to sign in), so assert where it landed. |
|
|
107
|
+
| `email: { to, subject }` | Open the newest [email](#emails) to `to` — with a subject containing `subject`, when given — that arrived during the journey, waiting for it if needed. |
|
|
108
|
+
| `as: name` | Act as [another person](#several-people), each in their own browser. The journey starts as `main`. On a journey with `data`, the name is `main` or one of the data set's users. |
|
|
109
|
+
| `wait: { ms }` | Pause for `ms` milliseconds. |
|
|
110
|
+
| `wait: { request: requestId }` | Wait until the request has been called since the last interaction (`click`, `open`, `fill`, `select`, `press`, `back`, `goto` or `as`; the page open before the first) and has finished loading. A call the page made before that interaction does not count. |
|
|
111
|
+
| `wait: { state: path }` | Wait until the state value at `path` is defined. |
|
|
112
|
+
| `screenshot: name` | Capture a screenshot. Screenshots are returned to agents using the MCP tool; the CLI runner ignores them. |
|
|
113
|
+
| `expect: { state: { path, equals } }` | The page state at `path` deep-equals `equals`. A path that does not exist reads as `null`, so `equals: null` also passes for a misspelt path. |
|
|
114
|
+
| `expect: { visible: target }` | The block, or the control a target narrows to, is visible. |
|
|
115
|
+
| `expect: { hidden: target }` | Nothing the target names is visible: no element matches, or every match is hidden. Passes at once when nothing matches yet, so pair it with a presence. |
|
|
116
|
+
| `expect: { calls: { request, count } }` | This person's browser called the request `count` times since the journey started, counted once the page settles. `pageId` names the request's page. |
|
|
117
|
+
| `expect: { calls: { endpoint, count } }` | The same for an endpoint called with `CallAPI`. Counts survive full page loads, so `count: 0` after a reload checks a write was never sent. |
|
|
118
|
+
| `expect: { text: { blockId, contains } }` | The block's rendered text (or a grid row's or cell's) contains the string. |
|
|
119
|
+
| `expect: { url: { contains } }` | The browser URL contains the string. It checks where the browser landed and never names a page: open a page with `pageId` or `goto`, with its `pathParams`. |
|
|
120
|
+
| `expect: { title: { equals } }` | The document title (the browser tab's text) is exactly the string; `{ contains }` checks part of it. |
|
|
121
|
+
| `expect: { error: text }` | The interaction just before it raised an [app error](#app-errors) whose message contains `text`. Must directly follow a click, open, fill, select, press or back. |
|
|
122
|
+
| `expect: { effect: true }` | The interaction just before it did something. Must directly follow a click, open, fill, select, press or back. |
|
|
123
|
+
|
|
124
|
+
`expect.calls` takes `{ request: requestId, pageId, count }`: request ids are scoped to a page, and two pages often share one such as `save`, so `pageId` names the page; it defaults to the page the person is on when the step runs. It compares once, without waiting for the count to change, because "not called" can only be judged after the moment has passed.
|
|
125
|
+
|
|
126
|
+
Before a step runs, every block, request, endpoint and `goto` page it names is checked against the app's build, and so is an `as` name on a journey with `data`. A misspelt one fails the step and names it, so `expect: { hidden: nosuchblock }` or `expect: { calls: { request: nosuchrequest, count: 0 } }` cannot pass by finding nothing. Blocks and requests are looked for on the page the person is on, and blocks also on the page the journey opened, so a step on a page that sent the person elsewhere still fails as not visible. A block inside a List matches by its `$` id, so `rows.0.name` matches `rows.$.name`, and a block of [dynamic page content](/dynamic-page-content) matches once the page shows it; `expect.state` paths are not checked.
|
|
127
|
+
|
|
128
|
+
`expect.effect` fails when the interaction before it ran no event, changed nothing on the page, called no request or endpoint and left the URL as it was. It reads what the runner saw during that interaction, so it does not wait. After a click that should do something and does not, it makes the journey fail until the control does something.
|
|
129
|
+
|
|
130
|
+
### App errors
|
|
131
|
+
|
|
132
|
+
A journey also fails at the step that causes an app error, even when every expectation after it holds. An app error is:
|
|
133
|
+
|
|
134
|
+
- an action that fails with an error, such as a `Request` or `CallAPI` whose request throws, or an operator that throws,
|
|
135
|
+
- an uncaught exception in the page, or an error the page reports to the dev server,
|
|
136
|
+
- a request or endpoint that throws on the server, or answers with a 5xx.
|
|
137
|
+
|
|
138
|
+
Errors raised while the journey's first page opens (its `onInit` and `onMount` requests) fail the journey `on open`, before any step runs. Expected outcomes never fail a journey: a failed `Validate`, a `Throw` action or any other user error, and a 401 or 403 refusal. Only errors the journey's own browsers cause count: an error you raise in your own tab on the same dev server while journeys run does not fail them, and a journey's errors are reported in its result, not in the dev server's build status.
|
|
139
|
+
|
|
140
|
+
```
|
|
141
|
+
FAIL member saves a ticket
|
|
142
|
+
file: /my-app/tests/journeys/tickets.yaml
|
|
143
|
+
step 2: { click: save }
|
|
144
|
+
server-error MongoDB: MongoDB rejected the MongoDBInsertOne command. pages/tickets.yaml:42
|
|
145
|
+
action-error Request "save_ticket" failed in save.onClick with ServiceError. pages/tickets.yaml:88
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Each error line gives its kind, its message and the config file and line it came from, when known. Fix the app in most cases. When the error is the outcome the app means, such as a unique index violation that a `catch` action shows as "already exists", either make it a user error in config (a `Throw` action is one), or assert it with `expect: { error: text }` straight after the interaction:
|
|
149
|
+
|
|
150
|
+
```yaml
|
|
151
|
+
- click: save
|
|
152
|
+
- expect: { error: duplicate key }
|
|
153
|
+
- expect: { visible: already_exists }
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The expectation claims the errors of that interaction whose message contains `text`, with the failed action that reported them, and fails when none matches. Any other error the interaction raised still fails it.
|
|
157
|
+
|
|
158
|
+
### Recorded values: `from`
|
|
159
|
+
|
|
160
|
+
`fill`, `select` and `expect.state` take an optional `from`, which marks a value taken from a recording:
|
|
161
|
+
|
|
162
|
+
- `from: recorded` marks a value observed in a trace. The step runs as usual; the marker tells you the value is whatever that recording happened to use, so review it.
|
|
163
|
+
- `from: shape` marks a placeholder, `value: null`, for a value the trace could not hold: a value typed in production, or a password the dev recorder did not keep. The runner refuses a journey that holds one before it opens a browser, naming the step. Fill the value from your test data or the journey's user, then remove `from`.
|
|
164
|
+
|
|
165
|
+
A `fill` or `select` with `value: null` and no `from: shape` is a grammar error.
|
|
166
|
+
|
|
167
|
+
The full grammar, including the failure shape the route returns, is documented with the [journey tool](/ai-agent-docs). The CLI and the MCP tool share one implementation, so a journey an agent verifies interactively can be committed as-is.
|
|
168
|
+
|
|
169
|
+
### Targets
|
|
170
|
+
|
|
171
|
+
A `blockId` reaches a block's own control — its button, input or link. Some controls are not blocks: the Edit and Delete buttons a grid renders in every row, the OK and Cancel of a confirm dialog, the items of a dropdown menu. Wherever a step takes a `blockId`, it also takes a target object that narrows the search:
|
|
172
|
+
|
|
173
|
+
| Key | Meaning |
|
|
174
|
+
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
175
|
+
| `blockId` | The block to search inside. |
|
|
176
|
+
| `row` | A grid row, zero-based as displayed (`AgGrid*` blocks). Needs `blockId`. |
|
|
177
|
+
| `column` | A grid cell in that row, by the column's `field` or `colId`. Needs `blockId`. |
|
|
178
|
+
| `text` | The interactive control whose visible text is exactly this (a button label, a tab, a menu item). |
|
|
179
|
+
| `containing` | The innermost element whose visible text contains this, as the page shows it, even across elements (`Assigned to <b>Jane</b>`, a label and value side by side): a row of a list a person picks by the name or email it shows. |
|
|
180
|
+
| `nth` | When several elements match, the zero-based one to use. |
|
|
181
|
+
|
|
182
|
+
`text` on its own, with no `blockId`, searches the whole page — front-most layer first: an open dropdown menu, then an open dialog, then the page. That is how a confirm dialog's button is clicked while the grid behind its mask has a button with the same label.
|
|
183
|
+
|
|
184
|
+
A `click`, `open`, `fill` or `select` whose `text` or `containing` matches more than one visible element fails rather than guess which one you meant. The failure says how many it matched and where, for example `Matched 3 controls with text "Delete" in the page; add nth: 0..2, or a blockId/row to narrow it.` Add `nth`, or narrow the target with `blockId`, `row` or `column`, so it names one element. Expectations are not strict: `expect.visible` passes when any match is visible, and `expect.hidden` when none is.
|
|
185
|
+
|
|
186
|
+
```yaml
|
|
187
|
+
- name: member deletes a control from the grid
|
|
188
|
+
pageId: controls
|
|
189
|
+
user: { roles: [admin] }
|
|
190
|
+
steps:
|
|
191
|
+
- click: { blockId: controls_grid, row: 1, text: Delete } # the row's Delete cell button
|
|
192
|
+
- expect: { visible: { text: Are you sure? } }
|
|
193
|
+
- click: { text: Delete } # the confirm dialog's OK, not the grid's
|
|
194
|
+
- wait: { request: delete_control }
|
|
195
|
+
- expect: { text: { blockId: controls_grid, row: 1, column: title, contains: Key rotation } }
|
|
196
|
+
- click: { blockId: controls_grid, row: 0, column: actions } # the cell's first control
|
|
197
|
+
- click: { blockId: controls_grid, row: 0, column: more, nth: 0 } # an icon-only menu trigger
|
|
198
|
+
- click: { text: Archive } # the open menu's item
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Some rows are neither a block nor a control, such as the cards of a `ListSelector`. `containing` clicks the text a person would click on, and the click reaches the row's own handler:
|
|
202
|
+
|
|
203
|
+
```yaml
|
|
204
|
+
- click: { blockId: members_list, containing: ada@example.test } # opens Ada's row
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
`fill`, `select` and `expect.text` always need a `blockId`; a value is typed into a block's input, never into a page-wide control. A target with a key the grammar does not know (`colum`) is rejected before the browser opens, so a typo cannot pass as a step that happened to find nothing.
|
|
208
|
+
|
|
209
|
+
A `Search` block keeps its input in a modal outside its own element. Open it with `click: <blockId>`, then `fill: { blockId, value }` types the query, and `click: { text: <result title> }` picks a result and fires `onSelect`. The result title is the text of the row when the search has no `description` mapped; with a description, pick it with `containing`.
|
|
210
|
+
|
|
211
|
+
## Testing sign-up and sign-in
|
|
212
|
+
|
|
213
|
+
A journey with `user: none` injects no user: it starts signed out, and the app's own auth decides who it is, exactly as in a real browser. Sign-up, email verification, sign-in, sign-out and organization switching all run for real, and the session cookie a sign-in sets carries through every later step.
|
|
214
|
+
|
|
215
|
+
```yaml
|
|
216
|
+
# tests/journeys/sign-up.yaml
|
|
217
|
+
- name: a new user signs up, verifies by email and signs in
|
|
218
|
+
pageId: signup
|
|
219
|
+
user: none
|
|
220
|
+
timeout: 30000
|
|
221
|
+
steps:
|
|
222
|
+
- fill: { blockId: email, value: ada@example.test }
|
|
223
|
+
- fill: { blockId: password, value: correct-horse-battery }
|
|
224
|
+
- click: signup_button
|
|
225
|
+
- email: { to: ada@example.test, subject: Verify your email address }
|
|
226
|
+
- click: { text: Verify email address }
|
|
227
|
+
- goto: login
|
|
228
|
+
- fill: { blockId: email, value: ada@example.test }
|
|
229
|
+
- fill: { blockId: password, value: correct-horse-battery }
|
|
230
|
+
- click: login_button
|
|
231
|
+
- goto: dashboard
|
|
232
|
+
- expect: { text: { blockId: user_details, contains: ada@example.test } }
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
A journey that opens a protected page signed out lands on the sign-in page, the way a visitor would; assert where it landed with `expect: { url: ... }`.
|
|
236
|
+
|
|
237
|
+
Give journeys that sign people in a `timeout` of about 30000. They load many pages, reload the app after each sign-in and wait for real emails, so on a busy machine the 5 second default fails them at random steps.
|
|
238
|
+
|
|
239
|
+
### Emails
|
|
240
|
+
|
|
241
|
+
The `email` step reads the mail the development server captured. Start the server with `LOWDEFY_DEV_SMTP_PORT` set to a free port (in the shell or the app's `.env`; the mail sink starts with the server, so restart it after adding the variable), and point the app's SMTP connection at `127.0.0.1` on that port through its secrets. The server then receives the app's mail over SMTP and keeps it instead of delivering it: every message the app sends — verification, magic link, invitation, your own `SMTPMailSend` requests — is written to `.lowdefy/mail/` in the app directory, and the directory is emptied each time the server starts. Only the development server does this, and only with the variable set; nothing in a production build captures mail.
|
|
242
|
+
|
|
243
|
+
```yaml
|
|
244
|
+
connections:
|
|
245
|
+
- id: email
|
|
246
|
+
type: SMTP
|
|
247
|
+
properties:
|
|
248
|
+
from: app@example.com
|
|
249
|
+
host:
|
|
250
|
+
_secret: SMTP_HOST # 127.0.0.1 for tests
|
|
251
|
+
port:
|
|
252
|
+
_number.parseInt:
|
|
253
|
+
on:
|
|
254
|
+
_secret: SMTP_PORT # the LOWDEFY_DEV_SMTP_PORT value for tests
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
`email` opens the newest matching message that arrived since the journey started, waiting up to the step timeout for one to arrive. The email is shown in the actor's tab, so `click: { text: Verify email address }` follows its button the way a person does, and `expect: { visible: { text: ... } }` checks its content. Opening the same email again (following an invitation link a second time after signing up) opens the same message. When nothing matches, the step fails with the messages that did arrive.
|
|
258
|
+
|
|
259
|
+
A one-time code is typed, not clicked. `fill` with `fromEmail: { to, subject, match }` reads the same newest matching message, waiting for it the same way, and types the first match of the regular expression `match` (or its first capture group, when it has one) into the block. The actor stays on the page they are on, as a person reads the code on their phone and types it into the tab they started from. The match runs on the email's text, not its HTML markup. Anchor the pattern with `\b`, since the text can also hold the sign-in link, whose address may contain digits.
|
|
260
|
+
|
|
261
|
+
```yaml
|
|
262
|
+
- click: login_magic_send
|
|
263
|
+
- fill:
|
|
264
|
+
blockId: otp
|
|
265
|
+
fromEmail:
|
|
266
|
+
to: ada@example.test
|
|
267
|
+
subject: Your sign-in link
|
|
268
|
+
match: '\b\d{6}\b'
|
|
269
|
+
- click: login_code_verify
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
### Several people
|
|
273
|
+
|
|
274
|
+
`as: invitee` switches the journey to another person with their own browser and cookies: an owner and the person they invite, or a member whose session stays open while the owner removes them. The journey starts as `main`. The first `as` for a name opens the journey's page in a new browser, as the journey's `user`, or as the data set user of that name on a journey with `data` (see [Journey data sets](/journey-data-sets)); switching back returns to that person's tab as they left it.
|
|
275
|
+
|
|
276
|
+
**Seeing what a second person sees.** On a journey with `data`, every person reads and writes the run's databases, so act as one user and then check, in the same run, what another user sees. The second user is a user of the data set, so no sign-in is needed:
|
|
277
|
+
|
|
278
|
+
```yaml
|
|
279
|
+
name: overriding a decision tells its owner
|
|
280
|
+
data: reviews
|
|
281
|
+
user: officer
|
|
282
|
+
pageId: review
|
|
283
|
+
steps:
|
|
284
|
+
- click: override_decision
|
|
285
|
+
- expect: { state: { path: decision.status, equals: overridden } }
|
|
286
|
+
- as: owner # a user of the data set, in a browser of their own
|
|
287
|
+
- goto: notifications/all
|
|
288
|
+
- expect: { text: { blockId: inbox, contains: Kirsty changed your decision } }
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
What `officer` wrote is in the database `owner` reads, and nothing of `officer`'s session (the signed-in user, cached requests, page state) reaches `owner`, who has their own browser. `as: main` returns to `officer`'s tab as it was left. Because everyone in the run shares its databases, a dropped `CallApi` that sends the notification fails the journey, and a [config mutant](#mutants-lowdefy-journeys-harden) that drops it is killed. What the journey exercised is the union over everyone it acted as: the requests, events and blocks `officer` and `owner` each touched are recorded for the journey, so coverage, lint and `journeys harden` need no per-person bookkeeping. An `as` that names the person already acting is a lint warning ([L7](#lint)).
|
|
292
|
+
|
|
293
|
+
Each person also sends requests from their own client address, so auth rate limits (a few sign-in attempts per address every few seconds) count each person's attempts apart, as they would for people on different devices, instead of one budget for the whole run.
|
|
294
|
+
|
|
295
|
+
### The database
|
|
296
|
+
|
|
297
|
+
Journeys that act as injected users and need data of their own belong on a [data set](/journey-data-sets): each run gets a fresh in-memory database, and your own database is never touched. Sign-up and sign-in journeys go through the app's real auth, which a data set does not redirect, so they need a test database of their own:
|
|
298
|
+
|
|
299
|
+
Journeys perform real sign-ups, so they need a database that starts empty and is never a real one. Run them against a fresh test database each time: a sign-up journey run a second time finds its address already registered, and no verification email is sent. Give each journey its own addresses, so journeys in one run do not collide.
|
|
300
|
+
|
|
301
|
+
Keep journeys that write, like these, out of `tests/journeys/` when your everyday `lowdefy test` runs against a shared database. Put them in their own directory, such as `tests/auth-journeys/`, and run them only from a script that starts the test database, the mail sink and the server they need:
|
|
302
|
+
|
|
303
|
+
```
|
|
304
|
+
lowdefy test --journeys-directory tests/auth-journeys --url http://localhost:3290
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
A plain `lowdefy test`, and the `lowdefy_run_tests` agent tool, never read that directory.
|
|
308
|
+
|
|
309
|
+
## Running
|
|
310
|
+
|
|
311
|
+
```
|
|
312
|
+
pnpx lowdefy@7 test
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
With no options, `lowdefy test` prepares `.lowdefy/dev` exactly as `lowdefy dev` does, starts the development server on a free port without opening a browser, runs every journey, prints the results and stops the server.
|
|
316
|
+
|
|
317
|
+
Journeys run on your machine, never in CI. `lowdefy test` needs only Node.js, pnpm and a Chromium the dev server can launch. The dev server uses Playwright's `chromium-headless-shell`, which it downloads itself the first time a browser tool needs it, and falls back to an installed Google Chrome meanwhile. Set `PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1` to turn the download off.
|
|
318
|
+
|
|
319
|
+
To verify the change in hand, run all journeys, a folder or glob of them, or a tagged section:
|
|
320
|
+
|
|
321
|
+
```
|
|
322
|
+
lowdefy test # every journey in tests/journeys/, sub-folders included, "_" folders left out
|
|
323
|
+
lowdefy test tests/journeys/review # a folder, with its sub-folders
|
|
324
|
+
lowdefy test 'tests/journeys/**/review-*.yaml' # a glob, quoted so the CLI expands it
|
|
325
|
+
lowdefy test --tag smoke --tag review # the journeys tagged smoke or review
|
|
326
|
+
lowdefy test --filter approve # the journeys whose name contains "approve"
|
|
327
|
+
lowdefy test --tier common # the most-used journeys, by production use
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
Paths, tags and filters combine: `lowdefy test tests/journeys/review --tag smoke` runs the smoke journeys in the review folder. Put the selections you use often in `package.json` scripts, such as `"test:smoke": "lowdefy test --tag smoke"`. The `lowdefy_run_tests` agent tool takes the same selections as `paths`, `tags`, `filter` and `tier`.
|
|
331
|
+
|
|
332
|
+
Once the journeys carry production [evidence](#evidence), `--tier` narrows the selection further to its most-used journeys, cut over that selection as described in [Usage and tiers](#usage-and-tiers): `lowdefy test --tier common` checks the happy paths in a fraction of the time, and `--tier edge` adds the edge cases real users still reach. Run `common` while you work on a change and `edge` before you call it done. `lowdefy test tests/journeys/review --tier common` is the happy paths of the review folder. A journey with a list of users is tiered once and runs as each of its users. Journeys with no counts for their current steps run in every tier, so a new or edited journey is always tested. A tiered run is refused when the selection has fewer than 100 journey matches in the usage window, or no evidence at all.
|
|
333
|
+
|
|
334
|
+
```
|
|
335
|
+
PASS member creates a control (5 steps, 1840ms)
|
|
336
|
+
FAIL guest sees the empty state
|
|
337
|
+
file: /my-app/tests/journeys/controls.yaml
|
|
338
|
+
step 0: { expect: { visible: empty_state } }
|
|
339
|
+
expected: block "empty_state" to be visible
|
|
340
|
+
actual: Timeout 5000ms exceeded.
|
|
341
|
+
1 passed, 1 failed of 2 journeys
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
A failing journey stops at its first failing step and prints the step's index, the step itself, and the `expected` and `actual` values. Steps after the failure are not run.
|
|
345
|
+
|
|
346
|
+
A journey with production [evidence](#evidence) prints its tier, its rank in the run's selection, its rate and its failures over the usage window after its `PASS` line, then its mutants:
|
|
347
|
+
|
|
348
|
+
```
|
|
349
|
+
PASS member assigns an open ticket (5 steps, 2100ms) common #2 · 13.7/day · 14 failed (3m) · 11/12 mutants
|
|
350
|
+
PASS member reopens a ticket (4 steps, 1650ms) unranked
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
`unranked` marks a journey with no counts for its current steps yet. A run without `--tier` never builds config to rank journeys, so a journey that clicks a data value (a grid row by its name) also shows `unranked` there; `--tier common`, `wide` or `edge` ranks it. With fewer than 100 journey matches in the usage window there is no ranking, and the line shows the rate and failures with no tier or rank. Persons and organisations are left out, since they do not add up across months; [`lowdefy journeys usage`](/cli#journeys-usage) shows them month by month. The mutants part is left out when there is no mutation report. `FAIL` lines carry no evidence.
|
|
354
|
+
|
|
355
|
+
A journey with `deprecated: true` is never run, even when named by path or `--filter`. It is listed with its recent rate, and the closing line counts it:
|
|
356
|
+
|
|
357
|
+
```
|
|
358
|
+
SKIP deprecated member prints a ticket 0.4/day (3m)
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
### Options
|
|
362
|
+
|
|
363
|
+
- `[paths...]`: Journey files, directories or globs to run instead of the whole suite, anywhere under the config directory, including the candidates in `tests/journeys/_candidates/`. A directory is read with all its sub-folders: `lowdefy test tests/journeys/_candidates/variants` runs every variant. A path containing `*`, `?` or `[` (including `**`) is a glob the CLI expands itself, so quote it: `lowdefy test 'tests/journeys/review/*.yaml'`. A glob that matches nothing is refused, like a path that does not exist. `--tag` and `--filter` apply on top.
|
|
364
|
+
- `--tag <tag>`: Only run journeys whose `tags` include the tag. Repeat it to run the journeys carrying any of the tags: `lowdefy test --tag smoke --tag review`.
|
|
365
|
+
- `--filter <name>`: Only run journeys whose `name` contains the string (case-insensitive). `lowdefy test --filter control` runs every journey with "control" in its name. Repeat it to run the journeys matching any of the strings.
|
|
366
|
+
- `--tier <tier>`: Only run the journeys in this popularity tier of the selection: `common` (p50), `wide` (p80), `edge` (p95) or `full` (every journey, the default). See [Usage and tiers](#usage-and-tiers).
|
|
367
|
+
- `--usage-window <months>`: The calendar months recent use is ranked over, for `--tier` and the `PASS` line, such as `6m`. The default is `3m`.
|
|
368
|
+
- `--repeat <n>`: Run each journey `n` times in a row (1 to 10) and classify it, as described in [Replaying candidates](#replaying-candidates).
|
|
369
|
+
- `--lint`: Check the journeys for the [lint rules](#lint) and run nothing.
|
|
370
|
+
- `--journeys-directory <path>`: Read journeys from this directory instead of `tests/journeys/`, for journeys that need a server set up for them, such as [auth journeys](#the-database). A relative path is resolved from the current directory. The run fails when the directory holds no journeys.
|
|
371
|
+
- `--url <url>`: Run against a development server that is already running instead of starting one, for example `lowdefy test --url http://localhost:3000` while `lowdefy dev` is open in another terminal. This is the fastest way to iterate on a journey.
|
|
372
|
+
- `--port <port>`: The port to start the development server on. If it is in use the next free port is taken. The default is `3000`.
|
|
373
|
+
- `--config-directory`, `--dev-directory`, `--ref-resolver`, `--log-level`, `--disable-telemetry`: As for [`lowdefy dev`](/cli#dev).
|
|
374
|
+
|
|
375
|
+
### Exit codes
|
|
376
|
+
|
|
377
|
+
| Exit code | Meaning |
|
|
378
|
+
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
379
|
+
| `0` | Every journey passed, or `tests/journeys/` has no journeys (a note is printed). |
|
|
380
|
+
| `1` | At least one journey failed, a journey file was invalid, a path, glob, `--tag` or `--filter` matched no journey, or a `--journeys-directory` held no journeys. With `--repeat`, a journey was `FLAKY` or `FAIL`; with `--lint`, a lint error was found; with `--tier`, the selection had too little production use to cut a tier. |
|
|
381
|
+
|
|
382
|
+
A journey file that is not valid YAML, or does not match the journey format (a missing `name`, a step with two keys, an unknown step key, a step the grammar refuses, named by its index) is reported as a failed journey with the validation message and the file path. It never aborts the run, so one broken file cannot hide the results of the others.
|
|
383
|
+
|
|
384
|
+
## Hardening journeys
|
|
385
|
+
|
|
386
|
+
A journey that passes does not prove much on its own: it may never check what its clicks did, it may pass only some of the time, and the edge cases around it are rarely written. These commands prove a journey is worth keeping.
|
|
387
|
+
|
|
388
|
+
### Replaying candidates
|
|
389
|
+
|
|
390
|
+
`lowdefy test --repeat 3 <paths>` runs each journey three times and classifies it:
|
|
391
|
+
|
|
392
|
+
```
|
|
393
|
+
PASS member assigns an open ticket (5 steps, 3/3, 2.1s each)
|
|
394
|
+
FLAKY owner closes a ticket (2/3 passed) run 2 failed at step 4 (click "close_submit"): ...
|
|
395
|
+
FAIL admin bulk-imports contacts (0/3) step 2 (click "import"): ... — fails every run: a finding, not a test to fix by retrying
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
A journey is kept, and a candidate moves into `tests/journeys/`, only after `--repeat 3` gives `PASS`. A `FLAKY` journey has a cause to fix, usually a missing `wait: { request }`, data that differs between runs, or a target that matches two elements; never add `wait: { ms }`. A `FAIL` is a finding: it breaks every time, so either the app has a bug or its behaviour changed.
|
|
399
|
+
|
|
400
|
+
Each run records what the journey exercised (the pages, requests, endpoints, events and blocks it touched) and a hash of the config it ran on in `.lowdefy/test/exercised.json`, which the lints, `journeys harden` and `journeys variants` read. Only a run of the whole suite, once, records as the suite's journey run.
|
|
401
|
+
|
|
402
|
+
### Lint
|
|
403
|
+
|
|
404
|
+
`lowdefy test --lint [paths...]` checks journeys without running them. It reads data sets from their files and needs no server. It prints one line per problem and exits `1` on any error:
|
|
405
|
+
|
|
406
|
+
```
|
|
407
|
+
L2 member assigns an open ticket step 3 (click "assign_submit") is not followed by an expect or wait: { request } before step 4.
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
An **assertion step** is any `expect`, or a `wait: { request }`.
|
|
411
|
+
|
|
412
|
+
| Rule | Checks | Severity |
|
|
413
|
+
| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
|
|
414
|
+
| L1 | No placeholders: no `from: shape`, and no `fill` or `select` with `value: null`. | error |
|
|
415
|
+
| L2 | A `click`, `open` or `press` whose target ran a Lowdefy event (in the journey's newest measured run), and every `goto` and `back`, is followed by an assertion step before the next action or input. A click that ran no event (a tab header) is exempt, as are `fill` and `select`. Unmeasured, every action is checked. | error |
|
|
416
|
+
| L3 | No `wait: { ms }`. | error |
|
|
417
|
+
| L4 | A journey whose newest run called a request or endpoint that writes declares `data:`. Unmeasured, it is a warning to run the journey once. | error |
|
|
418
|
+
| L5 | `user` names a user from the journey's data set (or a list of them, each checked), or is `none`. An inline user object, or no `user`, is a warning until the journey moves onto a data set. | warning |
|
|
419
|
+
| L6 | The last step is an assertion step. | error |
|
|
420
|
+
| L7 | An `as` step names the person who is already acting (`as: main` while still `main`, or the same name twice running), so it switches to nobody. | warning |
|
|
421
|
+
|
|
422
|
+
A journey written from a [session log](#session-logs), with event-less clicks and a final `wait: { request }`, lints without errors.
|
|
423
|
+
|
|
424
|
+
### Mutants: `lowdefy journeys harden`
|
|
425
|
+
|
|
426
|
+
`lowdefy journeys harden` measures whether your journeys fail when the feature they walk breaks. It breaks the config on purpose, one small change at a time and only in the journeys' own browsers, while you keep working in the same development server: your own tabs keep seeing the unchanged app.
|
|
427
|
+
|
|
428
|
+
Only journeys with a [data set](/journey-data-sets) (`data:`) are hardened: mutant runs write through the app's connections from parallel workers, and without a data set that is your own database. A selected journey with no `data:` is left out with an error naming it; when none is left, harden exits `1`.
|
|
429
|
+
|
|
430
|
+
1. Each selected journey runs once, unchanged. One that fails is left out, with a note to replay it.
|
|
431
|
+
2. The development server lists the mutants on what those runs exercised: a dropped action, a skipped validation, a flipped `visible` or `disabled`, swapped `_if` branches, a dropped payload key, a `Link` sent to `404`, a dropped block, a dropped endpoint step. A layout or template copied into several pages is one mutant, listed under the first page by id.
|
|
432
|
+
3. Each mutant runs against every journey whose path reached it, on any page it is copied onto, and is applied to the copy that journey reaches: a journey that fails with the mutant applied **kills** it; one that passes lets it **survive**. A failure where the mutant never reached the run says nothing about the mutant: it counts as an error and runs once more.
|
|
433
|
+
|
|
434
|
+
```
|
|
435
|
+
page tickets:
|
|
436
|
+
SURVIVED drop-action pages/tickets.yaml:88 drop-action SetState "set_status" (2 of 3) from assign_submit.onClick
|
|
437
|
+
ran: member assigns an open ticket (passed)
|
|
438
|
+
SURVIVED drop-block pages/tickets.yaml:141 drop-block Alert "assign_success" from tickets
|
|
439
|
+
ran: member assigns an open ticket (passed)
|
|
440
|
+
SCORE 12/14 killed, 3 unique member assigns an open ticket (tests/journeys/tickets.yaml)
|
|
441
|
+
42 mutants run · 38 killed · 4 survived · 0 unapplied · 0 errors · sampled 42 of 42 (seed 0) · 19 not exercised · 0 rebuilds · 3m 10s
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
While it runs, harden prints a line as the baseline runs (`Baseline 37/412 journeys (2 failed)`) and as mutants finish (`Mutants 12/45 run (3 survived)`), every few journeys or about every 15 seconds, so a long run is not silent.
|
|
445
|
+
|
|
446
|
+
A survivor is a change no journey noticed, with the source line it changed: add the assertion that would catch it (`expect.text` of the message, `expect.state`, `expect.visible`, `expect.hidden` or `expect.calls`) after the step that exercises it, and confirm with `lowdefy journeys harden --mutant <id>`. A mutant listed as **unapplied** never reached the journey's browser: that is a defect in harden, not a gap in the journey. Mutants no journey exercised are counted, not run.
|
|
447
|
+
|
|
448
|
+
The report is written to `.lowdefy/test/mutation.json`: each mutant with the journeys that ran it, the suite's score, and each journey's `killed` out of `total` and its `unique` kills (mutants no other journey kills). It is merged per journey: a run over some journeys replaces their scores and keeps every other journey's from earlier runs, until that journey is removed. An earlier mutant is kept only while the development server still lists it, so a removed or edited block, action or request stops counting, and each journey's score is recounted without it. `--mutant` runs write nothing. Survivors are findings, so the exit code is `0`; it is `1` only when the run could not finish. Editing the config during a run is fine: harden re-lists the mutants, keeps the verdicts no changed file touched and runs the rest again. The third change in one run stops it. harden never writes or deletes a journey.
|
|
449
|
+
|
|
450
|
+
- `[paths...]`, `--filter <name>`: The journeys to harden, as for `lowdefy test` (one `--filter`, and no `--tag`).
|
|
451
|
+
- `--page <pageId...>`: Only mutants on these pages, and the endpoint mutants a journey touching them called. The baseline run also skips journeys that cannot reach these pages: ones whose last measured run in `.lowdefy/test/exercised.json` touched none of them and whose `pageId` and `goto` steps name none. A journey with no record, a changed journey, or a record measured on a different config is always run, so run `lowdefy test` first to get the saving. A record belongs to a config by its content (a hash of the built pages, requests, endpoints and connections), not by when the dev server started, so `lowdefy test` and `lowdefy journeys harden`, which each start their own server, agree as long as the config is unchanged in between. The log says how many journeys were left out, and when journeys were kept whole it says why: no record, the journey changed, or the config changed since the last `lowdefy test`.
|
|
452
|
+
- `--operators <list>`: Only these operators, comma separated: `drop-action`, `skip-validate`, `flip-visible`, `swap-if`, `drop-payload`, `retarget-link`, `drop-block`, `drop-step`.
|
|
453
|
+
- `--max <n>`: Run at most `n` mutants, sampled the same way on every machine. The default is 200; `0` runs every mutant.
|
|
454
|
+
- `--seed <n>`: The sample's seed. Another seed draws another sample, so repeated runs can cover the rest. The default is `0`, recorded in the report.
|
|
455
|
+
- `--workers <n>`: How many journey runs at once, 1 to 16. The default is `4`. A journey never runs beside itself, and one that reads email runs alone.
|
|
456
|
+
- `--list`: Print the sampled mutants per operator and page and a time estimate, and run nothing.
|
|
457
|
+
- `--mutant <id>`: Run only this mutant, against the journeys on its path. The report file is left as it is.
|
|
458
|
+
- `--json`: Print the report as JSON.
|
|
459
|
+
- `--url`, `--port`, `--config-directory`, `--dev-directory`, `--ref-resolver`, `--log-level`, `--disable-telemetry`: As for `lowdefy test`.
|
|
460
|
+
|
|
461
|
+
### Variants: `lowdefy journeys variants`
|
|
462
|
+
|
|
463
|
+
`lowdefy journeys variants <file>` writes edge-case candidates of a journey to `tests/journeys/_candidates/variants/<source>/<journey>-<kind>.yaml` and replays each three times. `<source>` is the journey file's path under `tests/journeys/` (or under the config directory, for a file outside it) without its extension, so two files with one name in different folders never share a folder. `<journey>` is the journey's name in lower case with each run of other characters replaced by `-`, so the journeys of one file never share a file. A kind with several variants numbers them: `<journey>-negative-1.yaml`, `<journey>-negative-2.yaml`. The same journey always gives the same files.
|
|
464
|
+
|
|
465
|
+
| Kind | The variant | Passes when |
|
|
466
|
+
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
467
|
+
| `role` | Granted: for each other role set (the page's role matrix from `journeys coverage`, else each data set user's role set), the same steps as a data set user with exactly that set. Refused: when the page has `auth.roles` and a data set user holds none, that user opens the page. | Granted: that role can do the flow. Refused: the user lands on `/404` and does not see the journey's first block |
|
|
468
|
+
| `tenant` | A user of another organization (with the same roles when one exists) walks to the first step that uses a fixture value, waits for their own organization's row there, then checks each of this organization's fixture values is hidden. | The other organization sees its own rows and none of this one's |
|
|
469
|
+
| `empty` | The journey's user, by name, on the `--empty-data` data set, up to the first step that uses data, then that step's block is visible. | The page renders its empty state |
|
|
470
|
+
| `volume` | The same steps, as the journey's user by name, on the `--volume-data` data set. | The flow completes within its timeouts |
|
|
471
|
+
| `negative` | For each `fill` before the submit click on a block with `required` or a `validate` rule: the field left empty (or a placeholder to fill, for a `validate` rule), then the submit. | The message shows and nothing is sent |
|
|
472
|
+
| `interrupt` | A full reload of the start page just before the submit click, then the flow again. | The flow writes exactly once |
|
|
473
|
+
| `double-submit` | The submit click as a double click (`count: 2`). | The write happens once |
|
|
474
|
+
|
|
475
|
+
The **submit click** is the last `click` followed by a `wait: { request }` for a request that writes. Each variant is named `<journey> — <kind>: <detail>` and carries a `variant` key naming the journey it came from:
|
|
476
|
+
|
|
477
|
+
```yaml
|
|
478
|
+
name: 'member assigns an open ticket — double-submit: double click "assign_submit"'
|
|
479
|
+
pageId: tickets
|
|
480
|
+
variant:
|
|
481
|
+
of: member assigns an open ticket
|
|
482
|
+
kind: double-submit
|
|
483
|
+
detail: double click "assign_submit"
|
|
484
|
+
steps:
|
|
485
|
+
# ...
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
Each file starts with a header line naming the journey file it came from and holding a hash of what was generated. A file belongs to that journey file and the journey its `variant.of` names. A rerun rewrites a file it owns that is left as generated, keeps one you edited (a filled-in placeholder, a fix after a flaky replay) and says so, and removes an unedited file it owns that the journey no longer gives. A file another journey owns (two journey names that give one slug) is left alone and reported as `CONFLICT`: rename one of the journeys. A variant that passes is a candidate to keep; one that fails is a finding, a bug or a behaviour to assert as expected; a flaky one has a cause to fix. A granted `role` variant that passes is kept by adding its user to the original journey's `user` list ([one journey, several users](/journey-data-sets#one-journey-several-users)) and deleting the variant file, so the steps stay in one place; its first comment names the list. A refused `role` variant expects a different outcome and is kept as a journey of its own. Variants of a journey with a list of users keep the list, and replay once per user. The `role` and `tenant` kinds read the journey's data set file; a variant whose input is missing (no user with a role set, no fixture for the other organization, no `--empty-data`) is listed as skipped with what to add.
|
|
489
|
+
|
|
490
|
+
- `--name <journey>`: The journey to vary, when the file holds several.
|
|
491
|
+
- `--kinds <list>`: Only these kinds, comma separated.
|
|
492
|
+
- `--empty-data <name>`: The data set the `empty` variant runs on. It needs a user with the journey user's name.
|
|
493
|
+
- `--volume-data <name>`: The data set the `volume` variant runs on. It needs a user with the journey user's name.
|
|
494
|
+
- `--no-run`: Write the variants without replaying them.
|
|
495
|
+
- `--url`, `--port`, `--config-directory`, `--dev-directory`, `--log-level`, `--disable-telemetry`: As for `lowdefy test`.
|
|
496
|
+
|
|
497
|
+
### The `journeys-harden` skill
|
|
498
|
+
|
|
499
|
+
`lowdefy agent-setup` installs a `journeys-harden` skill that runs this section with you. It replays the journeys and candidates three times, lints them, sizes a `journeys harden` run with `--list` and asks before it starts, then takes each surviving mutant to you one at a time with the assertion that would catch it, confirming each with `--mutant <id>` and `lowdefy test --repeat 3`. It writes variants for the journeys you pick, refreshes `journeys evidence`, and reports journeys with no production backing (with their mutation kills), duplicate journeys and pages no journey reaches. It never deletes a journey or suggests deleting one, writes no step except an assertion you approved, and commits nothing.
|
|
500
|
+
|
|
501
|
+
## Journeys on autopilot
|
|
502
|
+
|
|
503
|
+
`lowdefy agent-setup` installs a `journeys-autopilot` skill for an app with no journeys, or with some. Start it once and leave it running: in Claude Code, `/loop /journeys-autopilot` (or a background session running `/journeys-autopilot`), from a shell that holds your app's secrets: exported variables, or a wrapper such as `infisical run -- claude`. It never reads or copies your `.env`; a variable the app needs and the shell lacks stops it with the variable named. It is one agent, with no sub-agents, one dev server and one test run at a time, in its own git worktree and branch, so your checkout and dev server are untouched.
|
|
504
|
+
|
|
505
|
+
The first cycle writes a [data set](/journey-data-sets) with made-up fixtures and a test user for each role, maps every page by role and every control that writes data, and records them in a ledger, `tests/autopilot.yaml`, committed in your app. When several `package.json` scripts run `lowdefy dev` and [`cli.devScript`](/cli#configuration) is not set, it sets `cli.devScript` to the one whose name marks it the development script, never one that names production or staging, and says why in the pull request; when no single script qualifies, it stops and names them. When your app already has a data set and journeys, it extends that data set (`extends: [<name>]`) without editing it, and enters each of your journeys in the ledger as covered: it never rewrites, moves or duplicates them. Each later cycle reads the ledger and takes the next gaps by value: refusals and permissions, then writes, then the pages people use most, then edge cases. Signed-out visits are skipped with a reason, since data set journeys run as signed-in users. A journey is kept only after three clean runs in a row (`lowdefy test --repeat 3`) and must assert a value only the fixtures hold. A flaky journey moves to `tests/journeys/_quarantine/` with its cause; a journey that exposes an app bug moves to `tests/journeys/_findings/` as a finding. Neither is deleted, and no journey is ever weakened to pass. Once every item is covered, the cycles turn to `lowdefy journeys harden` and `variants`, and to `coverage` when production analytics are pulled.
|
|
506
|
+
|
|
507
|
+
Each cycle ends with the ledger committed and its servers stopped, so you can interrupt it any time. Its work goes to one branch, `journeys-autopilot/main`, and one open pull request of journeys with a plain-language summary. Each cycle resumes from the pushed branch, so a run on another machine picks up where the last stopped. At about 20 journeys the pull request stops growing until you merge it; the next cycle then starts the branch afresh. Once the journeys that found an obvious, local app bug have merged, it fixes the bug in a separate small pull request that moves the failing journey into the suite, turning green; it writes up any other finding for you. It never merges, never pushes to the default branch, and never runs against production or a shared database. When the environment is broken (a missing variable, no clear dev script, a dev mock user in your config, a closed pull request, a rejected push) it records why in the ledger and stops.
|
|
508
|
+
|
|
509
|
+
## Dev recordings
|
|
510
|
+
|
|
511
|
+
`lowdefy dev` records how you use your app in the browser, so an agent can read what you just tried and write journeys for it. Recording is on by default and stays on your machine:
|
|
512
|
+
|
|
513
|
+
- Every tab you open on the development server records its clicks, typed values, key presses and page views, joined to the events they ran and the state those events wrote, to `.lowdefy/traces/dev/<date>/<session>.jsonl` in your config directory. `.lowdefy/` is not committed.
|
|
514
|
+
- Values you type are kept, so a recording can include data your app shows you. Password fields are never recorded: their values, and the state they write, are replaced with `null`.
|
|
515
|
+
- Recordings are kept for 7 days or 200 MB, whichever comes first. The development server deletes older ones at start and then every hour.
|
|
516
|
+
- Set `LOWDEFY_DEV_RECORD=false` in your shell or in the app's `.env` to turn recording off. Old recordings are still pruned.
|
|
517
|
+
|
|
518
|
+
`lowdefy test` records too, as `journey` traces under `.lowdefy/traces/journey/`, but only when it runs the whole suite once: no journey paths, no `--tag`, no `--filter`, no `--tier` but `full`, and only the first of `--repeat` runs. Skipping `deprecated: true` journeys still counts as the whole suite. Those traces show what the suite actually drives. Screenshots, state inspection and other agent tools never record.
|
|
519
|
+
|
|
520
|
+
### Session logs
|
|
521
|
+
|
|
522
|
+
`lowdefy journeys session` lists the recorded dev sessions, newest first, one line each: its id, when it ran, the pages it visited, how many interactions it holds and how many failed. Give it a session id and it prints that session as a log, one line per interaction with what the app did in response:
|
|
523
|
+
|
|
524
|
+
```
|
|
525
|
+
Session 20261003T140300Z-bbbbbb, 2026-10-03T14:03:00.000Z to 2026-10-03T14:21:40.000Z:
|
|
526
|
+
page ticket-new
|
|
527
|
+
fill title "Quarterly"
|
|
528
|
+
click save → Validate failed [priority]
|
|
529
|
+
fill priority "high"
|
|
530
|
+
click save → ran Validate, Link, request createTicket ok
|
|
531
|
+
page tickets
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
A failed attempt and its retry both show, values typed in dev show as typed, and ids the app generated show as values. The log asserts nothing: it is what you, or an agent, read to decide what a journey should prove, then write it and run it with `lowdefy test --repeat 3 <file>`. A `(config rebuilt)` line marks where a config edit reloaded the page.
|
|
535
|
+
|
|
536
|
+
- `--since <since>`: Only sessions with records at or after this time, as a duration back from now (`30m`, `2h`, `7d`) or an ISO date.
|
|
537
|
+
- `--source <dev|production>`: Where sessions are read from. `dev`, the default, is what the development server recorded; `production` reads the pulled cache, see [Production session logs](#production-session-logs).
|
|
538
|
+
- `--json`: Print the list or the log as JSON.
|
|
539
|
+
|
|
540
|
+
Agents connected to the development server read the same list and logs with the `lowdefy_journey_session` tool (optional `id` and `since`), which reads the app's own `.lowdefy/traces/dev/`.
|
|
541
|
+
|
|
542
|
+
`lowdefy agent-setup` installs a `journeys-from-dev` skill that uses these commands: it asks which session you meant, reads its log, decides what you were proving (a failed attempt and its fix are both worth a journey), writes the journeys with assertions on outcomes rather than on ids the app generated, runs each three times with `lowdefy test --repeat 3`, and leaves the ones that pass for you to keep.
|
|
543
|
+
|
|
544
|
+
## Production journeys
|
|
545
|
+
|
|
546
|
+
Journeys can be written from what your users do in production. Apps that send analytics with the [PostHog plugin](/PostHog) can pull them to your machine, read each session as a log, and see which journeys real use backs and what it does that no journey covers.
|
|
547
|
+
|
|
548
|
+
```
|
|
549
|
+
pnpx lowdefy@7 journeys pull posthog --since 30d
|
|
550
|
+
pnpx lowdefy@7 journeys session --source production --since 30d
|
|
551
|
+
pnpx lowdefy@7 journeys coverage --source production
|
|
552
|
+
pnpx lowdefy@7 journeys evidence --refresh
|
|
553
|
+
```
|
|
554
|
+
|
|
555
|
+
[`journeys pull posthog`](/cli#journeys-pull-posthog) needs `POSTHOG_PROJECT_ID`, `POSTHOG_API_HOST` and your own `POSTHOG_PERSONAL_API_KEY` with the Query Read scope. It writes one file per UTC day to `.lowdefy/traces/production/`. What it keeps:
|
|
556
|
+
|
|
557
|
+
- No value a user typed. PostHog never captures one, so a production session log's `fill` lines name the control only, and a journey written from it takes its values from your test data.
|
|
558
|
+
- Person and organisation ids hashed with a salt that never leaves your machine, so counts of people and organisations are the same on every machine while no raw id is stored.
|
|
559
|
+
- Page URLs with query parameter names only (`/tickets?id=&tab=`), never their values.
|
|
560
|
+
- The page each event happened on, by its page id, and the values of its [path](/page-paths) placeholders as `pathParams`. A page is never read from the URL path, so two records of one patterned page are two visits to it, and a journey that enters on a patterned page needs `pathParams`.
|
|
561
|
+
- No clicked text. The text of each clicked element is stored as a token, a hash under the same salt, so the same text groups and counts without being kept. Session logs, coverage and evidence turn a token back into text only when it is text from your app's config (pages, menus, i18n messages, block plugins' default messages, antd's own strings), collected with one full build of the app by the installed development server and cached until the config changes. Text that is not in the config, such as a customer's name in a grid cell, stays a token.
|
|
562
|
+
- No match index. A click on one of several controls with the same label, such as a grid's per-row Edit, carries no `nth`, so a step written from it as it stands targets every one of them. Narrow such a step by `blockId`, `row` or `containing`; a target that matches several controls fails the run rather than clicking the first. The `journeys-from-production` skill does this for you.
|
|
563
|
+
|
|
564
|
+
Pull, session logs and coverage read at most 30 days at a time: pick the window for the question, such as the last 30 days, the days since a deploy for a regression, or the days around a month-end for a periodic process. Evidence is not capped.
|
|
565
|
+
|
|
566
|
+
Upgrading from a version that stored clicked text: the first pull or production read deletes the day files pulled that way, any production candidates an earlier version wrote to `tests/journeys/_candidates/production/` and `.lowdefy/test/coverage.json`; the next pull and coverage write them again. Committed journeys are left as they are, so check any you promoted from production for text that came from your data rather than your config.
|
|
567
|
+
|
|
568
|
+
### Production session logs
|
|
569
|
+
|
|
570
|
+
`lowdefy journeys session --source production` reads the pulled window (`--since`, or `--from` and `--to`, at most 30 days) and works as it does for [dev sessions](#session-logs): without an id it lists the window's sessions, newest first; with one it prints that session as a log. Session ids are the analytics session ids the pulled cache holds (people and organizations in it are hashed). A production log shows the controls used and what the app did, never a typed value, and clicked text only when it is config text: a click on anything else reads `(text not in config)`. Rage and dead clicks are marked `(rage click)` and `(dead click)`. Production sees only the events that failed, so a line shows an outcome only when something failed.
|
|
571
|
+
|
|
572
|
+
```
|
|
573
|
+
page tickets
|
|
574
|
+
fill title
|
|
575
|
+
click save → Validate failed [priority]
|
|
576
|
+
click export (dead click)
|
|
577
|
+
```
|
|
578
|
+
|
|
579
|
+
Reading sessions one by one works until a window holds too many. From 100,000 rows in the window, [coverage](#coverage) and [usage](#usage-and-tiers) group sessions that do the same thing into flows and rank them by use instead; below that, they leave flows out and point to the session logs. `lowdefy journeys coverage --group` groups at any size, and `--no-group` never groups.
|
|
580
|
+
|
|
581
|
+
### Evidence
|
|
582
|
+
|
|
583
|
+
A committed journey can carry how much production use backs it, by calendar month:
|
|
584
|
+
|
|
585
|
+
```yaml
|
|
586
|
+
- name: member assigns an open ticket to a teammate
|
|
587
|
+
pageId: tickets
|
|
588
|
+
evidence:
|
|
589
|
+
production:
|
|
590
|
+
sequence: v1-3f9a12c0 # the flow these months were counted for
|
|
591
|
+
pageId: tickets
|
|
592
|
+
flow:
|
|
593
|
+
- tickets ["click","assign",null,"Assign"]
|
|
594
|
+
- tickets ["select","assignee",null,null]
|
|
595
|
+
months:
|
|
596
|
+
- { month: 2026-08, days: 31, sessions: 380, persons: 35, orgs: 9, failures: 12 }
|
|
597
|
+
- { month: 2026-09, days: 30, sessions: 412, persons: 37, orgs: 9, failures: 14 }
|
|
598
|
+
- { month: 2026-10, days: 3, sessions: 38, persons: 11, orgs: 5, failures: 1 }
|
|
599
|
+
mutation: { killed: 11, total: 12, unique: 2 }
|
|
600
|
+
refreshed: 2026-10-05
|
|
601
|
+
steps:
|
|
602
|
+
- click: { blockId: assign, text: Assign }
|
|
603
|
+
- select: { blockId: assignee, value: Ann }
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
A session backs a journey when it does the journey's interactions in the same order, other clicks in between allowed, starting on the journey's page. A journey's click text counts only when it is text from your app's config: any other text reads as no text, so a click on a grid cell by a customer's name is backed by that column's clicks exactly as one with no text, and no command tells whether production showed that value. Only [`lowdefy journeys evidence --refresh`](/cli#journeys-evidence) writes the key, and it changes nothing else in the file: comments, key order and quoting stay as they are. `lowdefy test` reads it to print the PASS line and validates it strictly, so a typo in a hand edit fails before the browser opens. A `dev` subkey is refused, with a message to delete it. A committed `mutation` subkey keeps its value when its source is not on your machine; it is filled from a hardening run's report in `.lowdefy/test/mutation.json` when there is one.
|
|
607
|
+
|
|
608
|
+
Each month is a UTC calendar month. `days` is how many of its final days the refresh read: the pull re-pulls today and yesterday for late events, so they count only once a later pull marks them final. `sessions` counts the backing sessions that started in that month, so a session that crosses midnight counts once; `failures` counts those that hit a failed event; `persons` and `orgs` are distinct within the month and do not add up across months. A month read with no backing session is written with `sessions: 0`, and a month never pulled is missing.
|
|
609
|
+
|
|
610
|
+
Counts build up across pulls and machines without double counting. A refresh reads every final day in the cache, gaps and all, and rewrites a month only when the cache holds more final days of it than the committed entry. Refreshing twice changes nothing, two machines holding the same days never overwrite each other, a colleague's fuller pull wins, and a laptop that never pulled a month, or pruned it, leaves that month as committed.
|
|
611
|
+
|
|
612
|
+
Each pulled day records the filters it was pulled with: the PostHog project, `--environment` and `--include-test-accounts`. Days pulled with different filters count different people, so `journeys evidence`, `journeys coverage` and `journeys usage` refuse a cache that mixes them, naming each set of filters with its days, and say to pull them again with one set (`lowdefy journeys pull posthog --refetch`). A pull whose filters differ from days already in the cache warns which days need pulling again.
|
|
613
|
+
|
|
614
|
+
`flow` is what the journey's steps are matched on: one `<page> <step>` line per click, select, fill, press, back or open, with the block, grid column and clicked text (config text only, as above, so a click by a customer's name is the same flow as one with no text), and `sequence` names it. Waits, other expectations, typed or picked values, rows and `nth` leave it unchanged. When an edit changes it (a click's text or block, the order, a `goto`, or an `expect.url` path that moves later steps to another page), the next refresh moves the counted months to a deprecated flow and counts the new flow from the cache:
|
|
615
|
+
|
|
616
|
+
```yaml
|
|
617
|
+
production:
|
|
618
|
+
sequence: v1-3f9a12c0
|
|
619
|
+
# …
|
|
620
|
+
deprecated:
|
|
621
|
+
- sequence: v1-91be04d7
|
|
622
|
+
pageId: tickets
|
|
623
|
+
flow:
|
|
624
|
+
- tickets ["click","assign_button",null,"Assign"]
|
|
625
|
+
replaced: 2026-10-05
|
|
626
|
+
months:
|
|
627
|
+
- { month: 2026-09, days: 30, sessions: 40, persons: 12, orgs: 4, failures: 0 }
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
A deprecated flow is never run, and every refresh keeps counting it, so you can see whether users still follow the old way. A renamed label keeps its history: a refresh reads a clicked-text token as today's config text or as any click text a committed flow already holds, so after `Assign` becomes `Allocate` the old flow's clicks still read as `Assign` and a fuller pull does not recount its months to 0. Undo the edit and it becomes live again with its months. No command deletes one; delete it by hand once it shows no use. A `production` block in the older window shape (`sessions`, `share`, `window`) still validates, and the next refresh replaces it with months.
|
|
631
|
+
|
|
632
|
+
A journey can also be retired as a whole with `deprecated: true` at its top level, while you watch its flow drain from production: `lowdefy test` skips it in every run, refresh keeps counting it, and [coverage](#coverage) leaves it out, since a journey that never runs covers nothing. Remove the flag to run it again.
|
|
633
|
+
|
|
634
|
+
No command removes a journey for lack of production use. Three months cannot see yearly work, and a journey that is the only one to catch a mutant matters whatever its traffic. `journeys evidence` lists the journeys nothing backs over the last 3 months, beside their mutation numbers, and leaves the decision to you.
|
|
635
|
+
|
|
636
|
+
### Usage and tiers
|
|
637
|
+
|
|
638
|
+
[`lowdefy journeys usage`](/cli#journeys-usage) ranks the journeys by how much real use leans on them now:
|
|
639
|
+
|
|
640
|
+
```
|
|
641
|
+
pnpx lowdefy@7 journeys usage
|
|
642
|
+
pnpx lowdefy@7 journeys usage tests/journeys/review --tier common
|
|
643
|
+
pnpx lowdefy@7 journeys usage --json
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
A journey's rate is its production sessions over the final days its months hold in the usage window: the last 3 calendar months (`--usage-window 3m`), ending at the newest month any selected journey has, so every journey is ranked over the same calendar and a flow that launched last month is not buried under years of history. The report lists the journeys by rate with their tier, their sessions and failures over the window and all time, one line per month with that month's people and organisations, and their deprecated flows with their recent use. Below that come the production flows no journey covers, from the coverage report, ranked by their sessions in coverage's window, when coverage grouped them; otherwise the report says why it lists none.
|
|
647
|
+
|
|
648
|
+
A tier is a cut through the selected journeys, ranked by rate, most first:
|
|
649
|
+
|
|
650
|
+
| Tier | Cut | Reads as |
|
|
651
|
+
| -------- | --- | ------------------------------------- |
|
|
652
|
+
| `common` | p50 | the happy paths |
|
|
653
|
+
| `wide` | p80 | the usual variations |
|
|
654
|
+
| `edge` | p95 | the edge cases real users still reach |
|
|
655
|
+
| `full` | p0 | every journey |
|
|
656
|
+
|
|
657
|
+
Tier pX holds the shortest run of journeys, from the top, whose summed rates reach X% of the total. Tiers nest, and journeys with equal rates are never split across a boundary. Tiers are cut over the selection after paths, `--tag` and `--filter`, so `tests/journeys/review --tier common` is the happy paths of the review area. A session counts for every journey it backs, so a tier's share is a share of journey matches, not of sessions. A journey edited since the last refresh, or never refreshed, has no counts for its current flow: it is `unranked` and in every tier, since a new or changed journey is what a change needs tested. A `deprecated: true` journey is in no tier. With fewer than 100 journey matches in the window, tiers are noise, and every tier but `full` is refused.
|
|
658
|
+
|
|
659
|
+
### Coverage
|
|
660
|
+
|
|
661
|
+
[`lowdefy journeys coverage --source production`](/cli#journeys-coverage) reports five measures, each as covered out of total with the uncovered items ranked by use:
|
|
662
|
+
|
|
663
|
+
| Measure | Counts | Covered when |
|
|
664
|
+
| ----------- | ----------------------------------------------------------------- | --------------------------------------------------------------------- |
|
|
665
|
+
| Interaction | each interaction in production sessions, by how often it happened | a journey does the same interaction on the same page |
|
|
666
|
+
| Flow | production sessions, when grouped (100,000 rows or `--group`) | a journey is backed by the session |
|
|
667
|
+
| Failure | distinct failed events (page, block, event, invalid fields) | a passing journey produced the same failed event (see below) |
|
|
668
|
+
| Frustration | rage- and dead-clicked blocks | a journey clicks the block and asserts with an `expect` right after |
|
|
669
|
+
| Role | (page, role set) pairs seen in production | a journey visiting the page runs as a user with exactly that role set |
|
|
670
|
+
|
|
671
|
+
The flow measure groups sessions into flows only when the window holds 100,000 rows or more, or with `--group`; otherwise it is left out, the summary says why, and you read the sessions with [`lowdefy journeys session --source production`](#production-session-logs). `--no-group` leaves it out at any size. The other four measures are always computed. Journeys with `deprecated: true` are left out of every measure: they are never run, so a flow only a deprecated journey walks reads as uncovered. A journey runs as its inline user's role set, as each data set user its `user` names (read from its data set file), or signed out with no user; `user: none` signs in through the app, so it counts for no role set. Coverage has no tiers: its flows are counted over the mining window of at most 30 days, not by month, so they have no rate to rank by.
|
|
672
|
+
|
|
673
|
+
Coverage also reads the newest full test run that the development server recorded (a plain `lowdefy test`, or `lowdefy_run_tests` with no paths, tags, filter or tier). The interaction measure then adds a measured share beside the static one: the production interactions that run actually drove. Failure coverage becomes measured: a failure counts as covered when a journey that passed in that run produced the same failed event, because a journey that reaches a failure and still passes asserts it. The test runner keeps which journeys passed in `.lowdefy/test/run.json`. Without a recorded run, failure coverage is reported as reached: a journey does the interaction that failed, which does not show it checks the outcome.
|
|
674
|
+
|
|
675
|
+
It writes whether flows were grouped (`flowGrouping`), the measures, a production profile (the top flows per entry page when grouped, failure paths, frustrated blocks, role sets per page, entry pages, and per block and column the clicks, distinct clicked-text tokens and most-clicked tokens) and each journey's interactions to `.lowdefy/test/coverage.json`, which is rewritten on every run and not committed. With a mutation report, the suite's mutation score is added as a sixth number.
|
|
676
|
+
|
|
677
|
+
`lowdefy agent-setup` installs a `journeys-from-production` skill for your coding agent. The agent picks the window for the question and says why, pulls and measures, then reads each session's log (or, from 100,000 rows, the grouped flows) with the page's config, requests, actions and plugin code to decide what the person was doing and whether it deserves a journey: failures first, then routines that write data, move money, change access or end a process, then the rest by count. It writes each journey from what the session showed, filling typed values from your data set's fixtures and labels from your config, proves each with `lowdefy test --repeat 3`, refreshes evidence with `lowdefy journeys usage --json` read before and after, and reports what it wrote, what it skipped and why, the findings, the deprecated flows users still follow, and how the tiers moved. It reads tokens, never production text: it does not read the trace salt or your `.env`, and never queries PostHog directly. It asks you only about findings and dead clicks, commits nothing, and never deletes a journey or suggests deleting one. It never deletes a deprecated flow either; it may suggest you delete one that shows no use.
|