@ihyeon/flow-qa 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +5 -1
- package/README.md +5 -1
- package/dist/src/cli.js +10 -3
- package/dist/src/workspace.js +50 -2
- package/docs/releases.md +35 -1
- package/docs/ui.md +1 -1
- package/package.json +1 -1
package/README.ko.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Flow QA
|
|
2
2
|
|
|
3
|
+
[](https://www.npmjs.com/package/@ihyeon/flow-qa) · [npm 패키지: @ihyeon/flow-qa](https://www.npmjs.com/package/@ihyeon/flow-qa)
|
|
4
|
+
|
|
3
5
|
[English](https://github.com/hyeonse-swing/flow-qa/blob/main/README.md) · **한국어**
|
|
4
6
|
|
|
5
7
|
사용자 플로우를 검토 가능한 테스트 케이스로 바꾸고, 실제 브라우저 실행 결과를 증거가 연결된 QA 리포트로 남기는 로컬 도구입니다. 플로우의 기대 결과가 모호하면 추측해 통과시키지 않고 질문과 차단 상태로 남깁니다.
|
|
@@ -14,7 +16,7 @@ CLI와 `127.0.0.1` 전용 웹 UI를 제공합니다. UI는 영어로 시작하
|
|
|
14
16
|
Node.js 24 이상이 필요합니다. 브라우저는 첫 실행 전에 설치합니다.
|
|
15
17
|
|
|
16
18
|
```sh
|
|
17
|
-
npm install -g @ihyeon/flow-qa@0.1.
|
|
19
|
+
npm install -g @ihyeon/flow-qa@0.1.3
|
|
18
20
|
flow-qa browser install
|
|
19
21
|
flow-qa ui --demo
|
|
20
22
|
```
|
|
@@ -56,6 +58,8 @@ flow-qa plan flows/my-flow.md --workspace my-qa
|
|
|
56
58
|
|
|
57
59
|
인증이 필요하면 모델 설정에 `--key-env FLOW_QA_API_KEY`를 추가하고 실행 환경에 그 값을 제공하세요. 설정 파일에는 환경 변수 이름만 저장합니다. 플로우 문서는 지정한 설계 모델 서버로 전송됩니다. 브라우저의 자연어 조작 `act` 또는 모델 판정 `assertAgent`를 쓰려면 도구 호출·이미지 입력을 지원하는 **별도** `agent` 모델 설정이 필요합니다. 고정된 클릭·입력·검증 단계는 모델 없이 실행됩니다.
|
|
58
60
|
|
|
61
|
+
`flow-qa doctor --workspace my-qa`로 로컬 의존성과 모델 인증용 환경 변수의 설정 여부를 확인할 수 있습니다. 키 값은 출력하지 않고 모델 서버도 호출하지 않습니다. 환경 변수가 있어도 실제 인증 성공을 확인한 것은 아닙니다.
|
|
62
|
+
|
|
59
63
|
## 설계 원칙과 결과 해석
|
|
60
64
|
|
|
61
65
|
- 케이스는 요구사항 ID와 명시적인 assertion을 가집니다. `open`, `click`, `fill`, `act`, `assertVisible`, `assertText`, `assertUrl`, `assertAgent` 등 허용된 단계만 스키마로 검증해 컴파일합니다. 모델 응답의 임의 JavaScript나 shell 명령은 실행하지 않습니다.
|
package/README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Flow QA
|
|
2
2
|
|
|
3
|
+
[](https://www.npmjs.com/package/@ihyeon/flow-qa) · [npm package: @ihyeon/flow-qa](https://www.npmjs.com/package/@ihyeon/flow-qa)
|
|
4
|
+
|
|
3
5
|
**English** · [한국어](https://github.com/hyeonse-swing/flow-qa/blob/main/README.ko.md)
|
|
4
6
|
|
|
5
7
|
Flow QA turns a user flow into reviewable test cases, runs them in a real browser, and ties the results to evidence in a QA report. When an expected outcome is unclear, it keeps the question visible and blocks that case instead of guessing.
|
|
@@ -14,7 +16,7 @@ Flow QA is available as a CLI and a web UI bound to `127.0.0.1`. Provide a targe
|
|
|
14
16
|
Requires Node.js 24 or later. Install Chromium before the first run:
|
|
15
17
|
|
|
16
18
|
```sh
|
|
17
|
-
npm install -g @ihyeon/flow-qa@0.1.
|
|
19
|
+
npm install -g @ihyeon/flow-qa@0.1.3
|
|
18
20
|
flow-qa browser install
|
|
19
21
|
flow-qa ui --demo
|
|
20
22
|
```
|
|
@@ -56,6 +58,8 @@ flow-qa plan flows/my-flow.md --workspace my-qa
|
|
|
56
58
|
|
|
57
59
|
For authenticated models, add `--key-env FLOW_QA_API_KEY` to the model command and provide that environment variable to the process. The config stores the variable's name, not its value. The flow is sent to the planner endpoint you chose. Natural-language browser action (`act`) and model judgment (`assertAgent`) require a **separately configured** `agent` model that supports tools and images. Fixed clicks, inputs, and assertions run without a model.
|
|
58
60
|
|
|
61
|
+
`flow-qa doctor --workspace my-qa` checks local dependencies and whether configured model-key environment variables are present. It does not print key values or call the model server; a present key does not confirm remote authentication.
|
|
62
|
+
|
|
59
63
|
## Design choices and result interpretation
|
|
60
64
|
|
|
61
65
|
- Every case references requirement IDs and a runnable case needs an explicit assertion. The schema accepts a fixed vocabulary including `open`, `click`, `fill`, `act`, `assertVisible`, `assertText`, `assertUrl`, and `assertAgent`. It never executes arbitrary JavaScript or shell commands returned by a model.
|
package/dist/src/cli.js
CHANGED
|
@@ -7,7 +7,7 @@ import { parseArgs } from 'node:util';
|
|
|
7
7
|
import { chromium } from 'playwright';
|
|
8
8
|
import { localeFromEnv, localized, localizeProductMessage } from "./i18n.js";
|
|
9
9
|
import { configSchema, idSchema, modelSchema, targetSchema } from "./schema.js";
|
|
10
|
-
import {
|
|
10
|
+
import { initWorkspace, readConfig, readSuite, sha256, verifySuiteSource, writeConfig, writeJSON } from "./workspace.js";
|
|
11
11
|
import { buildPlanningPrompt, planFlow } from "./planner.js";
|
|
12
12
|
import { compileSuite } from "./compiler.js";
|
|
13
13
|
import { runSuite } from "./engine.js";
|
|
@@ -165,7 +165,7 @@ export async function main(args) {
|
|
|
165
165
|
const config = await readConfig(workspace);
|
|
166
166
|
const name = idSchema.parse(required(arg, localized(locale, 'Target name', '대상 이름'), locale));
|
|
167
167
|
config.targets[name] = targetSchema.parse({ url: required(values.url, '--url', locale) });
|
|
168
|
-
await
|
|
168
|
+
await writeConfig(workspace, config);
|
|
169
169
|
console.log(localized(locale, `Target ${name} connected: ${config.targets[name].url}`, `대상 ${name} 연결: ${config.targets[name].url}`));
|
|
170
170
|
return;
|
|
171
171
|
}
|
|
@@ -175,7 +175,7 @@ export async function main(args) {
|
|
|
175
175
|
const config = await readConfig(workspace);
|
|
176
176
|
const base = { baseURL: required(values.url, '--url', locale), model: required(values.model, '--model', locale), ...(values['key-env'] ? { apiKeyEnv: values['key-env'] } : {}) };
|
|
177
177
|
config[arg] = modelSchema.parse(base);
|
|
178
|
-
await
|
|
178
|
+
await writeConfig(workspace, config);
|
|
179
179
|
console.log(localized(locale, `${arg} model connection saved. No model call has been made. API key values are not saved in the configuration file.`, `${arg} 모델 연결 설정 저장. 실제 호출 전입니다. API 키 값은 설정 파일에 저장하지 않습니다.`));
|
|
180
180
|
return;
|
|
181
181
|
}
|
|
@@ -268,11 +268,18 @@ export async function main(args) {
|
|
|
268
268
|
}
|
|
269
269
|
if (command === 'doctor') {
|
|
270
270
|
const config = configSchema.parse(await readConfig(workspace));
|
|
271
|
+
const keyStatus = (apiKeyEnv) => !apiKeyEnv
|
|
272
|
+
? localized(locale, 'key environment variable not configured', '키 환경 변수 미지정')
|
|
273
|
+
: localized(locale, `${apiKeyEnv}: ${process.env[apiKeyEnv] ? 'set' : 'missing'}`, `${apiKeyEnv}: ${process.env[apiKeyEnv] ? '설정됨' : '누락'}`);
|
|
271
274
|
for (const pkg of ['e2e', '@e2e-dev/web', 'playwright']) {
|
|
272
275
|
console.log(`${pkg}: ${await installedVersion(pkg, locale)}`);
|
|
273
276
|
}
|
|
274
277
|
const hasChromium = await chromiumInstalled();
|
|
275
278
|
console.log(localized(locale, `Node: ${process.version}\nChromium: ${hasChromium ? 'installed' : 'missing (flow-qa browser install)'}\nTargets: ${Object.keys(config.targets).join(', ') || 'none'}\nPlanner model: ${config.planner ? 'configured' : 'not configured (prompt handoff available)'}\nAgent model: ${config.agent ? 'configured' : 'not configured (deterministic tests available)'}`, `Node: ${process.version}\nChromium: ${hasChromium ? '설치됨' : '미설치 (flow-qa browser install)'}\n대상: ${Object.keys(config.targets).join(', ') || '없음'}\n설계 모델: ${config.planner ? '연결 설정됨' : '미연결 (프롬프트 전달 가능)'}\n실행 모델: ${config.agent ? '연결 설정됨' : '미연결 (고정 조작 테스트 가능)'}`));
|
|
279
|
+
if (config.planner)
|
|
280
|
+
console.log(localized(locale, `Planner API key: ${keyStatus(config.planner.apiKeyEnv)}`, `설계 모델 API 키: ${keyStatus(config.planner.apiKeyEnv)}`));
|
|
281
|
+
if (config.agent)
|
|
282
|
+
console.log(localized(locale, `Agent API key: ${keyStatus(config.agent.apiKeyEnv)}`, `실행 모델 API 키: ${keyStatus(config.agent.apiKeyEnv)}`));
|
|
276
283
|
return;
|
|
277
284
|
}
|
|
278
285
|
throw new Error(localized(locale, `Unknown command: ${command}`, `알 수 없는 명령: ${command}`));
|
package/dist/src/workspace.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
import { mkdir, readFile, writeFile } from 'node:fs/promises';
|
|
1
|
+
import { lstat, mkdir, open, readFile, realpath, rename, unlink, writeFile } from 'node:fs/promises';
|
|
2
2
|
import { resolve, dirname } from 'node:path';
|
|
3
|
-
import { createHash } from 'node:crypto';
|
|
3
|
+
import { createHash, randomUUID } from 'node:crypto';
|
|
4
4
|
import { configSchema, suiteSchema } from "./schema.js";
|
|
5
5
|
export const CONFIG_FILE = 'flowqa.config.json';
|
|
6
6
|
export async function readConfig(workspace) {
|
|
@@ -20,6 +20,54 @@ export async function writeJSON(path, data, exclusive = false) {
|
|
|
20
20
|
await mkdir(dirname(path), { recursive: true });
|
|
21
21
|
await writeFile(path, `${JSON.stringify(data, null, 2)}\n`, { flag: exclusive ? 'wx' : 'w', mode: 0o600 });
|
|
22
22
|
}
|
|
23
|
+
export async function writeConfig(workspace, config) {
|
|
24
|
+
const content = `${JSON.stringify(configSchema.parse(config), null, 2)}\n`;
|
|
25
|
+
const alias = resolve(workspace, CONFIG_FILE);
|
|
26
|
+
let file;
|
|
27
|
+
try {
|
|
28
|
+
file = await realpath(alias);
|
|
29
|
+
}
|
|
30
|
+
catch (error) {
|
|
31
|
+
if (error.code !== 'ENOENT')
|
|
32
|
+
throw error;
|
|
33
|
+
try {
|
|
34
|
+
if ((await lstat(alias)).isSymbolicLink())
|
|
35
|
+
throw new Error('Configuration symlink target does not exist.');
|
|
36
|
+
}
|
|
37
|
+
catch (inspectError) {
|
|
38
|
+
if (inspectError.code !== 'ENOENT')
|
|
39
|
+
throw inspectError;
|
|
40
|
+
}
|
|
41
|
+
file = alias;
|
|
42
|
+
}
|
|
43
|
+
await mkdir(dirname(file), { recursive: true });
|
|
44
|
+
const temp = resolve(dirname(file), `.flowqa-config-${randomUUID()}.tmp`);
|
|
45
|
+
let created = false;
|
|
46
|
+
try {
|
|
47
|
+
const handle = await open(temp, 'wx', 0o600);
|
|
48
|
+
created = true;
|
|
49
|
+
try {
|
|
50
|
+
await handle.writeFile(content, 'utf8');
|
|
51
|
+
await handle.sync();
|
|
52
|
+
}
|
|
53
|
+
finally {
|
|
54
|
+
await handle.close();
|
|
55
|
+
}
|
|
56
|
+
await rename(temp, file);
|
|
57
|
+
}
|
|
58
|
+
catch (error) {
|
|
59
|
+
if (created) {
|
|
60
|
+
try {
|
|
61
|
+
await unlink(temp);
|
|
62
|
+
}
|
|
63
|
+
catch (cleanupError) {
|
|
64
|
+
if (cleanupError.code !== 'ENOENT')
|
|
65
|
+
throw new AggregateError([error, cleanupError], 'Could not save configuration or clean up its temporary file.');
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
throw error;
|
|
69
|
+
}
|
|
70
|
+
}
|
|
23
71
|
export function sha256(text) { return createHash('sha256').update(text).digest('hex'); }
|
|
24
72
|
export async function verifySuiteSource(suite, workspace) {
|
|
25
73
|
if (!suite.source)
|
package/docs/releases.md
CHANGED
|
@@ -2,6 +2,40 @@
|
|
|
2
2
|
|
|
3
3
|
The alpha entries below are historical release records. They document earlier package versions and checks.
|
|
4
4
|
|
|
5
|
+
## Automated npm and GitHub releases
|
|
6
|
+
|
|
7
|
+
A push to `main` publishes only when `package.json` changes to a new plain `X.Y.Z` version. Pull requests and pushes without a version change run verification without publishing. The package and lockfile versions must agree; the version must increase, and registry errors stop the release. Rerunning a successful publication skips the existing immutable npm version.
|
|
8
|
+
|
|
9
|
+
The `publish` job in [ci.yml](../.github/workflows/ci.yml) waits for the full verification matrix on Ubuntu and macOS, then publishes through npm trusted publishing (OIDC). It uses the `latest` tag, requires the registry's `gitHead` to match the workflow commit, and checks the published package with a fresh consumer installation and three real Chromium demo cases. The commit check also applies when retrying an already published version. No npm token is stored in GitHub secrets. After the published package passes its consumer smoke check, a separate job creates the matching GitHub tag and release at the exact verified commit. It preserves existing releases and rejects a tag pointing to another commit. Rerunning an older release does not replace a newer latest release.
|
|
10
|
+
|
|
11
|
+
For the next release, update the release notes and current installation examples, then bump the version before committing and pushing:
|
|
12
|
+
|
|
13
|
+
```sh
|
|
14
|
+
npm version patch --no-git-tag-version
|
|
15
|
+
npm run check
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Push the version change to `main` and check the **Verify release** workflow. A successful verification job alone does not confirm publication: the publish job and its registry smoke check must also pass. GitHub release completion is reported by the separate release job. Retry a transient failure by rerunning the same workflow; do not reuse a published version for different package contents.
|
|
19
|
+
|
|
20
|
+
The npm trusted publisher is configured for GitHub repository `hyeonse-swing/flow-qa`, workflow `ci.yml`, with direct publishing allowed. See [npm's trusted publishing documentation](https://docs.npmjs.com/trusted-publishers/) for account setup.
|
|
21
|
+
|
|
22
|
+
## 0.1.3
|
|
23
|
+
|
|
24
|
+
- `doctor` reports whether each configured planner or agent key environment variable is present, missing, or not configured. English and Korean diagnostics show no credential values and make no model requests. The command remains informational.
|
|
25
|
+
- CLI target and model configuration updates write a private temporary file, synchronize it, and replace the configuration only after the new JSON is complete. Existing initialization and exclusive case-file creation behavior are preserved. This protects against partially written configuration during process interruption; it does not serialize concurrent CLI updates or guarantee rename durability after power loss.
|
|
26
|
+
- CI runs the existing test, browser, UI, and package-install checks on Ubuntu and macOS before publishing. Successful npm publication and its consumer smoke check now also create the corresponding GitHub release.
|
|
27
|
+
|
|
28
|
+
- Package: [`@ihyeon/flow-qa@0.1.3`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.1.3)
|
|
29
|
+
- Release: [GitHub 0.1.3](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.1.3)
|
|
30
|
+
- Verification: [GitHub Actions](https://github.com/hyeonse-swing/flow-qa/actions/workflows/ci.yml)
|
|
31
|
+
|
|
32
|
+
## 0.1.2
|
|
33
|
+
|
|
34
|
+
Adds version-gated npm publishing from GitHub Actions and npm links in both README languages. Runtime behavior is unchanged from 0.1.1.
|
|
35
|
+
|
|
36
|
+
- Package: [`@ihyeon/flow-qa@0.1.2`](https://www.npmjs.com/package/@ihyeon/flow-qa/v/0.1.2)
|
|
37
|
+
- Verification and publication: [GitHub Actions](https://github.com/hyeonse-swing/flow-qa/actions/workflows/ci.yml)
|
|
38
|
+
|
|
5
39
|
## 0.1.1
|
|
6
40
|
|
|
7
41
|
This release uses plain semver and the npm `latest` dist-tag. It includes the runtime fixes documented in the historical entry below, with no additional runtime changes. The existing runtime passed 47 tests and the browser checks recorded there. Version 0.1.1 metadata, package contents, and a fresh consumer installation were also verified: the installed `flow-qa` command worked and all three Chromium demo cases passed without model calls.
|
|
@@ -10,7 +44,7 @@ This release uses plain semver and the npm `latest` dist-tag. It includes the ru
|
|
|
10
44
|
- Source and package: [GitHub release](https://github.com/hyeonse-swing/flow-qa/releases/tag/v0.1.1)
|
|
11
45
|
- Public checks: [GitHub Actions](https://github.com/hyeonse-swing/flow-qa/actions/workflows/ci.yml)
|
|
12
46
|
|
|
13
|
-
Use
|
|
47
|
+
Use the matching source revision when checking this historical package:
|
|
14
48
|
|
|
15
49
|
```sh
|
|
16
50
|
FLOWQA_PACKAGE_SPEC=@ihyeon/flow-qa@0.1.1 npm run smoke:package
|
package/docs/ui.md
CHANGED
|
@@ -7,7 +7,7 @@ Use the target app's URL and a user flow to review cases, run them, and inspect
|
|
|
7
7
|
Node.js 24 or later is required. Install Flow QA and start the local demo:
|
|
8
8
|
|
|
9
9
|
```sh
|
|
10
|
-
npm install -g @ihyeon/flow-qa@0.1.
|
|
10
|
+
npm install -g @ihyeon/flow-qa@0.1.3
|
|
11
11
|
flow-qa browser install
|
|
12
12
|
flow-qa ui --demo
|
|
13
13
|
```
|