qa-agent-kit 0.6.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- qa_agent_kit-0.6.0.dist-info/METADATA +157 -0
- qa_agent_kit-0.6.0.dist-info/RECORD +32 -0
- qa_agent_kit-0.6.0.dist-info/WHEEL +4 -0
- qa_agent_kit-0.6.0.dist-info/entry_points.txt +6 -0
- qa_agent_kit-0.6.0.dist-info/licenses/LICENSE +21 -0
- qakit/AGENTS.md +48 -0
- qakit/__init__.py +21 -0
- qakit/_cli.py +46 -0
- qakit/docs/README.md +14 -0
- qakit/docs/agents.md +40 -0
- qakit/docs/case-studies.md +146 -0
- qakit/docs/device-qa-protocol.md +426 -0
- qakit/docs/getting-started.md +100 -0
- qakit/docs/ko/device-qa-protocol.md +433 -0
- qakit/docs/manifest.md +96 -0
- qakit/docs/mutation-testing.md +69 -0
- qakit/docs/prose-qa.md +49 -0
- qakit/docs/pypi.md +40 -0
- qakit/docs/report.md +64 -0
- qakit/skills/README.md +51 -0
- qakit/skills/device-qa/SKILL.md +87 -0
- qakit/skills/prose-qa/SKILL.md +49 -0
- qakit/skills/qa-rigor/SKILL.md +105 -0
- qakit/tools/device_lock.py +151 -0
- qakit/tools/device_qa.py +838 -0
- qakit/tools/install_skills.py +120 -0
- qakit/tools/mutate.py +684 -0
- qakit/tools/prose_check.py +291 -0
- qakit/tools/recording_frames.swift +78 -0
- qakit/tools/release_checks.py +287 -0
- qakit/tools/screenshot_contrast.swift +42 -0
- qakit/tools/shots_gif.py +81 -0
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Getting started
|
|
2
|
+
|
|
3
|
+
QAKit has three parts. Use the ones your project needs.
|
|
4
|
+
|
|
5
|
+
| Part | Needs | Gives |
|
|
6
|
+
|---|---|---|
|
|
7
|
+
| `skills/qa-rigor` + `tools/mutate.py` | Python 3.11+, your test command | Test-strength measurement for any language with tests (Python and C-family syntax: Swift, JS/TS, Kotlin, Go, Java, C#) |
|
|
8
|
+
| `skills/device-qa` + `tools/device_qa.py` + `tools/release_checks.py` | macOS, Xcode 27+, a USB-attached iPhone with Developer Mode | Real-device QA runs under variants, screenshots, crash reports, static App Store checks |
|
|
9
|
+
| Swift package `QAKit` | Xcode 15+ (iOS 17+), a UI-test target | Evidence, accessibility audit, button sweeps, system alert handling inside XCUITest |
|
|
10
|
+
|
|
11
|
+
## 1. Get the repo
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
git submodule add https://github.com/younatics/QAKit.git vendor/qakit # or clone it anywhere
|
|
15
|
+
python3 vendor/qakit/tools/device_qa.py --help
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Pin a release tag (`git -C vendor/qakit checkout v0.1.0`) and update on purpose; `CHANGELOG.md` lists what each tag
|
|
19
|
+
changes.
|
|
20
|
+
|
|
21
|
+
## 2. Mutation testing (any project)
|
|
22
|
+
|
|
23
|
+
Write a config next to your tests:
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{"name": "api", "root": ".", "sources": ["src/**/*.py"], "exclude": ["src/**/migrations/*"],
|
|
27
|
+
"test": "{orig}/.venv/bin/python -m pytest -x -q -p no:cacheprovider", "jobs": 3}
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
python3 vendor/qakit/tools/mutate.py --config mutation.json --list # what would be mutated
|
|
32
|
+
python3 vendor/qakit/tools/mutate.py --config mutation.json --changed-since origin/main
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Exit 0 means every mutant died. Survivors are listed in `<out>/mutation.md` with the line and the change; each one gets a
|
|
36
|
+
test or a `mutation-ok: <reason>` comment. `skills/qa-rigor/SKILL.md` has the full procedure and the pitfalls.
|
|
37
|
+
|
|
38
|
+
## 3. Link QAKit into a UI-test target
|
|
39
|
+
|
|
40
|
+
In Xcode: File > Add Package Dependencies > Add Local (the `vendor/qakit` folder) or the GitHub URL, then add the `QAKit`
|
|
41
|
+
library to the **UI-test target only** (it imports XCTest). With a generated project, add the package as a local
|
|
42
|
+
`XCLocalSwiftPackageReference` on that target, or compile `Sources/QAKit/QAKit.swift` into the UI-test target directly
|
|
43
|
+
(the file has no other dependency).
|
|
44
|
+
|
|
45
|
+
Then write the walkthrough:
|
|
46
|
+
|
|
47
|
+
```swift
|
|
48
|
+
import XCTest
|
|
49
|
+
import QAKit
|
|
50
|
+
|
|
51
|
+
final class MyAppQAWalkthroughTests: XCTestCase {
|
|
52
|
+
private var app: XCUIApplication!
|
|
53
|
+
|
|
54
|
+
override func setUpWithError() throws {
|
|
55
|
+
continueAfterFailure = true // a QA walk records every problem instead of stopping at the first
|
|
56
|
+
app = XCUIApplication()
|
|
57
|
+
app.launchArguments = ["-ui-testing"] + QA.languageArguments
|
|
58
|
+
app.launch()
|
|
59
|
+
QA.settleAfterLaunch(app) // the VoiceOver welcome or a permission prompt left over the app
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
func testWalkSettings() {
|
|
63
|
+
app.tabBars.buttons["Settings"].tap()
|
|
64
|
+
let screen = app.otherElements["settings.screen"]
|
|
65
|
+
XCTAssertTrue(screen.waitForExistence(timeout: 5))
|
|
66
|
+
QA.walk(app, "settings", in: self, deny: ["Delete", "Call"]) {
|
|
67
|
+
QA.settle(self.app, on: screen, in: self, closers: ["settings.done"]) {
|
|
68
|
+
self.app.terminate(); self.app.launch(); self.app.tabBars.buttons["Settings"].tap()
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`QA.walk` screenshots the screen, runs Apple's accessibility audit (every issue in one attached report), taps every visible
|
|
76
|
+
button once (screenshot of what opened, a check that the app is still in front, then your `settle` closure), scrolls and
|
|
77
|
+
repeats until the bottom. `QA.settle` closes whatever is on top and relaunches only as the last resort.
|
|
78
|
+
|
|
79
|
+
## 4. Write the manifest and run on the phone
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
cp vendor/qakit/examples/device_qa.json MyApp/release/testing/device_qa.json # edit paths, suites, variants
|
|
83
|
+
python3 vendor/qakit/tools/device_qa.py doctor --manifest MyApp/release/testing/device_qa.json
|
|
84
|
+
python3 vendor/qakit/tools/device_qa.py test --manifest MyApp/release/testing/device_qa.json --suite qa --variant dark --variant large-text
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`docs/manifest.md` explains every key; `docs/report.md` explains the output folder. The phone needs Developer Mode on, no
|
|
88
|
+
passcode, and (first run only) a tap on the UI automation prompt.
|
|
89
|
+
|
|
90
|
+
## 5. Static submission checks
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
python3 vendor/qakit/tools/release_checks.py --manifest MyApp/release/testing/device_qa.json
|
|
94
|
+
python3 vendor/qakit/tools/release_checks.py --manifest MyApp/release/testing/device_qa.json --ipa build/MyApp.ipa
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## 6. Give the agent the procedure
|
|
98
|
+
|
|
99
|
+
Link `skills/qa-rigor` and `skills/device-qa` into your agent's skills folder (`skills/README.md`) and point your
|
|
100
|
+
AGENTS.md or CLAUDE.md at `docs/device-qa-protocol.md`. The protocol's exit criteria are the release verdict.
|