comfy-qa 1.15.0 → 1.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -77,6 +77,34 @@ No mocks. QA runs against real servers:
|
|
|
77
77
|
bun install
|
|
78
78
|
```
|
|
79
79
|
|
|
80
|
+
## Roadmap
|
|
81
|
+
|
|
82
|
+
Short-term, in priority order:
|
|
83
|
+
|
|
84
|
+
1. **Improve reproduction precision.** Current pipeline misses bugs that depend
|
|
85
|
+
on specific workflows or custom nodes (see
|
|
86
|
+
[PR #9430](https://github.com/Comfy-Org/ComfyUI_frontend/pull/9430) — 8/11
|
|
87
|
+
reproduced). Environment setup tools (workflow loader, custom-node
|
|
88
|
+
installer, attachment downloader) close this gap.
|
|
89
|
+
|
|
90
|
+
2. **Measure reliability (flakiness).** Run the same checklist N times, track
|
|
91
|
+
pass→fail→pass transitions per operation. A QA run is only trustworthy if
|
|
92
|
+
its result is stable across repeats. Surface flaky operations on the
|
|
93
|
+
dashboard.
|
|
94
|
+
|
|
95
|
+
3. **Auto-file GitHub issues for failing operations** — gated behind a
|
|
96
|
+
confidence threshold. `scripts/report-failures.sh` exists but is intentionally
|
|
97
|
+
not wired up yet: while reproduction rate is still ~70%, auto-filing would
|
|
98
|
+
drown maintainers in false positives. Enable once (1) and (2) lift the
|
|
99
|
+
floor, and keep a manual review step before filing the first batch per repo.
|
|
100
|
+
|
|
101
|
+
4. **Cross-product QA matrix.** Today: registry, docs, website, download-data,
|
|
102
|
+
embedded-editor. Add cloud.comfy.org (WebGL, already working via
|
|
103
|
+
`--headless=new`), comfy-vibe, and first-party ComfyUI_frontend runs.
|
|
104
|
+
|
|
105
|
+
5. **Continuous QA.** Schedule daily runs, track score trends per product,
|
|
106
|
+
alert on regressions (score drop ≥ 10%).
|
|
107
|
+
|
|
80
108
|
## License
|
|
81
109
|
|
|
82
110
|
MIT
|