opencode-windows-scheduler 1.5.2 → 1.5.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +46 -1
- package/LICENSE +21 -21
- package/README.md +3 -2
- package/ROADMAP-STRATEGY.md +5 -4
- package/ROADMAP.md +251 -72
- package/dist/core/local-ai.d.ts +52 -0
- package/dist/core/local-ai.d.ts.map +1 -0
- package/dist/core/local-ai.js +119 -0
- package/dist/core/ps-wrapper.d.ts +2 -30
- package/dist/core/ps-wrapper.d.ts.map +1 -1
- package/dist/core/ps-wrapper.js +211 -39
- package/dist/core/triage-chain.d.ts +34 -0
- package/dist/core/triage-chain.d.ts.map +1 -0
- package/dist/core/triage-chain.js +70 -0
- package/dist/core/triage-rules.d.ts +23 -0
- package/dist/core/triage-rules.d.ts.map +1 -0
- package/dist/core/triage-rules.js +174 -0
- package/dist/core/triage-shadow.d.ts +47 -0
- package/dist/core/triage-shadow.d.ts.map +1 -0
- package/dist/core/triage-shadow.js +48 -0
- package/dist/core/triage.d.ts +76 -0
- package/dist/core/triage.d.ts.map +1 -0
- package/dist/core/triage.js +271 -0
- package/dist/gui-experimental.cjs +1 -0
- package/dist/i18n.d.ts.map +1 -1
- package/dist/i18n.js +34 -7
- package/dist/index.d.ts +27 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +187 -2
- package/dist/notifications.d.ts.map +1 -1
- package/dist/notifications.js +9 -1
- package/dist/rpc.d.ts +259 -0
- package/dist/rpc.d.ts.map +1 -1
- package/dist/rpc.js +58 -0
- package/dist/settings.d.ts +80 -0
- package/dist/settings.d.ts.map +1 -1
- package/dist/settings.js +199 -0
- package/docs/ARCHITECTURE.md +9 -7
- package/docs/FEATURES.md +22 -2
- package/docs/LOCAL-AI.md +202 -0
- package/docs/MIGRATION.md +17 -0
- package/docs/README.ru.md +5 -4
- package/docs/spikes/local-model-structured-output.md +81 -0
- package/docs/spikes/ollama-qwen3-4b-smoke.json +28 -0
- package/docs/spikes/ollama-qwen3-4b.json +163 -0
- package/gui-experimental.cjs +8 -8
- package/locales/en.json +32 -0
- package/locales/ru.json +32 -0
- package/package.json +9 -2
- package/scripts/bench-local.mjs +321 -0
- package/scripts/spike-local-structured.mjs +182 -0
- package/skills/scheduled-job-best-practices/references/recipes.md +40 -0
- package/src/core/local-ai.ts +173 -0
- package/src/core/ps-wrapper.ts +218 -38
- package/src/core/triage-chain.ts +116 -0
- package/src/core/triage-rules.ts +215 -0
- package/src/core/triage-shadow.ts +96 -0
- package/src/core/triage.ts +366 -0
- package/src/gui.tsx +4761 -4533
- package/src/i18n.ts +32 -9
- package/src/index.ts +222 -2
- package/src/notifications.ts +16 -0
- package/src/rpc.ts +58 -0
- package/src/settings.ts +318 -0
package/docs/LOCAL-AI.md
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
# Local assistant (triage)
|
|
2
|
+
|
|
3
|
+
The local assistant is an optional, fully offline classifier that filters noisy
|
|
4
|
+
notifications. After a run finishes, the scheduler can ask a local model to
|
|
5
|
+
classify the outcome (quality, timeout, permission, network, unknown) and decide
|
|
6
|
+
whether the notification is worth sending. It is **opt-in**: when no model is
|
|
7
|
+
available the scheduler keeps working exactly as before and notifications are
|
|
8
|
+
never blocked.
|
|
9
|
+
|
|
10
|
+
Triage is not a scheduler feature by itself — it only affects notifications. It
|
|
11
|
+
never changes run status, retries, Quality Gates, DAG dependencies or history.
|
|
12
|
+
|
|
13
|
+
## How it works
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
run finished ──► triage chain ──► notify / skip / log-only (shadow)
|
|
17
|
+
│
|
|
18
|
+
├─ llm — local model classifies the failure
|
|
19
|
+
└─ rules — built-in patterns (timeout, permission, network, quality)
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
- `off` — no triage; every notification is sent (default).
|
|
23
|
+
- `rules` — built-in rules only. No model, no network calls.
|
|
24
|
+
- `llm` — the configured local model only.
|
|
25
|
+
- `auto` — the model first, then the rules engine (recommended).
|
|
26
|
+
|
|
27
|
+
The result is validated against a fixed JSON schema. Invalid JSON, an
|
|
28
|
+
unreachable endpoint, a timeout or low confidence all fall back to the rules
|
|
29
|
+
engine (fail-open). With `fallback: "skip"` the scheduler skips the rules step
|
|
30
|
+
and delivers the notification instead.
|
|
31
|
+
|
|
32
|
+
## Requirements
|
|
33
|
+
|
|
34
|
+
- [Ollama](https://ollama.com/) running locally, or any OpenAI-compatible
|
|
35
|
+
server (a `baseUrl` ending with `/v1` switches to the OpenAI protocol).
|
|
36
|
+
- One pulled model, for example `qwen3:4b`.
|
|
37
|
+
- No internet access is required after the model is downloaded.
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
ollama serve
|
|
41
|
+
ollama pull qwen3:4b
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## Configuration
|
|
45
|
+
|
|
46
|
+
Triage settings live under `notifications.triage` and are edited from the GUI
|
|
47
|
+
side panel ("Local assistant") or through the `manage_local_ai` agent tool.
|
|
48
|
+
|
|
49
|
+
| Setting | Values | Default | Meaning |
|
|
50
|
+
| --- | --- | --- | --- |
|
|
51
|
+
| `mode` | `off` / `rules` / `llm` / `auto` | `off` | Triage strategy. |
|
|
52
|
+
| `baseUrl` | URL | `http://127.0.0.1:11434` | Ollama or OpenAI-compatible endpoint. |
|
|
53
|
+
| `model` | string | `""` | Model id, for example `qwen3:4b`. |
|
|
54
|
+
| `timeoutMs` | 1000..120000 | `45000` | Request timeout. |
|
|
55
|
+
| `fallback` | `rules` / `skip` | `rules` | Behaviour when the model fails or is unsure. |
|
|
56
|
+
| `confidenceThreshold` | 0..1 | `0.6` | Minimum model confidence; below it the fallback applies. |
|
|
57
|
+
| `shadow.enabled` | boolean | `true` | Log decisions without suppressing notifications. |
|
|
58
|
+
| `shadow.days` | 1..365 | `14` | Shadow period length. |
|
|
59
|
+
| `shadow.startedAt` | ISO string / null | `null` | Start of the shadow period; `null` restarts it. |
|
|
60
|
+
|
|
61
|
+
## Managing triage from the agent
|
|
62
|
+
|
|
63
|
+
The `manage_local_ai` tool has three actions.
|
|
64
|
+
|
|
65
|
+
Read the current state:
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
{ "action": "get" }
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Update one or more fields (merged with the current settings):
|
|
72
|
+
|
|
73
|
+
```json
|
|
74
|
+
{
|
|
75
|
+
"action": "update",
|
|
76
|
+
"mode": "auto",
|
|
77
|
+
"baseUrl": "http://127.0.0.1:11434",
|
|
78
|
+
"model": "qwen3:4b",
|
|
79
|
+
"timeoutMs": 45000,
|
|
80
|
+
"fallback": "rules",
|
|
81
|
+
"confidenceThreshold": 0.6,
|
|
82
|
+
"shadow": { "enabled": true, "days": 14 }
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Send one classification request against the configured model (or a candidate
|
|
87
|
+
patch without saving it):
|
|
88
|
+
|
|
89
|
+
```json
|
|
90
|
+
{ "action": "test", "model": "qwen3:4b", "timeoutMs": 60000 }
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
`get` and `update` return a summary: `mode`, `baseUrl`, `model`, `timeoutMs`,
|
|
94
|
+
`fallback`, `confidenceThreshold`, `rules` (number of active rules or `null`),
|
|
95
|
+
`shadow`, `llmReady`, `shadowActive` and `protocol` (`ollama` or `openai`).
|
|
96
|
+
|
|
97
|
+
## Picking a model by RAM
|
|
98
|
+
|
|
99
|
+
The classifier is a small, structured task: a 4B model is usually enough. The
|
|
100
|
+
benchmark below prints a tier for the current machine.
|
|
101
|
+
|
|
102
|
+
| RAM | Tier | Recommended model | Notes |
|
|
103
|
+
| --- | --- | --- | --- |
|
|
104
|
+
| < 8 GB | rules-only | `qwen3:0.6b` | Prefer `mode: "rules"`; a tiny model may be too slow. |
|
|
105
|
+
| 8–15 GB | light | `qwen3:1.7b` | Fast, lower accuracy. |
|
|
106
|
+
| 15–30 GB | balanced | `qwen3:4b` | Recommended default. |
|
|
107
|
+
| > 30 GB | max | `qwen3:8b` | Better accuracy, more latency. |
|
|
108
|
+
|
|
109
|
+
## Benchmarking a machine
|
|
110
|
+
|
|
111
|
+
`scripts/bench-local.mjs` measures latency, schema validity and throughput
|
|
112
|
+
(tokens/s) for the configured model. It reuses the same triage prompt and JSON
|
|
113
|
+
schema as the scheduler.
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
npm run bench:local -- --model qwen3:4b --runs 3
|
|
117
|
+
npm run bench:local -- --model qwen3:4b --runs 5 --out bench.json
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Options: `--base-url`, `--model`, `--runs` (1..50), `--timeout` (1000..600000),
|
|
121
|
+
`--out`, `--help`.
|
|
122
|
+
|
|
123
|
+
The report is a versioned JSON envelope (`schemaVersion: "1"`) suitable for
|
|
124
|
+
crowdsourcing results:
|
|
125
|
+
|
|
126
|
+
```json
|
|
127
|
+
{
|
|
128
|
+
"schemaVersion": "1",
|
|
129
|
+
"generatedAt": "2026-11-15T12:00:00.000Z",
|
|
130
|
+
"available": true,
|
|
131
|
+
"machine": { "platform": "win32", "arch": "x64", "cpu": "…", "cpuCount": 16, "ramGb": 15.9, "nodeVersion": "v24.21.0" },
|
|
132
|
+
"tier": { "id": "balanced", "recommendedModel": "qwen3:4b", "reason": "…" },
|
|
133
|
+
"model": "qwen3:4b",
|
|
134
|
+
"runs": 3,
|
|
135
|
+
"metrics": {
|
|
136
|
+
"schemaValid": 3,
|
|
137
|
+
"schemaValidRate": 1,
|
|
138
|
+
"jsonValid": 3,
|
|
139
|
+
"jsonValidRate": 1,
|
|
140
|
+
"avgLatencyMs": 23785,
|
|
141
|
+
"p95LatencyMs": 26904,
|
|
142
|
+
"avgTokPerSec": 28.7,
|
|
143
|
+
"p95TokPerSec": 28.98
|
|
144
|
+
},
|
|
145
|
+
"results": []
|
|
146
|
+
}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
Exit codes: `0` when at least 70% of responses are schema-valid, `1` otherwise,
|
|
150
|
+
`2` when the endpoint or model is unavailable (the JSON report is still written
|
|
151
|
+
with `available: false`).
|
|
152
|
+
|
|
153
|
+
## Shadow mode
|
|
154
|
+
|
|
155
|
+
Shadow mode is the safe way to evaluate triage. Decisions are logged, but no
|
|
156
|
+
notification is suppressed. Check the shadow report before switching to
|
|
157
|
+
`mode: "auto"`: it shows how many notifications would have been skipped and
|
|
158
|
+
whether any of those were false positives.
|
|
159
|
+
|
|
160
|
+
Start with `mode: "auto"`, `shadow.enabled: true` and `shadow.days: 14`. After
|
|
161
|
+
the period, review the shadow report and only then disable shadow mode.
|
|
162
|
+
|
|
163
|
+
## Troubleshooting
|
|
164
|
+
|
|
165
|
+
| Symptom | Cause | Fix |
|
|
166
|
+
| --- | --- | --- |
|
|
167
|
+
| `connection refused` / `fetch failed` | Ollama is not running | `ollama serve` |
|
|
168
|
+
| `model not found` | Model not pulled | `ollama pull <model>`, then check `ollama list` |
|
|
169
|
+
| Requests time out | Model too large for the machine or context too long | Lower the model tier, raise `timeoutMs`, cap context (below) |
|
|
170
|
+
| `llmReady: false` in `manage_local_ai` `get` | Endpoint or model unavailable | Check `baseUrl` and `model` |
|
|
171
|
+
| Very slow first response | Cold model load | Run once to warm the model; the benchmark shows steady-state numbers |
|
|
172
|
+
| Invalid JSON from the model | Model too small or prompt drift | Use a larger model or `mode: "rules"` |
|
|
173
|
+
|
|
174
|
+
### OLLAMA_CONTEXT_LENGTH
|
|
175
|
+
|
|
176
|
+
Ollama allocates a large context window by default, which costs RAM and slows
|
|
177
|
+
CPU inference. For triage, a short context is enough. Set it in the environment
|
|
178
|
+
before starting the server:
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
# Windows (PowerShell)
|
|
182
|
+
$env:OLLAMA_CONTEXT_LENGTH = "2048"
|
|
183
|
+
ollama serve
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
# Linux / macOS
|
|
188
|
+
OLLAMA_CONTEXT_LENGTH=2048 ollama serve
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Restart Ollama after changing the variable. If the model still does not fit,
|
|
192
|
+
choose a smaller tier or use `mode: "rules"`.
|
|
193
|
+
|
|
194
|
+
## Limits
|
|
195
|
+
|
|
196
|
+
- Triage only affects notifications. It never rewrites run results.
|
|
197
|
+
- The model runs locally; prompts and logs are not sent to a third party.
|
|
198
|
+
- A missing model never blocks scheduling: rules or direct delivery take over.
|
|
199
|
+
- The classifier is intentionally conservative. When in doubt it delivers the
|
|
200
|
+
notification (fail-open).
|
|
201
|
+
- The benchmark measures a single machine; results are not a guarantee for other
|
|
202
|
+
hardware.
|
package/docs/MIGRATION.md
CHANGED
|
@@ -240,6 +240,20 @@ For every migrated job:
|
|
|
240
240
|
right workspace path.
|
|
241
241
|
6. Wait for at least one natural scheduled run before deleting the old definition.
|
|
242
242
|
|
|
243
|
+
## Local AI / triage migration
|
|
244
|
+
|
|
245
|
+
If you are migrating from a setup that used cloud AI for lightweight task triage or
|
|
246
|
+
classification, you can move that load to a local model to save tokens:
|
|
247
|
+
|
|
248
|
+
1. Install Ollama: `ollama serve` and `ollama pull qwen3:4b` (or a smaller model for
|
|
249
|
+
RAM-constrained machines — see `docs/LOCAL-AI.md` for tier recommendations).
|
|
250
|
+
2. Configure triage via `manage_local_ai({action: "update", mode: "auto", ...})` or
|
|
251
|
+
through the GUI notifications panel.
|
|
252
|
+
3. Start in `shadow` mode so decisions are logged but not applied, then switch to `auto`
|
|
253
|
+
once confidence is acceptable.
|
|
254
|
+
4. Ollama is optional — if it is unavailable, the plugin falls back to rules-only mode
|
|
255
|
+
(fail-open) and never blocks notifications.
|
|
256
|
+
|
|
243
257
|
## Common pitfalls
|
|
244
258
|
|
|
245
259
|
| Symptom | Cause | Fix |
|
|
@@ -252,6 +266,9 @@ For every migrated job:
|
|
|
252
266
|
| Two runs at once after migration | old scheduler is still enabled | disable the old job |
|
|
253
267
|
| Cron rejected | expression not expressible by `schtasks` | use a supported pattern (`*/N`, fixed minute/hour, weekdays) |
|
|
254
268
|
| Prompts behave differently at night | interactive shell environment vs scheduler environment | use absolute paths, avoid alias-dependent commands |
|
|
269
|
+
| Local AI (Ollama) unreachable | `manage_local_ai` test fails or triage falls back to rules | run `ollama serve`, `ollama pull <model>`, check `OLLAMA_CONTEXT_LENGTH` |
|
|
270
|
+
| Local AI confidence too low | rules fire instead of LLM, more tokens used than expected | raise `confidenceThreshold` in `manage_local_ai`, review rule reasons in audit |
|
|
271
|
+
|
|
255
272
|
|
|
256
273
|
## Rollback
|
|
257
274
|
|
package/docs/README.ru.md
CHANGED
|
@@ -38,8 +38,8 @@ English version: [README.md](../README.md).
|
|
|
38
38
|
| Триггеры | cron, одноразовый запуск (`triggerType: "once"`), git-коммиты (`triggerType: "git"`, фильтры ветки/путей/игнорирования), изменение файла (chokidar, glob-маски `watchGlobs`/`watchIgnore`), MCP-события, входящие webhooks (`POST /api/hooks/:task` с HMAC) |
|
|
39
39
|
| Мультирепо | workspaces с валидацией путей, запуски по workspace, матрица поверх родительской задачи |
|
|
40
40
|
| Наблюдаемость | история SQLite (success / error / timeout / partial / running), метрики (длительность, токены, стоимость), JSONL-логи с `/api/logs`, Prometheus `/metrics`, экспорт OTLP, логи по задачам, веб-дашборд с аналитикой за 14 дней, ретеншн истории и экспорт CSV/JSON |
|
|
41
|
-
| Эксплуатация | dry-run, экспорт/импорт JSON, шаблоны, уведомления по нескольким каналам (webhook, Telegram, нативные toast), тихие часы, дайджест, зашифрованные секреты (`${secret:NAME}`), журнал аудита, настройки |
|
|
42
|
-
| Интерфейсы | панель OpenCode GUI, веб-дашборд,
|
|
41
|
+
| Эксплуатация | dry-run, экспорт/импорт JSON, шаблоны, уведомления по нескольким каналам (webhook, Telegram, нативные toast), тихие часы, дайджест, локальный ассистент для триажа уведомлений (fail-open, shadow-режим), зашифрованные секреты (`${secret:NAME}`), журнал аудита, настройки |
|
|
42
|
+
| Интерфейсы | панель OpenCode GUI, веб-дашборд, 25 инструментов для агента, встроенный скилл |
|
|
43
43
|
|
|
44
44
|
### Язык интерфейса
|
|
45
45
|
|
|
@@ -86,7 +86,7 @@ npm run build && npm run build:gui
|
|
|
86
86
|
|
|
87
87
|
## Инструменты для агента
|
|
88
88
|
|
|
89
|
-
Агенту доступен весь функционал —
|
|
89
|
+
Агенту доступен весь функционал — 25 инструментов: расписание, пайплайны, эксплуатация, аналитика и администрирование.
|
|
90
90
|
|
|
91
91
|
| Инструмент | Аргументы | Что делает |
|
|
92
92
|
|---|---|---|
|
|
@@ -105,6 +105,7 @@ npm run build && npm run build:gui
|
|
|
105
105
|
| `get_metrics` | — | Всего запусков, успехов и падений |
|
|
106
106
|
| `get_analytics` | `days?`, `topLimit?` | Отчёт за период: запуски, успешность, длительность, токены, стоимость, топ задач |
|
|
107
107
|
| `manage_settings` | `action`, `maxConcurrent?`, `winterMode?`, `notifications?`, `history?` | Чтение и изменение глобальных настроек (параллелизм, уведомления, retention) |
|
|
108
|
+
| `manage_local_ai` | `action` (`get`/`update`/`test`), `mode?`, `baseUrl?`, `model?`, `timeoutMs?`, `fallback?`, `confidenceThreshold?`, `shadow?` | Настройки локального ассистента для триажа уведомлений и проверка модели одним запросом (см. [LOCAL-AI.md](LOCAL-AI.md)) |
|
|
108
109
|
| `manage_templates` | `action`, поля шаблона | Список, сохранение и удаление шаблонов задач |
|
|
109
110
|
| `manage_profiles` | `action`, `name?`, `port?` | Список, создание, переключение и удаление изолированных профилей |
|
|
110
111
|
| `transfer_tasks` | `action`, `data?` | Экспорт всех задач в JSON и импорт обратно |
|
|
@@ -184,7 +185,7 @@ src/
|
|
|
184
185
|
npm install
|
|
185
186
|
npm run build # tsc
|
|
186
187
|
npm run build:gui # vite
|
|
187
|
-
npm test #
|
|
188
|
+
npm test # 39 сюит, 600+ проверок
|
|
188
189
|
npm run test:en # те же сюиты с английской локалью
|
|
189
190
|
```
|
|
190
191
|
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Спайк: structured output локальной модели (30.1)
|
|
2
|
+
|
|
3
|
+
Дата: 15.11.2026. Статус: **выполнен**.
|
|
4
|
+
|
|
5
|
+
## Вопрос
|
|
6
|
+
|
|
7
|
+
Можно ли использовать 4B-модель через Ollama для триажа уведомлений, не полагаясь на «повезёт с JSON»? Нужен ли retry и как валидировать ответ?
|
|
8
|
+
|
|
9
|
+
## Окружение
|
|
10
|
+
|
|
11
|
+
- OpenCode **2.2.3**, Ollama **0.40.16**.
|
|
12
|
+
- GPU: NVIDIA GeForce RTX 3050, 8 ГБ VRAM; модель загружается целиком.
|
|
13
|
+
- Модель: `hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M` (Qwen3 4.02B, Q4_K_M, 2.5 ГБ).
|
|
14
|
+
- Почему не `qwen3:4b`: официальный реестр Ollama отдаёт блобы с Cloudflare R2, а соединение с R2 на этой машине сбрасывается и по IPv4, и по IPv6. Hugging Face доступен, поэтому модель скачана оттуда; для Ollama это тот же GGUF и тот же runner.
|
|
15
|
+
|
|
16
|
+
## Формат id моделей Ollama в OpenCode V2
|
|
17
|
+
|
|
18
|
+
- OpenCode сам опрашивает `http://127.0.0.1:11434`, получает список моделей через `/api/tags` и `/api/show` и добавляет их в `/models` без подключения аккаунта.
|
|
19
|
+
- Ссылка на модель — **`ollama/<model>`**. Для модели из HF имя будет `ollama/hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M`.
|
|
20
|
+
- Для удалённой Ollama `providers.ollama.settings.baseURL` должен включать `/v1`; опциональный `apiKey` уходит bearer-токеном.
|
|
21
|
+
- В инвентарь попадают только модели с capability `completion`; embedding-only модели исключены.
|
|
22
|
+
- При недоступности Ollama OpenCode сохраняет последний успешный инвентарь, поэтому «модель есть в списке» не гарантирует, что сервер сейчас жив.
|
|
23
|
+
|
|
24
|
+
Вывод для планировщика: хранить `ollama/<model>` как обычный id модели, но перед запуском проверять доступность провайдера, а не только наличие id в списке.
|
|
25
|
+
|
|
26
|
+
## Результаты 10 прогонов
|
|
27
|
+
|
|
28
|
+
Команда:
|
|
29
|
+
|
|
30
|
+
```powershell
|
|
31
|
+
node scripts/spike-local-structured.mjs `
|
|
32
|
+
--model "hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M" `
|
|
33
|
+
--runs 10 --timeout 180000 `
|
|
34
|
+
--out docs/spikes/ollama-qwen3-4b.json
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
| Метрика | Значение |
|
|
38
|
+
|---|---|
|
|
39
|
+
| Валидный JSON | **10/10** |
|
|
40
|
+
| Соответствие JSON Schema | **10/10 (100%)** |
|
|
41
|
+
| Средняя латентность | **20 336 мс** |
|
|
42
|
+
| p95 латентности | **20 927 мс** |
|
|
43
|
+
| Разброс | 19 961–20 927 мс |
|
|
44
|
+
|
|
45
|
+
Первый запрос после загрузки модели занял **41.7 с** (cold start). Последующие — около 20 с.
|
|
46
|
+
|
|
47
|
+
Ответы во всех прогонах одинаковые: `category=quality`, `severity=high`, `confidence=0.95`. Это ожидаемо при `temperature: 0` и одном и том же сценарии; спайк проверял стабильность формата, а не качество классификации на разнообразных уведомлениях.
|
|
48
|
+
|
|
49
|
+
## Вывод
|
|
50
|
+
|
|
51
|
+
1. **Формат надёжен:** Qwen3-4B с `format: <JSON Schema>` выдала схема-валидный JSON 10/10. Retry ради формата не нужен.
|
|
52
|
+
2. **Валидацию схемы всё равно оставляем:** `format` — это подсказка, а не гарантия; парсер должен отклонять лишние поля, пустой `reason`, выход за enum и `confidence` вне `0..1`.
|
|
53
|
+
3. **Один retry оставляем как страховку** на таймаут/сетевую ошибку, а не на «модель не попала в JSON».
|
|
54
|
+
4. **Латентность — главный ограничитель:** 20 с в среднем и 42 с на холодную. Дефолтный `timeoutMs: 15000` из 30.2 для такой модели слишком мал. Для синхронного триажа нужен таймаут не меньше 45–60 с либо асинхронная обработка; иначе fallback на правила будет срабатывать почти всегда.
|
|
55
|
+
5. **Fail-open обязателен:** триаж не должен задерживать или блокировать доставку уведомления.
|
|
56
|
+
6. **Логировать** модель, версию, латентность и причину fallback, чтобы 30.8 собрал статистику.
|
|
57
|
+
|
|
58
|
+
## Решение для 30.3–30.5
|
|
59
|
+
|
|
60
|
+
- Валидировать JSON и схему на стороне планировщика.
|
|
61
|
+
- Один retry только при таймауте, обрыве соединения или невалидном JSON; при повторной ошибке — правила.
|
|
62
|
+
- Таймаут по умолчанию поднять до **45 000–60 000 мс** (или сделать триаж асинхронным).
|
|
63
|
+
- Порог уверенности из 30.2 (`0.6`) применять после валидации схемы.
|
|
64
|
+
- Не считать одинаковые ответы на один сценарий доказательством качества классификации.
|
|
65
|
+
|
|
66
|
+
## Как воспроизвести
|
|
67
|
+
|
|
68
|
+
```powershell
|
|
69
|
+
ollama serve
|
|
70
|
+
ollama pull hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M
|
|
71
|
+
node scripts/spike-local-structured.mjs --model "hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M" --runs 10 --timeout 180000 --out docs/spikes/ollama-qwen3-4b.json
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Если модели нет, скрипт завершится с кодом 2 и подскажет, что скачать. Если Ollama недоступна — тоже код 2, без ложного «спайк пройден».
|
|
75
|
+
|
|
76
|
+
## Что осталось на потом
|
|
77
|
+
|
|
78
|
+
- Проверить модель на наборе разнообразных и противоречивых уведомлений (30.8), а не на одном сценарии.
|
|
79
|
+
- Сравнить 4B с 8B, если качество классификации окажется недостаточным.
|
|
80
|
+
- Проверить поведение при длинном уведомлении и при обрезанном контексте.
|
|
81
|
+
- Учесть холодный старт: первая задача после простоя платит +20 с.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
{
|
|
2
|
+
"generatedAt": "2026-11-15T13:08:19.631Z",
|
|
3
|
+
"baseUrl": "http://127.0.0.1:11434",
|
|
4
|
+
"model": "hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M",
|
|
5
|
+
"runs": 1,
|
|
6
|
+
"jsonValid": 1,
|
|
7
|
+
"schemaValid": 1,
|
|
8
|
+
"schemaValidRate": 1,
|
|
9
|
+
"avgLatencyMs": 41655,
|
|
10
|
+
"p95LatencyMs": 41655,
|
|
11
|
+
"results": [
|
|
12
|
+
{
|
|
13
|
+
"run": 1,
|
|
14
|
+
"ok": true,
|
|
15
|
+
"jsonValid": true,
|
|
16
|
+
"schemaValid": true,
|
|
17
|
+
"latencyMs": 41655,
|
|
18
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
19
|
+
"parsed": {
|
|
20
|
+
"category": "quality",
|
|
21
|
+
"severity": "high",
|
|
22
|
+
"confidence": 0.95,
|
|
23
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
24
|
+
},
|
|
25
|
+
"error": ""
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
{
|
|
2
|
+
"generatedAt": "2026-11-15T13:11:46.276Z",
|
|
3
|
+
"baseUrl": "http://127.0.0.1:11434",
|
|
4
|
+
"model": "hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M",
|
|
5
|
+
"runs": 10,
|
|
6
|
+
"jsonValid": 10,
|
|
7
|
+
"schemaValid": 10,
|
|
8
|
+
"schemaValidRate": 1,
|
|
9
|
+
"avgLatencyMs": 20336,
|
|
10
|
+
"p95LatencyMs": 20927,
|
|
11
|
+
"results": [
|
|
12
|
+
{
|
|
13
|
+
"run": 1,
|
|
14
|
+
"ok": true,
|
|
15
|
+
"jsonValid": true,
|
|
16
|
+
"schemaValid": true,
|
|
17
|
+
"latencyMs": 20037,
|
|
18
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
19
|
+
"parsed": {
|
|
20
|
+
"category": "quality",
|
|
21
|
+
"severity": "high",
|
|
22
|
+
"confidence": 0.95,
|
|
23
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
24
|
+
},
|
|
25
|
+
"error": ""
|
|
26
|
+
},
|
|
27
|
+
{
|
|
28
|
+
"run": 2,
|
|
29
|
+
"ok": true,
|
|
30
|
+
"jsonValid": true,
|
|
31
|
+
"schemaValid": true,
|
|
32
|
+
"latencyMs": 20248,
|
|
33
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
34
|
+
"parsed": {
|
|
35
|
+
"category": "quality",
|
|
36
|
+
"severity": "high",
|
|
37
|
+
"confidence": 0.95,
|
|
38
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
39
|
+
},
|
|
40
|
+
"error": ""
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"run": 3,
|
|
44
|
+
"ok": true,
|
|
45
|
+
"jsonValid": true,
|
|
46
|
+
"schemaValid": true,
|
|
47
|
+
"latencyMs": 19961,
|
|
48
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
49
|
+
"parsed": {
|
|
50
|
+
"category": "quality",
|
|
51
|
+
"severity": "high",
|
|
52
|
+
"confidence": 0.95,
|
|
53
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
54
|
+
},
|
|
55
|
+
"error": ""
|
|
56
|
+
},
|
|
57
|
+
{
|
|
58
|
+
"run": 4,
|
|
59
|
+
"ok": true,
|
|
60
|
+
"jsonValid": true,
|
|
61
|
+
"schemaValid": true,
|
|
62
|
+
"latencyMs": 20927,
|
|
63
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
64
|
+
"parsed": {
|
|
65
|
+
"category": "quality",
|
|
66
|
+
"severity": "high",
|
|
67
|
+
"confidence": 0.95,
|
|
68
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
69
|
+
},
|
|
70
|
+
"error": ""
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
"run": 5,
|
|
74
|
+
"ok": true,
|
|
75
|
+
"jsonValid": true,
|
|
76
|
+
"schemaValid": true,
|
|
77
|
+
"latencyMs": 20444,
|
|
78
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
79
|
+
"parsed": {
|
|
80
|
+
"category": "quality",
|
|
81
|
+
"severity": "high",
|
|
82
|
+
"confidence": 0.95,
|
|
83
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
84
|
+
},
|
|
85
|
+
"error": ""
|
|
86
|
+
},
|
|
87
|
+
{
|
|
88
|
+
"run": 6,
|
|
89
|
+
"ok": true,
|
|
90
|
+
"jsonValid": true,
|
|
91
|
+
"schemaValid": true,
|
|
92
|
+
"latencyMs": 20365,
|
|
93
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
94
|
+
"parsed": {
|
|
95
|
+
"category": "quality",
|
|
96
|
+
"severity": "high",
|
|
97
|
+
"confidence": 0.95,
|
|
98
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
99
|
+
},
|
|
100
|
+
"error": ""
|
|
101
|
+
},
|
|
102
|
+
{
|
|
103
|
+
"run": 7,
|
|
104
|
+
"ok": true,
|
|
105
|
+
"jsonValid": true,
|
|
106
|
+
"schemaValid": true,
|
|
107
|
+
"latencyMs": 20325,
|
|
108
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
109
|
+
"parsed": {
|
|
110
|
+
"category": "quality",
|
|
111
|
+
"severity": "high",
|
|
112
|
+
"confidence": 0.95,
|
|
113
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
114
|
+
},
|
|
115
|
+
"error": ""
|
|
116
|
+
},
|
|
117
|
+
{
|
|
118
|
+
"run": 8,
|
|
119
|
+
"ok": true,
|
|
120
|
+
"jsonValid": true,
|
|
121
|
+
"schemaValid": true,
|
|
122
|
+
"latencyMs": 20300,
|
|
123
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
124
|
+
"parsed": {
|
|
125
|
+
"category": "quality",
|
|
126
|
+
"severity": "high",
|
|
127
|
+
"confidence": 0.95,
|
|
128
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
129
|
+
},
|
|
130
|
+
"error": ""
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
"run": 9,
|
|
134
|
+
"ok": true,
|
|
135
|
+
"jsonValid": true,
|
|
136
|
+
"schemaValid": true,
|
|
137
|
+
"latencyMs": 20250,
|
|
138
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
139
|
+
"parsed": {
|
|
140
|
+
"category": "quality",
|
|
141
|
+
"severity": "high",
|
|
142
|
+
"confidence": 0.95,
|
|
143
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
144
|
+
},
|
|
145
|
+
"error": ""
|
|
146
|
+
},
|
|
147
|
+
{
|
|
148
|
+
"run": 10,
|
|
149
|
+
"ok": true,
|
|
150
|
+
"jsonValid": true,
|
|
151
|
+
"schemaValid": true,
|
|
152
|
+
"latencyMs": 20500,
|
|
153
|
+
"content": "{\n\n\"category\":\"quality\",\"severity\":\"high\",\"confidence\":0.95,\"reason\":\"Quality Gate вернул код 1: 3 теста не прошли в auth-модуле.\"}",
|
|
154
|
+
"parsed": {
|
|
155
|
+
"category": "quality",
|
|
156
|
+
"severity": "high",
|
|
157
|
+
"confidence": 0.95,
|
|
158
|
+
"reason": "Quality Gate вернул код 1: 3 теста не прошли в auth-модуле."
|
|
159
|
+
},
|
|
160
|
+
"error": ""
|
|
161
|
+
}
|
|
162
|
+
]
|
|
163
|
+
}
|