eq-layer 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,8 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ .venv/
4
+ venv/
5
+ .env
6
+ .DS_Store
7
+ .idea/
8
+ .vscode/
eq_layer-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 eq-layer contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,527 @@
1
+ Metadata-Version: 2.5
2
+ Name: eq-layer
3
+ Version: 0.1.0
4
+ Summary: Inspectable intent, affect, repair, uncertainty, and factored-action control for LLM responses
5
+ Project-URL: Homepage, https://github.com/Furox-Art/eq-layer
6
+ Project-URL: Repository, https://github.com/Furox-Art/eq-layer
7
+ Project-URL: Issues, https://github.com/Furox-Art/eq-layer/issues
8
+ Author: Furox-Art
9
+ License: MIT
10
+ License-File: LICENSE
11
+ Keywords: affective-computing,agent-skills,agents,dialogue-management,intent-classification,llm,llm-steering
12
+ Classifier: Development Status :: 3 - Alpha
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: License :: OSI Approved :: MIT License
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.10
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
21
+ Requires-Python: >=3.10
22
+ Provides-Extra: dev
23
+ Requires-Dist: build<2,>=1.2; extra == 'dev'
24
+ Requires-Dist: pytest; extra == 'dev'
25
+ Requires-Dist: ruff; extra == 'dev'
26
+ Requires-Dist: scikit-learn<2,>=1.5; extra == 'dev'
27
+ Requires-Dist: twine<7,>=6; extra == 'dev'
28
+ Provides-Extra: ml
29
+ Requires-Dist: scikit-learn<2,>=1.5; extra == 'ml'
30
+ Description-Content-Type: text/markdown
31
+
32
+ # eq-layer
33
+
34
+ A control layer that gives an LLM a *decision* about how to speak before it speaks — instead of asking it to be more empathetic in a prompt.
35
+
36
+ ## Why this exists
37
+
38
+ Asking a model to be more empathetic is style transfer. It produces *more* of the same flat warmth, which reads as uncanny rather than attentive. The failure modes are structural, not knowledge gaps:
39
+
40
+ - **Flat affect.** "That sounds really tough!" arrives at the same register regardless of what was said.
41
+ - **Escalation blindness.** The user is frustrated, then angry, then threatening. The model holds one tone for all three turns.
42
+ - **Validation without motion.** The feeling is acknowledged; the next step never comes.
43
+ - **Repair collapse.** After a correction the model either capitulates entirely or starts defending itself. There is no middle.
44
+
45
+ So this repo does not retrain the model. It adds a layer:
46
+
47
+ 1. **Affect state** — `{valence, arousal, escalation_delta, stance, subtext}` as an explicit intermediate representation, tracked across turns. `escalation_delta` is the point: snapshot classification cannot see a trend.
48
+ 2. **Intent belief state** — the learned classifier preserves the full posterior over `action_request/status_check/explanation/question/statement/unknown` instead of collapsing uncertainty to one confidence number. A transparent one-step Bayes-risk router chooses whether to act or clarify from that distribution. This is POMDP-inspired uncertainty handling, not a full POMDP.
49
+ 3. **Conversation health state** — `repair` identifies observable user→assistant misalignment and its target; `interaction_quality` tracks structural conversation failure separately from emotion using current score, delta, repeated failures, unresolved repair, and clarification load.
50
+ 4. **Policy selection** — affect and intent jointly choose a response-policy family (`mirror`, `direct`, `repair`, `hold`, `boundary`, ...). Selection is a discrete decision, not generation.
51
+ 5. **Factored action compilation** — the selected policy is split into orthogonal controls: `task_move`, `social_move`, `repair_move`, and `realization`. The task is driven primarily by user intent, so affective adaptation cannot silently replace what the user actually asked for.
52
+ 6. **Steering** — the factored action, source policy, repair state, interaction quality, and canonical request are injected into the decode path so the control decision actually lands in the tokens.
53
+ 7. **Structural response audit** — the generated response is checked against inspectable control contracts such as question budget, required clarification, unsupported privacy promises, unknown-stance admissions, repair repetition, and low-verbosity overruns. This is not a factual truth checker.
54
+
55
+ ## Prior art and novelty boundary
56
+
57
+ EQ-Layer does **not** claim that affect-aware dialogue policy, user modelling,
58
+ repair, intent routing, or adaptive dialogue management are new. Those ideas
59
+ have substantial pre-LLM precedent in TRAINS/grounding work, adaptive TOOT,
60
+ RavenClaw, user-tailored generation, affect-sensitive AutoTutor, affective
61
+ POMDP dialogue management, and related systems.
62
+
63
+ The defensible research question is narrower: whether an explicit,
64
+ model-agnostic control layer can improve a **fixed general-purpose language
65
+ model** by separating intent, affect, conversational evidence, policy selection,
66
+ and generation steering while keeping those intermediate decisions inspectable.
67
+
68
+ See `docs/PRIOR_ART_AND_NOVELTY.md` for the historical map, claims to avoid,
69
+ and architecture lessons carried forward into this project.
70
+
71
+ ## Status
72
+
73
+ Early. The harness runs; the numbers are development signals, not a real-world EQ benchmark.
74
+
75
+ There is no standard benchmark for EQ. `eval/` is a first attempt at one, not a finished measurement. If you improve the scorer, that contribution matters as much as adding a policy.
76
+
77
+ ## Agent Skills: Codex, Claude, Copilot, Cursor, Antigravity, and Grok
78
+
79
+ EQ-Layer is packaged as a portable Agent Skill in `skills/eq-layer/` and as an
80
+ Agent Plugins 1.0 package via `plugin.json`. Repository discovery shims are
81
+ included for the major agent ecosystems:
82
+
83
+ | Agent | Repository path |
84
+ | --- | --- |
85
+ | OpenAI Codex | `.codex/skills/eq-layer/` |
86
+ | Anthropic Claude Code | `.claude/skills/eq-layer/` |
87
+ | GitHub Copilot | `.github/skills/eq-layer/` |
88
+ | Cursor | `.cursor/skills/eq-layer/` |
89
+ | Google Antigravity | `.agents/skills/eq-layer/` |
90
+ | xAI Grok Build | `.grok/skills/eq-layer/` |
91
+
92
+ For a user-level install across all supported discovery roots:
93
+
94
+ ```bash
95
+ python tools/install_agent_skills.py --target all
96
+ ```
97
+
98
+ The skill includes a local routing script and reports whether it used the
99
+ lightweight portable path or the full learned research pipeline.
100
+
101
+ ```bash
102
+ printf '%s' '[{"role":"user","content":"devam et"}]' \
103
+ | python skills/eq-layer/scripts/route.py --pretty
104
+ ```
105
+
106
+ See `skills/eq-layer/references/VENDOR_COMPATIBILITY.md` for verified discovery
107
+ paths and source documentation.
108
+
109
+ ## Quick start
110
+
111
+ Install the learned intent tracker:
112
+
113
+ ```bash
114
+ pip install "eq-layer[ml]"
115
+ ```
116
+
117
+ ```python
118
+ from eq_layer import ConversationTracker, Selector, Steer, TrainedAffect, TrainedIntent, TrainedSubtext, compose_action
119
+
120
+ messages = [
121
+ {"role": "user", "content": "Randevumu üç kez değiştirdiler"},
122
+ ]
123
+ affect = TrainedAffect.load("artifacts/emobank_affect.joblib")
124
+ subtext = TrainedSubtext.load("artifacts/xdailydialog_subtext.joblib")
125
+ tracker = ConversationTracker(affect=affect, subtext=subtext)
126
+ tracked = tracker.infer(messages)
127
+ state = tracked.state
128
+ intent = TrainedIntent.from_bundled().infer(messages)
129
+ selection = Selector().select(state, intent)
130
+ action = compose_action(
131
+ selection.policy,
132
+ intent,
133
+ repair=tracked.repair,
134
+ interaction_quality=tracked.interaction_quality,
135
+ )
136
+ print(selection.policy.name, action.to_dict())
137
+
138
+ prompt = Steer.build(
139
+ selection.policy,
140
+ intent,
141
+ repair=tracked.repair,
142
+ interaction_quality=tracked.interaction_quality,
143
+ ).apply_to_prompt(messages[-1]["content"])
144
+ ```
145
+
146
+ Run the seeded cases. Exits non-zero on any mismatch or any policy that no
147
+ case can reach:
148
+
149
+ ```bash
150
+ python eval/run.py
151
+ ```
152
+
153
+ Run the intent benchmark separately:
154
+
155
+ ```bash
156
+ python eval/intent_eval.py
157
+ ```
158
+
159
+ It compares the learned adapter with the heuristic fallback and reports
160
+ five-fold cross-validation on the bundled corpus.
161
+
162
+ Train the affect regressor from the pinned official EmoBank release:
163
+
164
+ ```bash
165
+ python tools/train_emobank_affect.py --output artifacts/emobank_affect.joblib
166
+ python eval/emobank_affect_eval.py
167
+ ```
168
+
169
+ EmoBank is not bundled into this repository. The training/evaluation path uses
170
+ the upstream train/dev/test split and records its pinned commit and
171
+ CC-BY-SA-4.0 provenance in `THIRD_PARTY_DATA.md`.
172
+
173
+ Train the learned dialogue-signal tracker from pinned XDailyDialog files:
174
+
175
+ ```bash
176
+ python tools/train_xdailydialog_subtext.py --output artifacts/xdailydialog_subtext.joblib
177
+ python eval/xdailydialog_subtext_eval.py
178
+ ```
179
+
180
+ XDailyDialog provides supervised dialogue-act and basic-emotion labels. EQ-Layer
181
+ does **not** pretend that `challenge`, `demand`, or `exhaustion` are direct
182
+ upstream labels: those are conservative compositions of learned dialogue
183
+ signals with learned V/A state. See `THIRD_PARTY_DATA.md` for the license-chain
184
+ caution on the English data.
185
+
186
+ ## Intent is a control signal, not mind-reading
187
+
188
+ The default intent path is now learned rather than keyword-selected.
189
+ `TrainedIntent` fits a balanced logistic-regression classifier over combined
190
+ word and character TF-IDF features. The bundled corpus currently contains
191
+ Turkish and English examples for `action_request`, `status_check`,
192
+ `explanation`, `question`, `statement`, and `unknown`.
193
+
194
+ The classifier emits a full posterior distribution, exposed as
195
+ `IntentBelief`. Production routing no longer discards that distribution behind
196
+ a single confidence/margin threshold. `decide_intent_action` computes the
197
+ expected interaction loss of each response move and clarification, then chooses
198
+ the minimum-risk action. For example, ambiguity between a status check and a
199
+ generic question can still be answered because the semantic mismatch cost is
200
+ small, while ambiguity between executing an action and merely responding to a
201
+ statement can trigger clarification.
202
+
203
+ The current loss matrix is **hand-specified and transparent, not learned or
204
+ claimed optimal**. It encodes relative interaction costs. EQ-Layer therefore
205
+ audits routing sensitivity rather than treating one matrix as ground truth:
206
+ `eval/intent_risk_sensitivity.py` sweeps clarification cost and performs
207
+ one-at-a-time ±20% perturbations of every non-zero loss cell, reporting action
208
+ flips overall and by source stratum. This is a local robustness diagnostic, not
209
+ loss-matrix optimization, and it must not be tuned against blinded human
210
+ preference outcomes. The old confidence and margin fields remain serialized for
211
+ backwards compatibility and diagnostics; they no longer choose the learned
212
+ intent route.
213
+
214
+ ### Intent-risk sensitivity audit
215
+
216
+ The production control path has also been run over all 120 external held-out
217
+ cases without generating LLM responses. Under one-at-a-time ±20% perturbations
218
+ of every non-zero loss cell, 7,332 / 7,440 routing decisions were unchanged
219
+ (**98.55% local decision stability**).
220
+
221
+ That result does **not** validate the loss matrix. Scaling the entire
222
+ clarification-cost row changes routing substantially:
223
+
224
+ - production clarification rate: **45.0%**
225
+ - clarification cost ×0.75: **69.17%**
226
+ - clarification cost ×1.25: **19.17%**
227
+ - clarification cost ×1.50: **9.17%**
228
+
229
+ The `task_repair` stratum is especially uncertain: 26 / 30 cases route to
230
+ clarification (**86.67%**) with mean fused intent entropy 0.8954. This may
231
+ reflect genuine ambiguity in repair turns, but it is not used to tune the
232
+ matrix. The current loss table remains hand-specified and uncalibrated.
233
+
234
+ Machine-readable audit:
235
+ `eval/results/intent_risk_heldout_120_audit.json`.
236
+
237
+ Explicit constraints such as `brief`, `scope_limited`, and `no_guess`
238
+ remain deterministic because they are control requirements, not semantic
239
+ labels.
240
+
241
+ `HeuristicIntent` remains available as a zero-dependency fallback. It does not
242
+ manufacture a learned posterior and is no longer the adapter used by the main
243
+ evaluation harness.
244
+
245
+ Affect can still override intent during sustained escalation. A clear action
246
+ request should drive the response in a calm turn; it should not erase a
247
+ multi-turn escalation signal.
248
+
249
+ The bundled data is small and synthetic. `eval/intent_eval.py` therefore
250
+ reports both a fixed held-out score and five-fold cross-validation; neither is
251
+ presented as evidence of production-level semantic understanding.
252
+
253
+ ## Verified development baselines
254
+
255
+ The current CI-verified EmoBank regression baseline uses the official upstream
256
+ train/dev/test split (8062 / 1000 / 1000). On the untouched test split:
257
+
258
+ | dimension | MAE | train-mean baseline MAE | Spearman rho |
259
+ | --- | ---: | ---: | ---: |
260
+ | Valence | 0.2125 | 0.2449 | 0.5460 |
261
+ | Arousal | 0.1782 | 0.1904 | 0.3120 |
262
+ | Dominance | 0.1494 | 0.1563 | 0.2573 |
263
+
264
+ The regressor beats the train-mean baseline on MAE for all three dimensions,
265
+ but Arousal and Dominance rank correlations are still modest. The machine-readable
266
+ record is `eval/results/emobank_affect_baseline.json`.
267
+
268
+ ### Dialogue-signal baseline
269
+
270
+ The learned dialogue-signal tracker was verified on the pinned XDailyDialog
271
+ English train/dev/test files (83035 / 8025 / 7716 utterances). On the untouched
272
+ test file:
273
+
274
+ | task | accuracy | macro-F1 | majority accuracy |
275
+ | --- | ---: | ---: | ---: |
276
+ | dialogue act | 0.8025 | 0.7274 | 0.4557 |
277
+ | basic emotion | 0.8249 | 0.4310 | 0.8166 |
278
+
279
+ The dialogue-act signal is substantially above the majority baseline. The
280
+ emotion signal is much weaker: macro-F1 is only 0.431 and accuracy is only
281
+ slightly above the majority baseline on test (and below it on dev). EQ-Layer
282
+ therefore treats emotion as supporting evidence rather than standalone proof of
283
+ subtext. The machine-readable result is
284
+ `eval/results/xdailydialog_signal_baseline.json`.
285
+
286
+ ### Higher-level subtext audits
287
+
288
+ Direct higher-level supervision was investigated rather than assumed:
289
+
290
+ | source / target | verified result | routing decision |
291
+ | --- | --- | --- |
292
+ | DialogBank correction | 4 direct examples | **NO-GO** — insufficient supervision |
293
+ | DialogBank disagreement | 1 direct example | **NO-GO** — insufficient supervision |
294
+ | Coarse Discourse disagreement | test ROC-AUC 0.8111, AP 0.1420, F1 0.2278 | **NO-GO** — ranking signal exists, but precision/recall are insufficient for `challenge` routing |
295
+ | DBDC3 hard breakdown | test ROC-AUC 0.5057, AP 0.2555 | **NO-GO** — effectively chance-level text-only generalisation |
296
+
297
+ These results are preserved rather than optimized away. The corresponding
298
+ machine-readable records are
299
+ `eval/results/dialogbank_label_inventory.json`,
300
+ `eval/results/coarse_disagreement_no_go.json`, and
301
+ `eval/results/dbdc3_breakdown_no_go.json`.
302
+
303
+
304
+ ## Evidence provenance
305
+
306
+ A subtext label is not enough by itself. `SubtextDecision.evidence_level`
307
+ records how the label was obtained:
308
+
309
+ - `verified` — caller/external annotation supplied the state.
310
+ - `direct_learned` — the upstream training target directly matches the emitted label.
311
+ - `derived` — multiple learned signals are composed into a higher-level control label.
312
+ - `structural` — explicit conversational form or marker triggered the label.
313
+ - `fallback` — learned evidence was too weak and an auditable fallback was used.
314
+
315
+ For example, XDailyDialog directly supervises `question`, but it does not
316
+ directly supervise EQ-Layer's `challenge` or `exhaustion`; those remain
317
+ `derived`. `StanceDecision` uses the same principle: factual
318
+ `user_right/user_wrong` stays `unknown` unless externally verified.
319
+
320
+ ## Blind response-level evaluation
321
+
322
+ Component metrics do not establish that EQ-Layer improves the final response.
323
+ The repository therefore includes a condition-blind paired human-evaluation
324
+ harness. The complete pre-registered protocol is in `eval/AB_PROTOCOL.md`.
325
+
326
+ Generate paired responses with the **same base model** in both arms:
327
+
328
+ ```bash
329
+ python eval/ab_generate.py heldout.jsonl pairs.jsonl manifest.json \
330
+ --model-command "python model_adapter.py" \
331
+ --model-id SAME_BASE_MODEL_VERSION \
332
+ --affect-model artifacts/emobank_affect.joblib \
333
+ --subtext-model artifacts/xdailydialog_subtext.joblib \
334
+ --temperature 0 \
335
+ --seed 42 \
336
+ --git-commit <eq-layer-commit>
337
+ ```
338
+
339
+ The configured command receives JSON on stdin and returns either plain response
340
+ text or JSON containing `text`/`response`. Secrets belong in environment
341
+ variables, not command arguments. `eval/cases.jsonl` is rejected by default
342
+ because it is a development set, and gold annotations are ignored unless an
343
+ explicit oracle-analysis flag is supplied.
344
+
345
+
346
+ Prepare pairs as JSONL:
347
+
348
+ ```json
349
+ {"id":"case-001","context":[{"role":"user","content":"..."}],"baseline":"...","eq":"..."}
350
+ ```
351
+
352
+ Blind and randomize them:
353
+
354
+ ```bash
355
+ python eval/ab_prepare.py pairs.jsonl ballot.jsonl key.json --seed 42
356
+ ```
357
+
358
+ Keep `key.json` away from raters. For every case, rate A/B/tie on:
359
+
360
+ - `intent_fidelity`
361
+ - `appropriateness`
362
+ - `actionability`
363
+ - `non_patronizing`
364
+ - `overall`
365
+
366
+ Then score a single rater:
367
+
368
+ ```bash
369
+ python eval/ab_score.py rated_ballot.jsonl key.json
370
+ ```
371
+
372
+ Or score multiple raters on the same blinded cases:
373
+
374
+ ```bash
375
+ python eval/ab_score_multi.py key.json rater1.jsonl rater2.jsonl rater3.jsonl
376
+ ```
377
+
378
+ The single-rater scorer reports EQ wins/losses/ties, non-tie win rate, Wilson
379
+ 95% confidence intervals, and an exact two-sided sign test. The multi-rater
380
+ scorer additionally aggregates **case-level majority preference** and reports
381
+ **Fleiss' kappa** for inter-rater agreement; it does not treat every rater vote
382
+ as an independent case.
383
+
384
+ The harness does **not** manufacture a result: response generation and human
385
+ ratings must come from a real pre-registered comparison.
386
+
387
+ ### Passive response auditor
388
+
389
+ A generated EQ response is also inspected by `ResponseAuditor`. The A/B
390
+ generation pipeline records the result under `generation.response_audit` and
391
+ the manifest records `response_auditor_mode=passive-metadata-only`.
392
+
393
+ The auditor currently checks only structurally observable control violations:
394
+
395
+ - empty response;
396
+ - question-budget overrun;
397
+ - missing required clarification;
398
+ - unsupported absolute privacy promise;
399
+ - `stance=unknown` combined with an explicit "you are right / I was wrong"
400
+ admission;
401
+ - near-duplicate recent assistant responses;
402
+ - repeated prior answer while a repair is active;
403
+ - large low-verbosity overruns.
404
+
405
+ It does **not** decide whether factual claims are true, whether the user's
406
+ position is correct, or whether advice is substantively good. Hard audit
407
+ failures currently request regeneration in metadata only; automatic
408
+ regeneration is deliberately disabled in the A/B experiment so the evaluation
409
+ protocol is not silently changed.
410
+
411
+ ## Selection
412
+
413
+ Policies declare preconditions, and the most constrained applicable policy
414
+ wins. There is no first-match rule, so adding a policy cannot silently change
415
+ what a looser one does to a state you did not think about.
416
+
417
+ Two consequences worth arguing about:
418
+
419
+ - **Implicit tie-breaking is forbidden.** Selection ranks by specificity, then
420
+ explicit `priority`. If two applicable policies still tie, the selector
421
+ raises an ambiguity error instead of silently picking whichever was declared
422
+ first.
423
+ - **`stance` is not guessed.** Whether the user is right is a semantic
424
+ judgement, so the tracker returns `unknown` unless a case or caller supplies
425
+ one. Policies that need it declare `user_is_right` or `stance_unknown`, so
426
+ "I could not tell" stays visible in the output instead of becoming a
427
+ confident wrong answer.
428
+
429
+ Cases can carry `annotated` fields (`stance`, `subtext`) for signals that
430
+ keywords genuinely cannot recover — exhaustion is the current example.
431
+
432
+ ## Layout
433
+
434
+ ```
435
+ eq_layer/
436
+ policies.py joint affect/intent policy taxonomy + selector
437
+ actions.py task/social/repair/realization factorization
438
+ auditor.py structural post-generation control-contract audit
439
+ interaction.py structural repair lifecycle + interaction-quality state
440
+ affect.py affect-state tracker interface + zero-dependency structural fallback
441
+ trained_affect.py learned VAD regressor + multi-turn affect adapter
442
+ trained_subtext.py learned dialogue-act/emotion signals + conservative subtext derivation
443
+ trained_disagreement.py experimental direct-disagreement model; verified no-go for routing
444
+ trained_breakdown.py experimental text-only breakdown model; verified no-go for routing
445
+ tracker.py composes affect, subtext, stance, repair and interaction quality
446
+ intent.py intent state + zero-dependency fallback
447
+ intent_belief.py posterior belief + transparent Bayes-risk router
448
+ trained_intent.py learned TF-IDF + logistic-regression posterior adapter
449
+ data/intent_train.jsonl bundled training corpus
450
+ steer.py policy + intent → decode path, and the scorer
451
+ response_experiment.py same-model baseline-vs-EQ generation engine
452
+ eval/
453
+ cases.jsonl development cases, not final A/B evidence
454
+ AB_PROTOCOL.md pre-registered response-level evaluation protocol
455
+ ab_generate.py paired response generation
456
+ ab_prepare.py A/B blinding
457
+ ab_score.py single-rater scoring
458
+ ab_score_multi.py multi-rater scoring
459
+ run.py
460
+ ```
461
+
462
+ ## Scoring
463
+
464
+ Three metrics, deliberately narrow:
465
+
466
+ - **specificity** — does the response name the concrete thing the user actually said, or a category of difficulty?
467
+ - **genericness** — count of empty validation phrases. Penalised on purpose: these are cheap under human raters, so models learn to produce them unless a cost is attached.
468
+ - **escalation latency** — how many turns pass after tension rises before the register changes.
469
+
470
+ If genericness is not penalised, a model will score well by saying nothing in particular.
471
+
472
+ ## Affect and conversational signals are now partly learned
473
+
474
+ `TrainedAffect` learns continuous Valence, Arousal and Dominance from EmoBank,
475
+ then converts Valence to `[-1, 1]` and Arousal to `[0, 1]` for policy state.
476
+ Multi-turn `escalation_delta` is computed from consecutive learned arousal
477
+ predictions rather than keyword counts.
478
+
479
+ `TrainedSubtext` separately learns dialogue-act and basic-emotion signals from
480
+ XDailyDialog. Those supervised signals are then combined with V/A state to
481
+ derive higher-level control labels conservatively:
482
+
483
+ - learned `question` -> `question`
484
+ - learned `directive` + independent heat -> `demand`
485
+ - learned `question` + anger/disgust + high arousal -> `challenge`
486
+ - learned sadness + negative low-arousal V/A -> `exhaustion` / `resignation`
487
+
488
+ `correction` and disclosure boundaries remain structural because the chosen
489
+ upstream data does not directly supervise those categories.
490
+
491
+ `stance=user_right/user_wrong` is **not** inferred from generic dialogue.
492
+ Whether a user is factually or procedurally correct requires external evidence.
493
+ `AnnotationStanceResolver` therefore emits `unknown` unless a verifier or
494
+ annotation supplies a stance. This is intentional rather than a missing
495
+ confidence threshold.
496
+
497
+ The remaining limitations are:
498
+
499
+ - **`escalating_past_n` excludes the current turn.** The question is whether
500
+ the preceding turns were rising, so one polite message after three hostile
501
+ ones does not reset the register.
502
+ - **`question` vs `challenge`** turns on whether escalation markers are
503
+ present, not on punctuation. `Sözleşme kaç gün geçerli?` is a question;
504
+ `Sen de mi?!` is a challenge.
505
+ - **Exhaustion/resignation still lacks direct supervision.** The current label
506
+ is a `derived` composition of learned sadness plus low-valence/low-arousal
507
+ state. It must not be described as a directly learned exhaustion classifier.
508
+ - **First-word matching strips punctuation.** `Neden?` must match `neden` or
509
+ the one-word repeated question — the exact shape `escalating` exists to
510
+ catch — silently never fires.
511
+
512
+ The remaining hard gap is direct, transferable supervision for higher-level
513
+ states such as correction, exhaustion/resignation, and factual stance. Two
514
+ plausible direct proxies — disagreement detection and text-only breakdown
515
+ detection — were tested and retained as no-go results instead of being wired
516
+ into policy routing. The tracker exposes where labels are direct, derived,
517
+ structural, fallback, or externally verified.
518
+
519
+ ## Contributing
520
+
521
+ Cases are the main entry point. Open one for a failure mode you can describe concretely — especially the ones where two policies seem equally right.
522
+
523
+ Policy changes go through an RFC first: `.github/RFC_TEMPLATE.md`.
524
+
525
+ ## License
526
+
527
+ MIT. See `LICENSE`.