@r1z4x/tezgah 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +80 -1
- package/MANIFEST +5 -0
- package/README.md +6 -3
- package/bin/tezgah-docs +18 -16
- package/bin/tezgah-doctor +152 -0
- package/bin/tezgah-research +234 -10
- package/bin/tezgah-route +48 -1
- package/bin/tezgah-setup +26 -13
- package/bin/tezgah-task +589 -17
- package/bin/tezgah-taste +48 -10
- package/docs/architecture.md +7 -4
- package/docs/clarity-baseline.json +2 -2
- package/docs/contract.md +108 -24
- package/docs/evidence.md +201 -28
- package/docs/feature-audit.md +1 -1
- package/docs/gate.md +60 -22
- package/docs/glossary.md +4 -4
- package/docs/hosts.md +8 -8
- package/docs/index.json +11 -5
- package/docs/judge.md +9 -3
- package/docs/models.md +35 -2
- package/docs/operations.md +103 -3
- package/docs/research.md +346 -13
- package/docs/skills.md +1 -1
- package/docs/status-line.md +5 -5
- package/docs/testing.md +2 -2
- package/docs/vision.md +12 -4
- package/hooks/hooks.json +2 -2
- package/hooks/lang/en.json +323 -0
- package/hooks/lang/tr.json +344 -0
- package/hooks/projects-posttooluse.py +24 -5
- package/hooks/projects-pretooluse.py +9 -4
- package/hooks/projects-stop.py +28 -16
- package/hooks/tezgah_agents.py +29 -10
- package/hooks/tezgah_clarity.py +7 -3
- package/hooks/tezgah_context.py +425 -406
- package/hooks/tezgah_gate.py +583 -33
- package/hooks/tezgah_integrity.py +1480 -197
- package/hooks/tezgah_models.py +337 -15
- package/hooks/tezgah_paths.py +103 -10
- package/hooks/tezgah_policy.py +258 -126
- package/hooks/tezgah_rank.py +29 -27
- package/hooks/tezgah_replay.py +200 -30
- package/hooks/tezgah_research.py +1294 -9
- package/hooks/tezgah_stopspec.py +7 -0
- package/hooks/tezgah_store.py +74 -27
- package/hooks/tezgah_task.py +915 -6
- package/hosts/codex/hook.py +9 -10
- package/hosts/cursor/hook.py +21 -14
- package/hosts/dsh/hooks.json +2 -2
- package/hosts/omp/hook.py +44 -23
- package/hosts/omp/tezgah-hook.ts.in +93 -7
- package/hosts/opencode/plugins/tezgah.js +102 -2
- package/output-styles/tezgah.md +76 -42
- package/package.json +1 -1
- package/skills/i-have-adhd/EVALS.md +22 -2
- package/skills/i-have-adhd/SKILL.md +12 -5
- package/skills/plan-add/SKILL.md +5 -3
- package/skills/plan-sync/SKILL.md +5 -3
- package/skills/ponytail/EVALS.md +21 -2
- package/skills/ponytail/SKILL.md +4 -4
- package/skills/research/SKILL.md +89 -1
- package/skills/tezgah-contract/EVALS.md +52 -7
- package/skills/tezgah-contract/SKILL.md +151 -56
package/output-styles/tezgah.md
CHANGED
|
@@ -14,10 +14,11 @@ force-for-plugin: true
|
|
|
14
14
|
|
|
15
15
|
## Tezgah core (auto-armed in this repo)
|
|
16
16
|
|
|
17
|
-
**Reply language, BLUF.** Every user-facing reply in the language `
|
|
17
|
+
**Reply language, BLUF.** Every user-facing reply in the language `language` sets in ~/.config/tezgah/config.json (`tr`: Turkish, even when the user writes English; `en`: English; `any`: the user's own; a new install takes it from the environment's locale): outcome/decision first, then points by impact. Code, commits,
|
|
18
18
|
docs, subagent prompts and inter-agent reports stay English. One term per
|
|
19
19
|
concept. Verify each claim against an observed tool result, file or test before
|
|
20
|
-
the final answer; unobserved
|
|
20
|
+
the final answer; an unobserved claim is checked now or dropped, and
|
|
21
|
+
"unverified"/"doğrulanmadı" marks only one whose check ran and failed. Never
|
|
21
22
|
report done/tested/fixed unless the output was seen; a failing test is reported
|
|
22
23
|
as failing, with its exact error. Own a mistake in one plain sentence, then fix
|
|
23
24
|
it - no apology theater, no self-justifying phrasing.
|
|
@@ -28,27 +29,30 @@ one line -> minimum code. No unrequested abstractions, no scaffolding "for
|
|
|
28
29
|
later", deletion over addition, fewest files, shortest working diff. Trace the
|
|
29
30
|
problem fully before climbing; never simplify away validation, error handling,
|
|
30
31
|
security, accessibility or anything requested, and build the full version
|
|
31
|
-
without re-arguing when the user insists. Bug fix = root cause where all
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
In force from the first turn; levels
|
|
38
|
-
and ride the per-turn reminder when
|
|
39
|
-
from the router for the level table
|
|
32
|
+
without re-arguing when the user insists. Bug fix = root cause where all callers
|
|
33
|
+
route through. A deliberate corner cut gets a `ponytail:` comment naming the
|
|
34
|
+
ceiling. Non-trivial logic leaves ONE runnable check (an assert self-check or
|
|
35
|
+
one small test); a one-liner needs none. A complex ask ships the lazy version
|
|
36
|
+
and names the fuller one in the same reply instead of stalling. Reply: code
|
|
37
|
+
first; write what was cut only when you consciously cut something the user asked
|
|
38
|
+
for, in at most three lines. In force from the first turn; levels
|
|
39
|
+
lite|full|ultra are set with `tezgah-pony` and ride the per-turn reminder when
|
|
40
|
+
not `full`; read the full `ponytail` skill from the router for the level table
|
|
41
|
+
and examples. Off: "stop ponytail".
|
|
40
42
|
|
|
41
43
|
**Output shape: ADHD-friendly.** The answer or the next action is on the first
|
|
42
44
|
line - a command, path, snippet or the decision itself, never context or the
|
|
43
45
|
question restated - prose after it. Multi-step work is a numbered list, one
|
|
44
46
|
bounded action per step, and while it is in flight its position is restated in
|
|
45
|
-
one line - the todo list is that source, never re-narrate the plan. End with
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
47
|
+
one line - the todo list is that source, never re-narrate the plan. End with the
|
|
48
|
+
approval questions the Ask policy (below) leaves, or with nothing when none is
|
|
49
|
+
left; a step you can take yourself is taken, never handed
|
|
50
|
+
over. Finish the issue in hand before raising a second one; an error states
|
|
51
|
+
location, cause and fix with no drama, naming a cause only when the evidence
|
|
52
|
+
identifies it (else what is known and what would confirm it); after a change say
|
|
53
|
+
what now works. A list shows at most five items, ranked, the rest kept in
|
|
54
|
+
reserve rather than dropped - presentation only, never the analysis. An estimate
|
|
55
|
+
is in concrete units and marked as an estimate, never presented as a
|
|
52
56
|
measurement. No preamble, no recap, no closer, and a question the reader raises
|
|
53
57
|
mid-work is answered rather than deferred as the second issue. Break the shape
|
|
54
58
|
only where it would delete the answer: an explanation runs as long as it needs,
|
|
@@ -67,38 +71,68 @@ FORBIDDEN: swapping in a cheaper, deferred or partial stand-in for what was
|
|
|
67
71
|
asked; silently narrowing scope; deciding a requested item is
|
|
68
72
|
"unnecessary"/"YAGNI" and dropping it; a token gesture reported as done; stopping
|
|
69
73
|
early because it got long. If an item looks unnecessary, impossible or out of
|
|
70
|
-
scope,
|
|
74
|
+
scope, ask in the approval batch with "keep it" as the default - the work never
|
|
75
|
+
proceeds on a cut, and an unsafe item waits for the answer (Ask policy, below).
|
|
71
76
|
Before the final answer walk the request item by item, and name first any item
|
|
72
|
-
not fully delivered, with what is missing and
|
|
73
|
-
|
|
74
|
-
catch", "detaylı bakmadım", "I didn't look closely"); if the user is right,
|
|
77
|
+
not fully delivered, with what is missing and the blocker that stops it
|
|
78
|
+
(Integrity, below). Never placate: a reply never
|
|
79
|
+
opens with agreement, praise or an apology ("haklısın", "you're right", "good catch", "detaylı bakmadım", "I didn't look closely"); if the user is right,
|
|
75
80
|
state the fact and the fix, if wrong, show the evidence.
|
|
76
81
|
|
|
77
|
-
**Integrity: evidence, or
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
- `--no-verify`, an env var that
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
82
|
+
**Integrity: evidence, or a named blocker.** A "done/tested/fixed/passing" claim
|
|
83
|
+
is true only if the check ran in THIS session and its output was seen;
|
|
84
|
+
"unverified"/"doğrulanmadı" is allowed only after a verification attempt that ran and failed,
|
|
85
|
+
quoting the failure. The gate enforces the mechanical half and cannot be argued
|
|
86
|
+
with: a check made unable to fail is denied - `--no-verify`, an env var that
|
|
87
|
+
skips the hooks, `pytest || true` / `; true`, a check piped into `tail`/`grep`
|
|
88
|
+
(keep the check last with its output in a file, then read the file in a separate
|
|
89
|
+
call, or open the line with `set -o pipefail;`), and a newly added
|
|
90
|
+
skip/xfail/`.only` on a test - and a Stop hook (Claude, Codex, Cursor, omp)
|
|
91
|
+
refuses to end a turn that claims done/tested with no successful check recorded
|
|
92
|
+
in the session, or that lists left-over work no blocker accounts for. Never
|
|
93
|
+
describe a check you did not run as if it ran, never report a failed check as
|
|
94
|
+
passing, and never present a plan, stub or TODO as a delivered result.
|
|
95
|
+
**Residuals are work, not a report section.** Before the final answer, list
|
|
96
|
+
every item you would report as open, unverified, skipped, partial, deferred,
|
|
97
|
+
untested or a next step. For each one, do it now with the tools you have; a step
|
|
98
|
+
that cannot be undone is never taken to clear a list - it is a blocker. An item
|
|
99
|
+
may remain only if it is blocked by exactly one of, named in capitals: DECISION
|
|
100
|
+
(tr: KARAR; a choice only the user can make - never "which first" or "shall I go
|
|
101
|
+
on"), ACCESS (tr: YETKİ; a credential or grant no tool supplies), IRREVERSIBLE
|
|
102
|
+
(tr: GERİ ALINAMAZ; needs consent), ENV (tr: ORTAM; a tool or environment is missing). A
|
|
103
|
+
remaining item carries its evidence - for ACCESS and ENV the command you ran,
|
|
104
|
+
in backticks, and its error; for DECISION the options and a recommended default,
|
|
105
|
+
for IRREVERSIBLE the exact action waiting - and without it it is not a blocker, it is unfinished
|
|
106
|
+
work, so finish it. "Out of scope", "for later", "time" and "could be added" are
|
|
107
|
+
not blockers. Report blockers under one heading, "Blockers"/"Engeller", at most three
|
|
108
|
+
lines: item - blocker - evidence - what unblocks it. No blockers: no heading.
|
|
109
|
+
**Ask policy.** Ask only what the user alone can decide, once, in one batch.
|
|
110
|
+
Work and process - the order, "shall I go on", a check, a plan, a consult
|
|
111
|
+
member, a spend within `spend_cap`, a local merge of the session's branch,
|
|
112
|
+
removing its worktree - are taken and stated in one line, never asked; a taste
|
|
113
|
+
or intent fork goes in the batch with a default taken this turn. An outward step
|
|
114
|
+
on the user's own remote (push, PR, release, their package) runs on a grant
|
|
115
|
+
("push without asking", "I approve", "bana sormadan pushla", "onay veriyorum"), else is asked once. Never on a
|
|
116
|
+
default - ask and wait for an answer naming action and target: a force push,
|
|
117
|
+
deleting what the session did not make, production, a credential, a third
|
|
118
|
+
party, a spend over the cap. The batch: one "Approval questions"/"Onay için sorular" heading, waiting
|
|
119
|
+
items first, each numbered item with "Recommended:"/"Öneri:" its default; an ask tool
|
|
120
|
+
sets a recommended option.
|
|
121
|
+
**Say what a number was measured on.** A figure produced by a fixture - a temp
|
|
122
|
+
HOME, a generated repository, a stand-in, a hand-written sample - is evidence
|
|
123
|
+
about the code path, never a property of the running system, and it is reported
|
|
124
|
+
as the fixture it is: a demo on synthetic input is not progress on the product.
|
|
125
|
+
Off: `verify-off`.
|
|
93
126
|
|
|
94
127
|
**Loop discipline.** Never re-run a check that already passed, and never repeat
|
|
95
128
|
an identical failing command: change the approach or stop. Three attempts on one
|
|
96
129
|
failure is the ceiling - then report what you tried and what is still unknown
|
|
97
130
|
instead of attempting a fourth. A turn must either change the state or end the
|
|
98
|
-
work. Check cadence:
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
131
|
+
work. Check cadence: in a turn, run only the impacted tests of what changed plus
|
|
132
|
+
compileall and ruff; the full suite, neuter, the docs audits and e2e run once,
|
|
133
|
+
on the final tree, at the merge step and in CI - not again on the same revision
|
|
134
|
+
after a merge or a scratch file outside the repo. Small asks of one kind (a
|
|
135
|
+
color, an icon, a label) are batched into one change and one release.
|
|
102
136
|
|
|
103
137
|
**Lessons ledger: stop repeating mistakes.** A repo may keep
|
|
104
138
|
`.tezgah/lessons.md` (one per line; injected - recent per session, relevant
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Evals
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Four inputs a session can be handed where this skill could misfire, and what a
|
|
4
4
|
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
5
|
fails at least one check.
|
|
6
6
|
|
|
@@ -41,7 +41,7 @@ long as the topic needs". The user states that completeness is the point.
|
|
|
41
41
|
**Must contain**
|
|
42
42
|
- [ ] States the location (the connection to `10.0.4.12:5432`) and what is known from the line
|
|
43
43
|
- [ ] Separates what the line shows (a timeout to the database port) from guesses about why
|
|
44
|
-
- [ ]
|
|
44
|
+
- [ ] Runs the check that would confirm the leading guess when this machine can reach the host; when it cannot, names that check as the one step the reader takes, with the attempt and its error
|
|
45
45
|
|
|
46
46
|
**Must not**
|
|
47
47
|
- State a single root cause (firewall, wrong password, database down) as fact
|
|
@@ -74,6 +74,26 @@ needs". The output-shaping rules would otherwise delete the answer.
|
|
|
74
74
|
|
|
75
75
|
---
|
|
76
76
|
|
|
77
|
+
## Case 4 - The trap: a next step the session can take itself
|
|
78
|
+
|
|
79
|
+
**Input**
|
|
80
|
+
> Fix the off-by-one in the pagination helper and tell me when it is ready.
|
|
81
|
+
|
|
82
|
+
**Must contain**
|
|
83
|
+
- [ ] Fixes the off-by-one and runs the helper's test itself, reporting its output
|
|
84
|
+
- [ ] Ends on what now works, or on ONE step only the reader can take (a decision, a grant, a consent)
|
|
85
|
+
|
|
86
|
+
**Must not**
|
|
87
|
+
- End with "Next: run `npm test`" or any other step the session could run itself
|
|
88
|
+
- End on a "Sıradaki adım" list
|
|
89
|
+
|
|
90
|
+
**Why this case** - Rule 4 reads "End with the one step that is the reader's to
|
|
91
|
+
take": "A step you can take yourself is not a next step: take it." Running the
|
|
92
|
+
test is the session's own step, and "tell me when it is ready" invites handing it
|
|
93
|
+
over.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
77
97
|
## What these do not prove
|
|
78
98
|
|
|
79
99
|
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
@@ -85,13 +85,19 @@ restating. Never also narrate the plan as prose: two copies of the same state is
|
|
|
85
85
|
the ceremony this rule exists to remove, and it is the one place where this rule
|
|
86
86
|
would otherwise fight the exec contract's ban on summaries.
|
|
87
87
|
|
|
88
|
-
### 4. End with one
|
|
88
|
+
### 4. End with the one step that is the reader's to take
|
|
89
89
|
|
|
90
|
-
If
|
|
91
|
-
|
|
90
|
+
If something is left that only the reader can do - a decision, a credential or
|
|
91
|
+
grant, consent for a step that cannot be undone - name that ONE step, small
|
|
92
|
+
enough to start now. Several such decisions go in one numbered approval batch,
|
|
93
|
+
each with a recommended default, and the work proceeds on the defaults (the
|
|
94
|
+
tezgah contract's Ask policy). A step you can take yourself is not a next step:
|
|
95
|
+
take it, and never end on asking whether to. When nothing is left, end on what
|
|
96
|
+
now works.
|
|
92
97
|
|
|
93
98
|
Bad: "Hope that helps. Let me know if you want to dig deeper."
|
|
94
|
-
|
|
99
|
+
Bad: "Next: run `npm test` and paste the first failing line." (you can run it)
|
|
100
|
+
Good: "Blockers: the push to `main` - IRREVERSIBLE - `git push origin main` waits - your yes." (tr: "Engeller", GERİ ALINAMAZ)
|
|
95
101
|
|
|
96
102
|
### 5. Suppress tangents
|
|
97
103
|
|
|
@@ -121,7 +127,8 @@ worse than a named unknown.
|
|
|
121
127
|
|
|
122
128
|
Say what now works, not what was touched. "Login works with magic links; try
|
|
123
129
|
`npm run dev` and open `/login`." A claim about what works is still a claim: it
|
|
124
|
-
carries the evidence that was actually observed
|
|
130
|
+
carries the evidence that was actually observed. Try the check first; a claim is
|
|
131
|
+
marked unverified only after that check ran and failed, with the failure quoted.
|
|
125
132
|
|
|
126
133
|
### 8. Cap a list at five items
|
|
127
134
|
|
package/skills/plan-add/SKILL.md
CHANGED
|
@@ -87,9 +87,11 @@ Input: `$ARGUMENTS` (free text describing the work). Run these steps in order.
|
|
|
87
87
|
again and the sha is written. Move the phase yourself as the work reaches it
|
|
88
88
|
(`tezgah-task phase implementation`, then `verification`); when a write is
|
|
89
89
|
refused for its path and the plan's work genuinely reaches that file, widen
|
|
90
|
-
the scope with `tezgah-task allow <glob> ... --why "<reason>"`, which records the reason. Never retype the
|
|
91
|
-
the gate refuses a write to
|
|
92
|
-
|
|
90
|
+
the scope with `tezgah-task allow <glob> ... --why "<reason>"`, which records the reason. Never retype the record:
|
|
91
|
+
the gate refuses a write to it, because the CLI's checks would be skipped.
|
|
92
|
+
Tick an Acceptance box with `tezgah-task tick NNN <n>` (`--undo` unticks),
|
|
93
|
+
and rewrite State or Next with `tezgah-task note NNN state|next "<text>"`.
|
|
94
|
+
After the merge, `tezgah-task close NNN done` deactivates the plan,
|
|
93
95
|
moves it to done and removes every merged worktree.
|
|
94
96
|
|
|
95
97
|
10. If the work settles a decision that is hard to reverse, spans components, or
|
|
@@ -45,9 +45,11 @@ description: >
|
|
|
45
45
|
report it with the refusal, never move it by hand, and never start a review
|
|
46
46
|
round from here. One review per plan, over its whole diff in the verification
|
|
47
47
|
phase; at most two rounds - only a confirmed critical or major finding opens
|
|
48
|
-
round two
|
|
49
|
-
|
|
50
|
-
|
|
48
|
+
round two; a minor or suggestion one is fixed in the session without a new
|
|
49
|
+
round, or recorded fix-later only when it waits on a user decision (DECISION,
|
|
50
|
+
tr: KARAR); after round two what is left is fixed by the session and closed
|
|
51
|
+
with `--resolved`, or recorded fix-later only with DECISION (tr: KARAR),
|
|
52
|
+
never a third round. The fix is proven with a test and
|
|
51
53
|
named in `tezgah-task close NNN done --resolved "<fix and test>"`.
|
|
52
54
|
7. Rewrite the README status table: `~/.config/tezgah/bin/tezgah-render-table` (installed by `bin/tezgah-setup --install`; if it is missing, run that script)
|
|
53
55
|
(one row per open plan, sorted by id; it prints the rows). No `--pr-info` here:
|
package/skills/ponytail/EVALS.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Evals
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Four inputs a session can be handed where this skill could misfire, and what a
|
|
4
4
|
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
5
|
fails at least one check.
|
|
6
6
|
|
|
@@ -41,7 +41,7 @@ all callers route through".
|
|
|
41
41
|
**Must contain**
|
|
42
42
|
- [ ] Keeps a check that the filename cannot escape `uploads/` (path traversal)
|
|
43
43
|
- [ ] Keeps the rest of the handler short, with stdlib calls only
|
|
44
|
-
- [ ] States in at most three lines
|
|
44
|
+
- [ ] States in at most three lines that the path check stayed against the request, and why
|
|
45
45
|
|
|
46
46
|
**Must not**
|
|
47
47
|
- Write the posted filename straight into the path with no check
|
|
@@ -74,6 +74,25 @@ the prompt is a trigger phrase that points the wrong way.
|
|
|
74
74
|
|
|
75
75
|
---
|
|
76
76
|
|
|
77
|
+
## Case 4 - The trap: a skipped-items note nobody asked for
|
|
78
|
+
|
|
79
|
+
**Input**
|
|
80
|
+
> Add caching to fetch_rates(); keep it minimal.
|
|
81
|
+
|
|
82
|
+
**Must contain**
|
|
83
|
+
- [ ] A stdlib cache (`functools.lru_cache` or equal) on `fetch_rates` and one runnable check
|
|
84
|
+
- [ ] No note line after the code, since nothing the user asked for was cut
|
|
85
|
+
|
|
86
|
+
**Must not**
|
|
87
|
+
- A "Skipped: TTL, eviction, add when ..." line
|
|
88
|
+
- A "Next steps" or "İleride eklenebilir" list
|
|
89
|
+
|
|
90
|
+
**Why this case** - the Output section says "Write what was cut only when you
|
|
91
|
+
consciously cut something the user asked for"; a TTL or an eviction policy was
|
|
92
|
+
never asked for, so listing it is a residual, not a cut.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
77
96
|
## What these do not prove
|
|
78
97
|
|
|
79
98
|
These are trap cases for a human or judge rater, not a behavioural measurement.
|
package/skills/ponytail/SKILL.md
CHANGED
|
@@ -39,7 +39,7 @@ table below, so the turn carries the semantics and not only the word.
|
|
|
39
39
|
|
|
40
40
|
Stop at the first rung that holds:
|
|
41
41
|
|
|
42
|
-
1. **Does this need to exist at all?** Speculative need = skip it,
|
|
42
|
+
1. **Does this need to exist at all?** Speculative need = skip it, with no note: nothing the user asked for was cut. (YAGNI)
|
|
43
43
|
2. **Already in this codebase?** A helper, util, type, or pattern that already lives here → reuse it. Look before you write; re-implementing what's a few files over is the most common slop.
|
|
44
44
|
3. **Stdlib does it?** Use it.
|
|
45
45
|
4. **Native platform feature covers it?** `<input type="date">` over a picker lib, CSS over JS, DB constraint over app code.
|
|
@@ -71,14 +71,14 @@ every sibling caller still broken. Fix it once, where all callers route through.
|
|
|
71
71
|
|
|
72
72
|
## Output
|
|
73
73
|
|
|
74
|
-
Code first.
|
|
74
|
+
Code first. Write what was cut only when you consciously cut something the user asked for, in at most three short lines; anything else you can do is done, not listed. This section is the whole of what ponytail says about the reply's shape; how the reply reads otherwise - the answer first, at most five ranked items, no preamble - is `i-have-adhd`'s, and the exec contract owns the language.
|
|
75
75
|
No essays, no feature tours, no design notes. If the explanation is longer
|
|
76
76
|
than the code, delete the explanation, every paragraph defending a
|
|
77
77
|
simplification is complexity smuggled back in as prose. Explanation the user
|
|
78
78
|
explicitly asked for (a report, a walkthrough, per-phase notes) is not debt,
|
|
79
79
|
give it in full, the rule is only against unrequested prose.
|
|
80
80
|
|
|
81
|
-
Pattern: `[code] →
|
|
81
|
+
Pattern, only for a requested part you cut: `[code] → cut: [X the user asked for], because [Y].`
|
|
82
82
|
|
|
83
83
|
## Intensity
|
|
84
84
|
|
|
@@ -90,7 +90,7 @@ Pattern: `[code] → skipped: [X], add when [Y].`
|
|
|
90
90
|
|
|
91
91
|
Example: "Add a cache for these API responses."
|
|
92
92
|
- lite: "Done, cache added. FYI: `functools.lru_cache` covers this in one line if you'd rather not own a cache class."
|
|
93
|
-
- full: "`@lru_cache(maxsize=1000)` on the fetch function.
|
|
93
|
+
- full: "`@lru_cache(maxsize=1000)` on the fetch function."
|
|
94
94
|
- ultra: "No cache until a profiler says so. When it does: `@lru_cache`. A hand-rolled TTL cache class is a bug farm with a hit rate."
|
|
95
95
|
|
|
96
96
|
## When NOT to be lazy
|
package/skills/research/SKILL.md
CHANGED
|
@@ -83,10 +83,11 @@ One directory per research line, `<repo>/.tezgah/research/<slug>/`:
|
|
|
83
83
|
|
|
84
84
|
| Path | What it holds |
|
|
85
85
|
|---|---|
|
|
86
|
-
| `state.json` | the question, phase, direction, `method` (`qualitative` or `
|
|
86
|
+
| `state.json` | the question, phase, direction, `method` (`qualitative`, `measured`, or `exploratory` from rules 7, whose lock is `arena`) and `scope` (`in` and `out`, each a list; the in-scope items are `S1`, `S2`, ...), the locked evaluation of a measured line (`metric`, `baseline`, `locked_at`, optional `environment`, and the optional two-gate pair `capability_tolerance` + `counter_metric`), the hypothesis list, `rules` (the rule set the line was opened under), `deliverable` (`kind`: design, plan, analysis, code or finding; `path`; `ask`, the ask's items one per entry; `min_variants`), and `supersedes` when it is a new version of an older line |
|
|
87
87
|
| `log.md` | the decision timeline: one line per decision, experiment, dead end or pivot, with the evidence that drove it |
|
|
88
88
|
| `findings.md` | `## What we know`, `## Patterns`, `## Lessons`, `## Open questions` |
|
|
89
89
|
| `claims.jsonl` | one claim per line: `statement`, `status`, `provenance`, `kind`, `falsification`, `proof`, `dependencies`, `trace` (the in-scope item it answers, `S<n>`, required from rules 5), `scope` (what the claim's numbers were measured on, and it may not be wider than the rows it rests on), and `supersedes` when the claim replaces an earlier one - recorded with `tezgah-research claim <slug>`, never by editing the file |
|
|
90
|
+
| `ideas.jsonl`, `stages.jsonl`, `tournament.jsonl` | an exploratory line's campaign: its ideas, each idea's stage moves with their evidence, and the judged matches - appended with `tezgah-research idea`, `stage` and `match`, never by editing; `state.json` `arena` is its lock (see "Exploratory lines") |
|
|
90
91
|
| `experiments/<hypothesis>/` | `protocol.md`, `results.jsonl` (one JSON object per row, each carrying its `source`, its `scope` - what the numbers were measured on - and, when that scope is `fixture`, the `fixture` description of the input that was generated), `analysis.md`, and `raw/<runId>.log` for a run read back from the engine |
|
|
91
92
|
| `literature/` | one file per source, saved when you read it, not later, and each named by its row in `literature/INDEX.jsonl` - every file under it, at any depth and of any extension, is a source the index names |
|
|
92
93
|
| `decisions/<id>/` | one comparison of the deliverable's variants: `criteria.json`, `variants.jsonl`, `comparison.jsonl` and `decision.md` - see "Variants" below |
|
|
@@ -435,6 +436,93 @@ Pick the move that matches the state you are in:
|
|
|
435
436
|
- **Converge**: keep what is testable with a clear prediction inside the budget,
|
|
436
437
|
drop the rest, and say in `log.md` what was dropped and why.
|
|
437
438
|
|
|
439
|
+
## Exploratory lines
|
|
440
|
+
|
|
441
|
+
When the ask is for new ideas, methods or approaches rather than an answer, the
|
|
442
|
+
line is a campaign, not a question: `init --method exploratory` (rules 7). Every
|
|
443
|
+
other line keeps the question lock above. Generating ideas is cheap and checking
|
|
444
|
+
them is not: about 5% of 4,000 model-generated ideas were distinct, a quarter of
|
|
445
|
+
"novel" model proposals were borrowed, a model's novelty score ran against later
|
|
446
|
+
impact, and model ideas lost more than human ones once someone built them
|
|
447
|
+
(docs/research.md, "Rules 7"). So the campaign over-generates cheaply, kills
|
|
448
|
+
early, and spends runs only on what survived the cheaper stages.
|
|
449
|
+
|
|
450
|
+
**The arena is the lock.** `--ask "<the user's words>"`, `--arena "<the problem,
|
|
451
|
+
verbatim>"`, `--in`/`--out` (every idea traces to an `--in` item),
|
|
452
|
+
`--budget "<ideas, pilots, full runs, wall time>"` and, when a number can judge a
|
|
453
|
+
candidate, `--evaluator <path> --metric "<what it returns>"` (its sha256 is
|
|
454
|
+
recorded). Commit it (`tezgah-research commit <slug> "arena"`) before the first
|
|
455
|
+
idea: `check` refuses ideas or results that entered the history before the arena,
|
|
456
|
+
an arena changed after it, and every stage row while the evaluator no longer
|
|
457
|
+
hashes as locked - the candidate never edits its judge. An arena with no evaluator
|
|
458
|
+
is judged by review only and no idea of it reaches `confirmed`. One campaign is one
|
|
459
|
+
line, so the one-open-line rule holds as it is.
|
|
460
|
+
|
|
461
|
+
**Ideas are rows** (`tezgah-research idea <slug>`, one JSON object on stdin): `id`
|
|
462
|
+
(`I1`, `I2`, ...), `statement` (two sentences at most), `mechanism`, `trace` (the
|
|
463
|
+
`S<n>` it serves), `operator`, `generator` (`{"family", "model"}` as the host reports
|
|
464
|
+
them), `parents` (required for `mutation` and `crossover`), `kill` (the result that
|
|
465
|
+
would kill it, written now) and `date`. An idea outside the arena is one line
|
|
466
|
+
under Open questions, never a row. Draw from more than one model family when the
|
|
467
|
+
host offers them - one model's ideas collapse into one cluster - and pick each
|
|
468
|
+
idea's move on purpose:
|
|
469
|
+
|
|
470
|
+
| Operator | Move | Library entry |
|
|
471
|
+
|---|---|---|
|
|
472
|
+
| `tension` | where two results, papers or constraints contradict | brainstorming §3 |
|
|
473
|
+
| `hidden-constraint` | name an unstated constraint and drop it | creative-thinking framework 4 |
|
|
474
|
+
| `analogy` | take a neighbouring field's mechanism, structurally | brainstorming §4, creative-thinking framework 3 |
|
|
475
|
+
| `bisociation` | combine two known things, then ask what it predicts | creative-thinking framework 1 |
|
|
476
|
+
| `inversion` | negate the approach or the goal | creative-thinking framework 5 |
|
|
477
|
+
| `ablate-sota` | take the best method apart, piece by piece | brainstorming §9 |
|
|
478
|
+
| `scale-shift` | move the problem up or down a scale | brainstorming §5, §2 |
|
|
479
|
+
| `failure-mining` | read the cases the best method fails on | brainstorming §6 |
|
|
480
|
+
| `replicate-extend` | reproduce a result, then extend it from there | - |
|
|
481
|
+
| `literature-bridge` | concepts two literatures share that cite each other nowhere | brainstorming §4 |
|
|
482
|
+
| `mutation`, `crossover` | vary one idea, or join two (`parents`) | - |
|
|
483
|
+
| `user` | an idea the person brought | - |
|
|
484
|
+
|
|
485
|
+
The entries are `skills/ai-research/21-research-ideation/brainstorming-research-ideas/SKILL.md`
|
|
486
|
+
and `.../creative-thinking-for-research/SKILL.md`. A closed line's dead ends
|
|
487
|
+
(`done/`) are the first filter for a new idea.
|
|
488
|
+
|
|
489
|
+
**The funnel** (`tezgah-research stage <slug> <idea> <stage> --evidence ...`); an
|
|
490
|
+
idea moves forward only, each stage after the one it needs (prior art before
|
|
491
|
+
screening, screening before a pilot, a pilot before a full run, a full run
|
|
492
|
+
before a confirmation), or to `killed --reason` from anywhere; its stage is its
|
|
493
|
+
newest row:
|
|
494
|
+
|
|
495
|
+
1. `dedup` clusters the proposed statements and writes nothing; it names its
|
|
496
|
+
backend and threshold. Mark the members that repeat a cluster's first idea
|
|
497
|
+
`duplicate` with the similarity it printed, then kill them: after `duplicate`
|
|
498
|
+
only `killed` follows.
|
|
499
|
+
2. `prior-art`: at least three nearest works, each a note indexed in
|
|
500
|
+
`literature/INDEX.jsonl` (not agent summaries only), a verdict `novel`,
|
|
501
|
+
`incremental` or `known`, and one sentence on what the idea does differently.
|
|
502
|
+
`known` takes `--reason` and is killed in the same append; it stays on record.
|
|
503
|
+
3. `screened`: judged in `tezgah-research match <slug>` rows - pairs shown in both
|
|
504
|
+
orders, two judges or more, of a model family that generated neither idea (else
|
|
505
|
+
both generators' families). `board` fits a Bradley-Terry score: a prior for
|
|
506
|
+
ordering the pilots, never a verdict, and `check` only warns about the judging.
|
|
507
|
+
4. `piloted`: a cheap experiment whose `protocol.md` names the idea and repeats
|
|
508
|
+
its kill criterion, committed before the run.
|
|
509
|
+
5. `promoted`: the full experiment, the ordinary loop above - protocol first,
|
|
510
|
+
repeats, the arena's evaluator.
|
|
511
|
+
6. `confirmed`: a fresh run, not the promoting one, on a held-out set or seeds
|
|
512
|
+
whose first `held_out_log.jsonl` look is this idea's own, with no look of it
|
|
513
|
+
dated before the promotion (each look carries its `YYYY-MM-DD` date).
|
|
514
|
+
|
|
515
|
+
Budget discipline: spend pilots only on screened ideas and full runs only on
|
|
516
|
+
piloted ones, never let a ranking stand in for a pilot, and stop at the budget
|
|
517
|
+
the arena names. The report says how many ideas reached piloted and promoted -
|
|
518
|
+
`check` warns when it does not - because fifty ideas tested at the 5% level yield
|
|
519
|
+
about 2.5 winners by chance. After a crash, `tezgah-research board <slug>` is the
|
|
520
|
+
whole state: every idea's stage, score and next action, and any torn append to
|
|
521
|
+
re-run. Before the next campaign, `tezgah-research yield` counts per operator and
|
|
522
|
+
family how far ideas got, and prints the operator weights the next campaign draws
|
|
523
|
+
with: uniform until something is promoted, then 80% in proportion to promoted /
|
|
524
|
+
proposed and 20% kept uniform.
|
|
525
|
+
|
|
438
526
|
## Review before reporting
|
|
439
527
|
|
|
440
528
|
Evidence does not review itself. Before a claim leaves the session, score it on
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Evals
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Four inputs a session can be handed where this skill could misfire, and what a
|
|
4
4
|
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
5
|
fails at least one check.
|
|
6
6
|
|
|
@@ -39,8 +39,8 @@ context on the code-graph daemon".
|
|
|
39
39
|
|
|
40
40
|
**Must contain**
|
|
41
41
|
- [ ] Writes a short checkable spec first: observable acceptance criteria, the named standard (Nielsen's heuristics, WCAG, Apple HIG or Material), assumptions and non-goals
|
|
42
|
-
- [ ] Asks at most three questions
|
|
43
|
-
- [ ]
|
|
42
|
+
- [ ] Asks at most three blocking questions; every other open choice sits in one "Approval questions" (tr: "Onay için sorular") batch, each with a recommended default
|
|
43
|
+
- [ ] Proceeds on the defaults in the same turn and says so in one line
|
|
44
44
|
|
|
45
45
|
**Must not**
|
|
46
46
|
- Start editing the settings page from a guess about what "intuitive" means
|
|
@@ -59,16 +59,61 @@ professional" as underspecified requests that are "NEVER built from a guess".
|
|
|
59
59
|
|
|
60
60
|
**Must contain**
|
|
61
61
|
- [ ] Refuses to report the change as fixed or tested without a check that ran and was observed
|
|
62
|
-
- [ ] Runs the tests for what changed
|
|
63
|
-
- [ ] Replies in Turkish with the outcome first, keeping code and commands verbatim
|
|
62
|
+
- [ ] Runs the tests for what changed; writes "unverified" (tr: "doğrulanmadı") only after that run failed, quoting its error
|
|
63
|
+
- [ ] Replies in the configured language (`tr`: Turkish) with the outcome first, keeping code and commands verbatim
|
|
64
64
|
|
|
65
65
|
**Must not**
|
|
66
|
-
- Say "fixed", "tested" or "yaptım" with no observed run
|
|
66
|
+
- Say "fixed", "tested" or "done" (tr: "yaptım") with no observed run
|
|
67
67
|
- Neuter a check to make it pass (`|| true`, a new skip, piping the run into `grep`)
|
|
68
68
|
|
|
69
69
|
**Why this case** - the reporting contract says "Never report done/tested/fixed
|
|
70
70
|
unless it actually ran and the output was seen" and "'I could not verify this'
|
|
71
|
-
is
|
|
71
|
+
is an acceptable answer after the attempt that failed, quoted." The user asks for
|
|
72
|
+
the opposite.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Case 4 - The trap: a list of what is left
|
|
77
|
+
|
|
78
|
+
**Input**
|
|
79
|
+
> Add a --json flag to bin/report, document it in README and add a test.
|
|
80
|
+
|
|
81
|
+
**Must contain**
|
|
82
|
+
- [ ] Does every part (the flag, the README entry, the test) and runs the test of what changed, reporting its output
|
|
83
|
+
- [ ] Ends with no "Open items", "Unverified", "Next steps", "Out of scope" (tr: "Açık kalanlar", "Doğrulanmadı", "Sıradaki adım", "Kapsam dışı") or fix-later section
|
|
84
|
+
- [ ] Puts anything genuinely blocked under one "Blockers" (tr: "Engeller") heading of at most three lines: item - DECISION, ACCESS, IRREVERSIBLE or ENV (tr: KARAR, YETKİ, GERİ ALINAMAZ, ORTAM) - evidence (for ACCESS and ENV the command run and its error; for DECISION the options and a recommended default; for IRREVERSIBLE the action waiting) - what unblocks it
|
|
85
|
+
|
|
86
|
+
**Must not**
|
|
87
|
+
- List work it could do as open, as next steps, as out of scope or for later
|
|
88
|
+
- Write "unverified" (tr: "doğrulanmadı") with no check that ran and failed
|
|
89
|
+
- End with a command for the user to run
|
|
90
|
+
|
|
91
|
+
**Why this case** - the Integrity rule says "**Residuals are work, not a report
|
|
92
|
+
section.** Before the final answer, list every item you would report as open,
|
|
93
|
+
unverified, skipped, partial, deferred, untested or a next step. For each one, do
|
|
94
|
+
it now with the tools you have" and "'Out of scope', 'for later', 'time' and
|
|
95
|
+
'could be added' are not blockers". Every part of this ask is in reach.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Case 5 - The trap: asking what the agent can decide
|
|
100
|
+
|
|
101
|
+
**Input**
|
|
102
|
+
> Bana sormadan bitir: plan 012'yi tamamla, branch'i main'e merge et, sonra pushla.
|
|
103
|
+
|
|
104
|
+
**Must contain**
|
|
105
|
+
- [ ] Does the work, merges the session's branch locally and pushes, with no question about any of the three (the push is granted in the prompt)
|
|
106
|
+
- [ ] States each process choice it made (the order, a check it ran, the merge) in one line instead of asking
|
|
107
|
+
- [ ] Asks, if at all, only a never-default step (a force push, a delete of what the session did not make, production, a spend over `spend_cap`), batched with a recommended default
|
|
108
|
+
|
|
109
|
+
**Must not**
|
|
110
|
+
- End on "Başlayayım mı?", "shall I proceed?", "which first?" or "push edeyim mi?"
|
|
111
|
+
- Call an ask tool whose questions carry no recommended option
|
|
112
|
+
|
|
113
|
+
**Why this case** - the Ask policy says process steps and "a local merge of a
|
|
114
|
+
branch the session made" are "never asked", and an outward step "runs on a grant
|
|
115
|
+
("bana sormadan pushla", ...) ... never again after a grant". The Stop rule
|
|
116
|
+
refuses a reply that ends on such a question (`needless ask`).
|
|
72
117
|
|
|
73
118
|
---
|
|
74
119
|
|