nukadoko 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3375431e3b4b0c689e3a373ec2499db0376b35e1e2e9d0afadf663376e3a7517
4
- data.tar.gz: 185c957dbb5f4217848c3ca69a733a215d4650147ad16fcd62f88eb0fb082c53
3
+ metadata.gz: 8e53cd82516de0c5338bbe7bf875e534dbdffec55bef983e17e0b4e5c42e7e5b
4
+ data.tar.gz: ee731ea548a72f78147c3f9a81b81728eb99b69040467c2ec00397c02137b799
5
5
  SHA512:
6
- metadata.gz: 55c28401e1fe5cfad034aa378ee47ee43ba89e839424ec14de0b0151adbba6978e0022b20b563f01e956f5b34b72dcbdf2bb28a10c83f649ad8a206294a7af32
7
- data.tar.gz: d8a0896d47e907f885cbd94120d84664ce86124a9afe3817ac572d5046871f3ef9e276d42d513bf55bcdad6fe702fa4f9a310f3f99d1e7fcef3902fa469ce8b4
6
+ metadata.gz: 4724272cb7bd2529912b710f5b0ce67e161a8cee08aafd82f849ef8d3e7b8d57e6ec9b715659889ca5dae623a85a7065b0c9c129dd0630c33654b620a25b5498
7
+ data.tar.gz: 1bddca443dbabb8b0d4b0f69b5e99245c54b5c2278323a7ccde3d5420ddcd1032ba67d789f571e81d4d164a6cd1e9220dfa608ec2f8c97675876d33bd15dd086
data/CHANGELOG.md CHANGED
@@ -6,6 +6,18 @@ notes come from the section below whose heading starts with `## [<version>]`.
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.6.0] - 2026-11-17
10
+
11
+ ### Added
12
+
13
+ - `nuka verdict --trailer` prints one git trailer line,
14
+ `Nukadoko-Verdict: <passed|failed|stale|none|unknown> run <id> tree <id>`,
15
+ to stamp a commit, and exits 0 whatever it says. `--json` carries the
16
+ same line as `trailer`. The same as the TypeScript engine.
17
+ - Under `--base`, each judgment a read cites carries `in_base`, and a row
18
+ that stands on a judgment the change added says so in its `judgment`
19
+ limit. The same as the TypeScript engine.
20
+
9
21
  ## [0.5.0] - 2026-11-16
10
22
 
11
23
  ### Added
data/README.ja.md CHANGED
@@ -68,8 +68,9 @@ seed が同じなら、TypeScript 版と同じ順番です。
68
68
  `--concurrency` は、feature ファイルを丸ごと、その数の worker プロセスに分けて実行します。
69
69
  Feature の行に `@nukadoko:serial` があるファイルは、ほかのファイルのあとに 1 つずつ実行します。
70
70
  `--repeat` は選んだシナリオをその回数だけ実行し、`--proof-runs` は control を何回実行するかを決め、`--records` はシナリオの記録を終わるたびに stdout に書きます。
71
- - `nuka verdict [--json]` は、いちばん新しい run が今の作業ツリーを判定し、赤を残さなかったかを答えます。
71
+ - `nuka verdict [--json] [--trailer]` は、いちばん新しい run が今の作業ツリーを判定し、赤を残さなかったかを答えます。
72
72
  run は始めません。
73
+ `--trailer` は、その答えでコミットに印を付ける git の trailer を 1 行出し、終了コード 0 で終わります。
73
74
  - `nuka judge <feature>:<line> [--run <id>] [--json]` は、Then が判断を求める基準(`judged-as` の claim)について、run が読んだ値を見せます。
74
75
  `--meets` か `--misses` に `--by <who>` と `--because <一文>` を付けると、その値の判断を `judgments/<feature>.json` に記録します。
75
76
  次の run は、この記録と照らして基準を読みます。
data/README.md CHANGED
@@ -50,7 +50,7 @@ end
50
50
  - `nuka init` writes `nukadoko.config.rb`, the empty `features/steps/`, and the `.gitignore` entries for `.nukadoko/` and `.env` files. A Rakefile with no `scenarios` task gets one that runs `bundle exec nuka run`, so people and CI run `rake scenarios`. It refuses when the config file exists.
51
51
  - `nuka check [--json]` reports what is wrong before anything runs.
52
52
  - `nuka run [<feature[:line]|dir>...] [--tag <expression>]... [--env <name>]... [--order file|shuffled[:<seed>]] [--repeat <n>] [--concurrency <n>] [--proof-runs <n>] [--step] [--records] [--json] [--no-check]` starts the program in each selected world, runs the selected scenarios, and writes a record per step under `.nukadoko/runs/`. With no target it runs `features/`. `--order shuffled` plays the files, then each file's scenarios, in an order a seed names, the same order the TypeScript engine plays for that seed. `--repeat` plays the selection that many times, `--concurrency` spreads whole feature files over that many worker processes (a file tagged `@nukadoko:serial` on its Feature line runs alone, after them), `--proof-runs` sets how many times each control runs, and `--records` writes each scenario record to stdout as it ends.
53
- - `nuka verdict [--json]` says whether the newest run judged the working tree as it is now and left nothing red. It never starts a run.
53
+ - `nuka verdict [--json] [--trailer]` says whether the newest run judged the working tree as it is now and left nothing red. It never starts a run. `--trailer` prints one git trailer line that stamps a commit with that answer, and exits 0.
54
54
  - `nuka judge <feature>:<line> [--run <id>] [--json]` shows the values a run read for a criterion whose Then asks for a judgment (a `judged-as` claim). With `--meets` or `--misses`, `--by <who>`, and `--because <one sentence>` it records a judgment of them in `judgments/<feature>.json`, which the next run reads the criterion against.
55
55
  - `nuka report [run] --format markdown|messages|allure [--out <path>] [--json]` renders a run again from its run directory alone, the newest run when none is named. `markdown` is its `summary.md`. `messages` is a cucumber-messages stream, one envelope per line, so a formatter or a JUnit step that reads cucumber's protocol keeps working: a claim step's claims and each criterion's verdict ride as attachments. `allure` is a directory of Allure results, `.nukadoko/export/allure-results` unless `--out` names another: one result per execution, each claim a child step of its Then, with the history id the TypeScript engine's results of the same scenario carry.
56
56
  - `nuka run <feature>:<line> --step` walks one scenario a step at a time: after each step it waits until `nuka step` answers. `nuka step wait [--walk <n>] [--timeout <s>] [--json]` prints where the walk stands: the step it paused after, its record, and a screenshot and an ARIA snapshot of the page it left open. `nuka step next <pause> --walk <n>` runs the next step, and `nuka step stop <pause> --walk <n>` skips the rest, which leaves the scenario stopped and its row red. Either engine's `nuka step` answers either engine's walk.
data/lib/nukadoko/cli.rb CHANGED
@@ -5,7 +5,7 @@
5
5
  # TypeScript engine prints, so an agent reads both engines the same way.
6
6
  module Nukadoko
7
7
  class CLI
8
- FLAGS = %w[--json --no-check --dry-run --then --replace --allure --records --meets --misses --step].freeze
8
+ FLAGS = %w[--json --no-check --dry-run --then --replace --allure --records --meets --misses --step --trailer].freeze
9
9
  REPEATED = %w[--tag --env --use].freeze
10
10
  VALUED = %w[--format --out --base --repeat --concurrency --order --proof-runs --args --pattern --description --feature --command --request --from --cwd --body --argv
11
11
  --run --by --because --walk --timeout].freeze
@@ -44,7 +44,8 @@ module Nukadoko
44
44
  alone
45
45
  nuka verdict say whether the newest run judged the working tree as
46
46
  it is now and left nothing red; exits 1 when it did
47
- not, and never starts a run
47
+ not, and never starts a run; --trailer prints one git
48
+ trailer line to stamp a commit, and exits 0
48
49
  nuka judge <target> show the values a run read for a criterion whose Then
49
50
  asks for a judgment (<feature file>:<scenario line>),
50
51
  or record a judgment of them with --meets or --misses
@@ -761,13 +762,18 @@ module Nukadoko
761
762
  #: () -> Integer
762
763
  def verdict
763
764
  answer = Engine::Verdict.new(@project)
765
+ # --trailer observes: a stamp never blocks, so only the exit code
766
+ # moves, and the envelope still says what the verdict was.
767
+ trailer = @options.key?("--trailer")
764
768
  if @json
765
769
  emit({ **head("verdict", answer.open? ? "failed" : "passed", answer.findings), **answer.envelope_fields }, answer.findings, "verdict")
770
+ elsif trailer
771
+ @out.puts answer.trailer
766
772
  else
767
773
  @out.puts answer.headline
768
774
  emit({}, answer.findings, "verdict")
769
775
  end
770
- answer.open? ? 1 : 0
776
+ answer.open? && !trailer ? 1 : 0
771
777
  end
772
778
 
773
779
  # What check says about the env files; a file it cannot read, or a key
@@ -73,6 +73,27 @@ module Nukadoko
73
73
  name.delete_prefix(prefix.empty? ? "" : "#{prefix}/") if now != before&.b
74
74
  end.sort
75
75
  end
76
+
77
+ # What `commit` holds as the judgments of a feature file: its
78
+ # entries, none when it holds no such file, or why the file did not
79
+ # parse. nil when git does not answer.
80
+ #: (String, String, String) -> Hash[Symbol, untyped]?
81
+ def at_commit(project, commit, feature)
82
+ prefix, ok = Base.git(project, "rev-parse", "--show-prefix")
83
+ return nil unless ok
84
+
85
+ prefix = prefix.delete_suffix("/")
86
+ name = prefix.empty? ? of(feature) : "#{prefix}/#{of(feature)}"
87
+ top, ok = Base.git(project, "rev-parse", "--show-toplevel")
88
+ return nil unless ok
89
+
90
+ listed, ok = Base.git(top, "ls-tree", "-z", "--name-only", commit, "--", name, strip: false)
91
+ return nil unless ok
92
+ return { entries: [] } unless listed.split("\0").include?(name)
93
+
94
+ text, ok = Base.git(top, "show", "#{commit}:#{name}", strip: false)
95
+ ok ? Judgments.parse_file(text.dup.force_encoding("UTF-8")) : nil
96
+ end
76
97
  end
77
98
  end
78
99
  end
@@ -184,6 +184,36 @@ module Nukadoko
184
184
  end
185
185
  end
186
186
 
187
+ # Under --base, each judgment a read cites gains `in_base`: true when
188
+ # the base's file already held that very judgment (the same claim,
189
+ # value, verdict, judge, reason, and time), false when the change
190
+ # brought it. A change cannot be trusted to judge its own output any
191
+ # more than to rewrite its own Thens, and the old world covers only
192
+ # the Thens. `base_entries_of` answers nil for a feature whose base
193
+ # file could not be read, and its reads stay unmarked.
194
+ #: (Array[Hash[Symbol, untyped]], ^(String) -> Array[Hash[Symbol, untyped]]?) -> Array[Hash[Symbol, untyped]]
195
+ def with_base_judgments(rows, base_entries_of)
196
+ rows.map do |row|
197
+ judgment = row[:judgment]
198
+ next row unless judgment && judgment[:reads].any? { _1[:judgment] }
199
+
200
+ entries = base_entries_of.(row[:feature])
201
+ next row unless entries
202
+
203
+ reads = judgment[:reads].map do |read|
204
+ cited = read[:judgment]
205
+ next read unless cited
206
+
207
+ held = entries.any? do |entry|
208
+ entry[:criterion] == row[:name] && entry[:then] == read[:step] && entry[:claim] == read[:claim] &&
209
+ %i[value_digest verdict by because at].all? { entry[_1] == cited[_1] }
210
+ end
211
+ { **read, judgment: { **cited, in_base: held } }
212
+ end
213
+ { **row, judgment: { **judgment, reads: reads } }
214
+ end
215
+ end
216
+
187
217
  #: (Array[Hash[Symbol, untyped]]) -> bool
188
218
  def open?(rows) = rows.any? { _1[:judgment] && _1[:judgment][:state] != "judged" }
189
219
 
data/lib/nukadoko/run.rb CHANGED
@@ -360,6 +360,14 @@ module Nukadoko
360
360
  if @base && changed.nil?
361
361
  warnings << { severity: "warning", code: "judgments-not-compared", message: "the judgment files were not compared with the base: git did not answer" }
362
362
  end
363
+ # And per row, which judgment the change brought: a row that stands
364
+ # on a judgment the change made of its own output.
365
+ if (base = @base) && changed
366
+ at_base = {} #: Hash[String, Array[Hash[Symbol, untyped]]?]
367
+ rows = Judgments.with_base_judgments(rows, lambda do |feature|
368
+ at_base.fetch(feature) { at_base[feature] = JudgmentFile.at_commit(@project, base.commit, feature)&.dig(:entries) }
369
+ end)
370
+ end
363
371
  judged += Judgments.findings(rows, @id) + problems
364
372
  { run_id: @id, run_dir: File.join(".nukadoko/runs", @id), started_at: @started_at, finished_at: now, commit: @commit,
365
373
  environments: @environments.map { environment_entry(_1) }, worlds: @world_entries, order: @order,
@@ -451,8 +451,11 @@ module Nukadoko
451
451
 
452
452
  by = judgment[:reads].map { _1.dig(:judgment, :by) || "" }.uniq.map { code_span(_1) }.join(", ")
453
453
  aria = judgment[:reads].any? { _1[:measured] == "aria-snapshot" } ? "; it is bound to the ARIA snapshot, so a change only to how the page looks keeps it current" : ""
454
+ # Under --base, a judgment the change itself brought is the change
455
+ # judging its own output, which the reviewer reads first.
456
+ added = judgment[:reads].any? { _1.dig(:judgment, :in_base) == false } ? " that this change added" : ""
454
457
  { kind: "judgment", feature: row[:feature], line: row[:line], text: row[:name],
455
- detail: "stands on a judgment by #{by}: nukadoko checked that the value is the one judged, not the judgment#{aria}" }
458
+ detail: "stands on a judgment by #{by}#{added}: nukadoko checked that the value is the one judged, not the judgment#{aria}" }
456
459
  end
457
460
  end
458
461
 
@@ -17,6 +17,8 @@ module Nukadoko
17
17
  "working-tree-unknown" => "No change is necessary. In a git repository with a commit, nuka verdict compares the tree with the one the run judged."
18
18
  }.freeze
19
19
 
20
+ TRAILER_KEY = "Nukadoko-Verdict" #: String
21
+
20
22
  attr_reader :judged #: String
21
23
  attr_reader :findings #: Array[Hash[Symbol, untyped]]
22
24
  attr_reader :newest #: Hash[Symbol, untyped]?
@@ -51,7 +53,22 @@ module Nukadoko
51
53
  #: () -> Hash[Symbol, untyped]
52
54
  def envelope_fields
53
55
  { judged: judged, working_tree: working_tree, run_id: newest&.fetch(:run_id), run_dir: newest&.fetch(:run_dir, nil),
54
- run_working_tree: newest && recorded, base: newest&.dig(:table, :base) }
56
+ run_working_tree: newest && recorded, base: newest&.dig(:table, :base), trailer: trailer }
57
+ end
58
+
59
+ # One git trailer line that stamps a commit with this answer. The tree
60
+ # named is the one the run recorded, so a reader compares it with the
61
+ # commit's own tree later and knows whether the stamp speaks for it.
62
+ #: () -> String
63
+ def trailer
64
+ run = newest or return "#{TRAILER_KEY}: none"
65
+ word = case judged
66
+ when "stale" then "stale"
67
+ when "unknown" then "unknown"
68
+ else open? ? "failed" : "passed"
69
+ end
70
+ tree = recorded
71
+ "#{TRAILER_KEY}: #{word} run #{run[:run_id]}#{" tree #{tree}" if tree}"
55
72
  end
56
73
 
57
74
  #: () -> String
@@ -2,5 +2,5 @@
2
2
  # rbs_inline: enabled
3
3
 
4
4
  module Nukadoko
5
- VERSION = "0.5.0"
5
+ VERSION = "0.6.0"
6
6
  end
data/llms.txt CHANGED
@@ -51,7 +51,7 @@ Run them in this order. Exit 0 is the only pass.
51
51
  - `nuka check [feature..] [--json] [--fail-on <code>] [--env <name>]...`: static checks for the files `nuka run` would select (a file, several, a directory, or every feature with no target), and the program's placeholders for the worlds a `nuka run` with the same `--env` flags would start. Findings are errors, warnings, and notes; a note is rot rather than breakage (an unbound pattern, an unused `from` or fixture, a field with no `.describe()`), and only a check with no target reports notes, because they are facts about every feature the project scans. Exit 1 on any error, or on a code `--fail-on` names (one value per flag, repeatable, or comma-separated; the finding keeps its severity, and `--json` lists the hits in `fail_on_hits`; a note code with targets, which compute no notes, is refused with `fail-on-note-with-targets`); warnings and notes alone exit 0. `nuka check --codes` prints every code `check`, `run`, and a refusal can carry; the table under "Findings" is that list. Condition tags are read against the environments `--env` names, as `nuka run --env` does, or without it the ones a run without `--env` selects: `condition-tag-malformed`, `condition-unknown` (a condition no environment declares), and `condition-unmet` (a scenario no selected environment meets) are errors.
52
52
  - `nuka run [feature..] [--no-check] [--env <name>]... [--tag <expression>]... [--concurrency <n>] [--repeat <n>] [--order file|shuffled[:<seed>]] [--proof-runs <n>] [--base <git ref>] [--step] [--records] [--json]`: runs `check` first for the selected scenarios and refuses on an error (`--no-check` runs anyway). `<file>:<line>` runs one scenario. Progress, failures, and the paths it wrote go to stderr (none of them under `--json`); each step's line says `passed`, `failed`, or `skipped`, the word its step record uses. stdout stays empty unless asked: `--records` writes each scenario's `run.json` entry there as one JSON line when it finishes (a step's `program_log_late` is only in `run.json`, written when the world stops), and `--json` prints `run.json` itself, which is the envelope: `run_dir` (the run directory, relative to the project root, the argument `nuka report` takes), `commit` (`sha`, `dirty`, and `working_tree`, the working tree as one git tree id: tracked files as on disk and untracked files `.gitignore` does not leave out, the state directory left out; absent when git gives none), `environments` (each `name`, `target_version`, `base_url`, its world's URL or `null`, and `conditions` when it declares them), `worlds` (each world the run started, with `name`, `tree`, `url`, `commands`, `timings`, `log`, and `error`; empty with no program), `selection` (`partial` when a `:line` target left out a scenario of its file), `order` (`{ kind: "file" }`, or `{ kind: "shuffled", seed }` under `--order shuffled`), `counts` (`scenarios`, `passed`, `failed`, and `stopped` (a walk stopped them, `--step`) of the executions of required cells, `not_run` required cells, `controls` of the executions of controls, and `controls_not_run` control cells), `table` (the verdict table: `base` (`{ ref, commit }` under `--base`, else `null`), `mapping_changed` and `mapping_changes` (`files`, `config`, `conditions`, `default_environments`, `unseen`; `null` without `--base`), `worlds`, its columns `{ world, environment }` with `world` of `head`, `base`, or `old`, `left_out`, the selected criteria `--env` left out, `proof_runs`, the `--proof-runs` a control's `runs` is measured against, and `rows`, one per selected criterion and then one per removed one, with `feature`, `line`, `name`, `change` (`null`, or under `--base` `unchanged`, `changed`, `new`, or `removed`), `renamed_from` (`{ feature, name }` at the base), `retagged` (`true` for an unchanged row whose tags differ from the base's), `conditions` (its condition tags and `@held`), `cells` (each `world`, `environment`, `role` of `required` or `control`, `conditions` (each condition its row's tags name, with the value its environment declares, `null` where none), `outcome` of `pass`, `fail`, `broke`, `stopped`, `flaky`, or `not-run`, the `line` of every one but `pass` and `not-run`, `runs`, for `fail` the `claim` every run broke and `actuals`, each run's actual value, and a `reason` for `not-run`, for a `broke` a hook caused, and for a control `broke` whose proof round could not finish), `verdict` of `held`, `proven`, `removed`, `failed`, `unproven`, or `moved`, `reason`, one sentence, and for a removed row `lines`, its sentences at the base), `flags` (only when a head step's execution events name a flag or a head world's `lists.flags` list declares one: one row per `name` and `side`, a declared side no criterion evaluated included with no `criteria`, each with `listed_in` (each environment whose list declares the side, empty for one no list declares) whenever a flags list was read, and `only_side_evaluated: true` on a flag's one side when the criteria evaluated no other, the side as its lines name it and never as an environment's conditions declare it, with `criteria`, each criterion that evaluated it there with its row's `verdict` and its `cells` on that side (`environment`, `role`, `outcome`, `line`, and `gates`, each gate its lines named or `null`), and `counts` by verdict, where a criterion seen on the side only in controls counts as `controls`; evidence to join with the program's own record of the flag by `name` and `side`, not a verdict on the flag; the summary gives each row one line under "Flags"), `routes` (only when a head world read a `lists.routes` list: one entry per `method` and `route` a list or a head step's `request` line names, `null` for the requests the program matched to no route, ordered by route and then method, with `handlers`, `listed_in` (each environment whose list holds it; empty for a route no list names), `requests` (every `request` line of a head step naming it, a late one included, each line in exactly one entry; 0 for one no step requested), `statuses`, and `criteria` and `counts` by the rules of `flags`, each cell with its `requests` and `statuses`; a request counts the same in a Given as in a Then and says the program matched the route, never that a claim read its answer; the summary's "Routes" names the listed routes no step requested, apart from those whose handler a request reached through another route (PUT and PATCH of one resource), the requests no list names, and each world whose list was not read), `paths` (only when a pair of sides read execution events: under `--base` each environment's base world against its head world, `between: { kind: "base" }`, base pairs first, for the rows the base ran there, which, when both programs name `{events}`, is every row whose head cell there is required and ran (head's `{events}` alone builds the base world for this even when no criterion is new or changed, so a refactor is compared), and two environments whose `conditions` differ in exactly one, `between: { kind: "condition", name, a, b }`; one entry per pair, with the sides `a` and `b` (`world`, `environment`), side `a` being the base world, or the environment where the condition is `false` or not declared, `unread` naming why nothing was compared or `null`, `counts` of `changed`, `open`, `same_so_far` (open, with every open count one range on both sides under the same gap words), `marked_only` (the only differences are on tables marked as bookkeeping or infrastructure; a criterion's `result` reads every other difference), `same`, and `not_compared`, `criteria` with each criterion's `result`, under a base pair `explained_by` on a `changed` one (`criterion_change` when it is new or changed, `null` when it is unchanged and nothing in the feature files asked for the difference), its side executions `a` and `b` (`scenario_record_id`, the cell's `outcome` and `line`), and `steps`, the steps with a difference aligned by line: `ran: "both"` with `changed` and `open` entries, each carrying the range of its count on side `a` and side `b`, and `holds` on a marked table, and `gaps`, or `ran: "a"` or `"b"` for a step only that side ran; a count a side's missing lines could still reach is `open`, never `changed`, a flag's side is the premise of a pair of environments and never compared there, while against the base a flag whose side changed is a difference, and `not_compared` names each criterion left out with a `reason`, such as `did not run at the base`; `unread` says when the base's program names no `{events}`; evidence, not a verdict; the summary lists up to 20 criteria per heading under "Changed paths", every base pair under one `base → head` heading that lists a criterion with the same lines in several environments once, counts the unchanged criteria compared (`Each of the 2 unchanged criteria compared did the same at the base and at head.` when each did), and lists the unchanged criteria that changed first, then the new or changed ones, then those that may differ, each group under a line of its own, and then, under `Bookkeeping and infrastructure writes that differ, which no sentence is about, though another reader of those tables may be:`, the differences with `holds`, which never rank a criterion), `limits` (what the run does not show, computed from the other fields of run.json: `left-out`, `check-skipped`, `control-not-proving` for a held row where no control failed at a Then, `no-claim`, `unmeasured`, `after-record`, `unrecorded-log`, and `unrecorded-events`, each with a `detail` and, where one exists, a `do` or `fix`; the summary renders it under `What this run does not show` right after the table, and stderr ends with one line of counts by kind, `limits: <n> <kind>, ... (run.json limits)`), and per scenario execution its `environment`, `role`, under `--base` the `world` and `row` of a base or old execution, and `for_paths` (`outcome`, and `line` when it did not pass) on a base execution run for `paths` alone, which is no cell, gets no proof runs, never changes the table, and whose stderr line ends `(for paths)`, and steps (each with `line`, `text`, `step`, `status`, `step_record_id`, `record`, `record_unreadable` with a `step-record-unreadable` warning when its step record could not be read back, a claim step's `claims`, `observation`, and `attempts`, and for a failure `error` with `kind` and `message` and the `used` results it read, a result over 4096 bytes of JSON cut to a preview with `truncated` and `result_bytes`), a failed scenario's `do` (a control has none), and every finding the run raised. `run.json` is written to `.nukadoko/runs/<run_id>/run.json` either way, and every path inside it (`record`, `evidence.dir`) is relative to that run directory; there is no scenario record file. The Markdown run summary, rendered from `run.json` alone, is always written beside it and appended to `$GITHUB_STEP_SUMMARY` when that is set; the run writes no Allure results or messages stream. Each failed scenario is a `scenario-failed` finding on stderr whose ` do:` line is the `nuka run <file>:<line>` that reruns it (under `--order shuffled`, which runs the feature files and then each file's scenarios in the order a seed names, with the seed on stderr before the first scenario, in `order`, and in the summary, the line names the scenarios that ran before it in its pass, then it, with `--order shuffled:<seed>`, or is the run itself under `--repeat` or `--concurrency` above 1), and a scenario a walk stopped is a `walk-stopped` one. `--step` walks one scenario with a person: see `nuka step` below. `--env` repeats: `--env a --env b` is one run with one `run_id` that runs every selected scenario in `a`, then in `b`; each environment gets its own env files, secrets, version probe, world, BeforeAll and AfterAll, and process-scope fixtures. A run without `--env` runs the environments the program's `defaultEnvironments` names, or else every environment that declares `conditions`, in the order the file lists them, and `default` when none does; any other, such as a deployment target, runs when `--env` names it. Under `--env`, a scenario no named environment meets is not run: its row is left out of the table, counted in `table.left_out`, in one line on stderr (`<n> criteria not run: their conditions hold in no selected environment`), and as the `left-out` entry of `limits`, and `nuka check` reports `condition-unmet` for the default selection only. `--env default` names the top-level environment (as run.json names it), whether or not the program defines it, so it combines with named ones. The tags `@with:<condition>`, `@with:<condition>=<value>` (exact text; `=true` and `=false` are malformed, write `@with:` or `@without:`), and `@without:<condition>` (off: `false` or not declared) on a Feature, Rule, Scenario, or Examples line make a scenario's cell `required` in an environment that meets them all and a `control` elsewhere; a scenario with none is required everywhere, and `@held` says the base already has the behaviour. `@expects:status=<500-599>`, `@expects:page-error=<name>`, and `@expects:error=<class>` on the same lines accept one failure underneath the scenario: a passed execution whose step got a 5xx response, whose page threw an error nothing caught, or whose trace holds an `error` with `handled: false`, is `broke@<that step's line>` unless a tag names that failure (`unexpected-failure` names it and spells the tag; `expects-unseen` notes a tag that matched nothing); only a pass moves, and like condition tags they are content under `--base`. A cell is `pass`, `fail@<line>` (each execution broke a claim at that line in Then position), `broke@<line>` (each stopped there before a claim was judged: a Given or When error, an undefined line, an unjudged claim; or, at the scenario's line, a hook, which the cell's `reason` names with when it ran), `flaky@<line>` (its executions disagree), `stopped@<line>` (a walk was stopped after the step at that line with `nuka step stop`), or `·` (none ran, with the `reason`). A required cell runs once, or `--repeat` times; a control that failed at a Then line runs again in the same world until it has broken the same claim there `--proof-runs` times (default 2) or a run disagrees (runs agree on the claim, by its position in the step, not on its value, which may differ), each a `proof run <k>/<n>` line on stderr; under `--concurrency` a proof round runs in a worker of its own, which runs BeforeAll and AfterAll again, and a hook that fails there, or a worker that records nothing, makes the round's control cells `broke` with the reason and a `proof-run-incomplete` warning, and the row reads as one no control proves (under `--base`, the base world runs for a new or changed row). A row is `failed` when a required cell is not `pass` or it has no required cell (only under `--no-check` without `--env`, with a `criterion-without-required-cell` finding whose `do:` runs the scenario where it is met, or `nuka check` on its file), `proven` when every required cell passed and a control failed in `--proof-runs` runs, and `held` otherwise. The exit code is 1 when a row is `failed`, `unproven`, or `moved`, or when the run failed outside every scenario: a BeforeAll or AfterAll of a pass failed (at any `--concurrency`), a head world did not start (`program-start-failed`), or a `--concurrency` worker exited before each of its scenarios had a record (`run-incomplete`); a control never makes it red. A `held` row whose control passed says the Then does not see what the switch changes, or the switch does not reach it: read it before calling the criterion covered. The stderr count line counts required cells alone, and a run with a control adds a `<n> control runs, <k> not run` line and a `criteria: <held> held, <proven> proven, <failed> failed` line under it, which every run under `--base` has, with `<unproven> unproven, <moved> moved, <removed> removed` after them. The summary opens with the table, one row per criterion and one `head <environment>` column per environment, headed with the conditions it declares (and `base <environment>` and `old <environment>` under `--base`), then one line per red row with its reason and where it stopped; a failure's `do:` names its environment. Naming one environment twice is refused. `--base <git ref>` compares each criterion with the commit the ref names: its identity is its feature file and name, and its content its steps with their effective keyword, docstrings, and tables, its Examples row, and its condition tags and `@held`, so a rename, a move, or another tag leaves it `unchanged` (with `renamed_from` when its name or file differs, and `retagged` when its tags differ) and an edit makes it `changed`; the rest are `new` or `removed`. The mapping (files under the features directory other than `.feature` files, the whole config but `stateDir` and `retention`, by its value and by the config file's own bytes, so a constant a config function closes over counts and an edit to those two fields alone does too, and each environment's `conditions` and `defaultEnvironments`) is compared as one unit. A `new` or `changed` row that passed at head also runs in the `base` world (the base's code under head's criteria and mapping, built from the base's program in `trees/base` under the run directory), where a base cell proves like a control after `--proof-runs` failures at one Then line; it is `proven` then, `unproven` (red) when nothing proves it (a compat Then never can, and the reason says so), and a row tagged `@held` is `held` when the base passes it. An `unchanged` row that passed at head runs in the `old` world (head's code under the base's feature files and mapping, in `trees/old`) when the mapping changed or the row is retagged or renamed, and is `moved` (red) when it does not pass there, or else `proven` when a control proves it, as without `--base`, and `held` when none does. Each `unproven` or `moved` row is a `criterion-unproven` or `criterion-moved` finding whose `do:` is `nuka run <file>:<line> --base <commit>`, naming the full hash the ref resolved to, which judges it again against the same commit after the ref moves. `--base` does not support yet a symbolic link at, above, or under the features directory or at the config file, a feature file outside the features directory, a submodule or a nested repository, or, for the base world alone, a workspace package linked into `node_modules` from inside the repository: the world such a shape reaches is not built, its cells are `·` with the reason, and a red row's finding has a `do:` that runs the row without `--base` and a `fix:` when an edit lets `--base` build the world; a shape that hides part of the mapping is named in `mapping_changes.unseen` and makes `mapping_changed` true. A criterion only the base holds is a `removed` row, quoted under the summary's "Removed criteria". The trees install nothing and resolve packages from head's project; a `url` function receives `tree` (`"base"` or `"head"`) and may return nothing for the base, and one that returns head's URL for the base leaves the base world not run, so it has to branch on `tree`. A tree whose cells do not run says why on stderr, `<world> <environment> not run: <reason>`. The summary opens with a line naming the base and what changed. A project outside git is refused with `base-outside-git`, a ref that names no commit with `base-ref-unknown`, a shallow clone without it with `base-shallow` (its `do` is the `git fetch`), a git failure with `base-git-failed`, and a `featuresDir` at the project root with `base-features-at-root`; `--since` is refused with `removed-flag`.
53
53
  - `nuka report [<run dir>] --format markdown|allure|messages [--out <path>] [--json]`: renders one finished run from its run directory alone, so a run directory copied off a CI machine renders too: `markdown` reads `run.json` alone, and `allure` and `messages` read `run.json`, the step records, and `sources.json`, the feature text the run parsed. With no run dir it renders the newest run. `markdown` is the run summary, on stdout or to `--out` (the file a CI step posts as a pull request comment); `allure` writes Allure 2 format results to `.nukadoko/export/allure-results/` or the directory `--out` names; `messages` is a cucumber-messages NDJSON stream on stdout or to `--out`, where a run over several environments gives each environment its own pickle, named `<scenario> (<environment>)`, and every execution of a control is left out, so a JUnit report of a green run is green; Allure keeps each control with the status it ran to, named `<scenario> (control)` with a `role` label, so a control that failed is a failed result there by design, and a CI gates on `nuka run`'s exit code. A file written under `.nukadoko/export/` is noted in the run's exports manifest, so retention removes it with the run, and rendering the same run there again replaces it; a file written anywhere else is not noted, and stays unless it sits inside a run directory that is removed. `--json` prints the envelope with `run_id`, `format`, and `written`, and needs `--out` for `markdown` and `messages`. A directory with no `run.json` is refused with `run-not-found`, and one whose `run.json` this release does not read with `run-invalid`; for `allure` and `messages`, one with no `sources.json` (a run from an earlier release) is refused with `run-sources-missing`, and one whose `sources.json` is corrupt or of another version with `run-sources-invalid`. A missing directory on the way to `--out` is created; anything else in the way (a directory where a file goes, a file where the Allure directory goes, a FIFO, a dangling symbolic link, a path under a regular file, or `allure-history.jsonl` in the export directory in any letter case) is refused with `report-out-invalid` before the run is read, and so is a markdown or messages file the file system refuses to write, with its error code; an Allure directory the file system refuses to write into ends as `report-incomplete`. A render that left out part of the run (a scenario, a step record, or an attachment it could not read) writes the rest and exits 1 with `report-incomplete`; a step whose step record is gone is never shown passed.
54
- - `nuka verdict [--json]`: before you say a change is done. It reads the newest run that ended and never starts one, and exits 1 when no run ended (`run-missing`), the newest run judged another working tree (`run-stale`: a file git counts changed after it started), it ran part of its files or was a walk (`run-partial`), or it left something red (its own failed findings, relayed as it printed them). A pass says that run judged exactly this tree and left nothing red; it does not say the change has a criterion of its own. Notes: `run-without-base`, `criteria-unchanged` (under `--base`, no criterion is new or changed), `working-tree-unknown` (outside git or before the first commit, where the verdict is the run's own). `--json` adds `judged` (`missing`, `stale`, `judged`, `unknown`), `working_tree`, `run_id`, `run_dir`, `run_working_tree`, and `base`. `skills/nukadoko/scripts/claude-code-verdict-hook.mjs` is an opt-in Claude Code hook that asks for it at the end of a turn that changed the tree.
54
+ - `nuka verdict [--json] [--trailer]`: before you say a change is done. It reads the newest run that ended and never starts one, and exits 1 when no run ended (`run-missing`), the newest run judged another working tree (`run-stale`: a file git counts changed after it started), it ran part of its files or was a walk (`run-partial`), or it left something red (its own failed findings, relayed as it printed them). A pass says that run judged exactly this tree and left nothing red; it does not say the change has a criterion of its own. Notes: `run-without-base`, `criteria-unchanged` (under `--base`, no criterion is new or changed), `working-tree-unknown` (outside git or before the first commit, where the verdict is the run's own). `--json` adds `judged` (`missing`, `stale`, `judged`, `unknown`), `working_tree`, `run_id`, `run_dir`, `run_working_tree`, `base`, and `trailer`. `--trailer` prints only that trailer, one git trailer line that stamps a commit (`Nukadoko-Verdict: passed run <run_id> tree <tree id>`; `failed`, `stale`, `none` when no run ended, or `unknown` when git gave no tree id; the tree is the one the run recorded, so compare it with `git rev-parse <commit>^{tree}`), and exits 0 whatever it says: it observes, and the plain command blocks. `skills/nukadoko/scripts/claude-code-verdict-hook.mjs` is an opt-in Claude Code hook that asks for it at the end of a turn that changed the tree.
55
55
  - `nuka judge <feature:line> [--run <run-id>] [--meets|--misses --by <name> --because <sentence>] [--json]`: when a run leaves a judgment `pending` or `stale`. A Then with a `judged-as` claim asks for a judgment no compare can give (whether a text reads as friendly); the run reads the value and judges nothing. Without a verdict flag it shows each value, the quality the sentence names, its digest, its step record, and the judged view. With `--meets` or `--misses`, `--by`, and `--because` it records one judgment per value in `judgments/<feature path>.json`, which you commit. The judge is not the agent that wrote the change. A row whose judgment is not `judged` fails the run (`judgment-pending`, `judgment-stale`, `judgment-misses`, `judgment-uncitable`); the verdict column is never changed by a judgment.
56
56
 
57
57
  Exploring one step at a time, when the path is not known yet:
@@ -34,6 +34,12 @@ module Nukadoko
34
34
  # root and sorted. nil when git does not answer.
35
35
  # : (String, String) -> Array[String]?
36
36
  def self?.changed: (String, String) -> Array[String]?
37
+
38
+ # What `commit` holds as the judgments of a feature file: its
39
+ # entries, none when it holds no such file, or why the file did not
40
+ # parse. nil when git does not answer.
41
+ # : (String, String, String) -> Hash[Symbol, untyped]?
42
+ def self?.at_commit: (String, String, String) -> Hash[Symbol, untyped]?
37
43
  end
38
44
  end
39
45
  end
@@ -89,6 +89,16 @@ module Nukadoko
89
89
  # : (Array[Hash[Symbol, untyped]], Array[Hash[Symbol, untyped]], ^(String) -> Array[Hash[Symbol, untyped]]) -> Array[Hash[Symbol, untyped]]
90
90
  def self?.with_judgments: (Array[Hash[Symbol, untyped]], Array[Hash[Symbol, untyped]], ^(String) -> Array[Hash[Symbol, untyped]]) -> Array[Hash[Symbol, untyped]]
91
91
 
92
+ # Under --base, each judgment a read cites gains `in_base`: true when
93
+ # the base's file already held that very judgment (the same claim,
94
+ # value, verdict, judge, reason, and time), false when the change
95
+ # brought it. A change cannot be trusted to judge its own output any
96
+ # more than to rewrite its own Thens, and the old world covers only
97
+ # the Thens. `base_entries_of` answers nil for a feature whose base
98
+ # file could not be read, and its reads stay unmarked.
99
+ # : (Array[Hash[Symbol, untyped]], ^(String) -> Array[Hash[Symbol, untyped]]?) -> Array[Hash[Symbol, untyped]]
100
+ def self?.with_base_judgments: (Array[Hash[Symbol, untyped]], ^(String) -> Array[Hash[Symbol, untyped]]?) -> Array[Hash[Symbol, untyped]]
101
+
92
102
  # : (Array[Hash[Symbol, untyped]]) -> bool
93
103
  def self?.open?: (Array[Hash[Symbol, untyped]]) -> bool
94
104
 
@@ -10,6 +10,8 @@ module Nukadoko
10
10
  class Verdict
11
11
  FIXES: untyped
12
12
 
13
+ TRAILER_KEY: String
14
+
13
15
  attr_reader judged: String
14
16
 
15
17
  attr_reader findings: Array[Hash[Symbol, untyped]]
@@ -32,6 +34,12 @@ module Nukadoko
32
34
  # : () -> Hash[Symbol, untyped]
33
35
  def envelope_fields: () -> Hash[Symbol, untyped]
34
36
 
37
+ # One git trailer line that stamps a commit with this answer. The tree
38
+ # named is the one the run recorded, so a reader compares it with the
39
+ # commit's own tree later and knows whether the stamp speaks for it.
40
+ # : () -> String
41
+ def trailer: () -> String
42
+
35
43
  # : () -> String
36
44
  def headline: () -> String
37
45
 
@@ -91,6 +91,28 @@ Code's hook fields as of 2.1.317 (`hook_event_name`, `session_id`, `cwd`,
91
91
  `stop_hook_active`; `decision: "block"` with `reason`); check them against
92
92
  the version you run.
93
93
 
94
+ ## Stamping each commit, before the hook blocks
95
+
96
+ A gate that blocks from its first day blocks on the cases nobody
97
+ measured. `nuka verdict --trailer` is the observing form: it prints one
98
+ git trailer line and exits 0 whatever it says, so it never stops a
99
+ commit. Add it in a `prepare-commit-msg` hook:
100
+
101
+ ```sh
102
+ git interpret-trailers --in-place --trailer "$(npx nuka verdict --trailer)" "$1"
103
+ ```
104
+
105
+ Git runs the hook at the repository's top. When the project is below it,
106
+ `cd` inside the `$(...)`, as `"$(cd app && npx nuka verdict --trailer)"`:
107
+ `$1` is a path from the top.
108
+
109
+ Every commit then carries `Nukadoko-Verdict: passed`, `failed`, `stale`,
110
+ `none` (no run ended), or `unknown` (no tree id), with the run id and the
111
+ tree the run recorded. The stamp speaks for the commit only when
112
+ `git rev-parse <commit>^{tree}` is that tree; a commit of part of the
113
+ working tree carries another. Count the commits that were not `passed`
114
+ for a while, then turn on the Stop hook above, which blocks.
115
+
94
116
  ## When the reviewer asks how risky it is
95
117
 
96
118
  A pass and a clean intent still leave the reviewer's own question: what
@@ -93,3 +93,9 @@ never `unproven` (no base can break a claim that compares nothing), and
93
93
  its judgment decides it. The summary names the judgment files the change
94
94
  brings ("A judgment file changed since the base"), so the person reviews
95
95
  each judgment with the feature.
96
+
97
+ A judgment the change itself added is the change judging its own output.
98
+ Such a row's `judgment` limit says `stands on a judgment by <name> that
99
+ this change added`, and each cited judgment in run.json carries `in_base:
100
+ false`. When the agent that wrote the change also recorded the judgment,
101
+ tell the person so, and ask a second agent or the person to judge it.
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: nukadoko
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - meganemura