samagotchi 0.155.0 → 0.157.0

data/docs/testing.md

@@ -75,6 +75,47 @@ spec/engine_turn_golden_spec.rb`, then read `git diff spec/fixtures/turn_goldens
 to `tmp/turn_goldens/<golden>.actual` and names the `diff` to run. Changes to the turn path (engine, kernel loop, chat loop,
 tool runner, gate, guardrail wiring, system prompt, tool declarations, registry) pick it in `script/affected-specs`.
 
+The model-setting characterizations (`spec/engine_effective_model_characterization_spec.rb`,
+`spec/engine_session_model_settings_characterization_spec.rb`, `spec/web/app_session_settings_characterization_spec.rb`
+over `spec/fixtures/effective_model/`, and `spec/model_setting_precedence_characterization_spec.rb`'s ladders over
+`spec/fixtures/model_setting_precedence/`) pin the settings on the code as it was before the `ModelLookup` refactor
+(`UPDATE_GOLDENS=1` regenerates them after an intended change only). They are retired after their refactor: the
+next round folds the first three into one wiring example each in `effective_model_spec.rb`,
+`session_model_settings_spec.rb` and `session_setting_spec.rb`, and the ladders the round after into the setting
+modules' and `model_lookup_spec.rb`.
+
+The session file goldens (`spec/session_file_goldens_spec.rb`, `spec/fixtures/session_goldens/`) pin the bytes `Session#save`
+writes: every field away from its default, a new session, a newer chi's file, an old chi's file in a git repo, every key
+present as `null`, the plain writers' raw values, a running turn's checkpoint and the scrub (the `*_in.json` files are
+hand-written inputs, never regenerated); and the file each path that makes or rewrites a session writes (a fork, a
+continue, a delegate child, a plain spawn, a review, a Verify fresh child, a resume, a respawn, an archive), with
+`carry_matrix.json`, what each keeps of its source. They are pinned before Session's fields became one table and are kept
+as the file format's pins, not retired: a diff is a change to every session file on disk, so only a commit that means to
+change the format regenerates them (`UPDATE_GOLDENS=1 bundle exec rspec spec/session_file_goldens_spec.rb`).
+
+`UPDATE_GOLDENS=1` is the characterization specs' variable (the prompt and turn goldens above use `UPDATE_PROMPTS`):
+the model-setting ones above, the session file goldens, claude-peer's goldens and the bundle-state corpus below. Regenerate only for an intended
+change, then read the diff.
+
+The bundle-state corpus (`spec/samagotchi/memory_bundle/file_state_corpus/`, `spec/fixtures/bundle_states/`) pins what
+every bundle file state does under every command that reads or writes it. Each case in `memory_cases.yml` (a memory:
+owned, edited, behind, missing, dropped, the user's own, another bundle's too…) and `asset_cases.yml` (a rule, the
+plugin, a script, a hook) is written directly as a starting tree, in the 0.155 and the 0.149 record shape, and run
+through install, `--force`, `--dry-run`, upgrade and its flags, chi update's plan, Remerge, the system bundle's restore,
+uninstall and status. A cell holds only what a user or the next command sees (the bytes, the trash, base and checksum,
+the mark, the summary's word, status's label, the warnings), never the Installer's result strings. `expect/*.yml` are
+generated (`UPDATE_GOLDENS=1 bundle exec rspec spec/samagotchi/memory_bundle/file_state_corpus/`, which also writes
+`tmp/bundle_states/review.<file>.txt`, every cell in words); a mismatch names the case, command and shape and writes
+`tmp/bundle_states/<file>.yml.actual` to diff. A regeneration asserts nothing, so run the folder again without
+`UPDATE_GOLDENS`: that run checks the new cells against `cases_spec.rb`, which holds what every regeneration keeps. No
+user bytes (neither the bundle's text nor what the record vouches for, its checksum first) are changed or removed without
+a copy in the trash first; the two record shapes give one answer except on a marked case (an allowlist entry needs a
+note); and for every asset, chi update's plan lists as replaced exactly the files the upgrade copies to the trash. To stay
+under 3 s, `upgrade_spec.rb` runs the 0.149 shape for marked cases only (`MARKED_ONLY_149`), where the shapes can differ. A regenerated cell is
+reviewed: a refactor that turns the corpus red stops and asks, and only a commit that means to change a cell
+regenerates, its changed cells read one by one; a cell that looks wrong today carries a `divergent:` tag and a note in
+its case.
+
 The web specs run in plain node. One that loads lit-html, itself or through a module it imports (`turn_block.js`,
 `turn_html.js`, `card.js`, `copy.js`, `images.js`, `scope.js`, `command_view.js`, `diff_view.js`, `read_view.js`,
 `question_cards.js`, `stream_handlers.js`, `stage_view.js`, `bubbles.js`, `cards_place.js`, the chips, `info_bar.js`, `info_bar_model.js`, `lists_view.js`, `context_chips.js`, `families_view.js`,
@@ -124,7 +165,15 @@ Each level owns its own share; a detail is asserted once, at the lowest level th
 - **e2e** is for real-browser behaviour (layout, focus, reload, two tabs), one happy path per Ruby↔web mechanism and
   a few journeys. Before adding a test, extend an existing journey's step; a variant goes to node, a Ruby↔web shape
   variant to a shared fixture (`spec/fixtures/composer_routes.json`). A journey names its stages with `test.step`
-  (the report names the one that failed: web.spec.js's notifications). The `lan` project's chi web is on the LAN only
+  (the report names the one that failed: web.spec.js's notifications). A turn test ends with
+  `expectSameAfterReload` (spec/e2e/support/history_shape.js): the history's shape live, then after a reload, the
+  same (live is the truth). Its allowlist is `history_shape.js ACCEPTED`, each entry with its reason; a test names
+  the entries its scenario has (only deliberate ones are left). A drawn turn's variants are node's: the shared
+  scenarios in `spec/shared/turn_drawings/` (written from `spec/support/turn_drawing_scenarios.rb`;
+  `UPDATE_TURN_DRAWINGS=1 bundle exec rspec spec/bridge/turn_drawings_spec.rb spec/web/session_show_drawings_spec.rb`
+  rewrites their lines and show after an intended change) feed `turn_drawing_parity.test.js`, which draws each one
+  live (the real stream handlers and stage) and from its drawing and compares the shapes with no allow list. The
+  `lan` project's chi web is on the LAN only
   when the machine has a private IPv4: without one its LAN tests skip and the participant review journey runs on
   loopback.
 
@@ -254,6 +303,7 @@ any other file requires it, so a spec file run alone still finds its requires. E
 | `lock_holder.rb` | *helper*: `spawn_lock_holder(session_dir)`, a stand-in worker process holding a session's owner lock, which says so on a pipe |
 | `integration_server.rb` | *helper*: the live model server `:integration` examples use, from `SAMAGOTCHI_INTEGRATION_*` ([Integration specs](#integration-specs)) |
 | `mcp_bundle.rb` | The shipped mcp bundle's path and the fake MCP server its specs start (`MCP_FAKE_COMMAND`, no RubyGems) |
+| `model_setting_ladders.rb` | shared context "model setting ladders": per setting a config, env and CLI with every layer set, peeled top down (`peel`), against `spec/fixtures/model_setting_precedence/` |
 | `no_console.rb` | Loaded first (`ruby -r`) by spawned bin/chi children: `IO.console` is nil, so output never wraps at a terminal's width |
 | `recording_surface.rb` | `RecordingSurface`: a `TerminalUI::Surface` that records commits and slots instead of drawing |
 | `remote_backend.rb` | `RemoteBackend`: `SessionAccess::Remote` against chi web's Rack app in this process (no socket, LAN token gate) |
@@ -334,7 +384,8 @@ files it changed (`lib/samagotchi/foo/bar.rb` → `spec/foo/bar_spec.rb`, `bar_*
 `lib/samagotchi/foo/bar/baz.rb` → `spec/foo/bar_baz_spec.rb`), the specs that read
 docs or source as text (its `RULES`: add one when a full run catches what the names missed) and the golden of a changed
 file's layer (any `foo/*.rb` → `spec/foo_golden_spec.rb` when there is one; `turn_flow.rb` and `turn_note.rb`, whose
-words the attached TUI prints, pick its `event_renderer` and `attached_loop` specs by a rule). Changes to `tool_declarations.rb` and `system_prompt.rb` also pick the
+words the attached TUI prints, pick its `event_renderer` and `attached_loop` specs by a rule; `memory_bundle/*.rb`,
+`bundle_command.rb`, `memory_paths.rb`, `model_overlay.rb` and the corpus's fixtures pick the bundle-state corpus). Changes to `tool_declarations.rb` and `system_prompt.rb` also pick the
 effective-model, request, prompt-snapshot and full system-prompt goldens because they hash the request's prompt and
 tools. Then come the other name matches (a spec named after several changed files first, e.g. `engine_toolkit_spec.rb` for `engine.rb` and
 `toolkit.rb`; then one beside the file, so `plugin/context.rb` picks `spec/samagotchi/plugin/*context*` before

data/lib/samagotchi/answer_command.rb

@@ -138,7 +138,8 @@ module Samagotchi
       end
       # Answered here, or no longer open (answered elsewhere, or a newer
       # question): what comes next either way.
-      wait_for_reply(id, cursor: start.cursor, baseline: start.baseline, timeout: options[:timeout], turn: start.own_turn)
+      wait_for_reply(id, target: ReplyWait::Target.answer(start),
+                         timeout: options[:timeout])
     end
 
     # The labels to send. A number is an option's place (1-based); an

data/lib/samagotchi/archive_store.rb

@@ -4,6 +4,7 @@ require "json"
 require "fileutils"
 require "time"
 require_relative "atomic_file"
+require_relative "session_paths"
 
 require_relative "session"
 
@@ -46,7 +47,7 @@ module Samagotchi
 
     # @return [Boolean] whether it was archived (and is not now)
     def self.unarchive(session_id, state_dir:)
-      return false unless archived?(Session.session_dir(session_id, state_dir: state_dir))
+      return false unless archived?(SessionPaths.session_dir(session_id, state_dir: state_dir))
 
       write(session_id, { "unarchived_at" => Time.now.iso8601(3) }, state_dir: state_dir)
     end
@@ -63,9 +64,9 @@ module Samagotchi
     end
 
     def self.write(session_id, record, state_dir:)
-      return false unless File.exist?(Session.session_file(session_id, state_dir: state_dir))
+      return false unless File.exist?(SessionPaths.session_file(session_id, state_dir: state_dir))
 
-      dir = Session.session_dir(session_id, state_dir: state_dir)
+      dir = SessionPaths.session_dir(session_id, state_dir: state_dir)
       FileUtils.mkdir_p(dir)
       path = File.join(dir, FILE)
       AtomicFile.write(path, JSON.generate(record))

data/lib/samagotchi/bootstrap/config_writer.rb

@@ -304,7 +304,7 @@ module Samagotchi
 
       def written_ok?(real, expected, name)
         parse(File.read(real)) == expected &&
-          ConfigFile.hosts_config(env: @env, path: real).key?(name.downcase)
+          HostConfig.all(env: @env, path: real).key?(name.downcase)
       rescue StandardError
         false
       end

data/lib/samagotchi/bootstrap_command.rb

@@ -3,6 +3,7 @@
 require "rbconfig"
 require_relative "config"
 require_relative "model_profile"
+require_relative "host_address"
 require_relative "host_registry"
 require_relative "host_keys"
 require_relative "chi_home"
@@ -342,7 +343,7 @@ module Samagotchi
     # The "loading the model?" hint fits a local server (a llama.cpp loading
     # weights); a remote provider's slow answer is the network, not a load.
     def loading_hint?(result)
-      !HostRegistry.remote_address?(result.candidate.scheme, result.candidate.host)
+      !HostAddress.remote_address?(result.candidate.scheme, result.candidate.host)
     end
 
     # 0 written, 1 written but not as planned, nil with @exit on a refusal.
@@ -489,7 +490,7 @@ module Samagotchi
              when :skipped then "skipped (#{result.error})"
              else "failed (#{result.error})"
              end
-      @stdout.puts("system bundle: #{text}")
+      @stdout.puts("system bundle: #{[text, *result.warnings].join("; ")}")
       result.status != :failed
     end
 

data/lib/samagotchi/bridge/card_places.rb

@@ -18,6 +18,15 @@ module Samagotchi
     # comes after (a prompt's turn_id, a line's opening_id), which the page
     # finds by its bubble, not by counting. Pure: no Engine, no file.
     class CardPlaces
+      # An opening's stamp: its turn_id (a prompt's, a wake note's), else its
+      # own opening_id (/continue no's reason line); nil for neither (a line
+      # an older chi wrote). The recap's too (RecapPlace).
+      # @return [String, nil]
+      def self.stamp(message)
+        id = message[:turn_id] || message["turn_id"] || message[:opening_id] || message["opening_id"]
+        id.is_a?(String) && !id.empty? ? id : nil
+      end
+
       # @param turn_log [Array<String>] the turn ids, oldest first
       # @param messages [Array<Hash>] the messages the page draws
       def initialize(turn_log, messages)
@@ -114,10 +123,7 @@ module Samagotchi
         position >= 0 ? @last_at_or_before[position] : nil
       end
 
-      def stamp_of(message)
-        id = message[:turn_id] || message["turn_id"] || message[:opening_id] || message["opening_id"]
-        id.is_a?(String) && !id.empty? ? id : nil
-      end
+      def stamp_of(message) = self.class.stamp(message)
 
       def idless_run(opening)
         @ids.drop(opening + 1).take_while(&:nil?).size

data/lib/samagotchi/bridge/card_store.rb

@@ -147,6 +147,9 @@ module Samagotchi
         stamp = places&.opening_of(entry)
         out[:opening_id] = stamp if stamp
         out[:current] = current
+        # Whose row it is (its turn's id, +of_turn+): the page leaves a drawn
+        # turn's own rows and cards to its drawing (Bridge::TurnDrawings).
+        out[:of_turn] = entry[:turn_id] if shown[:in_turn] && entry[:turn_id]
         out[:during] = true if entry[:during]
         out[:after_failed] = true if entry[:after_failed] && entry[:turn_id] == @failed_turn
         out[:label] = "from #{entry[:command]}" if entry[:command]

data/lib/samagotchi/bridge/turn_accumulator.rb

@@ -2,6 +2,7 @@
 
 require_relative "../events"
 require_relative "../kernel_loop"
+require_relative "../log"
 require_relative "../steer"
 require_relative "../tool_output_cut"
 require_relative "../tool_row_fields"
@@ -23,10 +24,17 @@ module Samagotchi
       #   its head and tail: a reload's cap, so a joined row is the row a
       #   reload draws
       # @param clock [#call] monotonic seconds: a tool call's duration
+      # @param on_turn_end [#call, nil] (turn, end_event): the turn's final
+      #   fold (a deep copy, #current_turn's shape without elapsed_ms) and its
+      #   end event, at the end event, before the turn is cleared (the Bridge's
+      #   TurnDrawings#append). It runs under this accumulator's mutex and the
+      #   SessionObserver's lock: it must never call back in here. An error is
+      #   logged and the fold goes on.
       def initialize(max_output_chars: ToolOutputCut::WEB_PREVIEW_CHARS,
-                     clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) })
+                     clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }, on_turn_end: nil)
         @max_output_chars = max_output_chars
         @clock = clock
+        @on_turn_end = on_turn_end
         @mutex = Mutex.new
         @turn = nil
         # When each running tool call started, by [iteration, call_index].
@@ -47,13 +55,14 @@ module Samagotchi
 
       # @return [Hash, nil] a copy of the turn in progress: prompt, turn_id, origin,
       #   continue, at (its wall time), ordered parts (generation / thinking / text / tool /
-      #   input / steer / reminder / notice), pending_question and the last event_seq folded
+      #   input / steer / reminder / notice / card), pending_question and the last event_seq folded
       #   in. A finished tool part keeps the live row's action and duration_ms
       #   too, a running one the time it has run so far (elapsed_ms), so a
       #   join times its row from its start. A notice part holds the event
       #   of one of the turn's rows (TurnNotice: a hook's notice, an
       #   empty-answer retry, a question and its answer), which a UI replays
-      #   through its live handler.
+      #   through its live handler. A card part is a card the turn showed in
+      #   its step (in_turn), its latest version in the place of its first.
       def current_turn
         @mutex.synchronize do
           next nil unless @turn
@@ -132,10 +141,13 @@ module Samagotchi
       # the live activity: action, tool, params, status), a merged_input
       # (the one synthetic type) per merged prompt, a steer-only
       # pending_input_merged per steer, a reminder_injected, and a notice
-      # part's own event.
+      # part's own event. Card parts only with +cards+ (a finished turn's
+      # drawing, chi web's unfold of it): a running turn's join gets its cards
+      # from CardStore's snapshot.
       # @param started_at [String, nil] the turn's start, on turn_started
+      # @param cards [Boolean] replay card parts as their card events
       # @return [Array<Hash>] symbol keys and types; [] with no turn
-      def self.replay_events(turn, started_at: nil)
+      def self.replay_events(turn, started_at: nil, cards: false)
         return [] unless turn
 
         started = { type: :turn_started, prompt: turn[:prompt], origin: turn[:origin], continue: !!turn[:continue] }
@@ -175,14 +187,66 @@ module Samagotchi
           when "steer"
             close_text.call
             events << { type: :pending_input_merged, count: 0, content: nil, steers: [{ source: part[:source], text: part[:text] }] }
+          when "card"
+            next unless cards && part[:event].is_a?(Hash)
+
+            close_text.call
+            events << part[:event].merge(type: :card, in_turn: true)
           end
         end
         events
       end
 
+      # The page's unfold of a session view (chi web's show sends it as
+      # `replay`; turn_events.js snapshotEvents was its JS twin): the turn in
+      # progress as #replay_events has it, its turn_started with what the
+      # page also reads (its wall time `at`, the run's `cut_after`, and every
+      # author key, a pair turn's participant_mode too), then a turn_enqueued
+      # per prompt queued behind it and a command_queued per command waiting
+      # for its end (spec/shared/turn_snapshot.json pins both sides). The TUI
+      # keeps #replay_events.
+      # @param current_turn [Hash, nil] #current_turn's shape, symbol keys
+      #   (a snapshot's: .from_wire)
+      # @param queued [Array<Hash>] #queued's entries
+      # @param queued_commands [Array<Hash>] #queued_commands' entries
+      # @return [Array<Hash>] symbol keys and types
+      def self.join_events(current_turn:, queued: [], queued_commands: [], started_at: nil)
+        events = replay_events(current_turn, started_at: started_at)
+        if (started = events.first)
+          started[:at] = current_turn[:at] if current_turn[:at].is_a?(String)
+          started[:cut_after] = current_turn[:cut_after] unless current_turn[:cut_after].nil?
+          Steer::AUTHOR_KEYS.each { |key| started.delete(key) }
+          Steer::AUTHOR_KEYS.each { |key| started[key] = current_turn[key] if current_turn[key] } if current_turn[:author]
+        end
+        Array(queued).each do |entry|
+          enqueued = { type: :turn_enqueued, enqueued_id: entry[:enqueued_id], client_id: entry[:client_id], prompt: entry[:prompt] }
+          enqueued[:delivery] = entry[:delivery] if entry[:delivery]
+          enqueued[:author] = entry[:author] if entry[:author]
+          events << with_images(enqueued, entry[:images])
+        end
+        Array(queued_commands).each do |entry|
+          command = { type: :command_queued, command_id: entry[:command_id], client_id: entry[:client_id], line: entry[:line],
+                      waits: "turn_end" }
+          command[:author] = entry[:author] if entry[:author]
+          events << command
+        end
+        events
+      end
+
+      # A Bridge snapshot's part (JSON: string keys) as the accumulator's own
+      # shapes: symbol keys all the way down, values as they are.
+      def self.from_wire(value)
+        case value
+        when Hash then value.to_h { |key, item| [key.is_a?(String) ? key.to_sym : key, from_wire(item)] }
+        when Array then value.map { |item| from_wire(item) }
+        else value
+        end
+      end
+
       def self.tool_replay_events(part)
         call = { iteration: part[:iteration], call_index: part[:call_index], tool: part[:tool] }
         call[:label] = part[:label] if part[:label]
+        call[:tool_id] = part[:tool_id] if part[:tool_id]
         # Its row fields (ToolRowFields), each in its place: title and, for
         # an alias's call, the name the model used, also in the activity.
         title = part.slice(*ToolRowFields::ACTIVITY_KEYS).compact
@@ -296,6 +360,7 @@ module Samagotchi
 
         fold_turn_event(event)
         if Events::TURN_END.include?(type)
+          turn_ended(event)
           @turn = nil
           @merged_origins = []
           @working_set = nil
@@ -326,6 +391,8 @@ module Samagotchi
           part = { kind: "tool", iteration: event[:iteration], call_index: event[:call_index],
                    tool: event[:tool], params: event[:params], status: "running" }
           part[:label] = event[:label] if event[:label]
+          # The id its output is saved under (ToolIds; not a row field).
+          part[:tool_id] = event[:tool_id] if event[:tool_id]
           ToolRowFields::KEYS.each { |key| part[key] = event[key] if event[key] }
           parts << part
           @tool_started_at[[event[:iteration], event[:call_index]]] = @clock.call
@@ -342,16 +409,26 @@ module Samagotchi
           # Only System 1's auto-allow is drawn on the row (the web's label).
           guardrail = event.dig(:activity, :guardrail)
           tool[:guardrail] = guardrail if guardrail.is_a?(Hash) && guardrail[:decided_by] == "system one"
-          tool[:output] = capped ? ToolOutputCut.preview(output, cap: @max_output_chars, tool: event[:tool]) : output.dup
+          # The event's web preview (ToolRunner: the reload's cut) when it
+          # has one; an older worker's event is cut here the same way.
+          tool[:output] = if event[:preview] then event[:preview].to_s.dup
+                          elsif capped then ToolOutputCut.preview(output, cap: @max_output_chars, tool: event[:tool])
+                          else output.dup
+                          end
           tool[:output_truncated] = capped || !!event[:output_truncated]
           tool[:images] = event[:images] if event[:images]
           tool[:diff] = event[:diff] if event[:diff]
           action = event.dig(:activity, :action)
           tool[:action] = action.to_s if action
-          # As the live row times it (EventRenderer): from tool_call_started,
-          # less an approval wait.
+          # The event's (ToolRunner's: the figure every UI shows); without
+          # one (an older worker's event), as the live row times it: from
+          # tool_call_started, less an approval wait.
           started_at = @tool_started_at.delete([event[:iteration], event[:call_index]])
-          tool[:duration_ms] = [((@clock.call - started_at) * 1000) - event[:waited_ms].to_f, 0].max.round if started_at
+          if event[:duration_ms].is_a?(Integer)
+            tool[:duration_ms] = event[:duration_ms]
+          elsif started_at
+            tool[:duration_ms] = [((@clock.call - started_at) * 1000) - event[:waited_ms].to_f, 0].max.round
+          end
         when :pending_input_merged
           # A steer-only merge (count 0) has no user part: the origins stay
           # for the user lines' own merge.
@@ -366,6 +443,8 @@ module Samagotchi
           end
         when :working_set_updated
           @working_set = { items: event[:items], at: event[:at] }
+        when :card
+          fold_card(parts, event) if event[:in_turn]
         when :reminder_injected
           parts << { kind: "reminder", reminders: Array(event[:reminders]).map(&:dup) }
         when :question_requested
@@ -382,6 +461,27 @@ module Samagotchi
         parts << { kind: "notice", event: TurnNotice.slice(event) } if TurnNotice.notice?(event)
       end
 
+      # The card fields a UI draws (CardStore keeps the same), a copy.
+      CARD_FIELDS = %i[id source title body level actions].freeze
+      private_constant :CARD_FIELDS
+
+      # A card the turn shows in its step; one with an earlier card's id
+      # replaces that part where it was.
+      def fold_card(parts, event)
+        card = { kind: "card", id: event[:id], event: Marshal.load(Marshal.dump(event.slice(*CARD_FIELDS))) }
+        index = parts.index { |part| part[:kind] == "card" && part[:id] == card[:id] }
+        index ? parts[index] = card : parts << card
+      end
+
+      # The turn's final fold to the hook (#initialize's on_turn_end).
+      def turn_ended(event)
+        return unless @on_turn_end
+
+        @on_turn_end.call(Marshal.load(Marshal.dump(@turn)), event)
+      rescue StandardError => e
+        Log.warn(:bridge, "turn_end_hook_failed", error: "#{e.class}: #{e.message}")
+      end
+
       def append_text(iteration, kind, text)
         return if text.nil? || text.empty?