@zalom/plastic 1.0.0-beta.17 → 1.0.0-beta.18

package/PLASTIC.md

@@ -256,20 +256,29 @@ hygiene after each intent. Advisory self-check, not hard-verifiable.
 ## Retrieval Gate
 
 A single capability-aware PreToolUse gate enforces retrieval-first routing on the agent's own
-Bash/Read/Grep/Glob calls (and on subagents, since PreToolUse binds them). Detection is binary:
-present means enforce, absent or down means off, with no warning and no advisory tier.
-
-- Store markdown (under a Plastic store) routes to QMD when QMD is present and the index is
-  fresh: the raw grep/find/Read is blocked and you use `qmd search`/`qmd query` (or
-  `scripts/qmd-sync search`) instead. When QMD is present but stale, the read is allowed this
-  turn and a background reindex is fired so the next turn enforces against a fresh index;
-  reindex is never synchronous. When QMD is absent or down, raw reads are allowed.
-- Serena-supported code and data files route to Serena symbolic tools when Serena is present;
-  absent means allowed.
-- Images, binaries, and everything else are allowed.
-- Bypass: append a trailing `# qmd-ok` shell comment to a Bash command for the rare case where
-  QMD is healthy but you genuinely need the raw read. A quoted or echoed occurrence does not
-  bypass. Bypasses are logged.
+Bash/Read/Grep/Glob calls (and on subagents, since PreToolUse binds them). The gate is
+OPERATION-based: it separates searching from reading, and it never stands between you and
+reading something you have already located.
+
+- Only CONTENT SEARCH over a Plastic store is gated. The Grep tool and bash `grep`/`rg`/`ag`
+  whose target is at or under a store route to QMD when QMD is present and the index is fresh:
+  the raw scan is blocked and you use `qmd search`/`qmd query` (or `scripts/qmd-sync search`)
+  instead. When QMD is present but stale, the search is allowed this turn and a background
+  reindex is fired so the next turn enforces against a fresh index; reindex is never
+  synchronous. When QMD is absent, the search is allowed.
+- Reading a known target (the Read tool, bash `cat`/`head`/`tail`) and structural discovery
+  (the Glob tool, bash `find`/`ls`) are always allowed, including over the store. QMD cannot
+  list directories or hand back one specific file, so these are never gated.
+- Code is never hard-gated here. Symbolic code navigation via Serena is a soft prompt mandate
+  (the UserPromptSubmit power-tools hook), not a block: content grep over code is allowed,
+  because Serena navigates symbols and cannot grep arbitrary strings.
+- QMD failure model. Absent or stale degrades to allow (stale also fires the background
+  reindex). A broken QMD, where the freshness probe errors or times out, also fails open, and
+  the hook emits a one-line warning so a degraded QMD is visible rather than silent.
+- Bypass: append a trailing `# qmd-ok` shell comment to a Bash command when you attempted
+  discovery and it did not serve you (no hits, or results that do not answer your need by your
+  reading of the snippets, not their score). A quoted or echoed occurrence does not bypass.
+  Bypasses are logged. The gate enforces that discovery was attempted, never that it succeeded.
 - Scope: only the agent's tool calls. Ruby `File.read` inside a script is invisible to the gate
   and is out of scope by design.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@zalom/plastic",
-  "version": "1.0.0-beta.17",
+  "version": "1.0.0-beta.18",
   "description": "Intent-driven idea development system for AI coding agents",
   "type": "module",
   "bin": {

package/scripts/hook-retrieval-gate

@@ -2,14 +2,19 @@
 # encoding: UTF-8
 # frozen_string_literal: true
 
-# PreToolUse retrieval gate (intent 84, Lever 2). Reads the tool call (JSON on
-# stdin: tool_name + tool_input), computes capabilities (QMD detect+freshness,
-# Serena detect), delegates the decision to RetrievalGate, and enforces:
+# PreToolUse retrieval gate (intent 84, Lever 2; operation-based redesign 89a).
+# Reads the tool call (JSON on stdin: tool_name + tool_input), computes
+# capabilities (QMD detect + freshness), delegates the decision to RetrievalGate,
+# and enforces:
 #   ALLOW = exit 0 ; BLOCK = exit 2 with reason on stderr (shown to the agent).
 # Fail-open: any parse error, timeout, or unexpected exception exits 0. On the
 # STALE QMD path RetrievalGate fires QmdSync.reindex_async (NEVER synchronous).
 # Binds subagents (PreToolUse hooks apply to subagent tool calls too).
 #
+# Only CONTENT SEARCH over store markdown is gated; reads and structural ops are
+# allowed. Code navigation is a soft prompt mandate (UserPromptSubmit power-tools),
+# not enforced here, so this hook no longer detects Serena.
+#
 # Scope: only the agent's own Bash/Read/Grep/Glob calls. Ruby `File.read` inside
 # scripts is invisible to a PreToolUse hook and is out of scope (no exemptions).
 #
@@ -19,15 +24,14 @@ require "json"
 require "timeout"
 require_relative "lib/retrieval_gate"
 require_relative "lib/qmd_sync"
-require_relative "lib/power_tools"
 
 module RetrievalGateHook
   module_function
 
   # Pure-ish core: capabilities and reindex are injected so this is unit-testable
-  # with no real qmd/serena. Returns [exit_code, stderr_string].
+  # with no real qmd. Returns [exit_code, stderr_string].
   #   stdin: raw PreToolUse JSON
-  #   capabilities: { qmd:, qmd_fresh:, serena: }
+  #   capabilities: { qmd:, qmd_fresh: }
   #   reindex: callable fired on the STALE path
   def run(stdin:, plastic_home:, cwd:, capabilities:, reindex: -> {})
     payload = parse(stdin)
@@ -62,23 +66,33 @@ module RetrievalGateHook
     nil
   end
 
-  # Detect real capabilities for the live executable. A slow `qmd status` cannot
-  # stall a tool call: a Timeout around the freshness probe degrades to "absent
-  # for this turn" (allow, no reindex), staying fail-open and non-blocking.
-  def detect_capabilities(cwd:)
-    qmd = QmdSync.detect
+  # Detect real capabilities for the live executable. Probes are injected so the
+  # three-tier QMD failure model is unit-testable:
+  #   - absent  : QMD not on PATH                 -> allow, no warn.
+  #   - broken  : QMD present but freshness probe  -> tier-b: WARN once, then treat
+  #               raises/times out                   as absent this turn (allow).
+  #   - fresh   : probe returns truthy            -> gate is live.
+  # A slow `qmd status` cannot stall a tool call: a 2s Timeout bounds the probe.
+  def detect_capabilities(cwd:,
+                          detect: -> { QmdSync.detect },
+                          fresh: -> { QmdSync.fresh? },
+                          warn: ->(m) { $stderr.puts(m) })
+    qmd = detect.call
     qmd_fresh = false
     if qmd
-      qmd_fresh = begin
-        Timeout.timeout(2) { QmdSync.fresh? }
+      begin
+        qmd_fresh = Timeout.timeout(2) { fresh.call }
       rescue StandardError
-        # Probe stalled/failed: treat as absent this turn (allow, no reindex).
+        # Tier-b: QMD is present but its freshness probe broke/stalled. Distinct
+        # from QMD being absent — warn so a degraded QMD is visible, then fail open
+        # (allow this turn, no reindex).
+        warn.call("PLASTIC GATE — QMD is present but its freshness probe failed; " \
+                  "allowing this turn without routing search to QMD (check qmd).")
         qmd = false
-        false
+        qmd_fresh = false
       end
     end
-    serena = PowerTools.serena?(cwd: cwd)
-    { qmd: qmd, qmd_fresh: qmd_fresh, serena: serena }
+    { qmd: qmd, qmd_fresh: qmd_fresh }
   end
 
   # Best-effort reindex callable for the STALE path. Resolves the collection from

package/scripts/lib/retrieval_gate.rb

@@ -3,62 +3,57 @@
 
 require_relative "bridge"
 
-# RetrievalGate — the single, pure decision for Lever 2 of intent 84.
+# RetrievalGate — the single, pure decision for Lever 2 of intent 84, redesigned
+# operation-based in intent 89a.
 #
 # Given an agent tool call (Bash/Read/Grep/Glob) and injected capability signals,
-# it decides whether to BLOCK the call (returning a redirect-to-QMD/Serena reason
-# String) or ALLOW it (returning nil). All capability and freshness signals are
-# injected by the caller (the hook); this module shells out to nothing, reads no
-# globals, and runs no binaries. Mirrors bridge.rb's decision-fn convention
-# (reason String to block, nil to allow).
+# it decides whether to BLOCK the call (returning a redirect-to-QMD reason String)
+# or ALLOW it (returning nil). All capability/freshness signals are injected by the
+# caller (the hook); this module shells out to nothing, reads no globals, and runs
+# no binaries. Mirrors bridge.rb's decision-fn convention (reason String to block,
+# nil to allow).
 #
-# Classification (per target path):
-#   - store `*.md` (under <plastic_home>/store or .../projects/<slug>/store) -> QMD
-#   - Serena-supported code/data file (NOT a store markdown) -> SERENA
-#   - images / binary / other -> ALLOWED
+# Operation-based policy (intent 89, ## Redesign):
+#   - The gate distinguishes DISCOVERY (content search) from READING a known target.
+#   - Only CONTENT SEARCH over store markdown is hard-gated -> QMD.
+#   - Reading a known target (Read, cat/head/tail) and structural discovery (Glob,
+#     find, ls) are ALWAYS allowed, including over the store.
+#   - Code navigation is a soft prompt MANDATE (PowerTools / UserPromptSubmit), not a
+#     hard gate here. Content grep over code is allowed (Serena cannot grep strings).
 #
-# Capability enforcement is BINARY (no advisory tier):
-#   - QMD class: detected+fresh -> BLOCK; detected+stale -> fire reindex, ALLOW
-#     this turn; absent/down -> ALLOW (no warning).
-#   - SERENA class: detected -> BLOCK; absent -> ALLOW.
+# Content-search vectors (the only ones that can be gated):
+#   - the Grep tool (its `path` search root)
+#   - bash `grep`/`rg`/`ag` (their path args; the first bareword is the PATTERN)
 #
-# Bypass: a TRAILING `# qmd-ok` shell comment on a Bash command (not a substring;
-# a quoted/echoed occurrence does not bypass).
+# QMD enforcement is BINARY (no advisory tier):
+#   - store-md content search: QMD detected+fresh -> BLOCK; detected+stale -> fire
+#     reindex, ALLOW this turn; absent/broken -> ALLOW (the hook warns on broken).
 #
-# Scope: only the agent's own tool calls. Ruby `File.read` inside scripts is
-# invisible to a PreToolUse hook and is explicitly out of scope (no exemptions).
+# Bypass: a TRAILING `# qmd-ok` shell comment on a Bash command (not a substring; a
+# quoted/echoed occurrence does not bypass). It is the auditable seam for "I tried
+# discovery and it did not serve me" (empty, low, or wrongly-scored results).
+#
+# Scope: only the agent's own tool calls. Ruby `File.read` inside scripts is invisible
+# to a PreToolUse hook and is explicitly out of scope (no exemptions).
 module RetrievalGate
   module_function
 
-  # Serena LSP covers many languages incl. JSON/YAML/TOML/Markdown/Ruby. Keep a
-  # small, conservative allowlist of code/data extensions. Markdown is listed but
-  # store markdown is reclassified to QMD before Serena ever sees it.
-  SERENA_EXTENSIONS = %w[
-    rb js jsx ts tsx mjs cjs py go rs java kt scala c h cpp hpp cc
-    cs php rb swift sh bash zsh lua ex exs erl clj sql
-    json yaml yml toml
-  ].freeze
-
-  # Image / binary extensions that are always allowed (plain read is fine).
-  BINARY_EXTENSIONS = %w[
-    png jpg jpeg gif webp svg ico bmp tiff pdf
-    zip gz tar tgz bz2 xz 7z
-    mp3 mp4 mov avi wav flac ogg
-    woff woff2 ttf otf eot
-    bin exe dll so dylib o a class jar wasm
-  ].freeze
-
   # A `# qmd-ok` token that is a real TRAILING shell comment, after stripping a
   # trailing newline. The token must be preceded by whitespace (or start the
-  # command) and run to end-of-string. `echo "# qmd-ok"` does NOT match: the
-  # token there is followed by a closing quote, not end-of-string.
+  # command) and run to end-of-string. `echo "# qmd-ok"` does NOT match: the token
+  # there is followed by a closing quote, not end-of-string.
   BYPASS_RE = /(?:\A|\s)#\s*qmd-ok\s*\z/.freeze
 
+  # Bash utilities that perform CONTENT SEARCH (scan file CONTENT for a pattern).
+  # These are the only bash read-vectors that can be gated; readers (cat/head/tail)
+  # and structural tools (find/ls) are never gated.
+  CONTENT_SEARCH_UTILS = %w[grep rg ag].freeze
+
   # Decide. Returns nil to ALLOW, or a reason String to BLOCK.
-  #   capabilities: { qmd:, qmd_fresh:, serena: } (booleans).
+  #   capabilities: { qmd:, qmd_fresh: } (booleans).
   #   reindex: no-arg callable fired once when a QMD-class target is STALE.
-  # When bypassed, returns nil and (if given) yields :bypass to the optional
-  # block so the caller can log it.
+  # When bypassed, returns nil and (if given) yields :bypass to the optional block
+  # so the caller can log it.
   def decision(tool_name:, tool_input:, plastic_home:, cwd:,
                capabilities:, reindex: -> {})
     targets = extract_targets(tool_name, tool_input, cwd: cwd)
@@ -71,17 +66,14 @@ module RetrievalGate
 
     stale_seen = false
     targets.each do |path|
-      case classify(path, plastic_home: plastic_home)
-      when :qmd
-        if capabilities[:qmd] && capabilities[:qmd_fresh]
-          return qmd_reason(path)
-        elsif capabilities[:qmd] # present but stale
-          stale_seen = true
-        end
-        # absent/down -> allow this target
-      when :serena
-        return serena_reason(path) if capabilities[:serena]
+      next unless classify(path, plastic_home: plastic_home) == :qmd
+
+      if capabilities[:qmd] && capabilities[:qmd_fresh]
+        return qmd_reason(path)
+      elsif capabilities[:qmd] # present but stale
+        stale_seen = true
       end
+      # absent/broken -> allow this target
     end
 
     reindex.call if stale_seen
@@ -90,27 +82,21 @@ module RetrievalGate
 
   # --- classification ---
 
+  # Operation-based: the store tree is the only gated class (content search whose
+  # target is at/under a store routes to QMD). Everything else is allowed.
   def classify(path, plastic_home:)
     return :allow if path.nil? || path.empty?
-    ext = extension(path)
-
-    if store_markdown?(path, plastic_home: plastic_home)
-      return :qmd
-    end
-    return :allow if BINARY_EXTENSIONS.include?(ext)
-    return :serena if SERENA_EXTENSIONS.include?(ext)
-
-    :allow
+    store_path?(path, plastic_home: plastic_home) ? :qmd : :allow
   end
 
-  # A markdown file under the global store or a project store. QMD owns store
-  # markdown even though Serena could also read markdown (QMD wins for the store).
-  def store_markdown?(path, plastic_home:)
-    return false unless %w[md markdown].include?(extension(path))
+  # A path AT or UNDER the global store or a project store. We gate the whole store
+  # tree (not just `*.md`) because a content search root is usually a directory:
+  # grepping the store scans its markdown, which is exactly what QMD should serve.
+  def store_path?(path, plastic_home:)
     abs = absolutize(path)
     home = File.expand_path(plastic_home)
     global = File.join(home, "store")
-    return true if abs.start_with?("#{global}/")
+    return true if abs == global || abs.start_with?("#{global}/")
 
     projects = File.join(home, "projects")
     return false unless abs.start_with?("#{projects}/")
@@ -118,19 +104,14 @@ module RetrievalGate
     tail.length >= 2 && tail[1] == "store"
   end
 
-  def extension(path)
-    File.extname(path.to_s).sub(/\A\./, "").downcase
-  end
-
   def absolutize(path)
     File.absolute_path?(path) ? path : File.expand_path(path)
   end
 
   # --- bypass ---
 
-  # Only Bash commands carry a trailing `# qmd-ok` comment. The token must be a
-  # real trailing comment (BYPASS_RE), so a quoted/echoed occurrence does not
-  # bypass.
+  # Only Bash commands carry a trailing `# qmd-ok` comment. The token must be a real
+  # trailing comment (BYPASS_RE), so a quoted/echoed occurrence does not bypass.
   def bypass?(tool_name, tool_input)
     return false unless tool_name.to_s == "Bash"
     cmd = tool_input.is_a?(Hash) ? tool_input["command"].to_s : ""
@@ -139,40 +120,36 @@ module RetrievalGate
 
   # --- target extraction ---
 
-  # Paths the call reads/scans. Conservative: missing an exotic form is fine;
-  # never flag /dev/null or pure pipes. Read vectors only (this is a READ gate),
-  # not the write vectors bridge.rb already covers.
+  # Paths a CONTENT-SEARCH operation scans. Reads (Read, cat/head/tail) and
+  # structural discovery (Glob, find, ls) are NOT content search -> no targets ->
+  # always allowed. Only the Grep tool and bash grep/rg/ag can be gated. Read
+  # vectors only (this is a READ gate); write vectors are bridge.rb's job.
   def extract_targets(tool_name, tool_input, cwd:)
     input = tool_input.is_a?(Hash) ? tool_input : {}
     case tool_name.to_s
-    when "Read"
-      [input["file_path"]].compact.reject(&:empty?)
-    when "Glob"
-      [input["path"], input["pattern"]].compact.reject { |s| s.to_s.empty? }
     when "Grep"
       # The search root is the target; the query text is not a path.
       [input["path"]].compact.reject { |s| s.to_s.empty? }
     when "Bash"
-      bash_read_targets(input["command"].to_s)
+      bash_search_targets(input["command"].to_s)
     else
+      # Read, Glob, and every other tool: read / structural op -> never gated.
       []
     end
   end
 
-  # READ utilities that take file/dir path arguments. Conservative parse: split
-  # on shell separators, identify the utility, collect its non-flag path args.
-  READ_UTILS = %w[grep rg ag find cat head tail less more bat ls wc nl sort uniq].freeze
-
-  def bash_read_targets(command)
+  # CONTENT-SEARCH path args across a compound command. Conservative: missing an
+  # exotic form is fine; never flag /dev/null or pure pipes.
+  def bash_search_targets(command)
     return [] unless command.is_a?(String) && !command.empty?
     targets = []
     command.split(/[;\n]|&&|\|\||\|/).each do |segment|
-      targets.concat(segment_read_targets(segment))
+      targets.concat(segment_search_targets(segment))
     end
     targets.reject { |t| t.nil? || t.empty? || dev_path?(t) }.uniq
   end
 
-  def segment_read_targets(segment)
+  def segment_search_targets(segment)
     tokens = tokenize(segment)
     return [] if tokens.empty?
 
@@ -180,21 +157,20 @@ module RetrievalGate
     idx = 0
     idx += 1 while tokens[idx] && tokens[idx].include?("=") && tokens[idx] !~ /\A-/
     util = File.basename(tokens[idx].to_s)
-    return [] unless READ_UTILS.include?(util)
+    return [] unless CONTENT_SEARCH_UTILS.include?(util)
 
     args = tokens[(idx + 1)..] || []
-    path_args_for(util, args)
+    path_args_for(args)
   end
 
-  # Collect path-shaped arguments for a read utility. Flags and flag-values are
-  # skipped; for grep/rg the first non-flag bareword is the PATTERN, not a path.
-  def path_args_for(util, args)
-    skip_pattern = %w[grep rg ag].include?(util)
+  # Collect path-shaped arguments for a content-search util. Flags are skipped; the
+  # first non-flag bareword is the PATTERN, not a path.
+  def path_args_for(args)
     paths = []
     pattern_consumed = false
     args.each do |a|
       next if a.start_with?("-")
-      if skip_pattern && !pattern_consumed
+      unless pattern_consumed
         pattern_consumed = true
         next
       end
@@ -225,14 +201,11 @@ module RetrievalGate
   # --- reasons ---
 
   def qmd_reason(path)
-    "retrieval gate: search the store via QMD, not raw grep/Read. " \
-      "Use `qmd search`/`qmd query` over the `plastic-*` collections (or " \
-      "`scripts/qmd-sync search`) instead of reading #{path}. " \
-      "If you genuinely need the raw read, append a trailing `# qmd-ok` to a Bash command."
-  end
-
-  def serena_reason(path)
-    "retrieval gate: navigate code via Serena's symbolic tools (find_symbol / " \
-      "get_symbols_overview / find_referencing_symbols), not raw grep/Read of #{path}."
+    "retrieval gate: search the store via QMD, not a raw content scan. Reading a " \
+      "known file and listing/globbing the store are fine; only CONTENT SEARCH over " \
+      "store markdown routes through QMD. Use `qmd search`/`qmd query` over the " \
+      "`plastic-*` collections (or `scripts/qmd-sync search`) instead of scanning " \
+      "#{path}. If QMD's results do not answer your need (your reading of the " \
+      "snippets, not their score), append a trailing `# qmd-ok` to a Bash command."
   end
 end