ask-rails-harness 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1c1a8388500ab4366c876be18f2b8fb2fd2c489fcd6c04da90c2a1ee9eea8b59
-  data.tar.gz: 6012eff1acc1cdecd8b606a91027f517a69ddb77d252c0aae8846e23b5e7f15e
+  metadata.gz: 3215cf3ae063b9066a65877e5eb501b3daec422ac1a6639a9286b384b2af3e8b
+  data.tar.gz: ba4b4347b86fd7dd944bfc5ac9b2031140081717721737cb9fcdce51badea17b
 SHA512:
-  metadata.gz: 7492561a0888730defddcff380d3800afefab364f41bf4df2e842a1894cdc4229c08e7d9c249df0b2d8ece1d0e9e345369840434a1af10d8802eb2f9f2fac4c4
-  data.tar.gz: 490b772093ac41af1ca679dfa55af7500ede1c634d01e82a3f7393ba23381fdf30507bf95929d8405e824ab7daf0d05f17052164a41724e8547bf7c4f3848abb
+  metadata.gz: 0eac35130da6e2a11713105cf07856c381f540487a3654b7385e1414920671831e568de01eafd8437286bb9537573aa42ca1287a2bfd2c2114c5cbde9ca9a9da
+  data.tar.gz: 7bb18027c9b38131db8d706fbdb19601449de9444b870441cd31684cb962063b08ddc78fd4f7dcbbcdbab0b977f8f81f5fe94f39ec15d0efd2e240da76278236

data/CHANGELOG.md

@@ -1,3 +1,41 @@
+## [0.2.0] — 2026-08-10
+
+### Added
+
+- **`run_tests` tool** — runs the app's test suite and returns structured
+  results (summary counts plus per-test file/line/message for failures)
+  instead of raw terminal output. Detects minitest vs rspec (`Gemfile.lock` +
+  `spec/`), supports `file:`/`name:` filters, `failed_only:` reruns persisted
+  from the previous run (`tmp/test/.ask/last-failures.json`), and a `timeout:`
+  that kills the run and reports `timed_out` with the artifact path. Minitest
+  gets machine-readable results via a bundled reporter/plugin; rspec uses its
+  built-in JSON formatter.
+- **`MinitestJsonReporter` + minitest plugin** — `lib/minitest/ask_rails_harness_plugin.rb`
+  is auto-discovered by minitest and registers the JSON reporter only when
+  `ASK_TEST_JSON_PATH` is set (i.e. when the harness invoked the run), so
+  ordinary `bin/rails test` runs are untouched. Supports both the minitest 5
+  (`ask_rails_harness_plugin_init`) and minitest 6 (`plugin_ask_rails_harness_init`)
+  plugin init conventions. Full human output always lands at
+  `tmp/test/.ask/last-test.log`.
+
+### Fixed
+
+- **`ReadLog` crashed with `NoMethodError: undefined method 'env' for module Ask::Rails`** —
+  `read_log.rb` used bare `Rails.env`, which resolves lexically to the
+  `Ask::Rails` module inside the gem's own namespace. Now `::Rails.env`
+  (matching `query_database.rb`).
+- **Audit-log failure warnings were silently swallowed** — `audit_log.rb` used
+  the same bare-`Rails` lookup, so `defined?(Rails.logger)` was always false.
+  Now `::Rails.logger&.warn` with a nil-safe guard.
+
+## [0.1.1] — 2026-07-31
+
+### Changed
+
+- Use `Ask::Agent::Policies::Permissions` for environment hooks (requires
+  ask-agent >= 0.28.0).
+- Require `time` for time-stdlib methods (`iso8601`, `parse`).
+
 ## [0.1.0] — 2026-07-25
 
 ### Changed

data/README.md

@@ -1,24 +1,10 @@
 # ask-rails-harness
 
-An admin AI agent for your Rails app. Mount the engine, get a chat interface at `/ask` that can inspect your code, query your database, read logs, and help you debug — all through an authenticated admin UI.
+[![Gem Version](https://badge.fury.io/rb/ask-rails-harness.svg)](https://badge.fury.io/rb/ask-rails-harness)
 
-> **Previously developed as `ask-rails` (v0.1.0–0.11.1).** Renamed to `ask-rails-harness` for clarity — this gem is the *harness* that wraps an AI agent with Rails-aware tools, environment, state, and feedback for internal/admin use.
+Admin AI copilot for Rails apps. Mounts a Rails Engine at `/ask` with an admin chat UI and 9 Rails-aware tools that inspect your code, query your database, read logs, and help you debug. For internal/admin use only: the agent has direct access to your database, file system, and shell.
 
-## Who is this for?
-
-- **Rails developers** who want an AI co-pilot that understands their app's codebase, schema, routes, and logs
-- **Internal/Admin use only** — the agent has direct access to your database, file system, and shell. Not for external/customer-facing use.
-
-For building customer-facing AI agents, use `ask-agent` directly with your own tools and UI.
-
-## What it gives you
-
-- **9 Rails-aware tools**: `ReadFile`, `QueryDatabase`, `ReadRoutes`, `ReadModel`, `ReadLog`, `RunCommand`, `SearchCodebase`, `SchemaGraph`, `RouteInspector`
-- **Admin chat UI**: Mount the engine, get a working chat at `/ask` with SSE streaming
-- **Auth integration**: Protect `/ask` behind your existing Devise/authentication
-- **AR persistence**: Agent sessions survive server restarts
-- **Service discovery**: Auto-detects installed ask-\* service gems
-- **Skills**: Built-in guides for Rails debugging, deployment, and database performance
+Previously released as `ask-rails` (v0.1.0-0.11.1) and renamed to `ask-rails-harness`. For customer-facing agents, use [ask-agent](https://github.com/ask-rb/ask-agent) directly.
 
 ## Installation
 
@@ -27,36 +13,21 @@ bundle add ask-rails-harness
 rails generate ask_rails_harness:install
 ```
 
-## Quick Start
+The generator creates `config/initializers/ask_rails_harness.rb`, the `ask_sessions` and `ask_audit_logs` migrations, and an `app/tools/` directory. Requires Rails 7.1+.
 
-Add the engine mount and auth protection to `config/routes.rb`:
+Run the migration, then mount the engine in `config/routes.rb`:
 
 ```ruby
-# config/routes.rb
 Rails.application.routes.draw do
-  # ... your routes ...
-
   authenticate :user, ->(u) { u.admin? } do
     mount Ask::Rails::Harness::Engine, at: "/ask"
   end
 end
 ```
 
-Then visit `/ask` in your browser.
-
-## Usage
-
-### Configuration
-
-```ruby
-# config/initializers/ask_rails_harness.rb
-Ask::Rails::Harness.configure do |c|
-  c.default_model = "claude-sonnet-4"
-  c.max_turns = 50
-end
-```
+Visit `/ask` in your browser.
 
-### Programmatic Access
+## Quick Start
 
 ```ruby
 # From any controller, view, or job
@@ -64,58 +35,59 @@ session = Ask::Rails::Harness.agent_session
 session.run("Find all open issues labeled 'bug' in our repo")
 ```
 
-### Route Helpers
+## Configuration
 
 ```ruby
-ask_rails_harness.root_path               # => /ask
-ask_rails_harness.sessions_path           # => /ask/sessions
-ask_rails_harness.session_messages_path(session_id) # => /ask/sessions/:id/messages
+# config/initializers/ask_rails_harness.rb
+Ask::Rails::Harness.configure do |c|
+  c.default_model = "claude-sonnet-4"
+  c.max_turns = 50
+  c.tool_concurrency = 5
+  c.allowed_commands = [/^rails /]
+  c.denied_commands = [/rm/, /dropdb/]
+  c.max_session_age = 7.days
+  c.max_sessions = 100
+  c.environment :production do |env|
+    env.mode = :read_only
+  end
+end
 ```
 
-### Auth
+Available settings: `default_model`, `max_turns`, `tool_concurrency`, `system_prompt`, `persistence_adapter`, `current_user`, `allowed_commands`, `denied_commands`, `max_session_age`, `max_sessions`, and per-environment permissions via `environment(name) { |env| env.mode = ... }`.
 
-By default, the admin chat is unprotected. Add auth in your routes (as shown above) or set a custom check:
+## Essential API
 
-```ruby
-# config/initializers/ask_rails_harness.rb
-Ask::Rails::Harness::Auth.check = -> {
-  redirect_to main_app.login_path unless current_user&.admin?
-}
-```
+- `Ask::Rails::Harness.agent_session`: build an agent session with the configured tools and prompt
+- `Ask::Rails::Harness.configure { |c| ... }`: configuration (see above)
+- `Ask::Rails::Harness.cleanup!`: prune old sessions and audit logs; also available as `rails ask_rails_harness:cleanup`
+- `Ask::Rails::Harness::Persistence`: ActiveRecord session persistence backed by the `ask_sessions` table
+- `Ask::Rails::Harness::Auth.check = -> { ... }`: auth proc evaluated in the controller context; the engine routes are unprotected by default
+- Audit log: every tool call is recorded in `ask_audit_logs` and broadcast as the `audit_log.ask_rails_harness` ActiveSupport notification
 
 ## Tools
 
 | Tool | What it does |
 |---|---|
-| `ReadFile` | Read any file (relative to `Rails.root`) |
-| `QueryDatabase` | Run read-only SQL (rejects non-SELECT in production) |
-| `ReadModel` | Inspect AR model schema, associations, validations |
-| `ReadRoutes` | View `config/routes.rb` |
+| `ReadFile` | Read any file relative to `Rails.root` |
+| `QueryDatabase` | Read-only SQL (non-SELECT rejected in production) |
+| `ReadModel` | Inspect an ActiveRecord model's columns, associations, validations |
+| `ReadRoutes` | Read `config/routes.rb` |
 | `ReadLog` | Read log files with level/search filtering |
 | `RunCommand` | Run shell commands in the app root |
-| `SearchCodebase` | Grep the codebase for patterns |
-| `SchemaGraph` | Full schema introspection — all models, tables, columns, associations |
+| `SearchCodebase` | Grep the codebase |
+| `SchemaGraph` | Full schema introspection: models, tables, columns, associations |
 | `RouteInspector` | Parsed route table with filters |
 
-## Engine Routes
+## Full documentation
 
-```
-GET  /ask                    → Chat UI
-POST /ask/sessions           → Create new session
-POST /ask/sessions/:id/messages → Send message (SSE streamed response)
-GET  /ask/sessions/:id/messages → Get message history
-GET  /ask/sessions/:id/stream  → SSE stream for existing session
-```
+The full ask-rb documentation lives at https://ask-rb.github.io/ask-docs. [Rails setup](https://ask-rb.github.io/ask-docs/rails/setup) covers ask-rails-harness in depth. API reference: https://ask-rb.github.io/ask-docs/reference/api.
 
-## Compared to ask-agent
+## Development
 
-| `ask-agent` | `ask-rails-harness` |
-|---|---|
-| Build external-facing agents | Build an internal admin co-pilot |
-| Bring your own tools | Ships Rails-specific tools |
-| Bring your own UI | Ships an admin chat UI |
-| Any Ruby app | Rails apps only |
-| General purpose | Development, debugging, ops |
+```bash
+bundle install
+bundle exec rake test
+```
 
 ## License
 

data/lib/ask/rails/harness/audit_log.rb

@@ -166,8 +166,10 @@ module Ask
                VALUES (#{values.join(', ')})"
             )
           rescue StandardError => e
-            # Silently fail — audit log should never crash the caller
-            Rails.logger.warn("[ask-rails-harness] Audit log write failed: #{e.message}") if defined?(Rails.logger)
+            # Silently fail — audit log should never crash the caller.
+            # ::Rails avoids the bare-`Rails` constant resolving to Ask::Rails;
+            # `&.` guards against Rails.logger returning nil (no app booted).
+            ::Rails.logger&.warn("[ask-rails-harness] Audit log write failed: #{e.message}") if defined?(::Rails.logger)
           end
 
           # Reset cached table check (useful in tests)

data/lib/ask/rails/harness/minitest_json_reporter.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+require "minitest"
+require "json"
+
+module Ask
+  module Rails
+    module Harness
+      # Writes a machine-readable JSON summary of a minitest run.
+      #
+      # Registered by the ask_rails_harness minitest plugin when the
+      # ASK_TEST_JSON_PATH env var is set (see the run_tests tool). Produces
+      # stable structured results — test name, klass, source file/line, status,
+      # and a message head — so the harness never has to parse terminal output.
+      #
+      # The reporter is inert unless ASK_TEST_JSON_PATH is set, so ordinary
+      # `bin/rails test` runs in apps that ship the harness are unaffected.
+      class MinitestJsonReporter < Minitest::AbstractReporter
+        def initialize(path = ENV["ASK_TEST_JSON_PATH"])
+          super()
+          @path = path
+          @results = []
+        end
+
+        def record(result)
+          @results << result
+        end
+
+        def report
+          return if @path.nil? || @path.to_s.empty?
+          File.write(@path, JSON.pretty_generate(build_report))
+        end
+
+        # Never influences the process exit status — pass/fail is decided by
+        # minitest's own summary reporter.
+        def passed?
+          true
+        end
+
+        private
+
+        def build_report
+          tests = @results.map { |result| test_entry(result) }
+          {
+            "framework" => "minitest",
+            "run" => tests.size,
+            "failures" => tests.count { |t| t["status"] == "failed" },
+            "errors" => tests.count { |t| t["status"] == "error" },
+            "skips" => tests.count { |t| t["status"] == "skipped" },
+            "tests" => tests
+          }
+        end
+
+        def test_entry(result)
+          file, line = result.source_location
+          failure = result.failure
+          {
+            "name" => result.name,
+            "klass" => result.klass,
+            "file" => file,
+            "line" => line,
+            "time" => result.time,
+            "status" => status_of(result),
+            "message" => failure ? message_of(failure) : nil
+          }
+        end
+
+        def status_of(result)
+          return "skipped" if result.skipped?
+          return "error" if result.error?
+          return "passed" if result.passed?
+          "failed"
+        end
+
+        # First lines of the failure message — enough for an agent to act on
+        # without dumping full backtraces into the report.
+        def message_of(failure)
+          failure.message.to_s.lines.first(3).map(&:strip).reject(&:empty?).join("\n")
+        end
+      end
+    end
+  end
+end

data/lib/ask/rails/harness/tools/query_database.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+require "time"
 
 module Ask
   module Rails

data/lib/ask/rails/harness/tools/read_log.rb

@@ -50,7 +50,9 @@ module Ask
 
           def resolve_log_path(custom_path)
             return rails_root.join(custom_path) if custom_path
-            rails_root.join("log", "#{Rails.env}.log")
+            # Use ::Rails (not Rails) — bare `Rails` inside the Ask::Rails::*
+            # namespace resolves to the Ask::Rails module itself.
+            rails_root.join("log", "#{::Rails.env}.log")
           end
 
           # Read from rotated archives too: log/production.log, .1, .2.gz, etc.

data/lib/ask/rails/harness/tools/run_tests.rb

@@ -0,0 +1,245 @@
+# frozen_string_literal: true
+
+require "json"
+require "timeout"
+
+module Ask
+  module Rails
+    module Harness
+      module Tools
+        class RunTests < Ask::Rails::Harness::Tool
+          description "Run the app's test suite and return structured results — summary counts " \
+                      "plus per-test file/line/message for failures, never raw terminal output. " \
+                      "Minitest gets a JSON reporter via the bundled minitest plugin; rspec uses " \
+                      "its built-in JSON formatter. Rerun only the previous run's failures with " \
+                      "failed_only."
+
+          param :file, type: :string, desc: "Test file path(s) relative to the app root (comma-separated for multiple)", required: false
+          param :name, type: :string, desc: "Test name pattern: minitest --name (string or /regex/), rspec -e", required: false
+          param :failed_only, type: :boolean, desc: "Rerun only the tests that failed in the previous run", required: false
+          param :timeout, type: :integer, desc: "Max seconds to wait before killing the run (default 300)", required: false
+
+          DEFAULT_TIMEOUT = 300
+          ARTIFACT_DIR = %w[tmp test .ask].freeze
+
+          def execute(file: nil, name: nil, failed_only: false, timeout: DEFAULT_TIMEOUT)
+            files = split_files(file)
+            framework = detect_framework
+
+            failed_tests = failed_only ? load_failed_tests : nil
+            if failed_only && failed_tests.empty?
+              return Ask::Result.failure("No failed tests from the previous run to rerun.")
+            end
+
+            artifact_dir = rails_root.join(*ARTIFACT_DIR).tap(&:mkpath)
+            log_path = artifact_dir.join("last-test.log")
+            json_path = artifact_dir.join("last-test.json")
+            status_path = artifact_dir.join("last-failures.json")
+
+            command, env = build_command(framework, files, name, failed_tests, json_path)
+            outcome = run(command, env, log_path, timeout)
+
+            results = parse_results(framework, json_path)
+            unless results
+              # A killed run can't produce results — report it structurally
+              # instead of failing, so the agent still gets the artifact path.
+              return Ask::Result.ok(data: timed_out_report(framework, command, env, outcome, status_path, log_path)) if outcome[:timed_out]
+
+              return Ask::Result.failure(
+                "Test run finished without machine-readable results (#{framework}); " \
+                "full output at #{rel(log_path)}"
+              )
+            end
+
+            report = build_report(framework, command, env, outcome, results, status_path, log_path)
+            Ask::Result.ok(data: report)
+          end
+
+          private
+
+          def split_files(file)
+            return [] if file.nil? || file.to_s.strip.empty?
+            file.split(",").map(&:strip).reject(&:empty?)
+          end
+
+          # Prefer rspec when it's in the bundle AND a spec/ dir exists;
+          # otherwise assume minitest (the Rails default).
+          def detect_framework
+            lockfile = rails_root.join("Gemfile.lock")
+            rspec = lockfile.exist? && lockfile.read.include?("rspec-rails")
+            rspec && rails_root.join("spec").directory? ? :rspec : :minitest
+          end
+
+          def build_command(framework, files, name, failed_tests, json_path)
+            if framework == :rspec
+              args = ["bundle", "exec", "rspec"]
+              args.concat(files)
+              args.concat(failed_tests.map { |t| "#{t[:file]}:#{t[:line]}" }) if failed_tests
+              args.concat(["-e", name]) if name
+              args.concat(["--format", "json", "--out", json_path.to_s])
+              [args, {}]
+            else
+              # Rails' `rails test` passes CLI args straight to minitest (files
+              # are required by Rails' runner, options parsed by minitest). The
+              # JSON reporter is injected via RUBYOPT: minitest 6 dropped both
+              # plugin auto-discovery and the -r option, so the plugin file is
+              # required by absolute path at process start (it pushes its
+              # extension; init_plugins registers the reporter later).
+              args = ["bin/rails", "test"]
+              args.concat(files)
+              args.concat(["-n", name]) if name
+              args.concat(["-n", name_pattern(failed_tests)]) if failed_tests
+              # minitest 6 dropped both plugin auto-discovery and the -r
+              # option. Activate the app's bundle first, then require the
+              # plugin by absolute path — it pushes its extension, and
+              # init_plugins registers the JSON reporter later.
+              env = {
+                "ASK_TEST_JSON_PATH" => json_path.to_s,
+                "RUBYOPT" => "-rbundler/setup -r#{minitest_plugin_path}"
+              }
+              [args, env]
+            end
+          end
+
+          def minitest_plugin_path
+            spec = Gem.loaded_specs["ask-rails-harness"]
+            spec ||= Gem::Specification.find_by_name("ask-rails-harness")
+            File.join(spec.full_gem_path, "lib", "minitest", "ask_rails_harness_plugin.rb")
+          end
+
+          # Minitest --name accepts a regexp; alternation runs exactly the
+          # failed tests.
+          def name_pattern(failed_tests)
+            escaped = failed_tests.map { |t| Regexp.escape(t[:test_name]) }
+            "/#{escaped.join('|')}/"
+          end
+
+          def run(command, env, log_path, timeout)
+            pid = Process.spawn(env, *command, chdir: rails_root.to_s,
+                                out: [log_path.to_s, "w"], err: [:child, :out])
+            status = nil
+            timed_out = false
+            begin
+              Timeout.timeout(timeout) { status = Process.wait2(pid).last }
+            rescue Timeout::Error
+              timed_out = true
+              begin
+                Process.kill("TERM", pid)
+                sleep 0.2
+                Process.kill("KILL", pid)
+              rescue Errno::ESRCH, Errno::EPERM
+                # Process already gone — nothing to kill.
+              end
+              status = Process.wait2(pid).last rescue nil
+            end
+            { exit_status: status&.exitstatus, timed_out: timed_out }
+          end
+
+          def build_report(framework, command, env, outcome, results, status_path, log_path)
+            summary = results[:summary]
+            failed_tests = results[:failed_tests]
+            persist_failed_tests(framework, failed_tests, status_path)
+
+            {
+              framework: framework.to_s,
+              command: full_command(command, env),
+              exit_status: outcome[:exit_status],
+              timed_out: outcome[:timed_out],
+              summary: summary,
+              failed_tests: failed_tests,
+              artifact: rel(log_path),
+              next: summary[:failures] + summary[:errors] > 0 ? "run_tests(failed_only: true)" : nil
+            }
+          end
+
+          def timed_out_report(framework, command, env, outcome, status_path, log_path)
+            persist_failed_tests(framework, [], status_path)
+            {
+              framework: framework.to_s,
+              command: full_command(command, env),
+              exit_status: outcome[:exit_status],
+              timed_out: true,
+              summary: nil,
+              failed_tests: nil,
+              artifact: rel(log_path),
+              next: nil
+            }
+          end
+
+          # The minitest options are part of the command itself (Rails passes
+          # CLI args through to minitest), so the reported command is the full
+          # invocation.
+          def full_command(command, env)
+            command.join(" ")
+          end
+
+          def parse_results(framework, json_path)
+            return nil unless json_path.exist?
+            payload = JSON.parse(json_path.read)
+            framework == :rspec ? parse_rspec(payload) : parse_minitest(payload)
+          rescue JSON::ParserError
+            nil
+          end
+
+          def parse_minitest(payload)
+            tests = payload["tests"] || []
+            summary = {
+              run: payload.fetch("run", tests.size),
+              failures: payload.fetch("failures", 0),
+              errors: payload.fetch("errors", 0),
+              skips: payload.fetch("skips", 0)
+            }
+            failed_tests = tests.filter_map do |t|
+              next unless %w[failed error].include?(t["status"])
+              { file: t["file"], test_name: t["name"], line: t["line"], message: t["message"] }
+            end
+            { summary: summary, failed_tests: failed_tests }
+          end
+
+          def parse_rspec(payload)
+            examples = payload["examples"] || []
+            summary_payload = payload["summary"] || {}
+            failed_examples = examples.select { |e| e["status"] == "failed" }
+            pending = examples.count { |e| e["status"] == "pending" }
+            summary = {
+              run: summary_payload.fetch("example_count", examples.size),
+              failures: summary_payload.fetch("failure_count", failed_examples.size),
+              errors: 0,
+              skips: summary_payload.fetch("pending_count", pending)
+            }
+            failed_tests = failed_examples.map do |e|
+              exception = e["exception"] || {}
+              {
+                file: e["file_path"],
+                test_name: e["full_description"],
+                line: e["line_number"],
+                message: exception["message"]
+              }
+            end
+            { summary: summary, failed_tests: failed_tests }
+          end
+
+          def persist_failed_tests(framework, failed_tests, status_path)
+            status_path.write(JSON.pretty_generate(framework: framework.to_s, failed_tests: failed_tests))
+          end
+
+          def load_failed_tests
+            path = rails_root.join(*ARTIFACT_DIR, "last-failures.json")
+            return [] unless path.exist?
+            JSON.parse(path.read).fetch("failed_tests", []).map { |t| symbolize_keys(t) }
+          rescue JSON::ParserError
+            []
+          end
+
+          def symbolize_keys(hash)
+            hash.each_with_object({}) { |(k, v), h| h[k.to_sym] = v }
+          end
+
+          def rel(path)
+            path.relative_path_from(rails_root).to_s
+          end
+        end
+      end
+    end
+  end
+end

data/lib/ask/rails/harness/version.rb

@@ -3,7 +3,7 @@
 module Ask
   module Rails
     module Harness
-      VERSION = "0.1.0"
+      VERSION = "0.2.0"
     end
   end
 end

data/lib/ask/rails/harness.rb

@@ -3,6 +3,7 @@
 require "rails"
 require "ask/agent"
 require "ask/auth"
+require "time"
 
 module Ask
   module Rails
@@ -64,7 +65,7 @@ module Ask
           env_mode = configuration.effective_mode
           return {} unless env_mode
 
-          perms = Ask::Agent::Extensions::Permissions.new(mode: env_mode)
+          perms = Ask::Agent::Policies::Permissions.new(mode: env_mode)
           { before_tool: [perms.method(:before_tool_call)] }
         rescue ArgumentError => e
           warn "[ask-rails-harness] Invalid environment mode: #{e.message}"
@@ -173,6 +174,8 @@ require_relative "harness/tools/read_model"
 require_relative "harness/tools/read_log"
 require_relative "harness/tools/schema_graph"
 require_relative "harness/tools/route_inspector"
+require_relative "harness/tools/run_tests"
+require_relative "harness/minitest_json_reporter"
 
 # Railtie is loaded only when Rails is fully available
 if defined?(::Rails::Railtie)
@@ -185,5 +188,5 @@ Ask::Rails::Harness::CORE_RAILS_TOOLS = [
   Ask::Rails::Harness::Tools::SearchCodebase, Ask::Rails::Harness::Tools::ReadRoutes,
   Ask::Rails::Harness::Tools::QueryDatabase, Ask::Rails::Harness::Tools::ReadModel,
   Ask::Rails::Harness::Tools::ReadLog, Ask::Rails::Harness::Tools::SchemaGraph,
-  Ask::Rails::Harness::Tools::RouteInspector
+  Ask::Rails::Harness::Tools::RouteInspector, Ask::Rails::Harness::Tools::RunTests
 ].freeze

data/lib/minitest/ask_rails_harness_plugin.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+# Minitest plugin for ask-rails-harness.
+#
+# Registers the JSON reporter when a run was started by the harness's
+# run_tests tool (ASK_TEST_JSON_PATH set); ordinary test runs are untouched.
+#
+# Loading differs by minitest version:
+#   - minitest 5 auto-discovers `minitest/*_plugin.rb` files and calls
+#     init_plugins with `#{name}_plugin_init`.
+#   - minitest 6 dropped auto-discovery (and the -r option); the run_tests
+#     tool injects this file via RUBYOPT="-r <absolute path>" and
+#     init_plugins dispatches `plugin_#{name}_init` instead.
+# Either way, pushing the extension below (idempotent) makes init_plugins
+# dispatch to the matching init method after the composite reporter exists —
+# the only reliable point to append a reporter.
+# __dir__-based requires keep the file loadable before Bundler.setup.
+require File.expand_path("../ask/rails/harness/minitest_json_reporter", __dir__)
+
+module Minitest
+  # Minitest 6 convention: init_plugins calls plugin_#{name}_init.
+  def self.plugin_ask_rails_harness_init(options)
+    register_ask_rails_harness_json_reporter(options)
+  end
+
+  # Minitest 5 convention: init_plugins calls #{name}_plugin_init.
+  def self.ask_rails_harness_plugin_init(options)
+    register_ask_rails_harness_json_reporter(options)
+  end
+
+  def self.register_ask_rails_harness_json_reporter(_options)
+    path = ENV["ASK_TEST_JSON_PATH"]
+    return if path.nil? || path.to_s.empty?
+
+    Minitest.reporter << Ask::Rails::Harness::MinitestJsonReporter.new(path)
+  end
+end
+
+Minitest.extensions << "ask_rails_harness" unless Minitest.extensions.include?("ask_rails_harness")

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-rails-harness
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Kaka Ruto
@@ -57,14 +57,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: 0.28.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: 0.28.0
 - !ruby/object:Gem::Dependency
   name: ask-auth
   requirement: !ruby/object:Gem::Requirement
@@ -160,6 +160,7 @@ files:
 - lib/ask/rails/harness/configuration.rb
 - lib/ask/rails/harness/engine.rb
 - lib/ask/rails/harness/environment_permissions.rb
+- lib/ask/rails/harness/minitest_json_reporter.rb
 - lib/ask/rails/harness/persistence.rb
 - lib/ask/rails/harness/railtie.rb
 - lib/ask/rails/harness/service_discovery.rb
@@ -171,6 +172,7 @@ files:
 - lib/ask/rails/harness/tools/read_routes.rb
 - lib/ask/rails/harness/tools/route_inspector.rb
 - lib/ask/rails/harness/tools/run_command.rb
+- lib/ask/rails/harness/tools/run_tests.rb
 - lib/ask/rails/harness/tools/schema_graph.rb
 - lib/ask/rails/harness/tools/search_codebase.rb
 - lib/ask/rails/harness/version.rb
@@ -181,6 +183,7 @@ files:
 - lib/generators/ask/rails/harness/install/templates/audit_log_migration.rb
 - lib/generators/ask/rails/harness/install/templates/initializer.rb
 - lib/generators/ask/rails/harness/install/templates/migration.rb
+- lib/minitest/ask_rails_harness_plugin.rb
 homepage: https://github.com/ask-rb/ask-rails-harness
 licenses:
 - MIT