console_agent 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 35e4cf8ecceb5241ce1b18f2cac6d8a235b3185f7839b2143eca6233390375d5
-  data.tar.gz: 7ebfc01003d9995ce65645a1f15955747cd59197c891e294d2152f3101fd583c
+  metadata.gz: 662ece5e5732e350b22871c8029421edf06ed4d9f98183f5cee7e95a3c1099f6
+  data.tar.gz: 483642598de39d23ace26f5d90863a8712b8799357c7809b2ba770f2bdc2522a
 SHA512:
-  metadata.gz: 1bfdf0d1b5a6278101d7cda640ac710a1aef589e5a1adc4194876404d897a53bcfa27cd423029abc76a117eb6ab2ba13a14e605c4447c1452ab78d0f7b98b960
-  data.tar.gz: 7589590b8dcbfd4268070ce59da381741ca1643152b2689829d741a6cc325e7d9130576629a3155ba0267fb0ea86623a584f9a6e6c2956143e043da43e551501
+  metadata.gz: 5095d8f4cb0706e84c81be1b4a859de6ac48336ccb9c950feeb0dcf0df676d78fddc53a2314a21cbe6ee9d79b808f5edf0d854f3262b72c5befbb4420d9b979b
+  data.tar.gz: 4c96f0e1664c1357cc7ec3f53aada9d1dcf4e05cbc92e19edfd052b2e99c75d1596d5a49440b8b9478fd576140b36929589a87e009dd971b4599cfa8ab149520

data/README.md

@@ -1,40 +1,6 @@
 # ConsoleAgent
 
-Claude Code, embedded in your Rails console.
-
-Ask questions in plain English. The AI explores your schema, models, and source code on its own, then writes and runs the Ruby code for you.
-
-## Install
-
-```ruby
-# Gemfile
-gem 'console_agent', group: :development
-```
-
-```bash
-bundle install
-rails generate console_agent:install
-```
-
-Set your API key in the generated initializer or as an env var (`ANTHROPIC_API_KEY`):
-
-```ruby
-# config/initializers/console_agent.rb
-ConsoleAgent.configure do |config|
-  config.api_key = 'sk-ant-...'
-end
-```
-
-To set up session logging (OPTIONAL), create the table from the console:
-
-```ruby
-ConsoleAgent.setup!
-# => ConsoleAgent: created console_agent_sessions table.
-```
-
-To reset the table (e.g. after upgrading), run `ConsoleAgent.teardown!` then `ConsoleAgent.setup!`.
-
-## Usage
+Claude Code for your Rails Console.
 
 ```
 irb> ai "find the 5 most recent orders over $100"
@@ -50,28 +16,10 @@ Execute? [y/N/edit] y
 => [#<Order id: 4821, ...>, ...]
 ```
 
-The AI calls tools behind the scenes to learn your app — schema, models, associations, source code — so it writes accurate queries without you providing any context.
-
-### Commands
-
-| Command | What it does |
-|---------|-------------|
-| `ai "query"` | One-shot: ask, review code, confirm |
-| `ai! "query"` | Interactive: ask and keep chatting |
-| `ai? "query"` | Explain only, never executes |
-
-### Multi-Step Plans
-
-For complex tasks, the AI builds a plan and executes it step by step:
+For complex tasks it builds multi-step plans, executing each step sequentially:
 
 ```
 ai> get the most recent salesforce token and count events via the API
-  Thinking...
-  -> describe_table("oauth2_tokens")
-     28 columns
-  -> read_file("lib/salesforce_api.rb")
-     202 lines
-
   Plan (2 steps):
   1. Find the most recent active Salesforce OAuth2 token
      token = Oauth2Token.where(provider: "salesforce", active: true)
@@ -81,155 +29,81 @@ ai> get the most recent salesforce token and count events via the API
      api.query("SELECT COUNT(Id) FROM Event")
 
   Accept plan? [y/N/a(uto)] a
-  Step 1/2: Find the most recent active Salesforce OAuth2 token
-  ...
-=> #<Oauth2Token id: 1, provider: "salesforce", ...>
-
-  Step 2/2: Query event count via SOQL
-  ...
-=> [{"expr0"=>42}]
-```
-
-Each step's return value is available to later steps as `step1`, `step2`, etc.
-
-Plan prompt options:
-- **y** — accept, then confirm each step one at a time
-- **a** — accept and auto-run all steps (stays in manual mode for future queries)
-- **N** — decline; you're asked what to change and the AI revises
-
-### Memories
-
-The AI remembers what it learns about your codebase across sessions:
-
-```
-ai> how does sharding work?
-  -> read_file("config/initializers/sharding.rb")
-  -> save_memory("Sharding architecture")
-     Memory saved
-
-  This app uses database-per-shard. User.count returns the current shard only.
-```
-
-Next time, it already knows — no re-reading files, fewer tokens.
-
-### Interactive Mode
-
-```
-irb> ai!
-ConsoleAgent interactive mode. Type 'exit' to leave.
-  Auto-execute: OFF (Shift-Tab or /auto to toggle) | > code to run directly | /usage | /name <label>
-
-ai> show me all tables
-  ...
-ai> count orders by status
-  ...
-ai> /auto
-  Auto-execute: ON
-ai> delete cancelled orders older than 90 days
-  ...
-ai> exit
 ```
 
-Toggle `/auto` to skip confirmation prompts. `/debug` shows raw API traffic. `/usage` shows token stats.
-
-### Direct Code Execution
+No context needed from you — it figures out your app on its own.
 
-Prefix any input with `>` to run Ruby code directly — no LLM round-trip. The result is added to the conversation context, so the AI knows what happened:
-
-```
-ai> >User.count
-=> 8
-ai> how many users do I have?
-  Thinking...
+## Install
 
-You have **8 users** in your database, as confirmed by the `User.count` you just ran.
+```ruby
+# Gemfile
+gem 'console_agent', group: :development
 ```
 
-Useful for quick checks, setting up variables, or giving the AI concrete data to work with.
-
-### Sessions
-
-Sessions are saved automatically when session logging is enabled. You can name, list, and resume them.
-
-```
-ai> /name sf_user_123_calendar
-  Session named: sf_user_123_calendar
-ai> exit
-Session #42 saved.
-  Resume with: ai_resume "sf_user_123_calendar"
-Left ConsoleAgent interactive mode.
+```bash
+bundle install
+rails generate console_agent:install
 ```
 
-List recent sessions:
+Set your API key in the generated initializer or via env var (`ANTHROPIC_API_KEY`):
 
+```ruby
+# config/initializers/console_agent.rb
+ConsoleAgent.configure do |config|
+  config.api_key = 'sk-ant-...'
+end
 ```
-irb> ai_sessions
-[Sessions — showing 3]
-
-  #42 sf_user_123_calendar find user 123 with calendar issues
-     [interactive] 5m ago 2340 tokens
 
-  #41 count all active users
-     [one_shot] 1h ago 850 tokens
-
-  #40 debug_payments explain payment flow
-     [interactive] 2h ago 4100 tokens
-
-Use ai_resume(id_or_name) to resume a session.
-```
+## Commands
 
-Resume a session by name or ID — previous output is replayed, then you continue where you left off:
+| Command | What it does |
+|---------|-------------|
+| `ai "query"` | Ask, review generated code, confirm execution |
+| `ai!` | Enter interactive mode (multi-turn conversation) |
+| `ai? "query"` | Explain only, no execution |
+| `ai_init` | Generate app guide for better AI context |
+| `ai_setup` | Install session logging table |
+| `ai_sessions` | List recent sessions |
+| `ai_resume` | Resume a session by name or ID |
+| `ai_memories` | Show stored memories |
+| `ai_status` | Show current configuration |
 
-```
-irb> ai_resume "sf_user_123_calendar"
---- Replaying previous session output ---
-ai> find user 123 with calendar issues
-  ...previous output...
---- End of previous output ---
-
-ConsoleAgent interactive mode (sf_user_123_calendar). Type 'exit' to leave.
-ai> now check their calendar sync status
-  ...
-```
+### Interactive Mode
 
-Name or rename a session after the fact:
+`ai!` starts a conversation. Slash commands available inside:
 
-```
-irb> ai_name 41, "active_user_count"
-Session #41 named: active_user_count
-```
+| Command | What it does |
+|---------|-------------|
+| `/auto` | Toggle auto-execute (skip confirmations) |
+| `/compact` | Compress history into a summary (saves tokens) |
+| `/usage` | Show token stats |
+| `/debug` | Toggle raw API output |
+| `/name <label>` | Name the session for easy resume |
 
-Filter sessions by search term:
+Prefix input with `>` to run Ruby directly (no LLM round-trip). The result is added to conversation context.
 
-```
-irb> ai_sessions 20, search: "salesforce"
-```
+## Features
 
-If you have an existing `console_agent_sessions` table, run `ConsoleAgent.migrate!` to add the `name` column.
+- **Tool use** — AI introspects your schema, models, files, and code to write accurate queries
+- **Multi-step plans** — complex tasks are broken into steps, executed sequentially with `step1`/`step2` references
+- **Memories** — AI saves what it learns about your app across sessions
+- **App guide** — `ai_init` generates a guide injected into every system prompt
+- **Sessions** — name, list, and resume interactive conversations (`ai_setup` to enable)
+- **History compaction** — `/compact` summarizes long conversations to reduce cost and latency
 
 ## Configuration
 
-All settings live in `config/initializers/console_agent.rb` and can be changed at runtime:
-
 ```ruby
 ConsoleAgent.configure do |config|
   config.provider = :anthropic       # or :openai
   config.auto_execute = false         # true to skip confirmations
-  config.max_tokens = 4096             # max tokens per LLM response
-  config.max_tool_rounds = 10         # max tool calls per query
-  config.session_logging = true       # log sessions to DB (run ConsoleAgent.setup!)
+  config.session_logging = true       # requires ai_setup
 end
 ```
 
-For the admin UI, mount the engine:
-
-```ruby
-mount ConsoleAgent::Engine => '/console_agent'
-```
-
 ## Requirements
 
-- Ruby >= 2.5, Rails >= 5.0, Faraday >= 1.0
+Ruby >= 2.5, Rails >= 5.0, Faraday >= 1.0
 
 ## License
 

data/lib/console_agent/console_methods.rb

@@ -130,15 +130,34 @@ module ConsoleAgent
       nil
     end
 
+    def ai_setup
+      ConsoleAgent.setup!
+    end
+
+    def ai_init
+      require 'console_agent/context_builder'
+      require 'console_agent/providers/base'
+      require 'console_agent/executor'
+      require 'console_agent/repl'
+
+      repl = Repl.new(__console_agent_binding)
+      repl.init_guide
+    rescue => e
+      $stderr.puts "\e[31mConsoleAgent error: #{e.message}\e[0m"
+      nil
+    end
+
     def ai(query = nil)
       if query.nil?
         $stderr.puts "\e[33mUsage: ai \"your question here\"\e[0m"
         $stderr.puts "\e[33m  ai  \"query\"  - ask + confirm execution\e[0m"
         $stderr.puts "\e[33m  ai! \"query\"  - enter interactive mode (or ai! with no args)\e[0m"
         $stderr.puts "\e[33m  ai? \"query\"  - explain only, no execution\e[0m"
+        $stderr.puts "\e[33m  ai_init      - generate/update app guide for better AI context\e[0m"
         $stderr.puts "\e[33m  ai_sessions  - list recent sessions\e[0m"
         $stderr.puts "\e[33m  ai_resume    - resume a session by name or id\e[0m"
         $stderr.puts "\e[33m  ai_name      - name a session: ai_name 42, \"my_label\"\e[0m"
+        $stderr.puts "\e[33m  ai_setup     - install session logging table\e[0m"
         $stderr.puts "\e[33m  ai_status    - show current configuration\e[0m"
         $stderr.puts "\e[33m  ai_memories  - show recent memories (ai_memories(n) for last n)\e[0m"
         return nil
@@ -235,6 +254,14 @@ module ConsoleAgent
     end
 
     def __console_agent_binding
+      # Try Pry first (pry-rails replaces IRB but IRB may still be loaded)
+      if defined?(Pry)
+        pry_inst = ObjectSpace.each_object(Pry).find { |p|
+          p.respond_to?(:binding_stack) && !p.binding_stack.empty?
+        } rescue nil
+        return pry_inst.current_binding if pry_inst
+      end
+
       # Try IRB workspace binding
       if defined?(IRB) && IRB.respond_to?(:CurrentContext)
         ctx = IRB.CurrentContext rescue nil
@@ -243,14 +270,6 @@ module ConsoleAgent
         end
       end
 
-      # Try Pry binding
-      if defined?(Pry) && respond_to?(:pry_instance, true)
-        pry_inst = pry_instance rescue nil
-        if pry_inst && pry_inst.respond_to?(:current_binding)
-          return pry_inst.current_binding
-        end
-      end
-
       # Fallback
       TOPLEVEL_BINDING
     end

data/lib/console_agent/context_builder.rb

@@ -15,10 +15,32 @@ module ConsoleAgent
       parts = []
       parts << smart_system_instructions
       parts << environment_context
+      parts << guide_context
       parts << memory_context
       parts.compact.join("\n\n")
     end
 
+    def environment_context
+      lines = ["## Environment"]
+      lines << "- Ruby #{RUBY_VERSION}"
+      lines << "- Rails #{Rails.version}" if defined?(Rails) && Rails.respond_to?(:version)
+
+      if defined?(ActiveRecord::Base) && ActiveRecord::Base.connected?
+        adapter = ActiveRecord::Base.connection.adapter_name rescue 'unknown'
+        lines << "- Database adapter: #{adapter}"
+      end
+
+      if defined?(Bundler)
+        key_gems = %w[devise cancancan pundit sidekiq delayed_job resque
+                      paperclip carrierwave activestorage shrine
+                      pg mysql2 sqlite3 mongoid]
+        loaded = key_gems.select { |g| Gem.loaded_specs.key?(g) }
+        lines << "- Key gems: #{loaded.join(', ')}" unless loaded.empty?
+      end
+
+      lines.join("\n")
+    end
+
     private
 
     def smart_system_instructions
@@ -67,25 +89,14 @@ module ConsoleAgent
       PROMPT
     end
 
-    def environment_context
-      lines = ["## Environment"]
-      lines << "- Ruby #{RUBY_VERSION}"
-      lines << "- Rails #{Rails.version}" if defined?(Rails) && Rails.respond_to?(:version)
-
-      if defined?(ActiveRecord::Base) && ActiveRecord::Base.connected?
-        adapter = ActiveRecord::Base.connection.adapter_name rescue 'unknown'
-        lines << "- Database adapter: #{adapter}"
-      end
-
-      if defined?(Bundler)
-        key_gems = %w[devise cancancan pundit sidekiq delayed_job resque
-                      paperclip carrierwave activestorage shrine
-                      pg mysql2 sqlite3 mongoid]
-        loaded = key_gems.select { |g| Gem.loaded_specs.key?(g) }
-        lines << "- Key gems: #{loaded.join(', ')}" unless loaded.empty?
-      end
+    def guide_context
+      content = ConsoleAgent.storage.read(ConsoleAgent::GUIDE_KEY)
+      return nil if content.nil? || content.strip.empty?
 
-      lines.join("\n")
+      "## Application Guide\n\n#{content.strip}"
+    rescue => e
+      ConsoleAgent.logger.debug("ConsoleAgent: guide context failed: #{e.message}")
+      nil
     end
 
     def memory_context

data/lib/console_agent/executor.rb

@@ -43,7 +43,7 @@ module ConsoleAgent
   class Executor
     CODE_REGEX = /```ruby\s*\n(.*?)```/m
 
-    attr_reader :binding_context
+    attr_reader :binding_context, :last_error
     attr_accessor :on_prompt
 
     def initialize(binding_context)
@@ -75,6 +75,7 @@ module ConsoleAgent
     def execute(code)
       return nil if code.nil? || code.strip.empty?
 
+      @last_error = nil
       captured_output = StringIO.new
       old_stdout = $stdout
       # Tee output: capture it and also print to the real stdout
@@ -89,12 +90,14 @@ module ConsoleAgent
       result
     rescue SyntaxError => e
       $stdout = old_stdout if old_stdout
-      $stderr.puts colorize("SyntaxError: #{e.message}", :red)
+      @last_error = "SyntaxError: #{e.message}"
+      $stderr.puts colorize(@last_error, :red)
       @last_output = nil
       nil
     rescue => e
       $stdout = old_stdout if old_stdout
-      $stderr.puts colorize("Error: #{e.class}: #{e.message}", :red)
+      @last_error = "#{e.class}: #{e.message}"
+      $stderr.puts colorize("Error: #{@last_error}", :red)
       e.backtrace.first(3).each { |line| $stderr.puts colorize("  #{line}", :red) }
       @last_output = captured_output&.string
       nil

data/lib/console_agent/railtie.rb

@@ -15,7 +15,7 @@ module ConsoleAgent
 
       # Welcome message
       if $stdout.respond_to?(:tty?) && $stdout.tty?
-        $stdout.puts "\e[36m[ConsoleAgent] AI assistant loaded. Try: ai \"show me all tables\"\e[0m"
+        $stdout.puts "\e[36m[ConsoleAgent v#{ConsoleAgent::VERSION}] AI assistant loaded. Try: ai \"show me all tables\"\e[0m"
       end
 
       # Pre-build context in background

data/lib/console_agent/repl.rb

@@ -18,29 +18,25 @@ module ConsoleAgent
       start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
       console_capture = StringIO.new
       exec_result = with_console_capture(console_capture) do
-        result, _ = send_query(query)
-        track_usage(result)
-        code = @executor.display_response(result.text)
-        display_usage(result)
-
-        exec_result = nil
-        executed = false
-        has_code = code && !code.strip.empty?
-
-        if has_code
-          exec_result = if ConsoleAgent.configuration.auto_execute
-                          @executor.execute(code)
-                        else
-                          @executor.confirm_and_execute(code)
-                        end
-          executed = !@executor.last_cancelled?
+        conversation = [{ role: :user, content: query }]
+        exec_result, code, executed = one_shot_round(conversation)
+
+        # Auto-retry once if execution errored
+        if executed && @executor.last_error
+          error_msg = "Code execution failed with error: #{@executor.last_error}"
+          error_msg = error_msg[0..1000] + '...' if error_msg.length > 1000
+          conversation << { role: :assistant, content: @_last_result_text }
+          conversation << { role: :user, content: error_msg }
+
+          $stdout.puts "\e[2m  Attempting to fix...\e[0m"
+          exec_result, code, executed = one_shot_round(conversation)
         end
 
         @_last_log_attrs = {
           query: query,
-          conversation: [{ role: :user, content: query }, { role: :assistant, content: result.text }],
+          conversation: conversation,
           mode: 'one_shot',
-          code_executed: has_code ? code : nil,
+          code_executed: code,
           code_output: executed ? @executor.last_output : nil,
           code_result: executed && exec_result ? exec_result.inspect : nil,
           executed: executed,
@@ -61,6 +57,31 @@ module ConsoleAgent
       nil
     end
 
+    # Executes one LLM round: send query, display, optionally execute code.
+    # Returns [exec_result, code, executed].
+    def one_shot_round(conversation)
+      result, _ = send_query(nil, conversation: conversation)
+      track_usage(result)
+      code = @executor.display_response(result.text)
+      display_usage(result)
+      @_last_result_text = result.text
+
+      exec_result = nil
+      executed = false
+      has_code = code && !code.strip.empty?
+
+      if has_code
+        exec_result = if ConsoleAgent.configuration.auto_execute
+                        @executor.execute(code)
+                      else
+                        @executor.confirm_and_execute(code)
+                      end
+        executed = !@executor.last_cancelled?
+      end
+
+      [exec_result, has_code ? code : nil, executed]
+    end
+
     def explain(query)
       start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
       console_capture = StringIO.new
@@ -90,6 +111,62 @@ module ConsoleAgent
       nil
     end
 
+    def init_guide
+      storage = ConsoleAgent.storage
+      existing_guide = begin
+        content = storage.read(ConsoleAgent::GUIDE_KEY)
+        (content && !content.strip.empty?) ? content.strip : nil
+      rescue
+        nil
+      end
+
+      if existing_guide
+        $stdout.puts "\e[36m  Existing guide found (#{existing_guide.length} chars). Will update.\e[0m"
+      else
+        $stdout.puts "\e[36m  No existing guide. Exploring the app...\e[0m"
+      end
+
+      require 'console_agent/tools/registry'
+      init_tools = Tools::Registry.new(mode: :init)
+      sys_prompt = init_system_prompt(existing_guide)
+      messages = [{ role: :user, content: "Explore this Rails application and generate the application guide." }]
+
+      # Temporarily increase timeout — init conversations are large
+      original_timeout = ConsoleAgent.configuration.timeout
+      ConsoleAgent.configuration.timeout = [original_timeout, 120].max
+
+      result, _ = send_query_with_tools(messages, system_prompt: sys_prompt, tools_override: init_tools)
+
+      guide_text = result.text.to_s.strip
+      # Strip markdown code fences if the LLM wrapped the response
+      guide_text = guide_text.sub(/\A```(?:markdown)?\s*\n?/, '').sub(/\n?```\s*\z/, '')
+      # Strip LLM preamble/thinking before the actual guide content
+      guide_text = guide_text.sub(/\A.*?(?=^#\s)/m, '') if guide_text =~ /^#\s/m
+
+      if guide_text.empty?
+        $stdout.puts "\e[33m  No guide content generated.\e[0m"
+        return nil
+      end
+
+      storage.write(ConsoleAgent::GUIDE_KEY, guide_text)
+
+      path = storage.respond_to?(:root_path) ? File.join(storage.root_path, ConsoleAgent::GUIDE_KEY) : ConsoleAgent::GUIDE_KEY
+      $stdout.puts "\e[32m  Guide saved to #{path} (#{guide_text.length} chars)\e[0m"
+      display_usage(result)
+      nil
+    rescue Interrupt
+      $stdout.puts "\n\e[33m  Interrupted.\e[0m"
+      nil
+    rescue Providers::ProviderError => e
+      $stderr.puts "\e[31mConsoleAgent Error: #{e.message}\e[0m"
+      nil
+    rescue => e
+      $stderr.puts "\e[31mConsoleAgent Error: #{e.class}: #{e.message}\e[0m"
+      nil
+    ensure
+      ConsoleAgent.configuration.timeout = original_timeout if original_timeout
+    end
+
     def interactive
       init_interactive_state
       interactive_loop
@@ -139,6 +216,7 @@ module ConsoleAgent
       @last_interactive_output = nil
       @last_interactive_result = nil
       @last_interactive_executed = false
+      @compact_warned = false
     end
 
     def interactive_loop
@@ -146,7 +224,7 @@ module ConsoleAgent
       name_display = @interactive_session_name ? " (#{@interactive_session_name})" : ""
       # Write banner to real stdout (bypass TeeIO) so it doesn't accumulate on resume
       @interactive_old_stdout.puts "\e[36mConsoleAgent interactive mode#{name_display}. Type 'exit' or 'quit' to leave.\e[0m"
-      @interactive_old_stdout.puts "\e[2m  Auto-execute: #{auto ? 'ON' : 'OFF'} (Shift-Tab or /auto to toggle) | > code to run directly | /usage | /name <label>\e[0m"
+      @interactive_old_stdout.puts "\e[2m  Auto-execute: #{auto ? 'ON' : 'OFF'} (Shift-Tab or /auto to toggle) | > code to run directly | /usage | /compact | /name <label>\e[0m"
 
       # Bind Shift-Tab to insert /auto command and submit
       if Readline.respond_to?(:parse_and_bind)
@@ -180,6 +258,11 @@ module ConsoleAgent
           next
         end
 
+        if input == '/compact'
+          compact_history
+          next
+        end
+
         if input.start_with?('/name')
           name = input.sub('/name', '').strip
           if name.empty?
@@ -240,65 +323,24 @@ module ConsoleAgent
         # Save immediately so the session is visible in the admin UI while the AI thinks
         log_interactive_turn
 
-        begin
-          result, tool_messages = send_query(input, conversation: @history)
-        rescue Interrupt
-          $stdout.puts "\n\e[33m  Aborted.\e[0m"
+        status = send_and_execute
+        if status == :interrupted
           @history.pop # Remove the user message that never got a response
           log_interactive_turn
           next
         end
 
-        track_usage(result)
-        code = @executor.display_response(result.text)
-        display_usage(result, show_session: true)
-
-        # Save after response is displayed so viewer shows progress before Execute prompt
-        log_interactive_turn
-
-        # Add tool call/result messages so the LLM remembers what it learned
-        @history.concat(tool_messages) if tool_messages && !tool_messages.empty?
-        @history << { role: :assistant, content: result.text }
-
-        if code && !code.strip.empty?
-          if ConsoleAgent.configuration.auto_execute
-            exec_result = @executor.execute(code)
-          else
-            exec_result = @executor.confirm_and_execute(code)
-          end
-
-          unless @executor.last_cancelled?
-            @last_interactive_code = code
-            @last_interactive_output = @executor.last_output
-            @last_interactive_result = exec_result ? exec_result.inspect : nil
-            @last_interactive_executed = true
-          end
-
-          if @executor.last_cancelled?
-            @history << { role: :user, content: "User declined to execute the code." }
-          else
-            output_parts = []
-
-            # Capture printed output (puts, print, etc.)
-            if @executor.last_output && !@executor.last_output.strip.empty?
-              output_parts << "Output:\n#{@executor.last_output.strip}"
-            end
-
-            # Capture return value
-            if exec_result
-              output_parts << "Return value: #{exec_result.inspect}"
-            end
-
-            unless output_parts.empty?
-              result_str = output_parts.join("\n\n")
-              result_str = result_str[0..1000] + '...' if result_str.length > 1000
-              @history << { role: :user, content: "Code was executed. #{result_str}" }
-            end
-          end
+        # Auto-retry once when execution fails — send error back to LLM for a fix
+        if status == :error
+          $stdout.puts "\e[2m  Attempting to fix...\e[0m"
+          log_interactive_turn
+          send_and_execute
         end
 
         # Update with the AI response, tokens, and any execution results
         log_interactive_turn
+
+        warn_if_history_large
       end
 
       $stdout = @interactive_old_stdout
@@ -318,6 +360,73 @@ module ConsoleAgent
       $stderr.puts "\e[31mConsoleAgent Error: #{e.class}: #{e.message}\e[0m"
     end
 
+    # Sends conversation to LLM, displays response, executes code if present.
+    # Returns :success, :error, :cancelled, :no_code, or :interrupted.
+    def send_and_execute
+      begin
+        result, tool_messages = send_query(nil, conversation: @history)
+      rescue Interrupt
+        $stdout.puts "\n\e[33m  Aborted.\e[0m"
+        return :interrupted
+      end
+
+      track_usage(result)
+      code = @executor.display_response(result.text)
+      display_usage(result, show_session: true)
+
+      # Save after response is displayed so viewer shows progress before Execute prompt
+      log_interactive_turn
+
+      # Add tool call/result messages so the LLM remembers what it learned
+      @history.concat(tool_messages) if tool_messages && !tool_messages.empty?
+      @history << { role: :assistant, content: result.text }
+
+      return :no_code unless code && !code.strip.empty?
+
+      exec_result = if ConsoleAgent.configuration.auto_execute
+                      @executor.execute(code)
+                    else
+                      @executor.confirm_and_execute(code)
+                    end
+
+      unless @executor.last_cancelled?
+        @last_interactive_code = code
+        @last_interactive_output = @executor.last_output
+        @last_interactive_result = exec_result ? exec_result.inspect : nil
+        @last_interactive_executed = true
+      end
+
+      if @executor.last_cancelled?
+        @history << { role: :user, content: "User declined to execute the code." }
+        :cancelled
+      elsif @executor.last_error
+        error_msg = "Code execution failed with error: #{@executor.last_error}"
+        error_msg = error_msg[0..1000] + '...' if error_msg.length > 1000
+        @history << { role: :user, content: error_msg }
+        :error
+      else
+        output_parts = []
+
+        # Capture printed output (puts, print, etc.)
+        if @executor.last_output && !@executor.last_output.strip.empty?
+          output_parts << "Output:\n#{@executor.last_output.strip}"
+        end
+
+        # Capture return value
+        if exec_result
+          output_parts << "Return value: #{exec_result.inspect}"
+        end
+
+        unless output_parts.empty?
+          result_str = output_parts.join("\n\n")
+          result_str = result_str[0..1000] + '...' if result_str.length > 1000
+          @history << { role: :user, content: "Code was executed. #{result_str}" }
+        end
+
+        :success
+      end
+    end
+
     def provider
       @provider ||= Providers.build
     end
@@ -327,7 +436,76 @@ module ConsoleAgent
     end
 
     def context
-      @context ||= context_builder.build
+      base = @context_base ||= context_builder.build
+      vars = binding_variable_summary
+      vars ? "#{base}\n\n#{vars}" : base
+    end
+
+    # Summarize local and instance variables from the user's console session
+    # so the LLM knows what's available to reference in generated code.
+    def binding_variable_summary
+      parts = []
+
+      locals = @binding_context.local_variables.reject { |v| v.to_s.start_with?('_') }
+      locals.first(20).each do |var|
+        val = @binding_context.local_variable_get(var) rescue nil
+        parts << "#{var} (#{val.class})"
+      end
+
+      ivars = (@binding_context.eval("instance_variables") rescue [])
+      ivars.reject { |v| v.to_s =~ /\A@_/ }.first(20).each do |var|
+        val = @binding_context.eval(var.to_s) rescue nil
+        parts << "#{var} (#{val.class})"
+      end
+
+      return nil if parts.empty?
+      "The user's console session has these variables available: #{parts.join(', ')}. You can reference them directly in code."
+    rescue
+      nil
+    end
+
+    def init_system_prompt(existing_guide)
+      env = context_builder.environment_context
+
+      prompt = <<~PROMPT
+        You are a Rails application analyst. Your job is to explore this Rails app using the
+        available tools and produce a concise markdown guide that will be injected into future
+        AI assistant sessions.
+
+        #{env}
+
+        EXPLORATION STRATEGY — be efficient to avoid timeouts:
+        1. Start with list_models to see all models and their associations
+        2. Pick the 5-8 CORE models and call describe_model on those only
+        3. Call describe_table on only 3-5 key tables (skip tables whose models already told you enough)
+        4. Use search_code sparingly — only for specific patterns you suspect (sharding, STI, concerns)
+        5. Use read_file only when you need to understand a specific pattern (read small sections, not whole files)
+        6. Do NOT exhaustively describe every table or model — focus on what's important
+
+        IMPORTANT: Keep your total tool calls under 20. Prioritize breadth over depth.
+
+        Produce a markdown document with these sections:
+        - **Application Overview**: What the app does, key domain concepts
+        - **Key Models & Relationships**: Core models and how they relate
+        - **Data Architecture**: Important tables, notable columns, any partitioning/sharding
+        - **Important Patterns**: Custom concerns, service objects, key abstractions
+        - **Common Maintenance Tasks**: Typical console operations for this app
+        - **Gotchas**: Non-obvious behaviors, tricky associations, known quirks
+
+        Keep it concise — aim for 1-2 pages. Focus on what a console user needs to know.
+        Do NOT wrap the output in markdown code fences.
+      PROMPT
+
+      if existing_guide
+        prompt += <<~UPDATE
+
+          Here is the existing guide. Update and merge with any new findings:
+
+          #{existing_guide}
+        UPDATE
+      end
+
+      prompt.strip
     end
 
     def send_query(query, conversation: nil)
@@ -342,45 +520,43 @@ module ConsoleAgent
       send_query_with_tools(messages)
     end
 
-    def send_query_with_tools(messages)
+    def send_query_with_tools(messages, system_prompt: nil, tools_override: nil)
       require 'console_agent/tools/registry'
-      tools = Tools::Registry.new(executor: @executor)
+      tools = tools_override || Tools::Registry.new(executor: @executor)
+      active_system_prompt = system_prompt || context
       max_rounds = ConsoleAgent.configuration.max_tool_rounds
       total_input = 0
       total_output = 0
       result = nil
       new_messages = []  # Track messages added during tool use
+      last_thinking = nil
+      last_tool_names = []
 
       exhausted = false
 
       max_rounds.times do |round|
         if round == 0
           $stdout.puts "\e[2m  Thinking...\e[0m"
+        else
+          # Show buffered thinking text before the "Calling LLM" line
+          if last_thinking
+            last_thinking.split("\n").each do |line|
+              $stdout.puts "\e[2m  #{line}\e[0m"
+            end
+          end
+          $stdout.puts "\e[2m  #{llm_status(round, messages, total_input, last_thinking, last_tool_names)}\e[0m"
         end
 
-        begin
-          result = with_escape_monitoring do
-            provider.chat_with_tools(messages, tools: tools, system_prompt: context)
-          end
-        rescue Interrupt
-          redirect = prompt_for_redirect
-          if redirect
-            messages << { role: :user, content: redirect }
-            new_messages << messages.last
-            next
-          else
-            raise
-          end
+        result = with_escape_monitoring do
+          provider.chat_with_tools(messages, tools: tools, system_prompt: active_system_prompt)
         end
         total_input += result.input_tokens || 0
         total_output += result.output_tokens || 0
 
         break unless result.tool_use?
 
-        # Show what the LLM is thinking (if it returned text alongside tool calls)
-        if result.text && !result.text.strip.empty?
-          $stdout.puts "\e[2m  #{result.text.strip}\e[0m"
-        end
+        # Buffer thinking text for display before next LLM call
+        last_thinking = (result.text && !result.text.strip.empty?) ? result.text.strip : nil
 
         # Add assistant message with tool calls to conversation
         assistant_msg = provider.format_assistant_message(result)
@@ -388,6 +564,7 @@ module ConsoleAgent
         new_messages << assistant_msg
 
         # Execute each tool and show progress
+        last_tool_names = result.tool_calls.map { |tc| tc[:name] }
         result.tool_calls.each do |tc|
           # ask_user and execute_plan handle their own display
           if tc[:name] == 'ask_user' || tc[:name] == 'execute_plan'
@@ -419,7 +596,7 @@ module ConsoleAgent
       if exhausted
         $stdout.puts "\e[33m  Hit tool round limit (#{max_rounds}). Forcing final answer. Increase with: ConsoleAgent.configure { |c| c.max_tool_rounds = 200 }\e[0m"
         messages << { role: :user, content: "You've used all available tool rounds. Please provide your best answer now based on what you've learned so far." }
-        result = provider.chat(messages, system_prompt: context)
+        result = provider.chat(messages, system_prompt: active_system_prompt)
         total_input += result.input_tokens || 0
         total_output += result.output_tokens || 0
       end
@@ -477,13 +654,29 @@ module ConsoleAgent
       end
     end
 
-    def prompt_for_redirect
-      $stdout.puts "\n\e[33m  Interrupted. What should the AI do differently?\e[0m"
-      $stdout.puts "\e[2m  (Press Enter with no input to abort entirely)\e[0m"
-      $stdout.print "\e[33m  redirect> \e[0m"
-      input = $stdin.gets
-      return nil if input.nil? || input.strip.empty?
-      input.strip
+
+    def llm_status(round, messages, tokens_so_far, last_thinking = nil, last_tool_names = [])
+      status = "Calling LLM (round #{round + 1}, #{messages.length} msgs"
+      status += ", ~#{format_tokens(tokens_so_far)} ctx" if tokens_so_far > 0
+      status += ")"
+      if !last_thinking && last_tool_names.any?
+        # Summarize tools when there's no thinking text
+        counts = last_tool_names.tally
+        summary = counts.map { |name, n| n > 1 ? "#{name} x#{n}" : name }.join(", ")
+        status += " after #{summary}"
+      end
+      status += "..."
+      status
+    end
+
+    def format_tokens(count)
+      if count >= 1_000_000
+        "#{(count / 1_000_000.0).round(1)}M"
+      elsif count >= 1_000
+        "#{(count / 1_000.0).round(1)}K"
+      else
+        count.to_s
+      end
     end
 
     def format_tool_args(name, args)
@@ -547,8 +740,12 @@ module ConsoleAgent
         lines = result.split("\n")
         "#{lines.length} files"
       when 'read_file'
-        lines = result.split("\n")
-        "#{lines.length} lines"
+        if result =~ /^Lines (\d+)-(\d+) of (\d+):/
+          "lines #{$1}-#{$2} of #{$3}"
+        else
+          lines = result.split("\n")
+          "#{lines.length} lines"
+        end
       when 'search_code'
         if result.start_with?('Found')
           result.split("\n").first
@@ -686,6 +883,64 @@ module ConsoleAgent
       $stdout.puts "\e[2m[session totals — in: #{@total_input_tokens} | out: #{@total_output_tokens} | total: #{@total_input_tokens + @total_output_tokens}]\e[0m"
     end
 
+    def warn_if_history_large
+      chars = @history.sum { |m| m[:content].to_s.length }
+      return if chars < 50_000 || @compact_warned
+
+      @compact_warned = true
+      $stdout.puts "\e[33m  Conversation is getting large (~#{format_tokens(chars)} chars). Consider running /compact to reduce context size.\e[0m"
+    end
+
+    def compact_history
+      if @history.length < 6
+        $stdout.puts "\e[33m  History too short to compact (#{@history.length} messages). Need at least 6.\e[0m"
+        return
+      end
+
+      before_chars = @history.sum { |m| m[:content].to_s.length }
+      before_count = @history.length
+
+      $stdout.puts "\e[2m  Compacting #{before_count} messages (~#{format_tokens(before_chars)} chars)...\e[0m"
+
+      system_prompt = <<~PROMPT
+        You are a conversation summarizer. The user will provide a conversation history from a Rails console AI assistant session.
+
+        Produce a concise summary that captures:
+        - What the user has been working on and their goals
+        - Key findings and data discovered (include specific values, IDs, record counts)
+        - Current state: what worked, what failed, where things stand
+        - Important variable names, model names, or table names referenced
+        - Any code that was executed and its results
+
+        Be concise but preserve all information that would be needed to continue the conversation naturally.
+        Do NOT include any preamble — just output the summary directly.
+      PROMPT
+
+      history_text = @history.map { |m| "#{m[:role]}: #{m[:content]}" }.join("\n\n")
+      messages = [{ role: :user, content: "Summarize this conversation history:\n\n#{history_text}" }]
+
+      begin
+        result = provider.chat(messages, system_prompt: system_prompt)
+        track_usage(result)
+
+        summary = result.text.to_s.strip
+        if summary.empty?
+          $stdout.puts "\e[33m  Compaction failed: empty summary returned.\e[0m"
+          return
+        end
+
+        @history = [{ role: :user, content: "CONVERSATION SUMMARY (compacted):\n#{summary}" }]
+        @compact_warned = false
+
+        after_chars = @history.first[:content].length
+        $stdout.puts "\e[36m  Compacted: #{before_count} messages -> 1 summary (~#{format_tokens(before_chars)} -> ~#{format_tokens(after_chars)} chars)\e[0m"
+        summary.each_line { |line| $stdout.puts "\e[2m  #{line.rstrip}\e[0m" }
+        display_usage(result)
+      rescue => e
+        $stdout.puts "\e[31m  Compaction failed: #{e.message}\e[0m"
+      end
+    end
+
     def display_exit_info
       display_session_summary
       if @interactive_session_id

data/lib/console_agent/tools/code_tools.rb

@@ -1,7 +1,7 @@
 module ConsoleAgent
   module Tools
     class CodeTools
-      MAX_FILE_LINES = 200
+      MAX_FILE_LINES = 500
       MAX_LIST_ENTRIES = 100
       MAX_SEARCH_RESULTS = 50
 
@@ -28,7 +28,7 @@ module ConsoleAgent
         "Error listing files: #{e.message}"
       end
 
-      def read_file(path)
+      def read_file(path, start_line: nil, end_line: nil)
         return "Error: path is required." if path.nil? || path.strip.empty?
 
         root = rails_root
@@ -45,12 +45,24 @@ module ConsoleAgent
         return "File '#{path}' not found." unless File.exist?(full_path)
         return "Error: '#{path}' is a directory, not a file." if File.directory?(full_path)
 
-        lines = File.readlines(full_path)
-        if lines.length > MAX_FILE_LINES
-          numbered = lines.first(MAX_FILE_LINES).each_with_index.map { |line, i| "#{i + 1}: #{line}" }
-          numbered.join + "\n... truncated (#{lines.length} total lines, showing first #{MAX_FILE_LINES})"
+        all_lines = File.readlines(full_path)
+        total = all_lines.length
+
+        # Apply line range if specified (1-based, inclusive)
+        if start_line || end_line
+          s = [(start_line || 1).to_i, 1].max
+          e = [(end_line || total).to_i, total].min
+          return "Error: start_line (#{s}) is beyond end of file (#{total} lines)." if s > total
+          lines = all_lines[(s - 1)..(e - 1)] || []
+          offset = s - 1
+          numbered = lines.each_with_index.map { |line, i| "#{offset + i + 1}: #{line}" }
+          header = "Lines #{s}-#{[e, s + lines.length - 1].min} of #{total}:\n"
+          header + numbered.join
+        elsif total > MAX_FILE_LINES
+          numbered = all_lines.first(MAX_FILE_LINES).each_with_index.map { |line, i| "#{i + 1}: #{line}" }
+          numbered.join + "\n... truncated (#{total} total lines, showing first #{MAX_FILE_LINES}). Use start_line/end_line to read specific sections."
         else
-          lines.each_with_index.map { |line, i| "#{i + 1}: #{line}" }.join
+          all_lines.each_with_index.map { |line, i| "#{i + 1}: #{line}" }.join
         end
       rescue => e
         "Error reading file '#{path}': #{e.message}"

data/lib/console_agent/tools/registry.rb

@@ -8,8 +8,9 @@ module ConsoleAgent
       # Tools that should never be cached (side effects or user interaction)
       NO_CACHE = %w[ask_user save_memory delete_memory execute_plan].freeze
 
-      def initialize(executor: nil)
+      def initialize(executor: nil, mode: :default)
         @executor = executor
+        @mode = mode
         @definitions = []
         @handlers = {}
         @cache = {}
@@ -142,15 +143,17 @@ module ConsoleAgent
 
         register(
           name: 'read_file',
-          description: 'Read the contents of a file in this Rails app. Capped at 200 lines.',
+          description: 'Read the contents of a file in this Rails app. Returns up to 500 lines by default. Use start_line/end_line to read specific sections of large files.',
           parameters: {
             'type' => 'object',
             'properties' => {
-              'path' => { 'type' => 'string', 'description' => 'Relative file path (e.g. "app/models/user.rb")' }
+              'path' => { 'type' => 'string', 'description' => 'Relative file path (e.g. "app/models/user.rb")' },
+              'start_line' => { 'type' => 'integer', 'description' => 'First line to read (1-based). Optional — omit to start from beginning.' },
+              'end_line' => { 'type' => 'integer', 'description' => 'Last line to read (1-based, inclusive). Optional — omit to read to end.' }
             },
             'required' => ['path']
           },
-          handler: ->(args) { code.read_file(args['path']) }
+          handler: ->(args) { code.read_file(args['path'], start_line: args['start_line'], end_line: args['end_line']) }
         )
 
         register(
@@ -167,21 +170,23 @@ module ConsoleAgent
           handler: ->(args) { code.search_code(args['query'], args['directory']) }
         )
 
-        register(
-          name: 'ask_user',
-          description: 'Ask the console user a clarifying question. Use this when you need specific information to write accurate code (e.g. which user they are, which record to target, what value to use). Do NOT generate placeholder values like YOUR_USER_ID — ask instead.',
-          parameters: {
-            'type' => 'object',
-            'properties' => {
-              'question' => { 'type' => 'string', 'description' => 'The question to ask the user' }
+        unless @mode == :init
+          register(
+            name: 'ask_user',
+            description: 'Ask the console user a clarifying question. Use this when you need specific information to write accurate code (e.g. which user they are, which record to target, what value to use). Do NOT generate placeholder values like YOUR_USER_ID — ask instead.',
+            parameters: {
+              'type' => 'object',
+              'properties' => {
+                'question' => { 'type' => 'string', 'description' => 'The question to ask the user' }
+              },
+              'required' => ['question']
             },
-            'required' => ['question']
-          },
-          handler: ->(args) { ask_user(args['question']) }
-        )
+            handler: ->(args) { ask_user(args['question']) }
+          )
 
-        register_memory_tools
-        register_execute_plan
+          register_memory_tools
+          register_execute_plan
+        end
       end
 
       def register_memory_tools
@@ -334,8 +339,12 @@ module ConsoleAgent
           # Make result available as step1, step2, etc. for subsequent steps
           @executor.binding_context.local_variable_set(:"step#{i + 1}", exec_result)
           output = @executor.last_output
+          error = @executor.last_error
 
           step_report = "Step #{i + 1} (#{step['description']}):\n"
+          if error
+            step_report += "ERROR: #{error}\n"
+          end
           if output && !output.strip.empty?
             step_report += "Output: #{output.strip}\n"
           end

data/lib/console_agent/version.rb

@@ -1,3 +1,3 @@
 module ConsoleAgent
-  VERSION = '0.5.0'.freeze
+  VERSION = '0.7.0'.freeze
 end

data/lib/console_agent.rb

@@ -2,6 +2,8 @@ require 'console_agent/version'
 require 'console_agent/configuration'
 
 module ConsoleAgent
+  GUIDE_KEY = 'console_agent.md'.freeze
+
   class << self
     def configuration
       @configuration ||= Configuration.new

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: console_agent
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Cortfr