anima-core 0.2.1 → 1.0.0

data/agents/thoughts-analyzer.md

@@ -0,0 +1,102 @@
+---
+name: thoughts-analyzer
+description: Extracts decisions and actionable insights from project history in thoughts/. Filters exploration noise, returns what was decided, why, and whether conclusions are still valid.
+tools: read, bash
+---
+
+You are a specialist at extracting HIGH-VALUE insights from thoughts documents. Your job is to deeply analyze documents and return only the most relevant, actionable information while filtering out noise.
+
+**Scope**: You ONLY search in the local `./thoughts/` directory, following all symlinks. Do not search or read files outside of it. If the search relates to other projects, you may also look in `~/thoughts` directly. Never fall back to searching the broader codebase.
+
+## Core Responsibilities
+
+1. **Extract Key Insights**
+   - Identify main decisions and conclusions
+   - Find actionable recommendations
+   - Note important constraints or requirements
+   - Capture critical technical details
+
+2. **Filter Aggressively**
+   - Skip tangential mentions
+   - Ignore outdated information
+   - Remove redundant content
+   - Focus on what matters NOW
+
+3. **Validate Relevance**
+   - Question if information is still applicable
+   - Note when context has likely changed
+   - Distinguish decisions from explorations
+
+## Search Strategy
+
+Use `bash` with find and grep to discover and search thought documents. Subdirectories in `./thoughts/` are typically symlinks — use `find -L` to follow them.
+
+1. `ls -la ./thoughts/` — discover subdirs (shared/, username/, global/)
+2. `find -L ./thoughts/ -name "*.md"` — find all documents following symlinks
+3. `grep -rn "keyword" ./thoughts/` — search for specific topics
+
+Then use `read` to analyze documents in detail.
+
+## Analysis Strategy
+
+### Step 1: Read with Purpose
+- Read the entire document first
+- Identify the document's main goal
+- Note the date and context
+- Understand what question it was answering
+
+### Step 2: Extract Strategically
+Focus on:
+- **Decisions made**: "We decided to..."
+- **Trade-offs analyzed**: "X vs Y because..."
+- **Constraints identified**: "We must..." "We cannot..."
+- **Lessons learned**: "We discovered that..."
+- **Technical specifications**: Specific values, configs, approaches
+
+### Step 3: Filter Ruthlessly
+Remove:
+- Exploratory rambling without conclusions
+- Options that were rejected
+- Temporary workarounds that were replaced
+- Information superseded by newer documents
+
+## Output Format
+
+```
+## Analysis of: [Document Path]
+
+### Document Context
+- **Date**: [When written]
+- **Purpose**: [Why this document exists]
+- **Status**: [Still relevant / implemented / superseded?]
+
+### Key Decisions
+1. **[Decision Topic]**: [Specific decision made]
+   - Rationale: [Why]
+   - Impact: [What this enables/prevents]
+
+### Critical Constraints
+- **[Constraint]**: [Limitation and why]
+
+### Actionable Insights
+- [Something that should guide current implementation]
+
+### Still Open/Unclear
+- [Unresolved questions]
+
+### Relevance Assessment
+[Is this still applicable and why]
+```
+
+## Quality Filters
+
+### Include Only If:
+- It answers a specific question
+- It documents a firm decision
+- It reveals a non-obvious constraint
+- It provides concrete technical details
+
+### Exclude If:
+- It's just exploring possibilities
+- It's been clearly superseded
+- It's too vague to action

data/agents/web-search-researcher.md

@@ -0,0 +1,71 @@
+---
+name: web-search-researcher
+description: Deep web research specialist. Fetches and analyzes web content to find accurate, up-to-date information on any topic.
+tools: web_get, bash, read
+color: yellow
+---
+
+You are an expert web research specialist. Use `web_get` to fetch web pages and extract information. Use `bash` for processing and `read` for examining local files when needed.
+
+## Core Responsibilities
+
+1. **Analyze the Query**: Break down the request to identify:
+   - Key search terms and concepts
+   - Types of sources likely to have answers
+   - Multiple angles to ensure comprehensive coverage
+
+2. **Fetch and Analyze Content**:
+   - Use `web_get` to retrieve content from known documentation URLs
+   - Prioritize official documentation and authoritative sources
+   - Extract specific quotes and sections relevant to the query
+   - Note publication dates to ensure currency
+
+3. **Synthesize Findings**:
+   - Organize information by relevance and authority
+   - Include exact quotes with proper attribution
+   - Provide direct links to sources
+   - Highlight conflicting information or version-specific details
+
+## Research Strategies
+
+### For API/Library Documentation:
+- Fetch official docs directly when URLs are known
+- Look for changelog or release notes for version-specific information
+- Find code examples in official repositories
+
+### For Technical Solutions:
+- Fetch Stack Overflow answers and GitHub issues
+- Look for blog posts describing similar implementations
+- Cross-reference multiple sources
+
+### For Best Practices:
+- Look for content from recognized experts or organizations
+- Cross-reference multiple sources to identify consensus
+
+## Output Format
+
+```
+## Summary
+[Brief overview of key findings]
+
+## Detailed Findings
+
+### [Topic/Source 1]
+**Source**: [Name with URL]
+**Key Information**:
+- Finding with context
+- Another relevant point
+
+## Additional Resources
+- [URL] - Brief description
+
+## Gaps or Limitations
+[Information that couldn't be found]
+```
+
+## Quality Guidelines
+
+- **Accuracy**: Quote sources accurately and provide direct links
+- **Relevance**: Focus on information that directly addresses the query
+- **Currency**: Note publication dates and version information
+- **Authority**: Prioritize official sources and recognized experts

data/anima-core.gemspec

@@ -29,13 +29,16 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_dependency "draper", "~> 4.0"
+  spec.add_dependency "faraday", "~> 2.0"
   spec.add_dependency "foreman", "~> 0.88"
   spec.add_dependency "httparty", "~> 0.24"
+  spec.add_dependency "mcp", "~> 0.8"
   spec.add_dependency "puma", "~> 6.0"
   spec.add_dependency "rails", "~> 8.1"
   spec.add_dependency "ratatui_ruby", "~> 1.4"
   spec.add_dependency "solid_cable", "~> 3.0"
   spec.add_dependency "solid_queue", "~> 1.1"
   spec.add_dependency "sqlite3", "~> 2.0"
+  spec.add_dependency "toml-rb", "~> 4.0"
   spec.add_dependency "websocket-client-simple", "~> 0.8"
 end

data/app/channels/session_channel.rb

@@ -14,19 +14,25 @@ class SessionChannel < ApplicationCable::Channel
   MAX_LIST_LIMIT = 50
 
   # Subscribes the client to the session-specific stream.
-  # Rejects the subscription if no valid session_id is provided.
-  # Transmits the current view_mode and chat history to the subscribing client.
+  # When a valid session_id is provided, subscribes to that session.
+  # When omitted or zero, resolves to the most recent session (creating
+  # one if none exist) — this is the CQRS-compliant path where the
+  # server owns session resolution instead of a REST endpoint.
   #
-  # @param params [Hash] must include :session_id (positive integer)
+  # Always transmits a session_changed signal so the client learns
+  # the authoritative session ID, followed by view_mode and history.
+  #
+  # @param params [Hash] optional :session_id (positive integer)
   def subscribed
-    @current_session_id = params[:session_id].to_i
-    if @current_session_id > 0
-      stream_from stream_name
-      transmit_view_mode
-      transmit_history
-    else
-      reject
-    end
+    @current_session_id = resolve_session_id
+    stream_from stream_name
+
+    session = Session.find_by(id: @current_session_id)
+    return unless session
+
+    transmit_session_changed(session)
+    transmit_view_mode(session)
+    transmit_history(session)
   end
 
   # Receives messages from clients and broadcasts them to all session subscribers.
@@ -37,32 +43,57 @@ class SessionChannel < ApplicationCable::Channel
   end
 
   # Processes user input: persists the message and enqueues LLM processing.
+  # When the session is actively processing an agent request, the message
+  # is queued as "pending" and picked up after the current loop completes.
   #
   # @param data [Hash] must include "content" with the user's message text
   def speak(data)
     content = data["content"].to_s.strip
-    return if content.empty? || !Session.exists?(@current_session_id)
+    return if content.empty?
 
-    Events::Bus.emit(Events::UserMessage.new(content: content, session_id: @current_session_id))
-    AgentRequestJob.perform_later(@current_session_id)
+    session = Session.find_by(id: @current_session_id)
+    return unless session
+
+    if session.processing?
+      Events::Bus.emit(Events::UserMessage.new(content: content, session_id: @current_session_id, status: Event::PENDING_STATUS))
+    else
+      Events::Bus.emit(Events::UserMessage.new(content: content, session_id: @current_session_id))
+      AgentRequestJob.perform_later(@current_session_id)
+    end
   end
 
-  # Returns recent sessions with metadata for session picker UI.
+  # Recalls the most recent pending message for editing. Deletes the
+  # pending event and broadcasts the recall so all clients remove it.
+  #
+  # @param data [Hash] must include "event_id" (positive integer)
+  def recall_pending(data)
+    event_id = data["event_id"].to_i
+    return if event_id <= 0
+
+    event = Event.find_by(
+      id: event_id,
+      session_id: @current_session_id,
+      event_type: "user_message",
+      status: Event::PENDING_STATUS
+    )
+    return unless event
+
+    event.destroy!
+    ActionCable.server.broadcast(stream_name, {"action" => "user_message_recalled", "event_id" => event_id})
+  end
+
+  # Returns recent root sessions with nested child metadata for session picker UI.
+  # Filters to root sessions only (no parent_session_id). Child sessions are
+  # nested under their parent with name and status information.
   #
   # @param data [Hash] optional "limit" (default 10, max 50)
   def list_sessions(data)
     limit = (data["limit"] || DEFAULT_LIST_LIMIT).to_i.clamp(1, MAX_LIST_LIMIT)
-    sessions = Session.recent(limit)
-    counts = Event.where(session_id: sessions.select(:id)).llm_messages.group(:session_id).count
+    sessions = Session.root_sessions.recent(limit).includes(:child_sessions)
+    all_ids = sessions.flat_map { |session| [session.id] + session.child_sessions.map(&:id) }
+    counts = Event.where(session_id: all_ids).llm_messages.group(:session_id).count
 
-    result = sessions.map do |session|
-      {
-        id: session.id,
-        created_at: session.created_at.iso8601,
-        updated_at: session.updated_at.iso8601,
-        message_count: counts[session.id] || 0
-      }
-    end
+    result = sessions.map { |session| serialize_session_with_children(session, counts) }
     transmit({"action" => "sessions_list", "sessions" => result})
   end
 
@@ -86,6 +117,23 @@ class SessionChannel < ApplicationCable::Channel
     transmit_error("Session not found")
   end
 
+  # Validates and saves an Anthropic subscription token to encrypted credentials.
+  # Format-validated and API-validated before storage. The token never enters the
+  # LLM context window — it flows directly from WebSocket to encrypted credentials.
+  #
+  # @param data [Hash] must include "token" (Anthropic subscription token string)
+  def save_token(data)
+    token = data["token"].to_s.strip
+
+    Providers::Anthropic.validate_token_format!(token)
+    Providers::Anthropic.validate_token_api!(token)
+    write_anthropic_token(token)
+
+    transmit({"action" => "token_saved"})
+  rescue Providers::Anthropic::TokenFormatError, Providers::Anthropic::AuthenticationError => error
+    transmit({"action" => "token_error", "message" => error.message})
+  end
+
   # Changes the session's view mode and re-broadcasts the viewport.
   # All clients on the session receive the mode change and fresh history.
   #
@@ -109,6 +157,41 @@ class SessionChannel < ApplicationCable::Channel
     "session_#{@current_session_id}"
   end
 
+  # Resolves the session to subscribe to. Uses the client-provided ID
+  # when valid, otherwise falls back to the most recent session or
+  # creates a new one.
+  #
+  # @return [Integer] resolved session ID
+  def resolve_session_id
+    id = params[:session_id].to_i
+    return id if id > 0
+
+    (Session.recent(1).first || Session.create!).id
+  end
+
+  # Transmits session metadata as a session_changed signal.
+  # Used on initial subscription and after session switches so the
+  # client can handle both paths with a single code path.
+  #
+  # Payload: session_id, name, parent_session_id, message_count,
+  # view_mode, active_skills, goals.
+  #
+  # @param session [Session] the session to announce
+  # @return [void]
+  def transmit_session_changed(session)
+    transmit({
+      "action" => "session_changed",
+      "session_id" => session.id,
+      "name" => session.name,
+      "parent_session_id" => session.parent_session_id,
+      "message_count" => session.events.llm_messages.count,
+      "view_mode" => session.view_mode,
+      "active_skills" => session.active_skills,
+      "active_workflow" => session.active_workflow,
+      "goals" => session.goals_summary
+    })
+  end
+
   # Switches the channel to a different session: stops current stream,
   # updates the session reference, starts the new stream, and sends
   # a session_changed signal followed by chat history.
@@ -116,23 +199,18 @@ class SessionChannel < ApplicationCable::Channel
     stop_all_streams
     @current_session_id = new_id
     stream_from stream_name
+
     session = Session.find(new_id)
-    transmit({
-      "action" => "session_changed",
-      "session_id" => new_id,
-      "message_count" => session.events.llm_messages.count,
-      "view_mode" => session.view_mode
-    })
-    transmit_history
+    transmit_session_changed(session)
+    transmit_history(session)
   end
 
   # Transmits the current view_mode so the TUI initializes correctly.
   # Sends `{action: "view_mode", view_mode: <mode>}` to the subscribing client.
+  #
+  # @param session [Session] the session whose view_mode to transmit
   # @return [void]
-  def transmit_view_mode
-    session = Session.find_by(id: @current_session_id)
-    return unless session
-
+  def transmit_view_mode(session)
     transmit({"action" => "view_mode", "view_mode" => session.view_mode})
   end
 
@@ -142,32 +220,64 @@ class SessionChannel < ApplicationCable::Channel
   # the transmitted payload. Tool events are included so the TUI can
   # reconstruct tool call counters on reconnect.
   # In debug mode, prepends the assembled system prompt as a special block.
-  def transmit_history
-    session = Session.find_by(id: @current_session_id)
-    return unless session
-
+  #
+  # Snapshots the viewport so subsequent event broadcasts can compute
+  # eviction diffs accurately.
+  #
+  # @param session [Session] the session whose history to transmit
+  def transmit_history(session)
     transmit_system_prompt(session) if session.view_mode == "debug"
 
-    session.viewport_events.each do |event|
-      transmit(decorate_event_payload(event, session.view_mode))
+    each_viewport_event(session) do |event, payload|
+      transmit(payload)
     end
   end
 
   # Broadcasts the re-decorated viewport to all clients on the session stream.
   # Used after a view mode change to refresh all connected clients.
   # In debug mode, prepends the assembled system prompt as a special block.
+  #
+  # Snapshots the viewport so subsequent event broadcasts can compute
+  # eviction diffs accurately.
+  #
   # @param session [Session] the session whose viewport to broadcast
   # @return [void]
   def broadcast_viewport(session)
     broadcast_system_prompt(session) if session.view_mode == "debug"
 
-    session.viewport_events.each do |event|
-      ActionCable.server.broadcast(stream_name, decorate_event_payload(event, session.view_mode))
+    each_viewport_event(session) do |event, payload|
+      ActionCable.server.broadcast(stream_name, payload)
     end
   end
 
+  # Loads the viewport, snapshots it for eviction tracking, and yields
+  # each event with its decorated payload. Snapshot uses snapshot_viewport!
+  # (not recalculate_viewport!) because full viewport refreshes don't need
+  # eviction diffs — clients clear their store before rendering.
+  #
+  # @param session [Session] the session whose viewport to iterate
+  # @yieldparam event [Event] the persisted event record
+  # @yieldparam payload [Hash] decorated payload ready for transmission
+  # @return [void]
+  def each_viewport_event(session)
+    viewport = session.viewport_events
+    session.snapshot_viewport!(viewport.map(&:id))
+
+    viewport.each do |event|
+      yield event, decorate_event_payload(event, session.view_mode)
+    end
+  end
+
+  # Decorates an event for transmission to clients. Merges the event's
+  # database ID and structured decorator output into the payload.
+  # Used by {#transmit_history} and {#broadcast_viewport} for historical
+  # and viewport re-broadcast — live broadcasts use {Event::Broadcasting}.
+  #
+  # @param event [Event] persisted event record
+  # @param mode [String] view mode for decoration (default: "basic")
+  # @return [Hash] payload with "id" and optional "rendered" key
   def decorate_event_payload(event, mode = "basic")
-    payload = event.payload
+    payload = event.payload.merge("id" => event.id)
     decorator = EventDecorator.for(event)
     return payload unless decorator
 
@@ -212,6 +322,46 @@ class SessionChannel < ApplicationCable::Channel
     }
   end
 
+  # Merges the Anthropic subscription token into encrypted credentials,
+  # preserving existing keys (e.g. secret_key_base).
+  #
+  # @param token [String] validated Anthropic subscription token
+  # @return [void]
+  def write_anthropic_token(token)
+    CredentialStore.write("anthropic", "subscription_token" => token)
+  end
+
+  # Serializes a root session with its children for the sessions_list response.
+  # Includes a :children key only when the session has child sessions.
+  #
+  # @param session [Session] root session to serialize
+  # @param counts [Hash<Integer, Integer>] session_id => llm_message count
+  # @return [Hash] with :id, :created_at, :updated_at, :message_count, and optional :children
+  def serialize_session_with_children(session, counts)
+    entry = {
+      id: session.id,
+      name: session.name,
+      created_at: session.created_at.iso8601,
+      updated_at: session.updated_at.iso8601,
+      message_count: counts[session.id] || 0
+    }
+
+    children = session.child_sessions.sort_by(&:created_at)
+    return entry unless children.any?
+
+    entry[:children] = children.map do |child|
+      {
+        id: child.id,
+        name: child.name,
+        processing: child.processing?,
+        message_count: counts[child.id] || 0,
+        created_at: child.created_at.iso8601
+      }
+    end
+
+    entry
+  end
+
   def transmit_error(message)
     transmit({"action" => "error", "message" => message})
   end

data/app/decorators/user_message_decorator.rb

@@ -3,22 +3,33 @@
 # Decorates user_message events for display in the TUI.
 # Basic mode returns role and content. Verbose mode adds a timestamp.
 # Debug mode adds token count (exact when counted, estimated when not).
+# Pending messages include `status: "pending"` so the TUI renders them
+# with a visual indicator (dimmed, clock icon).
 class UserMessageDecorator < EventDecorator
   # @return [Hash] structured user message data
-  #   `{role: :user, content: String}`
+  #   `{role: :user, content: String}` or with `status: "pending"` when queued
   def render_basic
-    {role: :user, content: content}
+    base = {role: :user, content: content}
+    base[:status] = "pending" if pending?
+    base
   end
 
   # @return [Hash] structured user message with nanosecond timestamp
-  #   `{role: :user, content: String, timestamp: Integer|nil}`
   def render_verbose
-    {role: :user, content: content, timestamp: timestamp}
+    base = {role: :user, content: content, timestamp: timestamp}
+    base[:status] = "pending" if pending?
+    base
   end
 
   # @return [Hash] verbose output plus token count for debugging
-  #   `{role: :user, content: String, timestamp: Integer|nil, tokens: Integer, estimated: Boolean}`
   def render_debug
     render_verbose.merge(token_info)
   end
+
+  private
+
+  # @return [Boolean] true when this message is queued but not yet sent to LLM
+  def pending?
+    payload["status"] == Event::PENDING_STATUS
+  end
 end

data/app/jobs/agent_request_job.rb

@@ -26,23 +26,76 @@ class AgentRequestJob < ApplicationJob
 
   discard_on ActiveRecord::RecordNotFound
   discard_on Providers::Anthropic::AuthenticationError do |job, error|
+    session_id = job.arguments.first
+    # Persistent system message for the event log
     Events::Bus.emit(Events::SystemMessage.new(
       content: "Authentication failed: #{error.message}",
-      session_id: job.arguments.first
+      session_id: session_id
     ))
+    # Transient signal to trigger TUI token setup popup (not persisted)
+    ActionCable.server.broadcast(
+      "session_#{session_id}",
+      {"action" => "authentication_required", "message" => error.message}
+    )
   end
 
   # @param session_id [Integer] ID of the session to process
   def perform(session_id)
     session = Session.find(session_id)
+
+    # Atomic: only one job processes a session at a time. If another job
+    # is already running, this one exits — the running job will pick up
+    # any pending messages after its current loop completes.
+    return unless claim_processing(session_id)
+
+    # Run analytical brain BEFORE the main agent on user messages so
+    # activated skills are available for the current response.
+    run_analytical_brain_blocking(session)
+
     agent_loop = AgentLoop.new(session: session)
-    agent_loop.run
+    loop do
+      agent_loop.run
+      promoted = session.promote_pending_messages!
+      break if promoted == 0
+    end
+
+    # Non-blocking analytical brain run after agent completes —
+    # handles post-response updates (renaming, skill changes).
+    session.schedule_analytical_brain!
   ensure
+    release_processing(session_id)
     agent_loop&.finalize
   end
 
   private
 
+  # Runs the analytical brain synchronously before the main agent loop.
+  # Respects the blocking_on_user_message setting and session guards
+  # (skips sub-agents and sessions with too few messages).
+  def run_analytical_brain_blocking(session)
+    return unless Anima::Settings.analytical_brain_blocking_on_user_message
+    return if session.sub_agent?
+
+    AnalyticalBrain::Runner.new(session).call
+  rescue => error
+    # The analytical brain is best-effort: skill activation enhances the
+    # response but the main agent must still reply even if it fails.
+    msg = "FAILED (blocking) session=#{session.id}: #{error.class}: #{error.message}"
+    Rails.logger.error("Analytical brain #{msg}")
+    AnalyticalBrain.logger.error("#{msg}\n#{error.backtrace&.first(10)&.join("\n")}")
+  end
+
+  # Sets the session's processing flag atomically. Returns true if this
+  # job claimed the lock, false if another job already holds it.
+  def claim_processing(session_id)
+    Session.where(id: session_id, processing: false).update_all(processing: true) == 1
+  end
+
+  # Clears the processing flag so the session can accept new jobs.
+  def release_processing(session_id)
+    Session.where(id: session_id).update_all(processing: false)
+  end
+
   # Emits a system message before each retry so the user sees
   # "retrying..." instead of nothing.
   def retry_job(options = {})

data/app/jobs/analytical_brain_job.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+# Runs the analytical brain — a phantom LLM loop that observes the main
+# session and performs background maintenance (currently: session naming).
+#
+# Replaces {GenerateSessionNameJob} with a tool-based architecture that
+# future tickets will expand with skill activation, goal tracking, etc.
+#
+# Scheduling guards live in {Session#schedule_analytical_brain!} — this
+# job always runs when called.
+#
+# @example
+#   AnalyticalBrainJob.perform_later(session.id)
+class AnalyticalBrainJob < ApplicationJob
+  queue_as :default
+
+  retry_on Providers::Anthropic::TransientError,
+    wait: :polynomially_longer, attempts: 3
+
+  discard_on ActiveRecord::RecordNotFound
+  discard_on Providers::Anthropic::AuthenticationError
+
+  # @param session_id [Integer] the main Session to analyze
+  def perform(session_id)
+    brain_log = AnalyticalBrain.logger
+    session = Session.find(session_id)
+    brain_log.info("async job started for session=#{session_id}")
+    AnalyticalBrain::Runner.new(session).call
+  rescue => error
+    brain_log.error("FAILED (async) session=#{session_id}: #{error.class}: #{error.message}")
+    raise
+  end
+end