lex-apollo 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: fcb22d66eb9b08e01ececa39900c455be2aa64f358a2976878c0b2934b71670a
+  data.tar.gz: 1d2d41cf8835c04e14827e22caaed2a848d0c8fc4c41cef3aeecc927041849b3
+SHA512:
+  metadata.gz: 6bff39d97c42ca8085b7937066e38d52c50fe302cd1d04e80d8a4aca6762eb5c28b33eb05a0ea16adf391b86bffa3d0936f3d83caf3d936d5703e8c08bd0140c
+  data.tar.gz: 694442a97667f0355bba359938c7bd9317f9d2308a6ef169a54c59bc63e097c7786625a525b33defe44581bc20593cba259ff68be019c6c55708b6ad7f31e43a

data/CHANGELOG.md

@@ -0,0 +1,25 @@
+# Changelog
+
+## [0.2.0] - 2026-03-16
+
+### Added
+- `Helpers::Embedding` — embedding generation wrapper with legion-llm + zero-vector fallback
+- `Knowledge.handle_ingest` — server-side ingest: embedding, corroboration check, entry creation, expertise upsert
+- `Knowledge.handle_query` — server-side query: semantic search via pgvector, retrieval boost, access logging
+- `Knowledge.retrieve_relevant` — GAIA tick phase handler for knowledge_retrieval (phase 4)
+- `Maintenance.check_corroboration` — periodic scan promoting candidates to confirmed via similarity threshold
+- `Expertise.aggregate` — periodic proficiency recalculation using log2-weighted average confidence
+
+## [0.1.0] - 2026-03-15
+
+### Added
+- Initial scaffold with helpers, runners, actors, transport, and standalone client
+- Confidence helper with decay, boost, and write gate logic
+- Similarity helper with cosine distance and corroboration classification
+- Graph query builder for recursive CTE traversal and pgvector semantic search
+- Knowledge, Expertise, and Maintenance runners (client-side RMQ payloads)
+- Ingest, QueryResponder, Decay, ExpertiseAggregator, CorroborationChecker actors
+- Transport layer: apollo exchange, ingest/query queues, ingest/query messages
+- Standalone Client with agent_id injection
+- GAIA tick integration: knowledge_retrieval phase (phase 4)
+- legion-data migration 012 with PostgreSQL+pgvector tables (guarded)

data/README.md

@@ -0,0 +1,135 @@
+# lex-apollo
+
+Shared durable knowledge store for the GAIA cognitive mesh. Agents publish confirmed knowledge via RabbitMQ; a dedicated Apollo service persists to PostgreSQL+pgvector. Supports semantic search, concept graph traversal, and expertise tracking.
+
+## Overview
+
+`lex-apollo` operates in two modes:
+
+- **Client mode**: Any agent loads this gem and calls runners. Runners publish to RabbitMQ — no direct database access required.
+- **Service mode**: A dedicated Apollo process runs the actors, subscribes to queues, generates embeddings, and writes to PostgreSQL+pgvector.
+
+The backing store is Azure Database for PostgreSQL Flexible Server with the pgvector extension.
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem 'lex-apollo'
+```
+
+## Usage
+
+### Standalone Client
+
+```ruby
+require 'legion/extensions/apollo'
+
+client = Legion::Extensions::Apollo::Client.new(agent_id: 'my-agent-001')
+
+# Store a confirmed knowledge entry
+client.store_knowledge(
+  domain: 'networking',
+  content: 'BGP route reflectors reduce full-mesh IBGP complexity',
+  confidence: 0.9,
+  source_agent_id: 'my-agent-001',
+  tags: ['bgp', 'routing', 'ibgp']
+)
+
+# Query for relevant knowledge
+client.query_knowledge(
+  query: 'BGP route reflector configuration',
+  domain: 'networking',
+  min_confidence: 0.6,
+  limit: 10
+)
+
+# Get related entries (concept graph traversal)
+client.related_entries(entry_id: 'entry-uuid', max_hops: 2)
+
+# Deprecate a stale entry
+client.deprecate_entry(entry_id: 'entry-uuid', reason: 'superseded by RFC 7938')
+```
+
+### Expertise Queries
+
+```ruby
+# Get proficiency scores for a domain
+client.get_expertise(domain: 'networking', agent_id: 'my-agent-001')
+
+# Find domains where knowledge coverage is thin
+client.domains_at_risk(min_entries: 5, min_confidence: 0.7)
+
+# Full agent knowledge profile
+client.agent_profile(agent_id: 'my-agent-001')
+```
+
+### Maintenance
+
+```ruby
+# Force confidence decay cycle
+client.force_decay(domain: 'networking')
+
+# Archive entries below confidence threshold
+client.archive_stale(max_confidence: 0.2)
+
+# Resolve a corroboration dispute
+client.resolve_dispute(entry_id: 'entry-uuid', resolution: :accept)
+```
+
+## Architecture
+
+### Client Mode
+
+Runners build structured payloads and publish to the `apollo` exchange via RabbitMQ. No PostgreSQL or pgvector dependency is needed in the calling agent. Transport requires `Legion::Transport` to be loaded (the `if defined?(Legion::Transport)` guard in the entry point handles this automatically).
+
+### Service Mode
+
+Five actors run in the dedicated Apollo service process:
+
+| Actor | Type | Interval | Purpose |
+|---|---|---|---|
+| `Ingest` | Subscription | on-message | Receive knowledge, generate embeddings, persist to PostgreSQL |
+| `QueryResponder` | Subscription | on-message | Handle semantic queries, return results via RPC |
+| `Decay` | Interval | 3600s | Confidence decay cycle across all entries |
+| `ExpertiseAggregator` | Interval | 1800s | Recalculate domain proficiency scores |
+| `CorroborationChecker` | Interval | 900s | Scan pending entries for auto-confirm threshold |
+
+### GAIA Tick Integration
+
+Apollo is wired into the GAIA tick cycle at the `knowledge_retrieval` phase (phase 4), which fires after `memory_retrieval` and before `working_memory_integration`. It activates only when local memory lacks high-confidence matches for the current tick context.
+
+## Confidence Model
+
+Entries have a confidence score between 0.0 and 1.0:
+
+- New entries start at the caller-supplied confidence value
+- Corroboration from multiple agents boosts confidence
+- Entries below `WRITE_GATE_THRESHOLD` are rejected on ingest
+- Confidence decays hourly; entries below `ARCHIVE_THRESHOLD` are archived
+
+See `helpers/confidence.rb` for decay constants and boost logic.
+
+## Requirements
+
+### Client mode
+- Ruby >= 3.4
+- RabbitMQ (via `legion-transport`)
+
+### Service mode
+- PostgreSQL with pgvector extension
+- RabbitMQ
+- `legion-data` for database connection management
+
+## Development
+
+```bash
+bundle install
+bundle exec rspec
+bundle exec rubocop
+```
+
+## License
+
+MIT

data/lib/legion/extensions/apollo/actors/corroboration_checker.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require 'legion/extensions/actors/every'
+require_relative '../runners/maintenance'
+
+module Legion
+  module Extensions
+    module Apollo
+      module Actor
+        class CorroborationChecker < Legion::Extensions::Actors::Every
+          def runner_class    = Legion::Extensions::Apollo::Runners::Maintenance
+          def runner_function = 'check_corroboration'
+          def time            = 900
+          def run_now?        = false
+          def use_runner?     = false
+          def check_subtask?  = false
+          def generate_task?  = false
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/actors/decay.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require 'legion/extensions/actors/every'
+require_relative '../runners/maintenance'
+
+module Legion
+  module Extensions
+    module Apollo
+      module Actor
+        class Decay < Legion::Extensions::Actors::Every
+          def runner_class    = Legion::Extensions::Apollo::Runners::Maintenance
+          def runner_function = 'force_decay'
+          def time            = 3600
+          def run_now?        = false
+          def use_runner?     = false
+          def check_subtask?  = false
+          def generate_task?  = false
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/actors/expertise_aggregator.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require 'legion/extensions/actors/every'
+require_relative '../runners/expertise'
+
+module Legion
+  module Extensions
+    module Apollo
+      module Actor
+        class ExpertiseAggregator < Legion::Extensions::Actors::Every
+          def runner_class    = Legion::Extensions::Apollo::Runners::Expertise
+          def runner_function = 'aggregate'
+          def time            = 1800
+          def run_now?        = false
+          def use_runner?     = false
+          def check_subtask?  = false
+          def generate_task?  = false
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/actors/ingest.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require 'legion/extensions/actors/subscription' if defined?(Legion::Extensions::Actors::Subscription)
+
+module Legion
+  module Extensions
+    module Apollo
+      module Actor
+        class Ingest < Legion::Extensions::Actors::Subscription
+          def runner_class    = 'Legion::Extensions::Apollo::Runners::Knowledge'
+          def runner_function = 'handle_ingest'
+          def check_subtask?  = false
+          def generate_task?  = false
+
+          def enabled?
+            defined?(Legion::Extensions::Apollo::Runners::Knowledge) &&
+              defined?(Legion::Transport)
+          rescue StandardError
+            false
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/actors/query_responder.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require 'legion/extensions/actors/subscription' if defined?(Legion::Extensions::Actors::Subscription)
+
+module Legion
+  module Extensions
+    module Apollo
+      module Actor
+        class QueryResponder < Legion::Extensions::Actors::Subscription
+          def runner_class    = 'Legion::Extensions::Apollo::Runners::Knowledge'
+          def runner_function = 'handle_query'
+          def check_subtask?  = false
+          def generate_task?  = false
+
+          def enabled?
+            defined?(Legion::Extensions::Apollo::Runners::Knowledge) &&
+              defined?(Legion::Transport)
+          rescue StandardError
+            false
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/client.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require_relative 'helpers/confidence'
+require_relative 'helpers/similarity'
+require_relative 'helpers/graph_query'
+require_relative 'runners/knowledge'
+require_relative 'runners/expertise'
+require_relative 'runners/maintenance'
+
+module Legion
+  module Extensions
+    module Apollo
+      class Client
+        include Runners::Knowledge
+        include Runners::Expertise
+        include Runners::Maintenance
+
+        attr_reader :agent_id
+
+        def initialize(agent_id: 'unknown', **)
+          @agent_id = agent_id
+        end
+
+        def store_knowledge(source_agent: nil, **)
+          super(**, source_agent: source_agent || @agent_id)
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/helpers/confidence.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Legion
+  module Extensions
+    module Apollo
+      module Helpers
+        module Confidence
+          INITIAL_CONFIDENCE = 0.5
+          CORROBORATION_BOOST = 0.3
+          RETRIEVAL_BOOST = 0.02
+          HOURLY_DECAY_FACTOR = 0.998
+          DECAY_THRESHOLD = 0.1
+          CORROBORATION_SIMILARITY_THRESHOLD = 0.9
+          WRITE_CONFIDENCE_GATE = 0.6
+          WRITE_NOVELTY_GATE = 0.3
+          STALE_DAYS = 90
+          CONTENT_TYPES = %i[fact concept procedure association observation].freeze
+          STATUSES = %w[candidate confirmed disputed decayed archived].freeze
+          RELATION_TYPES = %w[is_a has_a part_of causes similar_to contradicts supersedes depends_on].freeze
+
+          module_function
+
+          def apply_decay(confidence:, factor: HOURLY_DECAY_FACTOR, **)
+            [confidence * factor, 0.0].max
+          end
+
+          def apply_retrieval_boost(confidence:, **)
+            [confidence + RETRIEVAL_BOOST, 1.0].min
+          end
+
+          def apply_corroboration_boost(confidence:, **)
+            [confidence + CORROBORATION_BOOST, 1.0].min
+          end
+
+          def decayed?(confidence:, **)
+            confidence < DECAY_THRESHOLD
+          end
+
+          def meets_write_gate?(confidence:, novelty:, **)
+            confidence > WRITE_CONFIDENCE_GATE && novelty > WRITE_NOVELTY_GATE
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/helpers/embedding.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Legion
+  module Extensions
+    module Apollo
+      module Helpers
+        module Embedding
+          DIMENSION = 1536
+
+          module_function
+
+          def generate(text:, **)
+            return Array.new(DIMENSION, 0.0) unless defined?(Legion::LLM) && Legion::LLM.started?
+
+            result = Legion::LLM.embed(text: text)
+            result.is_a?(Array) && result.size == DIMENSION ? result : Array.new(DIMENSION, 0.0)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/helpers/graph_query.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+module Legion
+  module Extensions
+    module Apollo
+      module Helpers
+        module GraphQuery
+          SPREAD_FACTOR = 0.6
+          DEFAULT_DEPTH = 2
+          MIN_ACTIVATION = 0.1
+
+          module_function
+
+          def build_traversal_sql(depth: DEFAULT_DEPTH, relation_types: nil, min_activation: MIN_ACTIVATION, **)
+            type_filter = if relation_types&.any?
+                            types = relation_types.map { |t| "'#{t}'" }.join(', ')
+                            "AND r.relation_type IN (#{types})"
+                          else
+                            ''
+                          end
+
+            <<~SQL
+              WITH RECURSIVE graph AS (
+                SELECT e.id, e.content, e.content_type, e.confidence, e.tags, e.source_agent,
+                       0 AS depth, 1.0::float AS activation
+                FROM apollo_entries e
+                WHERE e.id = $entry_id
+
+                UNION ALL
+
+                SELECT e.id, e.content, e.content_type, e.confidence, e.tags, e.source_agent,
+                       g.depth + 1,
+                       (g.activation * #{SPREAD_FACTOR} * r.weight)::float
+                FROM graph g
+                JOIN apollo_relations r ON r.from_entry_id = g.id #{type_filter}
+                JOIN apollo_entries e ON e.id = r.to_entry_id
+                WHERE g.depth < #{depth}
+                  AND g.activation * #{SPREAD_FACTOR} * r.weight > #{min_activation}
+              )
+              SELECT DISTINCT ON (id) id, content, content_type, confidence, tags, source_agent,
+                     depth, activation
+              FROM graph
+              ORDER BY id, activation DESC
+            SQL
+          end
+
+          def build_semantic_search_sql(limit: 10, min_confidence: 0.3, statuses: nil, tags: nil, **)
+            conditions = ["e.confidence >= #{min_confidence}"]
+
+            if statuses&.any?
+              status_list = statuses.map { |s| "'#{s}'" }.join(', ')
+              conditions << "e.status IN (#{status_list})"
+            end
+
+            if tags&.any?
+              tag_list = tags.map { |t| "'#{t}'" }.join(', ')
+              conditions << "e.tags && ARRAY[#{tag_list}]::text[]"
+            end
+
+            where_clause = conditions.join(' AND ')
+
+            <<~SQL
+              SELECT e.id, e.content, e.content_type, e.confidence, e.tags, e.source_agent,
+                     e.access_count, e.created_at,
+                     (e.embedding <=> $embedding) AS distance
+              FROM apollo_entries e
+              WHERE #{where_clause}
+                AND e.embedding IS NOT NULL
+              ORDER BY e.embedding <=> $embedding
+              LIMIT #{limit}
+            SQL
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/helpers/similarity.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+require_relative 'confidence'
+
+module Legion
+  module Extensions
+    module Apollo
+      module Helpers
+        module Similarity
+          module_function
+
+          def cosine_similarity(vec_a:, vec_b:, **)
+            dot = vec_a.zip(vec_b).sum { |x, y| x * y }
+            mag_a = Math.sqrt(vec_a.sum { |x| x**2 })
+            mag_b = Math.sqrt(vec_b.sum { |x| x**2 })
+            return 0.0 if mag_a.zero? || mag_b.zero?
+
+            dot / (mag_a * mag_b)
+          end
+
+          def above_corroboration_threshold?(similarity:, **)
+            similarity >= Confidence::CORROBORATION_SIMILARITY_THRESHOLD
+          end
+
+          def classify_match(similarity:, same_content_type: true, contradicts: false, **)
+            if above_corroboration_threshold?(similarity: similarity) && same_content_type
+              contradicts ? :contradiction : :corroboration
+            else
+              :novel
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/legion/extensions/apollo/runners/expertise.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module Legion
+  module Extensions
+    module Apollo
+      module Runners
+        module Expertise
+          def get_expertise(domain:, min_proficiency: 0.0, **)
+            { action: :expertise_query, domain: domain, min_proficiency: min_proficiency }
+          end
+
+          def domains_at_risk(min_agents: 2, **)
+            { action: :domains_at_risk, min_agents: min_agents }
+          end
+
+          def agent_profile(agent_id:, **)
+            { action: :agent_profile, agent_id: agent_id }
+          end
+
+          def aggregate(**)
+            return { success: false, error: 'apollo_data_not_available' } unless defined?(Legion::Data::Model::ApolloEntry)
+
+            entries = Legion::Data::Model::ApolloEntry
+                      .select(:source_agent, :tags, :confidence)
+                      .exclude(source_agent: nil)
+                      .all
+
+            groups = {}
+            entries.each do |entry|
+              agent = entry.source_agent
+              domain = entry.tags.is_a?(Array) ? (entry.tags.first || 'general') : 'general'
+              key = "#{agent}:#{domain}"
+              groups[key] ||= { agent_id: agent, domain: domain, confidences: [] }
+              groups[key][:confidences] << entry.confidence.to_f
+            end
+
+            agent_set = Set.new
+            domain_set = Set.new
+
+            groups.each_value do |group|
+              avg = group[:confidences].sum / group[:confidences].size
+              count = group[:confidences].size
+              proficiency = [avg * Math.log2(count + 1), 1.0].min
+
+              existing = Legion::Data::Model::ApolloExpertise
+                         .where(agent_id: group[:agent_id], domain: group[:domain]).first
+
+              if existing
+                existing.update(proficiency: proficiency, entry_count: count, last_active_at: Time.now)
+              else
+                Legion::Data::Model::ApolloExpertise.create(
+                  agent_id: group[:agent_id], domain: group[:domain],
+                  proficiency: proficiency, entry_count: count, last_active_at: Time.now
+                )
+              end
+
+              agent_set << group[:agent_id]
+              domain_set << group[:domain]
+            end
+
+            { success: true, agents: agent_set.size, domains: domain_set.size }
+          rescue Sequel::Error => e
+            { success: false, error: e.message }
+          end
+
+          include Legion::Extensions::Helpers::Lex if defined?(Legion::Extensions::Helpers::Lex)
+        end
+      end
+    end
+  end
+end