fact_db 0.0.1

data/docs/api/index.md

@@ -0,0 +1,66 @@
+# API Reference
+
+Complete API documentation for FactDb.
+
+## Main Classes
+
+- [Facts](facts.md) - Main interface for FactDb operations
+
+## Models
+
+- [Content](models/content.md) - Immutable source documents
+- [Entity](models/entity.md) - Resolved identities
+- [Fact](models/fact.md) - Temporal assertions
+
+## Services
+
+- [ContentService](services/content-service.md) - Ingest and manage content
+- [EntityService](services/entity-service.md) - Create and resolve entities
+- [FactService](services/fact-service.md) - Extract and query facts
+
+## Extractors
+
+- [ManualExtractor](extractors/manual.md) - API-driven extraction
+- [LLMExtractor](extractors/llm.md) - AI-powered extraction
+- [RuleBasedExtractor](extractors/rule-based.md) - Pattern matching
+
+## Pipeline
+
+- [ExtractionPipeline](pipeline/extraction.md) - Concurrent extraction
+- [ResolutionPipeline](pipeline/resolution.md) - Parallel resolution
+
+## Module Structure
+
+```
+FactDb
+├── Facts                    # Main class
+├── Config                   # Configuration
+├── Database                 # Database connection
+├── Models
+│   ├── Content
+│   ├── Entity
+│   ├── EntityAlias
+│   ├── Fact
+│   ├── EntityMention
+│   └── FactSource
+├── Services
+│   ├── ContentService
+│   ├── EntityService
+│   └── FactService
+├── Extractors
+│   ├── Base
+│   ├── ManualExtractor
+│   ├── LLMExtractor
+│   └── RuleBasedExtractor
+├── Resolution
+│   ├── EntityResolver
+│   └── FactResolver
+├── Pipeline
+│   ├── ExtractionPipeline
+│   └── ResolutionPipeline
+├── Temporal
+│   ├── Query
+│   └── Timeline
+└── LLM
+    └── Adapter
+```

data/docs/api/models/content.md

@@ -0,0 +1,165 @@
+# Content Model
+
+Stores immutable source documents.
+
+## Class: `FactDb::Models::Content`
+
+```ruby
+content = FactDb::Models::Content.new(
+  raw_text: "Document content...",
+  content_type: "email",
+  captured_at: Time.current
+)
+```
+
+## Attributes
+
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `id` | Integer | Primary key |
+| `content_hash` | String | SHA256 hash for deduplication |
+| `content_type` | String | Type (email, document, etc.) |
+| `raw_text` | Text | Original content |
+| `title` | String | Optional title |
+| `source_uri` | String | Original location |
+| `source_metadata` | Hash | Additional metadata (JSONB) |
+| `embedding` | Vector | Semantic search vector |
+| `captured_at` | DateTime | When content was captured |
+| `created_at` | DateTime | Record creation time |
+
+## Associations
+
+```ruby
+has_many :fact_sources
+has_many :facts, through: :fact_sources
+```
+
+## Callbacks
+
+```ruby
+before_create :compute_hash
+before_create :generate_embedding
+```
+
+## Instance Methods
+
+### compute_hash
+
+```ruby
+def compute_hash
+```
+
+Computes SHA256 hash of raw_text for deduplication.
+
+### generate_embedding
+
+```ruby
+def generate_embedding
+```
+
+Generates embedding vector using configured generator.
+
+## Class Methods
+
+### find_or_create_by_text
+
+```ruby
+def self.find_or_create_by_text(text, **attributes)
+```
+
+Find existing content by hash or create new.
+
+**Example:**
+
+```ruby
+content = Content.find_or_create_by_text(
+  "Document text",
+  content_type: "document",
+  captured_at: Time.current
+)
+```
+
+## Scopes
+
+### by_type
+
+```ruby
+scope :by_type, ->(type) { where(content_type: type) }
+```
+
+Filter by content type.
+
+```ruby
+Content.by_type('email')
+```
+
+### captured_between
+
+```ruby
+scope :captured_between, ->(from, to) {
+  where(captured_at: from..to)
+}
+```
+
+Filter by capture date range.
+
+```ruby
+Content.captured_between(1.week.ago, Time.current)
+```
+
+### search_text
+
+```ruby
+scope :search_text, ->(query) {
+  where("raw_text @@ plainto_tsquery(?)", query)
+}
+```
+
+Full-text search.
+
+```ruby
+Content.search_text("quarterly earnings")
+```
+
+## Usage Examples
+
+### Create Content
+
+```ruby
+content = Content.create!(
+  raw_text: "Important document...",
+  content_type: "document",
+  title: "Q4 Report",
+  source_uri: "https://example.com/report.pdf",
+  captured_at: Time.current,
+  source_metadata: {
+    author: "Jane Smith",
+    department: "Finance"
+  }
+)
+```
+
+### Find by Hash
+
+```ruby
+hash = Digest::SHA256.hexdigest("Document text")
+content = Content.find_by(content_hash: hash)
+```
+
+### Get Related Facts
+
+```ruby
+content.facts.each do |fact|
+  puts fact.fact_text
+end
+```
+
+### Semantic Search
+
+```ruby
+# Requires embedding
+similar = Content
+  .where.not(embedding: nil)
+  .order(Arel.sql("embedding <=> '#{query_embedding}'"))
+  .limit(10)
+```

data/docs/api/models/entity.md

@@ -0,0 +1,202 @@
+# Entity Model
+
+Stores resolved identities (people, organizations, places, etc.).
+
+## Class: `FactDb::Models::Entity`
+
+```ruby
+entity = FactDb::Models::Entity.new(
+  canonical_name: "Paula Chen",
+  entity_type: "person"
+)
+```
+
+## Attributes
+
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `id` | Integer | Primary key |
+| `canonical_name` | String | Authoritative name |
+| `entity_type` | String | Type (person, organization, place, etc.) |
+| `resolution_status` | String | Status (unresolved, resolved, merged) |
+| `merged_into_id` | Integer | Points to canonical entity if merged |
+| `metadata` | Hash | Additional attributes (JSONB) |
+| `embedding` | Vector | Semantic search vector |
+| `created_at` | DateTime | Record creation time |
+
+## Entity Types
+
+- `person` - Individual people
+- `organization` - Companies, teams, groups
+- `place` - Locations
+- `product` - Products, services
+- `event` - Named events
+
+## Resolution Status
+
+- `unresolved` - Entity created but not confirmed
+- `resolved` - Entity identity confirmed
+- `merged` - Entity merged into another
+
+## Associations
+
+```ruby
+has_many :entity_aliases, dependent: :destroy
+has_many :entity_mentions
+has_many :facts, through: :entity_mentions
+belongs_to :merged_into, class_name: 'Entity', optional: true
+```
+
+## Instance Methods
+
+### add_alias
+
+```ruby
+def add_alias(text, type: nil, confidence: 1.0)
+```
+
+Add an alias to the entity.
+
+**Example:**
+
+```ruby
+entity.add_alias("Paula", type: "nickname", confidence: 0.95)
+```
+
+### merged?
+
+```ruby
+def merged?
+```
+
+Returns true if entity has been merged into another.
+
+### canonical
+
+```ruby
+def canonical
+```
+
+Returns the canonical entity (follows merge chain).
+
+**Example:**
+
+```ruby
+# If entity was merged
+canonical = entity.canonical  # Returns the canonical entity
+```
+
+## Scopes
+
+### by_type
+
+```ruby
+scope :by_type, ->(type) { where(entity_type: type) }
+```
+
+Filter by entity type.
+
+```ruby
+Entity.by_type('person')
+```
+
+### active
+
+```ruby
+scope :active, -> { where.not(resolution_status: 'merged') }
+```
+
+Exclude merged entities.
+
+```ruby
+Entity.active
+```
+
+### resolved
+
+```ruby
+scope :resolved, -> { where(resolution_status: 'resolved') }
+```
+
+Only resolved entities.
+
+### search_name
+
+```ruby
+scope :search_name, ->(query) {
+  where("canonical_name ILIKE ?", "%#{query}%")
+}
+```
+
+Search by name.
+
+```ruby
+Entity.search_name("paula")
+```
+
+## Usage Examples
+
+### Create Entity
+
+```ruby
+entity = Entity.create!(
+  canonical_name: "Paula Chen",
+  entity_type: "person",
+  metadata: {
+    department: "Engineering",
+    employee_id: "E12345"
+  }
+)
+```
+
+### Add Aliases
+
+```ruby
+entity.add_alias("Paula")
+entity.add_alias("P. Chen", type: "abbreviation")
+entity.add_alias("Chen, Paula", type: "formal")
+```
+
+### Check Aliases
+
+```ruby
+entity.entity_aliases.each do |a|
+  puts "#{a.alias_text} (#{a.alias_type})"
+end
+```
+
+### Get Related Facts
+
+```ruby
+entity.facts.each do |fact|
+  puts "#{fact.valid_at}: #{fact.fact_text}"
+end
+```
+
+### Find Similar Entities
+
+```ruby
+# By name
+similar = Entity.search_name("Microsoft")
+
+# By embedding
+similar = Entity
+  .where.not(embedding: nil)
+  .order(Arel.sql("embedding <=> '#{query_embedding}'"))
+  .limit(10)
+```
+
+### Merge Entities
+
+```ruby
+# entity2 will be merged into entity1
+entity2.update!(
+  resolution_status: 'merged',
+  merged_into_id: entity1.id
+)
+
+# Copy aliases
+entity2.entity_aliases.each do |a|
+  entity1.add_alias(a.alias_text, type: a.alias_type)
+end
+```

data/docs/api/models/fact.md

@@ -0,0 +1,270 @@
+# Fact Model
+
+Stores temporal assertions about entities.
+
+## Class: `FactDb::Models::Fact`
+
+```ruby
+fact = FactDb::Models::Fact.new(
+  fact_text: "Paula Chen is Principal Engineer",
+  valid_at: Date.parse("2024-01-10"),
+  status: "canonical"
+)
+```
+
+## Attributes
+
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `id` | Integer | Primary key |
+| `fact_text` | Text | The assertion |
+| `fact_hash` | String | Hash for deduplication |
+| `valid_at` | DateTime | When fact became true |
+| `invalid_at` | DateTime | When fact stopped being true (nil if current) |
+| `status` | String | Status (canonical, superseded, corroborated, synthesized) |
+| `superseded_by_id` | Integer | Points to replacing fact |
+| `derived_from_ids` | Array | Source facts for synthesized |
+| `corroborated_by_ids` | Array | Corroborating facts |
+| `confidence` | Float | Extraction confidence (0.0-1.0) |
+| `extraction_method` | String | How fact was extracted |
+| `metadata` | Hash | Additional data (JSONB) |
+| `embedding` | Vector | Semantic search vector |
+| `created_at` | DateTime | Record creation time |
+
+## Fact Status
+
+- `canonical` - Current authoritative version
+- `superseded` - Replaced by newer information
+- `corroborated` - Confirmed by multiple sources
+- `synthesized` - Derived from multiple facts
+
+## Associations
+
+```ruby
+has_many :entity_mentions, dependent: :destroy
+has_many :entities, through: :entity_mentions
+has_many :fact_sources, dependent: :destroy
+has_many :contents, through: :fact_sources
+belongs_to :superseded_by, class_name: 'Fact', optional: true
+```
+
+## Instance Methods
+
+### add_mention
+
+```ruby
+def add_mention(entity:, text:, role:, confidence: 1.0)
+```
+
+Add an entity mention to the fact.
+
+**Example:**
+
+```ruby
+fact.add_mention(
+  entity: paula,
+  text: "Paula Chen",
+  role: "subject",
+  confidence: 0.95
+)
+```
+
+### add_source
+
+```ruby
+def add_source(content:, type: "primary", excerpt: nil, confidence: 1.0)
+```
+
+Add a source content link.
+
+**Example:**
+
+```ruby
+fact.add_source(
+  content: email,
+  type: "primary",
+  excerpt: "...accepted the offer..."
+)
+```
+
+### currently_valid?
+
+```ruby
+def currently_valid?
+```
+
+Returns true if fact is currently valid (invalid_at is nil).
+
+### valid_at?(date)
+
+```ruby
+def valid_at?(date)
+```
+
+Returns true if fact was valid at the given date.
+
+### superseded?
+
+```ruby
+def superseded?
+```
+
+Returns true if fact has been superseded.
+
+### canonical?
+
+```ruby
+def canonical?
+```
+
+Returns true if fact is canonical.
+
+## Scopes
+
+### canonical
+
+```ruby
+scope :canonical, -> { where(status: 'canonical') }
+```
+
+Only canonical facts.
+
+### currently_valid
+
+```ruby
+scope :currently_valid, -> { where(invalid_at: nil) }
+```
+
+Facts that are currently valid.
+
+### valid_at
+
+```ruby
+scope :valid_at, ->(date) {
+  where("valid_at <= ? AND (invalid_at IS NULL OR invalid_at > ?)", date, date)
+}
+```
+
+Facts valid at a specific point in time.
+
+```ruby
+Fact.valid_at(Date.parse("2023-06-15"))
+```
+
+### mentioning_entity
+
+```ruby
+scope :mentioning_entity, ->(entity_id) {
+  joins(:entity_mentions).where(entity_mentions: { entity_id: entity_id })
+}
+```
+
+Facts mentioning a specific entity.
+
+```ruby
+Fact.mentioning_entity(paula.id)
+```
+
+### search_text
+
+```ruby
+scope :search_text, ->(query) {
+  where("fact_text @@ plainto_tsquery(?)", query)
+}
+```
+
+Full-text search.
+
+```ruby
+Fact.search_text("engineer")
+```
+
+### by_extraction_method
+
+```ruby
+scope :by_extraction_method, ->(method) {
+  where(extraction_method: method)
+}
+```
+
+Filter by extraction method.
+
+```ruby
+Fact.by_extraction_method('llm')
+```
+
+### high_confidence
+
+```ruby
+scope :high_confidence, -> { where("confidence > 0.8") }
+```
+
+High confidence facts only.
+
+## Usage Examples
+
+### Create Fact
+
+```ruby
+fact = Fact.create!(
+  fact_text: "Paula Chen joined Microsoft as Principal Engineer",
+  valid_at: Date.parse("2024-01-10"),
+  status: "canonical",
+  extraction_method: "manual",
+  confidence: 1.0
+)
+
+# Add mentions
+fact.add_mention(entity: paula, text: "Paula Chen", role: "subject")
+fact.add_mention(entity: microsoft, text: "Microsoft", role: "organization")
+
+# Add source
+fact.add_source(content: announcement, type: "primary")
+```
+
+### Query Facts
+
+```ruby
+# Current facts about Paula
+Fact.canonical.currently_valid.mentioning_entity(paula.id)
+
+# Historical facts
+Fact.valid_at(Date.parse("2023-06-15")).mentioning_entity(paula.id)
+
+# Search
+Fact.search_text("promoted")
+```
+
+### Supersede Fact
+
+```ruby
+new_fact = Fact.create!(
+  fact_text: "Paula Chen is Senior Principal Engineer",
+  valid_at: Date.parse("2024-06-01"),
+  status: "canonical"
+)
+
+old_fact.update!(
+  status: "superseded",
+  superseded_by_id: new_fact.id,
+  invalid_at: Date.parse("2024-06-01")
+)
+```
+
+### Get Sources
+
+```ruby
+fact.fact_sources.each do |source|
+  puts "Source: #{source.content.title}"
+  puts "Type: #{source.source_type}"
+  puts "Excerpt: #{source.excerpt}"
+end
+```
+
+### Get Mentioned Entities
+
+```ruby
+fact.entity_mentions.each do |mention|
+  puts "#{mention.entity.canonical_name} (#{mention.mention_role})"
+end
+```