llm.rb 13.0.0 → 14.0.0

data/resources/deepdive/fundamentals/database.md

@@ -0,0 +1,449 @@
+
+## Database
+### Introduction
+
+#### Overview
+
+Persistence lets an agent outlive a single session. The conversation
+history, model name, and compaction status are serialized as JSON
+that can be stored in a file, a database column, or transmitted
+over a network. Four storage options are available:
+
+- **Automatic filesystem persistence**: set `path:` on an agent
+  for transparent auto-save after every turn (recommended for
+  most file-based use-cases)
+- **Filesystem**: save and restore from a JSON file on disk
+- **ActiveRecord**: persist state in a database column using
+  [`acts_as_agent`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#acts_as_agent-instance_method)
+- **Sequel**: persist state in a database column using `plugin :agent`
+
+All three use the same serialization mechanism under the hood.
+
+#### How it works
+
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+implements
+[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
+and
+[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
+for serialization and
+[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
+for deserialization. Save writes the current state, restore loads
+it back and picks up where the conversation left off. The ORM
+wrappers automate this; each
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
+or
+[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
+call persists the
+updated state back to the column automatically.
+
+#### Why would I use it?
+
+Without persistence, every agent starts with a blank conversation.
+Persistence enables long-running agents that survive process
+restarts, debugging sessions that resume mid-investigation, and
+conversation history that can be queried alongside application
+data.
+
+#### Notes
+
+The saved JSON can be stored in a file, a database column, or
+transmitted over the network. The ORM integrations use the same
+underlying serialization as filesystem persistence.
+
+### Automatic filesystem persistence
+
+#### Overview
+
+The [`LLM::Agent#path`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#path)
+attribute provides transparent auto-persistence. Set a file path once
+and the agent restores conversation history from that file on startup
+and saves it back after every
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
+or
+[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
+turn. No manual
+[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)/
+[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
+calls needed.
+
+The feature is available on subclasses via the class DSL and on
+direct instances via the `path:` keyword argument.
+
+#### How it works
+
+When a `path` is set, the agent loads existing state from the file
+during initialization. After each turn
+([`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
+or
+[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)),
+the updated
+state is written back automatically. If the file does not exist yet,
+the agent starts with a blank conversation and creates the file on
+the first save.
+
+The path can be set using any of these approaches:
+
+##### Class DSL
+
+```ruby
+class PersistentAgent < LLM::Agent
+  set model: "deepseek-v4-pro",
+      path: "session.json"
+end
+
+# state saved automatically to session.json
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = PersistentAgent.new(llm)
+agent.talk "remember my name is robert"
+
+# restored from session.json; prints "robert"
+agent = PersistentAgent.new(llm)
+agent.talk "what's my name?"
+```
+
+##### Keyword argument
+
+```ruby
+# state saved automatically
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, path: "session.json", stream: $stdout)
+agent.talk "remember my name is robert"
+```
+
+##### Lazy resolution
+
+```ruby
+class DynamicAgent < LLM::Agent
+  set path: -> { "sessions/#{name}.json" }
+end
+```
+
+#### Why would I use it?
+
+Auto-persistence eliminates boilerplate. You set the path once and
+forget about serialization entirely. The agent picks up where it
+left off across process restarts, REPL sessions, or debugging runs
+without a single
+[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
+or
+[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
+call in your code.
+
+#### Notes
+
+The auto-path feature is a strict superset of manual filesystem
+persistence. If you need fine-grained control over when state is
+saved (e.g. batch several turns before persisting), use the manual
+[`LLM::Agent#save`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#save)
+and
+[`LLM::Agent#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#restore)
+methods described in the Filesystem section below. The `path`
+attribute delegates to the same underlying serialization.
+
+### Filesystem
+
+#### Overview
+
+A conversation that ends when the process exits is not very useful.
+Serialization saves the context (message history, model name,
+compaction status) as JSON that can be stored in a string, a
+file, or a database column. Restore it later, in a different
+process or on a different machine, and pick up where you left off.
+
+#### How it works
+
+[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
+implements
+[`LLM::Context#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_h)
+and
+[`LLM::Context#to_json`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#to_json)
+for serialization
+and
+[`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
+for deserialization. The serialized state includes
+the message history, model name, and compaction status. Save and
+restore work with file paths or in-memory strings. You can also
+serialize to a JSON string for database storage or network
+transmission:
+
+##### Save to a file
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm)
+agent.talk "remember my name is robert"
+agent.save(path: "agent.json")
+```
+
+##### Restore from a file
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, stream: $stdout)
+agent.restore(path: "agent.json")
+agent.talk "what's my name?"
+```
+
+##### Serialize to a JSON string
+
+```ruby
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm)
+agent.talk "remember my name is robert"
+json = agent.to_json
+agent = LLM::Agent.new(llm)
+agent.restore(string: json)
+```
+
+#### Why would I use it?
+
+Filesystem persistence is the foundation for long-running agents
+that outlive a single session. Save a debugging session
+mid-investigation and resume it tomorrow. Store a completed
+conversation as evidence or training data. Pass context between
+processes, between machines, or through a job queue.
+
+#### Notes
+
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+delegates serialization to its internal context. The
+saved JSON can be stored in a file, a database column, or
+transmitted over the network.
+
+### ActiveRecord
+
+#### Overview
+
+ActiveRecord models use
+[`LLM::ActiveRecord#acts_as_agent`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#acts_as_agent-instance_method)
+to install the agent wrapper. The method accepts a block for
+configuring agent defaults
+and an options hash for storage format. Most defaults such as
+`model`, `tools`, `instructions`, `schema`, and `concurrency` can
+be set through the block. Tracer and stream can also be set here
+rather than through legacy convention methods.
+
+#### How it works
+
+When you want to add agent persistence to an ActiveRecord model,
+call
+[`LLM::ActiveRecord#acts_as_agent`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#acts_as_agent-instance_method)
+in the model class. The `data` column stores
+the full agent state (conversation history, model name, compaction
+status) as JSON. On first call, a fresh agent is created and the
+conversation starts from scratch.
+
+On subsequent calls, the stored state is restored and the
+conversation continues. Every
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
+or
+[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
+call persists the
+updated state back to the column automatically.
+
+The `data_column:` option lets you use a different column name.
+The `format:` option controls the storage type. Use `:string` for
+a text column (any database) or `:jsonb` for a native PostgreSQL
+JSONB column (recommended for PostgreSQL). For `:jsonb`,
+ActiveRecord handles JSON typecasting automatically.
+
+The block yields an
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+instance. Set agent defaults in the block using the
+[`LLM::Agent.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#set-class_method)
+method or individual accessors. Anything you can set on an agent
+can be
+configured here -- model, tools, instructions, schema, concurrency,
+tracer, stream, and confirm:
+
+Note that `tracer` and `stream` are set directly in the block
+rather than through legacy `set_tracer` / `set_context` convention
+methods. The block style is the recommended approach for all agent
+configuration. Only `set_provider` is required as a private method.
+
+##### Migration with TEXT column
+
+```ruby
+class CreateAgents < ActiveRecord::Migration[7.1]
+  def change
+    create_table :agents do |t|
+      t.string :name
+      t.text :data   # stores the serialized agent state
+      t.timestamps
+    end
+  end
+end
+```
+
+##### Migration with JSONB column
+
+```ruby
+class CreateAgents < ActiveRecord::Migration[7.1]
+  def change
+    create_table :agents do |t|
+      t.string :name
+      t.jsonb :data  # native JSON storage, supports indexing
+      t.timestamps
+    end
+  end
+end
+```
+
+##### Model setup
+
+```ruby
+require "active_record"
+require "llm"
+require "llm/active_record"
+
+class Agent < ApplicationRecord
+  acts_as_agent(format: :jsonb) do |agent|
+    agent.model "deepseek-v4-pro"
+    agent.instructions "solve the user's query"
+    agent.tools [Research, FinalizeResearch, ActOnResearch]
+    agent.tracer LLM::Tracer::Logger.new(llm, io: $stdout)
+  end
+
+  private
+
+  def set_provider
+    LLM.deepseek(key: ENV["KEY"])
+  end
+end
+
+agent = Agent.create!(name: "researcher")
+agent.talk "perform research"
+# state is saved to the data column automatically
+```
+
+#### Why would I use it?
+
+The ActiveRecord wrapper integrates with your existing models.
+Agent state is automatically persisted after each conversation turn
+using the same `create!`, `save!`, and query methods you already
+use.
+
+#### Notes
+
+The `format` option defaults to `:string`. Use `:json` or `:jsonb`
+for PostgreSQL. JSONB is recommended; it supports indexing and
+is more efficient for querying. The `data_column` option defaults
+to `:data` -- create a migration to add a `TEXT` or `JSONB` column
+to your table.
+
+The model requires a `set_provider` private method that returns an
+[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
+instance. Legacy `set_context` and `set_tracer` convention methods
+also work for backwards compatibility, but the block style
+(`agent.tracer ...`, `agent.stream ...`) is preferred for all
+agent-level configuration.
+
+### Sequel
+
+#### Overview
+
+Sequel models use `plugin :agent` to install the agent wrapper.
+The plugin accepts the same options as
+[`acts_as_agent`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#acts_as_agent-instance_method)
+and follows the same conventions for provider resolution and state
+persistence.
+On first call, a fresh agent is created and the conversation starts
+from scratch. On subsequent calls, the stored state is restored and
+the conversation continues automatically.
+
+#### How it works
+
+When you want to add agent persistence to a Sequel model, register
+the plugin in the model class. The `data` column stores
+the full agent state as JSON, same structure as ActiveRecord.
+On first call, a fresh agent is created. On subsequent calls,
+the stored state is restored and the conversation continues.
+Every
+[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
+or
+[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
+call persists the updated state.
+
+The `data_column:` and `format:` options work identically to
+ActiveRecord. For `:jsonb`, Sequel loads the `pg_json` extension
+automatically and handles JSON typecasting. The block yields an
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+instance. Configure agent defaults in the block -- model, tools,
+instructions, tracer, stream, and confirm all go here:
+
+As with ActiveRecord, `tracer` and `stream` are configured in the
+block rather than through legacy convention methods. The block
+style is the recommended approach.
+
+##### Migration with TEXT column
+
+```ruby
+DB.create_table :agents do
+  primary_key :id
+  String :name
+  String :data       # stores the serialized agent state
+  DateTime :created_at
+  DateTime :updated_at
+end
+```
+
+##### Migration with JSONB column
+
+```ruby
+DB.create_table :agents do
+  primary_key :id
+  String :name
+  column :data, :jsonb  # native JSON storage
+  DateTime :created_at
+  DateTime :updated_at
+end
+```
+
+##### Model setup
+
+```ruby
+require "sequel"
+require "llm"
+require "llm/sequel/plugin"
+
+class Agent < Sequel::Model
+  plugin(:agent, format: :jsonb) do |agent|
+    agent.model "deepseek-v4-pro"
+    agent.instructions "solve the user's query"
+    agent.tools [Research, FinalizeResearch, ActOnResearch]
+    agent.tracer LLM::Tracer::Logger.new(llm, io: $stdout)
+  end
+
+  private
+
+  def set_provider
+    LLM.deepseek(key: ENV["KEY"])
+  end
+end
+
+agent = Agent.create(name: "researcher")
+agent.talk "perform research"
+# state is saved to the data column automatically
+```
+
+#### Why would I use it?
+
+The Sequel plugin integrates with your existing models.
+It follows the same conventions as ActiveRecord, so
+switching between the two requires minimal code changes.
+
+#### Notes
+
+The `format` option defaults to `:string`. Use `:json` or `:jsonb`
+for PostgreSQL. Sequel loads the `pg_json` extension automatically
+and wraps JSON values for native storage. JSONB is recommended.
+The `data_column` option defaults to `:data`.
+
+The model requires a `set_provider` private method that returns an
+[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
+instance. Legacy `set_context` and `set_tracer` convention methods
+also work for backwards compatibility, but configuring tracer,
+stream, and other agent options in the block is the preferred
+approach.
+

data/resources/deepdive/fundamentals/embeddings.md

@@ -0,0 +1,157 @@
+
+## Embeddings
+
+### Introduction
+
+#### Overview
+
+Embeddings convert text into numerical vectors that capture semantic
+meaning. Similar texts produce vectors that are close together in the
+vector space, enabling semantic search, clustering, and classification.
+Most providers offer embedding models through an
+[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
+method.
+
+llm.rb also includes support for OpenAI's vector store API, which
+provides a managed vector database as an HTTP service.
+
+#### How it works
+
+When you want to generate an embedding vector for a piece of text,
+call
+[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
+on any provider that supports it. The method accepts an input string
+or array of strings and returns a response with the resulting
+embeddings, which can be stored in a vector database and used for
+similarity search at query time:
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+body = "llm.rb is Ruby's capable AI runtime."
+embedding = llm.embed([body]).embeddings.first
+```
+
+#### Why would I use it?
+
+Embeddings are the foundation for Retrieval-Augmented Generation (RAG).
+Store embeddings for your documents, then retrieve the most relevant
+chunks at query time and feed them into the model as context. This
+pattern is one of the most common AI workflows.
+
+#### Notes
+
+Not all providers support embeddings. The following providers have
+an `embed` method:
+
+- OpenAI (`text-embedding-3-small`)
+- DeepInfra (`BAAI/bge-m3`)
+- Ollama (`qwen3:latest`)
+- Google (`gemini-embedding-2`)
+- Mistral (`mistral-embed`)
+
+### Vector stores (OpenAI)
+
+#### Overview
+
+OpenAI's vector store API provides a managed vector database as an
+HTTP service. You upload files, add them to a vector store, and
+search across them. The runtime handles chunking, embedding, and
+indexing on OpenAI's side.
+
+#### How it works
+
+When you want to create a vector store, upload files, and search
+across them, you can call methods on
+[`LLM::OpenAI::VectorStores`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenAI/VectorStores.html).
+Upload files first, then create the store with
+[`LLM::OpenAI::VectorStores#create_and_poll`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenAI/VectorStores.html#create_and_poll-instance_method)
+to
+wait until indexing completes:
+
+```ruby
+llm = LLM.openai(key: ENV["KEY"])
+
+# Upload files first
+files = %w[report.pdf manual.pdf].map { |f| llm.files.create(file: f) }
+
+# Create a vector store with the files and wait for indexing
+store = llm.vector_stores.create_and_poll(
+  name: "documents",
+  file_ids: files.map(&:id)
+)
+
+# Search across the store
+results = llm.vector_stores.search(
+  vector: store,
+  query: "What is the deadline?",
+  max_results: 5
+)
+results.each do |chunk|
+  text = chunk.content.map { |c| c.text }.join
+  puts "[score=#{chunk.score}] #{text[0..120]}..."
+end
+```
+
+#### Why would I use it?
+
+OpenAI's vector store API removes the operational overhead of running
+your own vector database. There is no infrastructure to manage, no
+indexing pipeline to maintain, and no embedding model to configure.
+The trade-off is vendor lock-in and per-request pricing.
+
+#### Notes
+
+Vector stores require the `openai` provider. Other providers such as
+DeepInfra, Google, and Mistral expose an embedding method but no
+vector store API. For a self-hosted vector database, pair
+[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
+with sqlite-vec or pgvector.
+
+### Local vector search
+
+#### Overview
+
+Local vector search combines
+[`LLM::Provider#embed`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#embed)
+with a local vector database such as
+[sqlite-vec](https://github.com/asg017/sqlite-vec) or
+[pgvector](https://github.com/pgvector/pgvector). This approach keeps
+your data on your own infrastructure.
+
+#### How it works
+
+When you want to store embeddings in a local database and search
+them later, generate vectors with any provider, store the embedding
+alongside the content, and query with a similarity search:
+
+```ruby
+require "llm"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+
+# Store: embed text and save it with a vector column
+body = "llm.rb is Ruby's capable AI runtime."
+embedding = llm.embed([body]).embeddings.first
+Document.create!(title: "llm.rb", body:, embedding:)
+
+# Query: embed the question and find similar documents
+query = "What is llm.rb?"
+query_embedding = llm.embed([query]).embeddings.first
+results = Document.order(Arel.sql("embedding <=> ?", query_embedding)).limit(5)
+```
+
+#### Why would I use it?
+
+Local vector search gives you full control over your data, no
+per-query costs beyond your embedding provider, and the ability
+to use any LLM provider for the generation step. It pairs well
+with local providers like Ollama for a fully self-hosted stack.
+
+#### Notes
+
+The exact query syntax depends on your vector database. sqlite-vec
+uses `vec_distance_L2` for Euclidean distance, while pgvector uses
+the `<=>` operator for cosine distance. Check your database
+documentation for the correct similarity function.

data/resources/deepdive/fundamentals/repl.md

@@ -0,0 +1,87 @@
+
+## REPL
+
+### Introduction
+
+#### Overview
+
+The REPL drops you into a curses-based TUI for talking to an agent
+interactively. It has a scrollable transcript that renders markdown,
+a multi-line input area, and a status bar showing context usage and
+cost. The UI thread stays responsive while a second thread communicates
+with the model. Think of it as `binding.pry` but for agents.
+
+#### How it works
+
+The REPL runs on two threads: one for the curses UI (input handling,
+transcript rendering, status bar) and one for model communication.
+
+The `name:` option labels the agent in the prompt. The `path:`
+option persists state across sessions. The `tools:` option attaches
+extra tools for the session.
+
+Commands start with `/` and are dispatched to registered
+[`LLM::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)
+subclasses. Type `/compact` to free context window
+space, `/exit` to leave.
+
+When characters arrive faster than a threshold, the REPL detects
+paste mode. In paste mode, pressing Enter inserts a newline instead
+of submitting.
+
+Start a session with:
+
+```ruby
+require "llm"
+require "llm/tools"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, name: "my-agent", path: "session.json")
+agent.repl(tools: LLM::Tool.subclasses)
+```
+
+#### Why would I use it?
+
+The REPL gives you an interactive environment to test agents, debug
+tool calls, and inspect conversation state without writing a
+separate UI. Drop in after running an agent to confirm it did what
+you expected. Inspect what went wrong when it did not. Keep talking
+to the same agent while its state is still intact. It is
+`binding.pry` but for agents.
+
+#### Notes
+
+The REPL requires the `curses` and `kramdown` gems. By default the
+tracer is disabled during the session. Set `tracer: true` to keep
+it active.
+
+Commands use the same vocabulary as tools: declare a name,
+description, and parameters with `parameter` and `required`.
+Subclassing an existing command inherits its name, description,
+and parameters. This is how `/quit` is an alias of `/exit`.
+
+```ruby
+class Greeter < LLM::Command
+  name "greet"
+  description "Greets the given name"
+  parameter :name, String, "The person's name"
+  required %i[name]
+
+  def call(name:)
+    write "Welcome #{name}!\n"
+  end
+end
+```
+
+The input area supports several keyboard shortcuts:
+
+| Key | Action |
+|---|---|
+| `Enter` | Submit the current prompt |
+| `Ctrl+A` | Jump to the start of the line |
+| `Ctrl+E` | Jump to the end of the line |
+| `Ctrl+P` / `Ctrl+N` | Recall previous / next user message |
+| `Tab` | Complete `/command` names |
+| `Esc` | Cancel the current request |
+| `Up` / `Down` | Scroll the transcript one line |
+| `PgUp` / `PgDn` | Scroll the transcript by one page |