little_ghost 0.4.0 → 0.6.0

data/docs/guides/core_concepts.md

@@ -1,6 +1,6 @@
 # Core Concepts
 
-Build one model-driven behavior in a Ruby class, then call it like Ruby. That is the idea LittleGhost grows from.
+Define an Agent in a Ruby class, then call it with `.ask`.
 
 ```ruby
 class CustomerSupportAgent < LittleGhost::Agent
@@ -29,7 +29,8 @@ CustomerSupportAgent
 └── limits and optional capabilities
 ```
 
-An Agent can return text or checked, structured data. Later, you can add streaming, sessions, or callbacks. None of them are required to begin.
+An Agent can return text or checked, structured data. You can add streaming,
+saved conversations, or callbacks later. None of them are required to begin.
 
 ## A Tool connects the model to Ruby
 
@@ -52,7 +53,12 @@ class HelpCenterLookupTool < LittleGhost::Tool
 end
 ```
 
-LittleGhost checks the model's arguments, calls the tool, and gives the result back to the model. The schema checks shape, not permission. Authorize sensitive reads and actions inside the tool with trusted application context.
+LittleGhost checks the model's arguments, calls the Tool, and gives the result
+back to the model. The schema checks shape, not permission. Check permission
+inside the Tool using identity and account information from your application.
+
+[Tools](tools.md) follows that path from model input to application code,
+including run-scoped bindings, concurrency, retries, and sandbox delegation.
 
 ## A Run owns one top-level execution
 
@@ -81,11 +87,21 @@ Run
 └── Agent and Tools ──> RunResult
 ```
 
-An **Invocation** is the request in LittleGhost's standard shape. Its `context` contains current request values supplied by your application. A Tool can read those values through `run.invocation.context` when it authorizes work.
+An **Invocation** is the request in LittleGhost's standard shape. Its `context`
+contains current request values supplied by your application. A Tool can read
+those values through `run.invocation.context` when it checks permission.
 
-The **RunContext** carries mutable working state in `context.state`. At the top level, saved Session state is loaded first, then current Invocation context is added. Child Assemblies may receive a copy, a mapped value, or no context at all. Recheck saved values before using them for permission decisions.
+A **Session** stores conversation state between Runs when persistence is
+configured. The **RunContext** carries mutable working state in `context.state`
+during one Run. LittleGhost loads saved Session state before adding the current
+Invocation context. Recheck saved values before using them for permission
+decisions.
 
-A Tool's **Binding** gives the Tool access to objects created for this run, including the Agent, Run, workspace, and sandbox. These objects are separate from the arguments chosen by the model.
+A Tool's **Binding** gives the Tool access to objects created for this run,
+including the Agent, Run, Workspace, and Sandbox. These objects are separate
+from arguments chosen by the model. [Tools](tools.md) explains the binding;
+[Workspaces and Sandboxes](sandboxing.md) explains delegated files and child
+processes.
 
 The final **RunResult** keeps the complete assembly result. Its `text` is the final text answer. Its `output` returns structured data when the Agent declared a result schema, and text otherwise. The top-level `Run#response` is always the caller-facing text.
 
@@ -102,9 +118,13 @@ Top-level calls normally return a Run, even when execution fails. The terminal e
 | Tool input or a `ToolError` failed | The model may recover | No terminal event by itself | Gives a safe error result back to the model |
 | Input, configuration, or resources failed before a Run could start | No Run exists | None | Raises the exception |
 
-Unexpected Tool exception messages are hidden from the model. The original exception remains available to trusted application callbacks and diagnostics.
+Unexpected Tool exception messages are hidden from the model. The original
+exception remains available to application callbacks and diagnostics.
 
-Failures while closing resources, delivering events, or reporting instrumentation sit outside the normal result path. They raise a Ruby exception because LittleGhost can no longer promise that it delivered a clean ending. [Running in Production](production.md) covers that boundary where applications supervise and shut down work.
+Failures while closing resources, delivering events, or reporting
+instrumentation sit outside the normal result path. They raise a Ruby exception
+because LittleGhost can no longer promise that it delivered a clean ending.
+[Running in Production](production.md) covers supervision and shutdown.
 
 ## An Assembly can look like one Agent
 
@@ -146,6 +166,10 @@ end
 
 Use a subagent when delegation is part of one model's decision-making. Use a Workflow when application code must guarantee that a step happens.
 
+When an Agent also uses code mode, subagent controls stay in the Agent's
+conversation. Code-mode programs can compose ordinary Tools, while spawning,
+messaging, and checking on subagents remain decisions for the parent model.
+
 ### Workflows make Ruby the coordinator
 
 A **Workflow** coordinates work with ordinary Ruby. Its `perform` method can call an Agent or another Assembly, read a result, choose a branch, or run independent steps together.
@@ -185,7 +209,10 @@ class ProblemSolverSwarm < LittleGhost::Swarm
 end
 ```
 
-A Swarm is intentionally agent-to-agent. Its members are Agents, not other kinds of Assembly. Caller history and application context stay hidden unless a member opts in. Treat every handoff message as untrusted model input.
+A Swarm is intentionally agent-to-agent. Its members are Agents, not other
+kinds of Assembly. Caller history and application context stay hidden unless a
+member opts in. A handoff message comes from another model, so a receiving
+Agent should use it as context rather than proof that an action is permitted.
 
 ### Graphs make routes visible
 
@@ -209,26 +236,12 @@ class SupportFlowGraph < LittleGhost::Graph
 end
 ```
 
-Graph nodes do not receive caller history or application context unless they opt in. They still receive the original request and the outputs routed to them. Use edge input mappers to choose or redact what moves forward.
-
-## Class definitions first, builders when needed
-
-Named classes are the default way to organize reusable behavior. They are readable, load through normal Ruby conventions, and give the coordination style a visible name such as `ResponseWorkflow` or `SupportFlowGraph`.
-
-Every assembly class can also produce a mutable builder:
-
-```ruby
-graph = SupportFlowGraph.to_builder
-graph.node :audit, AuditAgent
-graph.edge :respond, :audit
-graph.finish :audit
-graph.validate!
-run = graph.ask("Review order 481")
-```
-
-Use a builder when trusted application configuration decides the participants or routes. Each run gets a fixed copy of the builder as it looked when the run began, so later edits affect later runs. Ruby callbacks still see any application objects they captured.
+Graph nodes receive the original task and results from the nodes immediately
+before them. They do not receive caller history or application context unless
+their declarations opt in. [Compose Agents](assemblies.md) explains parallel
+routes, joins, input mapping, and data boundaries.
 
-## One result, including the journey
+## One result, even when several agents help
 
 Every assembly produces the same top-level `Run` and final `RunResult`. Composite assemblies also keep a size-limited record of the participants that ran:
 
@@ -240,8 +253,11 @@ run.result.steps
 run.result.trajectory.transitions
 ```
 
-This shows callers which participants ran without including raw provider responses. A Swarm or Graph may hide intermediate model events from the public stream so the response stays coherent. That is a presentation choice, not a privacy boundary: routed outputs and step summaries still exist.
+This record shows which participants ran. [Compose Agents](assemblies.md)
+explains builders, detailed routing records, and live events from nested Agents.
 
 The pieces now fit together: Agents define behavior. Tools connect them to Ruby. Runs record one execution. Assemblies let the system grow without changing the caller.
 
-Continue with [Compose Agents](assemblies.md) to put several agents to work together. If you are ready to connect the feature to a real application, jump to [Running in Production](production.md).
+Continue with [Models and Providers](models_and_providers.md) to choose model
+targets and configure provider connections. When you need several agents to
+work together, [Compose Agents](assemblies.md) builds on the same concepts.

data/docs/guides/getting_started.md

@@ -48,7 +48,7 @@ $ ruby customer_support_agent.rb
 
 `CustomerSupportAgent.ask` creates a `LittleGhost::Run` for this request. When the work finishes, the Run holds the outcome and response.
 
-The inline prompt keeps this first example easy to see in one place. When the instructions grow, [Prompts as Views](prompt_views.md) moves them into a conventional ERB file without adding setup to the Agent.
+The inline prompt keeps this first example visible in one place. When the instructions grow, [Prompts as Views](prompt_views.md) moves them into a conventional ERB file without adding setup to the Agent.
 
 The selected external provider may receive system instructions, caller input, conversation history, tool results, and attachments. Model wording can vary, so use application code—not a prompt—when a rule must always hold.
 
@@ -101,13 +101,18 @@ run.response
 # Refunds are available within 30 days, so your purchase is eligible.
 ```
 
-LittleGhost checks the model's arguments before it calls `HelpCenterLookupTool#call`. The schema checks shape, not permission. If a tool reads customer data or changes something, authorize that work from trusted application context. The tool's result then becomes context for the model.
+LittleGhost checks the model's arguments before it calls
+`HelpCenterLookupTool#call`. The Tool's result then becomes context for the
+model.
 
-### Use trusted context for private data
+### Use application context for private data
 
-Model tool arguments are untrusted, even after their shape has been checked. Pass identity and permissions from your application's authentication boundary instead.
+The schema checks shape, not permission. When a Tool reads private data or
+changes something, use identity and account information established by your
+application rather than asking the model to supply it.
 
-While an Agent is working, LittleGhost binds each Tool instance to the current Run. The Tool can read trusted request values through its `run` accessor:
+While an Agent is working, LittleGhost binds each Tool instance to the current
+Run. The Tool can read request values through its `run` accessor:
 
 ```ruby
 class OrderStatusTool < LittleGhost::Tool
@@ -147,7 +152,13 @@ run = CustomerSupportAgent.ask(
 )
 ```
 
-Here, `order_number` came from the model. The application supplied `actor_id` and `account_id` after authenticating the caller. LittleGhost places those request values on `run.invocation`; context keys become strings. The model cannot replace them through its tool arguments.
+Here, `order_number` came from the model. The application supplied `actor_id`
+and `account_id` after authenticating the caller. LittleGhost places those
+request values on `run.invocation`; context keys become strings.
+
+> **Safety note:** Treat model-selected Tool arguments like any other external
+> input. Check permission using the current user and account before returning
+> private data or performing a write.
 
 That is enough to authorize the first Tool safely. [Core Concepts](core_concepts.md) names the request and working-state objects behind `run`, and [Running in Production](production.md) explains what changes when you add saved conversations.
 

data/docs/guides/integrations.md

@@ -0,0 +1,217 @@
+# Connect MCP, AG-UI, and OpenTelemetry
+
+LittleGhost can load Tools from an MCP server, translate a Run stream for an
+interactive interface, and publish traces. Each integration uses the same
+Agents and Runs you already have.
+
+## Load Tools from an MCP server
+
+An MCP Toolset connects to one server and turns its published operations into
+LittleGhost Tool classes. Add the Toolset through the same Agent `tools`
+declaration used for local Tools:
+
+```ruby
+require "little_ghost/mcp"
+
+class HelpCenterTools < LittleGhost::MCP::Toolset
+  connection url: "https://mcp.example/rpc", timeout: 20
+end
+
+class CustomerSupportAgent < LittleGhost::Agent
+  system_prompt "Use help-center tools for published guidance."
+  tools HelpCenterTools
+end
+
+run = CustomerSupportAgent.ask("How long do refunds take?")
+run.response
+```
+
+`connection` requires `url` and also accepts `headers`, `timeout`, `signer`,
+`allow_insecure_http`, and `max_response_bytes`. Pass a block when credentials
+depend on the current Agent run:
+
+```ruby
+connection do |binding|
+  token = McpAccessTokens.for_actor(binding.run.invocation.actor_id)
+  {
+    url: "https://mcp.example/rpc",
+    headers: {"Authorization" => "Bearer #{token}"},
+    timeout: 20
+  }
+end
+```
+
+The block's `binding` gives it access to the current Run. LittleGhost evaluates
+the block before opening the MCP session, so each Agent run can use credentials
+for its authenticated caller.
+
+By default, the Agent receives every operation published by the server. Their
+normalized server names, such as `search` and `fetch`, become Tool names.
+
+Use `map_tool` when the Agent should receive only part of the server catalog or
+when a generated Tool needs a different name or configuration:
+
+```ruby
+class CuratedHelpCenterTools < LittleGhost::MCP::Toolset
+  connection url: "https://mcp.example/rpc", timeout: 20
+
+  map_tool do |tool_class, definition:, binding:|
+    next unless %w[search fetch].include?(definition.source_name)
+
+    tool_class.tool_name "help_center_#{definition.source_name}"
+    tool_class
+  end
+end
+```
+
+`definition` describes the operation published by the server, and `binding`
+identifies the current Agent run. Return the class after configuring it, or
+return `nil` to omit the operation. Renaming a generated Tool does not change
+the original `Definition#source_name` sent back to the server.
+
+The Agent can call the generated Tools like local Tools. LittleGhost uses one
+local client and transport for the Toolset during the Agent run. The built-in
+HTTP transport does not send an MCP session-termination request. Configure
+server-side expiry, or arrange explicit remote cleanup when the server requires
+it.
+
+Most MCP results need no mapping. LittleGhost returns `structuredContent` as a
+Ruby Hash when present, otherwise it returns the server's text. Server images
+become Artifacts.
+
+Use `map_result` when one operation needs application-specific conversion. This
+example turns the server's download identifier into a deferred Artifact:
+
+```ruby
+map_result do |result, call:, binding:|
+  next result unless call.definition.source_name == "export"
+
+  LittleGhost::Tool::Result.new(
+    value: result.structured_content,
+    artifacts: [
+      LittleGhost::Artifact.deferred(
+        reference: result.metadata.fetch("download_id"),
+        media_type: "application/octet-stream"
+      )
+    ]
+  )
+end
+```
+
+`map_result` receives the complete `MCP::Result`, the `MCP::Call` that produced
+it, and the current binding. Return any Ruby value or `Tool::Result`. Returning
+the supplied result unchanged keeps the default conversion described above.
+MCP images and local Tool artifacts use the same storage and presentation
+rules when `Configuration#artifacts` is enabled. Images and documents are sent
+as model content; their stored references are fallback information rather than
+a second representation. LittleGhost also checks results against
+server-advertised JSON Schema Draft 2020-12 output schemas.
+
+An optional server can fail discovery without preventing Agent construction:
+
+```ruby
+class HelpCenterTools < LittleGhost::MCP::Toolset
+  connection { |binding| McpConnections.help_center(binding) }
+  optional true
+  on_error do |error, binding:|
+    McpAvailability.report(error, run_id: binding.run.invocation.run_id)
+  end
+end
+```
+
+`optional true` converts expected provider and protocol discovery failures
+into an empty Tool set. `on_error` observes only those caught failures.
+Cancellation, deadlines, configuration errors, and application callback
+failures still propagate.
+
+LittleGhost limits the number and total size of discovered operations, the
+complexity of their schemas, and the size and number of returned images.
+`HTTPTransport` also limits each HTTP response and requires HTTPS unless local
+HTTP is explicitly enabled.
+
+> **Safety note:** An MCP server supplies descriptions and results that the model
+> can see. Structural validation does not make that content trustworthy or
+> authorize an operation it suggests. Expose only the operations the Agent
+> needs, use narrowly scoped credentials, and have the server authorize every
+> sensitive call. If a result becomes a deferred Artifact, its resolver must
+> verify that the referenced file belongs to the authenticated caller, fetch
+> only from an intended service, and limit the response size before returning
+> bytes to LittleGhost.
+
+LittleGhost implements its documented client behavior for the [MCP 2025-06-18
+specification](https://modelcontextprotocol.io/specification/2025-06-18).
+
+Use `LittleGhost::MCP::HTTPTransport` and `LittleGhost::MCP::Client` directly
+when you need a custom transport. They produce the same generated Tool classes
+and accept the same mapping callbacks as Toolset.
+
+## Send a Run stream through AG-UI
+
+The AG-UI adapter converts LittleGhost events into protocol event hashes:
+
+```ruby
+require "json"
+require "little_ghost/ag_ui"
+
+source = CustomerSupportAgent.stream_ask(
+  question,
+  actor_id: authenticated_user.id,
+  context: {account_id: authenticated_user.account_id}
+)
+
+events = LittleGhost::AGUI::Adapter.new.stream(
+  source,
+  thread_id: conversation.id,
+  run_id: request.request_id
+)
+
+events.each { |event| websocket.write(JSON.generate(event)) }
+```
+
+The adapter translates text, reasoning, Tool activity, usage, retries, trace
+context, subagent activity, and terminal outcomes. It is stateless between
+calls. Your application still owns the connection, backpressure, disconnect
+behavior, and any request state its callbacks need.
+
+LittleGhost also emits namespaced custom events. Consumers should preserve or
+deliberately ignore event types they don't recognize. See the [AG-UI event
+documentation](https://docs.ag-ui.com/concepts/events) when implementing the
+client.
+
+> **Safety note:** A Run stream can include model output, Tool arguments and
+> results, errors, and participant activity. Check that the connected user may
+> see the complete Run, then filter fields before sending or storing events.
+
+Calling `each` drives the source stream on the caller's fiber or thread. When a
+client disconnects, stop enumerating and apply the cancellation behavior your
+application needs. Closing the socket can't undo Tool work that already ran.
+
+## Trace Runs with OpenTelemetry
+
+Configure an OpenTelemetry SDK and exporter in the application, then register
+the LittleGhost subscriber before the first Agent call:
+
+```ruby
+LittleGhost.configure do |config|
+  config.instrument LittleGhost::Tracing::OpenTelemetry.new
+end
+```
+
+LittleGhost depends on `opentelemetry-api`, leaving the SDK, processor, and
+exporter up to the application. It emits spans and events for Runs, Agents,
+model calls, Tools, assemblies, sessions, usage, and failures. Active operations
+can propagate W3C `traceparent` and `tracestate` fields.
+
+Prompts, messages, responses, Tool arguments, and exception content are omitted
+by default. If you intentionally need some of that content, install a
+`LittleGhost::Support::ContentCapture` with a scrubber before enabling capture.
+Avoid putting raw user, order, session, or request IDs in span attributes.
+
+Attribute names follow the evolving [OpenTelemetry GenAI semantic
+conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) where they
+apply. Flush or shut down `LittleGhost::Instrumentation` during application
+shutdown when your backend buffers data.
+
+See [Running in Production](production.md) for startup, shutdown, and observability,
+[Tools](tools.md) for local and remote Tool behavior, and [Workspaces and
+Sandboxes](sandboxing.md) for child processes and files.

data/docs/guides/models_and_providers.md

@@ -0,0 +1,125 @@
+# Choose Models and Providers
+
+An Agent needs a model target: a configured provider connection plus the
+provider's model identifier. You can write that target directly while getting
+started, then give it an application-facing name when several Agents share it.
+
+## Start with one direct target
+
+A direct target has the form `connection:model-id`:
+
+```ruby
+class CustomerSupportAgent < LittleGhost::Agent
+  model "openrouter:openai/gpt-5.6-luna"
+  system_prompt "Answer customer questions clearly and concisely."
+end
+```
+
+`openrouter` names a connection configured by the application. The remainder
+is the model identifier understood by that provider. This is a good fit when
+one Agent owns one stable choice.
+
+## Give shared choices a role
+
+A model role lets several Agents share a choice without knowing its provider
+or model identifier:
+
+```ruby
+LittleGhost.configure do |config|
+  config.providers = {
+    primary: {
+      adapter: :openrouter,
+      api_key: ENV.fetch("OPENROUTER_API_KEY")
+    }
+  }
+  config.models = {
+    customer_support: {
+      target: "primary:openai/gpt-5.6-luna",
+      settings: {temperature: 0.2}
+    }
+  }
+  config.default_model = :customer_support
+end
+
+class CustomerSupportAgent < LittleGhost::Agent
+  model :customer_support
+end
+```
+
+Here `customer_support` is the role, `primary` is the connection, and
+`openrouter` is the adapter. You can move the role to another model or provider
+without editing the Agent.
+
+Profile settings are defaults. An individual call can override them:
+
+```ruby
+run = CustomerSupportAgent.ask(
+  "Explain the refund decision.",
+  settings: {temperature: 0.0}
+)
+```
+
+Build these settings in application code instead of passing request parameters
+through unchanged. Settings can affect cost, latency, and model behavior.
+
+## Configure connections in one place
+
+LittleGhost includes adapters for OpenRouter, OpenAI-compatible APIs,
+Anthropic, Gemini, Vertex AI, and Bedrock. Connections may live in an
+initializer or in the conventional files under `config/little_ghost`.
+
+Keep credentials in your application's secret manager. Agents refer to a role
+or configured connection; they don't need to contain credentials. If your
+application obtains short-lived credentials at runtime, configure a credential
+resolver that returns them for the selected connection.
+
+> **Safety note:** The selected provider may receive system instructions,
+> caller input, conversation history, Tool results, schemas, and attachments.
+> Choose a provider that is appropriate for that data, and keep credentials and
+> provider endpoints under application control.
+
+## Choose a role for each request
+
+An Agent can select between configured roles using its `Invocation`:
+
+```ruby
+class CustomerSupportAgent < LittleGhost::Agent
+  model do |invocation|
+    invocation.fetch(:premium_account, false) ? :premium_support : :customer_support
+  end
+end
+```
+
+Set `premium_account` from application state when creating the invocation. If
+a public request offers a model choice, map that choice to one of your
+configured roles rather than accepting an arbitrary provider target.
+
+Trusted application code may also declare a selection inline:
+
+```ruby
+class ResearchAgent < LittleGhost::Agent
+  model(
+    provider: "primary",
+    model: "openai/gpt-5.6-luna",
+    reasoning_effort: "high"
+  )
+end
+```
+
+`provider` still names a configured connection. The inline settings change the
+selection; they don't create a new connection.
+
+## Use model capabilities
+
+`LittleGhost::ModelResolver` turns a role or target into an executable
+`LittleGhost::Model`. Its catalog describes capabilities such as supported
+input types, output limits, and structured results. LittleGhost uses that
+information to reject unsupported attachments, constrain output limits, and
+choose a structured-result strategy.
+
+Provider capabilities can change. Handle failed Runs and provider errors even
+when the catalog says a feature is supported.
+
+Continue with [Prompts as Views](prompt_views.md) when an Agent's instructions
+outgrow one string. See [Structured Results and Content](structured_outputs_and_content.md)
+when you need checked result shapes, images, or documents.