relay.app 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3caa789124f0cee625277b4f1e3472e04c12d1aeeaa193ed6146ba593061d57d
-  data.tar.gz: 83e22033af1263c50c8746958d56e07836f5570a02399e894be48468de750c74
+  metadata.gz: 5bfc3dab37e4f9472c9bc2fc6e384059cd1d75a772407feaccf874d87606ee37
+  data.tar.gz: dcafd2c06b26dfbbe84e5f9b700a88cdac835621bedc3a8ffc3b01477028af2d
 SHA512:
-  metadata.gz: 848201fc63074254365539657cd709eb60dd3fbe729d7b1421d4076e58bc6dcb82e437058c1fa8f6cb7ce1a887a6ba3583724abc044ce9e10179d76884f46f91
-  data.tar.gz: 347db524516e78944bce97e8ee6f96944190206fb057f85be65eada7a1fcd3449bb3001be17016ad2125a190600deb6e08f7fcbfff69d4fba77285203860dbb9
+  metadata.gz: 0601e7bc7927a746fd219c63f1f048ce6ae284cda78dda87acf0fae8fe2850375b53ea5b98d5ea6c2fa817b4ea36e26e6787c45f8b518d09d24d9b5c4e27be5d
+  data.tar.gz: 7cf9df0755a86c7e932ddd9cfac5dc22632ac875ab82350a8205414a7527cde5f1feb45293a0c0ae5e655526b76d3debc2df97a13af9522c442c202042bb7d4e

data/CHANGELOG.md

@@ -2,6 +2,49 @@
 
 ## Unreleased
 
+## v0.6.0
+
+Changes since v0.5.0.
+
+### Change
+
+* **Move the default Relay home to `~/.config/relay`** <br>
+  Change Relay's default writable home directory from `~/.relay` to
+  `~/.config/relay`, and update the README examples for user-installed
+  tools to match the new location.
+
+* **Add `relay console`** <br>
+  Add a console command that boots Relay and starts IRB in the loaded
+  application process for interactive debugging.
+
+* **Add AWS Bedrock provider support** <br>
+  Add Bedrock to Relay's provider registry and persisted context
+  initialization, and extend `relay configure` to prompt for AWS access
+  key credentials.
+
+* **Prepopulate provider API key prompts from environment** <br>
+  Let `relay configure` reuse existing provider secrets from process
+  environment variables such as `OPENAI_API_KEY` and
+  `DEEPSEEK_API_KEY`, while still writing Relay's canonical `*_SECRET`
+  keys to `~/.relay/env`.
+
+### Fix
+
+* **Launch Falcon with the current Ruby interpreter** <br>
+  Change `relay start` to exec Falcon through `RbConfig.ruby -S` so the
+  server process uses the same Ruby and gem environment as Relay instead
+  of whichever `falcon` binary appears first on `PATH`.
+
+* **Load user-installed tools through Zeitwerk** <br>
+  Replace manual loading of `~/.config/relay/tools/*.rb` with a dedicated
+  Zeitwerk loader so development reloads unload and recreate user tools
+  instead of reopening existing classes.
+
+* **Activate the `llm.rb` gem before `require "llm"`** <br>
+  Avoid RubyGems loading the unrelated `llm` gem when both gems are
+  installed by explicitly activating `llm.rb` before requiring its
+  `llm` entrypoint.
+
 ## v0.5.0
 
 Model catalog workflow release.

data/README.md

@@ -1,68 +1,66 @@
 ## About
 
-Relay is a self-hostable LLM environment with support for OpenAI, DeepSeek,
-Anthropic, xAI and zAI out of the box. It is incredibly simple to setup
-and get started. The application is distributed as a RubyGem. It has a minimal
-set of dependencies - built on Roda, Sequel, Falcon, [llm.rb](https://github.com/llmrb/llm.rb),
+Relay is a self-hostable, hackable LLM web environment that can be extended
+with your own tools and skills that live in your `${HOME}` directory.
+It is for programmers, AI engineers, hackers, and anyone who wants
+their own AI environment with the option to extend it with code.
+
+
+## Setup
+
+It is simple to setup and get started. The application is
+distributed as a RubyGem. It has a minimal set of dependencies -
+built on Roda, Sequel, Falcon, [llm.rb](https://github.com/llmrb/llm.rb),
 HTMX and web sockets.
 
-There is support for connecting to MCP servers too - both HTTP and stdio. You can
-add your own tools to `~/.relay/tools` which is a neat way to extend the environment
-with your own functionality. The database uses SQLite3 to keep things simple - the
-goal is to have something you can setup in under two minutes.
+![demo](./demo.gif)
 
-## Getting started
+## Appearance
 
-#### Install
+#### Sign-in
 
-Install the gem:
+![Relay screenshot](./relay3.png)
 
-```sh
-gem install relay.app
-```
+#### Chat
 
-Go through interactive setup, start the server, and visit
-http://localhost:9292.
+![Relay screenshot](./relay1.png)
 
-```sh
-relay setup
-relay start
-```
+#### MCP
 
-## Features
+![Relay screenshot](./relay2.png)
 
-* Install and setup in 2 minutes
-* Localize your chats and mcp settings to your user account
-* Connect to multiple providers (OpenAI, xAI, Anthropic, Google, DeepSeek, zAI)
-* Connect to MCP servers
-* Cancel in-flight requests and tool execution cleanly
-* Run tools concurrently
-* Make it yours: extend and customize with your own tools and system prompt
-* Lightweight architecture
 
-## Sounds cool, how does it look?
+## Getting started
 
-**Sign-in**
+#### Install
 
-![Relay screenshot](./relay3.png)
+Install the gem
 
-**Chat**
+```sh
+gem install relay.app
+```
 
-![Relay screenshot](./relay1.png)
+#### Configure
 
-**MCP**
+Interactive setup
 
-![Relay screenshot](./relay2.png)
+```sh
+relay setup
+```
 
-## How easy is it to setup?
+#### Serve
 
-Very easy.
+Start the server, and visit http://localhost:9292
 
-![demo](./demo.gif)
+```sh
+relay start
+```
 
-## How do I add my own tool?
+## Tools
 
-Before running `relay start` you should add a `~/.relay/tools/<yourtool>.rb`.
+#### How do I add my own tool?
+
+Before running `relay start` you should add `~/.config/relay/tools/<yourtool>.rb`.
 The tool will be automatically made available to the LLM. This is how a tool
 might look - it is not very useful because it does not emit command output
 but it serves as a simple example that you can modify and change to meet
@@ -82,18 +80,18 @@ class Shell < LLM::Tool
 end
 ```
 
-## Wait, what is a tool?
+#### Wait, what is a tool?
 
 A tool contains a name, a description, and optional parameters. It is attached
 to a method, and that method that can be called. The model or LLM decides when
-and how to call a tool. A tool can do anything you can imagine, and it can extend
-the abilities of the LLM. Suddenly a LLM can search the web, run code, and anything
-you can think of. They're a powerful way to extend the capabilities of an LLM.
+and how to call a tool. A tool can extend the abilities of the LLM with
+your own code that could can search the web, read documentation, etc.
 
 An MCP server can also expose pre-packaged tools, and those can be especially
-powerful for talking to GitHub or your own Forgejo instance.
+useful for talking to GitHub, your own Forgejo instance or any other
+kind of MCP server. 
 
-## What are the default tools?
+#### What are the default tools?
 
 The `relay-knowledge` tool returns documentation for both Relay
 and [llm.rb](https://github.com/llmrb/llm.rb) - ask about either
@@ -106,14 +104,15 @@ can be played inline in the chat, and you can also add your own
 songs or remove existing ones through the same tools. The only
 requirement is that it is a YouTube URL.
 
-## What provider is the best value?
+## Costs
+
+#### What provider is the best value?
 
-DeepSeek. I highly recommend it. The context window is 1M. I have been using it
-all the time - especially for Relay development, and despite my heavy usage, it
-cost only 80 cents overall. It's almost free. I used it **a lot**. I'd estimate
-that a 1M context window costs 14 cents or so.
+DeepSeek. <br>
+Hard to beat it on price. <br>
+Recent models have a context window of 1M.
 
-## What about Ollama and friends?
+#### What about self-hosting with Ollama ?
 
 [llm.rb](https://github.com/llmrb/llm.rb#readme) provides support ollama, llama.cpp,
 and any OpenAI-compatible endpoint. But Relay does not surface it as a feature. I haven't

data/app/concerns/attachment.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Relay::Concerns
   module Attachment
     def attachment

data/app/concerns/context.rb

@@ -55,9 +55,9 @@ module Relay::Concerns
     # @return [Array<Relay::Models::MCP>]
     #  Saved MCP servers for the current user, newest first.
     def mcps
-      @mcps ||= user ? Relay::Models::MCP.summary_dataset(user.mcps_dataset).
-        reverse_order(:created_at).
-        all : []
+      @mcps ||= user ? Relay::Models::MCP.summary_dataset(user.mcps_dataset)
+        .reverse_order(:created_at)
+        .all : []
     end
 
     ##
@@ -94,13 +94,7 @@ module Relay::Concerns
     # @return [Hash<String, LLM::Provider>]
     #  A map of initialized LLM providers.
     def llms
-      @llms ||= {
-        "openai" => LLM.openai(key: ENV["OPENAI_SECRET"]),
-        "google" => LLM.google(key: ENV["GOOGLE_SECRET"]),
-        "anthropic" => LLM.anthropic(key: ENV["ANTHROPIC_SECRET"]),
-        "deepseek" => LLM.deepseek(key: ENV["DEEPSEEK_SECRET"]),
-        "xai" => LLM.xai(key: ENV["XAI_SECRET"])
-      }.transform_values(&:persist!)
+      Relay.providers
     end
 
     ##
@@ -114,7 +108,7 @@ module Relay::Concerns
     # @return [String]
     #  Returns the default chat model for the current provider.
     def default_model
-      case (provider = llms.fetch(self.provider)).name
+      case (provider = llms[self.provider]).name
       when :deepseek then "deepseek-v4-flash"
       when :openai then "gpt-5.4"
       when :xai then "grok-3"
@@ -142,6 +136,5 @@ module Relay::Concerns
     def user
       @user ||= Relay::Models::User[session["user_id"]] if session["user_id"]
     end
-
   end
 end

data/app/hooks/require_user.rb

@@ -4,7 +4,7 @@ module Relay::Hooks
   module RequireUser
     def call(*args, **kwargs)
       @user = Relay::Models::User[session["user_id"]]
-      @user.nil? ? r.redirect("/sign-in") : super(*args, **kwargs)
+      @user.nil? ? r.redirect("/sign-in") : super
     end
   end
 end

data/app/init/database.rb

@@ -8,7 +8,7 @@ module Relay::Database
   # @param [String] env
   # @return [Hash]
   def load(env:)
-    erb = ERB.new(File.read(File.join(__dir__, "..", "..", "db", "config.yml")))
+    erb = ERB.new(File.read(File.join(Relay.home, "db", "config.yml")))
     config = YAML.safe_load(erb.result, aliases: true)
     config.fetch(env)
   end

data/app/init/router.rb

@@ -157,8 +157,6 @@ module Relay
           Routes::ClearAttachment.new(self).call
         end
       end
-
     end
-
   end
 end

data/app/init.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 module Relay
+  require_relative "../lib/relay"
+
   require "async"
   require "async/websocket"
   require "async/websocket/adapters/rack"
@@ -12,7 +14,6 @@ module Relay
   require "zeitwerk"
   require "sequel"
   require "yaml"
-  require "relay"
 
   loader = Zeitwerk::Loader.new
   loader.inflector.inflect(
@@ -25,7 +26,6 @@ module Relay
     File.join(__dir__, "init")
   )
   loader.push_dir(__dir__, namespace: self)
-
   loader.enable_reloading if development?
   loader.setup
 
@@ -37,14 +37,24 @@ module Relay
   end
   @loader = loader
 
-  FileUtils.mkdir_p Relay.home
-  FileUtils.mkdir_p File.join(Relay.home, "db")
-  FileUtils.mkdir_p Relay.images_dir
-  FileUtils.mkdir_p Relay.logs_dir
+  Relay.bootstrap!
+  user_tools_dir = File.join(home, "tools")
+  user_loader = Zeitwerk::Loader.new
+  user_loader.tag = "relay-user-tools"
+  user_loader.push_dir(user_tools_dir)
+  user_loader.enable_reloading if development?
+  user_loader.setup
+
+  ##
+  # Returns the Zeitwerk loader used for user-installed tools
+  # @return [Zeitwerk::Loader]
+  def self.user_loader
+    @user_loader
+  end
+  @user_loader = user_loader
 
   require_relative "init/env"
   require_relative "init/database"
   require_relative "init/router"
-
   Relay.reload
 end

data/app/models/context.rb

@@ -44,13 +44,13 @@ module Relay::Models
 
     ##
     # @note
-    #  This method excludes tool calls and system messages.
+    #  This method excludes tool calls, tool returns, and system messages.
     #  It is safe to render in the UI.
     # @return [Array<Hash>]
     #  Returns persisted user and assistant messages
     def messages
       ctx.messages.filter_map do |message|
-        next if message.tool_call? || message.compaction?
+        next if message.tool_call? || message.tool_return? || message.compaction?
         next unless message.user? || message.assistant?
         {role: message.role.to_sym, content: message.content.to_s}
       end
@@ -67,11 +67,20 @@ module Relay::Models
     private
 
     def set_provider
-      LLM.method(provider).call(key: ENV["#{provider.upcase}_SECRET"], persistent: true)
+      case provider
+      when "bedrock"
+        LLM.bedrock(
+          access_key_id: ENV["AWS_ACCESS_KEY_ID"],
+          secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
+          persistent: true
+        )
+      else
+        LLM.method(provider).call(key: ENV["#{provider.upcase}_SECRET"], persistent: true)
+      end
     end
 
     def set_context
-      { model: self[:model], compactor: { retention_window: 8, token_threshold: "95%" } }
+      {model: self[:model], compactor: {retention_window: 8, token_threshold: "95%"}}
     end
 
     def set_tracer

data/app/pages/base.rb

@@ -20,6 +20,5 @@ module Relay::Pages
     def page(name, **locals)
       view(File.join("pages", name), locals:, layout_opts: {locals:})
     end
-
   end
 end

data/app/routes/base.rb

@@ -31,6 +31,5 @@ module Relay::Routes
     def htmx?
       request.env["HTTP_HX_REQUEST"] == "true"
     end
-
   end
 end

data/app/routes/mcp/base.rb

@@ -8,7 +8,7 @@ module Relay::Routes
       Relay::Models::MCP.where(id:, user_id: user.id).first || raise(Sequel::NoMatchingRow)
     end
 
-    def workspace(selected_id: nil, form:)
+    def workspace(form:, selected_id: nil)
       partial("fragments/mcp/workspace", locals: {mcps:, selected_id:, form:}) +
         partial("fragments/mcp_settings", locals: {servers: mcps, show_label: false, swap_oob: true})
     end

data/app/routes/websocket/connection.rb

@@ -37,9 +37,9 @@ class Relay::Routes::Websocket
     #  The mutable request params for the current turn
     # @return [void]
     def dispatch(conn, ctx, payload, params)
-      case
-      when interrupt?(payload) then interrupt!(conn, ctx)
-      when request_in_flight?
+      if interrupt?(payload)
+        interrupt!(conn, ctx)
+      elsif request_in_flight?
         write(conn, fragment(:status, status_bar(status: "Busy", ctx:)))
       else
         @task = Async { on_message(conn, ctx, payload, params) }
@@ -73,14 +73,14 @@ class Relay::Routes::Websocket
       file = attachment_from_payload(payload) || attachment.consume
       prompt = build_prompt(ctx, payload["message"], file)
       return if prompt.empty?
-      vars[:messages].concat [{role: :user, content: prompt}, {role: :assistant, content: +""}]
-      write(conn, fragment(:status, status_bar(status: "Thinking...", ctx:)))
-      write(conn, fragment(:remove_empty_state)) if vars[:messages].length == 2
-      write(conn, fragment(:append_message, message: vars[:messages][-2]))
-      write(conn, fragment(:append_message, message: vars[:messages][-1]))
-      write(conn, fragment(:input))
       yield_tools(ctx) do |tools|
         params[:tools] = tools
+        vars[:messages].concat [{role: :user, content: prompt}, {role: :assistant, content: +""}]
+        write(conn, fragment(:status, status_bar(status: "Thinking...", ctx:)))
+        write(conn, fragment(:remove_empty_state)) if vars[:messages].length == 2
+        write(conn, fragment(:append_message, message: vars[:messages][-2]))
+        write(conn, fragment(:append_message, message: vars[:messages][-1]))
+        write(conn, fragment(:input))
         wait_with_heartbeat(conn, proc { talk(ctx, prompt, params) })
         resolve_functions(ctx, conn, params)
       end
@@ -121,11 +121,11 @@ class Relay::Routes::Websocket
     #  The WebSocket connection object
     # @return [void]
     def resolve_functions(ctx, conn, params)
-      return if ctx.functions.empty?
-      returns = wait_with_heartbeat(conn, ctx.wait(:task))
-      wait_with_heartbeat(conn, proc { ctx.talk(returns, params) })
-      if ctx.functions.any?
-        resolve_functions(ctx, conn, params)
+      while ctx.functions?
+        returns = wait_with_heartbeat(conn, proc { ctx.wait(:task) })
+        break if returns.empty?
+        write(conn, fragment(:status, status_bar(status: tool_status(ctx.functions), ctx:))) if ctx.functions?
+        wait_with_heartbeat(conn, proc { ctx.talk(returns, params) })
       end
     end
 

data/app/tools/juke_box.rb

@@ -19,7 +19,7 @@ module Relay::Tools
           title: entry["title"],
           track: entry["track"],
           html: Relay.erb("fragments/_iframe.erb", {entry:}),
-          directions:,
+          directions:
         }
       end
     end

data/app/tools/relay_knowledge.rb

@@ -10,24 +10,26 @@ module Relay::Tools
     include Relay::Tool
 
     name "relay-knowledge"
-    description "Returns Relay or llm.rb documentation so answers can cite project details"
-    param :topic, Enum["relay", "llm.rb"], "The knowledge topic", required: true
+    description "Returns Relay, llm.rb or mruby-llm documentation"
+    parameter :topic, Enum["relay", "llm.rb", "mruby-llm"], "The knowledge topic"
+    required %i[topic]
 
     ##
     # Provides the Relay documentation
     # @return [Hash]
     def call(topic:)
       case topic
-      when "relay" then {directions:, documentation: relay_documentation}
-      when "llm.rb" then {directions:, documentation: llmrb_documentation}
+      when "relay" then {directions:, documentation: fetch(relay_resources)}
+      when "llm.rb" then {directions:, documentation: fetch(llmrb_resources)}
+      when "mruby-llm" then {directions:, documentation: fetch(mruby_llm_resources)}
       else {error: "unknown topic: #{topic}"}
       end
     end
 
     private
 
-    def relay_documentation
-      relay_resources.each_with_object({}) do |(key, url), h|
+    def fetch(resources)
+      resources.each_with_object({}) do |(key, url), h|
         res = Net::HTTP.get_response URI.parse(url)
         h[key] = res.body
       end
@@ -37,13 +39,6 @@ module Relay::Tools
       {"readme" => "https://raw.githubusercontent.com/llmrb/relay/refs/heads/main/README.md"}
     end
 
-    def llmrb_documentation
-      llmrb_resources.each_with_object({}) do |(key, url), h|
-        res = Net::HTTP.get_response URI.parse(url)
-        h[key] = res.body
-      end
-    end
-
     def llmrb_resources
       {
         "readme"   => "https://raw.githubusercontent.com/llmrb/llm.rb/refs/heads/main/README.md",
@@ -52,6 +47,12 @@ module Relay::Tools
       }
     end
 
+    def mruby_llm_resources
+      {
+        "readme" => "https://raw.githubusercontent.com/llmrb/mruby-llm/refs/heads/main/README.md"
+      }
+    end
+
     def directions
       "Reference links from the associated document in your response"
     end

data/app/views/fragments/_input.erb

@@ -22,6 +22,9 @@
     <form
       id="chat-composer"
       class="flex min-w-0 flex-1 items-end gap-3"
+      data-context-id="<%= ctx.id %>"
+      data-provider="<%= provider %>"
+      data-model="<%= model %>"
       hx-on::after-settle="this.querySelector('textarea')?.focus()"
       ws-send
     >

data/app/views/fragments/_providers.erb

@@ -12,8 +12,7 @@
       hx-swap="outerHTML"
       hx-trigger="change"
     >
-      <% Relay.providers.each do |_, builder| %>
-        <% llm = builder.call %>
+      <% Relay.providers.each do |_, llm| %>
         <option
           <%= "selected" if llm.name.to_s == provider.to_s %>
           value="<%= llm.name %>">

data/bin/relay

@@ -13,27 +13,19 @@ end
 
 def main(argv)
   case argv[0]
-  when "setup"
-    Process.spawn File.join(libexec, "setup"), *argv[1..]
-    Process.wait
-  when "download-models"
-    Process.spawn File.join(libexec, "download-models"), *argv[1..]
-    Process.wait
-  when "migrate"
-    Process.spawn File.join(libexec, "migrate"), *argv[1..]
-    Process.wait
-  when "start"
-    Process.spawn File.join(libexec, "start"), *argv[1..]
+  when "start", "setup", "download-models", "migrate", "bootstrap", "console"
+    Process.spawn File.join(libexec, argv[0]), *argv[1..]
     Process.wait
   else
-    require "relay"
+    require_relative "../lib/relay"
     warn Relay.banner
     warn "Usage: relay [COMMAND] [OPTIONS]\n\n" \
-         "Commands:\n" \
-         "  setup             Setup Relay for the first time\n" \
-         "  download-models   Download provider model catalogs\n" \
-         "  start             Start Relay\n" \
-         "  migrate           Run database migrations\n"
+        "Commands:\n" \
+        "  setup             Setup Relay for the first time\n" \
+        "  download-models   Download provider model catalogs\n" \
+        "  start             Start Relay\n" \
+        "  migrate           Run database migrations\n" \
+        "  console           Start an IRB console with Relay booted\n"
   end
 rescue Interrupt
   wait

data/config.ru

@@ -4,7 +4,7 @@ ENV["RACK_MULTIPART_BUFFERED_UPLOAD_BYTESIZE_LIMIT"] ||= (64 * 1024 * 1024).to_s
 
 require_relative "app/init"
 
-use Rack::Static, urls: ["/g"], root: Relay.home
+use Rack::Static, urls: ["/g"], root: Relay.public_dir
 use Rack::Static, urls: ["/images", "/stylesheets", "/js"], root: Relay.public_dir
 case Relay.environment
 when "development"

data/lib/relay/jukebox.rb

@@ -63,7 +63,8 @@ module Relay
     def extract_youtube_id(uri)
       path = uri.path.to_s
       return path.split("/").reject(&:empty?).last if path.start_with?("/embed/", "/shorts/")
-      CGI.parse(uri.query.to_s).fetch("v", []).first
+      form = URI.decode_www_form(uri.query.to_s)
+      (form.to_h["v"] || []).first
     end
 
     def songs

data/lib/relay/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Relay
-  VERSION = "0.5.0"
+  VERSION = "0.7.0"
 end