brut 0.17.0 → 0.18.0

data/brutrb.com/instrumentation.md

@@ -1,331 +0,0 @@
-# Instrumentation and Observability
-
-Brut has built-in support for OpenTelemetry, which is an open standard used by many observability vendors
-to allow you to understand the behavior of your app in production.  Brut also includes a configuration for
-the [otel-desktop-viewer](https://github.com/CtrlSpice/otel-desktop-viewer/), which allows you to see
-instrumentation in development.
-
-## Overview
-
-### Why Instrument?
-
-In production, you'll need to know what your app is doing and how well it's working.  Historically, logs
-can provide this information in a roundabout way. Over the last many years, Application Performance
-Monitoring (APM) vendors like New Relic and Data Dog allowed developers to see much richer detail about
-how an app is working.
-
-You could see, for example, the 95th percentil of your dashboard controller's performance, or the top 10
-slowest SQL statements your app is executing.  OpenTelemetry attempts to unify the API used to communicate
-this information from your app to your chosen vendor, and most vendors support it.
-
-Instrumentation, then, is a way to record what your app is doing, how long its taking, and perhaps even
-why it's doing what it's doing, down to a very specific level.  If properly configured, you could examine
-the performance of the app for a particular user on a particular day.
-
-### Setting up Instrumentation
-
-Brut automatically sets up OpenTelemetry (OTel) tracing.  The primary interface you will use is
-`Brut::Instrumentation::OpenTelemetry`, which is available via `Brut.container.instrumentation`.  We'll
-discuss that in a moment.
-
-To configure the specifics of where the traces will go, the OTel gem uses environment variables:
-
-| Variable                             | Value                      | Purpose                                                                                                                                                         |
-|--------------------------------------|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `OTEL_EXPORTER_OTLP_ENDPOINT`        | Depends on environment     | Where to send the tracers. This is provided by your vendor, but is `http://otel-desktop-viewer:4318` in development                                             |
-| `OTEL_EXPORTER_OTLP_HEADERS`         | Depends on vendor          | Your vendor may ask you to set this. It often contains identifying information or API keys                                                                      |
-| `OTEL_EXPORTER_OTLP_PROTOCOL`        | http/protobuf              | Your vendor may request a different protocol, but protobuf is common and supported by otel-desktop-viewer                                                       |
-| `OTEL_LOG_LEVEL`                     | debug                      | Useful when setting everything up to understand why things aren't working if they aren't working                                                                |
-| `OTEL_RUBY_BSP_START_THREAD_ON_BOOT` | false                      | Deals with esoteric issues with Puma. See [this GitHub issue](https://github.com/open-telemetry/opentelemetry-ruby/issues/462) for the details.
-| `OTEL_SERVICE_NAME`                  | Your app's `id` from `App` | Identifiers your app's name to the vendor                                                                                                                       |
-| `OTEL_TRACES_EXPORTER`               | otlp                       | Configures the class inside the OTel gem that will export the instrumentation to the vendor. If you omit this, Brut will log the instrumentation to the console |
-
-When you created your Brut app, your `.env.development` and `.env.test` should have values for all these
-environment variables that will send instrumentation to the otel-desktop-viewer that was also configured.
-
-If you run your app using `bin/dev` and use the app for a bit, then go to `http://localhost:8000`, you
-will see the otel-desktop-viewer UI and can browse the spans and traces sent by Brut.
-
-
-### What is Instrumented By Default
-
-Brut attempts to automatically instrument useful things so you don't have to do anything to start getting
-data.  Brut will attempt to conform to standard semantics for HTTP requests and SQL statements.
-
-Here is a non-exhaustive list of what Brut automatically instruments:
-
-* How long each page or handler request takes, broken down by components.
-* CLI execution time
-* Time to rebuild the schema for tests
-* Time to run tests
-* Time to apply migrations
-* Time spent inside a route hook
-* The locale detected from the browser
-* The layout class used when rendering a page
-* If a requested path is owned by Brut or not
-* Ignored parameters on all form submissions
-* How long reloading takes in development
-* CSP reporting results
-* SQL Statements
-
-> [!WARNING]
-> `Sequel::Extensions::BrutInstrumentation` sets up telemetry for
-> Sequel, and it does it in a relatively simplistic way.  The result
-> is that *all* SQL statements are part of the telemetry, including
-> the actual values inserted or used in `WHERE` clauses.
-> While you should not be putting sensitive data into your database,
-> be warned that this is happening. There are plans to improve this
-> to be more flexible and reduce the chance of sensitive data
-> being sent in traces.
-
-### Adding Your Own Instrumentation
-
-You can add instrumentation in two main ways, both of which can be used together.
-
-#### Instrumenting Existing Methods
-
-Although Brut instruments the entrypoints to pages, handlers, and components, you will likely have your own set of back-end business logic that needs to be instrumented.  If you aren't trying to diagnose a specific problem and just want to see your back-end class' methods show up in your instrumentation vendor's dashboard, `Brut::Instrumentation::Methods` will be the easiest way to do that.
-
-`Brut::Instrumentation::Methods` can be included in any class, and provides three class methods, which are *mutually exclusive*:
-
-* `instrument_all` instruments all methods, public and private.
-* `instrument_public` instruments only public methods.
-* `instrument` instruments one or more named methods.
-
-`initialize` is never instrumented.
-
-Consider this class:
-
-```ruby
-class Widget
-  def initialize
-    # ...
-  end
-
-  def search
-    # ...
-  end
-
-  def save
-    # ...
-  end
-
-private
-
-  def delete_orphans
-    # ...
-  end
-end
-```
-
-If we use `instrument_all`…
-
-```ruby
-class Widget
-  include Brut::Instrumentation::Methods
-  instrument_all
-
-  # ...
-end
-```
-
-…`search`, `save`, and `delete_orphans` will be instrumented.  If we use `instrument_public`…
-
-```ruby
-class Widget
-  include Brut::Instrumentation::Methods
-  instrument_public
-
-  # ...
-end
-```
-
-…then only `search` and `save` are instrumented.
-
-We can pick and choose by using `instrument`.
-
-```ruby{2,7,20}
-class Widget
-  include Brut::Instrumentation::Methods
-  def initialize
-    # ...
-  end
-
-  instrument def search
-    # ...
-  end
-
-  def save
-    # ...
-  end
-
-private
-
-  def delete_orphans
-    # ...
-  end
-  instrument :delete_orphans
-end
-```
-
-Above, `search` and `delete_orphans` are instrumented. Since `def` in Ruby returns a symbol, `instrument def search` is the same as `instrument :search`.
-
-#### Explicit Instrumentation with Spans, Attributes, and Events
-
-To add explicit instrumentation, you'll create one or more of the following:
-
-* *Spans* record a block of code. They are shown as a sub-span if one is already in effect.  When you
-create a span, that means it will be shown in the context of the HTTP request.
-* *Attributes* can be added to the current span to provide more context about what is happening. For
-example, the HTTP request method is an attribute of the span used for the HTTP request.  These attributes
-allow for sophisticated querying in the vendor's UI.
-* *Events* record things that happen and metadata about that thing. These are like log statements. They
-are associated with the span you are in when you add the event.
-
-These can all be added via `Brut.container.instrumentation`, which is a
-`Brut::Instrumentation::OpenTelemetry` instance.
-
-These methods are available:
-
-* `span(name,**attributes,&block)` - Create a new span around the block yielded.
-* `add_attributes(attributes)` - Add attributes to the current span. These will be prefixed with your
-app's prefix so it's clear in the observability UI that they are for your app and not standard.
-* `add_event(name,**attributes)` - Add an event with optional attributes for the current span.
-* `record_exception(ex,attributes=nil)` - Record an exception that was caught.
-* `record_and_reraise_exception!(ex,attributes=nil)` - Record an exception and raise it.
-
-Suppose you want to instrument `RequireAuthBeforeHook` from the [hooks](/hooks) documentation. Although
-the hook's `before` method is instrumented by Brut already, let's add some metadata to that span, and also
-add a span around the login check.
-
-```ruby {11-16,18,20,23-26}
-# app/src/front_end/route_hooks/require_auth_before_hook.rb
-class RequireAuthBeforeHook < Brut::FrontEnd::RouteHook
-  def before(request_context:,session:,request:,env:)
-    is_home_page       = request.path_info.match(/^\/?$/)
-    is_auth_route      = request.path_info.match?(/^\/auth\//)
-    is_brut_owned_path = env["brut.owned_path"]
-
-    requires_login = !is_home_page  &&
-                     !is_auth_route && 
-                     !is_brut_owned_path
-    Brut.container.instrumentation.add_attributes(
-      requires_login:,
-      is_home_page:,
-      is_auth_route:,
-      is_brut_owned_path:
-    )
-
-    Brut.container.instrumentation.span("login-check") do |span|
-      if session.logged_in?
-        span.add_attributes(logged_in: true)
-        request_context[:authenticated_account] = session.authenticated_account
-        requires_login = false
-      else
-        span.add_attributes(logged_in: false)
-      end
-    end
-
-    if requires_login
-      redirect_to(Auth::LoginPage)
-    else
-      continue
-    end
-
-  end
-end
-```
-
-Now, for every request someone makes to our app, we will see a span for the `RequireAuthBeforeHook`, and
-inside that span, we'll see the attributes we added as well as a sub-span representing the login check
-(which itself will have an attribute about the user's logged-in status).
-
-### Client-Side Observability
-
-
-The class `Brut::FrontEnd::Handlers::InstrumentationHandler` is set up to receive information from the
-client-side to provide insights about client-side behavior as part of a server-side request.  Brut
-attempts to join up any client-side instrumentation to the request that served it.
-
-It does this via the `Brut::FrontEnd::Components::Traceparent` component, which is included in your default layout when you created your Brut app.  This creates a `<meta>` tag containing standardized information used to
-connect the client-side behavior to the server-side request.
-
-The Brut custom element `<brut-tracing>` uses this information, along with statistics from the browser, to
-send a custom payload back to Brut at the route `/__brut/instrumentation`, which is handled by the
-aforementioned `InstrumentationHandler`.
-
-You should then see client-side tracing information as a sub-span of your HTTP request.  The information available depends on the browser, and some browsers don't send much. Also keep in mind that clock drift is real and while client-side timings are accurate, the timestamps will not be.
-
-### Logging
-
-Brut configures [SemanticLogger](https://logger.rocketjob.io/), but uses it sparingly.  Currently, Brut performs very little logging, and no request logging.  You may have noticed that your app doesn't produce a lot of output in development.  Brut's assumption is that you will use an OpenTelemetry vendor to understand your app in production or the otel-desktop-viewer in developoment.
-
-That said, since SemanticLogger is configured, you can use it at will:
-
-```ruby
-class HomePage
-  def page_template
-    SemanticLogger[self.class].debug "page being rendered"
-    
-    # ...
-
-  end
-end
-```
-
-The logging system is currently not very configurable, and works as follows:
-
-* In development, log messages are written to the standard output and to `logs/development.log`
-* In test, log messages are written to `logs/test.log`
-* In production, log messages are written to the standard output
-
-The default log level is "debug" for the web app at "fatal" for CLI apps.  You can set `LOG_LEVEL` in the environment to change this:
-
-* `"debug"` - Show all messages
-* `"info"` -  Show info and above (not debug messages)
-* `"warn"` -  Show warnings and above (not info, not debug)
-* `"error"` -  Show errors and fatals only
-* `"fatal"` -  Show fatals only
-
-Most CLIs also allow `--log-level` to accept one of these strings as wel ass `--verbose` to set the log level to debug.
-
-## Testing
-
-Generally you don't want to test instrumentation unless it's highly complex and critical to the app's
-ability to be maintained.  Ideally, your end-to-end tests will cover all the instrumentation code you
-write so you can be sure that none of that causes a problem.
-
-## Recommended Practices
-
-Entire books and conferences exist on how to properly instrument your app.  Our suggestion is to take what
-you have by default and add additional instrumentation only to solve specific problems or identify
-specific issues.
-
-## Technical Notes
-
-> [!IMPORTANT]
-> Technical Notes are for deeper understanding and debugging. While we will try to keep them up-to-date with changes to Brut's
-> internals, the source code is always more correct.
-
-_Last Updated Aug 27, 2025_
-
-OpenTelemetry is notoriously opaque and, ironically, unobservable in its own behavior.  Thus, the implementation is subject to change as I figure out what actually does what.
-
-### Web Requests
-
-`Brut::FrontEnd::Middlewares::OpenTelemetrySpan` is configured in `Brut::Framework::MCP` as the first middleware.  It sets up the outer span for all web requests.  Inside each request block (the code internal to Brut that calls handlers or pages), this span's name is modified with the HTTP method and path via the `brut.otel.root_span` in the Rack environment.
-
-### Client-Side
-
-The client-side portion of this is highly customized.  The Otel open source code for the client side is
-massive and hugely complex, so Brut decided to try to produce something simple and straightforward as a
-start. This can and will evolve over time.
-
-### CLI Commands
-
-Brut CLI commands are instrumented as well,
-in `Brut::CLI::App` in `execute!`, however the trace only begins if the underlying command is going to be executed. This may change.
-
-### Sidekiq Jobs
-
-Although Brut currently does not provide a default Sidekiq configuration, if you set up Sidekiq and include the `opentelemetry-instrumentation-sidekiq` gem in your app's `Gemfile`, you should see instrumentation of your Sidekiq jobs.  In practice, this default set up doesn't seem to work very well, so expect this to change for the better.

data/brutrb.com/javascript.md

@@ -1,122 +0,0 @@
-# JavaScript
-
-Brut provides basic bundling using [esbuild](https://esbuild.github.io/).  You can
-use any front-end framework with Brut, but you don't have to use one.
-
-Brut provides [BrutJS](/brut-js), which is a lightweight library of HTML custom elements and utility code.  These elements can provide a fair bit of front-end functionality using progressive enhancement without the need for a framework.
-
-## Overview
-
-All your app's JavaScript lives in `app/src/front_end/js`, or in modules you bring in via `package.json`.  Brut
-will *bundle* all of that up into a single `.js` file that is served up with your app.  Brut does this by using
-esbuild, a stable and standardized tool for bundling JavaScript.
-
-The way esbuild works is to be given an *entry point* that requires, or transitively requires, all of your
-JavaScript by using ES6 modules. `app/src/front_end/js/index.js` is the entry point for your app.
-
-For example, if you have a `Widget` class that uses a `Status` class, and you also use the third party library
-"foobar", here is how all the files would look.
-
-First, `package.json` (in your app's root) would include `"foobar"` (and it must set `"type"` to `"module"`):
-
-```json {3,6}
-{
-  "name": "your-app",
-  "type": "module",
-  "license": "UNLICENSED",
-  "dependencies": {
-    "foobar": "^0.0.11"
-  },
-  "devDependencies": {
-    "chokidar-cli": "^3.0.0",
-    "esbuild": "^0.20.2",
-    "jsdom": "^25.0.1",
-    "mocha": "^10.7.3",
-    "playwright": "^1.50.1"
-  },
-}
-```
-
-Next, `app/src/front_end/js/index.js` would import both `"foobar"` and `"Widget"`:
-
-```javascript
-import { foobar } from "foobar"
-import Widget from "./Widget"
-
-// ...
-```
-
-Notice that "foobar", since it's brought in as a third party dependency, is imported without a `./`.  Be careful
-here!  Every third party library has a different syntax for how to import whatever it is or does. Consult the
-documentation of each third party library you wish to import.
-
-The second `import` uses a `./` because it's importing a file in `app/src/front_end/js`, namely `Widget.js`.  Be
-careful here, too, as you must be sure to `export` the right thing.  Here's what `app/src/front_end/js/Widget.js`
-might look like:
-
-```javascript
-import Status from "./extra/Status"
-
-class Widget {
-  status = new Status()
-}
-
-export default Widget
-```
-
-Note that we import `Status` here.  Unlike Ruby, ES6 modules requires each class that references a class to
-import it explicitly.  Also notice that we do `export default Widget`, which allows `import Widget` to work.
-
-Finally, `app/src/front_end/extra/Status.js` looks like so:
-
-```javascript
-class Status {
-}
-export default Status
-```
-
-When `bin/build-assets` runs, esbuild will use `app/src/front_end/js/index.js` as its *entry point*, and will
-bundle both `Widget.js` and the "foobar" library.  When it bundles `Widget.js`, it will see that it imports
-`extra/Status.js` and bundle that, too.
-
-This bundle can be included in your app by ensuring this is in your layout:
-
-```ruby {5}
-def view_template
-  doctype
-  html(lang: "en") do
-    head do
-      script(defer: true, src: asset_path("/js/app.js"))
-      # ...
-```
-
-The `asset_path` helper takes a logical path—`/js/app.js`—and returns the actual path the browser can use.  More
-details on this can be found in [assets](/assets).
-
-## Testing
-
-Client-side behavior is best tested with end-to-end tests, however you can simplify your end-to-end tests by
-creating unit tests of your custom elements.  [BrutJS provides limited support for this](/brut-js/api/module-testing.html)
-
-## Recommended Practices
-
-Brut encourages you to use HTML custom elements as progressive enhancements over server-generated views.  This sort of client-side code will age well.  The toolchain and dependencies are minimal, so you will not have to worry too much about code written this way.
-
-It *will* be lower level and more verbose than existing frameworks.  We would argue that it is not significantly more difficult and the sustainability is worth it.
-
-## Technical Notes
-
-> [!IMPORTANT]
-> Technical Notes are for deeper understanding and debugging. While we will try to keep them up-to-date with changes to Brut's
-> internals, the source code is always more correct.
-
-_Last Updated May 7, 2025_
-
-Currently, Brut only supports a single entry point and bundle.  This could be easily made more flexible if there
-is a desire to finely tweak the JavaScript loaded on specific pages.
-
-Brut also does not expose any esbuild configuration.  This could be provided in the future, but for now, it is hard-coded.
-
-Brut may provide more direct support for import maps, but as of now, import maps are
-not widely used outside of Rails, and tend to cause a lot of problems, especially if
-you aren't able to field an HTTP/2 web server (or even know what that is).

data/brutrb.com/jobs.md

@@ -1,114 +0,0 @@
-# Background Jobs
-
-Brut ships without any background job system, however it should work with any system you'd like to use.  Brut
-can install/configure Sidekiq for you, however you are expected to understand Sidekiq in order to use it.
-
-## Setting up Sidekiq
-
-Brut's code-generation system used for installing capabilities are called *segments*, and Brut provides a
-Sidekiq segment you can use to get an initial working setup of Sidekiq in your Brut app.
-
-### Adding the Segment
-
-1. Ensure your project files are all committed. This is so you can easily see (and, if needed, undo) the
-   changes `mkbrut` will make.
-2. Use `mkbrut` to add the segment:
-
-   ```
-   docker run \
-          --pull always \
-          -v "$PWD":"$PWD" \
-          -w "$PWD" \
-          -u $(id -u):$(id -g) \
-          -it \
-          thirdtank/mkbrut \
-          mkbrut add-segment -r /path/to/your/project sidekiq
-   ```
-3. This will modify and create various files in your project. Check them out if you like:
-
-   ```
-   > git status
-   ```
-4. Exit your dev environment (i.e. hit `Ctrl-C` wherever you ran `dx/start`).
-5. Rebuild and restart your dev environment. This may take a moment, since Valkey will be downloaded.
-
-   ```
-   your-computer> dx/build
-   your-computer> dx/start
-   ```
-6. In another Terminal, connect to your dev container and run `bin/setup`
-
-   ```
-   your-computer> dx/exec bash
-   devcontainer> bin/setup
-   ```
-7. The segment provides an integration test that will use the actual Sidekiq server and client, running
-   against the actual Valkey database that was installed:
-
-   ```
-   devcontainer> bin/test e2e specs/integration/sidekiq_works.spec.rb
-   ```
-
-If this test passes, you are ready to go.
-
-### Using Sidekiq in Brut
-
-Jobs live in `app/src/back_end/jobs`, however this is just a convention and is not enforced - you can place a
-job anywhere that Zeitwerk will find the class.  Brut also provides basic configuration and a base job.
-
-| File | Purpose|
-|------|--------|
-| `app/config/sidekiq.yml` | Standard configuration for Sidekiq |
-| `app/src/back-end/jobs/app_job.r` | Base class for your jobs that includes `Sidekiq::Job` |
-| `app/src/back-end/segments/sidekiq_segment.rb` | Initial client and server configuration for Sidekiq (that you can't do with `sidekiq.yml`. This sets up basic observability for your jobs |
-
-### Accessing the Web UI
-
-The Sidekiq segment mounts the Sidekiq Web UI to your app inside `config.ru`:
-
-```ruby
-# ...
-map "/sidekiq" do
-  use Rack::Auth::Basic, "Sidekiq" do |username, password|
-    [username, password] == [ENV.fetch("SIDEKIQ_BASIC_AUTH_USER"), ENV.fetch("SIDEKIQ_BASIC_AUTH_PASSWORD")]
-  end
-  run Sidekiq::Web.new
-end
-# ...
-```
-
-Values for `SIDEKIQ_BASIC_AUTH_USER` and `SIDEKIQ_BASIC_AUTH_PASSWORD` for dev and test are placed into
-`.env.development` and `.env.test`, respectively. You must provide these values for production, based on
-however you are managing environment variables.
-
-Once you start the app, navigat to `http://localhost:6502/sidekiq` and enter the username/password from
-`.env.development`. You should see the web UI.
-
-### Deploying with The Heroku Segment
-
-If you have set up [Heroku Container-based Deployment](/deployment.md#heroku-container-based-deployment), you
-may need to modify `deploy/heroku_config.rb`.  The Sidekiq segement should have edited this, however if you
-installed the Heroku segment after setting up Sidekiq, you'll need to add to the file:
-
-```ruby [2-6]
-class HerokuConfig
-  def self.additional_images
-    {
-      "sidekiq" => {
-        cmd: "bin/run sidekiq",
-      }
-    }
-  end
-end
-```
-
-## Setting Up Other Job Systems
-
-To use another job system, you'll likely want to start with `app/src/app.rb`.  You can place all your
-initialize code in `#boot!` to get things working, then factor it out from there.  `App`, the class in that
-file, is a normal class, so you can extract your setup to other normal classes and bring them in as you would
-in any other Ruby app.
-
-Just note that `App`'s `initialize` method should avoid making network connections, so while you are safe to
-create objects and configuration here, do not connect to databases or anything like that.  You *can* do that
-inside `boot!`.