upright 0.2.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1d1bb4a1f14f8e4dc16cf5bebcd5868917b2c0703f6dd94592d6f8c818dfa0b9
-  data.tar.gz: 4d3a0593d1f49f256145340f3e64c9ab6f467a759f1c0a60e81e58635528e0d1
+  metadata.gz: c91a962bc77007bffeebba4df5da09c213a42c46a416de5fce7d19e9169258f7
+  data.tar.gz: 47b63615e16a15aede26f738a3ffc7256fada1aab09b775ff7ddd90282c91ea0
 SHA512:
-  metadata.gz: 5d06f946c9874af94e42bca38bb58671e227b3c4f2ac2f9a523d36b3efca04771a3d0a087bb2644ea067e5a8094d35e1370ee85ded8ea140af084afbf4a4e29a
-  data.tar.gz: c2ac9d3d2822197887e22308dbdce7be1b47692825dcb25cbb660f0de8e24643788b2cd5a63c72c4863041b619a35e3b31b9e241c69861b59ff66c8398fcd9b2
+  metadata.gz: 25a77e92224a4869f1da746deba9f5a4440c4a071d2e3b0c8fef363c2abd36e270cc9a1147a11081284a85f0deb8e22cc73a09d8a3dcd183ea29dd7a0419e872
+  data.tar.gz: a5e047ccb8feb9e864652496e81cdbb17ea4e0ae0972b8e380df9fa1206b011c23d5fd0b1e8bae2f366313dcb48b7af25d3babeba1f96e1211967bf70e954415

data/README.md

@@ -47,7 +47,8 @@ Upright is a self-hosted synthetic monitoring system. It provides a framework fo
 
 ## Installation
 
-Upright is designed to be run in it's own Rails app and deployed with Kamal.
+> [!NOTE]
+> Upright is designed to be run in its own Rails app and deployed with Kamal.
 
 ### Quick Start (New Project)
 
@@ -58,7 +59,7 @@ rails new my-upright --database=sqlite3 --skip-test
 cd my-upright
 bundle add upright
 bin/rails generate upright:install
-bin/rails db:migrate
+bin/rails db:prepare
 ```
 
 Start the server:
@@ -75,7 +76,10 @@ Visit http://app.my-upright.localhost:3000 to see your Upright instance.
 
 The `upright:install` generator creates:
 
+- `config/database.yml` - Development and test split into `primary` (probe results), `persistent` (rollups and incidents, migrated from the gem) and `queue` (Solid Queue) databases
+- `config/recurring.yml` - Probe schedules and the engine's housekeeping, health, rollup, incident and maintenance jobs
 - `config/initializers/upright.rb` - Engine configuration
+- `config/initializers/content_security_policy.rb` - Ready-to-enable CSP covering the engine's CDN dependencies
 - `config/sites.yml` - Site definitions for each VPS you host Upright on
 - `config/prometheus/prometheus.yml` - Prometheus configuration
 - `config/alertmanager/alertmanager.yml` - AlertManager configuration
@@ -121,6 +125,8 @@ shared:
       country: NL
       geohash: u17982
       provider: digitalocean
+      stores_metrics: true
+      primary: true
 
     - code: sfo
       city: San Francisco
@@ -131,14 +137,16 @@ shared:
 
 Each site node identifies itself via the `SITE_SUBDOMAIN` environment variable, configured in your Kamal deploy.yml.
 
+Two optional flags give a site a role beyond running probes: `primary` serves the app and status hostnames and runs the jobs writing the shared database, and `stores_metrics` runs a local Prometheus and Alertmanager. See [Sites and their roles](https://github.com/basecamp/upright/blob/main/docs/sites.md) for what each one changes, the health metrics every site exports, and how daily rollups read across them.
+
 ### Authentication
 
 #### Static Credentials
 
-Upright uses static credentials by default with username `admin` and password `upright`.
+Upright uses static credentials by default with username `admin` and the password taken from the `ADMIN_PASSWORD` environment variable.
 
-> [!WARNING]
-> Change the default password before deploying to production by setting the `ADMIN_PASSWORD` environment variable.
+> [!IMPORTANT]
+> There is no default password: `ADMIN_PASSWORD` must be set, and sign-in fails closed without it. Set it as a Kamal secret in production (the generated `config/deploy.yml` already lists it) and export it locally for development.
 
 
 #### OpenID Connect
@@ -157,6 +165,97 @@ Upright.configure do |config|
 end
 ```
 
+### Probe Result Cleanup
+
+Upright automatically cleans up old probe results on a recurring schedule. You can configure the retention thresholds:
+
+```ruby
+Upright.configure do |config|
+  config.stale_success_threshold = 24.hours     # Delete successful results older than this (default: 24 hours)
+  config.stale_failure_threshold = 30.days       # Delete failed results older than this (default: 30 days)
+  config.failure_retention_limit = 20_000        # Keep at most this many failed results (default: 20,000)
+end
+```
+
+## Custom Probe Types
+
+Upright ships with four built-in probe types: HTTP, Playwright, SMTP, and Traceroute. You can register your own to extend the system.
+
+### 1. Register the type
+
+Add it to your initializer so Upright knows about its name and icon:
+
+```ruby
+# config/initializers/upright.rb
+Upright.configure do |config|
+  config.probe_types.register :ping, name: "Ping", icon: "📶"
+end
+```
+
+### 2. Create the probe class
+
+Add a Ruby class in your `probes/` directory that extends `FrozenRecord::Base` and includes `Upright::Probeable` and `Upright::ProbeYamlSource`. Implement `probe_type`, `probe_target`, `check`, and `on_check_recorded`:
+
+```ruby
+# probes/ping_probe.rb
+class PingProbe < FrozenRecord::Base
+  include Upright::Probeable
+  include Upright::ProbeYamlSource
+
+  stagger_by_site 3.seconds
+
+  # Hostnames, IPv4, and IPv6 addresses only. The first character may not be
+  # a dash, so a configured host can never be mistaken for a ping flag.
+  HOST_PATTERN = /\A[a-z0-9:][a-z0-9:._-]*\z/i
+
+  def check
+    raise ArgumentError, "invalid ping host: #{host.inspect}" unless HOST_PATTERN.match?(host.to_s)
+
+    @ping_output, status = Open3.capture2e("ping", "-c", "1", "-W", "5", "--", host)
+    status.success?
+  end
+
+  def on_check_recorded(probe_result)
+    if @ping_output.present?
+      Upright::Artifact.new(name: "ping.log", content: @ping_output).attach_to(probe_result)
+    end
+  end
+
+  def probe_type = "ping"
+  def probe_target = host
+end
+```
+
+The `check` method should return a truthy value on success and a falsy value on failure.
+
+### 3. Define probes in YAML
+
+Create a YAML file matching the class name (e.g., `PingProbe` → `probes/ping_probes.yml`):
+
+```yaml
+# probes/ping_probes.yml
+- name: "Cloudflare DNS"
+  host: "1.1.1.1"
+
+- name: "Google DNS"
+  host: "8.8.8.8"
+```
+
+Fields defined in the YAML are available as methods on the probe instance (e.g., `host`).
+
+### 4. Schedule it
+
+Add a recurring job in `config/recurring.yml`:
+
+```yaml
+production:
+  ping_probes:
+    command: "PingProbe.check_and_record_all_later"
+    schedule: every 30 seconds
+```
+
+The custom type will automatically appear in all dashboard dropdowns and filter links.
+
 ## Defining Probes
 
 ### HTTP Probes
@@ -171,12 +270,15 @@ Add probes to `probes/http_probes.yml`:
 - name: API Health
   url: https://api.example.com/health
   expected_status: 200
+  alert_severity: critical
 
 - name: Admin Panel
   url: https://admin.example.com
   basic_auth_credentials: admin_auth  # Key in Rails credentials
 ```
 
+The optional `alert_severity` field controls the Prometheus alert severity when a probe fails. Values: `medium`, `high` (default), `critical`.
+
 ### SMTP Probes
 
 Add probes to `probes/smtp_probes.yml`:
@@ -317,70 +419,83 @@ image: your-org/upright
 servers:
   web:
     hosts:
-      - nyc.upright.example.com
-    env:
-      tags:
-        SITE_SUBDOMAIN: nyc
-
+      - ams.upright.example.com: [amsterdam]
+      - nyc.upright.example.com: [new_york]
+      - sfo.upright.example.com: [san_francisco]
+  jobs:
     hosts:
-      - ams.upright.example.com
-    env:
-      tags:
-        SITE_SUBDOMAIN: ams
+      - ams.upright.example.com: [amsterdam]
+      - nyc.upright.example.com: [new_york]
+      - sfo.upright.example.com: [san_francisco]
+    cmd: bin/jobs
 
-    hosts:
-      - sfo.upright.example.com
-    env:
-      tags:
-        SITE_SUBDOMAIN: sfo
-
-registry:
-  server: ghcr.io
-  username: your-org
-  password:
-    - KAMAL_REGISTRY_PASSWORD
+proxy:
+  app_port: 3000
+  ssl: true
+  hosts:
+    - "*.upright.example.com"
 
 env:
-  clear:
-    RAILS_ENV: production
-    RAILS_LOG_TO_STDOUT: true
   secret:
     - RAILS_MASTER_KEY
-    - OIDC_CLIENT_ID
-    - OIDC_CLIENT_SECRET
+    - ADMIN_PASSWORD
+  tags:
+    amsterdam:
+      SITE_SUBDOMAIN: ams
+    new_york:
+      SITE_SUBDOMAIN: nyc
+    san_francisco:
+      SITE_SUBDOMAIN: sfo
 
 accessories:
   playwright:
-    image: mcr.microsoft.com/playwright:v1.55.0-noble
-    cmd: npx -y playwright run-server --port 53333
-    host: nyc.upright.example.com
-    port: 53333
-    env:
-      clear:
-        DEBUG: pw:api
+    image: jacoblincool/playwright:chromium-server-1.55.0
+    port: "127.0.0.1:53333:53333"
+    roles:
+      - jobs
+
+  prometheus:
+    image: prom/prometheus:v3.2.1
+    hosts:
+      - ams.upright.example.com
+    cmd: >-
+      --config.file=/etc/prometheus/prometheus.yml
+      --storage.tsdb.path=/prometheus
+      --storage.tsdb.retention.time=30d
+      --web.enable-otlp-receiver
+    files:
+      - config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
+      - config/prometheus/rules/upright.rules.yml:/etc/prometheus/rules/upright.rules.yml
+
+  alertmanager:
+    image: prom/alertmanager:v0.28.1
+    hosts:
+      - ams.upright.example.com
+    cmd: --config.file=/etc/alertmanager/alertmanager.yml
+    files:
+      - config/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml
 ```
 
 ## Observability
 
 ### Prometheus
 
-The engine exposes metrics at `/metrics`. Configure Prometheus to scrape:
+Metrics are exposed via a Puma plugin at `http://0.0.0.0:9394/metrics`. Configure Prometheus to scrape:
 
 ```yaml
 scrape_configs:
   - job_name: upright
     static_configs:
-      - targets: ['localhost:3000']
-    metrics_path: /metrics
+      - targets: ['localhost:9394']
 ```
 
 ### Metrics Exposed
 
 - `upright_probe_duration_seconds` - Probe execution duration
-- `upright_probe_success` - Probe success/failure (1/0)
-- `upright_probe_status_code` - HTTP status code for HTTP probes
+- `upright_probe_up` - Probe status (1 = up, 0 = down)
+- `upright_http_response_status` - HTTP response status code
 
-Labels include: `probe_name`, `probe_type`, `site_code`, `site_city`, `site_country`
+Labels include: `type`, `name`, `site_code`, `site_city`, `site_country`
 
 ### AlertManager
 
@@ -391,20 +506,12 @@ groups:
   - name: upright
     rules:
       - alert: ProbeDown
-        expr: upright_probe_success == 0
-        for: 5m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Probe {{ $labels.probe_name }} is down"
-
-      - alert: ProbeSlow
-        expr: upright_probe_duration_seconds > 10
+        expr: upright_probe_up == 0
         for: 5m
         labels:
-          severity: warning
+          severity: "{{ $labels.alert_severity }}"
         annotations:
-          summary: "Probe {{ $labels.probe_name }} is slow"
+          summary: "Probe {{ $labels.name }} is down"
 ```
 
 ### OpenTelemetry
@@ -443,20 +550,65 @@ bin/dev
 
 Visit http://app.upright.localhost:3000 and sign in with:
 - **Username**: `admin`
-- **Password**: `upright` (or value of `ADMIN_PASSWORD` env var)
+- **Password**: the value of the `ADMIN_PASSWORD` env var, e.g. `ADMIN_PASSWORD=upright bin/dev` (a throwaway value like this is fine for local testing only — never for a deployed site)
 
 ### Testing Playwright Probes
 
 Run probes with a visible browser window:
 
 ```bash
-LOCAL_PLAYWRIGHT=1 bin/rails console
+HEADLESS=false bin/rails console
 ```
 
 ```ruby
 Probes::Playwright::MyServiceAuthProbe.check
 ```
 
+### Upgrading Playwright
+
+Playwright versions are pinned via `Upright::PLAYWRIGHT_VERSION` in `lib/upright/version.rb`. This drives the Ruby gem and npm package versions. To upgrade:
+
+1. Update `PLAYWRIGHT_VERSION` in `lib/upright/version.rb`
+2. Update the version in `package.json`
+3. Run `bin/setup` (or manually: `npm install && npx playwright install chromium`)
+4. Run `bin/rails test` to verify compatibility
+5. Commit the updated `package.json` and `package-lock.json`
+
+### Viewing Playwright Traces
+
+Upright stores a Playwright trace as a probe result artifact and does not serve
+the Playwright Trace Viewer. The viewer renders a trace's tags and attributes as
+HTML in its own origin, so serving it from Upright's hostname would let a trace
+run script against an admin session.
+
+A trace artifact links to `config.trace_viewer_url`, which defaults to
+upstream's hosted viewer at `https://trace.playwright.dev`. That viewer runs
+entirely in the browser, and being on its own registrable domain is what keeps a
+trace's contents off Upright's origin.
+
+Following the link hands that origin a URL it can read for 24 hours. Set
+`config.trace_viewer_url` to a viewer you host to keep traces to yourself, or
+assign `nil` to keep them download-only:
+
+```bash
+npx playwright show-trace trace.zip
+```
+
+A URL under `config.hostname` raises `Upright::ConfigurationError`, since that
+would put the trace back on the admin origin.
+
+That trace URL is served by `Upright::TracesController`, not Active Storage. The
+viewer fetches it from its own origin, so the request arrives without the admin
+session and needs `Access-Control-Allow-Origin` to be readable at all. The
+controller sends that header for the configured viewer's origin and nothing
+else, and takes a signed id in place of the session: scoped to a purpose,
+expiring after 24 hours, and resolvable only to a blob attached to an
+`Upright::ProbeResult` under a `.zip` filename. With no viewer configured the
+route 404s, so it exists only where it's used.
+
+Range requests are answered, which is how the viewer reads a ZIP's central
+directory without downloading the whole archive.
+
 ### Running Tests
 
 ```bash

data/Rakefile

@@ -4,3 +4,25 @@ APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
 load "rails/tasks/engine.rake"
 
 require "bundler/gem_tasks"
+
+Dir.glob("lib/tasks/**/*.rake").each { |r| load r }
+
+desc "Tag v<current version> and push it, which starts the Release workflow"
+task :tag do
+  require_relative "lib/upright/version"
+  tag = "v#{Upright::VERSION}"
+
+  abort "tag: the working tree has uncommitted changes" unless `git status --porcelain`.empty?
+
+  branch = `git rev-parse --abbrev-ref HEAD`.strip
+  abort "tag: run this on main (currently on #{branch})" unless branch == "main"
+
+  sh "git fetch origin main --quiet"
+  abort "tag: HEAD differs from origin/main; pull or push first" unless `git rev-parse HEAD`.strip == `git rev-parse origin/main`.strip
+  abort "tag: #{tag} already exists locally" unless `git tag --list #{tag}`.strip.empty?
+  abort "tag: #{tag} already exists on origin" unless `git ls-remote --tags origin refs/tags/#{tag}`.strip.empty?
+
+  sh "git tag --annotate #{tag} --message 'upright #{tag}'"
+  sh "git push origin #{tag}"
+  puts "Pushed #{tag}. Approve the release-rubygems environment when the Release workflow pauses."
+end

data/app/assets/stylesheets/upright/_global.css

@@ -35,6 +35,7 @@
 
   --lch-green-dark: 55% 0.162 147;
   --lch-red-dark: 59% 0.19 38;
+  --lch-orange-dark: 70% 0.17 65;
 
   /* Colors: Semantic */
   --color-black: oklch(var(--lch-black));
@@ -52,6 +53,7 @@
   --color-link: oklch(var(--lch-blue-dark));
   --color-positive: oklch(var(--lch-green-dark));
   --color-negative: oklch(var(--lch-red-dark));
+  --color-warning: oklch(var(--lch-orange-dark));
   --color-selected: oklch(var(--lch-blue-lighter));
 
   /* Borders & Shadows */

data/app/assets/stylesheets/upright/artifact.css

@@ -22,17 +22,11 @@
 
   .artifact__content {
     background: var(--color-ink-lightest);
-    border-radius: 0.25rem;
-    box-shadow: var(--shadow);
     display: flex;
     flex-direction: column;
-    height: 80vh;
-    left: 50%;
+    inset: 0;
     overflow: hidden;
     position: fixed;
-    top: 50%;
-    transform: translate(-50%, -50%);
-    width: 80vw;
     z-index: 1000;
   }
 
@@ -123,26 +117,14 @@
     }
   }
 
-  td .artifact + .artifact {
-    margin-left: calc(var(--inline-space) / 2);
+  .artifact__trace {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5em;
   }
 
-  @media (max-width: 140ch) {
-    .artifact__content {
-      width: calc(100vw - var(--block-space) * 2);
-      height: calc(100vh - var(--block-space) * 2);
-      left: 50%;
-      top: 50%;
-    }
+  td .artifact + .artifact {
+    margin-left: calc(var(--inline-space) / 2);
   }
 
-  @media (max-width: 55ch) {
-    .artifact__content {
-      inset: 0;
-      width: 100%;
-      height: 100%;
-      transform: none;
-      border-radius: 0;
-    }
-  }
 }

data/app/assets/stylesheets/upright/flash.css

@@ -0,0 +1,19 @@
+@layer components {
+  .flash {
+    border-radius: 0.5rem;
+    color: var(--color-ink);
+    font-size: var(--text-small);
+    margin-block-end: var(--block-space);
+    padding: 0.75rem 1rem;
+  }
+
+  .flash--notice {
+    background: color-mix(in oklab, var(--color-positive) 12%, var(--color-canvas));
+    border: 1px solid color-mix(in oklab, var(--color-positive) 40%, var(--color-canvas));
+  }
+
+  .flash--alert {
+    background: color-mix(in oklab, var(--color-negative) 12%, var(--color-canvas));
+    border: 1px solid color-mix(in oklab, var(--color-negative) 40%, var(--color-canvas));
+  }
+}

data/app/assets/stylesheets/upright/header.css

@@ -38,6 +38,15 @@
     text-transform: uppercase;
   }
 
+  .header__env {
+    color: var(--color-warning);
+    font-size: var(--text-x-small);
+    font-weight: 500;
+    letter-spacing: 0.06em;
+    margin-right: calc(var(--inline-space) * 2);
+    text-transform: uppercase;
+  }
+
   .header__nav {
     align-items: center;
     display: flex;
@@ -62,6 +71,11 @@
     &.active {
       color: var(--color-positive);
     }
+
+    &.unavailable {
+      color: var(--color-ink-light);
+      cursor: help;
+    }
   }
 
   .header__divider {