rtail 0.2.1 → 2.0.0-rc.2

package/README.md

@@ -1,26 +1,197 @@
 # `rtail(1)`
 
-[![Wercker CI](https://img.shields.io/wercker/ci/556547b7be632a8c751c857d.svg?style=flat-square)](https://app.wercker.com/project/bykey/54b073dac5b9156509c26031c78c98d4)
-[![Coveralls](https://img.shields.io/coveralls/kilianc/rtail.svg?style=flat-square)](https://coveralls.io/r/kilianc/rtail)
+[![CI](https://github.com/kilianc/rtail/actions/workflows/ci.yml/badge.svg)](https://github.com/kilianc/rtail/actions/workflows/ci.yml)
 [![NPM version](https://img.shields.io/npm/v/rtail.svg?style=flat-square)](https://www.npmjs.com/package/rtail)
 [![NPM downloads](https://img.shields.io/npm/dm/rtail.svg?style=flat-square)](https://www.npmjs.com/package/rtail)
 [![GitHub Stars](https://img.shields.io/github/stars/kilianc/rtail.svg?style=flat-square)](https://github.com/kilianc/rtail)
-[![License](https://img.shields.io/npm/l/rtail.svg?style=flat-square)](https://www.npmjs.com/package/rtail)
+[![License](https://img.shields.io/badge/license-Apache--2.0-04cd7e.svg?style=flat-square)](LICENSE)
 [![Gitter](https://img.shields.io/badge/≡_gitter-join_chat_➝-04cd7e.svg?style=flat-square)](https://gitter.im/kilianc/rtail?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 ## Terminal output to the browser in seconds, using UNIX pipes.
 
+> **v2 — private preview.** This branch is the rewrite described in
+> [`docs/proposal-logging-system.md`](docs/proposal-logging-system.md): rTail is
+> growing into a durable, searchable log store backed by Parquet and SQL. The
+> server is now a single Go binary with the webapp embedded, and the npm
+> package carries only the client pipe.
+>
+> The `cmd | rtail` wire format is frozen and always will be — the client you
+> already have installed works against a v2 server unchanged. Everything else
+> on this branch is subject to change until 2.0.0 ships.
+>
+> **Where it is:** P0–P5 are done — storage, search, compaction, the explorer
+> and the ingest protocols. What is left before 2.0.0 is the operational edges:
+> authentication, TLS, and a release pipeline for the three cgo platforms.
+
 `rtail` is a command line utility that grabs every line in `stdin` and broadcasts it over **UDP**. That's it. Nothing fancy. Nothing complicated. Tail log files, app output, or whatever you wish, using `rtail` broadcasting to an `rtail-server` – See multiple streams in the browser, in realtime.
 
-## Installation
+## Running the server
+
+The server ships as a container image — this is the recommended way to run it.
+
+    $ docker run -d --name rtail -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail
+
+Open <http://localhost:8888> and start piping. There is a
+[`docker-compose.yml`](docker-compose.yml) if you prefer:
+
+    $ docker compose up -d
+
+Options are flags, or `RTAIL_*` environment variables:
+
+    $ docker run -d -p 8080:8080 -p 9999:9999/udp \
+        -e RTAIL_WEB_PORT=8080 ghcr.io/kilianc/rtail
+
+    $ docker run -d -p 8888:8888 -p 9999:9999/udp \
+        ghcr.io/kilianc/rtail --backlog 500
+
+The image binds `0.0.0.0` inside the container, so **the published ports are
+reachable from your whole network**. It has no authentication — keep it on a
+trusted network, or put a reverse proxy in front of it.
+
+## Releases
+
+Tagged releases carry a binary for each platform and a container image.
+
+    $ docker run -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail
+
+Or take the binary for your platform from the
+[releases page](https://github.com/kilianc/rtail/releases) — `linux_amd64`,
+`linux_arm64`, `darwin_amd64`, `darwin_arm64`, with a `checksums.txt` beside
+them. There is no Windows build: DuckDB is cgo and nobody has asked.
+
+The binary is not static, because DuckDB is cgo. Linux builds are linked in
+Debian bookworm, so they need glibc 2.36 or newer; the container image is the
+answer on anything older. Every binary is started and asked for `/healthz`
+before it is published, so a release that cannot run does not become one.
+
+## Installing the client
 
+The client is a UNIX pipe and belongs on the host whose output you are tailing,
+not in the container. It is a single Go binary, about 2.6 MB, with no runtime
+of any kind:
+
+    # already in the release tarball, next to rtail-server
+    $ tar -xzf rtail_2.0.0_darwin_arm64.tar.gz && sudo mv rtail /usr/local/bin
+
+    # or, if you have Go
+    $ go install github.com/kilianc/rtail/v2/cmd/rtail@latest
+
+    # or, if you are already standing in a project
     $ npm install -g rtail
 
-## Web app
+The npm route is a convenience, not a dependency: the package holds a short
+launcher and one optional dependency per platform, each carrying a prebuilt
+binary and declaring the `os` and `cpu` it is for. npm downloads the one that
+matches and skips the rest. There is no postinstall script and nothing is
+fetched at install time, so it works behind a proxy and under
+`--ignore-scripts`. The launcher needs Node 20 or newer; the other two routes
+need nothing.
+
+Windows is published to npm and not as a tarball — the client cross-compiles
+anywhere, the server does not.
+
+## The explorer
+
+![The rTail explorer](docs/media/explorer.png)
+
+*A spike of upstream timeouts, twenty minutes back: `level>=WARN` on the `api`
+stream, with `service`, `latency_ms` and `region` promoted to columns.*
+
+**One filter drives everything.** Type it and the live tail narrows — the
+server compiles the same rQL to a predicate over the ingest stream. Switch to
+history and the same query runs against Parquet. Same semantics either way,
+because there is one parser behind both.
+
+**The histogram is the navigation.** Counts over time stacked by severity; drag
+across it to zoom the range.
+
+**Most queries are built by clicking.** Expand any event and every value
+carries "filter to this" and "exclude this"; the command bar updates to match,
+so whatever you built by pointing is something you could have typed. The field
+sidebar shows the top values of every field *within the current results*, which
+answers "what is even in here" before you know what to search for.
+
+**Completion is real.** It comes from the catalog's key inventory — every field
+that has actually appeared, with its type and how often — not from a guess at
+what a log line might contain.
+
+**Every view is a link.** Stream, filter, time range, promoted columns, and
+whether the timeline is open — all of it lives in the URL, so the screenshot
+above is a URL somebody could send you.
+
+**Any field can become a column.** The ⊞ beside a field in the sidebar pulls it
+out of the summary into a column of its own, and the × on the header puts it
+back.
+
+**The query pane holds everything that acts on the query** — the language
+switch, the examples, the time range and Run — inside its own border, because
+they belong to the query rather than to the page.
 
-![](https://s3.amazonaws.com/rtail/github/dark.png)
+Keyboard: `/` or `⌘K` focuses the editor, `⌘⏎` runs it (plain `⏎` is a
+newline — it is an editor), `j`/`k` move between events, `⎋` collapses
+everything.
 
-![](https://s3.amazonaws.com/rtail/github/light.png)
+## Getting logs in
+
+Four ways, and every one of them lands in the same envelope — a record is
+indistinguishable once stored, whichever door it came through.
+
+**UNIX pipes**, the original, frozen forever:
+
+    $ node server.js 2>&1 | rtail --id api.example.com
+
+**HTTP**, for anything that would mind losing a line. UDP is at-most-once and
+capped at one datagram; this acknowledges, so a 200 means the records are in
+the write-ahead log and anything else is safe to retry:
+
+    $ curl -XPOST localhost:8888/v1/ingest?stream=api --data-binary @- <<'JSON'
+    {"level":"error","msg":"upstream timeout","service":"api","latency_ms":30004}
+    a plain line works too
+    JSON
+
+Newline-delimited JSON or plain text — the content type is not consulted,
+because the normalizer already decides per line what it is looking at. `gzip`
+and `zstd` bodies are decoded.
+
+**OpenTelemetry**, protobuf or JSON, at `/v1/logs`. Point any collector at it
+with a config line and no code change:
+
+    exporters:
+      otlphttp:
+        logs_endpoint: http://rtail:8888/v1/logs
+
+`service.name` becomes the stream, log and resource attributes become fields,
+and `trace_id`/`span_id` are promoted so correlation actually works.
+
+**Syslog**, RFC5424 and RFC3164, UDP and TCP on the same port:
+
+    $ rtail-server --data ./logs --syslog-port 5140
+
+    # rsyslog.conf
+    *.* @@rtail-host:5140;RSYSLOG_SyslogProtocol23Format
+
+APP-NAME becomes the stream, the priority becomes a real severity — so
+`level>=ERROR` works on syslog without anyone writing a parser — and RFC5424
+structured data is promoted to fields.
+
+Anything that can write NDJSON to an HTTP endpoint works without a plugin:
+
+    # vector.toml
+    [sinks.rtail]
+    type = "http"
+    inputs = ["my_source"]
+    uri = "http://rtail:8888/v1/ingest?stream=vector"
+    encoding.codec = "json"
+    framing.method = "newline_delimited"
+
+    # fluent-bit.conf
+    [OUTPUT]
+        Name   http
+        Match  *
+        Host   rtail
+        Port   8888
+        URI    /v1/ingest?stream=fluentbit
+        Format json_lines
 
 ## Rationale
 
@@ -33,9 +204,126 @@ There are many log aggregation tools out there, but few of them are realtime. **
 **The `rtail` approach is very simple:**
 * pipe something into `rtail` using [UNIX I/O redirection](http://www.westwind.com/reference/os-x/commandline/pipes.html) [[2]](http://www.codecoffee.com/tipsforlinux/articles2/042.html)
 * broadcast every line using UDP
-* `rtail-server`, **if listening**, will dispatch the stream into your browser, using [socket.io](http://socket.io/).
+* `rtail-server`, **if listening**, will dispatch the stream into your browser, using [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events).
+
+`rtail` is a realtime debugging and monitoring tool, which can display multiple aggregate streams via a modern web interface. **With `--data` it now keeps them too, and lets you search them with SQL** — see [Storage](#storage) and [Search](#search) below.
+
+## Storage
+
+    $ rtail-server --data ./logs
+
+Point `--data` at a directory and every line is written to Parquet. Without it,
+rTail is what it always was: an in-memory ring, nothing on disk, nothing to
+clean up.
+
+**There is no schema to define.** The envelope — timestamp, stream, level,
+message, host, sequence, and the original line — is always there. Everything
+else is whatever your logs happened to contain: each root-level JSON key
+becomes its own typed column, and nested objects are kept as JSON you can
+path into. Two files written a week apart with completely different keys
+reconcile by name at read time.
+
+**Your logs are just files.** The layout is plain and browsable:
+
+    logs/
+      catalog.sqlite
+      wal/
+      streams/api.example.com/2026/08/11/L0-…parquet
+
+which means any tool reads them, with rTail uninstalled and nothing exported:
+
+    $ duckdb -c "
+        SELECT a_service, count(*)
+        FROM read_parquet('logs/streams/**/*.parquet', union_by_name := true)
+        WHERE level = 'ERROR' AND a_latency_ms > 500
+        GROUP BY 1"
+
+That is the point, and it is covered by a test rather than merely claimed.
+
+**Durability.** Lines land in a write-ahead log before anything else, and are
+batched into Parquet from there. The WAL is fsynced every 100ms, so an
+unclean shutdown costs at most that; a clean one costs nothing. Recovery
+replays whatever the previous process did not finish, and is idempotent —
+crashing midway through a flush and restarting produces the same files, not
+duplicates.
+
+**Compaction.** A flush every thirty seconds per stream is thousands of tiny
+files a day, each with a footer to open. A background pass merges them:
 
-`rtail` is a realtime debugging and monitoring tool, which can display multiple aggregate streams via a modern web interface. **There is no persistent layer, nor does the tool store any data**. If you need a persistent layer, use something like [loggly](https://www.loggly.com/).
+| level | holds | sorted by |
+|---|---|---|
+| L0 | one flush | `seq` — the order it arrived in |
+| L1 | an hour | `ts` |
+| L2 | a day | `(level, ts)`, or `--cluster-by` |
+
+Sort order is the point, not the file count. Sorting by time makes range
+pruning exact at row-group granularity rather than merely likely, and a
+low-cardinality leading column turns `level>=ERROR` over a day into a
+row-group skip instead of a scan. Compaction is also where a messy population
+of schemas becomes one clean union — types that disagreed get widened,
+always-null columns disappear.
+
+In practice six flushes of a small stream merge from 24KB to 5KB.
+
+A merge either publishes its output and retires its inputs, or changes
+nothing. Retired files stay on disk for a grace period, so a query that
+started before the merge still has its files. `--compact-interval 0` turns the
+whole thing off.
+
+**Retention** is per-server, and all of it happens during compaction:
+
+    $ rtail-server --data ./logs \
+        --retention 30d \
+        --retention-raw 7d \
+        --downsample-after 14d --downsample-level WARN
+
+`--retention` deletes whole files once their newest record is past the
+horizon; a file straddling the boundary survives until all of it has expired,
+because rewriting a file to drop a few rows costs far more than keeping them.
+`--retention-raw` drops the original line while keeping every promoted column,
+which roughly halves the footprint — the cost is that keys which were never
+promoted stop being queryable, since `raw` was their fallback.
+`--downsample-after` discards anything below `--downsample-level` once it is
+old enough.
+
+Durations take `d` and `w` as well as Go's own units, because `--retention
+720h` is nobody's idea of clear.
+
+## Search
+
+Two ways in, sharing one set of semantics.
+
+**The filter bar** — terse, and what most searching looks like:
+
+    level>=ERROR service=api "connection timeout"
+    -path:/health AND (region=us-east-1 OR region=us-west-2)
+    latency_ms>500 req.path:~"^/v1/"
+    trace_id=*
+
+`k=v` is equality, `k:v` a case-insensitive substring, `k:~re` a regular
+expression, `k>n` a comparison, `k=*` "the key exists", `k=null` "it does not".
+A bare word is full-text across the message and the original line. Terms are
+ANDed unless you write `OR`; `-` and `NOT` negate; parentheses group.
+
+`level` compares by **severity, not alphabetically** — `level>=ERROR` matches
+FATAL too, which is what you meant. Quoting a value suppresses type coercion,
+so `status=200` is the number and `status="200"` is the string.
+
+**Raw SQL**, for everything else:
+
+    curl -XPOST localhost:8888/v1/sql -d '{
+      "sql": "SELECT a_service, count(*) n, quantile_cont(a_latency_ms, 0.95) p95
+              FROM logs WHERE level = '"'"'ERROR'"'"' GROUP BY 1 ORDER BY n DESC",
+      "from": "-6h"
+    }'
+
+`logs` is a view over the pruned files. The query runs read-only in a DuckDB
+sandbox that can see nothing but your data directory.
+
+**The time range is the partition pruner**, not decoration. It is always
+applied — a query without one gets the last 24 hours injected — and every
+response reports the window it used along with how many files and rows the
+query implied reading, so a slow search can explain itself.
 
 ## Examples
 
@@ -70,11 +358,10 @@ For fun and debugging:
 
     Options:
       --host, -h     The server host                 [string] [default: "127.0.0.1"]
-      --port, -p     The server port                        [string] [default: 9999]
+      --port, -p     The server port                        [number] [default: 9999]
       --id, --name   The log stream id                 [string] [default: (moniker)]
       --mute, -m     Don't pipe stdin with stdout                          [boolean]
       --tty          Keeps ansi colors                     [boolean] [default: true]
-      --parse-date   Looks for dates to use as timestamp   [boolean] [default: true]
       --help         Show help                                             [boolean]
       --version, -v  Show version number                                   [boolean]
 
@@ -85,12 +372,10 @@ For fun and debugging:
       server | rtail --port 43567         Uses custom port
       server | rtail --mute               No stdout
       server | rtail --no-tty             Strips ansi colors
-      server | rtail --no-date-parse      Disable date parsing/stripping
-
 
 ## `rtail-server(1)`
 
-`rtail-server` receives all messages broadcast from every `rtail` client, displaying all incoming log streams in a realtime web view. **Under the hood, the server uses [socket.io](http://socket.io) to pipe every incoming UDP message to the browser.**
+`rtail-server` receives all messages broadcast from every `rtail` client, displaying all incoming log streams in a realtime web view. **Under the hood it is a single static Go binary with the webapp embedded, pushing every incoming UDP message to the browser over server-sent events.**
 
 There is little to no configuration – The default UDP/HTTP ports can be changed, but that's it.
 
@@ -100,17 +385,13 @@ Use default values:
 
     $ rtail-server
 
-Always use latest, stable webapp:
-
-    $ rtail-server --web-version stable
-
 Use custom ports:
 
     $ rtail-server --web-port 8080 --udp-port 9090
 
 Set debugging on:
 
-    $ DEBUG=rtail:* rtail-server
+    $ rtail-server --verbose
 
 Open your browser and start tailing logs!
 
@@ -119,29 +400,176 @@ Open your browser and start tailing logs!
     $ rtail-server --help
     Usage: rtail-server [OPTIONS]
 
-    Options:
-    --udp-host, --uh  The listening UDP hostname            [default: "127.0.0.1"]
-    --udp-port, --up  The listening UDP port                       [default: 9999]
-    --web-host, --wh  The listening HTTP hostname           [default: "127.0.0.1"]
-    --web-port, --wp  The listening HTTP port                      [default: 8888]
-    --web-version     Define web app version to serve                     [string]
-    --help, -h        Show help                                          [boolean]
-    --version, -v     Show version number                                [boolean]
-
-    Examples:
-    rtail-server --web-port 8080         Use custom HTTP port
-    rtail-server --udp-port 8080         Use custom UDP port
-    rtail-server --web-version stable    Always uses latest stable webapp
-    rtail-server --web-version unstable  Always uses latest develop webapp
-    rtail-server --web-version 0.1.3     Use webapp v0.1.3
+    Every option is also settable as RTAIL_*, e.g. RTAIL_WEB_PORT.
+
+      -udp-host string   the listening UDP hostname     (default "127.0.0.1")
+      -udp-port int      the listening UDP port         (default 9999)
+      -web-host string   the listening HTTP hostname    (default "127.0.0.1")
+      -web-port int      the listening HTTP port        (default 8888)
+      -backlog int       lines of history kept per stream (default 100)
+      -data string       directory for durable storage  (unset: memory only)
+      -web-root string   serve the webapp from this directory instead of the binary
+      -auth-token string require this bearer token on the API and webapp
+      -tls-cert string   serve HTTPS with this certificate (PEM)
+      -tls-key string    the private key for --tls-cert (PEM)
+      -insecure          allow a non-loopback address with no --auth-token
+      -verbose           log at debug level
+      -version           print the version and exit
+
+The v1 short aliases still work: `--uh`, `--up`, `--wh`, `--wp`, `-b`.
+
+Every option can also be set as an environment variable, prefixed with
+`RTAIL_` — `RTAIL_WEB_PORT=8080`, `RTAIL_BACKLOG=500`, and so on. This is how
+the container image is configured.
+
+Note that `--backlog` is about the *live view* — how many recent lines a
+freshly opened browser tab is handed. It has nothing to do with how much is
+kept on disk, which is everything until retention exists.
+
+## API
+
+The webapp is a client of a small documented API, so anything else can be too:
+
+| endpoint | what it does |
+|---|---|
+| `GET /v1/streams` | every known stream |
+| `GET /v1/tail?stream=<name>` | SSE feed: `streams`, `backlog`, then `line` events. Omit `stream` to watch the stream list only |
+| `GET /v1/search` | matching records, keyset-paginated |
+| `GET /v1/histogram` | match counts over time, split by severity |
+| `GET /v1/fields?field=<name>` | the most common values of a field, within the current filter |
+| `GET /v1/schema` | every key ever seen, with its type — what autocomplete completes against |
+| `POST /v1/sql` | read-only SQL over a `logs` view |
+| `POST /v1/ingest` | NDJSON or plain lines, optionally gzip/zstd |
+| `POST /v1/logs` | OTLP logs, protobuf or JSON |
+| `GET /healthz` | version, stream count, per-receiver ingest counters |
+
+The search endpoints share their parameters: `q` (filter), `stream`, `from` and
+`to` (RFC3339, epoch millis, or a relative `-6h`), `limit`, `cursor`, `order`.
+They return 503 with an explanation when the server is running without
+`--data`, since there is nothing to search.
 
 ## UDP Broadcasting
 
 To scale and broadcast on multiple servers, instruct the `rtail` client to stream to the broadcast address. Every message will then be delivered to all servers in your subnet.
 
-## Authentication layer
+## Authentication
+
+One shared token, or nothing.
+
+    $ rtail-server --auth-token "$(openssl rand -hex 32)" --web-host 0.0.0.0
+
+With a token set, every route except `/healthz` requires it. Programs send it
+as a header; browsers sign in once and get a cookie.
+
+    $ curl -H "Authorization: Bearer $TOKEN" http://logs.example.com/v1/streams
+
+`X-Rtail-Token` works too, for clients that reserve `Authorization`. In a
+browser, opening the server without one gives you a sign-in form, and
+`https://logs.example.com/?token=…` signs you in from a link — the token is
+spent immediately and redirected out of the address bar so it does not end up
+in your history.
+
+`/healthz` is deliberately left open, so a container healthcheck or a load
+balancer keeps working without being handed the secret. It reports liveness
+and stream names; everything that could show a log line is behind the token.
+
+**Running with no token is refused on any address but loopback.** This is the
+one rule the server enforces on your behalf:
 
-For the time being, the webapp doesn't have an authentication layer; it assumes that you will run it behind a VPN or reverse proxy, with a simple `Authorization` header check.
+    $ rtail-server --web-host 0.0.0.0
+    rtail-server: refusing to listen on 0.0.0.0 with no --auth-token:
+    anyone who can reach it could read every log.
+
+Pass `--insecure` if something in front of it handles access. Nothing stops
+you; it just will not happen by accident.
+
+What this deliberately is not: accounts, roles, OIDC, SAML, or per-stream
+permissions. rTail is a single-node tool, and anyone who needs those already
+has a reverse proxy that does them properly.
+
+**Two holes worth knowing about.** UDP ingest is unauthenticated, because UDP
+has no handshake to carry a token and v1's wire format is frozen. Syslog is
+unauthenticated for the same reason. Both listen on `127.0.0.1` by default; if
+you move either off loopback you are trusting your network.
+
+### TLS
+
+    $ rtail-server --tls-cert cert.pem --tls-key key.pem
+
+Both or neither — half a pair is refused at startup rather than failing on
+somebody's first request. There is no ACME client and there will not be one:
+a certificate is the one thing every reverse proxy already solves.
+
+# Running it locally
+
+Note there are two images, and they are not the same thing:
+[`Dockerfile`](Dockerfile) is the server you deploy, and
+[`tools/Dockerfile`](tools/Dockerfile) below is the development toolchain,
+which mounts your checkout.
+
+The toolchain lives in a container ([`tools/Dockerfile`](tools/Dockerfile)), so Docker is
+the only thing you need installed — no Node.js, no Go, no npm, no global CLIs.
+v2 needs both toolchains: Node builds the webapp, Go builds the server that
+embeds it, and the image carries pinned versions of each.
+
+    $ make dev
+
+That builds the assets and the server binary, starts `rtail-server`, feeds it
+three live demo streams, and serves the webapp. It prints the URL — `make url`
+prints it again. Stylesheets and the bundle recompile on save; reload the
+browser to pick them up. Go changes need a `make dev` restart, because the
+binary is rebuilt at startup. `Ctrl-C` stops everything.
+
+The first run also builds the toolchain image and installs dependencies, so it
+takes a minute; subsequent runs start immediately.
+
+To leave it running in the background instead:
+
+    $ make up         # start detached, wait for it, print the URL
+    $ make logs       # tail it
+    $ make down       # stop it
+
+Other targets:
+
+    $ make server     # build bin/rtail-server
+    $ make release    # build the self-contained server, webapp embedded
+    $ make build      # build the webapp into app/ (development)
+    $ make dist       # build the minified webapp into web/dist
+    $ make test       # run the Go and Node test suites
+    $ make vet        # go vet
+    $ make fmt        # gofmt the Go sources
+    $ make typecheck  # type-check the webapp
+    $ make shell      # open a shell inside the toolchain container
+    $ make clean      # remove generated assets, binaries and dependencies
+
+### Ports
+
+Ports are derived from the worktree path, so several checkouts of this repo can
+run at once without colliding. Each gets its own HTTP port, UDP port, and
+container name, fixed across runs — sharing any of the three fails confusingly,
+since two servers can both bind the same UDP port and the loser simply never
+receives a log line.
+
+`make url` reports the current one. Override with `make dev PORT=9000
+UDP_PORT=9001`.
+
+If you do have a Node.js toolchain on your machine, the same targets are plain
+npm scripts — `npm install && npm run dev`. Node builds the webapp and is not
+in the product: the server embeds the bundle, and the client is a Go binary.
+
+### The stack
+
+| | |
+| --- | --- |
+| Client | Go, one static binary, [json5](https://github.com/titanous/json5) its only dependency |
+| Server | Go + [DuckDB](https://duckdb.org) + [parquet-go](https://github.com/parquet-go/parquet-go) |
+| Webapp | [Preact](https://preactjs.com) + TypeScript, ~33 KB gzipped |
+| Build | [esbuild](https://esbuild.github.io); the stylesheets are plain CSS |
+| Tests | `go test ./...` |
+
+The webapp has no framework runtime beyond Preact: routing, preferences,
+popovers, and timestamp formatting are a few dozen lines each over the
+platform (`history`, `localStorage`, `Intl`) rather than dependencies.
 
 # How to contribute
 
@@ -150,11 +578,17 @@ This project follows the awesome [Vincent Driessen](http://nvie.com/about/) [bra
 * You must add a new feature on its own branch
 * You must contribute to hot-fixing, directly into the master branch (and pull-request to it)
 
-This project uses JSCS to enforce a consistent code style. Your contribution must be pass jscs validation.
+The test suite is `make test`, which is `go test -race ./...` inside the
+toolchain image. Use it to check whether your contribution breaks some part of
+the library, and be sure to add new tests for each new feature.
+
+    $ make test
+
+The webapp is TypeScript; please keep it type-clean:
 
-The test suite is written on top of [mochajs/mocha](http://mochajs.org/). Use the tests to check if your contribution breaks some part of the library and be sure to add new tests for each new feature.
+    $ make typecheck
 
-    $ npm test
+CI runs both, plus the production build, on Node 20 and 22.
 
 ## Contributors
 
@@ -166,48 +600,49 @@ The test suite is written on top of [mochajs/mocha](http://mochajs.org/). Use th
 
 ## Roadmap (aka where you can help)
 
-* Write a rock solid test suite
-* Allow use of DTLS (waiting for node to support this https://github.com/joyent/node/pull/6704)
-* Add GitHub OAuth and basic auth for teams (join proposal convo here: https://github.com/kilianc/rtail/issues/44)
-* Implement infinite-scroll like behavior in the webapp to support bigger backlogs and make it future proof.
-* Publish base rtail docker image to DockerHub
-* Create a catch all docker logs image
-* Rewrite webapp using ng2
+* Optional HTTP basic auth on the web port, and a shared secret on UDP ingest.
+  Full OAuth ([#44](https://github.com/kilianc/rtail/issues/44)) needs a user
+  model and session storage, which is a lot of machinery for a tool with no
+  persistence layer — basic auth covers the actual risk.
+* A catch-all docker logs image: watch `docker events`, pipe every container's
+  `docker logs -f` into a stream named after it.
+* Broaden the test suite, particularly around the UDP ingest path.
+
+Dropped, and why:
+
+* ~~Rewrite webapp using ng2~~ — done differently. The webapp is now Preact +
+  TypeScript; Angular 2 was already obsolete by the time it came up.
+* ~~Publish base rtail docker image to DockerHub~~ — done, but to GHCR rather
+  than DockerHub, since it needs no separate account or secrets.
+* ~~Write a rock solid test suite~~ — a suite exists and runs in CI. Deepening
+  it is the open item above.
+* ~~Allow use of DTLS~~ — the linked PR was against `joyent/node`, a repo that
+  no longer exists, and Node core still has no DTLS. Run rtail over WireGuard
+  or Tailscale instead; that is the modern answer to this problem.
+* ~~Infinite-scroll in the webapp~~ — cannot be built as stated. The server
+  keeps a fixed-size ring buffer per stream and sends it in one message; there
+  is no pagination to scroll through. Real history means a persistence layer,
+  which contradicts the design. `--backlog` covers the useful part.
 
 ## Sponsors
 ❤ rTail? Consider sponsoring this project to keep it alive and free for the community.
 
-* Lukibear (domain)
 * ? (wildcard TLS cert)
 * ? (.io domain)
 
 [![PayPal donate button](https://img.shields.io/badge/$_paypal-one_time_donation_➝-04cd7e.svg?style=flat-square)](https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=info%40rtail%2eorg&lc=US&item_name=rtail&item_number=rtail&currency_code=USD&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted)
 
-<a href="mailto:info@lukibear.com">Professional support or ad-hoc is also available.</a>
-
 ## License
 
-_This software is released under the MIT license cited below_.
-
-    Copyright (c) 2014 Kilian Ciuffolo, me@nailik.org. All Rights Reserved.
-
-    Permission is hereby granted, free of charge, to any person
-    obtaining a copy of this software and associated documentation
-    files (the 'Software'), to deal in the Software without
-    restriction, including without limitation the rights to use,
-    copy, modify, merge, publish, distribute, sublicense, and/or sell
-    copies of the Software, and to permit persons to whom the
-    Software is furnished to do so, subject to the following
-    conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
-    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
-    OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-    NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
-    HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
-    WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
-    OTHER DEALINGS IN THE SOFTWARE.
+rTail is licensed under the **Apache License 2.0** — see [`LICENSE`](LICENSE)
+and [`COPYRIGHT.md`](COPYRIGHT.md).
+
+Run it, modify it, ship it inside your own product, commercially, at any
+scale. Keep the notice, say what you changed, and the patent grant in §3
+covers you. There is nothing to buy and no feature held back for a paid tier,
+because there is no paid tier.
+
+rTail **v1.x remains under the MIT licence** it was released with. That does
+not change retroactively; if you are on v1, the terms you accepted still
+apply. v2 spent part of its life under AGPL-3.0; that is reversed, and
+[`COPYRIGHT.md`](COPYRIGHT.md) has the reasoning.