rtail 2.0.0-rc.2 → 2.0.0-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NOTICE +9 -0
- package/README.md +210 -94
- package/bin/rtail.js +7 -1
- package/package.json +7 -7
package/NOTICE
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
rTail
|
|
2
|
+
Copyright 2013-2026 Kilian Ciuffolo and the rTail contributors
|
|
3
|
+
|
|
4
|
+
This product includes software developed at the rTail project
|
|
5
|
+
(https://github.com/kilianc/rtail).
|
|
6
|
+
|
|
7
|
+
Portions of this software, specifically releases v1.x and the history on the
|
|
8
|
+
main branch preceding the v2 branch, were originally released under the MIT
|
|
9
|
+
licence and remain available under those terms.
|
package/README.md
CHANGED
|
@@ -9,19 +9,27 @@
|
|
|
9
9
|
|
|
10
10
|
## Terminal output to the browser in seconds, using UNIX pipes.
|
|
11
11
|
|
|
12
|
-
>
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
>
|
|
16
|
-
|
|
12
|
+
<picture>
|
|
13
|
+
<source media="(prefers-color-scheme: dark)" srcset="docs/media/explorer-dark.png">
|
|
14
|
+
<img alt="The rtail explorer: level>=WARN on the api stream over the last hour, a wall of upstream timeouts from payments at the top" src="docs/media/explorer-light.png">
|
|
15
|
+
</picture>
|
|
16
|
+
|
|
17
|
+
*`level>=WARN` on the `api` stream over the last hour: a run of upstream
|
|
18
|
+
timeouts from payments in the last few minutes, with `service`, `status`,
|
|
19
|
+
`latency_ms` and `region` promoted to columns.*
|
|
20
|
+
|
|
21
|
+
> **v2 — release candidate.** v2 is a rewrite: a durable, searchable log store
|
|
22
|
+
> backed by Parquet and SQL. The server is a single Go binary with the webapp
|
|
23
|
+
> embedded, and the npm package carries only the client pipe.
|
|
17
24
|
>
|
|
18
25
|
> The `cmd | rtail` wire format is frozen and always will be — the client you
|
|
19
|
-
> already have installed works against a v2 server unchanged.
|
|
20
|
-
> on this branch is subject to change until 2.0.0 ships.
|
|
26
|
+
> already have installed works against a v2 server unchanged.
|
|
21
27
|
>
|
|
22
|
-
> **Where it is:**
|
|
23
|
-
>
|
|
24
|
-
>
|
|
28
|
+
> **Where it is:** everything described below is built and tested — storage,
|
|
29
|
+
> search, compaction, the explorer, the four ingest protocols, authentication,
|
|
30
|
+
> TLS, and a release pipeline that publishes four tarballs, six npm packages
|
|
31
|
+
> and a container image per tag. `2.0.0-rc.3` is the current release; until
|
|
32
|
+
> `2.0.0` is tagged, reaching it takes `RTAIL_PRERELEASE=1` — see below.
|
|
25
33
|
|
|
26
34
|
`rtail` is a command line utility that grabs every line in `stdin` and broadcasts it over **UDP**. That's it. Nothing fancy. Nothing complicated. Tail log files, app output, or whatever you wish, using `rtail` broadcasting to an `rtail-server` – See multiple streams in the browser, in realtime.
|
|
27
35
|
|
|
@@ -29,7 +37,12 @@
|
|
|
29
37
|
|
|
30
38
|
The server ships as a container image — this is the recommended way to run it.
|
|
31
39
|
|
|
32
|
-
$ docker run -d --name rtail -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail
|
|
40
|
+
$ docker run -d --name rtail -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail:next
|
|
41
|
+
|
|
42
|
+
`:next` is the newest release candidate, the way `rtail@next` is on npm.
|
|
43
|
+
`latest` only moves on a stable release, so until `2.0.0` is tagged it is
|
|
44
|
+
stuck on an earlier candidate that predates `/data`; once it is, the plain
|
|
45
|
+
`ghcr.io/kilianc/rtail` is right again.
|
|
33
46
|
|
|
34
47
|
Open <http://localhost:8888> and start piping. There is a
|
|
35
48
|
[`docker-compose.yml`](docker-compose.yml) if you prefer:
|
|
@@ -39,20 +52,33 @@ Open <http://localhost:8888> and start piping. There is a
|
|
|
39
52
|
Options are flags, or `RTAIL_*` environment variables:
|
|
40
53
|
|
|
41
54
|
$ docker run -d -p 8080:8080 -p 9999:9999/udp \
|
|
42
|
-
-e RTAIL_WEB_PORT=8080 ghcr.io/kilianc/rtail
|
|
55
|
+
-e RTAIL_WEB_PORT=8080 ghcr.io/kilianc/rtail:next
|
|
56
|
+
|
|
57
|
+
$ docker run -d -p 8888:8888 -p 9999:9999/udp \
|
|
58
|
+
ghcr.io/kilianc/rtail:next --backlog 500
|
|
59
|
+
|
|
60
|
+
The image is memory-only until you give it somewhere to write. `/data` is
|
|
61
|
+
there for that, owned by the user the server runs as:
|
|
43
62
|
|
|
44
63
|
$ docker run -d -p 8888:8888 -p 9999:9999/udp \
|
|
45
|
-
ghcr.io/kilianc/rtail
|
|
64
|
+
-e RTAIL_DATA=/data -v rtail-data:/data ghcr.io/kilianc/rtail:next
|
|
65
|
+
|
|
66
|
+
A named volume, as above, takes that owner on its own. A bind mount keeps the
|
|
67
|
+
host directory's, which the server's uid 10001 cannot write to:
|
|
68
|
+
`chown 10001:10001` the directory first, or run it as yourself with
|
|
69
|
+
`--user "$(id -u):$(id -g)"`.
|
|
46
70
|
|
|
47
71
|
The image binds `0.0.0.0` inside the container, so **the published ports are
|
|
48
|
-
reachable from your whole network**. It
|
|
49
|
-
|
|
72
|
+
reachable from your whole network**. It is open by default;
|
|
73
|
+
`-e RTAIL_AUTH_TOKEN="$(openssl rand -hex 32)"` closes it — see
|
|
74
|
+
[Authentication](#authentication). Without a token, keep it on a trusted
|
|
75
|
+
network or put a reverse proxy in front of it.
|
|
50
76
|
|
|
51
77
|
## Releases
|
|
52
78
|
|
|
53
79
|
Tagged releases carry a binary for each platform and a container image.
|
|
54
80
|
|
|
55
|
-
$ docker run -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail
|
|
81
|
+
$ docker run -p 8888:8888 -p 9999:9999/udp ghcr.io/kilianc/rtail:next
|
|
56
82
|
|
|
57
83
|
Or take the binary for your platform from the
|
|
58
84
|
[releases page](https://github.com/kilianc/rtail/releases) — `linux_amd64`,
|
|
@@ -70,14 +96,40 @@ The client is a UNIX pipe and belongs on the host whose output you are tailing,
|
|
|
70
96
|
not in the container. It is a single Go binary, about 2.6 MB, with no runtime
|
|
71
97
|
of any kind:
|
|
72
98
|
|
|
73
|
-
#
|
|
74
|
-
$
|
|
99
|
+
# the short way
|
|
100
|
+
$ curl -fsSL https://raw.githubusercontent.com/kilianc/rtail/main/install.sh | sh
|
|
101
|
+
|
|
102
|
+
# or take the tarball yourself — it is what the line above downloads
|
|
103
|
+
$ tar -xzf rtail_2.0.0-rc.3_darwin_arm64.tar.gz
|
|
104
|
+
$ sudo mv rtail_2.0.0-rc.3_darwin_arm64/rtail /usr/local/bin
|
|
75
105
|
|
|
76
106
|
# or, if you have Go
|
|
77
107
|
$ go install github.com/kilianc/rtail/v2/cmd/rtail@latest
|
|
78
108
|
|
|
79
109
|
# or, if you are already standing in a project
|
|
80
|
-
$ npm install -g rtail
|
|
110
|
+
$ npm install -g rtail@next
|
|
111
|
+
|
|
112
|
+
**While 2.0 is a release candidate, two of those four need telling.** GitHub's
|
|
113
|
+
"latest release" skips prereleases, so the plain `install.sh` stops and asks
|
|
114
|
+
for `RTAIL_PRERELEASE=1`; and npm's `latest` tag still points at `0.2.1` — the
|
|
115
|
+
v1 Node client — so a plain `npm install -g rtail` quietly hands you that:
|
|
116
|
+
|
|
117
|
+
$ curl -fsSL https://raw.githubusercontent.com/kilianc/rtail/main/install.sh | RTAIL_PRERELEASE=1 sh
|
|
118
|
+
$ npm install -g rtail@next
|
|
119
|
+
|
|
120
|
+
`go install …@latest` needs nothing: there is no `v2` release tag yet, so Go
|
|
121
|
+
resolves it to the newest prerelease. Taking the tarball by name needs nothing
|
|
122
|
+
either. When `2.0.0` is tagged, all four plain forms are correct.
|
|
123
|
+
|
|
124
|
+
[`install.sh`](install.sh) detects your platform, checks the download against
|
|
125
|
+
the release's own `checksums.txt` and refuses to continue on a mismatch,
|
|
126
|
+
installs one binary into `/usr/local/bin` — or `~/.local/bin` if that is not
|
|
127
|
+
writable, because a script piped from the internet should not be the thing
|
|
128
|
+
that asks for your password — and touches nothing else. It never calls `sudo`
|
|
129
|
+
and never edits your shell profile. `RTAIL_VERSION`, `RTAIL_INSTALL_DIR` and
|
|
130
|
+
`RTAIL_SERVER=1` override the obvious things. It is a hundred lines of POSIX
|
|
131
|
+
sh; `curl -fsSL https://raw.githubusercontent.com/kilianc/rtail/main/install.sh | less` reads the whole of it,
|
|
132
|
+
and doing that before you pipe anything into `sh` is a good habit generally.
|
|
81
133
|
|
|
82
134
|
The npm route is a convenience, not a dependency: the package holds a short
|
|
83
135
|
launcher and one optional dependency per platform, each carrying a prebuilt
|
|
@@ -92,11 +144,6 @@ anywhere, the server does not.
|
|
|
92
144
|
|
|
93
145
|
## The explorer
|
|
94
146
|
|
|
95
|
-

|
|
96
|
-
|
|
97
|
-
*A spike of upstream timeouts, twenty minutes back: `level>=WARN` on the `api`
|
|
98
|
-
stream, with `service`, `latency_ms` and `region` promoted to columns.*
|
|
99
|
-
|
|
100
147
|
**One filter drives everything.** Type it and the live tail narrows — the
|
|
101
148
|
server compiles the same rQL to a predicate over the ingest stream. Switch to
|
|
102
149
|
history and the same query runs against Parquet. Same semantics either way,
|
|
@@ -141,14 +188,20 @@ indistinguishable once stored, whichever door it came through.
|
|
|
141
188
|
$ node server.js 2>&1 | rtail --id api.example.com
|
|
142
189
|
|
|
143
190
|
**HTTP**, for anything that would mind losing a line. UDP is at-most-once and
|
|
144
|
-
capped at one datagram; this acknowledges
|
|
145
|
-
the write-ahead log
|
|
191
|
+
capped at one datagram; this acknowledges. With `--data`, a 200 means every
|
|
192
|
+
line has been fsynced to the write-ahead log. Without it the server keeps only
|
|
193
|
+
its in-memory ring, and a 200 means the lines were received, not stored:
|
|
146
194
|
|
|
147
|
-
$ curl -XPOST localhost:8888/v1/ingest?stream=api --data-binary @- <<'JSON'
|
|
195
|
+
$ curl -XPOST 'localhost:8888/v1/ingest?stream=api' --data-binary @- <<'JSON'
|
|
148
196
|
{"level":"error","msg":"upstream timeout","service":"api","latency_ms":30004}
|
|
149
197
|
a plain line works too
|
|
150
198
|
JSON
|
|
151
199
|
|
|
200
|
+
Anything else may arrive after part of the batch was already stored — a store
|
|
201
|
+
that fails halfway through is a 503, a body past `--max-body-mb` a 413 — so
|
|
202
|
+
every response says how many lines it `accepted`, and those first lines are
|
|
203
|
+
durable. Resume after them rather than replaying the batch.
|
|
204
|
+
|
|
152
205
|
Newline-delimited JSON or plain text — the content type is not consulted,
|
|
153
206
|
because the normalizer already decides per line what it is looking at. `gzip`
|
|
154
207
|
and `zstd` bodies are decoded.
|
|
@@ -161,15 +214,21 @@ with a config line and no code change:
|
|
|
161
214
|
logs_endpoint: http://rtail:8888/v1/logs
|
|
162
215
|
|
|
163
216
|
`service.name` becomes the stream, log and resource attributes become fields,
|
|
164
|
-
and `trace_id`/`span_id` are promoted so correlation actually works.
|
|
217
|
+
and `trace_id`/`span_id` are promoted so correlation actually works. A 200
|
|
218
|
+
here promises what it does on `/v1/ingest`.
|
|
165
219
|
|
|
166
220
|
**Syslog**, RFC5424 and RFC3164, UDP and TCP on the same port:
|
|
167
221
|
|
|
168
|
-
$ rtail-server --data ./logs --syslog-port 5140
|
|
222
|
+
$ rtail-server --data ./logs --syslog-host 0.0.0.0 --syslog-port 5140
|
|
169
223
|
|
|
170
224
|
# rsyslog.conf
|
|
171
225
|
*.* @@rtail-host:5140;RSYSLOG_SyslogProtocol23Format
|
|
172
226
|
|
|
227
|
+
`--syslog-host` defaults to `--udp-host` — loopback, unless you moved it — so
|
|
228
|
+
a remote rsyslog needs the `0.0.0.0`. Syslog carries no token: `--auth-token`
|
|
229
|
+
covers HTTP only, and nothing refuses this bind without one, so off loopback
|
|
230
|
+
you are trusting your network.
|
|
231
|
+
|
|
173
232
|
APP-NAME becomes the stream, the priority becomes a real severity — so
|
|
174
233
|
`level>=ERROR` works on syslog without anyone writing a parser — and RFC5424
|
|
175
234
|
structured data is promoted to fields.
|
|
@@ -213,7 +272,7 @@ There are many log aggregation tools out there, but few of them are realtime. **
|
|
|
213
272
|
$ rtail-server --data ./logs
|
|
214
273
|
|
|
215
274
|
Point `--data` at a directory and every line is written to Parquet. Without it,
|
|
216
|
-
|
|
275
|
+
rtail is what it always was: an in-memory ring, nothing on disk, nothing to
|
|
217
276
|
clean up.
|
|
218
277
|
|
|
219
278
|
**There is no schema to define.** The envelope — timestamp, stream, level,
|
|
@@ -230,7 +289,7 @@ reconcile by name at read time.
|
|
|
230
289
|
wal/
|
|
231
290
|
streams/api.example.com/2026/08/11/L0-…parquet
|
|
232
291
|
|
|
233
|
-
which means any tool reads them, with
|
|
292
|
+
which means any tool reads them, with rtail uninstalled and nothing exported:
|
|
234
293
|
|
|
235
294
|
$ duckdb -c "
|
|
236
295
|
SELECT a_service, count(*)
|
|
@@ -311,11 +370,15 @@ so `status=200` is the number and `status="200"` is the string.
|
|
|
311
370
|
|
|
312
371
|
**Raw SQL**, for everything else:
|
|
313
372
|
|
|
314
|
-
curl -XPOST localhost:8888/v1/sql -
|
|
315
|
-
|
|
316
|
-
|
|
373
|
+
$ curl -XPOST localhost:8888/v1/sql --data-binary @- <<'JSON'
|
|
374
|
+
{
|
|
375
|
+
"sql": "SELECT a_service, count(*) n, quantile_cont(a_latency_ms, 0.95) p95 FROM logs WHERE level = 'ERROR' GROUP BY 1 ORDER BY n DESC",
|
|
317
376
|
"from": "-6h"
|
|
318
|
-
}
|
|
377
|
+
}
|
|
378
|
+
JSON
|
|
379
|
+
|
|
380
|
+
The query is one line because JSON strings cannot hold a raw newline; write
|
|
381
|
+
`\n` if you want the SQL to keep its shape.
|
|
319
382
|
|
|
320
383
|
`logs` is a view over the pruned files. The query runs read-only in a DuckDB
|
|
321
384
|
sandbox that can see nothing but your data directory.
|
|
@@ -375,9 +438,11 @@ For fun and debugging:
|
|
|
375
438
|
|
|
376
439
|
## `rtail-server(1)`
|
|
377
440
|
|
|
378
|
-
`rtail-server` receives all messages broadcast from every `rtail` client, displaying all incoming log streams in a realtime web view. **Under the hood it is a single
|
|
441
|
+
`rtail-server` receives all messages broadcast from every `rtail` client, displaying all incoming log streams in a realtime web view. **Under the hood it is a single Go binary with the webapp embedded, pushing every incoming UDP message to the browser over server-sent events.**
|
|
379
442
|
|
|
380
|
-
|
|
443
|
+
Everything is a flag, or the same name as an `RTAIL_*` environment variable.
|
|
444
|
+
The defaults are chosen so that `rtail-server` with no arguments is the v1
|
|
445
|
+
behaviour: both listeners on loopback, an in-memory ring, nothing on disk.
|
|
381
446
|
|
|
382
447
|
## Examples
|
|
383
448
|
|
|
@@ -397,34 +462,63 @@ Open your browser and start tailing logs!
|
|
|
397
462
|
|
|
398
463
|
## Params
|
|
399
464
|
|
|
400
|
-
|
|
401
|
-
Usage: rtail-server [OPTIONS]
|
|
465
|
+
`rtail-server --help` prints the lot. Grouped by what they touch:
|
|
402
466
|
|
|
403
|
-
|
|
467
|
+
**Listening**
|
|
404
468
|
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
469
|
+
| flag | default | |
|
|
470
|
+
|---|---|---|
|
|
471
|
+
| `--web-host` | `127.0.0.1` | HTTP listen address |
|
|
472
|
+
| `--web-port` | `8888` | HTTP port |
|
|
473
|
+
| `--udp-host` | `127.0.0.1` | UDP listen address |
|
|
474
|
+
| `--udp-port` | `9999` | UDP port |
|
|
475
|
+
| `--syslog-host` | `--udp-host` | syslog listen address |
|
|
476
|
+
| `--syslog-port` | `0` | syslog on this port, UDP and TCP; `0` disables it |
|
|
477
|
+
|
|
478
|
+
**Storage** — see [Storage](#storage)
|
|
479
|
+
|
|
480
|
+
| flag | default | |
|
|
481
|
+
|---|---|---|
|
|
482
|
+
| `--data` | unset | directory for durable storage; unset is memory only |
|
|
483
|
+
| `--backlog` | `100` | lines of live history kept per stream |
|
|
484
|
+
| `--compact-interval` | `5m` | how often to compact; `0` disables it |
|
|
485
|
+
| `--cluster-by` | `level` | leading sort column for daily files; empty is time only |
|
|
486
|
+
| `--retention` | `0` | delete data older than this; `0` keeps it forever |
|
|
487
|
+
| `--retention-raw` | `0` | drop the original line after this, keeping the columns |
|
|
488
|
+
| `--downsample-after` | `0` | discard low-severity records older than this |
|
|
489
|
+
| `--downsample-level` | `WARN` | the lowest severity `--downsample-after` keeps |
|
|
490
|
+
|
|
491
|
+
**Ingest**
|
|
492
|
+
|
|
493
|
+
| flag | default | |
|
|
494
|
+
|---|---|---|
|
|
495
|
+
| `--ingest-stream` | `http` | stream for HTTP records that name none |
|
|
496
|
+
| `--max-body-mb` | `32` | largest accepted ingest request |
|
|
418
497
|
|
|
419
|
-
|
|
498
|
+
**Access** — see [Authentication](#authentication)
|
|
420
499
|
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
the
|
|
500
|
+
| flag | default | |
|
|
501
|
+
|---|---|---|
|
|
502
|
+
| `--auth-token` | unset | require this bearer token on the HTTP API and webapp |
|
|
503
|
+
| `--insecure` | off | allow a non-loopback address with no `--auth-token` |
|
|
504
|
+
| `--tls-cert`, `--tls-key` | unset | serve HTTPS; both or neither |
|
|
505
|
+
|
|
506
|
+
**Everything else**
|
|
507
|
+
|
|
508
|
+
| flag | default | |
|
|
509
|
+
|---|---|---|
|
|
510
|
+
| `--web-root` | unset | serve the webapp from this directory instead of the binary |
|
|
511
|
+
| `--verbose` | off | log at debug level |
|
|
512
|
+
| `--version` | | print the version and exit |
|
|
513
|
+
|
|
514
|
+
Each one is also an environment variable, upper-cased and prefixed with
|
|
515
|
+
`RTAIL_` — `RTAIL_WEB_PORT=8080`, `RTAIL_BACKLOG=500`,
|
|
516
|
+
`RTAIL_DATA=/data`. That is how the container image is configured.
|
|
517
|
+
The v1 short aliases still work too: `--uh`, `--up`, `--wh`, `--wp`, `-b`.
|
|
424
518
|
|
|
425
519
|
Note that `--backlog` is about the *live view* — how many recent lines a
|
|
426
|
-
freshly opened browser tab is handed.
|
|
427
|
-
|
|
520
|
+
freshly opened browser tab is handed. How much is kept on disk is
|
|
521
|
+
`--retention`'s business, and is unlimited until you set it.
|
|
428
522
|
|
|
429
523
|
## API
|
|
430
524
|
|
|
@@ -433,7 +527,7 @@ The webapp is a client of a small documented API, so anything else can be too:
|
|
|
433
527
|
| endpoint | what it does |
|
|
434
528
|
|---|---|
|
|
435
529
|
| `GET /v1/streams` | every known stream |
|
|
436
|
-
| `GET /v1/tail?stream=<name>` | SSE feed: `streams`, `backlog`, then `line` events. Omit `stream` to
|
|
530
|
+
| `GET /v1/tail?stream=<name>` | SSE feed: `streams`, `backlog`, then `line` events. Omit `stream` to tail every stream at once, backlog merged |
|
|
437
531
|
| `GET /v1/search` | matching records, keyset-paginated |
|
|
438
532
|
| `GET /v1/histogram` | match counts over time, split by severity |
|
|
439
533
|
| `GET /v1/fields?field=<name>` | the most common values of a field, within the current filter |
|
|
@@ -441,10 +535,10 @@ The webapp is a client of a small documented API, so anything else can be too:
|
|
|
441
535
|
| `POST /v1/sql` | read-only SQL over a `logs` view |
|
|
442
536
|
| `POST /v1/ingest` | NDJSON or plain lines, optionally gzip/zstd |
|
|
443
537
|
| `POST /v1/logs` | OTLP logs, protobuf or JSON |
|
|
444
|
-
| `GET /healthz` | version, stream count, per-receiver ingest counters |
|
|
538
|
+
| `GET /healthz` | version, stream count, store size, per-receiver ingest counters |
|
|
445
539
|
|
|
446
540
|
The search endpoints share their parameters: `q` (filter), `stream`, `from` and
|
|
447
|
-
`to` (RFC3339, epoch millis, or a relative `-6h`), `limit`, `cursor`, `order`.
|
|
541
|
+
`to` (RFC3339, epoch millis, or a relative `-6h` or `-7d`), `limit`, `cursor`, `order`.
|
|
448
542
|
They return 503 with an explanation when the server is running without
|
|
449
543
|
`--data`, since there is nothing to search.
|
|
450
544
|
|
|
@@ -465,13 +559,16 @@ as a header; browsers sign in once and get a cookie.
|
|
|
465
559
|
|
|
466
560
|
`X-Rtail-Token` works too, for clients that reserve `Authorization`. In a
|
|
467
561
|
browser, opening the server without one gives you a sign-in form, and
|
|
468
|
-
`https://logs.example.com/?token=…` signs you in from a link
|
|
469
|
-
|
|
470
|
-
|
|
562
|
+
`https://logs.example.com/?token=…` signs you in from a link. It is the shared
|
|
563
|
+
token itself, not a one-time code, so treat the link as the secret it is: the
|
|
564
|
+
server sets the cookie and redirects the token out of the address bar so it
|
|
565
|
+
does not end up in your history or a bookmark.
|
|
471
566
|
|
|
472
567
|
`/healthz` is deliberately left open, so a container healthcheck or a load
|
|
473
|
-
balancer keeps working without being handed the secret. It reports liveness
|
|
474
|
-
and
|
|
568
|
+
balancer keeps working without being handed the secret. It reports liveness,
|
|
569
|
+
the version and a handful of numbers — how many streams, how much is stored,
|
|
570
|
+
what each receiver has counted — and never a name; everything that could show
|
|
571
|
+
a log line is behind the token.
|
|
475
572
|
|
|
476
573
|
**Running with no token is refused on any address but loopback.** This is the
|
|
477
574
|
one rule the server enforces on your behalf:
|
|
@@ -484,7 +581,7 @@ Pass `--insecure` if something in front of it handles access. Nothing stops
|
|
|
484
581
|
you; it just will not happen by accident.
|
|
485
582
|
|
|
486
583
|
What this deliberately is not: accounts, roles, OIDC, SAML, or per-stream
|
|
487
|
-
permissions.
|
|
584
|
+
permissions. rtail is a single-node tool, and anyone who needs those already
|
|
488
585
|
has a reverse proxy that does them properly.
|
|
489
586
|
|
|
490
587
|
**Two holes worth knowing about.** UDP ingest is unauthenticated, because UDP
|
|
@@ -504,8 +601,8 @@ a certificate is the one thing every reverse proxy already solves.
|
|
|
504
601
|
|
|
505
602
|
Note there are two images, and they are not the same thing:
|
|
506
603
|
[`Dockerfile`](Dockerfile) is the server you deploy, and
|
|
507
|
-
[`tools/Dockerfile`](tools/Dockerfile)
|
|
508
|
-
|
|
604
|
+
[`tools/Dockerfile`](tools/Dockerfile) is the development toolchain, which
|
|
605
|
+
mounts your checkout.
|
|
509
606
|
|
|
510
607
|
The toolchain lives in a container ([`tools/Dockerfile`](tools/Dockerfile)), so Docker is
|
|
511
608
|
the only thing you need installed — no Node.js, no Go, no npm, no global CLIs.
|
|
@@ -532,13 +629,17 @@ To leave it running in the background instead:
|
|
|
532
629
|
Other targets:
|
|
533
630
|
|
|
534
631
|
$ make server # build bin/rtail-server
|
|
632
|
+
$ make client # build bin/rtail
|
|
535
633
|
$ make release # build the self-contained server, webapp embedded
|
|
536
634
|
$ make build # build the webapp into app/ (development)
|
|
537
635
|
$ make dist # build the minified webapp into web/dist
|
|
538
|
-
$ make
|
|
636
|
+
$ make ci # everything CI runs, before you spend a runner minute
|
|
637
|
+
$ make test # go test -race ./...
|
|
539
638
|
$ make vet # go vet
|
|
540
639
|
$ make fmt # gofmt the Go sources
|
|
541
640
|
$ make typecheck # type-check the webapp
|
|
641
|
+
$ make npm-pack # lay out the npm packages under dist/npm
|
|
642
|
+
$ make screenshots # retake the screenshot at the top of this README
|
|
542
643
|
$ make shell # open a shell inside the toolchain container
|
|
543
644
|
$ make clean # remove generated assets, binaries and dependencies
|
|
544
645
|
|
|
@@ -553,9 +654,14 @@ receives a log line.
|
|
|
553
654
|
`make url` reports the current one. Override with `make dev PORT=9000
|
|
554
655
|
UDP_PORT=9001`.
|
|
555
656
|
|
|
556
|
-
If you
|
|
557
|
-
npm
|
|
558
|
-
|
|
657
|
+
If you have Node on your machine, the webapp targets are plain npm scripts —
|
|
658
|
+
`npm install`, then `npm run build`, `npm run dist`, `npm run typecheck`.
|
|
659
|
+
`npm run dev` also spawns `bin/rtail-server` and `bin/rtail`, so it needs Go
|
|
660
|
+
as well, to build those two for your machine. `make server client` is not a
|
|
661
|
+
substitute on a Mac: both build inside the Linux toolchain container, so what
|
|
662
|
+
lands in `bin/` is a Linux binary — right for `make dev`, which runs there, and
|
|
663
|
+
not something macOS will start. Node builds the webapp and is not in the
|
|
664
|
+
product: the server embeds the bundle, and the client is a Go binary.
|
|
559
665
|
|
|
560
666
|
### The stack
|
|
561
667
|
|
|
@@ -563,9 +669,9 @@ in the product: the server embeds the bundle, and the client is a Go binary.
|
|
|
563
669
|
| --- | --- |
|
|
564
670
|
| Client | Go, one static binary, [json5](https://github.com/titanous/json5) its only dependency |
|
|
565
671
|
| Server | Go + [DuckDB](https://duckdb.org) + [parquet-go](https://github.com/parquet-go/parquet-go) |
|
|
566
|
-
| Webapp | [Preact](https://preactjs.com) + TypeScript, ~
|
|
672
|
+
| Webapp | [Preact](https://preactjs.com) + TypeScript, ~31 KB of JS and ~13 KB of CSS, gzipped |
|
|
567
673
|
| Build | [esbuild](https://esbuild.github.io); the stylesheets are plain CSS |
|
|
568
|
-
| Tests | `go test ./...` |
|
|
674
|
+
| Tests | `go test -race ./...` |
|
|
569
675
|
|
|
570
676
|
The webapp has no framework runtime beyond Preact: routing, preferences,
|
|
571
677
|
popovers, and timestamp formatting are a few dozen lines each over the
|
|
@@ -573,22 +679,29 @@ platform (`history`, `localStorage`, `Intl`) rather than dependencies.
|
|
|
573
679
|
|
|
574
680
|
# How to contribute
|
|
575
681
|
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
682
|
+
There is one long-lived branch, `main`. Work on a branch of your own and open
|
|
683
|
+
a pull request against it. Releases are cut by pushing a `v*` tag — the
|
|
684
|
+
version in `package.json` has to match it, or the release fails before it
|
|
685
|
+
publishes anything.
|
|
580
686
|
|
|
581
|
-
|
|
582
|
-
toolchain image. Use it to check whether your contribution breaks some part of
|
|
583
|
-
the library, and be sure to add new tests for each new feature.
|
|
687
|
+
Before you push, run what CI runs:
|
|
584
688
|
|
|
585
|
-
$ make
|
|
689
|
+
$ make ci
|
|
586
690
|
|
|
587
|
-
|
|
691
|
+
That is the same [`tools/ci.sh`](tools/ci.sh) the workflow calls, so a green
|
|
692
|
+
run here is a green run there — `go vet`, `gofmt`, `go test -race ./...`,
|
|
693
|
+
shellcheck on `install.sh`, the webapp typecheck and production build, the npm
|
|
694
|
+
packaging smoke-tested through real tarballs, and the container image. The two
|
|
695
|
+
things it cannot cover are the macOS server build and the Node 20 webapp
|
|
696
|
+
build, because both need a second machine.
|
|
588
697
|
|
|
589
|
-
|
|
698
|
+
`make` builds the toolchain image only when there is none, so after a change
|
|
699
|
+
to [`tools/Dockerfile`](tools/Dockerfile), `docker build -t rtail-tools tools/`
|
|
700
|
+
brings yours level with CI's.
|
|
590
701
|
|
|
591
|
-
|
|
702
|
+
Add tests with each new feature, and keep the webapp type-clean — `make
|
|
703
|
+
typecheck` on its own is the quick version. CI builds the webapp on Node 20
|
|
704
|
+
and 22.
|
|
592
705
|
|
|
593
706
|
## Contributors
|
|
594
707
|
|
|
@@ -600,10 +713,6 @@ CI runs both, plus the production build, on Node 20 and 22.
|
|
|
600
713
|
|
|
601
714
|
## Roadmap (aka where you can help)
|
|
602
715
|
|
|
603
|
-
* Optional HTTP basic auth on the web port, and a shared secret on UDP ingest.
|
|
604
|
-
Full OAuth ([#44](https://github.com/kilianc/rtail/issues/44)) needs a user
|
|
605
|
-
model and session storage, which is a lot of machinery for a tool with no
|
|
606
|
-
persistence layer — basic auth covers the actual risk.
|
|
607
716
|
* A catch-all docker logs image: watch `docker events`, pipe every container's
|
|
608
717
|
`docker logs -f` into a stream named after it.
|
|
609
718
|
* Broaden the test suite, particularly around the UDP ingest path.
|
|
@@ -616,6 +725,13 @@ Dropped, and why:
|
|
|
616
725
|
than DockerHub, since it needs no separate account or secrets.
|
|
617
726
|
* ~~Write a rock solid test suite~~ — a suite exists and runs in CI. Deepening
|
|
618
727
|
it is the open item above.
|
|
728
|
+
* ~~Optional HTTP basic auth on the web port~~ — done, though as one shared
|
|
729
|
+
bearer token rather than basic auth; see
|
|
730
|
+
[Authentication](#authentication). Full OAuth
|
|
731
|
+
([#44](https://github.com/kilianc/rtail/issues/44)) is still declined: it
|
|
732
|
+
needs a user model and session storage, which is a lot of machinery for a
|
|
733
|
+
single-node tool. A shared secret on UDP ingest is the one part that cannot
|
|
734
|
+
be done — the v1 wire format is frozen and has nowhere to carry one.
|
|
619
735
|
* ~~Allow use of DTLS~~ — the linked PR was against `joyent/node`, a repo that
|
|
620
736
|
no longer exists, and Node core still has no DTLS. Run rtail over WireGuard
|
|
621
737
|
or Tailscale instead; that is the modern answer to this problem.
|
|
@@ -625,7 +741,7 @@ Dropped, and why:
|
|
|
625
741
|
which contradicts the design. `--backlog` covers the useful part.
|
|
626
742
|
|
|
627
743
|
## Sponsors
|
|
628
|
-
❤
|
|
744
|
+
❤ rtail? Consider sponsoring this project to keep it alive and free for the community.
|
|
629
745
|
|
|
630
746
|
* ? (wildcard TLS cert)
|
|
631
747
|
* ? (.io domain)
|
|
@@ -634,7 +750,7 @@ Dropped, and why:
|
|
|
634
750
|
|
|
635
751
|
## License
|
|
636
752
|
|
|
637
|
-
|
|
753
|
+
rtail is licensed under the **Apache License 2.0** — see [`LICENSE`](LICENSE)
|
|
638
754
|
and [`COPYRIGHT.md`](COPYRIGHT.md).
|
|
639
755
|
|
|
640
756
|
Run it, modify it, ship it inside your own product, commercially, at any
|
|
@@ -642,7 +758,7 @@ scale. Keep the notice, say what you changed, and the patent grant in §3
|
|
|
642
758
|
covers you. There is nothing to buy and no feature held back for a paid tier,
|
|
643
759
|
because there is no paid tier.
|
|
644
760
|
|
|
645
|
-
|
|
761
|
+
rtail **v1.x remains under the MIT licence** it was released with. That does
|
|
646
762
|
not change retroactively; if you are on v1, the terms you accepted still
|
|
647
763
|
apply. v2 spent part of its life under AGPL-3.0; that is reversed, and
|
|
648
764
|
[`COPYRIGHT.md`](COPYRIGHT.md) has the reasoning.
|
package/bin/rtail.js
CHANGED
|
@@ -30,6 +30,7 @@
|
|
|
30
30
|
*/
|
|
31
31
|
|
|
32
32
|
const { spawnSync } = require('node:child_process')
|
|
33
|
+
const { constants } = require('node:os')
|
|
33
34
|
|
|
34
35
|
// Go and Node spell the same architectures differently.
|
|
35
36
|
const PLATFORMS = {
|
|
@@ -88,4 +89,9 @@ if (result.error) {
|
|
|
88
89
|
|
|
89
90
|
// A child killed by a signal has a null status; 128 + signal is the shell's
|
|
90
91
|
// convention for reporting that, and exiting 0 would claim it succeeded.
|
|
91
|
-
|
|
92
|
+
if (null === result.status) {
|
|
93
|
+
const number = constants.signals[result.signal]
|
|
94
|
+
process.exit(number ? 128 + number : 1)
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
process.exit(result.status)
|
package/package.json
CHANGED
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "rtail",
|
|
3
|
-
"version": "2.0.0-rc.
|
|
3
|
+
"version": "2.0.0-rc.3",
|
|
4
4
|
"description": "Terminal output to the browser in seconds, using UNIX pipes.",
|
|
5
5
|
"keywords": ["log", "logging", "logio", "logs", "tail", "udp"],
|
|
6
6
|
"bin": { "rtail": "bin/rtail.js" },
|
|
7
|
-
"files": ["bin"],
|
|
7
|
+
"files": ["bin", "NOTICE"],
|
|
8
8
|
"engines": { "node": ">=20" },
|
|
9
9
|
"optionalDependencies": {
|
|
10
|
-
"@rtail/linux-x64": "2.0.0-rc.
|
|
11
|
-
"@rtail/linux-arm64": "2.0.0-rc.
|
|
12
|
-
"@rtail/darwin-x64": "2.0.0-rc.
|
|
13
|
-
"@rtail/darwin-arm64": "2.0.0-rc.
|
|
14
|
-
"@rtail/win32-x64": "2.0.0-rc.
|
|
10
|
+
"@rtail/linux-x64": "2.0.0-rc.3",
|
|
11
|
+
"@rtail/linux-arm64": "2.0.0-rc.3",
|
|
12
|
+
"@rtail/darwin-x64": "2.0.0-rc.3",
|
|
13
|
+
"@rtail/darwin-arm64": "2.0.0-rc.3",
|
|
14
|
+
"@rtail/win32-x64": "2.0.0-rc.3"
|
|
15
15
|
},
|
|
16
16
|
"repository": {
|
|
17
17
|
"type": "git",
|