onlylogs 0.11.0 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +251 -77
  3. data/app/assets/images/onlylogs/favicon/apple-touch-icon.png +0 -0
  4. data/app/assets/images/onlylogs/favicon/favicon-96x96.png +0 -0
  5. data/app/assets/images/onlylogs/favicon/favicon.ico +0 -0
  6. data/app/assets/images/onlylogs/favicon/favicon.svg +1 -3
  7. data/app/assets/images/onlylogs/favicon/web-app-manifest-192x192.png +0 -0
  8. data/app/assets/images/onlylogs/favicon/web-app-manifest-512x512.png +0 -0
  9. data/app/assets/images/onlylogs/logo.png +0 -0
  10. data/app/assets/javascripts/onlylogs/clusterize.js +26 -1
  11. data/app/channels/onlylogs/logs_channel.rb +14 -9
  12. data/app/controllers/concerns/onlylogs/authentication.rb +26 -0
  13. data/app/controllers/onlylogs/application_controller.rb +1 -18
  14. data/app/controllers/onlylogs/queries_controller.rb +72 -0
  15. data/app/controllers/onlylogs/viewer_controller.rb +7 -0
  16. data/app/javascript/onlylogs/controllers/keyboard_shortcuts_controller.js +131 -21
  17. data/app/javascript/onlylogs/controllers/log_cursor_controller.js +286 -0
  18. data/app/javascript/onlylogs/controllers/log_streamer_controller.js +42 -2
  19. data/app/javascript/onlylogs/controllers/match_highlight_controller.js +82 -0
  20. data/app/javascript/onlylogs/controllers/queries_controller.js +470 -0
  21. data/app/javascript/onlylogs/controllers/range_slider_controller.js +5 -4
  22. data/app/javascript/onlylogs/controllers/text_selection_controller.js +136 -15
  23. data/app/models/onlylogs/file.rb +14 -5
  24. data/app/models/onlylogs/grep.rb +31 -12
  25. data/app/models/onlylogs/queries_record.rb +30 -0
  26. data/app/models/onlylogs/query.rb +29 -0
  27. data/app/views/onlylogs/logs/index.html.erb +1 -1
  28. data/app/views/onlylogs/shared/_log_container.html.erb +165 -27
  29. data/app/views/onlylogs/shared/_log_container_styles.html.erb +729 -44
  30. data/app/views/onlylogs/shared/_shortcuts_dialog.html.erb +62 -0
  31. data/bin/super_grep +7 -0
  32. data/bin/super_ripgrep +14 -0
  33. data/config/routes.rb +1 -0
  34. data/lib/onlylogs/configuration.rb +14 -1
  35. data/lib/onlylogs/engine.rb +9 -0
  36. data/lib/onlylogs/http_device.rb +28 -13
  37. data/lib/onlylogs/version.rb +1 -1
  38. data/lib/onlylogs.rb +3 -0
  39. metadata +38 -2
  40. data/app/jobs/onlylogs/application_job.rb +0 -4
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4707246899a958c58bf6ea58aba6f21234d938c660ce23dbad8df598654a4b5c
4
- data.tar.gz: c46cf3df22e80cd90749bf144f62a271a9556b13ee718640377aa86f34608e05
3
+ metadata.gz: df2635aae242c5b3d3d1881c3dcb4c9297662200134c403b6f722036ef8f2357
4
+ data.tar.gz: f6faaccddcbd4042a0d6cb4cee8f087ce0ff7e9c75e66f0e403ee5c7de2edbc3
5
5
  SHA512:
6
- metadata.gz: 7eabe1b6124704eee89f759001cb7df9bf8fa5bb71ad98ac9c26bbc091db38c93ee3c7fdb4b35729f33cc9eeb2d0ac3170b2d2ef11ae5b979b7ec454664bc435
7
- data.tar.gz: fa8226aa140b44b8eac0f3a03000aac95615470343d90de827d98da79ba4045c776b052171b3555ef955db2747dc760640cd787f6207ae95f6e67e70e152dca1
6
+ metadata.gz: e649a51a9841c162ad4c418eea5047462505054a040f5314fec37f171109fb39bf4198e12d107b04f52d77cc19d84ab873e5435ca28bb77c1b4aad17677d02b6
7
+ data.tar.gz: 4e6b8ab9a92d4317c04de55e4f560642e77baa8d8b73c7f583fdb181383eed1965c6e731cbb871ede1aa2ce291a0f3a4dd07759eada1e0d4fc1f5fb069ab479d
data/README.md CHANGED
@@ -1,13 +1,13 @@
1
1
  <img alt="w:100px" src="app/assets/images/onlylogs/logo.png" width="400px"/>
2
2
 
3
- We believe logs are enough.
3
+ We believe logs are enough.
4
4
 
5
5
  We believe logs in human-readable format are enough.
6
6
 
7
7
  Stop streaming your logs to very expensive external services: just store your logs on disk.
8
8
 
9
- When your application grows and you don't want to self-host your log files anymore, you can
10
- stream them to https://onlylogs.io and continue enjoying the same features.
9
+ When your application grows and you don't want to self-host your log files anymore, you can stream them
10
+ to https://onlylogs.io and continue enjoying the same features.
11
11
 
12
12
  > [!IMPORTANT]
13
13
  > https://onlylogs.io is still in beta. Send us an email to a@renuo.ch if you want access to the platform.
@@ -18,7 +18,11 @@ If you already have a disk, you can just keep there also your log files (as well
18
18
 
19
19
  This section explains how to setup onlylogs to self host your logs and access them directly from your Rails app.
20
20
 
21
- If instead you want to stream your logs to https://onlylogs.io, head to [the onlylogs.io instructions page](https://onlylogs.io/instructions).
21
+ If instead you want to stream your logs to https://onlylogs.io, head
22
+ to [the onlylogs.io instructions page](https://onlylogs.io/instructions).
23
+
24
+ Onlylogs needs Rails 8 or newer. It works with Importmap, Sprockets and Propshaft (see
25
+ [INTEGRATION.md](INTEGRATION.md) if you have to register the Stimulus controller by hand).
22
26
 
23
27
  Add this line to your application's Gemfile:
24
28
 
@@ -38,6 +42,7 @@ mount the engine in your `routes.rb`
38
42
  Rails.application.routes.draw do
39
43
  # ...
40
44
  mount Onlylogs::Engine, at: "/onlylogs"
45
+ end
41
46
  ```
42
47
 
43
48
  Finally, you **must secure the engine**. Read the section dedicated to the [Authentication](#authentication).
@@ -51,17 +56,50 @@ Finally, you **must secure the engine**. Read the section dedicated to the [Auth
51
56
 
52
57
  Head to `/onlylogs` and enjoy your logs streamed right into your face!
53
58
 
54
- Here you can grep your logs with regular expressions.
59
+ The viewer has two modes:
60
+
61
+ - **Live**: the end of the file is streamed to you as new lines are written.
62
+ - **Search**: you look for something in the whole file.
63
+
64
+ Next to the query field you find two checkboxes:
65
+
66
+ - **Regexp**: the query is a regular expression. Without it, the query is searched as plain text.
67
+ - **Ignore case**: `error` also finds `ERROR`.
68
+
69
+ What the query matched is highlighted on every line. On any line you can also ask for the lines around it
70
+ (press `Enter` on it), and you can download the whole file.
55
71
 
56
72
  > [!TIP]
57
- > Onlylogs automatically detects and uses [ripgrep (rg)](https://github.com/BurntSushi/ripgrep) if available, which provides significantly faster search experience.
58
- > If ripgrep is not installed, onlylogs falls back to `grep`.
73
+ > Onlylogs automatically detects and uses [ripgrep (rg)](https://github.com/BurntSushi/ripgrep) if available, which
74
+ provides significantly faster search experience.
75
+ > If ripgrep is not installed, onlylogs falls back to `grep`.
59
76
  > A warning icon (⚠️) will be displayed in the toolbar when using `grep` to indicate slower search performance.
60
77
 
78
+ ### Saved searches
79
+
80
+ Click the star in the query field to save the current query. The list next to it shows all saved searches: pick one to
81
+ apply it, rename it or remove it. A saved search is shared by everybody who opens the same log file.
82
+
83
+ Saved searches are stored in a small SQLite database, in a hidden `.onlylogs` folder next to the log file
84
+ (for example `log/.onlylogs/queries.sqlite3`). The folder must be writable. In Docker, keep it on a volume, like the
85
+ logs (see [Notes about Docker](#notes-about-docker)). You can change the folder name with `config.database_dir`.
86
+
87
+ ### Keyboard shortcuts
88
+
89
+ Hold **⌃ Control** on macOS or **Alt** on Windows and Linux: a letter appears on every control of the toolbar, and
90
+ modifier + letter triggers it from anywhere, also while typing a query. Press `?` (or the ⌨ button) for the full list.
91
+
92
+ Inside the log lines, `↑` `↓` (or `j` `k`) move a line cursor, `←` `→` select words of that line (`Shift` grows the
93
+ selection) and `Enter` searches the selection, or shows the lines around the current one. `/` jumps to the query,
94
+ `Enter` or `Esc` in the query jumps back into the lines.
95
+
96
+ A host app can give its own controls a letter with the same attribute, e.g.
97
+ `<button data-onlylogs-shortcut="f">Fullscreen</button>`.
98
+
61
99
  ## Logging in production
62
100
 
63
- Rails apps log to `STDOUT` in production and let the platform collect the stream.
64
- onlylogs keeps that default: `Onlylogs::HttpLogger.new` writes every line to `$stdout` and, when
101
+ Rails apps log to `STDOUT` in production and let the platform collect the stream. onlylogs keeps that default:
102
+ `Onlylogs::HttpLogger.new` writes every line to `$stdout` and, when
65
103
  `ONLYLOGS_DRAIN_URL` is set, ships it to https://onlylogs.io as well. Nothing is written to disk.
66
104
 
67
105
  ```ruby
@@ -69,13 +107,19 @@ onlylogs keeps that default: `Onlylogs::HttpLogger.new` writes every line to `$s
69
107
  config.logger = Onlylogs::HttpLogger.new
70
108
  ```
71
109
 
72
- Keep this default on an **ephemeral filesystem** (Heroku, Deploio, Docker, Kubernetes). A log file
73
- there is lost on every deploy and, while it grows, its page cache counts towards the container's
74
- memory limit: it shows up as a memory leak that resets on each release.
110
+ Shipping is fire and forget. Lines are batched in memory and sent in the background; when onlylogs.io is down or does
111
+ not answer within the timeouts, the sender backs off for a while and the lines of that period are lost. Your `$stdout`
112
+ stream still has them. If you would rather keep them, set `ONLYLOGS_SPOOL_ENABLED=true` (`1`, `yes` and `on` work
113
+ too): batches that could not be delivered are then written to `ONLYLOGS_SPOOL_DIR` (default `tmp/onlylogs/spool`, up
114
+ to `ONLYLOGS_SPOOL_MAX_BYTES`, 128 MB) and replayed once the drain is back, also after a restart when the directory is
115
+ on a persistent disk.
116
+
117
+ Keep this default on an **ephemeral filesystem** (Heroku, Deploio, Docker, Kubernetes). A log file there is lost on
118
+ every deploy and, while it grows, its page cache counts towards the container's memory limit: it shows up as a memory
119
+ leak that resets on each release.
75
120
 
76
- If you have a **persistent disk** and want the files, for instance to browse them with the engine,
77
- pass a rotating log device as the local fallback. Rails rotates at 100 MB in development, which is a
78
- good size here too:
121
+ If you have a **persistent disk** and want the files, for instance to browse them with the engine, pass a rotating log
122
+ device as the local fallback. Rails rotates at 100 MB in development, which is a good size here too:
79
123
 
80
124
  ```ruby
81
125
  # config/environments/production.rb
@@ -85,11 +129,82 @@ config.logger = Onlylogs::HttpLogger.new(local_fallback: log_file)
85
129
 
86
130
  Any IO or log device works and is used as is. `Onlylogs::SocketLogger` accepts the same argument.
87
131
 
132
+ ### Settings of `HttpLogger`
133
+
134
+ `HttpLogger` is configured with environment variables. Only `ONLYLOGS_DRAIN_URL` is needed. The others have sensible
135
+ defaults.
136
+
137
+ | Variable | Default | What it does |
138
+ |-------------------------------|--------------------------|-----------------------------------------------------------|
139
+ | `ONLYLOGS_DRAIN_URL` | none | Where the logs are sent. Without it, logs stay local. |
140
+ | `ONLYLOGS_BATCH_SIZE` | `100` | Lines sent together in one request. |
141
+ | `ONLYLOGS_FLUSH_INTERVAL` | `0.5` | Seconds to wait before sending an incomplete batch. |
142
+ | `ONLYLOGS_MAX_QUEUE_SIZE` | `10000` | Lines kept in memory while waiting to be sent. |
143
+ | `ONLYLOGS_MAX_BATCH_BYTES` | `1048576` (1 MB) | Maximum size of a batch. Longer lines are truncated. |
144
+ | `ONLYLOGS_OPEN_TIMEOUT` | `0.5` | Seconds to wait when connecting to the drain. |
145
+ | `ONLYLOGS_READ_TIMEOUT` | `0.5` | Seconds to wait for the answer of the drain. |
146
+ | `ONLYLOGS_KEEP_ALIVE_TIMEOUT` | `30` | Seconds an idle connection is kept open for reuse. |
147
+ | `ONLYLOGS_CIRCUIT_COOLDOWN` | `30` | Seconds to stop sending after repeated failures. |
148
+ | `ONLYLOGS_SPOOL_DIR` | `tmp/onlylogs/spool` | Folder where undelivered batches are stored on disk. |
149
+ | `ONLYLOGS_SPOOL_MAX_BYTES` | `134217728` (128 MB) | Maximum size of that folder. |
150
+
151
+ A wrong value (for example a negative number) is ignored with a warning and the default is used. A wrong
152
+ `ONLYLOGS_DRAIN_URL` never stops your app from booting: it just logs locally.
153
+
154
+ ### What happens when the drain is down
155
+
156
+ Your app must never become slow or crash because of logging. `HttpLogger` sends the lines from a background thread,
157
+ never from the request thread, and it protects your app like this:
158
+
159
+ 1. Lines wait in a queue in memory and are sent in batches.
160
+ 2. After a few failed requests, a **circuit breaker** stops trying for a while, so your app does not wait on a dead
161
+ server.
162
+ 3. Batches that could not be delivered are saved in the **spool** folder on disk. They are sent again when the drain
163
+ is back, even after a restart. Because of this, a line can very rarely arrive twice, but it is not lost.
164
+ 4. If the queue is full, new lines are dropped. If the spool is full, its oldest batches are deleted. Your app keeps
165
+ running.
166
+
167
+ To see this in action, run `bin/fake_drain` and `bin/fake_app` (see [Development](#development--contributing)).
168
+
169
+ ### Sidecar (alternative to `HttpLogger`)
170
+
171
+ Instead of sending logs from inside the app, you can let a small separate process do it. The app writes the lines to
172
+ a Unix socket with `Onlylogs::SocketLogger`, and the sidecar sends them to the drain.
173
+
174
+ ```ruby
175
+ # config/environments/production.rb
176
+ config.logger = Onlylogs::SocketLogger.new
177
+ ```
178
+
179
+ ```ruby
180
+ # config/puma.rb
181
+ plugin :onlylogs_sidecar # starts and stops the sidecar together with Puma
182
+ ```
183
+
184
+ You can also start it yourself with `bin/onlylogs_sidecar`. It reads `ONLYLOGS_DRAIN_URL` and, optionally,
185
+ `ONLYLOGS_SIDECAR_SOCKET`, `ONLYLOGS_BATCH_SIZE` and `ONLYLOGS_FLUSH_INTERVAL`.
186
+
187
+ Prefer `HttpLogger`: it works in every process (Puma, Sidekiq, rake tasks) and has the spool. Use the sidecar only if
188
+ you want the sending to happen outside of your app process.
189
+
190
+ ### Filtering Log Lines with a Denylist
191
+
192
+ The `Onlylogs::Formatter` supports a denylist: an array of regular expressions that prevents matching lines from being
193
+ logged. This is useful for filtering out noisy or irrelevant entries like health checks or asset requests.
194
+
195
+ ```ruby
196
+ # config/environments/production.rb
197
+ config.logger = Onlylogs::HttpLogger.new
198
+ config.logger.formatter.denylist = [/health_check/, /ping/, /\.css\z/]
199
+ ```
200
+
201
+ Any log message matching at least one pattern in the denylist will be silently dropped.
202
+
88
203
  ## Authentication
89
204
 
90
205
  Yes, we should do this right away, because this engine gives access to your log files, so you want to be sure.
91
206
 
92
- The engine has one Controller and one ActionCable channel that **must be protected**.
207
+ The engine has controllers and one ActionCable channel that **must be protected**.
93
208
 
94
209
  Please be sure to secure them properly.
95
210
 
@@ -103,15 +218,17 @@ If your app is running in a Docker container, for example with Kamal, remember t
103
218
  ```yaml
104
219
  # config/deploy.yml
105
220
  volumes:
106
- - "storage:/rails/storage"
107
- - "cache:/rails/tmp/cache"
108
- - "logs:/rails/log"
221
+ - "storage:/rails/storage"
222
+ - "cache:/rails/tmp/cache"
223
+ - "logs:/rails/log"
109
224
  ```
110
225
 
226
+ The saved searches live in `log/.onlylogs`, so they are kept too.
227
+
111
228
  ### Basic Authentication Setup
112
229
 
113
- Credentials can be configured using environment variables, Rails credentials, or programmatically.
114
- Environment variables take precedence over Rails credentials.
230
+ Credentials can be configured using environment variables, Rails credentials, or programmatically. Environment variables
231
+ take precedence over Rails credentials.
115
232
 
116
233
  ```bash
117
234
  # env variables
@@ -134,11 +251,10 @@ Onlylogs.configure do |config|
134
251
  end
135
252
  ```
136
253
 
137
-
138
254
  ### Custom Authentication
139
255
 
140
- When you need custom authentication logic beyond basic auth,
141
- you can override the default authentication by configuring a parent controller that defines the `authenticate_onlylogs_user!` method.
256
+ When you need custom authentication logic beyond basic auth, you can override the default authentication by configuring
257
+ a parent controller that defines the `authenticate_onlylogs_user!` method.
142
258
 
143
259
  Configure a custom parent controller in your initializer:
144
260
 
@@ -158,7 +274,7 @@ class ApplicationController < ActionController::Base
158
274
  private
159
275
 
160
276
  def authenticate_onlylogs_user!
161
- raise unless current_user.can_access_logs?
277
+ raise unless current_user.can_access_logs?
162
278
  end
163
279
  end
164
280
  ```
@@ -174,25 +290,36 @@ Onlylogs.configure do |config|
174
290
  end
175
291
  ```
176
292
 
177
- ### WebSocket Authentication
293
+ ### Allowing only some users to see some files
294
+
295
+ By default everybody who gets through the authentication can open every file you allowed (see
296
+ [File Access Security](#file-access-security)). To decide per user and per file, set `authorize_log_file`. It runs
297
+ inside the controller, so you can use your own `current_user`. Return something falsy to refuse the file.
178
298
 
179
- Logs are streamed through a WebSocket connection, the Websocket is not protected, but in order to stream a file,
180
- the file path must be white-listed (see section below) and the file path encrypted using `Onlylogs::SecureFilePath.encrypt`
299
+ ```ruby
300
+ # config/initializers/onlylogs.rb
301
+ Onlylogs.configure do |config|
302
+ config.authorize_log_file = ->(file_path) { current_user.admin? || file_path.to_s.end_with?("public.log") }
303
+ end
304
+ ```
181
305
 
306
+ If you show the log viewer in your own pages to more people than the rest of the engine, you can give its endpoints a
307
+ different parent controller with `config.viewer_parent_controller`.
182
308
 
183
- ## Customization
309
+ ### WebSocket Authentication
184
310
 
185
- Onlylogs provides two ways to customize the appearance of the log viewer: CSS Variables and a complete style override.
186
- Check the file [_log_container_styles.html.erb](app/views/onlylogs/shared/_log_container_styles.html.erb) for the complete list of CSS variables.
311
+ Logs are streamed through a WebSocket. To stream a file, two things must be true:
187
312
 
188
- ## Configuration
313
+ - the file path is allowed (see [File Access Security](#file-access-security));
314
+ - the file path is sent encrypted, with `Onlylogs::SecureFilePath.encrypt`. The viewer does this for you. Nobody can
315
+ ask for a path they were not given.
189
316
 
190
- Check `configuration.rb` to see a list of all possible configuration.
317
+ An empty or unknown path is refused.
191
318
 
192
319
  ### File Access Security
193
320
 
194
- Onlylogs includes a secure file access system that prevents unauthorized access to files on your server.
195
- By default, onlylogs can access your Rails environment-specific log files (e.g., `log/development.log`, `log/production.log`).
321
+ Onlylogs includes a secure file access system that prevents unauthorized access to files on your server. By default,
322
+ onlylogs can access your Rails environment-specific log files (e.g., `log/development.log`, `log/production.log`).
196
323
 
197
324
  #### Configuring Allowed Files
198
325
 
@@ -206,15 +333,15 @@ Onlylogs.configure do |config|
206
333
  Rails.root.join("log/development.log"),
207
334
  Rails.root.join("log/production.log"),
208
335
  Rails.root.join("log/test.log"),
209
-
336
+
210
337
  # Custom log files
211
338
  Rails.root.join("log/custom.log"),
212
339
  Rails.root.join("log/api.log"),
213
-
340
+
214
341
  # Application-specific logs
215
342
  Rails.root.join("log/background_jobs.log"),
216
343
  Rails.root.join("log/imports.log"),
217
-
344
+
218
345
  # Allow all .log files in a directory using glob patterns
219
346
  Rails.root.join("log/*.log"),
220
347
  Rails.root.join("tmp/logs/*.log")
@@ -223,6 +350,7 @@ end
223
350
  ```
224
351
 
225
352
  **Default Behavior:**
353
+
226
354
  - If not configured, onlylogs defaults to `Rails.root.join("log/#{Rails.env}.log").to_s`
227
355
  - This means it will use `log/development.log` in development, `log/production.log` in production, etc.
228
356
 
@@ -236,11 +364,11 @@ Onlylogs.configure do |config|
236
364
  config.log_file_patterns = [
237
365
  # Allow all .log files in the log directory
238
366
  Rails.root.join("log/*.log"),
239
-
367
+
240
368
  # Allow specific pattern matches
241
369
  Rails.root.join("log/*production*.log"),
242
370
  Rails.root.join("log/*development*.log"),
243
-
371
+
244
372
  # Allow files in subdirectories
245
373
  Rails.root.join("log/**/*.log"),
246
374
  Rails.root.join("tmp/**/*.log")
@@ -249,20 +377,43 @@ end
249
377
  ```
250
378
 
251
379
  **Supported Glob Patterns:**
380
+
252
381
  - `*.log` - Matches all files ending with `.log` in the specified directory
253
382
  - `*production*.log` - Matches files containing "production" and ending with `.log`
254
383
  - `**/*.log` - Matches all `.log` files in the directory and all subdirectories
255
384
 
256
385
  **Important Notes:**
386
+
257
387
  - Patterns are directory-specific - `log/*.log` only matches files in the `log/` directory
258
388
  - Multiple patterns can be combined in the same configuration
259
389
 
390
+ ## Configuration
391
+
392
+ All options go into `Onlylogs.configure`. The most useful ones:
393
+
394
+ | Option | Default | What it does |
395
+ |--------------------------------|-------------------------------|-----------------------------------------------------|
396
+ | `log_file_patterns` | `log/<Rails.env>.log` | Files (or globs) the viewer may open. |
397
+ | `basic_auth_user`, `basic_auth_password` | env or credentials | Credentials of the basic authentication. |
398
+ | `disable_basic_authentication` | `false` | Turn off basic auth (use it with your own auth). |
399
+ | `parent_controller` | none | Controller the engine inherits from. |
400
+ | `viewer_parent_controller` | `parent_controller` | Same, but only for the viewer endpoints. |
401
+ | `authorize_log_file` | none | Per-user, per-file permission check. |
402
+ | `ripgrep_enabled` | `true` if `rg` exists | Use ripgrep instead of grep. |
403
+ | `editor` | from env | Editor used for file path links. |
404
+ | `max_line_matches` | `100000` | Maximum lines returned by a search. |
405
+ | `search_timeout` | `120` | Seconds after which a search is stopped. |
406
+ | `database_dir` | `".onlylogs"` | Folder (next to the logs) with the saved searche. |
407
+
408
+ See `lib/onlylogs/configuration.rb` for the details.
260
409
 
261
410
  ### Configuring Code Editor for File Path Links
262
411
 
263
- Onlylogs automatically detects file paths in log messages and converts them into clickable links that open in your preferred code editor.
412
+ Onlylogs automatically detects file paths in log messages and converts them into clickable links that open in your
413
+ preferred code editor.
264
414
 
265
- For a complete list of supported editors, see [app/models/onlylogs/file_path_parser.rb](app/models/onlylogs/file_path_parser.rb).
415
+ For a complete list of supported editors,
416
+ see [app/models/onlylogs/file_path_parser.rb](app/models/onlylogs/file_path_parser.rb).
266
417
 
267
418
  ```bash
268
419
  # env variables
@@ -284,25 +435,26 @@ Onlylogs.configure do |config|
284
435
  end
285
436
  ```
286
437
 
287
- #### Configuring Maximum Search Results
438
+ ### Configuring Maximum Search Results
288
439
 
289
- By default, onlylogs limits search results to 100,000 lines to prevent memory issues and ensure responsive performance. You can configure this limit based on your needs:
440
+ By default, onlylogs limits search results to 100,000 lines to prevent memory issues and ensure responsive performance.
441
+ You can configure this limit based on your needs:
290
442
 
291
443
  ```ruby
292
444
  # config/initializers/onlylogs.rb
293
445
  Onlylogs.configure do |config|
294
446
  # Set a custom limit (e.g., 50,000 lines)
295
447
  config.max_line_matches = 50_000
296
-
448
+
297
449
  # Or remove the limit entirely (use with caution)
298
450
  config.max_line_matches = nil
299
451
  end
300
452
  ```
301
453
 
302
- #### Bounding How Long a Viewer Search Can Run
454
+ ### Bounding How Long a Viewer Search Can Run
303
455
 
304
- Searches started from the log viewer stop after `search_timeout` seconds, keeping whatever they
305
- found and reporting that they did not reach the end of the file. The default is 120 seconds:
456
+ Searches started from the log viewer stop after `search_timeout` seconds, keeping whatever they found and reporting that
457
+ they did not reach the end of the file. The default is 120 seconds:
306
458
 
307
459
  ```ruby
308
460
  # config/initializers/onlylogs.rb
@@ -315,21 +467,30 @@ Onlylogs.configure do |config|
315
467
  end
316
468
  ```
317
469
 
318
- ### Filtering Log Lines with a Denylist
470
+ ## Customization
471
+
472
+ Onlylogs provides two ways to customize the appearance of the log viewer: CSS Variables and a complete style override.
473
+ Check the file [_log_container_styles.html.erb](app/views/onlylogs/shared/_log_container_styles.html.erb) for the
474
+ complete list of CSS variables.
475
+
476
+ ## Instrumentation
319
477
 
320
- The `Onlylogs::Formatter` supports a denylist: an array of regular expressions that prevents matching lines from being logged. This is useful for filtering out noisy or irrelevant entries like health checks or asset requests.
478
+ Every search sends an `ActiveSupport::Notifications` event called `search.onlylogs`. You can use it to measure how long
479
+ searches take, for example:
321
480
 
322
481
  ```ruby
323
- # config/environments/production.rb
324
- config.logger = Onlylogs::HttpLogger.new
325
- config.logger.formatter.denylist = [/health_check/, /ping/, /\.css\z/]
482
+ ActiveSupport::Notifications.subscribe("search.onlylogs") do |event|
483
+ Rails.logger.info "Search #{event.payload[:query]} took #{event.duration.round}ms, " \
484
+ "#{event.payload[:matches]} matches, timed out: #{event.payload[:timed_out]}"
485
+ end
326
486
  ```
327
487
 
328
- Any log message matching at least one pattern in the denylist will be silently dropped.
488
+ The payload contains `file_path`, `query`, `regexp`, `case_sensitive`, `start_position`, `end_position`,
489
+ `max_matches`, `matches` and `timed_out`.
329
490
 
330
491
  ## Development & Contributing
331
492
 
332
- You are more than welcome to help and contribute to this package.
493
+ You are more than welcome to help and contribute to this package.
333
494
 
334
495
  The app uses minitest and includes a dummy app, so getting started should be straightforward.
335
496
 
@@ -348,11 +509,13 @@ cd test/dummy
348
509
  bundle exec rails s
349
510
  ```
350
511
 
351
- The dummy app will be available at `http://localhost:3000` and onlylogs at `http://localhost:3000/onlylogs`.
512
+ The dummy app will be available at `http://onlylogs.localhost:3000` and onlylogs at
513
+ `http://onlylogs.localhost:3000/onlylogs`.
352
514
 
353
515
  ### Generating Test Logs
354
516
 
355
- To test onlylogs with live log data, use the continuous log writer script. This is especially useful for testing real-time log streaming and UI behavior.
517
+ To test onlylogs with live log data, use the continuous log writer script. This is especially useful for testing
518
+ real-time log streaming and UI behavior.
356
519
 
357
520
  ```bash
358
521
  # Generate 1 log entry every 2 seconds (default)
@@ -365,9 +528,12 @@ bin/continuous_log_writer 5 1
365
528
  bin/continuous_log_writer 10 3
366
529
  ```
367
530
 
368
- The script will write logs to `test/dummy/log/development.log`, which will appear in real-time in the onlylogs UI at `http://localhost:3000/onlylogs`.
531
+ The script will write logs to `test/dummy/log/development.log`, which will appear in real-time in the onlylogs UI at
532
+ `http://onlylogs.localhost:3000/onlylogs`.
369
533
 
370
- The lines are generated by `Onlylogs::ContinuousLogWriter`, in the same format a Rails app logging through `Onlylogs::Logger` produces. The script is a thin wrapper, so applications embedding onlylogs can reuse the writer against any file of their own:
534
+ The lines are generated by `Onlylogs::ContinuousLogWriter`, in the same format a Rails app logging through
535
+ `Onlylogs::Logger` produces. The script is a thin wrapper, so applications embedding onlylogs can reuse the writer
536
+ against any file of their own:
371
537
 
372
538
  ```ruby
373
539
  require "onlylogs/continuous_log_writer"
@@ -375,7 +541,8 @@ require "onlylogs/continuous_log_writer"
375
541
  Onlylogs::ContinuousLogWriter.new(some_log_path, logs_per_batch: 3, interval: 1).call
376
542
  ```
377
543
 
378
- It is plain Ruby (it does not need Rails to be booted) and is not loaded by `require "onlylogs"` — require it explicitly where you need it.
544
+ It is plain Ruby (it does not need Rails to be booted) and is not loaded by `require "onlylogs"` — require it explicitly
545
+ where you need it.
379
546
 
380
547
  **Example workflow:**
381
548
 
@@ -387,15 +554,18 @@ cd test/dummy && bundle exec rails s
387
554
  bin/continuous_log_writer 3 1
388
555
  ```
389
556
 
390
- Open `http://localhost:3000/onlylogs` in your browser and watch logs appear as the script writes them.
557
+ Open `http://onlylogs.localhost:3000/onlylogs` in your browser and watch logs appear as the script writes them.
391
558
 
392
559
  ### Latency Simulation
393
560
 
394
- For testing how onlylogs behaves under production-like network conditions, you can simulate latency for HTTP requests and WebSocket connections using the included latency simulation tool.
561
+ For testing how onlylogs behaves under production-like network conditions, you can simulate latency for HTTP requests
562
+ and WebSocket connections using the included latency simulation tool.
395
563
 
396
564
  **Parameters:**
565
+
397
566
  - **Latency** (default: 120ms): The base network delay added to all traffic
398
- - **Jitter** (default: 30ms): Random variation (±) applied to the latency (changes every 2 seconds to simulate real network conditions)
567
+ - **Jitter** (default: 30ms): Random variation (±) applied to the latency (changes every 2 seconds to simulate real
568
+ network conditions)
399
569
  - **Port** (default: 3000): The port to apply latency simulation to
400
570
 
401
571
  **Common scenarios:**
@@ -414,7 +584,6 @@ bin/simulate_latency enable 300/100
414
584
  bin/simulate_latency enable -p 8080
415
585
  ```
416
586
 
417
-
418
587
  **Testing and monitoring:**
419
588
 
420
589
  ```bash
@@ -428,11 +597,25 @@ bin/simulate_latency status
428
597
  bin/simulate_latency disable
429
598
  ```
430
599
 
431
- The `test` command will run 10 HTTP requests and 10 WebSocket connections, showing you the actual round-trip times and helping you verify the latency is working as expected.
600
+ The `test` command will run 10 HTTP requests and 10 WebSocket connections, showing you the actual round-trip times and
601
+ helping you verify the latency is working as expected.
602
+
603
+ ### Simulating a drain outage
604
+
605
+ To see how `HttpLogger` behaves when the drain is slow or broken, run a fake drain and a fake app in two terminals:
606
+
607
+ ```bash
608
+ bin/fake_drain # press a key to change its state: 2=200 5=500 3=503 4=413 h=hang s=slow r=reset d=down
609
+ bin/fake_app # 5 "Puma threads" logging like Rails requests through HttpLogger
610
+ ```
611
+
612
+ The dashboard shows request latency, CPU, memory, the queue, the circuit breaker, the spool and the drain counters.
613
+ A healthy app has a few % of CPU and no HTTP calls on request threads.
432
614
 
433
615
  ### Performance Testing
434
616
 
435
- Performance tests require large log files that are not included in the repository. You can download them using the provided script:
617
+ Performance tests require large log files that are not included in the repository. You can download them using the
618
+ provided script:
436
619
 
437
620
  ```bash
438
621
  bin/download_performance_fixtures
@@ -449,17 +632,8 @@ bin/rails test test/models/onlylogs/grep_performance_test.rb
449
632
 
450
633
  ### Plans for the future
451
634
 
452
- We believe that by simply analysing your logs you can also have a fancy errors report.
453
- Yes, correct. You don't need Sentry either.
454
-
455
- And you know what? You can get also performance reports.
456
-
457
- All of a sudden you are 100% free from external services for three more things:
458
-
459
- * logs
460
- * errors
461
- * performance
462
-
635
+ We believe that by simply analysing your logs you can also get error reports and performance reports. The goal is to
636
+ need no external service for logs, errors and performance.
463
637
 
464
638
  ## License
465
639