errmine 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +17 -0
- data/LICENSE +21 -0
- data/README.md +252 -0
- data/lib/errmine/middleware.rb +61 -0
- data/lib/errmine/notifier.rb +395 -0
- data/lib/errmine/railtie.rb +39 -0
- data/lib/errmine/version.rb +6 -0
- data/lib/errmine.rb +99 -0
- metadata +52 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 8cb0080b9e4a44a6e2003052e2363ffff60518821efa97420ab16b5dd53083bc
|
|
4
|
+
data.tar.gz: d5db59abdf0c8d23b1aaa2e7b009ff21efaaf21bea63c702b535e45e000c7ca2
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 10d3e4c4db4f7559fe0971f81cab347aec5f867eea1b39efdb98816fc0c0a595d8c6a774b3f1c20accb073e24b7c6bbbc9c5cede8362231fd627f3e907ef97c6
|
|
7
|
+
data.tar.gz: ff53f1241f18c703a647bb9dc3e1fc8cb7a82f5b652bbb08b4573d5376dc85271f3f280eddee7e057c44ae1512a633025fb117b2b13e3b6427b83705d1c03a85
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Errmine changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 (2025-12-23)
|
|
4
|
+
|
|
5
|
+
Initial release of Errmine - dead simple exception tracking for Redmine.
|
|
6
|
+
|
|
7
|
+
- [Feature] Automatic exception tracking to Redmine via REST API.
|
|
8
|
+
- [Feature] Issue deduplication using 8-character MD5 checksums based on exception class, message, and first application backtrace line.
|
|
9
|
+
- [Feature] Occurrence counter in issue subject (`[checksum][count] ExceptionClass: message`).
|
|
10
|
+
- [Feature] Rate limiting with configurable cooldown to prevent Redmine flooding.
|
|
11
|
+
- [Feature] Rails 7+ Error Reporting API integration via Railtie.
|
|
12
|
+
- [Feature] Rack middleware for exception handling in all Ruby web applications.
|
|
13
|
+
- [Feature] Manual notification API with custom context support.
|
|
14
|
+
- [Feature] Environment variable configuration (`ERRMINE_REDMINE_URL`, `ERRMINE_API_KEY`, `ERRMINE_PROJECT`, `ERRMINE_APP_NAME`).
|
|
15
|
+
- [Feature] Thread-safe in-memory cache with automatic cleanup.
|
|
16
|
+
- [Feature] Fail-safe error handling - never crashes your application.
|
|
17
|
+
- [Feature] Zero runtime dependencies - uses only Ruby stdlib.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Maciej Mensfeld
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,252 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="misc/logo.png" alt="Errmine" width="600"/>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
# Errmine
|
|
6
|
+
|
|
7
|
+
[](https://github.com/mensfeld/errmine/actions?query=workflow%3Aci)
|
|
8
|
+
[](http://badge.fury.io/rb/errmine)
|
|
9
|
+
|
|
10
|
+
Dead simple, zero-dependency exception tracking for Redmine.
|
|
11
|
+
|
|
12
|
+
Errmine automatically creates and updates Redmine issues from Ruby/Rails exceptions. When an error occurs, it creates a new issue. When the same error occurs again, it increments a counter and adds a journal note instead of creating duplicates.
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
[a1b2c3d4][47] NoMethodError: undefined method 'foo' for nil...
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## Why Errmine?
|
|
19
|
+
|
|
20
|
+
If you already use Redmine for project management, Errmine lets you track production errors without adding another service to your stack.
|
|
21
|
+
|
|
22
|
+
**vs. Sentry, Honeybadger, Airbrake, etc.**
|
|
23
|
+
- No external service, no monthly fees, no data leaving your infrastructure
|
|
24
|
+
- Errors live alongside your tasks and documentation in Redmine
|
|
25
|
+
- No new UI to learn - just Redmine issues
|
|
26
|
+
|
|
27
|
+
**vs. Exception Notification gem**
|
|
28
|
+
- Automatic deduplication - same error updates existing issue instead of creating duplicates
|
|
29
|
+
- Rate limiting prevents inbox/Redmine flooding during error storms
|
|
30
|
+
- Occurrence counting shows error frequency at a glance
|
|
31
|
+
|
|
32
|
+
**vs. Rolling your own**
|
|
33
|
+
- Zero dependencies - uses only Ruby stdlib
|
|
34
|
+
- Handles edge cases: rate limiting, timeouts, thread safety, fail-safe error handling
|
|
35
|
+
- Works out of the box with Rails 7+ Error Reporting API
|
|
36
|
+
|
|
37
|
+
## Features
|
|
38
|
+
|
|
39
|
+
- **Zero dependencies** - Uses only Ruby stdlib (net/http, json, digest, uri)
|
|
40
|
+
- **Automatic deduplication** - Same errors update existing issues instead of creating duplicates
|
|
41
|
+
- **Rate limiting** - Prevents flooding Redmine during error loops
|
|
42
|
+
- **Rails integration** - Works with Rails 7+ Error Reporting API or as Rack middleware
|
|
43
|
+
- **Thread-safe** - Safe to use in multi-threaded environments
|
|
44
|
+
- **Fail-safe** - Never crashes your application, logs errors to stderr
|
|
45
|
+
|
|
46
|
+
## Installation
|
|
47
|
+
|
|
48
|
+
Add to your Gemfile:
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
gem 'errmine'
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Then run:
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
bundle install
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## Configuration
|
|
61
|
+
|
|
62
|
+
### Environment Variables
|
|
63
|
+
|
|
64
|
+
Errmine reads these environment variables as defaults:
|
|
65
|
+
|
|
66
|
+
- `ERRMINE_REDMINE_URL` - Redmine server URL
|
|
67
|
+
- `ERRMINE_API_KEY` - API key for authentication
|
|
68
|
+
- `ERRMINE_PROJECT` - Project identifier (default: `'bug-tracker'`)
|
|
69
|
+
- `ERRMINE_APP_NAME` - Application name (default: `'unknown'`)
|
|
70
|
+
|
|
71
|
+
### Rails
|
|
72
|
+
|
|
73
|
+
Create an initializer:
|
|
74
|
+
|
|
75
|
+
```ruby
|
|
76
|
+
# config/initializers/errmine.rb
|
|
77
|
+
Errmine.configure do |config|
|
|
78
|
+
config.redmine_url = ENV.fetch('REDMINE_URL')
|
|
79
|
+
config.api_key = ENV.fetch('REDMINE_API_KEY')
|
|
80
|
+
config.project_id = 'my-project'
|
|
81
|
+
config.tracker_id = 1 # Default: 1 (Bug)
|
|
82
|
+
config.app_name = Rails.application.class.module_parent_name
|
|
83
|
+
config.cooldown = 300 # Default: 300 seconds
|
|
84
|
+
end
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### Plain Ruby
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
require 'errmine'
|
|
91
|
+
|
|
92
|
+
Errmine.configure do |config|
|
|
93
|
+
config.redmine_url = 'https://redmine.example.com'
|
|
94
|
+
config.api_key = 'your-redmine-api-key'
|
|
95
|
+
config.project_id = 'my-project'
|
|
96
|
+
end
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### Configuration Options
|
|
100
|
+
|
|
101
|
+
| Option | Default | Description |
|
|
102
|
+
|--------|---------|-------------|
|
|
103
|
+
| `redmine_url` | `ENV['ERRMINE_REDMINE_URL']` | Redmine server URL (required) |
|
|
104
|
+
| `api_key` | `ENV['ERRMINE_API_KEY']` | API key for authentication (required) |
|
|
105
|
+
| `project_id` | `ENV['ERRMINE_PROJECT']` or `'bug-tracker'` | Redmine project identifier |
|
|
106
|
+
| `tracker_id` | `1` | Tracker ID (usually 1 = Bug) |
|
|
107
|
+
| `app_name` | `ENV['ERRMINE_APP_NAME']` or `'unknown'` | Application name shown in issues |
|
|
108
|
+
| `enabled` | `true` | Enable/disable notifications |
|
|
109
|
+
| `cooldown` | `300` | Seconds between same-error notifications |
|
|
110
|
+
|
|
111
|
+
## Usage
|
|
112
|
+
|
|
113
|
+
### Rails 7+ (Automatic)
|
|
114
|
+
|
|
115
|
+
Errmine automatically subscribes to Rails' error reporting API via a Railtie. Unhandled exceptions are reported automatically. No additional setup required.
|
|
116
|
+
|
|
117
|
+
### Rack Middleware
|
|
118
|
+
|
|
119
|
+
For all Rails versions or any Rack-based application:
|
|
120
|
+
|
|
121
|
+
```ruby
|
|
122
|
+
# config/application.rb (Rails)
|
|
123
|
+
config.middleware.use Errmine::Middleware
|
|
124
|
+
|
|
125
|
+
# or config.ru (Sinatra, etc.)
|
|
126
|
+
use Errmine::Middleware
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
The middleware captures request context (URL, HTTP method, user via Warden).
|
|
130
|
+
|
|
131
|
+
### Manual Notification
|
|
132
|
+
|
|
133
|
+
Report exceptions manually with custom context:
|
|
134
|
+
|
|
135
|
+
```ruby
|
|
136
|
+
begin
|
|
137
|
+
risky_operation
|
|
138
|
+
rescue => e
|
|
139
|
+
Errmine.notify(e, {
|
|
140
|
+
url: request.url,
|
|
141
|
+
user: current_user&.email,
|
|
142
|
+
custom_field: 'any value'
|
|
143
|
+
})
|
|
144
|
+
raise
|
|
145
|
+
end
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### rescue_from in Controllers
|
|
149
|
+
|
|
150
|
+
```ruby
|
|
151
|
+
class ApplicationController < ActionController::Base
|
|
152
|
+
rescue_from StandardError do |e|
|
|
153
|
+
Errmine.notify(e, {
|
|
154
|
+
url: request.url,
|
|
155
|
+
user: current_user&.email
|
|
156
|
+
})
|
|
157
|
+
raise e
|
|
158
|
+
end
|
|
159
|
+
end
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
### Disabling Notifications
|
|
163
|
+
|
|
164
|
+
```ruby
|
|
165
|
+
Errmine.configure do |config|
|
|
166
|
+
config.enabled = Rails.env.production?
|
|
167
|
+
end
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
## How It Works
|
|
171
|
+
|
|
172
|
+
### Checksum Generation
|
|
173
|
+
|
|
174
|
+
Each exception gets an 8-character MD5 checksum based on:
|
|
175
|
+
- Exception class name
|
|
176
|
+
- Exception message
|
|
177
|
+
- First application backtrace line (containing `/app/`)
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
MD5("NoMethodError:undefined method 'foo':app/controllers/users_controller.rb:45")[0..7]
|
|
181
|
+
# => "a1b2c3d4"
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
### Deduplication
|
|
185
|
+
|
|
186
|
+
1. Search Redmine for open issues containing `[{checksum}]` in the subject
|
|
187
|
+
2. If found: increment counter, add journal note with timestamp and backtrace
|
|
188
|
+
3. If not found: create new issue
|
|
189
|
+
|
|
190
|
+
### Rate Limiting
|
|
191
|
+
|
|
192
|
+
To prevent flooding Redmine during error loops:
|
|
193
|
+
|
|
194
|
+
- Each checksum is cached with its last occurrence time
|
|
195
|
+
- Same error won't hit Redmine more than once per cooldown period (default: 5 minutes)
|
|
196
|
+
- Cache is automatically cleaned when it exceeds 500 entries
|
|
197
|
+
|
|
198
|
+
## Redmine Setup
|
|
199
|
+
|
|
200
|
+
1. **Enable REST API**: Administration > Settings > API > Enable REST web service
|
|
201
|
+
2. **Create an API key**: My Account > API access key > Show/Reset
|
|
202
|
+
3. **Create a project** for error tracking (or use existing one)
|
|
203
|
+
4. **Permissions**: The API user needs:
|
|
204
|
+
- View issues
|
|
205
|
+
- Add issues
|
|
206
|
+
- Edit issues
|
|
207
|
+
- Add notes
|
|
208
|
+
|
|
209
|
+
## Issue Format
|
|
210
|
+
|
|
211
|
+
### Subject
|
|
212
|
+
|
|
213
|
+
```
|
|
214
|
+
[{checksum}][{count}] {ExceptionClass}: {truncated message}
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Example: `[a1b2c3d4][47] NoMethodError: undefined method 'foo' for nil...`
|
|
218
|
+
|
|
219
|
+
### Description (Textile format)
|
|
220
|
+
|
|
221
|
+
```textile
|
|
222
|
+
**Exception:** @NoMethodError@
|
|
223
|
+
**Message:** undefined method 'foo' for nil:NilClass
|
|
224
|
+
**App:** my-app
|
|
225
|
+
**First seen:** 2025-01-15 10:30:00
|
|
226
|
+
|
|
227
|
+
**URL:** /users/123
|
|
228
|
+
**User:** user@example.com
|
|
229
|
+
|
|
230
|
+
h3. Backtrace
|
|
231
|
+
|
|
232
|
+
<pre>
|
|
233
|
+
app/controllers/users_controller.rb:45:in `show'
|
|
234
|
+
app/controllers/application_controller.rb:12:in `authenticate'
|
|
235
|
+
</pre>
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
### Journal Note (on subsequent occurrences)
|
|
239
|
+
|
|
240
|
+
```textile
|
|
241
|
+
Occurred again (*47x*) at 2025-01-15 10:35:00
|
|
242
|
+
|
|
243
|
+
URL: /users/456
|
|
244
|
+
|
|
245
|
+
<pre>
|
|
246
|
+
app/controllers/users_controller.rb:45:in `show'
|
|
247
|
+
</pre>
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
## License
|
|
251
|
+
|
|
252
|
+
MIT License. See [LICENSE](LICENSE) for details.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# @see Errmine
|
|
4
|
+
module Errmine
|
|
5
|
+
# Rack middleware that catches exceptions and reports them to Redmine.
|
|
6
|
+
# Re-raises the exception after reporting to allow normal error handling.
|
|
7
|
+
class Middleware
|
|
8
|
+
# Creates a new middleware instance
|
|
9
|
+
#
|
|
10
|
+
# @param app [#call] the Rack application
|
|
11
|
+
def initialize(app)
|
|
12
|
+
@app = app
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
# Processes the request and catches any exceptions
|
|
16
|
+
#
|
|
17
|
+
# @param env [Hash] the Rack environment
|
|
18
|
+
# @return [Array] the Rack response
|
|
19
|
+
# @raise [Exception] re-raises any caught exception after reporting
|
|
20
|
+
def call(env)
|
|
21
|
+
@app.call(env)
|
|
22
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
23
|
+
notify_exception(e, env)
|
|
24
|
+
raise
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
private
|
|
28
|
+
|
|
29
|
+
# Sends exception notification to Redmine
|
|
30
|
+
#
|
|
31
|
+
# @param exception [Exception] the caught exception
|
|
32
|
+
# @param env [Hash] the Rack environment
|
|
33
|
+
def notify_exception(exception, env)
|
|
34
|
+
context = build_context(env)
|
|
35
|
+
Errmine.notify(exception, context)
|
|
36
|
+
rescue StandardError => e
|
|
37
|
+
warn "[Errmine] Middleware error: #{e.message}"
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# Builds context hash from Rack environment
|
|
41
|
+
#
|
|
42
|
+
# @param env [Hash] the Rack environment
|
|
43
|
+
# @return [Hash] context with url, method, and user info
|
|
44
|
+
def build_context(env)
|
|
45
|
+
context = {}
|
|
46
|
+
|
|
47
|
+
request_uri = env['REQUEST_URI'] || env['PATH_INFO']
|
|
48
|
+
context[:url] = request_uri if request_uri
|
|
49
|
+
|
|
50
|
+
request_method = env['REQUEST_METHOD']
|
|
51
|
+
context[:method] = request_method if request_method
|
|
52
|
+
|
|
53
|
+
if defined?(env['warden']) && env['warden']&.user
|
|
54
|
+
user = env['warden'].user
|
|
55
|
+
context[:user] = user.respond_to?(:email) ? user.email : user.to_s
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
context
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
|
@@ -0,0 +1,395 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'net/http'
|
|
4
|
+
require 'uri'
|
|
5
|
+
require 'json'
|
|
6
|
+
require 'digest'
|
|
7
|
+
require 'singleton'
|
|
8
|
+
|
|
9
|
+
# @see Errmine
|
|
10
|
+
module Errmine
|
|
11
|
+
# Core notifier that handles exception reporting to Redmine.
|
|
12
|
+
# Manages checksum generation, rate limiting, and API communication.
|
|
13
|
+
class Notifier
|
|
14
|
+
include Singleton
|
|
15
|
+
|
|
16
|
+
# HTTP connection timeout in seconds
|
|
17
|
+
CONNECT_TIMEOUT = 5
|
|
18
|
+
|
|
19
|
+
# HTTP read timeout in seconds
|
|
20
|
+
READ_TIMEOUT = 10
|
|
21
|
+
|
|
22
|
+
# Maximum number of entries in the rate limit cache
|
|
23
|
+
MAX_CACHE_SIZE = 500
|
|
24
|
+
|
|
25
|
+
# Maximum length of exception message in issue subject
|
|
26
|
+
SUBJECT_MESSAGE_LENGTH = 60
|
|
27
|
+
|
|
28
|
+
# Initializes the notifier with empty cache
|
|
29
|
+
def initialize
|
|
30
|
+
@cache = {}
|
|
31
|
+
@mutex = Mutex.new
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Notifies Redmine about an exception
|
|
35
|
+
#
|
|
36
|
+
# @param exception [Exception] the exception to report
|
|
37
|
+
# @param context [Hash] additional context (url, user, etc.)
|
|
38
|
+
# @return [Hash, nil] the created/updated issue or nil on failure
|
|
39
|
+
def notify(exception, context = {})
|
|
40
|
+
checksum = generate_checksum(exception)
|
|
41
|
+
|
|
42
|
+
return nil if rate_limited?(checksum)
|
|
43
|
+
|
|
44
|
+
record_occurrence(checksum)
|
|
45
|
+
|
|
46
|
+
result = find_existing_issue(checksum)
|
|
47
|
+
|
|
48
|
+
case result
|
|
49
|
+
when :error
|
|
50
|
+
nil
|
|
51
|
+
when nil
|
|
52
|
+
create_issue(exception, context, checksum)
|
|
53
|
+
else
|
|
54
|
+
update_issue(result, exception, context)
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
# Clears the rate limit cache
|
|
59
|
+
#
|
|
60
|
+
# @return [void]
|
|
61
|
+
def reset_cache!
|
|
62
|
+
@mutex.synchronize { @cache.clear }
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
private
|
|
66
|
+
|
|
67
|
+
# Returns the Errmine configuration
|
|
68
|
+
#
|
|
69
|
+
# @return [Errmine::Configuration]
|
|
70
|
+
def config
|
|
71
|
+
Errmine.configuration
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
# Generates an 8-character checksum for the exception
|
|
75
|
+
#
|
|
76
|
+
# @param exception [Exception]
|
|
77
|
+
# @return [String]
|
|
78
|
+
def generate_checksum(exception)
|
|
79
|
+
first_app_line = first_app_backtrace_line(exception)
|
|
80
|
+
data = "#{exception.class}:#{exception.message}:#{first_app_line}"
|
|
81
|
+
Digest::MD5.hexdigest(data)[0, 8]
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# Finds the first backtrace line from the application
|
|
85
|
+
#
|
|
86
|
+
# @param exception [Exception]
|
|
87
|
+
# @return [String]
|
|
88
|
+
def first_app_backtrace_line(exception)
|
|
89
|
+
return '' unless exception.backtrace
|
|
90
|
+
|
|
91
|
+
exception.backtrace.find { |line| line.include?('/app/') } ||
|
|
92
|
+
exception.backtrace.first ||
|
|
93
|
+
''
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
# Checks if the checksum is rate limited
|
|
97
|
+
#
|
|
98
|
+
# @param checksum [String]
|
|
99
|
+
# @return [Boolean]
|
|
100
|
+
def rate_limited?(checksum)
|
|
101
|
+
@mutex.synchronize do
|
|
102
|
+
last_seen = @cache[checksum]
|
|
103
|
+
return false unless last_seen
|
|
104
|
+
|
|
105
|
+
Time.now - last_seen < config.cooldown
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# Records the occurrence of a checksum
|
|
110
|
+
#
|
|
111
|
+
# @param checksum [String]
|
|
112
|
+
# @return [void]
|
|
113
|
+
def record_occurrence(checksum)
|
|
114
|
+
@mutex.synchronize do
|
|
115
|
+
cleanup_cache if @cache.size >= MAX_CACHE_SIZE
|
|
116
|
+
@cache[checksum] = Time.now
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
# Cleans up old entries from the cache
|
|
121
|
+
#
|
|
122
|
+
# @return [void]
|
|
123
|
+
def cleanup_cache
|
|
124
|
+
cutoff = Time.now - config.cooldown
|
|
125
|
+
@cache.delete_if { |_, time| time < cutoff }
|
|
126
|
+
|
|
127
|
+
return unless @cache.size >= MAX_CACHE_SIZE
|
|
128
|
+
|
|
129
|
+
sorted = @cache.sort_by { |_, time| time }
|
|
130
|
+
to_remove = sorted.first(@cache.size - (MAX_CACHE_SIZE / 2))
|
|
131
|
+
to_remove.each_key { |key| @cache.delete(key) }
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
# Finds an existing open issue with the given checksum
|
|
135
|
+
#
|
|
136
|
+
# @param checksum [String]
|
|
137
|
+
# @return [Hash, Symbol, nil] the issue hash, :error on failure, or nil if not found
|
|
138
|
+
def find_existing_issue(checksum)
|
|
139
|
+
uri = build_uri('/issues.json')
|
|
140
|
+
uri.query = URI.encode_www_form(
|
|
141
|
+
project_id: config.project_id,
|
|
142
|
+
'subject' => "~[#{checksum}]",
|
|
143
|
+
status_id: 'open'
|
|
144
|
+
)
|
|
145
|
+
|
|
146
|
+
response = http_get(uri)
|
|
147
|
+
return :error unless response
|
|
148
|
+
|
|
149
|
+
data = JSON.parse(response.body)
|
|
150
|
+
issues = data['issues'] || []
|
|
151
|
+
|
|
152
|
+
issues.find { |issue| issue['subject']&.include?("[#{checksum}]") }
|
|
153
|
+
rescue JSON::ParserError => e
|
|
154
|
+
warn "[Errmine] Failed to parse response: #{e.message}"
|
|
155
|
+
:error
|
|
156
|
+
end
|
|
157
|
+
|
|
158
|
+
# Creates a new issue in Redmine
|
|
159
|
+
#
|
|
160
|
+
# @param exception [Exception]
|
|
161
|
+
# @param context [Hash]
|
|
162
|
+
# @param checksum [String]
|
|
163
|
+
# @return [Hash, nil]
|
|
164
|
+
def create_issue(exception, context, checksum)
|
|
165
|
+
uri = build_uri('/issues.json')
|
|
166
|
+
|
|
167
|
+
subject = build_subject(checksum, 1, exception)
|
|
168
|
+
description = build_description(exception, context)
|
|
169
|
+
|
|
170
|
+
payload = {
|
|
171
|
+
issue: {
|
|
172
|
+
project_id: config.project_id,
|
|
173
|
+
tracker_id: config.tracker_id,
|
|
174
|
+
subject: subject,
|
|
175
|
+
description: description
|
|
176
|
+
}
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
response = http_post(uri, payload)
|
|
180
|
+
return nil unless response
|
|
181
|
+
|
|
182
|
+
data = JSON.parse(response.body)
|
|
183
|
+
data['issue']
|
|
184
|
+
rescue JSON::ParserError => e
|
|
185
|
+
warn "[Errmine] Failed to parse response: #{e.message}"
|
|
186
|
+
nil
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
# Updates an existing issue with new occurrence
|
|
190
|
+
#
|
|
191
|
+
# @param issue [Hash]
|
|
192
|
+
# @param exception [Exception]
|
|
193
|
+
# @param context [Hash]
|
|
194
|
+
# @return [Net::HTTPResponse, nil]
|
|
195
|
+
def update_issue(issue, exception, context)
|
|
196
|
+
issue_id = issue['id']
|
|
197
|
+
current_subject = issue['subject']
|
|
198
|
+
|
|
199
|
+
new_count = extract_count(current_subject) + 1
|
|
200
|
+
checksum = extract_checksum(current_subject)
|
|
201
|
+
|
|
202
|
+
new_subject = build_subject(checksum, new_count, exception)
|
|
203
|
+
notes = build_journal_note(new_count, context, exception)
|
|
204
|
+
|
|
205
|
+
uri = build_uri("/issues/#{issue_id}.json")
|
|
206
|
+
|
|
207
|
+
payload = {
|
|
208
|
+
issue: {
|
|
209
|
+
subject: new_subject,
|
|
210
|
+
notes: notes
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
http_put(uri, payload)
|
|
215
|
+
end
|
|
216
|
+
|
|
217
|
+
# Builds the issue subject line
|
|
218
|
+
#
|
|
219
|
+
# @param checksum [String]
|
|
220
|
+
# @param count [Integer]
|
|
221
|
+
# @param exception [Exception]
|
|
222
|
+
# @return [String]
|
|
223
|
+
def build_subject(checksum, count, exception)
|
|
224
|
+
message = exception.message.to_s
|
|
225
|
+
truncated = message.length > SUBJECT_MESSAGE_LENGTH ? "#{message[0, SUBJECT_MESSAGE_LENGTH]}..." : message
|
|
226
|
+
truncated = truncated.gsub(/[\r\n]+/, ' ').strip
|
|
227
|
+
|
|
228
|
+
"[#{checksum}][#{count}] #{exception.class}: #{truncated}"
|
|
229
|
+
end
|
|
230
|
+
|
|
231
|
+
# Builds the issue description in Textile format
|
|
232
|
+
#
|
|
233
|
+
# @param exception [Exception]
|
|
234
|
+
# @param context [Hash]
|
|
235
|
+
# @return [String]
|
|
236
|
+
def build_description(exception, context)
|
|
237
|
+
lines = []
|
|
238
|
+
lines << "**Exception:** @#{exception.class}@"
|
|
239
|
+
lines << "**Message:** #{exception.message}"
|
|
240
|
+
lines << "**App:** #{config.app_name}"
|
|
241
|
+
lines << "**First seen:** #{Time.now.strftime('%Y-%m-%d %H:%M:%S')}"
|
|
242
|
+
lines << ''
|
|
243
|
+
|
|
244
|
+
lines << "**URL:** #{context[:url]}" if context[:url]
|
|
245
|
+
|
|
246
|
+
lines << "**User:** #{context[:user]}" if context[:user]
|
|
247
|
+
|
|
248
|
+
context.each do |key, value|
|
|
249
|
+
next if %i[url user].include?(key)
|
|
250
|
+
|
|
251
|
+
lines << "**#{key.to_s.capitalize}:** #{value}"
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
lines << ''
|
|
255
|
+
lines << 'h3. Backtrace'
|
|
256
|
+
lines << ''
|
|
257
|
+
lines << '<pre>'
|
|
258
|
+
lines << format_backtrace(exception)
|
|
259
|
+
lines << '</pre>'
|
|
260
|
+
|
|
261
|
+
lines.join("\n")
|
|
262
|
+
end
|
|
263
|
+
|
|
264
|
+
# Builds a journal note for issue updates
|
|
265
|
+
#
|
|
266
|
+
# @param count [Integer]
|
|
267
|
+
# @param context [Hash]
|
|
268
|
+
# @param exception [Exception]
|
|
269
|
+
# @return [String]
|
|
270
|
+
def build_journal_note(count, context, exception)
|
|
271
|
+
lines = []
|
|
272
|
+
lines << "Occurred again (*#{count}x*) at #{Time.now.strftime('%Y-%m-%d %H:%M:%S')}"
|
|
273
|
+
lines << ''
|
|
274
|
+
|
|
275
|
+
lines << "URL: #{context[:url]}" if context[:url]
|
|
276
|
+
lines << "User: #{context[:user]}" if context[:user]
|
|
277
|
+
|
|
278
|
+
lines << ''
|
|
279
|
+
lines << '<pre>'
|
|
280
|
+
lines << format_backtrace(exception, limit: 10)
|
|
281
|
+
lines << '</pre>'
|
|
282
|
+
|
|
283
|
+
lines.join("\n")
|
|
284
|
+
end
|
|
285
|
+
|
|
286
|
+
# Formats the exception backtrace
|
|
287
|
+
#
|
|
288
|
+
# @param exception [Exception]
|
|
289
|
+
# @param limit [Integer]
|
|
290
|
+
# @return [String]
|
|
291
|
+
def format_backtrace(exception, limit: 20)
|
|
292
|
+
return 'No backtrace available' unless exception.backtrace
|
|
293
|
+
|
|
294
|
+
exception.backtrace.first(limit).join("\n")
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
# Extracts the occurrence count from issue subject
|
|
298
|
+
#
|
|
299
|
+
# @param subject [String, nil]
|
|
300
|
+
# @return [Integer]
|
|
301
|
+
def extract_count(subject)
|
|
302
|
+
match = subject&.match(/\]\[(\d+)\]/)
|
|
303
|
+
match ? match[1].to_i : 0
|
|
304
|
+
end
|
|
305
|
+
|
|
306
|
+
# Extracts the checksum from issue subject
|
|
307
|
+
#
|
|
308
|
+
# @param subject [String, nil]
|
|
309
|
+
# @return [String]
|
|
310
|
+
def extract_checksum(subject)
|
|
311
|
+
match = subject&.match(/\[([a-f0-9]{8})\]/)
|
|
312
|
+
match ? match[1] : ''
|
|
313
|
+
end
|
|
314
|
+
|
|
315
|
+
# Builds a URI for the Redmine API
|
|
316
|
+
#
|
|
317
|
+
# @param path [String]
|
|
318
|
+
# @return [URI]
|
|
319
|
+
def build_uri(path)
|
|
320
|
+
base = config.redmine_url.chomp('/')
|
|
321
|
+
URI.parse("#{base}#{path}")
|
|
322
|
+
end
|
|
323
|
+
|
|
324
|
+
# Performs an HTTP GET request
|
|
325
|
+
#
|
|
326
|
+
# @param uri [URI]
|
|
327
|
+
# @return [Net::HTTPResponse, nil]
|
|
328
|
+
def http_get(uri)
|
|
329
|
+
http_request(uri) do |http|
|
|
330
|
+
request = Net::HTTP::Get.new(uri)
|
|
331
|
+
request['X-Redmine-API-Key'] = config.api_key
|
|
332
|
+
http.request(request)
|
|
333
|
+
end
|
|
334
|
+
end
|
|
335
|
+
|
|
336
|
+
# Performs an HTTP POST request
|
|
337
|
+
#
|
|
338
|
+
# @param uri [URI]
|
|
339
|
+
# @param payload [Hash]
|
|
340
|
+
# @return [Net::HTTPResponse, nil]
|
|
341
|
+
def http_post(uri, payload)
|
|
342
|
+
http_request(uri) do |http|
|
|
343
|
+
request = Net::HTTP::Post.new(uri)
|
|
344
|
+
request['X-Redmine-API-Key'] = config.api_key
|
|
345
|
+
request['Content-Type'] = 'application/json'
|
|
346
|
+
request.body = JSON.generate(payload)
|
|
347
|
+
http.request(request)
|
|
348
|
+
end
|
|
349
|
+
end
|
|
350
|
+
|
|
351
|
+
# Performs an HTTP PUT request
|
|
352
|
+
#
|
|
353
|
+
# @param uri [URI]
|
|
354
|
+
# @param payload [Hash]
|
|
355
|
+
# @return [Net::HTTPResponse, nil]
|
|
356
|
+
def http_put(uri, payload)
|
|
357
|
+
http_request(uri) do |http|
|
|
358
|
+
request = Net::HTTP::Put.new(uri)
|
|
359
|
+
request['X-Redmine-API-Key'] = config.api_key
|
|
360
|
+
request['Content-Type'] = 'application/json'
|
|
361
|
+
request.body = JSON.generate(payload)
|
|
362
|
+
http.request(request)
|
|
363
|
+
end
|
|
364
|
+
end
|
|
365
|
+
|
|
366
|
+
# Executes an HTTP request with error handling
|
|
367
|
+
#
|
|
368
|
+
# @param uri [URI]
|
|
369
|
+
# @return [Net::HTTPResponse, nil]
|
|
370
|
+
def http_request(uri)
|
|
371
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
372
|
+
http.use_ssl = uri.scheme == 'https'
|
|
373
|
+
http.open_timeout = CONNECT_TIMEOUT
|
|
374
|
+
http.read_timeout = READ_TIMEOUT
|
|
375
|
+
|
|
376
|
+
response = yield(http)
|
|
377
|
+
|
|
378
|
+
unless response.is_a?(Net::HTTPSuccess)
|
|
379
|
+
warn "[Errmine] HTTP #{response.code}: #{response.message}"
|
|
380
|
+
return nil
|
|
381
|
+
end
|
|
382
|
+
|
|
383
|
+
response
|
|
384
|
+
rescue Net::OpenTimeout, Net::ReadTimeout => e
|
|
385
|
+
warn "[Errmine] Timeout: #{e.message}"
|
|
386
|
+
nil
|
|
387
|
+
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError => e
|
|
388
|
+
warn "[Errmine] Connection error: #{e.message}"
|
|
389
|
+
nil
|
|
390
|
+
rescue StandardError => e
|
|
391
|
+
warn "[Errmine] HTTP error: #{e.message}"
|
|
392
|
+
nil
|
|
393
|
+
end
|
|
394
|
+
end
|
|
395
|
+
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative 'middleware'
|
|
4
|
+
|
|
5
|
+
# @see Errmine
|
|
6
|
+
module Errmine
|
|
7
|
+
# Rails integration via Railtie.
|
|
8
|
+
# Automatically subscribes to Rails 7+ Error Reporting API when available.
|
|
9
|
+
class Railtie < Rails::Railtie
|
|
10
|
+
initializer 'errmine.configure_rails_initialization' do |_app|
|
|
11
|
+
Rails.error.subscribe(ErrorSubscriber.new) if Rails.version >= '7.0'
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# Error subscriber for Rails 7+ Error Reporting API
|
|
15
|
+
class ErrorSubscriber
|
|
16
|
+
# Called by Rails when an error is reported
|
|
17
|
+
#
|
|
18
|
+
# @param error [Exception] the reported error
|
|
19
|
+
# @param handled [Boolean] whether the error was handled
|
|
20
|
+
# @param severity [Symbol] the error severity
|
|
21
|
+
# @param context [Hash]
|
|
22
|
+
# @param source [String, nil] the error source
|
|
23
|
+
# @return [void]
|
|
24
|
+
def report(error, handled:, severity:, context: {}, source: nil)
|
|
25
|
+
return if handled
|
|
26
|
+
|
|
27
|
+
errmine_context = {}
|
|
28
|
+
errmine_context[:url] = context[:url] if context[:url]
|
|
29
|
+
errmine_context[:user] = context[:user]&.to_s if context[:user]
|
|
30
|
+
errmine_context[:source] = source if source
|
|
31
|
+
errmine_context[:severity] = severity if severity
|
|
32
|
+
|
|
33
|
+
Errmine.notify(error, errmine_context)
|
|
34
|
+
rescue StandardError => e
|
|
35
|
+
warn "[Errmine] Error subscriber failed: #{e.message}"
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
end
|
data/lib/errmine.rb
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative 'errmine/version'
|
|
4
|
+
require_relative 'errmine/notifier'
|
|
5
|
+
|
|
6
|
+
# Dead simple exception tracking for Redmine.
|
|
7
|
+
# Automatically creates and updates Redmine issues from Ruby/Rails exceptions.
|
|
8
|
+
module Errmine
|
|
9
|
+
# Base error class for Errmine-specific errors
|
|
10
|
+
class Error < StandardError; end
|
|
11
|
+
|
|
12
|
+
# Raised when configuration is invalid
|
|
13
|
+
class ConfigurationError < Error; end
|
|
14
|
+
|
|
15
|
+
# Configuration class for Errmine settings
|
|
16
|
+
class Configuration
|
|
17
|
+
# @return [String, nil] Redmine server URL
|
|
18
|
+
attr_accessor :redmine_url
|
|
19
|
+
|
|
20
|
+
# @return [String, nil] Redmine API key
|
|
21
|
+
attr_accessor :api_key
|
|
22
|
+
|
|
23
|
+
# @return [String] Redmine project identifier
|
|
24
|
+
attr_accessor :project_id
|
|
25
|
+
|
|
26
|
+
# @return [Integer] Redmine tracker ID (default: 1 for Bug)
|
|
27
|
+
attr_accessor :tracker_id
|
|
28
|
+
|
|
29
|
+
# @return [String] Application name shown in issues
|
|
30
|
+
attr_accessor :app_name
|
|
31
|
+
|
|
32
|
+
# @return [Boolean] Whether notifications are enabled
|
|
33
|
+
attr_accessor :enabled
|
|
34
|
+
|
|
35
|
+
# @return [Integer] Cooldown period in seconds between same-error notifications
|
|
36
|
+
attr_accessor :cooldown
|
|
37
|
+
|
|
38
|
+
# Initializes configuration with defaults from environment variables
|
|
39
|
+
def initialize
|
|
40
|
+
@redmine_url = ENV.fetch('ERRMINE_REDMINE_URL', nil)
|
|
41
|
+
@api_key = ENV.fetch('ERRMINE_API_KEY', nil)
|
|
42
|
+
@project_id = ENV['ERRMINE_PROJECT'] || 'bug-tracker'
|
|
43
|
+
@tracker_id = 1
|
|
44
|
+
@app_name = ENV['ERRMINE_APP_NAME'] || 'unknown'
|
|
45
|
+
@enabled = true
|
|
46
|
+
@cooldown = 300
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Checks if the configuration has required values
|
|
50
|
+
#
|
|
51
|
+
# @return [Boolean] true if redmine_url and api_key are present
|
|
52
|
+
def valid?
|
|
53
|
+
!redmine_url.nil? && !redmine_url.empty? &&
|
|
54
|
+
!api_key.nil? && !api_key.empty?
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
class << self
|
|
59
|
+
# Returns the current configuration instance
|
|
60
|
+
#
|
|
61
|
+
# @return [Configuration] the configuration instance
|
|
62
|
+
def configuration
|
|
63
|
+
@configuration ||= Configuration.new
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Yields the configuration for modification
|
|
67
|
+
#
|
|
68
|
+
# @yield [Configuration] the configuration instance
|
|
69
|
+
# @return [Configuration] the configuration instance
|
|
70
|
+
def configure
|
|
71
|
+
yield(configuration) if block_given?
|
|
72
|
+
configuration
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
# Resets the configuration to default values
|
|
76
|
+
#
|
|
77
|
+
# @return [Configuration] the new configuration instance
|
|
78
|
+
def reset_configuration!
|
|
79
|
+
@configuration = Configuration.new
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# Notifies Redmine about an exception
|
|
83
|
+
#
|
|
84
|
+
# @param exception [Exception] the exception to report
|
|
85
|
+
# @param context [Hash] additional context (url, user, etc.)
|
|
86
|
+
# @return [Hash, nil] the created/updated issue or nil on failure
|
|
87
|
+
def notify(exception, context = {})
|
|
88
|
+
return unless configuration.enabled
|
|
89
|
+
return unless configuration.valid?
|
|
90
|
+
|
|
91
|
+
Notifier.instance.notify(exception, context)
|
|
92
|
+
rescue StandardError => e
|
|
93
|
+
warn "[Errmine] Failed to notify: #{e.message}"
|
|
94
|
+
nil
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
require_relative 'errmine/railtie' if defined?(Rails::Railtie)
|
metadata
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: errmine
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Maciej Mensfeld
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies: []
|
|
12
|
+
description: Automatically create and update Redmine issues from Ruby/Rails exceptions.
|
|
13
|
+
Zero dependencies.
|
|
14
|
+
email:
|
|
15
|
+
- maciej@mensfeld.pl
|
|
16
|
+
executables: []
|
|
17
|
+
extensions: []
|
|
18
|
+
extra_rdoc_files: []
|
|
19
|
+
files:
|
|
20
|
+
- CHANGELOG.md
|
|
21
|
+
- LICENSE
|
|
22
|
+
- README.md
|
|
23
|
+
- lib/errmine.rb
|
|
24
|
+
- lib/errmine/middleware.rb
|
|
25
|
+
- lib/errmine/notifier.rb
|
|
26
|
+
- lib/errmine/railtie.rb
|
|
27
|
+
- lib/errmine/version.rb
|
|
28
|
+
homepage: https://github.com/mensfeld/errmine
|
|
29
|
+
licenses:
|
|
30
|
+
- MIT
|
|
31
|
+
metadata:
|
|
32
|
+
source_code_uri: https://github.com/mensfeld/errmine
|
|
33
|
+
changelog_uri: https://github.com/mensfeld/errmine/blob/master/CHANGELOG.md
|
|
34
|
+
rubygems_mfa_required: 'true'
|
|
35
|
+
rdoc_options: []
|
|
36
|
+
require_paths:
|
|
37
|
+
- lib
|
|
38
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
39
|
+
requirements:
|
|
40
|
+
- - ">="
|
|
41
|
+
- !ruby/object:Gem::Version
|
|
42
|
+
version: 3.3.0
|
|
43
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
44
|
+
requirements:
|
|
45
|
+
- - ">="
|
|
46
|
+
- !ruby/object:Gem::Version
|
|
47
|
+
version: '0'
|
|
48
|
+
requirements: []
|
|
49
|
+
rubygems_version: 4.0.0.beta2
|
|
50
|
+
specification_version: 4
|
|
51
|
+
summary: Dead simple exception tracking for Redmine
|
|
52
|
+
test_files: []
|