tj-scale 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +77 -0
- data/LICENSE.txt +21 -0
- data/README.md +504 -0
- data/lib/tj-scale.rb +94 -0
- data/lib/tj_scale_ruby/configuration.rb +140 -0
- data/lib/tj_scale_ruby/job_backends/delayed_job.rb +50 -0
- data/lib/tj_scale_ruby/job_backends/sidekiq.rb +43 -0
- data/lib/tj_scale_ruby/job_backends.rb +27 -0
- data/lib/tj_scale_ruby/models/tj_scale_ruby.rb +134 -0
- data/lib/tj_scale_ruby/rack_queue_time_middleware.rb +45 -0
- data/lib/tj_scale_ruby/version.rb +6 -0
- metadata +209 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: e070c5d82758136d8df85df78f7f6b021f7494cfa8a2b64ff417aff9ad386333
|
|
4
|
+
data.tar.gz: d4cdd0e53528d48a634b8ee929c498cf63756ae7fd8be239ad6aaf277174052f
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 45b1804fc8ee06924b28d7230645fcb34d2d26fa4c37a412183cbfac6ff9bf82da108876d8012f275c774fc027e312ca0095b9bb6d0b89af6303023d0ae302fa
|
|
7
|
+
data.tar.gz: 46758b912e0a3ede622ed4ceae6ddb0b78c74b6dfe2cc4022fdf484892320db56204eea15600285bd7cbb6a09286974030b5ba73f983250cf3c3194ebd24632c
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
- **Gem renamed to `tj-scale`** (was `tj-scale-ruby`); require path is now `require "tj-scale"`. Internal `TjScaleRuby` module and `lib/tj_scale_ruby/` paths are unchanged.
|
|
13
|
+
|
|
14
|
+
### Removed
|
|
15
|
+
|
|
16
|
+
- **`min_dynos` / `max_dynos` removed from the gem.** The control plane's dashboard settings are the single source of truth for scaling bounds. `TJ_SCALE_MIN_DYNOS` / `TJ_SCALE_MAX_DYNOS` are ignored, and payloads no longer include `min_dynos` / `max_dynos`.
|
|
17
|
+
|
|
18
|
+
## [1.2.0] - 2026-06-12
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- **Sidekiq support.** Worker metrics now come from a pluggable queue backend: **Delayed Job** or **Sidekiq**, selected with `TJ_SCALE_JOB_BACKEND` (`delayed_job` / `sidekiq` / `auto`). The default `auto` prefers Sidekiq when the host app has it loaded, otherwise Delayed Job — existing Delayed Job apps need no changes.
|
|
23
|
+
- `TJ_SCALE_SIDEKIQ_QUEUES` — comma-separated list of Sidekiq queues to count (empty = all queues). Sidekiq `queue_time_s` is the latency of the slowest monitored queue.
|
|
24
|
+
- Worker payloads include a `job_backend` field (`"delayed_job"` or `"sidekiq"`).
|
|
25
|
+
- `TjScaleRuby::JobMonitor.backend` exposes the active backend; `TjScaleRuby::JobBackends.resolve/detect` for manual use.
|
|
26
|
+
- **Control plane (tj-scale-dashboard):** Slack notifications. Configure an incoming webhook under Settings → Slack notifications; the control plane posts a message whenever web or worker dynos scale up or down (auto or manual) and, optionally, when a scale attempt fails (throttled to one per app/process per 15 minutes). Includes per-event toggles, channel/username overrides, and a "Send test message" button.
|
|
27
|
+
|
|
28
|
+
### Changed (repository layout)
|
|
29
|
+
|
|
30
|
+
- The Rails control plane moved out of this repo's `platform/` directory into the sibling **`tj-scale-dashboard`** project (Rails module renamed `Platform` → `TjScaleDashboard`).
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- **`delayed_job_active_record` is no longer a runtime dependency.** Host apps bring their own queue gem (`delayed_job_active_record` or `sidekiq`); the gem lazily loads whichever backend is selected. If you rely on Delayed Job, make sure it is in your app's Gemfile (it almost certainly already is).
|
|
35
|
+
- Ingest requests send **`Authorization: Bearer`** as well as **`LOGPLEX_DRAIN_TOKEN`** (TJ Scale Agent API accepts either).
|
|
36
|
+
|
|
37
|
+
## [1.1.0] - 2026-04-27
|
|
38
|
+
|
|
39
|
+
### Changed
|
|
40
|
+
|
|
41
|
+
- **Breaking:** `TJ_SCALE_API_URL` is **required**; hardcoded default ingest URLs were removed.
|
|
42
|
+
- Documented end-to-end behavior in `lib/tj-scale-ruby.rb` (reporter vs control plane).
|
|
43
|
+
- Depend on **`openssl` >= 3.2** and `require "openssl"` before HTTP so metric POSTs avoid CRL verification failures on newer OpenSSL/Ruby combinations (same class of issue as ruby/openssl#949).
|
|
44
|
+
|
|
45
|
+
### Removed
|
|
46
|
+
|
|
47
|
+
- `TjScaleRuby::JobMonitor.restart_dynos` and `TJ_SCALE_RESTART_URL` (platform ingest never performed restarts).
|
|
48
|
+
- `TJ_SCALE_METRIC_SOURCE` / heartbeat mode (sent misleading zeros and could interact badly with autoscale rules). Web metrics use the web reporter path + queue middleware instead of skipping DJ queries.
|
|
49
|
+
|
|
50
|
+
## [1.0.1] - 2026-04-14
|
|
51
|
+
|
|
52
|
+
### Changed
|
|
53
|
+
|
|
54
|
+
- Gemspec packaging includes only `lib/**` plus `README.md`, `CHANGELOG.md`, and `LICENSE.txt` (no Gemfile, Rakefile, or other dev-only root files).
|
|
55
|
+
- The monorepo `platform/` app and test directories stay out of the gem.
|
|
56
|
+
- When `.git` is missing, `gem build` still works using the same file list.
|
|
57
|
+
- Require `active_support/core_ext/module/delegation` before `rails/railtie` so the entrypoint loads reliably when Active Support is not yet loaded.
|
|
58
|
+
|
|
59
|
+
## [1.0.0] - 2026-04-14
|
|
60
|
+
|
|
61
|
+
First public release of **tj-scale-ruby**.
|
|
62
|
+
|
|
63
|
+
### Added
|
|
64
|
+
|
|
65
|
+
- Automatic Delayed Job queue depth reporting to a configurable HTTP endpoint (default interval 20s).
|
|
66
|
+
- Heroku-aware monitoring on the first `web.1` or `worker.1` dyno via `TJ_SCALE_MONITOR_PROCESS`.
|
|
67
|
+
- `TjScaleRuby::Configuration` for target app, process type, min/max dynos (defaults 2–8), interval, priority filters (`TJ_MIN_PRIORITY` / `TJ_MAX_PRIORITY`), and optional heartbeat-only mode (`TJ_SCALE_METRIC_SOURCE=none`; **removed in 1.1.0**).
|
|
68
|
+
- JSON payloads including `job_count`, `timestamp`, and when set `heroku_app`, `target_process`, `min_dynos`, `max_dynos`.
|
|
69
|
+
- `TjScaleRuby::JobMonitor.restart_dynos` for restart requests via the scaling service (**removed in 1.1.0**).
|
|
70
|
+
- `TjScaleRuby.monitor_on_this_dyno?` and `TjScaleRuby.start_monitoring_loop!` (Railtie starts the loop after Rails initializes).
|
|
71
|
+
|
|
72
|
+
### Performance
|
|
73
|
+
|
|
74
|
+
- Single `Time.zone.now` per `send_log` for consistent timestamps and queue counts.
|
|
75
|
+
- Cached configuration for priority filters and Heroku app name (no repeated `ENV` reads in the hot path).
|
|
76
|
+
- Memoized `monitor_on_this_dyno?` and parsed API URIs; `Net::HTTP.start` for bounded connection lifecycle per request.
|
|
77
|
+
- Shared `http_post` helper for log and restart calls.
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2023-2024 Untechnickle
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,504 @@
|
|
|
1
|
+
# TjScaleRuby
|
|
2
|
+
|
|
3
|
+
[](https://www.ruby-lang.org/)
|
|
4
|
+
[](https://rubyonrails.org/)
|
|
5
|
+
[](LICENSE.txt)
|
|
6
|
+
|
|
7
|
+
TjScaleRuby is a Ruby gem that automatically monitors your background job queues — **Delayed Job or Sidekiq** — and your web traffic, and sends metrics to a remote monitoring service for auto-scaling purposes. It integrates seamlessly with Rails applications and Heroku deployments.
|
|
8
|
+
|
|
9
|
+
## Features
|
|
10
|
+
|
|
11
|
+
- 🔍 **Automatic Queue Monitoring**: Continuously monitors **Delayed Job or Sidekiq** queues for pending jobs
|
|
12
|
+
- 🔀 **Pluggable Job Backend**: Flip between Delayed Job and Sidekiq with one env var (`TJ_SCALE_JOB_BACKEND`); auto-detected by default
|
|
13
|
+
- 📊 **Metrics Reporting**: Sends job counts, queue latency, and web queue/response times to a remote monitoring service
|
|
14
|
+
- 🚀 **Heroku Integration**: Automatically runs on the first web or worker dyno
|
|
15
|
+
- ⚙️ **Configurable**: Customizable priority/queue filters and API endpoints
|
|
16
|
+
- 🔔 **Slack Notifications** (control plane): message your team whenever web or worker dynos scale up/down, or when scaling fails
|
|
17
|
+
- 🔒 **Error Handling**: Robust error handling with comprehensive logging
|
|
18
|
+
- 🧪 **Well Tested**: Comprehensive test suite with RSpec
|
|
19
|
+
|
|
20
|
+
## Control plane (Rails + PostgreSQL)
|
|
21
|
+
|
|
22
|
+
The control plane lives in its own project, **`tj-scale-dashboard`** (a sibling directory / repository to this gem): dashboard, per-app scaling rules (web queue time + worker job counts), metrics ingest API compatible with the gem, Heroku formation updates, and **Slack notifications** for every scale event (Settings → Slack notifications). It uses **PostgreSQL** — see its `README.md` and `.env.example` in `../tj-scale-dashboard/`.
|
|
23
|
+
|
|
24
|
+
## Installation
|
|
25
|
+
|
|
26
|
+
### As a Gem
|
|
27
|
+
|
|
28
|
+
Add this line to your application's Gemfile:
|
|
29
|
+
|
|
30
|
+
```ruby
|
|
31
|
+
gem "tj-scale-ruby", "~> 1.2"
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
And then execute:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
$ bundle install
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Or install it yourself as:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
$ gem install tj-scale-ruby
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### From Source
|
|
47
|
+
|
|
48
|
+
1. Clone the repository:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
git clone https://github.com/untechnickle/tj-scale-ruby.git
|
|
52
|
+
cd tj-scale-ruby
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
2. Install dependencies:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
bundle install
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
3. Run the setup script:
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
bin/setup
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
## End-to-end guide
|
|
68
|
+
|
|
69
|
+
Build the gem, publish or reference it, add it to your Rails app, and set configuration (env / Heroku). Follow these steps in order the first time you ship and adopt the gem.
|
|
70
|
+
|
|
71
|
+
### 1. Prerequisites
|
|
72
|
+
|
|
73
|
+
- **Ruby** 3.0+ and **Bundler** 2+
|
|
74
|
+
- A **Rails** 6.1+ app with **one job backend**:
|
|
75
|
+
- **`delayed_job_active_record`** (and a DB-backed `delayed_jobs` table), **or**
|
|
76
|
+
- **`sidekiq`** (and Redis)
|
|
77
|
+
- **Heroku CLI** (if you deploy to Heroku)
|
|
78
|
+
- A **RubyGems.org** account (if you publish publicly)
|
|
79
|
+
|
|
80
|
+
### 2. Prepare and build the gem (in this repository)
|
|
81
|
+
|
|
82
|
+
1. **Track files for the gem package** — the gemspec uses `git ls-files`, so the repo must be a git repository and files must be committed:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
cd /path/to/tj-scale
|
|
86
|
+
git init # if needed
|
|
87
|
+
git add .
|
|
88
|
+
git commit -m "Release tj-scale-ruby"
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
2. **Install dependencies and verify quality**:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
bundle install
|
|
95
|
+
bundle exec rspec
|
|
96
|
+
bundle exec rake standard
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
3. **Set the version** in `lib/tj_scale_ruby/version.rb` (for example `1.0.0`).
|
|
100
|
+
|
|
101
|
+
4. **Build the package**:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
gem build tj-scale-ruby.gemspec
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
This produces `tj-scale-ruby-<version>.gem` in the current directory.
|
|
108
|
+
|
|
109
|
+
### 3. Publish the gem (choose one)
|
|
110
|
+
|
|
111
|
+
**RubyGems.org (public)**
|
|
112
|
+
|
|
113
|
+
1. Create an API key at [rubygems.org/api_keys](https://rubygems.org/api_keys).
|
|
114
|
+
2. Push the file you built (version must match `VERSION`):
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
gem push tj-scale-ruby-1.0.0.gem
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
**Private gem host**
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
gem push tj-scale-ruby-1.0.0.gem --host https://your-gem-server.com
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Configure Bundler in the app if your host needs authentication (see your provider’s docs).
|
|
127
|
+
|
|
128
|
+
**Without publishing — use the repo directly**
|
|
129
|
+
|
|
130
|
+
In the app’s `Gemfile`:
|
|
131
|
+
|
|
132
|
+
```ruby
|
|
133
|
+
gem "tj-scale-ruby", git: "https://github.com/untechnickle/tj-scale-ruby.git", branch: "main"
|
|
134
|
+
# or path for local development:
|
|
135
|
+
# gem "tj-scale-ruby", path: "../tj-scale"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Then `bundle install`.
|
|
139
|
+
|
|
140
|
+
### 4. Add the gem to your Rails application
|
|
141
|
+
|
|
142
|
+
1. Open the app’s **`Gemfile`** and add (adjust version or source as above):
|
|
143
|
+
|
|
144
|
+
```ruby
|
|
145
|
+
gem "tj-scale-ruby", "~> 1.2"
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
2. Run:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
bundle install
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
3. **No extra `require`** is needed in `application.rb`: the gem registers a **Rails Railtie** and loads with Rails. Ensure your app already bundles its job backend — **`delayed_job_active_record`** or **`sidekiq`** (the gem auto-detects which one is loaded, or set `TJ_SCALE_JOB_BACKEND` explicitly).
|
|
155
|
+
|
|
156
|
+
4. Deploy or run the server as usual. The monitoring thread starts only on **`web.1`** or **`worker.1`**, depending on `TJ_SCALE_MONITOR_PROCESS` (see below).
|
|
157
|
+
|
|
158
|
+
### 5. Configure environment variables
|
|
159
|
+
|
|
160
|
+
Configuration is **entirely via environment variables** (and Heroku **config vars** in production). Copy [`.env.example`](.env.example) to `.env` for local use if you use **dotenv-rails** or similar.
|
|
161
|
+
|
|
162
|
+
| Variable | Required | Purpose |
|
|
163
|
+
|----------|----------|---------|
|
|
164
|
+
| `TJ_SCALE_API_TOKEN` | Yes | Sent as **`LOGPLEX_DRAIN_TOKEN`** and **`Authorization: Bearer …`** on each ingest POST. |
|
|
165
|
+
| `TJ_SCALE_API_URL` | Yes | Full URL of the Agent API ingest path: **`POST /api/v1/metrics`** or **`POST /api/v1/sys-logs`** (same handler). Example local: `http://127.0.0.1:5001/api/v1/metrics`. |
|
|
166
|
+
| `TJ_SCALE_TARGET_APP` | No | Heroku app name to scale; defaults to **`HEROKU_APP_NAME`** if unset. |
|
|
167
|
+
| `TJ_SCALE_TARGET_PROCESS` | No | `web` or `worker` to scale; defaults to `TJ_SCALE_MONITOR_PROCESS`. |
|
|
168
|
+
| `TJ_SCALE_MONITOR_PROCESS` | No | `web` or `worker` — only that process’s **`.1`** dyno runs the reporter (default **`worker`**). |
|
|
169
|
+
| `TJ_SCALE_INTERVAL_SECONDS` | No | Seconds between posts (default **20**). |
|
|
170
|
+
| `TJ_SCALE_JOB_BACKEND` | No | Queue backend for the **worker** reporter: `delayed_job`, `sidekiq`, or `auto` (default). `auto` uses Sidekiq when the `sidekiq` gem is loaded, otherwise Delayed Job. |
|
|
171
|
+
| `TJ_SCALE_SIDEKIQ_QUEUES` | No | Sidekiq backend only: comma-separated queue names to count (empty = all queues). |
|
|
172
|
+
| `TJ_MIN_PRIORITY` / `TJ_MAX_PRIORITY` | No | Limit which job priorities are counted (`0` = no filter; **worker** reporter, Delayed Job backend only). |
|
|
173
|
+
| `TJ_SCALE_QUEUE_TIME_MS` | No | When `TJ_SCALE_MONITOR_PROCESS=web`, supplies **`queue_time_ms`** if no middleware snapshot exists. |
|
|
174
|
+
| `TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE` | No | Set to **`1`**, **`true`**, **`yes`**, or **`on`** to enable Rack middleware that records **`queue_time_ms`** from **`X-Request-Start`** (Heroku). |
|
|
175
|
+
|
|
176
|
+
**Payload by reporter:** **`web`** — `queue_time_ms` (when known), `job_count` (requests since last tick), optional `response_time_ms`. **`worker`** — `job_count`, `queue_time_s` (oldest waiting job age / queue latency, or **0**), and `job_backend` (`delayed_job` or `sidekiq`). Both send `heroku_app` and `target_process`. Scaling limits (min/max dynos) are configured in the control plane's dashboard settings, not by this gem. Autoscaling runs on the **control plane** after ingest; this gem does not call Heroku.
|
|
177
|
+
|
|
178
|
+
Heroku sets **`DYNO`** (for example `web.1`, `worker.1`) and typically **`HEROKU_APP_NAME`**; do not set **`DYNO`** manually in production.
|
|
179
|
+
|
|
180
|
+
**Example: `revize-prod` (web) and `revize-prod-mirror` (workers)**
|
|
181
|
+
|
|
182
|
+
Use the same token on both apps if one backend handles scaling.
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
# revize-prod — reporter on web.1, scale web dynos
|
|
186
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_token \
|
|
187
|
+
TJ_SCALE_API_URL=https://your-control-plane.example.com/api/v1/metrics \
|
|
188
|
+
TJ_SCALE_TARGET_APP=revize-prod \
|
|
189
|
+
TJ_SCALE_TARGET_PROCESS=web \
|
|
190
|
+
TJ_SCALE_MONITOR_PROCESS=web \
|
|
191
|
+
-a revize-prod
|
|
192
|
+
|
|
193
|
+
# revize-prod-mirror — reporter on worker.1, scale worker dynos
|
|
194
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_token \
|
|
195
|
+
TJ_SCALE_API_URL=https://your-control-plane.example.com/api/v1/metrics \
|
|
196
|
+
TJ_SCALE_TARGET_APP=revize-prod-mirror \
|
|
197
|
+
TJ_SCALE_TARGET_PROCESS=worker \
|
|
198
|
+
TJ_SCALE_MONITOR_PROCESS=worker \
|
|
199
|
+
-a revize-prod-mirror
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
On the **web** app, enable **`TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE=1`** (and restart) so `queue_time_ms` is populated from Heroku’s router; the web dyno does not need a shared Delayed Job DB for that path.
|
|
203
|
+
|
|
204
|
+
### 6. Wire your monitoring service (Agent API)
|
|
205
|
+
|
|
206
|
+
The TJ Scale **Agent API** accepts the same JSON on either route:
|
|
207
|
+
|
|
208
|
+
- **`POST /api/v1/metrics`**
|
|
209
|
+
- **`POST /api/v1/sys-logs`**
|
|
210
|
+
|
|
211
|
+
Set **`TJ_SCALE_API_URL`** to the full URL of one of them (for example `http://127.0.0.1:5001/api/v1/metrics` in development). Authenticate with the ingest token using **`LOGPLEX_DRAIN_TOKEN`** or **`Authorization: Bearer`** (same secret as **`TJ_SCALE_API_TOKEN`**); the gem sends **both** headers.
|
|
212
|
+
|
|
213
|
+
The gem POSTs JSON with **`timestamp`**, **`heroku_app`**, **`target_process`**, plus **web** metrics (`queue_time_ms`, `job_count`, optional `response_time_ms`) or **worker** metrics (`job_count`, `queue_time_s`). Your control plane ingests samples and applies scaling rules (thresholds, cooldowns) when calling the Heroku Platform API.
|
|
214
|
+
|
|
215
|
+
### 7. Verify
|
|
216
|
+
|
|
217
|
+
- At least one **web** dyno on `revize-prod` and one **worker** dyno on `revize-prod-mirror` so **`web.1`** and **`worker.1`** exist.
|
|
218
|
+
- Check **Rails logs** for `TjScaleRuby` lines (debug/warn/error).
|
|
219
|
+
- Confirm your API receives periodic requests with the expected payload.
|
|
220
|
+
|
|
221
|
+
### Optional: manual calls in Ruby
|
|
222
|
+
|
|
223
|
+
```ruby
|
|
224
|
+
TjScaleRuby::JobMonitor.pending_job_count
|
|
225
|
+
TjScaleRuby::JobMonitor.oldest_pending_queue_seconds
|
|
226
|
+
TjScaleRuby.record_web_queue_time_ms!(42) # optional; middleware does this on each request
|
|
227
|
+
TjScaleRuby::JobMonitor.send_log
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
## Configuration
|
|
231
|
+
|
|
232
|
+
### Environment Variables
|
|
233
|
+
|
|
234
|
+
Create a `.env` file in your project root (or set these in your Heroku config vars):
|
|
235
|
+
|
|
236
|
+
```bash
|
|
237
|
+
# Required: API token and ingest URL for your control plane
|
|
238
|
+
TJ_SCALE_API_TOKEN=your_api_token_here
|
|
239
|
+
TJ_SCALE_API_URL=http://127.0.0.1:5001/api/v1/metrics
|
|
240
|
+
|
|
241
|
+
# --- Scaling metadata (sent to your monitoring API; defaults shown) ---
|
|
242
|
+
# Heroku app name to scale (defaults to HEROKU_APP_NAME when unset).
|
|
243
|
+
TJ_SCALE_TARGET_APP=my-heroku-app
|
|
244
|
+
|
|
245
|
+
# Process type to scale on that app: "web" or "worker" (defaults to TJ_SCALE_MONITOR_PROCESS).
|
|
246
|
+
TJ_SCALE_TARGET_PROCESS=web
|
|
247
|
+
|
|
248
|
+
# Which process type runs the reporter loop: only the first dyno of this type (web.1 or worker.1).
|
|
249
|
+
# Use "web" on the web app, "worker" on the worker app.
|
|
250
|
+
TJ_SCALE_MONITOR_PROCESS=worker
|
|
251
|
+
|
|
252
|
+
# Seconds between metric posts (default 20).
|
|
253
|
+
TJ_SCALE_INTERVAL_SECONDS=20
|
|
254
|
+
|
|
255
|
+
# Optional: Minimum priority filter for jobs (0 = no filter)
|
|
256
|
+
# Only count jobs with priority >= this value
|
|
257
|
+
TJ_MIN_PRIORITY=0
|
|
258
|
+
|
|
259
|
+
# Optional: Maximum priority filter for jobs (0 = no filter)
|
|
260
|
+
# Only count jobs with priority <= this value
|
|
261
|
+
TJ_MAX_PRIORITY=0
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
Heroku sets `HEROKU_APP_NAME` for the current app. If you omit `TJ_SCALE_TARGET_APP`, the payload uses that value so each app reports scaling targets for itself.
|
|
265
|
+
|
|
266
|
+
### Heroku Configuration
|
|
267
|
+
|
|
268
|
+
Set the environment variables on Heroku:
|
|
269
|
+
|
|
270
|
+
```bash
|
|
271
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_api_token_here
|
|
272
|
+
heroku config:set TJ_SCALE_API_URL=https://your-control-plane.example.com/api/v1/metrics
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
The `DYNO` environment variable is automatically set by Heroku and is used to determine which dyno should run the monitoring loop (only `web.1` or `worker.1`, depending on `TJ_SCALE_MONITOR_PROCESS`).
|
|
276
|
+
|
|
277
|
+
### Example: web on `revize-prod`, workers on `revize-prod-mirror`
|
|
278
|
+
|
|
279
|
+
Use two Heroku apps (or one app with both process types—same idea). Configure each app’s config vars so the **monitoring service** receives `heroku_app`, `target_process`, and `job_count` on each POST. Scaling bounds (min/max dynos) are set in the control plane's dashboard.
|
|
280
|
+
|
|
281
|
+
**App `revize-prod` (web dynos)** — run the reporter on the first web dyno and scale `web` dynos:
|
|
282
|
+
|
|
283
|
+
```bash
|
|
284
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_token -a revize-prod
|
|
285
|
+
heroku config:set TJ_SCALE_MONITOR_PROCESS=web -a revize-prod
|
|
286
|
+
heroku config:set TJ_SCALE_TARGET_APP=revize-prod -a revize-prod
|
|
287
|
+
heroku config:set TJ_SCALE_TARGET_PROCESS=web -a revize-prod
|
|
288
|
+
heroku config:set TJ_SCALE_API_URL=https://your-control-plane.example.com/api/v1/metrics -a revize-prod
|
|
289
|
+
heroku config:set TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE=1 -a revize-prod
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
**App `revize-prod-mirror` (worker dynos)** — run the reporter on `worker.1` and scale `worker` dynos:
|
|
293
|
+
|
|
294
|
+
```bash
|
|
295
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_token -a revize-prod-mirror
|
|
296
|
+
heroku config:set TJ_SCALE_MONITOR_PROCESS=worker -a revize-prod-mirror
|
|
297
|
+
heroku config:set TJ_SCALE_TARGET_APP=revize-prod-mirror -a revize-prod-mirror
|
|
298
|
+
heroku config:set TJ_SCALE_TARGET_PROCESS=worker -a revize-prod-mirror
|
|
299
|
+
heroku config:set TJ_SCALE_API_URL=https://your-control-plane.example.com/api/v1/metrics -a revize-prod-mirror
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
Ensure both apps have the `web` / `worker` process types in the `Procfile` as needed, deploy, and scale at least one dyno of each monitored type so `web.1` / `worker.1` exist.
|
|
303
|
+
|
|
304
|
+
**Monitoring API:** Your receiver at `TJ_SCALE_API_URL` should read the JSON body and enforce the min/max dyno bounds from its own dashboard settings when calling the Heroku Platform API (or your own scaler). Fields sent include `job_count`, `timestamp`, and when set, `heroku_app` and `target_process`.
|
|
305
|
+
|
|
306
|
+
## Usage
|
|
307
|
+
|
|
308
|
+
### Automatic Monitoring
|
|
309
|
+
|
|
310
|
+
Once installed and configured, the gem starts monitoring on the first dyno of the configured process type (`web.1` or `worker.1`, see `TJ_SCALE_MONITOR_PROCESS`). The loop runs every `TJ_SCALE_INTERVAL_SECONDS` (default 20) and POSTs metrics to the configured API.
|
|
311
|
+
|
|
312
|
+
### Manual Usage
|
|
313
|
+
|
|
314
|
+
You can also use the gem's methods manually in your code:
|
|
315
|
+
|
|
316
|
+
```ruby
|
|
317
|
+
# Get the count of pending jobs
|
|
318
|
+
count = TjScaleRuby::JobMonitor.pending_job_count
|
|
319
|
+
puts "Pending jobs: #{count}"
|
|
320
|
+
|
|
321
|
+
# Send current job count to the monitoring service
|
|
322
|
+
response = TjScaleRuby::JobMonitor.send_log
|
|
323
|
+
if response
|
|
324
|
+
puts "Log sent successfully"
|
|
325
|
+
end
|
|
326
|
+
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
### Priority Filtering
|
|
330
|
+
|
|
331
|
+
You can filter jobs by priority using environment variables:
|
|
332
|
+
|
|
333
|
+
```ruby
|
|
334
|
+
# Only count jobs with priority between 5 and 10
|
|
335
|
+
ENV['TJ_MIN_PRIORITY'] = '5'
|
|
336
|
+
ENV['TJ_MAX_PRIORITY'] = '10'
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
## How It Works
|
|
340
|
+
|
|
341
|
+
1. **Initialization**: When your Rails application starts, the `Railtie` checks if it's running on the first monitored dyno (`web.1` or `worker.1`, per `TJ_SCALE_MONITOR_PROCESS`)
|
|
342
|
+
2. **Background Thread**: If conditions are met, a background thread is started
|
|
343
|
+
3. **Monitoring Loop**: Every N seconds (`TJ_SCALE_INTERVAL_SECONDS`), the thread POSTs a payload: on **worker** reporters it counts waiting jobs on the active backend; on **web** reporters it sends router queue time (middleware / env) plus request volume and average response time for the interval.
|
|
344
|
+
4. **Job counting (worker reporters, Delayed Job backend)**: `pending_job_count` filters jobs based on:
|
|
345
|
+
- Jobs past their `run_at` time (with 5 second buffer)
|
|
346
|
+
- Jobs that haven't failed (`failed_at IS NULL`)
|
|
347
|
+
- Jobs that aren't locked (`locked_at IS NULL`)
|
|
348
|
+
- Optional priority range filters
|
|
349
|
+
5. **Job counting (worker reporters, Sidekiq backend)**: `pending_job_count` sums enqueued jobs across queues (all queues, or only those in `TJ_SCALE_SIDEKIQ_QUEUES`), and `queue_time_s` is the latency of the slowest monitored queue.
|
|
350
|
+
|
|
351
|
+
## Development
|
|
352
|
+
|
|
353
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
354
|
+
|
|
355
|
+
### Running Tests
|
|
356
|
+
|
|
357
|
+
```bash
|
|
358
|
+
# Run all tests
|
|
359
|
+
bundle exec rspec
|
|
360
|
+
|
|
361
|
+
# Run tests with documentation format
|
|
362
|
+
bundle exec rspec --format documentation
|
|
363
|
+
|
|
364
|
+
# Run a specific test file
|
|
365
|
+
bundle exec rspec spec/lib/tj_scale_ruby/models/tj_scale_ruby_spec.rb
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
### Code Quality
|
|
369
|
+
|
|
370
|
+
The project uses [Standard](https://github.com/testdouble/standard) for code formatting:
|
|
371
|
+
|
|
372
|
+
```bash
|
|
373
|
+
# Check code style
|
|
374
|
+
bundle exec rake standard
|
|
375
|
+
|
|
376
|
+
# Auto-fix code style issues
|
|
377
|
+
bundle exec rake standard:fix
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
## Building and Publishing the Gem
|
|
381
|
+
|
|
382
|
+
For a full checklist (git, tests, version, integrate in the app, Heroku), see **[End-to-end guide](#end-to-end-guide)** above.
|
|
383
|
+
|
|
384
|
+
### Building the Gem
|
|
385
|
+
|
|
386
|
+
1. Update the version in `lib/tj_scale_ruby/version.rb`
|
|
387
|
+
2. Ensure files are tracked by git (the gemspec uses `git ls-files`).
|
|
388
|
+
3. Build the gem:
|
|
389
|
+
|
|
390
|
+
```bash
|
|
391
|
+
gem build tj-scale-ruby.gemspec
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
This will create a `.gem` file in the current directory.
|
|
395
|
+
|
|
396
|
+
### Publishing to RubyGems
|
|
397
|
+
|
|
398
|
+
1. Make sure you have a RubyGems account
|
|
399
|
+
2. Get your API key from https://rubygems.org/api_keys
|
|
400
|
+
3. Configure your credentials:
|
|
401
|
+
|
|
402
|
+
```bash
|
|
403
|
+
gem push --key your_api_key tj-scale-ruby-1.0.0.gem
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Or use the credentials file:
|
|
407
|
+
|
|
408
|
+
```bash
|
|
409
|
+
# Create ~/.gem/credentials
|
|
410
|
+
# Add your API key:
|
|
411
|
+
# ---
|
|
412
|
+
# :rubygems_api_key: your_api_key_here
|
|
413
|
+
|
|
414
|
+
gem push tj-scale-ruby-1.0.0.gem
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
### Publishing to a Private Gem Server
|
|
418
|
+
|
|
419
|
+
If you're using a private gem server:
|
|
420
|
+
|
|
421
|
+
```bash
|
|
422
|
+
gem push tj-scale-ruby-1.0.0.gem --host https://your-gem-server.com
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
## Deployment
|
|
426
|
+
|
|
427
|
+
### Heroku Deployment
|
|
428
|
+
|
|
429
|
+
1. Add the gem to your `Gemfile`:
|
|
430
|
+
|
|
431
|
+
```ruby
|
|
432
|
+
gem "tj-scale-ruby", "~> 1.2"
|
|
433
|
+
```
|
|
434
|
+
|
|
435
|
+
2. Set the required environment variables:
|
|
436
|
+
|
|
437
|
+
```bash
|
|
438
|
+
heroku config:set TJ_SCALE_API_TOKEN=your_token
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
3. Deploy your application:
|
|
442
|
+
|
|
443
|
+
```bash
|
|
444
|
+
git push heroku main
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
The gem will automatically start monitoring on the first worker dyno.
|
|
448
|
+
|
|
449
|
+
### Docker Deployment
|
|
450
|
+
|
|
451
|
+
1. Add the gem to your `Gemfile`
|
|
452
|
+
2. Set environment variables in your Docker configuration
|
|
453
|
+
3. The gem will automatically start when the Rails application initializes
|
|
454
|
+
|
|
455
|
+
### Other Platforms
|
|
456
|
+
|
|
457
|
+
The gem works with any Rails application. Just ensure:
|
|
458
|
+
- Delayed Job or Sidekiq is configured
|
|
459
|
+
- Environment variables are set
|
|
460
|
+
- The application has worker dynos (for automatic monitoring)
|
|
461
|
+
|
|
462
|
+
## Requirements
|
|
463
|
+
|
|
464
|
+
- Ruby >= 3.0.0
|
|
465
|
+
- Rails >= 6.1
|
|
466
|
+
- One job backend (worker reporters only; bring your own gem):
|
|
467
|
+
- delayed_job_active_record >= 4.1 and PostgreSQL, **or**
|
|
468
|
+
- sidekiq and Redis
|
|
469
|
+
|
|
470
|
+
## Contributing
|
|
471
|
+
|
|
472
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/untechnickle/tj-scale-ruby.
|
|
473
|
+
|
|
474
|
+
1. Fork the repository
|
|
475
|
+
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
|
|
476
|
+
3. Make your changes
|
|
477
|
+
4. Add tests for your changes
|
|
478
|
+
5. Ensure all tests pass (`bundle exec rspec`)
|
|
479
|
+
6. Ensure code style is correct (`bundle exec rake standard`)
|
|
480
|
+
7. Commit your changes (`git commit -am 'Add some amazing feature'`)
|
|
481
|
+
8. Push to the branch (`git push origin feature/amazing-feature`)
|
|
482
|
+
9. Open a Pull Request
|
|
483
|
+
|
|
484
|
+
## License
|
|
485
|
+
|
|
486
|
+
The gem is available as open source under the terms of the [MIT License](LICENSE.txt).
|
|
487
|
+
|
|
488
|
+
## Changelog
|
|
489
|
+
|
|
490
|
+
See [CHANGELOG.md](CHANGELOG.md) for a list of changes and version history.
|
|
491
|
+
|
|
492
|
+
## Support
|
|
493
|
+
|
|
494
|
+
For issues, questions, or contributions, please open an issue on the [GitHub repository](https://github.com/untechnickle/tj-scale-ruby/issues).
|
|
495
|
+
|
|
496
|
+
## Authors
|
|
497
|
+
|
|
498
|
+
- **Tanuj** - *Initial work* - [tanuj@untechnickle.com](mailto:tanuj@untechnickle.com)
|
|
499
|
+
|
|
500
|
+
## Acknowledgments
|
|
501
|
+
|
|
502
|
+
- Built for Untechnickle
|
|
503
|
+
- Works with [Delayed Job](https://github.com/collectiveidea/delayed_job) and [Sidekiq](https://github.com/sidekiq/sidekiq) for background job processing
|
|
504
|
+
|
data/lib/tj-scale.rb
ADDED
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Railties expect ActiveSupport core extensions (e.g. `delegate_missing_to`) before load.
|
|
4
|
+
require "active_support/core_ext/module/delegation"
|
|
5
|
+
require "rails/railtie"
|
|
6
|
+
require "tj_scale_ruby/configuration"
|
|
7
|
+
require "tj_scale_ruby/models/tj_scale_ruby"
|
|
8
|
+
|
|
9
|
+
# ## How this gem participates in autoscaling
|
|
10
|
+
#
|
|
11
|
+
# TJ Scale (the control plane) scales Heroku formation when ingested metrics cross rules
|
|
12
|
+
# you configure there. This gem only **reports signals**; it never calls Heroku directly.
|
|
13
|
+
#
|
|
14
|
+
# **Where it runs:** A single background thread on **one** dyno — the first dyno of the
|
|
15
|
+
# process you choose (`web.1` or `worker.1` via +TJ_SCALE_MONITOR_PROCESS+). Heroku sets +DYNO+.
|
|
16
|
+
#
|
|
17
|
+
# **What it sends:** On each tick (+TJ_SCALE_INTERVAL_SECONDS+), it POSTs JSON to
|
|
18
|
+
# +TJ_SCALE_API_URL+ (e.g. +/api/v1/metrics+ or +/api/v1/sys-logs+ — same handler on the
|
|
19
|
+
# platform) with +LOGPLEX_DRAIN_TOKEN+ and +Authorization: Bearer+ set from +TJ_SCALE_API_TOKEN+.
|
|
20
|
+
# Payload always
|
|
21
|
+
# includes +heroku_app+ and +target_process+; scaling limits (min/max dynos) live in the
|
|
22
|
+
# control plane's settings, not in this gem.
|
|
23
|
+
#
|
|
24
|
+
# - **Worker monitor** (+TJ_SCALE_MONITOR_PROCESS=worker+): counts waiting jobs
|
|
25
|
+
# (+job_count+, +queue_time_s+) on the configured queue backend — **Delayed Job** or **Sidekiq**,
|
|
26
|
+
# toggled with +TJ_SCALE_JOB_BACKEND+ (+delayed_job+ / +sidekiq+ / +auto+, the default, which
|
|
27
|
+
# prefers Sidekiq when loaded). Delayed Job needs +delayed_job_active_record+ and a shared DB;
|
|
28
|
+
# Sidekiq needs the +sidekiq+ gem and Redis (+TJ_SCALE_SIDEKIQ_QUEUES+ limits counted queues).
|
|
29
|
+
# - **Web monitor** (+TJ_SCALE_MONITOR_PROCESS=web+): sends +queue_time_ms+ from Rack middleware
|
|
30
|
+
# (router queue via +X-Request-Start+) when +TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE=1+, plus
|
|
31
|
+
# request volume and average +response_time_ms+ for the tick. Without middleware, set
|
|
32
|
+
# +TJ_SCALE_QUEUE_TIME_MS+ or the platform will not scale on queue time until data exists.
|
|
33
|
+
#
|
|
34
|
+
# **Platform side:** You create a monitored app, set thresholds (e.g. upscale above N jobs or ms),
|
|
35
|
+
# and supply a Heroku API token there. Scaling runs in jobs after each accepted ingest.
|
|
36
|
+
#
|
|
37
|
+
# Main entry point for TjScaleRuby gem
|
|
38
|
+
module TjScaleRuby
|
|
39
|
+
# Rails integration: optional Rack middleware (web queue timing) + metrics loop on +web.1+ / +worker.1+.
|
|
40
|
+
class Railtie < Rails::Railtie
|
|
41
|
+
initializer "tj_scale_ruby.rack_queue_time_middleware" do |app|
|
|
42
|
+
flag = ENV.fetch("TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE", "").to_s.strip.downcase
|
|
43
|
+
next unless %w[1 true yes on].include?(flag)
|
|
44
|
+
|
|
45
|
+
require "tj_scale_ruby/rack_queue_time_middleware"
|
|
46
|
+
app.middleware.insert(0, RackQueueTimeMiddleware)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
config.after_initialize do
|
|
50
|
+
TjScaleRuby.start_monitoring_loop!
|
|
51
|
+
end
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# Starts a background thread on the designated first dyno (see {#monitor_on_this_dyno?}).
|
|
55
|
+
def self.start_monitoring_loop!
|
|
56
|
+
return unless monitor_on_this_dyno?
|
|
57
|
+
|
|
58
|
+
interval = configuration.interval_seconds
|
|
59
|
+
Thread.new do
|
|
60
|
+
loop do
|
|
61
|
+
begin
|
|
62
|
+
JobMonitor.send_log
|
|
63
|
+
rescue StandardError => e
|
|
64
|
+
Rails.logger.error("TjScaleRuby: Error in monitoring loop: #{e.message}")
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
sleep(interval)
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# True when this dyno should run the metrics loop (first of TJ_SCALE_MONITOR_PROCESS).
|
|
73
|
+
# Result is memoized; cleared by {.reset_configuration!}.
|
|
74
|
+
def self.monitor_on_this_dyno?
|
|
75
|
+
return @monitor_on_this_dyno if defined?(@monitor_on_this_dyno)
|
|
76
|
+
|
|
77
|
+
dyno = ENV["DYNO"]
|
|
78
|
+
@monitor_on_this_dyno = if dyno.nil? || dyno.empty?
|
|
79
|
+
false
|
|
80
|
+
else
|
|
81
|
+
parts = dyno.split(".", 2)
|
|
82
|
+
if parts.size != 2
|
|
83
|
+
false
|
|
84
|
+
else
|
|
85
|
+
process, index = parts
|
|
86
|
+
process == configuration.monitor_process && index.to_i == 1
|
|
87
|
+
end
|
|
88
|
+
end
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
def self.clear_monitoring_memo!
|
|
92
|
+
remove_instance_variable(:@monitor_on_this_dyno) if defined?(@monitor_on_this_dyno)
|
|
93
|
+
end
|
|
94
|
+
end
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module TjScaleRuby
|
|
4
|
+
# Reads Heroku / scaling settings from environment variables.
|
|
5
|
+
class Configuration
|
|
6
|
+
DEFAULT_INTERVAL_SECONDS = 20
|
|
7
|
+
JOB_BACKENDS = %w[auto delayed_job sidekiq].freeze
|
|
8
|
+
|
|
9
|
+
attr_reader :monitor_process, :interval_seconds,
|
|
10
|
+
:min_priority, :max_priority, :job_backend, :sidekiq_queues
|
|
11
|
+
|
|
12
|
+
def initialize(env = ENV)
|
|
13
|
+
@monitor_process = normalize_process(env["TJ_SCALE_MONITOR_PROCESS"]) || "worker"
|
|
14
|
+
@job_backend = normalize_job_backend(env["TJ_SCALE_JOB_BACKEND"])
|
|
15
|
+
@sidekiq_queues = parse_list(env["TJ_SCALE_SIDEKIQ_QUEUES"])
|
|
16
|
+
@interval_seconds = parse_positive_int(env["TJ_SCALE_INTERVAL_SECONDS"], DEFAULT_INTERVAL_SECONDS)
|
|
17
|
+
@target_app = nonempty_string(env["TJ_SCALE_TARGET_APP"])
|
|
18
|
+
@heroku_app_name = nonempty_string(env["HEROKU_APP_NAME"])
|
|
19
|
+
@target_process = normalize_process(env["TJ_SCALE_TARGET_PROCESS"]) || @monitor_process
|
|
20
|
+
@min_priority = env["TJ_MIN_PRIORITY"].to_i
|
|
21
|
+
@max_priority = env["TJ_MAX_PRIORITY"].to_i
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# Heroku app name the scaler should act on (defaults to HEROKU_APP_NAME when unset).
|
|
25
|
+
def target_app
|
|
26
|
+
@target_app || @heroku_app_name
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# Process type to scale on that app: "web" or "worker".
|
|
30
|
+
def target_process
|
|
31
|
+
@target_process
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
private
|
|
35
|
+
|
|
36
|
+
def normalize_process(value)
|
|
37
|
+
p = value.to_s.strip.downcase
|
|
38
|
+
return nil if p.empty?
|
|
39
|
+
|
|
40
|
+
%w[web worker].include?(p) ? p : nil
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# "delayed_job" | "sidekiq" | "auto" (default; detects from loaded gems).
|
|
44
|
+
def normalize_job_backend(value)
|
|
45
|
+
b = value.to_s.strip.downcase
|
|
46
|
+
JOB_BACKENDS.include?(b) ? b : "auto"
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def parse_list(value)
|
|
50
|
+
value.to_s.split(",").map(&:strip).reject(&:empty?)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def parse_positive_int(raw, default)
|
|
54
|
+
n = raw.to_i
|
|
55
|
+
n.positive? ? n : default
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def nonempty_string(value)
|
|
59
|
+
s = value.to_s.strip
|
|
60
|
+
s.empty? ? nil : s
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
class << self
|
|
65
|
+
def configuration
|
|
66
|
+
@configuration ||= Configuration.new
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def configuration=(config)
|
|
70
|
+
@configuration = config
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def reset_configuration!
|
|
74
|
+
@configuration = nil
|
|
75
|
+
TjScaleRuby.clear_monitoring_memo!
|
|
76
|
+
TjScaleRuby.clear_web_queue_time_snapshot!
|
|
77
|
+
TjScaleRuby.clear_web_traffic_stats!
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# Latest web router queue time (ms) captured by {RackQueueTimeMiddleware}; thread-safe.
|
|
81
|
+
def record_web_queue_time_ms!(ms)
|
|
82
|
+
v = Integer(ms)
|
|
83
|
+
return if v.negative?
|
|
84
|
+
|
|
85
|
+
web_queue_mutex.synchronize { @web_queue_time_ms = v }
|
|
86
|
+
rescue ArgumentError, TypeError
|
|
87
|
+
nil
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def last_web_queue_time_ms
|
|
91
|
+
web_queue_mutex.synchronize { @web_queue_time_ms }
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def clear_web_queue_time_snapshot!
|
|
95
|
+
web_queue_mutex.synchronize { @web_queue_time_ms = nil }
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Request volume + response time for +web+ metrics (flushed each ingest tick).
|
|
99
|
+
def record_web_request_finished!(duration_ms)
|
|
100
|
+
v = Integer(duration_ms)
|
|
101
|
+
return if v.negative?
|
|
102
|
+
|
|
103
|
+
web_traffic_mutex.synchronize do
|
|
104
|
+
@web_request_count = (@web_request_count || 0) + 1
|
|
105
|
+
@web_response_time_sum_ms = (@web_response_time_sum_ms || 0) + v
|
|
106
|
+
end
|
|
107
|
+
rescue ArgumentError, TypeError
|
|
108
|
+
nil
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
# @return [Hash] +:job_count+ requests since last flush; +:response_time_ms+ average ms or +nil+
|
|
112
|
+
def flush_web_traffic_stats_for_payload!
|
|
113
|
+
web_traffic_mutex.synchronize do
|
|
114
|
+
c = @web_request_count || 0
|
|
115
|
+
sum = @web_response_time_sum_ms || 0
|
|
116
|
+
@web_request_count = 0
|
|
117
|
+
@web_response_time_sum_ms = 0
|
|
118
|
+
avg = c.positive? ? (sum.to_f / c).round : nil
|
|
119
|
+
{ job_count: c, response_time_ms: avg }
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def clear_web_traffic_stats!
|
|
124
|
+
web_traffic_mutex.synchronize do
|
|
125
|
+
@web_request_count = 0
|
|
126
|
+
@web_response_time_sum_ms = 0
|
|
127
|
+
end
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
def web_queue_mutex
|
|
131
|
+
@web_queue_mutex ||= Mutex.new
|
|
132
|
+
end
|
|
133
|
+
private :web_queue_mutex
|
|
134
|
+
|
|
135
|
+
def web_traffic_mutex
|
|
136
|
+
@web_traffic_mutex ||= Mutex.new
|
|
137
|
+
end
|
|
138
|
+
private :web_traffic_mutex
|
|
139
|
+
end
|
|
140
|
+
end
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module TjScaleRuby
|
|
4
|
+
module JobBackends
|
|
5
|
+
# Counts runnable +Delayed::Job+ rows in the shared database.
|
|
6
|
+
# Requires the host app to bundle +delayed_job_active_record+.
|
|
7
|
+
class DelayedJob
|
|
8
|
+
class << self
|
|
9
|
+
def backend_name
|
|
10
|
+
"delayed_job"
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
# Relation for runnable Delayed::Job rows (priority filters applied).
|
|
14
|
+
def pending_jobs_relation(now = Time.zone.now)
|
|
15
|
+
ensure_loaded!
|
|
16
|
+
min_priority = TjScaleRuby.configuration.min_priority
|
|
17
|
+
max_priority = TjScaleRuby.configuration.max_priority
|
|
18
|
+
|
|
19
|
+
query = ::Delayed::Job.where("run_at + interval '5 seconds' < ?", now)
|
|
20
|
+
query = query.where("failed_at IS NULL")
|
|
21
|
+
query = query.where("locked_at IS NULL")
|
|
22
|
+
query = query.where("priority >= ?", min_priority) unless min_priority.zero?
|
|
23
|
+
query = query.where("priority <= ?", max_priority) unless max_priority.zero?
|
|
24
|
+
|
|
25
|
+
query
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def pending_job_count(now = Time.zone.now)
|
|
29
|
+
pending_jobs_relation(now).count
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
# Approximate age in seconds of the oldest runnable job (+run_at+ vs +now+). Zero if none.
|
|
33
|
+
def oldest_pending_queue_seconds(now = Time.zone.now)
|
|
34
|
+
oldest = pending_jobs_relation(now).minimum(:run_at)
|
|
35
|
+
return 0 if oldest.nil?
|
|
36
|
+
|
|
37
|
+
[(now - oldest).to_f.ceil, 0].max
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def ensure_loaded!
|
|
43
|
+
return if defined?(::Delayed::Job)
|
|
44
|
+
|
|
45
|
+
require "delayed/backend/active_record"
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module TjScaleRuby
|
|
4
|
+
module JobBackends
|
|
5
|
+
# Reads queue depth and latency from the Sidekiq API (Redis).
|
|
6
|
+
# Requires the host app to bundle +sidekiq+; priority filters do not apply —
|
|
7
|
+
# use +TJ_SCALE_SIDEKIQ_QUEUES+ to limit which queues are counted.
|
|
8
|
+
class Sidekiq
|
|
9
|
+
class << self
|
|
10
|
+
def backend_name
|
|
11
|
+
"sidekiq"
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def pending_job_count(_now = nil)
|
|
15
|
+
ensure_loaded!
|
|
16
|
+
queues.sum(&:size)
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# Latency (seconds since the oldest enqueued job) of the slowest monitored queue.
|
|
20
|
+
def oldest_pending_queue_seconds(_now = nil)
|
|
21
|
+
ensure_loaded!
|
|
22
|
+
latency = queues.map(&:latency).max
|
|
23
|
+
latency.nil? ? 0 : latency.ceil
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
def queues
|
|
29
|
+
names = TjScaleRuby.configuration.sidekiq_queues
|
|
30
|
+
return ::Sidekiq::Queue.all if names.empty?
|
|
31
|
+
|
|
32
|
+
names.map { |name| ::Sidekiq::Queue.new(name) }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def ensure_loaded!
|
|
36
|
+
return if defined?(::Sidekiq::Queue)
|
|
37
|
+
|
|
38
|
+
require "sidekiq/api"
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "tj_scale_ruby/job_backends/delayed_job"
|
|
4
|
+
require "tj_scale_ruby/job_backends/sidekiq"
|
|
5
|
+
|
|
6
|
+
module TjScaleRuby
|
|
7
|
+
# Pluggable queue backends for worker metrics. Selected with
|
|
8
|
+
# +TJ_SCALE_JOB_BACKEND+ (+delayed_job+, +sidekiq+, or +auto+ — the default).
|
|
9
|
+
# +auto+ prefers Sidekiq when the host app has it loaded, else Delayed Job.
|
|
10
|
+
module JobBackends
|
|
11
|
+
def self.current
|
|
12
|
+
resolve(TjScaleRuby.configuration.job_backend)
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def self.resolve(name)
|
|
16
|
+
case name.to_s
|
|
17
|
+
when "sidekiq" then Sidekiq
|
|
18
|
+
when "delayed_job" then DelayedJob
|
|
19
|
+
else detect
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def self.detect
|
|
24
|
+
defined?(::Sidekiq) ? Sidekiq : DelayedJob
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "openssl"
|
|
4
|
+
require "net/http"
|
|
5
|
+
require "uri"
|
|
6
|
+
require "json"
|
|
7
|
+
require "tj_scale_ruby/job_backends"
|
|
8
|
+
|
|
9
|
+
module TjScaleRuby
|
|
10
|
+
# Posts metrics to your TJ Scale (or compatible) ingest URL. Scaling decisions happen on the server.
|
|
11
|
+
class JobMonitor
|
|
12
|
+
class << self
|
|
13
|
+
# Active queue backend (Delayed Job or Sidekiq) per +TJ_SCALE_JOB_BACKEND+.
|
|
14
|
+
def backend
|
|
15
|
+
JobBackends.current
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
# Relation for runnable Delayed::Job rows (Delayed Job backend only; kept for manual use).
|
|
19
|
+
def pending_jobs_relation(now = Time.zone.now)
|
|
20
|
+
JobBackends::DelayedJob.pending_jobs_relation(now)
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# Counts jobs waiting to run on the active backend (DJ: runnable rows with priority
|
|
24
|
+
# filters; Sidekiq: enqueued jobs across monitored queues).
|
|
25
|
+
def pending_job_count(now = Time.zone.now)
|
|
26
|
+
backend.pending_job_count(now)
|
|
27
|
+
rescue StandardError => e
|
|
28
|
+
Rails.logger.error("TjScaleRuby: Error counting pending jobs: #{e.message}")
|
|
29
|
+
0
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
# Age in seconds of the oldest waiting job (DJ: +run_at+ vs +now+; Sidekiq: queue latency).
|
|
33
|
+
def oldest_pending_queue_seconds(now = Time.zone.now)
|
|
34
|
+
backend.oldest_pending_queue_seconds(now)
|
|
35
|
+
rescue StandardError => e
|
|
36
|
+
Rails.logger.error("TjScaleRuby: Error computing queue_time_s: #{e.message}")
|
|
37
|
+
0
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def send_log
|
|
41
|
+
token = ENV["TJ_SCALE_API_TOKEN"]
|
|
42
|
+
unless token
|
|
43
|
+
Rails.logger.warn("TjScaleRuby: TJ_SCALE_API_TOKEN not configured")
|
|
44
|
+
return false
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
api_url = ENV["TJ_SCALE_API_URL"].to_s.strip
|
|
48
|
+
if api_url.empty?
|
|
49
|
+
Rails.logger.warn("TjScaleRuby: TJ_SCALE_API_URL not configured")
|
|
50
|
+
return false
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
now = Time.zone.now
|
|
54
|
+
cfg = TjScaleRuby.configuration
|
|
55
|
+
payload = build_log_payload(now)
|
|
56
|
+
|
|
57
|
+
response = http_post(URI.parse(api_url), body: payload.compact.to_json, token: token)
|
|
58
|
+
|
|
59
|
+
if response.is_a?(Net::HTTPSuccess)
|
|
60
|
+
if cfg.monitor_process == "web"
|
|
61
|
+
Rails.logger.debug do
|
|
62
|
+
"TjScaleRuby: Sent web metrics queue_time_ms=#{payload[:queue_time_ms].inspect}"
|
|
63
|
+
end
|
|
64
|
+
else
|
|
65
|
+
Rails.logger.debug do
|
|
66
|
+
"TjScaleRuby: Sent worker metrics job_count=#{payload[:job_count]} " \
|
|
67
|
+
"queue_time_s=#{payload[:queue_time_s]}"
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
else
|
|
71
|
+
Rails.logger.warn("TjScaleRuby: API returned #{response.code}: #{response.message}")
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
response
|
|
75
|
+
rescue StandardError => e
|
|
76
|
+
Rails.logger.error("TjScaleRuby: Error sending log: #{e.message}")
|
|
77
|
+
false
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
private
|
|
81
|
+
|
|
82
|
+
def web_queue_time_ms_for_payload
|
|
83
|
+
snap = TjScaleRuby.last_web_queue_time_ms
|
|
84
|
+
return snap unless snap.nil?
|
|
85
|
+
|
|
86
|
+
env = ENV["TJ_SCALE_QUEUE_TIME_MS"]
|
|
87
|
+
return nil if env.blank?
|
|
88
|
+
|
|
89
|
+
env.to_i
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
def build_log_payload(now)
|
|
93
|
+
cfg = TjScaleRuby.configuration
|
|
94
|
+
base = {
|
|
95
|
+
timestamp: now.iso8601,
|
|
96
|
+
heroku_app: cfg.target_app,
|
|
97
|
+
target_process: cfg.target_process
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
if cfg.monitor_process == "web"
|
|
101
|
+
payload = base.dup
|
|
102
|
+
qms = web_queue_time_ms_for_payload
|
|
103
|
+
payload[:queue_time_ms] = qms unless qms.nil?
|
|
104
|
+
traffic = TjScaleRuby.flush_web_traffic_stats_for_payload!
|
|
105
|
+
payload[:job_count] = traffic[:job_count]
|
|
106
|
+
payload[:response_time_ms] = traffic[:response_time_ms] if traffic[:response_time_ms]
|
|
107
|
+
payload
|
|
108
|
+
else
|
|
109
|
+
job_count = pending_job_count(now)
|
|
110
|
+
queue_time_s = oldest_pending_queue_seconds(now)
|
|
111
|
+
base.merge(job_count: job_count, queue_time_s: queue_time_s, job_backend: backend.backend_name)
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
def http_post(uri, body:, token:)
|
|
116
|
+
Net::HTTP.start(
|
|
117
|
+
uri.host,
|
|
118
|
+
uri.port,
|
|
119
|
+
use_ssl: uri.scheme == "https",
|
|
120
|
+
read_timeout: 10,
|
|
121
|
+
open_timeout: 10
|
|
122
|
+
) do |http|
|
|
123
|
+
request = Net::HTTP::Post.new(uri)
|
|
124
|
+
request["Content-Type"] = "application/json"
|
|
125
|
+
request["Accept"] = "application/vnd.heroku+json; version=3"
|
|
126
|
+
request["LOGPLEX_DRAIN_TOKEN"] = token
|
|
127
|
+
request["Authorization"] = "Bearer #{token}"
|
|
128
|
+
request.body = body if body
|
|
129
|
+
http.request(request)
|
|
130
|
+
end
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
end
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module TjScaleRuby
|
|
4
|
+
# Records approximate router queue wait from {Heroku's X-Request-Start}[https://devcenter.heroku.com/articles/http-routing#request-queueing]
|
|
5
|
+
# so +web.1+ metrics can send +queue_time_ms+ without relying on a static ENV value.
|
|
6
|
+
#
|
|
7
|
+
# Enable with +TJ_SCALE_ENABLE_QUEUE_TIME_MIDDLEWARE=1+ (see Railtie).
|
|
8
|
+
class RackQueueTimeMiddleware
|
|
9
|
+
def initialize(app)
|
|
10
|
+
@app = app
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
def call(env)
|
|
14
|
+
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC, :float_millisecond)
|
|
15
|
+
ms = self.class.queue_wait_ms(env)
|
|
16
|
+
TjScaleRuby.record_web_queue_time_ms!(ms) unless ms.nil?
|
|
17
|
+
status, headers, body = @app.call(env)
|
|
18
|
+
elapsed = (Process.clock_gettime(Process::CLOCK_MONOTONIC, :float_millisecond) - t0).round
|
|
19
|
+
TjScaleRuby.record_web_request_finished!(elapsed)
|
|
20
|
+
[status, headers, body]
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# @return [Integer, nil] milliseconds since request entered the router, or nil if header missing/invalid
|
|
24
|
+
def self.queue_wait_ms(env)
|
|
25
|
+
start_ms = request_start_epoch_ms(env)
|
|
26
|
+
return nil unless start_ms
|
|
27
|
+
|
|
28
|
+
now_ms = (Time.now.utc.to_f * 1000).round
|
|
29
|
+
[now_ms - start_ms, 0].max
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def self.request_start_epoch_ms(env)
|
|
33
|
+
raw = env["HTTP_X_REQUEST_START"]
|
|
34
|
+
return nil if raw.nil? || raw.strip.empty?
|
|
35
|
+
|
|
36
|
+
m = raw.to_s.match(/t=(\d+(?:\.\d+)?)/i)
|
|
37
|
+
return nil unless m
|
|
38
|
+
|
|
39
|
+
v = m[1].to_f
|
|
40
|
+
v *= 1000 if v < 1_000_000_000_000 # seconds (or fractional) -> ms
|
|
41
|
+
v.round
|
|
42
|
+
end
|
|
43
|
+
private_class_method :request_start_epoch_ms
|
|
44
|
+
end
|
|
45
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: tj-scale
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Tanuj
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 2026-06-12 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: rails
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '6.1'
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '9'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: '6.1'
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '9'
|
|
32
|
+
- !ruby/object:Gem::Dependency
|
|
33
|
+
name: openssl
|
|
34
|
+
requirement: !ruby/object:Gem::Requirement
|
|
35
|
+
requirements:
|
|
36
|
+
- - "~>"
|
|
37
|
+
- !ruby/object:Gem::Version
|
|
38
|
+
version: '3.2'
|
|
39
|
+
type: :runtime
|
|
40
|
+
prerelease: false
|
|
41
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
42
|
+
requirements:
|
|
43
|
+
- - "~>"
|
|
44
|
+
- !ruby/object:Gem::Version
|
|
45
|
+
version: '3.2'
|
|
46
|
+
- !ruby/object:Gem::Dependency
|
|
47
|
+
name: delayed_job_active_record
|
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
|
49
|
+
requirements:
|
|
50
|
+
- - "~>"
|
|
51
|
+
- !ruby/object:Gem::Version
|
|
52
|
+
version: '4.1'
|
|
53
|
+
type: :development
|
|
54
|
+
prerelease: false
|
|
55
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
56
|
+
requirements:
|
|
57
|
+
- - "~>"
|
|
58
|
+
- !ruby/object:Gem::Version
|
|
59
|
+
version: '4.1'
|
|
60
|
+
- !ruby/object:Gem::Dependency
|
|
61
|
+
name: bundler
|
|
62
|
+
requirement: !ruby/object:Gem::Requirement
|
|
63
|
+
requirements:
|
|
64
|
+
- - ">="
|
|
65
|
+
- !ruby/object:Gem::Version
|
|
66
|
+
version: '2.0'
|
|
67
|
+
- - "<"
|
|
68
|
+
- !ruby/object:Gem::Version
|
|
69
|
+
version: '5'
|
|
70
|
+
type: :development
|
|
71
|
+
prerelease: false
|
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
73
|
+
requirements:
|
|
74
|
+
- - ">="
|
|
75
|
+
- !ruby/object:Gem::Version
|
|
76
|
+
version: '2.0'
|
|
77
|
+
- - "<"
|
|
78
|
+
- !ruby/object:Gem::Version
|
|
79
|
+
version: '5'
|
|
80
|
+
- !ruby/object:Gem::Dependency
|
|
81
|
+
name: rake
|
|
82
|
+
requirement: !ruby/object:Gem::Requirement
|
|
83
|
+
requirements:
|
|
84
|
+
- - "~>"
|
|
85
|
+
- !ruby/object:Gem::Version
|
|
86
|
+
version: '13.0'
|
|
87
|
+
type: :development
|
|
88
|
+
prerelease: false
|
|
89
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
90
|
+
requirements:
|
|
91
|
+
- - "~>"
|
|
92
|
+
- !ruby/object:Gem::Version
|
|
93
|
+
version: '13.0'
|
|
94
|
+
- !ruby/object:Gem::Dependency
|
|
95
|
+
name: rspec
|
|
96
|
+
requirement: !ruby/object:Gem::Requirement
|
|
97
|
+
requirements:
|
|
98
|
+
- - "~>"
|
|
99
|
+
- !ruby/object:Gem::Version
|
|
100
|
+
version: '3.0'
|
|
101
|
+
type: :development
|
|
102
|
+
prerelease: false
|
|
103
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
104
|
+
requirements:
|
|
105
|
+
- - "~>"
|
|
106
|
+
- !ruby/object:Gem::Version
|
|
107
|
+
version: '3.0'
|
|
108
|
+
- !ruby/object:Gem::Dependency
|
|
109
|
+
name: standard
|
|
110
|
+
requirement: !ruby/object:Gem::Requirement
|
|
111
|
+
requirements:
|
|
112
|
+
- - "~>"
|
|
113
|
+
- !ruby/object:Gem::Version
|
|
114
|
+
version: '1.3'
|
|
115
|
+
type: :development
|
|
116
|
+
prerelease: false
|
|
117
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
118
|
+
requirements:
|
|
119
|
+
- - "~>"
|
|
120
|
+
- !ruby/object:Gem::Version
|
|
121
|
+
version: '1.3'
|
|
122
|
+
- !ruby/object:Gem::Dependency
|
|
123
|
+
name: webmock
|
|
124
|
+
requirement: !ruby/object:Gem::Requirement
|
|
125
|
+
requirements:
|
|
126
|
+
- - "~>"
|
|
127
|
+
- !ruby/object:Gem::Version
|
|
128
|
+
version: '3.0'
|
|
129
|
+
type: :development
|
|
130
|
+
prerelease: false
|
|
131
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
132
|
+
requirements:
|
|
133
|
+
- - "~>"
|
|
134
|
+
- !ruby/object:Gem::Version
|
|
135
|
+
version: '3.0'
|
|
136
|
+
- !ruby/object:Gem::Dependency
|
|
137
|
+
name: pry
|
|
138
|
+
requirement: !ruby/object:Gem::Requirement
|
|
139
|
+
requirements:
|
|
140
|
+
- - "~>"
|
|
141
|
+
- !ruby/object:Gem::Version
|
|
142
|
+
version: '0.14'
|
|
143
|
+
type: :development
|
|
144
|
+
prerelease: false
|
|
145
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
146
|
+
requirements:
|
|
147
|
+
- - "~>"
|
|
148
|
+
- !ruby/object:Gem::Version
|
|
149
|
+
version: '0.14'
|
|
150
|
+
description: |
|
|
151
|
+
TjScaleRuby reports metrics from your Rails app to an TJ Scale (or compatible)
|
|
152
|
+
control plane so it can autoscale Heroku formation from rules you configure there.
|
|
153
|
+
|
|
154
|
+
Behavior at a glance:
|
|
155
|
+
- Runs a background loop on exactly one dyno: web.1 or worker.1 (configurable).
|
|
156
|
+
- Worker reporter: waiting job count and oldest-job age (queue_time_s) from Delayed Job
|
|
157
|
+
or Sidekiq — pick with TJ_SCALE_JOB_BACKEND (auto-detected by default).
|
|
158
|
+
- Web reporter: router queue_time_ms (Rack middleware + X-Request-Start), request volume,
|
|
159
|
+
and average response time per reporting interval.
|
|
160
|
+
- Sends target Heroku app and process type on every POST; scaling limits
|
|
161
|
+
(min/max dynos) are configured in the control plane's dashboard settings.
|
|
162
|
+
- Configure with environment variables; no Rails initializer required.
|
|
163
|
+
|
|
164
|
+
Requires Rails 6.1+ and Ruby 3.0+. Bring your own queue gem: delayed_job_active_record
|
|
165
|
+
or sidekiq (worker reporters only).
|
|
166
|
+
email:
|
|
167
|
+
- tanuj@untechnickle.com
|
|
168
|
+
executables: []
|
|
169
|
+
extensions: []
|
|
170
|
+
extra_rdoc_files: []
|
|
171
|
+
files:
|
|
172
|
+
- CHANGELOG.md
|
|
173
|
+
- LICENSE.txt
|
|
174
|
+
- README.md
|
|
175
|
+
- lib/tj-scale.rb
|
|
176
|
+
- lib/tj_scale_ruby/configuration.rb
|
|
177
|
+
- lib/tj_scale_ruby/job_backends.rb
|
|
178
|
+
- lib/tj_scale_ruby/job_backends/delayed_job.rb
|
|
179
|
+
- lib/tj_scale_ruby/job_backends/sidekiq.rb
|
|
180
|
+
- lib/tj_scale_ruby/models/tj_scale_ruby.rb
|
|
181
|
+
- lib/tj_scale_ruby/rack_queue_time_middleware.rb
|
|
182
|
+
- lib/tj_scale_ruby/version.rb
|
|
183
|
+
homepage: https://github.com/Untechnickle/tj-scale-gem
|
|
184
|
+
licenses:
|
|
185
|
+
- MIT
|
|
186
|
+
metadata:
|
|
187
|
+
source_code_uri: https://github.com/Untechnickle/tj-scale-gem
|
|
188
|
+
changelog_uri: https://github.com/Untechnickle/tj-scale-gem/blob/main/CHANGELOG.md
|
|
189
|
+
bug_tracker_uri: https://github.com/Untechnickle/tj-scale-gem/issues
|
|
190
|
+
documentation_uri: https://github.com/Untechnickle/tj-scale-gem#readme
|
|
191
|
+
rdoc_options: []
|
|
192
|
+
require_paths:
|
|
193
|
+
- lib
|
|
194
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
195
|
+
requirements:
|
|
196
|
+
- - ">="
|
|
197
|
+
- !ruby/object:Gem::Version
|
|
198
|
+
version: 3.0.0
|
|
199
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
200
|
+
requirements:
|
|
201
|
+
- - ">="
|
|
202
|
+
- !ruby/object:Gem::Version
|
|
203
|
+
version: '0'
|
|
204
|
+
requirements: []
|
|
205
|
+
rubygems_version: 3.6.2
|
|
206
|
+
specification_version: 4
|
|
207
|
+
summary: 'Rails gem: Delayed Job / Sidekiq queue metrics for Heroku auto-scaling (web
|
|
208
|
+
or worker dynos)'
|
|
209
|
+
test_files: []
|