esirb 0.2.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +98 -0
- data/README.md +412 -19
- data/lib/esirb/client.rb +142 -37
- data/lib/esirb/compatibility_date.rb +41 -0
- data/lib/esirb/configuration.rb +59 -0
- data/lib/esirb/error_limit_guard.rb +62 -0
- data/lib/esirb/errors.rb +107 -0
- data/lib/esirb/pagination.rb +105 -0
- data/lib/esirb/path.rb +90 -14
- data/lib/esirb/raise_error.rb +27 -0
- data/lib/esirb/rate_limit.rb +58 -0
- data/lib/esirb/retry.rb +74 -0
- data/lib/esirb/vary_by_authorization.rb +19 -0
- data/lib/esirb/version.rb +1 -1
- data/lib/esirb.rb +9 -4
- data/lib/generators/esirb/install/install_generator.rb +35 -0
- data/lib/generators/esirb/install/templates/esirb.rb.tt +35 -0
- metadata +20 -10
- data/.rspec +0 -3
- data/.rubocop.yml +0 -8
- data/.ruby-version +0 -1
- data/Rakefile +0 -12
- data/sig/esirb.rbs +0 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 94f6c3136baf9fdac67f6e0a354a5272bdbca9e3e7b5674442257b3ecb39cd36
|
|
4
|
+
data.tar.gz: ecfad5873e07d063ed5c3c27e214f2e4533aedcb5a7113a8258bc153b043fe9f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 69d2bc63875011b320e4c7488d1ca9b5c394fa33ceb1dbb24ecdd3929317cd90ee664d681ff0fa2bb2b973ff9fa2c66de67c870e0f75ac159da0f197ea514b4f
|
|
7
|
+
data.tar.gz: 89f8b30dd0e1daae91b4d5bf91fbc79d34965fa67dd4efa4e513882a7a4f9903f9cb4674a7fa800c6d8574a23d61275b2ef6f847f66190a582f8b08041afd6a7
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,103 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.4.0] - 2026-10-25
|
|
4
|
+
|
|
5
|
+
Almost every part of esirb changed in this release, and much of it is **not backwards compatible** with 0.3.0. Read the Changed and Removed sections before upgrading.
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Requires Ruby 3.3 or newer.
|
|
10
|
+
- GET, HEAD, PUT and DELETE requests are retried up to twice after a 502, 503 or 504, a timeout or a connection failure, honoring `Retry-After`.
|
|
11
|
+
- 420 raises `Esirb::ErrorLimited` and 429 raises `Esirb::TooManyRequests`, both subclasses of `Esirb::RateLimited`.
|
|
12
|
+
- A 429 is retried after its `Retry-After` when that's 10 seconds or less, and nothing is retried once fewer than 20 errors are left in ESI's error limit window.
|
|
13
|
+
- 502, 503 and 504 raise `Esirb::BadGateway`, `Esirb::ServiceUnavailable` and `Esirb::GatewayTimeout`, which are subclasses of `Esirb::ServerError`.
|
|
14
|
+
- Requests time out after 30 seconds waiting for a response and 5 seconds opening the connection. Before, they used the adapter's defaults (60 seconds each for Net::HTTP).
|
|
15
|
+
- The default ESI compatibility date is `2026-08-18` (was `2026-06-09`). Set `compatibility_date` to keep the old one.
|
|
16
|
+
- Headers and params passed to an HTTP verb are merged with the defaults instead of replacing them. A per-request header replaces a default with the same name.
|
|
17
|
+
- Requests no longer send empty `If-Modified-Since` and `If-None-Match` headers.
|
|
18
|
+
- The User-Agent is `esirb/<version> (+https://github.com/sicks/esirb)` instead of Faraday's.
|
|
19
|
+
- Each chained call returns a new, frozen `Esirb::Path`, so a partial path can be reused without changing it.
|
|
20
|
+
- `Esirb::Path#string` is now `Esirb::Path#to_s`, so a path works in string interpolation.
|
|
21
|
+
- Path params are URL-encoded. `nil`, blank, `.`, `..` and anything other than a string, symbol or integer raise `ArgumentError`.
|
|
22
|
+
- Keyword arguments in a path (`client.markets(1, type_id: 34)`) raise `ArgumentError`. Pass query params to the HTTP verb with `params:`.
|
|
23
|
+
- Only lowercase names that don't start with `to_` become path segments. Other missing methods raise `NoMethodError` as usual, so conversions like `Array#flatten` work.
|
|
24
|
+
- HTTP verbs raise `ArgumentError` on unknown options, and a `false` body is sent instead of dropped.
|
|
25
|
+
- `Esirb::Path.new` requires a client.
|
|
26
|
+
- `Esirb::Client.new` raises `ArgumentError` on unknown options.
|
|
27
|
+
- `compatibility_date` can't be `nil`, since a request without a date gets ESI's oldest behavior, and must be a `YYYY-MM-DD` string or a `Date` that isn't in the future (ESI's day starts at 11:00 UTC). These are checked when the client is created, instead of failing with a 400 on the first request.
|
|
28
|
+
- A client builds its Faraday connection when it's created instead of on first use, and keeps no state from building paths, so one client can be shared between threads.
|
|
29
|
+
- Responses are cached only when a `cache_store` is set, in `Esirb.configure` or per client. Before, every client cached in its own unbounded in-memory store by default.
|
|
30
|
+
- The cache stores raw response bodies as JSON instead of Marshal data.
|
|
31
|
+
- Array query params are sent as repeated keys (`label_ids=1&label_ids=2`), which is what ESI reads, instead of Faraday's Rails-style `label_ids[]=1&label_ids[]=2`.
|
|
32
|
+
- Responses with any JSON content type are parsed, including `+json` types like ESI's `application/openapi+json`. Before, only `application/json` was.
|
|
33
|
+
- Errors are esirb's own classes instead of Faraday's: error statuses raise `Esirb::ResponseError` subclasses, and connection failures and timeouts raise `Esirb::ConnectionFailed` and `Esirb::TimeoutError`, with the Faraday error as the `cause`. Code that rescues `Faraday::Error` needs to rescue `Esirb::Error` instead.
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
|
|
37
|
+
- A cache store shared between clients could serve one token's response to another, or a response in the wrong language, tenant or compatibility date. Cache entries are now keyed on the URL plus ESI's `Vary` headers and the token. Entries written by older versions aren't read, so the cache starts empty after upgrading.
|
|
38
|
+
|
|
39
|
+
### Added
|
|
40
|
+
|
|
41
|
+
- `Esirb.configure` sets defaults for new clients: `base_url`, `compatibility_date`, `tenant`, `language`, `user_agent` and `cache_store`. Each can also be passed to `Esirb::Client.new`.
|
|
42
|
+
- Authenticated responses (`Cache-Control: private`) are cached per access token.
|
|
43
|
+
- `rails generate esirb:install` writes `config/initializers/esirb.rb` with esirb's settings. esirb doesn't depend on Rails; the generator only loads when Rails runs it.
|
|
44
|
+
- Pagination helpers for ESI's three pagination styles: `get_all_pages` (`page` and `X-Pages`, restarting if the data changes partway through), `get_by_cursor` (`before`/`after` cursors, returning the `after` token to save) and `get_from_id` (`from_id`).
|
|
45
|
+
- With a `cache_store`, esirb stops sending requests while ESI's error limit is nearly used up (fewer than 20 errors left), across every client and process sharing the store, and raises `Esirb::ErrorLimitReached` with `retry_after` instead.
|
|
46
|
+
- `Esirb::RateLimit` reads ESI's rate limit and error limit headers from a response, and `Esirb::ResponseError#rate_limit` from an error.
|
|
47
|
+
- A `retries` option (default 2; 0 turns retries off).
|
|
48
|
+
- `timeout` and `open_timeout` options, and a `faraday` option or client block for adding middleware, changing the adapter or setting other connection options.
|
|
49
|
+
- `token:` accepts a callable, which is called before every request, so a long-lived client can use your app's refreshed access tokens. A string still works as before.
|
|
50
|
+
- An error hierarchy under `Esirb::Error`: `BadRequest`, `Unauthorized`, `Forbidden`, `NotFound`, `RateLimited` (with `retry_after`), `ClientError` and `ServerError`, each exposing `status`, `body`, `headers` and `response`.
|
|
51
|
+
- `military_campaigns`, `paragon_hub` and `recommended_fit` reach their hyphenated ESI segments, and `rake esi:check_segments` now checks every compatibility date.
|
|
52
|
+
- `user_agent:` client option for your app's name and contact info, sent before esirb's own User-Agent.
|
|
53
|
+
- Hyphenated ESI segments can be called with underscores: `client.characters(1).freelance_jobs` requests `/characters/1/freelance-jobs`. Only ESI's known hyphenated segments are converted.
|
|
54
|
+
- `#segment` on clients and paths adds a segment exactly as given, for segments esirb can't reach by method name.
|
|
55
|
+
- `rake esi:check_segments` compares esirb's list of hyphenated segments with ESI's OpenAPI spec.
|
|
56
|
+
- `respond_to?` reports path segment names.
|
|
57
|
+
|
|
58
|
+
### Removed
|
|
59
|
+
|
|
60
|
+
- `Esirb::Client#path`, which held the last path built. `client.path` is now an ordinary `/path` segment.
|
|
61
|
+
- The `raise_errors:` client option. Error statuses always raise; rescue the `Esirb::ResponseError` subclass when a status is expected, and use `e.response`, `e.status` or `e.body` to inspect it.
|
|
62
|
+
- The `cache:` client option. Pass `cache_store: nil` to turn caching off for one client.
|
|
63
|
+
- `Esirb::Client::BASE_URL` and `Esirb::Client::DEFAULT_HEADERS`. Use `Esirb.configuration` or the client's readers (`base_url`, `compatibility_date`, ...).
|
|
64
|
+
- The `trace` HTTP verb. ESI doesn't support it, and `trace` now works as a path segment.
|
|
65
|
+
- The `sig/esirb.rbs` type signatures, which only declared `VERSION`.
|
|
66
|
+
|
|
67
|
+
## [0.3.0] - 2026-07-30
|
|
68
|
+
|
|
69
|
+
### Fixed
|
|
70
|
+
|
|
71
|
+
- The `tenant:` and `language:` client options are sent as `X-Tenant` and `Accept-Language`. Before, they were accepted but ignored, and the defaults were always sent.
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
|
|
75
|
+
- Specs for the client and paths.
|
|
76
|
+
|
|
77
|
+
## [0.2.2] - 2026-07-01
|
|
78
|
+
|
|
79
|
+
### Added
|
|
80
|
+
|
|
81
|
+
- `raise_errors:` client option (default `true`): error statuses raise Faraday's errors.
|
|
82
|
+
|
|
83
|
+
## [0.2.1] - 2026-07-01
|
|
84
|
+
|
|
85
|
+
### Fixed
|
|
86
|
+
|
|
87
|
+
- `faraday` and `faraday-http-cache` are declared as runtime dependencies, so installing esirb installs them.
|
|
88
|
+
|
|
89
|
+
## [0.2.0] - 2026-07-01
|
|
90
|
+
|
|
91
|
+
### Added
|
|
92
|
+
|
|
93
|
+
- `Esirb::Client` and `Esirb::Path`: build an ESI path by chaining method calls (`client.characters(123).location`), then send it with an HTTP verb, passing `params:`, `body:` and `headers:`.
|
|
94
|
+
- `token:`, `tenant:`, `language:`, `cache:` and `cache_store:` client options, with responses cached through faraday-http-cache.
|
|
95
|
+
- A README.
|
|
96
|
+
|
|
97
|
+
### Changed
|
|
98
|
+
|
|
99
|
+
- The gem is named `esirb` (it started as `ruby_esi`).
|
|
100
|
+
|
|
3
101
|
## [0.1.0] - 2026-06-04
|
|
4
102
|
|
|
5
103
|
- Initial release
|
data/README.md
CHANGED
|
@@ -1,9 +1,28 @@
|
|
|
1
1
|
# Esirb
|
|
2
2
|
|
|
3
|
-
Esirb is a light wrapper around Faraday for the purposes of interacting with
|
|
3
|
+
Esirb is a light wrapper around Faraday for the purposes of interacting with EVE Online's ESI API.
|
|
4
4
|
|
|
5
5
|
I wanted something low maintenance, so this gem makes a few assumptions to let you dynamically generate any path that might end up on the ESI.
|
|
6
6
|
|
|
7
|
+
## Contents
|
|
8
|
+
|
|
9
|
+
- [Installation](#installation)
|
|
10
|
+
- [Usage](#usage)
|
|
11
|
+
- [Quick start](#quick-start)
|
|
12
|
+
- [Create a client](#create-a-client)
|
|
13
|
+
- [Build the path](#build-the-path)
|
|
14
|
+
- [Make the request](#make-the-request)
|
|
15
|
+
- [Pagination](#pagination)
|
|
16
|
+
- [Configuration](#configuration)
|
|
17
|
+
- [Customizing the connection](#customizing-the-connection)
|
|
18
|
+
- [Caching](#caching)
|
|
19
|
+
- [Errors](#errors)
|
|
20
|
+
- [Retries](#retries)
|
|
21
|
+
- [Rate limits](#rate-limits)
|
|
22
|
+
- [Development](#development)
|
|
23
|
+
- [Contributing](#contributing)
|
|
24
|
+
- [License](#license)
|
|
25
|
+
|
|
7
26
|
## Installation
|
|
8
27
|
|
|
9
28
|
Install the gem and add to the application's Gemfile by executing:
|
|
@@ -20,60 +39,434 @@ gem install esirb
|
|
|
20
39
|
|
|
21
40
|
## Usage
|
|
22
41
|
|
|
42
|
+
### Quick start
|
|
43
|
+
|
|
44
|
+
```ruby
|
|
45
|
+
client = Esirb::Client.new
|
|
46
|
+
alliance_ids = client.alliances.get.body # => [99000006, 99000008, ...]
|
|
47
|
+
```
|
|
23
48
|
|
|
24
49
|
### Create a client
|
|
50
|
+
|
|
25
51
|
```ruby
|
|
26
52
|
client = Esirb::Client.new
|
|
27
53
|
```
|
|
28
|
-
|
|
54
|
+
|
|
55
|
+
A client holds no state from building paths or making requests, so one client can be shared between threads.
|
|
56
|
+
|
|
57
|
+
#### Create a client with a token
|
|
58
|
+
|
|
29
59
|
```ruby
|
|
30
60
|
client = Esirb::Client.new(token: "...")
|
|
31
61
|
```
|
|
32
62
|
|
|
33
|
-
|
|
63
|
+
ESI access tokens expire after about 20 minutes. If a client lives longer than that, pass something that returns the current token instead, such as a lambda. esirb calls it before every request:
|
|
34
64
|
|
|
35
|
-
|
|
65
|
+
```ruby
|
|
66
|
+
client = Esirb::Client.new(token: -> { character.access_token_refreshing_if_expired })
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
esirb doesn't refresh tokens itself: your app keeps the refresh token and does the refresh, and esirb only asks for the access token. If the callable returns `nil` or a blank string, the request raises `Esirb::Error` instead of being sent.
|
|
70
|
+
|
|
71
|
+
#### Identify your app
|
|
72
|
+
|
|
73
|
+
CCP asks every ESI client to send a User-Agent with the app's name and a way to contact you. Pass yours as `user_agent:`, and esirb adds its own name after it:
|
|
36
74
|
|
|
37
75
|
```ruby
|
|
76
|
+
client = Esirb::Client.new(user_agent: "mappy/1.2 (admin@example.com)")
|
|
77
|
+
client.user_agent # => "mappy/1.2 (admin@example.com) esirb/0.4.0 (+https://github.com/sicks/esirb)"
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Build the path
|
|
38
81
|
|
|
39
|
-
path
|
|
40
|
-
|
|
82
|
+
Treat ESI's static path segments as method names, and path parameters as the arguments for those methods. Chain these methods to build the path to the desired ESI route.
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
path = client.route(30000142, 30002187)
|
|
86
|
+
path.to_s # => "/route/30000142/30002187"
|
|
41
87
|
|
|
42
88
|
path = client.ui.autopilot.waypoint
|
|
43
|
-
path.
|
|
89
|
+
path.to_s # => "/ui/autopilot/waypoint"
|
|
44
90
|
|
|
45
91
|
path = client.characters(123456789).location
|
|
46
|
-
path.
|
|
92
|
+
path.to_s # => "/characters/123456789/location"
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Each call returns a new path, so you can reuse part of one:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
character = client.characters(123456789)
|
|
99
|
+
character.location.to_s # => "/characters/123456789/location"
|
|
100
|
+
character.skills.to_s # => "/characters/123456789/skills"
|
|
47
101
|
```
|
|
48
102
|
|
|
49
|
-
|
|
103
|
+
Path params can be strings, symbols or integers, and are URL-encoded. Query params don't go in the path; pass them to the HTTP verb with `params:`.
|
|
50
104
|
|
|
51
|
-
|
|
105
|
+
Some ESI segments, like `freelance-jobs`, use hyphens, which Ruby method names can't contain. Call them with underscores instead:
|
|
52
106
|
|
|
53
107
|
```ruby
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
108
|
+
client.characters(123456789).freelance_jobs.to_s # => "/characters/123456789/freelance-jobs"
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
esirb keeps a list of ESI's hyphenated segments, and only those names are converted. Segments that really use underscores, like `system_kills`, stay as they are. For a segment esirb doesn't know about yet, add it exactly as written with `segment`:
|
|
57
112
|
|
|
58
|
-
|
|
59
|
-
|
|
113
|
+
```ruby
|
|
114
|
+
client.characters(123456789).segment("new-thing", 42).to_s # => "/characters/123456789/new-thing/42"
|
|
60
115
|
```
|
|
61
116
|
|
|
62
|
-
###
|
|
117
|
+
### Make the request
|
|
118
|
+
|
|
119
|
+
Call an HTTP verb on the path: `get`, `head`, `post`, `put`, `patch` or `delete`. Each returns a `Faraday::Response`, with JSON bodies already parsed.
|
|
120
|
+
|
|
63
121
|
```ruby
|
|
64
|
-
client = Esirb::Client.new
|
|
65
|
-
|
|
122
|
+
client = Esirb::Client.new(token: -> { character.access_token }) # needs the esi-location.read_location.v1 scope
|
|
123
|
+
resp = client.characters(123456789).location.get
|
|
124
|
+
resp.status # => 200
|
|
125
|
+
resp.body # => { "solar_system_id" => 30000142 }
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Pass query parameters with `params:`, a body with `body:` (sent as JSON), and extra headers with `headers:`. The [API Explorer](https://developers.eveonline.com/api-explorer) lists what each route takes.
|
|
129
|
+
|
|
130
|
+
```ruby
|
|
131
|
+
client.markets(10000002).orders.get(params: { type_id: 34, order_type: "sell" })
|
|
132
|
+
client.universe.names.post(body: [30000142, 30002187])
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Headers passed to a verb are added to the client's default headers, and replace a default with the same name.
|
|
136
|
+
|
|
137
|
+
### Pagination
|
|
138
|
+
|
|
139
|
+
ESI splits long lists into pages, using one of three styles depending on the route. You can tell which from the route's query parameters in ESI's API Explorer:
|
|
140
|
+
|
|
141
|
+
| The route takes | Style | Use | Typical routes |
|
|
142
|
+
|---|---|---|---|
|
|
143
|
+
| `page` | X-Pages | `get_all_pages` | market orders, assets, contracts, wallet journals |
|
|
144
|
+
| `before` and `after` | Cursor | `get_by_cursor` | freelance jobs, corporation projects, Paragon hub |
|
|
145
|
+
| `from_id` | From-ID | `get_from_id` | wallet transactions |
|
|
146
|
+
|
|
147
|
+
Each helper makes as many requests as it takes to collect the pages, so one call can be dozens of requests, each counting toward ESI's rate limit. They're sent one at a time, never in parallel. Each request goes through esirb's cache, retries and error-limit guard like any other, and keeps the `params:` and `headers:` you pass. If any request fails, the error is raised and nothing is returned.
|
|
148
|
+
|
|
149
|
+
#### `get_all_pages`: X-Pages ([ESI docs](https://developers.eveonline.com/docs/services/esi/pagination/x-pages/))
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
orders = client.markets(10000002).orders.get_all_pages(params: { type_id: 34 })
|
|
153
|
+
# => one array with every order from every page
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Under the hood, that's the same as fetching the pages yourself:
|
|
157
|
+
|
|
158
|
+
```ruby
|
|
159
|
+
orders = client.markets(10000002).orders
|
|
160
|
+
|
|
161
|
+
first = orders.get(params: { type_id: 34, page: 1 })
|
|
162
|
+
total_pages = first.headers["X-Pages"].to_i
|
|
163
|
+
records = first.body
|
|
164
|
+
|
|
165
|
+
(2..total_pages).each do |page|
|
|
166
|
+
response = orders.get(params: { type_id: 34, page: })
|
|
167
|
+
# get_all_pages also checks this, and starts over from page 1 if it doesn't match:
|
|
168
|
+
raise "data changed between pages" unless response.headers["Last-Modified"] == first.headers["Last-Modified"]
|
|
169
|
+
|
|
170
|
+
records.concat(response.body)
|
|
171
|
+
end
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Every page of one list should have the same `Last-Modified`. A different one means ESI refreshed the data partway through, so the pages would mix old and new records. `get_all_pages` then starts over from page 1 once, and raises `Esirb::PagesChanged` if it happens again. Fetching the whole list takes as many requests as there are pages, or up to twice that if it has to start over.
|
|
175
|
+
|
|
176
|
+
#### `get_by_cursor`: Cursor ([ESI docs](https://developers.eveonline.com/docs/services/esi/pagination/cursor-based/))
|
|
177
|
+
|
|
178
|
+
These responses aren't a plain list. They're an object holding the records under a key that depends on the route (`freelance_jobs`, `projects`, `listings`...), plus a `cursor` with two values: `before`, pointing at older records, and `after`, pointing at newer ones. `get_by_cursor` finds the records' key for you. There are two ways to call it.
|
|
179
|
+
|
|
180
|
+
**Everything,** for a first sync:
|
|
181
|
+
|
|
182
|
+
```ruby
|
|
183
|
+
jobs, after = client.freelance_jobs.get_by_cursor
|
|
184
|
+
# jobs: every record, oldest first
|
|
185
|
+
# after: save this to fetch only changes later
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Done by hand, that's:
|
|
189
|
+
|
|
190
|
+
```ruby
|
|
191
|
+
freelance_jobs = client.freelance_jobs
|
|
192
|
+
|
|
193
|
+
response = freelance_jobs.get # no cursor: the most recent records
|
|
194
|
+
jobs = response.body["freelance_jobs"]
|
|
195
|
+
after = response.body.dig("cursor", "after") # where the list ends now; save it for later
|
|
196
|
+
before = response.body.dig("cursor", "before")
|
|
197
|
+
|
|
198
|
+
while before
|
|
199
|
+
response = freelance_jobs.get(params: { before: })
|
|
200
|
+
older = response.body["freelance_jobs"]
|
|
201
|
+
break if older.empty?
|
|
202
|
+
|
|
203
|
+
jobs = older + jobs # get_by_cursor also skips records it already has, matched by id
|
|
204
|
+
before = response.body.dig("cursor", "before")
|
|
205
|
+
end
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
**Only what changed since** a saved `after`, to keep up to date:
|
|
209
|
+
|
|
210
|
+
```ruby
|
|
211
|
+
changed, after = client.freelance_jobs.get_by_cursor(after: saved_after)
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Done by hand:
|
|
215
|
+
|
|
216
|
+
```ruby
|
|
217
|
+
changed = []
|
|
218
|
+
after = saved_after
|
|
219
|
+
|
|
220
|
+
loop do
|
|
221
|
+
response = freelance_jobs.get(params: { after: })
|
|
222
|
+
newer = response.body["freelance_jobs"]
|
|
223
|
+
break if newer.empty?
|
|
224
|
+
|
|
225
|
+
changed += newer # get_by_cursor also replaces records it already has with these newer copies
|
|
226
|
+
after = response.body.dig("cursor", "after") || break
|
|
227
|
+
end
|
|
228
|
+
# Save `after` again for next time.
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
`after` and `before` values stay valid for days, according to CCP. Pass them back exactly as you got them, without trying to decode them. A record can show up on more than one page, for example when it changed while the pages were being fetched. `get_by_cursor` returns each record once, matched by its `id`: when following `before` toward older records it keeps the copy it already had, and when following `after` toward newer records it keeps the newest copy.
|
|
232
|
+
|
|
233
|
+
#### `get_from_id`: From-ID ([ESI docs](https://developers.eveonline.com/docs/services/esi/pagination/from-id/))
|
|
234
|
+
|
|
235
|
+
ESI returns the newest records first. To get older ones, you pass the oldest ID you've seen as `from_id`. `get_from_id` does that for you, one request after another, so you never pass `from_id` yourself.
|
|
236
|
+
|
|
237
|
+
**Everything, newest first:**
|
|
238
|
+
|
|
239
|
+
```ruby
|
|
240
|
+
transactions = client.characters(123456789).wallet.transactions
|
|
241
|
+
all = transactions.get_from_id
|
|
66
242
|
```
|
|
67
243
|
|
|
244
|
+
Done by hand:
|
|
245
|
+
|
|
246
|
+
```ruby
|
|
247
|
+
records = transactions.get.body # the newest records, newest first
|
|
248
|
+
|
|
249
|
+
loop do
|
|
250
|
+
from_id = records.last["transaction_id"]
|
|
251
|
+
# The response starts with the from_id record again, followed by older ones.
|
|
252
|
+
older = transactions.get(params: { from_id: }).body.reject { |record| record["transaction_id"] == from_id }
|
|
253
|
+
break if older.empty?
|
|
254
|
+
|
|
255
|
+
records.concat(older)
|
|
256
|
+
end
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
**Only records newer than one you already have:**
|
|
260
|
+
|
|
261
|
+
```ruby
|
|
262
|
+
newer = transactions.get_from_id(stop_at: newest_known_id)
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Done by hand:
|
|
266
|
+
|
|
267
|
+
```ruby
|
|
268
|
+
newer = []
|
|
269
|
+
from_id = nil
|
|
270
|
+
|
|
271
|
+
loop do
|
|
272
|
+
params = from_id ? { from_id: } : {}
|
|
273
|
+
page = transactions.get(params:).body.reject { |record| record["transaction_id"] == from_id }
|
|
274
|
+
stop_index = page.index { |record| record["transaction_id"] == newest_known_id }
|
|
275
|
+
break newer.concat(page.first(stop_index)) if stop_index
|
|
276
|
+
break if page.empty?
|
|
277
|
+
|
|
278
|
+
newer.concat(page)
|
|
279
|
+
from_id = page.last["transaction_id"]
|
|
280
|
+
end
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
`stop_at` is esirb's option, not an ESI parameter: an ID you already have, usually the newest one you stored last time. The walk ends when it reaches that record, which isn't included. If it's on the first page, that's one request. If ESI no longer has it, the walk fetches everything.
|
|
284
|
+
|
|
285
|
+
Records are identified by `transaction_id`, which is what wallet transactions use. Pass `id_key:` if a route uses a different field.
|
|
286
|
+
|
|
287
|
+
### Configuration
|
|
288
|
+
|
|
289
|
+
In a Rails app, generate an initializer with each setting and a note on what it does:
|
|
290
|
+
|
|
291
|
+
```bash
|
|
292
|
+
bin/rails generate esirb:install
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
That writes `config/initializers/esirb.rb`, with your app's name in the User-Agent, esirb's current compatibility date pinned, and `Rails.cache` as the cache store. Edit the User-Agent's contact email before deploying.
|
|
296
|
+
|
|
297
|
+
Outside Rails, or to write it yourself, set defaults for every new client once:
|
|
298
|
+
|
|
299
|
+
```ruby
|
|
300
|
+
Esirb.configure do |config|
|
|
301
|
+
config.user_agent = "mappy/1.2 (admin@example.com)"
|
|
302
|
+
config.compatibility_date = "2026-08-18"
|
|
303
|
+
config.cache_store = Rails.cache
|
|
304
|
+
end
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
Options passed to `Esirb::Client.new` override these for that client:
|
|
308
|
+
|
|
309
|
+
| Option | Default | Notes |
|
|
310
|
+
|---|---|---|
|
|
311
|
+
| `base_url` | `"https://esi.evetech.net"` | |
|
|
312
|
+
| `compatibility_date` | `"2026-08-18"` | Sent as `X-Compatibility-Date`, as a `"YYYY-MM-DD"` string or a `Date`. A newer date can change response shapes, so check your code against it before switching. Any date up to today works (ESI's day starts at 11:00 UTC), and ESI uses the latest published date on or before it. It can't be in the future, or `nil`: without a date, ESI uses its oldest behavior. |
|
|
313
|
+
| `tenant` | `"tranquility"` | Sent as `X-Tenant`. |
|
|
314
|
+
| `language` | `"en"` | Sent as `Accept-Language`. |
|
|
315
|
+
| `user_agent` | `nil` | Your app's name and contact info, sent before esirb's own User-Agent. |
|
|
316
|
+
| `cache_store` | `nil` | Where to cache responses. `nil` turns caching off; see [Caching](#caching). |
|
|
317
|
+
| `timeout` | `30` | Seconds to wait for a response. `nil` leaves it to the adapter. |
|
|
318
|
+
| `open_timeout` | `5` | Seconds to wait to open the connection. `nil` leaves it to the adapter. |
|
|
319
|
+
| `retries` | `2` | Times to retry a request that hit a 502, 503 or 504, a timeout or a connection failure; see [Retries](#retries). `0` turns retries off. |
|
|
320
|
+
| `faraday` | `nil` | A callable that customizes the Faraday connection; see [Customizing the connection](#customizing-the-connection). |
|
|
321
|
+
|
|
322
|
+
Setting `tenant` or `language` to `nil` leaves that header out. `token` is set per client only.
|
|
323
|
+
|
|
324
|
+
#### Compatibility dates
|
|
325
|
+
|
|
326
|
+
ESI changes a route's responses only under a new compatibility date, and a request without a date gets ESI's oldest behavior, so esirb always sends one. New fields and enum values can still appear within a date, so don't fail on ones you don't recognize. To move to a newer date, check ESI's changes since your current one against your code, then set `compatibility_date`. A new esirb version can move the default date; the CHANGELOG says when.
|
|
327
|
+
|
|
328
|
+
### Customizing the connection
|
|
329
|
+
|
|
330
|
+
esirb uses [Faraday](https://lostisland.github.io/faraday/). To add middleware, change the adapter or set other connection options, give `Esirb.configure` a `faraday` callable, or pass a block to a client. Both receive the Faraday connection builder after esirb has set up its own middleware:
|
|
331
|
+
|
|
332
|
+
```ruby
|
|
333
|
+
Esirb.configure do |config|
|
|
334
|
+
config.faraday = ->(f) { f.response :logger, Rails.logger }
|
|
335
|
+
end
|
|
336
|
+
|
|
337
|
+
client = Esirb::Client.new { |f| f.adapter :net_http_persistent }
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
(`:net_http_persistent` comes from the `faraday-net_http_persistent` gem, which you'd add to your Gemfile.) The configured callable runs first, then the client's block. Middleware added this way sits closest to the network, so a logger sees the requests that actually go to ESI, not responses served from the cache.
|
|
341
|
+
|
|
342
|
+
### Caching
|
|
343
|
+
|
|
344
|
+
esirb caches responses only when you give it a `cache_store`, usually once in `Esirb.configure` so every client shares it. Any object with `read`, `write` and `delete` works, such as `Rails.cache` or another `ActiveSupport::Cache` store.
|
|
345
|
+
|
|
346
|
+
Without a `cache_store`, every request goes to ESI, even when the last response is still fresh. CCP asks apps not to request data again before its `Expires` time or work around ESI's cache, and doing that can get an app banned, so set a store in production. The store also lets esirb [pause before the error limit runs out](#pausing-before-the-error-limit-runs-out).
|
|
347
|
+
|
|
348
|
+
```ruby
|
|
349
|
+
Esirb.configure do |config|
|
|
350
|
+
config.cache_store = ActiveSupport::Cache::RedisCacheStore.new(url: ENV["REDIS_URL"], namespace: "esirb", expires_in: 1.day)
|
|
351
|
+
end
|
|
352
|
+
|
|
353
|
+
Esirb::Client.new(cache_store: nil) # no caching for this client
|
|
354
|
+
```
|
|
355
|
+
|
|
356
|
+
Use a store that expires entries. Cache keys include the access token, and tokens change about every 20 minutes, so entries for old tokens are never read again. A store without expiry, including `Faraday::HttpCache::MemoryStore`, keeps them forever.
|
|
357
|
+
|
|
358
|
+
esirb follows ESI's cache headers:
|
|
359
|
+
|
|
360
|
+
- Responses are kept separate per URL, tenant, language, compatibility date and access token, so a shared store never serves one character's data to another.
|
|
361
|
+
- Authenticated responses are cached too. ESI marks them `private, max-age=5, must-revalidate`, so they're reused for 5 seconds and then revalidated, which returns a cheap `304 Not Modified` when nothing has changed.
|
|
362
|
+
|
|
363
|
+
### Errors
|
|
364
|
+
|
|
365
|
+
Error statuses from ESI raise a subclass of `Esirb::ResponseError`, with ESI's own message:
|
|
366
|
+
|
|
367
|
+
```ruby
|
|
368
|
+
begin
|
|
369
|
+
client.universe.systems(1).get
|
|
370
|
+
rescue Esirb::NotFound => e
|
|
371
|
+
e.message # => "404 System not found (GET https://esi.evetech.net/universe/systems/1)"
|
|
372
|
+
e.status # => 404
|
|
373
|
+
e.body # => { "error" => "System not found" }
|
|
374
|
+
e.headers # => response headers, e.g. e.headers["X-ESI-Error-Limit-Remain"]
|
|
375
|
+
end
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
| Class | Raised for |
|
|
379
|
+
|---|---|
|
|
380
|
+
| `Esirb::BadRequest` | 400 |
|
|
381
|
+
| `Esirb::Unauthorized` | 401 |
|
|
382
|
+
| `Esirb::Forbidden` | 403 |
|
|
383
|
+
| `Esirb::NotFound` | 404 |
|
|
384
|
+
| `Esirb::ErrorLimited` | 420: too many errors in ESI's error limit window. Every route is blocked until it resets. |
|
|
385
|
+
| `Esirb::TooManyRequests` | 429: a rate limit group is used up, after esirb's retries run out |
|
|
386
|
+
| `Esirb::RateLimited` | The parent of the two above. `retry_after` gives the seconds to wait, if ESI said. |
|
|
387
|
+
| `Esirb::ClientError` | Any other 4xx, and the parent of the classes above |
|
|
388
|
+
| `Esirb::BadGateway`, `Esirb::ServiceUnavailable`, `Esirb::GatewayTimeout` | 502, 503 and 504, after esirb's retries run out |
|
|
389
|
+
| `Esirb::ServerError` | Any other 5xx, and the parent of the three classes above |
|
|
390
|
+
| `Esirb::ConnectionFailed` | The connection to ESI failed, including TLS errors and timeouts while connecting |
|
|
391
|
+
| `Esirb::TimeoutError` | ESI connected but didn't answer within `timeout` |
|
|
392
|
+
| `Esirb::ErrorLimitReached` | Raised *without sending the request*, while ESI's error limit is nearly used up (needs a `cache_store`; see [Rate limits](#rate-limits)). `retry_after` gives the seconds until it resets. |
|
|
393
|
+
|
|
394
|
+
Error responses aren't free: a 4xx costs 5 rate limit tokens and counts toward ESI's [error limit](https://developers.eveonline.com/docs/services/esi/rate-limiting/), so don't keep repeating a request that fails.
|
|
395
|
+
|
|
396
|
+
Everything esirb raises is an `Esirb::Error`. When an error status is a normal answer, rescue its class. For example, ESI returns 404 for a character's fleet when they aren't in one:
|
|
397
|
+
|
|
398
|
+
```ruby
|
|
399
|
+
fleet = begin
|
|
400
|
+
client.characters(123456789).fleet.get.body
|
|
401
|
+
rescue Esirb::NotFound
|
|
402
|
+
nil
|
|
403
|
+
end
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
### Retries
|
|
407
|
+
|
|
408
|
+
ESI often answers with a 502, 503 or 504 that clears up within a second or two, so esirb retries those, along with timeouts and connection failures. By default it retries twice, waiting about half a second and then a second, with some randomness so many clients don't retry at the same moment.
|
|
409
|
+
|
|
410
|
+
- Only GET, HEAD, PUT and DELETE are retried. POST isn't, since repeating it might do something twice, like sending a mail.
|
|
411
|
+
- If ESI sends `Retry-After`, esirb waits that long instead. If it asks for more than 10 seconds, esirb raises straight away rather than tie up your process.
|
|
412
|
+
- A 429 is retried after its `Retry-After`, if that's 10 seconds or less. A 429 without `Retry-After`, which some routes send from inside EVE's servers, or one asking for longer, is raised straight away.
|
|
413
|
+
- When a failed response shows fewer than 20 errors left in ESI's [error limit](https://developers.eveonline.com/docs/services/esi/rate-limiting/), esirb stops retrying. Running out blocks every ESI route for your app until the window resets, and each failed retry counts as another error.
|
|
414
|
+
- Other errors, including 500, other 4xx and 420, aren't retried.
|
|
415
|
+
|
|
416
|
+
`retries` is the total for one request, whatever the reasons: with the default of 2, a request is sent at most 3 times. Set `retries` in `Esirb.configure` or per client to change the number, or to `0` to turn retries off. Worst case, a request takes about `(retries + 1) * timeout` seconds plus the waits, so lower `timeout` or `retries` together if that's too long.
|
|
417
|
+
|
|
418
|
+
### Rate limits
|
|
419
|
+
|
|
420
|
+
ESI limits requests in one of two ways, depending on the route ([details](https://developers.eveonline.com/docs/services/esi/rate-limiting/)). `Esirb::RateLimit` reads either from a response or an error:
|
|
421
|
+
|
|
422
|
+
```ruby
|
|
423
|
+
response = client.characters(123456789).location.get
|
|
424
|
+
limit = Esirb::RateLimit.new(response)
|
|
425
|
+
limit.group # => "char-location"
|
|
426
|
+
limit.limit # => 1200 tokens...
|
|
427
|
+
limit.window # => ...per 900 seconds
|
|
428
|
+
limit.remaining # => 1195
|
|
429
|
+
limit.used # => 2, what this request cost
|
|
430
|
+
|
|
431
|
+
# On routes still using the older error limit:
|
|
432
|
+
limit.error_limit_remaining # => 100 errors left in this window
|
|
433
|
+
limit.error_limit_reset # => 46 seconds until it resets
|
|
434
|
+
|
|
435
|
+
# Errors carry the limits too:
|
|
436
|
+
begin
|
|
437
|
+
client.characters(123456789).location.get
|
|
438
|
+
rescue Esirb::ResponseError => e
|
|
439
|
+
e.rate_limit.remaining
|
|
440
|
+
end
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
Values a route doesn't send are `nil`. Errors cost more than successful requests (5 tokens for a 4xx against 2 for a 2xx), so avoid repeating requests that fail. CCP also recommends slowing down as `remaining` nears zero, and staggering scheduled work rather than running it all at the same minute.
|
|
444
|
+
|
|
445
|
+
#### Pausing before the error limit runs out
|
|
446
|
+
|
|
447
|
+
ESI's older error limit allows about 100 error responses a minute. Once it's used up, *every* ESI route returns 420 until the window resets, and repeatedly ignoring it can get an app banned. With a `cache_store`, esirb keeps track of it for you: when a response shows fewer than 20 errors left, every client sharing that store stops sending requests until the window resets, and raises `Esirb::ErrorLimitReached` instead.
|
|
448
|
+
|
|
449
|
+
```ruby
|
|
450
|
+
begin
|
|
451
|
+
client.universe.systems(30000142).get
|
|
452
|
+
rescue Esirb::ErrorLimitReached => e
|
|
453
|
+
e.retry_after # => 37, seconds until ESI's error limit resets
|
|
454
|
+
end
|
|
455
|
+
```
|
|
456
|
+
|
|
457
|
+
ESI counts the error limit per IP address, so esirb keeps one count per ESI host and tenant, shared by every client and process using the store. If several servers with different IP addresses share one store, an error burst on one pauses the others too, which errs on the side of sending less.
|
|
458
|
+
|
|
68
459
|
## Development
|
|
69
460
|
|
|
70
461
|
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
71
462
|
|
|
463
|
+
After moving esirb's default compatibility date, run `bundle exec rake esi:check_segments` to compare esirb's list of hyphenated segments with ESI's OpenAPI spec.
|
|
464
|
+
|
|
72
465
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
73
466
|
|
|
74
467
|
## Contributing
|
|
75
468
|
|
|
76
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/
|
|
469
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/sicks/esirb.
|
|
77
470
|
|
|
78
471
|
## License
|
|
79
472
|
|