x 0.19.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.yardopts +9 -0
- data/CHANGELOG.md +648 -15
- data/README.md +506 -73
- data/SECURITY.md +66 -0
- data/UPGRADING.md +670 -0
- data/lib/x/version.rb +16 -1
- data/lib/x.rb +13 -1
- data/sig/manifest.yaml +5 -0
- data/sig/x.rbs +5 -361
- metadata +57 -49
- data/bin/console +0 -10
- data/bin/setup +0 -6
- data/lib/x/account_uploader.rb +0 -168
- data/lib/x/authenticator.rb +0 -21
- data/lib/x/bearer_token_authenticator.rb +0 -36
- data/lib/x/client.rb +0 -191
- data/lib/x/client_credentials.rb +0 -208
- data/lib/x/connection.rb +0 -183
- data/lib/x/errors/bad_gateway.rb +0 -6
- data/lib/x/errors/bad_request.rb +0 -6
- data/lib/x/errors/client_error.rb +0 -6
- data/lib/x/errors/connection_exception.rb +0 -6
- data/lib/x/errors/error.rb +0 -4
- data/lib/x/errors/forbidden.rb +0 -6
- data/lib/x/errors/gateway_timeout.rb +0 -6
- data/lib/x/errors/gone.rb +0 -6
- data/lib/x/errors/http_error.rb +0 -84
- data/lib/x/errors/internal_server_error.rb +0 -6
- data/lib/x/errors/invalid_media_type.rb +0 -6
- data/lib/x/errors/network_error.rb +0 -6
- data/lib/x/errors/not_acceptable.rb +0 -6
- data/lib/x/errors/not_found.rb +0 -6
- data/lib/x/errors/payload_too_large.rb +0 -6
- data/lib/x/errors/server_error.rb +0 -6
- data/lib/x/errors/service_unavailable.rb +0 -6
- data/lib/x/errors/too_many_redirects.rb +0 -6
- data/lib/x/errors/too_many_requests.rb +0 -58
- data/lib/x/errors/unauthorized.rb +0 -6
- data/lib/x/errors/unprocessable_entity.rb +0 -6
- data/lib/x/media_upload_validator.rb +0 -36
- data/lib/x/media_uploader.rb +0 -234
- data/lib/x/oauth2_authenticator.rb +0 -169
- data/lib/x/oauth_authenticator.rb +0 -182
- data/lib/x/rate_limit.rb +0 -89
- data/lib/x/redirect_handler.rb +0 -109
- data/lib/x/request_builder.rb +0 -93
- data/lib/x/response_parser.rb +0 -81
- data/lib/x/stream_parser.rb +0 -75
data/README.md
CHANGED
|
@@ -1,7 +1,9 @@
|
|
|
1
|
-
[](https://github.com/sferik/x-ruby/actions/workflows/x-core.yml)
|
|
2
|
+
[](https://github.com/sferik/x-ruby/actions/workflows/x-uploads.yml)
|
|
3
|
+
[](https://github.com/sferik/x-ruby/actions/workflows/x-streams.yml)
|
|
4
|
+
[](https://github.com/sferik/x-ruby/actions/workflows/x-resources.yml)
|
|
5
|
+
[](https://github.com/sferik/x-ruby/actions/workflows/x.yml)
|
|
3
6
|
[](https://github.com/sferik/x-ruby/actions/workflows/lint.yml)
|
|
4
|
-
[](https://github.com/sferik/x-ruby/actions/workflows/steep.yml)
|
|
5
7
|
[](https://qlty.sh/gh/sferik/projects/x-ruby)
|
|
6
8
|
[](https://rubygems.org/gems/x)
|
|
7
9
|
|
|
@@ -13,7 +15,7 @@ For updates and announcements, follow [this gem](https://x.com/gem) and [its cre
|
|
|
13
15
|
|
|
14
16
|
## Installation
|
|
15
17
|
|
|
16
|
-
Install the gem and add to the application's Gemfile:
|
|
18
|
+
The gems require Ruby 3.4 or later. Install the gem and add to the application's Gemfile:
|
|
17
19
|
|
|
18
20
|
bundle add x
|
|
19
21
|
|
|
@@ -21,6 +23,25 @@ Or, if Bundler is not being used to manage dependencies:
|
|
|
21
23
|
|
|
22
24
|
gem install x
|
|
23
25
|
|
|
26
|
+
## Documentation
|
|
27
|
+
|
|
28
|
+
The API documentation of every gem, `x-core`, `x-uploads`, `x-streams`, and `x-resources` together, is at [sferik.github.io/x-ruby/api](https://sferik.github.io/x-ruby/api/).
|
|
29
|
+
|
|
30
|
+
## Architecture
|
|
31
|
+
|
|
32
|
+
The `x` gem is a thin meta-gem that combines four gems, which are released from this repository in lockstep:
|
|
33
|
+
|
|
34
|
+
| Gem | What it does | Runtime dependencies |
|
|
35
|
+
| --- | --- | --- |
|
|
36
|
+
| [`x-core`](https://github.com/sferik/x-ruby/tree/main/x-core) | HTTP: authentication, requests, redirects, errors, and rate limits | `simple_oauth`, and `net-http`, a default gem |
|
|
37
|
+
| [`x-uploads`](https://github.com/sferik/x-ruby/tree/main/x-uploads) | Uploads: images, GIFs, videos, and subtitles, with videos and subtitles in chunks, plus profile images and banners | `x-core` |
|
|
38
|
+
| [`x-streams`](https://github.com/sferik/x-ruby/tree/main/x-streams) | Streams: the sample and filtered streams, reconnected as X recommends, and the rules of the filtered stream | `x-core` |
|
|
39
|
+
| [`x-resources`](https://github.com/sferik/x-ruby/tree/main/x-resources) | Resources: `User`, `Post`, `List`, `DirectMessage`, `Space`, `Community`, `Media`, `Poll`, `Place`, and cursors | `x-core` |
|
|
40
|
+
|
|
41
|
+
`require "x"` loads all four, and mixes the object methods (`find_user`, `find_all_posts`, `search_posts`, …), the upload methods (`upload_media`, `add_alt_text`, `update_profile_image`, …), and `streaming` into `X::Client`. Any other request can return objects too, given a resource class as its `object_class`. If you only want raw JSON, depend on `x-core` alone. If you want the objects with an HTTP client of your own, depend on `x-resources` alone: it makes no request itself, and takes `x-core` for the errors the X gems share. It asks the client you give it for `get`, `post`, `put`, and `delete`, each taking a path that already carries the query, an optional body for the two that send one, and `array_class:` and `object_class:`, and may be passed any other keyword `X::Client` takes within 1.x; [the client contract](https://github.com/sferik/x-ruby/tree/main/x-resources#the-client-contract) has the whole of it, and the `X::Resources::_Client` interface in [`sig/x-resources.rbs`](https://github.com/sferik/x-ruby/blob/main/x-resources/sig/x-resources.rbs) states it for a type checker.
|
|
42
|
+
|
|
43
|
+
Every class you write is named directly under `X`, whatever gem declares it: `X::Client` and `X::NotFound` from `x-core`, `X::User` and `X::MissingResource` from `x-resources`, `X::UploadedMedia` and `X::MediaProcessingFailed` from `x-uploads`, `X::StreamingClient` and `X::StreamError` from `x-streams`. Each gem's module holds its own mixins and internals. `X::Resources::Error`, `X::Uploads::Error`, and `X::Streams::Error` are the three exceptions, since each is the name a `rescue` reaches for to catch the failures of that gem alone.
|
|
44
|
+
|
|
24
45
|
## Usage
|
|
25
46
|
|
|
26
47
|
> [!NOTE]
|
|
@@ -38,21 +59,279 @@ x_credentials = {
|
|
|
38
59
|
|
|
39
60
|
# Initialize an X API client with your OAuth credentials
|
|
40
61
|
x_client = X::Client.new(**x_credentials)
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Objects
|
|
65
|
+
|
|
66
|
+
Every lookup requests all public fields and expansions, and returns immutable, thread-safe objects.
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
user = x_client.find_user("sferik") # a String is a username, an Integer is an ID
|
|
70
|
+
# find_user_by_username and find_user_by_id say which you mean
|
|
71
|
+
user.name # => "Erik Berlin"
|
|
72
|
+
user.followers_count # => 12345
|
|
73
|
+
|
|
74
|
+
post = x_client.find_post(1234567890) # X::Post
|
|
75
|
+
post.text # the full text, even of a post longer than 280 characters
|
|
76
|
+
post.created_at # => 2026-09-11 12:00:00 UTC
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Text.** Objects read every String as the API sends it, and the API escapes `&`, `<`, and `>` as `&`, `<`, and `>` in the text of a post and of a direct message, so `post.text`, `post.expanded_text`, and `message.text` hold them, and will throughout 1.x. Unescape the text to display it:
|
|
80
|
+
|
|
81
|
+
```ruby
|
|
82
|
+
require "cgi/escape"
|
|
83
|
+
|
|
84
|
+
post.text # => "Ruby & Rails"
|
|
85
|
+
CGI.unescapeHTML(post.text) # => "Ruby & Rails"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Nested data.** An object the API nests in a resource, or a list of them, such as `post.entities`, `post.urls`, `post.public_metrics`, `post.attachments`, `media.variants`, or `poll.options`, reads as the API sends it, a frozen Hash keyed by String, or an Array of them, and will throughout 1.x; a response that holds anything else in its place raises `X::InvalidAttribute`. A later 1.x release may add a reader that returns an object for some of it, as `post.matching_rules` returns `X::MatchingRule`s, but under a new name.
|
|
89
|
+
|
|
90
|
+
```ruby
|
|
91
|
+
post.public_metrics["like_count"] # => 3, which post.like_count reads too
|
|
92
|
+
post.urls.map { |url| url["expanded_url"] }
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
**Any endpoint.** For an endpoint without a method, pass a resource class as the `object_class` of a request. A response that holds one resource comes back as an object, and one that holds a list comes back as an `X::Page` of the page you requested, which reads as an array does, and holds the `next_token` of the page after it. The object holds only the fields you asked for, so `hydrate` fetches the rest.
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
user = x_client.get("users/by/username/sferik", object_class: X::User)
|
|
99
|
+
user.followers_count # => nil, since the request didn't ask for it
|
|
100
|
+
user.hydrate.followers_count # => 12345
|
|
101
|
+
|
|
102
|
+
blocked = x_client.get("users/#{user.id}/blocking", object_class: X::User) # => #<X::Page ...>
|
|
103
|
+
blocked.first # => #<X::User ...>
|
|
104
|
+
x_client.get("users/#{user.id}/blocking", params: {pagination_token: blocked.next_token}, object_class: X::User)
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**Identity.** Resources with the same class and ID are equal (`==`, `eql?`, and `hash`), even when they come from different requests.
|
|
108
|
+
|
|
109
|
+
```ruby
|
|
110
|
+
post.author == x_client.find_user("sferik") # => true
|
|
111
|
+
[post.author, user].uniq.size # => 1
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
**References.** Foreign keys have accessors that return objects. When the response included the referenced object, you get it with its fields. Otherwise, you get a stub that holds only the ID. `hydrate` fetches the full object once and memoizes it. `refresh` fetches it again.
|
|
115
|
+
|
|
116
|
+
```ruby
|
|
117
|
+
post.author # => #<X::User id="7505382" name="Erik Berlin" username="sferik" ...>
|
|
118
|
+
post.replied_to # => #<X::Post id="1234567889" text="..." ...>, or a stub of the ID if left out
|
|
119
|
+
post.replied_to.hydrate.text # one request
|
|
120
|
+
post.replied_to.hydrate.text # memoized, no request
|
|
121
|
+
post.replied_to.refresh.text # forces a new request
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Within one response, every reference to the same resource is the same object, so hydrating a user from one post hydrates it for every post on that page.
|
|
125
|
+
|
|
126
|
+
**Pagination.** Collections are `X::Cursor` objects, which include `Enumerable`, remember the client that fetched them, and fetch pages lazily. Iterating requests the maximum page size, and pages are cached, so iterating twice costs no extra requests. `first(n)` and `take(n)` request a page of `n` instead, raised to the endpoint's minimum, and keep it, so an iteration after them pays for none of it again, and the first page of `each_page` holds what they read. `published_count` reads the number the API publishes for a collection, such as a user's `followers_count`, without paging through it, while `count` pages through the whole collection, as `Enumerable` does. A cursor has no `size`, so sizing an enumerator over it, such as `each_slice(2).size`, never pages it. `refresh` returns a cursor with an empty cache, and `ids` fetches nothing but identifiers.
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
followers = user.followers # max_results=1000 per page
|
|
130
|
+
followers.first(10) # one request for ten users
|
|
131
|
+
followers.empty? # one request for one user, as any? and none? make
|
|
132
|
+
followers.to_a # fetches the remaining pages
|
|
133
|
+
followers.to_a # cached, no requests
|
|
134
|
+
followers.refresh.to_a # starts over
|
|
135
|
+
followers.published_count # => 12345, the user's followers_count, with no request
|
|
136
|
+
followers.ids # => [14100886, ...], requesting only identifiers
|
|
137
|
+
|
|
138
|
+
x_client.search_posts("ruby -is:retweet").each { |post| puts post.text }
|
|
139
|
+
x_client.search_users("ruby").first(10)
|
|
140
|
+
x_client.search_communities("ruby").first(10)
|
|
141
|
+
x_client.search_spaces("ruby", state: "live").first(10) # a client that signs with OAuth 1.0a searches as the app
|
|
142
|
+
x_client.find_all_spaces_by_creator([user, other]) # the live and scheduled spaces the users created
|
|
143
|
+
x_client.trends(1).first(10) # the topics trending worldwide, by WOEID, as X::Trend
|
|
144
|
+
x_client.personalized_trends # the topics X picks for the authenticated user
|
|
145
|
+
user.affiliates.first(10) # the users whose affiliation names the user
|
|
146
|
+
x_client.current_user!.home_timeline.first(10)
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
**Stubs.** `from_id` refers to a resource without a request, so a cursor can be built from an identifier alone. A reference the response did not expand is a stub too, and `stub?` tells the two apart.
|
|
41
150
|
|
|
151
|
+
```ruby
|
|
152
|
+
X::User.from_id(7505382, client: x_client).followers.ids
|
|
153
|
+
post.author.stub? # => false when the response included the author
|
|
154
|
+
X::User.hydrate_all(posts.map(&:author), client: x_client) # looks up what is not hydrated, included authors too, in one batch
|
|
155
|
+
X::User.hydrate_all(posts.filter_map(&:author).select(&:stub?), client: x_client) # looks up only the authors not included
|
|
156
|
+
message.peer(x_client.current_user!) # the other participant of a direct message
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Costs.** The X API bills each resource a request returns, so the size of a page is the size of the bill. `first(n)` and `take(n)` read a page of `n`, raised to the smallest page the endpoint takes, such as 5 for a user's posts and 10 for a search, so a smaller `n` is billed for that page. Iterating a cursor, `to_a`, and `ids` read the whole collection, up to 1,000 users a page for followers and following. `count` reads the whole collection too, a request per page, billed for every resource, so counting a user with a million followers reads a million users in a thousand requests. `published_count` reads the number the API publishes instead, for a user's followers, followed users, and list memberships, and for a list's members and followers, which costs nothing when the user or list holds it and one lookup when it is a stub, and it returns nil for any other collection. The published number counts what the collection holds, which can differ from what reading it finds, since the API leaves out what the authenticated user cannot see. Other `Enumerable` methods, such as `find`, `include?`, and `lazy`, cannot know how much they will read, so they request the largest page too; pass `max_results:` to read less per request. `user.follows?(other)` takes one lookup when either user is the authenticated user, by reading the other's `connection_status`, and otherwise scans the users `user` follows. The API has no lookup for a list membership, so `list.member?(user)` scans: the members of a private list, and for a public list, whichever is smaller of its members and the lists the user is on, as `member_count` and `listed_count` tell. Pass either scan `max_pages:` to read no more pages than that: one that reaches the limit with pages left raises `X::PageLimitReached`, rather than answer from the pages it read.
|
|
160
|
+
|
|
161
|
+
```ruby
|
|
162
|
+
user.followers.first(10) # ten users
|
|
163
|
+
user.followers.published_count # the user's followers_count, reading no follower
|
|
164
|
+
user.followers.count # every follower, a request per 1,000
|
|
165
|
+
user.muting.published_count # => nil, since the API publishes no number
|
|
166
|
+
x_client.current_user!.follows?(other) # one lookup
|
|
167
|
+
other.follows?(x_client.current_user!) # one lookup
|
|
168
|
+
other.follows?(someone) # scans everyone other follows
|
|
169
|
+
other.follows?(someone, max_pages: 5) # scans five pages at most, or raises X::PageLimitReached
|
|
170
|
+
user.followers(max_results: 100).find { |follower| follower.verified? }
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
The API bills a resource once per UTC day, however often it is read, and bills only the resources a response returns as data, not the ones it includes, so expanding `post.author` costs nothing extra. Batch lookups ask for each ID once. Reads of the authenticated user's own data cost $0.001 a resource, less than most other reads, when that user owns the app: their posts, mentions, likes, bookmarks, followers, following, blocks, mutes, and lists. So `x_client.current_user!.posts` is cheaper than searching for `from:sferik`. Actions take the authenticated user's ID from the prefix of an OAuth 1.0a access token, and so need no request to `users/me`.
|
|
174
|
+
|
|
175
|
+
Writes are billed by the request, and cost more than reads. Creating a post costs $0.015, or $0.20 when its text holds a URL, so a link costs more than ten times as much as the post around it. A like, repost, follow, or direct message costs $0.015, and undoing one costs $0.01. Prices change, so check the [pricing page](https://docs.x.com/x-api/getting-started/pricing) before a large run.
|
|
176
|
+
|
|
177
|
+
**Post usage.** `post_usage` reports how many posts the app's project has read this billing cycle, against its monthly cap, and how many it read each day. It returns nil when the response holds no usage, yielding the problems the API reported to a block, as `current_user` does, and `post_usage!` raises `X::MissingResource` instead.
|
|
178
|
+
|
|
179
|
+
```ruby
|
|
180
|
+
usage = x_client.post_usage!(days: 30)
|
|
181
|
+
usage.project_cap - usage.project_usage # the posts left to read this cycle
|
|
182
|
+
usage.daily # => {2026-09-14 00:00:00 UTC => 1234, ...}
|
|
183
|
+
usage.daily_by_app # the same, keyed by the ID of each of the project's apps
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
**Counting posts.** `count_posts` counts the posts from the last seven days that match a query, and `count_all_posts` counts every post, which needs full-archive access. Each costs one request per page of periods, whatever the count, and a count of many years of the archive takes many pages, so every count, `count_posts`, `count_all_posts`, and their `_by_period` forms, takes `max_pages:`, past which it raises `X::PageLimitReached`. The counts and usage endpoints refuse OAuth 1.0a, so a client that signs with it makes those requests through `app_only`, a copy of itself that authenticates as the app. The client fetches the app's bearer token the first time and reuses it.
|
|
187
|
+
|
|
188
|
+
```ruby
|
|
189
|
+
x_client.count_posts("ruby") # => 12345
|
|
190
|
+
x_client.count_posts_by_period("ruby", granularity: "hour") # => {2026-09-14 12:00:00 UTC...2026-09-14 13:00:00 UTC => 42, ...}
|
|
191
|
+
X::Post.count_all("ruby", client: x_client, start_time: "2020-01-01T00:00:00Z")
|
|
192
|
+
X::Post.count_all("ruby", client: x_client, max_pages: 3) # three requests at most
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
**Missing resources.** A finder returns nil when the resource does not exist, and its bang form raises `X::MissingResource`, an `X::Error`. X answers a lookup of one resource that is not there with a 200 OK and no data, or with a 404 that reports the resource as not found, and a finder reads both alike: nil, or `X::MissingResource` from the bang form, whose `cause` is the `X::NotFound` when the answer was a 404. Any other 404 raises `X::NotFound`: one that reports no resource as not found, as from a client pointed at the wrong host or API version, or one that answers a request other than the lookup of one resource by its identifier or username, such as `current_user`, a `find_all_…`, or a collection. A request that creates a resource, such as `create_post`, `create_list`, or `create_dm`, raises `X::MissingResource`, holding the response's problems, if the API answers with success and no resource, so it never returns nil. A value a response holds that cannot be read as what the API documents it to be, such as a timestamp that is not ISO 8601, raises `X::InvalidAttribute`, an `X::Error` too, from the reader that reads it. A page that names the token of a page already fetched as its next raises `X::UnreadableResponse`, the base of `X::InvalidAttribute`, rather than page forever. A list the response omitted, such as `space.host_ids` or `post.urls`, reads as empty rather than nil.
|
|
196
|
+
|
|
197
|
+
```ruby
|
|
198
|
+
x_client.find_user("nobody") # => nil
|
|
199
|
+
x_client.find_user!("nobody") # raises X::MissingResource
|
|
200
|
+
x_client.find_user("not a username") # raises ArgumentError, before any request
|
|
201
|
+
post.permalink # => "https://x.com/sferik/status/1234567890"
|
|
202
|
+
post.expanded_text # the text with every t.co link replaced by the URL it stands for
|
|
203
|
+
post.display_text_range # => 0...140, the Range of the text that is shown, or nil
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
**Partial errors.** A response can succeed and still report problems, such as a pinned post that was deleted or one ID of a batch that does not exist. `problems` returns them as `X::Problem` objects: on a resource, the ones whose `resource_id` or `value` is its identifier or that of a resource it refers to, such as the author of a post, and the ones that name no identifier, and on each page of a cursor, every one the response reported. A finder yields each one to a block, and `X::MissingResource#problems` holds the ones that explain a missing resource. A request the API refuses raises an `X::HTTPError` whose `problems` are every one the response named, such as each parameter it refused, and whose `problem` is the one the response describes as a whole, or, for a response that names errors alone, as the v1.1 API sends, the first of them; a 402 Payment Required, which X sends for a request the account has no credit left to pay for, raises `X::PaymentRequired`.
|
|
207
|
+
|
|
208
|
+
```ruby
|
|
209
|
+
user = x_client.find_user("sferik")
|
|
210
|
+
user.problems # => [#<X::Problem Not Found Error: Could not find post with pinned_tweet_id: [1234567890].>]
|
|
211
|
+
x_client.find_all_users(ids) { |problem| warn problem.detail if problem.not_found? }
|
|
212
|
+
x_client.find_user!("nobody") # raises X::MissingResource: Could not find X::User @nobody: Could not find user with username: [nobody].
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
**Parallel requests.** Batch lookups split the IDs into groups of 100, the API maximum, and request four groups at a time, as the chunks of an upload are sent four at a time. Pass a `concurrency` to request more or fewer at once: a lower number spends a rate limit more slowly. Paginated endpoints return a token for the next page with each page, so their pages must be fetched in order. `prefetch` fetches the next page in a background thread while you process the current one. It is opt-in because it spends one extra request when you stop iterating early.
|
|
216
|
+
|
|
217
|
+
```ruby
|
|
218
|
+
x_client.find_all_users(follower_ids) # parallel batches of 100, in the order asked for
|
|
219
|
+
x_client.find_all_users(follower_ids, concurrency: 1) # one batch at a time
|
|
220
|
+
X::Post.find_all(ids, client: x_client, concurrency: 8)
|
|
221
|
+
user.followers.prefetch.each { |follower| process(follower) }
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
**Actions.** Actions are taken as the authenticated user, so the client takes them. They take a resource of the class they act on, or its identifier, an Integer or a String of digits, and raise `ArgumentError` for anything else, such as a username, or a resource of another class, such as a list passed to `follow`, before any request but the lookup of the authenticated user a client's action may make first, so look a user up with `find_user` first.
|
|
225
|
+
|
|
226
|
+
```ruby
|
|
227
|
+
x_client.follow(user)
|
|
228
|
+
x_client.block(user)
|
|
229
|
+
x_client.like(post)
|
|
230
|
+
x_client.repost(post) # => true, as each action reports whether it took
|
|
231
|
+
x_client.bookmark(post) # bookmarking takes OAuth 2.0 user context, which this client does not hold
|
|
232
|
+
me = x_client.current_user! # looked up each time it is called, so keep it
|
|
233
|
+
folder = me.bookmark_folders.first
|
|
234
|
+
me.bookmarks(folder:).first(10) if folder # the posts in a bookmark folder, when the user has one
|
|
235
|
+
|
|
236
|
+
post = x_client.create_post("Hello, World! (from @gem)")
|
|
237
|
+
reply = x_client.create_post("Hello back!", reply_to: post, media_ids: [media])
|
|
238
|
+
quote = x_client.create_post("Worth reading", quote: post)
|
|
239
|
+
photo = x_client.create_post(media_ids: [media]) # a post needs no text when it has media
|
|
240
|
+
x_client.hide_reply(reply) # as the author of the post it replies to
|
|
241
|
+
post.delete
|
|
242
|
+
|
|
243
|
+
list = x_client.create_list("Rubyists", private: true)
|
|
244
|
+
list.update(description: "People who write Ruby") # also name and private
|
|
245
|
+
list.add_member(user)
|
|
246
|
+
x_client.pin_list(list) # also unpin_list, follow_list, and unfollow_list
|
|
247
|
+
list.delete
|
|
248
|
+
|
|
249
|
+
message = x_client.create_dm(user, "Hello!") # create_direct_message, shortened
|
|
250
|
+
x_client.dms_with(user).first(10) # the conversation with a user
|
|
251
|
+
|
|
252
|
+
group = x_client.create_group_dm([user, other], "Hello, both of you!") # create_group_direct_message, shortened
|
|
253
|
+
x_client.create_dm_in(group, "Anyone free on Friday?") # to the conversation of a message, or its identifier
|
|
254
|
+
x_client.dms_in(group).first(10) # the messages of a conversation, one-to-one or group
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
### Raw JSON
|
|
258
|
+
|
|
259
|
+
`X::Client` still speaks raw JSON, for endpoints without objects or when you want full control.
|
|
260
|
+
|
|
261
|
+
```ruby
|
|
42
262
|
# Get data about yourself
|
|
43
263
|
x_client.get("users/me")
|
|
44
264
|
# {"data"=>{"id"=>"7505382", "name"=>"Erik Berlin", "username"=>"sferik"}}
|
|
45
265
|
|
|
46
|
-
#
|
|
47
|
-
|
|
48
|
-
|
|
266
|
+
# Query parameters drop nil values and join arrays with commas
|
|
267
|
+
x_client.get("users", params: {ids: [7505382, 12], "user.fields": %w[id username]})
|
|
268
|
+
|
|
269
|
+
# Post, with a Hash encoded as JSON
|
|
270
|
+
created = x_client.post("tweets", {text: "Hello, World! (from @gem)"})
|
|
271
|
+
# {"data"=>{"edit_history_post_ids"=>["1234567890123456789"], "id"=>"1234567890123456789", "text"=>"Hello, World! (from @gem)"}}
|
|
49
272
|
|
|
50
273
|
# Delete the post
|
|
51
|
-
x_client.delete("tweets/#{
|
|
274
|
+
x_client.delete("tweets/#{created["data"]["id"]}")
|
|
52
275
|
# {"data"=>{"deleted"=>true}}
|
|
53
276
|
|
|
54
|
-
#
|
|
55
|
-
v1_client =
|
|
277
|
+
# Derive an API v1.1 client
|
|
278
|
+
v1_client = x_client.with(base_url: "https://api.x.com/1.1/")
|
|
279
|
+
|
|
280
|
+
# Post a form
|
|
281
|
+
v1_client.post("account/settings.json", form: {lang: "en"})
|
|
282
|
+
|
|
283
|
+
# Authenticate as the app, with a bearer token fetched with the API key and secret the first time, and again once X
|
|
284
|
+
# rejects it
|
|
285
|
+
x_client.app_only.get("tweets/search/stream/rules")
|
|
286
|
+
|
|
287
|
+
# Authenticate with OAuth 2.0, refreshing the access token when it expires or the API rejects it,
|
|
288
|
+
# and store the X::OAuth2Tokens of each refresh, since X accepts a refresh token only once; X::AuthorizationError
|
|
289
|
+
# means X refused to refresh, so rescue it beside X::Unauthorized to ask the user to authorize the app again, and a
|
|
290
|
+
# token endpoint that fails to answer raises the X::HTTPError of its status, such as the X::ServerError the client
|
|
291
|
+
# retries, or X::InvalidResponse for the page of a captive portal
|
|
292
|
+
oauth2_client = X::Client.new(client_id: "ID", client_secret: "SECRET", access_token: "TOKEN", refresh_token: "REFRESH",
|
|
293
|
+
expires_at: Time.now + 7200, save_tokens: ->(tokens) { store(tokens.access_token, tokens.refresh_token, tokens.expires_at) })
|
|
294
|
+
|
|
295
|
+
# Share the tokens of a user among processes: a refresh reads the store with load_tokens first, which returns the
|
|
296
|
+
# X::OAuth2Tokens stored, or nil, and takes the tokens another process stored in place of refreshing with a refresh
|
|
297
|
+
# token that process already spent
|
|
298
|
+
shared_tokens_client = X::Client.new(client_id: "ID", **load(user).to_h,
|
|
299
|
+
save_tokens: ->(tokens) { store(user, tokens) }, load_tokens: -> { load(user) })
|
|
300
|
+
|
|
301
|
+
# Store the tokens as JSON, which X::OAuth2Tokens.from_json reads back, the expiration time and all
|
|
302
|
+
json_tokens_client = X::Client.new(client_id: "ID", **X::OAuth2Tokens.from_json(redis.get(user)).to_h,
|
|
303
|
+
save_tokens: ->(tokens) { redis.set(user, tokens.to_json) })
|
|
304
|
+
|
|
305
|
+
# An access token issued without offline.access comes with no refresh token, and acts for the user until it expires
|
|
306
|
+
short_lived_client = X::Client.new(client_id: "ID", access_token: "TOKEN")
|
|
307
|
+
|
|
308
|
+
# Authenticate with an authenticator built elsewhere, in place of credentials; clients given the same one share its
|
|
309
|
+
# tokens, and each refresh reaches the save_tokens of every one of them
|
|
310
|
+
authenticator = X::OAuth2Authenticator.new(client_id: "ID", access_token: "TOKEN", refresh_token: "REFRESH")
|
|
311
|
+
shared_client = X::Client.new(authenticator:, save_tokens: ->(tokens) { store(tokens.refresh_token) })
|
|
312
|
+
|
|
313
|
+
# Authenticate with a scheme of your own: subclass X::Authenticator and return the headers that authenticate a request,
|
|
314
|
+
# reading of the request its http_method, uri, body, and [] for a header, which are all it answers
|
|
315
|
+
class VaultAuthenticator < X::Authenticator
|
|
316
|
+
def headers(_request) = {AUTHENTICATION_HEADER => "Bearer #{Vault.read("x/bearer_token")}"}
|
|
317
|
+
end
|
|
318
|
+
vault_client = X::Client.new(authenticator: VaultAuthenticator.new)
|
|
319
|
+
|
|
320
|
+
# Ask a user to authorize the app with OAuth 2.0 and PKCE, keeping the state and code verifier until X redirects back
|
|
321
|
+
authorization = X::OAuth2Authorization.new(client_id: "ID", redirect_uri: "https://example.com/callback",
|
|
322
|
+
scopes: %w[tweet.read tweet.write users.read offline.access])
|
|
323
|
+
session[:state] = authorization.state
|
|
324
|
+
session[:code_verifier] = authorization.code_verifier
|
|
325
|
+
redirect_to authorization.url
|
|
326
|
+
|
|
327
|
+
# Then, where X redirects back, exchange the code for a client that acts for the user; save_tokens is passed the
|
|
328
|
+
# tokens of the exchange, whose refresh_token is nil without offline.access, and those of each refresh after, and
|
|
329
|
+
# X::TokenReportFailed holds the client and the tokens when it raises for either, so neither is lost with the code or
|
|
330
|
+
# the refresh token they replaced
|
|
331
|
+
authorization = X::OAuth2Authorization.new(client_id: "ID", redirect_uri: "https://example.com/callback",
|
|
332
|
+
state: session[:state], code_verifier: session[:code_verifier])
|
|
333
|
+
user_client = authorization.client(request.url, save_tokens: ->(tokens) { store(tokens.refresh_token) })
|
|
334
|
+
user_client.scopes # => the scopes the user granted, which may be fewer than the app asked for
|
|
56
335
|
|
|
57
336
|
# Define a custom response object
|
|
58
337
|
Language = Struct.new(:code, :name, :local_name, :status, :debug)
|
|
@@ -65,29 +344,215 @@ languages = v1_client.get("help/languages.json", object_class: Language, array_c
|
|
|
65
344
|
languages.first.local_name
|
|
66
345
|
|
|
67
346
|
# Initialize an Ads API client
|
|
68
|
-
ads_client = X::Client.new(base_url: "https://ads-api.
|
|
347
|
+
ads_client = X::Client.new(base_url: "https://ads-api.x.com/12/", **x_credentials)
|
|
69
348
|
|
|
70
349
|
# Get your ad accounts
|
|
71
350
|
ads_client.get("accounts")
|
|
351
|
+
|
|
352
|
+
# Keep a connection open for the next request for five seconds rather than 30, behind a proxy that closes idle
|
|
353
|
+
# connections sooner than X does
|
|
354
|
+
patient_client = X::Client.new(keep_alive_timeout: 5, **x_credentials)
|
|
355
|
+
|
|
356
|
+
# Send headers with every request, such as one that names your application. They are defaults: a header of the same
|
|
357
|
+
# name passed to a request, or to a stream, is sent in place of the client's, and each of the client's is sent in
|
|
358
|
+
# place of a default of the gem, such as its User-Agent, whatever the case each is named in. They, and the gem's
|
|
359
|
+
# User-Agent, are sent with the requests that fetch and refresh the client's tokens too, as a gateway may require.
|
|
360
|
+
named_client = X::Client.new(headers: {"User-Agent" => "my-app/1.0 (+https://example.com)"}, **x_credentials)
|
|
361
|
+
named_client.get("users/me", headers: {"X-Trace" => "abc"}) # sends both
|
|
362
|
+
# with(headers:) replaces the headers of the client rather than adding to them, so give the copy every header it sends
|
|
363
|
+
traced_client = named_client.with(headers: named_client.headers.merge("X-Trace" => "abc")) # sharing its connections
|
|
364
|
+
|
|
365
|
+
# Send requests through a proxy. A client that is given none takes the proxy the environment names for the scheme of
|
|
366
|
+
# each request, in https_proxy or http_proxy, and reaches the hosts that no_proxy names directly.
|
|
367
|
+
proxied_client = X::Client.new(proxy_url: "http://user:password@proxy.example.com:8080", **x_credentials)
|
|
368
|
+
|
|
369
|
+
# Close the connections a client keeps open between requests, which the copies `with` makes of it share when they
|
|
370
|
+
# open their connections alike; a later request opens one again
|
|
371
|
+
x_client.close
|
|
72
372
|
```
|
|
73
373
|
|
|
374
|
+
**A client never changes.** It keeps the credentials and settings it was built with for as long as it lives, so a request never runs against a setting another thread is halfway through changing. `with` derives a client that differs, and takes anything `X::Client.new` takes.
|
|
375
|
+
|
|
376
|
+
```ruby
|
|
377
|
+
v1_client = x_client.with(base_url: "https://api.x.com/1.1/")
|
|
378
|
+
rotated = x_client.with(access_token: "new_token", access_token_secret: "new_secret")
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
**A client keeps its credentials to its own origin.** An endpoint that names a whole URL is sent to that URL, and it carries the client's credentials only when it names the scheme, host, and port of the `base_url`, which the API version above does. An endpoint, or a stream, of any other origin is sent without the `Authorization` header the authenticator signs and without any `Authorization`, `Cookie`, or `Proxy-Authorization` header of the client or the request, as a redirect that leads off the origin is, so the credentials of the API never reach a host they were not meant for. A client built with another `base_url`, such as the Ads API client above, carries its credentials to that origin.
|
|
382
|
+
|
|
383
|
+
**A client keeps its credentials to itself.** It has no reader for a secret it holds, and `inspect` names its authenticator without revealing what it signs with, so nothing that reflects over a client reads a credential out of one. `with` carries them to a client derived from it, and the hook given to `save_tokens` is passed the `X::OAuth2Tokens` of the refresh it reports.
|
|
384
|
+
|
|
385
|
+
```ruby
|
|
386
|
+
x_client.inspect # => #<X::Client base_url="https://api.x.com/2/" authenticator=#<X::OAuth1Authenticator>>
|
|
387
|
+
x_client.api_key # the API key, the client ID, and the expiration time are public
|
|
388
|
+
x_client.api_key_secret # raises NoMethodError, as every secret a client holds does
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
### Media
|
|
392
|
+
|
|
393
|
+
```ruby
|
|
394
|
+
# The media category is inferred from the file: an image, an animated GIF, a video, or subtitles.
|
|
395
|
+
# A GIF with a single frame is uploaded as an image, since X processes only animated GIFs as GIFs.
|
|
396
|
+
media = x_client.upload_media("cat.jpg", alt_text: "A cat asleep on a keyboard")
|
|
397
|
+
media.id # => 1880028106020515840, where media["id"] is the String the API gave
|
|
398
|
+
media.expires_after_secs # => 86400, the seconds after the upload within which a post can attach it
|
|
399
|
+
x_client.create_post("Look at this cat", media_ids: [media])
|
|
400
|
+
x_client.find_media(media).url # the X::Media it became, looked up by media key
|
|
401
|
+
x_client.find_all_media(post.media) # the media of a post, up to 100 keys per request
|
|
402
|
+
|
|
403
|
+
# A video, and an animated GIF larger than 5 MB, is uploaded in chunks, four at a time unless concurrency says
|
|
404
|
+
# otherwise, up to 16, in chunks of up to 5 MB sized so that the upload fits the 10,000 segments the API numbers, and
|
|
405
|
+
# upload waits until a video or an animated GIF has been processed, for up to ten minutes unless processing_timeout says
|
|
406
|
+
# otherwise. An image larger than 5 MB, a GIF larger than 15 MB, or subtitles larger than 1 MB raise X::InvalidMedia
|
|
407
|
+
# before any request, since X takes no more of them, as does a video larger than the 16 GB X takes of an upload.
|
|
408
|
+
video = x_client.upload_media("cat.mp4")
|
|
409
|
+
video.ready? # => true, since upload_media waited; state is "succeeded"
|
|
410
|
+
subtitles = x_client.upload_media("cat.srt")
|
|
411
|
+
x_client.add_subtitles(video, subtitles, "EN", display_name: "English") # returns the video, an X::UploadedMedia
|
|
412
|
+
x_client.create_post("Look at this cat move", media_ids: [video])
|
|
413
|
+
|
|
414
|
+
# Update the profile image and banner of the authenticated user. These two call the API v1.1, which the API v2 has
|
|
415
|
+
# no endpoint for, and return nothing to read: look the user up to read the image and banner it now has.
|
|
416
|
+
x_client.update_profile_image("avatar.png")
|
|
417
|
+
x_client.update_profile_banner("banner.png")
|
|
418
|
+
|
|
419
|
+
# Media is a path, or an IO open on it. Media given as a String or a Pathname is read from the file it names, and
|
|
420
|
+
# media given as a File or a Tempfile through that IO, even once the Tempfile is unlinked, or from the file it names
|
|
421
|
+
# once it is closed, a chunk at a time, so media of any size uploads without being held in memory. A String that holds
|
|
422
|
+
# the contents of media, rather than its path, raises ArgumentError: wrap the contents in a StringIO. A String is
|
|
423
|
+
# read as a path on this machine, so never pass one a user gave, such as a parameter of a form, which could name any
|
|
424
|
+
# file the process can read, and upload it to X: pass the IO of the file the user uploaded instead.
|
|
425
|
+
x_client.upload_media(Pathname("cat.jpg"))
|
|
426
|
+
File.open("cat.mp4", "rb") { |file| x_client.upload_media(file) }
|
|
427
|
+
|
|
428
|
+
# Media given as any other IO, such as a StringIO, is read to its end and held. Its category is read from the bytes
|
|
429
|
+
# it begins with, as the category of a file is before its name: a GIF, PNG, JPEG, BMP, TIFF, WebP, MP4, QuickTime, WebM, MPEG transport
|
|
430
|
+
# stream, or WebVTT file is recognized by its signature. Pass media_category for anything else, such as SubRip subtitles, which begin
|
|
431
|
+
# with nothing a text file could not.
|
|
432
|
+
x_client.upload_media(StringIO.new(png))
|
|
433
|
+
x_client.upload_media(StringIO.new(srt), media_category: "subtitles")
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
Each of these methods calls an uploader with the client: `upload_media`, `chunked_upload_media`, which uploads in chunks and returns before X processes the media, `await_media_processing`, and `await_media_processing!`, which raises `X::MediaProcessingFailed` where the other returns the status of processing that failed, call `X::Uploads::MediaUpload`, `add_alt_text` and `add_subtitles` call `X::Uploads::Metadata`, and `update_profile_image` and `update_profile_banner` call `X::Uploads::Account`. Media that already says its processing ended, as the response to the upload of an image does, is awaited without a request. The uploaders take an `X::Client` as `client:`, as in `X::Uploads::MediaUpload.chunked_upload("cat.mp4", client: x_client, chunk_size: 4 * 1024 * 1024)`, which `x_client.chunked_upload_media("cat.mp4", chunk_size: 4 * 1024 * 1024)` calls.
|
|
437
|
+
|
|
438
|
+
`upload_media` uploads an image in a single request, which takes no chunks, so it ignores `chunk_size:`, `concurrency:`, and `media_type:` for one; `chunked_upload_media` uploads in chunks whatever the media, and is the way to upload without waiting for X to process it.
|
|
439
|
+
|
|
440
|
+
A video uploads in chunks of 4 MB, each a request of its own, which a rate limit can refuse. A client retries a request refused for a rate limit only `max_rate_limit_retries` times, 0 by default, so a chunk refused fails the upload with `X::ChunkedUploadFailed`, which holds the media it initialized. Upload a large video with a client whose `max_rate_limit_retries` is set, such as `X::Client.new(**x_credentials, max_rate_limit_retries: 3)`, so that a rate limit is waited out, up to the client's `max_rate_limit_wait`, rather than fail the upload.
|
|
441
|
+
|
|
442
|
+
What each returns: `upload_media`, `chunked_upload_media`, `await_media_processing`, and `await_media_processing!` return an `X::UploadedMedia`, which reads as the Hash the API answered with as well as by its own methods; `add_alt_text` and `add_subtitles` return the media they describe as an `X::UploadedMedia`, the video for `add_subtitles`, so either can be passed on to `create_post`; and `update_profile_image` and `update_profile_banner` return nothing to read, `nil`: they are the only methods in these gems that call the v1.1 API, whose user no object of the object layer reads, so `current_user!` reads the image and banner the user now has.
|
|
443
|
+
|
|
74
444
|
### Streaming
|
|
75
445
|
|
|
446
|
+
A stream holds a connection open instead of answering a request, so `X::StreamingClient`, from `x-streams`, handles one, and `streaming` builds it from a client. It shares the client's credentials, base URL, parsing classes, and `on_response` hook, and keeps the settings a long-lived connection needs: `read_timeout`, 30 seconds by default, `max_reconnects`, and `on_reconnect`.
|
|
447
|
+
|
|
448
|
+
The stream endpoints take app-only authentication, so a client that authenticates as a user streams with the bearer token that `app_only` holds. A client that authenticates with OAuth 2.0 as a user and holds neither the app's bearer token nor its API key and secret has no credentials of the app, so `app_only` raises `X::UnsupportedOperation` for it, and a stream it opens goes out as the user, which X refuses with 403 Forbidden, raising `X::Forbidden`; give the client one of them, or stream with a client built from the app's bearer token, or its API key and secret, instead.
|
|
449
|
+
|
|
450
|
+
X holds a stream open indefinitely, but drops it for deploys, network trouble, and slow readers. A stream that ends, drops, is refused a connection, or that X disconnects with an `operational-disconnect` reconnects at once, then waits a quarter second longer each attempt, up to 16 seconds; one that ends within a line drops what it sent of the line, which is neither parsed nor passed to `on_response`, and reconnects the same way. A server error, a 408 Request Timeout, a 409 Conflict, or a line that is not JSON waits 5 seconds, doubling each attempt, up to 320 seconds, and longer when the response asks for longer in its `Retry-After` header, but raises at once when that header asks for longer than the `max_rate_limit_wait` of the client. A rate limit backs off from a minute, doubling each attempt, as X asks, and waits longer for a limit that resets later, but raises `X::TooManyRequests` at once rather than wait longer than the `max_rate_limit_wait` of the client, 900 seconds by default, so that a limit on the connections of a day does not hold a stream closed for hours, and a stream X keeps refusing gives up after waits of 60, 120, 240, and 480 seconds. Each of the three backs off on a count of its own, so the dropped connections before a server error do not lengthen the wait after it. A post, or the keep-alive X sends every 20 seconds, read from a connection that has been open for a minute starts every count over, so a filtered stream whose rules rarely match, connected for hours between drops, never runs out of reconnects; one read from a younger connection starts none over, so a stream whose connections each deliver a post and drop backs off, and runs out of reconnects, rather than reconnect at once without end; an `operational-disconnect` or a line that is not JSON does not, so a stream X keeps refusing backs off. A stream reconnects without limit by default; set `max_reconnects` to give up after that many attempts in a row, when it raises the error of the last, an `X::NetworkError` for a stream that ended or dropped, so a stream never returns but for a `break` from its block, or a `stop`. Pass `on_reconnect:`, a callable given the error that dropped the stream and the seconds it waits, to hear of each reconnect, since a stream that can never connect, through a host that does not resolve or a proxy set up wrong, otherwise reconnects in silence; call `stop` from it to give up, and the stream returns nil. An error it raises stops the stream and reaches the caller.
|
|
451
|
+
|
|
452
|
+
> [!IMPORTANT]
|
|
453
|
+
> Reconnects are unlimited by default, and silent, so a stream that cannot connect at all, as for a host that does not resolve or a network that is down, keeps trying every 16 seconds for as long as it runs. Only a certificate that does not verify, which will not verify on the next attempt either, raises at once, an `X::NetworkError` whose `cause` is the `OpenSSL::SSL::SSLError`. To give up on the rest, set `max_reconnects`, or pass `on_reconnect`, which is called before each reconnect with the error that dropped the stream, never nil, and the seconds it waits, and can `stop` the stream or raise. It is called with those two arguments, and no others, by every release of 1.x.
|
|
454
|
+
|
|
455
|
+
**The rules of the filtered stream.** The filtered stream delivers the posts that match the rules of the app, which belong to the stream and are read and changed through a streaming client: `rules` reads them, every page of them, `add_rules` adds them, and `delete_rules` deletes them, each authenticating as the app as a stream does. A rule is an `X::StreamRule`, a frozen value of the `value` it matches, the `tag` it is labelled with, and the `id` the API gave it, read as an Integer; `rules` returns them, and `add_rules` returns the ones the API added, and yields an `X::Problem` for each problem the API reported of the rules it did not add, a rule the app already has included. A rule the app already has does not keep the API from adding the others. A rule the API rejects, such as an invalid one, may keep it from adding any, and the API reports the rejected rule alone, so the rules returned, not the problems, say which were added. To read a rule the app already had, with its `id` and `tag`, call `rules`. `delete_rules` returns the number of rules it deleted, and yields each problem the API reported of those it did not, such as a rule the app does not have. Without a block, `add_rules` and `delete_rules` raise `X::RulesRejected` for the rules the API rejected, rather than drop them in silence; its `problems` are those problems, and its `added` or `deleted_count` is what the method would have returned. A rule the app already has is not a rejected rule, so `add_rules` can run at every boot. A rule to add is an `X::StreamRule`, a Hash of a `value` and a `tag`, or a String, which is the value of a rule with no tag, so the rules of one app add themselves to another. A rule is deleted by the identifier it was given, which an `X::StreamRule` the API returned, a Hash holding an `id`, an Integer, or an `X::MatchingRule` names, so what `rules` returned deletes itself, as do the `matching_rules` of a post of `x-resources`, or by the value it matches, which a rule without an identifier or a String names, so what `add_rules` was given deletes what it added. Anything else raises `ArgumentError` before a request, even a post or another resource with an `id`, which would otherwise delete whichever rule shared its identifier. No rules add or delete none, and send no request. `dry_run: true` has the API check the rules and change none of them.
|
|
456
|
+
|
|
457
|
+
**Stopping a stream.** A stream runs until its block stops it. `break` out of the block to stop the stream and return a value, or `throw` to unwind to a `catch` further out; neither reconnects. An error raised by the block stops the stream too, even one a dropped connection would have reconnected after, and reaches the caller unchanged, as does an error raised by `on_response` or by the class that builds each object. A `StopIteration` is an error like any other here, so a block that exhausts an `Enumerator` of its own hears about it rather than ending the stream in silence. A block runs only when a post arrives, so it cannot stop a stream that delivers nothing; `stop` can, from any thread or the trap of a signal: it stops every stream the streaming client runs, the next time each waits on the API, and each returns nil. A stopped streaming client stays stopped: a stream it is asked to run later, such as one a thread had not yet opened when `stop` was called, returns nil at once, without a request, and `stopped?` tells. `streaming` builds a new streaming client each time it is called, so keep the one you stop in a variable, and call `streaming` again to stream after a stop. A block, an `on_response`, an `on_reconnect`, or a `save_tokens` that is running when the stream is stopped runs to its end first. A stream that a Fiber scheduler runs, as in an `Async` task, waits on the API in the scheduler, so `stop` ends it once its block next returns, at the next keep-alive, within 20 seconds, or within a second as it waits to reconnect; `break` out of its block, or stop its task, to end it at once.
|
|
458
|
+
|
|
459
|
+
**Errors in a stream.** X sends some problems in a line of their own, holding errors and no data, such as the `operational-disconnect` it sends before it closes a stream. Such a line is not a post, so a stream never yields it, whatever it builds objects as: it raises `X::StreamError`, whose `problems` are the `X::Problem` objects of the line. A line of `operational-disconnect`s alone is X asking the stream to reconnect, so the stream reconnects after it, as after a dropped connection, and raises the error once it has no reconnects left; after a line of any other problems it does not reconnect, so you decide whether to open it again. `X::Problem#disconnect?` tells a disconnect apart. A post of the filtered stream names the rules it matched, which `matching_rules` reads as `X::MatchingRule` objects, each with the `id` and `tag` of a rule.
|
|
460
|
+
|
|
461
|
+
X sends a blank line, a CRLF, every 20 seconds to keep an idle stream alive, so a stream reads with a 30-second timeout of its own, which a keep-alive that arrives a little late does not trip. A connection that goes quiet is dropped and reconnected rather than held open until the 60-second `read_timeout` of an ordinary request. The `read_timeout` given to `streaming` must be at least 25 seconds, or nil for none, since one no longer than the 20 seconds between keep-alives would drop a stream that is quiet but connected whenever a keep-alive arrived a little late.
|
|
462
|
+
|
|
76
463
|
```ruby
|
|
77
|
-
# Set up rules for filtered stream
|
|
78
|
-
x_client.
|
|
464
|
+
# Set up rules for the filtered stream
|
|
465
|
+
streaming = x_client.streaming
|
|
466
|
+
streaming.add_rules([{value: "ruby -is:retweet", tag: "ruby"}, "crystal"]) # a rule the app already has raises nothing, and is not returned
|
|
467
|
+
streaming.rules # => [#<X::StreamRule id=1 value="ruby -is:retweet" tag="ruby">, ...]
|
|
468
|
+
|
|
469
|
+
# Stream matching posts in real time, until interrupted
|
|
470
|
+
x_client.streaming.stream("tweets/search/stream") do |post|
|
|
471
|
+
puts post["data"]["text"]
|
|
472
|
+
end
|
|
79
473
|
|
|
80
|
-
# Stream
|
|
81
|
-
|
|
82
|
-
|
|
474
|
+
# Stream posts as X::Post objects, reading the tags of the rules each matched; a line of errors alone, other than the
|
|
475
|
+
# operational-disconnect X sends before it closes a stream, after which it reconnects, raises X::StreamError and stops it
|
|
476
|
+
begin
|
|
477
|
+
x_client.streaming.stream("tweets/search/stream", object_class: X::Post) do |post|
|
|
478
|
+
puts "#{post.matching_rules.map(&:tag).join(", ")}: #{post.text}"
|
|
479
|
+
end
|
|
480
|
+
rescue X::StreamError => e
|
|
481
|
+
warn e.message
|
|
83
482
|
end
|
|
483
|
+
|
|
484
|
+
# Stop the stream from its block, which returns what break was given
|
|
485
|
+
first = x_client.streaming.stream("tweets/search/stream") { |post| break post }
|
|
486
|
+
|
|
487
|
+
# Stop a stream that runs in another thread, even one that delivers nothing
|
|
488
|
+
streaming = x_client.streaming
|
|
489
|
+
reader = Thread.new { streaming.stream("tweets/search/stream") { |post| queue << post } }
|
|
490
|
+
streaming.stop # => nil
|
|
491
|
+
reader.value # => nil
|
|
492
|
+
|
|
493
|
+
# Stop on Ctrl-C
|
|
494
|
+
streaming = x_client.streaming
|
|
495
|
+
Signal.trap("INT") { streaming.stop }
|
|
496
|
+
|
|
497
|
+
# Give up after five reconnects in a row
|
|
498
|
+
streaming_client = x_client.streaming(max_reconnects: 5)
|
|
499
|
+
|
|
500
|
+
# Report each reconnect
|
|
501
|
+
streaming_client = x_client.streaming(on_reconnect: ->(error, wait) { warn "#{error.message}; reconnecting in #{wait}s" })
|
|
502
|
+
|
|
503
|
+
# Log each reconnect, and give up on a stream that has failed to connect for minutes
|
|
504
|
+
streaming_client = x_client.streaming(on_reconnect: lambda do |error, wait|
|
|
505
|
+
warn "#{error.class}: #{error.message}; reconnecting in #{wait} seconds"
|
|
506
|
+
streaming_client.stop if error.is_a?(X::NetworkError) && wait >= 16
|
|
507
|
+
end)
|
|
508
|
+
|
|
509
|
+
# Delete the rules that were read; delete_rules("crystal") deletes the ones that match a value
|
|
510
|
+
streaming.delete_rules(streaming.rules) # => 2
|
|
511
|
+
```
|
|
512
|
+
|
|
513
|
+
### Responses
|
|
514
|
+
|
|
515
|
+
A client calls its `on_response` after every request with an `X::Response`, which counts the resources the response returned and reads its rate limits. The API returns rate limit headers with nearly every response, and some writes add limits on the requests of a day. The API bills the posts a stream delivers, each once a UTC day, so a stream calls `on_response` for each one, with that post as the body.
|
|
516
|
+
|
|
517
|
+
An `X::Response` reads the response itself with `status`, `headers`, and `body`, and an `X::HTTPError` reads a refused one the same three ways. Every error a request raises names the request: `http_method` and `uri` are the method it was sent with and the URL it was sent to, on `X::HTTPError`, `X::NetworkError`, `X::InvalidResponse`, and `X::TooManyRedirects` alike, and the message names it too, as `GET /2/users/1: Could not find user` does, so a log of failures says which endpoint each one came from. The message leaves the query out, since a lookup asks for every field of a resource; `uri` keeps it. Header names are lowercase, whatever case the API sent them in, and a header it sent more than once is joined with a comma. `http_response` is the escape hatch for what those do not read: it holds the response of the transport, a `Net::HTTPResponse` today, whose class is not part of what 1.x promises. An `X::InvalidResponse`, raised for a successful response whose body is not JSON, is an `X::HTTPError` too, and reads that response the same three ways; its `problem` is nil and its `problems` are empty.
|
|
518
|
+
|
|
519
|
+
```ruby
|
|
520
|
+
x_client = X::Client.new(**x_credentials, on_response: lambda { |response|
|
|
521
|
+
puts "#{response.http_method} #{response.uri.path}: #{response.resource_counts}" # {"data" => 100, "users" => 42}
|
|
522
|
+
limit = response.rate_limit
|
|
523
|
+
puts "#{limit.remaining} of #{limit.limit} left, resetting in #{limit.reset_in} seconds" if limit
|
|
524
|
+
})
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
**One request.** A block passed to `get`, `post`, `put`, or `delete` receives the same `X::Response`, for code that reads the response of one request rather than of every request a client makes. It is passed what `on_response` is passed, at the same points: a response the API refused, before the error is raised, and each attempt of a request that was sent again. A request with both reports to the client's hook first, and both receive the one summary.
|
|
528
|
+
|
|
529
|
+
```ruby
|
|
530
|
+
remaining = nil
|
|
531
|
+
user = x_client.get("users/me") { |response| remaining = response.rate_limit&.remaining }
|
|
84
532
|
```
|
|
85
533
|
|
|
534
|
+
**Rate limits.** A request the API refuses for a rate limit raises `X::TooManyRequests`, whose `retry_after` is the number of seconds the response asks a request to wait in its `Retry-After` header, or else the number until the limit that refused it resets, or nil when the response says neither; the header counts from when the response was sent, so it holds however far your clock is from the API's. Its `rate_limits` and `rate_limit` read the response's limits as `X::Response` reads them; the limits with no requests left are `exhausted_rate_limits`, and the one that resets last, which `reset_in` counts down to, is `limiting_rate_limit`. A client can instead wait and retry, up to `max_rate_limit_retries` times, which is 0 by default. It waits only as long as `max_rate_limit_wait`, which is 900 seconds by default, the length of a 15-minute window. A request whose limit resets later, such as a limit on the requests of a day, raises at once, as does one refused for the usage cap of the project, which lasts until the month ends, and which `error.problem&.usage_capped?` tells apart from a rate limit. A refusal that does not say when its limit resets waits a minute before the first retry, doubling the wait for each retry after, as X recommends. Up to five seconds are added at random to each wait, since every request of an app shares the app's limits, and so the moment they reset: without the random share, the requests one reset releases would be sent again in one burst, to be refused together once more. They are added to the wait rather than taken off it, since a request sent before the limit resets is refused again, so `max_rate_limit_wait` is the longest reset a request waits for, not the longest it sleeps. Each retry signs the request afresh and passes its response to `on_response`. The API refuses a rate-limited request without acting on it, so retrying a write does not repeat it. A stream is the exception to the default: `X::StreamingClient` reconnects after a rate limit however `max_rate_limit_retries` is set, though it too waits no longer than `max_rate_limit_wait`, and does not reconnect after the usage cap.
|
|
535
|
+
|
|
536
|
+
```ruby
|
|
537
|
+
x_client = X::Client.new(**x_credentials, max_rate_limit_retries: 3, max_rate_limit_wait: 60)
|
|
538
|
+
patient_client = x_client.with(max_rate_limit_retries: 0) # raise X::TooManyRequests at once again
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
**Failures the request did not cause.** A 5xx response, a 408 Request Timeout, which says the API gave up waiting for the request, and a request whose answer never arrived say nothing about the request itself, so the same request may pass a moment later. A client sends one again after an `X::ServerError` or an `X::RequestTimeout`, or an `X::NetworkError` for a request that never reached the API, such as a connection refused or one that timed out opening, `max_retries` times, which is twice by default, waiting up to a second before the first retry and up to twice as long before each after, but never more than a minute, with a random share of up to half of each wait taken off: a failure of the API fails every request in flight at once, and requests that waited the same time would be sent again together, to fail together once more. Only a `GET`, `PUT`, or `DELETE` is sent again: the API may have acted on a `POST` whose answer never arrived, so that one is left to you. A request that reached the API and whose answer never arrived, such as one that timed out reading its response, is left to you too, since the API bills a read it answered whether or not the answer arrived. `x-uploads` sends each chunk of an upload, its finalize, and alt text or subtitles again after any server error, 408, or network error, even a timeout, up to the `max_retries` of the client, since each is safe to send twice; X bills nothing for a chunk, but bills each alt text or subtitles request, so one whose answer was lost may be billed twice. Any other 4xx is raised at once, since the request is most often the reason for it, and one refused for a state that can change, such as a 409 for a filtered stream with no rules, passes only once that changes (a stream of `X::StreamingClient` reconnects after a 409, as under Streams); a 429 waits for its rate limit as above, and a 401 for an app-only token, or an OAuth 2.0 token with a refresh token, is refreshed and sent once more, whatever `max_retries` is. When the response carries a `Retry-After` header, such as a 503 that names the time its endpoint is expected back, the client waits as long as it asks, or as long as the backoff of the attempt, whichever is longer, so a request is never sent again before the API asked for it. A response that asks to be left alone for longer than a minute raises at once rather than hold you for minutes; `error.retry_after` reads the wait it asked for.
|
|
542
|
+
|
|
543
|
+
```ruby
|
|
544
|
+
x_client = X::Client.new(**x_credentials, max_retries: 4) # or 0, to raise at once
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
Whatever `max_retries` is, an idempotent request that fails on a connection the client had kept open for it, which X or a proxy between can close while it is idle, is sent again on a connection opened for it, when the failure says the connection was closed before the request was sent: that request never reached the API, so nothing is repeated. A request that timed out after it was sent is not sent again, this way or after `max_retries`, since the API may have acted on it; one that timed out opening its connection never reached the API, and is.
|
|
548
|
+
|
|
86
549
|
See other common usage [examples](https://github.com/sferik/x-ruby/tree/main/examples).
|
|
87
550
|
|
|
88
551
|
## History and Philosophy
|
|
89
552
|
|
|
90
|
-
This library is a rewrite of the [Twitter Ruby library](https://github.com/sferik/twitter).
|
|
553
|
+
This library is a from-scratch rewrite of the [Twitter Ruby library](https://github.com/sferik/twitter). Rather than carry that library’s design forward, it aims to be more minimal and modular: a lightweight core, `x-core`, that handles HTTP and little else, with optional gems, `x-uploads`, `x-streams`, and `x-resources`, built on top of it. You pay only for what you use. An application that needs nothing but raw JSON can depend on `x-core` alone and never load the code for uploads, streams, or objects.
|
|
554
|
+
|
|
555
|
+
That doesn’t mean new features won’t be added over time, but the benefits of more code must be weighed against the benefits of less:
|
|
91
556
|
|
|
92
557
|
* Less code is easier to maintain.
|
|
93
558
|
* Less code means fewer bugs.
|
|
@@ -97,27 +562,43 @@ In the immortal words of [Ezra Zygmuntowicz](https://github.com/ezmobius) and hi
|
|
|
97
562
|
|
|
98
563
|
> No code is faster than no code.
|
|
99
564
|
|
|
100
|
-
The tests for the previous version of this library executed in about 2 seconds. That sounds pretty fast until you see that tests for this library run in one-twentieth of a second. This means you can automatically run the tests any time you write a file and receive immediate feedback. For such of workflows, 2 seconds feels painfully slow.
|
|
101
|
-
|
|
102
|
-
This code is not littered with comments that are intended to generate documentation. Rather, this code is intended to be simple enough to serve as its own documentation. If you want to understand how something works, don’t read the documentation—it might be wrong—read the code. The code is always right.
|
|
103
|
-
|
|
104
565
|
## Features
|
|
105
566
|
|
|
106
|
-
If this entire library is implemented in
|
|
567
|
+
If this entire library is implemented in about 7,000 lines of code, why should you use it at all vs. writing your own library that suits your needs? If you feel inspired to do that, don’t let me discourage you, but this library has some advanced features that may not be initially apparent, including:
|
|
107
568
|
|
|
108
569
|
* OAuth 1.0 Revision A
|
|
109
570
|
* OAuth 2.0
|
|
571
|
+
* App-only authentication
|
|
110
572
|
* Thread safety
|
|
573
|
+
* Persistent HTTP connections, reused across requests to the same host
|
|
111
574
|
* HTTP redirect following
|
|
112
|
-
* HTTP proxy support
|
|
575
|
+
* HTTP and HTTPS proxy support, configured or taken from the environment
|
|
113
576
|
* HTTP logging
|
|
114
577
|
* HTTP timeout configuration
|
|
578
|
+
* HTTP headers set per client, per request, or both
|
|
115
579
|
* HTTP error handling
|
|
116
580
|
* Rate limit handling
|
|
117
|
-
*
|
|
118
|
-
*
|
|
581
|
+
* Retrying a request after waiting for its rate limit to reset
|
|
582
|
+
* Sending an idempotent request again after a 5xx response, a 408, or a network failure before it reached the API
|
|
583
|
+
* Sending an idempotent request again on a new connection when one kept open had been closed at the other end
|
|
584
|
+
* Streaming (filtered stream, sample stream)
|
|
585
|
+
* Reconnecting a dropped stream
|
|
586
|
+
* Immutable resource objects with identity, references, and hydration
|
|
587
|
+
* Lazy, cached, Enumerable cursors that request the maximum page size
|
|
119
588
|
* Configurable base URLs for accessing different APIs/versions
|
|
589
|
+
* Query strings, JSON bodies, and form bodies built from Ruby values
|
|
590
|
+
* Uploading any file with one call, with alt text and subtitles
|
|
120
591
|
* Parallel uploading of large media files in chunks
|
|
592
|
+
* Parallel batch lookups, as many at a time as you ask for
|
|
593
|
+
* Partial errors reported by a response that otherwise succeeded
|
|
594
|
+
|
|
595
|
+
## Versioning
|
|
596
|
+
|
|
597
|
+
The gems follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html), and 1.x promises their public interface: every class, module, constant, method, and keyword argument whose documentation says `@api public`, with the types the RBS signatures each gem ships give them, which declare that interface alone; the signatures of its internals, in `sig/internal`, are not shipped. A minor release may add to that interface and a patch release may fix it, but only a major release removes or changes what it promises.
|
|
598
|
+
|
|
599
|
+
Anything documented `@api private` is internal to the gem that declares it, even where Ruby lets you call it, as is every private constant, and either may change or go away in any release. A module documented `@api semipublic` is internal in the same way: its methods are public API of the class or client that includes or extends it, and are promised there, but the module itself is not one to include, extend, or name. Most of it sits in the module of its gem, such as `X::Core`, beside mixins that are public, such as `X::Resources::API` and `X::Uploads::MediaUpload`, so it is the documentation, not the namespace, that says which is which.
|
|
600
|
+
|
|
601
|
+
The five gems are released together at the same version. `x` depends on exactly that version of the other four, such as `= 1.0.0` for 1.0.0, so it installs the set that was built and tested together, and its version names the version of each. `x-uploads`, `x-streams`, and `x-resources` each depend on `x-core` with a constraint of their own version or a later one of the same major version, such as `>= 1.0.0, < 2` for 1.0.0, so an app that depends on them without `x` may install a later release of 1.x of one beside the others, but never an earlier release of `x-core` than their own. They are built and tested together at each version, so upgrade them together, as depending on `x` does.
|
|
121
602
|
|
|
122
603
|
## Sponsorship
|
|
123
604
|
|
|
@@ -143,54 +624,6 @@ Many thanks to our sponsors (listed in order of when they sponsored this project
|
|
|
143
624
|
<img src="https://raw.githubusercontent.com/sferik/x-ruby/main/sponsor_logos/spacer.png" width="20" align="middle">
|
|
144
625
|
<a href="https://ifttt.com"><img src="https://raw.githubusercontent.com/sferik/x-ruby/main/sponsor_logos/ifttt.svg" alt="IFTTT" width="200" align="middle"></a>
|
|
145
626
|
|
|
146
|
-
## Development
|
|
147
|
-
|
|
148
|
-
1. Clone the repo:
|
|
149
|
-
|
|
150
|
-
git clone git@github.com:sferik/x-ruby.git
|
|
151
|
-
|
|
152
|
-
2. Enter the repo’s directory:
|
|
153
|
-
|
|
154
|
-
cd x-ruby
|
|
155
|
-
|
|
156
|
-
3. Install dependencies via Bundler:
|
|
157
|
-
|
|
158
|
-
bin/setup
|
|
159
|
-
|
|
160
|
-
4. Run the default Rake task to ensure all tests pass:
|
|
161
|
-
|
|
162
|
-
bundle exec rake
|
|
163
|
-
|
|
164
|
-
5. Create a new branch for your feature or bug fix:
|
|
165
|
-
|
|
166
|
-
git checkout -b my-new-branch
|
|
167
|
-
|
|
168
|
-
## Contributing
|
|
169
|
-
|
|
170
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/sferik/x-ruby.
|
|
171
|
-
|
|
172
|
-
Pull requests will only be accepted if they meet all the following criteria:
|
|
173
|
-
|
|
174
|
-
1. Code must conform to [Standard Ruby](https://github.com/standardrb/standard#readme). This can be verified with:
|
|
175
|
-
|
|
176
|
-
bundle exec rake standard
|
|
177
|
-
|
|
178
|
-
2. Code must conform to the [RuboCop rules](https://github.com/rubocop/rubocop#readme). This can be verified with:
|
|
179
|
-
|
|
180
|
-
bundle exec rake rubocop
|
|
181
|
-
|
|
182
|
-
3. 100% C0 code coverage. This can be verified with:
|
|
183
|
-
|
|
184
|
-
bundle exec rake test
|
|
185
|
-
|
|
186
|
-
4. 100% mutation coverage. This can be verified with:
|
|
187
|
-
|
|
188
|
-
bundle exec rake mutant
|
|
189
|
-
|
|
190
|
-
5. RBS type signatures (in `sig/x.rbs`). This can be verified with:
|
|
191
|
-
|
|
192
|
-
bundle exec rake steep
|
|
193
|
-
|
|
194
627
|
## License
|
|
195
628
|
|
|
196
629
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|