skyfall 0.6.1 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +58 -10
- data/README.md +40 -34
- data/lib/skyfall/collection.rb +20 -0
- data/lib/skyfall/errors.rb +50 -5
- data/lib/skyfall/events.rb +19 -0
- data/lib/skyfall/firehose/account_message.rb +32 -4
- data/lib/skyfall/firehose/commit_message.rb +49 -8
- data/lib/skyfall/firehose/identity_message.rb +30 -2
- data/lib/skyfall/firehose/info_message.rb +35 -1
- data/lib/skyfall/firehose/labels_message.rb +28 -10
- data/lib/skyfall/firehose/message.rb +136 -27
- data/lib/skyfall/firehose/operation.rb +59 -6
- data/lib/skyfall/firehose/sync_message.rb +31 -0
- data/lib/skyfall/firehose/unknown_message.rb +8 -0
- data/lib/skyfall/firehose.rb +103 -5
- data/lib/skyfall/jetstream/account_message.rb +21 -1
- data/lib/skyfall/jetstream/commit_message.rb +35 -2
- data/lib/skyfall/jetstream/identity_message.rb +22 -1
- data/lib/skyfall/jetstream/message.rb +94 -7
- data/lib/skyfall/jetstream/operation.rb +58 -5
- data/lib/skyfall/jetstream/unknown_message.rb +8 -0
- data/lib/skyfall/jetstream.rb +94 -7
- data/lib/skyfall/label.rb +35 -1
- data/lib/skyfall/stream.rb +264 -47
- data/lib/skyfall/version.rb +1 -1
- metadata +21 -64
- data/lib/skyfall/car_archive.rb +0 -122
- data/lib/skyfall/cid.rb +0 -43
- data/lib/skyfall/extensions.rb +0 -32
- data/lib/skyfall/firehose/handle_message.rb +0 -14
- data/lib/skyfall/firehose/tombstone_message.rb +0 -11
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c52f2f3c63530b1e032666c7bfb71ba0b0279bfe0128c7432b8e267f1e2f9e95
|
|
4
|
+
data.tar.gz: 1c7e2e5cac7ec4f421be5c291a311dede53221150901a44418f01f7a6cdeae9e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1eb9556cbc363f6a9c2ae42d27dc9a4471a24f8e08f0b1b8046c131fe320b63de73498d839367f0fd89c3841b27877c53bc0a937b9eeed41ea486f05b55a38c3
|
|
7
|
+
data.tar.gz: 9b61caf5cc63834efd8185622c435a9102b4a310f890a988fd029e9e691e80acf7527e0afc5e5fecde967c4f2de538559168d07a72a2af4dd9d5ef06cc16ab50
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,51 @@
|
|
|
1
|
+
## [0.7.1] - 2026-08-05
|
|
2
|
+
|
|
3
|
+
- extracted `CID` and `CarArchive` to a new gem "oxygene"
|
|
4
|
+
- added `CommitMessage#cid` as alias of `#commit`
|
|
5
|
+
|
|
6
|
+
## [0.7.0] - 2026-02-13
|
|
7
|
+
|
|
8
|
+
The main change in this version is that inline [YARD documentation](https://rubydoc.info/gems/skyfall/) has been added. This was also a good opportunity to review some APIs and tweak some things in order to get Skyfall a bit closer to 1.0.
|
|
9
|
+
|
|
10
|
+
New APIs:
|
|
11
|
+
|
|
12
|
+
- the `Skyfall::Firehose` initializer now allows skipping `:subscribe_repos`, i.e. `.new(host)` or `.new(host, cursor)`
|
|
13
|
+
- added `Skyfall::Jetstream::CommitMessage#operation` (aliased as `op`) which returns the (always single) operation in the `operations` array
|
|
14
|
+
- added `#kind` as alias for `#type` in both `Message` classes
|
|
15
|
+
- added a base class for error types, `Skyfall::Error`
|
|
16
|
+
- added `#blocks` to `Skyfall::Firehose::SyncMessage`
|
|
17
|
+
- added `#rev`, `#since` and `#prev_data` to `Skyfall::Firehose::CommitMessage`
|
|
18
|
+
|
|
19
|
+
Deprecated & removed APIs:
|
|
20
|
+
|
|
21
|
+
- removed deprecated `HandleMessage` and `TombstoneMessage` message classes
|
|
22
|
+
- removed deprecated `CommitMessage#prev`
|
|
23
|
+
- deprecated `#path` in both `Operation` classes
|
|
24
|
+
|
|
25
|
+
Optimizations:
|
|
26
|
+
|
|
27
|
+
- much faster `Skyfall::Firehose::Message#time` parsing on Ruby 3.2+
|
|
28
|
+
- lazy decoding of sections in `CarArchive` – saves quite a lot of work if sections are only accessed through `Operation#raw_record`
|
|
29
|
+
- added `frozen_string_literal: true` in all files to reduce garbage collection
|
|
30
|
+
|
|
31
|
+
Access level changes:
|
|
32
|
+
|
|
33
|
+
- restricted `Stream#start_heartbeat_timer` & `Stream#stop_heartbeat_timer` methods' access to private
|
|
34
|
+
- restricted `Stream#handle_message` method access to protected
|
|
35
|
+
- restricted `Stream#last_update` to read-only access
|
|
36
|
+
- restricted `#inspectable_variables` access to either private or protected
|
|
37
|
+
- relaxed `Stream#build_websocket_url` & `Stream#build_websocket_client` access from private to protected
|
|
38
|
+
- fixed private class method `Skyfall::Firehose::Message.decode_cbor_objects` which wasn't actually private
|
|
39
|
+
|
|
40
|
+
Additional validations and other changes:
|
|
41
|
+
|
|
42
|
+
- `Stream#connect` throws an error if neither `on_message` nor `on_raw_message` handlers have been configured
|
|
43
|
+
- `Message` subclasses do additional checks if the fields they require to not be nil aren't nil
|
|
44
|
+
- `Message` subclasses raise an error if `.new` is called on a subclass (and not on the base `Message`) passing the data of a wrong kind of message (instead of returning e.g. a `CommitMessage` from `AccountMessage.new` as it worked previously)
|
|
45
|
+
- made `LabelsMessage` a subclass of `Firehose::Message`
|
|
46
|
+
- fixed the `require`s config in some files so they can be loaded in any order
|
|
47
|
+
|
|
48
|
+
|
|
1
49
|
## [0.6.1] - 2026-01-08
|
|
2
50
|
|
|
3
51
|
- added `:bsky_notif_declaration` shortcode for `app.bsky.notification.declaration` collection
|
|
@@ -7,7 +55,7 @@
|
|
|
7
55
|
|
|
8
56
|
## [0.6.0] - 2025-06-25
|
|
9
57
|
|
|
10
|
-
- significantly speeded up reading of events from the binary firehose (`Skyfall::Firehose`)
|
|
58
|
+
- significantly speeded up reading of events from the binary firehose (`Skyfall::Firehose`) – up to 4-5x faster than before
|
|
11
59
|
- removed the `Skyfall::Stream.new` constructor deprecated in 0.5.0
|
|
12
60
|
|
|
13
61
|
## [0.5.1] - 2025-05-18
|
|
@@ -30,11 +78,11 @@ This required some breaking changes in the existing API:
|
|
|
30
78
|
|
|
31
79
|
In most cases, you should only need to update the `Skyfall::Stream` class name in the constructor. If you've referenced message classes like `Skyfall::CommitMessage` directly, it's probably better to just check the `#type` property instead.
|
|
32
80
|
|
|
33
|
-
Also, small change to the user agent API: `Skyfall::Stream` now has an additional metod `version_string`, which will always return `Skyfall/0.x.y`
|
|
81
|
+
Also, small change to the user agent API: `Skyfall::Stream` now has an additional metod `version_string`, which will always return `Skyfall/0.x.y` – it's recommended to use that instead of `default_user_agent` to build your own user agent string that includes the library version. `default_user_agent` now passes through to `version_string`, but it could be changed in future to return something else.
|
|
34
82
|
|
|
35
83
|
## [0.4.1] - 2024-10-04
|
|
36
84
|
|
|
37
|
-
- performance fix
|
|
85
|
+
- performance fix – don't decode CAR sections which aren't needed, which is most of them; this cuts the amount of memory that GC has to free up by about one third, and should speed up processing by around ~10%
|
|
38
86
|
|
|
39
87
|
## [0.4.0] - 2024-09-23
|
|
40
88
|
|
|
@@ -48,15 +96,15 @@ Also, small change to the user agent API: `Skyfall::Stream` now has an additiona
|
|
|
48
96
|
- added `#account` event type (`AccountMessage`)
|
|
49
97
|
- added `handle` field to `IdentityMessage`
|
|
50
98
|
- fixed param validation on `Stream` initialization
|
|
51
|
-
- reverted the change that added Ruby stdlib dependencies explicitly to the gemspec, since this causes more problems than it's worth
|
|
99
|
+
- reverted the change that added Ruby stdlib dependencies explicitly to the gemspec, since this causes more problems than it's worth – only `base64` is left there, since it's the one now required to be listed
|
|
52
100
|
|
|
53
101
|
## [0.3.0] - 2024-03-21
|
|
54
102
|
|
|
55
103
|
- added support for labeller firehose, served by labeller services at the `com.atproto.label.subscribeLabels` endpoint (aliased as `:subscribe_labels`)
|
|
56
104
|
- the `#labels` messages from the labeller firehose are parsed into a `LabelsMessage`, which includes a `labels` array of `Label` objects
|
|
57
105
|
- `Stream` callbacks can now also be assigned via setters, e.g. `stream.on_message = proc { ... }`
|
|
58
|
-
- added default error handler to `Stream` which logs the error to `$stdout`
|
|
59
|
-
- added Ruby stdlib dependencies explicitly to the gemspec
|
|
106
|
+
- added default error handler to `Stream` which logs the error to `$stdout` – set `stream.on_error = nil` to disable
|
|
107
|
+
- added Ruby stdlib dependencies explicitly to the gemspec – fixes a warning in Ruby 3.3 when requiring `base64`, which will be extracted as an optional gem in 3.4
|
|
60
108
|
|
|
61
109
|
## [0.2.5] - 2024-03-14
|
|
62
110
|
|
|
@@ -83,7 +131,7 @@ Also, small change to the user agent API: `Skyfall::Stream` now has an additiona
|
|
|
83
131
|
|
|
84
132
|
## [0.2.1] - 2023-08-19
|
|
85
133
|
|
|
86
|
-
- optimized `WebsocketMessage` parsing performance
|
|
134
|
+
- optimized `WebsocketMessage` parsing performance – lazy parsing of most properties (message decoding should be over 50% faster on average)
|
|
87
135
|
- added separate subclasses of `WebsocketMessage` for different message types
|
|
88
136
|
- added support for `#handle`, `#info` and `#tombstone` message types
|
|
89
137
|
- `UnknownMessage` is returned for unrecognized message types
|
|
@@ -91,13 +139,13 @@ Also, small change to the user agent API: `Skyfall::Stream` now has an additiona
|
|
|
91
139
|
## [0.2.0] - 2023-07-24
|
|
92
140
|
|
|
93
141
|
- switched the websocket library from `websocket-client-simple` to `faye-websocket`, which should make event parsing up to ~30× faster (!)
|
|
94
|
-
- added `auto_reconnect` property to `Stream` (on by default)
|
|
142
|
+
- added `auto_reconnect` property to `Stream` (on by default) – if true, it will try to reconnect with an exponential backoff when the websocket disconnects, until you call `Stream#disconnect`
|
|
95
143
|
|
|
96
144
|
Note:
|
|
97
145
|
|
|
98
|
-
- calling `sleep` is no longer needed after connecting
|
|
146
|
+
- calling `sleep` is no longer needed after connecting – call `connect` on a new thread instead to get previously default behavior of running the event loop asynchronously
|
|
99
147
|
- the disconnect event no longer passes an error object in the argument
|
|
100
|
-
- there is currently no "heartbeat" feature as in 0.1.x that checks for a stuck connection
|
|
148
|
+
- there is currently no "heartbeat" feature as in 0.1.x that checks for a stuck connection – but it doesn't seem to be needed
|
|
101
149
|
|
|
102
150
|
## [0.1.3] - 2023-07-04
|
|
103
151
|
|
data/README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Skyfall
|
|
2
2
|
|
|
3
|
+
[](https://rubygems.org/gems/skyfall) [](https://rubydoc.info/gems/skyfall)
|
|
4
|
+
|
|
3
5
|
A Ruby gem for streaming data from the Bluesky/ATProto firehose 🦋
|
|
4
6
|
|
|
5
7
|
> [!NOTE]
|
|
@@ -15,7 +17,7 @@ Since version 0.5, Skyfall also supports connecting to [Jetstream](https://githu
|
|
|
15
17
|
|
|
16
18
|
## Installation
|
|
17
19
|
|
|
18
|
-
To use Skyfall, you need a reasonably new version of Ruby – it should run on Ruby 2.6 and above, although it's recommended to use a version that's still getting maintainance updates, i.e. currently 3.
|
|
20
|
+
To use Skyfall, you need a reasonably new version of Ruby – it should run on Ruby 2.6 and above, although it's recommended to use a version that's still getting maintainance updates, i.e. currently 3.3+. A compatible version should be preinstalled on macOS Big Sur and above and on many Linux systems. Otherwise, you can install one using tools such as [RVM](https://rvm.io), [asdf](https://asdf-vm.com), [ruby-install](https://github.com/postmodern/ruby-install) or [ruby-build](https://github.com/rbenv/ruby-build), or `rpm` or `apt-get` on Linux (see more installation options on [ruby-lang.org](https://www.ruby-lang.org/en/downloads/)).
|
|
19
21
|
|
|
20
22
|
To install the gem, run the command:
|
|
21
23
|
|
|
@@ -38,7 +40,7 @@ require 'skyfall'
|
|
|
38
40
|
sky = Skyfall::Firehose.new('bsky.network', :subscribe_repos)
|
|
39
41
|
```
|
|
40
42
|
|
|
41
|
-
The server name can be just a hostname, or a full URL with a `ws:` or `wss:` scheme, which is useful if you want to use a non-encrypted websocket connection, e.g. `"ws://localhost:8000"`. The endpoint can be either a full NSID string like `"com.atproto.sync.subscribeRepos"`, or one of the defined symbol shortcuts
|
|
43
|
+
The server name can be just a hostname, or a full URL with a `ws:` or `wss:` scheme, which is useful if you want to use a non-encrypted websocket connection, e.g. `"ws://localhost:8000"`. The endpoint can be either a full NSID string like `"com.atproto.sync.subscribeRepos"`, or one of the defined symbol shortcuts – you will almost always want to pass `:subscribe_repos` here.
|
|
42
44
|
|
|
43
45
|
Next, set up event listeners to handle incoming messages and get notified of errors. Here are all the available listeners (you will need at least either `on_message` or `on_raw_message`):
|
|
44
46
|
|
|
@@ -60,7 +62,7 @@ sky.on_timeout { puts "Connection stalled, triggering a reconnect..." }
|
|
|
60
62
|
sky.on_error { |e| puts "ERROR: #{e}" }
|
|
61
63
|
```
|
|
62
64
|
|
|
63
|
-
You can also call these as setters accepting a `Proc`
|
|
65
|
+
You can also call these as setters accepting a `Proc` – e.g. to disable default error handling, you can do:
|
|
64
66
|
|
|
65
67
|
```rb
|
|
66
68
|
sky.on_error = nil
|
|
@@ -94,7 +96,7 @@ sky.connect
|
|
|
94
96
|
|
|
95
97
|
### Cursors
|
|
96
98
|
|
|
97
|
-
ATProto websocket endpoints implement a "*cursor*" feature to help you make sure that you don't miss anything if your connection is down for a bit (because of a network issue, server restart, deploy etc.). Each message includes a `seq` field, which is the sequence number of the event. You can keep track of the last seq you've seen, and when you reconnect, you pass that number as a cursor parameter
|
|
99
|
+
ATProto websocket endpoints implement a "*cursor*" feature to help you make sure that you don't miss anything if your connection is down for a bit (because of a network issue, server restart, deploy etc.). Each message includes a `seq` field, which is the sequence number of the event. You can keep track of the last seq you've seen, and when you reconnect, you pass that number as a cursor parameter – the server will then "replay" all events you might have missed since that last one. (The `bsky.network` Relay firehose currently has a buffer of about 72 hours, though that's not something required by specification.)
|
|
98
100
|
|
|
99
101
|
To use a cursor when connecting to the firehose, pass it as the third parameter to `Skyfall::Firehose`. You should then regularly save the `seq` of the last event to some permanent storage, and then load it from there when reconnecting.
|
|
100
102
|
|
|
@@ -127,60 +129,57 @@ end
|
|
|
127
129
|
|
|
128
130
|
Each message passed to `on_message` is an instance of a subclass of either `Skyfall::Firehose::Message` or `Skyfall::Jetstream::Message`, depending on the selected source. The supported message types are:
|
|
129
131
|
|
|
130
|
-
- `CommitMessage` (`#commit`)
|
|
131
|
-
- `IdentityMessage` (`#identity`)
|
|
132
|
-
- `AccountMessage` (`#account`)
|
|
133
|
-
- `
|
|
134
|
-
- `
|
|
135
|
-
- `
|
|
136
|
-
- `InfoMessage` (`#info`) - a protocol error message, e.g. about an invalid cursor parameter
|
|
132
|
+
- `CommitMessage` (`#commit`) – represents a change in a user's repo; most messages are of this type
|
|
133
|
+
- `IdentityMessage` (`#identity`) – notifies about a change in user's DID document, e.g. a handle change or a migration to a new PDS
|
|
134
|
+
- `AccountMessage` (`#account`) – notifies about a change of an account's status (de/activation, suspension, deletion)
|
|
135
|
+
- `SyncMessage` (`#sync`) – updates repository state, can be used to trigger account resynchronization
|
|
136
|
+
- `LabelsMessage` (`#labels`) – only used in `subscribe_labels` endpoint
|
|
137
|
+
- `InfoMessage` (`#info`) – a protocol error message, e.g. about an invalid cursor parameter
|
|
137
138
|
- `UnknownMessage` is used for other unrecognized message types
|
|
138
139
|
|
|
139
|
-
`#handle` and `#tombstone` events are considered deprecated, replaced by `#identity` and `#account` respectively. They are still being emitted at the moment (in parallel with the newer event types), but they might stop being sent at any moment, so it's recommended that you don't rely on those.
|
|
140
|
-
|
|
141
140
|
`Skyfall::Firehose::Message` and `Skyfall::Jetstream::Message` variants of message classes should have more or less the same interface, except when a given field is not included in one of the formats.
|
|
142
141
|
|
|
143
142
|
All message objects have the following shared properties:
|
|
144
143
|
|
|
145
|
-
- `type` (symbol)
|
|
146
|
-
- `seq` (integer)
|
|
147
|
-
- `repo` or `did` (string)
|
|
148
|
-
- `time` (Time)
|
|
144
|
+
- `type` (symbol) – the message type identifier, e.g. `:commit`
|
|
145
|
+
- `seq` (integer) – a sequential index of the message; Jetstream messages instead have a `time_us` value, which is a Unix timestamp in microseconds (also aliased as `seq` for compatibility)
|
|
146
|
+
- `repo` or `did` (string) – DID of the repository (user account)
|
|
147
|
+
- `time` (Time) – timestamp of the described action
|
|
149
148
|
|
|
150
149
|
All properties except `type` may be nil for some message types that aren't related to a specific user, like `#info`.
|
|
151
150
|
|
|
152
151
|
Commit messages additionally have:
|
|
153
152
|
|
|
154
|
-
- `commit`
|
|
155
|
-
- `operations`
|
|
153
|
+
- `commit` – CID of the commit
|
|
154
|
+
- `operations` – list of operations (usually one)
|
|
156
155
|
|
|
157
156
|
Handle and Identity messages additionally have:
|
|
158
157
|
|
|
159
|
-
- `handle`
|
|
158
|
+
- `handle` – the new handle assigned to the DID
|
|
160
159
|
|
|
161
160
|
Account messages additionally have:
|
|
162
161
|
|
|
163
|
-
- `active?`
|
|
164
|
-
- `status`
|
|
162
|
+
- `active?` – whether the account is active, or inactive for any reason
|
|
163
|
+
- `status` – if not active, shows the status of the account (`:deactivated`, `:deleted`, `:takendown`)
|
|
165
164
|
|
|
166
165
|
Info messages additionally have:
|
|
167
166
|
|
|
168
|
-
- `name`
|
|
169
|
-
- `message`
|
|
167
|
+
- `name` – identifier of the message/error
|
|
168
|
+
- `message` – a human-readable description
|
|
170
169
|
|
|
171
170
|
|
|
172
171
|
### Commit operations
|
|
173
172
|
|
|
174
173
|
Operations are objects of type `Skyfall::Firehose::Operation` or `Skyfall::Jetstream::Operation` and have such properties:
|
|
175
174
|
|
|
176
|
-
- `repo` or `did` (string)
|
|
177
|
-
- `collection` (string)
|
|
178
|
-
- `type` (symbol)
|
|
179
|
-
- `rkey` (string)
|
|
180
|
-
- `path` (string)
|
|
181
|
-
- `uri` (string)
|
|
182
|
-
- `action` (symbol)
|
|
183
|
-
- `cid` (CID)
|
|
175
|
+
- `repo` or `did` (string) – DID of the repository (user account)
|
|
176
|
+
- `collection` (string) – name of the relevant collection in the repository, e.g. `app.bsky.feed.post` for posts
|
|
177
|
+
- `type` (symbol) – short name of the collection, e.g. `:bsky_post`
|
|
178
|
+
- `rkey` (string) – identifier of a record in a collection
|
|
179
|
+
- `path` (string) – the path part of the at:// URI – collection name + ID (rkey) of the item
|
|
180
|
+
- `uri` (string) – the complete at:// URI
|
|
181
|
+
- `action` (symbol) – `:create`, `:update` or `:delete`
|
|
182
|
+
- `cid` (CID) – CID of the operation/record (`nil` for delete operations)
|
|
184
183
|
|
|
185
184
|
Create and update operations will also have an attached record (JSON object) with details of the post, like etc. The record data is currently available as a Ruby hash via `raw_record` property (custom types will be added in future).
|
|
186
185
|
|
|
@@ -209,7 +208,7 @@ Note that the `Operation` objects have two properties that tell you the kind of
|
|
|
209
208
|
|
|
210
209
|
When Skyfall receives a message about a record type that's not on the list, whether in the `app.bsky` namespace or not, the operation `type` will be `:unknown`, while the `collection` will be the original string. So if an app like e.g. "Skygram" appears with a `zz.skygram.*` namespace that lets you share photos on ATProto, the operations will have a type `:unknown` and collection names like `zz.skygram.feed.photo`, and you can check the `collection` field for record types known to you and process them in some appropriate way, even if Skyfall doesn't recognize the record type.
|
|
211
210
|
|
|
212
|
-
Do not however check if such operations have a `type` equal to `:unknown` first
|
|
211
|
+
Do not however check if such operations have a `type` equal to `:unknown` first – just ignore the type and only check the `collection` string. The reason is that some next version of Skyfall might start recognizing those records and add a new `type` value for them like e.g. `:skygram_photo`, and then they won't match your condition anymore.
|
|
213
212
|
|
|
214
213
|
|
|
215
214
|
## Reconnection logic
|
|
@@ -263,7 +262,7 @@ sky.on_message do |msg|
|
|
|
263
262
|
end
|
|
264
263
|
```
|
|
265
264
|
|
|
266
|
-
See [ATProto label docs](https://atproto.com/specs/label) for info on what fields are included with each label
|
|
265
|
+
See [ATProto label docs](https://atproto.com/specs/label) for info on what fields are included with each label – `Skyfall::Label` includes properties with these original names, and also more friendly aliases for each (e.g. `value` instead of `val`).
|
|
267
266
|
|
|
268
267
|
|
|
269
268
|
## Other configuration
|
|
@@ -305,6 +304,13 @@ See [Jetstream docs](https://github.com/bluesky-social/jetstream?tab=readme-ov-f
|
|
|
305
304
|
> The `compress` and `requireHello` options (and zstd compression) are not available at the moment. Also the "subscriber sourced messages" aren't implemented yet.
|
|
306
305
|
|
|
307
306
|
|
|
307
|
+
## Other resources
|
|
308
|
+
|
|
309
|
+
- [YARD API documentation](https://rubydoc.info/gems/skyfall) at rubydoc.info
|
|
310
|
+
- [ruby.sdk.blue](https://ruby.sdk.blue)
|
|
311
|
+
- [Example scripts](https://ruby.sdk.blue/examples/)
|
|
312
|
+
- [bluesky-feeds-rb](https://tangled.org/mackuba.eu/bluesky-feeds-rb) – feed generator template project
|
|
313
|
+
|
|
308
314
|
## Credits
|
|
309
315
|
|
|
310
316
|
Copyright © 2026 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/did:plc:oio4hkxaop4ao4wz2pp3f4cr)).
|
data/lib/skyfall/collection.rb
CHANGED
|
@@ -1,4 +1,13 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
module Skyfall
|
|
4
|
+
|
|
5
|
+
#
|
|
6
|
+
# This module defines constants for known Bluesky record collection types, and a mapping of those
|
|
7
|
+
# names to symbol short codes which can be used as shorthand when processing events or in
|
|
8
|
+
# Jetstream filters.
|
|
9
|
+
#
|
|
10
|
+
|
|
2
11
|
module Collection
|
|
3
12
|
BSKY_PROFILE = "app.bsky.actor.profile"
|
|
4
13
|
BSKY_ACTOR_STATUS = "app.bsky.actor.status"
|
|
@@ -20,6 +29,8 @@ module Skyfall
|
|
|
20
29
|
BSKY_NOTIF_DECLARATION = "app.bsky.notification.declaration"
|
|
21
30
|
BSKY_CHAT_DECLARATION = "chat.bsky.actor.declaration"
|
|
22
31
|
|
|
32
|
+
# Mapping of NSID collection names to symbol short codes
|
|
33
|
+
|
|
23
34
|
SHORT_CODES = {
|
|
24
35
|
BSKY_ACTOR_STATUS => :bsky_actor_status,
|
|
25
36
|
BSKY_BLOCK => :bsky_block,
|
|
@@ -41,10 +52,19 @@ module Skyfall
|
|
|
41
52
|
BSKY_NOTIF_DECLARATION => :bsky_notif_declaration
|
|
42
53
|
}
|
|
43
54
|
|
|
55
|
+
# Returns a symbol short code for a given collection NSID, or `:unknown`
|
|
56
|
+
# if NSID is not on the list.
|
|
57
|
+
# @param collection [String] collection NSID
|
|
58
|
+
# @return [Symbol] short code or :unknown
|
|
59
|
+
|
|
44
60
|
def self.short_code(collection)
|
|
45
61
|
SHORT_CODES[collection] || :unknown
|
|
46
62
|
end
|
|
47
63
|
|
|
64
|
+
# Returns a collection NSID assigned to a given short code symbol, if one is defined.
|
|
65
|
+
# @param code [Symbol] one of the symbols listed in {SHORT_CODES}
|
|
66
|
+
# @return [String, nil] assigned NSID string, or nil when code is not known
|
|
67
|
+
|
|
48
68
|
def self.from_short_code(code)
|
|
49
69
|
SHORT_CODES.detect { |k, v| v == code }&.first
|
|
50
70
|
end
|
data/lib/skyfall/errors.rb
CHANGED
|
@@ -1,11 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
module Skyfall
|
|
2
|
-
|
|
4
|
+
#
|
|
5
|
+
# Wrapper base class for Skyfall error classes.
|
|
6
|
+
#
|
|
7
|
+
class Error < StandardError
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
#
|
|
11
|
+
# Raised when some code is not configured or configured incorrectly.
|
|
12
|
+
#
|
|
13
|
+
class ConfigError < Error
|
|
3
14
|
end
|
|
4
15
|
|
|
5
|
-
|
|
16
|
+
#
|
|
17
|
+
# Raised when some part of the message being decoded has invalid format.
|
|
18
|
+
#
|
|
19
|
+
class DecodeError < Error
|
|
6
20
|
end
|
|
7
21
|
|
|
8
|
-
|
|
22
|
+
#
|
|
23
|
+
# Raised when {Stream#connect} is called and there's already another instance of {Stream} or its
|
|
24
|
+
# subclass like {Firehose} that's connected to another websocket.
|
|
25
|
+
#
|
|
26
|
+
# This is currently not supported in Skyfall, because it uses EventMachine behind the scenes, which
|
|
27
|
+
# runs everything on a single "reactor" thread, and there can be only one such reactor thread in
|
|
28
|
+
# a given process. In theory, it should be possible for two connections to run inside a single
|
|
29
|
+
# shared EventMachine event loop, but it would require some more coordination and it might have
|
|
30
|
+
# unexpected side effects - e.g. synchronous work (including I/O and network requests) done during
|
|
31
|
+
# processing of an event from one connection would be blocking the other connection.
|
|
32
|
+
#
|
|
33
|
+
class ReactorActiveError < Error
|
|
9
34
|
def initialize
|
|
10
35
|
super(
|
|
11
36
|
"An EventMachine reactor thread is already running, but it seems to have been launched by another Stream. " +
|
|
@@ -14,9 +39,22 @@ module Skyfall
|
|
|
14
39
|
end
|
|
15
40
|
end
|
|
16
41
|
|
|
17
|
-
|
|
18
|
-
|
|
42
|
+
#
|
|
43
|
+
# Raised when the server sends a message which is formatted correctly, but describes some kind of
|
|
44
|
+
# error condition that the server has detected.
|
|
45
|
+
#
|
|
46
|
+
class SubscriptionError < Error
|
|
47
|
+
|
|
48
|
+
# @return [String] a short machine-readable error code
|
|
49
|
+
attr_reader :error_type
|
|
19
50
|
|
|
51
|
+
# @return [String] a human-readable error message
|
|
52
|
+
attr_reader :error_message
|
|
53
|
+
|
|
54
|
+
#
|
|
55
|
+
# @param error_type [String] a short machine-readable error code
|
|
56
|
+
# @param error_message [String, nil] a human-readable error message
|
|
57
|
+
#
|
|
20
58
|
def initialize(error_type, error_message = nil)
|
|
21
59
|
@error_type = error_type
|
|
22
60
|
@error_message = error_message
|
|
@@ -24,4 +62,11 @@ module Skyfall
|
|
|
24
62
|
super("Subscription error: #{error_type}" + (error_message ? " (#{error_message})" : ""))
|
|
25
63
|
end
|
|
26
64
|
end
|
|
65
|
+
|
|
66
|
+
#
|
|
67
|
+
# Raised when the server sends a message which is formatted correctly, but written in a version
|
|
68
|
+
# that's not supported by this library.
|
|
69
|
+
#
|
|
70
|
+
class UnsupportedError < Error
|
|
71
|
+
end
|
|
27
72
|
end
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Skyfall
|
|
4
|
+
|
|
5
|
+
# @private
|
|
6
|
+
module Events
|
|
7
|
+
protected
|
|
8
|
+
|
|
9
|
+
def event_handler(name)
|
|
10
|
+
define_method("on_#{name}") do |&block|
|
|
11
|
+
@handlers[name.to_sym] = block
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
define_method("on_#{name}=") do |block|
|
|
15
|
+
@handlers[name.to_sym] = block
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
@@ -1,13 +1,41 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require_relative '../firehose'
|
|
4
|
+
require_relative 'message'
|
|
2
5
|
|
|
3
6
|
module Skyfall
|
|
7
|
+
|
|
8
|
+
#
|
|
9
|
+
# Firehose message sent when the status of an account changes. This can be:
|
|
10
|
+
#
|
|
11
|
+
# - an account being created, sending its initial state (should be active)
|
|
12
|
+
# - an account being deactivated or suspended
|
|
13
|
+
# - an account being restored back to an active state from deactivation/suspension
|
|
14
|
+
# - an account being deleted (the status returning `:deleted`)
|
|
15
|
+
#
|
|
16
|
+
|
|
4
17
|
class Firehose::AccountMessage < Firehose::Message
|
|
5
|
-
|
|
6
|
-
|
|
18
|
+
|
|
19
|
+
#
|
|
20
|
+
# @private
|
|
21
|
+
# @param type_object [Hash] first decoded CBOR frame with metadata
|
|
22
|
+
# @param data_object [Hash] second decoded CBOR frame with payload
|
|
23
|
+
# @raise [DecodeError] if the message doesn't include required data
|
|
24
|
+
#
|
|
25
|
+
def initialize(type_object, data_object)
|
|
26
|
+
super
|
|
27
|
+
check_if_not_nil 'seq', 'did', 'time', 'active'
|
|
28
|
+
|
|
29
|
+
@active = @data_object['active']
|
|
30
|
+
@status = @data_object['status']&.to_sym
|
|
7
31
|
end
|
|
8
32
|
|
|
9
|
-
|
|
10
|
-
|
|
33
|
+
# @return [Boolean] true if the account is active, false if it's deactivated/suspended etc.
|
|
34
|
+
def active?
|
|
35
|
+
@active
|
|
11
36
|
end
|
|
37
|
+
|
|
38
|
+
# @return [Symbol, nil] for inactive accounts, specifies the exact state; nil for active accounts
|
|
39
|
+
attr_reader :status
|
|
12
40
|
end
|
|
13
41
|
end
|
|
@@ -1,27 +1,68 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
3
|
require_relative '../firehose'
|
|
4
|
+
require_relative 'message'
|
|
4
5
|
require_relative 'operation'
|
|
6
|
+
require 'oxygene'
|
|
5
7
|
|
|
6
8
|
module Skyfall
|
|
9
|
+
|
|
10
|
+
#
|
|
11
|
+
# Firehose message which includes one or more operations on records in the repo (a record was
|
|
12
|
+
# created, updated or deleted). In most cases this is a single record operation.
|
|
13
|
+
#
|
|
14
|
+
# Most of the messages received from the firehose are of this type, and this is the type you
|
|
15
|
+
# will usually be most interested in.
|
|
16
|
+
#
|
|
17
|
+
|
|
7
18
|
class Firehose::CommitMessage < Firehose::Message
|
|
8
|
-
|
|
9
|
-
|
|
19
|
+
|
|
20
|
+
#
|
|
21
|
+
# @private
|
|
22
|
+
# @param type_object [Hash] first decoded CBOR frame with metadata
|
|
23
|
+
# @param data_object [Hash] second decoded CBOR frame with payload
|
|
24
|
+
# @raise [DecodeError] if the message doesn't include required data
|
|
25
|
+
#
|
|
26
|
+
def initialize(type_object, data_object)
|
|
27
|
+
super
|
|
28
|
+
check_if_not_nil 'seq', 'repo', 'commit', 'blocks', 'ops', 'time', 'rev'
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
# @return [String] current revision of the repo
|
|
32
|
+
def rev
|
|
33
|
+
@data_object['rev']
|
|
10
34
|
end
|
|
11
35
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
@
|
|
36
|
+
# @return [String, nil] revision of the previous commit in the repo
|
|
37
|
+
def since
|
|
38
|
+
@data_object['since']
|
|
15
39
|
end
|
|
16
40
|
|
|
41
|
+
# @return [Oxygene::CID, nil] CID (Content Identifier) of data of the previous commit in the repo
|
|
42
|
+
def prev_data
|
|
43
|
+
@prev_data ||= Oxygene::CID.from_cbor_tag(@data_object['prevData'])
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# @return [Oxygene::CID] CID (Content Identifier) of the commit
|
|
47
|
+
def commit
|
|
48
|
+
@commit ||= Oxygene::CID.from_cbor_tag(@data_object['commit'])
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
alias cid commit
|
|
52
|
+
|
|
53
|
+
# @return [Oxygene::CARArchive] commit data in the form of a parsed CAR archive
|
|
17
54
|
def blocks
|
|
18
|
-
@blocks ||=
|
|
55
|
+
@blocks ||= Oxygene::CARArchive.new(@data_object['blocks'])
|
|
19
56
|
end
|
|
20
57
|
|
|
58
|
+
# @return [Array<Firehose::Operation>] record operations (usually one) included in the commit
|
|
21
59
|
def operations
|
|
22
60
|
@operations ||= @data_object['ops'].map { |op| Firehose::Operation.new(self, op) }
|
|
23
61
|
end
|
|
24
62
|
|
|
63
|
+
# Looks up record data assigned to a given operation in the commit's CAR archive.
|
|
64
|
+
# @param op [Firehose::Operation]
|
|
65
|
+
# @return [Hash, nil]
|
|
25
66
|
def raw_record_for_operation(op)
|
|
26
67
|
op.cid && blocks.section_with_cid(op.cid)
|
|
27
68
|
end
|
|
@@ -1,9 +1,37 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require_relative '../firehose'
|
|
4
|
+
require_relative 'message'
|
|
2
5
|
|
|
3
6
|
module Skyfall
|
|
7
|
+
|
|
8
|
+
#
|
|
9
|
+
# Firehose message sent when a new DID is created or when the details of someone's DID document
|
|
10
|
+
# are changed (usually either a handle change or a migration to a different PDS). The message
|
|
11
|
+
# may include currently assigned handle, though it's not required that this field is set.
|
|
12
|
+
#
|
|
13
|
+
# Note: the message is originally emitted from the account's PDS and is passed as is by relays,
|
|
14
|
+
# which means you can't fully trust that the handle is actually correctly assigned to the DID
|
|
15
|
+
# and verified by DNS or well-known. To confirm that, use `DID.resolve_handle` from
|
|
16
|
+
# [DIDKit](https://ruby.sdk.blue/didkit/).
|
|
17
|
+
#
|
|
18
|
+
|
|
4
19
|
class Firehose::IdentityMessage < Firehose::Message
|
|
5
|
-
|
|
6
|
-
|
|
20
|
+
|
|
21
|
+
#
|
|
22
|
+
# @private
|
|
23
|
+
# @param type_object [Hash] first decoded CBOR frame with metadata
|
|
24
|
+
# @param data_object [Hash] second decoded CBOR frame with payload
|
|
25
|
+
# @raise [DecodeError] if the message doesn't include required data
|
|
26
|
+
#
|
|
27
|
+
def initialize(type_object, data_object)
|
|
28
|
+
super
|
|
29
|
+
check_if_not_nil 'seq', 'did', 'time'
|
|
30
|
+
|
|
31
|
+
@handle = @data_object['handle']
|
|
7
32
|
end
|
|
33
|
+
|
|
34
|
+
# @return [String, nil] current handle assigned to the DID
|
|
35
|
+
attr_reader :handle
|
|
8
36
|
end
|
|
9
37
|
end
|
|
@@ -1,22 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require_relative '../firehose'
|
|
4
|
+
require_relative 'message'
|
|
2
5
|
|
|
3
6
|
module Skyfall
|
|
7
|
+
|
|
8
|
+
#
|
|
9
|
+
# An informational firehose message from the websocket service itself, unrelated to any repos.
|
|
10
|
+
#
|
|
11
|
+
# Currently there is only one type of message defined, `"OutdatedCursor"`, which is sent when
|
|
12
|
+
# the client connects with a cursor that is older than the oldest event currently kept in the
|
|
13
|
+
# backfill buffer. This message means that you're likely missing some events that were sent
|
|
14
|
+
# since the last time the client was connected but which were already deleted from the buffer.
|
|
15
|
+
#
|
|
16
|
+
# Note: the {#did}, {#seq} and {#time} properties are always `nil` for `#info` messages.
|
|
17
|
+
#
|
|
18
|
+
|
|
4
19
|
class Firehose::InfoMessage < Firehose::Message
|
|
5
|
-
attr_reader :name, :message
|
|
6
20
|
|
|
21
|
+
# @return [String] short machine-readable code of the info message
|
|
22
|
+
attr_reader :name
|
|
23
|
+
|
|
24
|
+
# @return [String, nil] a human-readable description
|
|
25
|
+
attr_reader :message
|
|
26
|
+
|
|
27
|
+
# Message which means that the cursor passed when connecting is older than the oldest event
|
|
28
|
+
# currently kept in the backfill buffer, and that you've likely missed some events that have
|
|
29
|
+
# already been deleted
|
|
7
30
|
OUTDATED_CURSOR = "OutdatedCursor"
|
|
8
31
|
|
|
32
|
+
#
|
|
33
|
+
# @private
|
|
34
|
+
# @param type_object [Hash] first decoded CBOR frame with metadata
|
|
35
|
+
# @param data_object [Hash] second decoded CBOR frame with payload
|
|
36
|
+
# @raise [DecodeError] if the message doesn't include required data
|
|
37
|
+
#
|
|
9
38
|
def initialize(type_object, data_object)
|
|
10
39
|
super
|
|
40
|
+
check_if_not_nil 'name'
|
|
11
41
|
|
|
12
42
|
@name = @data_object['name']
|
|
13
43
|
@message = @data_object['message']
|
|
14
44
|
end
|
|
15
45
|
|
|
46
|
+
# @return [String] a formatted summary
|
|
16
47
|
def to_s
|
|
17
48
|
(@name || "InfoMessage") + (@message ? ": #{@message}" : "")
|
|
18
49
|
end
|
|
19
50
|
|
|
51
|
+
protected
|
|
52
|
+
|
|
53
|
+
# @return [Array<Symbol>] list of instance variables to be printed in the {#inspect} output
|
|
20
54
|
def inspectable_variables
|
|
21
55
|
super - [:@did, :@seq]
|
|
22
56
|
end
|