minisky 0.4.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b22e3b707a428f0381e5c194ce6b5bd4118f6b6dfb7f2d249140c8025e3b1dd7
-  data.tar.gz: 2a6370ea6cb538c680252836295c411ff4c6df93b8c5aab9d8b73b843753febb
+  metadata.gz: 4ed55493afc5d821a8e67ea83bcdc8aa29a7d2c9c3afe40e178441a499516a2d
+  data.tar.gz: 351119843f56206f7205872a26cbf66758273dfd35fe54b064a346ebb31d8bcb
 SHA512:
-  metadata.gz: 3f86def9480e86d2989a52f143eacee8e5b78fede45bd6ba49ea7ad54f1da88a55555421d3fdfdddad025bed21a708c830e5e9b2c351ba57e016234e200b9f74
-  data.tar.gz: f8f7faa8d7e5df7807d6e9bb7ee1b87d1d5ab5503c0193fd031190be6e6346004618c66efdba96970fc93117d4f587c53ff5f17585a2ffa3ddafb503abc5a3cc
+  metadata.gz: bc684d8ffe0105e8c8d198acf35c6e9ca4a4856e7a74961adaab548d6471e140bb3d4e42c23ef710caab5449c12136f72c08522ddc68b244b99ca9019a8be832
+  data.tar.gz: 196a97d055b3cb6714df1379cbe5b64c8798e5e09e432bdf8ec652374f500cc8e6a34d39d60c976f2fc373c547d90f367fe54fe791e8d9d3eea98d618c96651a

data/CHANGELOG.md

@@ -1,3 +1,15 @@
+## [0.5.0] - 2024-12-27 🎄
+
+* `host` param in the initializer can be passed with a `https://` prefix (useful if you're passing it directly from a DID document, e.g. using DIDKit)
+* added validation of the `method` parameter in request calls: it needs to be either a proper NSID, or a full URL as a string or a URI object
+* added new optional `params` keyword argument in `post_request`, which lets you append query parameters to the URL if a POST endpoint requires passing them this way
+* `default_progress` is set by default to show progress using dots (`.`) if Minisky is loaded inside an IRB or Pry context
+* when experimenting with Minisky in the console, you can now skip the `field:` parameter to `fetch_all` if you don't remember the expected key name in the response, and the method will make a request and return an error which tells you the list of available keys
+* added `access_token_expired?` helper method
+* moved `token_expiration_date` to public methods
+* `check_access` now returns a result symbol: `:logged_in`, `:refreshed` or `:ok`
+* fixed `method_missing` setter on `User`
+
 ## [0.4.0] - 2024-03-31 🐣
 
 * allow passing non-JSON body to requests (e.g. when uploading blobs)

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The zlib License
 
-Copyright (c) 2023 Jakub Suder
+Copyright (c) 2023-2024 Jakub Suder
 
 This software is provided 'as-is', without any express or implied
 warranty.  In no event will the authors be held liable for any damages

data/README.md

@@ -1,7 +1,12 @@
-# Minisky
+# Minisky 🌤
 
 Minisky is a minimal client of the Bluesky (ATProto) API. It provides a simple API client class that you can use to log in to the Bluesky API and make any GET and POST requests there. It's meant to be an easy way to start playing and experimenting with the AT Protocol API.
 
+This is designed as a low-level XRPC client library - it purposefully does not include any convenience methods like "get posts" or "get profile" etc., it only provides base components that you could use to build a higher level API.
+
+> [!NOTE]
+> ATProto Ruby gems collection: [skyfall](https://github.com/mackuba/skyfall) | [blue_factory](https://github.com/mackuba/blue_factory) | [minisky](https://github.com/mackuba/minisky) | [didkit](https://github.com/mackuba/didkit)
+
 
 ## Installation
 
@@ -13,7 +18,7 @@ To install the Minisky gem, run the command:
 
 Or alternatively, add it to the `Gemfile` file for Bundler:
 
-    gem 'minisky', '~> 0.3'
+    gem 'minisky', '~> 0.5'
 
 
 ## Usage
@@ -31,14 +36,21 @@ This allows you to do things like:
 - look up profile information about any account
 - load complete threads or users' profile feeds from the AppView
 
-To use Minisky this way, create a `Minisky` instance passing the API hostname string (at the moment there is only one server at `bsky.social`, but there will be more once federation support goes live) and `nil` as the configuration in the arguments:
+To use Minisky this way, create a `Minisky` instance, passing the API hostname string and `nil` as the configuration in the arguments. Use the hostname `api.bsky.app` or `public.api.bsky.app` for the AppView, or a PDS hostname for the `com.atproto.*` raw data endpoints:
 
 ```rb
 require 'minisky'
 
-bsky = Minisky.new('bsky.social', nil)
+bsky = Minisky.new('api.bsky.app', nil)
 ```
 
+> [!NOTE]
+> To call PDS endpoints like `getRecord` or `listRecords`, you need to connect to the PDS of the user whose data you're loading, not to yours (unless it's the same one). Alternatively, you can use the `bsky.social` "entryway" PDS hostname for any Bluesky-hosted accounts, but this will not work for self-hosted accounts.
+>
+> To look up the PDS hostname of a user given their handle or DID, you can use the [didkit](https://github.com/mackuba/didkit) library.
+>
+> For the AppView, `api.bsky.app` connects directly to Bluesky's AppView, and `public.api.bsky.app` to a version with extra caching that will usually be faster.
+
 
 ### Authenticated access
 
@@ -53,9 +65,12 @@ pass: very-secret-password
 
 The `id` can be either your handle, or your DID, or the email you've used to sign up. It's recommended that you use the "app password" that you can create in the settings instead of your main account password.
 
+> [!NOTE]
+> Bluesky has recently implemented OAuth, but Minisky doesn't support it yet - it will be added in a future version. App passwords should still be supported for a fairly long time.
+
 After you log in, this file will also be used to store your access & request tokens and DID. The data in the config file can be accessed through a `user` wrapper property that exposes them as methods, e.g. the password is available as `user.pass` and the DID as `user.did`.
 
-Next, create the Minisky client instance, passing the server name and the config file name:
+Next, create the Minisky client instance, passing your PDS hostname (for Bluesky-hosted PDSes, you can use either `bsky.social` or your specific PDS like `amanita.us-east.host.bsky.network`) and the name of the config file:
 
 ```rb
 require 'minisky'
@@ -93,7 +108,7 @@ bsky.post_request('com.atproto.repo.createRecord', {
 
 In authenticated mode, the requests use the saved access token for auth headers automatically. You can also pass `auth: false` or `auth: nil` to not send any authentication headers for a given request, or `auth: sometoken` to use a specific other token. In unauthenticated mode, sending of auth headers is disabled.
 
-The third useful method you can use is `#fetch_all`, which loads multiple paginated responses and collects all returned items on a single list (you need to pass the name of the field that contains the items in the response). Optionally, you can also specify a limit of pages to load as `max_pages: n`, or a break condition `break_when` to stop fetching when any item matches it. You can use it to e.g. to fetch all of your posts from the last 30 days, but not earlier:
+The third useful method you can use is `#fetch_all`, which loads multiple paginated responses and collects all returned items on a single list (you need to pass the name of the field that contains the items in the response). Optionally, you can also specify a limit of pages to load as `max_pages: n`, or a break condition `break_when` to stop fetching when any item matches it. You can use it to e.g. to fetch all of your posts from the last 30 days but not earlier:
 
 ```rb
 time_limit = Time.now - 86400 * 30
@@ -127,7 +142,7 @@ You can find more examples in the [example](https://github.com/mackuba/minisky/t
 
 The `Minisky` client currently supports such configuration options:
 
-- `default_progress` - a progress character to automatically use for `#fetch_all` calls (default: `nil`)
+- `default_progress` - a progress character to automatically use for `#fetch_all` calls (default: `.` when in an interactive console, `nil` otherwise)
 - `send_auth_headers` - whether auth headers should be added by default (default: `true` in authenticated mode)
 - `auto_manage_tokens` - whether access tokens should be generated and refreshed automatically when needed (default: `true` in authenticated mode)
 
@@ -177,7 +192,7 @@ The class needs to provide:
 
 ## Credits
 
-Copyright © 2023 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
+Copyright © 2024 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
 
 The code is available under the terms of the [zlib license](https://choosealicense.com/licenses/zlib/) (permissive, similar to MIT).
 

data/example/find_missing_follows.rb

@@ -0,0 +1,67 @@
+#!/usr/bin/env ruby
+
+# Example: fetch the list of accounts followed by a given user and check which of them have been deleted / deactivated.
+
+# load minisky from a local folder - you normally won't need this
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+
+require 'didkit'
+require 'minisky'
+
+handle = ARGV[0].to_s.gsub(/^@/, '')
+if handle.empty?
+  puts "Usage: #{$PROGRAM_NAME} <handle>"
+  exit 1
+end
+
+pds_host = DID.resolve_handle(handle).get_document.pds_endpoint
+pds = Minisky.new(pds_host, nil, progress: '.')
+
+print "Fetching all follows of @#{handle} from #{pds_host}: "
+
+follows = pds.fetch_all('com.atproto.repo.listRecords',
+  { repo: handle, collection: 'app.bsky.graph.follow', limit: 100 }, field: 'records')
+
+puts
+puts "Found #{follows.length} follows"
+
+appview = Minisky.new('api.bsky.app', nil)
+
+profiles = []
+i = 0
+
+puts
+print "Fetching profiles of all followed accounts: "
+
+# getProfiles lets us load multiple profiles in one request, but only up to 25 in one batch
+
+while i < follows.length
+  batch = follows[i...i+25]
+  dids = batch.map { |x| x['value']['subject'] }
+  print '.'
+  result = appview.get_request('app.bsky.actor.getProfiles', { actors: dids })
+  profiles += result['profiles']
+  i += 25
+end
+
+# these are DIDs that are on the follows list, but aren't being returned from getProfiles
+missing = follows.map { |x| x['value']['subject'] } - profiles.map { |x| x['did'] }
+
+puts
+puts "#{missing.length} followed accounts are missing:"
+puts
+
+missing.each do |did|
+  begin
+    doc = DID.new(did).get_document
+  rescue OpenURI::HTTPError
+    puts "#{did} (?) => DID not found"
+    next
+  end
+
+  # check account status on their assigned PDS
+  pds = Minisky.new(doc.pds_endpoint.gsub('http://', ''), nil)
+  status = pds.get_request('com.atproto.sync.getRepoStatus', { did: did }).slice('status', 'active') rescue 'deleted'
+
+  puts "#{did} (@#{doc.handles.first}) => #{status}"
+end

data/lib/minisky/errors.rb

@@ -52,4 +52,13 @@ class Minisky
       @location = location
     end
   end
+
+  class FieldNotSetError < Error
+    attr_reader :fields
+
+    def initialize(fields)
+      @fields = fields
+      super("Field parameter not provided; available fields: #{@fields.inspect}")
+    end
+  end
 end

data/lib/minisky/minisky.rb

@@ -19,6 +19,10 @@ class Minisky
       @auto_manage_tokens = false
     end
 
+    if active_repl?
+      @default_progress = '.'
+    end
+
     if options
       options.each do |k, v|
         self.send("#{k}=", v)
@@ -26,6 +30,12 @@ class Minisky
     end
   end
 
+  def active_repl?
+    return true if defined?(IRB) && IRB.respond_to?(:CurrentContext) && IRB.CurrentContext
+    return true if defined?(Pry) && Pry.respond_to?(:cli) && Pry.cli
+    false
+  end
+
   def save_config
     File.write(@config_file, YAML.dump(@config)) if @config_file
   end

data/lib/minisky/requests.rb

@@ -18,7 +18,7 @@ class Minisky
     end
 
     def method_missing(name, *args)
-      if name.end_with?('=')
+      if name.to_s.end_with?('=')
         @config[name.to_s.chop] = args[0]
       else
         @config[name.to_s]
@@ -26,6 +26,8 @@ class Minisky
     end
   end
 
+  NSID_REGEXP = /^[a-zA-Z]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)+(\.[a-zA-Z]([a-zA-Z]{0,61}[a-zA-Z])?)$/
+
   module Requests
     attr_accessor :default_progress
     attr_writer :send_auth_headers
@@ -43,7 +45,11 @@ class Minisky
     alias progress= default_progress=
 
     def base_url
-      @base_url ||= "https://#{host}/xrpc"
+      if host.include?('://')
+        host.chomp('/') + '/xrpc'
+      else
+        "https://#{host}/xrpc"
+      end
     end
 
     def user
@@ -54,7 +60,7 @@ class Minisky
       check_access if auto_manage_tokens && auth == true
 
       headers = authentication_header(auth).merge(headers || {})
-      url = URI("#{base_url}/#{method}")
+      url = build_request_uri(method)
 
       if params && !params.empty?
         url.query = URI.encode_www_form(params)
@@ -66,24 +72,30 @@ class Minisky
       handle_response(response)
     end
 
-    def post_request(method, params = nil, auth: default_auth_mode, headers: nil)
+    def post_request(method, data = nil, auth: default_auth_mode, headers: nil, params: nil)
       check_access if auto_manage_tokens && auth == true
 
       headers = authentication_header(auth).merge(headers || {})
       headers["Content-Type"] = "application/json" unless headers.keys.any? { |k| k.to_s.downcase == 'content-type' }
 
-      body = if params.is_a?(String) || params.nil?
-        params.to_s
+      body = if data.is_a?(String) || data.nil?
+        data.to_s
       else
-        params.to_json
+        data.to_json
       end
 
-      response = Net::HTTP.post(URI("#{base_url}/#{method}"), body, headers)
+      url = build_request_uri(method)
+
+      if params && !params.empty?
+        url.query = URI.encode_www_form(params)
+      end
+
+      response = Net::HTTP.post(url, body, headers)
       handle_response(response)
     end
 
-    def fetch_all(method, params = nil, field:,
-                  auth: default_auth_mode, break_when: nil, max_pages: nil, headers: nil, progress: @default_progress)
+    def fetch_all(method, params = nil, auth: default_auth_mode,
+                  field: nil, break_when: nil, max_pages: nil, headers: nil, progress: @default_progress)
       data = []
       params = {} if params.nil?
       pages = 0
@@ -92,6 +104,11 @@ class Minisky
         print(progress) if progress
 
         response = get_request(method, params, auth: auth, headers: headers)
+
+        if field.nil?
+          raise FieldNotSetError, response.keys.select { |f| response[f].is_a?(Array) }
+        end
+
         records = response[field]
         cursor = response['cursor']
 
@@ -110,8 +127,12 @@ class Minisky
     def check_access
       if !user.logged_in?
         log_in
-      elsif token_expiration_date(user.access_token) < Time.now + 60
+        :logged_in
+      elsif access_token_expired?
         perform_token_refresh
+        :refreshed
+      else
+        :ok
       end
     end
 
@@ -149,6 +170,26 @@ class Minisky
       json
     end
 
+    def token_expiration_date(token)
+      parts = token.split('.')
+      raise AuthError, "Invalid access token format" unless parts.length == 3
+
+      begin
+        payload = JSON.parse(Base64.decode64(parts[1]))
+      rescue JSON::ParserError
+        raise AuthError, "Couldn't decode payload from access token"
+      end
+
+      exp = payload['exp']
+      raise AuthError, "Invalid token expiry data" unless exp.is_a?(Numeric) && exp > 0
+
+      Time.at(exp)
+    end
+
+    def access_token_expired?
+      token_expiration_date(user.access_token) < Time.now + 60
+    end
+
     def reset_tokens
       user.access_token = nil
       user.refresh_token = nil
@@ -182,6 +223,18 @@ class Minisky
       end
     end
 
+    def build_request_uri(method)
+      if method.is_a?(URI)
+        method
+      elsif method.include?('://')
+        URI(method)
+      elsif method =~ NSID_REGEXP
+        URI("#{base_url}/#{method}")
+      else
+        raise ArgumentError, "Invalid method name #{method.inspect} (should be an NSID, URL or an URI object)"
+      end
+    end
+
     def default_auth_mode
       !!send_auth_headers
     end
@@ -200,22 +253,6 @@ class Minisky
       end
     end
 
-    def token_expiration_date(token)
-      parts = token.split('.')
-      raise AuthError, "Invalid access token format" unless parts.length == 3
-
-      begin
-        payload = JSON.parse(Base64.decode64(parts[1]))
-      rescue JSON::ParserError
-        raise AuthError, "Couldn't decode payload from access token"
-      end
-
-      exp = payload['exp']
-      raise AuthError, "Invalid token expiry data" unless exp.is_a?(Numeric) && exp > 0
-
-      Time.at(exp)
-    end
-
     def handle_response(response)
       status = response.code.to_i
       message = response.message

data/lib/minisky/version.rb

@@ -1,5 +1,5 @@
 require_relative 'minisky'
 
 class Minisky
-  VERSION = "0.4.0"
+  VERSION = "0.5.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: minisky
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Kuba Suder
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-03-31 00:00:00.000000000 Z
+date: 2024-12-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: base64
@@ -38,6 +38,7 @@ files:
 - example/ask_password.rb
 - example/fetch_my_posts.rb
 - example/fetch_profile.rb
+- example/find_missing_follows.rb
 - example/post_skeet.rb
 - example/science_feed.rb
 - lib/minisky.rb