minisky 0.3.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7a753ae5a6f7d2dd3089ee9b6a637e578051261c615fedd28a44f63ac1774b4d
-  data.tar.gz: 479240c065cb3ba705aa4a9d07efa8c4904ec2aca2e6ed0e847d391ab8cd62fe
+  metadata.gz: 4ed55493afc5d821a8e67ea83bcdc8aa29a7d2c9c3afe40e178441a499516a2d
+  data.tar.gz: 351119843f56206f7205872a26cbf66758273dfd35fe54b064a346ebb31d8bcb
 SHA512:
-  metadata.gz: f87814325757d1b19cee4e403d6e13afbf84b1353b932eedbce23faa6cb1abfe772d0cf2c459b21bb45ea08fa34173457dea3fdb266f32f1f4334e2e90d22692
-  data.tar.gz: c4bea17a09982db0462e15e092fdf43c454fd94163d04099b119104fde555387621c5c1b8d51e1a1247ad2cece0b5fd9258b1c98555b0ffb7eca4b96107dd83a
+  metadata.gz: bc684d8ffe0105e8c8d198acf35c6e9ca4a4856e7a74961adaab548d6471e140bb3d4e42c23ef710caab5449c12136f72c08522ddc68b244b99ca9019a8be832
+  data.tar.gz: 196a97d055b3cb6714df1379cbe5b64c8798e5e09e432bdf8ec652374f500cc8e6a34d39d60c976f2fc373c547d90f367fe54fe791e8d9d3eea98d618c96651a

data/CHANGELOG.md

@@ -1,3 +1,24 @@
+## [0.5.0] - 2024-12-27 🎄
+
+* `host` param in the initializer can be passed with a `https://` prefix (useful if you're passing it directly from a DID document, e.g. using DIDKit)
+* added validation of the `method` parameter in request calls: it needs to be either a proper NSID, or a full URL as a string or a URI object
+* added new optional `params` keyword argument in `post_request`, which lets you append query parameters to the URL if a POST endpoint requires passing them this way
+* `default_progress` is set by default to show progress using dots (`.`) if Minisky is loaded inside an IRB or Pry context
+* when experimenting with Minisky in the console, you can now skip the `field:` parameter to `fetch_all` if you don't remember the expected key name in the response, and the method will make a request and return an error which tells you the list of available keys
+* added `access_token_expired?` helper method
+* moved `token_expiration_date` to public methods
+* `check_access` now returns a result symbol: `:logged_in`, `:refreshed` or `:ok`
+* fixed `method_missing` setter on `User`
+
+## [0.4.0] - 2024-03-31 🐣
+
+* allow passing non-JSON body to requests (e.g. when uploading blobs)
+* allow passing custom headers to requests, including overriding `Content-Type`
+* fixed error when the response is success but not JSON (e.g. an empty body like in deleteRecord)
+* allow passing options to the client in the initializer
+* aliased `default_progress` setting as `progress`
+* added `base64` dependency explicitly to the gemspec - fixes a warning in Ruby 3.3, since it will be extracted as an optional gem in 3.4
+
 ## [0.3.1] - 2023-10-10
 
 * fixed Minisky not working on Ruby 2.x

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The zlib License
 
-Copyright (c) 2023 Jakub Suder
+Copyright (c) 2023-2024 Jakub Suder
 
 This software is provided 'as-is', without any express or implied
 warranty.  In no event will the authors be held liable for any damages

data/README.md

@@ -1,7 +1,12 @@
-# Minisky
+# Minisky 🌤
 
 Minisky is a minimal client of the Bluesky (ATProto) API. It provides a simple API client class that you can use to log in to the Bluesky API and make any GET and POST requests there. It's meant to be an easy way to start playing and experimenting with the AT Protocol API.
 
+This is designed as a low-level XRPC client library - it purposefully does not include any convenience methods like "get posts" or "get profile" etc., it only provides base components that you could use to build a higher level API.
+
+> [!NOTE]
+> ATProto Ruby gems collection: [skyfall](https://github.com/mackuba/skyfall) | [blue_factory](https://github.com/mackuba/blue_factory) | [minisky](https://github.com/mackuba/minisky) | [didkit](https://github.com/mackuba/didkit)
+
 
 ## Installation
 
@@ -13,7 +18,7 @@ To install the Minisky gem, run the command:
 
 Or alternatively, add it to the `Gemfile` file for Bundler:
 
-    gem 'minisky', '~> 0.3'
+    gem 'minisky', '~> 0.5'
 
 
 ## Usage
@@ -31,14 +36,21 @@ This allows you to do things like:
 - look up profile information about any account
 - load complete threads or users' profile feeds from the AppView
 
-To use Minisky this way, create a `Minisky` instance passing the API hostname string (at the moment there is only one server at `bsky.social`, but there will be more once federation support goes live) and `nil` as the configuration in the arguments:
+To use Minisky this way, create a `Minisky` instance, passing the API hostname string and `nil` as the configuration in the arguments. Use the hostname `api.bsky.app` or `public.api.bsky.app` for the AppView, or a PDS hostname for the `com.atproto.*` raw data endpoints:
 
 ```rb
 require 'minisky'
 
-bsky = Minisky.new('bsky.social', nil)
+bsky = Minisky.new('api.bsky.app', nil)
 ```
 
+> [!NOTE]
+> To call PDS endpoints like `getRecord` or `listRecords`, you need to connect to the PDS of the user whose data you're loading, not to yours (unless it's the same one). Alternatively, you can use the `bsky.social` "entryway" PDS hostname for any Bluesky-hosted accounts, but this will not work for self-hosted accounts.
+>
+> To look up the PDS hostname of a user given their handle or DID, you can use the [didkit](https://github.com/mackuba/didkit) library.
+>
+> For the AppView, `api.bsky.app` connects directly to Bluesky's AppView, and `public.api.bsky.app` to a version with extra caching that will usually be faster.
+
 
 ### Authenticated access
 
@@ -53,9 +65,12 @@ pass: very-secret-password
 
 The `id` can be either your handle, or your DID, or the email you've used to sign up. It's recommended that you use the "app password" that you can create in the settings instead of your main account password.
 
+> [!NOTE]
+> Bluesky has recently implemented OAuth, but Minisky doesn't support it yet - it will be added in a future version. App passwords should still be supported for a fairly long time.
+
 After you log in, this file will also be used to store your access & request tokens and DID. The data in the config file can be accessed through a `user` wrapper property that exposes them as methods, e.g. the password is available as `user.pass` and the DID as `user.did`.
 
-Next, create the Minisky client instance, passing the server name and the config file name:
+Next, create the Minisky client instance, passing your PDS hostname (for Bluesky-hosted PDSes, you can use either `bsky.social` or your specific PDS like `amanita.us-east.host.bsky.network`) and the name of the config file:
 
 ```rb
 require 'minisky'
@@ -93,7 +108,7 @@ bsky.post_request('com.atproto.repo.createRecord', {
 
 In authenticated mode, the requests use the saved access token for auth headers automatically. You can also pass `auth: false` or `auth: nil` to not send any authentication headers for a given request, or `auth: sometoken` to use a specific other token. In unauthenticated mode, sending of auth headers is disabled.
 
-The third useful method you can use is `#fetch_all`, which loads multiple paginated responses and collects all returned items on a single list (you need to pass the name of the field that contains the items in the response). Optionally, you can also specify a limit of pages to load as `max_pages: n`, or a break condition `break_when` to stop fetching when any item matches it. You can use it to e.g. to fetch all of your posts from the last 30 days, but not earlier:
+The third useful method you can use is `#fetch_all`, which loads multiple paginated responses and collects all returned items on a single list (you need to pass the name of the field that contains the items in the response). Optionally, you can also specify a limit of pages to load as `max_pages: n`, or a break condition `break_when` to stop fetching when any item matches it. You can use it to e.g. to fetch all of your posts from the last 30 days but not earlier:
 
 ```rb
 time_limit = Time.now - 86400 * 30
@@ -127,7 +142,7 @@ You can find more examples in the [example](https://github.com/mackuba/minisky/t
 
 The `Minisky` client currently supports such configuration options:
 
-- `default_progress` - a progress character to automatically use for `#fetch_all` calls (default: `nil`)
+- `default_progress` - a progress character to automatically use for `#fetch_all` calls (default: `.` when in an interactive console, `nil` otherwise)
 - `send_auth_headers` - whether auth headers should be added by default (default: `true` in authenticated mode)
 - `auto_manage_tokens` - whether access tokens should be generated and refreshed automatically when needed (default: `true` in authenticated mode)
 
@@ -177,7 +192,7 @@ The class needs to provide:
 
 ## Credits
 
-Copyright © 2023 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
+Copyright © 2024 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
 
 The code is available under the terms of the [zlib license](https://choosealicense.com/licenses/zlib/) (permissive, similar to MIT).
 

data/example/ask_password.rb

@@ -0,0 +1,76 @@
+#!/usr/bin/env ruby
+
+# Example: print 10 latest posts from the user's home feed.
+# 
+# Instead of using a config file to read & store authentication info, this example uses a customized client class
+# which reads the password from the console and creates a throwaway access token.
+#
+# This approach makes sense for one-off scripts, but it shouldn't be used for things that need to be done repeatedly
+# and often (the authentication-related endpoints have lower rate limits than others).
+
+# load minisky from a local folder - you normally won't need this
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+
+require 'io/console'
+require 'minisky'
+
+class TransientClient
+  include Minisky::Requests
+
+  attr_reader :config, :host
+
+  def initialize(host, user)
+    @host = host
+    @config = { 'id' => user.gsub(/^@/, '') }
+  end
+
+  def ask_for_password
+    print "Enter password for @#{config['id']}: "
+    @config['pass'] = STDIN.noecho(&:gets).chomp
+    puts
+  end
+
+  def save_config
+    # ignore
+  end
+end
+
+host, handle = ARGV
+
+unless host && handle
+  puts "Usage: #{$PROGRAM_NAME} <pds_hostname> <handle>"
+  exit 1
+end
+
+# create a client instance & read password
+bsky = TransientClient.new(host, handle)
+bsky.ask_for_password
+
+# fetch 10 posts from the user's home feed
+result = bsky.get_request('app.bsky.feed.getTimeline', { limit: 10 })
+
+result['feed'].each do |r|
+  reason = r['reason']
+  reply = r['reply']
+  post = r['post']
+
+  if reason && reason['$type'] == 'app.bsky.feed.defs#reasonRepost'
+    puts "[Reposted by @#{reason['by']['handle']}]"
+  end
+
+  handle = post['author']['handle']
+  timestamp = Time.parse(post['record']['createdAt']).getlocal
+
+  puts "@#{handle} • #{timestamp}"
+  puts
+
+  if reply
+    puts "[in reply to @#{reply['parent']['author']['handle']}]"
+    puts
+  end
+
+  puts post['record']['text']
+  puts
+  puts "=" * 120
+  puts
+end

data/example/find_missing_follows.rb

@@ -0,0 +1,67 @@
+#!/usr/bin/env ruby
+
+# Example: fetch the list of accounts followed by a given user and check which of them have been deleted / deactivated.
+
+# load minisky from a local folder - you normally won't need this
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+
+require 'didkit'
+require 'minisky'
+
+handle = ARGV[0].to_s.gsub(/^@/, '')
+if handle.empty?
+  puts "Usage: #{$PROGRAM_NAME} <handle>"
+  exit 1
+end
+
+pds_host = DID.resolve_handle(handle).get_document.pds_endpoint
+pds = Minisky.new(pds_host, nil, progress: '.')
+
+print "Fetching all follows of @#{handle} from #{pds_host}: "
+
+follows = pds.fetch_all('com.atproto.repo.listRecords',
+  { repo: handle, collection: 'app.bsky.graph.follow', limit: 100 }, field: 'records')
+
+puts
+puts "Found #{follows.length} follows"
+
+appview = Minisky.new('api.bsky.app', nil)
+
+profiles = []
+i = 0
+
+puts
+print "Fetching profiles of all followed accounts: "
+
+# getProfiles lets us load multiple profiles in one request, but only up to 25 in one batch
+
+while i < follows.length
+  batch = follows[i...i+25]
+  dids = batch.map { |x| x['value']['subject'] }
+  print '.'
+  result = appview.get_request('app.bsky.actor.getProfiles', { actors: dids })
+  profiles += result['profiles']
+  i += 25
+end
+
+# these are DIDs that are on the follows list, but aren't being returned from getProfiles
+missing = follows.map { |x| x['value']['subject'] } - profiles.map { |x| x['did'] }
+
+puts
+puts "#{missing.length} followed accounts are missing:"
+puts
+
+missing.each do |did|
+  begin
+    doc = DID.new(did).get_document
+  rescue OpenURI::HTTPError
+    puts "#{did} (?) => DID not found"
+    next
+  end
+
+  # check account status on their assigned PDS
+  pds = Minisky.new(doc.pds_endpoint.gsub('http://', ''), nil)
+  status = pds.get_request('com.atproto.sync.getRepoStatus', { did: did }).slice('status', 'active') rescue 'deleted'
+
+  puts "#{did} (@#{doc.handles.first}) => #{status}"
+end

data/lib/minisky/errors.rb

@@ -52,4 +52,13 @@ class Minisky
       @location = location
     end
   end
+
+  class FieldNotSetError < Error
+    attr_reader :fields
+
+    def initialize(fields)
+      @fields = fields
+      super("Field parameter not provided; available fields: #{@fields.inspect}")
+    end
+  end
 end

data/lib/minisky/minisky.rb

@@ -3,7 +3,7 @@ require 'yaml'
 class Minisky
   attr_reader :host, :config
 
-  def initialize(host, config_file)
+  def initialize(host, config_file, options = {})
     @host = host
     @config_file = config_file
 
@@ -18,6 +18,22 @@ class Minisky
       @send_auth_headers = false
       @auto_manage_tokens = false
     end
+
+    if active_repl?
+      @default_progress = '.'
+    end
+
+    if options
+      options.each do |k, v|
+        self.send("#{k}=", v)
+      end
+    end
+  end
+
+  def active_repl?
+    return true if defined?(IRB) && IRB.respond_to?(:CurrentContext) && IRB.CurrentContext
+    return true if defined?(Pry) && Pry.respond_to?(:cli) && Pry.cli
+    false
   end
 
   def save_config

data/lib/minisky/requests.rb

@@ -18,7 +18,7 @@ class Minisky
     end
 
     def method_missing(name, *args)
-      if name.end_with?('=')
+      if name.to_s.end_with?('=')
         @config[name.to_s.chop] = args[0]
       else
         @config[name.to_s]
@@ -26,6 +26,8 @@ class Minisky
     end
   end
 
+  NSID_REGEXP = /^[a-zA-Z]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)+(\.[a-zA-Z]([a-zA-Z]{0,61}[a-zA-Z])?)$/
+
   module Requests
     attr_accessor :default_progress
     attr_writer :send_auth_headers
@@ -39,19 +41,26 @@ class Minisky
       instance_variable_defined?('@auto_manage_tokens') ? @auto_manage_tokens : true
     end
 
+    alias progress default_progress
+    alias progress= default_progress=
+
     def base_url
-      @base_url ||= "https://#{host}/xrpc"
+      if host.include?('://')
+        host.chomp('/') + '/xrpc'
+      else
+        "https://#{host}/xrpc"
+      end
     end
 
     def user
       @user ||= User.new(config)
     end
 
-    def get_request(method, params = nil, auth: default_auth_mode)
+    def get_request(method, params = nil, auth: default_auth_mode, headers: nil)
       check_access if auto_manage_tokens && auth == true
 
-      headers = authentication_header(auth)
-      url = URI("#{base_url}/#{method}")
+      headers = authentication_header(auth).merge(headers || {})
+      url = build_request_uri(method)
 
       if params && !params.empty?
         url.query = URI.encode_www_form(params)
@@ -63,18 +72,30 @@ class Minisky
       handle_response(response)
     end
 
-    def post_request(method, params = nil, auth: default_auth_mode)
+    def post_request(method, data = nil, auth: default_auth_mode, headers: nil, params: nil)
       check_access if auto_manage_tokens && auth == true
 
-      headers = authentication_header(auth).merge({ "Content-Type" => "application/json" })
-      body = params ? params.to_json : ''
+      headers = authentication_header(auth).merge(headers || {})
+      headers["Content-Type"] = "application/json" unless headers.keys.any? { |k| k.to_s.downcase == 'content-type' }
+
+      body = if data.is_a?(String) || data.nil?
+        data.to_s
+      else
+        data.to_json
+      end
+
+      url = build_request_uri(method)
 
-      response = Net::HTTP.post(URI("#{base_url}/#{method}"), body, headers)
+      if params && !params.empty?
+        url.query = URI.encode_www_form(params)
+      end
+
+      response = Net::HTTP.post(url, body, headers)
       handle_response(response)
     end
 
-    def fetch_all(method, params = nil, field:,
-                  auth: default_auth_mode, break_when: nil, max_pages: nil, progress: @default_progress)
+    def fetch_all(method, params = nil, auth: default_auth_mode,
+                  field: nil, break_when: nil, max_pages: nil, headers: nil, progress: @default_progress)
       data = []
       params = {} if params.nil?
       pages = 0
@@ -82,7 +103,12 @@ class Minisky
       loop do
         print(progress) if progress
 
-        response = get_request(method, params, auth: auth)
+        response = get_request(method, params, auth: auth, headers: headers)
+
+        if field.nil?
+          raise FieldNotSetError, response.keys.select { |f| response[f].is_a?(Array) }
+        end
+
         records = response[field]
         cursor = response['cursor']
 
@@ -101,8 +127,12 @@ class Minisky
     def check_access
       if !user.logged_in?
         log_in
-      elsif token_expiration_date(user.access_token) < Time.now + 60
+        :logged_in
+      elsif access_token_expired?
         perform_token_refresh
+        :refreshed
+      else
+        :ok
       end
     end
 
@@ -140,6 +170,26 @@ class Minisky
       json
     end
 
+    def token_expiration_date(token)
+      parts = token.split('.')
+      raise AuthError, "Invalid access token format" unless parts.length == 3
+
+      begin
+        payload = JSON.parse(Base64.decode64(parts[1]))
+      rescue JSON::ParserError
+        raise AuthError, "Couldn't decode payload from access token"
+      end
+
+      exp = payload['exp']
+      raise AuthError, "Invalid token expiry data" unless exp.is_a?(Numeric) && exp > 0
+
+      Time.at(exp)
+    end
+
+    def access_token_expired?
+      token_expiration_date(user.access_token) < Time.now + 60
+    end
+
     def reset_tokens
       user.access_token = nil
       user.refresh_token = nil
@@ -152,14 +202,14 @@ class Minisky
       alias_method :do_post_request, :post_request
       private :do_get_request, :do_post_request
 
-      def get_request(method, params = nil, auth: default_auth_mode, **kwargs)
+      def get_request(method, params = nil, auth: default_auth_mode, headers: nil, **kwargs)
         params ||= kwargs unless kwargs.empty?
-        do_get_request(method, params, auth: auth)
+        do_get_request(method, params, auth: auth, headers: headers)
       end
 
-      def post_request(method, params = nil, auth: default_auth_mode, **kwargs)
+      def post_request(method, params = nil, auth: default_auth_mode, headers: nil, **kwargs)
         params ||= kwargs unless kwargs.empty?
-        do_post_request(method, params, auth: auth)
+        do_post_request(method, params, auth: auth, headers: headers)
       end
     end
 
@@ -173,6 +223,18 @@ class Minisky
       end
     end
 
+    def build_request_uri(method)
+      if method.is_a?(URI)
+        method
+      elsif method.include?('://')
+        URI(method)
+      elsif method =~ NSID_REGEXP
+        URI("#{base_url}/#{method}")
+      else
+        raise ArgumentError, "Invalid method name #{method.inspect} (should be an NSID, URL or an URI object)"
+      end
+    end
+
     def default_auth_mode
       !!send_auth_headers
     end
@@ -191,35 +253,18 @@ class Minisky
       end
     end
 
-    def token_expiration_date(token)
-      parts = token.split('.')
-      raise AuthError, "Invalid access token format" unless parts.length == 3
-
-      begin
-        payload = JSON.parse(Base64.decode64(parts[1]))
-      rescue JSON::ParserError
-        raise AuthError, "Couldn't decode payload from access token"
-      end
-
-      exp = payload['exp']
-      raise AuthError, "Invalid token expiry data" unless exp.is_a?(Numeric) && exp > 0
-
-      Time.at(exp)
-    end
-
     def handle_response(response)
       status = response.code.to_i
       message = response.message
+      response_body = (response.content_type == 'application/json') ? JSON.parse(response.body) : response.body
 
       case response
       when Net::HTTPSuccess
-        JSON.parse(response.body)
+        response_body
       when Net::HTTPRedirection
         raise UnexpectedRedirect.new(status, message, response['location'])
       else
-        data = (response.content_type == 'application/json') ? JSON.parse(response.body) : response.body
-
-        error_class = if data.is_a?(Hash) && data['error'] == 'ExpiredToken'
+        error_class = if response_body.is_a?(Hash) && response_body['error'] == 'ExpiredToken'
           ExpiredTokenError
         elsif response.is_a?(Net::HTTPClientError)
           ClientErrorResponse
@@ -229,7 +274,7 @@ class Minisky
           BadResponse
         end
 
-        raise error_class.new(status, message, data)
+        raise error_class.new(status, message, response_body)
       end
     end
   end

data/lib/minisky/version.rb

@@ -1,5 +1,5 @@
 require_relative 'minisky'
 
 class Minisky
-  VERSION = "0.3.1"
+  VERSION = "0.5.0"
 end

metadata

@@ -1,15 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: minisky
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.5.0
 platform: ruby
 authors:
 - Kuba Suder
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-10-10 00:00:00.000000000 Z
-dependencies: []
+date: 2024-12-27 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: base64
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
 description: A very simple client class that lets you log in to the Bluesky API and
   make any requests there.
 email:
@@ -21,8 +35,10 @@ files:
 - CHANGELOG.md
 - LICENSE.txt
 - README.md
+- example/ask_password.rb
 - example/fetch_my_posts.rb
 - example/fetch_profile.rb
+- example/find_missing_follows.rb
 - example/post_skeet.rb
 - example/science_feed.rb
 - lib/minisky.rb