didkit 0.0.4 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76dd4f3787cf1cc288015511eb35ea6d41ef4c390f85ca67854617fe7c46d710
-  data.tar.gz: 40ea426b53fa3176dccd4c538cbd40050a3883bb6632af2b45ca1a6f49ac84b7
+  metadata.gz: 0170f70f452400a399264cccd61a96a0ef96d570caca53547b2794eb8067d4bd
+  data.tar.gz: 68197fe06b766fbe69d09d6d85b8abf59467f09560cb600b9af34bb900efaf6b
 SHA512:
-  metadata.gz: a26c81e89e2397bb7690149e143a0de6c9e9635a993e8cd13d572d76b21b38ce37628eb3c3ebc4ea9ffc20b3807f619acb46612a76af9f816b336315a10f7aab
-  data.tar.gz: 89b4d9eb1ec2b7f5207587cde9b0a2b2db8a5912a92dc1ef996f0bac493996aae8b1340fc673838c404ac28d4ab191be6e7ce3a984f35a83402b9ef489feea8b
+  metadata.gz: 367f5acd94953397d5ca72edec0a43f2a96e8dfef3615d16210edc203abe14b1875c1e78316efe351e223c881cbc80d2fd6cf7314753c78caea6c9ccb7ed78d0
+  data.tar.gz: 15ade27998fbc1881f28bc07fe7d625f310e686be7f101508460db48df7ca263adfc6731bf9f8bb935363fb23e51f33b1f560183595ba624d1fd7ad4e8dcf00f

data/CHANGELOG.md

@@ -1,3 +1,18 @@
+## [0.2.0] - 2024-03-19
+
+- added `PLCImporter` class, which lets you import operations from PLC in pages of 1000 through the "export" API
+- implemented parsing of all services from DID doc & operations, not only `atproto_pds` (specifically labeller endpoints)
+- allow setting the nameserver in `Resolver` initializer
+
+## [0.1.0] - 2024-03-12
+
+- rejecting handles from disallowed domains like `.arpa` or `.test`
+- validating handles with the `.well-known` file having a trailing newline
+- validating handles with `.well-known` address returning a redirect
+- added `#pick_valid_handle` helper
+- allow overriding the nameserver for `Resolv::DNS`
+- other bug fixes
+
 ## [0.0.4] - 2024-03-07
 
 - extracted resolving code from `DID` to a new `Resolver` class (`DID` has helper methods to call the resolver)

data/README.md

@@ -1,13 +1,11 @@
-# DidKit
+# DIDKit
 
 A small Ruby gem for handling Distributed Identifiers (DIDs) in Bluesky / AT Protocol
 
 
 ## What does it do
 
-**TODO** - not much yet :)
-
-See the [did.rb](https://github.com/mackuba/didkit/blob/master/lib/didkit/did.rb) file for now.
+Accounts on Bluesky use identifiers like [did:plc:oio4hkxaop4ao4wz2pp3f4cr](https://plc.directory/did:plc:oio4hkxaop4ao4wz2pp3f4cr) as unique IDs, and they also have assigned human-readable handles like [@mackuba.eu](https://bsky.app/profile/mackuba.eu), which are verified either through a DNS TXT entry or a `/.well-known/atproto-did` file. This library allows you to look up any account's assigned handle using a DID string or vice versa, load the account's DID JSON document that specifies the handles and the PDS server hosting user's repo, and check if the assigned handle verifies correctly.
 
 
 ## Installation
@@ -15,8 +13,72 @@ See the [did.rb](https://github.com/mackuba/didkit/blob/master/lib/didkit/did.rb
     gem install didkit
 
 
+## Usage
+
+Use the `DIDKit::Resolver` class to look up DIDs and handles.
+
+To look up a handle:
+
+```rb
+resolver = DIDKit::Resolver.new
+resolver.resolve_handle('nytimes.com')
+ # => #<DIDKit::DID:0x00000001035956b0 @did="did:plc:eclio37ymobqex2ncko63h4r", @type=:plc, @resolved_by=:dns>
+```
+
+This returns an object of `DIDKit::DID` class (aliased as just `DID`), which tells you:
+
+- the DID as a string (`#to_s` or `#did`)
+- the DID type (`#type`, `:plc` or `:web`)
+- if the handle was resolved via a DNS entry or a `.well-known` file (`#resolved_by`, `:dns` or `:http`)
+
+To go in the other direction – to find an assigned and verified handle given a DID – use `get_validated_handle` (pass DID as a string or an object):
+
+```rb
+resolver.get_validated_handle('did:plc:ewvi7nxzyoun6zhxrhs64oiz')
+ # => "atproto.com" 
+```
+
+You can also load the DID document using `resolve_did`:
+
+```rb
+doc = resolver.resolve_did('did:plc:ragtjsm2j2vknwkz3zp4oxrd')
+ # => #<DIDKit::Document:0x0000000105d751f8 @did=#<DIDKit::DID:...>, @json={...}>
+
+doc.handles
+ # => ["pfrazee.com"] 
+
+doc.pds_endpoint
+ # => "https://morel.us-east.host.bsky.network" 
+```
+
+There are also some helper methods in the `DID` class that create a `Resolver` for you to save you some typing:
+
+```rb
+did = DID.resolve_handle('jay.bsky.team')
+ #  => #<DIDKit::DID:0x000000010615ed28 @did="did:plc:oky5czdrnfjpqslsw2a5iclo", @type=:plc, @resolved_by=:dns>
+
+did.to_s
+ # => "did:plc:oky5czdrnfjpqslsw2a5iclo" 
+
+did.get_document
+ # => #<DIDKit::Document:0x00000001066d4898 @did=#<DIDKit::DID:...>, @json={...}>
+
+did.get_validated_handle
+ # => "jay.bsky.team" 
+```
+
+
+### Configuration
+
+You can override the nameserver used for DNS lookups by setting the `nameserver` property in `Resolver`, e.g. to use Google's or CloudFlare's global DNS:
+
+```
+resolver.nameserver = '8.8.8.8'
+```
+
+
 ## Credits
 
-Copyright © 2023 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
+Copyright © 2024 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)).
 
 The code is available under the terms of the [zlib license](https://choosealicense.com/licenses/zlib/) (permissive, similar to MIT).

data/lib/didkit/at_handles.rb

@@ -0,0 +1,14 @@
+module DIDKit
+  module AtHandles
+    class FormatError < StandardError
+    end
+
+    def parse_also_known_as(aka)
+      raise FormatError, "Invalid alsoKnownAs: #{aka.inspect}" unless aka.is_a?(Array)
+      raise FormatError, "Invalid alsoKnownAs: #{aka.inspect}" unless aka.all? { |x| x.is_a?(String) }
+      raise FormatError, "Invalid alsoKnownAs: #{aka.inspect}" unless aka.all? { |x| x =~ %r(\Aat://[^/]+\z) }
+
+      aka.map { |x| x.gsub('at://', '') }
+    end
+  end
+end

data/lib/didkit/document.rb

@@ -1,9 +1,17 @@
+require_relative 'at_handles'
+require_relative 'resolver'
+require_relative 'service_record'
+require_relative 'services'
+
 module DIDKit
   class Document
     class FormatError < StandardError
     end
 
-    attr_reader :json, :did, :pds_endpoint, :handles
+    include AtHandles
+    include Services
+
+    attr_reader :json, :did, :handles, :services
 
     def initialize(did, json)
       raise FormatError, "Missing id field" if json['id'].nil?
@@ -17,29 +25,24 @@ module DIDKit
       raise FormatError, "Missing service key" if service.nil?
       raise FormatError, "Invalid service data" unless service.is_a?(Array) && service.all? { |x| x.is_a?(Hash) }
 
-      if pds = service.detect { |x| x['id'] == '#atproto_pds' }
-        raise FormatError, "Missing PDS type" unless pds['type']
-        raise FormatError, "Invalid PDS type" unless pds['type'] == 'AtprotoPersonalDataServer'
-        raise FormatError, "Missing PDS endpoint" unless pds['serviceEndpoint']
-        raise FormatError, "Invalid PDS endpoint" unless pds['serviceEndpoint'].is_a?(String)
-        raise FormatError, "Invalid PDS endpoint" unless pds['serviceEndpoint'] =~ %r(://)
-
-        @pds_endpoint = pds['serviceEndpoint']
-      end
-
-      if aka = json['alsoKnownAs']
-        raise FormatError, "Invalid alsoKnownAs" unless aka.is_a?(Array)
-        raise FormatError, "Invalid alsoKnownAs" unless aka.all? { |x| x.is_a?(String) }
-        raise FormatError, "Invalid alsoKnownAs" unless aka.all? { |x| x =~ %r(\Aat://[^/]+\z) }
-
-        @handles = aka.map { |x| x.gsub('at://', '') }
-      else
-        @handles = []
-      end
+      @services = service.map { |x|
+        id, type, endpoint = x.values_at('id', 'type', 'serviceEndpoint')
+
+        raise FormatError, "Missing service id" unless id
+        raise FormatError, "Invalid service id: #{id.inspect}" unless id.is_a?(String) && id.start_with?('#')
+        raise FormatError, "Missing service type" unless type
+        raise FormatError, "Invalid service type: #{type.inspect}" unless type.is_a?(String)
+        raise FormatError, "Missing service endpoint" unless endpoint
+        raise FormatError, "Invalid service endpoint: #{endpoint.inspect}" unless endpoint.is_a?(String)
+
+        ServiceRecord.new(id.gsub(/^#/, ''), type, endpoint)
+      }
+
+      @handles = parse_also_known_as(json['alsoKnownAs'] || [])
     end
 
     def get_validated_handle
-      Resolver.new.get_validated_handle(self)
+      Resolver.new.pick_valid_handle(did, handles)
     end
   end
 end

data/lib/didkit/plc_importer.rb

@@ -0,0 +1,73 @@
+require 'json'
+require 'open-uri'
+require 'time'
+
+require_relative 'plc_operation'
+
+module DIDKit
+  class PLCImporter
+    PLC_SERVICE = 'plc.directory'
+    MAX_PAGE = 1000
+
+    attr_accessor :ignore_errors, :last_date
+
+    def initialize(since: nil)
+      if since.to_s == 'beginning'
+        @last_date = nil
+      elsif since.is_a?(String)
+        @last_date = Time.parse(since)
+      elsif since
+        @last_date = since
+      else
+        @last_date = Time.now
+        @eof = true
+      end
+
+      @ignore_errors = false
+    end
+
+    def plc_service
+      PLC_SERVICE
+    end
+
+    def get_export(args = {})
+      url = URI("https://#{plc_service}/export")
+      url.query = URI.encode_www_form(args)
+
+      data = URI.open(url).read
+      data.lines.map(&:strip).reject(&:empty?).map { |x| JSON.parse(x) }
+    end
+
+    def fetch_page
+      request_time = Time.now
+
+      query = @last_date ? { :after => @last_date.utc.iso8601(6) } : {}
+      rows = get_export(query)
+
+      operations = rows.filter_map do |json|
+        begin
+          PLCOperation.new(json)
+        rescue PLCOperation::FormatError => e
+          ignore_errors ? nil : raise
+        end
+      end
+
+      @last_date = operations.last&.created_at || request_time
+      @eof = (rows.length < MAX_PAGE)
+
+      operations
+    end
+
+    def fetch(&block)
+      loop do
+        operations = fetch_page
+        block.call(operations)
+        break if eof?
+      end
+    end
+
+    def eof?
+      !!@eof
+    end
+  end
+end

data/lib/didkit/plc_operation.rb

@@ -0,0 +1,57 @@
+require 'time'
+
+require_relative 'at_handles'
+require_relative 'service_record'
+require_relative 'services'
+
+module DIDKit
+  class PLCOperation
+    class FormatError < StandardError
+    end
+
+    include AtHandles
+    include Services
+
+    attr_reader :json, :did, :created_at, :type, :handles, :services
+
+    def initialize(json)
+      @json = json
+      @did = json['did']
+      raise FormatError, "Missing DID: #{json}" if @did.nil?
+      raise FormatError, "Invalid DID: #{@did}" unless @did.is_a?(String) && @did.start_with?('did:')
+
+      timestamp = json['createdAt']
+      raise FormatError, "Missing createdAt: #{json}" if timestamp.nil?
+      raise FormatError, "Invalid createdAt: #{timestamp.inspect}" unless timestamp.is_a?(String)
+
+      @created_at = Time.parse(timestamp)
+
+      operation = json['operation']
+      raise FormatError, "Missing operation key: #{json}" if operation.nil?
+      raise FormatError, "Invalid operation data: #{operation.inspect}" unless operation.is_a?(Hash)
+
+      type = operation['type']
+      raise FormatError, "Missing operation type: #{json}" if type.nil?
+
+      @type = type.to_sym
+      return unless @type == :plc_operation
+
+      services = operation['services']
+      raise FormatError, "Missing services key: #{json}" if services.nil?
+      raise FormatError, "Invalid services data: #{services}" unless services.is_a?(Hash)
+
+      @services = services.map { |k, x|
+        type, endpoint = x.values_at('type', 'endpoint')
+
+        raise FormatError, "Missing service type" unless type
+        raise FormatError, "Invalid service type: #{type.inspect}" unless type.is_a?(String)
+        raise FormatError, "Missing service endpoint" unless endpoint
+        raise FormatError, "Invalid service endpoint: #{endpoint.inspect}" unless endpoint.is_a?(String)
+
+        ServiceRecord.new(k, type, endpoint)
+      }
+
+      @handles = parse_also_known_as(operation['alsoKnownAs'] || [])
+    end
+  end
+end

data/lib/didkit/resolver.rb

@@ -8,9 +8,20 @@ require_relative 'document'
 
 module DIDKit
   class Resolver
+    RESERVED_DOMAINS = %w(alt arpa example internal invalid local localhost onion test)
+    MAX_REDIRECTS = 5
+
+    attr_accessor :nameserver
+
+    def initialize(options = {})
+      @nameserver = options[:nameserver]
+    end
+
     def resolve_handle(handle)
       domain = handle.gsub(/^@/, '')
 
+      return nil if RESERVED_DOMAINS.include?(domain.split('.').last)
+
       if dns_did = resolve_handle_by_dns(domain)
         DID.new(dns_did, :dns)
       elsif http_did = resolve_handle_by_well_known(domain)
@@ -21,13 +32,13 @@ module DIDKit
     end
 
     def resolve_handle_by_dns(domain)
-      dns_records = Resolv::DNS.open { |d| d.getresources("_atproto.#{domain}", Resolv::DNS::Resource::IN::TXT) }
+      dns_records = Resolv::DNS.open(resolv_options) { |d|
+        d.getresources("_atproto.#{domain}", Resolv::DNS::Resource::IN::TXT)
+      }
 
       if record = dns_records.first
         if string = record.strings.first
-          if string =~ /^did\=(did\:\w+\:.*)$/
-            return $1
-          end
+          return parse_did_from_dns(string)
         end
       end
 
@@ -35,7 +46,11 @@ module DIDKit
     end
 
     def resolve_handle_by_well_known(domain)
-      url = URI("https://#{domain}/.well-known/atproto-did")
+      resolve_handle_from_url("https://#{domain}/.well-known/atproto-did")
+    end
+
+    def resolve_handle_from_url(url, redirects = 0)
+      url = URI(url) unless url.is_a?(URI)
 
       response = Net::HTTP.start(url.host, url.port, use_ssl: true, open_timeout: 10, read_timeout: 10) do |http|
         request = Net::HTTP::Get.new(url)
@@ -44,9 +59,12 @@ module DIDKit
 
       if response.is_a?(Net::HTTPSuccess)
         if text = response.body
-          if text.lines.length == 1 && text.start_with?('did:')
-            return text
-          end
+          return parse_did_from_well_known(text)
+        end
+      elsif response.is_a?(Net::HTTPRedirection) && redirects < MAX_REDIRECTS
+        if location = response['Location']
+          target_url = location.include?('://') ? location : (url.origin + location)
+          return resolve_handle_from_url(target_url, redirects + 1)
         end
       end
 
@@ -55,6 +73,21 @@ module DIDKit
       nil
     end
 
+    def resolv_options
+      options = Resolv::DNS::Config.default_config_hash.dup
+      options[:nameserver] = nameserver if nameserver
+      options
+    end
+
+    def parse_did_from_dns(txt)
+      txt =~ /\Adid\=(did\:\w+\:.*)\z/ ? $1 : nil
+    end
+
+    def parse_did_from_well_known(text)
+      text = text.strip
+      text.lines.length == 1 && text =~ /\Adid\:\w+\:.*\z/ ? text : nil
+    end
+
     def resolve_did(did)
       did = DID.new(did) if did.is_a?(String)
 
@@ -76,7 +109,11 @@ module DIDKit
     def get_validated_handle(did_or_doc)
       document = did_or_doc.is_a?(Document) ? did_or_doc : resolve_did(did_or_doc)
 
-      document.handles.detect { |h| resolve_handle(h) == document.did }
+      pick_valid_handle(document.did, document.handles)
+    end
+
+    def pick_valid_handle(did, handles)
+      handles.detect { |h| resolve_handle(h) == did }
     end
   end
 end

data/lib/didkit/service_record.rb

@@ -0,0 +1,20 @@
+require 'uri'
+require_relative 'errors'
+
+module DIDKit
+  class ServiceRecord
+    attr_reader :key, :type, :endpoint
+
+    def initialize(key, type, endpoint)
+      begin
+        uri = URI(endpoint)
+      rescue URI::Error
+        raise FormatError, "Invalid service endpoint: #{endpoint.inspect}"
+      end
+
+      @key = key
+      @type = type
+      @endpoint = endpoint
+    end
+  end
+end

data/lib/didkit/services.rb

@@ -0,0 +1,15 @@
+module DIDKit
+  module Services
+    def get_service(key, type)
+      @services&.detect { |s| s.key == key && s.type == type }
+    end
+
+    def pds_endpoint
+      @pds_endpoint ||= get_service('atproto_pds', 'AtprotoPersonalDataServer')&.endpoint
+    end
+
+    def labeler_endpoint
+      @labeler_endpoint ||= get_service('atproto_labeler', 'AtprotoLabeler')&.endpoint
+    end
+  end
+end

data/lib/didkit/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DIDKit
-  VERSION = "0.0.4"
+  VERSION = "0.2.0"
 end

data/lib/didkit.rb

@@ -2,6 +2,9 @@
 
 require_relative "didkit/did"
 require_relative "didkit/document"
+require_relative "didkit/plc_importer"
+require_relative "didkit/plc_operation"
+require_relative "didkit/resolver"
 require_relative "didkit/version"
 
 module DIDKit

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: didkit
 version: !ruby/object:Gem::Version
-  version: 0.0.4
+  version: 0.2.0
 platform: ruby
 authors:
 - Kuba Suder
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-03-07 00:00:00.000000000 Z
+date: 2024-03-19 00:00:00.000000000 Z
 dependencies: []
 description:
 email:
@@ -21,10 +21,15 @@ files:
 - LICENSE.txt
 - README.md
 - lib/didkit.rb
+- lib/didkit/at_handles.rb
 - lib/didkit/did.rb
 - lib/didkit/document.rb
 - lib/didkit/errors.rb
+- lib/didkit/plc_importer.rb
+- lib/didkit/plc_operation.rb
 - lib/didkit/resolver.rb
+- lib/didkit/service_record.rb
+- lib/didkit/services.rb
 - lib/didkit/version.rb
 - sig/didkit.rbs
 homepage: https://github.com/mackuba/didkit