webloc 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 498d11e48961e5843a3a25fafb71f4738ebc10733037145ea30a17f176abcaa2
-  data.tar.gz: 78beeadfdafd1a7f414fc6f7b69c6fd24819ed79bb39a9148e72e1bc38089416
+  metadata.gz: 246849e44bba8ffbcd81036265a4cc813aa963dccf5cb092cb8551268f83b276
+  data.tar.gz: 833a7e93bc666b75ce56cfe58619229884eea236fda992a2a660647315eb093c
 SHA512:
-  metadata.gz: 99b273c4af784bdafe2cde8166da2c3672a263d2e412f0f1547e70ef4b8bf8de74185a13b1050e5c16b713e3f7b8c67baabe9f25b1366ee4e86876a95a368740
-  data.tar.gz: fdbd68bb1239b9b35b1eb8c3d3ef8302d69d8a48eca6830bf636eb693f6e8c9a35e19af02148fda21a2d5a2ce3be1e3b7bef8186d7917dd8c40f73a699557e60
+  metadata.gz: 37d9c62d25c198f4e11b56f826c229def950817d1d28733779093ed97e9e0c389db4985abebbc4712b49171f7df9e94c3748a399c0783c5b60249f135356a649
+  data.tar.gz: 949d852b749d043290c2462b33aeebe18737d9845a06d588d621d4f2260c3e54dcf02da9aa21502cdacf32ce2fc341fdbaf7cc673862e308056cba9b50228752

data/CHANGELOG.md

@@ -0,0 +1,15 @@
+# Changelog
+
+## 0.4.0
+
+- Rewrite binary plist reading to follow object references and offsets, supporting
+  short URLs, additional metadata, and different object orders.
+- Fix binary output for long URLs and support lengths beyond 65,535 characters.
+- Add Unicode URL support, writing UTF-16 strings and reading them as UTF-8.
+- Recover historical long-URL files affected by earlier versions' offset bugs.
+- Reject malformed files with a `WeblocError` subclass instead of returning
+  partial URLs. Public methods and error classes remain unchanged.
+- Require Ruby 3.0 or newer.
+- Retain XML support, the original tests and fixtures, and the existing binary
+  encoding for short ASCII URLs. Add README example tests, independent plist
+  interoperability checks, and CI coverage.

data/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2011-2024 Peter Cooper
+Copyright (c) 2011-2026 Peter Cooper
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -2,7 +2,7 @@
 
 *webloc* is a Ruby library that can read from and write to <tt>.webloc</tt> files as used on macOS. These are a variant of 'plist' format files, specifically used for storing links to URLs.
 
-It works on Ruby 2.7 and up, including Ruby 3.x, and supports URLs of up to 2048 characters in length (and probably longer, but this is around the de facto limit for URLs in most systems).
+It works on Ruby 3.0 and up (including Ruby 4.0.2 and TruffleRuby). It reads XML and binary plist files and writes binary plist files, supporting Unicode and URLs longer than 65,535 characters.
 
 ## Installation
 
@@ -10,14 +10,108 @@ It works on Ruby 2.7 and up, including Ruby 3.x, and supports URLs of up to 2048
     
 ## Usage
 
+### Basic Usage
+
 Reading a .webloc file:
 
-    Webloc.load(ARGV.first).url
+    require 'webloc'
+    webloc = Webloc.load('bookmark.webloc')
+    puts webloc.url
+    # => "https://example.com"
 
 Writing to a .webloc file:
 
+    require 'webloc'
     Webloc.new('https://rubyweekly.com/').save('rubyweekly.webloc')
 
+### Advanced Examples
+
+#### Error handling
+
+```ruby
+require 'webloc'
+
+begin
+  webloc = Webloc.load('suspicious.webloc')
+  puts webloc.url
+rescue Webloc::FileNotFoundError => e
+  puts "File not found: #{e.message}"
+rescue Webloc::CorruptedFileError => e
+  puts "File is corrupted: #{e.message}"
+rescue Webloc::InvalidFormatError => e
+  puts "Invalid file format: #{e.message}"
+rescue Webloc::WeblocError => e
+  puts "General webloc error: #{e.message}"
+end
+```
+
+#### Validating URLs before creating webloc files
+
+```ruby
+require 'webloc'
+require 'uri'
+
+def create_webloc_safely(url, filename)
+  # Basic URL validation
+  uri = URI.parse(url)
+  unless uri.is_a?(URI::HTTP) || uri.is_a?(URI::HTTPS)
+    puts "Invalid URL scheme: #{url}"
+    return false
+  end
+  
+  # Create the webloc file
+  Webloc.new(url).save(filename)
+  puts "Created #{filename} for #{url}"
+  true
+rescue URI::InvalidURIError
+  puts "Invalid URL format: #{url}"
+  false
+rescue Webloc::WeblocError => e
+  puts "Failed to create webloc: #{e.message}"
+  false
+end
+
+create_webloc_safely('https://example.com', 'example.webloc')
+create_webloc_safely('invalid-url', 'invalid.webloc')
+```
+
+#### Converting between formats
+
+```ruby
+require 'webloc'
+require 'json'
+
+# Convert webloc to JSON
+webloc = Webloc.load('bookmark.webloc')
+json_data = { url: webloc.url, title: File.basename('bookmark.webloc', '.webloc') }
+File.write('bookmark.json', JSON.pretty_generate(json_data))
+
+# Convert JSON back to webloc
+json_content = JSON.parse(File.read('bookmark.json'))
+Webloc.new(json_content['url']).save('restored.webloc')
+```
+
+## Compatibility
+
+The public methods and error classes are unchanged. The reader also recovers
+historical long-URL files with the known offset bugs. New writes correct those
+offsets; short ASCII URLs retain their original binary encoding.
+Binary reading supports short URLs, additional metadata, and different object orders.
+URLs are stored as supplied, without escaping or scheme validation. Unicode
+strings use UTF-16 in binary files and are returned as UTF-8 when read.
+
+Malformed files without a recoverable historical layout raise a `WeblocError`
+subclass instead of returning a partial URL.
+
+## Development
+
+Run `bundle install` followed by `bundle exec rake test`.
+
+The suite retains the original tests and fixtures and exercises the Ruby examples
+in this README. Additional fixtures cover other binary layouts and files written
+by older releases. Interoperability tests use Python 3's `plistlib` when available
+and Apple's `plutil` on macOS; neither is a runtime dependency of the gem.
+
 ## Thanks
 
 Thanks is due to Christos Karaiskos for [this article](https://medium.com/@karaiskc/understanding-apples-binary-property-list-format-281e6da00dbd
@@ -25,6 +119,6 @@ Thanks is due to Christos Karaiskos for [this article](https://medium.com/@karai
 
 ## License
 
-Copyright (C) 2011-2024 Peter Cooper
+Copyright (C) 2011-2026 Peter Cooper
 
-webloc is licensed under the terms of the MIT License
+webloc is licensed under the terms of the MIT License

data/lib/webloc/binary_plist.rb

@@ -0,0 +1,159 @@
+class Webloc
+  # Read the root dictionary's URL without recursively decoding unrelated metadata.
+  class BinaryPlist
+    HEADER = 'bplist00'.b.freeze
+    PREFIX = "bplist00\xD1\x01\x02SURL".b.freeze
+    INTEGER_FORMATS = { 1 => 'C', 2 => 'n', 4 => 'N', 8 => 'Q>' }.freeze
+
+    def self.integer_width(value)
+      INTEGER_FORMATS.each_key.find { |width| value < (1 << (8 * width)) } ||
+        raise(ArgumentError, 'URL is too long to encode')
+    end
+
+    def self.dump(url)
+      if url.ascii_only?
+        payload = url.b
+        length = payload.bytesize
+        marker = 0x5f
+      else
+        payload = url.encode(Encoding::UTF_16BE).b
+        length = payload.bytesize / 2 # UTF-16 code units, including surrogate pairs.
+        marker = 0x6f
+      end
+
+      # Keep the historical object order and extended string length encoding,
+      # so existing valid ASCII output (including the original fixture) is stable.
+      width = integer_width(length)
+      output = PREFIX + [marker, 0x10 + width.bit_length - 1].pack('CC')
+      output << [length].pack(INTEGER_FORMATS.fetch(width)) << payload
+      table_offset = output.bytesize
+      offset_width = integer_width(table_offset)
+      output << [8, 11, 15].pack("#{INTEGER_FORMATS.fetch(offset_width)}*")
+      output << [offset_width, 1, 3, 0, table_offset].pack('x6CCQ>Q>Q>')
+    end
+
+    def initialize(data, filename)
+      @data = data
+      @filename = filename
+    end
+
+    def url
+      unless @data.start_with?(HEADER)
+        detail = @data.include?('SURL_') ? 'missing bplist00 header' : 'missing SURL marker'
+        raise InvalidFormatError, "Invalid binary webloc format - #{detail} in file: #{@filename}"
+      end
+
+      # Recognize the complete historical layout, never just a URL marker.
+      historical_url = legacy_url
+      return historical_url if historical_url
+
+      read_table
+      type, count, cursor, limit = object(@root)
+      corrupt('Root object is not a dictionary') unless type == 0xd
+      read(cursor, count * 2 * @reference_width, limit)
+
+      count.times do |index|
+        key_ref = integer(cursor + index * @reference_width, @reference_width, limit)
+        next unless string(key_ref) == 'URL'
+
+        value_ref = integer(cursor + (count + index) * @reference_width, @reference_width, limit)
+        return string(value_ref)
+      end
+      corrupt("No 'URL' key found")
+    end
+
+    private
+
+    def corrupt(message)
+      raise CorruptedFileError, "#{message} in binary webloc file: #{@filename}"
+    end
+
+    def read(offset, length, limit)
+      corrupt('Truncated or invalid object') if offset < 8 || length < 0 || offset > limit || length > limit - offset
+      @data.byteslice(offset, length)
+    end
+
+    def integer(offset, width, limit)
+      read(offset, width, limit).each_byte.reduce(0) { |value, byte| (value << 8) | byte }
+    end
+
+    def read_table
+      corrupt('Truncated plist trailer') if @data.bytesize < 40
+      offset_width, @reference_width, count, @root, table_offset =
+        @data.byteslice(-32, 32).unpack('x6CCQ>Q>Q>')
+      unless (1..8).cover?(offset_width) && (1..8).cover?(@reference_width)
+        corrupt('Invalid offset or reference width')
+      end
+      trailer_offset = @data.bytesize - 32
+      unless count > 0 && @root < count && table_offset >= 8 &&
+             table_offset + count * offset_width == trailer_offset
+        corrupt('Invalid offset table')
+      end
+
+      @offsets = Array.new(count) do |index|
+        offset = integer(table_offset + index * offset_width, offset_width, trailer_offset)
+        corrupt('Object offset is outside the object table') unless offset >= 8 && offset < table_offset
+        offset
+      end
+      # Objects need not be stored in reference order. Bound each read by the
+      # following physical object, so a truncated URL cannot consume the table.
+      boundaries = @offsets.uniq.sort + [table_offset]
+      @limits = boundaries.each_cons(2).to_h
+    end
+
+    def object(reference)
+      corrupt('Invalid object reference') unless reference < @offsets.length
+      offset = @offsets.fetch(reference)
+      limit = @limits.fetch(offset)
+      marker = integer(offset, 1, limit)
+      cursor = offset + 1
+      length = marker & 0xf
+      if length == 0xf
+        size_marker = integer(cursor, 1, limit)
+        unless (0x10..0x13).cover?(size_marker)
+          raise InvalidFormatError, "Unsupported length encoding in binary webloc file: #{@filename}"
+        end
+        width = 1 << (size_marker & 0xf)
+        length = integer(cursor + 1, width, limit)
+        cursor += width + 1
+      end
+      [marker >> 4, length, cursor, limit]
+    end
+
+    def string(reference)
+      type, length, cursor, limit = object(reference)
+      case type
+      when 0x5
+        value = read(cursor, length, limit)
+        corrupt('Invalid ASCII string') unless value.ascii_only?
+        value
+      when 0x6
+        read(cursor, length * 2, limit).force_encoding(Encoding::UTF_16BE).encode(Encoding::UTF_8)
+      else
+        corrupt('Expected a string')
+      end
+    rescue EncodingError
+      corrupt('Invalid Unicode string')
+    end
+
+    def legacy_url
+      return unless @data.start_with?(PREFIX + "\x5f".b)
+      width = { 0x10 => 1, 0x11 => 2 }[@data.getbyte(16)]
+      return unless width && @data.bytesize >= 18 + width + 35
+
+      length = @data.byteslice(17, width).unpack1(INTEGER_FORMATS.fetch(width))
+      return unless length >= 238 && width == (length > 255 ? 2 : 1)
+      table_offset = 17 + width + length
+      return unless @data.bytesize == table_offset + 35
+
+      # Match even the two-byte overflow in the old trailer, but require every
+      # other byte of its structure and the full declared URL to be present.
+      trailer = [1, 1, 3, 0, (length + 18) & 0xffff].pack('x6CCQ>Q>Q>')
+      return unless @data.byteslice(table_offset, 35) == "\x08\x0b\x0f".b + trailer
+      value = @data.byteslice(17 + width, length)
+      value if value.ascii_only?
+    end
+  end
+
+  private_constant :BinaryPlist
+end

data/lib/webloc/version.rb

@@ -1,3 +1,3 @@
 class Webloc
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/lib/webloc.rb

@@ -1,78 +1,83 @@
 require 'plist'
+require_relative 'webloc/binary_plist'
 
 class Webloc
+  class WeblocError < StandardError; end
+  class FileNotFoundError < WeblocError; end
+  class CorruptedFileError < WeblocError; end
+  class InvalidFormatError < WeblocError; end
+  class EmptyFileError < WeblocError; end
+
   attr_accessor :url
 
   def initialize(url)
+    raise ArgumentError, "URL cannot be nil or empty" if url.nil? || url.empty?
     @url = url
   end
 
   def self.load(filename)
-    data = File.read(filename)
-    data = data.force_encoding('binary') rescue data
+    raise FileNotFoundError, "File not found: #{filename}" unless File.exist?(filename)
     
-    if data !~ /\<plist/
-      offset = (data =~ /SURL_/)
-      length_offset = 7
-      if data[offset + 5] == "\x10"
-        length = data[offset + 6]
-        length = length.unpack('C')[0]  
-      elsif data[offset + 5] == "\x11"
-        length_offset = 8
-        length = data[offset + 6] + data[offset + 7]
-        length = length.unpack('S>')[0]  
-      end
-      url = data[offset + length_offset,length]
+    begin
+      data = File.binread(filename)
+    rescue => e
+      raise FileNotFoundError, "Unable to read file '#{filename}': #{e.message}"
+    end
+    
+    raise EmptyFileError, "File is empty: #{filename}" if data.empty?
+    
+    url = nil
+    
+    if data.start_with?('bplist00') || !data.include?('<plist')
+      # Handle binary plist format
+      url = parse_binary_format(data, filename)
     else
-      url = Plist::parse_xml(filename)['URL'] rescue nil
+      # Handle XML plist format
+      url = parse_xml_format(filename)
     end
     
-    raise ArgumentError unless url
+    raise CorruptedFileError, "No URL found in webloc file: #{filename}" if url.nil? || (url.is_a?(String) && url.empty?)
+    raise CorruptedFileError, "URL must be a string in webloc file: #{filename}" unless url.is_a?(String)
     new(url)
   end
 
-  def data
-    # PLIST HEADER
-    @data = "bplist\x30\x30".bytes
+  private
 
-    # PLIST OBJECT TABLE
-    @data += "\xD1\x01\x02".bytes   # object 1 is a dictionary
-    @data += "SURL".bytes           # object 2
+  def self.parse_binary_format(data, filename)
+    BinaryPlist.new(data.b, filename).url
+  end
 
-    length_suffix = @url.length > 255 ? "\x11" : "\x10"
-    @data += ("\x5f" + length_suffix).bytes       # object 3 is an ASCII string with a variable length length encoding (I know..)
-                              #   .. the '0' in \x10 denotes the length can be encoded within 2**0 bytes (i.e. 1)
-                              #   .. the '1' in \x11 denotes the length can be encoded within 2**1 bytes (i.e. 2)
-    
-    if @url.length > 255
-      @data += [@url.length].pack('S>').bytes
-    else
-      @data += [@url.length].pack('C').bytes
+  def self.parse_xml_format(filename)
+    begin
+      plist_data = Plist::parse_xml(filename)
+      raise InvalidFormatError, "Invalid XML plist format - could not parse file: #{filename}" unless plist_data.is_a?(Hash)
+      
+      url = plist_data['URL']
+      raise CorruptedFileError, "No 'URL' key found in plist file: #{filename}" unless url
+      
+      url
+    rescue => e
+      if e.message.include?('parse') || e.message.include?('XML') || e.message.include?('plist')
+        raise InvalidFormatError, "Invalid XML plist format in file '#{filename}': #{e.message}"
+      else
+        raise CorruptedFileError, "Failed to parse XML webloc format in file '#{filename}': #{e.message}"
+      end
     end
-    @data += @url.bytes             # and finally the URL itself
-
-    # This is the offset table
-    @data += "\x08\x0B\x0F".bytes   # so objects at 0x08, 0x0b and 0x0f
+  end
 
-    # PLIST TRAILER
-    # Bytes 0-4 are unused
-    @data += "\x00\x00\x00\x00\x00".bytes
-    # Byte 5 is the sort version
-    @data += "\x00".bytes
-    # Byte 6 is how many bytes are needed for each offset table offset
-    @data += "\x01".bytes
-    @data += "\x01".bytes
-    # Bytes 8-15 are how many objects are contained in the plist
-    @data += "\x00\x00\x00\x00\x00\x00\x00\x03".bytes
-    # Bytes 16-23 are for an offset from the offset table
-    @data += "\x00\x00\x00\x00\x00\x00\x00\x00".bytes
-    # Bytes 24-31 denote the position of the offset table from the start of the file
-    @data += "\x00\x00\x00\x00\x00\x00".bytes + [@url.length + 18].pack('S>').bytes
+  public
 
-    @data = @data.pack('C*')
+  def data
+    BinaryPlist.dump(@url)
   end
-  
+
   def save(filename)
-    File.open(filename, 'wb') { |f| f.write data }
+    raise ArgumentError, "Filename cannot be nil or empty" if filename.nil? || filename.empty?
+    
+    begin
+      File.open(filename, 'wb') { |f| f.write data }
+    rescue => e
+      raise WeblocError, "Failed to save webloc file '#{filename}': #{e.message}"
+    end
   end
-end
+end

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: webloc
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Peter Cooper
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-05-01 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: plist
@@ -31,21 +30,16 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- Gemfile
+- CHANGELOG.md
 - LICENSE
 - README.md
-- Rakefile
 - lib/webloc.rb
+- lib/webloc/binary_plist.rb
 - lib/webloc/version.rb
-- test/oldstyle.webloc
-- test/pliststyle.webloc
-- test/webloc_test.rb
-- webloc.gemspec
 homepage: https://github.com/peterc/webloc
-licenses: []
+licenses:
+- MIT
 metadata: {}
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -53,18 +47,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '3.0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.3
-signing_key:
+rubygems_version: 4.0.6
 specification_version: 4
 summary: Reads and writes .webloc files on macOS
-test_files:
-- test/oldstyle.webloc
-- test/pliststyle.webloc
-- test/webloc_test.rb
+test_files: []

data/.gitignore

@@ -1,4 +0,0 @@
-pkg/*
-*.gem
-.bundle
-.DS_Store

data/Gemfile

@@ -1,4 +0,0 @@
-source "https://rubygems.org"
-
-# Specify your gem's dependencies in webloc.gemspec
-gemspec

data/Rakefile

@@ -1,11 +0,0 @@
-require 'bundler'
-Bundler::GemHelper.install_tasks
-
-require 'rake/testtask'
-Rake::TestTask.new(:test) do |test|
-  test.libs << 'lib' << 'test'
-  test.pattern = 'test/**/*_test.rb'
-  test.verbose = true
-end
-
-task :default => :test

data/test/pliststyle.webloc

@@ -1,8 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
-	<key>URL</key>
-	<string>https://github.com/peterc/webloc</string>
-</dict>
-</plist>

data/test/webloc_test.rb

@@ -1,50 +0,0 @@
-require 'test/unit'
-require 'webloc'
-require 'tempfile'
-
-class WeblocTest < Test::Unit::TestCase
-  def test_webloc_object_requires_url
-    assert_raise(ArgumentError) { Webloc.new }
-  end
-  
-  def test_webloc_object_created_with_url
-    assert_equal 'http://example.com', Webloc.new('http://example.com').url
-  end
-  
-  def test_webloc_object_loaded_from_old_style_file
-    assert_equal 'https://github.com/peterc/webloc', Webloc.load(File.dirname(__FILE__) + '/oldstyle.webloc').url
-  end
-
-  def test_webloc_object_loaded_from_plist_file
-    assert_equal 'https://github.com/peterc/webloc', Webloc.load(File.dirname(__FILE__) + '/pliststyle.webloc').url
-  end
-  
-  def test_webloc_generates_valid_data
-    data = File.read(File.dirname(__FILE__) + '/oldstyle.webloc').b
-    assert_equal data, Webloc.new('https://github.com/peterc/webloc').data
-  end
-
-  def test_webloc_can_handle_long_urls
-    url = "http://example.com/this-is-a-very-long-url-abcde" + ('a' * 2000)
-    assert_nothing_raised { Webloc.new(url).data }
-    file = Tempfile.new('test-long-webloc')
-    begin
-      Webloc.new(url).save(file.path)
-      assert_equal url, Webloc.load(file.path).url
-    ensure
-      file.close
-      file.unlink
-    end
-  end
-
-  def test_webloc_can_write_file
-    file = Tempfile.new('test-webloc')
-    begin
-      Webloc.new('https://github.com/peterc/webloc').save(file.path)
-      assert_equal Webloc.new('https://github.com/peterc/webloc').data, File.read(file.path).b
-    ensure
-      file.close
-      file.unlink
-    end    
-  end
-end

data/webloc.gemspec

@@ -1,23 +0,0 @@
-# -*- encoding: utf-8 -*-
-$:.push File.expand_path("../lib", __FILE__)
-require "webloc/version"
-
-Gem::Specification.new do |s|
-  s.name        = "webloc"
-  s.version     = Webloc::VERSION
-  s.platform    = Gem::Platform::RUBY
-  s.authors     = ["Peter Cooper"]
-  s.email       = ["git@peterc.org"]
-  s.homepage    = "https://github.com/peterc/webloc"
-  s.summary     = %q{Reads and writes .webloc files on macOS}
-  s.description = %q{Webloc reads and writes .webloc files on macOS}
-
-  s.rubyforge_project = "webloc"
-
-  s.files         = `git ls-files`.split("\n")
-  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
-  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
-  s.require_paths = ["lib"]
-  
-  s.add_dependency 'plist'
-end