abbu 0.1.2 → 0.4.0

data/docs/TODO.md

@@ -0,0 +1,180 @@
+<!-- docs/TODO.md -->
+
+# Abbu Roadmap & TODO
+
+Feature checklist organized by release version.
+
+---
+
+## v0.1.x — Foundation (Released)
+
+### v0.1.0 — Initial Release
+- [x] `Abbu.open(path)` entry point returning an `Archive`
+- [x] `Archive#contacts` — reads contacts from SQLite
+- [x] `Archive#sqlite?` — detects modern `.abcddb` bundles
+- [x] `Contact` model with `first_name`, `last_name`, `emails`, `phones`, `company`
+- [x] `Parsers::SqliteParser` — queries `ZABCDRECORD`, `ZABCDEMAILADDRESS`, `ZABCDPHONENUMBER`
+- [x] `Parsers::PlistParser` — stub with warning
+- [x] `Exporters::CsvExporter` — `to_file` and `to_stdout`
+- [x] `Exporters::JsonExporter` — `to_file` and `to_stdout`
+- [x] `Exporters::VcardExporter` — `to_file` and `to_stdout` (vCard 3.0)
+- [x] `Utils::Deduplicator` — groups contacts by first email
+- [x] `bin/abbu` CLI with `--format`, `--output`, `--stats`, `--dedupe`, `--version`
+- [x] Rake tasks: `abbu:export`, `abbu:dedupe`, `abbu:stats`
+- [x] `docs/ABBU.md` — file format reference
+- [x] RSpec test suite with 100% coverage target
+- [x] Guard + RuboCop DX loop
+- [x] GitHub Actions CI (Ruby 3.2 + 3.3)
+
+### v0.1.1 — Pathname Fix
+- [x] Fix missing `require 'pathname'` in `archive.rb`
+- [x] Regression guard spec for file require isolation
+
+### v0.1.2 — Full Apple Contacts Schema
+- [x] Nickname, prefix (`Title`), suffix fields
+- [x] Smart `full_name` formatting: `Honorable Stan "Stretch" Carver II`
+- [x] Job title, department, maiden name
+- [x] Phonetic first/last name, phonetic company
+- [x] Pronouns, ringtone, texttone
+- [x] Hash-based emails/phones preserving custom labels
+- [x] Address parsing from `ZABCDPOSTALADDRESS`
+- [x] Group membership from `Z_ABCDCONTACTGROUP`
+- [x] URL parsing from `ZABCDURLADDRESS`
+- [x] Notes from `ZABCDNOTE`
+- [x] Related names from `ZABCDRELATEDNAME`
+- [x] Social profiles from `ZABCDSOCIALPROFILE` (Twitter, etc.)
+- [x] CSV export with all fields (addresses, groups, URLs, notes, related names, social profiles)
+- [x] JSON export with all contact fields
+- [x] vCard 3.0 export with `ADR`, `URL`, `NICKNAME`, `TITLE`, `NOTE`, `X-SOCIALPROFILE`
+- [x] `rubocop-rspec` plugin integration
+- [x] SqliteParser refactored with `RECORD_FIELD_MAP` constant
+- [x] CsvExporter refactored into `core_fields` / `extended_fields`
+
+---
+
+## v0.2.0 — Plist Parser (Released)
+
+- [x] `PlistParser` — parse legacy `.abcdp` plist contact files
+- [x] Full field extraction matching SqliteParser output shape
+- [x] `FIELD_MAP` constant for flat-field mapping
+- [x] Multi-value field extraction (emails, phones, addresses, URLs, notes, related names, social profiles)
+- [x] Flexible input: directory path or array of file paths
+- [x] `Archive` scans `**/*.abcdp` across entire bundle tree
+- [x] `plist` gem (~> 3.7) runtime dependency
+- [x] Plist fixture files for integration testing
+- [x] Birthday / anniversary date parsing (plist `Birthday` key)
+- [x] Lunar birthday support
+- [x] Middle name extraction
+- [x] Instant messaging addresses (AIM, Jabber, etc.)
+- [x] Verification code field support
+
+---
+
+## v0.2.1 — Date Fields
+
+- [x] `dates` attribute on `Contact` (array of hashes: `{ label:, date: }`)
+- [x] SqliteParser: parse `ZABCDDATECOMPONENTS` (year/month/day separate columns)
+- [x] PlistParser: parse `Birthday` key
+- [x] Anniversary and custom date labels
+- [x] Lunar birthday handling
+- [x] CSV/JSON/vCard export of date fields (`BDAY`, `ANNIVERSARY` in vCard)
+
+---
+
+## v0.2.2 — Instant Messaging & Verification
+
+- [x] `instant_messages` attribute on `Contact`
+- [x] SqliteParser: parse `ZABCDMESSAGINGADDRESS`
+- [x] PlistParser: parse `InstantMessage` key
+- [x] Verification code field
+- [x] CSV/JSON/vCard export (`IMPP` in vCard)
+
+---
+
+## v0.2.3 — Middle Name & Completeness
+
+- [x] `middle_name` attribute on `Contact`
+- [x] Update `full_name` to include middle name
+- [x] SqliteParser: `ZMIDDLENAME` column
+- [x] PlistParser: `Middle` key
+- [x] Phonetic middle name support
+- [x] vCard `N` field with middle name component
+
+---
+
+## v0.3.0 — Image Resolution (Released)
+
+- [x] Resolve contact photos through observed `ZIMAGEURI` relationships
+- [x] `Contact#image_uri` and `Contact#image_path` accessors
+- [x] Root and nested source image discovery
+- [x] CSV / JSON image paths and vCard PHOTO file URI references
+
+---
+
+## v0.4.0 — Read-only Contacts Toolkit
+
+Implemented in the accepted #19–#25 tranche and earlier prerequisites; publication
+is gated by [release issue #27](https://github.com/scarver2/abbu/issues/27),
+Deputy exact-head review, and Sheriff release authority.
+
+- [x] Evidence-safe schema introspection and `--schema`
+- [x] Lossless Apple label normalization with `raw_label` and anniversary vCard fidelity
+- [x] Creation/modification timestamps and source provenance
+- [x] Tolerant structured diagnostics, per-database/table deduplication, and strict mode
+- [x] Provenance-aware identity suggestions, phone comparison, and explicit merge-policy boundary
+- [x] Source-aware duplicate image-stem resolution
+- [x] First-class image extraction with content detection, safe filenames, and no destination overwrites
+- [x] Chainable query API, exact email/phone lookup, and partial name/email search
+- [x] TSV and stable JSON CLI search output
+- [x] Explicit read-only live macOS Contacts input through `--live` / `--live-path PATH`
+- [x] Ruby 3.3 minimum and lint/tooling gate; Ruby 3.4 and 4.0 CI
+- [x] Pull-request CI plus canonical-main pushes, without duplicate feature-branch runs
+- [x] `AGENTS.md`, local skills, and canonical spec/lint/package workflows
+
+## Pre-1.0 Backlog — Not Included in 0.4.0
+
+These remain future work; no implementation is authorized by release preparation.
+Version assignments below 1.0 are intentionally deferred until scopes are accepted.
+
+- [ ] [#28](https://github.com/scarver2/abbu/issues/28): synthetic WAL, sidecar, and concurrent-writer validation
+- [ ] [#29](https://github.com/scarver2/abbu/issues/29): stronger Apple-compatible vCard fidelity and round trips
+- [ ] [#30](https://github.com/scarver2/abbu/issues/30): embedded vCard photos (currently file URI references)
+- [ ] [#31](https://github.com/scarver2/abbu/issues/31): first-class read-only source objects
+- [ ] [#32](https://github.com/scarver2/abbu/issues/32): first-class groups and membership queries
+- [ ] [#33](https://github.com/scarver2/abbu/issues/33): evidence-backed My Card identification
+- [ ] [#34](https://github.com/scarver2/abbu/issues/34): modified-since and date-range queries
+- [ ] [#35](https://github.com/scarver2/abbu/issues/35): evidence-backed save/edit history
+- [ ] [#36](https://github.com/scarver2/abbu/issues/36): alternate-calendar and lunar metadata research
+- [ ] [#37](https://github.com/scarver2/abbu/issues/37): explainable fuzzy names and configurable thresholds
+- [ ] [#38](https://github.com/scarver2/abbu/issues/38): merge plans, side-by-side evidence, and safe built-in policies
+- [ ] [#39](https://github.com/scarver2/abbu/issues/39): broader machine-readable JSON CLI contract
+- [ ] [#40](https://github.com/scarver2/abbu/issues/40): streaming contact iteration/export and large-store performance
+- [ ] [#41](https://github.com/scarver2/abbu/issues/41): portable SQLite export
+- [ ] [#42](https://github.com/scarver2/abbu/issues/42): birthday/anniversary iCalendar export
+- [ ] [#43](https://github.com/scarver2/abbu/issues/43): identity-aware snapshot diffs
+- [ ] [#44](https://github.com/scarver2/abbu/issues/44): history across snapshot directories
+- [ ] [#45](https://github.com/scarver2/abbu/issues/45): optional MCP/agent adapter over read-only interfaces
+- [ ] [#46](https://github.com/scarver2/abbu/issues/46): evidence-backed new-ABBU writer research
+- [ ] [#47](https://github.com/scarver2/abbu/issues/47): public API stability checklist
+- [ ] Thumbnail/full-size image selection after reproducible format evidence
+- [ ] Region filtering and an explicit CLI filter grammar
+
+Previous in-place `Archive#deduplicate!` / source-archive mutation proposals are
+superseded. Merge plans must preserve originals; any future writer targets a new
+output bundle and requires evidence-backed round-trip validation. Writing live
+Contacts databases is not part of the roadmap.
+
+## v1.0.0 — Future Stable API
+
+- [ ] Complete the API-stability checklist and acceptance evidence before promising stability
+- [ ] Comprehensive public API documentation
+- [ ] Benchmarks for large archives (10k+ contacts)
+- [ ] Optional external CRM adapters (Printavo, HubSpot, generic webhook/API), separately scoped
+
+See [README](../README.md) and [CHANGELOG](CHANGELOG.md) for current behavior
+and historical releases.
+
+—
+Stan Carver II
+Made in Texas 🤠
+https://stancarver.com

data/lib/abbu/archive.rb

@@ -2,26 +2,62 @@
 # frozen_string_literal: true
 
 require 'pathname'
-require_relative 'parsers/sqlite_parser'
+require_relative 'diagnostic'
+require_relative 'image_extractor'
+require_relative 'parse_error'
 require_relative 'parsers/plist_parser'
+require_relative 'parsers/sqlite_parser'
+require_relative 'query'
+require_relative 'schema_inspector'
+require_relative 'utils/image_resolver'
 
 module Abbu
   class Archive
-    attr_reader :path
+    attr_reader :diagnostics, :path
 
-    def initialize(path)
+    def initialize(path, strict: false)
       @path = Pathname.new(path)
+      @strict = strict
+      @diagnostics = []
       validate!
     end
 
     def contacts
-      parser.contacts
+      @contacts ||= parser.contacts.tap { |cs| attach_images(cs) }
+    end
+
+    def query
+      Query.new(contacts)
+    end
+
+    def where(criteria)
+      query.where(criteria)
+    end
+
+    def search(term)
+      query.search(term)
+    end
+
+    def find_by_email(email)
+      query.find_by_email(email)
+    end
+
+    def find_by_phone(phone)
+      query.find_by_phone(phone)
     end
 
     def sqlite?
       db_paths.any?
     end
 
+    def extract_images(output_dir)
+      ImageExtractor.new(contacts).extract(output_dir)
+    end
+
+    def schema_report
+      SchemaInspector.new(db_paths, root_path: @path).report
+    end
+
     private
 
     def validate!
@@ -33,16 +69,40 @@ module Abbu
       @db_paths ||= @path.glob('**/*.abcddb')
     end
 
-    def records_path
-      @records_path ||= @path.join('Records')
+    def plist_paths
+      @plist_paths ||= @path.glob('**/*.abcdp').sort
     end
 
     def parser
       if sqlite?
-        Parsers::SqliteParser.new(db_paths)
+        Parsers::SqliteParser.new(db_paths, root_path: @path, diagnostics: diagnostics, strict: @strict)
       else
-        Parsers::PlistParser.new(records_path)
+        Parsers::PlistParser.new(plist_paths, root_path: @path, diagnostics: diagnostics, strict: @strict)
       end
     end
+
+    def attach_images(contacts)
+      return if contacts.empty?
+
+      resolver = Utils::ImageResolver.new(@path)
+      contacts.each do |contact|
+        next unless contact.image_uri
+
+        contact.image_path = resolver.resolve(contact.image_uri, source: contact.source)
+        record_missing_image(contact) unless contact.image_path
+      end
+    end
+
+    def record_missing_image(contact)
+      diagnostic = Diagnostic.new(
+        category: :missing_image,
+        message: 'Referenced contact image was not found',
+        parser: :archive,
+        source: contact.source&.fetch(:path, @path.to_s) || @path.to_s,
+        context: {}
+      )
+      diagnostics << diagnostic
+      raise ParseError, diagnostic if @strict
+    end
   end
 end

data/lib/abbu/contact.rb

@@ -3,9 +3,16 @@
 
 module Abbu
   class Contact
-    attr_accessor :first_name, :last_name, :emails, :phones, :company, :addresses, :groups, :nickname, :prefix, :suffix,
-                  :job_title, :department, :maiden_name, :phonetic_first_name, :phonetic_last_name, :phonetic_company,
-                  :pronouns, :ringtone, :texttone, :urls, :notes, :related_names, :social_profiles
+    attr_accessor :first_name, :middle_name, :last_name, :emails,
+                  :phones, :company, :addresses, :groups, :nickname,
+                  :prefix, :suffix, :job_title, :department, :maiden_name,
+                  :phonetic_first_name, :phonetic_middle_name, :phonetic_last_name,
+                  :phonetic_company, :pronouns, :ringtone, :texttone,
+                  :urls, :notes, :related_names, :social_profiles,
+                  :birthday, :anniversary, :dates, :instant_messages,
+                  :verification_code, :lunar_birthday,
+                  :image_uri, :image_path,
+                  :created_at, :modified_at, :source
 
     def initialize
       @emails = []
@@ -16,11 +23,13 @@ module Abbu
       @notes = []
       @related_names = []
       @social_profiles = []
+      @dates = []
+      @instant_messages = []
     end
 
     def full_name
       quoted_nickname = nickname ? "\"#{nickname}\"" : nil
-      [prefix, first_name, quoted_nickname, last_name, suffix].compact.join(' ')
+      [prefix, first_name, middle_name, quoted_nickname, last_name, suffix].compact.join(' ')
     end
 
     def to_s

data/lib/abbu/diagnostic.rb

@@ -0,0 +1,27 @@
+# lib/abbu/diagnostic.rb
+# frozen_string_literal: true
+
+module Abbu
+  class Diagnostic
+    attr_reader :category, :context, :message, :parser, :source
+
+    def initialize(category:, message:, parser:, source:, context: {})
+      @category = category.to_sym
+      @context = context.freeze
+      @message = message.to_s.freeze
+      @parser = parser.to_sym
+      @source = source.to_s.freeze
+      freeze
+    end
+
+    def to_h
+      {
+        category: category,
+        message: message,
+        parser: parser,
+        source: source,
+        context: context
+      }
+    end
+  end
+end

data/lib/abbu/exporters/csv_exporter.rb

@@ -27,7 +27,8 @@ module Abbu
       private
 
       def headers
-        %w[Name Email Phone Company Address Groups URLs Notes RelatedNames SocialProfiles]
+        %w[Name First Middle Last Email Phone Company Address Groups URLs Notes RelatedNames SocialProfiles Birthday
+           Anniversary InstantMessages VerificationCode LunarBirthday ImagePath]
       end
 
       def row(contact)
@@ -36,21 +37,20 @@ module Abbu
 
       def core_fields(contact)
         [
-          contact.full_name,
-          contact.emails.first&.fetch(:address, nil),
-          contact.phones.first&.fetch(:number, nil),
-          contact.company,
-          format_address(contact.addresses.first),
-          contact.groups.join(', ')
+          contact.full_name, contact.first_name, contact.middle_name,
+          contact.last_name, contact.emails.first&.fetch(:address, nil),
+          contact.phones.first&.fetch(:number, nil), contact.company,
+          format_address(contact.addresses.first), contact.groups.join(', ')
         ]
       end
 
-      def extended_fields(contact)
+      def extended_fields(contact) # rubocop:disable Metrics/AbcSize
         [
-          contact.urls.map { |u| u[:url] }.join(', '),
-          contact.notes.join("\n"),
-          format_related_names(contact.related_names),
-          format_social_profiles(contact.social_profiles)
+          contact.urls.map { |u| u[:url] }.join(', '), contact.notes.join("\n"),
+          format_related_names(contact.related_names), format_social_profiles(contact.social_profiles),
+          format_date(contact.birthday), format_date(contact.anniversary),
+          format_instant_messages(contact.instant_messages), contact.verification_code,
+          format_date(contact.lunar_birthday), contact.image_path.to_s
         ]
       end
 
@@ -67,6 +67,20 @@ module Abbu
       def format_social_profiles(profiles)
         profiles.map { |sp| "#{sp[:username]} on #{sp[:service]}" }.join(', ')
       end
+
+      def format_instant_messages(ims)
+        ims.map { |im| "#{im[:address]} (#{im[:service]})" }.join(', ')
+      end
+
+      def format_date(date)
+        return nil unless date
+
+        if date[:year]&.positive?
+          format('%<year>04d-%<month>02d-%<day>02d', year: date[:year], month: date[:month], day: date[:day])
+        else
+          format('--%<month>02d-%<day>02d', month: date[:month], day: date[:day])
+        end
+      end
     end
   end
 end

data/lib/abbu/exporters/json_exporter.rb

@@ -2,6 +2,7 @@
 # frozen_string_literal: true
 
 require 'json'
+require 'time'
 
 module Abbu
   module Exporters
@@ -28,7 +29,11 @@ module Abbu
         {
           name: contact.full_name,
           first_name: contact.first_name,
+          middle_name: contact.middle_name,
           last_name: contact.last_name,
+          phonetic_first_name: contact.phonetic_first_name,
+          phonetic_middle_name: contact.phonetic_middle_name,
+          phonetic_last_name: contact.phonetic_last_name,
           nickname: contact.nickname,
           prefix: contact.prefix,
           suffix: contact.suffix,
@@ -42,7 +47,18 @@ module Abbu
           urls: contact.urls,
           notes: contact.notes,
           related_names: contact.related_names,
-          social_profiles: contact.social_profiles
+          social_profiles: contact.social_profiles,
+          birthday: contact.birthday,
+          anniversary: contact.anniversary,
+          dates: contact.dates,
+          instant_messages: contact.instant_messages,
+          verification_code: contact.verification_code,
+          lunar_birthday: contact.lunar_birthday,
+          image_uri: contact.image_uri,
+          image_path: contact.image_path&.to_s,
+          created_at: contact.created_at&.iso8601,
+          modified_at: contact.modified_at&.iso8601,
+          source: contact.source
         }.compact
       end
     end

data/lib/abbu/exporters/vcard_exporter.rb

@@ -1,6 +1,8 @@
 # lib/abbu/exporters/vcard_exporter.rb
 # frozen_string_literal: true
 
+require 'uri'
+
 module Abbu
   module Exporters
     class VcardExporter
@@ -22,23 +24,75 @@ module Abbu
         @contacts.map { |c| vcard_for(c) }.join("\n")
       end
 
-      def vcard_for(contact) # rubocop:disable Metrics/AbcSize,Metrics/MethodLength
+      def vcard_for(contact) # rubocop:disable Metrics/MethodLength
         lines = ['BEGIN:VCARD', 'VERSION:3.0']
-        lines << "FN:#{contact.full_name}"
-        lines << "N:#{contact.last_name};#{contact.first_name};;;"
-        lines << "NICKNAME:#{contact.nickname}" if contact.nickname
-        lines << "ORG:#{contact.company}" if contact.company
-        lines << "TITLE:#{contact.job_title}" if contact.job_title
+
+        append_name_fields(lines, contact)
         append_emails(lines, contact)
         append_phones(lines, contact)
         append_addresses(lines, contact)
         append_urls(lines, contact)
         append_social_profiles(lines, contact)
-        contact.notes.each { |n| lines << "NOTE:#{n}" }
+        append_dates(lines, contact)
+        append_instant_messages(lines, contact)
+        append_verification_code(lines, contact)
+        append_notes(lines, contact)
+        append_photo(lines, contact)
+
         lines << 'END:VCARD'
         lines.join("\n")
       end
 
+      def append_name_fields(lines, contact)
+        lines << "FN:#{contact.full_name}"
+        lines << "N:#{name_components(contact).join(';')}"
+        append_nickname(lines, contact)
+        append_company(lines, contact)
+        append_title(lines, contact)
+        append_phonetic_names(lines, contact)
+      end
+
+      def name_components(contact)
+        [contact.last_name, contact.first_name, contact.middle_name, contact.prefix, contact.suffix]
+      end
+
+      def append_nickname(lines, contact)
+        lines << "NICKNAME:#{contact.nickname}" if contact.nickname
+      end
+
+      def append_company(lines, contact)
+        lines << "ORG:#{contact.company}" if contact.company
+      end
+
+      def append_title(lines, contact)
+        lines << "TITLE:#{contact.job_title}" if contact.job_title
+      end
+
+      def append_phonetic_names(lines, contact)
+        lines << "X-PHONETIC-FIRST-NAME:#{contact.phonetic_first_name}" if contact.phonetic_first_name
+        lines << "X-PHONETIC-MIDDLE-NAME:#{contact.phonetic_middle_name}" if contact.phonetic_middle_name
+        lines << "X-PHONETIC-LAST-NAME:#{contact.phonetic_last_name}" if contact.phonetic_last_name
+      end
+
+      def append_verification_code(lines, contact)
+        lines << "X-VERIFICATION-CODE:#{contact.verification_code}" if contact.verification_code
+      end
+
+      def append_photo(lines, contact)
+        return unless contact.image_path
+
+        lines << "PHOTO;VALUE=URI:#{file_uri(contact.image_path)}"
+      end
+
+      def file_uri(path)
+        escaped_path = URI::DEFAULT_PARSER.escape(path.to_s)
+        URI::Generic.build(scheme: 'file', path: escaped_path)
+      end
+
+      def append_notes(lines, contact)
+        contact.notes.each { |n| lines << "NOTE:#{n}" }
+      end
+
       def append_emails(lines, contact)
         contact.emails.each do |e|
           label = e[:label] || 'INTERNET'
@@ -71,6 +125,36 @@ module Abbu
           lines << "X-SOCIALPROFILE;TYPE=#{sp[:service]}:#{sp[:username]}"
         end
       end
+
+      def append_instant_messages(lines, contact)
+        contact.instant_messages.each do |im|
+          service = im[:service]&.downcase || 'unknown'
+          lines << "IMPP;TYPE=#{im[:label]}:#{service}:#{im[:address]}"
+        end
+      end
+
+      def append_dates(lines, contact)
+        lines << "BDAY:#{format_vcard_date(contact.birthday)}" if contact.birthday
+        lines << "X-LUNAR-BDAY:#{format_vcard_date(contact.lunar_birthday)}" if contact.lunar_birthday
+        return unless contact.anniversary
+
+        lines << "X-ABDATE;type=pref:#{format_vcard_date(contact.anniversary)}"
+        lines << "X-ABLABEL:#{anniversary_label(contact.anniversary)}"
+      end
+
+      def anniversary_label(anniversary)
+        anniversary[:raw_label] || anniversary[:label] || 'Anniversary'
+      end
+
+      def format_vcard_date(date)
+        return nil unless date
+
+        if date[:year]&.positive?
+          format('%<year>04d-%<month>02d-%<day>02d', year: date[:year], month: date[:month], day: date[:day])
+        else
+          format('--%<month>02d-%<day>02d', month: date[:month], day: date[:day])
+        end
+      end
     end
   end
 end