pilipinas 1.1.1 → 1.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 989723b7dcd5133a2d2240aa124e0c5400a981f521d0904d8b2858fb71f15e9e
-  data.tar.gz: c01c0377bf9572381844a38d1c77dbe45aed0e554076159a9964371f5f7aa851
+  metadata.gz: 3c053d3fbf05daa1c3bd6298945223e06a0c8154962f9958e8681701738a808b
+  data.tar.gz: 5613bb74fb432f59980cb54067d2d5fb0c2229807df82471e73131b884e00452
 SHA512:
-  metadata.gz: b2589f13688bd95ad3ad176d9dccc2412a0df9a3484a173129e3dc66e7f78dad67ea1f7920f7bee9136f412b5c6221a7776b2f7a682ef1e0fdb55d2ffd1cd307
-  data.tar.gz: 73bc97b32dcaa44f0a9181e63d450770535e12f73254fcea500551a46bb788a0dfe2bf9ed7de897a84ab1c22f0a19545d050e2b7a1c8cd76c7d3515a55b60d0d
+  metadata.gz: 5b1fb8a1a22541315c95dddf2f8f027e88c9455a7fbcde7dd8af9a8483c844247224321aae75d627eab1a15121ef075df01430cfe9a50448dbf9191e83c53851
+  data.tar.gz: 453835a9b952e40983c10e50ba5f7321efc669c59b11db7442943c29553d37c5d5d79defb697382ac7eac11298f87dc92d7e8dc1e0ad26e7cf5262b532ac97ac

data/CHANGELOG.md

@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ---
 
+## [1.1.2] - 2026-06-12
+
+### Fixed
+
+- `rake pilipinas:load` now seeds the ActiveRecord tables from `lib/data/pilipinas_data.yml`, the complete bundled data source. Previously the loader used the compact file-backed YAML files, so database rows were populated with only `code` and `name` while columns such as `location_id`, `parent_id`, `lft`, `rgt`, coordinates, and classification fields remained `NULL`.
+
+- Re-running `rake pilipinas:load` now removes stale compact rows created by earlier loader versions. This fixes cases where old rows such as provinces with `location_id: nil` remained in the database because their compact internal codes did not match the full PSA-style codes used by `pilipinas_data.yml`.
+
+### Changed
+
+- Improved loader memory behavior by transforming rows directly from the parsed full-data table and keeping only the current insert batch in memory.
+
+### Tests
+
+- Added regression coverage for complete-column seeding, stale compact-row cleanup, legacy seed behavior, and exact batch-boundary inserts.
+
+---
+
+## [1.1.1] - 2026-06-11
+
+### Added
+
+- `rails generate pilipinas:code_indexes` — new migration generator that adds `UNIQUE` indexes on the `code` column to all four `pilipinas_*` tables. Run this if your database was created with a pre-1.0 migration and `rake pilipinas:load` raises `ArgumentError: No unique index found for code`.
+
+### Fixed
+
+- `Loader.bulk_insert` now rescues the bare `ArgumentError` raised by `upsert_all` when the unique index on `code` is absent, and re-raises it as a descriptive `Pilipinas::Error` that tells the user exactly which commands to run (`rails generate pilipinas:code_indexes && rails db:migrate`). Previously the error surfaced as an unguided `ArgumentError` from deep inside ActiveRecord.
+
+---
+
 ## [1.1.0] - 2026-05-13
 
 ### Added
@@ -18,19 +48,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     self.enforce_readonly = false
   end
   ```
-- `lib/pilipinas/testing/rspec.rb` — a ready-made RSpec helper that disables
-  the read-only guard on all four DB models for the entire test suite.
-  Require it once in `rails_helper.rb`:
+- `lib/pilipinas/testing/rspec.rb` — a ready-made RSpec helper that disables the read-only guard on all four DB models for the entire test suite. Require it once in `rails_helper.rb`:
   ```ruby
   require 'pilipinas/testing/rspec'
   ```
-- Spec coverage for `StaticRecord` — 8 examples covering default behaviour,
-  `enforce_readonly = false`, subclass inheritance, and class-level isolation.
+- Spec coverage for `StaticRecord` — 8 examples covering default behaviour, `enforce_readonly = false`, subclass inheritance, and class-level isolation.
 
 ### Changed
 
-- `readonly?` now gates on `self.class.enforce_readonly && !new_record?` instead
-  of unconditionally returning `!new_record?`.
+- `readonly?` now gates on `self.class.enforce_readonly && !new_record?` instead of unconditionally returning `!new_record?`.
 
 ---
 
@@ -40,26 +66,20 @@ Complete rewrite of the gem. Zero runtime dependencies.
 
 ### Added
 
-- In-memory layer with thread-safe `Pilipinas::Cache` (Mutex + double-checked
-  locking) and O(1) look-ups via separate code/name hash indices.
+- In-memory layer with thread-safe `Pilipinas::Cache` (Mutex + double-checked locking) and O(1) look-ups via separate code/name hash indices.
 - Immutable value objects — every entity instance is frozen.
-- `Pilipinas::Region`, `Province`, `City`, `Barangay` with `.all`, `.count`,
-  `.first`, `.last`, `.find_by`, `.find_by_code`, `.find_by_name`.
+- `Pilipinas::Region`, `Province`, `City`, `Barangay` with `.all`, `.count`, `.first`, `.last`, `.find_by`, `.find_by_code`, `.find_by_name`.
 - Hierarchy traversal: `region.provinces`, `province.cities`, `city.barangays`.
-- Optional ActiveRecord layer (`Pilipinas::Db::*`) with memory-efficient scopes
-  (`.lite`, `.by_code`, `.by_name`, `.find_lite_by_code`, `.find_lite_by_name`).
-- `StaticRecord` concern: disables STI, adds lean SELECT scopes, enforces
-  read-only on persisted records.
-- Migration generator (`rails generate pilipinas:migration`) and
-  `rake pilipinas:load` seeding task.
+- Optional ActiveRecord layer (`Pilipinas::Db::*`) with memory-efficient scopes (`.lite`, `.by_code`, `.by_name`, `.find_lite_by_code`, `.find_lite_by_name`).
+- `StaticRecord` concern: disables STI, adds lean SELECT scopes, enforces read-only on persisted records.
+- Migration generator (`rails generate pilipinas:migration`) and `rake pilipinas:load` seeding task.
 - Full RSpec suite (57 examples).
 - GitHub Actions CI pipeline.
 
 ### Changed
 
 - Requires Ruby ≥ 3.4 (developed against Ruby 4.0).
-- Removed all runtime gem dependencies (previously depended on `yaml_db` and
-  others).
+- Removed all runtime gem dependencies (previously depended on `yaml_db` and others).
 
 ---
 
@@ -74,6 +94,8 @@ Complete rewrite of the gem. Zero runtime dependencies.
 - Rails generator for migrations.
 - Railtie for automatic Rake task loading in Rails apps.
 
+[1.1.2]: https://github.com/denmarkmeralpis/pilipinas/compare/v1.1.1...v1.1.2
+[1.1.1]: https://github.com/denmarkmeralpis/pilipinas/compare/v1.1.0...v1.1.1
 [1.1.0]: https://github.com/denmarkmeralpis/pilipinas/compare/v1.0.0...v1.1.0
 [1.0.0]: https://github.com/denmarkmeralpis/pilipinas/compare/v0.0.1...v1.0.0
 [0.0.1]: https://github.com/denmarkmeralpis/pilipinas/releases/tag/v0.0.1

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    pilipinas (1.1.1)
+    pilipinas (1.1.2)
 
 GEM
   remote: https://rubygems.org/
@@ -296,7 +296,7 @@ CHECKSUMS
   nokogiri (1.19.3-x86_64-linux-musl) sha256=248c906d2166eca5efb56d52fdee5f9a1f51d69a72e2b64fdac647b4ce39ea3f
   parallel (2.1.0) sha256=b35258865c2e31134c5ecb708beaaf6772adf9d5efae28e93e99260877b09356
   parser (3.3.11.1) sha256=d17ace7aabe3e72c3cc94043714be27cc6f852f104d81aa284c2281aecc65d54
-  pilipinas (1.1.1)
+  pilipinas (1.1.2)
   pp (0.6.3) sha256=2951d514450b93ccfeb1df7d021cae0da16e0a7f95ee1e2273719669d0ab9df6
   prettyprint (0.2.0) sha256=2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193
   prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85

data/lib/pilipinas/loader.rb

@@ -9,8 +9,9 @@ module Pilipinas
   # * **Idempotent** — uses +upsert_all+ (Rails 6.1+) so re-running the Rake
   #   task is safe.  Falls back to +insert_all+ (Rails 6.0) or individual
   #   +create!+ calls on older versions.
-  # * **Memory-efficient** — rows are inserted in batches of {BATCH_SIZE}
-  #   (default 500) so the process never holds a 42 k-row Array in memory.
+  # * **Memory-aware** — rows are transformed and inserted in batches of
+  #   {BATCH_SIZE} (default 500) so the process never holds a full
+  #   ActiveRecord insert payload in memory.
   # * **Atomic** — all four tables are seeded inside a single transaction; a
   #   failure rolls back everything, leaving no partial data.
   #
@@ -21,22 +22,102 @@ module Pilipinas
     # Number of rows inserted per SQL statement.
     # 500 balances SQL statement size against the number of round-trips.
     BATCH_SIZE = 500
+    FULL_DATA_FILE = 'pilipinas_data.yml'
+    LOCATION_TABLE = 'pilipinas_locations'
+
+    FULL_DATA_SEEDS = [
+      ['Region', 'Locations::Region', %w[location_id lft rgt code name longitude latitude]],
+      ['Province', 'Locations::Province', %w[location_id parent_id lft rgt code name longitude latitude]],
+      [
+        'City',
+        'Locations::Town',
+        %w[location_id parent_id lft rgt code name city income_class urban_rural district longitude latitude]
+      ],
+      ['Barangay', 'Locations::Barangay', %w[location_id parent_id lft rgt code name urban_rural]]
+    ].freeze
+    private_constant :FULL_DATA_FILE, :LOCATION_TABLE, :FULL_DATA_SEEDS
 
     class << self
       # Seed all four geographic tables inside a single transaction.
       #
       # @return [void]
       def run
+        column_indexes, records = full_location_table
+
         ActiveRecord::Base.transaction do
-          seed(Db::Region,   'regions.yml')
-          seed(Db::Province, 'provinces.yml')
-          seed(Db::City,     'cities.yml')
-          seed(Db::Barangay, 'barangays.yml')
+          FULL_DATA_SEEDS.each do |model_name, type, attributes|
+            model = Db.const_get(model_name)
+            delete_stale_compact_rows(model)
+            seed_full_data(model, records, column_indexes, type, attributes)
+          end
         end
       end
 
       private
 
+      # Load the full Rails fixture-style location dump bundled with the gem.
+      #
+      # The older per-table YAML files only contain +code+ and +name+ for the
+      # file-backed API.  Database seeding needs the complete dump so lft/rgt,
+      # parent links, coordinates, and classification fields are populated.
+      #
+      # @return [Array(Hash, Array<Array>)]
+      def full_location_table
+        data = Psych.load_file(File.join(DATA_DIR, FULL_DATA_FILE)) || {}
+        table = data.fetch(LOCATION_TABLE)
+        columns = table.fetch('columns').map(&:to_s)
+
+        [columns.each_with_index.to_h, table.fetch('records')]
+      end
+
+      # Insert or update rows for one split table from the full location dump.
+      #
+      # @param model          [Class] ActiveRecord model class
+      # @param records        [Array<Array>] full location dump rows
+      # @param column_indexes [Hash] source column names mapped to row indexes
+      # @param type           [String] source STI type to select
+      # @param attributes     [Array<String>] destination table attributes
+      # @return [void]
+      def seed_full_data(model, records, column_indexes, type, attributes)
+        type_index = column_indexes.fetch('type')
+        attribute_indexes = attributes.to_h { |attribute| [attribute, column_indexes.fetch(attribute)] }
+        now = timestamp
+        batch = []
+
+        records.each do |record|
+          next unless record[type_index] == type
+
+          batch << full_data_attributes(record, attribute_indexes, now)
+
+          next if batch.size < BATCH_SIZE
+
+          bulk_insert(model, batch)
+          batch = []
+        end
+
+        bulk_insert(model, batch) unless batch.empty?
+      end
+
+      # @param record            [Array] full location dump row
+      # @param attribute_indexes [Hash] destination attributes mapped to row indexes
+      # @param now               [Time] timestamp shared by the current table seed
+      # @return [Hash]
+      def full_data_attributes(record, attribute_indexes, now)
+        attribute_indexes.to_h { |attribute, index| [attribute, record[index]] }
+                         .merge('created_at' => now, 'updated_at' => now)
+      end
+
+      # Older loader versions seeded compact file-backed rows with only +code+
+      # and +name+. Those rows cannot be upserted into the full-data records
+      # because their +code+ values differ from the PSA codes in
+      # +pilipinas_data.yml+, so remove them before inserting the canonical set.
+      #
+      # @param model [Class] ActiveRecord model class
+      # @return [void]
+      def delete_stale_compact_rows(model)
+        model.where(location_id: nil).delete_all
+      end
+
       # Insert or update rows for one table from a YAML file.
       #
       # @param model    [Class]  ActiveRecord model class

data/lib/pilipinas/version.rb

@@ -4,5 +4,5 @@ module Pilipinas
   # Semantic version of the gem.
   #
   # Follows {https://semver.org Semantic Versioning}: MAJOR.MINOR.PATCH.
-  VERSION = '1.1.1'
+  VERSION = '1.1.2'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pilipinas
 version: !ruby/object:Gem::Version
-  version: 1.1.1
+  version: 1.1.2
 platform: ruby
 authors:
 - Nujian Den Mark Meralpis