has_public_record 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: dadeae14d61a8a853dff01a3fc146bf16903fde443c4863a537987ccb9edaa04
+  data.tar.gz: 7d348eb1f8c4f76be37625666c2e99ecbb824355189eb4cdaa8796058c7bcceb
+SHA512:
+  metadata.gz: 402e8cd8f72ffc69f45d326cae433f28b3e13aad4fce4de6bd99ca55192976c4346e01e51eb2eab1e37577246e85dd8324e305fd73ace87841f48f0322ebcee9
+  data.tar.gz: 292344f738d26c9a99e4eb4bb020dcbbbc51da61626cbb7e5d5fbb15d781d65886f21b1ef96ec31c1c36a71ebf02c91991bb287a63d5b5d3f0076ff95e22a770

data/CHANGELOG.md

@@ -0,0 +1,6 @@
+# Changelog
+
+## 0.1.0
+
+First release: prefixed public ids with length growth, lookup helpers, resolve and resolve_many, a standalone
+generator, migration and install generators, backfill / report / rename tasks, and testing helpers.

data/README.md

@@ -0,0 +1,189 @@
+# has_public_record
+
+Short, prefixed, URL-safe public ids for ActiveRecord models: `usr_K3F9A2`. The prefix says what the id
+belongs to, the random part is short, and it grows by itself as a table fills.
+
+~~~ruby
+class User < ApplicationRecord
+  has_public_record prefix: "usr"
+end
+
+user = User.create!
+user.public_id        # => "usr_K3F9A2"
+user.public_id_bare   # => "K3F9A2"
+user_path(user)       # => "/users/usr_K3F9A2"   (to_param)
+User.find_by_public_id!("usr_K3F9A2")
+~~~
+
+## Why prefixed, and why not the database id
+
+Database ids leak counts and invite enumeration. A prefixed id is safe to put in URLs and logs, and the prefix
+lets you resolve an id with no other context: `HasPublicRecord.resolve("usr_K3F9A2")` returns the `User`.
+
+## Options
+
+~~~ruby
+has_public_record prefix: "usr",          # required, unique across the app: a-z and 0-9, starts with a letter, max 10
+                  column: :public_id,     # where the id is stored
+                  length: 8,              # starting length of the random part for this model
+                  max_bumps: 4            # how far the length may grow before ExhaustedError
+~~~
+
+Declare it on the base class of an STI hierarchy. Declaring costs no database access, so it is safe while the
+app boots.
+
+## Methods
+
+On the model:
+
+| Method | What it does |
+| --- | --- |
+| `find_by_public_id(ref)` | Finds by `usr_K3F9A2` or the bare `K3F9A2`; another model's id or junk gives nil |
+| `find_by_public_id!(ref)` | Same, raises `ActiveRecord::RecordNotFound` |
+| `with_public_id(*refs)` | A relation for several ids; ids of other models are ignored |
+| `public_id?(ref)` | Whether the ref belongs to this model |
+| `public_id_prefix`, `public_id_column` | What the model declared |
+| `missing_public_id` | Rows with no id yet |
+| `generate_public_id` | A fresh unused id |
+| `backfill_public_ids!` | Gives every row without an id one; returns the count |
+
+On a record: `public_id`, `public_id_bare`, `public_id_prefix`, `to_param`.
+
+On the module:
+
+| Method | What it does |
+| --- | --- |
+| `HasPublicRecord.resolve(ref)` / `resolve!` | The record a ref points at, or nil / raises |
+| `HasPublicRecord.model_for(ref)` | The model class for a ref |
+| `HasPublicRecord.parse(ref)` | `Parsed(prefix:, bare:)` or nil |
+| `HasPublicRecord.valid?(ref)` | Whether the ref is shaped like an id |
+| `HasPublicRecord.prefixes` | `{ "usr" => "User", ... }` |
+| `HasPublicRecord.prefix_for(model)` | The prefix a model declared |
+| `HasPublicRecord.generate(...)` | Makes an id for a scope |
+
+The id is read-only once created. `insert_all` and `upsert_all` skip callbacks, so rows inserted that way get
+no id; fill them with `backfill_public_ids!`.
+
+## Length growth
+
+Ids start at `default_length` (6 by default, about 2 billion codes). When random tries keep hitting existing
+codes, the length grows by one and is remembered per table and column for the life of the process. Each growth
+is announced (see Notifications), so you can see a table getting dense before it matters.
+
+## Configuration
+
+~~~ruby
+# config/initializers/has_public_record.rb  (rails g has_public_record:install writes it, all commented out)
+HasPublicRecord.configure do |config|
+  config.default_length = 6
+  config.attempts_per_length = 10
+  config.max_length_bumps = 4
+  config.alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"   # 2+ distinct, no "_", no whitespace
+  config.lengths = { "users" => 8 }                          # per table; beats a model's length:
+end
+~~~
+
+Settings are read when an id is made, never copied at load, so an app that keeps its settings elsewhere can
+simply assign them from its own boot code. Prefixes are code, not settings.
+
+## Adding it to an existing table
+
+Two steps, so a populated table never needs a NOT NULL column it cannot satisfy:
+
+~~~zsh
+rails g has_public_record:migration users              # nullable column + unique index
+rails db:migrate
+rails has_public_record:backfill MODEL=User            # gives every row an id
+rails has_public_record:report                         # all clean?
+rails g has_public_record:migration users --enforce    # now NOT NULL
+rails db:migrate
+~~~
+
+Options: `--column=name`, and `--concurrently` (PostgreSQL only: builds the index without locking writes).
+For new tables, `t.string :public_id, null: false, index: { unique: true }` is enough.
+
+## Rake tasks
+
+| Task | What it does |
+| --- | --- |
+| `has_public_record:backfill` | Fills missing ids (`MODEL=User` to limit) |
+| `has_public_record:report` | Counts missing ids, wrong prefixes and duplicates, and shows how full each table's code space is; exits 1 if anything is off |
+| `has_public_record:rename MODEL=User FROM=old` | Previews moving stored ids from an old prefix to the declared one; add `APPLY=1` to do it, in one transaction |
+
+Renaming a prefix breaks every stored URL and any id saved in another table.
+
+## Notifications
+
+Through ActiveSupport::Notifications, never a logger of its own:
+
+- `grow.has_public_record` with `key`, `table`, `column` (both nil for a standalone generator), `from`, `to`
+- `rename.has_public_record` with `table`, `column`, `from`, `to`, `count`
+
+## Engines
+
+Works inside an engine. Prefixes are global per process, so choose distinctive ones. An engine should not call
+`HasPublicRecord.configure` (it is process-wide); use the per-model `length:` and `max_bumps:` instead.
+
+## Lookup helpers
+
+Ids are matched in any letter case (`usr_k3f9a2` finds `usr_K3F9A2`) when the alphabet has one case; an alphabet
+with both cases is left as typed.
+
+| Helper | What it does |
+| --- | --- |
+| `User.from_param(params[:id])` / `from_param!` | What a controller has: nil-safe and raising versions |
+| `User.public_id_format` | A regexp for route constraints: `resources :users, constraints: { id: User.public_id_format }` |
+| `HasPublicRecord.resolve_many(*refs)` | Mixed refs (an audit log) in one query per model; returns `{ ref => record }` |
+
+## Ids without a model
+
+The generator needs no model. Whether a code is free is a block:
+
+~~~ruby
+HasPublicRecord.generate("inv", length: 8) { |id| !Invoice.exists?(number: id) }   # => "inv_7K2QX9AB"
+HasPublicRecord.generate(nil, length: 5, key: "codes") { |id| !Redis.exists?(id) } # bare code: key names its growth
+
+invoices = HasPublicRecord.identifier("inv", length: 8) { |id| !Invoice.exists?(number: id) }
+invoices.generate                  # grows and notifies exactly like models do
+invoices.valid?("inv_7K2QX9AB")    # => true
+invoices.bare("inv_7K2QX9AB")      # => "7K2QX9AB"
+~~~
+
+Standalone prefixes are not registered for `HasPublicRecord.resolve`; only models are. A prefix a model already
+declared is refused.
+
+## Testing
+
+Rails fixtures and `insert_all` skip callbacks, so rows get no id unless you supply one:
+
+~~~ruby
+# fixtures: the same label always gives the same id
+alice:
+  public_id: <%= User.fixture_public_id(:alice) %>
+
+# bulk inserts
+User.insert_all(User.with_generated_public_ids(rows))
+~~~
+
+## Alphabets
+
+`config.alphabet = :unambiguous` leaves out 0/O and 1/I/L, which makes ids easier to read aloud. The default
+is `0-9A-Z`.
+
+## Associations
+
+Plain Rails associations are enough. Keep integer foreign keys inside the system and use the public id only at
+the edge, scoped through the owner so one person cannot reach another's records:
+
+~~~ruby
+current_user.sessions.find_by_public_id!(params[:id])
+~~~
+
+## Capacity
+
+`has_public_record:report` shows how full each table's code space is. Growth starts well before 100%: at half
+full, ten random tries in a row all hit taken codes about once in a thousand.
+
+## License
+
+MIT.

data/lib/generators/has_public_record/install/install_generator.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+
+module HasPublicRecord
+  module Generators
+    # rails g has_public_record:install  ->  config/initializers/has_public_record.rb
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+      desc "Creates config/initializers/has_public_record.rb with every setting, commented out."
+
+      def create_initializer
+        @defaults = HasPublicRecord::Configuration.new
+        template "initializer.rb.tt", "config/initializers/has_public_record.rb"
+      end
+
+      private
+
+      attr_reader :defaults
+    end
+  end
+end

data/lib/generators/has_public_record/install/templates/initializer.rb.tt

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+# has_public_record settings. Everything is commented out and shows its default: uncomment what you want to change.
+# Prefixes are not settings: each model declares its own (has_public_record prefix: "usr").
+
+HasPublicRecord.configure do |config|
+  # Starting length of the random part of an id (the prefix is extra). Grows by itself on dense tables.
+  # config.default_length = <%= defaults.default_length %>
+
+  # Random tries at one length before it grows by one.
+  # config.attempts_per_length = <%= defaults.attempts_per_length %>
+
+  # How many times the length may grow before ExhaustedError is raised.
+  # config.max_length_bumps = <%= defaults.max_length_bumps %>
+
+  # Characters used in ids: 2+ distinct, no "_" and no whitespace.
+  # config.alphabet = "<%= defaults.alphabet %>"   # or a preset: :unambiguous (no 0/O, 1/I/L)
+
+  # Starting length per table; beats a model's own length: option. Keys are table names.
+  # config.lengths = { "users" => 8 }
+end

data/lib/generators/has_public_record/migration/migration_generator.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module HasPublicRecord
+  module Generators
+    # rails g has_public_record:migration users            adds the nullable column and its unique index
+    # rails g has_public_record:migration users --enforce  later: makes it NOT NULL, once every row has an id
+    class MigrationGenerator < ActiveRecord::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+      desc "Adds a public id column to an existing table (give the table name, e.g. users)."
+
+      class_option :column, type: :string, default: "public_id", desc: "Column name"
+      class_option :enforce, type: :boolean, default: false, desc: "Make the column NOT NULL (run after the backfill)"
+      class_option :concurrently, type: :boolean, default: false,
+                                  desc: "PostgreSQL only: build the index without locking writes"
+
+      def create_migration_file
+        migration_template(options[:enforce] ? "enforce.rb.tt" : "add.rb.tt",
+                           File.join(db_migrate_path, "#{migration_name}.rb"))
+      end
+
+      private
+
+      def public_column = options[:column]
+
+      def migration_name
+        options[:enforce] ? "enforce_#{public_column}_on_#{table_name}" : "add_#{public_column}_to_#{table_name}"
+      end
+    end
+  end
+end

data/lib/generators/has_public_record/migration/templates/add.rb.tt

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+# Adds the public id column (nullable) and its unique index. Then fill existing rows:
+#   rails has_public_record:backfill MODEL=YourModel
+# and, once every row has an id (rails has_public_record:report), make it required:
+#   rails g has_public_record:migration <%= table_name %> --enforce
+class <%= migration_class_name %> < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+<% if options[:concurrently] -%>
+  disable_ddl_transaction!
+
+<% end -%>
+  def change
+    add_column :<%= table_name %>, :<%= public_column %>, :string
+    add_index :<%= table_name %>, :<%= public_column %>, unique: true<%= ", algorithm: :concurrently" if options[:concurrently] %>
+  end
+end

data/lib/generators/has_public_record/migration/templates/enforce.rb.tt

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+# Run after the backfill: fails if any row still has no id.
+class <%= migration_class_name %> < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    change_column_null :<%= table_name %>, :<%= public_column %>, false
+  end
+end

data/lib/has_public_record/configuration.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  # HasPublicRecord.configure { |c| c.default_length = 8 }
+  class Configuration
+    DEFAULT_ALPHABET = (("0".."9").to_a + ("A".."Z").to_a).join.freeze
+    # No 0/O or 1/I/L: easier to read aloud and to type from a screenshot.
+    UNAMBIGUOUS = "23456789ABCDEFGHJKMNPQRSTUVWXYZ"
+    PRESETS = { default: DEFAULT_ALPHABET, unambiguous: UNAMBIGUOUS }.freeze
+
+    attr_reader :default_length, :attempts_per_length, :max_length_bumps, :alphabet, :lengths
+
+    def initialize = reset!
+
+    def reset!
+      @default_length = 6        # starting length of the random part (the prefix is extra)
+      @attempts_per_length = 10  # random tries before the length grows by one
+      @max_length_bumps = 4      # growth steps before ExhaustedError
+      @alphabet = DEFAULT_ALPHABET
+      @lengths = {}.freeze       # per-table starting length: { "users" => 8 }
+      self
+    end
+
+    def default_length=(value)
+      @default_length = integer!(value, :default_length, min: 1)
+    end
+
+    def attempts_per_length=(value)
+      @attempts_per_length = integer!(value, :attempts_per_length, min: 1)
+    end
+
+    def max_length_bumps=(value)
+      @max_length_bumps = integer!(value, :max_length_bumps, min: 0)
+    end
+
+    def alphabet=(value)
+      if value.is_a?(Symbol)
+        value = PRESETS.fetch(value) { raise ArgumentError, "unknown alphabet preset #{value.inspect} (expected #{PRESETS.keys.join(', ')})" }
+      end
+      chars = value.to_s.each_char.uniq
+      raise ArgumentError, "alphabet needs at least 2 distinct characters" if chars.size < 2
+      if chars.any? { |char| char == SEPARATOR || char.match?(/\s/) }
+        raise ArgumentError, "alphabet cannot contain #{SEPARATOR.inspect} or whitespace"
+      end
+
+      @alphabet = chars.join.freeze
+    end
+
+    def lengths=(value)
+      @lengths = value.to_h.to_h { |table, length| [table.to_s, integer!(length, "lengths[#{table}]", min: 1)] }.freeze
+    end
+
+    private
+
+    def integer!(value, name, min:)
+      number = Integer(value)
+      raise ArgumentError, "#{name} must be at least #{min}" if number < min
+
+      number
+    rescue TypeError, ArgumentError => e
+      raise ArgumentError, e.message.start_with?(name.to_s) ? e.message : "#{name}: #{e.message}"
+    end
+  end
+end

data/lib/has_public_record/generator.rb

@@ -0,0 +1,95 @@
+# frozen_string_literal: true
+
+require "securerandom"
+require "concurrent/map"
+
+module HasPublicRecord
+  # Makes codes. Starts at a length (config.lengths[name] > the explicit length > config.default_length) and grows
+  # when random tries keep hitting taken codes, remembering the grown length per key (per process). Growth is
+  # announced with "grow.has_public_record" (key, table, column, from, to; table and column only for a scope).
+  #
+  # Whether a code is free is a callable, so nothing here needs a model:
+  #   Generator.generate(prefix: "inv", length: 8) { |id| !Redis.exists?(id) }
+  #   Generator.generate(prefix: "usr", scope: User.unscoped, column: :public_id)   # what models use
+  module Generator
+    EVENT = "grow.has_public_record"
+    LEARNED = Concurrent::Map.new # key => length reached after growing
+
+    class << self
+      def random(length:, alphabet: HasPublicRecord.config.alphabet)
+        chars = alphabet.chars
+        Array.new(length) { chars[SecureRandom.random_number(chars.size)] }.join
+      end
+
+      # Returns the full id ("usr_K3F9A2"), or the bare code when prefix is nil (then key: is required: it names
+      # the learned length). The block receives a candidate and answers true when it is free.
+      def generate(prefix: nil, length: nil, max_bumps: nil, key: nil, scope: nil, column: nil, &free)
+        prefix = check_prefix(prefix)
+        free, key, table = source_for(prefix, key, scope, column, free)
+        config = HasPublicRecord.config
+        limit = max_bumps || config.max_length_bumps
+        current = [starting_length(table || key, length), LEARNED[key].to_i].max
+        bumps = 0
+
+        loop do
+          config.attempts_per_length.times do
+            candidate = compose(prefix, random(length: current))
+            return candidate if free.call(candidate)
+          end
+
+          if bumps >= limit
+            raise ExhaustedError, "no unique code after #{bumps} length bumps (length #{current}) for #{key}"
+          end
+
+          bumps += 1
+          from = current
+          current += 1
+          learn(key, current)
+          ActiveSupport::Notifications.instrument(EVENT, key: key, table: table, column: column, from: from, to: current)
+        end
+      end
+
+      # The length a name (a table, or a standalone key) starts at.
+      def starting_length(name, explicit)
+        configured = HasPublicRecord.config.lengths[name.to_s]
+        [Integer(configured || explicit || HasPublicRecord.config.default_length), 1].max
+      end
+
+      # nil stays nil; anything else must be a well-formed prefix.
+      def check_prefix(prefix)
+        return nil if prefix.nil?
+
+        text = prefix.to_s
+        return text if PREFIX_FORMAT.match?(text)
+
+        raise InvalidPrefixError, "invalid prefix #{text.inspect} (expected #{PREFIX_FORMAT.inspect})"
+      end
+
+      def reset_learned! = LEARNED.clear
+
+      private
+
+      def source_for(prefix, key, scope, column, free)
+        if scope
+          raise ArgumentError, "give a block or scope:, not both" if free
+          raise ArgumentError, "scope: needs column:" unless column
+
+          table = scope.table_name.to_s
+          [->(candidate) { !scope.exists?(column => candidate) }, (key || "#{table}.#{column}").to_s, table]
+        else
+          raise ArgumentError, "give a block that says whether an id is free (or scope: and column:)" unless free
+
+          key ||= prefix
+          raise ArgumentError, "a generator with no prefix needs key: to name its growth" if key.nil?
+
+          [free, key.to_s, nil]
+        end
+      end
+
+      def compose(prefix, bare) = prefix ? "#{prefix}#{SEPARATOR}#{bare}" : bare
+
+      # Atomic per key: a smaller length never overwrites a larger one.
+      def learn(key, length) = LEARNED.compute(key) { |learned| [learned.to_i, length].max }
+    end
+  end
+end

data/lib/has_public_record/identifier.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  # Ids without a model: order numbers, API keys, Redis keys.
+  #
+  #   invoices = HasPublicRecord.identifier("inv", length: 8) { |id| !Invoice.exists?(number: id) }
+  #   invoices.generate            # => "inv_7K2QX9AB"
+  #   invoices.valid?("inv_7k2qx9ab")   # => true
+  #   invoices.bare("inv_7K2QX9AB")     # => "7K2QX9AB"
+  #
+  # The prefix is optional (then key: names the generator), but is not registered for HasPublicRecord.resolve:
+  # only models are. A prefix a model already declared is refused (best effort: models not loaded yet are unknown).
+  class Identifier
+    attr_reader :prefix, :key, :length, :max_bumps
+
+    def initialize(prefix = nil, length: nil, max_bumps: nil, key: nil, &free)
+      raise ArgumentError, "an identifier needs a block that says whether an id is free" unless free
+
+      @prefix = Generator.check_prefix(prefix)
+      if @prefix && (owner = HasPublicRecord.registry[@prefix])
+        raise DuplicatePrefixError, "prefix #{@prefix.inspect} is already used by #{owner}"
+      end
+
+      @key = (key || @prefix)&.to_s
+      raise ArgumentError, "an identifier with no prefix needs key:" if @key.nil?
+
+      @length = length
+      @max_bumps = max_bumps
+      @free = free
+    end
+
+    def generate = Generator.generate(prefix: prefix, length: length, max_bumps: max_bumps, key: key, &@free)
+
+    # The code without the prefix, case folded; nil when the ref belongs elsewhere.
+    def bare(ref)
+      canonical = HasPublicRecord.canonicalize(ref, prefix)
+      canonical && (prefix ? canonical.delete_prefix("#{prefix}#{SEPARATOR}") : canonical)
+    end
+
+    # Shaped like one of this identifier's ids: its prefix, then only alphabet characters.
+    def valid?(ref)
+      code = bare(ref)
+      !code.nil? && !code.empty? && code.each_char.all? { |char| HasPublicRecord.config.alphabet.include?(char) }
+    end
+  end
+end

data/lib/has_public_record/macro.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  # Extended onto ActiveRecord::Base, so `has_public_record` exists in every model.
+  module Macro
+    # prefix:    required, unique across the app ("usr")
+    # column:    where the id is stored (default :public_id)
+    # length:    starting length of the random part for this model (config.lengths[table] beats it)
+    # max_bumps: how far the length may grow (default config.max_length_bumps)
+    def has_public_record(prefix:, column: :public_id, length: nil, max_bumps: nil)
+      include HasPublicRecord::Model
+      declare_public_record(prefix: prefix, column: column, length: length, max_bumps: max_bumps)
+    end
+  end
+end

data/lib/has_public_record/maintenance.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  # What the rake tasks do, kept here so it can be tested and called from a console or a deploy script.
+  module Maintenance
+    Report = Data.define(:model, :table, :column, :total, :missing, :wrong_prefix, :wrong_samples, :duplicates, :problem) do
+      def ok? = problem.nil? && missing.zero? && wrong_prefix.zero? && duplicates.zero?
+    end
+
+    # How full a table's code space is. Growth starts well before 100%: at half full, one random try in two hits
+    # a taken code, so ten tries in a row fail about once in a thousand.
+    Capacity = Data.define(:length, :slots, :used) do
+      def fill = used.fdiv(slots)
+
+      def to_s
+        "#{length} chars = #{ActiveSupport::NumberHelper.number_to_human(slots)} ids, #{format('%.4f', fill * 100)}% used"
+      end
+    end
+
+    class << self
+      # Every model that declared has_public_record (one per table and column), optionally only the named one.
+      def models(only: nil)
+        found = ActiveRecord::Base.descendants.select do |model|
+          model.respond_to?(:public_record_options) && model.public_record_options && model.name.present? &&
+            !model.abstract_class? && model == model.base_class
+        end
+        found = found.uniq { |model| [model.table_name, model.public_id_column] }.sort_by(&:to_s)
+        only ? found.select { |model| model.name == only.to_s } : found
+      end
+
+      def reports(only: nil) = models(only: only).map { |model| report(model) }
+
+      # Rows without an id, rows whose id does not start with this model's prefix (legacy bare codes, or another
+      # model's prefix), and duplicated ids (only possible when the unique index is missing).
+      def report(model)
+        table = model.table_name
+        column = model.public_id_column
+        return broken(model, "table #{table} does not exist") unless model.table_exists?
+        unless model.column_names.include?(column.to_s)
+          return broken(model, "column #{column} is missing: rails g has_public_record:migration #{table}")
+        end
+
+        scope = model.base_class.unscoped
+        present = scope.where.not(column => [nil, ""])
+        wrong = present.where.not(with_prefix(model))
+        Report.new(model: model, table: table, column: column, total: scope.count,
+                   missing: scope.where(column => [nil, ""]).count,
+                   wrong_prefix: wrong.count, wrong_samples: wrong.limit(3).pluck(column),
+                   duplicates: present.group(column).having("COUNT(*) > 1").count.size, problem: nil)
+      end
+
+      # Moves stored ids from an old prefix to the model's declared one (change prefix: in the model first).
+      # Counts only unless apply: true. One transaction: all rows move or none do. Returns the rows concerned.
+      def rename_prefix(model, from:, apply: false, batch_size: 1000)
+        from = from.to_s
+        to = model.public_id_prefix
+        raise ArgumentError, "invalid prefix #{from.inspect}" unless PREFIX_FORMAT.match?(from)
+        raise ArgumentError, "#{from.inspect} is already the prefix #{model.name} declares" if from == to
+
+        column = model.public_id_column
+        old = "#{from}#{SEPARATOR}"
+        scope = model.base_class.unscoped.where(with_prefix(model, old))
+        count = scope.count
+        return count unless apply && count.positive?
+
+        model.transaction do
+          scope.find_each(batch_size: batch_size) do |record|
+            next unless record[column].start_with?(old) # the LIKE may be case-insensitive on some databases
+
+            renamed = "#{to}#{SEPARATOR}#{record[column].delete_prefix(old)}"
+            model.public_record_row_scope(record).update_all(column => renamed)
+          end
+        end
+        ActiveSupport::Notifications.instrument("rename.has_public_record",
+                                                table: model.table_name, column: column, from: from, to: to, count: count)
+        count
+      end
+
+      def capacity(model)
+        length = Generator.starting_length(model.table_name, model.public_record_options.length)
+        used = model.base_class.unscoped.where.not(model.public_id_column => [nil, ""]).count
+        Capacity.new(length: length, slots: HasPublicRecord.config.alphabet.size**length, used: used)
+      end
+
+      private
+
+      def broken(model, problem)
+        Report.new(model: model, table: model.table_name, column: model.public_id_column, total: 0, missing: 0,
+                   wrong_prefix: 0, wrong_samples: [], duplicates: 0, problem: problem)
+      end
+
+      # column LIKE 'usr\_%' with the underscore escaped (it is a wildcard in LIKE).
+      def with_prefix(model, text = "#{model.public_id_prefix}#{SEPARATOR}")
+        model.base_class.arel_table[model.public_id_column].matches("#{model.sanitize_sql_like(text)}%", "\\")
+      end
+    end
+  end
+end

data/lib/has_public_record/model.rb

@@ -0,0 +1,154 @@
+# frozen_string_literal: true
+
+require "active_support/concern"
+
+module HasPublicRecord
+  module Model
+    extend ActiveSupport::Concern
+
+    Options = Data.define(:prefix, :column, :length, :max_bumps)
+
+    included do
+      class_attribute :public_record_options, instance_writer: false, instance_predicate: false
+    end
+
+    class_methods do
+      # Called by the has_public_record macro. Touches no database table, so it is safe while the app boots.
+      def declare_public_record(prefix:, column:, length:, max_bumps:)
+        if public_record_options
+          raise HasPublicRecord::Error,
+                "#{name || inspect} already has has_public_record (prefix #{public_record_options.prefix.inspect}); " \
+                "declare it once, on the base class"
+        end
+
+        prefix = HasPublicRecord.registry.register(prefix, name)
+        self.public_record_options = Options.new(prefix: prefix, column: column.to_sym, length: length, max_bumps: max_bumps)
+
+        attr_readonly column # a public id never changes once created
+        before_create :assign_public_record
+        validate :validate_public_record_format, on: :create
+      end
+
+      def public_id_prefix = public_record_options.prefix
+      def public_id_column = public_record_options.column
+
+      # The stored form of a ref: "usr_K3F9A2". Accepts the bare code and any letter case (see HasPublicRecord.fold_case).
+      # nil for another model's id, for junk and for blank.
+      def normalize_public_id(ref) = HasPublicRecord.canonicalize(ref, public_id_prefix)
+
+      def public_id?(ref) = !normalize_public_id(ref).nil?
+
+      def find_by_public_id(ref)
+        id = normalize_public_id(ref)
+        id && find_by(public_id_column => id)
+      end
+
+      def find_by_public_id!(ref)
+        find_by_public_id(ref) ||
+          raise(ActiveRecord::RecordNotFound.new("Couldn't find #{name} with public id #{ref.inspect}", name, public_id_column, ref))
+      end
+
+      # What a controller passes in: User.from_param!(params[:id]).
+      def from_param(param) = find_by_public_id(param)
+      def from_param!(param) = find_by_public_id!(param)
+
+      # A relation for several ids at once; ids of another model are ignored.
+      def with_public_id(*refs)
+        where(public_id_column => refs.flatten.filter_map { |ref| normalize_public_id(ref) })
+      end
+
+      # A regexp for route constraints (no anchors, as Rails wants): resources :users, constraints: { id: User.public_id_format }
+      def public_id_format
+        alphabet = HasPublicRecord.config.alphabet
+        chars = Regexp.escape((alphabet.chars + alphabet.swapcase.chars).uniq.join)
+        /(?i:#{Regexp.escape(public_id_prefix)})#{SEPARATOR}[#{chars}]+/
+      end
+
+      # Rows that have no public id yet (after adding the column to an existing table).
+      def missing_public_id = where(public_id_column => nil)
+
+      # A fresh unused id for this model (what before_create uses).
+      def generate_public_id
+        HasPublicRecord::Generator.generate(scope: base_class.unscoped, column: public_id_column, prefix: public_id_prefix,
+                                            length: public_record_options.length, max_bumps: public_record_options.max_bumps)
+      end
+
+      # The one row a record is stored in, ignoring default scopes. Handles composite primary keys. A table with no
+      # primary key is refused: an empty condition would match every row.
+      def public_record_row_scope(record)
+        keys = Array(base_class.primary_key)
+        raise HasPublicRecord::Error, "#{name} has no primary key, so a single row cannot be addressed" if keys.empty?
+
+        base_class.unscoped.where(keys.zip(Array(record.id)).to_h)
+      end
+
+      # Gives every row without an id one; returns how many. Plain SQL updates: the column is readonly through the
+      # model, and callbacks and updated_at stay untouched.
+      def backfill_public_ids!(batch_size: 1000)
+        count = 0
+        missing_public_id.find_each(batch_size: batch_size) do |record|
+          count += public_record_row_scope(record).update_all(public_id_column => generate_public_id)
+        end
+        count
+      end
+
+      # insert_all / upsert_all skip callbacks, so rows get no id. This fills it in, one distinct id per row, and
+      # keeps ids you supplied:  User.insert_all(User.with_generated_public_ids(rows))
+      def with_generated_public_ids(rows)
+        column = public_id_column
+        taken = Set.new
+        rows.map do |row|
+          row = row.to_h.symbolize_keys
+          if row[column].present?
+            taken << row[column]
+            next row
+          end
+
+          id = generate_public_id
+          id = generate_public_id while taken.include?(id) # the database probe cannot see the rest of this batch
+          taken << id
+          row.merge(column => id)
+        end
+      end
+
+      # A stable id for a fixture label, for ERB in fixtures (Rails fixtures skip callbacks):
+      #   public_id: <%= User.fixture_public_id(:alice) %>
+      def fixture_public_id(label)
+        length = HasPublicRecord::Generator.starting_length(table_name, public_record_options.length)
+        HasPublicRecord.fixture_id(public_id_prefix, label, length: length)
+      end
+    end
+
+    # A copy is a new record: it must not carry the original's id into the unique index.
+    def initialize_dup(other)
+      super
+      self[self.class.public_id_column] = nil
+    end
+
+    def public_id_prefix = self.class.public_id_prefix
+
+    # "usr_K3F9A2" -> "K3F9A2"
+    def public_id_bare
+      value = self[self.class.public_id_column]
+      value&.delete_prefix("#{public_id_prefix}#{SEPARATOR}")
+    end
+
+    # Rails' url helpers use this: user_path(user) -> /users/usr_K3F9A2
+    def to_param = (self[self.class.public_id_column] if persisted?)
+
+    private
+
+    def assign_public_record
+      column = self.class.public_id_column
+      self[column] = self.class.generate_public_id if self[column].blank?
+    end
+
+    def validate_public_record_format
+      column = self.class.public_id_column
+      value = self[column]
+      return if value.blank? || self.class.normalize_public_id(value) == value
+
+      errors.add(column, :invalid)
+    end
+  end
+end

data/lib/has_public_record/railtie.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require "rails/railtie"
+
+module HasPublicRecord
+  # Loads the rake tasks. Generators need no hook: Rails finds lib/generators of every gem on the load path.
+  class Railtie < ::Rails::Railtie
+    rake_tasks { load File.expand_path("../tasks/has_public_record.rake", __dir__) }
+  end
+end

data/lib/has_public_record/registry.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  # prefix -> class NAME ("usr" => "User"). Names, not classes: a class object goes stale when Rails reloads code,
+  # a name is constantized fresh on lookup. Declaring the same prefix again from the same class (a reload) is a
+  # no-op; from a different class it raises, which is what makes prefixes unique.
+  class Registry
+    def initialize
+      @map = {}.freeze
+      @mutex = Mutex.new
+    end
+
+    def register(prefix, class_name)
+      prefix = prefix.to_s
+      unless PREFIX_FORMAT.match?(prefix)
+        raise InvalidPrefixError, "invalid prefix #{prefix.inspect} (expected #{PREFIX_FORMAT.inspect}: lower-case letters and digits, starting with a letter, 10 at most)"
+      end
+      raise InvalidPrefixError, "a model needs a class name to declare a prefix (anonymous class)" if class_name.blank?
+
+      @mutex.synchronize do
+        existing = @map[prefix]
+        if existing && existing != class_name
+          raise DuplicatePrefixError, "prefix #{prefix.inspect} is already used by #{existing}, so #{class_name} cannot use it"
+        end
+
+        @map = @map.merge(prefix => class_name).freeze
+      end
+      prefix
+    end
+
+    def [](prefix) = @map[prefix.to_s]
+    def prefix_for(class_name) = @map.key(class_name.to_s)
+    def to_h = @map
+    def clear! = @mutex.synchronize { @map = {}.freeze }
+  end
+end

data/lib/has_public_record/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module HasPublicRecord
+  VERSION = "0.1.0"
+end

data/lib/has_public_record.rb

@@ -0,0 +1,157 @@
+# frozen_string_literal: true
+
+require "active_support"
+require "active_support/core_ext/object/blank"
+require "active_support/number_helper"
+require "digest"
+require "zeitwerk"
+require_relative "has_public_record/version"
+
+loader = Zeitwerk::Loader.for_gem
+loader.ignore("#{__dir__}/generators") # Rails generators do not follow constant naming
+loader.ignore("#{__dir__}/tasks")      # rake files
+loader.ignore("#{__dir__}/has_public_record/version.rb") # required above
+loader.ignore("#{__dir__}/has_public_record/railtie.rb") # needs Rails, so it is required below, only under Rails
+loader.setup
+
+# Short, prefixed public ids: has_public_record prefix: "usr"  ->  usr_K3F9A2
+module HasPublicRecord
+  class Error < StandardError; end
+  class InvalidPrefixError < Error; end
+  class DuplicatePrefixError < Error; end
+  class ExhaustedError < Error; end
+
+  SEPARATOR = "_"
+  PREFIX_FORMAT = /\A[a-z][a-z0-9]{0,9}\z/
+  REF_FORMAT = /\A([a-z][a-z0-9]{0,9})#{SEPARATOR}(\S+)\z/i # any letter case: people type ids
+  Parsed = Data.define(:prefix, :bare)
+
+  @config = Configuration.new
+  @registry = Registry.new
+  @models_loaded = false
+
+  class << self
+    attr_reader :config, :registry
+
+    def configure
+      yield(config)
+      config
+    end
+
+    # "usr_K3F9A2" -> Parsed(prefix: "usr", bare: "K3F9A2"); nil when it is not shaped like a ref.
+    def parse(ref)
+      match = REF_FORMAT.match(ref.to_s.strip) or return nil
+      Parsed.new(prefix: match[1].downcase, bare: fold_case(match[2]))
+    end
+
+    def valid?(ref) = !parse(ref).nil?
+
+    # Ids are stored in one case. When the alphabet has only upper-case (or only lower-case) letters, anything typed
+    # is folded to it; an alphabet with both cases is left exactly as typed.
+    def fold_case(text)
+      alphabet = config.alphabet
+      if alphabet == alphabet.upcase then text.upcase
+      elsif alphabet == alphabet.downcase then text.downcase
+      else text
+      end
+    end
+
+    # The stored form ("usr_K3F9A2") of a ref that belongs to prefix; the bare code is accepted. nil for a ref of
+    # another prefix or a blank one. With no prefix, just the folded text.
+    def canonicalize(ref, prefix)
+      text = ref.to_s.strip
+      return nil if text.empty?
+      return fold_case(text) unless prefix
+
+      head, separator, tail = text.partition(SEPARATOR)
+      if separator.empty?
+        "#{prefix}#{SEPARATOR}#{fold_case(head)}"
+      elsif head.downcase == prefix && !tail.empty?
+        "#{prefix}#{SEPARATOR}#{fold_case(tail)}"
+      end
+    end
+
+    # { "usr" => "User", ... } for every model declared so far.
+    def prefixes = registry.to_h
+
+    # The prefix a model (class, or class name) declared, or nil.
+    def prefix_for(model) = registry.prefix_for(model.is_a?(Module) ? model.name : model)
+
+    # The model class a ref belongs to, or nil (bad shape, unknown prefix, class gone).
+    def model_for(ref)
+      parsed = parse(ref) or return nil
+      name = registry[parsed.prefix]
+      if name.nil?
+        load_models!
+        name = registry[parsed.prefix]
+      end
+      name&.safe_constantize
+    end
+
+    # The record a ref points at, or nil. resolve! raises ActiveRecord::RecordNotFound.
+    def resolve(ref)
+      model = model_for(ref) or return nil
+      model.find_by_public_id(ref)
+    end
+
+    def resolve!(ref)
+      resolve(ref) || raise(ActiveRecord::RecordNotFound, "Couldn't find a record for public id #{ref.inspect}")
+    end
+
+    # Mixed refs (an audit log, polymorphic references) in one query per model, not one per ref.
+    # Returns { ref => record } for the refs that resolved, keyed by the ref as given. Bare codes cannot be
+    # resolved without a model, so they are skipped.
+    def resolve_many(*refs)
+      found = {}
+      refs.flatten.compact.uniq.group_by { |ref| model_for(ref) }.each do |model, list|
+        next unless model
+
+        by_id = model.with_public_id(list).index_by { |record| record[model.public_id_column] }
+        list.each do |ref|
+          record = by_id[model.normalize_public_id(ref)]
+          found[ref] = record if record
+        end
+      end
+      found
+    end
+
+    # Makes an id without a model; see Generator and Identifier.
+    #   HasPublicRecord.generate("inv", length: 8) { |id| !Invoice.exists?(number: id) }
+    def generate(prefix = nil, **options, &block) = Generator.generate(**{ prefix: prefix }.merge(options), &block)
+
+    def identifier(prefix = nil, **options, &free) = Identifier.new(prefix, **options, &free)
+
+    # A stable id for a label, for fixtures and seeds: the same label always gives the same id.
+    def fixture_id(prefix, label, length: config.default_length)
+      number = Digest::SHA512.hexdigest("#{prefix}:#{label}").to_i(16)
+      chars = config.alphabet.chars
+      bare = Array.new(length) do
+        number, digit = number.divmod(chars.size)
+        chars[digit]
+      end.join
+      prefix ? "#{prefix}#{SEPARATOR}#{bare}" : bare
+    end
+
+    # For specs: forget every declaration, learned length and setting.
+    def reset!
+      registry.clear!
+      Generator.reset_learned!
+      config.reset!
+      @models_loaded = false
+    end
+
+    private
+
+    # Models register when their class loads. With lazy autoloading (development, tests) a ref for a model nobody
+    # touched yet would not resolve, so on the first unknown prefix everything is loaded once.
+    def load_models!
+      return if @models_loaded
+
+      @models_loaded = true
+      Zeitwerk::Loader.eager_load_all
+    end
+  end
+end
+
+ActiveSupport.on_load(:active_record) { extend HasPublicRecord::Macro }
+require_relative "has_public_record/railtie" if defined?(Rails::Railtie)

data/lib/tasks/has_public_record.rake

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+namespace :has_public_record do
+  # Models register their prefix when their class loads, so load them all first.
+  task prepare: :environment do
+    Rails.application.eager_load!
+  end
+
+  def hpr_models
+    only = ENV["MODEL"].presence
+    models = HasPublicRecord::Maintenance.models(only: only)
+    abort "No model#{" named #{only}" if only} declares has_public_record." if models.empty?
+    models
+  end
+
+  desc "Give every row without a public id one (MODEL=User to limit)"
+  task backfill: :prepare do
+    hpr_models.each do |model|
+      problem = HasPublicRecord::Maintenance.report(model).problem
+      next puts("#{model.name}: skipped, #{problem}") if problem
+
+      puts "#{model.name}: #{model.backfill_public_ids!} rows given an id"
+    end
+  end
+
+  desc "Missing ids, wrong prefixes, duplicates and capacity (MODEL=User to limit). Exits 1 when anything is off"
+  task report: :prepare do
+    reports = HasPublicRecord::Maintenance.reports(only: ENV["MODEL"].presence)
+    abort "No model declares has_public_record." if reports.empty?
+
+    reports.each do |r|
+      label = "#{r.model.name} (#{r.table}.#{r.column}, prefix #{r.model.public_id_prefix})"
+      if r.problem
+        puts "#{label}: #{r.problem}"
+      elsif r.ok?
+        puts "#{label}: #{r.total} rows, ok; #{HasPublicRecord::Maintenance.capacity(r.model)}"
+      else
+        puts "#{label}: #{r.total} rows, #{r.missing} missing, #{r.wrong_prefix} wrong prefix" \
+             "#{" (e.g. #{r.wrong_samples.join(', ')})" if r.wrong_samples.any?}, #{r.duplicates} duplicated; " \
+             "#{HasPublicRecord::Maintenance.capacity(r.model)}"
+      end
+    end
+    exit 1 unless reports.all?(&:ok?)
+  end
+
+  desc "Rename stored ids from an old prefix to the model's declared one: MODEL=User FROM=old [APPLY=1]"
+  task rename: :prepare do
+    from = ENV["FROM"].presence or abort "Give the old prefix: FROM=old"
+    ENV["MODEL"].presence or abort "Give the model: MODEL=User"
+    model = hpr_models.first
+    apply = ENV["APPLY"].present?
+
+    count = HasPublicRecord::Maintenance.rename_prefix(model, from: from, apply: apply)
+    if apply
+      puts "#{model.name}: #{count} ids renamed #{from}_ -> #{model.public_id_prefix}_"
+    else
+      puts "#{model.name}: #{count} ids would be renamed #{from}_ -> #{model.public_id_prefix}_ (dry run; add APPLY=1)"
+    end
+  end
+end

metadata

@@ -0,0 +1,120 @@
+--- !ruby/object:Gem::Specification
+name: has_public_record
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Baden
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+- !ruby/object:Gem::Dependency
+  name: railties
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+- !ruby/object:Gem::Dependency
+  name: zeitwerk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.6'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2.6'
+description: has_public_record gives a model a random prefixed public id that grows
+  in length as the table fills.
+email:
+- baden.rushworth@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- README.md
+- lib/generators/has_public_record/install/install_generator.rb
+- lib/generators/has_public_record/install/templates/initializer.rb.tt
+- lib/generators/has_public_record/migration/migration_generator.rb
+- lib/generators/has_public_record/migration/templates/add.rb.tt
+- lib/generators/has_public_record/migration/templates/enforce.rb.tt
+- lib/has_public_record.rb
+- lib/has_public_record/configuration.rb
+- lib/has_public_record/generator.rb
+- lib/has_public_record/identifier.rb
+- lib/has_public_record/macro.rb
+- lib/has_public_record/maintenance.rb
+- lib/has_public_record/model.rb
+- lib/has_public_record/railtie.rb
+- lib/has_public_record/registry.rb
+- lib/has_public_record/version.rb
+- lib/tasks/has_public_record.rake
+homepage: https://github.com/BadenRushworth/has_public_record
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/BadenRushworth/has_public_record
+  source_code_uri: https://github.com/BadenRushworth/has_public_record
+  changelog_uri: https://github.com/BadenRushworth/has_public_record/blob/main/CHANGELOG.md
+  bug_tracker_uri: https://github.com/BadenRushworth/has_public_record/issues
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.16
+specification_version: 4
+summary: Short, prefixed, URL-safe public ids for ActiveRecord models (usr_K3F9A2).
+test_files: []