snowflake_id 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 35ccd9038bfa98f7542f483829f7c3d810636e9d9bd7c1bf26e3233fe1b1d287
+  data.tar.gz: 8569b7d8c6d63f406ed02c294006bfa065e63d1bc27e0931354f39f880ca487d
+SHA512:
+  metadata.gz: b0b86cbebcaef643798a8d29ab5aafbcfe168f4e8233e438f5478e24710e7363ade49e661ba94eacdb4cf449af2bf005b645a91de3c4b6d3f087c490f52b6703
+  data.tar.gz: fc337bd840a6b65ad48d597378508e0fe4f95ed42bf0bcff16da3fe7fba9440922d86b8fc608749c65857f028e1e04be69a20f1ab120e63dc4e2a3ab9a50a24e

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright Luiz Eduardo Kowalski
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,143 @@
+# SnowflakeId
+
+A Rails plugin that provides Snowflake-like IDs for your ActiveRecord models with minimal configuration.
+
+Snowflake IDs are 64-bit integers that contain:
+- **48 bits** for millisecond-level timestamp
+- **16 bits** for sequence data (includes hashed table name + secret salt + sequence number)
+
+This ensures globally unique, time-sortable IDs that don't reveal the total count of records in your database.
+
+## Features
+
+- **Transparent** - Just use `t.snowflake` and it works automatically
+- **Automatic database setup** - Hooks into `db:migrate` and `db:prepare` tasks to ensure everything is set up.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem "snowflake_id"
+```
+
+And then execute:
+```bash
+rails generate snowflake_id:install
+```
+
+## Quick Start
+
+**That's it!** Just use `t.snowflake` in your migrations and everything works automatically.
+
+### For Snowflake ID as primary key:
+```ruby
+class CreateUsers < ActiveRecord::Migration[8.0]
+  def change
+    create_table :users, id: false do |t|
+      t.snowflake :id, primary_key: true  # Snowflake primary key
+      t.string :name
+      t.timestamps
+    end
+  end
+end
+```
+
+**Note**: When using `t.snowflake :id` directly, Rails will complain about redefining the primary key. Always use `create_table :table_name, id: false` when you want a snowflake primary key.
+
+### For additional snowflake columns (non-primary key):
+```ruby
+class CreatePosts < ActiveRecord::Migration[8.0]
+  def change
+    create_table :posts do |t|
+      t.string :title
+      t.text :content
+      t.snowflake :uid  # Additional snowflake column
+      t.timestamps
+    end
+  end
+end
+```
+
+### Generator Support
+
+You can also use Snowflake helper in Rails generators:
+
+```bash
+# Generate a model with a snowflake field
+rails generate model Post title:string uid:snowflake
+
+# This will create a migration like:
+# create_table :posts do |t|
+#   t.string :title
+#   t.snowflake :uid
+#   t.timestamps
+# end
+```
+
+### Working with Snowflake IDs
+
+There's nothing else to be done at this point. `t.snowflake` columns will automatically get unique IDs on record creation, and they are just a `:bigint` column in the database.
+At this point, you can use them like any other integer ID.
+
+```ruby
+user = User.create!(name: "Alice")
+user.id  # => 115198501587747344
+
+# Convert ID back to timestamp
+SnowflakeId::Generator.to_time(user.id)
+# => 2024-12-25 10:15:42 UTC
+
+# Generate ID for specific timestamp
+SnowflakeId::Generator.at(1.hour.ago)
+# => 1766651542000012345
+```
+
+### Database Integration
+
+The gem automatically hooks into these Rails tasks:
+- `db:migrate`
+- `db:schema:load`
+- `db:structure:load`
+- `db:seed`
+
+## Migration from Standard IDs
+
+If you have existing models with standard Rails IDs, you'll need to run a migration to convert them to Snowflake IDs.
+
+```ruby
+execute("ALTER TABLE table_name ALTER COLUMN id SET DEFAULT timestamp_id('table_name')")
+```
+
+⚠️ **Warning**: This is a complex operation that may require downtime and careful planning.
+
+## Requirements
+
+- **Database**: PostgreSQL (uses PostgreSQL-specific functions)
+- **Rails**: 7.0+ (may work with earlier versions)
+- **Ruby**: 3.0+
+
+## How it Works
+
+1. **Function Creation**: Creates a PostgreSQL `timestamp_id()` function
+2. **Sequence Management**: Auto-creates sequences for each table (`table_name_id_seq`)
+3. **ID Generation**: Uses timestamp + hashed sequence for uniqueness
+4. **Rails Integration**: Hooks into model lifecycle and database tasks
+
+
+## Contributing
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/my-new-feature`)
+3. Add tests for your changes
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin feature/my-new-feature`)
+6. Create a Pull Request
+
+## License
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+
+## Acknowledgements
+
+The implementation of Snowflake-like ids was initially done by [Mastodon](https://github.com/mastodon/mastodon/blob/06803422da3794538cd9cd5c7ccd61a0694ef921/lib/mastodon/snowflake.rb)

data/Rakefile

@@ -0,0 +1,15 @@
+require "bundler/setup"
+
+APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
+load "rails/tasks/engine.rake"
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+task default: %i[test]
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end

data/lib/generators/snowflake_id/install/install_generator.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module SnowflakeId::Generators
+  class InstallGenerator < Rails::Generators::Base
+    include ActiveRecord::Generators::Migration
+
+    TEMPLATES = File.join(File.dirname(__FILE__), "templates")
+    source_paths << TEMPLATES
+
+    desc "Install SnowflakeId by creating a migration to setup the timestamp_id function"
+
+    def create_migration_file
+      migration_template "install_snowflake_id.rb.erb", File.join(db_migrate_path, "install_snowflake_id.rb")
+    end
+
+    private
+
+    def migration_version
+      "[#{ActiveRecord::VERSION::STRING.to_f}]"
+    end
+  end
+end

data/lib/generators/snowflake_id/install/templates/install_snowflake_id.rb.erb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class InstallSnowflakeId < ActiveRecord::Migration<%= migration_version %>
+  def up
+    # Create the timestamp_id PostgreSQL function
+    SnowflakeId::Generator.define_timestamp_id
+  end
+
+  def down
+    # Remove the timestamp_id function
+    execute "DROP FUNCTION IF EXISTS timestamp_id(text)"
+  end
+end

data/lib/snowflake_id/column_methods.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module SnowflakeId
+  module ColumnMethods
+    def snowflake(name, **options)
+      if name == :id && !options[:primary_key]
+        raise ArgumentError, "Cannot use t.snowflake :id directly. Use `create_table` with `id: false` and then t.snowflake :id, primary_key: true"
+      end
+
+      table_name = @name
+
+      unless table_name
+        raise ArgumentError, "Could not determine table name for snowflake column. Make sure you're using it within a create_table block."
+      end
+
+      options[:default] = -> { "timestamp_id('#{table_name}'::text)" }
+
+      column(name, :bigint, **options)
+    end
+  end
+end

data/lib/snowflake_id/database_tasks.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+# Hook into Rails database tasks to ensure snowflake sequences exist
+def ensure_snowflake_sequences
+  return unless defined?(ActiveRecord::Base)
+
+  begin
+    if ActiveRecord::Base.connection.adapter_name == "PostgreSQL"
+      Rails.logger.debug "SnowflakeId: Ensure sequences exist for `timestamp_id` columns"
+      SnowflakeId::Generator.ensure_id_sequences_exist
+    end
+  rescue ActiveRecord::NoDatabaseError, ActiveRecord::ConnectionNotEstablished
+    Rails.logger.warn "SnowflakeId: Could not ensure sequences: #{e.message}"
+  end
+end
+
+# Enhance existing Rails database tasks by adding our hook to them
+if Rake::Task.task_defined?("db:migrate")
+  Rake::Task["db:migrate"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:schema:load")
+  Rake::Task["db:schema:load"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:structure:load")
+  Rake::Task["db:structure:load"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:seed")
+  Rake::Task["db:seed"].enhance do
+    ensure_snowflake_sequences
+  end
+end

data/lib/snowflake_id/generator.rb

@@ -0,0 +1,188 @@
+# frozen_string_literal: true
+
+# Copied from https://github.com/mastodon/mastodon/blob/06803422da3794538cd9cd5c7ccd61a0694ef921/lib/mastodon/snowflake.rb
+
+module SnowflakeId
+  module Generator
+    DEFAULT_REGEX = /timestamp_id\('(?<seq_prefix>\w+)'/
+
+    class Callbacks
+      def self.around_create(record)
+        now = Time.now.utc
+
+        if record.created_at.nil? || record.created_at >= now || record.created_at == record.updated_at || record.override_timestamps
+          yield
+        else
+          record.id = SnowflakeId::Generator.at(record.created_at)
+          tries     = 0
+
+          begin
+            yield
+          rescue ActiveRecord::RecordNotUnique
+            raise if tries > 100
+
+            tries     += 1
+            record.id += rand(100)
+
+            retry
+          end
+        end
+      end
+    end
+
+    class << self
+      # Our ID will be composed of the following:
+      # 6 bytes (48 bits) of millisecond-level timestamp
+      # 2 bytes (16 bits) of sequence data
+      #
+      # The 'sequence data' is intended to be unique within a
+      # given millisecond, yet obscure the 'serial number' of
+      # this row.
+      #
+      # To do this, we hash the following data:
+      # * Table name (if provided, skipped if not)
+      # * Secret salt (should not be guessable)
+      # * Timestamp (again, millisecond-level granularity)
+      #
+      # We then take the first two bytes of that value, and add
+      # the lowest two bytes of the table ID sequence number
+      # (`table_name`_id_seq). This means that even if we insert
+      # two rows at the same millisecond, they will have
+      # distinct 'sequence data' portions.
+      #
+      # If this happens, and an attacker can see both such IDs,
+      # they can determine which of the two entries was inserted
+      # first, but not the total number of entries in the table
+      # (even mod 2**16).
+      #
+      # The table name is included in the hash to ensure that
+      # different tables derive separate sequence bases so rows
+      # inserted in the same millisecond in different tables do
+      # not reveal the table ID sequence number for one another.
+      #
+      # The secret salt is included in the hash to ensure that
+      # external users cannot derive the sequence base given the
+      # timestamp and table name, which would allow them to
+      # compute the table ID sequence number.
+      def define_timestamp_id
+        return if already_defined?
+
+        connection.execute(sanitized_timestamp_id_sql)
+      end
+
+      def ensure_id_sequences_exist
+        # Find tables using timestamp IDs.
+        connection.tables.each do |table|
+          ensure_id_sequences_exist_for(table)
+        end
+      end
+
+      def ensure_id_sequences_exist_for(table_name)
+        # We're only concerned with "id" columns.
+        id_col = connection.columns(table_name).find { |col| col.name == "id" }
+        return unless id_col
+
+        # And only those that are using timestamp_id.
+        data = DEFAULT_REGEX.match(id_col.default_function)
+        return unless data
+
+        seq_name = "#{data[:seq_prefix]}_id_seq"
+
+        # If we were on Postgres 9.5+, we could do CREATE SEQUENCE IF
+        # NOT EXISTS, but we can't depend on that. Instead, catch the
+        # possible exception and ignore it.
+        # Note that seq_name isn't a column name, but it's a
+        # relation, like a column, and follows the same quoting rules
+        # in Postgres.
+        connection.execute(<<~SQL)
+          DO $$
+            BEGIN
+              CREATE SEQUENCE #{connection.quote_column_name(seq_name)};
+            EXCEPTION WHEN duplicate_table THEN
+              -- Do nothing, we have the sequence already.
+            END
+          $$ LANGUAGE plpgsql;
+        SQL
+      rescue StandardError => e
+        Rails.logger.warn "SnowflakeId: Could not ensure sequence for #{table_name}: #{e.message}"
+      end
+
+      def at(timestamp, with_random: true)
+        id  = timestamp.to_i * 1000
+        id += rand(1000) if with_random
+        id <<= 16
+        id += rand(2**16) if with_random
+        id
+      end
+
+      def to_time(id)
+        Time.at((id >> 16) / 1000).utc
+      end
+
+      private
+
+      def already_defined?
+        connection.execute(<<~SQL.squish).values.first.first
+          SELECT EXISTS(
+            SELECT * FROM pg_proc WHERE proname = 'timestamp_id'
+          );
+        SQL
+      end
+
+      def sanitized_timestamp_id_sql
+        ActiveRecord::Base.sanitize_sql_array(timestamp_id_sql_array)
+      end
+
+      def timestamp_id_sql_array
+        [ timestamp_id_sql_string, { random_string: SecureRandom.hex(16) } ]
+      end
+
+      def timestamp_id_sql_string
+        <<~SQL
+          CREATE OR REPLACE FUNCTION timestamp_id(table_name text)
+          RETURNS bigint AS
+          $$
+            DECLARE
+              time_part bigint;
+              sequence_base bigint;
+              tail bigint;
+            BEGIN
+              time_part := (
+                -- Get the time in milliseconds
+                ((date_part('epoch', now()) * 1000))::bigint
+                -- And shift it over two bytes
+                << 16);
+
+              sequence_base := (
+                'x' ||
+                -- Take the first two bytes (four hex characters)
+                substr(
+                  -- Of the MD5 hash of the data we documented
+                  md5(table_name || :random_string || time_part::text),
+                  1, 4
+                )
+              -- And turn it into a bigint
+              )::bit(16)::bigint;
+
+              -- Finally, add our sequence number to our base, and chop
+              -- it to the last two bytes
+              tail := (
+                (sequence_base + nextval(table_name || '_id_seq'))
+                & 65535);
+
+              -- Return the time part and the sequence part. OR appears
+              -- faster here than addition, but they're equivalent:
+              -- time_part has no trailing two bytes, and tail is only
+              -- the last two bytes.
+              RETURN time_part | tail;
+            END
+          $$ LANGUAGE plpgsql VOLATILE;
+        SQL
+      end
+
+      def connection
+        ActiveRecord::Base.connection
+      end
+    end
+  end
+end

data/lib/snowflake_id/railtie.rb

@@ -0,0 +1,19 @@
+module SnowflakeId
+  class Railtie < ::Rails::Railtie
+    initializer "snowflake_id.register_field_type" do
+      ActiveSupport.on_load(:active_record) do
+        ActiveRecord::ConnectionAdapters::TableDefinition.prepend(SnowflakeId::ColumnMethods)
+
+        if defined?(ActiveRecord::ConnectionAdapters::PostgreSQLAdapter)
+          ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:snowflake] = { name: "bigint" }
+        else
+          raise "SnowflakeId: Unsupported database adapter. Only PostgreSQL is supported."
+        end
+      end
+    end
+
+    rake_tasks do
+      load "snowflake_id/database_tasks.rb"
+    end
+  end
+end

data/lib/snowflake_id/version.rb

@@ -0,0 +1,3 @@
+module SnowflakeId
+  VERSION = "0.1.0"
+end

data/lib/snowflake_id.rb

@@ -0,0 +1,10 @@
+require "zeitwerk"
+
+loader = Zeitwerk::Loader.for_gem
+loader.ignore("#{__dir__}/generators")
+loader.ignore("#{__dir__}/snowflake_id/database_tasks.rb")
+loader.setup
+
+require "snowflake_id/railtie"
+
+module SnowflakeId; end

data/lib/tasks/snowflake_id_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :snowflake_id do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,84 @@
+--- !ruby/object:Gem::Specification
+name: snowflake_id
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Luiz Eduardo Kowalski
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+- !ruby/object:Gem::Dependency
+  name: zeitwerk
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.7'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.7'
+description: A Rails plugin that provides a simple way to generate unique Snowflake
+  IDs for your ActiveRecord models.
+email:
+- luizeduardokowalski@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE
+- README.md
+- Rakefile
+- lib/generators/snowflake_id/install/install_generator.rb
+- lib/generators/snowflake_id/install/templates/install_snowflake_id.rb.erb
+- lib/snowflake_id.rb
+- lib/snowflake_id/column_methods.rb
+- lib/snowflake_id/database_tasks.rb
+- lib/snowflake_id/generator.rb
+- lib/snowflake_id/railtie.rb
+- lib/snowflake_id/version.rb
+- lib/tasks/snowflake_id_tasks.rake
+homepage: https://github.com/luizkowalski/snowflake_id/
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/luizkowalski/snowflake_id/
+  source_code_uri: https://github.com/luizkowalski/snowflake_id
+  changelog_uri: https://github.com/luizkowalski/snowflake_id/CHAGNES.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.7.2
+specification_version: 4
+summary: Generate Snowflake IDs in Rails models.
+test_files: []