rails-snowflake 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 8f0727305e2203bd51c71c99e038f7902d0abdf9f283bcf0dca93495c5f851bf
+  data.tar.gz: 751f6d5c2e3329ad5be5c6104f8cc164af707181e01f3d428673736980f0539e
+SHA512:
+  metadata.gz: 05771bac80703b818c3c86021624f0d68ce23ceb5c34cd2423de1d2ce65674cc3824540f706d56593b0f97b6bc6afcd7c01b7e542e18a24a625a777323979957
+  data.tar.gz: 287eccd9577239bb6d2543383c3979483f52dc6272e6077681b56f7b86cf0ef94abbe8a11da01e7b90e7254c506d1e732253d048d003b2e79c345fc726ea8c13

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright Luiz Eduardo Kowalski
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,145 @@
+# Rails::Snowflake
+
+[![CI](https://github.com/luizkowalski/snowflake_id/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/luizkowalski/snowflake_id/actions/workflows/ci.yml)
+
+A Rails plugin that provides Snowflake-like IDs for your ActiveRecord models with minimal configuration.
+
+Snowflake IDs are 64-bit integers that contain:
+- **48 bits** for millisecond-level timestamp
+- **16 bits** for sequence data (includes hashed table name + secret salt + sequence number)
+
+This ensures globally unique, time-sortable IDs that don't reveal the total count of records in your database.
+
+## Features
+
+- **Transparent** - Just use `t.snowflake` and it works automatically
+- **Automatic database setup** - Hooks into `db:migrate` and `db:prepare` tasks to ensure everything is set up.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem "rails-snowflake"
+```
+
+And then execute the installer (note the underscore):
+```bash
+rails generate rails_snowflake:install
+```
+
+## Quick Start
+
+**That's it!** Just use `t.snowflake` in your migrations and everything works automatically.
+
+### For Snowflake ID as primary key:
+```ruby
+class CreateUsers < ActiveRecord::Migration[8.0]
+  def change
+    create_table :users, id: false do |t|
+      t.snowflake :id, primary_key: true  # Snowflake primary key
+      t.string :name
+      t.timestamps
+    end
+  end
+end
+```
+
+**Note**: When using `t.snowflake :id` directly, Rails will complain about redefining the primary key. Always use `create_table :table_name, id: false` when you want a snowflake primary key.
+
+### For additional snowflake columns (non-primary key):
+```ruby
+class CreatePosts < ActiveRecord::Migration[8.0]
+  def change
+    create_table :posts do |t|
+      t.string :title
+      t.text :content
+      t.snowflake :uid  # Additional snowflake column
+      t.timestamps
+    end
+  end
+end
+```
+
+### Generator Support
+
+You can also use Snowflake helper in Rails generators:
+
+```bash
+# Generate a model with a snowflake field
+rails generate model Post title:string uid:snowflake
+
+# This will create a migration like:
+# create_table :posts do |t|
+#   t.string :title
+#   t.snowflake :uid
+#   t.timestamps
+# end
+```
+
+### Working with Snowflake IDs
+
+There's nothing else to be done at this point. `t.snowflake` columns will automatically get unique IDs on record creation, and they are just a `:bigint` column in the database.
+At this point, you can use them like any other integer ID.
+
+```ruby
+user = User.create!(name: "Alice")
+user.id  # => 115198501587747344
+
+# Convert ID back to timestamp
+Rails::Snowflake::Id.to_time(user.id)
+# => 2024-12-25 10:15:42 UTC
+
+# Generate ID for specific timestamp
+Rails::Snowflake::Id.at(1.hour.ago)
+# => 1766651542000012345
+```
+
+### Database Integration
+
+The gem automatically hooks into these Rails tasks:
+- `db:migrate`
+- `db:schema:load`
+- `db:structure:load`
+- `db:seed`
+
+## Migration from Standard IDs
+
+If you have existing models with standard Rails IDs, you'll need to run a migration to convert them to Snowflake IDs.
+
+```ruby
+execute("ALTER TABLE table_name ALTER COLUMN id SET DEFAULT timestamp_id('table_name')")
+```
+
+⚠️ **Warning**: This is a complex operation that may require downtime and careful planning.
+
+## Requirements
+
+- **Database**: PostgreSQL (uses PostgreSQL-specific functions)
+- **Rails**: 7.2+ (may work with earlier versions)
+- **Ruby**: 3.2+
+
+## How it Works
+
+1. **Function Creation**: Creates a PostgreSQL `timestamp_id()` function
+2. **Sequence Management**: Auto-creates sequences for each table (`table_name_id_seq`)
+3. **ID Generation**: Uses timestamp + hashed sequence for uniqueness
+4. **Rails Integration**: Hooks into model lifecycle and database tasks
+
+
+## Contributing
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/my-new-feature`)
+3. Add tests for your changes
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin feature/my-new-feature`)
+6. Create a Pull Request
+
+## License
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+
+## Acknowledgements
+
+The implementation of Snowflake-like ids was initially done by [Mastodon](https://github.com/mastodon/mastodon/blob/06803422da3794538cd9cd5c7ccd61a0694ef921/lib/mastodon/snowflake.rb)

data/Rakefile

@@ -0,0 +1,15 @@
+require "bundler/setup"
+
+APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
+load "rails/tasks/engine.rake"
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+task default: %i[test]
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end

data/lib/rails/snowflake/column_methods.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Rails
+  module Snowflake
+    module ColumnMethods
+    def snowflake(name, **options)
+      if name == :id && !options[:primary_key]
+        raise ArgumentError, "Cannot use t.snowflake :id directly. Use `create_table` with `id: false` and then `t.snowflake :id, primary_key: true`"
+      end
+
+      table_name = @name
+
+      unless table_name
+        raise ArgumentError, "Could not determine table name for Snowflake column. Make sure you're using it within a `create_table` block."
+      end
+
+      options[:default] = -> { "timestamp_id('#{table_name}'::text)" }
+
+      column(name, :bigint, **options)
+    end
+    end
+  end
+end

data/lib/rails/snowflake/database_tasks.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+# Hook into Rails database tasks to ensure snowflake sequences exist
+def ensure_snowflake_sequences
+  return unless defined?(ActiveRecord::Base)
+
+  begin
+    if ActiveRecord::Base.connection.adapter_name == "PostgreSQL"
+      Rails.logger.debug "Rails::Snowflake: Ensure sequences exist for `timestamp_id` columns"
+      Rails::Snowflake::Id.ensure_id_sequences_exist
+    end
+  rescue ActiveRecord::NoDatabaseError, ActiveRecord::ConnectionNotEstablished
+    Rails.logger.warn "Rails::Snowflake: Could not ensure sequences: #{e.message}"
+  end
+end
+
+# Enhance existing Rails database tasks by adding our hook to them
+if Rake::Task.task_defined?("db:migrate")
+  Rake::Task["db:migrate"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:schema:load")
+  Rake::Task["db:schema:load"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:structure:load")
+  Rake::Task["db:structure:load"].enhance do
+    ensure_snowflake_sequences
+  end
+end
+
+if Rake::Task.task_defined?("db:seed")
+  Rake::Task["db:seed"].enhance do
+    ensure_snowflake_sequences
+  end
+end

data/lib/rails/snowflake/generators/install/install_generator.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module Rails
+  module Snowflake
+    module Generators
+      class InstallGenerator < ::Rails::Generators::Base
+        include ActiveRecord::Generators::Migration
+
+        # Ensure the Thor/Rails namespace is rails_snowflake:install
+        def self.namespace(name = nil)
+          if name
+            super
+          else
+            @namespace ||= "rails_snowflake:install"
+          end
+        end
+
+        TEMPLATES = File.join(File.dirname(__FILE__), "templates")
+        source_paths << TEMPLATES
+
+        desc "Install Rails::Snowflake by creating a migration to setup the timestamp_id function"
+
+        def create_migration_file
+          migration_template "install_snowflake_id.rb.erb", File.join(db_migrate_path, "install_snowflake_id.rb")
+        end
+
+        private
+
+        def migration_version
+          "[#{ActiveRecord::VERSION::STRING.to_f}]"
+        end
+      end
+    end
+  end
+end

data/lib/rails/snowflake/generators/install/templates/install_snowflake_id.rb.erb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class InstallSnowflakeId < ActiveRecord::Migration<%= migration_version %>
+  def up
+    # Create the timestamp_id PostgreSQL function
+    Rails::Snowflake::Id.define_timestamp_id
+  end
+
+  def down
+    # Remove the timestamp_id function
+    execute "DROP FUNCTION IF EXISTS timestamp_id(text)"
+  end
+end

data/lib/rails/snowflake/id.rb

@@ -0,0 +1,190 @@
+# frozen_string_literal: true
+
+# Copied from https://github.com/mastodon/mastodon/blob/06803422da3794538cd9cd5c7ccd61a0694ef921/lib/mastodon/snowflake.rb
+
+module Rails
+  module Snowflake
+    module Id
+    DEFAULT_REGEX = /timestamp_id\('(?<seq_prefix>\w+)'/
+
+    class Callbacks
+      def self.around_create(record)
+        now = Time.now.utc
+
+        if record.created_at.nil? || record.created_at >= now || record.created_at == record.updated_at || record.override_timestamps
+          yield
+        else
+          record.id = Rails::Snowflake::Id.at(record.created_at)
+          tries     = 0
+
+          begin
+            yield
+          rescue ActiveRecord::RecordNotUnique
+            raise if tries > 100
+
+            tries     += 1
+            record.id += rand(100)
+
+            retry
+          end
+        end
+      end
+    end
+
+    class << self
+      # Our ID will be composed of the following:
+      # 6 bytes (48 bits) of millisecond-level timestamp
+      # 2 bytes (16 bits) of sequence data
+      #
+      # The 'sequence data' is intended to be unique within a
+      # given millisecond, yet obscure the 'serial number' of
+      # this row.
+      #
+      # To do this, we hash the following data:
+      # * Table name (if provided, skipped if not)
+      # * Secret salt (should not be guessable)
+      # * Timestamp (again, millisecond-level granularity)
+      #
+      # We then take the first two bytes of that value, and add
+      # the lowest two bytes of the table ID sequence number
+      # (`table_name`_id_seq). This means that even if we insert
+      # two rows at the same millisecond, they will have
+      # distinct 'sequence data' portions.
+      #
+      # If this happens, and an attacker can see both such IDs,
+      # they can determine which of the two entries was inserted
+      # first, but not the total number of entries in the table
+      # (even mod 2**16).
+      #
+      # The table name is included in the hash to ensure that
+      # different tables derive separate sequence bases so rows
+      # inserted in the same millisecond in different tables do
+      # not reveal the table ID sequence number for one another.
+      #
+      # The secret salt is included in the hash to ensure that
+      # external users cannot derive the sequence base given the
+      # timestamp and table name, which would allow them to
+      # compute the table ID sequence number.
+      def define_timestamp_id
+        return if already_defined?
+
+        connection.execute(sanitized_timestamp_id_sql)
+      end
+
+      def ensure_id_sequences_exist
+        # Find tables using timestamp IDs.
+        connection.tables.each do |table|
+          ensure_id_sequences_exist_for(table)
+        end
+      end
+
+      def ensure_id_sequences_exist_for(table_name)
+        # We're only concerned with "id" columns.
+        id_col = connection.columns(table_name).find { |col| col.name == "id" }
+        return unless id_col
+
+        # And only those that are using timestamp_id.
+        data = DEFAULT_REGEX.match(id_col.default_function)
+        return unless data
+
+        seq_name = "#{data[:seq_prefix]}_id_seq"
+
+        # If we were on Postgres 9.5+, we could do CREATE SEQUENCE IF
+        # NOT EXISTS, but we can't depend on that. Instead, catch the
+        # possible exception and ignore it.
+        # Note that seq_name isn't a column name, but it's a
+        # relation, like a column, and follows the same quoting rules
+        # in Postgres.
+        connection.execute(<<~SQL)
+          DO $$
+            BEGIN
+              CREATE SEQUENCE #{connection.quote_column_name(seq_name)};
+            EXCEPTION WHEN duplicate_table THEN
+              -- Do nothing, we have the sequence already.
+            END
+          $$ LANGUAGE plpgsql;
+        SQL
+      rescue StandardError => e
+        Rails.logger.warn "Rails::Snowflake: Could not ensure sequence for #{table_name}: #{e.message}"
+      end
+
+      def at(timestamp, with_random: true)
+        id  = timestamp.to_i * 1000
+        id += rand(1000) if with_random
+        id <<= 16
+        id += rand(2**16) if with_random
+        id
+      end
+
+      def to_time(id)
+        Time.at((id >> 16) / 1000).utc
+      end
+
+      private
+
+      def already_defined?
+        connection.execute(<<~SQL.squish).values.first.first
+          SELECT EXISTS(
+            SELECT * FROM pg_proc WHERE proname = 'timestamp_id'
+          );
+        SQL
+      end
+
+      def sanitized_timestamp_id_sql
+        ActiveRecord::Base.sanitize_sql_array(timestamp_id_sql_array)
+      end
+
+      def timestamp_id_sql_array
+        [ timestamp_id_sql_string, { random_string: SecureRandom.hex(16) } ]
+      end
+
+      def timestamp_id_sql_string
+        <<~SQL
+          CREATE OR REPLACE FUNCTION timestamp_id(table_name text)
+          RETURNS bigint AS
+          $$
+            DECLARE
+              time_part bigint;
+              sequence_base bigint;
+              tail bigint;
+            BEGIN
+              time_part := (
+                -- Get the time in milliseconds
+                ((date_part('epoch', now()) * 1000))::bigint
+                -- And shift it over two bytes
+                << 16);
+
+              sequence_base := (
+                'x' ||
+                -- Take the first two bytes (four hex characters)
+                substr(
+                  -- Of the MD5 hash of the data we documented
+                  md5(table_name || :random_string || time_part::text),
+                  1, 4
+                )
+              -- And turn it into a bigint
+              )::bit(16)::bigint;
+
+              -- Finally, add our sequence number to our base, and chop
+              -- it to the last two bytes
+              tail := (
+                (sequence_base + nextval(table_name || '_id_seq'))
+                & 65535);
+
+              -- Return the time part and the sequence part. OR appears
+              -- faster here than addition, but they're equivalent:
+              -- time_part has no trailing two bytes, and tail is only
+              -- the last two bytes.
+              RETURN time_part | tail;
+            END
+          $$ LANGUAGE plpgsql VOLATILE;
+        SQL
+      end
+
+      def connection
+        ActiveRecord::Base.connection
+      end
+    end
+    end
+  end
+end

data/lib/rails/snowflake/railtie.rb

@@ -0,0 +1,21 @@
+module Rails
+  module Snowflake
+    class Railtie < ::Rails::Railtie
+      initializer "snowflake_id.register_field_type" do
+        ActiveSupport.on_load(:active_record) do
+          ActiveRecord::ConnectionAdapters::TableDefinition.prepend(Rails::Snowflake::ColumnMethods)
+
+          if defined?(ActiveRecord::ConnectionAdapters::PostgreSQLAdapter)
+            ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:snowflake] = { name: "bigint" }
+          else
+            raise "Rails::Snowflake: Unsupported database adapter. Only PostgreSQL is supported."
+          end
+        end
+      end
+
+      rake_tasks do
+        load "rails/snowflake/database_tasks.rb"
+      end
+    end
+  end
+end

data/lib/rails/snowflake/version.rb

@@ -0,0 +1,5 @@
+module Rails
+  module Snowflake
+    VERSION = "0.1.0"
+  end
+end

data/lib/rails/snowflake.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+# Manually require all Rails::Snowflake modules
+require_relative "snowflake/version"
+require_relative "snowflake/id"
+require_relative "snowflake/column_methods"
+require_relative "snowflake/railtie"
+require_relative "snowflake/generators/install/install_generator"
+
+
+module Rails
+  module Snowflake
+    class Error < StandardError; end
+  end
+end

data/lib/tasks/rails/snowflake_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :snowflake_id do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,70 @@
+--- !ruby/object:Gem::Specification
+name: rails-snowflake
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Luiz Eduardo Kowalski
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+description: A Rails plugin that provides a simple way to generate unique Snowflake
+  IDs for your ActiveRecord models.
+email:
+- luizeduardokowalski@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE
+- README.md
+- Rakefile
+- lib/rails/snowflake.rb
+- lib/rails/snowflake/column_methods.rb
+- lib/rails/snowflake/database_tasks.rb
+- lib/rails/snowflake/generators/install/install_generator.rb
+- lib/rails/snowflake/generators/install/templates/install_snowflake_id.rb.erb
+- lib/rails/snowflake/id.rb
+- lib/rails/snowflake/railtie.rb
+- lib/rails/snowflake/version.rb
+- lib/tasks/rails/snowflake_tasks.rake
+homepage: https://github.com/luizkowalski/snowflake_id/
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/luizkowalski/snowflake_id/
+  source_code_uri: https://github.com/luizkowalski/snowflake_id
+  changelog_uri: https://github.com/luizkowalski/snowflake_id/CHANGES.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.7.2
+specification_version: 4
+summary: Database-backed Snowflake IDs for Rails models.
+test_files: []