global_uid 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,130 @@
1
+ # Global UID Plugin
2
+
3
+ ## Summary
4
+
5
+ This plugin does a lot of the heavy lifting needed to have an external MySQL based global id generator as described in this article from Flickr
6
+
7
+ (http://code.flickr.com/blog/2010/02/08/ticket-servers-distributed-unique-primary-keys-on-the-cheap/)
8
+
9
+ There are three parts to it: configuration, migration and object creation
10
+
11
+ ### Interactions with other databases
12
+
13
+ This plugin shouldn't fail with Databases other than MySQL but neither will it do anything either. There's theoretically nothing that should stop it from being *ported* to other DBs, we just don't need to.
14
+
15
+ ## Installation
16
+
17
+ Shove this in your Gemfile and smoke it
18
+
19
+ gem "global_uid", :git => "git://github.com/zendesk/global_uid.git"
20
+
21
+ ### Configuration
22
+
23
+ First configure some databases in database.yml in the normal way.
24
+
25
+ id_server_1:
26
+ adapter: mysql
27
+ host: id_server_db1.prod
28
+ port: 3306
29
+
30
+ id_server_2:
31
+ adapter: mysql
32
+ host: id_server_db2.prod
33
+ port: 3306
34
+
35
+ Then setup these servers, and other defaults in your environment.rb:
36
+
37
+ GlobalUid.default_options = {
38
+ :id_servers => [ 'id_server_1', 'id_server_2' ],
39
+ :increment_by => 3
40
+ }
41
+
42
+ Here's a complete list of the options you can use:
43
+
44
+ Name Default
45
+ :disabled false
46
+ Disable GlobalUid entirely
47
+
48
+ :dry_run false
49
+ Setting this parameter causes the REPLACE INTO statements to run, but the id picked up will not be used.
50
+
51
+ :connection_timeout 3 seconds
52
+ Timeout for connecting to a global UID server
53
+
54
+ :query_timeout 10 seconds
55
+ Timeout for retrieving a global UID from a server before we move on to the next server
56
+
57
+ :connection_retry 10.minutes
58
+ After failing to connect or query a UID server, how long before we retry
59
+
60
+ :use_server_variables false
61
+ If set, this gem will call "set @@auto_increment_offset" in order to setup the global uid servers.
62
+ good for test/development, not so much for production.
63
+ :notifier A proc calling ActiveRecord::Base.logger
64
+ This proc is called with two parameters upon UID server failure -- an exception and a message
65
+
66
+ :increment_by 5
67
+ Chooses the step size for the increment. This will define the maximum number of UID servers you can have.
68
+
69
+ ### Testing
70
+
71
+ mysqladmin -uroot create global_uid_test
72
+ mysqladmin -uroot create global_uid_test_id_server_1
73
+ mysqladmin -uroot create global_uid_test_id_server_2
74
+
75
+ Copy test/config/database.yml.example to test/config/database.yml and make the modifications you need to point it to 2 local MySQL databases. Then +rake test+
76
+
77
+ ### Migration
78
+
79
+ Migrations will now add global_uid tables for you by default. They will also change
80
+ your primary keys from signature "PRIMARY KEY AUTO_INCREMENT NOT NULL" to "PRIMARY KEY NOT NULL".
81
+
82
+ If you'd like to disable this behavior, you can write:
83
+
84
+ class CreateFoos < ActiveRecord::Migration
85
+ def self.up
86
+ create_table :foos, :use_global_uid => false do |t|
87
+
88
+
89
+ ## Model-level stuff
90
+
91
+ If you want GlobalUIDs created, you don't have to do anything except set up the GlobalUID tables
92
+ with your migration. Everything will be taken care you. It's calm, and soothing like aloe.
93
+ It's the Rails way.
94
+
95
+
96
+ ### Disabling global uid per table
97
+
98
+ class Foo < ActiveRecord::Base
99
+ disable_global_uid
100
+ end
101
+
102
+
103
+ ## Taking matters into your own hands:
104
+
105
+
106
+ class Foo < ActiveRecord::Base
107
+ disable_global_uid
108
+
109
+ def before_create
110
+ self.id = generate_uid()
111
+ # other stuff
112
+ ....
113
+ end
114
+
115
+ end
116
+
117
+ If you're using a non standard uid table then pass that in.
118
+
119
+ generate_uid(:uid_table => '<name>')
120
+
121
+ ## Submitting Bug reports, patches or improvements
122
+
123
+ I welcome your feedback, bug reports, patches and improvements. Please e-mail these
124
+ to
125
+ simon at zendesk.com
126
+
127
+
128
+ with [mysqlbigint global uid] in the subject line. I'll get back to you as soon as I can.
129
+
130
+ Copyright (c) 2010 Zendesk, released under the MIT license
@@ -0,0 +1,54 @@
1
+ module GlobalUid
2
+ module ActiveRecordExtension
3
+
4
+ def self.included(base)
5
+ base.extend(ClassMethods)
6
+ base.class_inheritable_accessor :global_uid_disabled
7
+ base.before_create :global_uid_before_create
8
+ end
9
+
10
+ def global_uid_before_create
11
+ return if GlobalUid::Base.global_uid_options[:disabled]
12
+ return if self.class.global_uid_disabled
13
+
14
+ global_uid = nil
15
+ realtime = Benchmark::realtime do
16
+ global_uid = self.class.generate_uid
17
+ end
18
+ if GlobalUid::Base.global_uid_options[:dry_run]
19
+ ActiveRecord::Base.logger.info("GlobalUid dry-run: #{self.class.name}\t#{global_uid}\t#{"%.4f" % realtime}")
20
+ return
21
+ end
22
+
23
+ # Morten, Josh, and Ben have discussed this particular line of code, whether "||=" or "=" is correct.
24
+ # "||=" allows for more flexibility and more correct behavior (crashing) upon EBCAK
25
+ self.id ||= global_uid
26
+ end
27
+
28
+ module ClassMethods
29
+ def generate_uid(options = {})
30
+ uid_table_name = self.global_uid_table
31
+
32
+ self.ensure_global_uid_table
33
+
34
+ GlobalUid::Base.get_uid_for_class(self, options)
35
+ end
36
+
37
+ def disable_global_uid
38
+ self.global_uid_disabled = true
39
+ end
40
+
41
+ def global_uid_table
42
+ GlobalUid::Base.id_table_from_name(self.table_name)
43
+ end
44
+
45
+ def ensure_global_uid_table
46
+ return @global_uid_table_exists if @global_uid_table_exists
47
+ GlobalUid::Base.with_connections do |connection|
48
+ raise "Global UID table #{global_uid_table} not found!" unless connection.table_exists?(global_uid_table)
49
+ end
50
+ @global_uid_table_exists = true
51
+ end
52
+ end
53
+ end
54
+ end
@@ -0,0 +1,209 @@
1
+ require "active_record"
2
+ require "system_timer" if Gem.available?("system_timer")
3
+ require "timeout"
4
+
5
+ module GlobalUid
6
+ class Base
7
+ @@servers = nil
8
+
9
+ GLOBAL_UID_DEFAULTS = {
10
+ :connection_timeout => 3,
11
+ :connection_retry => 10.minutes,
12
+ :notifier => Proc.new { |exception, message| ActiveRecord::Base.logger.error("GlobalUID error: #{exception} #{message}") },
13
+ :query_timeout => 10,
14
+ :increment_by => 5, # This will define the maximum number of servers that you can have
15
+ :disabled => false,
16
+ :per_process_affinity => true,
17
+ :dry_run => false
18
+ }
19
+
20
+ def self.create_uid_tables(id_table_name, options={})
21
+ type = options[:uid_type] || "bigint(21) UNSIGNED"
22
+ start_id = options[:start_id] || 1
23
+
24
+ # TODO it would be nice to be able to set the engine or something to not be MySQL specific
25
+ with_connections do |connection|
26
+ connection.execute("CREATE TABLE IF NOT EXISTS `#{id_table_name}` (
27
+ `id` #{type} NOT NULL AUTO_INCREMENT,
28
+ `stub` char(1) NOT NULL DEFAULT '',
29
+ PRIMARY KEY (`id`),
30
+ UNIQUE KEY `stub` (`stub`)
31
+ )")
32
+
33
+ # prime the pump on each server
34
+ connection.execute("INSERT IGNORE INTO `#{id_table_name}` VALUES(#{start_id}, 'a')")
35
+ end
36
+ end
37
+
38
+ def self.drop_uid_tables(id_table_name, options={})
39
+ with_connections do |connection|
40
+ connection.execute("DROP TABLE IF EXISTS `#{id_table_name}`")
41
+ end
42
+ end
43
+
44
+ if const_defined?("SystemTimer")
45
+ GlobalUidTimer = SystemTimer
46
+ else
47
+ GlobalUidTimer = Timeout
48
+ end
49
+
50
+ def self.connection_method
51
+ @@connection_method ||= begin
52
+ if ActiveRecord::Base.respond_to?(:mysql2_connection)
53
+ :mysql2_connection
54
+ elsif ActiveRecord::Base.respond_to?(:mysql_connection)
55
+ :mysql_connection
56
+ else
57
+ raise "No Mysql Connection Adapter found..."
58
+ end
59
+ end
60
+ end
61
+
62
+
63
+ def self.new_connection(name, connection_timeout, offset, increment_by, use_server_variables)
64
+ raise "No id server '#{name}' configured in database.yml" unless ActiveRecord::Base.configurations.has_key?(name)
65
+ config = ActiveRecord::Base.configurations[name]
66
+
67
+ con = nil
68
+ begin
69
+ GlobalUidTimer.timeout(connection_timeout, ConnectionTimeoutException) do
70
+ con = ActiveRecord::Base.send(self.connection_method, config)
71
+ end
72
+ rescue ConnectionTimeoutException => e
73
+ notify e, "Timed out establishing a connection to #{name}"
74
+ return nil
75
+ rescue Exception => e
76
+ notify e, "establishing a connection to #{name}: #{e.message}"
77
+ return nil
78
+ end
79
+
80
+ # Please note that this is unreliable -- if you lose your CX to the server
81
+ # and auto-reconnect, you will be utterly hosed. Much better to dedicate a server
82
+ # or two to the cause, and set their auto_increment_increment globally.
83
+ if use_server_variables
84
+ con.execute("set @@auto_increment_increment = #{increment_by}")
85
+ con.execute("set @@auto_increment_offset = #{offset}")
86
+ end
87
+
88
+ con
89
+ end
90
+
91
+ def self.init_server_info(options)
92
+ id_servers = self.global_uid_servers
93
+
94
+ raise "You haven't configured any id servers" if id_servers.nil? or id_servers.empty?
95
+ raise "More servers configured than increment_by: #{id_servers.size} > #{options[:increment_by]} -- this will create duplicate IDs." if id_servers.size > options[:increment_by]
96
+
97
+ offset = 1
98
+
99
+ id_servers.map do |name, i|
100
+ info = {}
101
+ info[:cx] = nil
102
+ info[:name] = name
103
+ info[:retry_at] = nil
104
+ info[:offset] = offset
105
+ info[:rand] = rand
106
+ info[:new?] = true
107
+ offset +=1
108
+ info
109
+ end
110
+ end
111
+
112
+ def self.setup_connections!(options)
113
+ connection_timeout = options[:connection_timeout]
114
+ increment_by = options[:increment_by]
115
+
116
+ if @@servers.nil?
117
+ @@servers = init_server_info(options)
118
+ # sorting here sets up each process to have affinity to a particular server.
119
+ @@servers = @@servers.sort_by { |s| s[:rand] }
120
+ end
121
+
122
+ @@servers.each do |info|
123
+ next if info[:cx]
124
+
125
+ if info[:new?] || ( info[:retry_at] && Time.now > info[:retry_at] )
126
+ info[:new?] = false
127
+
128
+ connection = new_connection(info[:name], connection_timeout, info[:offset], increment_by, options[:use_server_variables])
129
+ info[:cx] = connection
130
+ info[:retry_at] = Time.now + options[:connection_retry] if connection.nil?
131
+ end
132
+ end
133
+
134
+ @@servers
135
+ end
136
+
137
+ def self.with_connections(options = {})
138
+ options = self.global_uid_options.merge(options)
139
+ servers = setup_connections!(options)
140
+
141
+ if !options[:per_process_affinity]
142
+ servers = servers.sort_by { rand } #yes, I know it's not true random.
143
+ end
144
+
145
+ raise NoServersAvailableException if servers.empty?
146
+
147
+ exception_count = 0
148
+
149
+ errors = []
150
+ servers.each do |s|
151
+ begin
152
+ yield s[:cx] if s[:cx]
153
+ rescue TimeoutException, Exception => e
154
+ notify e, "#{e.message}"
155
+ errors << e
156
+ s[:cx] = nil
157
+ s[:retry_at] = Time.now + 10.minutes
158
+ end
159
+ end
160
+
161
+ if errors.size == servers.size
162
+ # The INCREDIBLY BAD CASE WHERE EVERYONE IS ERRORING!
163
+ # Might as well start drinking if we hit this spot in the code.
164
+ # note that at this point we're trying to retry on the next request.
165
+ servers.each { |s| s[:retry_at] = Time.now - 5.minutes }
166
+ raise NoServersAvailableException, "Errors hit: #{errors.map(&:to_s).join(',')}"
167
+ end
168
+
169
+ servers.map { |s| s[:cx] }.compact
170
+ end
171
+
172
+ def self.notify(exception, message)
173
+ if self.global_uid_options[:notifier]
174
+ self.global_uid_options[:notifier].call(exception, message)
175
+ end
176
+ end
177
+
178
+ def self.get_connections(options = {})
179
+ with_connections {}
180
+ end
181
+
182
+ def self.get_uid_for_class(klass, options = {})
183
+ with_connections do |connection|
184
+ timeout = self.global_uid_options[:query_timeout]
185
+ GlobalUidTimer.timeout(self.global_uid_options[:query_timeout], TimeoutException) do
186
+ id = connection.insert("REPLACE INTO #{klass.global_uid_table} (stub) VALUES ('a')")
187
+ return id
188
+ end
189
+ end
190
+ raise NoServersAvailableException, "All global UID servers are gone!"
191
+ end
192
+
193
+ def self.global_uid_options=(options)
194
+ @global_uid_options = GLOBAL_UID_DEFAULTS.merge(options.symbolize_keys)
195
+ end
196
+
197
+ def self.global_uid_options
198
+ @global_uid_options
199
+ end
200
+
201
+ def self.global_uid_servers
202
+ self.global_uid_options[:id_servers]
203
+ end
204
+
205
+ def self.id_table_from_name(name)
206
+ "#{name}_ids".to_sym
207
+ end
208
+ end
209
+ end
@@ -0,0 +1,43 @@
1
+ module GlobalUid
2
+ module MigrationExtension
3
+ def self.included(base)
4
+ base.alias_method_chain :create_table, :global_uid
5
+ base.alias_method_chain :drop_table, :global_uid
6
+ end
7
+
8
+ def create_table_with_global_uid(name, options = {}, &blk)
9
+ uid_enabled = !(GlobalUid::Base.global_uid_options[:disabled] || options[:use_global_uid] == false)
10
+
11
+ # rules for stripping out auto_increment -- enabled, not dry-run, and not a "PK-less" table
12
+ remove_auto_increment = uid_enabled && !GlobalUid::Base.global_uid_options[:dry_run] && !(options[:id] == false)
13
+
14
+ if remove_auto_increment
15
+ old_id_option = options[:id]
16
+ options.merge!(:id => false)
17
+ end
18
+
19
+ if uid_enabled
20
+ id_table_name = options[:global_uid_table] || GlobalUid::Base.id_table_from_name(name)
21
+ GlobalUid::Base.create_uid_tables(id_table_name, options)
22
+ end
23
+
24
+ create_table_without_global_uid(name, options) { |t|
25
+ if remove_auto_increment
26
+ # need to honor specifically named tables
27
+ id_column_name = (old_id_option || :id)
28
+ t.column id_column_name, "int(10) NOT NULL PRIMARY KEY"
29
+ end
30
+ blk.call(t) if blk
31
+ }
32
+ end
33
+
34
+ def drop_table_with_global_uid(name, options = {})
35
+ if !GlobalUid::Base.global_uid_options[:disabled] && options[:use_global_uid] != false
36
+ id_table_name = options[:global_uid_table] || GlobalUid::Base.id_table_from_name(name)
37
+ GlobalUid::Base.drop_uid_tables(id_table_name,options)
38
+ end
39
+ return drop_table_without_global_uid(name,options)
40
+ end
41
+
42
+ end
43
+ end
data/lib/global_uid.rb ADDED
@@ -0,0 +1,13 @@
1
+ require "global_uid/base"
2
+ require "global_uid/active_record_extension"
3
+ require "global_uid/migration_extension"
4
+
5
+ module GlobalUid
6
+ class NoServersAvailableException < Exception ; end
7
+ class ConnectionTimeoutException < Exception ; end
8
+ class TimeoutException < Exception ; end
9
+ end
10
+
11
+ ActiveRecord::Base.send(:include, GlobalUid::ActiveRecordExtension)
12
+ ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, GlobalUid::MigrationExtension)
13
+
@@ -0,0 +1,20 @@
1
+ test:
2
+ adapter: mysql2
3
+ encoding: utf8
4
+ database: global_uid_test
5
+ username: root
6
+ password:
7
+
8
+ test_id_server_1:
9
+ adapter: mysql2
10
+ encoding: utf8
11
+ database: global_uid_test_id_server_1
12
+ username: root
13
+ password:
14
+
15
+ test_id_server_2:
16
+ adapter: mysql2
17
+ encoding: utf8
18
+ database: global_uid_test_id_server_2
19
+ username: root
20
+ password:
@@ -0,0 +1,362 @@
1
+ require 'test_helper'
2
+
3
+ class CreateWithNoParams < ActiveRecord::Migration
4
+ group :change if self.respond_to?(:group)
5
+
6
+ def self.up
7
+ create_table :with_global_uids do |t|
8
+ t.string :description
9
+ end
10
+ end
11
+
12
+ def self.down
13
+ drop_table :with_global_uids
14
+ end
15
+ end
16
+
17
+ class CreateWithNamedID < ActiveRecord::Migration
18
+ group :change if self.respond_to?(:group)
19
+
20
+ def self.up
21
+ create_table :with_global_uids, :id => 'hello' do |t|
22
+ t.string :description
23
+ end
24
+ end
25
+
26
+ def self.down
27
+ drop_table :with_global_uids
28
+ end
29
+ end
30
+
31
+ class CreateWithoutGlobalUIDs < ActiveRecord::Migration
32
+ group :change if self.respond_to?(:group)
33
+
34
+ def self.up
35
+ create_table :without_global_uids, :use_global_uid => false do |t|
36
+ t.string :description
37
+ end
38
+ end
39
+
40
+ def self.down
41
+ drop_table :without_global_uids, :use_global_uid => false
42
+ end
43
+ end
44
+
45
+ class WithGlobalUID < ActiveRecord::Base
46
+ end
47
+
48
+ class WithoutGlobalUID < ActiveRecord::Base
49
+ end
50
+
51
+ class GlobalUIDTest < ActiveSupport::TestCase
52
+ ActiveRecord::Migration.verbose = false
53
+
54
+ context "migrations" do
55
+ setup do
56
+ restore_defaults!
57
+ reset_connections!
58
+ drop_old_test_tables!
59
+ end
60
+
61
+ context "without explicit parameters" do
62
+ context "with global-uid enabled" do
63
+ setup do
64
+ GlobalUid::Base.global_uid_options[:disabled] = false
65
+ CreateWithNoParams.up
66
+ @create_table = show_create_sql(WithGlobalUID, "with_global_uids").split("\n")
67
+ end
68
+
69
+ should "create the global_uids table" do
70
+ GlobalUid::Base.with_connections do |cx|
71
+ assert cx.table_exists?('with_global_uids_ids')
72
+ end
73
+ end
74
+
75
+ should "create global_uids tables with matching ids" do
76
+ GlobalUid::Base.with_connections do |cx|
77
+ foo = cx.select_all("select id from with_global_uids_ids")
78
+ assert(foo.first['id'].to_i == 1)
79
+ end
80
+ end
81
+
82
+ should "tear off the auto_increment part of the primary key from the created table" do
83
+ id_line = @create_table.grep(/\`id\` int/i).first
84
+ assert_no_match /auto_increment/i, id_line
85
+ end
86
+
87
+ should "create a primary key on id" do
88
+ assert @create_table.grep(/primary key/i).size > 0
89
+ end
90
+
91
+ teardown do
92
+ CreateWithNoParams.down
93
+ end
94
+ end
95
+
96
+ context "with global-uid disabled, globally" do
97
+ setup do
98
+ GlobalUid::Base.global_uid_options[:disabled] = true
99
+ CreateWithNoParams.up
100
+ end
101
+
102
+ should "not create the global_uids table" do
103
+ GlobalUid::Base.with_connections do |cx|
104
+ assert !cx.table_exists?('with_global_uids_ids')
105
+ end
106
+ end
107
+
108
+ teardown do
109
+ CreateWithNoParams.down
110
+ GlobalUid::Base.global_uid_options[:disabled] = false
111
+ end
112
+ end
113
+
114
+ context "with a named ID key" do
115
+ setup do
116
+ CreateWithNamedID.up
117
+ end
118
+
119
+ should "preserve the name of the ID key" do
120
+ @create_table = show_create_sql(WithGlobalUID, "with_global_uids").split("\n")
121
+ assert(@create_table.grep(/hello.*int/i))
122
+ assert(@create_table.grep(/primary key.*hello/i))
123
+ end
124
+
125
+ teardown do
126
+ CreateWithNamedID.down
127
+ end
128
+ end
129
+ end
130
+
131
+ context "with global-uid disabled in the migration" do
132
+ setup do
133
+ CreateWithoutGlobalUIDs.up
134
+ @create_table = show_create_sql(WithoutGlobalUID, "without_global_uids").split("\n")
135
+ end
136
+
137
+ should "not create the global_uids table" do
138
+ GlobalUid::Base.with_connections do |cx|
139
+ assert !cx.table_exists?('without_global_uids_ids')
140
+ end
141
+ end
142
+
143
+ should "create standard auto-increment tables" do
144
+ id_line = @create_table.grep(/.id. int/i).first
145
+ assert_match /auto_increment/i, id_line
146
+ end
147
+
148
+ teardown do
149
+ CreateWithoutGlobalUIDs.down
150
+ end
151
+ end
152
+ end
153
+
154
+ context "With GlobalUID" do
155
+ setup do
156
+ reset_connections!
157
+ drop_old_test_tables!
158
+ restore_defaults!
159
+ CreateWithNoParams.up
160
+ CreateWithoutGlobalUIDs.up
161
+ end
162
+
163
+ context "normally" do
164
+ should "get a unique id" do
165
+ test_unique_ids
166
+ end
167
+ end
168
+
169
+ context "With a timing out server" do
170
+ setup do
171
+ reset_connections!
172
+ @a_decent_cx = GlobalUid::Base.new_connection(GlobalUid::Base.global_uid_servers.first, 50, 1, 5, true)
173
+ ActiveRecord::Base.stubs(:mysql_connection).raises(GlobalUid::ConnectionTimeoutException).then.returns(@a_decent_cx)
174
+ @connections = GlobalUid::Base.get_connections
175
+ end
176
+
177
+ should "limp along with one functioning server" do
178
+ assert @connections.include?(@a_decent_cx)
179
+ assert_equal GlobalUid::Base.global_uid_servers.size - 1, @connections.size, "get_connections size"
180
+ end
181
+
182
+ should "eventually retry the connection and get it back in place" do
183
+ # clear the state machine expectation
184
+ ActiveRecord::Base.mysql_connection rescue nil
185
+ ActiveRecord::Base.mysql_connection rescue nil
186
+
187
+ awhile = Time.now + 10.hours
188
+ Time.stubs(:now).returns(awhile)
189
+
190
+ assert GlobalUid::Base.get_connections.size == GlobalUid::Base.global_uid_servers.size
191
+
192
+ end
193
+
194
+ should "get some unique ids" do
195
+ test_unique_ids
196
+ end
197
+ end
198
+
199
+ context "With a server timing out on query" do
200
+ setup do
201
+ reset_connections!
202
+ @old_size = GlobalUid::Base.get_connections.size # prime them
203
+ GlobalUid::Base.get_connections.first.stubs(:execute).raises(GlobalUid::TimeoutException)
204
+ # trigger the failure -- have to do it it a bunch of times, as one call might not hit the server
205
+ # Even so there's a 1/(2^32) possibility of this test failing.
206
+ 32.times do WithGlobalUID.create! end
207
+ end
208
+
209
+ should "pull the server out of the pool" do
210
+ assert GlobalUid::Base.get_connections.size == @old_size - 1
211
+ end
212
+
213
+ should "get ids from the remaining server" do
214
+ test_unique_ids
215
+ end
216
+
217
+ should "eventually retry the connection" do
218
+ awhile = Time.now + 10.hours
219
+ Time.stubs(:now).returns(awhile)
220
+
221
+ assert GlobalUid::Base.get_connections.size == GlobalUid::Base.global_uid_servers.size
222
+ end
223
+ end
224
+
225
+ context "With both servers throwing exceptions" do
226
+ setup do
227
+ # would prefer to do the below, but need Mocha 0.9.10 to do so
228
+ # ActiveRecord::ConnectionAdapters::MysqlAdapter.any_instance.stubs(:execute).raises(ActiveRecord::StatementInvalid)
229
+ GlobalUid::Base.with_connections do |cx|
230
+ cx.stubs(:execute).raises(ActiveRecord::StatementInvalid)
231
+ end
232
+ end
233
+
234
+ should "raise a NoServersAvailableException" do
235
+ assert_raises(GlobalUid::NoServersAvailableException) do
236
+ WithGlobalUID.create!
237
+ end
238
+ end
239
+
240
+ should "retry the servers immediately after failure" do
241
+ assert_raises(GlobalUid::NoServersAvailableException) do
242
+ WithGlobalUID.create!
243
+ end
244
+
245
+ assert WithGlobalUID.create!
246
+ end
247
+ end
248
+
249
+
250
+ context "with per-process_affinity" do
251
+ setup do
252
+ GlobalUid::Base.global_uid_options[:per_process_affinity] = true
253
+ end
254
+
255
+ should "increment sequentially" do
256
+ last_id = 0
257
+ 10.times do
258
+ this_id = WithGlobalUID.create!.id
259
+ assert this_id > last_id
260
+ end
261
+ end
262
+
263
+ teardown do
264
+ GlobalUid::Base.global_uid_options[:per_process_affinity] = false
265
+ end
266
+ end
267
+
268
+ context "with global-uid disabled" do
269
+ setup do
270
+ WithoutGlobalUID.disable_global_uid
271
+ end
272
+
273
+ should "never call various unsafe methods" do
274
+ GlobalUid::Base.expects(:new_connection).never
275
+ GlobalUid::Base.expects(:get_uid_for_class).never
276
+ WithoutGlobalUID.expects(:generate_uid).never
277
+ WithoutGlobalUID.expects(:ensure_global_uid).never
278
+ GlobalUid::Base.expects(:get_uid_for_class).never
279
+ end
280
+
281
+ teardown do
282
+ end
283
+ end
284
+
285
+ teardown do
286
+ mocha_teardown # tear down mocha early to prevent some of this being tied to mocha expectations
287
+ reset_connections!
288
+ CreateWithNoParams.down
289
+ CreateWithoutGlobalUIDs.down
290
+ end
291
+ end
292
+
293
+ context "In dry-run mode" do
294
+ setup do
295
+ reset_connections!
296
+ drop_old_test_tables!
297
+ GlobalUid::Base.global_uid_options[:dry_run] = true
298
+ CreateWithNoParams.up
299
+ end
300
+
301
+ should "create a normal looking table" do
302
+
303
+ end
304
+
305
+ should "increment normally1" do
306
+ (1..10).each do |i|
307
+ assert_equal i, WithGlobalUID.create!.id
308
+ end
309
+ end
310
+
311
+ should "insert into the UID servers nonetheless" do
312
+ GlobalUid::Base.expects(:get_uid_for_class).at_least(10)
313
+ 10.times { WithGlobalUID.create! }
314
+ end
315
+
316
+ should "log the results" do
317
+ ActiveRecord::Base.logger.expects(:info).at_least(10)
318
+ 10.times { WithGlobalUID.create! }
319
+ end
320
+
321
+ teardown do
322
+ reset_connections!
323
+ CreateWithNoParams.down
324
+ GlobalUid::Base.global_uid_options[:dry_run] = false
325
+ end
326
+ end
327
+
328
+ private
329
+ def test_unique_ids
330
+ seen = {}
331
+ (0..10).each do
332
+ foo = WithGlobalUID.new
333
+ foo.save
334
+ assert !foo.id.nil?
335
+ assert foo.description.nil?
336
+ assert !seen.has_key?(foo.id)
337
+ seen[foo.id] = 1
338
+ end
339
+ end
340
+
341
+ def drop_old_test_tables!
342
+ GlobalUid::Base.with_connections do |cx|
343
+ cx.execute("DROP TABLE IF exists with_global_uids_ids")
344
+ end
345
+ end
346
+
347
+ def reset_connections!
348
+ GlobalUid::Base.class_eval "@@servers = nil"
349
+ end
350
+
351
+ def restore_defaults!
352
+ GlobalUid::Base.global_uid_options[:disabled] = false
353
+ GlobalUid::Base.global_uid_options[:use_server_variables] = true
354
+ GlobalUid::Base.global_uid_options[:dry_run] = false
355
+ end
356
+
357
+ def show_create_sql(klass, table)
358
+ klass.connection.select_all("show create table #{table}")[0]["Create Table"]
359
+ end
360
+ end
361
+
362
+
@@ -0,0 +1,23 @@
1
+ require 'rubygems'
2
+
3
+ require 'bundler'
4
+ Bundler.setup
5
+
6
+ require "active_record"
7
+ require "active_support"
8
+ require "active_support/test_case"
9
+ require "shoulda"
10
+ require "global_uid"
11
+
12
+ GlobalUid::Base.global_uid_options = {
13
+ :use_server_variables => true,
14
+ :disabled => false,
15
+ :id_servers => [
16
+ "test_id_server_1",
17
+ "test_id_server_2"
18
+ ]
19
+ }
20
+
21
+ ActiveRecord::Base.configurations = YAML::load(IO.read(File.dirname(__FILE__) + "/config/database.yml"))
22
+ ActiveRecord::Base.establish_connection("test")
23
+ ActiveRecord::Base.logger = Logger.new(File.dirname(__FILE__) + "/test.log")
metadata ADDED
@@ -0,0 +1,295 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: global_uid
3
+ version: !ruby/object:Gem::Version
4
+ hash: 23
5
+ prerelease:
6
+ segments:
7
+ - 1
8
+ - 0
9
+ - 0
10
+ version: 1.0.0
11
+ platform: ruby
12
+ authors:
13
+ - Ben Osheroff
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2011-07-07 00:00:00 -07:00
19
+ default_executable:
20
+ dependencies:
21
+ - !ruby/object:Gem::Dependency
22
+ name: global_uid
23
+ version_requirements: &id001 !ruby/object:Gem::Requirement
24
+ none: false
25
+ requirements:
26
+ - - ">="
27
+ - !ruby/object:Gem::Version
28
+ hash: 3
29
+ segments:
30
+ - 0
31
+ version: "0"
32
+ prerelease: false
33
+ type: :runtime
34
+ requirement: *id001
35
+ - !ruby/object:Gem::Dependency
36
+ name: rake
37
+ version_requirements: &id002 !ruby/object:Gem::Requirement
38
+ none: false
39
+ requirements:
40
+ - - ">="
41
+ - !ruby/object:Gem::Version
42
+ hash: 3
43
+ segments:
44
+ - 0
45
+ version: "0"
46
+ prerelease: false
47
+ type: :development
48
+ requirement: *id002
49
+ - !ruby/object:Gem::Dependency
50
+ name: bundler
51
+ version_requirements: &id003 !ruby/object:Gem::Requirement
52
+ none: false
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ hash: 3
57
+ segments:
58
+ - 0
59
+ version: "0"
60
+ prerelease: false
61
+ type: :development
62
+ requirement: *id003
63
+ - !ruby/object:Gem::Dependency
64
+ name: shoulda
65
+ version_requirements: &id004 !ruby/object:Gem::Requirement
66
+ none: false
67
+ requirements:
68
+ - - ">="
69
+ - !ruby/object:Gem::Version
70
+ hash: 3
71
+ segments:
72
+ - 0
73
+ version: "0"
74
+ prerelease: false
75
+ type: :development
76
+ requirement: *id004
77
+ - !ruby/object:Gem::Dependency
78
+ name: mocha
79
+ version_requirements: &id005 !ruby/object:Gem::Requirement
80
+ none: false
81
+ requirements:
82
+ - - ">="
83
+ - !ruby/object:Gem::Version
84
+ hash: 3
85
+ segments:
86
+ - 0
87
+ version: "0"
88
+ prerelease: false
89
+ type: :development
90
+ requirement: *id005
91
+ - !ruby/object:Gem::Dependency
92
+ name: ruby-debug
93
+ version_requirements: &id006 !ruby/object:Gem::Requirement
94
+ none: false
95
+ requirements:
96
+ - - ">="
97
+ - !ruby/object:Gem::Version
98
+ hash: 3
99
+ segments:
100
+ - 0
101
+ version: "0"
102
+ prerelease: false
103
+ type: :development
104
+ requirement: *id006
105
+ - !ruby/object:Gem::Dependency
106
+ name: activerecord
107
+ version_requirements: &id007 !ruby/object:Gem::Requirement
108
+ none: false
109
+ requirements:
110
+ - - ~>
111
+ - !ruby/object:Gem::Version
112
+ hash: 23
113
+ segments:
114
+ - 2
115
+ - 3
116
+ - 10
117
+ version: 2.3.10
118
+ prerelease: false
119
+ type: :runtime
120
+ requirement: *id007
121
+ - !ruby/object:Gem::Dependency
122
+ name: activesupport
123
+ version_requirements: &id008 !ruby/object:Gem::Requirement
124
+ none: false
125
+ requirements:
126
+ - - ~>
127
+ - !ruby/object:Gem::Version
128
+ hash: 23
129
+ segments:
130
+ - 2
131
+ - 3
132
+ - 10
133
+ version: 2.3.10
134
+ prerelease: false
135
+ type: :runtime
136
+ requirement: *id008
137
+ - !ruby/object:Gem::Dependency
138
+ name: SystemTimer
139
+ version_requirements: &id009 !ruby/object:Gem::Requirement
140
+ none: false
141
+ requirements:
142
+ - - ~>
143
+ - !ruby/object:Gem::Version
144
+ hash: 11
145
+ segments:
146
+ - 1
147
+ - 2
148
+ version: "1.2"
149
+ prerelease: false
150
+ type: :runtime
151
+ requirement: *id009
152
+ - !ruby/object:Gem::Dependency
153
+ name: mysql
154
+ version_requirements: &id010 !ruby/object:Gem::Requirement
155
+ none: false
156
+ requirements:
157
+ - - "="
158
+ - !ruby/object:Gem::Version
159
+ hash: 45
160
+ segments:
161
+ - 2
162
+ - 8
163
+ - 1
164
+ version: 2.8.1
165
+ prerelease: false
166
+ type: :runtime
167
+ requirement: *id010
168
+ - !ruby/object:Gem::Dependency
169
+ name: rake
170
+ version_requirements: &id011 !ruby/object:Gem::Requirement
171
+ none: false
172
+ requirements:
173
+ - - ">="
174
+ - !ruby/object:Gem::Version
175
+ hash: 3
176
+ segments:
177
+ - 0
178
+ version: "0"
179
+ prerelease: false
180
+ type: :development
181
+ requirement: *id011
182
+ - !ruby/object:Gem::Dependency
183
+ name: bundler
184
+ version_requirements: &id012 !ruby/object:Gem::Requirement
185
+ none: false
186
+ requirements:
187
+ - - ">="
188
+ - !ruby/object:Gem::Version
189
+ hash: 3
190
+ segments:
191
+ - 0
192
+ version: "0"
193
+ prerelease: false
194
+ type: :development
195
+ requirement: *id012
196
+ - !ruby/object:Gem::Dependency
197
+ name: shoulda
198
+ version_requirements: &id013 !ruby/object:Gem::Requirement
199
+ none: false
200
+ requirements:
201
+ - - ">="
202
+ - !ruby/object:Gem::Version
203
+ hash: 3
204
+ segments:
205
+ - 0
206
+ version: "0"
207
+ prerelease: false
208
+ type: :development
209
+ requirement: *id013
210
+ - !ruby/object:Gem::Dependency
211
+ name: mocha
212
+ version_requirements: &id014 !ruby/object:Gem::Requirement
213
+ none: false
214
+ requirements:
215
+ - - ">="
216
+ - !ruby/object:Gem::Version
217
+ hash: 3
218
+ segments:
219
+ - 0
220
+ version: "0"
221
+ prerelease: false
222
+ type: :development
223
+ requirement: *id014
224
+ - !ruby/object:Gem::Dependency
225
+ name: ruby-debug
226
+ version_requirements: &id015 !ruby/object:Gem::Requirement
227
+ none: false
228
+ requirements:
229
+ - - ">="
230
+ - !ruby/object:Gem::Version
231
+ hash: 3
232
+ segments:
233
+ - 0
234
+ version: "0"
235
+ prerelease: false
236
+ type: :development
237
+ requirement: *id015
238
+ description: Zendesk GUID
239
+ email:
240
+ - ben@zendesk.com
241
+ executables: []
242
+
243
+ extensions: []
244
+
245
+ extra_rdoc_files:
246
+ - README.md
247
+ files:
248
+ - lib/global_uid.rb
249
+ - lib/global_uid/active_record_extension.rb
250
+ - lib/global_uid/base.rb
251
+ - lib/global_uid/migration_extension.rb
252
+ - README.md
253
+ - test/config/database.yml.example
254
+ - test/global_uid_test.rb
255
+ - test/test_helper.rb
256
+ has_rdoc: true
257
+ homepage: http://github.com/zendesk/global_uid
258
+ licenses: []
259
+
260
+ post_install_message:
261
+ rdoc_options: []
262
+
263
+ require_paths:
264
+ - lib
265
+ required_ruby_version: !ruby/object:Gem::Requirement
266
+ none: false
267
+ requirements:
268
+ - - ">="
269
+ - !ruby/object:Gem::Version
270
+ hash: 3
271
+ segments:
272
+ - 0
273
+ version: "0"
274
+ required_rubygems_version: !ruby/object:Gem::Requirement
275
+ none: false
276
+ requirements:
277
+ - - ">="
278
+ - !ruby/object:Gem::Version
279
+ hash: 23
280
+ segments:
281
+ - 1
282
+ - 3
283
+ - 6
284
+ version: 1.3.6
285
+ requirements: []
286
+
287
+ rubyforge_project:
288
+ rubygems_version: 1.5.2
289
+ signing_key:
290
+ specification_version: 3
291
+ summary: Zendesk GUID
292
+ test_files:
293
+ - test/config/database.yml.example
294
+ - test/global_uid_test.rb
295
+ - test/test_helper.rb