global_uid 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md ADDED
@@ -0,0 +1,130 @@
1
+ # Global UID Plugin
2
+
3
+ ## Summary
4
+
5
+ This plugin does a lot of the heavy lifting needed to have an external MySQL based global id generator as described in this article from Flickr
6
+
7
+ (http://code.flickr.com/blog/2010/02/08/ticket-servers-distributed-unique-primary-keys-on-the-cheap/)
8
+
9
+ There are three parts to it: configuration, migration and object creation
10
+
11
+ ### Interactions with other databases
12
+
13
+ This plugin shouldn't fail with Databases other than MySQL but neither will it do anything either. There's theoretically nothing that should stop it from being *ported* to other DBs, we just don't need to.
14
+
15
+ ## Installation
16
+
17
+ Shove this in your Gemfile and smoke it
18
+
19
+ gem "global_uid", :git => "git://github.com/zendesk/global_uid.git"
20
+
21
+ ### Configuration
22
+
23
+ First configure some databases in database.yml in the normal way.
24
+
25
+ id_server_1:
26
+ adapter: mysql
27
+ host: id_server_db1.prod
28
+ port: 3306
29
+
30
+ id_server_2:
31
+ adapter: mysql
32
+ host: id_server_db2.prod
33
+ port: 3306
34
+
35
+ Then setup these servers, and other defaults in your environment.rb:
36
+
37
+ GlobalUid.default_options = {
38
+ :id_servers => [ 'id_server_1', 'id_server_2' ],
39
+ :increment_by => 3
40
+ }
41
+
42
+ Here's a complete list of the options you can use:
43
+
44
+ Name Default
45
+ :disabled false
46
+ Disable GlobalUid entirely
47
+
48
+ :dry_run false
49
+ Setting this parameter causes the REPLACE INTO statements to run, but the id picked up will not be used.
50
+
51
+ :connection_timeout 3 seconds
52
+ Timeout for connecting to a global UID server
53
+
54
+ :query_timeout 10 seconds
55
+ Timeout for retrieving a global UID from a server before we move on to the next server
56
+
57
+ :connection_retry 10.minutes
58
+ After failing to connect or query a UID server, how long before we retry
59
+
60
+ :use_server_variables false
61
+ If set, this gem will call "set @@auto_increment_offset" in order to setup the global uid servers.
62
+ good for test/development, not so much for production.
63
+ :notifier A proc calling ActiveRecord::Base.logger
64
+ This proc is called with two parameters upon UID server failure -- an exception and a message
65
+
66
+ :increment_by 5
67
+ Chooses the step size for the increment. This will define the maximum number of UID servers you can have.
68
+
69
+ ### Testing
70
+
71
+ mysqladmin -uroot create global_uid_test
72
+ mysqladmin -uroot create global_uid_test_id_server_1
73
+ mysqladmin -uroot create global_uid_test_id_server_2
74
+
75
+ Copy test/config/database.yml.example to test/config/database.yml and make the modifications you need to point it to 2 local MySQL databases. Then +rake test+
76
+
77
+ ### Migration
78
+
79
+ Migrations will now add global_uid tables for you by default. They will also change
80
+ your primary keys from signature "PRIMARY KEY AUTO_INCREMENT NOT NULL" to "PRIMARY KEY NOT NULL".
81
+
82
+ If you'd like to disable this behavior, you can write:
83
+
84
+ class CreateFoos < ActiveRecord::Migration
85
+ def self.up
86
+ create_table :foos, :use_global_uid => false do |t|
87
+
88
+
89
+ ## Model-level stuff
90
+
91
+ If you want GlobalUIDs created, you don't have to do anything except set up the GlobalUID tables
92
+ with your migration. Everything will be taken care you. It's calm, and soothing like aloe.
93
+ It's the Rails way.
94
+
95
+
96
+ ### Disabling global uid per table
97
+
98
+ class Foo < ActiveRecord::Base
99
+ disable_global_uid
100
+ end
101
+
102
+
103
+ ## Taking matters into your own hands:
104
+
105
+
106
+ class Foo < ActiveRecord::Base
107
+ disable_global_uid
108
+
109
+ def before_create
110
+ self.id = generate_uid()
111
+ # other stuff
112
+ ....
113
+ end
114
+
115
+ end
116
+
117
+ If you're using a non standard uid table then pass that in.
118
+
119
+ generate_uid(:uid_table => '<name>')
120
+
121
+ ## Submitting Bug reports, patches or improvements
122
+
123
+ I welcome your feedback, bug reports, patches and improvements. Please e-mail these
124
+ to
125
+ simon at zendesk.com
126
+
127
+
128
+ with [mysqlbigint global uid] in the subject line. I'll get back to you as soon as I can.
129
+
130
+ Copyright (c) 2010 Zendesk, released under the MIT license
@@ -0,0 +1,54 @@
1
+ module GlobalUid
2
+ module ActiveRecordExtension
3
+
4
+ def self.included(base)
5
+ base.extend(ClassMethods)
6
+ base.class_inheritable_accessor :global_uid_disabled
7
+ base.before_create :global_uid_before_create
8
+ end
9
+
10
+ def global_uid_before_create
11
+ return if GlobalUid::Base.global_uid_options[:disabled]
12
+ return if self.class.global_uid_disabled
13
+
14
+ global_uid = nil
15
+ realtime = Benchmark::realtime do
16
+ global_uid = self.class.generate_uid
17
+ end
18
+ if GlobalUid::Base.global_uid_options[:dry_run]
19
+ ActiveRecord::Base.logger.info("GlobalUid dry-run: #{self.class.name}\t#{global_uid}\t#{"%.4f" % realtime}")
20
+ return
21
+ end
22
+
23
+ # Morten, Josh, and Ben have discussed this particular line of code, whether "||=" or "=" is correct.
24
+ # "||=" allows for more flexibility and more correct behavior (crashing) upon EBCAK
25
+ self.id ||= global_uid
26
+ end
27
+
28
+ module ClassMethods
29
+ def generate_uid(options = {})
30
+ uid_table_name = self.global_uid_table
31
+
32
+ self.ensure_global_uid_table
33
+
34
+ GlobalUid::Base.get_uid_for_class(self, options)
35
+ end
36
+
37
+ def disable_global_uid
38
+ self.global_uid_disabled = true
39
+ end
40
+
41
+ def global_uid_table
42
+ GlobalUid::Base.id_table_from_name(self.table_name)
43
+ end
44
+
45
+ def ensure_global_uid_table
46
+ return @global_uid_table_exists if @global_uid_table_exists
47
+ GlobalUid::Base.with_connections do |connection|
48
+ raise "Global UID table #{global_uid_table} not found!" unless connection.table_exists?(global_uid_table)
49
+ end
50
+ @global_uid_table_exists = true
51
+ end
52
+ end
53
+ end
54
+ end
@@ -0,0 +1,209 @@
1
+ require "active_record"
2
+ require "system_timer" if Gem.available?("system_timer")
3
+ require "timeout"
4
+
5
+ module GlobalUid
6
+ class Base
7
+ @@servers = nil
8
+
9
+ GLOBAL_UID_DEFAULTS = {
10
+ :connection_timeout => 3,
11
+ :connection_retry => 10.minutes,
12
+ :notifier => Proc.new { |exception, message| ActiveRecord::Base.logger.error("GlobalUID error: #{exception} #{message}") },
13
+ :query_timeout => 10,
14
+ :increment_by => 5, # This will define the maximum number of servers that you can have
15
+ :disabled => false,
16
+ :per_process_affinity => true,
17
+ :dry_run => false
18
+ }
19
+
20
+ def self.create_uid_tables(id_table_name, options={})
21
+ type = options[:uid_type] || "bigint(21) UNSIGNED"
22
+ start_id = options[:start_id] || 1
23
+
24
+ # TODO it would be nice to be able to set the engine or something to not be MySQL specific
25
+ with_connections do |connection|
26
+ connection.execute("CREATE TABLE IF NOT EXISTS `#{id_table_name}` (
27
+ `id` #{type} NOT NULL AUTO_INCREMENT,
28
+ `stub` char(1) NOT NULL DEFAULT '',
29
+ PRIMARY KEY (`id`),
30
+ UNIQUE KEY `stub` (`stub`)
31
+ )")
32
+
33
+ # prime the pump on each server
34
+ connection.execute("INSERT IGNORE INTO `#{id_table_name}` VALUES(#{start_id}, 'a')")
35
+ end
36
+ end
37
+
38
+ def self.drop_uid_tables(id_table_name, options={})
39
+ with_connections do |connection|
40
+ connection.execute("DROP TABLE IF EXISTS `#{id_table_name}`")
41
+ end
42
+ end
43
+
44
+ if const_defined?("SystemTimer")
45
+ GlobalUidTimer = SystemTimer
46
+ else
47
+ GlobalUidTimer = Timeout
48
+ end
49
+
50
+ def self.connection_method
51
+ @@connection_method ||= begin
52
+ if ActiveRecord::Base.respond_to?(:mysql2_connection)
53
+ :mysql2_connection
54
+ elsif ActiveRecord::Base.respond_to?(:mysql_connection)
55
+ :mysql_connection
56
+ else
57
+ raise "No Mysql Connection Adapter found..."
58
+ end
59
+ end
60
+ end
61
+
62
+
63
+ def self.new_connection(name, connection_timeout, offset, increment_by, use_server_variables)
64
+ raise "No id server '#{name}' configured in database.yml" unless ActiveRecord::Base.configurations.has_key?(name)
65
+ config = ActiveRecord::Base.configurations[name]
66
+
67
+ con = nil
68
+ begin
69
+ GlobalUidTimer.timeout(connection_timeout, ConnectionTimeoutException) do
70
+ con = ActiveRecord::Base.send(self.connection_method, config)
71
+ end
72
+ rescue ConnectionTimeoutException => e
73
+ notify e, "Timed out establishing a connection to #{name}"
74
+ return nil
75
+ rescue Exception => e
76
+ notify e, "establishing a connection to #{name}: #{e.message}"
77
+ return nil
78
+ end
79
+
80
+ # Please note that this is unreliable -- if you lose your CX to the server
81
+ # and auto-reconnect, you will be utterly hosed. Much better to dedicate a server
82
+ # or two to the cause, and set their auto_increment_increment globally.
83
+ if use_server_variables
84
+ con.execute("set @@auto_increment_increment = #{increment_by}")
85
+ con.execute("set @@auto_increment_offset = #{offset}")
86
+ end
87
+
88
+ con
89
+ end
90
+
91
+ def self.init_server_info(options)
92
+ id_servers = self.global_uid_servers
93
+
94
+ raise "You haven't configured any id servers" if id_servers.nil? or id_servers.empty?
95
+ raise "More servers configured than increment_by: #{id_servers.size} > #{options[:increment_by]} -- this will create duplicate IDs." if id_servers.size > options[:increment_by]
96
+
97
+ offset = 1
98
+
99
+ id_servers.map do |name, i|
100
+ info = {}
101
+ info[:cx] = nil
102
+ info[:name] = name
103
+ info[:retry_at] = nil
104
+ info[:offset] = offset
105
+ info[:rand] = rand
106
+ info[:new?] = true
107
+ offset +=1
108
+ info
109
+ end
110
+ end
111
+
112
+ def self.setup_connections!(options)
113
+ connection_timeout = options[:connection_timeout]
114
+ increment_by = options[:increment_by]
115
+
116
+ if @@servers.nil?
117
+ @@servers = init_server_info(options)
118
+ # sorting here sets up each process to have affinity to a particular server.
119
+ @@servers = @@servers.sort_by { |s| s[:rand] }
120
+ end
121
+
122
+ @@servers.each do |info|
123
+ next if info[:cx]
124
+
125
+ if info[:new?] || ( info[:retry_at] && Time.now > info[:retry_at] )
126
+ info[:new?] = false
127
+
128
+ connection = new_connection(info[:name], connection_timeout, info[:offset], increment_by, options[:use_server_variables])
129
+ info[:cx] = connection
130
+ info[:retry_at] = Time.now + options[:connection_retry] if connection.nil?
131
+ end
132
+ end
133
+
134
+ @@servers
135
+ end
136
+
137
+ def self.with_connections(options = {})
138
+ options = self.global_uid_options.merge(options)
139
+ servers = setup_connections!(options)
140
+
141
+ if !options[:per_process_affinity]
142
+ servers = servers.sort_by { rand } #yes, I know it's not true random.
143
+ end
144
+
145
+ raise NoServersAvailableException if servers.empty?
146
+
147
+ exception_count = 0
148
+
149
+ errors = []
150
+ servers.each do |s|
151
+ begin
152
+ yield s[:cx] if s[:cx]
153
+ rescue TimeoutException, Exception => e
154
+ notify e, "#{e.message}"
155
+ errors << e
156
+ s[:cx] = nil
157
+ s[:retry_at] = Time.now + 10.minutes
158
+ end
159
+ end
160
+
161
+ if errors.size == servers.size
162
+ # The INCREDIBLY BAD CASE WHERE EVERYONE IS ERRORING!
163
+ # Might as well start drinking if we hit this spot in the code.
164
+ # note that at this point we're trying to retry on the next request.
165
+ servers.each { |s| s[:retry_at] = Time.now - 5.minutes }
166
+ raise NoServersAvailableException, "Errors hit: #{errors.map(&:to_s).join(',')}"
167
+ end
168
+
169
+ servers.map { |s| s[:cx] }.compact
170
+ end
171
+
172
+ def self.notify(exception, message)
173
+ if self.global_uid_options[:notifier]
174
+ self.global_uid_options[:notifier].call(exception, message)
175
+ end
176
+ end
177
+
178
+ def self.get_connections(options = {})
179
+ with_connections {}
180
+ end
181
+
182
+ def self.get_uid_for_class(klass, options = {})
183
+ with_connections do |connection|
184
+ timeout = self.global_uid_options[:query_timeout]
185
+ GlobalUidTimer.timeout(self.global_uid_options[:query_timeout], TimeoutException) do
186
+ id = connection.insert("REPLACE INTO #{klass.global_uid_table} (stub) VALUES ('a')")
187
+ return id
188
+ end
189
+ end
190
+ raise NoServersAvailableException, "All global UID servers are gone!"
191
+ end
192
+
193
+ def self.global_uid_options=(options)
194
+ @global_uid_options = GLOBAL_UID_DEFAULTS.merge(options.symbolize_keys)
195
+ end
196
+
197
+ def self.global_uid_options
198
+ @global_uid_options
199
+ end
200
+
201
+ def self.global_uid_servers
202
+ self.global_uid_options[:id_servers]
203
+ end
204
+
205
+ def self.id_table_from_name(name)
206
+ "#{name}_ids".to_sym
207
+ end
208
+ end
209
+ end
@@ -0,0 +1,43 @@
1
+ module GlobalUid
2
+ module MigrationExtension
3
+ def self.included(base)
4
+ base.alias_method_chain :create_table, :global_uid
5
+ base.alias_method_chain :drop_table, :global_uid
6
+ end
7
+
8
+ def create_table_with_global_uid(name, options = {}, &blk)
9
+ uid_enabled = !(GlobalUid::Base.global_uid_options[:disabled] || options[:use_global_uid] == false)
10
+
11
+ # rules for stripping out auto_increment -- enabled, not dry-run, and not a "PK-less" table
12
+ remove_auto_increment = uid_enabled && !GlobalUid::Base.global_uid_options[:dry_run] && !(options[:id] == false)
13
+
14
+ if remove_auto_increment
15
+ old_id_option = options[:id]
16
+ options.merge!(:id => false)
17
+ end
18
+
19
+ if uid_enabled
20
+ id_table_name = options[:global_uid_table] || GlobalUid::Base.id_table_from_name(name)
21
+ GlobalUid::Base.create_uid_tables(id_table_name, options)
22
+ end
23
+
24
+ create_table_without_global_uid(name, options) { |t|
25
+ if remove_auto_increment
26
+ # need to honor specifically named tables
27
+ id_column_name = (old_id_option || :id)
28
+ t.column id_column_name, "int(10) NOT NULL PRIMARY KEY"
29
+ end
30
+ blk.call(t) if blk
31
+ }
32
+ end
33
+
34
+ def drop_table_with_global_uid(name, options = {})
35
+ if !GlobalUid::Base.global_uid_options[:disabled] && options[:use_global_uid] != false
36
+ id_table_name = options[:global_uid_table] || GlobalUid::Base.id_table_from_name(name)
37
+ GlobalUid::Base.drop_uid_tables(id_table_name,options)
38
+ end
39
+ return drop_table_without_global_uid(name,options)
40
+ end
41
+
42
+ end
43
+ end
data/lib/global_uid.rb ADDED
@@ -0,0 +1,13 @@
1
+ require "global_uid/base"
2
+ require "global_uid/active_record_extension"
3
+ require "global_uid/migration_extension"
4
+
5
+ module GlobalUid
6
+ class NoServersAvailableException < Exception ; end
7
+ class ConnectionTimeoutException < Exception ; end
8
+ class TimeoutException < Exception ; end
9
+ end
10
+
11
+ ActiveRecord::Base.send(:include, GlobalUid::ActiveRecordExtension)
12
+ ActiveRecord::ConnectionAdapters::AbstractAdapter.send(:include, GlobalUid::MigrationExtension)
13
+
@@ -0,0 +1,20 @@
1
+ test:
2
+ adapter: mysql2
3
+ encoding: utf8
4
+ database: global_uid_test
5
+ username: root
6
+ password:
7
+
8
+ test_id_server_1:
9
+ adapter: mysql2
10
+ encoding: utf8
11
+ database: global_uid_test_id_server_1
12
+ username: root
13
+ password:
14
+
15
+ test_id_server_2:
16
+ adapter: mysql2
17
+ encoding: utf8
18
+ database: global_uid_test_id_server_2
19
+ username: root
20
+ password:
@@ -0,0 +1,362 @@
1
+ require 'test_helper'
2
+
3
+ class CreateWithNoParams < ActiveRecord::Migration
4
+ group :change if self.respond_to?(:group)
5
+
6
+ def self.up
7
+ create_table :with_global_uids do |t|
8
+ t.string :description
9
+ end
10
+ end
11
+
12
+ def self.down
13
+ drop_table :with_global_uids
14
+ end
15
+ end
16
+
17
+ class CreateWithNamedID < ActiveRecord::Migration
18
+ group :change if self.respond_to?(:group)
19
+
20
+ def self.up
21
+ create_table :with_global_uids, :id => 'hello' do |t|
22
+ t.string :description
23
+ end
24
+ end
25
+
26
+ def self.down
27
+ drop_table :with_global_uids
28
+ end
29
+ end
30
+
31
+ class CreateWithoutGlobalUIDs < ActiveRecord::Migration
32
+ group :change if self.respond_to?(:group)
33
+
34
+ def self.up
35
+ create_table :without_global_uids, :use_global_uid => false do |t|
36
+ t.string :description
37
+ end
38
+ end
39
+
40
+ def self.down
41
+ drop_table :without_global_uids, :use_global_uid => false
42
+ end
43
+ end
44
+
45
+ class WithGlobalUID < ActiveRecord::Base
46
+ end
47
+
48
+ class WithoutGlobalUID < ActiveRecord::Base
49
+ end
50
+
51
+ class GlobalUIDTest < ActiveSupport::TestCase
52
+ ActiveRecord::Migration.verbose = false
53
+
54
+ context "migrations" do
55
+ setup do
56
+ restore_defaults!
57
+ reset_connections!
58
+ drop_old_test_tables!
59
+ end
60
+
61
+ context "without explicit parameters" do
62
+ context "with global-uid enabled" do
63
+ setup do
64
+ GlobalUid::Base.global_uid_options[:disabled] = false
65
+ CreateWithNoParams.up
66
+ @create_table = show_create_sql(WithGlobalUID, "with_global_uids").split("\n")
67
+ end
68
+
69
+ should "create the global_uids table" do
70
+ GlobalUid::Base.with_connections do |cx|
71
+ assert cx.table_exists?('with_global_uids_ids')
72
+ end
73
+ end
74
+
75
+ should "create global_uids tables with matching ids" do
76
+ GlobalUid::Base.with_connections do |cx|
77
+ foo = cx.select_all("select id from with_global_uids_ids")
78
+ assert(foo.first['id'].to_i == 1)
79
+ end
80
+ end
81
+
82
+ should "tear off the auto_increment part of the primary key from the created table" do
83
+ id_line = @create_table.grep(/\`id\` int/i).first
84
+ assert_no_match /auto_increment/i, id_line
85
+ end
86
+
87
+ should "create a primary key on id" do
88
+ assert @create_table.grep(/primary key/i).size > 0
89
+ end
90
+
91
+ teardown do
92
+ CreateWithNoParams.down
93
+ end
94
+ end
95
+
96
+ context "with global-uid disabled, globally" do
97
+ setup do
98
+ GlobalUid::Base.global_uid_options[:disabled] = true
99
+ CreateWithNoParams.up
100
+ end
101
+
102
+ should "not create the global_uids table" do
103
+ GlobalUid::Base.with_connections do |cx|
104
+ assert !cx.table_exists?('with_global_uids_ids')
105
+ end
106
+ end
107
+
108
+ teardown do
109
+ CreateWithNoParams.down
110
+ GlobalUid::Base.global_uid_options[:disabled] = false
111
+ end
112
+ end
113
+
114
+ context "with a named ID key" do
115
+ setup do
116
+ CreateWithNamedID.up
117
+ end
118
+
119
+ should "preserve the name of the ID key" do
120
+ @create_table = show_create_sql(WithGlobalUID, "with_global_uids").split("\n")
121
+ assert(@create_table.grep(/hello.*int/i))
122
+ assert(@create_table.grep(/primary key.*hello/i))
123
+ end
124
+
125
+ teardown do
126
+ CreateWithNamedID.down
127
+ end
128
+ end
129
+ end
130
+
131
+ context "with global-uid disabled in the migration" do
132
+ setup do
133
+ CreateWithoutGlobalUIDs.up
134
+ @create_table = show_create_sql(WithoutGlobalUID, "without_global_uids").split("\n")
135
+ end
136
+
137
+ should "not create the global_uids table" do
138
+ GlobalUid::Base.with_connections do |cx|
139
+ assert !cx.table_exists?('without_global_uids_ids')
140
+ end
141
+ end
142
+
143
+ should "create standard auto-increment tables" do
144
+ id_line = @create_table.grep(/.id. int/i).first
145
+ assert_match /auto_increment/i, id_line
146
+ end
147
+
148
+ teardown do
149
+ CreateWithoutGlobalUIDs.down
150
+ end
151
+ end
152
+ end
153
+
154
+ context "With GlobalUID" do
155
+ setup do
156
+ reset_connections!
157
+ drop_old_test_tables!
158
+ restore_defaults!
159
+ CreateWithNoParams.up
160
+ CreateWithoutGlobalUIDs.up
161
+ end
162
+
163
+ context "normally" do
164
+ should "get a unique id" do
165
+ test_unique_ids
166
+ end
167
+ end
168
+
169
+ context "With a timing out server" do
170
+ setup do
171
+ reset_connections!
172
+ @a_decent_cx = GlobalUid::Base.new_connection(GlobalUid::Base.global_uid_servers.first, 50, 1, 5, true)
173
+ ActiveRecord::Base.stubs(:mysql_connection).raises(GlobalUid::ConnectionTimeoutException).then.returns(@a_decent_cx)
174
+ @connections = GlobalUid::Base.get_connections
175
+ end
176
+
177
+ should "limp along with one functioning server" do
178
+ assert @connections.include?(@a_decent_cx)
179
+ assert_equal GlobalUid::Base.global_uid_servers.size - 1, @connections.size, "get_connections size"
180
+ end
181
+
182
+ should "eventually retry the connection and get it back in place" do
183
+ # clear the state machine expectation
184
+ ActiveRecord::Base.mysql_connection rescue nil
185
+ ActiveRecord::Base.mysql_connection rescue nil
186
+
187
+ awhile = Time.now + 10.hours
188
+ Time.stubs(:now).returns(awhile)
189
+
190
+ assert GlobalUid::Base.get_connections.size == GlobalUid::Base.global_uid_servers.size
191
+
192
+ end
193
+
194
+ should "get some unique ids" do
195
+ test_unique_ids
196
+ end
197
+ end
198
+
199
+ context "With a server timing out on query" do
200
+ setup do
201
+ reset_connections!
202
+ @old_size = GlobalUid::Base.get_connections.size # prime them
203
+ GlobalUid::Base.get_connections.first.stubs(:execute).raises(GlobalUid::TimeoutException)
204
+ # trigger the failure -- have to do it it a bunch of times, as one call might not hit the server
205
+ # Even so there's a 1/(2^32) possibility of this test failing.
206
+ 32.times do WithGlobalUID.create! end
207
+ end
208
+
209
+ should "pull the server out of the pool" do
210
+ assert GlobalUid::Base.get_connections.size == @old_size - 1
211
+ end
212
+
213
+ should "get ids from the remaining server" do
214
+ test_unique_ids
215
+ end
216
+
217
+ should "eventually retry the connection" do
218
+ awhile = Time.now + 10.hours
219
+ Time.stubs(:now).returns(awhile)
220
+
221
+ assert GlobalUid::Base.get_connections.size == GlobalUid::Base.global_uid_servers.size
222
+ end
223
+ end
224
+
225
+ context "With both servers throwing exceptions" do
226
+ setup do
227
+ # would prefer to do the below, but need Mocha 0.9.10 to do so
228
+ # ActiveRecord::ConnectionAdapters::MysqlAdapter.any_instance.stubs(:execute).raises(ActiveRecord::StatementInvalid)
229
+ GlobalUid::Base.with_connections do |cx|
230
+ cx.stubs(:execute).raises(ActiveRecord::StatementInvalid)
231
+ end
232
+ end
233
+
234
+ should "raise a NoServersAvailableException" do
235
+ assert_raises(GlobalUid::NoServersAvailableException) do
236
+ WithGlobalUID.create!
237
+ end
238
+ end
239
+
240
+ should "retry the servers immediately after failure" do
241
+ assert_raises(GlobalUid::NoServersAvailableException) do
242
+ WithGlobalUID.create!
243
+ end
244
+
245
+ assert WithGlobalUID.create!
246
+ end
247
+ end
248
+
249
+
250
+ context "with per-process_affinity" do
251
+ setup do
252
+ GlobalUid::Base.global_uid_options[:per_process_affinity] = true
253
+ end
254
+
255
+ should "increment sequentially" do
256
+ last_id = 0
257
+ 10.times do
258
+ this_id = WithGlobalUID.create!.id
259
+ assert this_id > last_id
260
+ end
261
+ end
262
+
263
+ teardown do
264
+ GlobalUid::Base.global_uid_options[:per_process_affinity] = false
265
+ end
266
+ end
267
+
268
+ context "with global-uid disabled" do
269
+ setup do
270
+ WithoutGlobalUID.disable_global_uid
271
+ end
272
+
273
+ should "never call various unsafe methods" do
274
+ GlobalUid::Base.expects(:new_connection).never
275
+ GlobalUid::Base.expects(:get_uid_for_class).never
276
+ WithoutGlobalUID.expects(:generate_uid).never
277
+ WithoutGlobalUID.expects(:ensure_global_uid).never
278
+ GlobalUid::Base.expects(:get_uid_for_class).never
279
+ end
280
+
281
+ teardown do
282
+ end
283
+ end
284
+
285
+ teardown do
286
+ mocha_teardown # tear down mocha early to prevent some of this being tied to mocha expectations
287
+ reset_connections!
288
+ CreateWithNoParams.down
289
+ CreateWithoutGlobalUIDs.down
290
+ end
291
+ end
292
+
293
+ context "In dry-run mode" do
294
+ setup do
295
+ reset_connections!
296
+ drop_old_test_tables!
297
+ GlobalUid::Base.global_uid_options[:dry_run] = true
298
+ CreateWithNoParams.up
299
+ end
300
+
301
+ should "create a normal looking table" do
302
+
303
+ end
304
+
305
+ should "increment normally1" do
306
+ (1..10).each do |i|
307
+ assert_equal i, WithGlobalUID.create!.id
308
+ end
309
+ end
310
+
311
+ should "insert into the UID servers nonetheless" do
312
+ GlobalUid::Base.expects(:get_uid_for_class).at_least(10)
313
+ 10.times { WithGlobalUID.create! }
314
+ end
315
+
316
+ should "log the results" do
317
+ ActiveRecord::Base.logger.expects(:info).at_least(10)
318
+ 10.times { WithGlobalUID.create! }
319
+ end
320
+
321
+ teardown do
322
+ reset_connections!
323
+ CreateWithNoParams.down
324
+ GlobalUid::Base.global_uid_options[:dry_run] = false
325
+ end
326
+ end
327
+
328
+ private
329
+ def test_unique_ids
330
+ seen = {}
331
+ (0..10).each do
332
+ foo = WithGlobalUID.new
333
+ foo.save
334
+ assert !foo.id.nil?
335
+ assert foo.description.nil?
336
+ assert !seen.has_key?(foo.id)
337
+ seen[foo.id] = 1
338
+ end
339
+ end
340
+
341
+ def drop_old_test_tables!
342
+ GlobalUid::Base.with_connections do |cx|
343
+ cx.execute("DROP TABLE IF exists with_global_uids_ids")
344
+ end
345
+ end
346
+
347
+ def reset_connections!
348
+ GlobalUid::Base.class_eval "@@servers = nil"
349
+ end
350
+
351
+ def restore_defaults!
352
+ GlobalUid::Base.global_uid_options[:disabled] = false
353
+ GlobalUid::Base.global_uid_options[:use_server_variables] = true
354
+ GlobalUid::Base.global_uid_options[:dry_run] = false
355
+ end
356
+
357
+ def show_create_sql(klass, table)
358
+ klass.connection.select_all("show create table #{table}")[0]["Create Table"]
359
+ end
360
+ end
361
+
362
+
@@ -0,0 +1,23 @@
1
+ require 'rubygems'
2
+
3
+ require 'bundler'
4
+ Bundler.setup
5
+
6
+ require "active_record"
7
+ require "active_support"
8
+ require "active_support/test_case"
9
+ require "shoulda"
10
+ require "global_uid"
11
+
12
+ GlobalUid::Base.global_uid_options = {
13
+ :use_server_variables => true,
14
+ :disabled => false,
15
+ :id_servers => [
16
+ "test_id_server_1",
17
+ "test_id_server_2"
18
+ ]
19
+ }
20
+
21
+ ActiveRecord::Base.configurations = YAML::load(IO.read(File.dirname(__FILE__) + "/config/database.yml"))
22
+ ActiveRecord::Base.establish_connection("test")
23
+ ActiveRecord::Base.logger = Logger.new(File.dirname(__FILE__) + "/test.log")
metadata ADDED
@@ -0,0 +1,295 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: global_uid
3
+ version: !ruby/object:Gem::Version
4
+ hash: 23
5
+ prerelease:
6
+ segments:
7
+ - 1
8
+ - 0
9
+ - 0
10
+ version: 1.0.0
11
+ platform: ruby
12
+ authors:
13
+ - Ben Osheroff
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2011-07-07 00:00:00 -07:00
19
+ default_executable:
20
+ dependencies:
21
+ - !ruby/object:Gem::Dependency
22
+ name: global_uid
23
+ version_requirements: &id001 !ruby/object:Gem::Requirement
24
+ none: false
25
+ requirements:
26
+ - - ">="
27
+ - !ruby/object:Gem::Version
28
+ hash: 3
29
+ segments:
30
+ - 0
31
+ version: "0"
32
+ prerelease: false
33
+ type: :runtime
34
+ requirement: *id001
35
+ - !ruby/object:Gem::Dependency
36
+ name: rake
37
+ version_requirements: &id002 !ruby/object:Gem::Requirement
38
+ none: false
39
+ requirements:
40
+ - - ">="
41
+ - !ruby/object:Gem::Version
42
+ hash: 3
43
+ segments:
44
+ - 0
45
+ version: "0"
46
+ prerelease: false
47
+ type: :development
48
+ requirement: *id002
49
+ - !ruby/object:Gem::Dependency
50
+ name: bundler
51
+ version_requirements: &id003 !ruby/object:Gem::Requirement
52
+ none: false
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ hash: 3
57
+ segments:
58
+ - 0
59
+ version: "0"
60
+ prerelease: false
61
+ type: :development
62
+ requirement: *id003
63
+ - !ruby/object:Gem::Dependency
64
+ name: shoulda
65
+ version_requirements: &id004 !ruby/object:Gem::Requirement
66
+ none: false
67
+ requirements:
68
+ - - ">="
69
+ - !ruby/object:Gem::Version
70
+ hash: 3
71
+ segments:
72
+ - 0
73
+ version: "0"
74
+ prerelease: false
75
+ type: :development
76
+ requirement: *id004
77
+ - !ruby/object:Gem::Dependency
78
+ name: mocha
79
+ version_requirements: &id005 !ruby/object:Gem::Requirement
80
+ none: false
81
+ requirements:
82
+ - - ">="
83
+ - !ruby/object:Gem::Version
84
+ hash: 3
85
+ segments:
86
+ - 0
87
+ version: "0"
88
+ prerelease: false
89
+ type: :development
90
+ requirement: *id005
91
+ - !ruby/object:Gem::Dependency
92
+ name: ruby-debug
93
+ version_requirements: &id006 !ruby/object:Gem::Requirement
94
+ none: false
95
+ requirements:
96
+ - - ">="
97
+ - !ruby/object:Gem::Version
98
+ hash: 3
99
+ segments:
100
+ - 0
101
+ version: "0"
102
+ prerelease: false
103
+ type: :development
104
+ requirement: *id006
105
+ - !ruby/object:Gem::Dependency
106
+ name: activerecord
107
+ version_requirements: &id007 !ruby/object:Gem::Requirement
108
+ none: false
109
+ requirements:
110
+ - - ~>
111
+ - !ruby/object:Gem::Version
112
+ hash: 23
113
+ segments:
114
+ - 2
115
+ - 3
116
+ - 10
117
+ version: 2.3.10
118
+ prerelease: false
119
+ type: :runtime
120
+ requirement: *id007
121
+ - !ruby/object:Gem::Dependency
122
+ name: activesupport
123
+ version_requirements: &id008 !ruby/object:Gem::Requirement
124
+ none: false
125
+ requirements:
126
+ - - ~>
127
+ - !ruby/object:Gem::Version
128
+ hash: 23
129
+ segments:
130
+ - 2
131
+ - 3
132
+ - 10
133
+ version: 2.3.10
134
+ prerelease: false
135
+ type: :runtime
136
+ requirement: *id008
137
+ - !ruby/object:Gem::Dependency
138
+ name: SystemTimer
139
+ version_requirements: &id009 !ruby/object:Gem::Requirement
140
+ none: false
141
+ requirements:
142
+ - - ~>
143
+ - !ruby/object:Gem::Version
144
+ hash: 11
145
+ segments:
146
+ - 1
147
+ - 2
148
+ version: "1.2"
149
+ prerelease: false
150
+ type: :runtime
151
+ requirement: *id009
152
+ - !ruby/object:Gem::Dependency
153
+ name: mysql
154
+ version_requirements: &id010 !ruby/object:Gem::Requirement
155
+ none: false
156
+ requirements:
157
+ - - "="
158
+ - !ruby/object:Gem::Version
159
+ hash: 45
160
+ segments:
161
+ - 2
162
+ - 8
163
+ - 1
164
+ version: 2.8.1
165
+ prerelease: false
166
+ type: :runtime
167
+ requirement: *id010
168
+ - !ruby/object:Gem::Dependency
169
+ name: rake
170
+ version_requirements: &id011 !ruby/object:Gem::Requirement
171
+ none: false
172
+ requirements:
173
+ - - ">="
174
+ - !ruby/object:Gem::Version
175
+ hash: 3
176
+ segments:
177
+ - 0
178
+ version: "0"
179
+ prerelease: false
180
+ type: :development
181
+ requirement: *id011
182
+ - !ruby/object:Gem::Dependency
183
+ name: bundler
184
+ version_requirements: &id012 !ruby/object:Gem::Requirement
185
+ none: false
186
+ requirements:
187
+ - - ">="
188
+ - !ruby/object:Gem::Version
189
+ hash: 3
190
+ segments:
191
+ - 0
192
+ version: "0"
193
+ prerelease: false
194
+ type: :development
195
+ requirement: *id012
196
+ - !ruby/object:Gem::Dependency
197
+ name: shoulda
198
+ version_requirements: &id013 !ruby/object:Gem::Requirement
199
+ none: false
200
+ requirements:
201
+ - - ">="
202
+ - !ruby/object:Gem::Version
203
+ hash: 3
204
+ segments:
205
+ - 0
206
+ version: "0"
207
+ prerelease: false
208
+ type: :development
209
+ requirement: *id013
210
+ - !ruby/object:Gem::Dependency
211
+ name: mocha
212
+ version_requirements: &id014 !ruby/object:Gem::Requirement
213
+ none: false
214
+ requirements:
215
+ - - ">="
216
+ - !ruby/object:Gem::Version
217
+ hash: 3
218
+ segments:
219
+ - 0
220
+ version: "0"
221
+ prerelease: false
222
+ type: :development
223
+ requirement: *id014
224
+ - !ruby/object:Gem::Dependency
225
+ name: ruby-debug
226
+ version_requirements: &id015 !ruby/object:Gem::Requirement
227
+ none: false
228
+ requirements:
229
+ - - ">="
230
+ - !ruby/object:Gem::Version
231
+ hash: 3
232
+ segments:
233
+ - 0
234
+ version: "0"
235
+ prerelease: false
236
+ type: :development
237
+ requirement: *id015
238
+ description: Zendesk GUID
239
+ email:
240
+ - ben@zendesk.com
241
+ executables: []
242
+
243
+ extensions: []
244
+
245
+ extra_rdoc_files:
246
+ - README.md
247
+ files:
248
+ - lib/global_uid.rb
249
+ - lib/global_uid/active_record_extension.rb
250
+ - lib/global_uid/base.rb
251
+ - lib/global_uid/migration_extension.rb
252
+ - README.md
253
+ - test/config/database.yml.example
254
+ - test/global_uid_test.rb
255
+ - test/test_helper.rb
256
+ has_rdoc: true
257
+ homepage: http://github.com/zendesk/global_uid
258
+ licenses: []
259
+
260
+ post_install_message:
261
+ rdoc_options: []
262
+
263
+ require_paths:
264
+ - lib
265
+ required_ruby_version: !ruby/object:Gem::Requirement
266
+ none: false
267
+ requirements:
268
+ - - ">="
269
+ - !ruby/object:Gem::Version
270
+ hash: 3
271
+ segments:
272
+ - 0
273
+ version: "0"
274
+ required_rubygems_version: !ruby/object:Gem::Requirement
275
+ none: false
276
+ requirements:
277
+ - - ">="
278
+ - !ruby/object:Gem::Version
279
+ hash: 23
280
+ segments:
281
+ - 1
282
+ - 3
283
+ - 6
284
+ version: 1.3.6
285
+ requirements: []
286
+
287
+ rubyforge_project:
288
+ rubygems_version: 1.5.2
289
+ signing_key:
290
+ specification_version: 3
291
+ summary: Zendesk GUID
292
+ test_files:
293
+ - test/config/database.yml.example
294
+ - test/global_uid_test.rb
295
+ - test/test_helper.rb