mysql2_bigint 0.2.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. data/CHANGELOG.md +120 -0
  2. data/Gemfile +3 -0
  3. data/MIT-LICENSE +20 -0
  4. data/README.rdoc +248 -0
  5. data/Rakefile +5 -0
  6. data/benchmark/active_record.rb +51 -0
  7. data/benchmark/allocations.rb +33 -0
  8. data/benchmark/escape.rb +36 -0
  9. data/benchmark/query_with_mysql_casting.rb +80 -0
  10. data/benchmark/query_without_mysql_casting.rb +47 -0
  11. data/benchmark/sequel.rb +37 -0
  12. data/benchmark/setup_db.rb +119 -0
  13. data/examples/eventmachine.rb +21 -0
  14. data/examples/threaded.rb +20 -0
  15. data/ext/mysql2/client.c +768 -0
  16. data/ext/mysql2/client.h +41 -0
  17. data/ext/mysql2/extconf.rb +69 -0
  18. data/ext/mysql2/mysql2_ext.c +12 -0
  19. data/ext/mysql2/mysql2_ext.h +38 -0
  20. data/ext/mysql2/result.c +488 -0
  21. data/ext/mysql2/result.h +20 -0
  22. data/lib/active_record/connection_adapters/em_mysql2_adapter.rb +64 -0
  23. data/lib/active_record/connection_adapters/mysql2_adapter.rb +654 -0
  24. data/lib/active_record/fiber_patches.rb +104 -0
  25. data/lib/arel/engines/sql/compilers/mysql2_compiler.rb +11 -0
  26. data/lib/mysql2.rb +16 -0
  27. data/lib/mysql2/client.rb +240 -0
  28. data/lib/mysql2/em.rb +37 -0
  29. data/lib/mysql2/em_fiber.rb +31 -0
  30. data/lib/mysql2/error.rb +15 -0
  31. data/lib/mysql2/result.rb +5 -0
  32. data/lib/mysql2/version.rb +3 -0
  33. data/lib/mysql2_bigint.rb +1 -0
  34. data/mysql2_bigint.gemspec +32 -0
  35. data/spec/em/em_fiber_spec.rb +22 -0
  36. data/spec/em/em_spec.rb +49 -0
  37. data/spec/mysql2/client_spec.rb +385 -0
  38. data/spec/mysql2/error_spec.rb +25 -0
  39. data/spec/mysql2/result_spec.rb +328 -0
  40. data/spec/rcov.opts +3 -0
  41. data/spec/spec_helper.rb +66 -0
  42. data/tasks/benchmarks.rake +20 -0
  43. data/tasks/compile.rake +53 -0
  44. data/tasks/rspec.rake +16 -0
  45. data/tasks/vendor_mysql.rake +41 -0
  46. metadata +199 -0
data/CHANGELOG.md ADDED
@@ -0,0 +1,120 @@
1
+ # Changelog
2
+
3
+ ## HEAD
4
+ * various fixes for em_mysql2 and fiber usage
5
+
6
+ ## 0.2.6 (October 19th, 2010)
7
+ * version bump since the 0.2.5 win32 binary gems were broken
8
+
9
+ ## 0.2.5 (October 19th, 2010)
10
+ * fixes for easier Win32 binary gem deployment for targeting 1.8 and 1.9 in the same gem
11
+ * refactor of connection checks and management to avoid race conditions with the GC/threading to prevent the unexpected loss of connections
12
+ * update the default flags during connection
13
+ * add support for setting wait_timeout on AR adapter
14
+ * upgrade to rspec2
15
+ * bugfix for an edge case where the GC would clean up a Mysql2::Client object before the underlying MYSQL pointer had been initialized
16
+ * fix to CFLAGS to allow compilation on SPARC with sunstudio compiler - Anko painting <anko.com+github@gmail.com>
17
+
18
+ ## 0.2.4 (September 17th, 2010)
19
+ * a few patches for win32 support from Luis Lavena - thanks man!
20
+ * bugfix from Eric Wong to avoid a potential stack overflow during Mysql2::Client#escape
21
+ * added the ability to turn internal row caching on/off via the :cache_rows => true/false option
22
+ * a couple of small patches for rbx compatibility
23
+ * set IndexDefinition#length in AR adapter - Kouhei Yanagita <yanagi@shakenbu.org>
24
+ * fix a long-standing data corruption bug - thank you thank you thank you to @joedamato (http://github.com/ice799)
25
+ * bugfix from calling mysql_close on a closed/freed connection surfaced by the above fix
26
+
27
+ ## 0.2.3 (August 20th, 2010)
28
+ * connection flags can now be passed to the constructor via the :flags key
29
+ * switch AR adapter connection over to use FOUND_ROWS option
30
+ * patch to ensure we use DateTime objects in place of Time for timestamps that are out of the supported range on 32bit platforms < 1.9.2
31
+
32
+ ## 0.2.2 (August 19th, 2010)
33
+ * Change how AR adapter would send initial commands upon connecting
34
+ ** we can make multiple session variable assignments in a single query
35
+ * fix signal handling when waiting on queries
36
+ * retry connect if interrupted by signals
37
+
38
+ ## 0.2.1 (August 16th, 2010)
39
+ * bring mysql2 ActiveRecord adapter back into gem
40
+
41
+ ## 0.2.0 (August 16th, 2010)
42
+ * switch back to letting libmysql manage all allocation/thread-state/freeing for the connection
43
+ * cache various numeric type conversions in hot-spots of the code for a little speed boost
44
+ * ActiveRecord adapter moved into Rails 3 core
45
+ ** Don't worry 2.3.x users! We'll either release the adapter as a separate gem, or try to get it into 2.3.9
46
+ * Fix for the "closed MySQL connection" error (GH #31)
47
+ * Fix for the "can't modify frozen object" error in 1.9.2 (GH #37)
48
+ * Introduce cascading query and result options (more info in README)
49
+ * Sequel adapter pulled into core (will be in the next release - 3.15.0 at the time of writing)
50
+ * add a safety check when attempting to send a query before a result has been fetched
51
+
52
+ ## 0.1.9 (July 17th, 2010)
53
+ * Support async ActiveRecord access with fibers and EventMachine (mperham)
54
+ * string encoding support for 1.9, respecting Encoding.default_internal
55
+ * added support for rake-compiler (tenderlove)
56
+ * bugfixes for ActiveRecord driver
57
+ ** one minor bugfix for TimeZone support
58
+ ** fix the select_rows method to return what it should according to the docs (r-stu31)
59
+ * Mysql2::Client#fields method added - returns the array of field names from a resultset, as strings
60
+ * Sequel adapter
61
+ ** bugfix regarding sybolized field names (Eric Wong)
62
+ ** fix query logging in Sequel adapter
63
+ * Lots of nice code cleanup (tenderlove)
64
+ ** Mysql2::Error definition moved to pure-Ruby
65
+ ** Mysql2::client#initialize definition moved to pure-Ruby
66
+ ** Mysql2::Result partially moved to pure-Ruby
67
+
68
+ ## 0.1.8 (June 2nd, 2010)
69
+ * fixes for AR adapter for timezone juggling
70
+ * fixes to be able to run benchmarks and specs under 1.9.2
71
+
72
+ ## 0.1.7 (May 22nd, 2010)
73
+ * fix a bug when using the disconnect! method on a closed connection in the AR driver
74
+
75
+ ## 0.1.6 (May 14th, 2010)
76
+ * more fixes to the AR adapter related to casting
77
+ * add missing index creation override method to AR adapter
78
+ * added sql_state and error_number methods to the Mysql2::Error exception class
79
+
80
+ ## 0.1.5 (May 12th, 2010)
81
+ * quite a few patches from Eric Wong related to thread-safety, non-blocking I/O and general cleanup
82
+ ** wrap mysql_real_connect with rb_thread_blocking_region
83
+ ** release GVL for possibly blocking mysql_* library calls
84
+ ** [cleanup] quiet down warnings
85
+ ** [cleanup] make all C symbols static
86
+ ** add Mysql2::Client#close method
87
+ ** correctly free the wrapped result in case of EOF
88
+ ** Fix memory leak from the result wrapper struct itself
89
+ ** make Mysql2::Client destructor safely non-blocking
90
+ * bug fixes for ActiveRecord adapter
91
+ ** added casting for default values since they all come back from Mysql as strings (!?!)
92
+ ** missing constant was added
93
+ ** fixed a typo in the show_variable method
94
+ * switched over sscanf for date/time parsing in C
95
+ * made some specs a little finer-grained
96
+ * initial Sequel adapter added
97
+ * updated query benchmarks to reflect the difference between casting in C and in Ruby
98
+
99
+ ## 0.1.4 (April 23rd, 2010)
100
+ * optimization: implemented a local cache for rows that are lazily created in ruby during iteration. The MySQL C result is freed as soon as all the results have been cached
101
+ * optimization: implemented a local cache for field names so every row reuses the same objects as field names/keys
102
+ * refactor the Mysql2 connection adapter for ActiveRecord to not extend the Mysql adapter - now being a free-standing connection adapter
103
+
104
+ ## 0.1.3 (April 15th, 2010)
105
+ * added an EventMachine Deferrable API
106
+ * added an ActiveRecord connection adapter
107
+ ** should be compatible with 2.3.5 and 3.0 (including Arel)
108
+
109
+ ## 0.1.2 (April 9th, 2010)
110
+ * fix a bug (copy/paste fail) around checking for empty TIME values and returning nil (thanks @marius)
111
+
112
+ ## 0.1.1 (April 6th, 2010)
113
+ * added affected_rows method (mysql_affected_rows)
114
+ * added last_id method (last_insert_id)
115
+ * enable reconnect option by default
116
+ * added initial async query support
117
+ * updated extconf (thanks to the mysqlplus project) for easier gem building
118
+
119
+ ## 0.1.0 (April 6th, 2010)
120
+ * initial release
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source :rubygems
2
+
3
+ gemspec
data/MIT-LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2010-2011 Brian Lopez - http://github.com/brianmario
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.rdoc ADDED
@@ -0,0 +1,248 @@
1
+ = Mysql2 - A modern, simple and very fast Mysql library for Ruby - binding to libmysql
2
+
3
+ The Mysql2 gem is meant to serve the extremely common use-case of connecting, querying and iterating on results.
4
+ Some database libraries out there serve as direct 1:1 mappings of the already complex C API's available.
5
+ This one is not.
6
+
7
+ It also forces the use of UTF-8 [or binary] for the connection [and all strings in 1.9, unless Encoding.default_internal is set then it'll convert from UTF-8 to that encoding] and uses encoding-aware MySQL API calls where it can.
8
+
9
+ The API consists of two clases:
10
+
11
+ Mysql2::Client - your connection to the database
12
+
13
+ Mysql2::Result - returned from issuing a #query on the connection. It includes Enumerable.
14
+
15
+ == Installing
16
+
17
+ gem install mysql2
18
+
19
+ You may have to specify --with-mysql-config=/some/random/path/bin/mysql_config
20
+
21
+ == Usage
22
+
23
+ Connect to a database:
24
+
25
+ # this takes a hash of options, almost all of which map directly
26
+ # to the familiar database.yml in rails
27
+ # See http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/MysqlAdapter.html
28
+ client = Mysql2::Client.new(:host => "localhost", :username => "root")
29
+
30
+ Then query it:
31
+
32
+ results = client.query("SELECT * FROM users WHERE group='githubbers'")
33
+
34
+ Need to escape something first?
35
+
36
+ escaped = client.escape("gi'thu\"bbe\0r's")
37
+ results = client.query("SELECT * FROM users WHERE group='#{escaped}'")
38
+
39
+ Finally, iterate over the results:
40
+
41
+ results.each do |row|
42
+ # conveniently, row is a hash
43
+ # the keys are the fields, as you'd expect
44
+ # the values are pre-built ruby primitives mapped from their corresponding field types in MySQL
45
+ # Here's an otter: http://farm1.static.flickr.com/130/398077070_b8795d0ef3_b.jpg
46
+ end
47
+
48
+ Or, you might just keep it simple:
49
+
50
+ client.query("SELECT * FROM users WHERE group='githubbers'").each do |row|
51
+ # do something with row, it's ready to rock
52
+ end
53
+
54
+ How about with symbolized keys?
55
+
56
+ # NOTE: the :symbolize_keys and future options will likely move to the #query method soon
57
+ client.query("SELECT * FROM users WHERE group='githubbers'").each(:symbolize_keys => true) do |row|
58
+ # do something with row, it's ready to rock
59
+ end
60
+
61
+ == Cascading config
62
+
63
+ The default config hash is at:
64
+
65
+ Mysql2::Client.default_query_options
66
+
67
+ which defaults to:
68
+
69
+ {:async => false, :as => :hash, :symbolize_keys => false}
70
+
71
+ that can be used as so:
72
+
73
+ # these are the defaults all Mysql2::Client instances inherit
74
+ Mysql2::Client.default_query_options.merge!(:as => :array)
75
+
76
+ or
77
+
78
+ # this will change the defaults for all future results returned by the #query method _for this connection only_
79
+ c = Mysql2::Client.new
80
+ c.query_options.merge!(:symbolize_keys => true)
81
+
82
+ or
83
+
84
+ # this will set the options for the Mysql2::Result instance returned from the #query method
85
+ c = Mysql2::Client.new
86
+ c.query(sql, :symbolize_keys => true)
87
+
88
+ == Result types
89
+
90
+ === Array of Arrays
91
+
92
+ Pass the :as => :array option to any of the above methods of configuration
93
+
94
+ === Array of Hashes
95
+
96
+ The default result type is set to :hash, but you can override a previous setting to something else with :as => :hash
97
+
98
+ === Others...
99
+
100
+ I may add support for :as => :csv or even :as => :json to allow for *much* more efficient generation of those data types from result sets.
101
+ If you'd like to see either of these (or others), open an issue and start bugging me about it ;)
102
+
103
+ === Timezones
104
+
105
+ Mysql2 now supports two timezone options:
106
+
107
+ :database_timezone - this is the timezone Mysql2 will assume fields are already stored as, and will use this when creating the initial Time objects in ruby
108
+ :application_timezone - this is the timezone Mysql2 will convert to before finally handing back to the caller
109
+
110
+ In other words, if :database_timezone is set to :utc - Mysql2 will create the Time objects using Time.utc(...) from the raw value libmysql hands over initially.
111
+ Then, if :application_timezone is set to say - :local - Mysql2 will then convert the just-created UTC Time object to local time.
112
+
113
+ Both options only allow two values - :local or :utc - with the exception that :application_timezone can be [and defaults to] nil
114
+
115
+ === Casting "boolean" columns
116
+
117
+ You can now tell Mysql2 to cast tinyint(1) fields to boolean values in Ruby with the :cast_booleans option.
118
+
119
+ client = Mysql2::Client.new
120
+ result = client.query("SELECT * FROM table_with_boolean_field", :cast_booleans => true)
121
+
122
+ === Async
123
+
124
+ Mysql2::Client takes advantage of the MySQL C API's (undocumented) non-blocking function mysql_send_query for *all* queries.
125
+ But, in order to take full advantage of it in your Ruby code, you can do:
126
+
127
+ client.query("SELECT sleep(5)", :async => true)
128
+
129
+ Which will return nil immediately. At this point you'll probably want to use some socket monitoring mechanism
130
+ like EventMachine or even IO.select. Once the socket becomes readable, you can do:
131
+
132
+ # result will be a Mysql2::Result instance
133
+ result = client.async_result
134
+
135
+ NOTE: Because of the way MySQL's query API works, this method will block until the result is ready.
136
+ So if you really need things to stay async, it's best to just monitor the socket with something like EventMachine.
137
+ If you need multiple query concurrency take a look at using a connection pool.
138
+
139
+ === Row Caching
140
+
141
+ By default, Mysql2 will cache rows that have been created in Ruby (since this happens lazily).
142
+ This is especially helpful since it saves the cost of creating the row in Ruby if you were to iterate over the collection again.
143
+
144
+ If you only plan on using each row once, then it's much more efficient to disable this behavior by setting the :cache_rows option to false.
145
+ This would be helpful if you wanted to iterate over the results in a streaming manner. Meaning the GC would cleanup rows you don't need anymore as you're iterating over the result set.
146
+
147
+ == ActiveRecord
148
+
149
+ To use the ActiveRecord driver (with our without rails), all you should need to do is have this gem installed and set the adapter in your database.yml to "mysql2".
150
+ That was easy right? :)
151
+
152
+ == Asynchronous ActiveRecord
153
+
154
+ You can also use Mysql2 with asynchronous Rails (first introduced at http://www.mikeperham.com/2010/04/03/introducing-phat-an-asynchronous-rails-app/) by
155
+ setting the adapter in your database.yml to "em_mysql2". You must be running Ruby 1.9, thin and the rack-fiber_pool middleware for it to work.
156
+
157
+ == Sequel
158
+
159
+ The Sequel adapter was pulled out into Sequel core (will be part of the next release) and can be used by specifying the "mysql2://" prefix to your connection specification.
160
+
161
+ == EventMachine
162
+
163
+ The mysql2 EventMachine deferrable api allows you to make async queries using EventMachine,
164
+ while specifying callbacks for success for failure. Here's a simple example:
165
+
166
+ require 'mysql2/em'
167
+
168
+ EM.run do
169
+ client1 = Mysql2::EM::Client.new
170
+ defer1 = client1.query "SELECT sleep(3) as first_query"
171
+ defer1.callback do |result|
172
+ puts "Result: #{result.to_a.inspect}"
173
+ end
174
+
175
+ client2 = Mysql2::EM::Client.new
176
+ defer2 = client2.query "SELECT sleep(1) second_query"
177
+ defer2.callback do |result|
178
+ puts "Result: #{result.to_a.inspect}"
179
+ end
180
+ end
181
+
182
+ == Lazy Everything
183
+
184
+ Well... almost ;)
185
+
186
+ Field name strings/symbols are shared across all the rows so only one object is ever created to represent the field name for an entire dataset.
187
+
188
+ Rows themselves are lazily created in ruby-land when an attempt to yield it is made via #each.
189
+ For example, if you were to yield 4 rows from a 100 row dataset, only 4 hashes will be created. The rest will sit and wait in C-land until you want them (or when the GC goes to cleanup your Mysql2::Result instance).
190
+ Now say you were to iterate over that same collection again, this time yielding 15 rows - the 4 previous rows that had already been turned into ruby hashes would be pulled from an internal cache, then 11 more would be created and stored in that cache.
191
+ Once the entire dataset has been converted into ruby objects, Mysql2::Result will free the Mysql C result object as it's no longer needed.
192
+
193
+ This caching behavior can be disabled by setting the :cache_rows option to false.
194
+
195
+ As for field values themselves, I'm workin on it - but expect that soon.
196
+
197
+ == Compatibility
198
+
199
+ The specs pass on my system (SL 10.6.3, x86_64) in these rubies:
200
+
201
+ * 1.8.7-p249
202
+ * ree-1.8.7-2010.01
203
+ * 1.9.1-p378
204
+ * ruby-trunk
205
+ * rbx-head - broken at the moment, working with the rbx team for a solution
206
+
207
+ The ActiveRecord driver should work on 2.3.5 and 3.0
208
+
209
+ == Yeah... but why?
210
+
211
+ Someone: Dude, the Mysql gem works fiiiiiine.
212
+
213
+ Me: It sure does, but it only hands you nil and strings for field values. Leaving you to convert
214
+ them into proper Ruby types in Ruby-land - which is slow as balls.
215
+
216
+
217
+ Someone: OK fine, but do_mysql can already give me back values with Ruby objects mapped to MySQL types.
218
+
219
+ Me: Yep, but it's API is considerably more complex *and* can be ~2x slower.
220
+
221
+ == Benchmarks
222
+
223
+ Performing a basic "SELECT * FROM" query on a table with 30k rows and fields of nearly every Ruby-representable data type,
224
+ then iterating over every row using an #each like method yielding a block:
225
+
226
+ # These results are from the query_with_mysql_casting.rb script in the benchmarks folder
227
+ user system total real
228
+ Mysql2
229
+ 0.750000 0.180000 0.930000 ( 1.821655)
230
+ do_mysql
231
+ 1.650000 0.200000 1.850000 ( 2.811357)
232
+ Mysql
233
+ 7.500000 0.210000 7.710000 ( 8.065871)
234
+
235
+ == Development
236
+
237
+ To run the tests, you can use RVM and Bundler to create a pristine environment for mysql2 development/hacking.
238
+ Use 'bundle install' to install the necessary development and testing gems:
239
+
240
+ bundle install
241
+ rake
242
+
243
+ == Special Thanks
244
+
245
+ * Eric Wong - for the contribution (and the informative explanations) of some thread-safety, non-blocking I/O and cleanup patches. You rock dude
246
+ * Yury Korolev (http://github.com/yury) - for TONS of help testing the ActiveRecord adapter
247
+ * Aaron Patterson (http://github.com/tenderlove) - tons of contributions, suggestions and general badassness
248
+ * Mike Perham (http://github.com/mperham) - Async ActiveRecord adapter (uses Fibers and EventMachine)
data/Rakefile ADDED
@@ -0,0 +1,5 @@
1
+ # encoding: UTF-8
2
+ require 'rake'
3
+
4
+ # Load custom tasks
5
+ Dir['tasks/*.rake'].sort.each { |f| load f }
@@ -0,0 +1,51 @@
1
+ # encoding: UTF-8
2
+ $LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
3
+
4
+ require 'rubygems'
5
+ require 'benchmark'
6
+ require 'active_record'
7
+
8
+ ActiveRecord::Base.default_timezone = :local
9
+ ActiveRecord::Base.time_zone_aware_attributes = true
10
+
11
+ number_of = 10
12
+ mysql2_opts = {
13
+ :adapter => 'mysql2',
14
+ :database => 'test'
15
+ }
16
+ mysql_opts = {
17
+ :adapter => 'mysql',
18
+ :database => 'test'
19
+ }
20
+
21
+ class Mysql2Model < ActiveRecord::Base
22
+ set_table_name :mysql2_test
23
+ end
24
+
25
+ class MysqlModel < ActiveRecord::Base
26
+ set_table_name :mysql2_test
27
+ end
28
+
29
+ Benchmark.bmbm do |x|
30
+ x.report "Mysql2" do
31
+ Mysql2Model.establish_connection(mysql2_opts)
32
+ number_of.times do
33
+ Mysql2Model.all(:limit => 1000).each{ |r|
34
+ r.attributes.keys.each{ |k|
35
+ r.send(k.to_sym)
36
+ }
37
+ }
38
+ end
39
+ end
40
+
41
+ x.report "Mysql" do
42
+ MysqlModel.establish_connection(mysql_opts)
43
+ number_of.times do
44
+ MysqlModel.all(:limit => 1000).each{ |r|
45
+ r.attributes.keys.each{ |k|
46
+ r.send(k.to_sym)
47
+ }
48
+ }
49
+ end
50
+ end
51
+ end