sequel_pg 1.3.0-x86-mswin32-60 → 1.5.0-x86-mswin32-60
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +10 -0
- data/MIT-LICENSE +29 -1
- data/README.rdoc +33 -7
- data/ext/sequel_pg/extconf.rb +2 -0
- data/ext/sequel_pg/sequel_pg.c +356 -21
- data/lib/1.8/sequel_pg.so +0 -0
- data/lib/1.9/sequel_pg.so +0 -0
- data/lib/sequel_pg/sequel_pg.rb +14 -0
- data/lib/sequel_pg/streaming.rb +82 -0
- metadata +8 -7
data/CHANGELOG
CHANGED
@@ -1,3 +1,13 @@
|
|
1
|
+
=== 1.5.0 (2012-07-02)
|
2
|
+
|
3
|
+
* Add C-based PostgreSQL array parser, for major speedup in parsing arrays (Dan McClain, jeremyevans)
|
4
|
+
|
5
|
+
=== 1.4.0 (2012-06-01)
|
6
|
+
|
7
|
+
* Add support for streaming on PostgreSQL 9.2 using PQsetRowProcessor (jeremyevans)
|
8
|
+
|
9
|
+
* Respect DEBUG environment variable when building (jeremyevans)
|
10
|
+
|
1
11
|
=== 1.3.0 (2012-04-02)
|
2
12
|
|
3
13
|
* Build Windows version against PostgreSQL 9.1.1, ruby 1.8.7, and ruby 1.9.2 (previously 9.0.1, 1.8.6, and 1.9.1) (jeremyevans)
|
data/MIT-LICENSE
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
Copyright (c) 2010-
|
1
|
+
Copyright (c) 2010-2012 Jeremy Evans
|
2
2
|
|
3
3
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
4
4
|
of this software and associated documentation files (the "Software"), to
|
@@ -17,3 +17,31 @@ THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
|
17
17
|
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
18
|
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
19
|
|
20
|
+
|
21
|
+
The original array parsing code (parse_pg_array, read_array) was taken from
|
22
|
+
the pg_array_parser library (https://github.com/dockyard/pg_array_parser)
|
23
|
+
and has the following license:
|
24
|
+
|
25
|
+
Copyright (c) 2012 Dan McClain
|
26
|
+
|
27
|
+
MIT License
|
28
|
+
|
29
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
30
|
+
a copy of this software and associated documentation files (the
|
31
|
+
"Software"), to deal in the Software without restriction, including
|
32
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
33
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
34
|
+
permit persons to whom the Software is furnished to do so, subject to
|
35
|
+
the following conditions:
|
36
|
+
|
37
|
+
The above copyright notice and this permission notice shall be
|
38
|
+
included in all copies or substantial portions of the Software.
|
39
|
+
|
40
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
41
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
42
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
43
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
44
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
45
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
46
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
47
|
+
|
data/README.rdoc
CHANGED
@@ -38,13 +38,13 @@ Here's an example that uses a modified version of swift's benchmarks
|
|
38
38
|
sequel #select 0.090000 2.020000 2.110000 2.246688 46.54m
|
39
39
|
sequel_pg #select 0.000000 0.250000 0.250000 0.361999 7.33m
|
40
40
|
|
41
|
-
sequel_pg also has code to speed up the map, to_hash,
|
42
|
-
select_map, and select_order_map
|
43
|
-
default. It also has code to speed
|
44
|
-
which is off by default as it isn't
|
45
|
-
handle overriding Model.call,
|
46
|
-
Model#after_initialize, which may cause problems
|
47
|
-
following plugins that ship with Sequel:
|
41
|
+
sequel_pg also has code to speed up the map, to_hash, to_hash_groups,
|
42
|
+
select_hash, select_hash_groups, select_map, and select_order_map
|
43
|
+
Dataset methods, which is on by default. It also has code to speed
|
44
|
+
up the loading of model objects, which is off by default as it isn't
|
45
|
+
fully compatible. It doesn't handle overriding Model.call,
|
46
|
+
Model#set_values, or Model#after_initialize, which may cause problems
|
47
|
+
with the following plugins that ship with Sequel:
|
48
48
|
|
49
49
|
* class_table_inheritance
|
50
50
|
* force_encoding
|
@@ -63,6 +63,32 @@ enable the model optimization via:
|
|
63
63
|
# Specific dataset
|
64
64
|
Artist.dataset.optimize_model_load = true
|
65
65
|
|
66
|
+
== Streaming
|
67
|
+
|
68
|
+
If you are using PostgreSQL 9.2 or higher on the client, then sequel_pg
|
69
|
+
should enable streaming support. This allows you to stream returned
|
70
|
+
rows one at a time, instead of collecting the entire result set in
|
71
|
+
memory (which is how PostgreSQL works by default). You can check
|
72
|
+
if streaming is supported by:
|
73
|
+
|
74
|
+
Sequel::Postgres.supports_streaming?
|
75
|
+
|
76
|
+
If streaming is supported, you can load the streaming support into the
|
77
|
+
database:
|
78
|
+
|
79
|
+
require 'sequel_pg/streaming'
|
80
|
+
DB.extend Sequel::Postgres::Streaming
|
81
|
+
|
82
|
+
Then you can call the Dataset#stream method to have the dataset use
|
83
|
+
the streaming support:
|
84
|
+
|
85
|
+
DB[:table].stream.each{|row| ...}
|
86
|
+
|
87
|
+
If you want to enable streaming for all of a database's datasets, you
|
88
|
+
can do the following:
|
89
|
+
|
90
|
+
DB.extend_datasets Sequel::Postgres::Streaming::AllQueries
|
91
|
+
|
66
92
|
== Installing the gem
|
67
93
|
|
68
94
|
gem install sequel_pg
|
data/ext/sequel_pg/extconf.rb
CHANGED
@@ -1,4 +1,5 @@
|
|
1
1
|
require 'mkmf'
|
2
|
+
$CFLAGS << " -O0 -g -ggdb" if ENV['DEBUG']
|
2
3
|
$CFLAGS << " -Wall " unless RUBY_PLATFORM =~ /solaris/
|
3
4
|
dir_config('pg', ENV["POSTGRES_INCLUDE"] || (IO.popen("pg_config --includedir").readline.chomp rescue nil),
|
4
5
|
ENV["POSTGRES_LIB"] || (IO.popen("pg_config --libdir").readline.chomp rescue nil))
|
@@ -13,6 +14,7 @@ if enable_config("static-build")
|
|
13
14
|
end
|
14
15
|
|
15
16
|
if (have_library('pq') || have_library('libpq') || have_library('ms/libpq')) && have_header('libpq-fe.h')
|
17
|
+
have_func 'PQsetRowProcessor'
|
16
18
|
create_makefile("sequel_pg")
|
17
19
|
else
|
18
20
|
puts 'Could not find PostgreSQL build environment (libraries & headers): Makefile not created'
|
data/ext/sequel_pg/sequel_pg.c
CHANGED
@@ -41,11 +41,23 @@
|
|
41
41
|
#define SPG_YIELD_KMV_HASH_GROUPS 12
|
42
42
|
#define SPG_YIELD_MKMV_HASH_GROUPS 13
|
43
43
|
|
44
|
+
struct spg_row_proc_info {
|
45
|
+
VALUE dataset;
|
46
|
+
VALUE block;
|
47
|
+
VALUE model;
|
48
|
+
VALUE colsyms[SPG_MAX_FIELDS];
|
49
|
+
VALUE colconvert[SPG_MAX_FIELDS];
|
50
|
+
#if SPG_ENCODING
|
51
|
+
int enc_index;
|
52
|
+
#endif
|
53
|
+
};
|
54
|
+
|
44
55
|
static VALUE spg_Sequel;
|
45
56
|
static VALUE spg_Blob;
|
46
57
|
static VALUE spg_BigDecimal;
|
47
58
|
static VALUE spg_Date;
|
48
59
|
static VALUE spg_SQLTime;
|
60
|
+
static VALUE spg_PGError;
|
49
61
|
|
50
62
|
static VALUE spg_sym_utc;
|
51
63
|
static VALUE spg_sym_local;
|
@@ -102,6 +114,115 @@ static int enc_get_index(VALUE val)
|
|
102
114
|
}
|
103
115
|
#endif
|
104
116
|
|
117
|
+
static VALUE read_array(int *index, char *c_pg_array_string, int array_string_length, char *word, VALUE converter)
|
118
|
+
{
|
119
|
+
int word_index = 0;
|
120
|
+
|
121
|
+
/* The current character in the input string. */
|
122
|
+
char c;
|
123
|
+
|
124
|
+
/* 0: Currently outside a quoted string, current word never quoted
|
125
|
+
* 1: Currently inside a quoted string
|
126
|
+
* -1: Currently outside a quoted string, current word previously quoted */
|
127
|
+
int openQuote = 0;
|
128
|
+
|
129
|
+
/* Inside quoted input means the next character should be treated literally,
|
130
|
+
* instead of being treated as a metacharacter.
|
131
|
+
* Outside of quoted input, means that the word shouldn't be pushed to the array,
|
132
|
+
* used when the last entry was a subarray (which adds to the array itself). */
|
133
|
+
int escapeNext = 0;
|
134
|
+
|
135
|
+
VALUE array = rb_ary_new();
|
136
|
+
|
137
|
+
/* Special case the empty array, so it doesn't need to be handled manually inside
|
138
|
+
* the loop. */
|
139
|
+
if(((*index) < array_string_length) && c_pg_array_string[(*index)] == '}')
|
140
|
+
{
|
141
|
+
return array;
|
142
|
+
}
|
143
|
+
|
144
|
+
for(;(*index) < array_string_length; ++(*index))
|
145
|
+
{
|
146
|
+
c = c_pg_array_string[*index];
|
147
|
+
if(openQuote < 1)
|
148
|
+
{
|
149
|
+
if(c == ',' || c == '}')
|
150
|
+
{
|
151
|
+
if(!escapeNext)
|
152
|
+
{
|
153
|
+
if(openQuote == 0 && word_index == 4 && !strncmp(word, "NULL", word_index))
|
154
|
+
{
|
155
|
+
rb_ary_push(array, Qnil);
|
156
|
+
}
|
157
|
+
else if (RTEST(converter))
|
158
|
+
{
|
159
|
+
rb_ary_push(array, rb_funcall(converter, spg_id_call, 1, rb_str_new(word, word_index)));
|
160
|
+
}
|
161
|
+
else
|
162
|
+
{
|
163
|
+
rb_ary_push(array, rb_str_new(word, word_index));
|
164
|
+
}
|
165
|
+
}
|
166
|
+
if(c == '}')
|
167
|
+
{
|
168
|
+
return array;
|
169
|
+
}
|
170
|
+
escapeNext = 0;
|
171
|
+
openQuote = 0;
|
172
|
+
word_index = 0;
|
173
|
+
}
|
174
|
+
else if(c == '"')
|
175
|
+
{
|
176
|
+
openQuote = 1;
|
177
|
+
}
|
178
|
+
else if(c == '{')
|
179
|
+
{
|
180
|
+
(*index)++;
|
181
|
+
rb_ary_push(array, read_array(index, c_pg_array_string, array_string_length, word, converter));
|
182
|
+
escapeNext = 1;
|
183
|
+
}
|
184
|
+
else
|
185
|
+
{
|
186
|
+
word[word_index] = c;
|
187
|
+
word_index++;
|
188
|
+
}
|
189
|
+
}
|
190
|
+
else if (escapeNext) {
|
191
|
+
word[word_index] = c;
|
192
|
+
word_index++;
|
193
|
+
escapeNext = 0;
|
194
|
+
}
|
195
|
+
else if (c == '\\')
|
196
|
+
{
|
197
|
+
escapeNext = 1;
|
198
|
+
}
|
199
|
+
else if (c == '"')
|
200
|
+
{
|
201
|
+
openQuote = -1;
|
202
|
+
}
|
203
|
+
else
|
204
|
+
{
|
205
|
+
word[word_index] = c;
|
206
|
+
word_index++;
|
207
|
+
}
|
208
|
+
}
|
209
|
+
|
210
|
+
return array;
|
211
|
+
}
|
212
|
+
|
213
|
+
static VALUE parse_pg_array(VALUE self, VALUE pg_array_string, VALUE converter) {
|
214
|
+
|
215
|
+
/* convert to c-string, create additional ruby string buffer of
|
216
|
+
* the same length, as that will be the worst case. */
|
217
|
+
char *c_pg_array_string = StringValueCStr(pg_array_string);
|
218
|
+
int array_string_length = RSTRING_LEN(pg_array_string);
|
219
|
+
VALUE buf = rb_str_buf_new(array_string_length);
|
220
|
+
char *word = RSTRING_PTR(buf);
|
221
|
+
int index = 1;
|
222
|
+
|
223
|
+
return read_array(&index, c_pg_array_string, array_string_length, word, converter);
|
224
|
+
}
|
225
|
+
|
105
226
|
static VALUE spg_time(const char *s) {
|
106
227
|
VALUE now;
|
107
228
|
int hour, minute, second, tokens;
|
@@ -399,32 +520,13 @@ static VALUE spg__field_ids(VALUE v, VALUE *colsyms, long nfields) {
|
|
399
520
|
return pg_columns;
|
400
521
|
}
|
401
522
|
|
402
|
-
static
|
403
|
-
PGresult *res;
|
404
|
-
VALUE colsyms[SPG_MAX_FIELDS];
|
405
|
-
VALUE colconvert[SPG_MAX_FIELDS];
|
406
|
-
long ntuples;
|
407
|
-
long nfields;
|
523
|
+
static void spg_set_column_info(VALUE self, PGresult *res, VALUE *colsyms, VALUE *colconvert) {
|
408
524
|
long i;
|
409
525
|
long j;
|
410
|
-
|
526
|
+
long nfields;
|
411
527
|
VALUE conv_procs = 0;
|
412
|
-
VALUE opts;
|
413
|
-
VALUE pg_type;
|
414
|
-
VALUE pg_value;
|
415
|
-
char type = SPG_YIELD_NORMAL;
|
416
|
-
|
417
|
-
#ifdef SPG_ENCODING
|
418
|
-
int enc_index;
|
419
|
-
enc_index = enc_get_index(rres);
|
420
|
-
#endif
|
421
528
|
|
422
|
-
Data_Get_Struct(rres, PGresult, res);
|
423
|
-
ntuples = PQntuples(res);
|
424
529
|
nfields = PQnfields(res);
|
425
|
-
if (nfields > SPG_MAX_FIELDS) {
|
426
|
-
rb_raise(rb_eRangeError, "more than %d columns in query", SPG_MAX_FIELDS);
|
427
|
-
}
|
428
530
|
|
429
531
|
for(j=0; j<nfields; j++) {
|
430
532
|
colsyms[j] = rb_funcall(self, spg_id_output_identifier, 1, rb_str_new2(PQfname(res, j)));
|
@@ -459,6 +561,36 @@ static VALUE spg_yield_hash_rows(VALUE self, VALUE rres, VALUE ignore) {
|
|
459
561
|
break;
|
460
562
|
}
|
461
563
|
}
|
564
|
+
}
|
565
|
+
|
566
|
+
static VALUE spg_yield_hash_rows(VALUE self, VALUE rres, VALUE ignore) {
|
567
|
+
PGresult *res;
|
568
|
+
VALUE colsyms[SPG_MAX_FIELDS];
|
569
|
+
VALUE colconvert[SPG_MAX_FIELDS];
|
570
|
+
long ntuples;
|
571
|
+
long nfields;
|
572
|
+
long i;
|
573
|
+
long j;
|
574
|
+
VALUE h;
|
575
|
+
VALUE opts;
|
576
|
+
VALUE pg_type;
|
577
|
+
VALUE pg_value;
|
578
|
+
char type = SPG_YIELD_NORMAL;
|
579
|
+
|
580
|
+
#ifdef SPG_ENCODING
|
581
|
+
int enc_index;
|
582
|
+
enc_index = enc_get_index(rres);
|
583
|
+
#endif
|
584
|
+
|
585
|
+
Data_Get_Struct(rres, PGresult, res);
|
586
|
+
ntuples = PQntuples(res);
|
587
|
+
nfields = PQnfields(res);
|
588
|
+
if (nfields > SPG_MAX_FIELDS) {
|
589
|
+
rb_raise(rb_eRangeError, "more than %d columns in query", SPG_MAX_FIELDS);
|
590
|
+
}
|
591
|
+
|
592
|
+
spg_set_column_info(self, res, colsyms, colconvert);
|
593
|
+
|
462
594
|
rb_ivar_set(self, spg_id_columns, rb_ary_new4(nfields, colsyms));
|
463
595
|
|
464
596
|
opts = rb_funcall(self, spg_id_opts, 0);
|
@@ -675,6 +807,199 @@ static VALUE spg_yield_hash_rows(VALUE self, VALUE rres, VALUE ignore) {
|
|
675
807
|
return self;
|
676
808
|
}
|
677
809
|
|
810
|
+
static VALUE spg_supports_streaming_p(VALUE self) {
|
811
|
+
return
|
812
|
+
#if HAVE_PQSETROWPROCESSOR
|
813
|
+
Qtrue;
|
814
|
+
#else
|
815
|
+
Qfalse;
|
816
|
+
#endif
|
817
|
+
}
|
818
|
+
|
819
|
+
#if HAVE_PQSETROWPROCESSOR
|
820
|
+
static VALUE spg__rp_value(VALUE self, PGresult* res, const PGdataValue* dvs, int j, VALUE* colconvert
|
821
|
+
#ifdef SPG_ENCODING
|
822
|
+
, int enc_index
|
823
|
+
#endif
|
824
|
+
) {
|
825
|
+
const char *v;
|
826
|
+
PGdataValue dv = dvs[j];
|
827
|
+
VALUE rv;
|
828
|
+
size_t l;
|
829
|
+
int len = dv.len;
|
830
|
+
|
831
|
+
if(len < 0) {
|
832
|
+
rv = Qnil;
|
833
|
+
} else {
|
834
|
+
v = dv.value;
|
835
|
+
|
836
|
+
switch(PQftype(res, j)) {
|
837
|
+
case 16: /* boolean */
|
838
|
+
rv = *v == 't' ? Qtrue : Qfalse;
|
839
|
+
break;
|
840
|
+
case 17: /* bytea */
|
841
|
+
v = PQunescapeBytea((unsigned char*)v, &l);
|
842
|
+
rv = rb_funcall(spg_Blob, spg_id_new, 1, rb_str_new(v, l));
|
843
|
+
PQfreemem((char *)v);
|
844
|
+
break;
|
845
|
+
case 20: /* integer */
|
846
|
+
case 21:
|
847
|
+
case 22:
|
848
|
+
case 23:
|
849
|
+
case 26:
|
850
|
+
rv = rb_str2inum(rb_str_new(v, len), 10);
|
851
|
+
break;
|
852
|
+
case 700: /* float */
|
853
|
+
case 701:
|
854
|
+
if (strncmp("NaN", v, 3) == 0) {
|
855
|
+
rv = spg_nan;
|
856
|
+
} else if (strncmp("Infinity", v, 8) == 0) {
|
857
|
+
rv = spg_pos_inf;
|
858
|
+
} else if (strncmp("-Infinity", v, 9) == 0) {
|
859
|
+
rv = spg_neg_inf;
|
860
|
+
} else {
|
861
|
+
rv = rb_float_new(rb_str_to_dbl(rb_str_new(v, len), Qfalse));
|
862
|
+
}
|
863
|
+
break;
|
864
|
+
case 790: /* numeric */
|
865
|
+
case 1700:
|
866
|
+
rv = rb_funcall(spg_BigDecimal, spg_id_new, 1, rb_str_new(v, len));
|
867
|
+
break;
|
868
|
+
case 1082: /* date */
|
869
|
+
rv = rb_str_new(v, len);
|
870
|
+
rv = spg_date(StringValuePtr(rv));
|
871
|
+
break;
|
872
|
+
case 1083: /* time */
|
873
|
+
case 1266:
|
874
|
+
rv = rb_str_new(v, len);
|
875
|
+
rv = spg_time(StringValuePtr(rv));
|
876
|
+
break;
|
877
|
+
case 1114: /* timestamp */
|
878
|
+
case 1184:
|
879
|
+
rv = rb_str_new(v, len);
|
880
|
+
rv = spg_timestamp(StringValuePtr(rv), self);
|
881
|
+
break;
|
882
|
+
case 18: /* char */
|
883
|
+
case 25: /* text */
|
884
|
+
case 1043: /* varchar*/
|
885
|
+
rv = rb_tainted_str_new(v, len);
|
886
|
+
#ifdef SPG_ENCODING
|
887
|
+
rb_enc_associate_index(rv, enc_index);
|
888
|
+
#endif
|
889
|
+
break;
|
890
|
+
default:
|
891
|
+
rv = rb_tainted_str_new(v, len);
|
892
|
+
#ifdef SPG_ENCODING
|
893
|
+
rb_enc_associate_index(rv, enc_index);
|
894
|
+
#endif
|
895
|
+
if (colconvert[j] != Qnil) {
|
896
|
+
rv = rb_funcall(colconvert[j], spg_id_call, 1, rv);
|
897
|
+
}
|
898
|
+
}
|
899
|
+
}
|
900
|
+
return rv;
|
901
|
+
}
|
902
|
+
|
903
|
+
static int spg_row_processor(PGresult *res, const PGdataValue *columns, const char **errmsgp, void *param) {
|
904
|
+
long nfields;
|
905
|
+
struct spg_row_proc_info *info;
|
906
|
+
info = (struct spg_row_proc_info *)param;
|
907
|
+
VALUE *colsyms = info->colsyms;
|
908
|
+
VALUE *colconvert = info->colconvert;
|
909
|
+
VALUE self = info->dataset;
|
910
|
+
|
911
|
+
switch (PQresultStatus(res))
|
912
|
+
{
|
913
|
+
case PGRES_TUPLES_OK:
|
914
|
+
case PGRES_COPY_OUT:
|
915
|
+
case PGRES_COPY_IN:
|
916
|
+
#ifdef HAVE_CONST_PGRES_COPY_BOTH
|
917
|
+
case PGRES_COPY_BOTH:
|
918
|
+
#endif
|
919
|
+
case PGRES_EMPTY_QUERY:
|
920
|
+
case PGRES_COMMAND_OK:
|
921
|
+
break;
|
922
|
+
case PGRES_BAD_RESPONSE:
|
923
|
+
case PGRES_FATAL_ERROR:
|
924
|
+
case PGRES_NONFATAL_ERROR:
|
925
|
+
rb_raise(spg_PGError, "error while streaming results");
|
926
|
+
default:
|
927
|
+
rb_raise(spg_PGError, "unexpected result status while streaming results");
|
928
|
+
}
|
929
|
+
|
930
|
+
nfields = PQnfields(res);
|
931
|
+
if(columns == NULL) {
|
932
|
+
spg_set_column_info(self, res, colsyms, colconvert);
|
933
|
+
rb_ivar_set(self, spg_id_columns, rb_ary_new4(nfields, colsyms));
|
934
|
+
} else {
|
935
|
+
long j;
|
936
|
+
VALUE h, m;
|
937
|
+
h = rb_hash_new();
|
938
|
+
|
939
|
+
for(j=0; j<nfields; j++) {
|
940
|
+
rb_hash_aset(h, colsyms[j], spg__rp_value(self, res, columns, j, colconvert
|
941
|
+
#ifdef SPG_ENCODING
|
942
|
+
, info->enc_index
|
943
|
+
#endif
|
944
|
+
));
|
945
|
+
}
|
946
|
+
|
947
|
+
/* optimize_model_load used, return model instance */
|
948
|
+
if ((m = info->model)) {
|
949
|
+
m = rb_obj_alloc(m);
|
950
|
+
rb_ivar_set(m, spg_id_values, h);
|
951
|
+
h = m;
|
952
|
+
}
|
953
|
+
|
954
|
+
rb_funcall(info->block, spg_id_call, 1, h);
|
955
|
+
}
|
956
|
+
return 1;
|
957
|
+
}
|
958
|
+
|
959
|
+
static VALUE spg_unset_row_processor(VALUE rconn) {
|
960
|
+
PGconn *conn;
|
961
|
+
Data_Get_Struct(rconn, PGconn, conn);
|
962
|
+
if ((PQskipResult(conn)) != NULL) {
|
963
|
+
/* Results remaining when row processor finished,
|
964
|
+
* either because an exception was raised or the iterator
|
965
|
+
* exited early, so skip all remaining rows. */
|
966
|
+
while(PQgetResult(conn) != NULL) {
|
967
|
+
/* Use a separate while loop as PQgetResult is faster than
|
968
|
+
* PQskipResult. */
|
969
|
+
}
|
970
|
+
}
|
971
|
+
PQsetRowProcessor(conn, NULL, NULL);
|
972
|
+
return Qnil;
|
973
|
+
}
|
974
|
+
|
975
|
+
static VALUE spg_with_row_processor(VALUE self, VALUE rconn, VALUE dataset, VALUE block) {
|
976
|
+
struct spg_row_proc_info info;
|
977
|
+
PGconn *conn;
|
978
|
+
Data_Get_Struct(rconn, PGconn, conn);
|
979
|
+
bzero(&info, sizeof(info));
|
980
|
+
|
981
|
+
info.dataset = dataset;
|
982
|
+
info.block = block;
|
983
|
+
info.model = 0;
|
984
|
+
#if SPG_ENCODING
|
985
|
+
info.enc_index = enc_get_index(rconn);
|
986
|
+
#endif
|
987
|
+
|
988
|
+
/* Abuse local variable, detect if optimize_model_load used */
|
989
|
+
block = rb_funcall(dataset, spg_id_opts, 0);
|
990
|
+
if (rb_type(block) == T_HASH && rb_hash_aref(block, spg_sym__sequel_pg_type) == spg_sym_model) {
|
991
|
+
block = rb_hash_aref(block, spg_sym__sequel_pg_value);
|
992
|
+
if (rb_type(block) == T_CLASS) {
|
993
|
+
info.model = block;
|
994
|
+
}
|
995
|
+
}
|
996
|
+
|
997
|
+
PQsetRowProcessor(conn, spg_row_processor, (void*)&info);
|
998
|
+
rb_ensure(rb_yield, Qnil, spg_unset_row_processor, rconn);
|
999
|
+
return Qnil;
|
1000
|
+
}
|
1001
|
+
#endif
|
1002
|
+
|
678
1003
|
void Init_sequel_pg(void) {
|
679
1004
|
VALUE c, spg_Postgres;
|
680
1005
|
ID cg;
|
@@ -725,6 +1050,7 @@ void Init_sequel_pg(void) {
|
|
725
1050
|
spg_BigDecimal = rb_funcall(rb_cObject, cg, 1, rb_str_new2("BigDecimal"));
|
726
1051
|
spg_Date = rb_funcall(rb_cObject, cg, 1, rb_str_new2("Date"));
|
727
1052
|
spg_Postgres = rb_funcall(spg_Sequel, cg, 1, rb_str_new2("Postgres"));
|
1053
|
+
spg_PGError = rb_funcall(rb_cObject, cg, 1, rb_str_new2("PGError"));
|
728
1054
|
|
729
1055
|
spg_nan = rb_eval_string("0.0/0.0");
|
730
1056
|
spg_pos_inf = rb_eval_string("1.0/0.0");
|
@@ -749,5 +1075,14 @@ void Init_sequel_pg(void) {
|
|
749
1075
|
rb_define_private_method(c, "yield_hash_rows", spg_yield_hash_rows, 2);
|
750
1076
|
rb_define_private_method(c, "fetch_rows_set_cols", spg_fetch_rows_set_cols, 1);
|
751
1077
|
|
1078
|
+
rb_define_singleton_method(spg_Postgres, "supports_streaming?", spg_supports_streaming_p, 0);
|
1079
|
+
|
1080
|
+
#if HAVE_PQSETROWPROCESSOR
|
1081
|
+
c = rb_funcall(spg_Postgres, cg, 1, rb_str_new2("Database"));
|
1082
|
+
rb_define_private_method(c, "with_row_processor", spg_with_row_processor, 3);
|
1083
|
+
#endif
|
1084
|
+
|
1085
|
+
rb_define_singleton_method(spg_Postgres, "parse_pg_array", parse_pg_array, 2);
|
1086
|
+
|
752
1087
|
rb_require("sequel_pg/sequel_pg");
|
753
1088
|
}
|
data/lib/1.8/sequel_pg.so
CHANGED
Binary file
|
data/lib/1.9/sequel_pg.so
CHANGED
Binary file
|
data/lib/sequel_pg/sequel_pg.rb
CHANGED
@@ -83,3 +83,17 @@ class Sequel::Postgres::Dataset
|
|
83
83
|
(rp = row_proc).is_a?(Class) && (rp < Sequel::Model) && optimize_model_load && !opts[:use_cursor] && !opts[:graph]
|
84
84
|
end
|
85
85
|
end
|
86
|
+
|
87
|
+
if defined?(Sequel::Postgres::PGArray)
|
88
|
+
# pg_array extension previously loaded
|
89
|
+
|
90
|
+
class Sequel::Postgres::PGArray::Creator
|
91
|
+
# Override Creator to use sequel_pg's C-based parser instead of the pure ruby parser.
|
92
|
+
def call(string)
|
93
|
+
Sequel::Postgres::PGArray.new(Sequel::Postgres.parse_pg_array(string, @converter), @type)
|
94
|
+
end
|
95
|
+
end
|
96
|
+
|
97
|
+
# Remove the pure-ruby parser, no longer needed.
|
98
|
+
Sequel::Postgres::PGArray.send(:remove_const, :Parser)
|
99
|
+
end
|
@@ -0,0 +1,82 @@
|
|
1
|
+
unless Sequel::Postgres.respond_to?(:supports_streaming?)
|
2
|
+
raise LoadError, "either sequel_pg not loaded, or an old version of sequel_pg loaded"
|
3
|
+
end
|
4
|
+
unless Sequel::Postgres.supports_streaming?
|
5
|
+
raise LoadError, "streaming is not supported by the version of libpq in use"
|
6
|
+
end
|
7
|
+
|
8
|
+
# Database methods necessary to support streaming. You should extend your
|
9
|
+
# Database object with this:
|
10
|
+
#
|
11
|
+
# DB.extend Sequel::Postgres::Streaming
|
12
|
+
#
|
13
|
+
# Then you can call #stream on your datasets to use the streaming support:
|
14
|
+
#
|
15
|
+
# DB[:table].stream.each{|row| ...}
|
16
|
+
module Sequel::Postgres::Streaming
|
17
|
+
# Also extend the database's datasets to support streaming
|
18
|
+
def self.extended(db)
|
19
|
+
db.extend_datasets(DatasetMethods)
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
|
24
|
+
# If streaming is requested, set a row processor while executing
|
25
|
+
# the query.
|
26
|
+
def _execute(conn, sql, opts={})
|
27
|
+
if stream = opts[:stream]
|
28
|
+
with_row_processor(conn, *stream){super}
|
29
|
+
else
|
30
|
+
super
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
# Dataset methods used to implement streaming.
|
35
|
+
module DatasetMethods
|
36
|
+
# If streaming has been requested and the current dataset
|
37
|
+
# can be streamed, request the database use streaming when
|
38
|
+
# executing this query.
|
39
|
+
def fetch_rows(sql, &block)
|
40
|
+
if stream_results?
|
41
|
+
execute(sql, :stream=>[self, block])
|
42
|
+
else
|
43
|
+
super
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
# Return a clone of the dataset that will use streaming to load
|
48
|
+
# rows.
|
49
|
+
def stream
|
50
|
+
clone(:stream=>true)
|
51
|
+
end
|
52
|
+
|
53
|
+
private
|
54
|
+
|
55
|
+
# Only stream results if streaming has been specifically requested
|
56
|
+
# and the query is streamable.
|
57
|
+
def stream_results?
|
58
|
+
@opts[:stream] && streamable?
|
59
|
+
end
|
60
|
+
|
61
|
+
# Queries using cursors are not streamable, and queries that use
|
62
|
+
# the map/select_map/to_hash/to_hash_groups optimizations are not
|
63
|
+
# streamable, but other queries are streamable.
|
64
|
+
def streamable?
|
65
|
+
spgt = (o = @opts)[:_sequel_pg_type]
|
66
|
+
(spgt.nil? || spgt == :model) && !o[:cursor]
|
67
|
+
end
|
68
|
+
end
|
69
|
+
|
70
|
+
# Extend a database's datasets with this module to enable streaming
|
71
|
+
# on all streamable queries:
|
72
|
+
#
|
73
|
+
# DB.extend_datasets(Sequel::Postgres::Streaming::AllQueries)
|
74
|
+
module AllQueries
|
75
|
+
private
|
76
|
+
|
77
|
+
# Always stream results if the query is streamable.
|
78
|
+
def stream_results?
|
79
|
+
streamable?
|
80
|
+
end
|
81
|
+
end
|
82
|
+
end
|
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sequel_pg
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 3
|
5
5
|
prerelease:
|
6
6
|
segments:
|
7
7
|
- 1
|
8
|
-
-
|
8
|
+
- 5
|
9
9
|
- 0
|
10
|
-
version: 1.
|
10
|
+
version: 1.5.0
|
11
11
|
platform: x86-mswin32-60
|
12
12
|
authors:
|
13
13
|
- Jeremy Evans
|
@@ -15,7 +15,7 @@ autorequire:
|
|
15
15
|
bindir: bin
|
16
16
|
cert_chain: []
|
17
17
|
|
18
|
-
date: 2012-
|
18
|
+
date: 2012-04-07 00:00:00 Z
|
19
19
|
dependencies:
|
20
20
|
- !ruby/object:Gem::Dependency
|
21
21
|
name: pg
|
@@ -41,12 +41,12 @@ dependencies:
|
|
41
41
|
requirements:
|
42
42
|
- - ">="
|
43
43
|
- !ruby/object:Gem::Version
|
44
|
-
hash:
|
44
|
+
hash: 151
|
45
45
|
segments:
|
46
46
|
- 3
|
47
|
-
-
|
47
|
+
- 36
|
48
48
|
- 0
|
49
|
-
version: 3.
|
49
|
+
version: 3.36.0
|
50
50
|
type: :runtime
|
51
51
|
version_requirements: *id002
|
52
52
|
description: |
|
@@ -72,6 +72,7 @@ files:
|
|
72
72
|
- ext/sequel_pg/extconf.rb
|
73
73
|
- ext/sequel_pg/sequel_pg.c
|
74
74
|
- lib/sequel_pg/sequel_pg.rb
|
75
|
+
- lib/sequel_pg/streaming.rb
|
75
76
|
- lib/1.8/sequel_pg.so
|
76
77
|
- lib/1.9/sequel_pg.so
|
77
78
|
homepage: http://github.com/jeremyevans/sequel_pg
|