pg_query 1.0.2 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/CHANGELOG.md +162 -40
- data/README.md +80 -69
- data/Rakefile +85 -4
- data/ext/pg_query/extconf.rb +4 -32
- data/ext/pg_query/guc-file.c +0 -0
- data/ext/pg_query/pg_query.c +104 -0
- data/ext/pg_query/pg_query.pb-c.c +37628 -0
- data/ext/pg_query/pg_query_deparse.c +9953 -0
- data/ext/pg_query/pg_query_fingerprint.c +292 -0
- data/ext/pg_query/pg_query_fingerprint.h +8 -0
- data/ext/pg_query/pg_query_internal.h +24 -0
- data/ext/pg_query/pg_query_json_plpgsql.c +738 -0
- data/ext/pg_query/pg_query_json_plpgsql.h +9 -0
- data/ext/pg_query/pg_query_normalize.c +437 -0
- data/ext/pg_query/pg_query_outfuncs.h +10 -0
- data/ext/pg_query/pg_query_outfuncs_json.c +297 -0
- data/ext/pg_query/pg_query_outfuncs_protobuf.c +237 -0
- data/ext/pg_query/pg_query_parse.c +148 -0
- data/ext/pg_query/pg_query_parse_plpgsql.c +460 -0
- data/ext/pg_query/pg_query_readfuncs.h +11 -0
- data/ext/pg_query/pg_query_readfuncs_protobuf.c +142 -0
- data/ext/pg_query/pg_query_ruby.c +108 -12
- data/ext/pg_query/pg_query_scan.c +173 -0
- data/ext/pg_query/pg_query_split.c +221 -0
- data/ext/pg_query/protobuf-c.c +3660 -0
- data/ext/pg_query/src_backend_catalog_namespace.c +1051 -0
- data/ext/pg_query/src_backend_catalog_pg_proc.c +142 -0
- data/ext/pg_query/src_backend_commands_define.c +117 -0
- data/ext/pg_query/src_backend_libpq_pqcomm.c +651 -0
- data/ext/pg_query/src_backend_nodes_bitmapset.c +513 -0
- data/ext/pg_query/src_backend_nodes_copyfuncs.c +6013 -0
- data/ext/pg_query/src_backend_nodes_equalfuncs.c +4003 -0
- data/ext/pg_query/src_backend_nodes_extensible.c +99 -0
- data/ext/pg_query/src_backend_nodes_list.c +922 -0
- data/ext/pg_query/src_backend_nodes_makefuncs.c +417 -0
- data/ext/pg_query/src_backend_nodes_nodeFuncs.c +1363 -0
- data/ext/pg_query/src_backend_nodes_value.c +84 -0
- data/ext/pg_query/src_backend_parser_gram.c +47456 -0
- data/ext/pg_query/src_backend_parser_parse_expr.c +313 -0
- data/ext/pg_query/src_backend_parser_parser.c +497 -0
- data/ext/pg_query/src_backend_parser_scan.c +7091 -0
- data/ext/pg_query/src_backend_parser_scansup.c +160 -0
- data/ext/pg_query/src_backend_postmaster_postmaster.c +2230 -0
- data/ext/pg_query/src_backend_storage_ipc_ipc.c +192 -0
- data/ext/pg_query/src_backend_storage_lmgr_s_lock.c +370 -0
- data/ext/pg_query/src_backend_tcop_postgres.c +776 -0
- data/ext/pg_query/src_backend_utils_adt_datum.c +326 -0
- data/ext/pg_query/src_backend_utils_adt_expandeddatum.c +98 -0
- data/ext/pg_query/src_backend_utils_adt_format_type.c +136 -0
- data/ext/pg_query/src_backend_utils_adt_ruleutils.c +1683 -0
- data/ext/pg_query/src_backend_utils_error_assert.c +74 -0
- data/ext/pg_query/src_backend_utils_error_elog.c +1748 -0
- data/ext/pg_query/src_backend_utils_fmgr_fmgr.c +570 -0
- data/ext/pg_query/src_backend_utils_hash_dynahash.c +1086 -0
- data/ext/pg_query/src_backend_utils_init_globals.c +168 -0
- data/ext/pg_query/src_backend_utils_mb_mbutils.c +839 -0
- data/ext/pg_query/src_backend_utils_misc_guc.c +1831 -0
- data/ext/pg_query/src_backend_utils_mmgr_aset.c +1560 -0
- data/ext/pg_query/src_backend_utils_mmgr_mcxt.c +1006 -0
- data/ext/pg_query/src_common_encnames.c +158 -0
- data/ext/pg_query/src_common_keywords.c +39 -0
- data/ext/pg_query/src_common_kwlist_d.h +1081 -0
- data/ext/pg_query/src_common_kwlookup.c +91 -0
- data/ext/pg_query/src_common_psprintf.c +158 -0
- data/ext/pg_query/src_common_string.c +86 -0
- data/ext/pg_query/src_common_stringinfo.c +336 -0
- data/ext/pg_query/src_common_wchar.c +1651 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_comp.c +1133 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_funcs.c +877 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_gram.c +6533 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_handler.c +107 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_reserved_kwlist_d.h +123 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_scanner.c +671 -0
- data/ext/pg_query/src_pl_plpgsql_src_pl_unreserved_kwlist_d.h +255 -0
- data/ext/pg_query/src_port_erand48.c +127 -0
- data/ext/pg_query/src_port_pg_bitutils.c +246 -0
- data/ext/pg_query/src_port_pgsleep.c +69 -0
- data/ext/pg_query/src_port_pgstrcasecmp.c +83 -0
- data/ext/pg_query/src_port_qsort.c +240 -0
- data/ext/pg_query/src_port_random.c +31 -0
- data/ext/pg_query/src_port_snprintf.c +1449 -0
- data/ext/pg_query/src_port_strerror.c +324 -0
- data/ext/pg_query/src_port_strnlen.c +39 -0
- data/ext/pg_query/xxhash.c +43 -0
- data/lib/pg_query.rb +7 -4
- data/lib/pg_query/constants.rb +21 -0
- data/lib/pg_query/deparse.rb +16 -991
- data/lib/pg_query/filter_columns.rb +86 -85
- data/lib/pg_query/fingerprint.rb +122 -87
- data/lib/pg_query/json_field_names.rb +1402 -0
- data/lib/pg_query/node.rb +31 -0
- data/lib/pg_query/param_refs.rb +42 -37
- data/lib/pg_query/parse.rb +220 -200
- data/lib/pg_query/parse_error.rb +1 -1
- data/lib/pg_query/pg_query_pb.rb +3211 -0
- data/lib/pg_query/scan.rb +23 -0
- data/lib/pg_query/treewalker.rb +24 -40
- data/lib/pg_query/truncate.rb +64 -43
- data/lib/pg_query/version.rb +2 -2
- metadata +102 -11
- data/ext/pg_query/pg_query_ruby.h +0 -10
- data/lib/pg_query/deep_dup.rb +0 -16
- data/lib/pg_query/deparse/alter_table.rb +0 -42
- data/lib/pg_query/deparse/interval.rb +0 -105
- data/lib/pg_query/legacy_parsetree.rb +0 -109
- data/lib/pg_query/node_types.rb +0 -282
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: ce01a7fe50bcabcd0416813bf71543f4b8f16a922d17a52bf398d510a0c5aa46
|
4
|
+
data.tar.gz: fea65174ec2fa31bea2254e3206c92f185f6d9ab457eb9d2a09c49727a08a305
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b7b84af5ffca59fa2beaa5a5a2866f1ed6ca0e7200ac163717c26066265a8adb77a620c7c49347d35e6bbb7dbc49697480f369c12c5b07d2d6986a331118335d
|
7
|
+
data.tar.gz: d762f31c00650b8682e1e346508d3cd83c7933204f67e6278504eddb6c6f99b6e578a856e4f360ec8783e813192fc1085eb7817bbc629dee5f5500f44720553a
|
data/CHANGELOG.md
CHANGED
@@ -1,18 +1,140 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## 2.0.0 2021-03-18
|
4
|
+
|
5
|
+
* Update to PostgreSQL 13 parser
|
6
|
+
* Update to libpg_query v2, and new Protobuf-based format
|
7
|
+
* WARNING: This is a breaking change if you are directly interacting with the
|
8
|
+
parsetree (helpers like `table` and such still work the same way)
|
9
|
+
* Use actual Ruby classes for parser result, instead of decoded JSON
|
10
|
+
* This is essentialy to enable easy and fast two-way communication with
|
11
|
+
the C library, and as a bonus makes for a better interaction on the Ruby
|
12
|
+
side, as we are handling actual objects instead of hashes and arrays.
|
13
|
+
* Use new deparser maintained directly in libpg_query
|
14
|
+
* This replaces the complete Ruby deparser with a new, more complete deparser
|
15
|
+
that is directly maintained in libpg_query. Further deparser improvements
|
16
|
+
should be directly contributed to [libpg_query](https://github.com/pganalyze/libpg_query)
|
17
|
+
* Tables helper: Return more details through "tables_with_details" method
|
18
|
+
* This is renamed from the previously badly named "tables_with_types"
|
19
|
+
method. Note that this change should not affect the output of the
|
20
|
+
primary "tables" helper.
|
21
|
+
* Replace on-demand libpg_query source download with bundled source code
|
22
|
+
* Its unnecessary to download the source on-demand, and makes this more
|
23
|
+
complex than it needs to be. Instead, introduce a new "update_source" rake
|
24
|
+
task that can be called to refresh the source for a specified revision.
|
25
|
+
* Re-implement smart truncation without requiring a special node type
|
26
|
+
* This ensures the truncate method works with the new deparser, without
|
27
|
+
the C level code needing to know about it. We may add it in the C library
|
28
|
+
in the future for edge cases that can't be covered by this slightly
|
29
|
+
hack-ish approach, but for now this avoids unnecessary C library
|
30
|
+
deparser modifications with non-standard node types.
|
31
|
+
* Update Ruby finterprinting to new fingerprint format and XXH3 hash
|
32
|
+
* Note that its recommended to use PgQuery.fingerprint for performance
|
33
|
+
reasons, but when the tree has been modified, it can be convenient to
|
34
|
+
run a Ruby-side fingerprint instead of the C-based one that is faster.
|
35
|
+
|
36
|
+
|
37
|
+
## 1.3.0 2020-12-28
|
38
|
+
|
39
|
+
* Incorporate newer libpg_query updates in 10-1.0.3 and 10-1.0.4
|
40
|
+
* Adds support for running on ARM
|
41
|
+
* Fixes asprintf warning during builds
|
42
|
+
* Updates to newer Postgres 10 patch release (10.15)
|
43
|
+
* Deparsing improvements by [@emin100](https://github.com/emin100)
|
44
|
+
* Add support for additional DROP statements (#147)
|
45
|
+
* Fix CREATE TABLE AS - Support without TEMP, Add ON COMMIT (#149)
|
46
|
+
* Empty target list support (#156)
|
47
|
+
* UNION parentheses (#158)
|
48
|
+
* OVERLAY keyword function (#161)
|
49
|
+
* Array indirection (#162)
|
50
|
+
* ARRAY functions (#163)
|
51
|
+
* Correctly handle column names that need escaping in INSERT and UPDATE statements (#164)
|
52
|
+
* INSERT INTO ON CONFLICT (#166)
|
53
|
+
* LATERAL JOIN (#168)
|
54
|
+
* UPDATE FROM clause (#170)
|
55
|
+
* SELECT aggregate FILTER (#175)
|
56
|
+
* INTERSECT operator (#176)
|
57
|
+
* Deparsing: Improve handling of boolean type casts [@himanshu-pro](https://github.com/himanshu-pro) & [@emin100](https://github.com/emin100)
|
58
|
+
* `tables` method: Find tables in the subquery of CREATE TABLE AS (#172) [@Tassosb](https://github.com/Tassosb)
|
59
|
+
* Support Ruby 3.0, verify SHA256 checksum of downloaded libpg_query (#178) [@stanhu](https://github.com/stanhu)
|
60
|
+
* Verify SHA256 checksum to guard against any malicious attempts to change the archive
|
61
|
+
* Use `URI.open` to fix Ruby 3.0 support
|
62
|
+
|
63
|
+
|
64
|
+
## 1.2.0 2019-11-10
|
65
|
+
|
66
|
+
* Reduce escaped keywords to Postgres-specific keywords, and ignore unreserved keywords
|
67
|
+
* This matches the behaviour of Postgres' quote_identifier function, and avoids problems
|
68
|
+
when doing text comparisons with output involving that function
|
69
|
+
* Note that this will lead to different output than in earlier pg_query versions,
|
70
|
+
in some cases
|
71
|
+
|
72
|
+
|
73
|
+
## 1.1.1 2019-11-10
|
74
|
+
|
75
|
+
* Deparsing improvements by [@emin100](https://github.com/emin100)
|
76
|
+
* Deparse ILIKE, COLLATE and DISCARD (#133)
|
77
|
+
* CREATE CAST (#136)
|
78
|
+
* CREATE SCHEMA (#136)
|
79
|
+
* UNION, UNION ALL and EXCEPT in SELECT queries (#136)
|
80
|
+
* CREATE DOMAIN (#145)
|
81
|
+
* Subquery indirection (#157)
|
82
|
+
* Fix Type Cast Parentheses Problem (#152)
|
83
|
+
* SELECT INTO (#151)
|
84
|
+
* SET DEFAULT in INSERT INTO (#154)
|
85
|
+
* REVOKE (#155)
|
86
|
+
* PREPARE and EXECUTE (#148)
|
87
|
+
* INSERT INTO ... RETURNING (#153)
|
88
|
+
* Fix Alter .. RENAME SQL (#146)
|
89
|
+
* Deparsing improvements by [@herwinw](https://github.com/herwinw)
|
90
|
+
* Fix subquery in COPY in deparse (#112)
|
91
|
+
* Function call indirection (#116)
|
92
|
+
* Function without parameters (#117)
|
93
|
+
* CREATE AGGREGATE
|
94
|
+
* CREATE OPERATOR
|
95
|
+
* CREATE TYPE
|
96
|
+
* GRANT statements
|
97
|
+
* DROP SCHEMA
|
98
|
+
* Deparsing improvements by [@akiellor](https://github.com/akiellor)
|
99
|
+
* Named window functions (#150)
|
100
|
+
* Deparsing improvements by [@himanshu](https://github.com/himanshu)
|
101
|
+
* Arguments in custom types (#143)
|
102
|
+
* Use "double precision" instead of "double" type name (#139)
|
103
|
+
* Use explicit -z flag to support OpenBSD tar (#134) [@sirn](https://github.com/sirn)
|
104
|
+
* Add Ruby 2.6 to Travis tests
|
105
|
+
* Escape identifiers in more cases, if necessary
|
106
|
+
|
107
|
+
|
108
|
+
## 1.1.0 2018-10-04
|
109
|
+
|
110
|
+
* Deparsing improvements by [@herwinw](https://github.com/herwinw)
|
111
|
+
* Add NULLS FIRST/LAST to ORDER BY [#95](https://github.com/pganalyze/pg_query/pull/95)
|
112
|
+
* VACUUM [#97](https://github.com/pganalyze/pg_query/pull/97)
|
113
|
+
* UPDATE with multiple columns [#99](https://github.com/pganalyze/pg_query/pull/99)
|
114
|
+
* DISTINCT ON [#101](https://github.com/pganalyze/pg_query/pull/101)
|
115
|
+
* CREATE TABLE AS [#102](https://github.com/pganalyze/pg_query/pull/102)
|
116
|
+
* SQL value functions [#103](https://github.com/pganalyze/pg_query/pull/103)
|
117
|
+
* LOCK [#105](https://github.com/pganalyze/pg_query/pull/105)
|
118
|
+
* EXPLAIN [#107](https://github.com/pganalyze/pg_query/pull/107)
|
119
|
+
* COPY [#108](https://github.com/pganalyze/pg_query/pull/108)
|
120
|
+
* DO [#109](https://github.com/pganalyze/pg_query/pull/109)
|
121
|
+
* Ignore pg_query.so in git checkout [#110](https://github.com/pganalyze/pg_query/pull/110) [@herwinw](https://github.com/herwinw)
|
122
|
+
* Prefer __dir__ over File.dirname(__FILE__) [#110](https://github.com/pganalyze/pg_query/pull/104) [@herwinw](https://github.com/herwinw)
|
123
|
+
|
124
|
+
|
3
125
|
## 1.0.2 2018-04-11
|
4
126
|
|
5
127
|
* Deparsing improvements
|
6
|
-
* SELECT DISTINCT clause [#77](https://github.com/
|
7
|
-
* "CASE expr WHEN ... END" clause [#78](https://github.com/
|
8
|
-
* LEFT/RIGHT/FULL/NATURAL JOIN [#79](https://github.com/
|
9
|
-
* SELECT that includes schema name [#80](https://github.com/
|
128
|
+
* SELECT DISTINCT clause [#77](https://github.com/pganalyze/pg_query/pull/77) [@Papierkorb](https://github.com/Papierkorb)
|
129
|
+
* "CASE expr WHEN ... END" clause [#78](https://github.com/pganalyze/pg_query/pull/78) [@Papierkorb](https://github.com/Papierkorb)
|
130
|
+
* LEFT/RIGHT/FULL/NATURAL JOIN [#79](https://github.com/pganalyze/pg_query/pull/79) [@Papierkorb](https://github.com/Papierkorb)
|
131
|
+
* SELECT that includes schema name [#80](https://github.com/pganalyze/pg_query/pull/80) [@jcsjcs](https://github.com/jcsjcs)
|
10
132
|
|
11
133
|
|
12
134
|
## 1.0.1 2018-02-02
|
13
135
|
|
14
|
-
* Parse CTEs and nested selects in INSERT/UPDATE [#76](https://github.com/
|
15
|
-
* Drop explicit json dependency [#74](https://github.com/
|
136
|
+
* Parse CTEs and nested selects in INSERT/UPDATE [#76](https://github.com/pganalyze/pg_query/pull/76) [@jcoleman](https://github.com/jcoleman)
|
137
|
+
* Drop explicit json dependency [#74](https://github.com/pganalyze/pg_query/pull/74) [@yuki24](https://github.com/yuki24)
|
16
138
|
|
17
139
|
|
18
140
|
## 1.0.0 2017-10-31
|
@@ -33,18 +155,18 @@
|
|
33
155
|
## 0.13.4 2017-10-20
|
34
156
|
|
35
157
|
* Update to libpg_query 9.5-1.7.0
|
36
|
-
- Fixes compilation old gcc before 4.6.0 [#73](https://github.com/
|
158
|
+
- Fixes compilation old gcc before 4.6.0 [#73](https://github.com/pganalyze/pg_query/issues/73)
|
37
159
|
|
38
160
|
|
39
161
|
## 0.13.3 2017-09-04
|
40
162
|
|
41
|
-
* Fix table detection for SELECTs that have sub-SELECTs without FROM clause [#69](https://github.com/
|
163
|
+
* Fix table detection for SELECTs that have sub-SELECTs without FROM clause [#69](https://github.com/pganalyze/pg_query/issues/69)
|
42
164
|
|
43
165
|
|
44
166
|
## 0.13.2 2017-08-10
|
45
167
|
|
46
|
-
* Support table detection in sub-SELECTs in JOINs [#68](https://github.com/
|
47
|
-
* Legacy ".parsetree" helper: Fix "Between" and "In" operator does not have "AEXPR" [#66](https://github.com/
|
168
|
+
* Support table detection in sub-SELECTs in JOINs [#68](https://github.com/pganalyze/pg_query/pull/65) [@seanmdick](https://github.com/seanmdick)
|
169
|
+
* Legacy ".parsetree" helper: Fix "Between" and "In" operator does not have "AEXPR" [#66](https://github.com/pganalyze/pg_query/issues/66)
|
48
170
|
* For new applications please use ".tree" method which uses the native structure
|
49
171
|
returned from libpg_query which resembles Postgres node names more closely
|
50
172
|
|
@@ -57,7 +179,7 @@
|
|
57
179
|
|
58
180
|
## 0.13.0 2017-07-30
|
59
181
|
|
60
|
-
* Introduce split between SELECT/DML/DDL for tables method [#65](https://github.com/
|
182
|
+
* Introduce split between SELECT/DML/DDL for tables method [#65](https://github.com/pganalyze/pg_query/pull/65) [@chrisfrommann](https://github.com/chrisfrommann)
|
61
183
|
* Backwards compatible, use the new select_tables/dml_tables/ddl_tables to
|
62
184
|
access the categorized table references
|
63
185
|
* Update libpg_query to 9.5-1.6.2
|
@@ -85,14 +207,14 @@
|
|
85
207
|
|
86
208
|
## 0.11.5 2017-07-09
|
87
209
|
|
88
|
-
* Deparse coldeflist [#64](https://github.com/
|
89
|
-
* Use Integer class for checking integer instead of Fixnum [#62](https://github.com/
|
210
|
+
* Deparse coldeflist [#64](https://github.com/pganalyze/pg_query/pull/64) [@jcsjcs](https://github.com/jcsjcs)
|
211
|
+
* Use Integer class for checking integer instead of Fixnum [#62](https://github.com/pganalyze/pg_query/pull/62) [@makimoto](https://github.com/makimoto)
|
90
212
|
|
91
213
|
|
92
214
|
## 0.11.4 2017-01-18
|
93
215
|
|
94
|
-
* Compatibility with Ruby 2.4 [#59](https://github.com/
|
95
|
-
* Deparse varchar and numeric casts without arguments [#61](https://github.com/
|
216
|
+
* Compatibility with Ruby 2.4 [#59](https://github.com/pganalyze/pg_query/pull/59) [@merqlove](https://github.com/merqlove)
|
217
|
+
* Deparse varchar and numeric casts without arguments [#61](https://github.com/pganalyze/pg_query/pull/61) [@jcsjcs](https://github.com/jcsjcs)
|
96
218
|
|
97
219
|
|
98
220
|
## 0.11.3 2016-12-06
|
@@ -101,11 +223,11 @@
|
|
101
223
|
* Cut off fingerprints at 100 nodes deep to avoid excessive runtimes/memory
|
102
224
|
* Fix warning on Linux due to missing asprintf include
|
103
225
|
* Improved deparsing [@jcsjcs](https://github.com/jcsjcs)
|
104
|
-
* Float [#54](https://github.com/
|
105
|
-
* BETWEEN [#55](https://github.com/
|
106
|
-
* NULLIF [#56](https://github.com/
|
107
|
-
* SELECT NULL and BooleanTest [#57](https://github.com/
|
108
|
-
* Fix build on BSD systems [#58](https://github.com/
|
226
|
+
* Float [#54](https://github.com/pganalyze/pg_query/pull/54)
|
227
|
+
* BETWEEN [#55](https://github.com/pganalyze/pg_query/pull/55)
|
228
|
+
* NULLIF [#56](https://github.com/pganalyze/pg_query/pull/56)
|
229
|
+
* SELECT NULL and BooleanTest [#57](https://github.com/pganalyze/pg_query/pull/57)
|
230
|
+
* Fix build on BSD systems [#58](https://github.com/pganalyze/pg_query/pull/58) [@myfreeweb](https://github.com/myfreeweb)
|
109
231
|
|
110
232
|
|
111
233
|
## 0.11.2 2016-06-27
|
@@ -124,9 +246,9 @@
|
|
124
246
|
## 0.11.0 2016-06-22
|
125
247
|
|
126
248
|
* Improved table name analysis (#tables method)
|
127
|
-
* Don't include CTE names, make them accessible as #cte_names instead [#52](https://github.com/
|
128
|
-
* Include table names in target list sub selects [#38](https://github.com/
|
129
|
-
* Add support for ORDER/GROUP BY, HAVING, and booleans in WHERE [#53](https://github.com/
|
249
|
+
* Don't include CTE names, make them accessible as #cte_names instead [#52](https://github.com/pganalyze/pg_query/issues/52)
|
250
|
+
* Include table names in target list sub selects [#38](https://github.com/pganalyze/pg_query/issues/38)
|
251
|
+
* Add support for ORDER/GROUP BY, HAVING, and booleans in WHERE [#53](https://github.com/pganalyze/pg_query/pull/53) [@jcoleman](https://github.com/jcoleman)
|
130
252
|
* Fix parsing of DROP TYPE statements
|
131
253
|
|
132
254
|
|
@@ -135,19 +257,19 @@
|
|
135
257
|
* Based on PostgreSQL 9.5.3
|
136
258
|
* Use LLVM extracted parser for significantly improved build times (via libpg_query)
|
137
259
|
* Deparsing Improvements
|
138
|
-
* SET statements [#48](https://github.com/
|
139
|
-
* LIKE/NOT LIKE [#49](https://github.com/
|
140
|
-
* CREATE FUNCTION improvements [#50](https://github.com/
|
260
|
+
* SET statements [#48](https://github.com/pganalyze/pg_query/pull/48) [@Winslett](https://github.com/Winslett)
|
261
|
+
* LIKE/NOT LIKE [#49](https://github.com/pganalyze/pg_query/pull/49) [@Winslett](https://github.com/Winslett)
|
262
|
+
* CREATE FUNCTION improvements [#50](https://github.com/pganalyze/pg_query/pull/50) [@Winslett](https://github.com/Winslett)
|
141
263
|
|
142
264
|
|
143
265
|
## 0.9.2 2016-05-03
|
144
266
|
|
145
|
-
* Fix issue with A_CONST string values in `.parsetree` compatibility layer (Fixes [#47](https://github.com/
|
267
|
+
* Fix issue with A_CONST string values in `.parsetree` compatibility layer (Fixes [#47](https://github.com/pganalyze/pg_query/issues/47))
|
146
268
|
|
147
269
|
|
148
270
|
## 0.9.1 2016-04-20
|
149
271
|
|
150
|
-
* Add support for Ruby 1.9 (Fixes [#44](https://github.com/
|
272
|
+
* Add support for Ruby 1.9 (Fixes [#44](https://github.com/pganalyze/pg_query/issues/44))
|
151
273
|
|
152
274
|
|
153
275
|
## 0.9.0 2016-04-17
|
@@ -173,14 +295,14 @@
|
|
173
295
|
## 0.7.2 2015-12-20
|
174
296
|
|
175
297
|
* Deparsing
|
176
|
-
* Quote all column refs [#40](https://github.com/
|
177
|
-
* Quote all range vars [#43](https://github.com/
|
178
|
-
* Support for COUNT(DISTINCT ...) [#42](https://github.com/
|
298
|
+
* Quote all column refs [#40](https://github.com/pganalyze/pg_query/pull/40) [@avinoamr](https://github.com/avinoamr)
|
299
|
+
* Quote all range vars [#43](https://github.com/pganalyze/pg_query/pull/43) [@avinoamr](https://github.com/avinoamr)
|
300
|
+
* Support for COUNT(DISTINCT ...) [#42](https://github.com/pganalyze/pg_query/pull/42) [@avinoamr](https://github.com/avinoamr)
|
179
301
|
|
180
302
|
|
181
303
|
## 0.7.1 2015-11-17
|
182
304
|
|
183
|
-
* Abstracted parser access into libpg_query [#24](https://github.com/
|
305
|
+
* Abstracted parser access into libpg_query [#24](https://github.com/pganalyze/pg_query/pull/35)
|
184
306
|
* libpg_query
|
185
307
|
* Use UTF-8 encoding for parsing [#4](https://github.com/lfittl/libpg_query/pull/4) [@zhm](https://github.com/zhm)
|
186
308
|
* Add type to A_CONST nodes[#5](https://github.com/lfittl/libpg_query/pull/5) [@zhm](https://github.com/zhm)
|
@@ -188,20 +310,20 @@
|
|
188
310
|
|
189
311
|
## 0.7.0 2015-10-17
|
190
312
|
|
191
|
-
* Restructure build process to use upstream tarballs [#35](https://github.com/
|
192
|
-
* Avoid bison/flex dependency to make deployment easier [#31](https://github.com/
|
193
|
-
* Solve issues with deployments to Heroku [#32](https://github.com/
|
313
|
+
* Restructure build process to use upstream tarballs [#35](https://github.com/pganalyze/pg_query/pull/35)
|
314
|
+
* Avoid bison/flex dependency to make deployment easier [#31](https://github.com/pganalyze/pg_query/issues/31)
|
315
|
+
* Solve issues with deployments to Heroku [#32](https://github.com/pganalyze/pg_query/issues/32)
|
194
316
|
* Deparsing
|
195
|
-
* HAVING and FOR UPDATE [#36](https://github.com/
|
317
|
+
* HAVING and FOR UPDATE [#36](https://github.com/pganalyze/pg_query/pull/36) [@JackDanger](https://github.com/JackDanger)
|
196
318
|
|
197
319
|
|
198
320
|
## 0.6.4 2015-10-01
|
199
321
|
|
200
322
|
* Deparsing
|
201
|
-
* Constraints & Interval Types [#28](https://github.com/
|
202
|
-
* Cross joins [#29](https://github.com/
|
203
|
-
* ALTER TABLE [#30](https://github.com/
|
204
|
-
* LIMIT and OFFSET [#33](https://github.com/
|
323
|
+
* Constraints & Interval Types [#28](https://github.com/pganalyze/pg_query/pull/28) [@JackDanger](https://github.com/JackDanger)
|
324
|
+
* Cross joins [#29](https://github.com/pganalyze/pg_query/pull/29) [@mme](https://github.com/mme)
|
325
|
+
* ALTER TABLE [#30](https://github.com/pganalyze/pg_query/pull/30) [@JackDanger](https://github.com/JackDanger)
|
326
|
+
* LIMIT and OFFSET [#33](https://github.com/pganalyze/pg_query/pull/33) [@jcsjcs](https://github.com/jcsjcs)
|
205
327
|
|
206
328
|
|
207
329
|
## 0.6.3 2015-08-20
|
data/README.md
CHANGED
@@ -1,10 +1,10 @@
|
|
1
|
-
# pg_query [ ![](https://img.shields.io/gem/v/pg_query.svg)](https://rubygems.org/gems/pg_query) [ ![](https://img.shields.io/gem/dt/pg_query.svg)](https://rubygems.org/gems/pg_query)
|
1
|
+
# pg_query [ ![](https://img.shields.io/gem/v/pg_query.svg)](https://rubygems.org/gems/pg_query) [ ![](https://img.shields.io/gem/dt/pg_query.svg)](https://rubygems.org/gems/pg_query)
|
2
2
|
|
3
3
|
This Ruby extension uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parsetree.
|
4
4
|
|
5
5
|
In addition the extension allows you to normalize queries (replacing constant values with ?) and parse these normalized queries into a parsetree again.
|
6
6
|
|
7
|
-
When you build this extension, it builds parts of the PostgreSQL server source (see [libpg_query](https://github.com/
|
7
|
+
When you build this extension, it builds parts of the PostgreSQL server source (see [libpg_query](https://github.com/pganalyze/libpg_query)), and then statically links it into this extension.
|
8
8
|
|
9
9
|
This is slightly crazy, but is the only reliable way of parsing all valid PostgreSQL queries.
|
10
10
|
|
@@ -25,17 +25,49 @@ Due to compiling parts of PostgreSQL, installation might take a while on slower
|
|
25
25
|
```ruby
|
26
26
|
PgQuery.parse("SELECT 1")
|
27
27
|
|
28
|
-
=> #<PgQuery:
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
28
|
+
=> #<PgQuery::ParserResult:0x00007fb69a958820
|
29
|
+
@query="SELECT 1",
|
30
|
+
@tree=<PgQuery::ParseResult:
|
31
|
+
version: 130002,
|
32
|
+
stmts: [
|
33
|
+
<PgQuery::RawStmt:
|
34
|
+
stmt: <PgQuery::Node:
|
35
|
+
select_stmt: <PgQuery::SelectStmt:
|
36
|
+
distinct_clause: [],
|
37
|
+
target_list: [
|
38
|
+
<PgQuery::Node:
|
39
|
+
res_target: <PgQuery::ResTarget:
|
40
|
+
name: "",
|
41
|
+
indirection: [],
|
42
|
+
val: <PgQuery::Node:
|
43
|
+
a_const: <PgQuery::A_Const:
|
44
|
+
val: <PgQuery::Node:
|
45
|
+
integer: <PgQuery::Integer: ival: 1>
|
46
|
+
>,
|
47
|
+
location: 7
|
48
|
+
>
|
49
|
+
>,
|
50
|
+
location: 7
|
51
|
+
>
|
52
|
+
>
|
53
|
+
],
|
54
|
+
from_clause: [],
|
55
|
+
group_clause: [],
|
56
|
+
window_clause: [],
|
57
|
+
values_lists: [],
|
58
|
+
sort_clause: [],
|
59
|
+
limit_option: :LIMIT_OPTION_DEFAULT,
|
60
|
+
locking_clause: [],
|
61
|
+
op: :SETOP_NONE,
|
62
|
+
all: false
|
63
|
+
>
|
64
|
+
>,
|
65
|
+
stmt_location: 0,
|
66
|
+
stmt_len: 0
|
67
|
+
>
|
68
|
+
]
|
69
|
+
>,
|
70
|
+
@warnings=[]>
|
39
71
|
```
|
40
72
|
|
41
73
|
### Modifying a parsed query and turning it into SQL again
|
@@ -43,35 +75,14 @@ PgQuery.parse("SELECT 1")
|
|
43
75
|
```ruby
|
44
76
|
parsed_query = PgQuery.parse("SELECT * FROM users")
|
45
77
|
|
46
|
-
=> #<PgQuery:0x007ff3e956c8b0
|
47
|
-
@tree=
|
48
|
-
[{"SelectStmt"=>
|
49
|
-
{"targetList"=>
|
50
|
-
[{"ResTarget"=>
|
51
|
-
{"val"=>
|
52
|
-
{"ColumnRef"=> {"fields"=>[{"A_Star"=>{}}], "location"=>7}},
|
53
|
-
"location"=>7}
|
54
|
-
}],
|
55
|
-
"fromClause"=>
|
56
|
-
[{"RangeVar"=>
|
57
|
-
{"relname"=>"users",
|
58
|
-
"inhOpt"=>2,
|
59
|
-
"relpersistence"=>"p",
|
60
|
-
"location"=>14}}],
|
61
|
-
}}],
|
62
|
-
@query="SELECT * FROM users",
|
63
|
-
@warnings=[]>
|
64
|
-
|
65
78
|
# Modify the parse tree in some way
|
66
|
-
parsed_query.tree[0][
|
79
|
+
parsed_query.tree.stmts[0].stmt.select_stmt.from_clause[0].range_var.relname = 'other_users'
|
67
80
|
|
68
81
|
# Turn it into SQL again
|
69
82
|
parsed_query.deparse
|
70
|
-
=> "SELECT * FROM
|
83
|
+
=> "SELECT * FROM other_users"
|
71
84
|
```
|
72
85
|
|
73
|
-
Note: The deparsing feature is experimental and does not support outputting all SQL yet.
|
74
|
-
|
75
86
|
### Parsing a normalized query
|
76
87
|
|
77
88
|
```ruby
|
@@ -83,29 +94,10 @@ PgQuery.normalize("SELECT 1 FROM x WHERE y = 'foo'")
|
|
83
94
|
# Parsing a normalized query (pre-Postgres 10 style)
|
84
95
|
PgQuery.parse("SELECT ? FROM x WHERE y = ?")
|
85
96
|
|
86
|
-
=> #<PgQuery:
|
87
|
-
|
88
|
-
|
89
|
-
|
90
|
-
[{"ResTarget"=>
|
91
|
-
{"val"=>{"ParamRef"=>{"location"=>7}},
|
92
|
-
"location"=>7}}],
|
93
|
-
"fromClause"=>
|
94
|
-
[{"RangeVar"=>
|
95
|
-
{"relname"=>"x",
|
96
|
-
"inhOpt"=>2,
|
97
|
-
"relpersistence"=>"p",
|
98
|
-
"location"=>14}}],
|
99
|
-
"whereClause"=>
|
100
|
-
{"A_Expr"=>
|
101
|
-
{"kind"=>0,
|
102
|
-
"name"=>[{"String"=>{"str"=>"="}}],
|
103
|
-
"lexpr"=>{"ColumnRef"=>{"fields"=>[{"String"=>{"str"=>"y"}}], "location"=>22}},
|
104
|
-
"rexpr"=>{"ParamRef"=>{"location"=>26}},
|
105
|
-
"location"=>24}},
|
106
|
-
}}],
|
107
|
-
@query="SELECT ? FROM x WHERE y = ?",
|
108
|
-
@warnings=[]>
|
97
|
+
=> #<PgQuery::ParserResult:0x00007fb69a97a5d8
|
98
|
+
@query="SELECT ? FROM x WHERE y = ?",
|
99
|
+
@tree=<PgQuery::ParseResult: ...>,
|
100
|
+
@warnings=[]>
|
109
101
|
```
|
110
102
|
|
111
103
|
### Extracting tables from a query
|
@@ -129,21 +121,21 @@ PgQuery.parse("SELECT ? FROM x WHERE x.y = ? AND z = ?").filter_columns
|
|
129
121
|
```ruby
|
130
122
|
PgQuery.parse("SELECT 1").fingerprint
|
131
123
|
|
132
|
-
=> "
|
124
|
+
=> "50fde20626009aba"
|
133
125
|
|
134
126
|
PgQuery.parse("SELECT 2; --- comment").fingerprint
|
135
127
|
|
136
|
-
=> "
|
128
|
+
=> "50fde20626009aba"
|
137
129
|
|
138
|
-
# Faster fingerprint method that is implemented inside the native library
|
130
|
+
# Faster fingerprint method that is implemented inside the native C library
|
139
131
|
PgQuery.fingerprint("SELECT ?")
|
140
132
|
|
141
|
-
=> "
|
133
|
+
=> "50fde20626009aba"
|
142
134
|
```
|
143
135
|
|
144
136
|
## Differences from Upstream PostgreSQL
|
145
137
|
|
146
|
-
This gem is based on [libpg_query](https://github.com/
|
138
|
+
This gem is based on [libpg_query](https://github.com/pganalyze/libpg_query),
|
147
139
|
which uses the latest stable PostgreSQL version, but with a patch applied
|
148
140
|
to support parsing normalized queries containing `?` replacement characters.
|
149
141
|
|
@@ -151,15 +143,34 @@ to support parsing normalized queries containing `?` replacement characters.
|
|
151
143
|
|
152
144
|
Currently tested and officially supported Ruby versions:
|
153
145
|
|
154
|
-
*
|
155
|
-
*
|
156
|
-
*
|
157
|
-
*
|
146
|
+
* CRuby 2.5
|
147
|
+
* CRuby 2.6
|
148
|
+
* CRuby 2.7
|
149
|
+
* CRuby 3.0
|
150
|
+
|
151
|
+
Not supported:
|
152
|
+
|
153
|
+
* JRuby: `pg_query` relies on a C extension, which is discouraged / not properly supported for JRuby
|
154
|
+
* TruffleRuby: GraalVM [does not support sigjmp](https://www.graalvm.org/reference-manual/llvm/NativeExecution/), which is used by the Postgres error handling code (`pg_query` uses a copy of the Postgres parser & error handling code)
|
155
|
+
|
156
|
+
## Developer tasks
|
157
|
+
|
158
|
+
### Update libpg_query source
|
159
|
+
|
160
|
+
In order to update to a newer Postgres parser, first update [libpg_query](https://github.com/pganalyze/libpg_query) to the new Postgres version and tag a release.
|
161
|
+
|
162
|
+
Once that is done, follow the following steps:
|
163
|
+
|
164
|
+
1. Update `LIB_PG_QUERY_TAG` and `LIB_PG_QUERY_SHA256SUM` in `Rakefile`
|
165
|
+
|
166
|
+
2. Run `rake update_source` to update the source code
|
167
|
+
|
168
|
+
3. Commit the `Rakefile` and the modified files in `ext/pg_query` to this source tree and make a PR
|
158
169
|
|
159
170
|
|
160
171
|
## Resources
|
161
172
|
|
162
|
-
See [libpg_query](https://github.com/
|
173
|
+
See [libpg_query](https://github.com/pganalyze/libpg_query/blob/13-latest/README.md#resources) for pg_query in other languages, as well as products/tools built on pg_query.
|
163
174
|
|
164
175
|
## Original Author
|
165
176
|
|