bigquery-fdw 2.1.2__tar.gz → 2.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bigquery_fdw-2.2.0/PKG-INFO +153 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/README.md +9 -7
- bigquery_fdw-2.2.0/bigquery_fdw.egg-info/PKG-INFO +153 -0
- bigquery_fdw-2.2.0/bigquery_fdw.egg-info/requires.txt +3 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/setup.py +11 -12
- bigquery-fdw-2.1.2/PKG-INFO +0 -193
- bigquery-fdw-2.1.2/bigquery_fdw.egg-info/PKG-INFO +0 -193
- bigquery-fdw-2.1.2/bigquery_fdw.egg-info/requires.txt +0 -6
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/LICENSE +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/bigquery_fdw.egg-info/SOURCES.txt +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/bigquery_fdw.egg-info/dependency_links.txt +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/bigquery_fdw.egg-info/entry_points.txt +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/bigquery_fdw.egg-info/top_level.txt +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/setup.cfg +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/src/__init__.py +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/src/bqclient.py +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/src/bqclient_test.py +0 -0
- {bigquery-fdw-2.1.2 → bigquery_fdw-2.2.0}/src/fdw.py +0 -0
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: bigquery-fdw
|
|
3
|
+
Version: 2.2.0
|
|
4
|
+
Summary: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
5
|
+
Home-page: https://github.com/gabfl/bigquery_fdw
|
|
6
|
+
Author: Gabriel Bordeaux
|
|
7
|
+
Author-email: pypi@gab.lc
|
|
8
|
+
License: MIT
|
|
9
|
+
Classifier: Topic :: Database
|
|
10
|
+
Classifier: Topic :: Database :: Database Engines/Servers
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Operating System :: MacOS
|
|
13
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
+
Classifier: Natural Language :: English
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python
|
|
17
|
+
Classifier: Development Status :: 4 - Beta
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: google-cloud-bigquery==3.47.0
|
|
22
|
+
Requires-Dist: google-auth==2.62.2
|
|
23
|
+
Requires-Dist: google-auth-oauthlib==1.5.0
|
|
24
|
+
Dynamic: author
|
|
25
|
+
Dynamic: author-email
|
|
26
|
+
Dynamic: classifier
|
|
27
|
+
Dynamic: description
|
|
28
|
+
Dynamic: description-content-type
|
|
29
|
+
Dynamic: home-page
|
|
30
|
+
Dynamic: license
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
Dynamic: requires-dist
|
|
33
|
+
Dynamic: requires-python
|
|
34
|
+
Dynamic: summary
|
|
35
|
+
|
|
36
|
+
# bigquery_fdw: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
37
|
+
|
|
38
|
+
[](https://pypi.org/project/bigquery-fdw/)
|
|
39
|
+
[](https://github.com/gabfl/bigquery_fdw/actions)
|
|
40
|
+
[](https://codecov.io/gh/gabfl/bigquery_fdw)
|
|
41
|
+
[](https://raw.githubusercontent.com/gabfl/bigquery_fdw/main/LICENSE)
|
|
42
|
+
|
|
43
|
+
bigquery_fdw is a BigQuery foreign data wrapper for PostgreSQL using [Multicorn](https://github.com/Segfault-Inc/Multicorn).
|
|
44
|
+
|
|
45
|
+
It allows to write queries in PostgreSQL SQL syntax using a foreign table. It supports most of BigQuery's [data types](docs/data_types.md) and [operators](docs/operators.md).
|
|
46
|
+
|
|
47
|
+
## Features and limitations
|
|
48
|
+
|
|
49
|
+
- Table partitioning is supported. [You can use partitions in your SQL queries](docs/table_partitioning.md).
|
|
50
|
+
- Queries are parameterized when sent to BigQuery
|
|
51
|
+
- BigQuery's standard SQL support (legacy SQL is not supported)
|
|
52
|
+
- Authentication works with a "[Service Account](docs/service_account.md)" Json private key
|
|
53
|
+
|
|
54
|
+
[Read more](docs/README.md).
|
|
55
|
+
|
|
56
|
+
## Requirements
|
|
57
|
+
|
|
58
|
+
- PostgreSQL 14 to 18
|
|
59
|
+
- Python >= 3.10
|
|
60
|
+
|
|
61
|
+
## Get started
|
|
62
|
+
|
|
63
|
+
### Using docker
|
|
64
|
+
|
|
65
|
+
See [getting started with Docker](docs/docker.md)
|
|
66
|
+
|
|
67
|
+
### Installation on Debian/Ubuntu
|
|
68
|
+
|
|
69
|
+
#### Dependencies required to install bigquery_fdw:
|
|
70
|
+
|
|
71
|
+
You need to install the following dependencies:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
# Install required packages
|
|
75
|
+
apt update
|
|
76
|
+
apt install -y postgresql-server-dev-18 python3-setuptools python3-dev make gcc git
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
PostgreSQL versions 14 to 18 are tested in CI.
|
|
80
|
+
|
|
81
|
+
#### Installation
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
# Install Multicorn
|
|
85
|
+
# multicorn2 is a fork of Segfault-Inc/Multicorn that adds support for recent PostgreSQL versions.
|
|
86
|
+
# The original pgsql-io/multicorn2 repository was removed in October 2026; this copy is pinned to its v3.2 release.
|
|
87
|
+
# Alternatively, up to PostgreSQL 12, you can use gabfl/Multicorn that adds better support for Python3.
|
|
88
|
+
# You may also choose to build against the original project instead.
|
|
89
|
+
git clone https://github.com/ShaheedHaque/multicorn2.git Multicorn && cd Multicorn
|
|
90
|
+
git checkout 4e0feaa8c8a4e7db80ff442d71fb7786d5519e29
|
|
91
|
+
make && make install
|
|
92
|
+
|
|
93
|
+
# Install bigquery_fdw
|
|
94
|
+
pip3 install bigquery-fdw
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Major dependencies installed automatically during the installation process:
|
|
98
|
+
|
|
99
|
+
- [Google Cloud BigQuery](https://pypi.org/project/google-cloud-bigquery/)
|
|
100
|
+
- [Multicorn](https://github.com/ShaheedHaque/multicorn2)
|
|
101
|
+
|
|
102
|
+
## Authentication
|
|
103
|
+
|
|
104
|
+
bigquery_fdw relies on [Google Cloud API's default authentication](https://cloud.google.com/docs/authentication/getting-started#linux-or-macos).
|
|
105
|
+
|
|
106
|
+
Your need to have an environment variable `GOOGLE_APPLICATION_CREDENTIALS` that has to be accessible by bigquery_fdw. Setting environment variables varies depending on OS but for Ubuntu or Debian, the preferred way is to edit `/etc/postgresql/[version]/main/environment` and add:
|
|
107
|
+
```
|
|
108
|
+
GOOGLE_APPLICATION_CREDENTIALS = '/path/to/key.json'
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Restarting PostgreSQL is required for the environment variable to be loaded.
|
|
112
|
+
|
|
113
|
+
## Usage
|
|
114
|
+
|
|
115
|
+
We recommend testing the [BigQuery client connectivity](docs/test_client.md) before trying to use the FDW.
|
|
116
|
+
|
|
117
|
+
With `psql`:
|
|
118
|
+
|
|
119
|
+
```sql
|
|
120
|
+
CREATE EXTENSION multicorn;
|
|
121
|
+
|
|
122
|
+
CREATE SERVER bigquery_srv FOREIGN DATA WRAPPER multicorn
|
|
123
|
+
OPTIONS (
|
|
124
|
+
wrapper 'bigquery_fdw.fdw.ConstantForeignDataWrapper'
|
|
125
|
+
);
|
|
126
|
+
|
|
127
|
+
CREATE FOREIGN TABLE my_bigquery_table (
|
|
128
|
+
column1 text,
|
|
129
|
+
column2 bigint
|
|
130
|
+
) SERVER bigquery_srv
|
|
131
|
+
OPTIONS (
|
|
132
|
+
fdw_dataset 'my_dataset',
|
|
133
|
+
fdw_table 'my_table'
|
|
134
|
+
);
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
## Options
|
|
138
|
+
|
|
139
|
+
List of options implemented in `CREATE FOREIGN TABLE` syntax:
|
|
140
|
+
|
|
141
|
+
| Option | Default | Description |
|
|
142
|
+
|-----|----|----|
|
|
143
|
+
| `fdw_dataset` | - | BigQuery dataset name |
|
|
144
|
+
| `fdw_table` | - | BigQuery table name |
|
|
145
|
+
| `fdw_convert_tz` | - | Convert BigQuery time zone for dates and timestamps to selected time zone. Example: `'US/Eastern'`. |
|
|
146
|
+
| `fdw_group` | `'false'` | See [Remote grouping and counting](docs/remote_grouping.md). |
|
|
147
|
+
| `fdw_casting` | - | See [Casting](docs/casting.md). |
|
|
148
|
+
| `fdw_verbose` | `'false'` | Set to `'true'` to output debug information in PostrgeSQL's logs |
|
|
149
|
+
| `fdw_sql_dialect` | `'standard'` | BigQuery SQL dialect. Currently only `standard` is supported. |
|
|
150
|
+
|
|
151
|
+
## More documentation
|
|
152
|
+
|
|
153
|
+
See [bigquery_fdw documentation](docs/README.md).
|
|
@@ -20,8 +20,8 @@ It allows to write queries in PostgreSQL SQL syntax using a foreign table. It su
|
|
|
20
20
|
|
|
21
21
|
## Requirements
|
|
22
22
|
|
|
23
|
-
- PostgreSQL
|
|
24
|
-
- Python >= 3.
|
|
23
|
+
- PostgreSQL 14 to 18
|
|
24
|
+
- Python >= 3.10
|
|
25
25
|
|
|
26
26
|
## Get started
|
|
27
27
|
|
|
@@ -38,19 +38,21 @@ You need to install the following dependencies:
|
|
|
38
38
|
```bash
|
|
39
39
|
# Install required packages
|
|
40
40
|
apt update
|
|
41
|
-
apt install -y postgresql-server-dev-
|
|
41
|
+
apt install -y postgresql-server-dev-18 python3-setuptools python3-dev make gcc git
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
|
|
44
|
+
PostgreSQL versions 14 to 18 are tested in CI.
|
|
45
45
|
|
|
46
46
|
#### Installation
|
|
47
47
|
|
|
48
48
|
```bash
|
|
49
49
|
# Install Multicorn
|
|
50
|
-
#
|
|
50
|
+
# multicorn2 is a fork of Segfault-Inc/Multicorn that adds support for recent PostgreSQL versions.
|
|
51
|
+
# The original pgsql-io/multicorn2 repository was removed in October 2026; this copy is pinned to its v3.2 release.
|
|
51
52
|
# Alternatively, up to PostgreSQL 12, you can use gabfl/Multicorn that adds better support for Python3.
|
|
52
53
|
# You may also choose to build against the original project instead.
|
|
53
|
-
git clone https://github.com/
|
|
54
|
+
git clone https://github.com/ShaheedHaque/multicorn2.git Multicorn && cd Multicorn
|
|
55
|
+
git checkout 4e0feaa8c8a4e7db80ff442d71fb7786d5519e29
|
|
54
56
|
make && make install
|
|
55
57
|
|
|
56
58
|
# Install bigquery_fdw
|
|
@@ -60,7 +62,7 @@ pip3 install bigquery-fdw
|
|
|
60
62
|
Major dependencies installed automatically during the installation process:
|
|
61
63
|
|
|
62
64
|
- [Google Cloud BigQuery](https://pypi.org/project/google-cloud-bigquery/)
|
|
63
|
-
- [Multicorn](https://github.com/
|
|
65
|
+
- [Multicorn](https://github.com/ShaheedHaque/multicorn2)
|
|
64
66
|
|
|
65
67
|
## Authentication
|
|
66
68
|
|
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: bigquery-fdw
|
|
3
|
+
Version: 2.2.0
|
|
4
|
+
Summary: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
5
|
+
Home-page: https://github.com/gabfl/bigquery_fdw
|
|
6
|
+
Author: Gabriel Bordeaux
|
|
7
|
+
Author-email: pypi@gab.lc
|
|
8
|
+
License: MIT
|
|
9
|
+
Classifier: Topic :: Database
|
|
10
|
+
Classifier: Topic :: Database :: Database Engines/Servers
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Operating System :: MacOS
|
|
13
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
+
Classifier: Natural Language :: English
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python
|
|
17
|
+
Classifier: Development Status :: 4 - Beta
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: google-cloud-bigquery==3.47.0
|
|
22
|
+
Requires-Dist: google-auth==2.62.2
|
|
23
|
+
Requires-Dist: google-auth-oauthlib==1.5.0
|
|
24
|
+
Dynamic: author
|
|
25
|
+
Dynamic: author-email
|
|
26
|
+
Dynamic: classifier
|
|
27
|
+
Dynamic: description
|
|
28
|
+
Dynamic: description-content-type
|
|
29
|
+
Dynamic: home-page
|
|
30
|
+
Dynamic: license
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
Dynamic: requires-dist
|
|
33
|
+
Dynamic: requires-python
|
|
34
|
+
Dynamic: summary
|
|
35
|
+
|
|
36
|
+
# bigquery_fdw: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
37
|
+
|
|
38
|
+
[](https://pypi.org/project/bigquery-fdw/)
|
|
39
|
+
[](https://github.com/gabfl/bigquery_fdw/actions)
|
|
40
|
+
[](https://codecov.io/gh/gabfl/bigquery_fdw)
|
|
41
|
+
[](https://raw.githubusercontent.com/gabfl/bigquery_fdw/main/LICENSE)
|
|
42
|
+
|
|
43
|
+
bigquery_fdw is a BigQuery foreign data wrapper for PostgreSQL using [Multicorn](https://github.com/Segfault-Inc/Multicorn).
|
|
44
|
+
|
|
45
|
+
It allows to write queries in PostgreSQL SQL syntax using a foreign table. It supports most of BigQuery's [data types](docs/data_types.md) and [operators](docs/operators.md).
|
|
46
|
+
|
|
47
|
+
## Features and limitations
|
|
48
|
+
|
|
49
|
+
- Table partitioning is supported. [You can use partitions in your SQL queries](docs/table_partitioning.md).
|
|
50
|
+
- Queries are parameterized when sent to BigQuery
|
|
51
|
+
- BigQuery's standard SQL support (legacy SQL is not supported)
|
|
52
|
+
- Authentication works with a "[Service Account](docs/service_account.md)" Json private key
|
|
53
|
+
|
|
54
|
+
[Read more](docs/README.md).
|
|
55
|
+
|
|
56
|
+
## Requirements
|
|
57
|
+
|
|
58
|
+
- PostgreSQL 14 to 18
|
|
59
|
+
- Python >= 3.10
|
|
60
|
+
|
|
61
|
+
## Get started
|
|
62
|
+
|
|
63
|
+
### Using docker
|
|
64
|
+
|
|
65
|
+
See [getting started with Docker](docs/docker.md)
|
|
66
|
+
|
|
67
|
+
### Installation on Debian/Ubuntu
|
|
68
|
+
|
|
69
|
+
#### Dependencies required to install bigquery_fdw:
|
|
70
|
+
|
|
71
|
+
You need to install the following dependencies:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
# Install required packages
|
|
75
|
+
apt update
|
|
76
|
+
apt install -y postgresql-server-dev-18 python3-setuptools python3-dev make gcc git
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
PostgreSQL versions 14 to 18 are tested in CI.
|
|
80
|
+
|
|
81
|
+
#### Installation
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
# Install Multicorn
|
|
85
|
+
# multicorn2 is a fork of Segfault-Inc/Multicorn that adds support for recent PostgreSQL versions.
|
|
86
|
+
# The original pgsql-io/multicorn2 repository was removed in October 2026; this copy is pinned to its v3.2 release.
|
|
87
|
+
# Alternatively, up to PostgreSQL 12, you can use gabfl/Multicorn that adds better support for Python3.
|
|
88
|
+
# You may also choose to build against the original project instead.
|
|
89
|
+
git clone https://github.com/ShaheedHaque/multicorn2.git Multicorn && cd Multicorn
|
|
90
|
+
git checkout 4e0feaa8c8a4e7db80ff442d71fb7786d5519e29
|
|
91
|
+
make && make install
|
|
92
|
+
|
|
93
|
+
# Install bigquery_fdw
|
|
94
|
+
pip3 install bigquery-fdw
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Major dependencies installed automatically during the installation process:
|
|
98
|
+
|
|
99
|
+
- [Google Cloud BigQuery](https://pypi.org/project/google-cloud-bigquery/)
|
|
100
|
+
- [Multicorn](https://github.com/ShaheedHaque/multicorn2)
|
|
101
|
+
|
|
102
|
+
## Authentication
|
|
103
|
+
|
|
104
|
+
bigquery_fdw relies on [Google Cloud API's default authentication](https://cloud.google.com/docs/authentication/getting-started#linux-or-macos).
|
|
105
|
+
|
|
106
|
+
Your need to have an environment variable `GOOGLE_APPLICATION_CREDENTIALS` that has to be accessible by bigquery_fdw. Setting environment variables varies depending on OS but for Ubuntu or Debian, the preferred way is to edit `/etc/postgresql/[version]/main/environment` and add:
|
|
107
|
+
```
|
|
108
|
+
GOOGLE_APPLICATION_CREDENTIALS = '/path/to/key.json'
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Restarting PostgreSQL is required for the environment variable to be loaded.
|
|
112
|
+
|
|
113
|
+
## Usage
|
|
114
|
+
|
|
115
|
+
We recommend testing the [BigQuery client connectivity](docs/test_client.md) before trying to use the FDW.
|
|
116
|
+
|
|
117
|
+
With `psql`:
|
|
118
|
+
|
|
119
|
+
```sql
|
|
120
|
+
CREATE EXTENSION multicorn;
|
|
121
|
+
|
|
122
|
+
CREATE SERVER bigquery_srv FOREIGN DATA WRAPPER multicorn
|
|
123
|
+
OPTIONS (
|
|
124
|
+
wrapper 'bigquery_fdw.fdw.ConstantForeignDataWrapper'
|
|
125
|
+
);
|
|
126
|
+
|
|
127
|
+
CREATE FOREIGN TABLE my_bigquery_table (
|
|
128
|
+
column1 text,
|
|
129
|
+
column2 bigint
|
|
130
|
+
) SERVER bigquery_srv
|
|
131
|
+
OPTIONS (
|
|
132
|
+
fdw_dataset 'my_dataset',
|
|
133
|
+
fdw_table 'my_table'
|
|
134
|
+
);
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
## Options
|
|
138
|
+
|
|
139
|
+
List of options implemented in `CREATE FOREIGN TABLE` syntax:
|
|
140
|
+
|
|
141
|
+
| Option | Default | Description |
|
|
142
|
+
|-----|----|----|
|
|
143
|
+
| `fdw_dataset` | - | BigQuery dataset name |
|
|
144
|
+
| `fdw_table` | - | BigQuery table name |
|
|
145
|
+
| `fdw_convert_tz` | - | Convert BigQuery time zone for dates and timestamps to selected time zone. Example: `'US/Eastern'`. |
|
|
146
|
+
| `fdw_group` | `'false'` | See [Remote grouping and counting](docs/remote_grouping.md). |
|
|
147
|
+
| `fdw_casting` | - | See [Casting](docs/casting.md). |
|
|
148
|
+
| `fdw_verbose` | `'false'` | Set to `'true'` to output debug information in PostrgeSQL's logs |
|
|
149
|
+
| `fdw_sql_dialect` | `'standard'` | BigQuery SQL dialect. Currently only `standard` is supported. |
|
|
150
|
+
|
|
151
|
+
## More documentation
|
|
152
|
+
|
|
153
|
+
See [bigquery_fdw documentation](docs/README.md).
|
|
@@ -1,30 +1,29 @@
|
|
|
1
1
|
from setuptools import setup
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
long_description = pypandoc.convert_file('README.md', 'rst')
|
|
6
|
-
except (IOError, ImportError):
|
|
7
|
-
long_description = open('README.md').read()
|
|
3
|
+
with open('README.md', encoding='utf-8') as f:
|
|
4
|
+
long_description = f.read()
|
|
8
5
|
|
|
9
6
|
setup(
|
|
10
7
|
name='bigquery-fdw',
|
|
11
|
-
version='2.
|
|
8
|
+
version='2.2.0',
|
|
12
9
|
description='BigQuery Foreign Data Wrapper for PostgreSQL',
|
|
13
10
|
long_description=long_description,
|
|
11
|
+
long_description_content_type='text/markdown',
|
|
14
12
|
author='Gabriel Bordeaux',
|
|
15
13
|
author_email='pypi@gab.lc',
|
|
16
14
|
url='https://github.com/gabfl/bigquery_fdw',
|
|
17
15
|
license='MIT',
|
|
16
|
+
python_requires='>=3.10',
|
|
18
17
|
packages=['bigquery_fdw'],
|
|
19
18
|
package_dir={'bigquery_fdw': 'src'},
|
|
20
19
|
# external dependencies
|
|
21
20
|
install_requires=[
|
|
22
|
-
'google-cloud-bigquery==3.
|
|
23
|
-
'google-auth==2.
|
|
24
|
-
'google-auth-oauthlib==1.
|
|
25
|
-
'protobuf==4.21.8', # Forcing version for google-cloud-bigquery
|
|
26
|
-
'grpcio==1.
|
|
27
|
-
'grpcio-status==1.
|
|
21
|
+
'google-cloud-bigquery==3.47.0',
|
|
22
|
+
'google-auth==2.62.2',
|
|
23
|
+
'google-auth-oauthlib==1.5.0',
|
|
24
|
+
# 'protobuf==4.21.8', # Forcing version for google-cloud-bigquery
|
|
25
|
+
# 'grpcio==1.60.1', # Forcing version for google-cloud-bigquery
|
|
26
|
+
# 'grpcio-status==1.60.1', # Forcing version for google-cloud-bigquery
|
|
28
27
|
],
|
|
29
28
|
entry_points={
|
|
30
29
|
'console_scripts': [
|
bigquery-fdw-2.1.2/PKG-INFO
DELETED
|
@@ -1,193 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: bigquery-fdw
|
|
3
|
-
Version: 2.1.2
|
|
4
|
-
Summary: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
5
|
-
Home-page: https://github.com/gabfl/bigquery_fdw
|
|
6
|
-
Author: Gabriel Bordeaux
|
|
7
|
-
Author-email: pypi@gab.lc
|
|
8
|
-
License: MIT
|
|
9
|
-
Classifier: Topic :: Database
|
|
10
|
-
Classifier: Topic :: Database :: Database Engines/Servers
|
|
11
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
-
Classifier: Operating System :: MacOS
|
|
13
|
-
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
-
Classifier: Natural Language :: English
|
|
15
|
-
Classifier: Programming Language :: Python :: 3
|
|
16
|
-
Classifier: Programming Language :: Python
|
|
17
|
-
Classifier: Development Status :: 4 - Beta
|
|
18
|
-
License-File: LICENSE
|
|
19
|
-
|
|
20
|
-
bigquery_fdw: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
21
|
-
==========================================================
|
|
22
|
-
|
|
23
|
-
|Pypi| |Build Status| |codecov| |MIT licensed|
|
|
24
|
-
|
|
25
|
-
bigquery_fdw is a BigQuery foreign data wrapper for PostgreSQL using
|
|
26
|
-
`Multicorn <https://github.com/Segfault-Inc/Multicorn>`__.
|
|
27
|
-
|
|
28
|
-
It allows to write queries in PostgreSQL SQL syntax using a foreign
|
|
29
|
-
table. It supports most of BigQuery’s `data
|
|
30
|
-
types <docs/data_types.md>`__ and `operators <docs/operators.md>`__.
|
|
31
|
-
|
|
32
|
-
Features and limitations
|
|
33
|
-
------------------------
|
|
34
|
-
|
|
35
|
-
- Table partitioning is supported. `You can use partitions in your SQL
|
|
36
|
-
queries <docs/table_partitioning.md>`__.
|
|
37
|
-
- Queries are parameterized when sent to BigQuery
|
|
38
|
-
- BigQuery’s standard SQL support (legacy SQL is not supported)
|
|
39
|
-
- Authentication works with a “`Service
|
|
40
|
-
Account <docs/service_account.md>`__” Json private key
|
|
41
|
-
|
|
42
|
-
`Read more <docs/README.md>`__.
|
|
43
|
-
|
|
44
|
-
Requirements
|
|
45
|
-
------------
|
|
46
|
-
|
|
47
|
-
- PostgreSQL >= 9.5 up to 14
|
|
48
|
-
- Python >= 3.4
|
|
49
|
-
|
|
50
|
-
Get started
|
|
51
|
-
-----------
|
|
52
|
-
|
|
53
|
-
Using docker
|
|
54
|
-
~~~~~~~~~~~~
|
|
55
|
-
|
|
56
|
-
See `getting started with Docker <docs/docker.md>`__
|
|
57
|
-
|
|
58
|
-
Installation on Debian/Ubuntu
|
|
59
|
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
60
|
-
|
|
61
|
-
Dependencies required to install bigquery_fdw:
|
|
62
|
-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
63
|
-
|
|
64
|
-
You need to install the following dependencies:
|
|
65
|
-
|
|
66
|
-
.. code:: bash
|
|
67
|
-
|
|
68
|
-
# Install required packages
|
|
69
|
-
apt update
|
|
70
|
-
apt install -y postgresql-server-dev-14 python3-setuptools python3-dev make gcc git
|
|
71
|
-
|
|
72
|
-
All PostgreSQL versions from 9.2 to 14 should be supported.
|
|
73
|
-
|
|
74
|
-
Installation
|
|
75
|
-
^^^^^^^^^^^^
|
|
76
|
-
|
|
77
|
-
.. code:: bash
|
|
78
|
-
|
|
79
|
-
# Install Multicorn
|
|
80
|
-
# pgsql-io/multicorn2 is a fork of Segfault-Inc/Multicorn that adds support for PostgreSQL 13/14.
|
|
81
|
-
# Alternatively, up to PostgreSQL 12, you can use gabfl/Multicorn that adds better support for Python3.
|
|
82
|
-
# You may also choose to build against the original project instead.
|
|
83
|
-
git clone https://github.com/pgsql-io/multicorn2.git Multicorn && cd Multicorn
|
|
84
|
-
make && make install
|
|
85
|
-
|
|
86
|
-
# Install bigquery_fdw
|
|
87
|
-
pip3 install bigquery-fdw
|
|
88
|
-
|
|
89
|
-
Major dependencies installed automatically during the installation
|
|
90
|
-
process:
|
|
91
|
-
|
|
92
|
-
- `Google Cloud
|
|
93
|
-
BigQuery <https://pypi.org/project/google-cloud-bigquery/>`__
|
|
94
|
-
- `Multicorn <https://github.com/pgsql-io/multicorn2>`__
|
|
95
|
-
|
|
96
|
-
Authentication
|
|
97
|
-
--------------
|
|
98
|
-
|
|
99
|
-
bigquery_fdw relies on `Google Cloud API’s default
|
|
100
|
-
authentication <https://cloud.google.com/docs/authentication/getting-started#linux-or-macos>`__.
|
|
101
|
-
|
|
102
|
-
Your need to have an environment variable
|
|
103
|
-
``GOOGLE_APPLICATION_CREDENTIALS`` that has to be accessible by
|
|
104
|
-
bigquery_fdw. Setting environment variables varies depending on OS but
|
|
105
|
-
for Ubuntu or Debian, the preferred way is to edit
|
|
106
|
-
``/etc/postgresql/[version]/main/environment`` and add:
|
|
107
|
-
|
|
108
|
-
::
|
|
109
|
-
|
|
110
|
-
GOOGLE_APPLICATION_CREDENTIALS = '/path/to/key.json'
|
|
111
|
-
|
|
112
|
-
Restarting PostgreSQL is required for the environment variable to be
|
|
113
|
-
loaded.
|
|
114
|
-
|
|
115
|
-
Usage
|
|
116
|
-
-----
|
|
117
|
-
|
|
118
|
-
We recommend testing the `BigQuery client
|
|
119
|
-
connectivity <docs/test_client.md>`__ before trying to use the FDW.
|
|
120
|
-
|
|
121
|
-
With ``psql``:
|
|
122
|
-
|
|
123
|
-
.. code:: sql
|
|
124
|
-
|
|
125
|
-
CREATE EXTENSION multicorn;
|
|
126
|
-
|
|
127
|
-
CREATE SERVER bigquery_srv FOREIGN DATA WRAPPER multicorn
|
|
128
|
-
OPTIONS (
|
|
129
|
-
wrapper 'bigquery_fdw.fdw.ConstantForeignDataWrapper'
|
|
130
|
-
);
|
|
131
|
-
|
|
132
|
-
CREATE FOREIGN TABLE my_bigquery_table (
|
|
133
|
-
column1 text,
|
|
134
|
-
column2 bigint
|
|
135
|
-
) SERVER bigquery_srv
|
|
136
|
-
OPTIONS (
|
|
137
|
-
fdw_dataset 'my_dataset',
|
|
138
|
-
fdw_table 'my_table'
|
|
139
|
-
);
|
|
140
|
-
|
|
141
|
-
Options
|
|
142
|
-
-------
|
|
143
|
-
|
|
144
|
-
List of options implemented in ``CREATE FOREIGN TABLE`` syntax:
|
|
145
|
-
|
|
146
|
-
+--------------------------+---------------------+---------------------+
|
|
147
|
-
| Option | Default | Description |
|
|
148
|
-
+==========================+=====================+=====================+
|
|
149
|
-
| ``fdw_dataset`` | - | BigQuery dataset |
|
|
150
|
-
| | | name |
|
|
151
|
-
+--------------------------+---------------------+---------------------+
|
|
152
|
-
| ``fdw_table`` | - | BigQuery table name |
|
|
153
|
-
+--------------------------+---------------------+---------------------+
|
|
154
|
-
| ``fdw_convert_tz`` | - | Convert BigQuery |
|
|
155
|
-
| | | time zone for dates |
|
|
156
|
-
| | | and timestamps to |
|
|
157
|
-
| | | selected time zone. |
|
|
158
|
-
| | | Example: |
|
|
159
|
-
| | | ``'US/Eastern'``. |
|
|
160
|
-
+--------------------------+---------------------+---------------------+
|
|
161
|
-
| ``fdw_group`` | ``'false'`` | See `Remote |
|
|
162
|
-
| | | grouping and |
|
|
163
|
-
| | | counting <docs/remo |
|
|
164
|
-
| | | te_grouping.md>`__. |
|
|
165
|
-
+--------------------------+---------------------+---------------------+
|
|
166
|
-
| ``fdw_casting`` | - | See |
|
|
167
|
-
| | | `Casting <d |
|
|
168
|
-
| | | ocs/casting.md>`__. |
|
|
169
|
-
+--------------------------+---------------------+---------------------+
|
|
170
|
-
| ``fdw_verbose`` | ``'false'`` | Set to ``'true'`` |
|
|
171
|
-
| | | to output debug |
|
|
172
|
-
| | | information in |
|
|
173
|
-
| | | PostrgeSQL’s logs |
|
|
174
|
-
+--------------------------+---------------------+---------------------+
|
|
175
|
-
| ``fdw_sql_dialect`` | ``'standard'`` | BigQuery SQL |
|
|
176
|
-
| | | dialect. Currently |
|
|
177
|
-
| | | only ``standard`` |
|
|
178
|
-
| | | is supported. |
|
|
179
|
-
+--------------------------+---------------------+---------------------+
|
|
180
|
-
|
|
181
|
-
More documentation
|
|
182
|
-
------------------
|
|
183
|
-
|
|
184
|
-
See `bigquery_fdw documentation <docs/README.md>`__.
|
|
185
|
-
|
|
186
|
-
.. |Pypi| image:: https://img.shields.io/pypi/v/bigquery-fdw.svg
|
|
187
|
-
:target: https://pypi.org/project/bigquery-fdw/
|
|
188
|
-
.. |Build Status| image:: https://github.com/gabfl/bigquery_fdw/actions/workflows/ci.yml/badge.svg?branch=main
|
|
189
|
-
:target: https://github.com/gabfl/bigquery_fdw/actions
|
|
190
|
-
.. |codecov| image:: https://codecov.io/gh/gabfl/bigquery_fdw/branch/main/graph/badge.svg
|
|
191
|
-
:target: https://codecov.io/gh/gabfl/bigquery_fdw
|
|
192
|
-
.. |MIT licensed| image:: https://img.shields.io/badge/license-MIT-green.svg
|
|
193
|
-
:target: https://raw.githubusercontent.com/gabfl/bigquery_fdw/main/LICENSE
|
|
@@ -1,193 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: bigquery-fdw
|
|
3
|
-
Version: 2.1.2
|
|
4
|
-
Summary: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
5
|
-
Home-page: https://github.com/gabfl/bigquery_fdw
|
|
6
|
-
Author: Gabriel Bordeaux
|
|
7
|
-
Author-email: pypi@gab.lc
|
|
8
|
-
License: MIT
|
|
9
|
-
Classifier: Topic :: Database
|
|
10
|
-
Classifier: Topic :: Database :: Database Engines/Servers
|
|
11
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
-
Classifier: Operating System :: MacOS
|
|
13
|
-
Classifier: Operating System :: POSIX :: Linux
|
|
14
|
-
Classifier: Natural Language :: English
|
|
15
|
-
Classifier: Programming Language :: Python :: 3
|
|
16
|
-
Classifier: Programming Language :: Python
|
|
17
|
-
Classifier: Development Status :: 4 - Beta
|
|
18
|
-
License-File: LICENSE
|
|
19
|
-
|
|
20
|
-
bigquery_fdw: BigQuery Foreign Data Wrapper for PostgreSQL
|
|
21
|
-
==========================================================
|
|
22
|
-
|
|
23
|
-
|Pypi| |Build Status| |codecov| |MIT licensed|
|
|
24
|
-
|
|
25
|
-
bigquery_fdw is a BigQuery foreign data wrapper for PostgreSQL using
|
|
26
|
-
`Multicorn <https://github.com/Segfault-Inc/Multicorn>`__.
|
|
27
|
-
|
|
28
|
-
It allows to write queries in PostgreSQL SQL syntax using a foreign
|
|
29
|
-
table. It supports most of BigQuery’s `data
|
|
30
|
-
types <docs/data_types.md>`__ and `operators <docs/operators.md>`__.
|
|
31
|
-
|
|
32
|
-
Features and limitations
|
|
33
|
-
------------------------
|
|
34
|
-
|
|
35
|
-
- Table partitioning is supported. `You can use partitions in your SQL
|
|
36
|
-
queries <docs/table_partitioning.md>`__.
|
|
37
|
-
- Queries are parameterized when sent to BigQuery
|
|
38
|
-
- BigQuery’s standard SQL support (legacy SQL is not supported)
|
|
39
|
-
- Authentication works with a “`Service
|
|
40
|
-
Account <docs/service_account.md>`__” Json private key
|
|
41
|
-
|
|
42
|
-
`Read more <docs/README.md>`__.
|
|
43
|
-
|
|
44
|
-
Requirements
|
|
45
|
-
------------
|
|
46
|
-
|
|
47
|
-
- PostgreSQL >= 9.5 up to 14
|
|
48
|
-
- Python >= 3.4
|
|
49
|
-
|
|
50
|
-
Get started
|
|
51
|
-
-----------
|
|
52
|
-
|
|
53
|
-
Using docker
|
|
54
|
-
~~~~~~~~~~~~
|
|
55
|
-
|
|
56
|
-
See `getting started with Docker <docs/docker.md>`__
|
|
57
|
-
|
|
58
|
-
Installation on Debian/Ubuntu
|
|
59
|
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
60
|
-
|
|
61
|
-
Dependencies required to install bigquery_fdw:
|
|
62
|
-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
63
|
-
|
|
64
|
-
You need to install the following dependencies:
|
|
65
|
-
|
|
66
|
-
.. code:: bash
|
|
67
|
-
|
|
68
|
-
# Install required packages
|
|
69
|
-
apt update
|
|
70
|
-
apt install -y postgresql-server-dev-14 python3-setuptools python3-dev make gcc git
|
|
71
|
-
|
|
72
|
-
All PostgreSQL versions from 9.2 to 14 should be supported.
|
|
73
|
-
|
|
74
|
-
Installation
|
|
75
|
-
^^^^^^^^^^^^
|
|
76
|
-
|
|
77
|
-
.. code:: bash
|
|
78
|
-
|
|
79
|
-
# Install Multicorn
|
|
80
|
-
# pgsql-io/multicorn2 is a fork of Segfault-Inc/Multicorn that adds support for PostgreSQL 13/14.
|
|
81
|
-
# Alternatively, up to PostgreSQL 12, you can use gabfl/Multicorn that adds better support for Python3.
|
|
82
|
-
# You may also choose to build against the original project instead.
|
|
83
|
-
git clone https://github.com/pgsql-io/multicorn2.git Multicorn && cd Multicorn
|
|
84
|
-
make && make install
|
|
85
|
-
|
|
86
|
-
# Install bigquery_fdw
|
|
87
|
-
pip3 install bigquery-fdw
|
|
88
|
-
|
|
89
|
-
Major dependencies installed automatically during the installation
|
|
90
|
-
process:
|
|
91
|
-
|
|
92
|
-
- `Google Cloud
|
|
93
|
-
BigQuery <https://pypi.org/project/google-cloud-bigquery/>`__
|
|
94
|
-
- `Multicorn <https://github.com/pgsql-io/multicorn2>`__
|
|
95
|
-
|
|
96
|
-
Authentication
|
|
97
|
-
--------------
|
|
98
|
-
|
|
99
|
-
bigquery_fdw relies on `Google Cloud API’s default
|
|
100
|
-
authentication <https://cloud.google.com/docs/authentication/getting-started#linux-or-macos>`__.
|
|
101
|
-
|
|
102
|
-
Your need to have an environment variable
|
|
103
|
-
``GOOGLE_APPLICATION_CREDENTIALS`` that has to be accessible by
|
|
104
|
-
bigquery_fdw. Setting environment variables varies depending on OS but
|
|
105
|
-
for Ubuntu or Debian, the preferred way is to edit
|
|
106
|
-
``/etc/postgresql/[version]/main/environment`` and add:
|
|
107
|
-
|
|
108
|
-
::
|
|
109
|
-
|
|
110
|
-
GOOGLE_APPLICATION_CREDENTIALS = '/path/to/key.json'
|
|
111
|
-
|
|
112
|
-
Restarting PostgreSQL is required for the environment variable to be
|
|
113
|
-
loaded.
|
|
114
|
-
|
|
115
|
-
Usage
|
|
116
|
-
-----
|
|
117
|
-
|
|
118
|
-
We recommend testing the `BigQuery client
|
|
119
|
-
connectivity <docs/test_client.md>`__ before trying to use the FDW.
|
|
120
|
-
|
|
121
|
-
With ``psql``:
|
|
122
|
-
|
|
123
|
-
.. code:: sql
|
|
124
|
-
|
|
125
|
-
CREATE EXTENSION multicorn;
|
|
126
|
-
|
|
127
|
-
CREATE SERVER bigquery_srv FOREIGN DATA WRAPPER multicorn
|
|
128
|
-
OPTIONS (
|
|
129
|
-
wrapper 'bigquery_fdw.fdw.ConstantForeignDataWrapper'
|
|
130
|
-
);
|
|
131
|
-
|
|
132
|
-
CREATE FOREIGN TABLE my_bigquery_table (
|
|
133
|
-
column1 text,
|
|
134
|
-
column2 bigint
|
|
135
|
-
) SERVER bigquery_srv
|
|
136
|
-
OPTIONS (
|
|
137
|
-
fdw_dataset 'my_dataset',
|
|
138
|
-
fdw_table 'my_table'
|
|
139
|
-
);
|
|
140
|
-
|
|
141
|
-
Options
|
|
142
|
-
-------
|
|
143
|
-
|
|
144
|
-
List of options implemented in ``CREATE FOREIGN TABLE`` syntax:
|
|
145
|
-
|
|
146
|
-
+--------------------------+---------------------+---------------------+
|
|
147
|
-
| Option | Default | Description |
|
|
148
|
-
+==========================+=====================+=====================+
|
|
149
|
-
| ``fdw_dataset`` | - | BigQuery dataset |
|
|
150
|
-
| | | name |
|
|
151
|
-
+--------------------------+---------------------+---------------------+
|
|
152
|
-
| ``fdw_table`` | - | BigQuery table name |
|
|
153
|
-
+--------------------------+---------------------+---------------------+
|
|
154
|
-
| ``fdw_convert_tz`` | - | Convert BigQuery |
|
|
155
|
-
| | | time zone for dates |
|
|
156
|
-
| | | and timestamps to |
|
|
157
|
-
| | | selected time zone. |
|
|
158
|
-
| | | Example: |
|
|
159
|
-
| | | ``'US/Eastern'``. |
|
|
160
|
-
+--------------------------+---------------------+---------------------+
|
|
161
|
-
| ``fdw_group`` | ``'false'`` | See `Remote |
|
|
162
|
-
| | | grouping and |
|
|
163
|
-
| | | counting <docs/remo |
|
|
164
|
-
| | | te_grouping.md>`__. |
|
|
165
|
-
+--------------------------+---------------------+---------------------+
|
|
166
|
-
| ``fdw_casting`` | - | See |
|
|
167
|
-
| | | `Casting <d |
|
|
168
|
-
| | | ocs/casting.md>`__. |
|
|
169
|
-
+--------------------------+---------------------+---------------------+
|
|
170
|
-
| ``fdw_verbose`` | ``'false'`` | Set to ``'true'`` |
|
|
171
|
-
| | | to output debug |
|
|
172
|
-
| | | information in |
|
|
173
|
-
| | | PostrgeSQL’s logs |
|
|
174
|
-
+--------------------------+---------------------+---------------------+
|
|
175
|
-
| ``fdw_sql_dialect`` | ``'standard'`` | BigQuery SQL |
|
|
176
|
-
| | | dialect. Currently |
|
|
177
|
-
| | | only ``standard`` |
|
|
178
|
-
| | | is supported. |
|
|
179
|
-
+--------------------------+---------------------+---------------------+
|
|
180
|
-
|
|
181
|
-
More documentation
|
|
182
|
-
------------------
|
|
183
|
-
|
|
184
|
-
See `bigquery_fdw documentation <docs/README.md>`__.
|
|
185
|
-
|
|
186
|
-
.. |Pypi| image:: https://img.shields.io/pypi/v/bigquery-fdw.svg
|
|
187
|
-
:target: https://pypi.org/project/bigquery-fdw/
|
|
188
|
-
.. |Build Status| image:: https://github.com/gabfl/bigquery_fdw/actions/workflows/ci.yml/badge.svg?branch=main
|
|
189
|
-
:target: https://github.com/gabfl/bigquery_fdw/actions
|
|
190
|
-
.. |codecov| image:: https://codecov.io/gh/gabfl/bigquery_fdw/branch/main/graph/badge.svg
|
|
191
|
-
:target: https://codecov.io/gh/gabfl/bigquery_fdw
|
|
192
|
-
.. |MIT licensed| image:: https://img.shields.io/badge/license-MIT-green.svg
|
|
193
|
-
:target: https://raw.githubusercontent.com/gabfl/bigquery_fdw/main/LICENSE
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|