ping-dataexport 0.2.1__tar.gz → 0.2.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ping_dataexport-0.2.1/src/ping_dataexport.egg-info → ping_dataexport-0.2.2}/PKG-INFO +6 -6
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/README.md +4 -4
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/pyproject.toml +2 -2
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/__init__.py +1 -1
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/cli.py +1 -1
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2/src/ping_dataexport.egg-info}/PKG-INFO +6 -6
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/LICENSE +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/MANIFEST.in +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/config.sample.ini +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/job.sample.ini +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/setup.cfg +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/__config__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/__main__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/api.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/applogger.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/config.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/__init__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/connector.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/oracle_connector.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/pyodbc_connector.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/sqlite_connector.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/db/type_map.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/errors.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/export/__init__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/export/engine.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/export/options.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/export/runner.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/export/util.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/job/__init__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/job/executor.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/job/shared_context.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/job/spec.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/job/validator.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/paths.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/__init__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/base.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/datelist.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/groupby.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/monthbydate.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/single.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/planner/watermark.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/writer/__init__.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/writer/arrow_map.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/writer/base.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/writer/delimited.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport/writer/parquet.py +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/SOURCES.txt +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/dependency_links.txt +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/entry_points.txt +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/requires.txt +0 -0
- {ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/top_level.txt +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ping-dataexport
|
|
3
|
-
Version: 0.2.
|
|
4
|
-
Summary:
|
|
3
|
+
Version: 0.2.2
|
|
4
|
+
Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
|
|
5
5
|
Author: vorapol
|
|
6
6
|
License-Expression: Apache-2.0
|
|
7
7
|
Keywords: export,database,csv,parquet,etl,data-platform,airflow
|
|
@@ -188,7 +188,7 @@ All options:
|
|
|
188
188
|
| `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
|
|
189
189
|
| `-j` | `--job` | run one job.ini section — see section 5 |
|
|
190
190
|
| `-jf` | `--jobfile` | job file path (default `./job.ini`) |
|
|
191
|
-
| `-
|
|
191
|
+
| `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
|
|
192
192
|
| `-n` | `--name` | job name shown in logs and the default log filename |
|
|
193
193
|
| `-cf` | `--configfile` | config file path (default `./config.ini`) |
|
|
194
194
|
| `-lf` | `--logfile` | log file path |
|
|
@@ -299,7 +299,7 @@ file — see *Output and logs* below):
|
|
|
299
299
|
|
|
300
300
|
Put recurring exports in a job file so you never retype long commands. Each
|
|
301
301
|
`[Section]` is **one job**: run one with `-j SectionName`, or all of them at
|
|
302
|
-
once with `-
|
|
302
|
+
once with `-p`. Every key is simply a CLI option written as a config key:
|
|
303
303
|
|
|
304
304
|
```ini
|
|
305
305
|
[DailySales]
|
|
@@ -323,7 +323,7 @@ Output=out/full_dump.parquet
|
|
|
323
323
|
```bash
|
|
324
324
|
ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
|
|
325
325
|
ping-dataexport -j DailySales -od other_dir # CLI options override the section
|
|
326
|
-
ping-dataexport -
|
|
326
|
+
ping-dataexport -p 4 # ALL sections in parallel, 4 workers
|
|
327
327
|
```
|
|
328
328
|
|
|
329
329
|
Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
|
|
@@ -385,7 +385,7 @@ only, no metadata lookups.
|
|
|
385
385
|
- Every run logs structured lines to stdout and to a log file:
|
|
386
386
|
`-lf path` > `LogFile=` in config.ini > default
|
|
387
387
|
`{ExportFolder}/logs/log_{jobname}.txt`.
|
|
388
|
-
- CLI exit codes: `0` success, `1` failure (in `-
|
|
388
|
+
- CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
|
|
389
389
|
job failed).
|
|
390
390
|
|
|
391
391
|
## ODBC prerequisites (ODBC sources only)
|
|
@@ -159,7 +159,7 @@ All options:
|
|
|
159
159
|
| `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
|
|
160
160
|
| `-j` | `--job` | run one job.ini section — see section 5 |
|
|
161
161
|
| `-jf` | `--jobfile` | job file path (default `./job.ini`) |
|
|
162
|
-
| `-
|
|
162
|
+
| `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
|
|
163
163
|
| `-n` | `--name` | job name shown in logs and the default log filename |
|
|
164
164
|
| `-cf` | `--configfile` | config file path (default `./config.ini`) |
|
|
165
165
|
| `-lf` | `--logfile` | log file path |
|
|
@@ -270,7 +270,7 @@ file — see *Output and logs* below):
|
|
|
270
270
|
|
|
271
271
|
Put recurring exports in a job file so you never retype long commands. Each
|
|
272
272
|
`[Section]` is **one job**: run one with `-j SectionName`, or all of them at
|
|
273
|
-
once with `-
|
|
273
|
+
once with `-p`. Every key is simply a CLI option written as a config key:
|
|
274
274
|
|
|
275
275
|
```ini
|
|
276
276
|
[DailySales]
|
|
@@ -294,7 +294,7 @@ Output=out/full_dump.parquet
|
|
|
294
294
|
```bash
|
|
295
295
|
ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
|
|
296
296
|
ping-dataexport -j DailySales -od other_dir # CLI options override the section
|
|
297
|
-
ping-dataexport -
|
|
297
|
+
ping-dataexport -p 4 # ALL sections in parallel, 4 workers
|
|
298
298
|
```
|
|
299
299
|
|
|
300
300
|
Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
|
|
@@ -356,7 +356,7 @@ only, no metadata lookups.
|
|
|
356
356
|
- Every run logs structured lines to stdout and to a log file:
|
|
357
357
|
`-lf path` > `LogFile=` in config.ini > default
|
|
358
358
|
`{ExportFolder}/logs/log_{jobname}.txt`.
|
|
359
|
-
- CLI exit codes: `0` success, `1` failure (in `-
|
|
359
|
+
- CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
|
|
360
360
|
job failed).
|
|
361
361
|
|
|
362
362
|
## ODBC prerequisites (ODBC sources only)
|
|
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "ping-dataexport"
|
|
7
|
-
version = "0.2.
|
|
8
|
-
description = "
|
|
7
|
+
version = "0.2.2"
|
|
8
|
+
description = "PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = "Apache-2.0"
|
|
11
11
|
license-files = ["LICENSE"]
|
|
@@ -91,7 +91,7 @@ def build_parser() -> argparse.ArgumentParser:
|
|
|
91
91
|
p.add_argument("-lf", "--logfile", help="log file path")
|
|
92
92
|
p.add_argument("-t", "--test", action="store_true", default=False,
|
|
93
93
|
help="test mode: preview only, no files")
|
|
94
|
-
p.add_argument("-pl", "--parallel", nargs="?", const=0, type=int, metavar="N",
|
|
94
|
+
p.add_argument("-p", "-pl", "--parallel", nargs="?", const=0, type=int, metavar="N",
|
|
95
95
|
help="run ALL job sections in parallel with N workers (default: CPU count)")
|
|
96
96
|
return p
|
|
97
97
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ping-dataexport
|
|
3
|
-
Version: 0.2.
|
|
4
|
-
Summary:
|
|
3
|
+
Version: 0.2.2
|
|
4
|
+
Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
|
|
5
5
|
Author: vorapol
|
|
6
6
|
License-Expression: Apache-2.0
|
|
7
7
|
Keywords: export,database,csv,parquet,etl,data-platform,airflow
|
|
@@ -188,7 +188,7 @@ All options:
|
|
|
188
188
|
| `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
|
|
189
189
|
| `-j` | `--job` | run one job.ini section — see section 5 |
|
|
190
190
|
| `-jf` | `--jobfile` | job file path (default `./job.ini`) |
|
|
191
|
-
| `-
|
|
191
|
+
| `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
|
|
192
192
|
| `-n` | `--name` | job name shown in logs and the default log filename |
|
|
193
193
|
| `-cf` | `--configfile` | config file path (default `./config.ini`) |
|
|
194
194
|
| `-lf` | `--logfile` | log file path |
|
|
@@ -299,7 +299,7 @@ file — see *Output and logs* below):
|
|
|
299
299
|
|
|
300
300
|
Put recurring exports in a job file so you never retype long commands. Each
|
|
301
301
|
`[Section]` is **one job**: run one with `-j SectionName`, or all of them at
|
|
302
|
-
once with `-
|
|
302
|
+
once with `-p`. Every key is simply a CLI option written as a config key:
|
|
303
303
|
|
|
304
304
|
```ini
|
|
305
305
|
[DailySales]
|
|
@@ -323,7 +323,7 @@ Output=out/full_dump.parquet
|
|
|
323
323
|
```bash
|
|
324
324
|
ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
|
|
325
325
|
ping-dataexport -j DailySales -od other_dir # CLI options override the section
|
|
326
|
-
ping-dataexport -
|
|
326
|
+
ping-dataexport -p 4 # ALL sections in parallel, 4 workers
|
|
327
327
|
```
|
|
328
328
|
|
|
329
329
|
Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
|
|
@@ -385,7 +385,7 @@ only, no metadata lookups.
|
|
|
385
385
|
- Every run logs structured lines to stdout and to a log file:
|
|
386
386
|
`-lf path` > `LogFile=` in config.ini > default
|
|
387
387
|
`{ExportFolder}/logs/log_{jobname}.txt`.
|
|
388
|
-
- CLI exit codes: `0` success, `1` failure (in `-
|
|
388
|
+
- CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
|
|
389
389
|
job failed).
|
|
390
390
|
|
|
391
391
|
## ODBC prerequisites (ODBC sources only)
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
{ping_dataexport-0.2.1 → ping_dataexport-0.2.2}/src/ping_dataexport.egg-info/entry_points.txt
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|