ping-dataexport 0.2.1__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. {ping_dataexport-0.2.1/src/ping_dataexport.egg-info → ping_dataexport-0.3.0}/PKG-INFO +39 -23
  2. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/README.md +37 -21
  3. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/pyproject.toml +2 -2
  4. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/__init__.py +1 -1
  5. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/cli.py +5 -4
  6. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/job/validator.py +3 -3
  7. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0/src/ping_dataexport.egg-info}/PKG-INFO +39 -23
  8. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/LICENSE +0 -0
  9. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/MANIFEST.in +0 -0
  10. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/config.sample.ini +0 -0
  11. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/job.sample.ini +0 -0
  12. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/setup.cfg +0 -0
  13. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/__config__.py +0 -0
  14. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/__main__.py +0 -0
  15. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/api.py +0 -0
  16. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/applogger.py +0 -0
  17. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/config.py +0 -0
  18. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/__init__.py +0 -0
  19. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/connector.py +0 -0
  20. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/oracle_connector.py +0 -0
  21. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/pyodbc_connector.py +0 -0
  22. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/sqlite_connector.py +0 -0
  23. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/db/type_map.py +0 -0
  24. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/errors.py +0 -0
  25. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/export/__init__.py +0 -0
  26. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/export/engine.py +0 -0
  27. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/export/options.py +0 -0
  28. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/export/runner.py +0 -0
  29. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/export/util.py +0 -0
  30. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/job/__init__.py +0 -0
  31. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/job/executor.py +0 -0
  32. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/job/shared_context.py +0 -0
  33. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/job/spec.py +0 -0
  34. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/paths.py +0 -0
  35. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/__init__.py +0 -0
  36. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/base.py +0 -0
  37. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/datelist.py +0 -0
  38. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/groupby.py +0 -0
  39. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/monthbydate.py +0 -0
  40. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/single.py +0 -0
  41. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/watermark.py +0 -0
  42. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/__init__.py +0 -0
  43. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/arrow_map.py +0 -0
  44. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/base.py +0 -0
  45. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/delimited.py +0 -0
  46. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/parquet.py +0 -0
  47. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/SOURCES.txt +0 -0
  48. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/dependency_links.txt +0 -0
  49. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/entry_points.txt +0 -0
  50. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/requires.txt +0 -0
  51. {ping_dataexport-0.2.1 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/top_level.txt +0 -0
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ping-dataexport
3
- Version: 0.2.1
4
- Summary: Universal database data export to CSV/TXT/Parquet streaming, typed, no pandas
3
+ Version: 0.3.0
4
+ Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
5
5
  Author: vorapol
6
6
  License-Expression: Apache-2.0
7
7
  Keywords: export,database,csv,parquet,etl,data-platform,airflow
@@ -46,7 +46,7 @@ Dynamic: license-file
46
46
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
47
47
  และไม่ใช้ pandas เด็ดขาด
48
48
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
49
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
49
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
50
50
  และแหล่งข้อมูล ODBC ใดก็ได้
51
51
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
52
52
  ที่ออกแบบมาสำหรับ Airflow
@@ -151,7 +151,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
151
151
  file defaults to `./config.ini` (`-cf` to point elsewhere).
152
152
 
153
153
  ```bash
154
- # preview to the console (no -o/-od): prints the first rows
154
+ # preview to the console (no -o/-dir): prints the first rows
155
155
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
156
156
 
157
157
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -169,11 +169,11 @@ All options:
169
169
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
170
170
  | `-q` | `--query` | SQL query |
171
171
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
172
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
173
- | `-bn` | `--basename` | base filename for mode exports |
172
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
173
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
174
174
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
175
175
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
176
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
176
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
177
177
  | `-quo` | `--quote` | quote character (default `"`) |
178
178
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
179
179
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -188,21 +188,37 @@ All options:
188
188
  | `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
189
189
  | `-j` | `--job` | run one job.ini section — see section 5 |
190
190
  | `-jf` | `--jobfile` | job file path (default `./job.ini`) |
191
- | `-pl` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count) |
191
+ | `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
192
192
  | `-n` | `--name` | job name shown in logs and the default log filename |
193
193
  | `-cf` | `--configfile` | config file path (default `./config.ini`) |
194
194
  | `-lf` | `--logfile` | log file path |
195
195
  | `-t` | `--test` | test mode: validate + preview, no files |
196
196
 
197
+ ### Which options must go together
198
+
199
+ | If you use… | You must also have… | Notes |
200
+ |-------------|--------------------|-------|
201
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
202
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
203
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
204
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
205
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
206
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
207
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
208
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
209
+ | `-m splitrow` | `-row` | no `-col` needed |
210
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
211
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
212
+
197
213
  ## 4. Export modes (partitioned outputs)
198
214
 
199
- Modes split one query into multiple files. All need `-od` (output directory)
200
- + `-bn` (base filename); partition values are always bound SQL parameters.
215
+ Modes split one query into multiple files. All need `-dir` (output directory)
216
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
201
217
 
202
218
  **`groupby` — one file per distinct value of a column:**
203
219
 
204
220
  ```bash
205
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
221
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
206
222
  ```
207
223
  ```
208
224
  by_region/
@@ -217,7 +233,7 @@ by_region/
217
233
 
218
234
  ```bash
219
235
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
220
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
236
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
221
237
  ```
222
238
  ```
223
239
  daily/
@@ -233,7 +249,7 @@ daily/
233
249
 
234
250
  ```bash
235
251
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
236
- -df 2026-01-10 -dr 3 -od rel -bn sales
252
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
237
253
  ```
238
254
  ```
239
255
  rel/
@@ -248,7 +264,7 @@ at the anchor date; omit `-df` to anchor on today):**
248
264
 
249
265
  ```bash
250
266
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
251
- -df 2026-03-15 -mr 3 -od monthly -bn sales
267
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
252
268
  ```
253
269
  ```
254
270
  monthly/
@@ -261,7 +277,7 @@ monthly/
261
277
  **`splitrow` — rotate to a new file every N rows:**
262
278
 
263
279
  ```bash
264
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
280
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
265
281
  ```
266
282
  ```
267
283
  split/
@@ -273,7 +289,7 @@ split/
273
289
  **`watermark` — incremental: only rows newer than the last run:**
274
290
 
275
291
  ```bash
276
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
292
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
277
293
  ```
278
294
  ```
279
295
  incr/
@@ -299,7 +315,7 @@ file — see *Output and logs* below):
299
315
 
300
316
  Put recurring exports in a job file so you never retype long commands. Each
301
317
  `[Section]` is **one job**: run one with `-j SectionName`, or all of them at
302
- once with `-pl`. Every key is simply a CLI option written as a config key:
318
+ once with `-p`. Every key is simply a CLI option written as a config key:
303
319
 
304
320
  ```ini
305
321
  [DailySales]
@@ -322,8 +338,8 @@ Output=out/full_dump.parquet
322
338
 
323
339
  ```bash
324
340
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
325
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
326
- ping-dataexport -pl 4 # ALL sections in parallel, 4 workers
341
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
342
+ ping-dataexport -p 4 # ALL sections in parallel, 4 workers
327
343
  ```
328
344
 
329
345
  Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
@@ -333,11 +349,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
333
349
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
334
350
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
335
351
  | `Output` | `-o` / `--output` | single output file path |
336
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
337
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
352
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
353
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
338
354
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
339
355
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
340
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
356
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
341
357
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
342
358
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
343
359
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
@@ -385,7 +401,7 @@ only, no metadata lookups.
385
401
  - Every run logs structured lines to stdout and to a log file:
386
402
  `-lf path` > `LogFile=` in config.ini > default
387
403
  `{ExportFolder}/logs/log_{jobname}.txt`.
388
- - CLI exit codes: `0` success, `1` failure (in `-pl` parallel mode: `1` if any
404
+ - CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
389
405
  job failed).
390
406
 
391
407
  ## ODBC prerequisites (ODBC sources only)
@@ -17,7 +17,7 @@
17
17
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
18
18
  และไม่ใช้ pandas เด็ดขาด
19
19
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
20
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
20
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
21
21
  และแหล่งข้อมูล ODBC ใดก็ได้
22
22
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
23
23
  ที่ออกแบบมาสำหรับ Airflow
@@ -122,7 +122,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
122
122
  file defaults to `./config.ini` (`-cf` to point elsewhere).
123
123
 
124
124
  ```bash
125
- # preview to the console (no -o/-od): prints the first rows
125
+ # preview to the console (no -o/-dir): prints the first rows
126
126
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
127
127
 
128
128
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -140,11 +140,11 @@ All options:
140
140
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
141
141
  | `-q` | `--query` | SQL query |
142
142
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
143
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
144
- | `-bn` | `--basename` | base filename for mode exports |
143
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
144
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
145
145
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
146
146
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
147
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
147
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
148
148
  | `-quo` | `--quote` | quote character (default `"`) |
149
149
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
150
150
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -159,21 +159,37 @@ All options:
159
159
  | `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
160
160
  | `-j` | `--job` | run one job.ini section — see section 5 |
161
161
  | `-jf` | `--jobfile` | job file path (default `./job.ini`) |
162
- | `-pl` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count) |
162
+ | `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
163
163
  | `-n` | `--name` | job name shown in logs and the default log filename |
164
164
  | `-cf` | `--configfile` | config file path (default `./config.ini`) |
165
165
  | `-lf` | `--logfile` | log file path |
166
166
  | `-t` | `--test` | test mode: validate + preview, no files |
167
167
 
168
+ ### Which options must go together
169
+
170
+ | If you use… | You must also have… | Notes |
171
+ |-------------|--------------------|-------|
172
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
173
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
174
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
175
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
176
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
177
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
178
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
179
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
180
+ | `-m splitrow` | `-row` | no `-col` needed |
181
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
182
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
183
+
168
184
  ## 4. Export modes (partitioned outputs)
169
185
 
170
- Modes split one query into multiple files. All need `-od` (output directory)
171
- + `-bn` (base filename); partition values are always bound SQL parameters.
186
+ Modes split one query into multiple files. All need `-dir` (output directory)
187
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
172
188
 
173
189
  **`groupby` — one file per distinct value of a column:**
174
190
 
175
191
  ```bash
176
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
192
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
177
193
  ```
178
194
  ```
179
195
  by_region/
@@ -188,7 +204,7 @@ by_region/
188
204
 
189
205
  ```bash
190
206
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
191
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
207
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
192
208
  ```
193
209
  ```
194
210
  daily/
@@ -204,7 +220,7 @@ daily/
204
220
 
205
221
  ```bash
206
222
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
207
- -df 2026-01-10 -dr 3 -od rel -bn sales
223
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
208
224
  ```
209
225
  ```
210
226
  rel/
@@ -219,7 +235,7 @@ at the anchor date; omit `-df` to anchor on today):**
219
235
 
220
236
  ```bash
221
237
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
222
- -df 2026-03-15 -mr 3 -od monthly -bn sales
238
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
223
239
  ```
224
240
  ```
225
241
  monthly/
@@ -232,7 +248,7 @@ monthly/
232
248
  **`splitrow` — rotate to a new file every N rows:**
233
249
 
234
250
  ```bash
235
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
251
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
236
252
  ```
237
253
  ```
238
254
  split/
@@ -244,7 +260,7 @@ split/
244
260
  **`watermark` — incremental: only rows newer than the last run:**
245
261
 
246
262
  ```bash
247
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
263
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
248
264
  ```
249
265
  ```
250
266
  incr/
@@ -270,7 +286,7 @@ file — see *Output and logs* below):
270
286
 
271
287
  Put recurring exports in a job file so you never retype long commands. Each
272
288
  `[Section]` is **one job**: run one with `-j SectionName`, or all of them at
273
- once with `-pl`. Every key is simply a CLI option written as a config key:
289
+ once with `-p`. Every key is simply a CLI option written as a config key:
274
290
 
275
291
  ```ini
276
292
  [DailySales]
@@ -293,8 +309,8 @@ Output=out/full_dump.parquet
293
309
 
294
310
  ```bash
295
311
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
296
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
297
- ping-dataexport -pl 4 # ALL sections in parallel, 4 workers
312
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
313
+ ping-dataexport -p 4 # ALL sections in parallel, 4 workers
298
314
  ```
299
315
 
300
316
  Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
@@ -304,11 +320,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
304
320
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
305
321
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
306
322
  | `Output` | `-o` / `--output` | single output file path |
307
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
308
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
323
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
324
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
309
325
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
310
326
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
311
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
327
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
312
328
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
313
329
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
314
330
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
@@ -356,7 +372,7 @@ only, no metadata lookups.
356
372
  - Every run logs structured lines to stdout and to a log file:
357
373
  `-lf path` > `LogFile=` in config.ini > default
358
374
  `{ExportFolder}/logs/log_{jobname}.txt`.
359
- - CLI exit codes: `0` success, `1` failure (in `-pl` parallel mode: `1` if any
375
+ - CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
360
376
  job failed).
361
377
 
362
378
  ## ODBC prerequisites (ODBC sources only)
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "ping-dataexport"
7
- version = "0.2.1"
8
- description = "Universal database data export to CSV/TXT/Parquet streaming, typed, no pandas"
7
+ version = "0.3.0"
8
+ description = "PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project"
9
9
  readme = "README.md"
10
10
  license = "Apache-2.0"
11
11
  license-files = ["LICENSE"]
@@ -8,4 +8,4 @@ The package uses relative imports only and has no import-time side effects, so
8
8
  the directory can be copied into any project and imported directly.
9
9
  """
10
10
 
11
- __version__ = "0.2.1"
11
+ __version__ = "0.3.0"
@@ -58,15 +58,16 @@ def build_parser() -> argparse.ArgumentParser:
58
58
  p.add_argument("-q", "--query", help="SQL query (required unless --job is used)")
59
59
 
60
60
  p.add_argument("-o", "--output", help="single output file path")
61
- p.add_argument("-od", "--outputdir", help="output directory (required with --mode)")
62
- p.add_argument("-bn", "--basename", help="base filename for mode exports")
61
+ p.add_argument("-dir", "-od", "--outputdir", help="output directory (required with --mode)")
62
+ p.add_argument("-pf", "-bn", "--prefix", "--basename", dest="basename",
63
+ help="output filename prefix for mode exports")
63
64
 
64
65
  p.add_argument("-f", "--format", choices=FORMATS, help="output format (auto from extension)")
65
66
  p.add_argument("-gz", "--gzip", action="store_true", default=None,
66
67
  help="gzip compress output (csv/txt only)")
67
68
 
68
69
  p.add_argument("-quo", "--quote", help='quote character (default ")')
69
- p.add_argument("-sp", "--sep", help="field separator (default ,) — \\t maps to tab")
70
+ p.add_argument("-sep", "--sep", help="field separator (default ,) — \\t maps to tab")
70
71
  p.add_argument("-stnl", "--stripnewline", choices=STRIP_NEWLINE_MODES,
71
72
  help="newline handling in text columns")
72
73
  p.add_argument("-stmc", "--stripmetachar", action="store_true", default=None,
@@ -91,7 +92,7 @@ def build_parser() -> argparse.ArgumentParser:
91
92
  p.add_argument("-lf", "--logfile", help="log file path")
92
93
  p.add_argument("-t", "--test", action="store_true", default=False,
93
94
  help="test mode: preview only, no files")
94
- p.add_argument("-pl", "--parallel", nargs="?", const=0, type=int, metavar="N",
95
+ p.add_argument("-p", "-pl", "--parallel", nargs="?", const=0, type=int, metavar="N",
95
96
  help="run ALL job sections in parallel with N workers (default: CPU count)")
96
97
  return p
97
98
 
@@ -25,15 +25,15 @@ class JobValidator:
25
25
  e(spec, "--query is required (or provide Query in the job section)")
26
26
 
27
27
  if spec.output and (spec.outputdir or spec.basename):
28
- e(spec, "--output cannot be mixed with --outputdir/--basename")
28
+ e(spec, "--output cannot be mixed with --outputdir/--prefix")
29
29
 
30
30
  if spec.mode:
31
31
  if spec.mode not in MODES:
32
32
  e(spec, f"unknown mode {spec.mode!r} (expected one of: {', '.join(MODES)})")
33
33
  if not (spec.outputdir and spec.basename):
34
- e(spec, f"--mode {spec.mode} requires both --outputdir and --basename")
34
+ e(spec, f"--mode {spec.mode} requires both --outputdir and --prefix")
35
35
  elif spec.outputdir or spec.basename:
36
- e(spec, "--outputdir/--basename require --mode")
36
+ e(spec, "--outputdir/--prefix require --mode")
37
37
 
38
38
  if spec.mode in _MODES_NEEDING_COLUMN and not spec.column:
39
39
  e(spec, f"--mode {spec.mode} requires --column")
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ping-dataexport
3
- Version: 0.2.1
4
- Summary: Universal database data export to CSV/TXT/Parquet streaming, typed, no pandas
3
+ Version: 0.3.0
4
+ Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
5
5
  Author: vorapol
6
6
  License-Expression: Apache-2.0
7
7
  Keywords: export,database,csv,parquet,etl,data-platform,airflow
@@ -46,7 +46,7 @@ Dynamic: license-file
46
46
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
47
47
  และไม่ใช้ pandas เด็ดขาด
48
48
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
49
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
49
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
50
50
  และแหล่งข้อมูล ODBC ใดก็ได้
51
51
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
52
52
  ที่ออกแบบมาสำหรับ Airflow
@@ -151,7 +151,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
151
151
  file defaults to `./config.ini` (`-cf` to point elsewhere).
152
152
 
153
153
  ```bash
154
- # preview to the console (no -o/-od): prints the first rows
154
+ # preview to the console (no -o/-dir): prints the first rows
155
155
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
156
156
 
157
157
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -169,11 +169,11 @@ All options:
169
169
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
170
170
  | `-q` | `--query` | SQL query |
171
171
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
172
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
173
- | `-bn` | `--basename` | base filename for mode exports |
172
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
173
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
174
174
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
175
175
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
176
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
176
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
177
177
  | `-quo` | `--quote` | quote character (default `"`) |
178
178
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
179
179
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -188,21 +188,37 @@ All options:
188
188
  | `-rm` | `--removeexisting` | `Yes`: clear the output directory before exporting |
189
189
  | `-j` | `--job` | run one job.ini section — see section 5 |
190
190
  | `-jf` | `--jobfile` | job file path (default `./job.ini`) |
191
- | `-pl` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count) |
191
+ | `-p` | `--parallel` | run ALL job sections in parallel with N workers (default: CPU count); `-pl` still works as a legacy alias |
192
192
  | `-n` | `--name` | job name shown in logs and the default log filename |
193
193
  | `-cf` | `--configfile` | config file path (default `./config.ini`) |
194
194
  | `-lf` | `--logfile` | log file path |
195
195
  | `-t` | `--test` | test mode: validate + preview, no files |
196
196
 
197
+ ### Which options must go together
198
+
199
+ | If you use… | You must also have… | Notes |
200
+ |-------------|--------------------|-------|
201
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
202
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
203
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
204
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
205
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
206
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
207
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
208
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
209
+ | `-m splitrow` | `-row` | no `-col` needed |
210
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
211
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
212
+
197
213
  ## 4. Export modes (partitioned outputs)
198
214
 
199
- Modes split one query into multiple files. All need `-od` (output directory)
200
- + `-bn` (base filename); partition values are always bound SQL parameters.
215
+ Modes split one query into multiple files. All need `-dir` (output directory)
216
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
201
217
 
202
218
  **`groupby` — one file per distinct value of a column:**
203
219
 
204
220
  ```bash
205
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
221
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
206
222
  ```
207
223
  ```
208
224
  by_region/
@@ -217,7 +233,7 @@ by_region/
217
233
 
218
234
  ```bash
219
235
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
220
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
236
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
221
237
  ```
222
238
  ```
223
239
  daily/
@@ -233,7 +249,7 @@ daily/
233
249
 
234
250
  ```bash
235
251
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
236
- -df 2026-01-10 -dr 3 -od rel -bn sales
252
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
237
253
  ```
238
254
  ```
239
255
  rel/
@@ -248,7 +264,7 @@ at the anchor date; omit `-df` to anchor on today):**
248
264
 
249
265
  ```bash
250
266
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
251
- -df 2026-03-15 -mr 3 -od monthly -bn sales
267
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
252
268
  ```
253
269
  ```
254
270
  monthly/
@@ -261,7 +277,7 @@ monthly/
261
277
  **`splitrow` — rotate to a new file every N rows:**
262
278
 
263
279
  ```bash
264
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
280
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
265
281
  ```
266
282
  ```
267
283
  split/
@@ -273,7 +289,7 @@ split/
273
289
  **`watermark` — incremental: only rows newer than the last run:**
274
290
 
275
291
  ```bash
276
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
292
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
277
293
  ```
278
294
  ```
279
295
  incr/
@@ -299,7 +315,7 @@ file — see *Output and logs* below):
299
315
 
300
316
  Put recurring exports in a job file so you never retype long commands. Each
301
317
  `[Section]` is **one job**: run one with `-j SectionName`, or all of them at
302
- once with `-pl`. Every key is simply a CLI option written as a config key:
318
+ once with `-p`. Every key is simply a CLI option written as a config key:
303
319
 
304
320
  ```ini
305
321
  [DailySales]
@@ -322,8 +338,8 @@ Output=out/full_dump.parquet
322
338
 
323
339
  ```bash
324
340
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
325
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
326
- ping-dataexport -pl 4 # ALL sections in parallel, 4 workers
341
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
342
+ ping-dataexport -p 4 # ALL sections in parallel, 4 workers
327
343
  ```
328
344
 
329
345
  Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
@@ -333,11 +349,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
333
349
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
334
350
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
335
351
  | `Output` | `-o` / `--output` | single output file path |
336
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
337
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
352
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
353
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
338
354
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
339
355
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
340
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
356
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
341
357
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
342
358
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
343
359
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
@@ -385,7 +401,7 @@ only, no metadata lookups.
385
401
  - Every run logs structured lines to stdout and to a log file:
386
402
  `-lf path` > `LogFile=` in config.ini > default
387
403
  `{ExportFolder}/logs/log_{jobname}.txt`.
388
- - CLI exit codes: `0` success, `1` failure (in `-pl` parallel mode: `1` if any
404
+ - CLI exit codes: `0` success, `1` failure (in `-p` parallel mode: `1` if any
389
405
  job failed).
390
406
 
391
407
  ## ODBC prerequisites (ODBC sources only)
File without changes