ping-dataexport 0.2.2__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. {ping_dataexport-0.2.2/src/ping_dataexport.egg-info → ping_dataexport-0.3.0}/PKG-INFO +34 -18
  2. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/README.md +33 -17
  3. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/pyproject.toml +1 -1
  4. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/__init__.py +1 -1
  5. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/cli.py +4 -3
  6. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/job/validator.py +3 -3
  7. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0/src/ping_dataexport.egg-info}/PKG-INFO +34 -18
  8. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/LICENSE +0 -0
  9. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/MANIFEST.in +0 -0
  10. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/config.sample.ini +0 -0
  11. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/job.sample.ini +0 -0
  12. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/setup.cfg +0 -0
  13. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/__config__.py +0 -0
  14. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/__main__.py +0 -0
  15. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/api.py +0 -0
  16. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/applogger.py +0 -0
  17. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/config.py +0 -0
  18. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/__init__.py +0 -0
  19. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/connector.py +0 -0
  20. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/oracle_connector.py +0 -0
  21. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/pyodbc_connector.py +0 -0
  22. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/sqlite_connector.py +0 -0
  23. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/db/type_map.py +0 -0
  24. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/errors.py +0 -0
  25. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/export/__init__.py +0 -0
  26. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/export/engine.py +0 -0
  27. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/export/options.py +0 -0
  28. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/export/runner.py +0 -0
  29. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/export/util.py +0 -0
  30. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/job/__init__.py +0 -0
  31. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/job/executor.py +0 -0
  32. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/job/shared_context.py +0 -0
  33. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/job/spec.py +0 -0
  34. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/paths.py +0 -0
  35. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/__init__.py +0 -0
  36. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/base.py +0 -0
  37. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/datelist.py +0 -0
  38. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/groupby.py +0 -0
  39. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/monthbydate.py +0 -0
  40. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/single.py +0 -0
  41. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/planner/watermark.py +0 -0
  42. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/__init__.py +0 -0
  43. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/arrow_map.py +0 -0
  44. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/base.py +0 -0
  45. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/delimited.py +0 -0
  46. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport/writer/parquet.py +0 -0
  47. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/SOURCES.txt +0 -0
  48. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/dependency_links.txt +0 -0
  49. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/entry_points.txt +0 -0
  50. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/requires.txt +0 -0
  51. {ping_dataexport-0.2.2 → ping_dataexport-0.3.0}/src/ping_dataexport.egg-info/top_level.txt +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ping-dataexport
3
- Version: 0.2.2
3
+ Version: 0.3.0
4
4
  Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
5
5
  Author: vorapol
6
6
  License-Expression: Apache-2.0
@@ -46,7 +46,7 @@ Dynamic: license-file
46
46
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
47
47
  และไม่ใช้ pandas เด็ดขาด
48
48
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
49
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
49
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
50
50
  และแหล่งข้อมูล ODBC ใดก็ได้
51
51
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
52
52
  ที่ออกแบบมาสำหรับ Airflow
@@ -151,7 +151,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
151
151
  file defaults to `./config.ini` (`-cf` to point elsewhere).
152
152
 
153
153
  ```bash
154
- # preview to the console (no -o/-od): prints the first rows
154
+ # preview to the console (no -o/-dir): prints the first rows
155
155
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
156
156
 
157
157
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -169,11 +169,11 @@ All options:
169
169
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
170
170
  | `-q` | `--query` | SQL query |
171
171
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
172
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
173
- | `-bn` | `--basename` | base filename for mode exports |
172
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
173
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
174
174
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
175
175
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
176
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
176
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
177
177
  | `-quo` | `--quote` | quote character (default `"`) |
178
178
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
179
179
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -194,15 +194,31 @@ All options:
194
194
  | `-lf` | `--logfile` | log file path |
195
195
  | `-t` | `--test` | test mode: validate + preview, no files |
196
196
 
197
+ ### Which options must go together
198
+
199
+ | If you use… | You must also have… | Notes |
200
+ |-------------|--------------------|-------|
201
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
202
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
203
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
204
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
205
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
206
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
207
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
208
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
209
+ | `-m splitrow` | `-row` | no `-col` needed |
210
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
211
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
212
+
197
213
  ## 4. Export modes (partitioned outputs)
198
214
 
199
- Modes split one query into multiple files. All need `-od` (output directory)
200
- + `-bn` (base filename); partition values are always bound SQL parameters.
215
+ Modes split one query into multiple files. All need `-dir` (output directory)
216
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
201
217
 
202
218
  **`groupby` — one file per distinct value of a column:**
203
219
 
204
220
  ```bash
205
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
221
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
206
222
  ```
207
223
  ```
208
224
  by_region/
@@ -217,7 +233,7 @@ by_region/
217
233
 
218
234
  ```bash
219
235
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
220
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
236
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
221
237
  ```
222
238
  ```
223
239
  daily/
@@ -233,7 +249,7 @@ daily/
233
249
 
234
250
  ```bash
235
251
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
236
- -df 2026-01-10 -dr 3 -od rel -bn sales
252
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
237
253
  ```
238
254
  ```
239
255
  rel/
@@ -248,7 +264,7 @@ at the anchor date; omit `-df` to anchor on today):**
248
264
 
249
265
  ```bash
250
266
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
251
- -df 2026-03-15 -mr 3 -od monthly -bn sales
267
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
252
268
  ```
253
269
  ```
254
270
  monthly/
@@ -261,7 +277,7 @@ monthly/
261
277
  **`splitrow` — rotate to a new file every N rows:**
262
278
 
263
279
  ```bash
264
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
280
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
265
281
  ```
266
282
  ```
267
283
  split/
@@ -273,7 +289,7 @@ split/
273
289
  **`watermark` — incremental: only rows newer than the last run:**
274
290
 
275
291
  ```bash
276
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
292
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
277
293
  ```
278
294
  ```
279
295
  incr/
@@ -322,7 +338,7 @@ Output=out/full_dump.parquet
322
338
 
323
339
  ```bash
324
340
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
325
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
341
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
326
342
  ping-dataexport -p 4 # ALL sections in parallel, 4 workers
327
343
  ```
328
344
 
@@ -333,11 +349,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
333
349
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
334
350
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
335
351
  | `Output` | `-o` / `--output` | single output file path |
336
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
337
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
352
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
353
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
338
354
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
339
355
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
340
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
356
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
341
357
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
342
358
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
343
359
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
@@ -17,7 +17,7 @@
17
17
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
18
18
  และไม่ใช้ pandas เด็ดขาด
19
19
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
20
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
20
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
21
21
  และแหล่งข้อมูล ODBC ใดก็ได้
22
22
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
23
23
  ที่ออกแบบมาสำหรับ Airflow
@@ -122,7 +122,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
122
122
  file defaults to `./config.ini` (`-cf` to point elsewhere).
123
123
 
124
124
  ```bash
125
- # preview to the console (no -o/-od): prints the first rows
125
+ # preview to the console (no -o/-dir): prints the first rows
126
126
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
127
127
 
128
128
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -140,11 +140,11 @@ All options:
140
140
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
141
141
  | `-q` | `--query` | SQL query |
142
142
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
143
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
144
- | `-bn` | `--basename` | base filename for mode exports |
143
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
144
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
145
145
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
146
146
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
147
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
147
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
148
148
  | `-quo` | `--quote` | quote character (default `"`) |
149
149
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
150
150
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -165,15 +165,31 @@ All options:
165
165
  | `-lf` | `--logfile` | log file path |
166
166
  | `-t` | `--test` | test mode: validate + preview, no files |
167
167
 
168
+ ### Which options must go together
169
+
170
+ | If you use… | You must also have… | Notes |
171
+ |-------------|--------------------|-------|
172
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
173
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
174
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
175
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
176
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
177
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
178
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
179
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
180
+ | `-m splitrow` | `-row` | no `-col` needed |
181
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
182
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
183
+
168
184
  ## 4. Export modes (partitioned outputs)
169
185
 
170
- Modes split one query into multiple files. All need `-od` (output directory)
171
- + `-bn` (base filename); partition values are always bound SQL parameters.
186
+ Modes split one query into multiple files. All need `-dir` (output directory)
187
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
172
188
 
173
189
  **`groupby` — one file per distinct value of a column:**
174
190
 
175
191
  ```bash
176
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
192
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
177
193
  ```
178
194
  ```
179
195
  by_region/
@@ -188,7 +204,7 @@ by_region/
188
204
 
189
205
  ```bash
190
206
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
191
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
207
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
192
208
  ```
193
209
  ```
194
210
  daily/
@@ -204,7 +220,7 @@ daily/
204
220
 
205
221
  ```bash
206
222
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
207
- -df 2026-01-10 -dr 3 -od rel -bn sales
223
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
208
224
  ```
209
225
  ```
210
226
  rel/
@@ -219,7 +235,7 @@ at the anchor date; omit `-df` to anchor on today):**
219
235
 
220
236
  ```bash
221
237
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
222
- -df 2026-03-15 -mr 3 -od monthly -bn sales
238
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
223
239
  ```
224
240
  ```
225
241
  monthly/
@@ -232,7 +248,7 @@ monthly/
232
248
  **`splitrow` — rotate to a new file every N rows:**
233
249
 
234
250
  ```bash
235
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
251
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
236
252
  ```
237
253
  ```
238
254
  split/
@@ -244,7 +260,7 @@ split/
244
260
  **`watermark` — incremental: only rows newer than the last run:**
245
261
 
246
262
  ```bash
247
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
263
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
248
264
  ```
249
265
  ```
250
266
  incr/
@@ -293,7 +309,7 @@ Output=out/full_dump.parquet
293
309
 
294
310
  ```bash
295
311
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
296
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
312
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
297
313
  ping-dataexport -p 4 # ALL sections in parallel, 4 workers
298
314
  ```
299
315
 
@@ -304,11 +320,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
304
320
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
305
321
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
306
322
  | `Output` | `-o` / `--output` | single output file path |
307
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
308
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
323
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
324
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
309
325
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
310
326
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
311
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
327
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
312
328
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
313
329
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
314
330
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "ping-dataexport"
7
- version = "0.2.2"
7
+ version = "0.3.0"
8
8
  description = "PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project"
9
9
  readme = "README.md"
10
10
  license = "Apache-2.0"
@@ -8,4 +8,4 @@ The package uses relative imports only and has no import-time side effects, so
8
8
  the directory can be copied into any project and imported directly.
9
9
  """
10
10
 
11
- __version__ = "0.2.2"
11
+ __version__ = "0.3.0"
@@ -58,15 +58,16 @@ def build_parser() -> argparse.ArgumentParser:
58
58
  p.add_argument("-q", "--query", help="SQL query (required unless --job is used)")
59
59
 
60
60
  p.add_argument("-o", "--output", help="single output file path")
61
- p.add_argument("-od", "--outputdir", help="output directory (required with --mode)")
62
- p.add_argument("-bn", "--basename", help="base filename for mode exports")
61
+ p.add_argument("-dir", "-od", "--outputdir", help="output directory (required with --mode)")
62
+ p.add_argument("-pf", "-bn", "--prefix", "--basename", dest="basename",
63
+ help="output filename prefix for mode exports")
63
64
 
64
65
  p.add_argument("-f", "--format", choices=FORMATS, help="output format (auto from extension)")
65
66
  p.add_argument("-gz", "--gzip", action="store_true", default=None,
66
67
  help="gzip compress output (csv/txt only)")
67
68
 
68
69
  p.add_argument("-quo", "--quote", help='quote character (default ")')
69
- p.add_argument("-sp", "--sep", help="field separator (default ,) — \\t maps to tab")
70
+ p.add_argument("-sep", "--sep", help="field separator (default ,) — \\t maps to tab")
70
71
  p.add_argument("-stnl", "--stripnewline", choices=STRIP_NEWLINE_MODES,
71
72
  help="newline handling in text columns")
72
73
  p.add_argument("-stmc", "--stripmetachar", action="store_true", default=None,
@@ -25,15 +25,15 @@ class JobValidator:
25
25
  e(spec, "--query is required (or provide Query in the job section)")
26
26
 
27
27
  if spec.output and (spec.outputdir or spec.basename):
28
- e(spec, "--output cannot be mixed with --outputdir/--basename")
28
+ e(spec, "--output cannot be mixed with --outputdir/--prefix")
29
29
 
30
30
  if spec.mode:
31
31
  if spec.mode not in MODES:
32
32
  e(spec, f"unknown mode {spec.mode!r} (expected one of: {', '.join(MODES)})")
33
33
  if not (spec.outputdir and spec.basename):
34
- e(spec, f"--mode {spec.mode} requires both --outputdir and --basename")
34
+ e(spec, f"--mode {spec.mode} requires both --outputdir and --prefix")
35
35
  elif spec.outputdir or spec.basename:
36
- e(spec, "--outputdir/--basename require --mode")
36
+ e(spec, "--outputdir/--prefix require --mode")
37
37
 
38
38
  if spec.mode in _MODES_NEEDING_COLUMN and not spec.column:
39
39
  e(spec, f"--mode {spec.mode} requires --column")
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ping-dataexport
3
- Version: 0.2.2
3
+ Version: 0.3.0
4
4
  Summary: PING's Data Export Tool - To Parquet/CSV/TEXT as Accelerator for Data & AI Project
5
5
  Author: vorapol
6
6
  License-Expression: Apache-2.0
@@ -46,7 +46,7 @@ Dynamic: license-file
46
46
  `decimal128(p,s)`, คอลัมน์ int ที่มี NULL ยังคงเป็น int, วันที่ยังคงเป็นวันที่
47
47
  และไม่ใช้ pandas เด็ดขาด
48
48
  - **ฐานข้อมูลที่รองรับ** — SQLite (Library มาตรฐาน ไม่ต้องติดตั้งอะไรเพิ่ม),
49
- SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบเ Native หรือ ODBC)
49
+ SQL Server (MS ODBC / FreeTDS), Oracle (โหมด thin แบบ Native หรือ ODBC)
50
50
  และแหล่งข้อมูล ODBC ใดก็ได้
51
51
  - **ใช้งานได้สองทาง** — ผ่าน CLI `ping-dataexport` หรือผ่าน Library API (`ping_dataexport.api`)
52
52
  ที่ออกแบบมาสำหรับ Airflow
@@ -151,7 +151,7 @@ PasswordBase64=c2VjcmV0 ; base64("secret")
151
151
  file defaults to `./config.ini` (`-cf` to point elsewhere).
152
152
 
153
153
  ```bash
154
- # preview to the console (no -o/-od): prints the first rows
154
+ # preview to the console (no -o/-dir): prints the first rows
155
155
  ping-dataexport -s SQLITE01 -q "SELECT * FROM sales"
156
156
 
157
157
  # single file — format inferred from the extension (.csv/.txt/.parquet)
@@ -169,11 +169,11 @@ All options:
169
169
  | `-s` | `--source` | DB config section name (e.g. `DB01`) |
170
170
  | `-q` | `--query` | SQL query |
171
171
  | `-o` | `--output` | single output file (format from extension, or `-f`) |
172
- | `-od` | `--outputdir` | output directory (required for `-m` modes) |
173
- | `-bn` | `--basename` | base filename for mode exports |
172
+ | `-dir` | `--outputdir` | output directory (required for `-m` modes); `-od` still works as a legacy alias |
173
+ | `-pf` | `--prefix` | output filename prefix for mode exports; `-bn`/`--basename` still work as legacy aliases |
174
174
  | `-f` | `--format` | `csv`, `txt`, `parquet` |
175
175
  | `-gz` | `--gzip` | gzip the output (CSV/TXT only) |
176
- | `-sp` | `--sep` | field separator (default `,`; `\t` maps to tab) |
176
+ | `-sep` | `--sep` | field separator (default `,`; `\t` maps to tab) |
177
177
  | `-quo` | `--quote` | quote character (default `"`) |
178
178
  | `-stnl` | `--stripnewline` | newline handling in text columns: `space` \| `blank` \| `escape` \| `doubleescape` |
179
179
  | `-stmc` | `--stripmetachar` | strip invisible/control characters from text columns |
@@ -194,15 +194,31 @@ All options:
194
194
  | `-lf` | `--logfile` | log file path |
195
195
  | `-t` | `--test` | test mode: validate + preview, no files |
196
196
 
197
+ ### Which options must go together
198
+
199
+ | If you use… | You must also have… | Notes |
200
+ |-------------|--------------------|-------|
201
+ | (anything) | `-s` + `-q` | the two basics — always required, unless `-j` supplies them from a job file |
202
+ | `-o` | nothing else | single file; **cannot** be combined with `-dir`/`-pf` |
203
+ | `-m` (any mode) | `-dir` + `-pf` | every mode writes multiple files, so it needs a directory + filename prefix |
204
+ | `-dir` or `-pf` | `-m` | they only make sense for mode exports |
205
+ | `-m groupby` / `watermark` | `-col` | the column to group / track |
206
+ | `-m date` | `-col` + `-df` + `-dt` | the date column and an explicit range |
207
+ | `-m relativedate` | `-col` + `-dr` | `-df` optional (defaults to today) |
208
+ | `-m monthbydate` | `-col` + `-mr` | `-df` optional (defaults to today) |
209
+ | `-m splitrow` | `-row` | no `-col` needed |
210
+ | `-gz` | CSV/TXT output | rejected with parquet (it compresses itself) |
211
+ | no `-o` and no `-dir` | — | console preview only (first `PreviewRow` rows, nothing written) |
212
+
197
213
  ## 4. Export modes (partitioned outputs)
198
214
 
199
- Modes split one query into multiple files. All need `-od` (output directory)
200
- + `-bn` (base filename); partition values are always bound SQL parameters.
215
+ Modes split one query into multiple files. All need `-dir` (output directory)
216
+ + `-pf` (filename prefix); partition values are always bound SQL parameters.
201
217
 
202
218
  **`groupby` — one file per distinct value of a column:**
203
219
 
204
220
  ```bash
205
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -od by_region -bn sales
221
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m groupby -col region -dir by_region -pf sales
206
222
  ```
207
223
  ```
208
224
  by_region/
@@ -217,7 +233,7 @@ by_region/
217
233
 
218
234
  ```bash
219
235
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m date -col txn_date \
220
- -df 2026-01-01 -dt 2026-01-05 -od daily -bn sales
236
+ -df 2026-01-01 -dt 2026-01-05 -dir daily -pf sales
221
237
  ```
222
238
  ```
223
239
  daily/
@@ -233,7 +249,7 @@ daily/
233
249
 
234
250
  ```bash
235
251
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m relativedate -col txn_date \
236
- -df 2026-01-10 -dr 3 -od rel -bn sales
252
+ -df 2026-01-10 -dr 3 -dir rel -pf sales
237
253
  ```
238
254
  ```
239
255
  rel/
@@ -248,7 +264,7 @@ at the anchor date; omit `-df` to anchor on today):**
248
264
 
249
265
  ```bash
250
266
  ping-dataexport -s DB01 -q "SELECT * FROM sales" -m monthbydate -col txn_date \
251
- -df 2026-03-15 -mr 3 -od monthly -bn sales
267
+ -df 2026-03-15 -mr 3 -dir monthly -pf sales
252
268
  ```
253
269
  ```
254
270
  monthly/
@@ -261,7 +277,7 @@ monthly/
261
277
  **`splitrow` — rotate to a new file every N rows:**
262
278
 
263
279
  ```bash
264
- ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -od split -bn part
280
+ ping-dataexport -s DB01 -q "SELECT * FROM sales ORDER BY id" -m splitrow -row 100000 -dir split -pf part
265
281
  ```
266
282
  ```
267
283
  split/
@@ -273,7 +289,7 @@ split/
273
289
  **`watermark` — incremental: only rows newer than the last run:**
274
290
 
275
291
  ```bash
276
- ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -od incr -bn sales
292
+ ping-dataexport -s DB01 -q "SELECT * FROM sales" -m watermark -col txn_date -dir incr -pf sales
277
293
  ```
278
294
  ```
279
295
  incr/
@@ -322,7 +338,7 @@ Output=out/full_dump.parquet
322
338
 
323
339
  ```bash
324
340
  ping-dataexport -j DailySales # one section (job.ini by default, -jf elsewhere)
325
- ping-dataexport -j DailySales -od other_dir # CLI options override the section
341
+ ping-dataexport -j DailySales -dir other_dir # CLI options override the section
326
342
  ping-dataexport -p 4 # ALL sections in parallel, 4 workers
327
343
  ```
328
344
 
@@ -333,11 +349,11 @@ Every job.ini key and the CLI option it mirrors (keys are case-insensitive):
333
349
  | `Source` | `-s` / `--source` | which config.ini DB section to export from |
334
350
  | `Query` | `-q` / `--query` | the SQL to run; indent continuation lines for multi-line queries |
335
351
  | `Output` | `-o` / `--output` | single output file path |
336
- | `OutputDir` | `-od` / `--outputdir` | output directory (needed when `Mode=` is set) |
337
- | `BaseName` | `-bn` / `--basename` | base filename for mode exports |
352
+ | `OutputDir` | `-dir` / `--outputdir` | output directory (needed when `Mode=` is set) |
353
+ | `BaseName` | `-pf` / `--prefix` | output filename prefix for mode exports |
338
354
  | `Format` | `-f` / `--format` | `csv` / `txt` / `parquet` (else inferred from `Output` extension) |
339
355
  | `Gzip` | `-gz` / `--gzip` | `Yes`/`No` — gzip the output (CSV/TXT only) |
340
- | `Sep` | `-sp` / `--sep` | field separator (default `,`; `\t` = tab) |
356
+ | `Sep` | `-sep` / `--sep` | field separator (default `,`; `\t` = tab) |
341
357
  | `Quote` | `-quo` / `--quote` | quote character (default `"`) |
342
358
  | `StripNewLine` | `-stnl` / `--stripnewline` | `space` \| `blank` \| `escape` \| `doubleescape` |
343
359
  | `StripMetaChar` | `-stmc` / `--stripmetachar` | `Yes`/`No` — strip invisible/control characters |
File without changes