datacontract-cli 0.10.25__py3-none-any.whl → 0.10.27__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.

This version of datacontract-cli might be problematic. Click here for more details.

Files changed (21) hide show
  1. datacontract/catalog/catalog.py +1 -1
  2. datacontract/cli.py +10 -0
  3. datacontract/data_contract.py +12 -2
  4. datacontract/engines/soda/connections/kafka.py +2 -1
  5. datacontract/export/dbt_converter.py +6 -3
  6. datacontract/export/exporter.py +1 -0
  7. datacontract/export/exporter_factory.py +7 -1
  8. datacontract/export/{html_export.py → html_exporter.py} +6 -0
  9. datacontract/export/mermaid_exporter.py +32 -0
  10. datacontract/export/odcs_v3_exporter.py +22 -4
  11. datacontract/export/sodacl_converter.py +9 -1
  12. datacontract/export/sql_converter.py +6 -2
  13. datacontract/export/sql_type_converter.py +8 -2
  14. datacontract/imports/spark_importer.py +73 -2
  15. datacontract/templates/datacontract.html +48 -2
  16. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/METADATA +335 -385
  17. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/RECORD +21 -20
  18. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/WHEEL +1 -1
  19. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/entry_points.txt +0 -0
  20. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/licenses/LICENSE +0 -0
  21. {datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/top_level.txt +0 -0
{datacontract_cli-0.10.25.dist-info → datacontract_cli-0.10.27.dist-info}/METADATA RENAMED
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: datacontract-cli
3
- Version: 0.10.25
3
+ Version: 0.10.27
4
4
  Summary: The datacontract CLI is an open source command-line tool for working with Data Contracts. It uses data contract YAML files to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
5
5
  Author-email: Jochen Christ <jochen.christ@innoq.com>, Stefan Negele <stefan.negele@innoq.com>, Simon Harrer <simon.harrer@innoq.com>
6
6
  License-Expression: MIT
@@ -12,14 +12,14 @@ Requires-Python: >=3.10
12
12
  Description-Content-Type: text/markdown
13
13
  License-File: LICENSE
14
14
  Requires-Dist: typer<0.16,>=0.15.1
15
- Requires-Dist: pydantic<2.11.0,>=2.8.2
15
+ Requires-Dist: pydantic<2.12.0,>=2.8.2
16
16
  Requires-Dist: pyyaml~=6.0.1
17
17
  Requires-Dist: requests<2.33,>=2.31
18
18
  Requires-Dist: fastjsonschema<2.22.0,>=2.19.1
19
19
  Requires-Dist: fastparquet<2025.0.0,>=2024.5.0
20
20
  Requires-Dist: numpy<2.0.0,>=1.26.4
21
21
  Requires-Dist: python-multipart<1.0.0,>=0.0.20
22
- Requires-Dist: rich<14.0,>=13.7
22
+ Requires-Dist: rich<15.0,>=13.7
23
23
  Requires-Dist: sqlglot<27.0.0,>=26.6.0
24
24
  Requires-Dist: duckdb<2.0.0,>=1.0.0
25
25
  Requires-Dist: soda-core-duckdb<3.6.0,>=3.3.20
@@ -51,8 +51,8 @@ Requires-Dist: soda-core-spark-df<3.6.0,>=3.3.20; extra == "kafka"
51
51
  Provides-Extra: postgres
52
52
  Requires-Dist: soda-core-postgres<3.6.0,>=3.3.20; extra == "postgres"
53
53
  Provides-Extra: s3
54
- Requires-Dist: s3fs==2025.2.0; extra == "s3"
55
- Requires-Dist: aiobotocore<2.20.0,>=2.17.0; extra == "s3"
54
+ Requires-Dist: s3fs<2026.0.0,>=2025.2.0; extra == "s3"
55
+ Requires-Dist: aiobotocore<2.23.0,>=2.17.0; extra == "s3"
56
56
  Provides-Extra: snowflake
57
57
  Requires-Dist: snowflake-connector-python[pandas]<3.15,>=3.6; extra == "snowflake"
58
58
  Requires-Dist: soda-core-snowflake<3.6.0,>=3.3.20; extra == "snowflake"
@@ -70,7 +70,7 @@ Provides-Extra: rdf
70
70
  Requires-Dist: rdflib==7.0.0; extra == "rdf"
71
71
  Provides-Extra: api
72
72
  Requires-Dist: fastapi==0.115.12; extra == "api"
73
- Requires-Dist: uvicorn==0.34.0; extra == "api"
73
+ Requires-Dist: uvicorn==0.34.2; extra == "api"
74
74
  Provides-Extra: protobuf
75
75
  Requires-Dist: grpcio-tools>=1.53; extra == "protobuf"
76
76
  Provides-Extra: all
@@ -79,14 +79,14 @@ Provides-Extra: dev
79
79
  Requires-Dist: datacontract-cli[all]; extra == "dev"
80
80
  Requires-Dist: httpx==0.28.1; extra == "dev"
81
81
  Requires-Dist: kafka-python; extra == "dev"
82
- Requires-Dist: moto==5.1.3; extra == "dev"
82
+ Requires-Dist: moto==5.1.4; extra == "dev"
83
83
  Requires-Dist: pandas>=2.1.0; extra == "dev"
84
84
  Requires-Dist: pre-commit<4.3.0,>=3.7.1; extra == "dev"
85
85
  Requires-Dist: pytest; extra == "dev"
86
86
  Requires-Dist: pytest-xdist; extra == "dev"
87
- Requires-Dist: pymssql==2.3.2; extra == "dev"
87
+ Requires-Dist: pymssql==2.3.4; extra == "dev"
88
88
  Requires-Dist: ruff; extra == "dev"
89
- Requires-Dist: testcontainers[kafka,minio,mssql,postgres]==4.9.2; extra == "dev"
89
+ Requires-Dist: testcontainers[kafka,minio,mssql,postgres]==4.10.0; extra == "dev"
90
90
  Requires-Dist: trino==0.333.0; extra == "dev"
91
91
  Dynamic: license-file
92
92
 
@@ -344,110 +344,86 @@ Commands
344
344
 
345
345
  ### init
346
346
  ```
347
-
348
- Usage: datacontract init [OPTIONS] [LOCATION]
349
-
350
- Create an empty data contract.
351
-
352
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
353
- │ location [LOCATION] The location of the data contract file to │
354
- │ create. │
355
- │ [default: datacontract.yaml] │
356
- ╰──────────────────────────────────────────────────────────────────────────────╯
357
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
358
- │ --template TEXT URL of a template or data contract │
359
- │ [default: None] │
360
- │ --overwrite --no-overwrite Replace the existing │
361
- │ datacontract.yaml │
362
- │ [default: no-overwrite] │
363
- │ --help Show this message and exit. │
364
- ╰──────────────────────────────────────────────────────────────────────────────╯
347
+
348
+ Usage: datacontract init [OPTIONS] [LOCATION]
349
+
350
+ Create an empty data contract.
351
+
352
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
353
+ │ location [LOCATION] The location of the data contract file to create. │
354
+ │ [default: datacontract.yaml] │
355
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
356
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
357
+ │ --template TEXT URL of a template or data contract [default: None] │
358
+ │ --overwrite --no-overwrite Replace the existing datacontract.yaml │
359
+ │ [default: no-overwrite] │
360
+ │ --help Show this message and exit. │
361
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
365
362
 
366
363
  ```
367
364
 
368
365
  ### lint
369
366
  ```
370
-
371
- Usage: datacontract lint [OPTIONS] [LOCATION]
372
-
373
- Validate that the datacontract.yaml is correctly formatted.
374
-
375
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
376
- │ location [LOCATION] The location (url or path) of the data contract │
377
- │ yaml. │
378
- │ [default: datacontract.yaml] │
379
- ╰──────────────────────────────────────────────────────────────────────────────╯
380
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
381
- │ --schema TEXT The location (url or path) of the Data │
382
- │ Contract Specification JSON Schema │
383
- │ [default: None] │
384
- │ --output PATH Specify the file path where the test results │
385
- │ should be written to (e.g., │
386
- │ './test-results/TEST-datacontract.xml'). If │
387
- │ no path is provided, the output will be │
388
- │ printed to stdout. │
389
- │ [default: None] │
390
- │ --output-format [junit] The target format for the test results. │
391
- │ [default: None] │
392
- │ --help Show this message and exit. │
393
- ╰──────────────────────────────────────────────────────────────────────────────╯
367
+
368
+ Usage: datacontract lint [OPTIONS] [LOCATION]
369
+
370
+ Validate that the datacontract.yaml is correctly formatted.
371
+
372
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
373
+ │ location [LOCATION] The location (url or path) of the data contract yaml. │
374
+ │ [default: datacontract.yaml] │
375
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
376
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
377
+ │ --schema TEXT The location (url or path) of the Data Contract Specification │
378
+ │ JSON Schema │
379
+ │ [default: None] │
380
+ │ --output PATH Specify the file path where the test results should be written │
381
+ │ to (e.g., './test-results/TEST-datacontract.xml'). If no path is │
382
+ │ provided, the output will be printed to stdout. │
383
+ │ [default: None] │
384
+ │ --output-format [junit] The target format for the test results. [default: None] │
385
+ │ --help Show this message and exit. │
386
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
394
387
 
395
388
  ```
396
389
 
397
390
  ### test
398
391
  ```
399
-
400
- Usage: datacontract test [OPTIONS] [LOCATION]
401
-
402
- Run schema and quality tests on configured servers.
403
-
404
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
405
- │ location [LOCATION] The location (url or path) of the data contract │
406
- │ yaml. │
407
- │ [default: datacontract.yaml] │
408
- ╰──────────────────────────────────────────────────────────────────────────────╯
409
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
410
- │ --schema TEXT The location (url or │
411
- │ path) of the Data │
412
- │ Contract │
413
- │ Specification JSON │
414
- │ Schema │
415
- │ [default: None] │
416
- │ --server TEXT The server │
417
- │ configuration to run │
418
- │ the schema and │
419
- │ quality tests. Use │
420
- │ the key of the server │
421
- │ object in the data │
422
- │ contract yaml file to │
423
- │ refer to a server, │
424
- │ e.g., `production`, │
425
- │ or `all` for all │
426
- │ servers (default). │
427
- │ [default: all] │
428
- │ --publish TEXT The url to publish │
429
- │ the results after the │
430
- │ test │
431
- │ [default: None] │
432
- │ --output PATH Specify the file path │
433
- │ where the test │
434
- │ results should be │
435
- │ written to (e.g., │
436
- │ './test-results/TEST… │
437
- │ [default: None] │
438
- │ --output-format [junit] The target format for │
439
- │ the test results. │
440
- │ [default: None] │
441
- │ --logs --no-logs Print logs │
442
- │ [default: no-logs] │
443
- │ --ssl-verification --no-ssl-verificati… SSL verification when │
444
- │ publishing the data │
445
- │ contract. │
446
- │ [default: │
447
- │ ssl-verification] │
448
- │ --help Show this message and │
449
- │ exit. │
450
- ╰──────────────────────────────────────────────────────────────────────────────╯
392
+
393
+ Usage: datacontract test [OPTIONS] [LOCATION]
394
+
395
+ Run schema and quality tests on configured servers.
396
+
397
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
398
+ │ location [LOCATION] The location (url or path) of the data contract yaml. │
399
+ │ [default: datacontract.yaml] │
400
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
401
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
402
+ │ --schema TEXT The location (url or path) of the Data │
403
+ │ Contract Specification JSON Schema │
404
+ │ [default: None] │
405
+ │ --server TEXT The server configuration to run the │
406
+ │ schema and quality tests. Use the key of │
407
+ │ the server object in the data contract │
408
+ │ yaml file to refer to a server, e.g., │
409
+ │ `production`, or `all` for all servers │
410
+ │ (default). │
411
+ │ [default: all] │
412
+ │ --publish TEXT The url to publish the results after the │
413
+ │ test │
414
+ │ [default: None] │
415
+ │ --output PATH Specify the file path where the test │
416
+ │ results should be written to (e.g., │
417
+ │ './test-results/TEST-datacontract.xml'). │
418
+ │ [default: None] │
419
+ │ --output-format [junit] The target format for the test results. │
420
+ │ [default: None] │
421
+ │ --logs --no-logs Print logs [default: no-logs] │
422
+ │ --ssl-verification --no-ssl-verification SSL verification when publishing the │
423
+ │ data contract. │
424
+ │ [default: ssl-verification] │
425
+ │ --help Show this message and exit. │
426
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
451
427
 
452
428
  ```
453
429
 
@@ -930,64 +906,57 @@ models:
930
906
 
931
907
  ### export
932
908
  ```
933
-
934
- Usage: datacontract export [OPTIONS] [LOCATION]
935
-
936
- Convert data contract to a specific format. Saves to file specified by
937
- `output` option if present, otherwise prints to stdout.
938
-
939
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
940
- │ location [LOCATION] The location (url or path) of the data contract │
941
- │ yaml. │
942
- │ [default: datacontract.yaml] │
943
- ╰──────────────────────────────────────────────────────────────────────────────╯
944
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
945
- │ * --format [jsonschema|pydantic-model The export format. │
946
- │ |sodacl|dbt|dbt-sources|db [default: None] │
947
- │ t-staging-sql|odcs|rdf|avr [required] │
948
- │ o|protobuf|great-expectati │
949
- │ ons|terraform|avro-idl|sql │
950
- │ |sql-query|html|go|bigquer │
951
- │ y|dbml|spark|sqlalchemy|da │
952
- │ ta-caterer|dcs|markdown|ic │
953
- │ eberg|custom] │
954
- │ --output PATH Specify the file path where │
955
- │ the exported data will be │
956
- │ saved. If no path is │
957
- │ provided, the output will │
958
- │ be printed to stdout. │
959
- │ [default: None] │
960
- │ --server TEXT The server name to export. │
961
- │ [default: None] │
962
- │ --model TEXT Use the key of the model in │
963
- │ the data contract yaml file │
964
- │ to refer to a model, e.g., │
965
- │ `orders`, or `all` for all │
966
- │ models (default). │
967
- │ [default: all] │
968
- │ --schema TEXT The location (url or path) │
969
- │ of the Data Contract │
970
- │ Specification JSON Schema │
971
- │ [default: None] │
972
- │ --engine TEXT [engine] The engine used │
973
- │ for great expection run. │
974
- │ [default: None] │
975
- │ --template PATH [custom] The file path of │
976
- │ Jinja template. │
977
- │ [default: None] │
978
- │ --help Show this message and exit. │
979
- ╰──────────────────────────────────────────────────────────────────────────────╯
980
- ╭─ RDF Options ────────────────────────────────────────────────────────────────╮
981
- │ --rdf-base TEXT [rdf] The base URI used to generate the RDF graph. │
982
- │ [default: None] │
983
- ╰──────────────────────────────────────────────────────────────────────────────╯
984
- ╭─ SQL Options ────────────────────────────────────────────────────────────────╮
985
- │ --sql-server-type TEXT [sql] The server type to determine the sql │
986
- │ dialect. By default, it uses 'auto' to │
987
- │ automatically detect the sql dialect via the │
988
- │ specified servers in the data contract. │
989
- │ [default: auto] │
990
- ╰──────────────────────────────────────────────────────────────────────────────╯
909
+
910
+ Usage: datacontract export [OPTIONS] [LOCATION]
911
+
912
+ Convert data contract to a specific format. Saves to file specified by `output` option if present,
913
+ otherwise prints to stdout.
914
+
915
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
916
+ │ location [LOCATION] The location (url or path) of the data contract yaml. │
917
+ │ [default: datacontract.yaml] │
918
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
919
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
920
+ │ * --format [jsonschema|pydantic-model|sodacl|db The export format. [default: None] │
921
+ │ t|dbt-sources|dbt-staging-sql|odcs|r [required] │
922
+ │ df|avro|protobuf|great-expectations| │
923
+ │ terraform|avro-idl|sql|sql-query|mer │
924
+ │ maid|html|go|bigquery|dbml|spark|sql │
925
+ │ alchemy|data-caterer|dcs|markdown|ic │
926
+ │ eberg|custom] │
927
+ │ --output PATH Specify the file path where the │
928
+ │ exported data will be saved. If no │
929
+ │ path is provided, the output will be │
930
+ │ printed to stdout. │
931
+ │ [default: None] │
932
+ │ --server TEXT The server name to export. │
933
+ │ [default: None] │
934
+ │ --model TEXT Use the key of the model in the data │
935
+ │ contract yaml file to refer to a │
936
+ │ model, e.g., `orders`, or `all` for │
937
+ │ all models (default). │
938
+ │ [default: all] │
939
+ │ --schema TEXT The location (url or path) of the │
940
+ │ Data Contract Specification JSON │
941
+ │ Schema │
942
+ │ [default: None] │
943
+ │ --engine TEXT [engine] The engine used for great │
944
+ │ expection run. │
945
+ │ [default: None] │
946
+ │ --template PATH [custom] The file path of Jinja │
947
+ │ template. │
948
+ │ [default: None] │
949
+ │ --help Show this message and exit. │
950
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
951
+ ╭─ RDF Options ────────────────────────────────────────────────────────────────────────────────────╮
952
+ │ --rdf-base TEXT [rdf] The base URI used to generate the RDF graph. [default: None] │
953
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
954
+ ╭─ SQL Options ────────────────────────────────────────────────────────────────────────────────────╮
955
+ │ --sql-server-type TEXT [sql] The server type to determine the sql dialect. By default, │
956
+ │ it uses 'auto' to automatically detect the sql dialect via the │
957
+ │ specified servers in the data contract. │
958
+ │ [default: auto] │
959
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
991
960
 
992
961
  ```
993
962
 
@@ -1284,104 +1253,88 @@ FROM
1284
1253
 
1285
1254
  ### import
1286
1255
  ```
1287
-
1288
- Usage: datacontract import [OPTIONS]
1289
-
1290
- Create a data contract from the given source location. Saves to file specified
1291
- by `output` option if present, otherwise prints to stdout.
1292
-
1293
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1294
- │ * --format [sql|avro|dbt|dbml|gl The format of the │
1295
- │ ue|jsonschema|bigquer source file. │
1296
- │ y|odcs|unity|spark|ic [default: None] │
1297
- │ eberg|parquet|csv|pro [required] │
1298
- │ tobuf] │
1299
- │ --output PATH Specify the file path │
1300
- │ where the Data │
1301
- │ Contract will be │
1302
- │ saved. If no path is │
1303
- │ provided, the output │
1304
- │ will be printed to │
1305
- │ stdout. │
1306
- │ [default: None] │
1307
- │ --source TEXT The path to the file │
1308
- │ or Glue Database that │
1309
- │ should be imported. │
1310
- │ [default: None] │
1311
- │ --dialect TEXT The SQL dialect to │
1312
- │ use when importing │
1313
- │ SQL files, e.g., │
1314
- │ postgres, tsql, │
1315
- │ bigquery. │
1316
- │ [default: None] │
1317
- │ --glue-table TEXT List of table ids to │
1318
- │ import from the Glue │
1319
- │ Database (repeat for │
1320
- │ multiple table ids, │
1321
- │ leave empty for all │
1322
- │ tables in the │
1323
- │ dataset). │
1324
- │ [default: None] │
1325
- │ --bigquery-project TEXT The bigquery project │
1326
- │ id. │
1327
- │ [default: None] │
1328
- │ --bigquery-dataset TEXT The bigquery dataset │
1329
- │ id. │
1330
- │ [default: None] │
1331
- │ --bigquery-table TEXT List of table ids to │
1332
- │ import from the │
1333
- │ bigquery API (repeat │
1334
- │ for multiple table │
1335
- │ ids, leave empty for │
1336
- │ all tables in the │
1337
- │ dataset). │
1338
- │ [default: None] │
1339
- │ --unity-table-full-n… TEXT Full name of a table │
1340
- │ in the unity catalog │
1341
- │ [default: None] │
1342
- │ --dbt-model TEXT List of models names │
1343
- │ to import from the │
1344
- │ dbt manifest file │
1345
- │ (repeat for multiple │
1346
- │ models names, leave │
1347
- │ empty for all models │
1348
- │ in the dataset). │
1349
- │ [default: None] │
1350
- │ --dbml-schema TEXT List of schema names │
1351
- │ to import from the │
1352
- │ DBML file (repeat for │
1353
- │ multiple schema │
1354
- │ names, leave empty │
1355
- │ for all tables in the │
1356
- │ file). │
1357
- │ [default: None] │
1358
- │ --dbml-table TEXT List of table names │
1359
- │ to import from the │
1360
- │ DBML file (repeat for │
1361
- │ multiple table names, │
1362
- │ leave empty for all │
1363
- │ tables in the file). │
1364
- │ [default: None] │
1365
- │ --iceberg-table TEXT Table name to assign │
1366
- │ to the model created │
1367
- │ from the Iceberg │
1368
- │ schema. │
1369
- │ [default: None] │
1370
- │ --template TEXT The location (url or │
1371
- │ path) of the Data │
1372
- │ Contract │
1373
- │ Specification │
1374
- │ Template │
1375
- │ [default: None] │
1376
- │ --schema TEXT The location (url or │
1377
- │ path) of the Data │
1378
- │ Contract │
1379
- │ Specification JSON │
1380
- │ Schema │
1381
- │ [default: None] │
1382
- │ --help Show this message and │
1383
- │ exit. │
1384
- ╰──────────────────────────────────────────────────────────────────────────────╯
1256
+
1257
+ Usage: datacontract import [OPTIONS]
1258
+
1259
+ Create a data contract from the given source location. Saves to file specified by `output` option
1260
+ if present, otherwise prints to stdout.
1261
+
1262
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1263
+ │ * --format [sql|avro|dbt|dbml|glue|jsonsc The format of the source file. │
1264
+ │ hema|bigquery|odcs|unity|spark [default: None] │
1265
+ │ |iceberg|parquet|csv|protobuf| [required] │
1266
+ │ excel] │
1267
+ │ --output PATH Specify the file path where │
1268
+ │ the Data Contract will be │
1269
+ │ saved. If no path is provided, │
1270
+ │ the output will be printed to │
1271
+ │ stdout. │
1272
+ │ [default: None] │
1273
+ │ --source TEXT The path to the file that │
1274
+ │ should be imported. │
1275
+ │ [default: None] │
1276
+ │ --dialect TEXT The SQL dialect to use when │
1277
+ │ importing SQL files, e.g., │
1278
+ │ postgres, tsql, bigquery. │
1279
+ │ [default: None] │
1280
+ │ --glue-table TEXT List of table ids to import │
1281
+ │ from the Glue Database (repeat │
1282
+ │ for multiple table ids, leave │
1283
+ │ empty for all tables in the │
1284
+ │ dataset). │
1285
+ │ [default: None] │
1286
+ │ --bigquery-project TEXT The bigquery project id. │
1287
+ │ [default: None] │
1288
+ │ --bigquery-dataset TEXT The bigquery dataset id. │
1289
+ │ [default: None] │
1290
+ │ --bigquery-table TEXT List of table ids to import │
1291
+ │ from the bigquery API (repeat │
1292
+ │ for multiple table ids, leave │
1293
+ │ empty for all tables in the │
1294
+ │ dataset). │
1295
+ │ [default: None] │
1296
+ │ --unity-table-full-name TEXT Full name of a table in the │
1297
+ │ unity catalog │
1298
+ │ [default: None] │
1299
+ │ --dbt-model TEXT List of models names to import │
1300
+ │ from the dbt manifest file │
1301
+ │ (repeat for multiple models │
1302
+ │ names, leave empty for all │
1303
+ │ models in the dataset). │
1304
+ │ [default: None] │
1305
+ │ --dbml-schema TEXT List of schema names to import │
1306
+ │ from the DBML file (repeat for │
1307
+ │ multiple schema names, leave │
1308
+ │ empty for all tables in the │
1309
+ │ file). │
1310
+ │ [default: None] │
1311
+ │ --dbml-table TEXT List of table names to import │
1312
+ │ from the DBML file (repeat for │
1313
+ │ multiple table names, leave │
1314
+ │ empty for all tables in the │
1315
+ │ file). │
1316
+ │ [default: None] │
1317
+ │ --iceberg-table TEXT Table name to assign to the │
1318
+ │ model created from the Iceberg │
1319
+ │ schema. │
1320
+ │ [default: None] │
1321
+ │ --template TEXT The location (url or path) of │
1322
+ │ the Data Contract │
1323
+ │ Specification Template │
1324
+ │ [default: None] │
1325
+ │ --schema TEXT The location (url or path) of │
1326
+ │ the Data Contract │
1327
+ │ Specification JSON Schema │
1328
+ │ [default: None] │
1329
+ │ --owner TEXT The owner or team responsible │
1330
+ │ for managing the data │
1331
+ │ contract. │
1332
+ │ [default: None] │
1333
+ │ --id TEXT The identifier for the the │
1334
+ │ data contract. │
1335
+ │ [default: None] │
1336
+ │ --help Show this message and exit. │
1337
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1385
1338
 
1386
1339
  ```
1387
1340
 
@@ -1397,20 +1350,21 @@ Available import options:
1397
1350
 
1398
1351
  | Type | Description | Status |
1399
1352
  |--------------------|------------------------------------------------|--------|
1400
- | `sql` | Import from SQL DDL | ✅ |
1401
1353
  | `avro` | Import from AVRO schemas | ✅ |
1402
- | `glue` | Import from AWS Glue DataCatalog | ✅ |
1403
- | `jsonschema` | Import from JSON Schemas | ✅ |
1404
1354
  | `bigquery` | Import from BigQuery Schemas | ✅ |
1405
- | `unity` | Import from Databricks Unity Catalog | partial |
1355
+ | `csv` | Import from CSV File | ✅ |
1356
+ | `dbml` | Import from DBML models | ✅ |
1406
1357
  | `dbt` | Import from dbt models | ✅ |
1358
+ | `excel` | Import from ODCS Excel Template | ✅ |
1359
+ | `glue` | Import from AWS Glue DataCatalog | ✅ |
1360
+ | `iceberg` | Import from an Iceberg JSON Schema Definition | partial |
1361
+ | `jsonschema` | Import from JSON Schemas | ✅ |
1407
1362
  | `odcs` | Import from Open Data Contract Standard (ODCS) | ✅ |
1408
- | `spark` | Import from Spark StructTypes | ✅ |
1409
- | `dbml` | Import from DBML models | ✅ |
1410
- | `csv` | Import from CSV File | ✅ |
1363
+ | `parquet` | Import from Parquet File Metadata | ✅ |
1411
1364
  | `protobuf` | Import from Protobuf schemas | ✅ |
1412
- | `iceberg` | Import from an Iceberg JSON Schema Definition | partial |
1413
- | `parquet` | Import from Parquet File Metadta | ✅ |
1365
+ | `spark` | Import from Spark StructTypes | ✅ |
1366
+ | `sql` | Import from SQL DDL | ✅ |
1367
+ | `unity` | Import from Databricks Unity Catalog | partial |
1414
1368
  | Missing something? | Please create an issue on GitHub | TBD |
1415
1369
 
1416
1370
 
@@ -1460,8 +1414,9 @@ datacontract import --format unity --source my_unity_table.json
1460
1414
 
1461
1415
  ```bash
1462
1416
  # Example import single table from Unity Catalog via HTTP endpoint
1463
- export DATABRICKS_IMPORT_INSTANCE="https://xyz.cloud.databricks.com"
1464
- export DATABRICKS_IMPORT_ACCESS_TOKEN=<token>
1417
+ export DATACONTRACT_DATABRICKS_SERVER_HOSTNAME="https://xyz.cloud.databricks.com"
1418
+ export DATACONTRACT_DATABRICKS_HTTP_PATH="/sql/1.0/warehouses/b053a331fa014fb4"
1419
+ export DATACONTRACT_DATABRICKS_TOKEN=<token>
1465
1420
  datacontract import --format unity --unity-table-full-name <table_full_name>
1466
1421
  ```
1467
1422
 
@@ -1482,6 +1437,17 @@ datacontract import --format dbt --source <manifest_path> --dbt-model <model_nam
1482
1437
  datacontract import --format dbt --source <manifest_path>
1483
1438
  ```
1484
1439
 
1440
+ ### Excel
1441
+
1442
+ Importing from [ODCS Excel Template](https://github.com/datacontract/open-data-contract-standard-excel-template).
1443
+
1444
+ Examples:
1445
+
1446
+ ```bash
1447
+ # Example import from ODCS Excel Template
1448
+ datacontract import --format excel --source odcs.xlsx
1449
+ ```
1450
+
1485
1451
  #### Glue
1486
1452
 
1487
1453
  Importing from Glue reads the necessary Data directly off of the AWS API.
@@ -1573,91 +1539,83 @@ datacontract import --format protobuf --source "test.proto"
1573
1539
 
1574
1540
  ### breaking
1575
1541
  ```
1576
-
1577
- Usage: datacontract breaking [OPTIONS] LOCATION_OLD LOCATION_NEW
1578
-
1579
- Identifies breaking changes between data contracts. Prints to stdout.
1580
-
1581
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
1582
- │ * location_old TEXT The location (url or path) of the old data │
1583
- │ contract yaml. │
1584
- │ [default: None] │
1585
- │ [required] │
1586
- │ * location_new TEXT The location (url or path) of the new data │
1587
- │ contract yaml. │
1588
- │ [default: None] │
1589
- │ [required] │
1590
- ╰──────────────────────────────────────────────────────────────────────────────╯
1591
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1592
- │ --help Show this message and exit. │
1593
- ╰──────────────────────────────────────────────────────────────────────────────╯
1542
+
1543
+ Usage: datacontract breaking [OPTIONS] LOCATION_OLD LOCATION_NEW
1544
+
1545
+ Identifies breaking changes between data contracts. Prints to stdout.
1546
+
1547
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
1548
+ │ * location_old TEXT The location (url or path) of the old data contract yaml. │
1549
+ │ [default: None] │
1550
+ │ [required] │
1551
+ │ * location_new TEXT The location (url or path) of the new data contract yaml. │
1552
+ │ [default: None] │
1553
+ │ [required] │
1554
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1555
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1556
+ │ --help Show this message and exit. │
1557
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1594
1558
 
1595
1559
  ```
1596
1560
 
1597
1561
  ### changelog
1598
1562
  ```
1599
-
1600
- Usage: datacontract changelog [OPTIONS] LOCATION_OLD LOCATION_NEW
1601
-
1602
- Generate a changelog between data contracts. Prints to stdout.
1603
-
1604
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
1605
- │ * location_old TEXT The location (url or path) of the old data │
1606
- │ contract yaml. │
1607
- │ [default: None] │
1608
- │ [required] │
1609
- │ * location_new TEXT The location (url or path) of the new data │
1610
- │ contract yaml. │
1611
- │ [default: None] │
1612
- │ [required] │
1613
- ╰──────────────────────────────────────────────────────────────────────────────╯
1614
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1615
- │ --help Show this message and exit. │
1616
- ╰──────────────────────────────────────────────────────────────────────────────╯
1563
+
1564
+ Usage: datacontract changelog [OPTIONS] LOCATION_OLD LOCATION_NEW
1565
+
1566
+ Generate a changelog between data contracts. Prints to stdout.
1567
+
1568
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
1569
+ │ * location_old TEXT The location (url or path) of the old data contract yaml. │
1570
+ │ [default: None] │
1571
+ │ [required] │
1572
+ │ * location_new TEXT The location (url or path) of the new data contract yaml. │
1573
+ │ [default: None] │
1574
+ │ [required] │
1575
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1576
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1577
+ │ --help Show this message and exit. │
1578
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1617
1579
 
1618
1580
  ```
1619
1581
 
1620
1582
  ### diff
1621
1583
  ```
1622
-
1623
- Usage: datacontract diff [OPTIONS] LOCATION_OLD LOCATION_NEW
1624
-
1625
- PLACEHOLDER. Currently works as 'changelog' does.
1626
-
1627
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
1628
- │ * location_old TEXT The location (url or path) of the old data │
1629
- │ contract yaml. │
1630
- │ [default: None] │
1631
- │ [required] │
1632
- │ * location_new TEXT The location (url or path) of the new data │
1633
- │ contract yaml. │
1634
- │ [default: None] │
1635
- │ [required] │
1636
- ╰──────────────────────────────────────────────────────────────────────────────╯
1637
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1638
- │ --help Show this message and exit. │
1639
- ╰──────────────────────────────────────────────────────────────────────────────╯
1584
+
1585
+ Usage: datacontract diff [OPTIONS] LOCATION_OLD LOCATION_NEW
1586
+
1587
+ PLACEHOLDER. Currently works as 'changelog' does.
1588
+
1589
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
1590
+ │ * location_old TEXT The location (url or path) of the old data contract yaml. │
1591
+ │ [default: None] │
1592
+ │ [required] │
1593
+ │ * location_new TEXT The location (url or path) of the new data contract yaml. │
1594
+ │ [default: None] │
1595
+ │ [required] │
1596
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1597
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1598
+ │ --help Show this message and exit. │
1599
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1640
1600
 
1641
1601
  ```
1642
1602
 
1643
1603
  ### catalog
1644
1604
  ```
1645
-
1646
- Usage: datacontract catalog [OPTIONS]
1647
-
1648
- Create a html catalog of data contracts.
1649
-
1650
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1651
- │ --files TEXT Glob pattern for the data contract files to include in │
1652
- │ the catalog. Applies recursively to any subfolders. │
1653
- │ [default: *.yaml] │
1654
- │ --output TEXT Output directory for the catalog html files. │
1655
- │ [default: catalog/] │
1656
- │ --schema TEXT The location (url or path) of the Data Contract │
1657
- │ Specification JSON Schema │
1658
- │ [default: None] │
1659
- │ --help Show this message and exit. │
1660
- ╰──────────────────────────────────────────────────────────────────────────────╯
1605
+
1606
+ Usage: datacontract catalog [OPTIONS]
1607
+
1608
+ Create a html catalog of data contracts.
1609
+
1610
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1611
+ │ --files TEXT Glob pattern for the data contract files to include in the catalog. │
1612
+ │ Applies recursively to any subfolders. │
1613
+ │ [default: *.yaml] │
1614
+ │ --output TEXT Output directory for the catalog html files. [default: catalog/] │
1615
+ │ --schema TEXT The location (url or path) of the Data Contract Specification JSON Schema │
1616
+ │ [default: None] │
1617
+ │ --help Show this message and exit. │
1618
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1661
1619
 
1662
1620
  ```
1663
1621
 
@@ -1673,56 +1631,48 @@ datacontract catalog --files "*.odcs.yaml"
1673
1631
 
1674
1632
  ### publish
1675
1633
  ```
1676
-
1677
- Usage: datacontract publish [OPTIONS] [LOCATION]
1678
-
1679
- Publish the data contract to the Data Mesh Manager.
1680
-
1681
- ╭─ Arguments ──────────────────────────────────────────────────────────────────╮
1682
- │ location [LOCATION] The location (url or path) of the data contract │
1683
- │ yaml. │
1684
- │ [default: datacontract.yaml] │
1685
- ╰──────────────────────────────────────────────────────────────────────────────╯
1686
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1687
- │ --schema TEXT The location (url or │
1688
- │ path) of the Data │
1689
- │ Contract Specification │
1690
- │ JSON Schema │
1691
- │ [default: None] │
1692
- │ --ssl-verification --no-ssl-verification SSL verification when │
1693
- │ publishing the data │
1694
- │ contract. │
1695
- │ [default: │
1696
- │ ssl-verification] │
1697
- │ --help Show this message and │
1698
- │ exit. │
1699
- ╰──────────────────────────────────────────────────────────────────────────────╯
1634
+
1635
+ Usage: datacontract publish [OPTIONS] [LOCATION]
1636
+
1637
+ Publish the data contract to the Data Mesh Manager.
1638
+
1639
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────╮
1640
+ │ location [LOCATION] The location (url or path) of the data contract yaml. │
1641
+ │ [default: datacontract.yaml] │
1642
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1643
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1644
+ │ --schema TEXT The location (url or path) of the Data │
1645
+ │ Contract Specification JSON Schema │
1646
+ │ [default: None] │
1647
+ │ --ssl-verification --no-ssl-verification SSL verification when publishing the data │
1648
+ │ contract. │
1649
+ │ [default: ssl-verification] │
1650
+ │ --help Show this message and exit. │
1651
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1700
1652
 
1701
1653
  ```
1702
1654
 
1703
1655
  ### api
1704
1656
  ```
1705
-
1706
- Usage: datacontract api [OPTIONS]
1707
-
1708
- Start the datacontract CLI as server application with REST API.
1709
- The OpenAPI documentation as Swagger UI is available on http://localhost:4242.
1710
- You can execute the commands directly from the Swagger UI.
1711
- To protect the API, you can set the environment variable
1712
- DATACONTRACT_CLI_API_KEY to a secret API key. To authenticate, requests must
1713
- include the header 'x-api-key' with the correct API key. This is highly
1714
- recommended, as data contract tests may be subject to SQL injections or leak
1715
- sensitive information.
1716
- To connect to servers (such as a Snowflake data source), set the credentials
1717
- as environment variables as documented in https://cli.datacontract.com/#test
1718
-
1719
- ╭─ Options ────────────────────────────────────────────────────────────────────╮
1720
- │ --port INTEGER Bind socket to this port. [default: 4242] │
1721
- │ --host TEXT Bind socket to this host. Hint: For running in │
1722
- │ docker, set it to 0.0.0.0 │
1723
- │ [default: 127.0.0.1] │
1724
- │ --help Show this message and exit. │
1725
- ╰──────────────────────────────────────────────────────────────────────────────╯
1657
+
1658
+ Usage: datacontract api [OPTIONS]
1659
+
1660
+ Start the datacontract CLI as server application with REST API.
1661
+ The OpenAPI documentation as Swagger UI is available on http://localhost:4242. You can execute the
1662
+ commands directly from the Swagger UI.
1663
+ To protect the API, you can set the environment variable DATACONTRACT_CLI_API_KEY to a secret API
1664
+ key. To authenticate, requests must include the header 'x-api-key' with the correct API key. This
1665
+ is highly recommended, as data contract tests may be subject to SQL injections or leak sensitive
1666
+ information.
1667
+ To connect to servers (such as a Snowflake data source), set the credentials as environment
1668
+ variables as documented in https://cli.datacontract.com/#test
1669
+
1670
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
1671
+ │ --port INTEGER Bind socket to this port. [default: 4242] │
1672
+ │ --host TEXT Bind socket to this host. Hint: For running in docker, set it to 0.0.0.0 │
1673
+ │ [default: 127.0.0.1] │
1674
+ │ --help Show this message and exit. │
1675
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
1726
1676
 
1727
1677
  ```
1728
1678