datacontract-cli 0.10.12__tar.gz → 0.10.14__tar.gz

{datacontract_cli-0.10.12/datacontract_cli.egg-info → datacontract_cli-0.10.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: datacontract-cli
-Version: 0.10.12
+Version: 0.10.14
 Summary: The datacontract CLI is an open source command-line tool for working with Data Contracts. It uses data contract YAML files to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
 Author-email: Jochen Christ <jochen.christ@innoq.com>, Stefan Negele <stefan.negele@innoq.com>, Simon Harrer <simon.harrer@innoq.com>
 Project-URL: Homepage, https://cli.datacontract.com
@@ -12,24 +12,24 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: typer<0.13,>=0.12
-Requires-Dist: pydantic<2.9.0,>=2.8.2
+Requires-Dist: pydantic<2.10.0,>=2.8.2
 Requires-Dist: pyyaml~=6.0.1
 Requires-Dist: requests<2.33,>=2.31
-Requires-Dist: fastapi==0.112.0
-Requires-Dist: uvicorn==0.30.5
+Requires-Dist: fastapi==0.115.2
+Requires-Dist: uvicorn==0.30.6
 Requires-Dist: fastjsonschema<2.21.0,>=2.19.1
 Requires-Dist: fastparquet==2024.5.0
-Requires-Dist: python-multipart==0.0.9
-Requires-Dist: rich~=13.7.0
-Requires-Dist: simple-ddl-parser==1.6.0
-Requires-Dist: duckdb==1.0.0
+Requires-Dist: python-multipart==0.0.12
+Requires-Dist: rich<13.10,>=13.7
+Requires-Dist: simple-ddl-parser==1.7.1
+Requires-Dist: duckdb==1.1.2
 Requires-Dist: soda-core-duckdb<3.4.0,>=3.3.1
 Requires-Dist: setuptools>=60
 Requires-Dist: python-dotenv~=1.0.0
 Requires-Dist: rdflib==7.0.0
 Requires-Dist: opentelemetry-exporter-otlp-proto-grpc~=1.16
 Requires-Dist: opentelemetry-exporter-otlp-proto-http~=1.16
-Requires-Dist: boto3<1.35.6,>=1.34.41
+Requires-Dist: boto3<1.35.45,>=1.34.41
 Requires-Dist: jinja_partials>=0.2.1
 Provides-Extra: avro
 Requires-Dist: avro==1.12.0; extra == "avro"
@@ -37,15 +37,18 @@ Provides-Extra: bigquery
 Requires-Dist: soda-core-bigquery<3.4.0,>=3.3.1; extra == "bigquery"
 Provides-Extra: databricks
 Requires-Dist: soda-core-spark-df<3.4.0,>=3.3.1; extra == "databricks"
-Requires-Dist: databricks-sql-connector<3.4.0,>=3.1.2; extra == "databricks"
+Requires-Dist: databricks-sql-connector<3.6.0,>=3.1.2; extra == "databricks"
+Requires-Dist: databricks-sdk<0.36.0,>=0.32.0; extra == "databricks"
 Requires-Dist: soda-core-spark[databricks]<3.4.0,>=3.3.1; extra == "databricks"
+Provides-Extra: iceberg
+Requires-Dist: pyiceberg==0.7.1; extra == "iceberg"
 Provides-Extra: kafka
 Requires-Dist: datacontract-cli[avro]; extra == "kafka"
 Requires-Dist: soda-core-spark-df<3.4.0,>=3.3.1; extra == "kafka"
 Provides-Extra: postgres
 Requires-Dist: soda-core-postgres<3.4.0,>=3.3.1; extra == "postgres"
 Provides-Extra: s3
-Requires-Dist: s3fs==2024.6.1; extra == "s3"
+Requires-Dist: s3fs==2024.9.0; extra == "s3"
 Provides-Extra: snowflake
 Requires-Dist: snowflake-connector-python[pandas]<3.13,>=3.6; extra == "snowflake"
 Requires-Dist: soda-core-snowflake<3.4.0,>=3.3.1; extra == "snowflake"
@@ -58,12 +61,12 @@ Requires-Dist: dbt-core>=1.8.0; extra == "dbt"
 Provides-Extra: dbml
 Requires-Dist: pydbml>=1.1.1; extra == "dbml"
 Provides-Extra: all
-Requires-Dist: datacontract-cli[bigquery,databricks,dbml,dbt,kafka,postgres,s3,snowflake,sqlserver,trino]; extra == "all"
+Requires-Dist: datacontract-cli[bigquery,databricks,dbml,dbt,iceberg,kafka,postgres,s3,snowflake,sqlserver,trino]; extra == "all"
 Provides-Extra: dev
 Requires-Dist: datacontract-cli[all]; extra == "dev"
 Requires-Dist: httpx==0.27.2; extra == "dev"
 Requires-Dist: kafka-python; extra == "dev"
-Requires-Dist: moto==5.0.13; extra == "dev"
+Requires-Dist: moto==5.0.18; extra == "dev"
 Requires-Dist: pandas>=2.1.0; extra == "dev"
 Requires-Dist: pre-commit<3.9.0,>=3.7.1; extra == "dev"
 Requires-Dist: pyarrow>=12.0.0; extra == "dev"
@@ -71,8 +74,8 @@ Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-xdist; extra == "dev"
 Requires-Dist: pymssql==2.3.1; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
-Requires-Dist: testcontainers[kafka,minio,mssql,postgres]==4.8.1; extra == "dev"
-Requires-Dist: trino==0.329.0; extra == "dev"
+Requires-Dist: testcontainers[kafka,minio,mssql,postgres]==4.8.2; extra == "dev"
+Requires-Dist: trino==0.330.0; extra == "dev"
 
 # Data Contract CLI
 
@@ -84,8 +87,8 @@ Requires-Dist: trino==0.329.0; extra == "dev"
   <a href="https://datacontract.com/slack" rel="nofollow"><img src="https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&amp;style=social" alt="Slack Status" data-canonical-src="https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&amp;style=social" style="max-width: 100%;"></a>
 </p>
 
-The `datacontract` CLI is an open source command-line tool for working with [Data Contracts](https://datacontract.com/).
-It uses data contract YAML files to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
+The `datacontract` CLI is an open-source command-line tool for working with data contracts.
+It uses data contract YAML files as [Data Contract Specification](https://datacontract.com/) or [ODCS](https://bitol-io.github.io/open-data-contract-standard/latest/) to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
 
 ![Main features of the Data Contract CLI](datacontractcli.png)
 
@@ -97,15 +100,15 @@ Let's look at this data contract:
 
 We have a _servers_ section with endpoint details to the S3 bucket, _models_ for the structure of the data, _servicelevels_ and _quality_ attributes that describe the expected freshness and number of rows.
 
-This data contract contains all information to connect to S3 and check that the actual data meets the defined schema and quality requirements. We can use this information to test if the actual data set in S3 is compliant to the data contract.
+This data contract contains all information to connect to S3 and check that the actual data meets the defined schema and quality requirements. We can use this information to test if the actual data product in S3 is compliant to the data contract.
 
-Let's use [pip](https://pip.pypa.io/en/stable/getting-started/) to install the CLI (or use the [Docker image](#docker), if you prefer).
+Let's use [pip](https://pip.pypa.io/en/stable/getting-started/) to install the CLI (or use the [Docker image](#docker)),
 ```bash
 $ python3 -m pip install datacontract-cli[all]
 ```
 
 
-We run the tests:
+now, let's run the tests:
 
 ```bash
 $ datacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml
@@ -143,7 +146,7 @@ Testing https://datacontract.com/examples/orders-latest/datacontract.yaml
 
 Voilà, the CLI tested that the _datacontract.yaml_ itself is valid, all records comply with the schema, and all quality attributes are met.
 
-We can also use the datacontract.yaml to export in many [formats](#format), e.g., to SQL:
+We can also use the datacontract.yaml to export in many [formats](#format), e.g., to generate a SQL DDL:
 
 ```bash
 $ datacontract export --format sql https://datacontract.com/examples/orders-latest/datacontract.yaml
@@ -190,7 +193,7 @@ $ datacontract test datacontract.yaml
 # execute schema and quality checks on the examples within the contract
 $ datacontract test --examples datacontract.yaml
 
-# export data contract as html (other formats: avro, dbt, dbt-sources, dbt-staging-sql, jsonschema, odcs, rdf, sql, sodacl, terraform, ...)
+# export data contract as html (other formats: avro, dbt, dbt-sources, dbt-staging-sql, jsonschema, odcs_v2, odcs_v3, rdf, sql, sodacl, terraform, ...)
 $ datacontract export --format html datacontract.yaml > datacontract.html
 
 # import avro (other formats: sql, glue, bigquery...)
@@ -223,8 +226,7 @@ if not run.has_passed():
 Choose the most appropriate installation method for your needs:
 
 ### pip
-Python 3.11 recommended.
-Python 3.12 available as pre-release release candidate for 0.9.3
+Python 3.10, 3.11, and 3.12 are supported. We recommend to use Python 3.11.
 
 ```bash
 python3 -m pip install datacontract-cli[all]
@@ -238,17 +240,22 @@ pipx install datacontract-cli[all]
 
 ### Docker
 
+You can also use our Docker image to run the CLI tool. It is also convenient for CI/CD pipelines.
+
 ```bash
 docker pull datacontract/cli
 docker run --rm -v ${PWD}:/home/datacontract datacontract/cli
 ```
 
-Or via an alias that automatically uses the latest version:
+You can create an alias for the Docker command to make it easier to use:
 
 ```bash
 alias datacontract='docker run --rm -v "${PWD}:/home/datacontract" datacontract/cli:latest'
 ```
 
+_Note:_ The output of Docker command line messages is limited to 80 columns and may include line breaks. Don't pipe docker output to files if you want to export code. Use the `--output` option instead.
+
+
 
 ## Optional Dependencies
 
@@ -741,10 +748,11 @@ servers:
 
 #### Environment Variables
 
-| Environment Variable               | Example | Description                 |
-|------------------------------------|---------|-----------------------------|
-| `DATACONTRACT_KAFKA_SASL_USERNAME` | `xxx`   | The SASL username (key).    |
-| `DATACONTRACT_KAFKA_SASL_PASSWORD` | `xxx`   | The SASL password (secret). |
+| Environment Variable                | Example | Description                                                                      |
+|-------------------------------------|---------|----------------------------------------------------------------------------------|
+| `DATACONTRACT_KAFKA_SASL_USERNAME`  | `xxx`   | The SASL username (key).                                                         |
+| `DATACONTRACT_KAFKA_SASL_PASSWORD`  | `xxx`   | The SASL password (secret).                                                      |
+| `DATACONTRACT_KAFKA_SASL_MECHANISM` | `PLAIN` | Default `PLAIN`. Other supported mechanisms: `SCRAM-SHA-256` and `SCRAM-SHA-512` |
 
 
 ### Postgres
@@ -799,6 +807,10 @@ models:
     fields:
       my_column_1: # corresponds to a column
         type: varchar
+      my_column_2: # corresponds to a column with custom trino type
+        type: object
+        config:
+          trinoType: row(en_us varchar, pt_br varchar)
 ```
 
 #### Environment Variables
@@ -825,7 +837,7 @@ models:
 │ *  --format        [jsonschema|pydantic-model|sodacl|dbt|dbt-sources|db  The export format. [default: None] [required]         │
 │                    t-staging-sql|odcs|rdf|avro|protobuf|great-expectati                                                        │
 │                    ons|terraform|avro-idl|sql|sql-query|html|go|bigquer                                                        │
-│                    y|dbml|spark|sqlalchemy]                                                                                    │
+│                    y|dbml|spark|sqlalchemy|data-caterer|dcs]                                                                       │
 │    --output        PATH                                                  Specify the file path where the exported data will be │
 │                                                                          saved. If no path is provided, the output will be     │
 │                                                                          printed to stdout.                                    │
@@ -857,26 +869,30 @@ Available export options:
 
 | Type                 | Description                                             | Status |
 |----------------------|---------------------------------------------------------|--------|
-| `html`               | Export to HTML                                          | ✅     |
-| `jsonschema`         | Export to JSON Schema                                   | ✅     |
-| `odcs`               | Export to Open Data Contract Standard (ODCS)            | ✅     |
-| `sodacl`             | Export to SodaCL quality checks in YAML format          | ✅     |
-| `dbt`                | Export to dbt models in YAML format                     | ✅     |
-| `dbt-sources`        | Export to dbt sources in YAML format                    | ✅     |
-| `dbt-staging-sql`    | Export to dbt staging SQL models                        | ✅     |
-| `rdf`                | Export data contract to RDF representation in N3 format | ✅     |
-| `avro`               | Export to AVRO models                                   | ✅     |
-| `protobuf`           | Export to Protobuf                                      | ✅     |
-| `terraform`          | Export to terraform resources                           | ✅     |
-| `sql`                | Export to SQL DDL                                       | ✅     |
-| `sql-query`          | Export to SQL Query                                     | ✅     |
-| `great-expectations` | Export to Great Expectations Suites in JSON Format      | ✅     |
-| `bigquery`           | Export to BigQuery Schemas                              | ✅     |
-| `go`                 | Export to Go types                                      | ✅     |
-| `pydantic-model`     | Export to pydantic models                               | ✅     |
-| `DBML`               | Export to a DBML Diagram description                    | ✅     |
-| `spark`              | Export to a Spark StructType                            | ✅     |
-| `sqlalchemy`         | Export to SQLAlchemy Models                             | ✅     |
+| `html`               | Export to HTML                                          | ✅      |
+| `jsonschema`         | Export to JSON Schema                                   | ✅      |
+| `odcs_v2`            | Export to Open Data Contract Standard (ODCS) V2         | ✅      |
+| `odcs_v3`            | Export to Open Data Contract Standard (ODCS) V3         | ✅      |
+| `odcs`               | Export to Open Data Contract Standard (ODCS) V3         | ✅      |
+| `sodacl`             | Export to SodaCL quality checks in YAML format          | ✅      |
+| `dbt`                | Export to dbt models in YAML format                     | ✅      |
+| `dbt-sources`        | Export to dbt sources in YAML format                    | ✅      |
+| `dbt-staging-sql`    | Export to dbt staging SQL models                        | ✅      |
+| `rdf`                | Export data contract to RDF representation in N3 format | ✅      |
+| `avro`               | Export to AVRO models                                   | ✅      |
+| `protobuf`           | Export to Protobuf                                      | ✅      |
+| `terraform`          | Export to terraform resources                           | ✅      |
+| `sql`                | Export to SQL DDL                                       | ✅      |
+| `sql-query`          | Export to SQL Query                                     | ✅      |
+| `great-expectations` | Export to Great Expectations Suites in JSON Format      | ✅      |
+| `bigquery`           | Export to BigQuery Schemas                              | ✅      |
+| `go`                 | Export to Go types                                      | ✅      |
+| `pydantic-model`     | Export to pydantic models                               | ✅      |
+| `DBML`               | Export to a DBML Diagram description                    | ✅      |
+| `spark`              | Export to a Spark StructType                            | ✅      |
+| `sqlalchemy`         | Export to SQLAlchemy Models                             | ✅      |
+| `data-caterer`       | Export to Data Caterer in YAML format                   | ✅      |
+| `dcs`                | Export to Data Contract Specification in YAML format    | ✅      |
 | Missing something?   | Please create an issue on GitHub                        | TBD    |
 
 #### Great Expectations
@@ -940,6 +956,20 @@ To specify custom Avro properties in your data contract, you can define them wit
 
 >NOTE: At this moment, we just support [logicalType](https://avro.apache.org/docs/1.11.0/spec.html#Logical+Types) and [default](https://avro.apache.org/docs/1.11.0/spec.htm)
 
+#### Data Caterer
+
+The export function converts the data contract to a data generation task in YAML format that can be 
+ingested by [Data Caterer](https://github.com/data-catering/data-caterer). This gives you the 
+ability to generate production-like data in any environment based off your data contract.
+
+```shell
+datacontract export datacontract.yaml --format data-caterer --model orders
+```
+
+You can further customise the way data is generated via adding 
+[additional metadata in the YAML](https://data.catering/setup/generator/data-generator/) 
+to suit your needs.
+
 #### Example Configuration
 
 ```yaml
@@ -980,7 +1010,7 @@ models:
                                                                                                                                
 ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ *  --format                       [sql|avro|dbt|glue|jsonschema|bigquery|odcs  The format of the source file.               │
-│                                   |unity|spark]                                [default: None]                              │
+│                                   |unity|spark|iceberg]                        [default: None]                              │
 │                                                                                [required]                                   │
 │    --source                       TEXT                                         The path to the file or Glue Database that   │
 │                                                                                should be imported.                          │
@@ -1010,6 +1040,8 @@ models:
 │                                                                                file (repeat for multiple table names, leave │
 │                                                                                empty for all tables in the file).           │
 │                                                                                [default: None]                              │
+│    --iceberg-table                TEXT                                         Table name to assign to the model created    │
+│                                                                                from the Iceberg schema. [default: None]     │
 │    --help                                                                      Show this message and exit.                  │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
@@ -1035,12 +1067,25 @@ Available import options:
 | `spark`            | Import from Spark StructTypes                  | ✅      |
 | `dbml`             | Import from DBML models                        | ✅      |
 | `protobuf`         | Import from Protobuf schemas                   | TBD    |
+| `iceberg`          | Import from an Iceberg JSON Schema Definition  | partial |
 | Missing something? | Please create an issue on GitHub               | TBD    |
 
 
+#### ODCS
+
+Import from Open Data Contract Standard (ODCS) v2 or v3.
+The importer automatically detects the ODCS version and imports the data contract.
+
+Examples:
+
+```bash
+# Example import from ODCS
+datacontract import --format odcs --source my_data_contract.odcs.yaml
+```
+
 #### BigQuery
 
-Bigquery data can either be imported off of JSON Files generated from the table descriptions or directly from the Bigquery API. In case you want to use JSON Files, specify the `source` parameter with a path to the JSON File.
+BigQuery data can either be imported off of JSON Files generated from the table descriptions or directly from the Bigquery API. In case you want to use JSON Files, specify the `source` parameter with a path to the JSON File.
 
 To import from the Bigquery API, you have to _omit_ `source` and instead need to provide `bigquery-project` and `bigquery-dataset`. Additionally you may specify `bigquery-table` to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the dataset will be imported.
 
@@ -1152,6 +1197,15 @@ datacontract import --format dbml --source <file_path> --dbml-table <table_name_
 datacontract import --format dbml --source <file_path> --dbml-table <table_name_1> --dbml-schema <schema_1>
 ```
 
+#### Iceberg
+
+Importing from an [Iceberg Table Json Schema Definition](https://iceberg.apache.org/spec/#appendix-c-json-serialization). Specify location of json files using the `source` parameter.
+
+Examples:
+
+```bash
+datacontract import --format iceberg --source ./tests/fixtures/iceberg/simple_schema.json --iceberg-table test-table
+```
 
 ### breaking
 

{datacontract_cli-0.10.12 → datacontract_cli-0.10.14}/README.md

@@ -8,8 +8,8 @@
   <a href="https://datacontract.com/slack" rel="nofollow"><img src="https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&amp;style=social" alt="Slack Status" data-canonical-src="https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&amp;style=social" style="max-width: 100%;"></a>
 </p>
 
-The `datacontract` CLI is an open source command-line tool for working with [Data Contracts](https://datacontract.com/).
-It uses data contract YAML files to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
+The `datacontract` CLI is an open-source command-line tool for working with data contracts.
+It uses data contract YAML files as [Data Contract Specification](https://datacontract.com/) or [ODCS](https://bitol-io.github.io/open-data-contract-standard/latest/) to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
 
 ![Main features of the Data Contract CLI](datacontractcli.png)
 
@@ -21,15 +21,15 @@ Let's look at this data contract:
 
 We have a _servers_ section with endpoint details to the S3 bucket, _models_ for the structure of the data, _servicelevels_ and _quality_ attributes that describe the expected freshness and number of rows.
 
-This data contract contains all information to connect to S3 and check that the actual data meets the defined schema and quality requirements. We can use this information to test if the actual data set in S3 is compliant to the data contract.
+This data contract contains all information to connect to S3 and check that the actual data meets the defined schema and quality requirements. We can use this information to test if the actual data product in S3 is compliant to the data contract.
 
-Let's use [pip](https://pip.pypa.io/en/stable/getting-started/) to install the CLI (or use the [Docker image](#docker), if you prefer).
+Let's use [pip](https://pip.pypa.io/en/stable/getting-started/) to install the CLI (or use the [Docker image](#docker)),
 ```bash
 $ python3 -m pip install datacontract-cli[all]
 ```
 
 
-We run the tests:
+now, let's run the tests:
 
 ```bash
 $ datacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml
@@ -67,7 +67,7 @@ Testing https://datacontract.com/examples/orders-latest/datacontract.yaml
 
 Voilà, the CLI tested that the _datacontract.yaml_ itself is valid, all records comply with the schema, and all quality attributes are met.
 
-We can also use the datacontract.yaml to export in many [formats](#format), e.g., to SQL:
+We can also use the datacontract.yaml to export in many [formats](#format), e.g., to generate a SQL DDL:
 
 ```bash
 $ datacontract export --format sql https://datacontract.com/examples/orders-latest/datacontract.yaml
@@ -114,7 +114,7 @@ $ datacontract test datacontract.yaml
 # execute schema and quality checks on the examples within the contract
 $ datacontract test --examples datacontract.yaml
 
-# export data contract as html (other formats: avro, dbt, dbt-sources, dbt-staging-sql, jsonschema, odcs, rdf, sql, sodacl, terraform, ...)
+# export data contract as html (other formats: avro, dbt, dbt-sources, dbt-staging-sql, jsonschema, odcs_v2, odcs_v3, rdf, sql, sodacl, terraform, ...)
 $ datacontract export --format html datacontract.yaml > datacontract.html
 
 # import avro (other formats: sql, glue, bigquery...)
@@ -147,8 +147,7 @@ if not run.has_passed():
 Choose the most appropriate installation method for your needs:
 
 ### pip
-Python 3.11 recommended.
-Python 3.12 available as pre-release release candidate for 0.9.3
+Python 3.10, 3.11, and 3.12 are supported. We recommend to use Python 3.11.
 
 ```bash
 python3 -m pip install datacontract-cli[all]
@@ -162,17 +161,22 @@ pipx install datacontract-cli[all]
 
 ### Docker
 
+You can also use our Docker image to run the CLI tool. It is also convenient for CI/CD pipelines.
+
 ```bash
 docker pull datacontract/cli
 docker run --rm -v ${PWD}:/home/datacontract datacontract/cli
 ```
 
-Or via an alias that automatically uses the latest version:
+You can create an alias for the Docker command to make it easier to use:
 
 ```bash
 alias datacontract='docker run --rm -v "${PWD}:/home/datacontract" datacontract/cli:latest'
 ```
 
+_Note:_ The output of Docker command line messages is limited to 80 columns and may include line breaks. Don't pipe docker output to files if you want to export code. Use the `--output` option instead.
+
+
 
 ## Optional Dependencies
 
@@ -665,10 +669,11 @@ servers:
 
 #### Environment Variables
 
-| Environment Variable               | Example | Description                 |
-|------------------------------------|---------|-----------------------------|
-| `DATACONTRACT_KAFKA_SASL_USERNAME` | `xxx`   | The SASL username (key).    |
-| `DATACONTRACT_KAFKA_SASL_PASSWORD` | `xxx`   | The SASL password (secret). |
+| Environment Variable                | Example | Description                                                                      |
+|-------------------------------------|---------|----------------------------------------------------------------------------------|
+| `DATACONTRACT_KAFKA_SASL_USERNAME`  | `xxx`   | The SASL username (key).                                                         |
+| `DATACONTRACT_KAFKA_SASL_PASSWORD`  | `xxx`   | The SASL password (secret).                                                      |
+| `DATACONTRACT_KAFKA_SASL_MECHANISM` | `PLAIN` | Default `PLAIN`. Other supported mechanisms: `SCRAM-SHA-256` and `SCRAM-SHA-512` |
 
 
 ### Postgres
@@ -723,6 +728,10 @@ models:
     fields:
       my_column_1: # corresponds to a column
         type: varchar
+      my_column_2: # corresponds to a column with custom trino type
+        type: object
+        config:
+          trinoType: row(en_us varchar, pt_br varchar)
 ```
 
 #### Environment Variables
@@ -749,7 +758,7 @@ models:
 │ *  --format        [jsonschema|pydantic-model|sodacl|dbt|dbt-sources|db  The export format. [default: None] [required]         │
 │                    t-staging-sql|odcs|rdf|avro|protobuf|great-expectati                                                        │
 │                    ons|terraform|avro-idl|sql|sql-query|html|go|bigquer                                                        │
-│                    y|dbml|spark|sqlalchemy]                                                                                    │
+│                    y|dbml|spark|sqlalchemy|data-caterer|dcs]                                                                       │
 │    --output        PATH                                                  Specify the file path where the exported data will be │
 │                                                                          saved. If no path is provided, the output will be     │
 │                                                                          printed to stdout.                                    │
@@ -781,26 +790,30 @@ Available export options:
 
 | Type                 | Description                                             | Status |
 |----------------------|---------------------------------------------------------|--------|
-| `html`               | Export to HTML                                          | ✅     |
-| `jsonschema`         | Export to JSON Schema                                   | ✅     |
-| `odcs`               | Export to Open Data Contract Standard (ODCS)            | ✅     |
-| `sodacl`             | Export to SodaCL quality checks in YAML format          | ✅     |
-| `dbt`                | Export to dbt models in YAML format                     | ✅     |
-| `dbt-sources`        | Export to dbt sources in YAML format                    | ✅     |
-| `dbt-staging-sql`    | Export to dbt staging SQL models                        | ✅     |
-| `rdf`                | Export data contract to RDF representation in N3 format | ✅     |
-| `avro`               | Export to AVRO models                                   | ✅     |
-| `protobuf`           | Export to Protobuf                                      | ✅     |
-| `terraform`          | Export to terraform resources                           | ✅     |
-| `sql`                | Export to SQL DDL                                       | ✅     |
-| `sql-query`          | Export to SQL Query                                     | ✅     |
-| `great-expectations` | Export to Great Expectations Suites in JSON Format      | ✅     |
-| `bigquery`           | Export to BigQuery Schemas                              | ✅     |
-| `go`                 | Export to Go types                                      | ✅     |
-| `pydantic-model`     | Export to pydantic models                               | ✅     |
-| `DBML`               | Export to a DBML Diagram description                    | ✅     |
-| `spark`              | Export to a Spark StructType                            | ✅     |
-| `sqlalchemy`         | Export to SQLAlchemy Models                             | ✅     |
+| `html`               | Export to HTML                                          | ✅      |
+| `jsonschema`         | Export to JSON Schema                                   | ✅      |
+| `odcs_v2`            | Export to Open Data Contract Standard (ODCS) V2         | ✅      |
+| `odcs_v3`            | Export to Open Data Contract Standard (ODCS) V3         | ✅      |
+| `odcs`               | Export to Open Data Contract Standard (ODCS) V3         | ✅      |
+| `sodacl`             | Export to SodaCL quality checks in YAML format          | ✅      |
+| `dbt`                | Export to dbt models in YAML format                     | ✅      |
+| `dbt-sources`        | Export to dbt sources in YAML format                    | ✅      |
+| `dbt-staging-sql`    | Export to dbt staging SQL models                        | ✅      |
+| `rdf`                | Export data contract to RDF representation in N3 format | ✅      |
+| `avro`               | Export to AVRO models                                   | ✅      |
+| `protobuf`           | Export to Protobuf                                      | ✅      |
+| `terraform`          | Export to terraform resources                           | ✅      |
+| `sql`                | Export to SQL DDL                                       | ✅      |
+| `sql-query`          | Export to SQL Query                                     | ✅      |
+| `great-expectations` | Export to Great Expectations Suites in JSON Format      | ✅      |
+| `bigquery`           | Export to BigQuery Schemas                              | ✅      |
+| `go`                 | Export to Go types                                      | ✅      |
+| `pydantic-model`     | Export to pydantic models                               | ✅      |
+| `DBML`               | Export to a DBML Diagram description                    | ✅      |
+| `spark`              | Export to a Spark StructType                            | ✅      |
+| `sqlalchemy`         | Export to SQLAlchemy Models                             | ✅      |
+| `data-caterer`       | Export to Data Caterer in YAML format                   | ✅      |
+| `dcs`                | Export to Data Contract Specification in YAML format    | ✅      |
 | Missing something?   | Please create an issue on GitHub                        | TBD    |
 
 #### Great Expectations
@@ -864,6 +877,20 @@ To specify custom Avro properties in your data contract, you can define them wit
 
 >NOTE: At this moment, we just support [logicalType](https://avro.apache.org/docs/1.11.0/spec.html#Logical+Types) and [default](https://avro.apache.org/docs/1.11.0/spec.htm)
 
+#### Data Caterer
+
+The export function converts the data contract to a data generation task in YAML format that can be 
+ingested by [Data Caterer](https://github.com/data-catering/data-caterer). This gives you the 
+ability to generate production-like data in any environment based off your data contract.
+
+```shell
+datacontract export datacontract.yaml --format data-caterer --model orders
+```
+
+You can further customise the way data is generated via adding 
+[additional metadata in the YAML](https://data.catering/setup/generator/data-generator/) 
+to suit your needs.
+
 #### Example Configuration
 
 ```yaml
@@ -904,7 +931,7 @@ models:
                                                                                                                                
 ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ *  --format                       [sql|avro|dbt|glue|jsonschema|bigquery|odcs  The format of the source file.               │
-│                                   |unity|spark]                                [default: None]                              │
+│                                   |unity|spark|iceberg]                        [default: None]                              │
 │                                                                                [required]                                   │
 │    --source                       TEXT                                         The path to the file or Glue Database that   │
 │                                                                                should be imported.                          │
@@ -934,6 +961,8 @@ models:
 │                                                                                file (repeat for multiple table names, leave │
 │                                                                                empty for all tables in the file).           │
 │                                                                                [default: None]                              │
+│    --iceberg-table                TEXT                                         Table name to assign to the model created    │
+│                                                                                from the Iceberg schema. [default: None]     │
 │    --help                                                                      Show this message and exit.                  │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
@@ -959,12 +988,25 @@ Available import options:
 | `spark`            | Import from Spark StructTypes                  | ✅      |
 | `dbml`             | Import from DBML models                        | ✅      |
 | `protobuf`         | Import from Protobuf schemas                   | TBD    |
+| `iceberg`          | Import from an Iceberg JSON Schema Definition  | partial |
 | Missing something? | Please create an issue on GitHub               | TBD    |
 
 
+#### ODCS
+
+Import from Open Data Contract Standard (ODCS) v2 or v3.
+The importer automatically detects the ODCS version and imports the data contract.
+
+Examples:
+
+```bash
+# Example import from ODCS
+datacontract import --format odcs --source my_data_contract.odcs.yaml
+```
+
 #### BigQuery
 
-Bigquery data can either be imported off of JSON Files generated from the table descriptions or directly from the Bigquery API. In case you want to use JSON Files, specify the `source` parameter with a path to the JSON File.
+BigQuery data can either be imported off of JSON Files generated from the table descriptions or directly from the Bigquery API. In case you want to use JSON Files, specify the `source` parameter with a path to the JSON File.
 
 To import from the Bigquery API, you have to _omit_ `source` and instead need to provide `bigquery-project` and `bigquery-dataset`. Additionally you may specify `bigquery-table` to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the dataset will be imported.
 
@@ -1076,6 +1118,15 @@ datacontract import --format dbml --source <file_path> --dbml-table <table_name_
 datacontract import --format dbml --source <file_path> --dbml-table <table_name_1> --dbml-schema <schema_1>
 ```
 
+#### Iceberg
+
+Importing from an [Iceberg Table Json Schema Definition](https://iceberg.apache.org/spec/#appendix-c-json-serialization). Specify location of json files using the `source` parameter.
+
+Examples:
+
+```bash
+datacontract import --format iceberg --source ./tests/fixtures/iceberg/simple_schema.json --iceberg-table test-table
+```
 
 ### breaking
 

{datacontract_cli-0.10.12 → datacontract_cli-0.10.14}/datacontract/cli.py

@@ -244,6 +244,10 @@ def import_(
             help="List of table names to import from the DBML file (repeat for multiple table names, leave empty for all tables in the file)."
         ),
     ] = None,
+    iceberg_table: Annotated[
+        Optional[str],
+        typer.Option(help="Table name to assign to the model created from the Iceberg schema."),
+    ] = None,
 ):
     """
     Create a data contract from the given source location. Prints to stdout.
@@ -259,6 +263,7 @@ def import_(
         dbt_model=dbt_model,
         dbml_schema=dbml_schema,
         dbml_table=dbml_table,
+        iceberg_table=iceberg_table,
     )
     console.print(result.to_yaml())
 

{datacontract_cli-0.10.12 → datacontract_cli-0.10.14}/datacontract/data_contract.py

@@ -199,7 +199,15 @@ class DataContract:
 
         except DataContractException as e:
             run.checks.append(
-                Check(type=e.type, result=e.result, name=e.name, reason=e.reason, engine=e.engine, details="")
+                Check(
+                    type=e.type,
+                    name=e.name,
+                    result=e.result,
+                    reason=e.reason,
+                    model=e.model,
+                    engine=e.engine,
+                    details="",
+                )
             )
             run.log_error(str(e))
         except Exception as e: