datacontract-cli 0.10.9__tar.gz → 0.10.11__tar.gz

{datacontract_cli-0.10.9 → datacontract_cli-0.10.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: datacontract-cli
-Version: 0.10.9
+Version: 0.10.11
 Summary: The datacontract CLI is an open source command-line tool for working with Data Contracts. It uses data contract YAML files to lint the data contract, connect to data sources and execute schema and quality tests, detect breaking changes, and export to different formats. The tool is written in Python. It can be used as a standalone CLI tool, in a CI/CD pipeline, or directly as a Python library.
 Author-email: Jochen Christ <jochen.christ@innoq.com>, Stefan Negele <stefan.negele@innoq.com>, Simon Harrer <simon.harrer@innoq.com>
 Project-URL: Homepage, https://cli.datacontract.com
@@ -12,14 +12,14 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: typer[all]<0.13,>=0.9
-Requires-Dist: pydantic<2.8.0,>=2.5.3
+Requires-Dist: pydantic<2.9.0,>=2.8.2
 Requires-Dist: pyyaml~=6.0.1
 Requires-Dist: requests<2.33,>=2.31
-Requires-Dist: fastapi==0.111.0
+Requires-Dist: fastapi==0.111.1
 Requires-Dist: fastparquet==2024.5.0
 Requires-Dist: python-multipart==0.0.9
 Requires-Dist: rich~=13.7.0
-Requires-Dist: simple-ddl-parser==1.5.1
+Requires-Dist: simple-ddl-parser==1.5.2
 Requires-Dist: soda-core-duckdb<3.4.0,>=3.3.1
 Requires-Dist: setuptools>=60
 Requires-Dist: duckdb==1.0.0
@@ -37,7 +37,7 @@ Provides-Extra: bigquery
 Requires-Dist: soda-core-bigquery<3.4.0,>=3.3.1; extra == "bigquery"
 Provides-Extra: databricks
 Requires-Dist: soda-core-spark-df<3.4.0,>=3.3.1; extra == "databricks"
-Requires-Dist: databricks-sql-connector<3.3.0,>=3.1.2; extra == "databricks"
+Requires-Dist: databricks-sql-connector<3.4.0,>=3.1.2; extra == "databricks"
 Requires-Dist: soda-core-spark[databricks]<3.4.0,>=3.3.1; extra == "databricks"
 Provides-Extra: deltalake
 Requires-Dist: deltalake<0.19,>=0.17; extra == "deltalake"
@@ -61,19 +61,19 @@ Provides-Extra: dev
 Requires-Dist: datacontract-cli[all]; extra == "dev"
 Requires-Dist: httpx==0.27.0; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
-Requires-Dist: pre-commit~=3.7.1; extra == "dev"
+Requires-Dist: pre-commit<3.9.0,>=3.7.1; extra == "dev"
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-xdist; extra == "dev"
-Requires-Dist: moto; extra == "dev"
+Requires-Dist: moto==5.0.11; extra == "dev"
 Requires-Dist: pymssql==2.3.0; extra == "dev"
 Requires-Dist: kafka-python; extra == "dev"
-Requires-Dist: trino==0.328.0; extra == "dev"
-Requires-Dist: testcontainers<4.8,>=4.5; extra == "dev"
-Requires-Dist: testcontainers[core]; extra == "dev"
-Requires-Dist: testcontainers[minio]; extra == "dev"
-Requires-Dist: testcontainers[postgres]; extra == "dev"
-Requires-Dist: testcontainers[kafka]; extra == "dev"
-Requires-Dist: testcontainers[mssql]; extra == "dev"
+Requires-Dist: trino==0.329.0; extra == "dev"
+Requires-Dist: testcontainers==4.7.2; extra == "dev"
+Requires-Dist: testcontainers[core]==4.7.2; extra == "dev"
+Requires-Dist: testcontainers[minio]==4.7.2; extra == "dev"
+Requires-Dist: testcontainers[postgres]==4.7.2; extra == "dev"
+Requires-Dist: testcontainers[kafka]==4.7.2; extra == "dev"
+Requires-Dist: testcontainers[mssql]==4.7.2; extra == "dev"
 
 # Data Contract CLI
 
@@ -197,10 +197,10 @@ $ datacontract export --format html datacontract.yaml > datacontract.html
 # import avro (other formats: sql, glue, bigquery...)
 $ datacontract import --format avro --source avro_schema.avsc
 
-# find differences between to data contracts
+# find differences between two data contracts
 $ datacontract diff datacontract-v1.yaml datacontract-v2.yaml
 
-# find differences between to data contracts categorized into error, warning, and info.
+# find differences between two data contracts categorized into error, warning, and info.
 $ datacontract changelog datacontract-v1.yaml datacontract-v2.yaml
 
 # fail pipeline on breaking changes. Uses changelog internally and showing only error and warning.
@@ -300,16 +300,16 @@ Commands
  Download a datacontract.yaml template and write it to file.
 
 ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────╮
-│   location      [LOCATION]  The location (url or path) of the data contract yaml to create. │
-│                             [default: datacontract.yaml]                                    │
+│   location      [LOCATION]  The location (url or path) of the data contract yaml to create.  │
+│                             [default: datacontract.yaml]                                     │
 ╰──────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ────────────────────────────────────────────────────────────────────────────────────╮
-│ --template                       TEXT  URL of a template or data contract                   │
-│                                        [default:                                            │
-│                                        https://datacontract.com/datacontract.init.yaml]     │
-│ --overwrite    --no-overwrite          Replace the existing datacontract.yaml               │
-│                                        [default: no-overwrite]                              │
-│ --help                                 Show this message and exit.                          │
+│ --template                       TEXT  URL of a template or data contract                    │
+│                                        [default:                                             │
+│                                        https://datacontract.com/datacontract.init.yaml]      │
+│ --overwrite    --no-overwrite          Replace the existing datacontract.yaml                │
+│                                        [default: no-overwrite]                               │
+│ --help                                 Show this message and exit.                           │
 ╰──────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -321,12 +321,12 @@ Commands
  Validate that the datacontract.yaml is correctly formatted.
 
 ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│   location      [LOCATION]  The location (url or path) of the data contract yaml. [default: datacontract.yaml]                 │
+│   location      [LOCATION]  The location (url or path) of the data contract yaml. [default: datacontract.yaml]                  │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ --schema        TEXT  The location (url or path) of the Data Contract Specification JSON Schema                                │
-│                       [default: https://datacontract.com/datacontract.schema.json]                                             │
-│ --help                Show this message and exit.                                                                              │
+│ --schema        TEXT  The location (url or path) of the Data Contract Specification JSON Schema                                 │
+│                       [default: https://datacontract.com/datacontract.schema.json]                                              │
+│ --help                Show this message and exit.                                                                               │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -338,28 +338,28 @@ Commands
  Run schema and quality tests on configured servers.
 
 ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│   location      [LOCATION]  The location (url or path) of the data contract yaml. [default: datacontract.yaml]                 │
+│   location      [LOCATION]  The location (url or path) of the data contract yaml. [default: datacontract.yaml]                  │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ --schema                                                       TEXT  The location (url or path) of the Data Contract           │
-│                                                                      Specification JSON Schema                                 │
-│                                                                      [default:                                                 │
-│                                                                      https://datacontract.com/datacontract.schema.json]        │
-│ --server                                                       TEXT  The server configuration to run the schema and quality    │
-│                                                                      tests. Use the key of the server object in the data       │
-│                                                                      contract yaml file to refer to a server, e.g.,            │
-│                                                                      `production`, or `all` for all servers (default).         │
-│                                                                      [default: all]                                            │
-│ --examples                    --no-examples                          Run the schema and quality tests on the example data      │
-│                                                                      within the data contract.                                 │
-│                                                                      [default: no-examples]                                    │
-│ --publish                                                      TEXT  The url to publish the results after the test             │
-│                                                                      [default: None]                                           │
-│ --publish-to-opentelemetry    --no-publish-to-opentelemetry          Publish the results to opentelemetry. Use environment     │
-│                                                                      variables to configure the OTLP endpoint, headers, etc.   │
-│                                                                      [default: no-publish-to-opentelemetry]                    │
-│ --logs                        --no-logs                              Print logs [default: no-logs]                             │
-│ --help                                                               Show this message and exit.                               │
+│ --schema                                                       TEXT  The location (url or path) of the Data Contract            │
+│                                                                      Specification JSON Schema                                  │
+│                                                                      [default:                                                  │
+│                                                                      https://datacontract.com/datacontract.schema.json]         │
+│ --server                                                       TEXT  The server configuration to run the schema and quality     │
+│                                                                      tests. Use the key of the server object in the data        │
+│                                                                      contract yaml file to refer to a server, e.g.,             │
+│                                                                      `production`, or `all` for all servers (default).          │
+│                                                                      [default: all]                                             │
+│ --examples                    --no-examples                          Run the schema and quality tests on the example data       │
+│                                                                      within the data contract.                                  │
+│                                                                      [default: no-examples]                                     │
+│ --publish                                                      TEXT  The url to publish the results after the test              │
+│                                                                      [default: None]                                            │
+│ --publish-to-opentelemetry    --no-publish-to-opentelemetry          Publish the results to opentelemetry. Use environment      │
+│                                                                      variables to configure the OTLP endpoint, headers, etc.    │
+│                                                                      [default: no-publish-to-opentelemetry]                     │
+│ --logs                        --no-logs                              Print logs [default: no-logs]                              │
+│ --help                                                               Show this message and exit.                                │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -385,7 +385,7 @@ Supported server types:
 - [sqlserver](#sqlserver)
 - [databricks](#databricks)
 - [databricks (programmatic)](#databricks-programmatic)
-- [dataframr (programmatic)](#dataframe-programmatic)
+- [dataframe (programmatic)](#dataframe-programmatic)
 - [snowflake](#snowflake)
 - [kafka](#kafka)
 - [postgres](#postgres)
@@ -473,7 +473,6 @@ models:
 | `DATACONTRACT_BIGQUERY_ACCOUNT_INFO_JSON_PATH` | `~/service-access-key.json` | Service Access key as saved on key creation by BigQuery. If this environment variable isn't set, the cli tries to use `GOOGLE_APPLICATION_CREDENTIALS` as a fallback, so if you have that set for using their Python library anyway, it should work seamlessly. |
 
 
-
 ### Azure
 
 Data Contract CLI can test data that is stored in Azure Blob storage or Azure Data Lake Storage (Gen2) (ADLS) in various formats.
@@ -493,11 +492,11 @@ servers:
 
 Authentication works with an Azure Service Principal (SPN) aka App Registration with a secret.
 
-| Environment Variable              | Example                       | Description                                          |
-|-----------------------------------|-------------------------------|------------------------------------------------------|
-| `DATACONTRACT_AZURE_TENANT_ID`            | `79f5b80f-10ff-40b9-9d1f-774b42d605fc`                  | The Azure Tenant ID                                  |
-| `DATACONTRACT_AZURE_CLIENT_ID`     | `3cf7ce49-e2e9-4cbc-a922-4328d4a58622`            | The ApplicationID / ClientID of the app registration |
-| `DATACONTRACT_AZURE_CLIENT_SECRET` | `yZK8Q~GWO1MMXXXXXXXXXXXXX` | The Client Secret value                              |
+| Environment Variable               | Example                                | Description                                          |
+|------------------------------------|----------------------------------------|------------------------------------------------------|
+| `DATACONTRACT_AZURE_TENANT_ID`     | `79f5b80f-10ff-40b9-9d1f-774b42d605fc` | The Azure Tenant ID                                  |
+| `DATACONTRACT_AZURE_CLIENT_ID`     | `3cf7ce49-e2e9-4cbc-a922-4328d4a58622` | The ApplicationID / ClientID of the app registration |
+| `DATACONTRACT_AZURE_CLIENT_SECRET` | `yZK8Q~GWO1MMXXXXXXXXXXXXX`            | The Client Secret value                              |
 
 
 
@@ -527,13 +526,13 @@ models:
 
 #### Environment Variables
 
-| Environment Variable             | Example            | Description |
-|----------------------------------|--------------------|-------------|
-| `DATACONTRACT_SQLSERVER_USERNAME` | `root`         | Username    |
-| `DATACONTRACT_SQLSERVER_PASSWORD` | `toor` | Password    |
-| `DATACONTRACT_SQLSERVER_TRUSTED_CONNECTION` | `True` | Use windows authentication, instead of login    |
-| `DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE` | `True` | Trust self-signed certificate    |
-| `DATACONTRACT_SQLSERVER_ENCRYPTED_CONNECTION` | `True` | Use SSL    |
+| Environment Variable                              | Example| Description                                  |
+|---------------------------------------------------|--------|----------------------------------------------|
+| `DATACONTRACT_SQLSERVER_USERNAME`                 | `root` | Username                                     |
+| `DATACONTRACT_SQLSERVER_PASSWORD`                 | `toor` | Password                                     |
+| `DATACONTRACT_SQLSERVER_TRUSTED_CONNECTION`       | `True` | Use windows authentication, instead of login |
+| `DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE` | `True` | Trust self-signed certificate                |
+| `DATACONTRACT_SQLSERVER_ENCRYPTED_CONNECTION`     | `True` | Use SSL                                      |
 
 
 
@@ -564,8 +563,8 @@ models:
 
 | Environment Variable                         | Example                              | Description                                           |
 |----------------------------------------------|--------------------------------------|-------------------------------------------------------|
-| `DATACONTRACT_DATABRICKS_TOKEN` | `dapia00000000000000000000000000000` | The personal access token to authenticate             |
-| `DATACONTRACT_DATABRICKS_HTTP_PATH` | `/sql/1.0/warehouses/b053a3ffffffff` | The HTTP path to the SQL warehouse or compute cluster |
+| `DATACONTRACT_DATABRICKS_TOKEN`              | `dapia00000000000000000000000000000` | The personal access token to authenticate             |
+| `DATACONTRACT_DATABRICKS_HTTP_PATH`          | `/sql/1.0/warehouses/b053a3ffffffff` | The HTTP path to the SQL warehouse or compute cluster |
 
 
 ### Databricks (programmatic)
@@ -610,7 +609,7 @@ run.result
 
 Works with Spark DataFrames.
 DataFrames need to be created as named temporary views.
-Multiple temporary views are suppored if your data contract contains multiple models.
+Multiple temporary views are supported if your data contract contains multiple models.
 
 Testing DataFrames is useful to test your datasets in a pipeline before writing them to a data source.
 
@@ -666,14 +665,31 @@ models:
 ```
 
 #### Environment Variables
-
-| Environment Variable               | Example            | Description                                         |
-|------------------------------------|--------------------|-----------------------------------------------------|
-| `DATACONTRACT_SNOWFLAKE_USERNAME`  | `datacontract`     | Username                                            |
-| `DATACONTRACT_SNOWFLAKE_PASSWORD`  | `mysecretpassword` | Password                                            |
-| `DATACONTRACT_SNOWFLAKE_ROLE`      | `DATAVALIDATION`   | The snowflake role to use.                          |
-| `DATACONTRACT_SNOWFLAKE_WAREHOUSE` | `COMPUTE_WH`       | The Snowflake Warehouse to use executing the tests. |
-
+All [parameters supported by Soda](https://docs.soda.io/soda/connect-snowflake.html), uppercased and prepended by `DATACONTRACT_SNOWFLAKE_` prefix.  
+For example:
+
+| Soda parameter       | Environment Variable                        | 
+|----------------------|---------------------------------------------|
+| `username`           | `DATACONTRACT_SNOWFLAKE_USERNAME`           |
+| `password`           | `DATACONTRACT_SNOWFLAKE_PASSWORD`           |
+| `warehouse`          | `DATACONTRACT_SNOWFLAKE_WAREHOUSE`          |
+| `role`               | `DATACONTRACT_SNOWFLAKE_ROLE`               |
+| `connection_timeout` | `DATACONTRACT_SNOWFLAKE_CONNECTION_TIMEOUT` |
+
+Beware, that parameters:
+* `account`
+* `database`
+* `schema`
+
+are obtained from the `servers` section of the YAML-file.  
+E.g. from the example above:
+```yaml
+servers:
+  snowflake:
+    account: abcdefg-xn12345
+    database: ORDER_DB
+    schema: ORDERS_PII_V2
+```
 
 
 ### Kafka
@@ -778,7 +794,7 @@ models:
 │ *  --format        [jsonschema|pydantic-model|sodacl|dbt|dbt-sources|db  The export format. [default: None] [required]         │
 │                    t-staging-sql|odcs|rdf|avro|protobuf|great-expectati                                                        │
 │                    ons|terraform|avro-idl|sql|sql-query|html|go|bigquer                                                        │
-│                    y|dbml|spark]                                                                                                     │
+│                    y|dbml|spark]                                                                                               │
 │    --output        PATH                                                  Specify the file path where the exported data will be │
 │                                                                          saved. If no path is provided, the output will be     │
 │                                                                          printed to stdout.                                    │
@@ -926,22 +942,34 @@ models:
 ```
  Usage: datacontract import [OPTIONS]
 
- Create a data contract from the given source location. Prints to stdout.
-
-╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ *  --format                  [sql|avro|glue|bigquery|jsonschema|unity]  The format of the source file. [default: None] [required]         │
-│    --source                  TEXT                                       The path to the file or Glue Database that should be imported.    │
-│                                                                         [default: None]                                                   │
-│    --glue-table              TEXT                                       List of table ids to import from the Glue Database (repeat for    │
-│                                                                         multiple table ids, leave empty for all tables in the dataset).   │
-│                                                                         [default: None]                                                   │
-│    --bigquery-project        TEXT                                       The bigquery project id. [default: None]                          │
-│    --bigquery-dataset        TEXT                                       The bigquery dataset id. [default: None]                          │
-│    --bigquery-table          TEXT                                       List of table ids to import from the bigquery API (repeat for     │
-│                                                                         multiple table ids, leave empty for all tables in the dataset).   │
-│                                                                         [default: None]                                                   │
-│    --help                                                               Show this message and exit.                                       │
-╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+ Create a data contract from the given source location. Prints to stdout.                                                      
+                                                                                                                               
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ *  --format                       [sql|avro|dbt|glue|jsonschema|bigquery|odcs  The format of the source file.               │
+│                                   |unity|spark]                                [default: None]                              │
+│                                                                                [required]                                   │
+│    --source                       TEXT                                         The path to the file or Glue Database that   │
+│                                                                                should be imported.                          │
+│                                                                                [default: None]                              │
+│    --glue-table                   TEXT                                         List of table ids to import from the Glue    │
+│                                                                                Database (repeat for multiple table ids,     │
+│                                                                                leave empty for all tables in the dataset).  │
+│                                                                                [default: None]                              │
+│    --bigquery-project             TEXT                                         The bigquery project id. [default: None]     │
+│    --bigquery-dataset             TEXT                                         The bigquery dataset id. [default: None]     │
+│    --bigquery-table               TEXT                                         List of table ids to import from the         │
+│                                                                                bigquery API (repeat for multiple table ids, │
+│                                                                                leave empty for all tables in the dataset).  │
+│                                                                                [default: None]                              │
+│    --unity-table-full-name        TEXT                                         Full name of a table in the unity catalog    │
+│                                                                                [default: None]                              │
+│    --dbt-model                    TEXT                                         List of models names to import from the dbt  │
+│                                                                                manifest file (repeat for multiple models    │
+│                                                                                names, leave empty for all models in the     │
+│                                                                                dataset).                                    │
+│                                                                                [default: None]                              │
+│    --help                                                                      Show this message and exit.                  │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
 Example:
@@ -952,18 +980,19 @@ datacontract import --format sql --source my_ddl.sql
 
 Available import options:
 
-| Type               | Description                                    | Status  |
-|--------------------|------------------------------------------------|---------|
+| Type               | Description                                    | Status |
+|--------------------|------------------------------------------------|--------|
 | `sql`              | Import from SQL DDL                            | ✅      |
 | `avro`             | Import from AVRO schemas                       | ✅      |
 | `glue`             | Import from AWS Glue DataCatalog               | ✅      |
-| `protobuf`         | Import from Protobuf schemas                   | TBD     |
-| `jsonschema`       | Import from JSON Schemas                       | ✅       |
-| `bigquery`         | Import from BigQuery Schemas                   | ✅       |
+| `jsonschema`       | Import from JSON Schemas                       | ✅      |
+| `bigquery`         | Import from BigQuery Schemas                   | ✅      |
 | `unity`            | Import from Databricks Unity Catalog           | partial |
-| `dbt`              | Import from dbt models                         | TBD     |
-| `odcs`             | Import from Open Data Contract Standard (ODCS) | ✅       |
-| Missing something? | Please create an issue on GitHub               | TBD     |
+| `dbt`              | Import from dbt models                         | ✅      |
+| `odcs`             | Import from Open Data Contract Standard (ODCS) | ✅      |
+| `spark`            | Import from Spark StructTypes                  | ✅      |
+| `protobuf`         | Import from Protobuf schemas                   | TBD    |
+| Missing something? | Please create an issue on GitHub               | TBD    |
 
 
 #### BigQuery
@@ -1005,7 +1034,24 @@ export DATABRICKS_IMPORT_ACCESS_TOKEN=<token>
 datacontract import --format unity --unity-table-full-name <table_full_name>
 ```
 
-### Glue
+#### dbt
+
+Importing from dbt manifest file.
+You may give the `dbt-model` parameter to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the database will be imported.
+
+Examples:
+
+```bash
+# Example import from dbt manifest with specifying the tables to import
+datacontract import --format dbt --source <manifest_path> --dbt-model <model_name_1> --dbt-model <model_name_2> --dbt-model <model_name_3>
+```
+
+```bash
+# Example import from dbt manifest importing all tables in the database
+datacontract import --format dbt --source <manifest_path>
+```
+
+#### Glue
 
 Importing from Glue reads the necessary Data directly off of the AWS API.
 You may give the `glue-table` parameter to enumerate the tables that should be imported. If no tables are given, _all_ available tables of the database will be imported.
@@ -1022,6 +1068,15 @@ datacontract import --format glue --source <database_name> --glue-table <table_n
 datacontract import --format glue --source <database_name>
 ```
 
+#### Spark
+
+Importing from Spark table or view these must be created or accessible in the Spark context. Specify tables list in `source` parameter.
+
+Example:
+
+```bash
+datacontract import --format spark --source "users,orders"
+```
 
 ### breaking
 
@@ -1031,11 +1086,11 @@ datacontract import --format glue --source <database_name>
  Identifies breaking changes between data contracts. Prints to stdout.
 
 ╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]        │
-│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]        │
+│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]         │
+│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]         │
 ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ --help          Show this message and exit.                                                                              │
+│ --help          Show this message and exit.                                                                               │
 ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -1047,11 +1102,11 @@ datacontract import --format glue --source <database_name>
  Generate a changelog between data contracts. Prints to stdout.
 
 ╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]        │
-│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]        │
+│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]         │
+│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]         │
 ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ --help          Show this message and exit.                                                                              │
+│ --help          Show this message and exit.                                                                               │
 ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -1063,8 +1118,8 @@ datacontract import --format glue --source <database_name>
  PLACEHOLDER. Currently works as 'changelog' does.
 
 ╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]        │
-│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]        │
+│ *    location_old      TEXT  The location (url or path) of the old data contract yaml. [default: None] [required]         │
+│ *    location_new      TEXT  The location (url or path) of the new data contract yaml. [default: None] [required]         │
 ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ --help          Show this message and exit.                                                                               │
@@ -1080,9 +1135,9 @@ datacontract import --format glue --source <database_name>
  Create an html catalog of data contracts.
 
 ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ --files         TEXT  Glob pattern for the data contract files to include in the catalog. [default: *.yaml]             │
-│ --output        TEXT  Output directory for the catalog html files. [default: catalog/]                                  │
-│ --help                Show this message and exit.                                                                       │
+│ --files         TEXT  Glob pattern for the data contract files to include in the catalog. [default: *.yaml]              │
+│ --output        TEXT  Output directory for the catalog html files. [default: catalog/]                                   │
+│ --help                Show this message and exit.                                                                        │
 ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
 
@@ -1104,19 +1159,22 @@ datacontract import --format glue --source <database_name>
 
 ## Integrations
 
-| Integration       | Option                       | Description                                                                                           |
-|-------------------|------------------------------|-------------------------------------------------------------------------------------------------------|
-| Data Mesh Manager | `--publish`                  | Push full results to the [Data Mesh Manager API](https://api.datamesh-manager.com/swagger/index.html) |
-| OpenTelemetry     | `--publish-to-opentelemetry` | Push result as gauge metrics                                                                          |
+| Integration           | Option                       | Description                                                                                                   |
+|-----------------------|------------------------------|---------------------------------------------------------------------------------------------------------------|
+| Data Mesh Manager     | `--publish`                  | Push full results to the [Data Mesh Manager API](https://api.datamesh-manager.com/swagger/index.html)         |
+| Data Contract Manager | `--publish`                  | Push full results to the [Data Contract Manager API](https://api.datacontract-manager.com/swagger/index.html) |
+| OpenTelemetry         | `--publish-to-opentelemetry` | Push result as gauge metrics                                                                                  |
 
 ### Integration with Data Mesh Manager
 
-If you use [Data Mesh Manager](https://datamesh-manager.com/), you can use the data contract URL and append the `--publish` option to send and display the test results. Set an environment variable for your API key.
+If you use [Data Mesh Manager](https://datamesh-manager.com/) or [Data Contract Manager](https://datacontract-manager.com/), you can use the data contract URL and append the `--publish` option to send and display the test results. Set an environment variable for your API key.
 
 ```bash
 # Fetch current data contract, execute tests on production, and publish result to data mesh manager
 $ EXPORT DATAMESH_MANAGER_API_KEY=xxx
-$ datacontract test https://demo.datamesh-manager.com/demo279750347121/datacontracts/4df9d6ee-e55d-4088-9598-b635b2fdcbbc/datacontract.yaml --server production --publish
+$ datacontract test https://demo.datamesh-manager.com/demo279750347121/datacontracts/4df9d6ee-e55d-4088-9598-b635b2fdcbbc/datacontract.yaml \ 
+ --server production \
+ --publish https://api.datamesh-manager.com/api/test-results
 ```
 
 ### Integration with OpenTelemetry
@@ -1126,12 +1184,12 @@ If you use OpenTelemetry, you can use the data contract URL and append the `--pu
 The metric name is "datacontract.cli.test.result" and it uses the following encoding for the result:
 
 | datacontract.cli.test.result | Description                           |
-|-------|---------------------------------------|
-| 0     | test run passed, no warnings          |
-| 1     | test run has warnings                 |
-| 2     | test run failed                       |
-| 3     | test run not possible due to an error |
-| 4     | test status unknown                   |
+|------------------------------|---------------------------------------|
+| 0                            | test run passed, no warnings          |
+| 1                            | test run has warnings                 |
+| 2                            | test run failed                       |
+| 3                            | test run not possible due to an error |
+| 4                            | test status unknown                   |
 
 
 ```bash
@@ -1159,7 +1217,7 @@ Create a data contract based on the actual data. This is the fastest way to get
 
 1. Use an existing physical schema (e.g., SQL DDL) as a starting point to define your logical data model in the contract. Double check right after the import whether the actual data meets the imported logical data model. Just to be sure.
     ```bash
-    $ datacontract import --format sql ddl.sql
+    $ datacontract import --format sql --source ddl.sql
     $ datacontract test
     ```
 
@@ -1182,7 +1240,7 @@ Create a data contract based on the actual data. This is the fastest way to get
 
 5. Set up a CI pipeline that executes daily and reports the results to the [Data Mesh Manager](https://datamesh-manager.com). Or to some place else. You can even publish to any opentelemetry compatible system.
    ```bash
-   $ datacontract test --publish https://api.datamesh-manager.com/api/runs
+   $ datacontract test --publish https://api.datamesh-manager.com/api/test-results
    ```
 
 ### Contract-First
@@ -1234,7 +1292,7 @@ Examples: adding models or fields
 - Add the models or fields in the datacontract.yaml
 - Increment the minor version of the datacontract.yaml on any change. Simply edit the datacontract.yaml for this.
 - You need a policy that these changes are non-breaking. That means that one cannot use the star expression in SQL to query a table under contract. Make the consequences known.
-- Fail the build in the Pull Request if a datacontract.yaml accidentially adds a breaking change even despite only a minor version change
+- Fail the build in the Pull Request if a datacontract.yaml accidentally adds a breaking change even despite only a minor version change
    ```bash
   $ datacontract breaking datacontract-from-pr.yaml datacontract-from-main.yaml
   ```
@@ -1292,7 +1350,7 @@ if __name__ == "__main__":
     data_contract = DataContract(
         data_contract_file="/path/datacontract.yaml"
     )
-    # call export
+    # Call export
     result = data_contract.export(
         export_format="custom", model="orders", server="production", custom_arg="my_custom_arg"
     )
@@ -1318,10 +1376,11 @@ Output
 Using the importer factory to add a new custom importer
 ```python
 
-from datacontract.model.data_contract_specification import DataContractSpecification
+from datacontract.model.data_contract_specification import DataContractSpecification, Field, Model
 from datacontract.data_contract import DataContract
 from datacontract.imports.importer import Importer
 from datacontract.imports.importer_factory import importer_factory
+
 import json
 
 # Create a custom class that implements import_source method
@@ -1332,43 +1391,89 @@ class CustomImporter(Importer):
         source_dict = json.loads(source)
         data_contract_specification.id = source_dict.get("id_custom")
         data_contract_specification.info.title = source_dict.get("title")
+        data_contract_specification.info.version = source_dict.get("version")
         data_contract_specification.info.description = source_dict.get("description_from_app")
-
+        
+        for model in source_dict.get("models", []):
+            fields = {}
+            for column in model.get('columns'):
+                field = Field(
+                    description=column.get('column_description'), 
+                    type=column.get('type') 
+                )
+                fields[column.get('name')] = field               
+                   
+            dc_model = Model(
+                description=model.get('description'), 
+                fields= fields
+            )
+
+            data_contract_specification.models[model.get('name')] = dc_model
         return data_contract_specification
-
+ 
 
 # Register the new custom class into factory
 importer_factory.register_importer("custom_company_importer", CustomImporter)
 
 
 if __name__ == "__main__":
-    # get a custom da
-    json_from_custom_app = '{"id_custom":"uuid-custom","version":"0.0.2", "title":"my_custom_imported_data", "description_from_app": "Custom contract description"}'
+    # Get a custom data from other app 
+    json_from_custom_app = '''
+    {
+        "id_custom": "uuid-custom",
+        "version": "0.0.2",
+        "title": "my_custom_imported_data",
+        "description_from_app": "Custom contract description",
+        "models": [
+            {
+            "name": "model1",
+            "desctiption": "model description from app",
+            "columns": [
+                {
+                "name": "columnA",
+                "type": "varchar",
+                "column_description": "my_column description"
+                },
+                {
+                "name": "columnB",
+                "type": "varchar",
+                "column_description": "my_columnB description"
+                }
+            ]
+            }
+        ]
+        }
+    '''
     # Create a DataContract instance
     data_contract = DataContract()
 
-    # call import_from
+    # Call import_from_source
     result = data_contract.import_from_source(
-        format="custom_company_importer", data_contract_specification=DataContract.init(), source=json_from_custom_app
-    )
-    print(dict(result))
-
+        format="custom_company_importer", 
+        data_contract_specification=DataContract.init(), 
+        source=json_from_custom_app
+    ) 
+    print(result.to_yaml() )
 ```
 Output
+  
+```yaml
+dataContractSpecification: 0.9.3
+id: uuid-custom
+info:
+  title: my_custom_imported_data
+  version: 0.0.2
+  description: Custom contract description
+models:
+  model1:
+    fields:
+      columnA:
+        type: varchar
+        description: my_column description
+      columnB:
+        type: varchar
+        description: my_columnB description
 
-```python
-{
-  'dataContractSpecification': '0.9.3', 
-  'id': 'uuid-custom', 
-  'info': Info(title='my_custom_imported_data', version='0.0.1', status=None, description='Custom contract description', owner=None, contact=None), 
-  'servers': {}, 
-  'terms': None, 
-  'models': {}, 
-  'definitions': {}, 
-  'examples': [], 
-  'quality': None, 
-  'servicelevels': None
-}
 ```
 ## Development Setup
 
@@ -1435,10 +1540,11 @@ repos:
 
 ### Available Hook IDs
 
-| Hook ID           | Description              | Dependency |
-| ----------------- | ------------------------ | ---------- |
-| datacontract-lint | Runs the lint subcommand. | Python3    |
-| datacontract-test | Runs the test subcommand. Please look at [test](#test) section for all available arguments. | Python3 |
+| Hook ID           | Description                                        | Dependency |
+| ----------------- | -------------------------------------------------- | ---------- |
+| datacontract-lint | Runs the lint subcommand.                          | Python3    |
+| datacontract-test | Runs the test subcommand. Please look at           | Python3    |
+|                   | [test](#test) section for all available arguments. |            |
 
 ## Release Steps
 
@@ -1456,13 +1562,15 @@ We are happy to receive your contributions. Propose your change in an issue or d
 ## Companies using this tool
 
 - [INNOQ](https://innoq.com)
+- [Data Catering](https://data.catering/)
 - And many more. To add your company, please create a pull request.
 
 ## Related Tools
 
-- [Data Mesh Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data products and data contracts. It supports the data contract specification and allows the user to import or export data contracts using this specification.
+- [Data Contract Manager](https://www.datacontract-manager.com/) is a commercial tool to manage data contracts. It contains a web UI, access management, and data governance for a full enterprise data marketplace.
 - [Data Contract GPT](https://gpt.datacontract.com) is a custom GPT that can help you write data contracts.
 - [Data Contract Editor](https://editor.datacontract.com) is an editor for Data Contracts, including a live html preview.
+- [Data Contract Playground](https://data-catering.github.io/data-contract-playground/) allows you to validate and export your data contract to different formats within your browser.
 
 ## License