sqlmesh 0.187.6.dev24__tar.gz → 0.187.7__tar.gz

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sqlmesh
-Version: 0.187.6.dev24
+Version: 0.187.7
 Summary: Next-generation data transformation framework
 Author-email: "TobikoData Inc." <engineering@tobikodata.com>
 License:                                  Apache License
@@ -233,7 +233,7 @@ Requires-Dist: pydantic>=2.0.0
 Requires-Dist: requests
 Requires-Dist: rich[jupyter]
 Requires-Dist: ruamel.yaml
-Requires-Dist: sqlglot[rs]~=26.24.0
+Requires-Dist: sqlglot[rs]~=26.21.1
 Requires-Dist: tenacity
 Requires-Dist: time-machine
 Requires-Dist: json-stream
@@ -348,7 +348,7 @@ Dynamic: license-file
   <img src="docs/readme/sqlmesh.png" alt="SQLMesh logo" width="50%" height="50%">
 </p>
 
-SQLMesh is a next-generation data transformation framework designed to ship data quickly, efficiently, and without error. Data teams can run and deploy data transformations written in SQL or Python with visibility and control at any size.
+SQLMesh is a next-generation data transformation framework designed to ship data quickly, efficiently, and without error. Data teams can efficiently run and deploy data transformations written in SQL or Python with visibility and control at any size.
 
 It is more than just a [dbt alternative](https://tobikodata.com/reduce_costs_with_cron_and_partitions.html).
 
@@ -357,10 +357,9 @@ It is more than just a [dbt alternative](https://tobikodata.com/reduce_costs_wit
 </p>
 
 ## Core Features
+<img src="https://github.com/TobikoData/sqlmesh-public-assets/blob/main/sqlmesh_plan_mode.gif?raw=true" alt="SQLMesh Plan Mode">
 
-<img src="https://github.com/TobikoData/sqlmesh-public-assets/blob/main/vscode.gif?raw=true" alt="SQLMesh Plan Mode">
-
-> Get instant SQL impact and context of your changes, both in the CLI and in the [SQLMesh VSCode Extension](https://sqlmesh.readthedocs.io/en/latest/guides/vscode/?h=vs+cod)
+> Get instant SQL impact analysis of your changes, whether in the CLI or in [SQLMesh Plan Mode](https://sqlmesh.readthedocs.io/en/stable/guides/ui/?h=modes#working-with-an-ide)
 
   <details>
   <summary><b>Virtual Data Environments</b></summary>
@@ -467,7 +466,6 @@ outputs:
 * Never build a table [more than once](https://tobikodata.com/simplicity-or-efficiency-how-dbt-makes-you-choose.html)
 * Track what data’s been modified and run only the necessary transformations for [incremental models](https://tobikodata.com/correctly-loading-incremental-data-at-scale.html)
 * Run [unit tests](https://tobikodata.com/we-need-even-greater-expectations.html) for free and configure automated audits
-* Run [table diffs](https://sqlmesh.readthedocs.io/en/stable/examples/sqlmesh_cli_crash_course/?h=crash#run-data-diff-against-prod) between prod and dev based on tables/views impacted by a change 
 
 <details>
 <summary><b>Level Up Your SQL</b></summary>
@@ -479,7 +477,7 @@ Write SQL in any dialect and SQLMesh will transpile it to your target SQL dialec
 * Definitions using [simply SQL](https://sqlmesh.readthedocs.io/en/stable/concepts/models/sql_models/#sql-based-definition) (no need for redundant and confusing `Jinja` + `YAML`)
 * See impact of changes before you run them in your warehouse with column-level lineage
 
-For more information, check out the [website](https://www.tobikodata.com/sqlmesh) and [documentation](https://sqlmesh.readthedocs.io/en/stable/).
+For more information, check out the [website](https://sqlmesh.com) and [documentation](https://sqlmesh.readthedocs.io/en/stable/).
 
 ## Getting Started
 Install SQLMesh through [pypi](https://pypi.org/project/sqlmesh/) by running:
@@ -489,36 +487,16 @@ mkdir sqlmesh-example
 cd sqlmesh-example
 python -m venv .venv
 source .venv/bin/activate
-pip install 'sqlmesh[lsp]' # install the sqlmesh package with extensions to work with VSCode
+pip install sqlmesh
 source .venv/bin/activate # reactivate the venv to ensure you're using the right installation
 sqlmesh init duckdb # get started right away with a local duckdb instance
 sqlmesh plan # see the plan for the changes you're making
 ```
 
-</details>
-
 > Note: You may need to run `python3` or `pip3` instead of `python` or `pip`, depending on your python installation.
 
-<details>
-<summary><b>Windows Installation</b></summary>
-
-```bash
-mkdir sqlmesh-example
-cd sqlmesh-example
-python -m venv .venv
-.\.venv\Scripts\Activate.ps1
-pip install 'sqlmesh[lsp]' # install the sqlmesh package with extensions to work with VSCode
-.\.venv\Scripts\Activate.ps1 # reactivate the venv to ensure you're using the right installation
-sqlmesh init duckdb # get started right away with a local duckdb instance
-sqlmesh plan # see the plan for the changes you're making
-```
-</details>
-
-
 Follow the [quickstart guide](https://sqlmesh.readthedocs.io/en/stable/quickstart/cli/#1-create-the-sqlmesh-project) to learn how to use SQLMesh. You already have a head start!
 
-Follow the [crash course](https://sqlmesh.readthedocs.io/en/stable/examples/sqlmesh_cli_crash_course/) to learn the core movesets and use the easy to reference cheat sheet. 
-
 Follow this [example](https://sqlmesh.readthedocs.io/en/stable/examples/incremental_time_full_walkthrough/) to learn how to use SQLMesh in a full walkthrough.
 
 ## Join Our Community

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/README.md

@@ -2,7 +2,7 @@
   <img src="docs/readme/sqlmesh.png" alt="SQLMesh logo" width="50%" height="50%">
 </p>
 
-SQLMesh is a next-generation data transformation framework designed to ship data quickly, efficiently, and without error. Data teams can run and deploy data transformations written in SQL or Python with visibility and control at any size.
+SQLMesh is a next-generation data transformation framework designed to ship data quickly, efficiently, and without error. Data teams can efficiently run and deploy data transformations written in SQL or Python with visibility and control at any size.
 
 It is more than just a [dbt alternative](https://tobikodata.com/reduce_costs_with_cron_and_partitions.html).
 
@@ -11,10 +11,9 @@ It is more than just a [dbt alternative](https://tobikodata.com/reduce_costs_wit
 </p>
 
 ## Core Features
+<img src="https://github.com/TobikoData/sqlmesh-public-assets/blob/main/sqlmesh_plan_mode.gif?raw=true" alt="SQLMesh Plan Mode">
 
-<img src="https://github.com/TobikoData/sqlmesh-public-assets/blob/main/vscode.gif?raw=true" alt="SQLMesh Plan Mode">
-
-> Get instant SQL impact and context of your changes, both in the CLI and in the [SQLMesh VSCode Extension](https://sqlmesh.readthedocs.io/en/latest/guides/vscode/?h=vs+cod)
+> Get instant SQL impact analysis of your changes, whether in the CLI or in [SQLMesh Plan Mode](https://sqlmesh.readthedocs.io/en/stable/guides/ui/?h=modes#working-with-an-ide)
 
   <details>
   <summary><b>Virtual Data Environments</b></summary>
@@ -121,7 +120,6 @@ outputs:
 * Never build a table [more than once](https://tobikodata.com/simplicity-or-efficiency-how-dbt-makes-you-choose.html)
 * Track what data’s been modified and run only the necessary transformations for [incremental models](https://tobikodata.com/correctly-loading-incremental-data-at-scale.html)
 * Run [unit tests](https://tobikodata.com/we-need-even-greater-expectations.html) for free and configure automated audits
-* Run [table diffs](https://sqlmesh.readthedocs.io/en/stable/examples/sqlmesh_cli_crash_course/?h=crash#run-data-diff-against-prod) between prod and dev based on tables/views impacted by a change 
 
 <details>
 <summary><b>Level Up Your SQL</b></summary>
@@ -133,7 +131,7 @@ Write SQL in any dialect and SQLMesh will transpile it to your target SQL dialec
 * Definitions using [simply SQL](https://sqlmesh.readthedocs.io/en/stable/concepts/models/sql_models/#sql-based-definition) (no need for redundant and confusing `Jinja` + `YAML`)
 * See impact of changes before you run them in your warehouse with column-level lineage
 
-For more information, check out the [website](https://www.tobikodata.com/sqlmesh) and [documentation](https://sqlmesh.readthedocs.io/en/stable/).
+For more information, check out the [website](https://sqlmesh.com) and [documentation](https://sqlmesh.readthedocs.io/en/stable/).
 
 ## Getting Started
 Install SQLMesh through [pypi](https://pypi.org/project/sqlmesh/) by running:
@@ -143,36 +141,16 @@ mkdir sqlmesh-example
 cd sqlmesh-example
 python -m venv .venv
 source .venv/bin/activate
-pip install 'sqlmesh[lsp]' # install the sqlmesh package with extensions to work with VSCode
+pip install sqlmesh
 source .venv/bin/activate # reactivate the venv to ensure you're using the right installation
 sqlmesh init duckdb # get started right away with a local duckdb instance
 sqlmesh plan # see the plan for the changes you're making
 ```
 
-</details>
-
 > Note: You may need to run `python3` or `pip3` instead of `python` or `pip`, depending on your python installation.
 
-<details>
-<summary><b>Windows Installation</b></summary>
-
-```bash
-mkdir sqlmesh-example
-cd sqlmesh-example
-python -m venv .venv
-.\.venv\Scripts\Activate.ps1
-pip install 'sqlmesh[lsp]' # install the sqlmesh package with extensions to work with VSCode
-.\.venv\Scripts\Activate.ps1 # reactivate the venv to ensure you're using the right installation
-sqlmesh init duckdb # get started right away with a local duckdb instance
-sqlmesh plan # see the plan for the changes you're making
-```
-</details>
-
-
 Follow the [quickstart guide](https://sqlmesh.readthedocs.io/en/stable/quickstart/cli/#1-create-the-sqlmesh-project) to learn how to use SQLMesh. You already have a head start!
 
-Follow the [crash course](https://sqlmesh.readthedocs.io/en/stable/examples/sqlmesh_cli_crash_course/) to learn the core movesets and use the easy to reference cheat sheet. 
-
 Follow this [example](https://sqlmesh.readthedocs.io/en/stable/examples/incremental_time_full_walkthrough/) to learn how to use SQLMesh in a full walkthrough.
 
 ## Join Our Community

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/examples/sushi/macros/utils.py

@@ -26,7 +26,6 @@ def sql_literal(
     column_str: str,
     column_quoted: str,
 ):
-    """A macro that accepts various types of SQL literals and returns the column."""
     assert isinstance(column, str)
     assert isinstance(str_lit, str)
     assert str_lit == "'x'"

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/pyproject.toml

@@ -22,7 +22,7 @@ dependencies = [
     "requests",
     "rich[jupyter]",
     "ruamel.yaml",
-    "sqlglot[rs]~=26.24.0",
+    "sqlglot[rs]~=26.21.1",
     "tenacity",
     "time-machine",
     "json-stream"

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.187.6.dev24'
-__version_tuple__ = version_tuple = (0, 187, 6, 'dev24')
+__version__ = version = '0.187.7'
+__version_tuple__ = version_tuple = (0, 187, 7)

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/core/config/root.py

@@ -107,7 +107,6 @@ class Config(BaseConfig):
         physical_schema_mapping: A mapping from regular expressions to names of schemas in which physical tables for corresponding models will be placed.
         environment_suffix_target: Indicates whether to append the environment name to the schema or table name.
         gateway_managed_virtual_layer: Whether the models' views in the virtual layer are created by the model-specific gateway rather than the default gateway.
-        infer_python_dependencies: Whether to statically analyze Python code to automatically infer Python package requirements.
         environment_catalog_mapping: A mapping from regular expressions to catalog names. The catalog name is used to determine the target catalog for a given environment.
         default_target_environment: The name of the environment that will be the default target for the `sqlmesh plan` and `sqlmesh run` commands.
         log_limit: The default number of logs to keep.
@@ -147,7 +146,6 @@ class Config(BaseConfig):
         default=EnvironmentSuffixTarget.default
     )
     gateway_managed_virtual_layer: bool = False
-    infer_python_dependencies: bool = True
     environment_catalog_mapping: RegexKeyDict = {}
     default_target_environment: str = c.PROD
     log_limit: int = c.DEFAULT_LOG_LIMIT

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/core/config/scheduler.py

@@ -47,8 +47,8 @@ class SchedulerConfig(abc.ABC):
         """
 
     @abc.abstractmethod
-    def get_default_catalog_per_gateway(self, context: GenericContext) -> t.Dict[str, str]:
-        """Returns the default catalog for each gateway.
+    def get_default_catalog(self, context: GenericContext) -> t.Optional[str]:
+        """Returns the default catalog for the Scheduler.
 
         Args:
             context: The SQLMesh Context.
@@ -66,7 +66,7 @@ class SchedulerConfig(abc.ABC):
 class _EngineAdapterStateSyncSchedulerConfig(SchedulerConfig):
     def create_state_sync(self, context: GenericContext) -> StateSync:
         state_connection = (
-            context.config.get_state_connection(context.gateway) or context.connection_config
+            context.config.get_state_connection(context.gateway) or context._connection_config
         )
 
         warehouse_connection = context.config.get_connection(context.gateway)
@@ -110,7 +110,7 @@ class _EngineAdapterStateSyncSchedulerConfig(SchedulerConfig):
 
     def state_sync_fingerprint(self, context: GenericContext) -> str:
         state_connection = (
-            context.config.get_state_connection(context.gateway) or context.connection_config
+            context.config.get_state_connection(context.gateway) or context._connection_config
         )
         return md5(
             [
@@ -132,16 +132,12 @@ class BuiltInSchedulerConfig(_EngineAdapterStateSyncSchedulerConfig, BaseConfig)
             state_sync=context.state_sync,
             snapshot_evaluator=context.snapshot_evaluator,
             create_scheduler=context.create_scheduler,
-            default_catalog=context.default_catalog,
+            default_catalog=self.get_default_catalog(context),
             console=context.console,
         )
 
-    def get_default_catalog_per_gateway(self, context: GenericContext) -> t.Dict[str, str]:
-        default_catalogs_per_gateway: t.Dict[str, str] = {}
-        for gateway, adapter in context.engine_adapters.items():
-            if catalog := adapter.default_catalog:
-                default_catalogs_per_gateway[gateway] = catalog
-        return default_catalogs_per_gateway
+    def get_default_catalog(self, context: GenericContext) -> t.Optional[str]:
+        return context.engine_adapter.default_catalog
 
 
 SCHEDULER_CONFIG_TO_TYPE = {

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/core/context.py

@@ -62,9 +62,7 @@ from sqlmesh.core.config import (
     Config,
     load_configs,
 )
-from sqlmesh.core.config.connection import ConnectionConfig
 from sqlmesh.core.config.loader import C
-from sqlmesh.core.config.root import RegexKeyDict
 from sqlmesh.core.console import get_console
 from sqlmesh.core.context_diff import ContextDiff
 from sqlmesh.core.dialect import (
@@ -369,9 +367,6 @@ class GenericContext(BaseContext, t.Generic[C]):
         self._excluded_requirements: t.Set[str] = set()
         self._default_catalog: t.Optional[str] = None
         self._default_catalog_per_gateway: t.Optional[t.Dict[str, str]] = None
-        self._engine_adapter: t.Optional[EngineAdapter] = None
-        self._connection_config: t.Optional[ConnectionConfig] = None
-        self._test_connection_config: t.Optional[ConnectionConfig] = None
         self._linters: t.Dict[str, Linter] = {}
         self._loaded: bool = False
 
@@ -388,6 +383,7 @@ class GenericContext(BaseContext, t.Generic[C]):
         self.pinned_environments = Environment.sanitize_names(self.config.pinned_environments)
         self.auto_categorize_changes = self.config.plan.auto_categorize_changes
         self.selected_gateway = gateway or self.config.default_gateway_name
+        self.gateway_managed_virtual_layer = self.config.gateway_managed_virtual_layer
 
         gw_model_defaults = self.config.gateways[self.selected_gateway].model_defaults
         if gw_model_defaults:
@@ -411,15 +407,24 @@ class GenericContext(BaseContext, t.Generic[C]):
             for path, config in self.configs.items()
         ]
 
-        self._concurrent_tasks = concurrent_tasks
+        self._connection_config = self.config.get_connection(self.gateway)
         self._state_connection_config = (
-            self.config.get_state_connection(self.gateway) or self.connection_config
+            self.config.get_state_connection(self.gateway) or self._connection_config
         )
+        self.concurrent_tasks = concurrent_tasks or self._connection_config.concurrent_tasks
+
+        self._engine_adapters: t.Dict[str, EngineAdapter] = {
+            self.selected_gateway: self._connection_config.create_engine_adapter()
+        }
 
         self._snapshot_evaluator: t.Optional[SnapshotEvaluator] = None
 
         self.console = get_console()
-        setattr(self.console, "dialect", self.config.dialect)
+        setattr(self.console, "dialect", self.engine_adapter.dialect)
+
+        self._test_connection_config = self.config.get_test_connection(
+            self.gateway, self.default_catalog, default_catalog_dialect=self.engine_adapter.DIALECT
+        )
 
         self._provided_state_sync: t.Optional[StateSync] = state_sync
         self._state_sync: t.Optional[StateSync] = None
@@ -430,6 +435,14 @@ class GenericContext(BaseContext, t.Generic[C]):
         self.users = list({user.username: user for user in self.users}.values())
         self._register_notification_targets()
 
+        if (
+            self.config.environment_catalog_mapping
+            and not self.engine_adapter.catalog_support.is_multi_catalog_supported
+        ):
+            raise SQLMeshError(
+                "Environment catalog mapping is only supported for engine adapters that support multiple catalogs"
+            )
+
         if load:
             self.load()
 
@@ -440,9 +453,7 @@ class GenericContext(BaseContext, t.Generic[C]):
     @property
     def engine_adapter(self) -> EngineAdapter:
         """Returns the default engine adapter."""
-        if self._engine_adapter is None:
-            self._engine_adapter = self.connection_config.create_engine_adapter()
-        return self._engine_adapter
+        return self._engine_adapters[self.selected_gateway]
 
     @property
     def snapshot_evaluator(self) -> SnapshotEvaluator:
@@ -921,22 +932,19 @@ class GenericContext(BaseContext, t.Generic[C]):
 
         return snapshot
 
-    def config_for_path(self, path: Path) -> t.Tuple[Config, Path]:
-        """Returns the config and path of the said project for a given file path."""
+    def config_for_path(self, path: Path) -> Config:
         for config_path, config in self.configs.items():
             try:
                 path.relative_to(config_path)
-                return config, config_path
+                return config
             except ValueError:
                 pass
-        return self.config, self.path
+        return self.config
 
     def config_for_node(self, node: str | Model | Audit) -> Config:
         if isinstance(node, str):
-            return self.config_for_path(self.get_snapshot(node, raise_if_missing=True).node._path)[
-                0
-            ]  # type: ignore
-        return self.config_for_path(node._path)[0]  # type: ignore
+            return self.config_for_path(self.get_snapshot(node, raise_if_missing=True).node._path)  # type: ignore
+        return self.config_for_path(node._path)  # type: ignore
 
     @property
     def models(self) -> MappingProxyType[str, Model]:
@@ -969,8 +977,8 @@ class GenericContext(BaseContext, t.Generic[C]):
 
     @property
     def default_catalog(self) -> t.Optional[str]:
-        if self._default_catalog is None and self.default_catalog_per_gateway:
-            self._default_catalog = self.default_catalog_per_gateway[self.selected_gateway]
+        if self._default_catalog is None:
+            self._default_catalog = self._scheduler.get_default_catalog(self)
         return self._default_catalog
 
     @python_api_analytics
@@ -1527,7 +1535,7 @@ class GenericContext(BaseContext, t.Generic[C]):
             allow_destructive_models=expanded_destructive_models,
             environment_ttl=environment_ttl,
             environment_suffix_target=self.config.environment_suffix_target,
-            environment_catalog_mapping=self.environment_catalog_mapping,
+            environment_catalog_mapping=self.config.environment_catalog_mapping,
             categorizer_config=categorizer_config or self.auto_categorize_changes,
             auto_categorization_enabled=not no_auto_categorization,
             effective_from=effective_from,
@@ -1539,6 +1547,7 @@ class GenericContext(BaseContext, t.Generic[C]):
             ),
             end_bounded=not run,
             ensure_finalized_snapshots=self.config.plan.use_finalized_state,
+            engine_schema_differ=self.engine_adapter.SCHEMA_DIFFER,
             interval_end_per_model=max_interval_end_per_model,
             console=self.console,
             user_provided_flags=user_provided_flags,
@@ -1627,7 +1636,7 @@ class GenericContext(BaseContext, t.Generic[C]):
             self.console.show_model_difference_summary(
                 context_diff,
                 EnvironmentNamingInfo.from_environment_catalog_mapping(
-                    self.environment_catalog_mapping,
+                    self.config.environment_catalog_mapping,
                     name=environment,
                     suffix_target=self.config.environment_suffix_target,
                     normalize_name=context_diff.normalize_environment_name,
@@ -1715,7 +1724,7 @@ class GenericContext(BaseContext, t.Generic[C]):
                 if target_snapshot and source_snapshot:
                     if (source_snapshot.fingerprint != target_snapshot.fingerprint) and (
                         (source_snapshot.version != target_snapshot.version)
-                        or source_snapshot.is_forward_only
+                        or (source_snapshot.is_forward_only or target_snapshot.is_forward_only)
                     ):
                         # Compare the virtual layer instead of the physical layer because the virtual layer is guaranteed to point
                         # to the correct/active snapshot for the model in the specified environment, taking into account things like dev previews
@@ -1981,7 +1990,7 @@ class GenericContext(BaseContext, t.Generic[C]):
 
         try:
             model_to_test = self.get_model(model, raise_if_missing=True)
-            test_adapter = self.test_connection_config.create_engine_adapter(
+            test_adapter = self._test_connection_config.create_engine_adapter(
                 register_comments_override=False
             )
 
@@ -2027,7 +2036,7 @@ class GenericContext(BaseContext, t.Generic[C]):
             preserve_fixtures=preserve_fixtures,
             stream=stream,
             default_catalog=self.default_catalog,
-            default_catalog_dialect=self.config.dialect or "",
+            default_catalog_dialect=self.engine_adapter.DIALECT,
         )
 
     @python_api_analytics
@@ -2466,7 +2475,7 @@ class GenericContext(BaseContext, t.Generic[C]):
                 self.console.log_test_results(
                     result,
                     test_output,
-                    self.test_connection_config._engine_adapter.DIALECT,
+                    self._test_connection_config._engine_adapter.DIALECT,
                 )
             if not result.wasSuccessful():
                 raise PlanError(
@@ -2487,7 +2496,7 @@ class GenericContext(BaseContext, t.Generic[C]):
                 if snapshot.version
                 else snapshot.qualified_view_name.for_environment(
                     EnvironmentNamingInfo.from_environment_catalog_mapping(
-                        self.environment_catalog_mapping,
+                        self.config.environment_catalog_mapping,
                         name=c.PROD,
                         suffix_target=self.config.environment_suffix_target,
                     )
@@ -2499,63 +2508,24 @@ class GenericContext(BaseContext, t.Generic[C]):
     @cached_property
     def engine_adapters(self) -> t.Dict[str, EngineAdapter]:
         """Returns all the engine adapters for the gateways defined in the configuration."""
-        adapters: t.Dict[str, EngineAdapter] = {self.selected_gateway: self.engine_adapter}
         for gateway_name in self.config.gateways:
             if gateway_name != self.selected_gateway:
                 connection = self.config.get_connection(gateway_name)
                 adapter = connection.create_engine_adapter(concurrent_tasks=self.concurrent_tasks)
-                adapters[gateway_name] = adapter
-        return adapters
+                self._engine_adapters[gateway_name] = adapter
+        return self._engine_adapters
 
     @cached_property
     def default_catalog_per_gateway(self) -> t.Dict[str, str]:
         """Returns the default catalogs for each engine adapter."""
         if self._default_catalog_per_gateway is None:
-            self._default_catalog_per_gateway = self._scheduler.get_default_catalog_per_gateway(
-                self
-            )
+            self._default_catalog_per_gateway = {
+                name: adapter.default_catalog
+                for name, adapter in self.engine_adapters.items()
+                if adapter.default_catalog
+            }
         return self._default_catalog_per_gateway
 
-    @cached_property
-    def concurrent_tasks(self) -> int:
-        if self._concurrent_tasks is None:
-            self._concurrent_tasks = self.connection_config.concurrent_tasks
-        return self._concurrent_tasks
-
-    @cached_property
-    def connection_config(self) -> ConnectionConfig:
-        if self._connection_config is None:
-            self._connection_config = self.config.get_connection(self.selected_gateway)
-        return self._connection_config
-
-    @cached_property
-    def test_connection_config(self) -> ConnectionConfig:
-        if self._test_connection_config is None:
-            self._test_connection_config = self.config.get_test_connection(
-                self.gateway,
-                self.default_catalog,
-                default_catalog_dialect=self.engine_adapter.DIALECT,
-            )
-        return self._test_connection_config
-
-    @cached_property
-    def environment_catalog_mapping(self) -> RegexKeyDict:
-        engine_adapter = None
-        try:
-            engine_adapter = self.engine_adapter
-        except Exception:
-            pass
-
-        if (
-            self.config.environment_catalog_mapping
-            and engine_adapter
-            and not self.engine_adapter.catalog_support.is_multi_catalog_supported
-        ):
-            raise SQLMeshError(
-                "Environment catalog mapping is only supported for engine adapters that support multiple catalogs"
-            )
-        return self.config.environment_catalog_mapping
-
     def _get_engine_adapter(self, gateway: t.Optional[str] = None) -> EngineAdapter:
         if gateway:
             if adapter := self.engine_adapters.get(gateway):
@@ -2616,8 +2586,7 @@ class GenericContext(BaseContext, t.Generic[C]):
             ensure_finalized_snapshots=ensure_finalized_snapshots,
             diff_rendered=diff_rendered,
             environment_statements=self._environment_statements,
-            gateway_managed_virtual_layer=self.config.gateway_managed_virtual_layer,
-            infer_python_dependencies=self.config.infer_python_dependencies,
+            gateway_managed_virtual_layer=self.gateway_managed_virtual_layer,
         )
 
     def _destroy(self) -> None:

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/core/context_diff.py

@@ -102,7 +102,6 @@ class ContextDiff(PydanticModel):
         diff_rendered: bool = False,
         environment_statements: t.Optional[t.List[EnvironmentStatements]] = [],
         gateway_managed_virtual_layer: bool = False,
-        infer_python_dependencies: bool = True,
     ) -> ContextDiff:
         """Create a ContextDiff object.
 
@@ -117,12 +116,6 @@ class ContextDiff(PydanticModel):
                 the environment is not finalized.
             provided_requirements: Python dependencies sourced from the lock file.
             excluded_requirements: Python dependencies to exclude.
-            diff_rendered: Whether to compute the diff of the rendered version of the compared expressions.
-            environment_statements: A list of `before_all` or `after_all` statements associated with the environment.
-            gateway_managed_virtual_layer: Whether the models' views in the virtual layer are created by the
-                model-specific gateway rather than the default gateway.
-            infer_python_dependencies: Whether to statically analyze Python code to automatically infer Python
-                package requirements.
 
         Returns:
             The ContextDiff object.
@@ -215,7 +208,6 @@ class ContextDiff(PydanticModel):
             provided_requirements or {},
             excluded_requirements or set(),
             snapshots.values(),
-            infer_python_dependencies=infer_python_dependencies,
         )
 
         previous_environment_statements = state_reader.get_environment_statements(environment)
@@ -483,41 +475,29 @@ def _build_requirements(
     provided_requirements: t.Dict[str, str],
     excluded_requirements: t.Set[str],
     snapshots: t.Collection[Snapshot],
-    infer_python_dependencies: bool = True,
 ) -> t.Dict[str, str]:
     requirements = {
         k: v for k, v in provided_requirements.items() if k not in excluded_requirements
     }
-
-    if not infer_python_dependencies:
-        return requirements
-
     distributions = metadata.packages_distributions()
 
     for snapshot in snapshots:
-        if not snapshot.is_model:
-            continue
-
-        for executable in snapshot.model.python_env.values():
-            if executable.kind != "import":
-                continue
-
-            try:
-                start = "from " if executable.payload.startswith("from ") else "import "
-                lib = executable.payload.split(start)[1].split()[0].split(".")[0]
-                if lib not in distributions:
-                    continue
-
-                for dist in distributions[lib]:
-                    if (
-                        dist not in requirements
-                        and dist not in IGNORED_PACKAGES
-                        and dist not in excluded_requirements
-                    ):
-                        requirements[dist] = metadata.version(dist)
-            except metadata.PackageNotFoundError:
-                from sqlmesh.core.console import get_console
-
-                get_console().log_warning(f"Failed to find package for {lib}.")
-
+        if snapshot.is_model:
+            for executable in snapshot.model.python_env.values():
+                if executable.kind == "import":
+                    try:
+                        start = "from " if executable.payload.startswith("from ") else "import "
+                        lib = executable.payload.split(start)[1].split()[0].split(".")[0]
+                        if lib in distributions:
+                            for dist in distributions[lib]:
+                                if (
+                                    dist not in requirements
+                                    and dist not in IGNORED_PACKAGES
+                                    and dist not in excluded_requirements
+                                ):
+                                    requirements[dist] = metadata.version(dist)
+                    except metadata.PackageNotFoundError:
+                        from sqlmesh.core.console import get_console
+
+                        get_console().log_warning(f"Failed to find package for {lib}.")
     return requirements

{sqlmesh-0.187.6.dev24 → sqlmesh-0.187.7}/sqlmesh/core/model/cache.py

@@ -142,7 +142,6 @@ class OptimizedQueryCache:
 
     def _put(self, name: str, model: SqlModel) -> None:
         optimized_query = model.render_query()
-
         new_entry = OptimizedQueryCacheEntry(
             optimized_rendered_query=optimized_query,
             renderer_violations=model.violated_rules_for_query,
@@ -181,17 +180,10 @@ def load_optimized_query(
     assert _optimized_query_cache
     model, snapshot_id = model_snapshot_id
 
-    entry_name = None
-
     if isinstance(model, SqlModel):
-        try:
-            entry_name = _optimized_query_cache.put(model)
-        except:
-            # this can happen if there is a query rendering error.
-            # for example, the model query references some python library or function that was available
-            # at the time the model was created but has since been removed locally
-            logger.exception(f"Failed to cache optimized query for model '{model.name}'")
-
+        entry_name = _optimized_query_cache.put(model)
+    else:
+        entry_name = None
     return snapshot_id, entry_name