dbt-sqlserver 1.11.1__tar.gz → 1.11.2rc2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dbt_sqlserver-1.11.1/dbt_sqlserver.egg-info → dbt_sqlserver-1.11.2rc2}/PKG-INFO +108 -7
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/README.md +107 -6
- dbt_sqlserver-1.11.2rc2/dbt/adapters/sqlserver/__version__.py +1 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_adapter.py +258 -62
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/metadata.sql +49 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/schema.sql +34 -9
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/incremental/incremental.sql +4 -0
- dbt_sqlserver-1.11.2rc2/dbt/include/sqlserver/macros/materializations/models/table/columns_spec_ddl.sql +79 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/table/table.sql +4 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/table/table_dml_refresh.sql +35 -1
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/unit_test/unit_test_create_table_as.sql +5 -1
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/view/view.sql +9 -2
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/tests.sql +3 -6
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/unit_tests.sql +2 -5
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/relations/table/create.sql +22 -8
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2/dbt_sqlserver.egg-info}/PKG-INFO +108 -7
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/pyproject.toml +27 -0
- dbt_sqlserver-1.11.1/dbt/adapters/sqlserver/__version__.py +0 -1
- dbt_sqlserver-1.11.1/dbt/include/sqlserver/macros/materializations/models/table/columns_spec_ddl.sql +0 -30
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/LICENSE +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/__init__.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/__init__.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/py.typed +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/relation_configs/__init__.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/relation_configs/index.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/relation_configs/policies.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_auth.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_backend.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_column.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_configs.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_connections.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_constants.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_credentials.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_helpers.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_mask.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_relation.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/adapters/sqlserver/sqlserver_runtime.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/__init__.py +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/dbt_project.yml +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/apply_grants.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/apply_masks.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/catalog.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/columns.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/indexes.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/persist_docs.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/relation.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/show.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/adapters/validate_sql.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/functions/helpers.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/functions/scalar.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/hooks.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/incremental/incremental_strategies.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/incremental/merge.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/unit_test/get_fixture_sql.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/models/view/create_view_as.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/snapshots/helpers.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/snapshots/snapshot.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/snapshots/snapshot_merge.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/materializations/snapshots/strategies.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/relations/seeds/helpers.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/relations/table/clone.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/relations/views/create.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/any_value.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/array_construct.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/cast_bool_to_text.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/concat.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/date_trunc.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/dateadd.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/get_tables_by_pattern.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/hash.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/last_day.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/length.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/listagg.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/position.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/safe_cast.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/split_part.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/macros/utils/timestamps.sql +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt/include/sqlserver/profile_template.yml +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt_sqlserver.egg-info/SOURCES.txt +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt_sqlserver.egg-info/dependency_links.txt +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt_sqlserver.egg-info/requires.txt +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/dbt_sqlserver.egg-info/top_level.txt +0 -0
- {dbt_sqlserver-1.11.1 → dbt_sqlserver-1.11.2rc2}/setup.cfg +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: dbt-sqlserver
|
|
3
|
-
Version: 1.11.
|
|
3
|
+
Version: 1.11.2rc2
|
|
4
4
|
Summary: A Microsoft SQL Server adapter plugin for dbt
|
|
5
5
|
Author: Mikael Ene, Anders Swanson, Sam Debruyn, Cor Zuurmond, Cody Scott
|
|
6
6
|
License: MIT
|
|
@@ -157,7 +157,7 @@ your_profile:
|
|
|
157
157
|
|
|
158
158
|
## Changelog
|
|
159
159
|
|
|
160
|
-
See [the changelog](CHANGELOG.md)
|
|
160
|
+
See [the changelog](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/CHANGELOG.md)
|
|
161
161
|
|
|
162
162
|
## Configuration
|
|
163
163
|
|
|
@@ -204,7 +204,7 @@ Safe expansions are further gated by `column_type_expansion_max_rows` (default 1
|
|
|
204
204
|
|
|
205
205
|
### `dbt_sqlserver_use_dbt_transactions`
|
|
206
206
|
|
|
207
|
-
_(default: `false`)_ When enabled, makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.
|
|
207
|
+
_(default: `false`)_ When enabled, makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.
|
|
208
208
|
|
|
209
209
|
The default is `false`, preserving existing behavior where `begin`/`commit` hooks are logical no-ops and the ODBC driver auto-commits each statement. When `dbt_sqlserver_use_dbt_transactions: true`, the adapter emits real T-SQL transaction statements, and rollback uses `IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION`.
|
|
210
210
|
|
|
@@ -212,10 +212,11 @@ The driver connection remains in autocommit mode (`autocommit=true`) in both mod
|
|
|
212
212
|
|
|
213
213
|
This mode is opt-in and should be tested carefully with project-specific materializations and hooks.
|
|
214
214
|
|
|
215
|
+
**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.
|
|
216
|
+
|
|
215
217
|
```yaml
|
|
216
218
|
# dbt_project.yml
|
|
217
219
|
flags:
|
|
218
|
-
dbt_sqlserver_enable_safe_type_expansion: true
|
|
219
220
|
dbt_sqlserver_use_dbt_transactions: true # <-- opt-in; default is false
|
|
220
221
|
```
|
|
221
222
|
|
|
@@ -258,8 +259,6 @@ your_profile:
|
|
|
258
259
|
prefer_single_alter_column=true) }}
|
|
259
260
|
```
|
|
260
261
|
|
|
261
|
-
**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.
|
|
262
|
-
|
|
263
262
|
### `as_columnstore`
|
|
264
263
|
|
|
265
264
|
*(default: `true`)* When building a table, the adapter creates a [clustered columnstore index](https://learn.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview) (CCI) on it. Set `as_columnstore: false` to build a plain rowstore table instead.
|
|
@@ -283,6 +282,108 @@ You can also set it per model:
|
|
|
283
282
|
{{ config(materialized="table", as_columnstore=false) }}
|
|
284
283
|
```
|
|
285
284
|
|
|
285
|
+
With `table_refresh_method: dml`, a schema change makes the refresh fall back to a rename-swap. On that run — and only that run — the scratch table is rebuilt the way this adapter builds every other table, so it carries the model's columnstore index, and under an enforced contract its `NOT NULL`s and inline constraints, into the swap. That run therefore executes the model's SQL twice — once for the `SELECT … INTO` that probes for the schema change, once for the rebuild — and builds the columnstore index once. Steady-state refreshes are unaffected and keep the single `SELECT … INTO`. A table that lost its columnstore index to this bug before you upgraded is not repaired automatically: its schema still matches, so it stays on the cheap path. To rebuild it, temporarily set `full_refresh_build: prebuilt` and run with `--full-refresh`.
|
|
286
|
+
|
|
287
|
+
### Constraints
|
|
288
|
+
|
|
289
|
+
Constraints declared in a model's yaml are applied when — and only when — the model's [contract](https://docs.getdbt.com/reference/resource-configs/contract) is enforced, which is what every dbt adapter does and keeps their cost opt-in. (dbt-core does not raise if you declare constraints with the contract off — they are simply never emitted.) `not_null`, `check`, `unique`, `primary_key` and `foreign_key` are all supported.
|
|
290
|
+
|
|
291
|
+
**Where a constraint lands depends on whether you name it.**
|
|
292
|
+
|
|
293
|
+
An unnamed constraint is rendered inline in the `CREATE TABLE` column list and SQL Server names it (`PK__my_model__3213E83F…`). It is validated as the table is built, so a violation fails before the new table is swapped in and the previous one is left untouched.
|
|
294
|
+
|
|
295
|
+
A *model-level* constraint carrying `name:` is applied by `ALTER TABLE … ADD CONSTRAINT` after the build swaps the new table into place and drops the old one. SQL Server scopes constraint names per schema, and a table is built alongside the one it replaces, so that is the first moment the name is free to reuse — naming a constraint inline would collide with the outgoing table (`Msg 2714`) on every rebuild after the first. The trade-off is that this runs after the model has committed: if the data violates the constraint, the model fails with the table already in place but unconstrained, and — as with any failure this late in a build — `post_hook`s declared with `transaction: false` do not run.
|
|
296
|
+
|
|
297
|
+
Name a constraint when you want it stable across environments (schema-comparison tools report the generated names as differences) or need to reference it later. A `name:` on a *column-level* constraint is ignored with a warning — declare it under the model's `constraints:` key instead.
|
|
298
|
+
|
|
299
|
+
The full set of rules, with the test that verifies each, is in [docs/constraints.md](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/docs/constraints.md).
|
|
300
|
+
|
|
301
|
+
```yaml
|
|
302
|
+
models:
|
|
303
|
+
- name: fact_sales
|
|
304
|
+
config:
|
|
305
|
+
contract:
|
|
306
|
+
enforced: true
|
|
307
|
+
constraints:
|
|
308
|
+
# named: applied by ALTER TABLE after the swap
|
|
309
|
+
- type: primary_key
|
|
310
|
+
name: PK_fact_sales
|
|
311
|
+
columns: [sale_id]
|
|
312
|
+
- type: foreign_key
|
|
313
|
+
name: FK_fact_sales_customer
|
|
314
|
+
columns: [customer_id]
|
|
315
|
+
to: ref('dim_customer')
|
|
316
|
+
to_columns: [customer_id]
|
|
317
|
+
# unnamed: rendered into the CREATE TABLE
|
|
318
|
+
- type: check
|
|
319
|
+
expression: amount >= 0
|
|
320
|
+
columns:
|
|
321
|
+
- name: sale_id
|
|
322
|
+
data_type: int
|
|
323
|
+
constraints:
|
|
324
|
+
- type: not_null
|
|
325
|
+
- name: customer_id
|
|
326
|
+
data_type: int
|
|
327
|
+
constraints:
|
|
328
|
+
- type: not_null
|
|
329
|
+
- name: amount
|
|
330
|
+
data_type: decimal(18,2)
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
#### Clustering
|
|
334
|
+
|
|
335
|
+
`primary_key` and `unique` are emitted as `NONCLUSTERED` by default so they can coexist with the clustered columnstore index built for [`as_columnstore`](#as_columnstore). Use dbt's own `expression` field to ask for something else:
|
|
336
|
+
|
|
337
|
+
```yaml
|
|
338
|
+
constraints:
|
|
339
|
+
- type: primary_key
|
|
340
|
+
name: PK_fact_sales
|
|
341
|
+
columns: [sale_id]
|
|
342
|
+
expression: clustered # requires as_columnstore: false
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
`clustered` and `nonclustered` are the only values understood here. `expression` is free text that dbt splices between the keyword and the column list, which is the one place T-SQL accepts nothing else — index options such as `with (fillfactor = 90)` belong on a separate index, not on the constraint.
|
|
346
|
+
|
|
347
|
+
#### Foreign keys
|
|
348
|
+
|
|
349
|
+
Two things are worth knowing before adding them:
|
|
350
|
+
|
|
351
|
+
- **They are not free at build time.** Every load is validated against them; add them where you want the guarantee, not everywhere the relationship exists.
|
|
352
|
+
- **A foreign key pointing at a model blocks that model's rebuild.** The build renames the outgoing table to a backup and drops it, but the child's foreign key follows the renamed object, so the drop fails with `Msg 3726`. SQL Server has no `DROP TABLE … CASCADE`.
|
|
353
|
+
|
|
354
|
+
The adapter ships a macro for exactly this, meant as a `pre_hook` on the **referenced** (parent) model:
|
|
355
|
+
|
|
356
|
+
```sql
|
|
357
|
+
{{ config(pre_hook="{{ drop_fk_constraints() }}") }}
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
It drops the foreign keys in both directions — the inbound ones other tables hold against this model, and this model's own outbound ones — so the rebuild's backup drop succeeds. The trade-off is explicit and worth stating: **the child's foreign key does not exist between the parent's rebuild and the child's next build.** (dbt-postgres makes the same trade silently, by issuing every `drop table` with `cascade`.)
|
|
361
|
+
|
|
362
|
+
Add the hook *before* the first rebuild. Once a rebuild has failed with `Msg 3726`, the new table is already in place but without its own named constraints — those are applied after the backup drop that failed — and the child's key now points at `<model>__dbt_backup`, because a foreign key follows the object, not the name. A plain `dbt run` does not recover: the parent trips over the same backup and the child is skipped behind it. Rebuild the child alone — that run fails too, since the parent has no key to reference, but its swap drops the old child and the stale key with it — and a plain `dbt run` then rebuilds both in order. Dropping the child's key by hand (`ALTER TABLE <child> DROP CONSTRAINT <name>`) does the same.
|
|
363
|
+
|
|
364
|
+
`table_refresh_method: dml` is *not* a workaround. Its steady-state refresh issues `DELETE FROM <parent>`, which fails with `Msg 547` as soon as the child holds referencing rows, and its schema-change path falls back to the same rename-swap, hitting `Msg 3726` anyway.
|
|
365
|
+
|
|
366
|
+
- **SQL Server has no cross-database foreign keys.** `to: ref(...)` resolves to a fully qualified relation, database included, which SQL Server accepts as long as it names the current database. A target in another database fails with `references invalid table`.
|
|
367
|
+
|
|
368
|
+
A foreign key also does not order the build on its own: add an explicit `-- depends_on: {{ ref('dim_customer') }}` to the child model so the parent is built first.
|
|
369
|
+
|
|
370
|
+
#### Changing a constraint after the first build
|
|
371
|
+
|
|
372
|
+
Named constraints are applied by an `ALTER TABLE` whose `ADD` is guarded on the name already being present on the table, so:
|
|
373
|
+
|
|
374
|
+
- **Adding** a constraint to a model that already exists lands on its next run — no `--full-refresh` needed, on `table` and `incremental` alike.
|
|
375
|
+
- **Changing** an existing constraint's definition while keeping its name is *not* detected: a constraint name, unlike a `dbt_idx_` index name, says nothing about what the constraint does. Run `--full-refresh` to apply the new definition; that rebuilds the table, so the constraint is created fresh. (A `table` model rebuilds on every run and needs nothing special.)
|
|
376
|
+
- **Renaming** a constraint on a table that persists across runs adds the new name beside the old one. For a `check`, `unique` or `foreign_key` that means a duplicate; for a `primary_key` the run fails with `Msg 1779` (*table already has a primary key defined on it*) until the old one is dropped. Rename with `--full-refresh`.
|
|
377
|
+
- **Removing** a constraint from the yaml does not drop it from the database. Drop it yourself, or `--full-refresh`.
|
|
378
|
+
|
|
379
|
+
Every bullet above is about *named* constraints. Unnamed ones ride the `CREATE TABLE`, so they follow the table and change only when the table is rebuilt — on a materialization whose table persists across runs (`incremental` in its steady state, `table_refresh_method: dml`), adding an unnamed constraint to an existing model does nothing until a `--full-refresh`, and the run still reports success. Name it, or full-refresh.
|
|
380
|
+
|
|
381
|
+
#### Build-shape notes
|
|
382
|
+
|
|
383
|
+
An *unnamed* `primary_key` or `unique` constraint on a column that also carries a [data mask](#dynamic-data-masking-masked_with--masks) is rejected by the build. The constraint rides the `CREATE TABLE`, so its index already exists by the time `apply_masks` runs, and the adapter refuses to mask any column that an index has as a key: *is configured for masking but is also an index key column*. Declare that constraint at the model level **with a `name:`** instead — named constraints are applied by `ALTER TABLE` after the masks are in place, which the adapter allows.
|
|
384
|
+
|
|
385
|
+
A contract-enforced model is always loaded as `CREATE TABLE` followed by `INSERT … WITH (TABLOCK)`, on every build path. A `primary_key` or `unique` constraint — named or not — puts a nonclustered index on that table *before* the load, so the index is maintained row by row while the rows go in, and that part of the load is fully logged where an index-free heap would have been minimally logged. The cost lands on every rebuild of the model; it is most visible with `full_refresh_build: prebuilt`, whose point is a cheap bulk load. If a model's load time matters, weigh the key constraints against it; `check`, `not_null` and `foreign_key` do not create indexes and do not carry this cost.
|
|
386
|
+
|
|
286
387
|
### Dynamic Data Masking (`masked_with` / `masks`)
|
|
287
388
|
|
|
288
389
|
The adapter can apply SQL Server [Dynamic Data Masking](https://learn.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking) (DDM) to columns as part of the materialization, so masks are re-applied on every build and survive dbt's drop-and-recreate on a full refresh. A principal granted `SELECT` but not `UNMASK` then sees masked values instead of real data (dbt's own build principal, being `db_owner`, keeps `UNMASK` and reads real data). Requires **SQL Server 2016+**.
|
|
@@ -337,7 +438,7 @@ Behaviour:
|
|
|
337
438
|
|
|
338
439
|
This adapter is community-maintained.
|
|
339
440
|
You are welcome to contribute by creating issues, opening or reviewing pull requests, or helping other users in the Slack channel.
|
|
340
|
-
If you're unsure how to get started, check out our [contributing guide](CONTRIBUTING.md).
|
|
441
|
+
If you're unsure how to get started, check out our [contributing guide](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/CONTRIBUTING.md).
|
|
341
442
|
|
|
342
443
|
## License
|
|
343
444
|
|
|
@@ -120,7 +120,7 @@ your_profile:
|
|
|
120
120
|
|
|
121
121
|
## Changelog
|
|
122
122
|
|
|
123
|
-
See [the changelog](CHANGELOG.md)
|
|
123
|
+
See [the changelog](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/CHANGELOG.md)
|
|
124
124
|
|
|
125
125
|
## Configuration
|
|
126
126
|
|
|
@@ -167,7 +167,7 @@ Safe expansions are further gated by `column_type_expansion_max_rows` (default 1
|
|
|
167
167
|
|
|
168
168
|
### `dbt_sqlserver_use_dbt_transactions`
|
|
169
169
|
|
|
170
|
-
_(default: `false`)_ When enabled, makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.
|
|
170
|
+
_(default: `false`)_ When enabled, makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.
|
|
171
171
|
|
|
172
172
|
The default is `false`, preserving existing behavior where `begin`/`commit` hooks are logical no-ops and the ODBC driver auto-commits each statement. When `dbt_sqlserver_use_dbt_transactions: true`, the adapter emits real T-SQL transaction statements, and rollback uses `IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION`.
|
|
173
173
|
|
|
@@ -175,10 +175,11 @@ The driver connection remains in autocommit mode (`autocommit=true`) in both mod
|
|
|
175
175
|
|
|
176
176
|
This mode is opt-in and should be tested carefully with project-specific materializations and hooks.
|
|
177
177
|
|
|
178
|
+
**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.
|
|
179
|
+
|
|
178
180
|
```yaml
|
|
179
181
|
# dbt_project.yml
|
|
180
182
|
flags:
|
|
181
|
-
dbt_sqlserver_enable_safe_type_expansion: true
|
|
182
183
|
dbt_sqlserver_use_dbt_transactions: true # <-- opt-in; default is false
|
|
183
184
|
```
|
|
184
185
|
|
|
@@ -221,8 +222,6 @@ your_profile:
|
|
|
221
222
|
prefer_single_alter_column=true) }}
|
|
222
223
|
```
|
|
223
224
|
|
|
224
|
-
**Compatibility notes:** Enabling `dbt_sqlserver_use_dbt_transactions: true` may expose transaction-state assumptions hidden by autocommit-only mode. Explicit transaction macros may interact with dbt-managed transactions, and cleanup after failed DDL/DML may differ. Review pre/post hooks for in-transaction vs out-of-transaction semantics.
|
|
225
|
-
|
|
226
225
|
### `as_columnstore`
|
|
227
226
|
|
|
228
227
|
*(default: `true`)* When building a table, the adapter creates a [clustered columnstore index](https://learn.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview) (CCI) on it. Set `as_columnstore: false` to build a plain rowstore table instead.
|
|
@@ -246,6 +245,108 @@ You can also set it per model:
|
|
|
246
245
|
{{ config(materialized="table", as_columnstore=false) }}
|
|
247
246
|
```
|
|
248
247
|
|
|
248
|
+
With `table_refresh_method: dml`, a schema change makes the refresh fall back to a rename-swap. On that run — and only that run — the scratch table is rebuilt the way this adapter builds every other table, so it carries the model's columnstore index, and under an enforced contract its `NOT NULL`s and inline constraints, into the swap. That run therefore executes the model's SQL twice — once for the `SELECT … INTO` that probes for the schema change, once for the rebuild — and builds the columnstore index once. Steady-state refreshes are unaffected and keep the single `SELECT … INTO`. A table that lost its columnstore index to this bug before you upgraded is not repaired automatically: its schema still matches, so it stays on the cheap path. To rebuild it, temporarily set `full_refresh_build: prebuilt` and run with `--full-refresh`.
|
|
249
|
+
|
|
250
|
+
### Constraints
|
|
251
|
+
|
|
252
|
+
Constraints declared in a model's yaml are applied when — and only when — the model's [contract](https://docs.getdbt.com/reference/resource-configs/contract) is enforced, which is what every dbt adapter does and keeps their cost opt-in. (dbt-core does not raise if you declare constraints with the contract off — they are simply never emitted.) `not_null`, `check`, `unique`, `primary_key` and `foreign_key` are all supported.
|
|
253
|
+
|
|
254
|
+
**Where a constraint lands depends on whether you name it.**
|
|
255
|
+
|
|
256
|
+
An unnamed constraint is rendered inline in the `CREATE TABLE` column list and SQL Server names it (`PK__my_model__3213E83F…`). It is validated as the table is built, so a violation fails before the new table is swapped in and the previous one is left untouched.
|
|
257
|
+
|
|
258
|
+
A *model-level* constraint carrying `name:` is applied by `ALTER TABLE … ADD CONSTRAINT` after the build swaps the new table into place and drops the old one. SQL Server scopes constraint names per schema, and a table is built alongside the one it replaces, so that is the first moment the name is free to reuse — naming a constraint inline would collide with the outgoing table (`Msg 2714`) on every rebuild after the first. The trade-off is that this runs after the model has committed: if the data violates the constraint, the model fails with the table already in place but unconstrained, and — as with any failure this late in a build — `post_hook`s declared with `transaction: false` do not run.
|
|
259
|
+
|
|
260
|
+
Name a constraint when you want it stable across environments (schema-comparison tools report the generated names as differences) or need to reference it later. A `name:` on a *column-level* constraint is ignored with a warning — declare it under the model's `constraints:` key instead.
|
|
261
|
+
|
|
262
|
+
The full set of rules, with the test that verifies each, is in [docs/constraints.md](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/docs/constraints.md).
|
|
263
|
+
|
|
264
|
+
```yaml
|
|
265
|
+
models:
|
|
266
|
+
- name: fact_sales
|
|
267
|
+
config:
|
|
268
|
+
contract:
|
|
269
|
+
enforced: true
|
|
270
|
+
constraints:
|
|
271
|
+
# named: applied by ALTER TABLE after the swap
|
|
272
|
+
- type: primary_key
|
|
273
|
+
name: PK_fact_sales
|
|
274
|
+
columns: [sale_id]
|
|
275
|
+
- type: foreign_key
|
|
276
|
+
name: FK_fact_sales_customer
|
|
277
|
+
columns: [customer_id]
|
|
278
|
+
to: ref('dim_customer')
|
|
279
|
+
to_columns: [customer_id]
|
|
280
|
+
# unnamed: rendered into the CREATE TABLE
|
|
281
|
+
- type: check
|
|
282
|
+
expression: amount >= 0
|
|
283
|
+
columns:
|
|
284
|
+
- name: sale_id
|
|
285
|
+
data_type: int
|
|
286
|
+
constraints:
|
|
287
|
+
- type: not_null
|
|
288
|
+
- name: customer_id
|
|
289
|
+
data_type: int
|
|
290
|
+
constraints:
|
|
291
|
+
- type: not_null
|
|
292
|
+
- name: amount
|
|
293
|
+
data_type: decimal(18,2)
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
#### Clustering
|
|
297
|
+
|
|
298
|
+
`primary_key` and `unique` are emitted as `NONCLUSTERED` by default so they can coexist with the clustered columnstore index built for [`as_columnstore`](#as_columnstore). Use dbt's own `expression` field to ask for something else:
|
|
299
|
+
|
|
300
|
+
```yaml
|
|
301
|
+
constraints:
|
|
302
|
+
- type: primary_key
|
|
303
|
+
name: PK_fact_sales
|
|
304
|
+
columns: [sale_id]
|
|
305
|
+
expression: clustered # requires as_columnstore: false
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
`clustered` and `nonclustered` are the only values understood here. `expression` is free text that dbt splices between the keyword and the column list, which is the one place T-SQL accepts nothing else — index options such as `with (fillfactor = 90)` belong on a separate index, not on the constraint.
|
|
309
|
+
|
|
310
|
+
#### Foreign keys
|
|
311
|
+
|
|
312
|
+
Two things are worth knowing before adding them:
|
|
313
|
+
|
|
314
|
+
- **They are not free at build time.** Every load is validated against them; add them where you want the guarantee, not everywhere the relationship exists.
|
|
315
|
+
- **A foreign key pointing at a model blocks that model's rebuild.** The build renames the outgoing table to a backup and drops it, but the child's foreign key follows the renamed object, so the drop fails with `Msg 3726`. SQL Server has no `DROP TABLE … CASCADE`.
|
|
316
|
+
|
|
317
|
+
The adapter ships a macro for exactly this, meant as a `pre_hook` on the **referenced** (parent) model:
|
|
318
|
+
|
|
319
|
+
```sql
|
|
320
|
+
{{ config(pre_hook="{{ drop_fk_constraints() }}") }}
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
It drops the foreign keys in both directions — the inbound ones other tables hold against this model, and this model's own outbound ones — so the rebuild's backup drop succeeds. The trade-off is explicit and worth stating: **the child's foreign key does not exist between the parent's rebuild and the child's next build.** (dbt-postgres makes the same trade silently, by issuing every `drop table` with `cascade`.)
|
|
324
|
+
|
|
325
|
+
Add the hook *before* the first rebuild. Once a rebuild has failed with `Msg 3726`, the new table is already in place but without its own named constraints — those are applied after the backup drop that failed — and the child's key now points at `<model>__dbt_backup`, because a foreign key follows the object, not the name. A plain `dbt run` does not recover: the parent trips over the same backup and the child is skipped behind it. Rebuild the child alone — that run fails too, since the parent has no key to reference, but its swap drops the old child and the stale key with it — and a plain `dbt run` then rebuilds both in order. Dropping the child's key by hand (`ALTER TABLE <child> DROP CONSTRAINT <name>`) does the same.
|
|
326
|
+
|
|
327
|
+
`table_refresh_method: dml` is *not* a workaround. Its steady-state refresh issues `DELETE FROM <parent>`, which fails with `Msg 547` as soon as the child holds referencing rows, and its schema-change path falls back to the same rename-swap, hitting `Msg 3726` anyway.
|
|
328
|
+
|
|
329
|
+
- **SQL Server has no cross-database foreign keys.** `to: ref(...)` resolves to a fully qualified relation, database included, which SQL Server accepts as long as it names the current database. A target in another database fails with `references invalid table`.
|
|
330
|
+
|
|
331
|
+
A foreign key also does not order the build on its own: add an explicit `-- depends_on: {{ ref('dim_customer') }}` to the child model so the parent is built first.
|
|
332
|
+
|
|
333
|
+
#### Changing a constraint after the first build
|
|
334
|
+
|
|
335
|
+
Named constraints are applied by an `ALTER TABLE` whose `ADD` is guarded on the name already being present on the table, so:
|
|
336
|
+
|
|
337
|
+
- **Adding** a constraint to a model that already exists lands on its next run — no `--full-refresh` needed, on `table` and `incremental` alike.
|
|
338
|
+
- **Changing** an existing constraint's definition while keeping its name is *not* detected: a constraint name, unlike a `dbt_idx_` index name, says nothing about what the constraint does. Run `--full-refresh` to apply the new definition; that rebuilds the table, so the constraint is created fresh. (A `table` model rebuilds on every run and needs nothing special.)
|
|
339
|
+
- **Renaming** a constraint on a table that persists across runs adds the new name beside the old one. For a `check`, `unique` or `foreign_key` that means a duplicate; for a `primary_key` the run fails with `Msg 1779` (*table already has a primary key defined on it*) until the old one is dropped. Rename with `--full-refresh`.
|
|
340
|
+
- **Removing** a constraint from the yaml does not drop it from the database. Drop it yourself, or `--full-refresh`.
|
|
341
|
+
|
|
342
|
+
Every bullet above is about *named* constraints. Unnamed ones ride the `CREATE TABLE`, so they follow the table and change only when the table is rebuilt — on a materialization whose table persists across runs (`incremental` in its steady state, `table_refresh_method: dml`), adding an unnamed constraint to an existing model does nothing until a `--full-refresh`, and the run still reports success. Name it, or full-refresh.
|
|
343
|
+
|
|
344
|
+
#### Build-shape notes
|
|
345
|
+
|
|
346
|
+
An *unnamed* `primary_key` or `unique` constraint on a column that also carries a [data mask](#dynamic-data-masking-masked_with--masks) is rejected by the build. The constraint rides the `CREATE TABLE`, so its index already exists by the time `apply_masks` runs, and the adapter refuses to mask any column that an index has as a key: *is configured for masking but is also an index key column*. Declare that constraint at the model level **with a `name:`** instead — named constraints are applied by `ALTER TABLE` after the masks are in place, which the adapter allows.
|
|
347
|
+
|
|
348
|
+
A contract-enforced model is always loaded as `CREATE TABLE` followed by `INSERT … WITH (TABLOCK)`, on every build path. A `primary_key` or `unique` constraint — named or not — puts a nonclustered index on that table *before* the load, so the index is maintained row by row while the rows go in, and that part of the load is fully logged where an index-free heap would have been minimally logged. The cost lands on every rebuild of the model; it is most visible with `full_refresh_build: prebuilt`, whose point is a cheap bulk load. If a model's load time matters, weigh the key constraints against it; `check`, `not_null` and `foreign_key` do not create indexes and do not carry this cost.
|
|
349
|
+
|
|
249
350
|
### Dynamic Data Masking (`masked_with` / `masks`)
|
|
250
351
|
|
|
251
352
|
The adapter can apply SQL Server [Dynamic Data Masking](https://learn.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking) (DDM) to columns as part of the materialization, so masks are re-applied on every build and survive dbt's drop-and-recreate on a full refresh. A principal granted `SELECT` but not `UNMASK` then sees masked values instead of real data (dbt's own build principal, being `db_owner`, keeps `UNMASK` and reads real data). Requires **SQL Server 2016+**.
|
|
@@ -300,7 +401,7 @@ Behaviour:
|
|
|
300
401
|
|
|
301
402
|
This adapter is community-maintained.
|
|
302
403
|
You are welcome to contribute by creating issues, opening or reviewing pull requests, or helping other users in the Slack channel.
|
|
303
|
-
If you're unsure how to get started, check out our [contributing guide](CONTRIBUTING.md).
|
|
404
|
+
If you're unsure how to get started, check out our [contributing guide](https://github.com/dbt-msft/dbt-sqlserver/blob/release/v1.11/CONTRIBUTING.md).
|
|
304
405
|
|
|
305
406
|
## License
|
|
306
407
|
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
version = "1.11.2rc2"
|