sandwich 0.1.0__py3-none-any.whl → 0.2.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. sandwich-0.2.1.dist-info/METADATA +164 -0
  2. sandwich-0.2.1.dist-info/RECORD +5 -0
  3. sandwich-0.1.0.dist-info/METADATA +0 -16
  4. sandwich-0.1.0.dist-info/RECORD +0 -5
  5. {sandwich-0.1.0.dist-info → sandwich-0.2.1.dist-info}/WHEEL +0 -0
  6. {sandwich-0.1.0.dist-info → sandwich-0.2.1.dist-info}/licenses/LICENSE +0 -0
sandwich-0.2.1.dist-info/METADATA ADDED
@@ -0,0 +1,164 @@
1
+ Metadata-Version: 2.4
2
+ Name: sandwich
3
+ Version: 0.2.1
4
+ Summary: DataVault 2.0 code gen
5
+ Author-email: Andrey Morozov <andrey@morozov.lv>
6
+ License-File: LICENSE
7
+ Keywords: DWH,Data Vault 2.0
8
+ Classifier: Development Status :: 1 - Planning
9
+ Classifier: Environment :: Console
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Operating System :: OS Independent
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Typing :: Typed
15
+ Requires-Python: >=3.14
16
+ Requires-Dist: sqlalchemy
17
+ Description-Content-Type: text/markdown
18
+
19
+ ## Data Vault 2.0 scaffolding tool
20
+ This tool is designed to streamline the process of creating Data Vault 2.0 entities,
21
+ such as hubs, links, and satellites.
22
+ As well as building information layer objects such as dim and fact tables
23
+ from a multidimensional paradigm.
24
+
25
+ ### How it works:
26
+ User: provides a staging view `stg.[entity_name]` (or a table if the staging layer persisted)
27
+ with all requirements for the `[entity_name]` defined in the schema (how to define see below).
28
+ Tool:
29
+ 1. Validates metadata of the provided staging view or table.
30
+ 2. Generates the necessary DDL statements to create the Data Vault 2.0 entities.
31
+ 3. Generates ELT procedures to load data to the generated entities.
32
+ 4. Generates support procedures such as `meta.Drop_all_related_to_[entity_name]` and `elt.Run_all_related_to_[entity_name]`
33
+
34
+ ```text
35
+ +----------------------+
36
+ | hub.[entity_name] |
37
+ +----------------------+
38
+ ^
39
+ o 1.define +-------------------+ | 3.create
40
+ /|\ -------> | stg.[entity_name] | # +----------------------+
41
+ / \ +-------------------+ /|\ ---------> | sat.[entity_name] |
42
+ User ---------------------------------------> / \ 3.create +----------------------+
43
+ 2.use Tool
44
+ | 3.create
45
+ v
46
+ +----------------------+
47
+ | dim.[entity_name] |
48
+ +----------------------+
49
+
50
+ ```
51
+
52
+ ### How to define a staging view or table:
53
+ * `bk_` (BusinessKey) - at least one `bk_` column
54
+ * `hk_[entity_name]` (HashKey) - exactly one `hk_[entity_name]` column if you want a `hub` table created
55
+ * `LoadDate` - required by dv2 standard for an auditability
56
+ * `RecordSource` - required by dv2 standard for an auditability
57
+ * `HashDiff` - optional, required if you want to have a scd2 type `dim` table created
58
+ * `IsAvailable` - optional, required if you want to track missing/deleted records
59
+ * all other columns will be considered as business columns and will be included to the `sat` table definition
60
+
61
+
62
+ | staging fields | scd2dim profile |
63
+ |--------------------|-----------------|
64
+ | bk_ | ✅ |
65
+ | hk_`[entity_name]` | ✅ |
66
+ | LoadDate | ✅ |
67
+ | RecordSource | ✅ |
68
+ | HashDiff | ✅ |
69
+ | IsAvailable | ✅ |
70
+
71
+ ```sql
72
+ -- staging view example for the scd2dim profile (mssql)
73
+ create view [stg].[UR_officers] as
74
+ select cast(31 as bigint) [bk_id]
75
+ , core.StringToHash1(cast(31 as bigint)) [hk_UR_officers]
76
+ , sysdatetime() [LoadDate]
77
+ , cast('LobSystem.dbo.officers_daily' as varchar(200)) [RecordSource]
78
+ , core.StringToHash8(
79
+ cast('uri' as nvarchar(100))
80
+ , cast('00000000000000' as varchar(20))
81
+ , cast('NATURAL_PERSON' as varchar(50))
82
+ , cast(null as varchar(20))
83
+ , cast('INDIVIDUALLY' as varchar(50))
84
+ , cast(0 as int)
85
+ , cast('2008-04-07' as date)
86
+ , cast('2008-04-07 18:00:54.000' as datetime)
87
+ ) [HashDiff]
88
+ , cast('uri' as nvarchar(100)) [uri]
89
+ , cast('00000000000000' as varchar(20)) [at_legal_entity_registration_number]
90
+ , cast('NATURAL_PERSON' as varchar(50)) [entity_type]
91
+ , cast(null as varchar(20)) [legal_entity_registration_number]
92
+ , cast('INDIVIDUALLY' as varchar(50)) [rights_of_representation_type]
93
+ , cast(0 as int) [representation_with_at_least]
94
+ , cast('2008-04-07' as date) [registered_on]
95
+ , cast('2008-04-07 18:00:54.000' as datetime) [last_modified_at]
96
+ , cast(1 as bit) [IsAvailable]
97
+ ```
98
+ ### scd2dim profile example:
99
+ | stg | hub | sat | dim |
100
+ |--------------------|------------------------|----------------------------|--------------------|
101
+ | | | | hk_`[entity_name]` |
102
+ | BKs... | (uk)BKs... | BKs... | (pk)BKs... |
103
+ | hk_`[entity_name]` | (pk)hk_`[entity_name]` | (pk)(fk)hk_`[entity_name]` | |
104
+ | LoadDate | LoadDate | (pk)LoadDate | |
105
+ | RecordSource | RecordSource | RecordSource | |
106
+ | HashDiff | | HashDiff | |
107
+ | FLDs... | | FLDs... | FLDs... |
108
+ | IsAvailable | | IsAvailable | IsAvailable |
109
+ | | | | IsCurrent |
110
+ | | | | (pk)DateFrom |
111
+ | | | | DateTo |
112
+
113
+ ### link2fact profile example:
114
+ | stg | link | sat | fact |
115
+ |--------------------|------------------------|----------------------------|------|
116
+ | HKs... | (uk)(fk)HKs... | | |
117
+ | hk_`[entity_name]` | (pk)hk_`[entity_name]` | (pk)(fk)hk_`[entity_name]` | |
118
+ | degenerate_field | (uk)degenerate_field | degenerate_field | |
119
+ | LoadDate | LoadDate | LoadDate | |
120
+ | RecordSource | RecordSource | RecordSource | |
121
+ | FLDs... | | FLDs... | |
122
+
123
+
124
+ ### Schemas:
125
+ * `core` - framework-related code
126
+ * `stg` - staging layer for both virtual (views) and materialized (tables)
127
+ * `hub` - hub layer
128
+ * `sat` - satellite layer
129
+ * `dim` - dimension layer (information vault)
130
+ * `fact` - fact layer (information vault)
131
+ * `elt` - ELT procedures
132
+ * `job` - top level ELT procedures
133
+ * `meta` - metadata vault
134
+ * `proxy` - source data for materialized staging area (meant for wrapping external data sources as SQL views)
135
+
136
+ ### DV2-related schemas layering
137
+ | LoB* | staging | raw vault | business vault | information vault |
138
+ |-------|---------|-----------|----------------|-------------------|
139
+ | proxy | stg | hub | sal | dim |
140
+ | | | sat | | fact |
141
+ | | | link | | |
142
+ _* Line of Business applications_
143
+
144
+ ### Usage diagram
145
+ ```text
146
+ + +-----------+ automation
147
+ +---- + -------> | Dv2Utils | -------+------+
148
+ | + uses +-----------+ |
149
+ | + | uses | creates
150
+ | + v |
151
+ | + uses +-----------+ uses |
152
+ +---- + -------> | Dv2Helper | --------------+
153
+ | + +-----------+ |
154
+ o + | |
155
+ /|\ + | DDL | python
156
+ / \ ==========================================================
157
+ DWH Dev + creates | | database
158
+ | + v V
159
+ | + uses +--------+ uses +---------------+
160
+ +---- + -------> | entity | -----> | core objects |
161
+ + +--------+ +---------------+
162
+ +
163
+
164
+ ```
sandwich-0.2.1.dist-info/RECORD ADDED
@@ -0,0 +1,5 @@
1
+ sandwich/py.typed,sha256=70pF0eMpuZgOyb0zFSE07ugId_AoU5z6CpLlVfg3pik,34
2
+ sandwich-0.2.1.dist-info/METADATA,sha256=3v9XmGospGdT5N66wOySHQTwVxK94AoxOwig_mF5t9E,8237
3
+ sandwich-0.2.1.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
4
+ sandwich-0.2.1.dist-info/licenses/LICENSE,sha256=kyN6iPdQMWMA8kquZpbnZvnPQDi9WMlyyXAPtlevrFQ,1080
5
+ sandwich-0.2.1.dist-info/RECORD,,
sandwich-0.1.0.dist-info/METADATA DELETED
@@ -1,16 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: sandwich
3
- Version: 0.1.0
4
- Summary: DataVault 2.0 code gen
5
- Author-email: Andrey Morozov <andrey@morozov.lv>
6
- License-File: LICENSE
7
- Keywords: DWH,Data Vault 2.0
8
- Classifier: Development Status :: 1 - Planning
9
- Classifier: Environment :: Console
10
- Classifier: Intended Audience :: Developers
11
- Classifier: License :: OSI Approved :: MIT License
12
- Classifier: Operating System :: OS Independent
13
- Classifier: Programming Language :: Python :: 3
14
- Requires-Python: >=3.12
15
- Requires-Dist: pyodbc>=5.3.0
16
- Requires-Dist: sqlalchemy>=2.0.44
sandwich-0.1.0.dist-info/RECORD DELETED
@@ -1,5 +0,0 @@
1
- sandwich/py.typed,sha256=70pF0eMpuZgOyb0zFSE07ugId_AoU5z6CpLlVfg3pik,34
2
- sandwich-0.1.0.dist-info/METADATA,sha256=yaTGSc1dbOXKSbTTQeH9opaMJ8icvyFetLh9K1wL_ww,543
3
- sandwich-0.1.0.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
4
- sandwich-0.1.0.dist-info/licenses/LICENSE,sha256=kyN6iPdQMWMA8kquZpbnZvnPQDi9WMlyyXAPtlevrFQ,1080
5
- sandwich-0.1.0.dist-info/RECORD,,