dremiojs 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/.eslintrc.json +14 -0
  2. package/.prettierrc +7 -0
  3. package/README.md +59 -0
  4. package/dremiodocs/dremio-cloud/cloud-api-reference.md +748 -0
  5. package/dremiodocs/dremio-cloud/dremio-cloud-about.md +225 -0
  6. package/dremiodocs/dremio-cloud/dremio-cloud-admin.md +3754 -0
  7. package/dremiodocs/dremio-cloud/dremio-cloud-bring-data.md +6098 -0
  8. package/dremiodocs/dremio-cloud/dremio-cloud-changelog.md +32 -0
  9. package/dremiodocs/dremio-cloud/dremio-cloud-developer.md +1147 -0
  10. package/dremiodocs/dremio-cloud/dremio-cloud-explore-analyze.md +2522 -0
  11. package/dremiodocs/dremio-cloud/dremio-cloud-get-started.md +300 -0
  12. package/dremiodocs/dremio-cloud/dremio-cloud-help-support.md +869 -0
  13. package/dremiodocs/dremio-cloud/dremio-cloud-manage-govern.md +800 -0
  14. package/dremiodocs/dremio-cloud/dremio-cloud-overview.md +36 -0
  15. package/dremiodocs/dremio-cloud/dremio-cloud-security.md +1844 -0
  16. package/dremiodocs/dremio-cloud/sql-docs.md +7180 -0
  17. package/dremiodocs/dremio-software/dremio-software-acceleration.md +1575 -0
  18. package/dremiodocs/dremio-software/dremio-software-admin.md +884 -0
  19. package/dremiodocs/dremio-software/dremio-software-client-applications.md +3277 -0
  20. package/dremiodocs/dremio-software/dremio-software-data-products.md +560 -0
  21. package/dremiodocs/dremio-software/dremio-software-data-sources.md +8701 -0
  22. package/dremiodocs/dremio-software/dremio-software-deploy-dremio.md +3446 -0
  23. package/dremiodocs/dremio-software/dremio-software-get-started.md +848 -0
  24. package/dremiodocs/dremio-software/dremio-software-monitoring.md +422 -0
  25. package/dremiodocs/dremio-software/dremio-software-reference.md +677 -0
  26. package/dremiodocs/dremio-software/dremio-software-security.md +2074 -0
  27. package/dremiodocs/dremio-software/dremio-software-v25-api.md +32637 -0
  28. package/dremiodocs/dremio-software/dremio-software-v26-api.md +36757 -0
  29. package/jest.config.js +10 -0
  30. package/package.json +25 -0
  31. package/src/api/catalog.ts +74 -0
  32. package/src/api/jobs.ts +105 -0
  33. package/src/api/reflection.ts +77 -0
  34. package/src/api/source.ts +61 -0
  35. package/src/api/user.ts +32 -0
  36. package/src/client/base.ts +66 -0
  37. package/src/client/cloud.ts +37 -0
  38. package/src/client/software.ts +73 -0
  39. package/src/index.ts +16 -0
  40. package/src/types/catalog.ts +31 -0
  41. package/src/types/config.ts +18 -0
  42. package/src/types/job.ts +18 -0
  43. package/src/types/reflection.ts +29 -0
  44. package/tests/integration_manual.ts +95 -0
  45. package/tsconfig.json +19 -0
@@ -0,0 +1,560 @@
1
+ # Dremio Software - Data Products
2
+
3
+
4
+
5
+ ---
6
+
7
+ # Source: https://docs.dremio.com/current/data-products/
8
+
9
+ Version: current [26.x]
10
+
11
+ On this page
12
+
13
+ # Build Data Products
14
+
15
+ A data product is a self-contained data asset that has been prepared, can be trusted, and has an SLA. It is meant to optimize for reuse, consistency, and sharing of data. Organizations have data products that are domain-specific and data products that are common at the organization level.
16
+
17
+ In Dremio, there are two levels of data products:
18
+
19
+ 1. Data products that are created by curating and transforming one or more source tables into a view.
20
+ 2. Data products that are created at the business or application layer and are used in reports, dashboards, and other applications.
21
+
22
+ Here are two examples of data products:
23
+
24
+ * Sam is a data analyst in the sales team, and he is asked to create a data product that provides a unified data asset that segments customers into different groups based on behavior, spending, and demographics. He creates a data product with a curated and transformed view of customer data from multiple source tables (e.g., customer demographics, purchase history, and engagement activity). The data in the data product is cleansed, transformed, enriched, certified, and structured for easy consumption by marketing and sales teams.
25
+ * Alice is an analyst in the finance team, and she needs to create an executive financial dashboard at the business or application layer for her company’s E-staff team. She creates a data product that powers a visualization report for financial performance metrics for her company, including revenue, expenses, and profitability over time. The data product integrates multiple data sources from various domains, including transactional and financial data, and presents the information in a format that is easily consumable by executives for decision-making.
26
+
27
+ ## Data Product Lifecyle
28
+
29
+ Data products are created and managed through a lifecycle similar to the software development lifecycle. The data product lifecycle encompasses the end-to-end process of developing, deploying, and maintaining data products, ensuring they provide ongoing value to users. The image below depicts the phases of the lifecycle, which teams iterate through.
30
+
31
+ ![This image illustrates the data products lifecycle.](/images/data-products-lifecycle.png "Data products lifecycle")
32
+
33
+ ### Discover
34
+
35
+ To make data products reusable, they must be easily discoverable and explorable. Cross-functional or organization level data products must be accessible across various team to drive consistency. Users need to be able to quickly understand how to interpret the data in a data product and determine if it is relevant to the business problem at hand. For this to happen, data products must be published with adequate metadata such as domain, descriptions and definitions, tags, and usage information. To learn more about data discovery, see [Discover Data](/current/data-products/discover).
36
+
37
+ ### Develop
38
+
39
+ Data products can be developed using SQL in Dremio's SQL Runner. You can just as easily use your IDE of choice for development. To learn more about developing data products, see [Develop Data Products](/current/data-products/develop).
40
+
41
+ ### Deploy
42
+
43
+ To learn more about deploying your semantic layer with dbt, see [Deploy with dbt](/current/data-products/deploy-with-dbt).
44
+
45
+ ### Govern
46
+
47
+ Effective data governance ensures secure, compliant, and transparent management of data by ensuring documentation and traceability, enforcing fine-grained access policies, and tracking dataset lineage to enhance data quality, minimize risks, and optimize value. To learn more about governance, see [Govern Data](/current/data-products/govern).
48
+
49
+ ### Serve
50
+
51
+ Data products can be served in multiple forms. For example, data products are served in the Dremio console for adhoc analysis or they can be incorporated into a dashboard or report. This phase of the lifecycle is focused on delivering insights and data output to users ensuring accessibility and usability. For more information on the client applications that support connectivity to Dremio, see [Connecting Client Applications to Dremio](/current/client-applications).
52
+
53
+ ### Observe
54
+
55
+ To fully empower users to manage their data products, they must be able to monitor them continuously to assess usage and performance over time. Observability enables data product owners to make iterative improvements based on their users' needs.
56
+
57
+ ## Additional Resources
58
+
59
+ Find out more about data products by enrolling in the [Data Product Fundamentals course in Dremio University](https://university.dremio.com/course/data-product-fundamentals).
60
+
61
+ Was this page helpful?
62
+
63
+ [Previous
64
+
65
+ Clustering](/current/load-data/clustering)[Next
66
+
67
+ Discover Data](/current/data-products/discover/)
68
+
69
+ * Data Product Lifecyle
70
+ + Discover
71
+ + Develop
72
+ + Deploy
73
+ + Govern
74
+ + Serve
75
+ + Observe
76
+ * Additional Resources
77
+
78
+ ---
79
+
80
+ # Source: https://docs.dremio.com/current/data-products/discover/
81
+
82
+ Version: current [26.x]
83
+
84
+ On this page
85
+
86
+ # Discover Data
87
+
88
+ Dremio simplifies the discovery of data objects and other Dremio entities.
89
+
90
+ Data discovery capabilities in Dremio include the following:
91
+
92
+ * Catalog exploration in the Dremio console
93
+ * AI-enabled semantic search for objects and entities
94
+ * Star objects
95
+
96
+ Data discovery capabilities are optimized for data that is governed by Dremio. Files and folders that have not been formatted as a table in Dremio may not be easily discoverable.
97
+
98
+ ## Catalog exploration in the Dremio console
99
+
100
+ The Datasets page allows you to navigate through and explore objects that you have access to in Dremio. To learn more about the Datasets page, see [Quick Tour of the Datasets Page](/current/get-started/quick_tour#datasets-page). Once you have located the table of view that you are interested in, you can use the Details panel to learn more about it.
101
+
102
+ ## AI-enabled semantic search for objects Enterprise
103
+
104
+ You can use the search bar in Dremio to find objects that are accessible through Dremio. This capability performs keyword and semantic search to find tables, views, and other objects that are related to your search criteria. Search only returns results that you have privileges to see. See [Searching for Dremio Objects](/current/data-products/discover/semantic-search).
105
+
106
+ ## Object Metadata
107
+
108
+ Metadata such as the owner of the table or view, when it was created, and columns is easily accessible through metadata cards. Metadata cards can help you learn more about the object of interest and provide quick links to actions that you may want to take on the table or view. See [Metadata Cards](/current/data-products/discover/metadata) for more information.
109
+
110
+ ## Star objects
111
+
112
+ You can star objects that you use frequently to provide easier discoverability and access. Starring objects such as sources, spaces, folders, tables, and views will surface them in the **Starred** tab of the SQL Runner. See [Star Objects](/current/data-products/discover/bookmarks) to learn more.
113
+
114
+ Was this page helpful?
115
+
116
+ [Previous
117
+
118
+ Build Data Products](/current/data-products/)[Next
119
+
120
+ Searching for Dremio Objects](/current/data-products/discover/semantic-search)
121
+
122
+ * Catalog exploration in the Dremio console
123
+ * AI-enabled semantic search for objects Enterprise
124
+ * Object Metadata
125
+ * Star objects
126
+
127
+ ---
128
+
129
+ # Source: https://docs.dremio.com/current/data-products/develop
130
+
131
+ Version: current [26.x]
132
+
133
+ On this page
134
+
135
+ # Develop Data Products
136
+
137
+ You can curate and transform your data to create a data product by
138
+
139
+ * Writing SQL in the SQL Runner
140
+ * Using the low-code tranformation flows in the SQL Runner to help generate SQL
141
+
142
+ ## Write SQL in the SQL Runner
143
+
144
+ You can use the SQL Runner to transform your data and create data products in Dremio.
145
+
146
+ * For a quick tour of the SQL Runner and the supported capabilities, see [Quick Tour of the SQL Runner](/current/get-started/quick_tour#sql-runner).
147
+ * See the [SQL Reference](/current/reference/sql) for functions and commands that you can use to transform and work with your data.
148
+ * You can also create data products using an IDE of your choice. Use [Arrow Flight JDBC](/current/client-applications/drivers/arrow-flight-sql-jdbc-driver) to establish a connection to Dremio from and IDE to do your development.
149
+
150
+ ### Create a View
151
+
152
+ You can create a view from an existing table or view by transforming the data as required and performing the following steps:
153
+
154
+ 1. Compose the query in the SQL Runner and click **Run** to validate the query. After running the query, click the arrow next to **Save Script As** in the top right of the SQL editor, and select **Save View as...** from the drop-down menu.
155
+ 2. Name the new view and select where the view will be located. If the location is not updated, the view will get saved to your home space. Once the view is saved, you will be navigated to the Dataset page.
156
+
157
+ ### Retrieve a View Definition
158
+
159
+ If you have the `SELECT` privilege for a view, you can see the underlying definition in the SQL editor but cannot edit the view directly. To see a view definition, on the Datasets page, hover over the line containing the view and click ![](/images/icons/go-to-table.png) on the right.
160
+
161
+ tip
162
+
163
+ If you have the required privileges, you can run `SHOW CREATE VIEW <view_name>` in the SQL editor to see the view definition. See [SHOW CREATE VIEW](/current/reference/sql/commands/show-create-view).
164
+
165
+ ### Edit a View
166
+
167
+ If you have the privileges required to edit a view, you can see and edit the definition of the view.
168
+
169
+ To edit a view, perform the following steps:
170
+
171
+ 1. On the Datasets page, hover over the line containing the view and click ![](/images/icons/edit.png) on the right. The view definition will open.
172
+ 2. Edit the view definition as needed and click **Run** to validate the query.
173
+ 3. Click **Save View** in the top right corner of the SQL editor. This will overwrite the current definition of the view with the new definition.
174
+
175
+ ### Delete a View
176
+
177
+ Perform the following steps to delete a view:
178
+
179
+ 1. On the Datasets page, go to the folder or space where your view is located.
180
+ 2. Hover over the line containing the view that you want to delete, click the ellipses (...) icon. From the list of actions, click **Delete**. Confirm that you want to delete the view.
181
+
182
+ caution
183
+
184
+ If you are deleting a table or view with children, you get a warning. Removing a table or view with children leaves you with disconnected views that you can no longer query.
185
+
186
+ ## Use Low-Code to Transform Data in the SQL Runner
187
+
188
+ To begin a data transformation, via one of the following methods:
189
+
190
+ * Highlighting a portion or all of a field's value
191
+ * Using the dropdown menu for the transformation.
192
+ The dropdown is to the right of the field's name.
193
+
194
+ ### Use Highlighting
195
+
196
+ The highlighting method is often the most intuitive method.
197
+ It provides enough context for Dremio to make some best guesses
198
+ about how to execute the transformation that you have in mind.
199
+
200
+ For instance, you could highlight a portion of a field that contains customer names to quickly perform an extract
201
+ that creates a new field with only last names.
202
+
203
+ **Suggestion Cards**
204
+ For transformations that are initiated by highlighting part or all of a field value,
205
+ Dremio uses a heuristic to determine a set of "suggestion cards" that represent its
206
+ best guesses as to your intended result.
207
+
208
+ You can click on one of these suggestion cards to inspect a preview of the new dataset and
209
+ confirm that it matches your expectations.
210
+ If no suggestion card is a perfect match, you can "flip" the card (by clicking the
211
+ pencil icon in the upper right corner) to tweak the
212
+ card's parameters before applying the transformation.
213
+
214
+ note
215
+
216
+ The highlight method is great for beginning an extract. However, in cases where other capabilities are required, the dropdown menu may be more useful.
217
+
218
+ ### Use Dropdown Menus
219
+
220
+ The dropdown menu provides a more complete list of transformations that are applicable to the data type.
221
+
222
+ ### Fix Inconsistent Data with a Join
223
+
224
+ In situations where the entries in a field are inconsistent
225
+ (for example, different spellings or abbreviations for the same name),
226
+ the following technique can be used to increase the quality of the dataset:
227
+
228
+ 1. Identify the field with the problematic data. It may be useful to run this command in the SQL Editor:
229
+
230
+ Identify a field
231
+
232
+ ```
233
+ SELECT DISTINCT myProblemFieldName FROM myDatasource.myTable
234
+ ```
235
+ 2. Download the results as CSV using the Download button.
236
+ 3. Open the file in a text editor or Excel and create lookup values for the distinct values
237
+ from your table in a second column.
238
+ For example, standardizing variations in color names to a single canonical name.
239
+ 4. Upload this file to your Home space on Dremio
240
+ 5. Open this new dataset and hit the Join button located on the left above the field names
241
+ 6. Select Custom Join and then the name of the inconsistent dataset you would like to fix, followed by Next
242
+ 7. Drag over the name of the left column from your uploaded dataset, and match it with the name of the field you
243
+ would like to correct in the inconsistent dataset
244
+ 8. Apply the Join then drop the old field, renaming the new one to take its place
245
+ 9. Save the corrected dataset
246
+
247
+ ### Clean Text
248
+
249
+ For text data, excess whitespace and changing capitalization schemes are two common data cleanliness issues.
250
+ Dremio provides two transformations for dealing with these possible inconsistencies:
251
+ **Trim Whitespace** and **Convert Case**.
252
+
253
+ ### Handle Invalid, Empty and NULL Values
254
+
255
+ Empty or NULL text values are best eliminated by using Exclude.
256
+ You can initiate this transformation by:
257
+
258
+ 1. Highlight a value from the field
259
+ that contains empty or NULL values, and
260
+ 2. Select Exclude from the dropdown that appears.
261
+ This renders a list of the values in this field, and the frequency at which they occur.
262
+ 3. Check the boxes next to the empty and/or NULL values you which to exclude from the dataset and click Apply.
263
+
264
+ ### Work with Date Types
265
+
266
+ You can convert a text type field that contains date information into a proper date type field. This allows you to do more sophisticated analyses in external tools such as spotting by trends by month, year, or quarter.
267
+
268
+ You can begin this conversion by selecting 'Date & Time' from the type menu located to the left of the text field's name. In the subsequent dialog, select whether the output should be a time, date, date and time. It also gives a few default options for the format as well as a 'Custom' field for indicating a custom format. See [Data & Time Data Types](/current/reference/sql/data-types/#date--time-data-types) for more information on the conversions you can do.
269
+
270
+ Was this page helpful?
271
+
272
+ [Previous
273
+
274
+ Star Objects](/current/data-products/discover/bookmarks)[Next
275
+
276
+ Deploy with dbt](/current/data-products/deploy-with-dbt)
277
+
278
+ * Write SQL in the SQL Runner
279
+ + Create a View
280
+ + Retrieve a View Definition
281
+ + Edit a View
282
+ + Delete a View
283
+ * Use Low-Code to Transform Data in the SQL Runner
284
+ + Use Highlighting
285
+ + Use Dropdown Menus
286
+ + Fix Inconsistent Data with a Join
287
+ + Clean Text
288
+ + Handle Invalid, Empty and NULL Values
289
+ + Work with Date Types
290
+
291
+ ---
292
+
293
+ # Source: https://docs.dremio.com/current/data-products/deploy-with-dbt
294
+
295
+ Version: current [26.x]
296
+
297
+ On this page
298
+
299
+ # Deploy with dbt
300
+
301
+ dbt enables analytics engineers to develop and manage semantic layers within dbt projects and deploy them to Dremio.
302
+
303
+ You can use Dremio's dbt connector `dbt-dremio` to transform data that is in data sources that are connected to a Dremio project.
304
+
305
+ ## Prerequisites
306
+
307
+ * Download the `dbt-dremio` package from <https://github.com/dremio/dbt-dremio>.
308
+ * Ensure that Python 3.9.x or later is installed.
309
+ * Ensure that you are using Dremio Software version 22.0 or later.
310
+ * If you want to use TLS to secure the connection between dbt and Dremio Software, configure full wire encryption in your Dremio cluster. For more information, see the configuration of TLS for [Dremio on Kubernetes](/current/deploy-dremio/configuring-kubernetes/#transport-level-security) or [Dremio standalone clusters](/current/deploy-dremio/other-options/standalone/dremio-config/dremio-conf/wire-encryption-config/#full-wire-encryption-enterprise).
311
+
312
+ ## Installing
313
+
314
+ Install this package from PyPi by running this command:
315
+
316
+ Install dbt-dremio package
317
+
318
+ ```
319
+ pip install dbt-dremio
320
+ ```
321
+
322
+ note
323
+
324
+ `dbt-dremio` works with dbt-core versions 1.8 and 1.9. Earlier versions of dbt-core are out of support from dbt.
325
+
326
+ ## Initializing a dbt Project
327
+
328
+ 1. Run the command `dbt init <project_name>`.
329
+ 2. Select `dremio` as the database to use.
330
+ 3. Select one of these options to generate a profile for your project:
331
+ * `software_with_username_password` for working with a Dremio Software cluster and authenticating to the cluster with a username and a password
332
+ * `software_with_pat` for working with a Dremio Software cluster and authenticating to the cluster with a personal access token
333
+
334
+ Next, configure the profile for your dbt project.
335
+
336
+ ## Profiles
337
+
338
+ When you initialize a dbt project, you create one of these three profiles. You must configure it before trying to connect to Dremio Cloud or Dremio Software.
339
+
340
+ * Profile for Dremio Software with Username/Password Authentication
341
+ * Profile for Dremio Software with Authentication Through a Personal Access Token
342
+
343
+ For descriptions of the configurations in these profiles, see Configurations.
344
+
345
+ ### Dremio Software Profile with Username & Password
346
+
347
+ Example Profile
348
+
349
+ ```
350
+ [project name]:
351
+ outputs:
352
+ dev:
353
+ password: b9JtkIgI3uup9gGxxK
354
+ port: 9047
355
+ software_host: 192.0.2.0
356
+ object_storage_source: Samples
357
+ object_storage_path: "samples.dremio.com"."Dremio University"
358
+ dremio_space: Space1
359
+ dremio_space_folder: Folder1.Folder2
360
+ threads: 1
361
+ type: dremio
362
+ use_ssl: true
363
+ user: userName
364
+ target: dev
365
+ ```
366
+
367
+ ### Dremio Software Profile with Personal Access Token
368
+
369
+ Example Profile
370
+
371
+ ```
372
+ [project name]:
373
+ outputs:
374
+ dev:
375
+ pat: A1BCDrE2FwgH3IJkLM4NoPqrsT5uV6WXyza7I8bcDEFgJ9hIj0Kl1MNOPq2Rstu
376
+ port: 9047
377
+ software_host: 192.0.2.0
378
+ object_storage_source: Samples
379
+ object_storage_path: "samples.dremio.com"."Dremio University"
380
+ dremio_space: Space1
381
+ dremio_space_folder: Folder1.Folder2
382
+ threads: 1
383
+ type: dremio
384
+ use_ssl: true
385
+ user: userName
386
+ target: dev
387
+ ```
388
+
389
+ ## Configurations
390
+
391
+ | Configuration | Required? | Default Value | Description |
392
+ | --- | --- | --- | --- |
393
+ | `password` | Yes, if you are not using the pat configuration. | None | The password of the account to use when logging into the Dremio cluster. |
394
+ | `pat` | Yes, if you are not using the user and password configurations. | None | The personal access token to use for authenticating to Dremio. See [Personal Access Tokens](/current/security/authentication/personal-access-tokens/) for instructions about obtaining a token. The use of a personal access token takes precedence if values for the three configurations user, password and pat are specified. |
395
+ | `port` | Yes | `9047` | Port for Dremio Software cluster API endpoints. |
396
+ | `software_host` | Yes | None | The hostname or IP address of the coordinator node of the Dremio cluster. |
397
+ | `enterprise_catalog_namespace` | No | None | The name of the catalog in which to create tables, materialized views, tests, and other objects, and views. The dbt aliases are `datalake` (for objects) and `database` (for views). This name corresponds to the name of a catalog in the **Open Catalogs** section of the Datasets page in Dremio. |
398
+ | `enterprise_catalog_folder` | No | None | The path in the catalog in which to create objects / views. The dbt aliases are `root_path` (for objects) and `schema` (for views). Nested folders in the path are separated with periods. This value corresponds to the path in this location in the Datasets page in Dremio. |
399
+ | `object_storage_source` | No | $scratch | The name of the filesystem in which to create tables, materialized views, tests, and other objects. The dbt alias is `datalake`. This name corresponds to the name of a source in the **Object Storage** section of the Datasets page in Dremio: |
400
+ | `object_storage_path` | No | `no_schema` | The path in the filesystem in which to create objects. The default is the root level of the filesystem. The dbt alias is `root_path`. Nested folders in the path are separated with periods. This value corresponds to the path in this location in the Datasets page in Dremio: 'samples.dremio.com'.'Dremio University' |
401
+ | `dremio_space` | No | `@<username>` | The value of the Dremio space in which to create views. The dbt alias is `database`. This value corresponds to the name in this location in the **Spaces** section of the Datasets page in Dremio: Spaces1 |
402
+ | `dremio_space_folder` | No | `no_schema` | The folder in the Dremio space in which to create views. The default is the top level in the space. The dbt alias is `schema`. Nested folders are separated with periods. This value corresponds to the path in this location in the Datasets page in Dremio: |
403
+ | `threads` | Yes | 1 | The number of threads the dbt project runs on. |
404
+ | `type` | Yes | dremio | Auto-populated when creating a Dremio project. Do not change this value. |
405
+ | `use_ssl` | Yes | `true` | Acceptable values are `true` and `false`. If the value is set to true, ensure that full wire encryption is configured in your Dremio cluster. See [Prerequisites](/current/data-products/deploy-with-dbt#prerequisites). |
406
+ | `verify_ssl` | No | `true` | Acceptable values are `true` and `false`. Set to `false` if using a self-signed certificate or if the root certificate authority (CA) is not included in Python’s CA certificates. |
407
+ | `user` | Yes | None | The username of the account to use when logging into the Dremio cluster. |
408
+
409
+ ## Known Issues
410
+
411
+ [Model contracts](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) are not supported.
412
+
413
+ ## Additional Resources
414
+
415
+ Learn more about DataOps by enrolling in the [DataOps with Apache Iceberg course in Dremio University](https://university.dremio.com/course/dataops-with-apache-iceberg).
416
+
417
+ Was this page helpful?
418
+
419
+ [Previous
420
+
421
+ Develop Data Products](/current/data-products/develop)[Next
422
+
423
+ Govern Data](/current/data-products/govern/)
424
+
425
+ * Prerequisites
426
+ * Installing
427
+ * Initializing a dbt Project
428
+ * Profiles
429
+ + Dremio Software Profile with Username & Password
430
+ + Dremio Software Profile with Personal Access Token
431
+ * Configurations
432
+ * Known Issues
433
+ * Additional Resources
434
+
435
+ ---
436
+
437
+ # Source: https://docs.dremio.com/current/data-products/govern/
438
+
439
+ Version: current [26.x]
440
+
441
+ On this page
442
+
443
+ # Govern Data
444
+
445
+ Effective data governance is essential for managing secure, compliant, transparent, and traceable data products. In addition to securing your data using role-based access control (RBAC), data governance for data products allows you to apply fine-grained access controls (row access & column-masking policies) on your data, as well as trace the lineage of any dataset in the lakehouse. Together, these governance tools help organizations optimize the value they derive from their data to enhance data quality and transparency, while minimizing the risks associated with data misuse and non-compliance.
446
+ Dremio also provides tools for describing, identifying, and displaying datasets using wikis and tags.
447
+
448
+ ## Row-Access and Column-Masking Policies
449
+
450
+ [Row access & column-masking (or fine-grained access control) policies](/current/data-products/govern/row-column-policies-udf) provide mechanisms to enforce data privacy and security rules directly on your data.
451
+
452
+ ## Lineage
453
+
454
+ [Data lineage](/current/data-products/govern/lineage) allows you to track and visualize data as it moves through the various stages of a data pipeline. It provides clarity on where the data comes from, how it is transformed, and where it is used.
455
+
456
+ ## Wikis and Tags
457
+
458
+ [Wikis](/current/data-products/govern/wikis-tags) for datasets allow users to document and describe datasets in the Open Catalog using a rich text editor with [Github-flavored markdown](https://github.github.com/gfm/).
459
+
460
+ [Tags](/current/data-products/govern/wikis-tags) for datasets help organize and retrieve data efficiently by allowing users to search, filter, and locate datasets through specific tag associations.
461
+
462
+ Was this page helpful?
463
+
464
+ [Previous
465
+
466
+ Deploy with dbt](/current/data-products/deploy-with-dbt)[Next
467
+
468
+ Row-Access and Column-Masking Policies](/current/data-products/govern/row-column-policies-udf)
469
+
470
+ * Row-Access and Column-Masking Policies
471
+ * Lineage
472
+ * Wikis and Tags
473
+
474
+ ---
475
+
476
+ # Source: https://docs.dremio.com/current/data-products/discover
477
+
478
+ Version: current [26.x]
479
+
480
+ On this page
481
+
482
+ # Discover Data
483
+
484
+ Dremio simplifies the discovery of data objects and other Dremio entities.
485
+
486
+ Data discovery capabilities in Dremio include the following:
487
+
488
+ * Catalog exploration in the Dremio console
489
+ * AI-enabled semantic search for objects and entities
490
+ * Star objects
491
+
492
+ Data discovery capabilities are optimized for data that is governed by Dremio. Files and folders that have not been formatted as a table in Dremio may not be easily discoverable.
493
+
494
+ ## Catalog exploration in the Dremio console
495
+
496
+ The Datasets page allows you to navigate through and explore objects that you have access to in Dremio. To learn more about the Datasets page, see [Quick Tour of the Datasets Page](/current/get-started/quick_tour#datasets-page). Once you have located the table of view that you are interested in, you can use the Details panel to learn more about it.
497
+
498
+ ## AI-enabled semantic search for objects Enterprise
499
+
500
+ You can use the search bar in Dremio to find objects that are accessible through Dremio. This capability performs keyword and semantic search to find tables, views, and other objects that are related to your search criteria. Search only returns results that you have privileges to see. See [Searching for Dremio Objects](/current/data-products/discover/semantic-search).
501
+
502
+ ## Object Metadata
503
+
504
+ Metadata such as the owner of the table or view, when it was created, and columns is easily accessible through metadata cards. Metadata cards can help you learn more about the object of interest and provide quick links to actions that you may want to take on the table or view. See [Metadata Cards](/current/data-products/discover/metadata) for more information.
505
+
506
+ ## Star objects
507
+
508
+ You can star objects that you use frequently to provide easier discoverability and access. Starring objects such as sources, spaces, folders, tables, and views will surface them in the **Starred** tab of the SQL Runner. See [Star Objects](/current/data-products/discover/bookmarks) to learn more.
509
+
510
+ Was this page helpful?
511
+
512
+ [Previous
513
+
514
+ Build Data Products](/current/data-products/)[Next
515
+
516
+ Searching for Dremio Objects](/current/data-products/discover/semantic-search)
517
+
518
+ * Catalog exploration in the Dremio console
519
+ * AI-enabled semantic search for objects Enterprise
520
+ * Object Metadata
521
+ * Star objects
522
+
523
+ ---
524
+
525
+ # Source: https://docs.dremio.com/current/data-products/govern
526
+
527
+ Version: current [26.x]
528
+
529
+ On this page
530
+
531
+ # Govern Data
532
+
533
+ Effective data governance is essential for managing secure, compliant, transparent, and traceable data products. In addition to securing your data using role-based access control (RBAC), data governance for data products allows you to apply fine-grained access controls (row access & column-masking policies) on your data, as well as trace the lineage of any dataset in the lakehouse. Together, these governance tools help organizations optimize the value they derive from their data to enhance data quality and transparency, while minimizing the risks associated with data misuse and non-compliance.
534
+ Dremio also provides tools for describing, identifying, and displaying datasets using wikis and tags.
535
+
536
+ ## Row-Access and Column-Masking Policies
537
+
538
+ [Row access & column-masking (or fine-grained access control) policies](/current/data-products/govern/row-column-policies-udf) provide mechanisms to enforce data privacy and security rules directly on your data.
539
+
540
+ ## Lineage
541
+
542
+ [Data lineage](/current/data-products/govern/lineage) allows you to track and visualize data as it moves through the various stages of a data pipeline. It provides clarity on where the data comes from, how it is transformed, and where it is used.
543
+
544
+ ## Wikis and Tags
545
+
546
+ [Wikis](/current/data-products/govern/wikis-tags) for datasets allow users to document and describe datasets in the Open Catalog using a rich text editor with [Github-flavored markdown](https://github.github.com/gfm/).
547
+
548
+ [Tags](/current/data-products/govern/wikis-tags) for datasets help organize and retrieve data efficiently by allowing users to search, filter, and locate datasets through specific tag associations.
549
+
550
+ Was this page helpful?
551
+
552
+ [Previous
553
+
554
+ Deploy with dbt](/current/data-products/deploy-with-dbt)[Next
555
+
556
+ Row-Access and Column-Masking Policies](/current/data-products/govern/row-column-policies-udf)
557
+
558
+ * Row-Access and Column-Masking Policies
559
+ * Lineage
560
+ * Wikis and Tags