pointblank 0.11.2__tar.gz → 0.11.4__tar.gz

{pointblank-0.11.2 → pointblank-0.11.4}/CITATION.cff

@@ -3,7 +3,7 @@ message: 'If you wish to cite the "Pointblank" package use:'
 type: software
 license: MIT
 title: "Pointblank: Find out if your data is what you think it is"
-version: 0.11.1
+version: 0.11.3
 abstract: Validate data in Polars and Pandas DataFrames and database tables.
   Validation pipelines can be made using easily-readable, consecutive validation
   steps. Upon execution of the validation plan, several reporting options are available.

{pointblank-0.11.2 → pointblank-0.11.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pointblank
-Version: 0.11.2
+Version: 0.11.4
 Summary: Find out if your data is what you think it is.
 Author-email: Richard Iannone <riannone@me.com>
 License: MIT License
@@ -282,6 +282,49 @@ pb validate small_table --check col-vals-gt --column a --value 5 --show-extract
 pb validate small_table --check rows-distinct --exit-code
 ```
 
+Click the following headings to see some video demonstrations of the CLI:
+
+<details>
+<summary>Getting Started with the Pointblank CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-getting-started.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Doing Some Data Exploration</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-data-exploration.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Validating Data with the CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-essential-validations.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Using Polars in the CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-using-polars.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Integrating Pointblank with CI/CD</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-cicd-workflows.gif" width="100%">
+</div>
+
+</details>
+
 ## Features That Set Pointblank Apart
 
 - **Complete validation workflow**: From data access to validation to reporting in a single pipeline
@@ -349,6 +392,10 @@ There are many ways to contribute to the ongoing development of Pointblank. Some
 Please read over the [contributing guidelines](https://github.com/posit-dev/pointblank/blob/main/CONTRIBUTING.md) for
 information on how to get started.
 
+## Pointblank for R
+
+There's also a version of Pointblank for R, which has been around since 2017 and is widely used in the R community. You can find it at https://github.com/rstudio/pointblank.
+
 ## Roadmap
 
 We're actively working on enhancing Pointblank with:

{pointblank-0.11.2 → pointblank-0.11.4}/README.md

@@ -196,6 +196,49 @@ pb validate small_table --check col-vals-gt --column a --value 5 --show-extract
 pb validate small_table --check rows-distinct --exit-code
 ```
 
+Click the following headings to see some video demonstrations of the CLI:
+
+<details>
+<summary>Getting Started with the Pointblank CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-getting-started.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Doing Some Data Exploration</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-data-exploration.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Validating Data with the CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-essential-validations.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Using Polars in the CLI</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-using-polars.gif" width="100%">
+</div>
+
+</details>
+<details>
+<summary>Integrating Pointblank with CI/CD</summary>
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/vhs/cli-cicd-workflows.gif" width="100%">
+</div>
+
+</details>
+
 ## Features That Set Pointblank Apart
 
 - **Complete validation workflow**: From data access to validation to reporting in a single pipeline
@@ -263,6 +306,10 @@ There are many ways to contribute to the ongoing development of Pointblank. Some
 Please read over the [contributing guidelines](https://github.com/posit-dev/pointblank/blob/main/CONTRIBUTING.md) for
 information on how to get started.
 
+## Pointblank for R
+
+There's also a version of Pointblank for R, which has been around since 2017 and is widely used in the R community. You can find it at https://github.com/rstudio/pointblank.
+
 ## Roadmap
 
 We're actively working on enhancing Pointblank with:

{pointblank-0.11.2 → pointblank-0.11.4}/docs/_quarto.yml

@@ -79,6 +79,7 @@ website:
             - user-guide/schema-validation.qmd
             - user-guide/assertions.qmd
             - user-guide/draft-validation.qmd
+            - user-guide/yaml-validation-workflows.qmd
         - section: "Post Interrogation"
           contents:
             - user-guide/step-reports.qmd
@@ -219,6 +220,15 @@ quartodoc:
         - name: load_dataset
         - name: get_data_path
         - name: connect_to_table
+    - title: YAML
+      desc: >
+        The *YAML* group contains functions that allow for the use of YAML to orchestrate validation
+        workflows. The `yaml_interrogate()` function can be used to run a validation workflow from
+        YAML strings or files. The `validate_yaml()` function checks if the YAML configuration
+        passes its own validity checks.
+      contents:
+        - name: yaml_interrogate
+        - name: validate_yaml
     - title: Utility Functions
       desc: >
         The *Utility Functions* group contains functions that are useful accessing metadata about

{pointblank-0.11.2 → pointblank-0.11.4}/docs/assets/vhs/cli-cicd-workflows.tape

@@ -1,6 +1,3 @@
-# VHS tape for CI/CD integration workflows
-# Shows production-ready automation patterns
-
 Output cli-cicd-workflows.gif
 
 Set FontSize 16
@@ -8,9 +5,9 @@ Set Width 1200
 Set Height 700
 Set Theme "Dracula"
 
-Type "# Example of a Production CI/CD Data Validation Workflow"
+Type "# Example of a Production CI/CD Data Validation Workflow with the Pointblank CLI"
 Enter
-Sleep 2s
+Sleep 3s
 
 Type ""
 Enter
@@ -38,8 +35,8 @@ Sleep 10s
 
 Type ""
 Enter
-Type "# Use in GitHub Actions or any automation platform"
+Type "# Make use of --exit-code in GitHub Actions or any other automation platform"
 Enter
-Type "echo 'Exit codes: 0 = success, 1 = validation failed'"
+Type "# Exit codes: 0 = success, 1 = validation failed"
 Enter
-Sleep 8s
+Sleep 6s

pointblank-0.11.4/docs/assets/vhs/cli-complete-workflow.tape

@@ -0,0 +1,80 @@
+Output cli-complete-workflow.gif
+
+Set FontSize 13
+Set Width 1000
+Set Height 1100
+Set Theme "Dracula"
+
+Type "# This is a Demo of a Complete Data Quality Workflow Using the Pointblank CLI"
+Enter
+Sleep 1s
+
+Type ""
+Enter
+Type "# Step 1: Quickly preview a dataset"
+Enter
+Type "pb preview worldcities.csv"
+Enter
+Sleep 5s
+
+Type ""
+Enter
+Type "# Step 2: Quickly check for missing values in the dataset"
+Enter
+Type "pb missing worldcities.csv"
+Enter
+Sleep 6s
+
+Type ""
+Enter
+Type "# Step 3: Validate that population values are provided for each city (i.e., not null)"
+Enter
+Type "pb validate worldcities.csv --check col-vals-not-null --column population"
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# Step 4: Check for missing city names (and show the problematic rows)"
+Enter
+Type "pb validate worldcities.csv --check col-vals-not-null --column city_name --show-extract"
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# Step 5: Validate latitude range (latitude: -90 to 90) in two steps"
+Enter
+Type "pb validate worldcities.csv --check col-vals-ge --column latitude --value -90 --check col-vals-le --column latitude --value 90"
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# Step 6: Create a comprehensive validation script for production use"
+Enter
+Type "cat validation.py"
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# Step 7: Run that comprehensive validation and get a validation reporting table"
+Enter
+Type "pb run validation.py"
+Enter
+Sleep 10s
+
+Type ""
+Enter
+Type "# Step 8: Generate an HTML validation report file and show the failing rows"
+Enter
+Type "pb run validation.py --output-html validation_report.html --show-extract"
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# That was a complete workflow, moving from quick checks to more involved validations"
+Enter
+Sleep 3s

{pointblank-0.11.2 → pointblank-0.11.4}/docs/assets/vhs/cli-data-exploration.tape

@@ -14,7 +14,7 @@ Sleep 2s
 
 Type ""
 Enter
-Type "# 0. Exploring world cities dataset: get the table dimensions"
+Type "# 1. Ensure that data can be loaded and see some info about the dataset"
 Enter
 Type "pb info worldcities.csv"
 Enter
@@ -22,7 +22,7 @@ Sleep 4s
 
 Type ""
 Enter
-Type "# 1. Quick data preview: see structure and sample rows"
+Type "# 2. Get a quick data preview to see structure and sample rows"
 Enter
 Type "pb preview worldcities.csv --head 3 --tail 3"
 Enter
@@ -30,7 +30,7 @@ Sleep 8s
 
 Type ""
 Enter
-Type "# 2. Missing values analysis: identify data gaps"
+Type "# 3. Missing values analysis: identify data gaps"
 Enter
 Type "pb missing worldcities.csv"
 Enter
@@ -38,7 +38,7 @@ Sleep 8s
 
 Type ""
 Enter
-Type "# 3. Comprehensive data scan: statistical overview"
+Type "# 4. Comprehensive data scan: statistical overview"
 Enter
 Type "pb scan worldcities.csv"
 Enter
@@ -46,6 +46,6 @@ Sleep 10s
 
 Type ""
 Enter
-Type "echo 'Complete data profiling in seconds with the Pointblank CLI'"
+Type "# That's data profiling in seconds with the Pointblank CLI"
 Enter
-Sleep 3s
+Sleep 4s

{pointblank-0.11.2 → pointblank-0.11.4}/docs/assets/vhs/cli-essential-validations.tape

@@ -1,6 +1,3 @@
-# VHS tape for high-impact data validation demonstrations
-# Shows the most practical CLI validations
-
 Output cli-essential-validations.gif
 
 Set FontSize 16
@@ -14,7 +11,7 @@ Sleep 4s
 
 Type ""
 Enter
-Type "# 0. Get the table dimensions, just to know what you're working with"
+Type "# 1. Get the table dimensions, just to know what you're working with"
 Enter
 Type "pb info small_table"
 Enter
@@ -22,7 +19,7 @@ Sleep 8s
 
 Type ""
 Enter
-Type "# 1. Check for duplicate rows (the most common data quality issue)"
+Type "# 2. Check for duplicate rows (the most common data quality issue)"
 Enter
 Type "pb validate small_table --check rows-distinct"
 Enter
@@ -30,7 +27,7 @@ Sleep 8s
 
 Type ""
 Enter
-Type "# 2. Find missing values in important columns"
+Type "# 3. Find missing values in important columns"
 Enter
 Type "pb validate small_table --check col-vals-not-null --column c"
 Enter
@@ -38,7 +35,7 @@ Sleep 8s
 
 Type ""
 Enter
-Type "# 3. When validation fails, see exactly which rows are problematic with --show-extract"
+Type "# 4. When validation fails, see exactly which rows are problematic with --show-extract"
 Enter
 Type "pb validate small_table --check col-vals-not-null --column c --show-extract"
 Enter
@@ -46,6 +43,6 @@ Sleep 10s
 
 Type ""
 Enter
-Type "echo 'Essential validations: duplicates, nulls, with data extracts'"
+Type "# That is just a sampling of the essentials with duplicates, nulls, with data extracts"
 Enter
-Sleep 6s
+Sleep 3s

{pointblank-0.11.2 → pointblank-0.11.4}/docs/assets/vhs/cli-getting-started.tape

@@ -1,6 +1,3 @@
-# VHS tape for CLI overview and getting started
-# Shows help and basic commands with custom prompt
-
 Output cli-getting-started.gif
 
 Set FontSize 16
@@ -14,7 +11,7 @@ Sleep 5s
 
 Type ""
 Enter
-Type "# See all available commands"
+Type "# See all available commands for the Pointblank CLI"
 Enter
 Type "pb --help"
 Enter
@@ -46,6 +43,6 @@ Sleep 10s
 
 Type ""
 Enter
-Type "# You are now ready to validate any dataset you like: CSV, Parquet, or a DB table"
+Type "# You can validate any sort of dataset you want: CSV, Parquet, or a DB table"
 Enter
 Sleep 5s

pointblank-0.11.4/docs/assets/vhs/cli-using-polars.tape

@@ -0,0 +1,64 @@
+Output cli-using-polars.gif
+
+Set FontSize 13
+Set Width 1000
+Set Height 1100
+Set Theme "Dracula"
+
+Type "# Polars Data Manipulation with the Pointblank CLI"
+Enter
+Sleep 4s
+
+Type ""
+Enter
+Type "# 1. Load and explore the worldcities CSV dataset"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").head(10)'`
+Enter
+Sleep 8s
+
+Type ""
+Enter
+Type "# 2. Use the Polars methods for filtering and selecting columns"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").select(["city_name", "country", "population"]).filter(pl.col("population") > 1000000)'`
+Enter
+Sleep 10s
+
+Type ""
+Enter
+Type "# 3. Data manipulation -> missing data analysis via --output-format missing"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").select(["city_name", "country", "population"])' --output-format missing`
+Enter
+Sleep 10s
+
+Type ""
+Enter
+Type "# 4. Behold the power of piping! Filter large cities and validate for duplicates with --pipe | pb validate"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").filter(pl.col("population") > 5000000)' --pipe | pb validate --check rows-distinct`
+Enter
+Sleep 10s
+
+Type ""
+Enter
+Type "# 5. Do a complex transformation and pipe into a data scan"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").filter(pl.col("country").is_in(["United States", "China", "India"])).select(["city_name", "latitude", "longitude"])' --pipe | pb scan`
+Enter
+Sleep 12s
+
+Type ""
+Enter
+Type "# 6. Create derived metrics and pipe to a validation: so cool!"
+Enter
+Type `pb pl 'pl.read_csv("worldcities.csv").with_columns((pl.col("population") / 1000000).alias("pop_millions")).filter(pl.col("pop_millions") > 0.5)' --pipe | pb validate --check col-vals-gt --column pop_millions --value 0`
+Enter
+Sleep 12s
+
+Type ""
+Enter
+Type "# The pb pl command truly enables a powerful new class of data quality pipelines!"
+Enter
+Sleep 5s

{pointblank-0.11.2 → pointblank-0.11.4}/docs/demos/cli-interactive/index.qmd

@@ -69,6 +69,15 @@ Discover how to profile and explore data using CLI tools that are quick and easy
 *Preview data, find missing values, and generate column summaries*
 :::
 
+### Using Polars
+
+You can use Polars in the CLI to load and transform data, and, pass the data to other CLI tools:
+
+::: {.gif-container}
+![Using Polars](../../assets/vhs/cli-using-polars.gif)
+*Preview data, find missing values, and generate column summaries*
+:::
+
 ### CI/CD Integration & Automation
 
 Learn how to integrate data quality checks into automated pipelines:

{pointblank-0.11.2 → pointblank-0.11.4}/docs/index.qmd

@@ -220,6 +220,11 @@ PRs with code changes). Just know that anything you can do to help would be very
 Please read over the [contributing guidelines](https://github.com/posit-dev/pointblank/blob/main/CONTRIBUTING.md)
 for information on how to get started.
 
+## Pointblank for R
+
+There's also a version of Pointblank for R, which has been around since 2017 and is widely used in
+the R community. You can find it at <https://github.com/rstudio/pointblank>.
+
 ## Roadmap
 
 We're actively working on enhancing Pointblank with: