pointblank 0.13.4__tar.gz → 0.14.0__tar.gz

{pointblank-0.13.4 → pointblank-0.14.0}/CITATION.cff

@@ -3,7 +3,7 @@ message: 'If you wish to cite the "Pointblank" package use:'
 type: software
 license: MIT
 title: "Pointblank: Find out if your data is what you think it is"
-version: 0.13.1
+version: 0.13.4
 abstract: Validate data in Polars and Pandas DataFrames and database tables.
   Validation pipelines can be made using easily-readable, consecutive validation
   steps. Upon execution of the validation plan, several reporting options are available.

{pointblank-0.13.4 → pointblank-0.14.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pointblank
-Version: 0.13.4
+Version: 0.14.0
 Summary: Find out if your data is what you think it is.
 Author-email: Richard Iannone <riannone@me.com>
 License: MIT License
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: commonmark>=0.9.1
 Requires-Dist: importlib-metadata
-Requires-Dist: great_tables>=0.17.0
+Requires-Dist: great_tables>=0.19.0
 Requires-Dist: narwhals>=2.0.1
 Requires-Dist: typing_extensions>=3.10.0.0
 Requires-Dist: requests>=2.31.0
@@ -132,13 +132,63 @@ _Data validation made beautiful and powerful_
    <a href="translations/README.ar.md">العربية</a>
 </div>
 
-## What is Pointblank?
+<br>
+
+Pointblank takes a different approach to data quality. It doesn't have to be a tedious technical task. Rather, it can become a process focused on clear communication between team members. While other validation libraries focus solely on catching errors, Pointblank is great at both **finding issues and sharing insights**. Our beautiful, customizable reports turn validation results into conversations with stakeholders, making data quality issues immediately understandable and actionable for everyone on your team.
+
+**Get started in minutes, not hours.** Pointblank's AI-powered [`DraftValidation`](https://posit-dev.github.io/pointblank/user-guide/draft-validation.html) feature analyzes your data and suggests intelligent validation rules automatically. So there's no need to stare at an empty validation script wondering where to begin. Pointblank can kickstart your data quality journey so you can focus on what matters most.
+
+Whether you're a data scientist who needs to quickly communicate data quality findings, a data engineer building robust pipelines, or an analyst presenting data quality results to business stakeholders, Pointblank helps you to turn data quality from an afterthought into a competitive advantage.
+
+## Getting Started with AI-Powered Validation Drafting
+
+The `DraftValidation` class uses LLMs to analyze your data and generate a complete validation plan with intelligent suggestions. This helps you quickly get started with data validation or jumpstart a new project.
+
+```python
+import pointblank as pb
 
-Pointblank is a powerful, yet elegant data validation framework for Python that transforms how you ensure data quality. With its intuitive, chainable API, you can quickly validate your data against comprehensive quality checks and visualize results through stunning, interactive reports that make data issues immediately actionable.
+# Load your data
+data = pb.load_dataset("game_revenue")              # A sample dataset
 
-Whether you're a data scientist, data engineer, or analyst, Pointblank helps you catch data quality issues before they impact your analyses or downstream systems.
+# Use DraftValidation to generate a validation plan
+pb.DraftValidation(data=data, model="anthropic:claude-sonnet-4-5")
+```
 
-## Getting Started in 30 Seconds
+The output is a complete validation plan with intelligent suggestions based on your data:
+
+```python
+import pointblank as pb
+
+# The validation plan
+validation = (
+    pb.Validate(
+        data=data,
+        label="Draft Validation",
+        thresholds=pb.Thresholds(warning=0.10, error=0.25, critical=0.35)
+    )
+    .col_vals_in_set(columns="item_type", set=["iap", "ad"])
+    .col_vals_gt(columns="item_revenue", value=0)
+    .col_vals_between(columns="session_duration", left=3.2, right=41.0)
+    .col_count_match(count=11)
+    .row_count_match(count=2000)
+    .rows_distinct()
+    .interrogate()
+)
+
+validation
+```
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/pointblank-draft-validation-report.png" width="800px">
+</div>
+
+<br>
+
+Copy, paste, and customize the generated validation plan for your needs.
+
+## Chainable Validation API
+
+Pointblank's chainable API makes validation simple and readable. The same pattern always applies: (1) start with `Validate`, (2) add validation steps, and (3) finish with `interrogate()`.
 
 ```python
 import pointblank as pb
@@ -164,6 +214,12 @@ validation
 
 <br>
 
+Once you have an interrogated `validation` object, you can leverage a variety of methods to extract insights like:
+
+- getting detailed reports for single steps to see what went wrong
+- filtering tables based on validation results
+- extracting problematic data for debugging
+
 ## Why Choose Pointblank?
 
 - **Works with your existing stack**: Seamlessly integrates with Polars, Pandas, DuckDB, MySQL, PostgreSQL, SQLite, Parquet, PySpark, Snowflake, and more!
@@ -172,7 +228,9 @@ validation
 - **Threshold-based alerts**: Set 'warning', 'error', and 'critical' thresholds with custom actions
 - **Practical outputs**: Use validation results to filter tables, extract problematic data, or trigger downstream processes
 
-## Real-World Example
+## Production-Ready Validation Pipeline
+
+Here's how Pointblank handles complex, real-world scenarios with advanced features like threshold management, automated alerts, and comprehensive business rule validation:
 
 ```python
 import pointblank as pb
@@ -280,7 +338,7 @@ validation = pb.yaml_interrogate("validation.yaml")
 validation.get_tabular_report().show()
 ```
 
-This approach is perfect for:
+This approach is suitable for:
 
 - **CI/CD pipelines**: Store validation rules alongside your code
 - **Team collaboration**: Share validation logic in a readable format

{pointblank-0.13.4 → pointblank-0.14.0}/README.md

@@ -36,13 +36,63 @@ _Data validation made beautiful and powerful_
    <a href="translations/README.ar.md">العربية</a>
 </div>
 
-## What is Pointblank?
+<br>
+
+Pointblank takes a different approach to data quality. It doesn't have to be a tedious technical task. Rather, it can become a process focused on clear communication between team members. While other validation libraries focus solely on catching errors, Pointblank is great at both **finding issues and sharing insights**. Our beautiful, customizable reports turn validation results into conversations with stakeholders, making data quality issues immediately understandable and actionable for everyone on your team.
+
+**Get started in minutes, not hours.** Pointblank's AI-powered [`DraftValidation`](https://posit-dev.github.io/pointblank/user-guide/draft-validation.html) feature analyzes your data and suggests intelligent validation rules automatically. So there's no need to stare at an empty validation script wondering where to begin. Pointblank can kickstart your data quality journey so you can focus on what matters most.
+
+Whether you're a data scientist who needs to quickly communicate data quality findings, a data engineer building robust pipelines, or an analyst presenting data quality results to business stakeholders, Pointblank helps you to turn data quality from an afterthought into a competitive advantage.
+
+## Getting Started with AI-Powered Validation Drafting
+
+The `DraftValidation` class uses LLMs to analyze your data and generate a complete validation plan with intelligent suggestions. This helps you quickly get started with data validation or jumpstart a new project.
+
+```python
+import pointblank as pb
 
-Pointblank is a powerful, yet elegant data validation framework for Python that transforms how you ensure data quality. With its intuitive, chainable API, you can quickly validate your data against comprehensive quality checks and visualize results through stunning, interactive reports that make data issues immediately actionable.
+# Load your data
+data = pb.load_dataset("game_revenue")              # A sample dataset
 
-Whether you're a data scientist, data engineer, or analyst, Pointblank helps you catch data quality issues before they impact your analyses or downstream systems.
+# Use DraftValidation to generate a validation plan
+pb.DraftValidation(data=data, model="anthropic:claude-sonnet-4-5")
+```
 
-## Getting Started in 30 Seconds
+The output is a complete validation plan with intelligent suggestions based on your data:
+
+```python
+import pointblank as pb
+
+# The validation plan
+validation = (
+    pb.Validate(
+        data=data,
+        label="Draft Validation",
+        thresholds=pb.Thresholds(warning=0.10, error=0.25, critical=0.35)
+    )
+    .col_vals_in_set(columns="item_type", set=["iap", "ad"])
+    .col_vals_gt(columns="item_revenue", value=0)
+    .col_vals_between(columns="session_duration", left=3.2, right=41.0)
+    .col_count_match(count=11)
+    .row_count_match(count=2000)
+    .rows_distinct()
+    .interrogate()
+)
+
+validation
+```
+
+<div align="center">
+<img src="https://posit-dev.github.io/pointblank/assets/pointblank-draft-validation-report.png" width="800px">
+</div>
+
+<br>
+
+Copy, paste, and customize the generated validation plan for your needs.
+
+## Chainable Validation API
+
+Pointblank's chainable API makes validation simple and readable. The same pattern always applies: (1) start with `Validate`, (2) add validation steps, and (3) finish with `interrogate()`.
 
 ```python
 import pointblank as pb
@@ -68,6 +118,12 @@ validation
 
 <br>
 
+Once you have an interrogated `validation` object, you can leverage a variety of methods to extract insights like:
+
+- getting detailed reports for single steps to see what went wrong
+- filtering tables based on validation results
+- extracting problematic data for debugging
+
 ## Why Choose Pointblank?
 
 - **Works with your existing stack**: Seamlessly integrates with Polars, Pandas, DuckDB, MySQL, PostgreSQL, SQLite, Parquet, PySpark, Snowflake, and more!
@@ -76,7 +132,9 @@ validation
 - **Threshold-based alerts**: Set 'warning', 'error', and 'critical' thresholds with custom actions
 - **Practical outputs**: Use validation results to filter tables, extract problematic data, or trigger downstream processes
 
-## Real-World Example
+## Production-Ready Validation Pipeline
+
+Here's how Pointblank handles complex, real-world scenarios with advanced features like threshold management, automated alerts, and comprehensive business rule validation:
 
 ```python
 import pointblank as pb
@@ -184,7 +242,7 @@ validation = pb.yaml_interrogate("validation.yaml")
 validation.get_tabular_report().show()
 ```
 
-This approach is perfect for:
+This approach is suitable for:
 
 - **CI/CD pipelines**: Store validation rules alongside your code
 - **Team collaboration**: Share validation logic in a readable format

{pointblank-0.13.4 → pointblank-0.14.0}/docs/_quarto.yml

@@ -6,6 +6,9 @@ project:
 
 format:
   html:
+    include-in-header:
+      - text: |
+          <script src="https://cdn.jsdelivr.net/gh/posit-dev/supported-by-posit/js/badge.min.js"></script>
     theme: flatly
     css:
       - styles.css
@@ -165,6 +168,7 @@ quartodoc:
         - name: Validate.col_count_match
         - name: Validate.conjointly
         - name: Validate.specially
+        - name: Validate.prompt
     - title: Column Selection
       desc: >
         A flexible way to select columns for validation is to use the `col()` function along with
@@ -238,10 +242,12 @@ quartodoc:
         The *YAML* group contains functions that allow for the use of YAML to orchestrate validation
         workflows. The `yaml_interrogate()` function can be used to run a validation workflow from
         YAML strings or files. The `validate_yaml()` function checks if the YAML configuration
-        passes its own validity checks.
+        passes its own validity checks. The `yaml_to_python()` function converts YAML configuration
+        to equivalent Python code.
       contents:
         - name: yaml_interrogate
         - name: validate_yaml
+        - name: yaml_to_python
     - title: Utility Functions
       desc: >
         The *Utility Functions* group contains functions that are useful accessing metadata about
@@ -254,6 +260,8 @@ quartodoc:
         - name: get_row_count
         - name: get_action_metadata
         - name: get_validation_summary
+        - name: write_file
+        - name: read_file
         - name: config
     - title: Prebuilt Actions
       desc: >

pointblank-0.14.0/docs/demos/check-for-freshness/index.qmd

@@ -0,0 +1,125 @@
+---
+pagetitle: "Examples: Validating Data Freshness"
+notebook-links: false
+page-navigation: false
+toc: false
+html-table-processing: none
+---
+
+### Validating Data Freshness
+
+Use date/datetime-based validations to ensure your data is current and recent. This is critical for applications that depend on timely data updates.
+
+```{python}
+#| echo: false
+
+import pointblank as pb
+import polars as pl
+from datetime import date, datetime, timedelta
+
+# Create sample data with mixed freshness levels
+freshness_data = pl.DataFrame({
+    "data_timestamp": [
+        datetime(2023, 12, 28, 10, 30),  # 3 days ago from Dec 31
+        datetime(2023, 12, 29, 14, 15),  # 2 days ago
+        datetime(2023, 12, 30, 9, 45),   # 1 day ago
+        datetime(2023, 12, 31, 16, 20),  # Today
+    ],
+    "sensor_id": ["TEMP_01", "TEMP_02", "TEMP_01", "TEMP_03"],
+    "reading": [22.5, 21.8, 23.1, 22.9],
+    "quality_score": [0.95, 0.88, 0.92, 0.97]
+})
+
+# Assuming today is 2023-12-31, check for data freshness
+current_date = date(2023, 12, 31)
+freshness_cutoff = current_date - timedelta(days=2)  # Data should be within 2 days
+
+validation = (
+    pb.Validate(freshness_data)
+    .specially(
+        expr=lambda df: df.filter(
+            pl.col("data_timestamp").dt.date() >= freshness_cutoff
+        ).height > 0,
+        brief=f"Recent data available (within 2 days of {current_date})"
+    )
+    .col_vals_ge(
+        columns="data_timestamp",
+        value=current_date - timedelta(days=7),  # Within last week
+        brief="All data points are from the last week"
+    )
+    .specially(
+        expr=lambda df: (
+            df.select(pl.col("data_timestamp").max()).item().date() >= current_date
+        ),
+        brief="Most recent data is from today"
+    )
+    .col_vals_not_null(
+        columns="data_timestamp",
+        brief="No missing timestamps"
+    )
+    .interrogate()
+)
+
+validation
+```
+
+```python
+import pointblank as pb
+import polars as pl
+from datetime import date, datetime, timedelta
+
+# Create sample data with mixed freshness levels
+freshness_data = pl.DataFrame({
+    "data_timestamp": [
+        datetime(2023, 12, 28, 10, 30),  # 3 days ago from Dec 31
+        datetime(2023, 12, 29, 14, 15),  # 2 days ago
+        datetime(2023, 12, 30, 9, 45),   # 1 day ago
+        datetime(2023, 12, 31, 16, 20),  # Today
+    ],
+    "sensor_id": ["TEMP_01", "TEMP_02", "TEMP_01", "TEMP_03"],
+    "reading": [22.5, 21.8, 23.1, 22.9],
+    "quality_score": [0.95, 0.88, 0.92, 0.97]
+})
+
+# Assuming today is 2023-12-31, check for data freshness
+current_date = date(2023, 12, 31)
+freshness_cutoff = current_date - timedelta(days=2)  # Data should be within 2 days
+
+validation = (
+    pb.Validate(freshness_data)
+    .specially(
+        expr=lambda df: df.filter(
+            pl.col("data_timestamp").dt.date() >= freshness_cutoff
+        ).height > 0,
+        brief=f"Recent data available (within 2 days of {current_date})"
+    )
+    .col_vals_ge(
+        columns="data_timestamp",
+        value=current_date - timedelta(days=7),  # Within last week
+        brief="All data points are from the last week"
+    )
+    .specially(
+        expr=lambda df: (
+            df.select(pl.col("data_timestamp").max()).item().date() >= current_date
+        ),
+        brief="Most recent data is from today"
+    )
+    .col_vals_not_null(
+        columns="data_timestamp",
+        brief="No missing timestamps"
+    )
+    .interrogate()
+)
+
+validation
+```
+
+<details>
+<summary>Preview of Input Table</summary>
+
+```{python}
+# | echo: false
+pb.preview(freshness_data)
+```
+
+</details>

pointblank-0.14.0/docs/demos/datetime-validations/index.qmd

@@ -0,0 +1,155 @@
+---
+pagetitle: "Examples: Date and Datetime Validations"
+notebook-links: false
+page-navigation: false
+toc: false
+html-table-processing: none
+---
+
+### Date and Datetime Validations
+
+**pointblank** provides comprehensive support for validating date and datetime values, including timezone-aware comparisons. This ensures temporal data quality in applications that handle time-sensitive information.
+
+```{python}
+#| echo: false
+
+import pointblank as pb
+import polars as pl
+from datetime import date, datetime
+import pytz
+
+# Create sample data with various temporal data types
+temporal_data = pl.DataFrame({
+    "order_date": [
+        date(2023, 1, 15),
+        date(2023, 6, 10),
+        date(2023, 12, 5),
+        date(2024, 3, 20)
+    ],
+    "created_at": [
+        datetime(2023, 1, 15, 9, 30, 0),
+        datetime(2023, 6, 10, 14, 45, 30),
+        datetime(2023, 12, 5, 8, 15, 0),
+        datetime(2024, 3, 20, 17, 22, 45)
+    ],
+    "event_time_tz": [
+        datetime(2023, 1, 15, 9, 0, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2023, 6, 10, 12, 30, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2023, 12, 5, 15, 45, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2024, 3, 20, 18, 15, tzinfo=pytz.timezone("America/New_York"))
+    ],
+    "order_id": [1001, 1002, 1003, 1004],
+    "amount": [150.0, 275.5, 89.99, 420.00]
+})
+
+validation = (
+    pb.Validate(temporal_data)
+    .col_vals_ge(
+        columns="order_date",
+        value=date(2023, 1, 1),
+        brief="Orders are from 2023 or later"
+    )
+    .col_vals_between(
+        columns="created_at",
+        left=datetime(2023, 1, 1, 0, 0, 0),
+        right=datetime(2024, 12, 31, 23, 59, 59),
+        brief="Creation timestamps within expected range"
+    )
+    .col_vals_ge(
+        columns="event_time_tz",
+        value=datetime(2023, 1, 1, 8, 0, tzinfo=pytz.timezone("America/New_York")),
+        brief="Timezone-aware events after 8 AM Eastern"
+    )
+    .col_schema_match(
+        pb.Schema(
+            columns=[
+                ("order_date", "Date"),
+                ("created_at", "Datetime(time_unit='us', time_zone=None)"),
+                ("event_time_tz", "Datetime(time_unit='us', time_zone='America/New_York')"),
+                ("order_id", "Int64"),
+                ("amount", "Float64")
+            ]
+        ),
+        brief="Schema includes proper date/datetime types"
+    )
+    .interrogate()
+)
+
+validation
+```
+
+```python
+import pointblank as pb
+import polars as pl
+from datetime import date, datetime
+import pytz
+
+# Create sample data with various temporal data types
+temporal_data = pl.DataFrame({
+    "order_date": [
+        date(2023, 1, 15),
+        date(2023, 6, 10),
+        date(2023, 12, 5),
+        date(2024, 3, 20)
+    ],
+    "created_at": [
+        datetime(2023, 1, 15, 9, 30, 0),
+        datetime(2023, 6, 10, 14, 45, 30),
+        datetime(2023, 12, 5, 8, 15, 0),
+        datetime(2024, 3, 20, 17, 22, 45)
+    ],
+    "event_time_tz": [
+        datetime(2023, 1, 15, 9, 0, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2023, 6, 10, 12, 30, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2023, 12, 5, 15, 45, tzinfo=pytz.timezone("America/New_York")),
+        datetime(2024, 3, 20, 18, 15, tzinfo=pytz.timezone("America/New_York"))
+    ],
+    "order_id": [1001, 1002, 1003, 1004],
+    "amount": [150.0, 275.5, 89.99, 420.00]
+})
+
+validation = (
+    pb.Validate(temporal_data)
+    .col_vals_ge(
+        columns="order_date",
+        value=date(2023, 1, 1),
+        brief="Orders are from 2023 or later"
+    )
+    .col_vals_between(
+        columns="created_at",
+        left=datetime(2023, 1, 1, 0, 0, 0),
+        right=datetime(2024, 12, 31, 23, 59, 59),
+        brief="Creation timestamps within expected range"
+    )
+    .col_vals_ge(
+        columns="event_time_tz",
+        value=datetime(2023, 1, 1, 8, 0, tzinfo=pytz.timezone("America/New_York")),
+        brief="Timezone-aware events after 8 AM Eastern"
+    )
+    .col_schema_match(
+        pb.Schema(
+            columns=[
+                ("order_date", "Date"),
+                ("created_at", "Datetime(time_unit='us', time_zone=None)"),
+                ("event_time_tz", "Datetime(time_unit='us', time_zone='America/New_York')"),
+                ("order_id", "Int64"),
+                ("amount", "Float64")
+            ]
+        ),
+        brief="Schema includes proper date/datetime types"
+    )
+    .interrogate()
+)
+
+validation
+```
+
+<details>
+<summary>Preview of Input Table</summary>
+
+```{python}
+# | echo: false
+pb.preview(temporal_data)
+```
+
+</details>

{pointblank-0.13.4 → pointblank-0.14.0}/docs/demos/index.qmd

@@ -153,8 +153,11 @@ For far more specialized validations, modify the table with the `pre=` argument
 [Verifying Row and Column Counts](./check-row-column-counts/index.qmd)<br>
 Check the dimensions of the table with the `*_count_match()` validation methods.
 
-[Validating Datetime Values and Data Freshness](./check-for-freshness/index.qmd)<br>
-Use date-based validations to ensure your data is current and within expected time ranges.
+[Validating Data Freshness](./check-for-freshness/index.qmd)<br>
+Use date-based validations to ensure your data is current and recent.
+
+[Date and Datetime Validations](./datetime-validations/index.qmd)<br>
+Comprehensive examples of date, datetime, and timezone-aware datetime comparisons.
 
 [Custom Validation with `specially()`](./custom-validation-specially/index.qmd)<br>
 Create bespoke validations using `specially()` to implement domain-specific business rules.

{pointblank-0.13.4 → pointblank-0.14.0}/docs/demos/mutate-table-in-step/index.qmd

@@ -17,6 +17,18 @@ import pointblank as pb
 import polars as pl
 import narwhals as nw
 
+# Define preprocessing functions
+def get_median_a(df):
+    """Use a Polars expression to aggregate column `a`."""
+    return df.select(pl.median("a"))
+
+def add_b_length_column(df):
+    """Use Narwhals to add a string length column `b_len`."""
+    return (
+        nw.from_native(df)
+        .with_columns(b_len=nw.col("b").str.len_chars())
+    )
+
 validation = (
     pb.Validate(
         data=pb.load_dataset(dataset="small_table", tbl_type="polars")
@@ -24,14 +36,12 @@ validation = (
     .col_vals_between(
         columns="a",
         left=3, right=6,
-        pre=lambda df: df.select(pl.median("a"))    # Use a Polars expression to aggregate
+        pre=get_median_a
     )
     .col_vals_eq(
         columns="b_len",
         value=9,
-        pre=lambda dfn: dfn.with_columns(           # Use a Narwhals expression, identified
-            b_len=nw.col("b").str.len_chars()       # by the 'dfn' here
-        )
+        pre=add_b_length_column
     )
     .interrogate()
 )
@@ -44,6 +54,18 @@ import pointblank as pb
 import polars as pl
 import narwhals as nw
 
+# Define preprocessing functions
+def get_median_a(df):
+    """Use a Polars expression to aggregate column `a`."""
+    return df.select(pl.median("a"))
+
+def add_b_length_column(df):
+    """Use Narwhals to add a string length column `b_len`."""
+    return (
+        nw.from_native(df)
+        .with_columns(b_len=nw.col("b").str.len_chars())
+    )
+
 validation = (
     pb.Validate(
         data=pb.load_dataset(dataset="small_table", tbl_type="polars")
@@ -51,14 +73,12 @@ validation = (
     .col_vals_between(
         columns="a",
         left=3, right=6,
-        pre=lambda df: df.select(pl.median("a"))    # Use a Polars expression to aggregate
+        pre=get_median_a
     )
     .col_vals_eq(
         columns="b_len",
         value=9,
-        pre=lambda dfn: dfn.with_columns(           # Use a Narwhals expression, identified
-            b_len=nw.col("b").str.len_chars()       # by the 'dfn' here
-        )
+        pre=add_b_length_column
     )
     .interrogate()
 )