@datarecce/ui 0.1.29 → 0.1.31

package/recce-source/RECCE_CLOUD.md

@@ -0,0 +1,81 @@
+# Recce Cloud CLI
+
+A lightweight, standalone CLI for managing Recce Cloud operations in CI/CD environments.
+
+## Overview
+
+The `recce-cloud` package provides a minimal CLI tool specifically designed for cloud operations, without the heavy dependencies of the full `recce` package. This makes it ideal for CI/CD pipelines where installation speed matters.
+
+## Dependency Comparison
+
+**recce-cloud** (3 dependencies):
+- click
+- requests
+- rich
+
+**recce** (20+ dependencies):
+- boto3, click, deepdiff, fastapi, GitPython, itsdangerous, jinja2, packaging, portalocker, py-markdown-table, pydantic, PyGithub, python-dateutil, python-multipart, pytz, requests, rich, ruamel.yaml, sentry-sdk, sqlglot, uvicorn, watchdog, websockets
+
+## Installation
+
+### Development Mode
+
+From the repository root:
+
+```bash
+python setup_cloud.py develop
+```
+
+### Production Install (when published to PyPI)
+
+```bash
+pip install recce-cloud
+```
+
+## Usage
+
+```bash
+# Check version
+recce-cloud --version
+
+# Get help
+recce-cloud --help
+```
+
+## Package Structure
+
+```
+recce_cloud/
+├── __init__.py          # Package initialization and version
+├── cli.py              # Main CLI entry point
+└── VERSION             # Version file (synced with main recce)
+```
+
+## CI/CD Integration Example
+
+```yaml
+# GitHub Actions example
+- name: Install recce-cloud
+  run: pip install recce-cloud
+
+- name: Use recce-cloud
+  run: recce-cloud --version
+```
+
+## Development
+
+The `recce-cloud` package is developed within the same repository as `recce` but is packaged and distributed separately:
+
+- **Main package**: `setup.py` → installs `recce` command
+- **Cloud package**: `setup_cloud.py` → installs `recce-cloud` command
+
+Both packages share the same VERSION file to keep versions synchronized.
+
+## Future Commands
+
+The CLI is designed to be extended with cloud-specific commands such as:
+- `recce-cloud list-sessions` - List Recce Cloud sessions
+- `recce-cloud purge` - Purge cloud state files
+- `recce-cloud upload` - Upload state files to cloud
+
+Currently, only the basic CLI structure with version command is implemented.

package/recce-source/SECURITY.md

@@ -0,0 +1,25 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 0.x     | :white_check_mark: |
+
+## Reporting a Vulnerability
+
+Please report (suspected) security vulnerabilities to **dev@infuseai.io**. You will receive a response from us within 48 hours. In the response, we will confirm whether the vulnerability was indeed found, and we will discuss the steps towards a solution.
+
+It's important to never publicly disclose a security vulnerability before it has been fully resolved, as this can potentially put the wider community at risk.
+
+### Security Vulnerability Response
+
+* Acknowledgement of receipt of the report
+* Regular updates about the progress towards fixing the vulnerability
+* An announced release date for the patch fixing the vulnerability
+
+### Public Disclosure Timing
+
+After the initial reply to your report, we will keep you informed of the progress towards a fix and full announcement. We aim to handle all security vulnerabilities in a timely manner. After the vulnerability has been fixed, we will publish a security advisory along with credit to the reporter if desired.
+
+Thank you for helping to keep Recce and its users safe!

package/recce-source/docs/PACKAGING.md

@@ -0,0 +1,340 @@
+# Recce Packaging Guide
+
+This document explains how `recce` and `recce-cloud` packages are structured, built, and published.
+
+## Overview
+
+The Recce repository contains two separate Python packages:
+
+1. **`recce`** - Full-featured CLI for local development and review
+2. **`recce-cloud`** - Lightweight CLI for CI/CD environments
+
+Both packages share the same version number from `recce/VERSION`.
+
+## Package Structure
+
+```
+recce/
+├── recce/                    # Main package (full-featured)
+│   ├── VERSION              # Single source of truth for both packages
+│   └── ...
+├── recce_cloud/             # Lightweight cloud package
+│   ├── VERSION              # Synced from recce/VERSION during build
+│   ├── __init__.py          # Reads VERSION with fallback
+│   └── ...
+├── setup.py                 # Build config for 'recce' package
+├── setup_cloud.py           # Build config for 'recce-cloud' package
+├── pyproject.toml           # Modern Python build system config
+└── Makefile                 # Build automation
+```
+
+## Version Management
+
+### Single Source of Truth
+
+All version information comes from `recce/VERSION`:
+
+```
+1.24.0
+```
+
+### How Versioning Works
+
+1. **Build time**: `setup.py` and `setup_cloud.py` read `recce/VERSION`
+2. **Build process**: `make build-cloud` copies `recce/VERSION` to `recce_cloud/VERSION`
+3. **Runtime**: `recce_cloud/__init__.py` reads VERSION with fallback logic:
+   - First tries `recce_cloud/VERSION` (for installed package)
+   - Falls back to `../recce/VERSION` (for development)
+
+### Updating Version
+
+To release a new version:
+
+```bash
+# 1. Update the version file
+echo "1.25.0" > recce/VERSION
+
+# 2. Commit the change
+git add recce/VERSION
+git commit -m "chore: bump version to 1.25.0"
+
+# 3. Tag the release
+git tag v1.25.0
+git push && git push --tags
+```
+
+## Development Workflow
+
+### Installing for Development
+
+**Install recce (full package):**
+```bash
+make install-dev
+# OR
+pip install -e .[dev]
+```
+
+**Install recce-cloud (lightweight package):**
+```bash
+make install-cloud-dev
+# OR
+cd /path/to/recce
+python setup_cloud.py develop
+```
+
+**Install both packages:**
+```bash
+make install-dev
+make install-cloud-dev
+```
+
+### Running Tests
+
+```bash
+# Test recce
+make test
+
+# Test recce-cloud
+pytest tests/recce_cloud/
+
+# Test with coverage
+make test-coverage
+```
+
+### Code Quality
+
+```bash
+# Format code (both packages)
+make format
+make format-cloud
+
+# Check code quality (both packages)
+make check
+make check-cloud
+
+# Lint (both packages)
+make flake8
+make flake8-cloud
+```
+
+## Build Process
+
+### Building Packages
+
+**Build recce only:**
+```bash
+make build
+```
+This will:
+1. Clean build artifacts
+2. Build frontend (pnpm install + pnpm build)
+3. Build Python package (python setup.py sdist bdist_wheel)
+
+**Build recce-cloud only:**
+```bash
+make build-cloud
+```
+This will:
+1. Clean build artifacts
+2. Sync VERSION file to recce_cloud/
+3. Build Python package (python setup_cloud.py sdist bdist_wheel)
+
+**Build both packages:**
+```bash
+make build-all
+```
+This will:
+1. Clean build artifacts
+2. Build frontend
+3. Build both recce and recce-cloud packages
+
+**Build frontend only:**
+```bash
+make build-frontend
+```
+
+**Clean build artifacts:**
+```bash
+make clean-build
+```
+
+This creates packages in `dist/`:
+- `recce-1.24.0.tar.gz` (source distribution)
+- `recce-1.24.0-py3-none-any.whl` (wheel)
+- `recce-cloud-1.24.0.tar.gz` (source distribution)
+- `recce_cloud-1.24.0-py3-none-any.whl` (wheel)
+
+## Publishing to PyPI
+
+### Prerequisites
+
+1. Install publishing tools:
+   ```bash
+   pip install twine
+   ```
+
+2. Configure PyPI credentials:
+   ```bash
+   # Create ~/.pypirc with:
+   [pypi]
+   username = __token__
+   password = pypi-<your-token>
+   ```
+
+### Publishing Workflow
+
+**1. Build both packages:**
+```bash
+make build-all
+```
+
+**2. Check packages:**
+```bash
+twine check dist/*
+```
+
+**3. Upload to TestPyPI (optional):**
+```bash
+twine upload --repository testpypi dist/*
+```
+
+**4. Upload to PyPI:**
+```bash
+# Upload recce-cloud first
+twine upload dist/recce-cloud-*
+
+# Then upload recce
+twine upload dist/recce-*
+```
+
+**Note:** Always publish `recce-cloud` before `recce` to ensure the dependency is available.
+
+### Future: Add recce → recce-cloud Dependency
+
+When ready, update `setup.py`:
+
+```python
+install_requires=[
+    "recce-cloud",  # Add this line
+    "boto3",
+    # ... other deps
+]
+```
+
+This makes `pip install recce` automatically install `recce-cloud`.
+
+## Installation Scenarios
+
+### Scenario 1: Lightweight (CI/CD)
+
+```bash
+pip install recce-cloud
+```
+
+**Result:**
+- Only `recce-cloud` installed
+- 3 dependencies (click, requests, rich)
+- ~5MB installed size
+- Only `recce-cloud` command available
+
+### Scenario 2: Full (Development)
+
+```bash
+pip install recce
+```
+
+**Result:**
+- Only `recce` installed (currently)
+- 20+ dependencies
+- ~50MB+ installed size
+- Only `recce` command available
+
+**Future (when dependency added):**
+- Both `recce` and `recce-cloud` installed
+- `recce-cloud` downloaded automatically from PyPI
+- Both commands available
+
+## Troubleshooting
+
+### VERSION file not found
+
+**Error:**
+```
+FileNotFoundError: [Errno 2] No such file or directory: 'VERSION'
+```
+
+**Solution:**
+Ensure you're running build commands from the repository root, or use Makefile targets.
+
+### recce-cloud command not found after install
+
+**Cause:** Package not installed correctly or not in PATH
+
+**Solution:**
+```bash
+# Verify installation
+pip show recce-cloud
+
+# Check entry point
+pip show -f recce-cloud | grep recce-cloud
+
+# Reinstall
+pip uninstall recce-cloud
+pip install recce-cloud
+```
+
+### Development install doesn't work
+
+**Issue:** `python setup_cloud.py develop` fails with path errors
+
+**Solution:**
+Run from repository root:
+```bash
+cd /path/to/recce
+python setup_cloud.py develop
+```
+
+Or use Makefile:
+```bash
+make install-cloud-dev
+```
+
+## Reference
+
+### Makefile Targets
+
+| Command | Description |
+|---------|-------------|
+| `make install` | Install recce package |
+| `make install-dev` | Install recce with dev dependencies |
+| `make install-cloud` | Install recce-cloud package |
+| `make install-cloud-dev` | Install recce-cloud in development mode |
+| `make build` | Build recce package |
+| `make build-cloud` | Build recce-cloud package |
+| `make build-all` | Build both packages |
+| `make clean-build` | Clean build artifacts |
+| `make test` | Run tests |
+| `make format` | Format code (recce) |
+| `make format-cloud` | Format code (recce-cloud) |
+| `make check` | Check code quality (recce) |
+| `make check-cloud` | Check code quality (recce-cloud) |
+
+### Package Dependencies
+
+**recce-cloud:**
+- `click>=7.1`
+- `requests>=2.28.1`
+- `rich>=12.0.0`
+
+**recce:**
+- All recce-cloud dependencies (when dependency added)
+- Plus: boto3, fastapi, uvicorn, pydantic, dbt, etc.
+
+## Best Practices
+
+1. **Always build both packages together** using `make build-all`
+2. **Test installation** in a clean virtual environment before publishing
+3. **Publish recce-cloud first** to ensure dependency is available
+4. **Use Makefile targets** for consistent builds
+5. **Keep VERSION in sync** by only editing `recce/VERSION`
+6. **Run tests** before publishing: `make test && pytest tests/recce_cloud/`

package/recce-source/docs/README.md

@@ -0,0 +1 @@
+Please check the [Recce documentation](https://datarecce.io/docs/)

package/recce-source/integration_tests/dbt/dbt_project.yml

@@ -0,0 +1,26 @@
+name: "jaffle_shop"
+
+config-version: 2
+version: "0.1"
+
+profile: "jaffle_shop"
+
+model-paths: ["models"]
+seed-paths: ["seeds"]
+test-paths: ["tests"]
+analysis-paths: ["analysis"]
+macro-paths: ["macros"]
+
+target-path: "target"
+clean-targets:
+  - "target"
+  - "dbt_modules"
+  - "logs"
+
+require-dbt-version: [">=1.0.0", "<2.0.0"]
+
+models:
+  jaffle_shop:
+    materialized: table
+    staging:
+      materialized: view

package/recce-source/integration_tests/dbt/models/customers.sql

@@ -0,0 +1,69 @@
+with customers as (
+
+    select * from {{ ref('stg_customers') }}
+
+),
+
+orders as (
+
+    select * from {{ ref('stg_orders') }}
+
+),
+
+payments as (
+
+    select * from {{ ref('stg_payments') }}
+
+),
+
+customer_orders as (
+
+        select
+        customer_id,
+
+        min(order_date) as first_order,
+        max(order_date) as most_recent_order,
+        count(order_id) as number_of_orders
+    from orders
+
+    group by customer_id
+
+),
+
+customer_payments as (
+
+    select
+        orders.customer_id,
+        sum(amount) as total_amount
+
+    from payments
+
+    left join orders on
+         payments.order_id = orders.order_id
+
+    group by orders.customer_id
+
+),
+
+final as (
+
+    select
+        customers.customer_id,
+        customers.first_name,
+        customers.last_name,
+        customer_orders.first_order,
+        customer_orders.most_recent_order,
+        customer_orders.number_of_orders,
+        customer_payments.total_amount as customer_lifetime_value
+
+    from customers
+
+    left join customer_orders
+        on customers.customer_id = customer_orders.customer_id
+
+    left join customer_payments
+        on  customers.customer_id = customer_payments.customer_id
+
+)
+
+select * from final

package/recce-source/integration_tests/dbt/models/docs.md

@@ -0,0 +1,14 @@
+{% docs orders_status %}
+
+Orders can be one of the following statuses:
+
+| status         | description                                                                                                            |
+|----------------|------------------------------------------------------------------------------------------------------------------------|
+| placed         | The order has been placed but has not yet left the warehouse                                                           |
+| shipped        | The order has ben shipped to the customer and is currently in transit                                                  |
+| completed      | The order has been received by the customer                                                                            |
+| return_pending | The customer has indicated that they would like to return the order, but it has not yet been received at the warehouse |
+| returned       | The order has been returned by the customer and received at the warehouse                                              |
+
+
+{% enddocs %}

package/recce-source/integration_tests/dbt/models/orders.sql

@@ -0,0 +1,56 @@
+{% set payment_methods = ['credit_card', 'coupon', 'bank_transfer', 'gift_card'] %}
+
+with orders as (
+
+    select * from {{ ref('stg_orders') }}
+
+),
+
+payments as (
+
+    select * from {{ ref('stg_payments') }}
+
+),
+
+order_payments as (
+
+    select
+        order_id,
+
+        {% for payment_method in payment_methods -%}
+        sum(case when payment_method = '{{ payment_method }}' then amount else 0 end) as {{ payment_method }}_amount,
+        {% endfor -%}
+
+        sum(amount) as total_amount
+
+    from payments
+
+    group by order_id
+
+),
+
+final as (
+
+    select
+        orders.order_id,
+        orders.customer_id,
+        orders.order_date,
+        orders.status,
+
+        {% for payment_method in payment_methods -%}
+
+        order_payments.{{ payment_method }}_amount,
+
+        {% endfor -%}
+
+        order_payments.total_amount as amount
+
+    from orders
+
+
+    left join order_payments
+        on orders.order_id = order_payments.order_id
+
+)
+
+select * from final

package/recce-source/integration_tests/dbt/models/schema.yml

@@ -0,0 +1,82 @@
+version: 2
+
+models:
+  - name: customers
+    description: This table has basic information about a customer, as well as some derived facts based on a customer's orders
+
+    columns:
+      - name: customer_id
+        description: This is a unique identifier for a customer
+        data_tests:
+          - unique
+          - not_null
+
+      - name: first_name
+        description: Customer's first name. PII.
+
+      - name: last_name
+        description: Customer's last name. PII.
+
+      - name: first_order
+        description: Date (UTC) of a customer's first order
+
+      - name: most_recent_order
+        description: Date (UTC) of a customer's most recent order
+
+      - name: number_of_orders
+        description: Count of the number of orders a customer has placed
+
+      - name: total_order_amount
+        description: Total value (AUD) of a customer's orders
+
+  - name: orders
+    description: This table has basic information about orders, as well as some derived facts based on payments
+
+    columns:
+      - name: order_id
+        data_tests:
+          - unique
+          - not_null
+        description: This is a unique identifier for an order
+
+      - name: customer_id
+        description: Foreign key to the customers table
+        data_tests:
+          - not_null
+          - relationships:
+              to: ref('customers')
+              field: customer_id
+
+      - name: order_date
+        description: Date (UTC) that the order was placed
+
+      - name: status
+        description: '{{ doc("orders_status") }}'
+        data_tests:
+          - accepted_values:
+              values: ['placed', 'shipped', 'completed', 'return_pending', 'returned']
+
+      - name: amount
+        description: Total amount (AUD) of the order
+        data_tests:
+          - not_null
+
+      - name: credit_card_amount
+        description: Amount of the order (AUD) paid for by credit card
+        data_tests:
+          - not_null
+
+      - name: coupon_amount
+        description: Amount of the order (AUD) paid for by coupon
+        data_tests:
+          - not_null
+
+      - name: bank_transfer_amount
+        description: Amount of the order (AUD) paid for by bank transfer
+        data_tests:
+          - not_null
+
+      - name: gift_card_amount
+        description: Amount of the order (AUD) paid for by gift card
+        data_tests:
+          - not_null