fastdsu 0.0.0__tar.gz

fastdsu-0.0.0/.github/SECURITY_CHECKLIST.md

@@ -0,0 +1,193 @@
+# SECURITY.md
+
+## Summary  
+This policy explains how the public can responsibly [report vulnerabilities](reporting-a-vulnerability), and how DBT developers must protect [sensitive information](handling-secrets) and follow DBT’s required [security controls](security-controls) when working in GitHub.
+
+---
+
+## Reporting a Vulnerability
+
+If you believe you have found a security vulnerability, please submit a report via our HackerOne [form](https://hackerone.com/2680e4cd-0436-42a5-bd2a-37fd86367276/embedded_submissions/new)
+
+Please include:
+- Where the issue can be observed (URL, IP address or page)  
+- A brief description (e.g. “XSS vulnerability”)  
+- Safe, non‑destructive reproduction steps  
+
+**Disclosure guidelines**
+- Do **not** share vulnerability details beyond DBT and the asset owner  
+- HackerOne accounts are optional but allow updates  
+- You must agree to HackerOne’s Terms, Privacy Policy, and Disclosure Guidelines  
+- NCC Group triages reports within **five working days**  
+- DBT Cyber assists with coordination, but the asset owner is responsible for remediation  
+
+---
+
+## Handling Secrets
+
+These requirements apply to DBT developers.
+
+Information on secrets and their management can be found [here](https://dbis.sharepoint.com/:w:/r/sites/DDaTDirectorate/Shared%20Documents/Work%20-%20GitHub%20Security/Github%20Security%20Framework/Guidelines%20and%20Policies/GitHub%20Security%20Standards%20v0.6.docx?d=w022dea8105074e36af5450797083c297&csf=1&web=1&e=SR5out).
+
+Instructions on what to do in the event of a leak can be found [here](https://dbis.sharepoint.com/:w:/r/sites/DDaTDirectorate/Shared%20Documents/Work%20-%20GitHub%20Security/Github%20Security%20Framework/Incident%20Response/GitHub%20Repository%20Incident%20Playbook.docx?d=w9ba04ffa4a7c4ff38faaaf12ff030c94&csf=1&web=1&e=yZF5dO).
+
+**In summary, developers must:**
+- Never commit secrets or sensitive data to GitHub  
+- Use secure storage for managing secrets
+- Ensure no secrets appear in PRs, logs or config files
+- Follow incident‑response steps immediately if a leak occurs  
+
+---
+
+## Security Controls
+
+DBT uses several processes to strengthen the security posture of our GitHub repositories.
+
+Use this checklist to ensure your repository implements these processes:
+
+- [x] **Create or update `SECURITY_CHECKLIST.md`**  
+      See: [SECURITY_CHECKLIST.md](#security_checklistmd)
+
+- [x] **Review the CI/CD overview**  
+      See: [CI/CD Overview](#cicd-overview)
+
+- [x] **Set up the pre‑commit hook framework**  
+      See: [Pre‑Commit Hooks](#precommit-hooks)
+
+- [x] **Set up custom GitHub properties**  
+      See: [Custom GitHub Properties](#custom-github-properties)
+
+- [x] **Apply the DBT GitHub security policy**  
+      See: [GitHub Security Policy](#github-security-policy)
+
+- [x] **Ensure a `CODEOWNERS` file exists**  
+      See: [CODEOWNERS](#codeowners)
+
+- [x] **Review GitHub Safety Tips**  
+      See: [GitHub Safety Tips](#github-safety-tips)
+
+- [x] **Review Repository Access and Governance**  
+      See: [Repository Access & Governance](#repository-access--governance)
+
+- [x] **Review the Pull Request template**  
+      See: [Pull Request Template](#pull-request-template)
+
+- [x] **Review branch protection rules**  
+      See: [Branch Protection Rules](#branch-protection-rules)
+
+- [x] **Review push protection**  
+      See: [Push Protection](#push-protection)  
+
+---
+
+### SECURITY_CHECKLIST.md
+
+Create `SECURITY_CHECKLIST.md` in the root of your repository if not present. Copy the checklist and update it as checks are completed.
+
+---
+
+### CI/CD Overview
+
+Review the GitHub CI/CD Overview which summarises the controls DBT has in place to strengthen the security posture of our GitHub repositories.
+
+![CI/CD overview](https://raw.githubusercontent.com/uktrade/.github/refs/heads/main/assets/CI-CD%20pipeline.svg)
+
+---
+
+### Pre‑Commit Hooks
+
+DBT requires all contributors to use the organisation‑approved pre‑commit hooks before committing. A GitHub Action blocks PRs where the hook has not run.
+
+For more information and setup guidance please refer [here](https://github.com/uktrade/github-standards).
+
+---
+
+### Custom GitHub Properties
+
+DBT uses custom github properties to enforce branch protection rules and run organisation level github actions.
+
+Manage custom properties:
+`https://github.com/uktrade/REPO_NAME/settings/access`
+
+**Mandatory**
+- `reusable_workflow_opt_in` - `true`  
+- `scs_portfolio` - The portfolio associated with your CSC. If your portfolio is missing, this can be added by raising an SRE ticket.
+
+**Optional**
+- `is_docker` — for repos that build Docker images  
+- `language` — all languages used by this repository should be selected, and github workflows will run with dedicated checks on that language.  
+
+---
+
+### GitHub Security Policy
+
+DBT has introduced a new organisation-wide GitHub security policy that applies the required security checks to every repository. New repositories get this policy by default, but existing ones must have it enabled before they can be made public. Over time, this policy will fully replace the old one across the uktrade account.
+
+**You must be an organisation administrator to apply this policy**
+
+To add the new security policy, follow these instructions:
+
+1. As an organisation administrator, navigate to the [security config page](https://github.com/organizations/uktrade/settings/security_products).
+1. Scroll down to the **Apply configurations** sections, and enter the name of the repository to be made public in the filter input field
+1. Use the checkbox next to the results list to select all repositories being made public, then use the **Apply configuration** button to select the **Default DBT security** configuration
+1. A confirmation modal will appear displaying a summary of the action being made. Click the apply button
+1. In the repository that has had the new policy applied, navigate to the **Advanced Security** page in the repository settings. At the top of the page there should be a banner message **Modifications to some settings have been blocked by organization administrators.**
+
+---
+
+### CodeQL for Fork‑Based PRs (Optional)
+
+The default DBT GitHub Security policy does not currently support scanning PRs raised from a fork of a repository.
+If PRs from forks must be supported, switch to **Advanced** CodeQL to generate a `codeql.yml` workflow:
+
+1. Open the GitHub settings page, and navigate to the Advanced Security section using the left hand menu
+1. Scroll down to the Code Scanning section, under the Tools sub-section there will be an item for CodeQL analysis
+1. Click the ... button next to Default setup text, then choose the Switch to advanced option from the menu
+1. On the popup, click the Disable CodeQL button. Although you are disabling CodeQL, there is still a branch protection rule in place that blocks a PR unless a CodeQL scan is detected. Disabling here will not allow PRs to be merged
+1. The GitHub online editor will open to create a new file called codeql.yml in your repo, and the contents of this file will be prefilled with the languages CodeQL has detected in your repo. You can modify the contents of this file if needed, however you must leave the workflow name as `CodeQL Advanced`
+1. Once happy with the workflow file contents, click the green Commit changes button to trigger a PR to merge this into the main branch
+1. Approve and merge the PR with this workflow file. Once merged, the CodeQL scan will perform an initial scan that can take a while but you can track the progress by viewing the Actions tab for your repository
+
+### CODEOWNERS
+
+Repositories must include a `CODEOWNERS` file:
+https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
+
+### GitHub Safety Tips
+
+Internal contributors should review the [GitHub Safety Tips](https://uktrade.atlassian.net/wiki/x/n4AEKQE) to understand how to protect themselves when coding in the open.
+
+### Repository Access & Governance
+
+TBC
+
+### Pull Request Template
+
+If your repository does not already contain a pull_request_template.md file, you will inherit the DBT template by default. If you are already using your own template, you should add this section to remind reviewers they should be ensuring no secret values are visible:
+
+```
+### Reviewer Checklist
+
+- [ ] I have reviewed the PR and ensured no secret or sensitive data is present
+```
+
+### Branch Protection Rules
+
+An organisation ruleset has been created to apply a minimum set of branch protection rules:
+
+- A PR is required for merges into the default branch (usually main)
+- At least 1 approver is required before a PR can be merged
+- Any conversations on the PR must be marked as resolved
+
+Organisation admins and repository admins have been added to the bypass list for this branch protection ruleset.
+
+Repository admins might decide to add additional rules to their own repositories. It is not possible for repository admins to add their own rules that reduce this level of protection. As an example, a repository admin could add a ruleset that drops the required number of approvers to 0 but that would have no effect as the organisation ruleset would take precedence. They could add a ruleset that sets the number of approvers to 3, and as this is not reducing the organisation ruleset protection this would take precedence.
+
+## Push Protection
+
+Push protection is required for all repositories using the default DBT GitHub security policy.  
+DBT also defines [custom secret‑scanning patterns](https://docs.github.com/en/code-security/concepts/secret-security/about-push-protection).
+
+You should confirm that push protection is enabled on your repository.
+
+Please raise a ticket with SRE if you need additional patterns.

fastdsu-0.0.0/.github/actions/setup-toolchain/action.yml

@@ -0,0 +1,29 @@
+name: Setup toolchain
+description: Install uv with Python and optionally just
+
+inputs:
+  python-version:
+    description: Python version to install
+    required: false
+    default: "3.11"
+  install-just:
+    description: Whether to install just
+    required: false
+    default: "true"
+
+runs:
+  using: composite
+  steps:
+    - name: Install uv and Python
+      uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+      with:
+        enable-cache: true
+        cache-dependency-glob: "uv.lock"
+        python-version: ${{ inputs.python-version }}
+
+    - name: Install just
+      if: ${{ inputs.install-just == 'true' }}
+      uses: extractions/setup-crate@7577c1bdf2d95e6d65d532788f35ed79d4b1dda2 # v2.0.1
+      with:
+        repo: casey/just
+        version: 1.55.1

fastdsu-0.0.0/.github/pull_request_template.md

@@ -0,0 +1,23 @@
+<!-- Give high level context to this PR -->
+
+## 🛠️ Changes proposed in this pull request
+
+* <!-- List the main changes you've made -->
+
+## 👀 Guidance to review
+
+<!-- Help reviewers know where to focus their efforts, or flag controversial decisions -->
+<!-- Add links and diagrams to explain decisions -->
+<!-- Point out any **breaking changes** -->
+
+## 🤖 AI declaration
+
+<!-- Declare code where AI was used, and how you ensured its quality -->
+
+## ✅ Checklist:
+
+- [ ] This is the smallest, simplest solution to the problem
+- [ ] I've read [our code standards](../CONTRIBUTING.md) and this code follows them  
+- [ ] All new code is tested
+- [ ] I've updated all relevant documentation
+- [ ] I've reviewed the PR and ensured no secret values are present

fastdsu-0.0.0/.github/workflows/cd.yml

@@ -0,0 +1,97 @@
+name: CD
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  build-wheels:
+    name: Build wheels on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    permissions:
+      contents: read
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [
+          ubuntu-latest, 
+          ubuntu-24.04-arm,
+          windows-latest, 
+          macos-15-intel, 
+          macos-14
+        ]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: "3.11"
+          install-just: "false"
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+
+      - name: Build wheels
+        run: uvx cibuildwheel --output-dir wheelhouse
+        env:
+          CIBW_BUILD: cp311-* cp312-* cp313-* cp314-*
+          CIBW_SKIP: pp* *-win32 *i686* *-musllinux*
+          CIBW_BUILD_VERBOSITY: 1
+          CIBW_BUILD_FRONTEND: "build"
+          MACOSX_DEPLOYMENT_TARGET: "11.0"
+
+      - name: Upload wheels
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+        with:
+          name: dist-wheels-${{ matrix.os }}
+          path: ./wheelhouse/*.whl
+
+  build-sdist:
+    name: Build source distribution
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: "3.11"
+          install-just: "false"
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+
+      - name: Build source distribution
+        run: uv build --sdist --out-dir dist
+
+      - name: Upload sdist
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+        with:
+          name: dist-sdist
+          path: dist/*.tar.gz
+
+  deploy-package:
+    needs: [build-wheels, build-sdist]
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+
+    environment: pypi
+
+    steps:
+      - name: Download build artefacts
+        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
+        with:
+          pattern: dist-*
+          path: dist
+          merge-multiple: true
+
+      - name: Publish package to PyPI
+        uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0

fastdsu-0.0.0/.github/workflows/ci.yml

@@ -0,0 +1,102 @@
+name: CI
+
+on:
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test-python:
+    name: test ${{ matrix.python-version }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+
+      - name: Run Python tests
+        run: just test python
+
+  test-os:
+    name: test on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: "3.11"
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+
+      - name: Run Rust tests
+        run: just test rust
+
+      - name: Run Python tests
+        run: just test python
+
+  format-and-lint:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: "3.11"
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+        with:
+          components: |
+            rustfmt
+            clippy
+
+      - name: Format and lint
+        run: just format
+
+  build-package:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        
+      - name: Setup uv and just toolchain
+        uses: ./.github/actions/setup-toolchain
+        with:
+          python-version: "3.11"
+
+      - name: Set up Rust
+        uses: actions-rust-lang/setup-rust-toolchain@1780873c7b576612439a134613cc4cc74ce5538c # v1.15.2
+
+      - name: Build the package
+        run: uv build
+
+  ci-success:
+    needs: [format-and-lint, test-python, test-os, build-package]
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: CI Success
+        run: echo "All CI checks passed!"

fastdsu-0.0.0/.gitignore

@@ -0,0 +1,30 @@
+# Custom
+scratch/
+
+# Python
+__pycache__/
+*.py[oc]
+*.egg-info/
+dist/
+wheels/
+.pytest_cache
+.ruff_cache
+
+# Compiled extensions
+*.so
+*.pyd
+
+# Rust
+target/
+
+# uv / virtualenvs
+.venv
+
+# Editor
+.vscode/
+.idea/
+*.sw[po]
+
+# OS
+.DS_Store
+Thumbs.db

fastdsu-0.0.0/.pre-commit-config.yaml

@@ -0,0 +1,34 @@
+repos:
+- repo: local
+  hooks:
+    # Run the Python and Rust linter and formatter using just command
+    - id: format
+      name: format
+      description: "Format Python and Rust code via just command"
+      entry: just format
+      language: system
+      types_or: [python, pyi]
+      pass_filenames: false
+      stages: [pre-commit]
+
+    # Run the Python and Rust type checker using just command
+    - id: check
+      name: check
+      description: "Type check Python and Rust code via just command"
+      entry: just check
+      language: system
+      types_or: [python, pyi]
+      pass_filenames: false
+      stages: [pre-commit]
+
+# Mandatory internal hooks
+- repo: https://github.com/uktrade/github-standards
+  rev: v1.4.1  # update periodically with pre-commit autoupdate
+  hooks:
+    - id: run-security-scan
+      verbose: false
+    - id: run-personal-data-scan
+      verbose: false
+    - id: validate-security-scan
+      verbose: false
+      

fastdsu-0.0.0/.python-version

@@ -0,0 +1 @@
+3.12.12

fastdsu-0.0.0/CODEOWNERS

@@ -0,0 +1 @@
+* @uktrade/data-matching-service