PII-discovery 0.1.0__tar.gz

pii_discovery-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pratik P. Lahudkar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pii_discovery-0.1.0/MANIFEST.in

@@ -0,0 +1,11 @@
+include LICENSE
+include PUBLISHING.md
+include README.md
+include pii_discovery.py
+exclude PII_Discovery_and_Detection_Methodology.docx
+exclude EDGE_CASE_CATALOG.md
+exclude SAMPLES_README.md
+recursive-exclude samples *
+recursive-exclude sample_report_value_only *
+recursive-exclude __pycache__ *
+global-exclude *.py[cod] *.csv *.xlsx *.xlsm *.pdf *.docx

pii_discovery-0.1.0/PII_discovery.egg-info/PKG-INFO

@@ -0,0 +1,173 @@
+Metadata-Version: 2.4
+Name: PII-discovery
+Version: 0.1.0
+Summary: Local-first discovery of candidate PII, PHI, and financial data in common file formats
+Author: Pratik P. Lahudkar
+License-Expression: MIT
+Keywords: pii,phi,privacy,data-discovery,sensitive-data,structured-data,unstructured-data,forensics
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Information Technology
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: tqdm<5,>=4.66
+Provides-Extra: formats
+Requires-Dist: openpyxl<4,>=3.1; extra == "formats"
+Requires-Dist: pypdf<7,>=5; extra == "formats"
+Requires-Dist: python-docx<2,>=1.1; extra == "formats"
+Provides-Extra: ner
+Requires-Dist: spacy<4,>=3.7; extra == "ner"
+Provides-Extra: all
+Requires-Dist: openpyxl<4,>=3.1; extra == "all"
+Requires-Dist: pypdf<7,>=5; extra == "all"
+Requires-Dist: python-docx<2,>=1.1; extra == "all"
+Requires-Dist: spacy<4,>=3.7; extra == "all"
+Requires-Dist: tqdm<5,>=4.66; extra == "all"
+Provides-Extra: maintainer
+Requires-Dist: build<2,>=1.2; extra == "maintainer"
+Requires-Dist: twine<7,>=5; extra == "maintainer"
+Dynamic: license-file
+
+# PII-discovery
+
+**Local-first discovery of candidate PII, PHI, and financial data across structured, semi-structured, and text-based files.**
+
+PII-discovery is a Python library and command-line scanner for locating potentially sensitive values in supported local files. It combines structured-field and key-name matching with value patterns, explicit labels, selected validation rules, nearby text context, and optional spaCy named-entity recognition.
+
+The scanner reads source files locally and writes reports to a directory you choose. It does not redact values or modify source files. Findings include exact candidate values and unredacted context so an analyst can locate and review each match.
+
+PII-discovery is a discovery aid. It does not certify that a dataset is free of sensitive information, guarantee zero false positives, verify a person's identity, or establish regulatory compliance.
+
+## Capabilities
+
+- Scans structured, semi-structured, and unstructured text in supported local files.
+- Uses embedded header and key aliases, regular expressions, explicit label capture, selected format and checksum validation, and source context.
+- Optionally identifies PERSON and ORG candidates with spaCy NER.
+- Reports exact values, source locations, detection evidence, confidence/severity scores, and prefix/suffix context.
+- Keeps structured header findings separate from value-level findings.
+- Includes a progress bar through tqdm.
+- Embeds a reference taxonomy of 651 data-type entries across 27 categories. The taxonomy supports classification and matching; it does not mean every entry has a dedicated value detector.
+
+## Installation
+
+Install the base package from PyPI:
+
+~~~~powershell
+python -m pip install pii-discovery
+~~~~
+
+The base installation includes the command-line scanner and progress display. Optional readers for Excel, PDF, and Word documents can be installed with:
+
+~~~~powershell
+python -m pip install "pii-discovery[formats]"
+~~~~
+
+To enable spaCy named-entity recognition, install the NER extra and its language model:
+
+~~~~powershell
+python -m pip install "pii-discovery[ner]"
+python -m spacy download en_core_web_sm
+~~~~
+
+Install all optional integrations with `python -m pip install "pii-discovery[all]"`. The spaCy language model is installed separately. If spaCy or the requested model is unavailable, the scanner warns and continues without NER.
+
+## Supported input formats
+
+The current readers support:
+
+- Delimited text, including CSV and TSV
+- JSON and JSON Lines
+- XML and HTML
+- Plain text, logs, and Markdown
+- SQLite databases, read through a read-only connection
+- EML email messages, ICS calendars, and VCF files
+- XLSX and XLSM workbooks
+- PDF files with extractable embedded text
+- DOCX paragraphs and tables
+
+Some formats require the `formats` extra. XLSX scanning reads cached cell values and does not calculate formulas. PDF scanning extracts embedded text and does not OCR scanned pages. Images, audio, video, external databases, hidden document metadata, and password-protected content are outside the current scanner's coverage.
+
+## Quick start
+
+Scan a directory recursively and write reports to a separate output directory:
+
+~~~~powershell
+pii-discovery ./source_data --output ./pii_report
+~~~~
+
+Scan individual files and set a higher minimum candidate score:
+
+~~~~powershell
+pii-discovery ./customers.csv ./case_notes.txt --output ./case_report --min-confidence 0.70
+~~~~
+
+Use the command's help option to see every setting. Key controls include:
+
+| Option | Default | Purpose |
+| --- | ---: | --- |
+| `--output` | `pii_discovery_report` | Report destination |
+| `--no-recursive` | Off | Scan only files immediately inside each input directory |
+| `--min-confidence` | `0.55` | Minimum candidate score to report |
+| `--context-chars` | `70` | Maximum prefix and suffix context length in findings |
+| `--max-file-mb` | `100` | Per-file size limit |
+| `--spacy-model` | `en_core_web_sm` | spaCy model used for optional PERSON/ORG NER |
+| `--no-spacy` | Off | Disable spaCy NER |
+
+Confidence and severity values are evidence scores assigned by the implementation. They are not calibrated probabilities that a match is true.
+
+## Python API
+
+The primary programmatic entry point is `build_report`:
+
+~~~~python
+from pathlib import Path
+
+from pii_discovery import build_report
+
+summary = build_report(
+    inputs=[Path("source_data")],
+    output_dir=Path("pii_report"),
+    recursive=True,
+    min_confidence=0.55,
+    context_chars=70,
+    max_file_mb=100,
+    spacy_model="en_core_web_sm",
+)
+
+print(summary)
+~~~~
+
+Pass `spacy_model=None` to disable NER. The function returns a summary dictionary and writes the reports to `output_dir`.
+
+## Reports
+
+The report directory contains:
+
+- `findings.csv` — value-level candidate findings, including exact unredacted values in `value_preview`, source locations, detection evidence, and unredacted prefix/suffix context.
+- `structured_fields.csv` — sensitive column and key names detected from structured sources, kept separate from value-level findings.
+- `summary_by_category.csv`, `summary_by_file.csv`, and `summary_by_pii_type.csv` — finding counts by category, source file, and mapped catalog type.
+- `file_inventory.csv` — discovered files, scan status, values scanned, finding count, and parser errors.
+- `document_indicators.csv` — document-level terms reported separately from value findings.
+- `pii_type_catalog.csv`, `classification_legend.csv`, and `regulatory_terminology.csv` — embedded reference material used in classification and reporting.
+- `report.md` — scan totals, summaries, interpretation notes, and known limitations.
+
+The scanner does not redact values. Findings and reports may contain personal, health, financial, identity, or investigative information. Store and share them only under appropriate access controls and evidence-handling procedures.
+
+## Detection scope and limitations
+
+Detection combines header/key aliases, embedded taxonomy terms, regular expressions, explicit label capture, selected format or checksum checks, nearby context, and optional spaCy PERSON/ORG NER. Document-level keywords are reported separately and are not treated as proof that a value was found.
+
+Not every catalog item has a dedicated value detector. Names, addresses, health content, dates, and jurisdiction-specific identifiers can depend heavily on labels, language, context, and source quality. NER candidates are statistical signals, not proof of identity or personal-data status. The scanner does not query government, financial, carrier, or other issuing registries to authenticate values.
+
+No automated detector can guarantee complete recall or zero false positives across all languages, jurisdictions, source layouts, and data conditions. Validate results against an approved, representative, adjudicated dataset and have a qualified analyst review findings before relying on them in an investigation. Do not use this package as the sole basis for a legal, employment, financial, or other high-impact decision.
+
+The scanner processes files locally and does not upload source contents to a remote detection service. It is not a forensic acquisition tool and does not create a chain-of-custody record.
+
+## License
+
+MIT. See the included `LICENSE` file.

pii_discovery-0.1.0/PII_discovery.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+MANIFEST.in
+PUBLISHING.md
+README.md
+pii_discovery.py
+pyproject.toml
+PII_discovery.egg-info/PKG-INFO
+PII_discovery.egg-info/SOURCES.txt
+PII_discovery.egg-info/dependency_links.txt
+PII_discovery.egg-info/entry_points.txt
+PII_discovery.egg-info/requires.txt
+PII_discovery.egg-info/top_level.txt

pii_discovery-0.1.0/PII_discovery.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

pii_discovery-0.1.0/PII_discovery.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+pii-discovery = pii_discovery:main

pii_discovery-0.1.0/PII_discovery.egg-info/requires.txt

@@ -0,0 +1,20 @@
+tqdm<5,>=4.66
+
+[all]
+openpyxl<4,>=3.1
+pypdf<7,>=5
+python-docx<2,>=1.1
+spacy<4,>=3.7
+tqdm<5,>=4.66
+
+[formats]
+openpyxl<4,>=3.1
+pypdf<7,>=5
+python-docx<2,>=1.1
+
+[maintainer]
+build<2,>=1.2
+twine<7,>=5
+
+[ner]
+spacy<4,>=3.7

pii_discovery-0.1.0/PII_discovery.egg-info/top_level.txt

@@ -0,0 +1 @@
+pii_discovery

pii_discovery-0.1.0/PKG-INFO

@@ -0,0 +1,173 @@
+Metadata-Version: 2.4
+Name: PII-discovery
+Version: 0.1.0
+Summary: Local-first discovery of candidate PII, PHI, and financial data in common file formats
+Author: Pratik P. Lahudkar
+License-Expression: MIT
+Keywords: pii,phi,privacy,data-discovery,sensitive-data,structured-data,unstructured-data,forensics
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Information Technology
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: tqdm<5,>=4.66
+Provides-Extra: formats
+Requires-Dist: openpyxl<4,>=3.1; extra == "formats"
+Requires-Dist: pypdf<7,>=5; extra == "formats"
+Requires-Dist: python-docx<2,>=1.1; extra == "formats"
+Provides-Extra: ner
+Requires-Dist: spacy<4,>=3.7; extra == "ner"
+Provides-Extra: all
+Requires-Dist: openpyxl<4,>=3.1; extra == "all"
+Requires-Dist: pypdf<7,>=5; extra == "all"
+Requires-Dist: python-docx<2,>=1.1; extra == "all"
+Requires-Dist: spacy<4,>=3.7; extra == "all"
+Requires-Dist: tqdm<5,>=4.66; extra == "all"
+Provides-Extra: maintainer
+Requires-Dist: build<2,>=1.2; extra == "maintainer"
+Requires-Dist: twine<7,>=5; extra == "maintainer"
+Dynamic: license-file
+
+# PII-discovery
+
+**Local-first discovery of candidate PII, PHI, and financial data across structured, semi-structured, and text-based files.**
+
+PII-discovery is a Python library and command-line scanner for locating potentially sensitive values in supported local files. It combines structured-field and key-name matching with value patterns, explicit labels, selected validation rules, nearby text context, and optional spaCy named-entity recognition.
+
+The scanner reads source files locally and writes reports to a directory you choose. It does not redact values or modify source files. Findings include exact candidate values and unredacted context so an analyst can locate and review each match.
+
+PII-discovery is a discovery aid. It does not certify that a dataset is free of sensitive information, guarantee zero false positives, verify a person's identity, or establish regulatory compliance.
+
+## Capabilities
+
+- Scans structured, semi-structured, and unstructured text in supported local files.
+- Uses embedded header and key aliases, regular expressions, explicit label capture, selected format and checksum validation, and source context.
+- Optionally identifies PERSON and ORG candidates with spaCy NER.
+- Reports exact values, source locations, detection evidence, confidence/severity scores, and prefix/suffix context.
+- Keeps structured header findings separate from value-level findings.
+- Includes a progress bar through tqdm.
+- Embeds a reference taxonomy of 651 data-type entries across 27 categories. The taxonomy supports classification and matching; it does not mean every entry has a dedicated value detector.
+
+## Installation
+
+Install the base package from PyPI:
+
+~~~~powershell
+python -m pip install pii-discovery
+~~~~
+
+The base installation includes the command-line scanner and progress display. Optional readers for Excel, PDF, and Word documents can be installed with:
+
+~~~~powershell
+python -m pip install "pii-discovery[formats]"
+~~~~
+
+To enable spaCy named-entity recognition, install the NER extra and its language model:
+
+~~~~powershell
+python -m pip install "pii-discovery[ner]"
+python -m spacy download en_core_web_sm
+~~~~
+
+Install all optional integrations with `python -m pip install "pii-discovery[all]"`. The spaCy language model is installed separately. If spaCy or the requested model is unavailable, the scanner warns and continues without NER.
+
+## Supported input formats
+
+The current readers support:
+
+- Delimited text, including CSV and TSV
+- JSON and JSON Lines
+- XML and HTML
+- Plain text, logs, and Markdown
+- SQLite databases, read through a read-only connection
+- EML email messages, ICS calendars, and VCF files
+- XLSX and XLSM workbooks
+- PDF files with extractable embedded text
+- DOCX paragraphs and tables
+
+Some formats require the `formats` extra. XLSX scanning reads cached cell values and does not calculate formulas. PDF scanning extracts embedded text and does not OCR scanned pages. Images, audio, video, external databases, hidden document metadata, and password-protected content are outside the current scanner's coverage.
+
+## Quick start
+
+Scan a directory recursively and write reports to a separate output directory:
+
+~~~~powershell
+pii-discovery ./source_data --output ./pii_report
+~~~~
+
+Scan individual files and set a higher minimum candidate score:
+
+~~~~powershell
+pii-discovery ./customers.csv ./case_notes.txt --output ./case_report --min-confidence 0.70
+~~~~
+
+Use the command's help option to see every setting. Key controls include:
+
+| Option | Default | Purpose |
+| --- | ---: | --- |
+| `--output` | `pii_discovery_report` | Report destination |
+| `--no-recursive` | Off | Scan only files immediately inside each input directory |
+| `--min-confidence` | `0.55` | Minimum candidate score to report |
+| `--context-chars` | `70` | Maximum prefix and suffix context length in findings |
+| `--max-file-mb` | `100` | Per-file size limit |
+| `--spacy-model` | `en_core_web_sm` | spaCy model used for optional PERSON/ORG NER |
+| `--no-spacy` | Off | Disable spaCy NER |
+
+Confidence and severity values are evidence scores assigned by the implementation. They are not calibrated probabilities that a match is true.
+
+## Python API
+
+The primary programmatic entry point is `build_report`:
+
+~~~~python
+from pathlib import Path
+
+from pii_discovery import build_report
+
+summary = build_report(
+    inputs=[Path("source_data")],
+    output_dir=Path("pii_report"),
+    recursive=True,
+    min_confidence=0.55,
+    context_chars=70,
+    max_file_mb=100,
+    spacy_model="en_core_web_sm",
+)
+
+print(summary)
+~~~~
+
+Pass `spacy_model=None` to disable NER. The function returns a summary dictionary and writes the reports to `output_dir`.
+
+## Reports
+
+The report directory contains:
+
+- `findings.csv` — value-level candidate findings, including exact unredacted values in `value_preview`, source locations, detection evidence, and unredacted prefix/suffix context.
+- `structured_fields.csv` — sensitive column and key names detected from structured sources, kept separate from value-level findings.
+- `summary_by_category.csv`, `summary_by_file.csv`, and `summary_by_pii_type.csv` — finding counts by category, source file, and mapped catalog type.
+- `file_inventory.csv` — discovered files, scan status, values scanned, finding count, and parser errors.
+- `document_indicators.csv` — document-level terms reported separately from value findings.
+- `pii_type_catalog.csv`, `classification_legend.csv`, and `regulatory_terminology.csv` — embedded reference material used in classification and reporting.
+- `report.md` — scan totals, summaries, interpretation notes, and known limitations.
+
+The scanner does not redact values. Findings and reports may contain personal, health, financial, identity, or investigative information. Store and share them only under appropriate access controls and evidence-handling procedures.
+
+## Detection scope and limitations
+
+Detection combines header/key aliases, embedded taxonomy terms, regular expressions, explicit label capture, selected format or checksum checks, nearby context, and optional spaCy PERSON/ORG NER. Document-level keywords are reported separately and are not treated as proof that a value was found.
+
+Not every catalog item has a dedicated value detector. Names, addresses, health content, dates, and jurisdiction-specific identifiers can depend heavily on labels, language, context, and source quality. NER candidates are statistical signals, not proof of identity or personal-data status. The scanner does not query government, financial, carrier, or other issuing registries to authenticate values.
+
+No automated detector can guarantee complete recall or zero false positives across all languages, jurisdictions, source layouts, and data conditions. Validate results against an approved, representative, adjudicated dataset and have a qualified analyst review findings before relying on them in an investigation. Do not use this package as the sole basis for a legal, employment, financial, or other high-impact decision.
+
+The scanner processes files locally and does not upload source contents to a remote detection service. It is not a forensic acquisition tool and does not create a chain-of-custody record.
+
+## License
+
+MIT. See the included `LICENSE` file.

pii_discovery-0.1.0/PUBLISHING.md

@@ -0,0 +1,71 @@
+# Publish PII-discovery on PyPI
+
+This guide publishes release 0.1.0 from this project directory. The distribution name is PII-discovery; PyPI normalizes it to pii-discovery. A read-only lookup returned Not Found for this name on 12 November 2026. A lookup does not reserve a name, so PyPI confirms availability during the first upload.
+
+## 1. Prepare your PyPI account
+
+1. Create an account at https://pypi.org/account/register/ or sign in to your existing account.
+2. Verify your email address and configure two-factor authentication. PyPI requires two-factor authentication for accounts.
+3. Review the author name in pyproject.toml, the MIT notice in LICENSE, the version, and the rendered README.
+
+## 2. Build the release files
+
+Open PowerShell in the project directory and install the release tools from the local project:
+
+~~~~powershell
+python -m pip install --upgrade pip
+python -m pip install ".[maintainer]"
+python -m build
+~~~~
+
+The dist directory should contain:
+
+~~~~text
+dist/pii_discovery-0.1.0-py3-none-any.whl
+dist/pii_discovery-0.1.0.tar.gz
+~~~~
+
+Check the archives' metadata and rendered project description:
+
+~~~~powershell
+python -m twine check ./dist/pii_discovery-0.1.0-py3-none-any.whl ./dist/pii_discovery-0.1.0.tar.gz
+~~~~
+
+Inspect both archives before upload. Confirm they contain only the intended scanner source, package metadata, README, and license. The packaging configuration excludes sample inputs and generated reports. Never upload evidence, findings, API tokens, or local configuration files.
+
+## 3. Create upload credentials
+
+For a manual first upload:
+
+1. In PyPI account settings, open API tokens and create a token.
+2. Because the project does not exist on PyPI until its first successful upload, select an account-scoped token for this first release.
+3. Keep the token private and outside the project files. At the Twine prompt, enter __token__ as the username and the complete API token, including its pypi- prefix, as the password.
+
+PyPI Trusted Publishing is recommended for automated publishing from a supported CI provider. It avoids storing a long-lived upload token, but requires a hosted source repository and a configured, trusted release workflow. This local project does not currently have that setup.
+
+## 4. Upload the first release
+
+Run this from the project directory:
+
+~~~~powershell
+python -m twine upload ./dist/pii_discovery-0.1.0-py3-none-any.whl ./dist/pii_discovery-0.1.0.tar.gz
+~~~~
+
+Enter the API token only in the credential prompt. A successful upload creates the project page and publishes version 0.1.0. A released version cannot be reused for changed files; increment the version for each subsequent release.
+
+## 5. Verify the published package
+
+1. Open https://pypi.org/project/pii-discovery/ and confirm the title, summary, author, license, and rendered README.
+2. In a clean Python environment, install the uploaded package:
+
+   ~~~~powershell
+   python -m pip install --upgrade pii-discovery
+   ~~~~
+
+3. Confirm the command-line entry point is installed:
+
+   ~~~~powershell
+   pii-discovery --help
+   ~~~~
+
+For later releases, update the version in pyproject.toml, build fresh archives, run Twine's metadata check, and upload the new files. After the first release, create a project-scoped token for routine manual uploads and revoke the initial account-scoped token if it is no longer needed.

pii_discovery-0.1.0/README.md

@@ -0,0 +1,138 @@
+# PII-discovery
+
+**Local-first discovery of candidate PII, PHI, and financial data across structured, semi-structured, and text-based files.**
+
+PII-discovery is a Python library and command-line scanner for locating potentially sensitive values in supported local files. It combines structured-field and key-name matching with value patterns, explicit labels, selected validation rules, nearby text context, and optional spaCy named-entity recognition.
+
+The scanner reads source files locally and writes reports to a directory you choose. It does not redact values or modify source files. Findings include exact candidate values and unredacted context so an analyst can locate and review each match.
+
+PII-discovery is a discovery aid. It does not certify that a dataset is free of sensitive information, guarantee zero false positives, verify a person's identity, or establish regulatory compliance.
+
+## Capabilities
+
+- Scans structured, semi-structured, and unstructured text in supported local files.
+- Uses embedded header and key aliases, regular expressions, explicit label capture, selected format and checksum validation, and source context.
+- Optionally identifies PERSON and ORG candidates with spaCy NER.
+- Reports exact values, source locations, detection evidence, confidence/severity scores, and prefix/suffix context.
+- Keeps structured header findings separate from value-level findings.
+- Includes a progress bar through tqdm.
+- Embeds a reference taxonomy of 651 data-type entries across 27 categories. The taxonomy supports classification and matching; it does not mean every entry has a dedicated value detector.
+
+## Installation
+
+Install the base package from PyPI:
+
+~~~~powershell
+python -m pip install pii-discovery
+~~~~
+
+The base installation includes the command-line scanner and progress display. Optional readers for Excel, PDF, and Word documents can be installed with:
+
+~~~~powershell
+python -m pip install "pii-discovery[formats]"
+~~~~
+
+To enable spaCy named-entity recognition, install the NER extra and its language model:
+
+~~~~powershell
+python -m pip install "pii-discovery[ner]"
+python -m spacy download en_core_web_sm
+~~~~
+
+Install all optional integrations with `python -m pip install "pii-discovery[all]"`. The spaCy language model is installed separately. If spaCy or the requested model is unavailable, the scanner warns and continues without NER.
+
+## Supported input formats
+
+The current readers support:
+
+- Delimited text, including CSV and TSV
+- JSON and JSON Lines
+- XML and HTML
+- Plain text, logs, and Markdown
+- SQLite databases, read through a read-only connection
+- EML email messages, ICS calendars, and VCF files
+- XLSX and XLSM workbooks
+- PDF files with extractable embedded text
+- DOCX paragraphs and tables
+
+Some formats require the `formats` extra. XLSX scanning reads cached cell values and does not calculate formulas. PDF scanning extracts embedded text and does not OCR scanned pages. Images, audio, video, external databases, hidden document metadata, and password-protected content are outside the current scanner's coverage.
+
+## Quick start
+
+Scan a directory recursively and write reports to a separate output directory:
+
+~~~~powershell
+pii-discovery ./source_data --output ./pii_report
+~~~~
+
+Scan individual files and set a higher minimum candidate score:
+
+~~~~powershell
+pii-discovery ./customers.csv ./case_notes.txt --output ./case_report --min-confidence 0.70
+~~~~
+
+Use the command's help option to see every setting. Key controls include:
+
+| Option | Default | Purpose |
+| --- | ---: | --- |
+| `--output` | `pii_discovery_report` | Report destination |
+| `--no-recursive` | Off | Scan only files immediately inside each input directory |
+| `--min-confidence` | `0.55` | Minimum candidate score to report |
+| `--context-chars` | `70` | Maximum prefix and suffix context length in findings |
+| `--max-file-mb` | `100` | Per-file size limit |
+| `--spacy-model` | `en_core_web_sm` | spaCy model used for optional PERSON/ORG NER |
+| `--no-spacy` | Off | Disable spaCy NER |
+
+Confidence and severity values are evidence scores assigned by the implementation. They are not calibrated probabilities that a match is true.
+
+## Python API
+
+The primary programmatic entry point is `build_report`:
+
+~~~~python
+from pathlib import Path
+
+from pii_discovery import build_report
+
+summary = build_report(
+    inputs=[Path("source_data")],
+    output_dir=Path("pii_report"),
+    recursive=True,
+    min_confidence=0.55,
+    context_chars=70,
+    max_file_mb=100,
+    spacy_model="en_core_web_sm",
+)
+
+print(summary)
+~~~~
+
+Pass `spacy_model=None` to disable NER. The function returns a summary dictionary and writes the reports to `output_dir`.
+
+## Reports
+
+The report directory contains:
+
+- `findings.csv` — value-level candidate findings, including exact unredacted values in `value_preview`, source locations, detection evidence, and unredacted prefix/suffix context.
+- `structured_fields.csv` — sensitive column and key names detected from structured sources, kept separate from value-level findings.
+- `summary_by_category.csv`, `summary_by_file.csv`, and `summary_by_pii_type.csv` — finding counts by category, source file, and mapped catalog type.
+- `file_inventory.csv` — discovered files, scan status, values scanned, finding count, and parser errors.
+- `document_indicators.csv` — document-level terms reported separately from value findings.
+- `pii_type_catalog.csv`, `classification_legend.csv`, and `regulatory_terminology.csv` — embedded reference material used in classification and reporting.
+- `report.md` — scan totals, summaries, interpretation notes, and known limitations.
+
+The scanner does not redact values. Findings and reports may contain personal, health, financial, identity, or investigative information. Store and share them only under appropriate access controls and evidence-handling procedures.
+
+## Detection scope and limitations
+
+Detection combines header/key aliases, embedded taxonomy terms, regular expressions, explicit label capture, selected format or checksum checks, nearby context, and optional spaCy PERSON/ORG NER. Document-level keywords are reported separately and are not treated as proof that a value was found.
+
+Not every catalog item has a dedicated value detector. Names, addresses, health content, dates, and jurisdiction-specific identifiers can depend heavily on labels, language, context, and source quality. NER candidates are statistical signals, not proof of identity or personal-data status. The scanner does not query government, financial, carrier, or other issuing registries to authenticate values.
+
+No automated detector can guarantee complete recall or zero false positives across all languages, jurisdictions, source layouts, and data conditions. Validate results against an approved, representative, adjudicated dataset and have a qualified analyst review findings before relying on them in an investigation. Do not use this package as the sole basis for a legal, employment, financial, or other high-impact decision.
+
+The scanner processes files locally and does not upload source contents to a remote detection service. It is not a forensic acquisition tool and does not create a chain-of-custody record.
+
+## License
+
+MIT. See the included `LICENSE` file.