presidio-analyzer 2.2.363__tar.gz

presidio_analyzer-2.2.363/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.4
+Name: presidio_analyzer
+Version: 2.2.363
+Summary: Presidio Analyzer package
+License-Expression: MIT
+Keywords: presidio_analyzer
+Author: Presidio
+Author-email: presidio@microsoft.com
+Requires-Python: >=3.10,<3.15
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Provides-Extra: ahds
+Provides-Extra: azure-ai-language
+Provides-Extra: gliner
+Provides-Extra: langextract
+Provides-Extra: server
+Provides-Extra: stanza
+Provides-Extra: transformers
+Requires-Dist: accelerate (>=0.20.0,<2.0.0) ; extra == "transformers"
+Requires-Dist: azure-ai-textanalytics (>=5.4.0,<6.0.0) ; extra == "azure-ai-language"
+Requires-Dist: azure-core (>=1.39.0,<2.0.0) ; extra == "azure-ai-language"
+Requires-Dist: azure-health-deidentification (>=1.1.0b1,<2.0.0) ; extra == "ahds"
+Requires-Dist: azure-identity (>=1.18.0,<2.0.0) ; extra == "langextract"
+Requires-Dist: azure-identity (>=1.25.3,<2.0.0) ; extra == "ahds"
+Requires-Dist: click (>=8.1.0,<9.0.0)
+Requires-Dist: flask (>=1.1,<4.0.0) ; extra == "server"
+Requires-Dist: gliner (>=0.2.26,<1.0.0) ; extra == "gliner"
+Requires-Dist: gunicorn (>=20.0.0,<26.0.0) ; (platform_system != "Windows") and (extra == "server")
+Requires-Dist: huggingface_hub (>=0.20.0,<2.0.0) ; extra == "transformers"
+Requires-Dist: huggingface_hub ; extra == "gliner"
+Requires-Dist: jinja2 (>=3.0.0,<4.0.0) ; extra == "langextract"
+Requires-Dist: langextract (>=1.0.0,<2.0.0) ; extra == "langextract"
+Requires-Dist: more-itertools (>=10.0.0,<12.0.0) ; extra == "langextract"
+Requires-Dist: numpy (>=1.19.0,<2.5.0)
+Requires-Dist: onnxruntime (>=1.19) ; (python_version > "3.10") and (extra == "gliner")
+Requires-Dist: onnxruntime (>=1.19,<1.24.1) ; (python_version == "3.10") and (extra == "gliner")
+Requires-Dist: openai (>=1.50.0,<3.0.0) ; extra == "langextract"
+Requires-Dist: phonenumbers (>=9.0.28,<10.0.0)
+Requires-Dist: pydantic (>=2.12.5,<3.0.0)
+Requires-Dist: pyyaml (>=6.0.3,<7.0.0)
+Requires-Dist: regex (>=2023.0.0)
+Requires-Dist: spacy (>=3.4.4,!=3.7.0,!=3.8.14,<4.0.0)
+Requires-Dist: spacy_huggingface_pipelines (>=0.0.4,<1.0.0) ; extra == "transformers"
+Requires-Dist: stanza (>=1.11.1,<2.0.0) ; extra == "stanza"
+Requires-Dist: tldextract (>=5.3.1,<6.0.0)
+Requires-Dist: transformers (>=4.0.0,<6.0.0) ; extra == "transformers"
+Requires-Dist: transformers ; extra == "gliner"
+Requires-Dist: waitress (>=2.0.0,<4.0.0) ; (platform_system == "Windows") and (extra == "server")
+Project-URL: Homepage, https://github.com/data-privacy-stack/presidio
+Description-Content-Type: text/markdown
+
+# Presidio analyzer
+
+## Description
+
+The Presidio analyzer is a Python based service for detecting PII entities in text.
+
+During analysis, it runs a set of different _PII Recognizers_,
+each one in charge of detecting one or more PII entities using different mechanisms.
+
+Presidio analyzer comes with a set of predefined recognizers,
+but can easily be extended with other types of custom recognizers.
+Predefined and custom recognizers leverage regex,
+Named Entity Recognition and other types of logic to detect PII in unstructured text.
+
+### Language Model-based PII/PHI Detection
+
+Presidio analyzer supports language model-based PII/PHI detection (LLMs, SLMs) for flexible entity recognition. The current implementation uses [LangExtract](https://github.com/google/langextract) with support for multiple providers:
+
+- **Ollama** - Local model deployment for privacy-sensitive environments
+- **Azure OpenAI** - Cloud-based deployment with enterprise features
+
+```bash
+pip install presidio-analyzer[langextract]
+```
+
+#### Quick Usage
+
+**Ollama** (local models):
+
+```python
+from presidio_analyzer.predefined_recognizers import BasicLangExtractRecognizer
+recognizer = BasicLangExtractRecognizer()  # Uses default config
+```
+
+**Azure OpenAI** (cloud models):
+
+```python
+from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer
+
+# Simple usage - pass everything as parameters
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4",  # Your Azure deployment name
+    azure_endpoint="https://your-resource.openai.azure.com/",
+    api_key="your-api-key"
+)
+
+# Or use environment variables (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY):
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4"  # Your Azure deployment name
+)
+
+# Advanced: Customize entities/prompts with config file
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4",
+    config_path="./custom_config.yaml",  # Optional: for custom entities/prompts
+    azure_endpoint="https://your-resource.openai.azure.com/",
+    api_key="your-api-key"
+)
+```
+
+**Note:** LangExtract recognizers do not validate connectivity during initialization. Connection errors or missing models will be reported when `analyze()` is first called.
+
+See the [Language Model-based PII/PHI Detection guide](https://presidio.dataprivacystack.org/samples/python/langextract/) for complete setup and usage instructions.
+
+## Deploy Presidio analyzer to Azure
+
+Use the following button to deploy presidio analyzer to your Azure subscription.
+
+[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fmicrosoft%2Fpresidio%2Fmain%2Fpresidio-analyzer%2Fdeploytoazure.json)
+
+## Simple usage example
+
+```python
+from presidio_analyzer import AnalyzerEngine
+
+# Set up the engine, loads the NLP module (spaCy model by default) and other PII recognizers
+analyzer = AnalyzerEngine()
+
+# Call analyzer to get results
+results = analyzer.analyze(text="My phone number is 212-555-5555",
+                           entities=["PHONE_NUMBER"],
+                           language='en')
+print(results)
+
+```
+
+## GPU Acceleration
+
+For GPU acceleration, install the appropriate dependencies for your hardware:
+
+- **Linux with NVIDIA GPU**: cupy-cuda12x (or the version matching your CUDA installation)
+- **macOS with Apple Silicon**: MPS (Metal Performance Shaders) is currently not supported. The analyzer will use CPU for PyTorch operations.
+
+## Documentation
+
+Additional documentation on installation, usage and extending the Analyzer can be found under the [Analyzer](https://presidio.dataprivacystack.org/analyzer/) section of [Presidio Documentation](https://presidio.dataprivacystack.org)
+

presidio_analyzer-2.2.363/README.md

@@ -0,0 +1,96 @@
+# Presidio analyzer
+
+## Description
+
+The Presidio analyzer is a Python based service for detecting PII entities in text.
+
+During analysis, it runs a set of different _PII Recognizers_,
+each one in charge of detecting one or more PII entities using different mechanisms.
+
+Presidio analyzer comes with a set of predefined recognizers,
+but can easily be extended with other types of custom recognizers.
+Predefined and custom recognizers leverage regex,
+Named Entity Recognition and other types of logic to detect PII in unstructured text.
+
+### Language Model-based PII/PHI Detection
+
+Presidio analyzer supports language model-based PII/PHI detection (LLMs, SLMs) for flexible entity recognition. The current implementation uses [LangExtract](https://github.com/google/langextract) with support for multiple providers:
+
+- **Ollama** - Local model deployment for privacy-sensitive environments
+- **Azure OpenAI** - Cloud-based deployment with enterprise features
+
+```bash
+pip install presidio-analyzer[langextract]
+```
+
+#### Quick Usage
+
+**Ollama** (local models):
+
+```python
+from presidio_analyzer.predefined_recognizers import BasicLangExtractRecognizer
+recognizer = BasicLangExtractRecognizer()  # Uses default config
+```
+
+**Azure OpenAI** (cloud models):
+
+```python
+from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer
+
+# Simple usage - pass everything as parameters
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4",  # Your Azure deployment name
+    azure_endpoint="https://your-resource.openai.azure.com/",
+    api_key="your-api-key"
+)
+
+# Or use environment variables (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY):
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4"  # Your Azure deployment name
+)
+
+# Advanced: Customize entities/prompts with config file
+recognizer = AzureOpenAILangExtractRecognizer(
+    model_id="gpt-4",
+    config_path="./custom_config.yaml",  # Optional: for custom entities/prompts
+    azure_endpoint="https://your-resource.openai.azure.com/",
+    api_key="your-api-key"
+)
+```
+
+**Note:** LangExtract recognizers do not validate connectivity during initialization. Connection errors or missing models will be reported when `analyze()` is first called.
+
+See the [Language Model-based PII/PHI Detection guide](https://presidio.dataprivacystack.org/samples/python/langextract/) for complete setup and usage instructions.
+
+## Deploy Presidio analyzer to Azure
+
+Use the following button to deploy presidio analyzer to your Azure subscription.
+
+[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fmicrosoft%2Fpresidio%2Fmain%2Fpresidio-analyzer%2Fdeploytoazure.json)
+
+## Simple usage example
+
+```python
+from presidio_analyzer import AnalyzerEngine
+
+# Set up the engine, loads the NLP module (spaCy model by default) and other PII recognizers
+analyzer = AnalyzerEngine()
+
+# Call analyzer to get results
+results = analyzer.analyze(text="My phone number is 212-555-5555",
+                           entities=["PHONE_NUMBER"],
+                           language='en')
+print(results)
+
+```
+
+## GPU Acceleration
+
+For GPU acceleration, install the appropriate dependencies for your hardware:
+
+- **Linux with NVIDIA GPU**: cupy-cuda12x (or the version matching your CUDA installation)
+- **macOS with Apple Silicon**: MPS (Metal Performance Shaders) is currently not supported. The analyzer will use CPU for PyTorch operations.
+
+## Documentation
+
+Additional documentation on installation, usage and extending the Analyzer can be found under the [Analyzer](https://presidio.dataprivacystack.org/analyzer/) section of [Presidio Documentation](https://presidio.dataprivacystack.org)

presidio_analyzer-2.2.363/presidio_analyzer/__init__.py

@@ -0,0 +1,54 @@
+# isort: skip_file
+"""Presidio analyzer package."""
+
+import logging
+
+from presidio_analyzer.analysis_explanation import AnalysisExplanation
+from presidio_analyzer.recognizer_result import RecognizerResult
+from presidio_analyzer.dict_analyzer_result import DictAnalyzerResult
+from presidio_analyzer.entity_recognizer import EntityRecognizer
+from presidio_analyzer.local_recognizer import LocalRecognizer
+from presidio_analyzer.pattern import Pattern
+from presidio_analyzer.pattern_recognizer import PatternRecognizer
+from presidio_analyzer.remote_recognizer import RemoteRecognizer
+from presidio_analyzer.lm_recognizer import LMRecognizer
+from presidio_analyzer.recognizer_registry import RecognizerRegistry
+from presidio_analyzer.analyzer_engine import AnalyzerEngine
+from presidio_analyzer.batch_analyzer_engine import BatchAnalyzerEngine
+from presidio_analyzer.analyzer_request import AnalyzerRequest
+from presidio_analyzer.context_aware_enhancers import ContextAwareEnhancer
+from presidio_analyzer.context_aware_enhancers import LemmaContextAwareEnhancer
+from presidio_analyzer.analyzer_engine_provider import AnalyzerEngineProvider
+
+# Define default loggers behavior
+
+# 1. presidio_analyzer logger
+logging.getLogger("presidio-analyzer").addHandler(logging.NullHandler())
+
+# 2. decision_process logger.
+# Setting the decision_process trace here as we would want it
+# to be activated using a parameter to AnalyzeEngine and not by default.
+decision_process_logger = logging.getLogger("decision_process")
+ch = logging.StreamHandler()
+formatter = logging.Formatter("[%(asctime)s][%(name)s][%(levelname)s]%(message)s")
+ch.setFormatter(formatter)
+decision_process_logger.addHandler(ch)
+decision_process_logger.setLevel("INFO")
+__all__ = [
+    "Pattern",
+    "AnalysisExplanation",
+    "RecognizerResult",
+    "DictAnalyzerResult",
+    "EntityRecognizer",
+    "LocalRecognizer",
+    "PatternRecognizer",
+    "RemoteRecognizer",
+    "LMRecognizer",
+    "RecognizerRegistry",
+    "AnalyzerEngine",
+    "AnalyzerRequest",
+    "ContextAwareEnhancer",
+    "LemmaContextAwareEnhancer",
+    "BatchAnalyzerEngine",
+    "AnalyzerEngineProvider",
+]

presidio_analyzer-2.2.363/presidio_analyzer/analysis_explanation.py

@@ -0,0 +1,65 @@
+from typing import Dict
+
+
+class AnalysisExplanation:
+    """
+    Hold tracing information to explain why PII entities were identified as such.
+
+    :param recognizer: name of recognizer that made the decision
+    :param original_score: recognizer's confidence in result
+    :param pattern_name: name of pattern
+            (if decision was made by a PatternRecognizer)
+    :param pattern: regex pattern that was applied (if PatternRecognizer)
+    :param validation_result: result of a validation (e.g. checksum)
+    :param textual_explanation: Free text for describing
+            a decision of a logic or model
+    """
+
+    def __init__(
+        self,
+        recognizer: str,
+        original_score: float,
+        pattern_name: str = None,
+        pattern: str = None,
+        validation_result: bool = None,
+        textual_explanation: str = None,
+        regex_flags: int = None,
+    ):
+        self.recognizer = recognizer
+        self.pattern_name = pattern_name
+        self.pattern = pattern
+        self.original_score = original_score
+        self.score = original_score
+        self.textual_explanation = textual_explanation
+        self.score_context_improvement = 0
+        self.supportive_context_word = ""
+        self.validation_result = validation_result
+        self.regex_flags = regex_flags
+
+    def __repr__(self):
+        """Create string representation of the object."""
+        return str(self.__dict__)
+
+    def set_improved_score(self, score: float) -> None:
+        """Update the score and calculate the difference from the original score."""
+        self.score = score
+        self.score_context_improvement = self.score - self.original_score
+
+    def set_supportive_context_word(self, word: str) -> None:
+        """Set the context word which helped increase the score."""
+        self.supportive_context_word = word
+
+    def append_textual_explanation_line(self, text: str) -> None:
+        """Append a new line to textual_explanation field."""
+        if self.textual_explanation is None:
+            self.textual_explanation = text
+        else:
+            self.textual_explanation = f"{self.textual_explanation}\n{text}"
+
+    def to_dict(self) -> Dict:
+        """
+        Serialize self to dictionary.
+
+        :return: a dictionary
+        """
+        return self.__dict__