pdf-anonymizer-cli 0.3.1__tar.gz → 0.3.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,144 @@
1
+ Metadata-Version: 2.4
2
+ Name: pdf-anonymizer-cli
3
+ Version: 0.3.3
4
+ Summary: CLI for a tool to anonymize PDF, Markdown, and plain text files using LLMs.
5
+ Author-email: Leonid Ganeline <leo.gan.57@gmail.com>
6
+ License: MIT
7
+ Project-URL: repository, https://github.com/leo-gan/anonymizer
8
+ Requires-Python: >=3.10
9
+ Description-Content-Type: text/markdown
10
+ Requires-Dist: typer
11
+ Requires-Dist: rich
12
+ Requires-Dist: pdf-anonymizer-core
13
+
14
+ # 🦉🫥 PDF Anonymizer CLI
15
+
16
+ A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
17
+
18
+ - **High-Quality Anonymization**: Leverages LLMs to identify and replace Personally Identifiable Information (PII) with high accuracy.
19
+ - **Large File Support**: Consistently anonymizes large files (tested up to 1GB).
20
+ - **Multi-Provider & Cost-Effective**: Free to use with local [Ollama](https://ollama.com/) models. It also supports major providers like [OpenAI](https://openai.com/), [Anthropic](https://www.anthropic.com/), [Google](https://ai.google.com/), [Hugging Face](https://huggingface.co/), and [OpenRouter](https://openrouter.ai/).
21
+ - **Reversible**: Supports deanonymization to recover original data when needed.
22
+ - **Multi-Format**: Works with PDF, Markdown, and plain text files.
23
+
24
+
25
+ ## Installation
26
+
27
+ Install the CLI with your favorite package manager. To use a specific LLM provider, you must install the corresponding extra.
28
+
29
+ - **Google**: `pip install "pdf-anonymizer-cli[google]"`
30
+ - **Ollama**: `pip install "pdf-anonymizer-cli[ollama]"`
31
+ - **Hugging Face**: `pip install "pdf-anonymizer-cli[huggingface]"`
32
+ - **OpenRouter**: `pip install "pdf-anonymizer-cli[openrouter]"`
33
+ - **OpenAI**: `pip install "pdf-anonymizer-cli[openai]"`
34
+ - **Anthropic**: `pip install "pdf-anonymizer-cli[anthropic]"`
35
+
36
+ You can also install multiple extras at once:
37
+
38
+ ```bash
39
+ pip install "pdf-anonymizer-cli[google,openrouter]"
40
+ ```
41
+
42
+ This installs the `pdf-anonymizer` executable.
43
+
44
+
45
+ ## Environment Variables
46
+
47
+ The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
48
+
49
+ - `GOOGLE_API_KEY`: Required when using Google models.
50
+ - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
51
+ - `OPENROUTER_API_KEY`: Required when using OpenRouter models.
52
+ - `OPENAI_API_KEY`: Required when using OpenAI models.
53
+ - `ANTHROPIC_API_KEY`: Required when using Anthropic models.
54
+ - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using Ollama models.
55
+
56
+ Example `.env` file:
57
+ ```env
58
+ GOOGLE_API_KEY="YOUR_API_KEY_HERE"
59
+ HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
60
+ OPENROUTER_API_KEY="YOUR_OPENROUTER_KEY"
61
+ ```
62
+
63
+ ## Usage
64
+
65
+ ### Anonymize
66
+
67
+ The `run` command anonymizes one or more files.
68
+
69
+ ```bash
70
+ pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
71
+ [--characters-to-anonymize INTEGER] \
72
+ [--prompt-name {simple|detailed}] \
73
+ [--model-name TEXT] \
74
+ [--anonymized-entities PATH]
75
+ ```
76
+
77
+ **Arguments**:
78
+ - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
79
+
80
+ **Options**:
81
+ - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
82
+ - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
83
+ - `--model-name TEXT`: The language model to use.
84
+ - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
85
+
86
+ **Models**:
87
+ You can use any of the predefined models below, or specify a new model using the format `"provider/model-name"`.
88
+ For example: `--model-name "google/gemini-flash-latest"`.
89
+
90
+ - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
91
+ - **Ollama**: `gemma:7b`, `phi4-mini`.
92
+ - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
93
+ - **OpenRouter**: `openai/gpt-4o`, `google/gemini-pro`.
94
+ - **OpenAI**: `gpt-4o`, `gpt-5`.
95
+ - **Anthropic**: `claude-4-sonet`, `claude-4.5-sonet`.
96
+
97
+ ### Examples
98
+
99
+ **Basic anonymization with the default model (Google)**:
100
+ ```bash
101
+ pdf-anonymizer run document.pdf
102
+ ```
103
+
104
+ **A new model (Google) and a simple prompt**:
105
+ ```bash
106
+ pdf-anonymizer run notes.md --model-name "google/gemini-flash-latest" --prompt-name simple
107
+ ```
108
+
109
+ **Using an OpenRouter model**:
110
+ ```bash
111
+ pdf-anonymizer run report.pdf --model-name "openai/gpt-4o"
112
+ ```
113
+
114
+ ### Deanonymize
115
+
116
+ The `deanonymize` command reverts anonymization using a mapping file.
117
+
118
+ ```bash
119
+ pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
120
+ ```
121
+
122
+ **Arguments**:
123
+ - `ANONYMIZED_FILE`: Path to the anonymized text file.
124
+ - `MAPPING_FILE`: Path to the JSON mapping file.
125
+
126
+ **Example**:
127
+ ```bash
128
+ pdf-anonymizer deanonymize \
129
+ data/anonymized/document.anonymized.md \
130
+ data/mappings/document.mapping.json
131
+ ```
132
+
133
+ This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.
134
+
135
+ ---
136
+
137
+ ## See Also
138
+
139
+ - **[Main Documentation](https://leo-gan.github.io/anonymizer/)** — Full guides including the 101 course.
140
+ - **[Core Package README](../pdf-anonymizer-core/README.md)** — Details on the underlying library.
141
+ - **[Recipes & Common Workflows](https://leo-gan.github.io/anonymizer/project/recipes/)** — Practical CLI usage examples (profiles, batching, external workflows, etc.).
142
+ - **[CLI Reference (full)](https://leo-gan.github.io/anonymizer/project/cli-usage/)** — Complete command reference on the docs site.
143
+ - **[Troubleshooting](https://leo-gan.github.io/anonymizer/project/troubleshooting/)** — Help with common CLI problems.
144
+ - **[Architecture](https://leo-gan.github.io/anonymizer/project/architecture/)** — How the CLI and core work together.
@@ -0,0 +1,131 @@
1
+ # 🦉🫥 PDF Anonymizer CLI
2
+
3
+ A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
4
+
5
+ - **High-Quality Anonymization**: Leverages LLMs to identify and replace Personally Identifiable Information (PII) with high accuracy.
6
+ - **Large File Support**: Consistently anonymizes large files (tested up to 1GB).
7
+ - **Multi-Provider & Cost-Effective**: Free to use with local [Ollama](https://ollama.com/) models. It also supports major providers like [OpenAI](https://openai.com/), [Anthropic](https://www.anthropic.com/), [Google](https://ai.google.com/), [Hugging Face](https://huggingface.co/), and [OpenRouter](https://openrouter.ai/).
8
+ - **Reversible**: Supports deanonymization to recover original data when needed.
9
+ - **Multi-Format**: Works with PDF, Markdown, and plain text files.
10
+
11
+
12
+ ## Installation
13
+
14
+ Install the CLI with your favorite package manager. To use a specific LLM provider, you must install the corresponding extra.
15
+
16
+ - **Google**: `pip install "pdf-anonymizer-cli[google]"`
17
+ - **Ollama**: `pip install "pdf-anonymizer-cli[ollama]"`
18
+ - **Hugging Face**: `pip install "pdf-anonymizer-cli[huggingface]"`
19
+ - **OpenRouter**: `pip install "pdf-anonymizer-cli[openrouter]"`
20
+ - **OpenAI**: `pip install "pdf-anonymizer-cli[openai]"`
21
+ - **Anthropic**: `pip install "pdf-anonymizer-cli[anthropic]"`
22
+
23
+ You can also install multiple extras at once:
24
+
25
+ ```bash
26
+ pip install "pdf-anonymizer-cli[google,openrouter]"
27
+ ```
28
+
29
+ This installs the `pdf-anonymizer` executable.
30
+
31
+
32
+ ## Environment Variables
33
+
34
+ The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
35
+
36
+ - `GOOGLE_API_KEY`: Required when using Google models.
37
+ - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
38
+ - `OPENROUTER_API_KEY`: Required when using OpenRouter models.
39
+ - `OPENAI_API_KEY`: Required when using OpenAI models.
40
+ - `ANTHROPIC_API_KEY`: Required when using Anthropic models.
41
+ - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using Ollama models.
42
+
43
+ Example `.env` file:
44
+ ```env
45
+ GOOGLE_API_KEY="YOUR_API_KEY_HERE"
46
+ HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
47
+ OPENROUTER_API_KEY="YOUR_OPENROUTER_KEY"
48
+ ```
49
+
50
+ ## Usage
51
+
52
+ ### Anonymize
53
+
54
+ The `run` command anonymizes one or more files.
55
+
56
+ ```bash
57
+ pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
58
+ [--characters-to-anonymize INTEGER] \
59
+ [--prompt-name {simple|detailed}] \
60
+ [--model-name TEXT] \
61
+ [--anonymized-entities PATH]
62
+ ```
63
+
64
+ **Arguments**:
65
+ - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
66
+
67
+ **Options**:
68
+ - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
69
+ - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
70
+ - `--model-name TEXT`: The language model to use.
71
+ - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
72
+
73
+ **Models**:
74
+ You can use any of the predefined models below, or specify a new model using the format `"provider/model-name"`.
75
+ For example: `--model-name "google/gemini-flash-latest"`.
76
+
77
+ - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
78
+ - **Ollama**: `gemma:7b`, `phi4-mini`.
79
+ - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
80
+ - **OpenRouter**: `openai/gpt-4o`, `google/gemini-pro`.
81
+ - **OpenAI**: `gpt-4o`, `gpt-5`.
82
+ - **Anthropic**: `claude-4-sonet`, `claude-4.5-sonet`.
83
+
84
+ ### Examples
85
+
86
+ **Basic anonymization with the default model (Google)**:
87
+ ```bash
88
+ pdf-anonymizer run document.pdf
89
+ ```
90
+
91
+ **A new model (Google) and a simple prompt**:
92
+ ```bash
93
+ pdf-anonymizer run notes.md --model-name "google/gemini-flash-latest" --prompt-name simple
94
+ ```
95
+
96
+ **Using an OpenRouter model**:
97
+ ```bash
98
+ pdf-anonymizer run report.pdf --model-name "openai/gpt-4o"
99
+ ```
100
+
101
+ ### Deanonymize
102
+
103
+ The `deanonymize` command reverts anonymization using a mapping file.
104
+
105
+ ```bash
106
+ pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
107
+ ```
108
+
109
+ **Arguments**:
110
+ - `ANONYMIZED_FILE`: Path to the anonymized text file.
111
+ - `MAPPING_FILE`: Path to the JSON mapping file.
112
+
113
+ **Example**:
114
+ ```bash
115
+ pdf-anonymizer deanonymize \
116
+ data/anonymized/document.anonymized.md \
117
+ data/mappings/document.mapping.json
118
+ ```
119
+
120
+ This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.
121
+
122
+ ---
123
+
124
+ ## See Also
125
+
126
+ - **[Main Documentation](https://leo-gan.github.io/anonymizer/)** — Full guides including the 101 course.
127
+ - **[Core Package README](../pdf-anonymizer-core/README.md)** — Details on the underlying library.
128
+ - **[Recipes & Common Workflows](https://leo-gan.github.io/anonymizer/project/recipes/)** — Practical CLI usage examples (profiles, batching, external workflows, etc.).
129
+ - **[CLI Reference (full)](https://leo-gan.github.io/anonymizer/project/cli-usage/)** — Complete command reference on the docs site.
130
+ - **[Troubleshooting](https://leo-gan.github.io/anonymizer/project/troubleshooting/)** — Help with common CLI problems.
131
+ - **[Architecture](https://leo-gan.github.io/anonymizer/project/architecture/)** — How the CLI and core work together.
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "pdf-anonymizer-cli"
3
- version = "0.3.1"
3
+ version = "0.3.3"
4
4
  description = "CLI for a tool to anonymize PDF, Markdown, and plain text files using LLMs."
5
5
  authors = [{ name = "Leonid Ganeline", email = "leo.gan.57@gmail.com" }]
6
6
  license = { text = "MIT" }
@@ -7,15 +7,14 @@ from typing import Dict, List, Optional
7
7
  import typer
8
8
  from dotenv import load_dotenv
9
9
  from pdf_anonymizer_core.conf import (
10
- DEFAULT_CHARACTERS_TO_ANONYMIZE,
11
- DEFAULT_MODEL_NAME,
12
- DEFAULT_PROMPT_NAME,
13
- ModelName,
14
- ModelProvider,
10
+ DEFAULT_LOG_FILE,
11
+ ConfigProfile,
15
12
  PromptEnum,
16
- get_enum_value,
13
+ get_config_for_profile,
14
+ get_provider_and_model_name,
17
15
  )
18
16
  from pdf_anonymizer_core.core import anonymize_file
17
+ from pdf_anonymizer_core.llm_provider import configure_cache
19
18
  from pdf_anonymizer_core.prompts import detailed, simple
20
19
  from pdf_anonymizer_core.utils import (
21
20
  consolidate_mapping,
@@ -24,10 +23,11 @@ from pdf_anonymizer_core.utils import (
24
23
  )
25
24
  from typing_extensions import Annotated
26
25
 
26
+ # Setup logging using log file configured in conf.py
27
27
  logging.basicConfig(
28
28
  level=logging.INFO,
29
29
  format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
30
- handlers=[logging.FileHandler("app.log"), logging.StreamHandler()],
30
+ handlers=[logging.FileHandler(DEFAULT_LOG_FILE), logging.StreamHandler()],
31
31
  )
32
32
 
33
33
  app = typer.Typer()
@@ -54,24 +54,34 @@ def run(
54
54
  resolve_path=True,
55
55
  ),
56
56
  ],
57
+ config_profile: Annotated[
58
+ ConfigProfile,
59
+ typer.Option(
60
+ "--config-profile",
61
+ "-p",
62
+ help="The configuration profile to use (best-quality, best-speed, best-cost).",
63
+ case_sensitive=False,
64
+ ),
65
+ ] = ConfigProfile.BEST_SPEED,
57
66
  characters_to_anonymize: Annotated[
58
- int,
59
- typer.Option(help="Number of characters to send for anonymization in one go."),
60
- ] = DEFAULT_CHARACTERS_TO_ANONYMIZE,
67
+ Optional[int],
68
+ typer.Option(
69
+ help="Override number of characters to send for anonymization in one go (chunk size)."
70
+ ),
71
+ ] = None,
61
72
  prompt_name: Annotated[
62
- PromptEnum,
73
+ Optional[PromptEnum],
63
74
  typer.Option(
64
- help="The name of the prompt to use for anonymization.",
75
+ help="Override prompt template to use for anonymization.",
65
76
  case_sensitive=False,
66
77
  ),
67
- ] = get_enum_value(PromptEnum, DEFAULT_PROMPT_NAME),
78
+ ] = None,
68
79
  model_name: Annotated[
69
- ModelName,
80
+ Optional[str],
70
81
  typer.Option(
71
- help="The name of the model to use for anonymization.",
72
- case_sensitive=False,
82
+ help="Override the language model to use for anonymization.",
73
83
  ),
74
- ] = get_enum_value(ModelName, DEFAULT_MODEL_NAME),
84
+ ] = None,
75
85
  anonymized_entities: Annotated[
76
86
  Optional[Path],
77
87
  typer.Option(
@@ -88,34 +98,45 @@ def run(
88
98
  ) -> None:
89
99
  """
90
100
  Anonymize one or more files by replacing PII with anonymized placeholders.
91
-
92
- Args:
93
- file_paths: List of paths to files to process.
94
- characters_to_anonymize: Number of characters to process in each chunk.
95
- prompt_name: The prompt template to use for anonymization.
96
- model_name: The language model to use for anonymization.
97
- anonymized_entities: A file with a list of entities to anonymize.
98
101
  """
99
102
  load_environment()
100
103
 
101
- if model_name.provider == ModelProvider.GOOGLE:
102
- if "gemini" in model_name.value and not os.getenv("GOOGLE_API_KEY"):
104
+ # Get configuration based on profile and optional user overrides
105
+ config = get_config_for_profile(
106
+ profile=config_profile,
107
+ model_name=model_name,
108
+ prompt_name=prompt_name,
109
+ chunk_size=characters_to_anonymize,
110
+ )
111
+
112
+ # Configure LLM caching with values from configuration
113
+ configure_cache(
114
+ enabled=config.enable_cache,
115
+ cache_dir=config.cache_dir,
116
+ cache_file=config.cache_file,
117
+ )
118
+
119
+ provider_name, _ = get_provider_and_model_name(config.model_name)
120
+ if provider_name == "google":
121
+ if "gemini" in config.model_name and not os.getenv("GOOGLE_API_KEY"):
103
122
  logging.error(
104
123
  "Error: GOOGLE_API_KEY not found. Please set it in the .env file."
105
124
  )
106
125
  sys.exit(1)
107
126
 
127
+ logging.info(f"Using configuration profile: {config_profile.value}")
108
128
  logging.info(f" --file-paths: {file_paths}")
109
- logging.info(f" --characters-to-anonymize: {characters_to_anonymize}")
110
- logging.info(f" --model-name: {model_name.value}")
129
+ logging.info(f" --chunk-size: {config.chunk_size}")
130
+ logging.info(f" --chunk-overlap: {config.chunk_overlap}")
131
+ logging.info(f" --model-name: {config.model_name}")
111
132
 
112
133
  # Select the appropriate prompt template
113
134
  prompt_templates: Dict[str, str] = {
114
- PromptEnum.simple: simple.prompt_template,
115
- PromptEnum.detailed: detailed.prompt_template,
135
+ PromptEnum.simple.value: simple.prompt_template,
136
+ PromptEnum.detailed.value: detailed.prompt_template,
116
137
  }
117
- prompt_template: str = prompt_templates[prompt_name]
118
- logging.info(f" --prompt-name: {prompt_name.value}")
138
+ prompt_template: str = prompt_templates[config.prompt_name]
139
+ logging.info(f" --prompt-name: {config.prompt_name}")
119
140
 
120
141
  entities_to_anonymize = None
121
142
  if anonymized_entities:
@@ -129,11 +150,16 @@ def run(
129
150
  logging.info("=" * 40)
130
151
  logging.info(f"Processing file {i}/{len(file_paths)}: {file_path}")
131
152
  full_anonymized_text, final_mapping = anonymize_file(
132
- str(file_path),
133
- characters_to_anonymize,
134
- prompt_template,
135
- model_name.value,
136
- entities_to_anonymize,
153
+ file_path=str(file_path),
154
+ characters_to_anonymize=config.chunk_size,
155
+ prompt_template=prompt_template,
156
+ model_name=config.model_name,
157
+ anonymized_entities=entities_to_anonymize,
158
+ chunk_overlap=config.chunk_overlap,
159
+ regex_patterns=config.regex_patterns,
160
+ max_retries=config.max_retries,
161
+ base_retry_delay=config.base_retry_delay,
162
+ max_retry_delay=config.max_retry_delay,
137
163
  )
138
164
 
139
165
  if full_anonymized_text and final_mapping:
@@ -183,10 +209,6 @@ def deanonymize(
183
209
  ) -> None:
184
210
  """
185
211
  Deanonymize a file using a mapping file.
186
-
187
- Args:
188
- anonymized_file: Path to the anonymized file.
189
- mapping_file: Path to the mapping file.
190
212
  """
191
213
  logging.info(f"Deanonymizing '{anonymized_file}' using '{mapping_file}'")
192
214
  deanonymized_output_file, stats_file = deanonymize_file(
@@ -0,0 +1,144 @@
1
+ Metadata-Version: 2.4
2
+ Name: pdf-anonymizer-cli
3
+ Version: 0.3.3
4
+ Summary: CLI for a tool to anonymize PDF, Markdown, and plain text files using LLMs.
5
+ Author-email: Leonid Ganeline <leo.gan.57@gmail.com>
6
+ License: MIT
7
+ Project-URL: repository, https://github.com/leo-gan/anonymizer
8
+ Requires-Python: >=3.10
9
+ Description-Content-Type: text/markdown
10
+ Requires-Dist: typer
11
+ Requires-Dist: rich
12
+ Requires-Dist: pdf-anonymizer-core
13
+
14
+ # 🦉🫥 PDF Anonymizer CLI
15
+
16
+ A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
17
+
18
+ - **High-Quality Anonymization**: Leverages LLMs to identify and replace Personally Identifiable Information (PII) with high accuracy.
19
+ - **Large File Support**: Consistently anonymizes large files (tested up to 1GB).
20
+ - **Multi-Provider & Cost-Effective**: Free to use with local [Ollama](https://ollama.com/) models. It also supports major providers like [OpenAI](https://openai.com/), [Anthropic](https://www.anthropic.com/), [Google](https://ai.google.com/), [Hugging Face](https://huggingface.co/), and [OpenRouter](https://openrouter.ai/).
21
+ - **Reversible**: Supports deanonymization to recover original data when needed.
22
+ - **Multi-Format**: Works with PDF, Markdown, and plain text files.
23
+
24
+
25
+ ## Installation
26
+
27
+ Install the CLI with your favorite package manager. To use a specific LLM provider, you must install the corresponding extra.
28
+
29
+ - **Google**: `pip install "pdf-anonymizer-cli[google]"`
30
+ - **Ollama**: `pip install "pdf-anonymizer-cli[ollama]"`
31
+ - **Hugging Face**: `pip install "pdf-anonymizer-cli[huggingface]"`
32
+ - **OpenRouter**: `pip install "pdf-anonymizer-cli[openrouter]"`
33
+ - **OpenAI**: `pip install "pdf-anonymizer-cli[openai]"`
34
+ - **Anthropic**: `pip install "pdf-anonymizer-cli[anthropic]"`
35
+
36
+ You can also install multiple extras at once:
37
+
38
+ ```bash
39
+ pip install "pdf-anonymizer-cli[google,openrouter]"
40
+ ```
41
+
42
+ This installs the `pdf-anonymizer` executable.
43
+
44
+
45
+ ## Environment Variables
46
+
47
+ The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
48
+
49
+ - `GOOGLE_API_KEY`: Required when using Google models.
50
+ - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
51
+ - `OPENROUTER_API_KEY`: Required when using OpenRouter models.
52
+ - `OPENAI_API_KEY`: Required when using OpenAI models.
53
+ - `ANTHROPIC_API_KEY`: Required when using Anthropic models.
54
+ - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using Ollama models.
55
+
56
+ Example `.env` file:
57
+ ```env
58
+ GOOGLE_API_KEY="YOUR_API_KEY_HERE"
59
+ HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
60
+ OPENROUTER_API_KEY="YOUR_OPENROUTER_KEY"
61
+ ```
62
+
63
+ ## Usage
64
+
65
+ ### Anonymize
66
+
67
+ The `run` command anonymizes one or more files.
68
+
69
+ ```bash
70
+ pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
71
+ [--characters-to-anonymize INTEGER] \
72
+ [--prompt-name {simple|detailed}] \
73
+ [--model-name TEXT] \
74
+ [--anonymized-entities PATH]
75
+ ```
76
+
77
+ **Arguments**:
78
+ - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
79
+
80
+ **Options**:
81
+ - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
82
+ - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
83
+ - `--model-name TEXT`: The language model to use.
84
+ - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
85
+
86
+ **Models**:
87
+ You can use any of the predefined models below, or specify a new model using the format `"provider/model-name"`.
88
+ For example: `--model-name "google/gemini-flash-latest"`.
89
+
90
+ - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
91
+ - **Ollama**: `gemma:7b`, `phi4-mini`.
92
+ - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
93
+ - **OpenRouter**: `openai/gpt-4o`, `google/gemini-pro`.
94
+ - **OpenAI**: `gpt-4o`, `gpt-5`.
95
+ - **Anthropic**: `claude-4-sonet`, `claude-4.5-sonet`.
96
+
97
+ ### Examples
98
+
99
+ **Basic anonymization with the default model (Google)**:
100
+ ```bash
101
+ pdf-anonymizer run document.pdf
102
+ ```
103
+
104
+ **A new model (Google) and a simple prompt**:
105
+ ```bash
106
+ pdf-anonymizer run notes.md --model-name "google/gemini-flash-latest" --prompt-name simple
107
+ ```
108
+
109
+ **Using an OpenRouter model**:
110
+ ```bash
111
+ pdf-anonymizer run report.pdf --model-name "openai/gpt-4o"
112
+ ```
113
+
114
+ ### Deanonymize
115
+
116
+ The `deanonymize` command reverts anonymization using a mapping file.
117
+
118
+ ```bash
119
+ pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
120
+ ```
121
+
122
+ **Arguments**:
123
+ - `ANONYMIZED_FILE`: Path to the anonymized text file.
124
+ - `MAPPING_FILE`: Path to the JSON mapping file.
125
+
126
+ **Example**:
127
+ ```bash
128
+ pdf-anonymizer deanonymize \
129
+ data/anonymized/document.anonymized.md \
130
+ data/mappings/document.mapping.json
131
+ ```
132
+
133
+ This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.
134
+
135
+ ---
136
+
137
+ ## See Also
138
+
139
+ - **[Main Documentation](https://leo-gan.github.io/anonymizer/)** — Full guides including the 101 course.
140
+ - **[Core Package README](../pdf-anonymizer-core/README.md)** — Details on the underlying library.
141
+ - **[Recipes & Common Workflows](https://leo-gan.github.io/anonymizer/project/recipes/)** — Practical CLI usage examples (profiles, batching, external workflows, etc.).
142
+ - **[CLI Reference (full)](https://leo-gan.github.io/anonymizer/project/cli-usage/)** — Complete command reference on the docs site.
143
+ - **[Troubleshooting](https://leo-gan.github.io/anonymizer/project/troubleshooting/)** — Help with common CLI problems.
144
+ - **[Architecture](https://leo-gan.github.io/anonymizer/project/architecture/)** — How the CLI and core work together.
@@ -1,103 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: pdf-anonymizer-cli
3
- Version: 0.3.1
4
- Summary: CLI for a tool to anonymize PDF, Markdown, and plain text files using LLMs.
5
- Author-email: Leonid Ganeline <leo.gan.57@gmail.com>
6
- License: MIT
7
- Project-URL: repository, https://github.com/leo-gan/anonymizer
8
- Requires-Python: >=3.10
9
- Description-Content-Type: text/markdown
10
- Requires-Dist: typer
11
- Requires-Dist: rich
12
- Requires-Dist: pdf-anonymizer-core
13
-
14
- # 🦉🫥 PDF Anonymizer CLI
15
-
16
- A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
17
-
18
- ## Installation
19
-
20
- This project uses `uv` and is structured as a monorepo. The dependencies for the CLI and its core library are managed at the root of the project.
21
-
22
- 1. **Install `uv`**: Follow the [official installation instructions](https://astral.sh/docs/uv#installation).
23
- 2. **Install dependencies from the repository root**:
24
- ```bash
25
- # From the repository root
26
- uv sync
27
- ```
28
- This installs the `pdf-anonymizer` executable.
29
-
30
- ## Environment Variables
31
-
32
- The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
33
-
34
- - `GOOGLE_API_KEY`: Required when using Google's Gemini models.
35
- - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using local Ollama models.
36
- - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
37
-
38
- Example `.env` file:
39
- ```env
40
- GOOGLE_API_KEY="YOUR_API_KEY_HERE"
41
- HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
42
- ```
43
-
44
- ## Usage
45
-
46
- ### Anonymize
47
-
48
- The `run` command anonymizes one or more files.
49
-
50
- ```bash
51
- pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
52
- [--characters-to-anonymize INTEGER] \
53
- [--prompt-name {simple|detailed}] \
54
- [--model-name TEXT] \
55
- [--anonymized-entities PATH]
56
- ```
57
-
58
- **Arguments**:
59
- - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
60
-
61
- **Options**:
62
- - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
63
- - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
64
- - `--model-name TEXT`: The language model to use.
65
- - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
66
-
67
- **Models**:
68
- - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
69
- - **Ollama**: `gemma:7b`, `phi4-mini`.
70
- - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
71
-
72
- ### Examples
73
-
74
- **Basic anonymization**:
75
- ```bash
76
- pdf-anonymizer run document.pdf
77
- ```
78
-
79
- **Custom model and prompt**:
80
- ```bash
81
- pdf-anonymizer run notes.md --model-name phi4-mini --prompt-name simple
82
- ```
83
-
84
- ### Deanonymize
85
-
86
- The `deanonymize` command reverts anonymization using a mapping file.
87
-
88
- ```bash
89
- pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
90
- ```
91
-
92
- **Arguments**:
93
- - `ANONYMIZED_FILE`: Path to the anonymized text file.
94
- - `MAPPING_FILE`: Path to the JSON mapping file.
95
-
96
- **Example**:
97
- ```bash
98
- pdf-anonymizer deanonymize \
99
- data/anonymized/document.anonymized.md \
100
- data/mappings/document.mapping.json
101
- ```
102
-
103
- This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.
@@ -1,90 +0,0 @@
1
- # 🦉🫥 PDF Anonymizer CLI
2
-
3
- A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
4
-
5
- ## Installation
6
-
7
- This project uses `uv` and is structured as a monorepo. The dependencies for the CLI and its core library are managed at the root of the project.
8
-
9
- 1. **Install `uv`**: Follow the [official installation instructions](https://astral.sh/docs/uv#installation).
10
- 2. **Install dependencies from the repository root**:
11
- ```bash
12
- # From the repository root
13
- uv sync
14
- ```
15
- This installs the `pdf-anonymizer` executable.
16
-
17
- ## Environment Variables
18
-
19
- The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
20
-
21
- - `GOOGLE_API_KEY`: Required when using Google's Gemini models.
22
- - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using local Ollama models.
23
- - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
24
-
25
- Example `.env` file:
26
- ```env
27
- GOOGLE_API_KEY="YOUR_API_KEY_HERE"
28
- HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
29
- ```
30
-
31
- ## Usage
32
-
33
- ### Anonymize
34
-
35
- The `run` command anonymizes one or more files.
36
-
37
- ```bash
38
- pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
39
- [--characters-to-anonymize INTEGER] \
40
- [--prompt-name {simple|detailed}] \
41
- [--model-name TEXT] \
42
- [--anonymized-entities PATH]
43
- ```
44
-
45
- **Arguments**:
46
- - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
47
-
48
- **Options**:
49
- - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
50
- - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
51
- - `--model-name TEXT`: The language model to use.
52
- - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
53
-
54
- **Models**:
55
- - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
56
- - **Ollama**: `gemma:7b`, `phi4-mini`.
57
- - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
58
-
59
- ### Examples
60
-
61
- **Basic anonymization**:
62
- ```bash
63
- pdf-anonymizer run document.pdf
64
- ```
65
-
66
- **Custom model and prompt**:
67
- ```bash
68
- pdf-anonymizer run notes.md --model-name phi4-mini --prompt-name simple
69
- ```
70
-
71
- ### Deanonymize
72
-
73
- The `deanonymize` command reverts anonymization using a mapping file.
74
-
75
- ```bash
76
- pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
77
- ```
78
-
79
- **Arguments**:
80
- - `ANONYMIZED_FILE`: Path to the anonymized text file.
81
- - `MAPPING_FILE`: Path to the JSON mapping file.
82
-
83
- **Example**:
84
- ```bash
85
- pdf-anonymizer deanonymize \
86
- data/anonymized/document.anonymized.md \
87
- data/mappings/document.mapping.json
88
- ```
89
-
90
- This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.
@@ -1,103 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: pdf-anonymizer-cli
3
- Version: 0.3.1
4
- Summary: CLI for a tool to anonymize PDF, Markdown, and plain text files using LLMs.
5
- Author-email: Leonid Ganeline <leo.gan.57@gmail.com>
6
- License: MIT
7
- Project-URL: repository, https://github.com/leo-gan/anonymizer
8
- Requires-Python: >=3.10
9
- Description-Content-Type: text/markdown
10
- Requires-Dist: typer
11
- Requires-Dist: rich
12
- Requires-Dist: pdf-anonymizer-core
13
-
14
- # 🦉🫥 PDF Anonymizer CLI
15
-
16
- A command-line interface for anonymizing PDF, Markdown, and plain text files using LLMs.
17
-
18
- ## Installation
19
-
20
- This project uses `uv` and is structured as a monorepo. The dependencies for the CLI and its core library are managed at the root of the project.
21
-
22
- 1. **Install `uv`**: Follow the [official installation instructions](https://astral.sh/docs/uv#installation).
23
- 2. **Install dependencies from the repository root**:
24
- ```bash
25
- # From the repository root
26
- uv sync
27
- ```
28
- This installs the `pdf-anonymizer` executable.
29
-
30
- ## Environment Variables
31
-
32
- The CLI will automatically load a `.env` file from the current directory or any parent directory. For consistency, it's recommended to place a single `.env` file at the root of the repository.
33
-
34
- - `GOOGLE_API_KEY`: Required when using Google's Gemini models.
35
- - `OLLAMA_HOST`: Optional, defaults to `http://localhost:11434` when using local Ollama models.
36
- - `HUGGING_FACE_TOKEN`: Required when using Hugging Face models. You can get a token from [here](https://huggingface.co/docs/hub/security-tokens).
37
-
38
- Example `.env` file:
39
- ```env
40
- GOOGLE_API_KEY="YOUR_API_KEY_HERE"
41
- HUGGING_FACE_TOKEN="YOUR_HF_TOKEN_HERE"
42
- ```
43
-
44
- ## Usage
45
-
46
- ### Anonymize
47
-
48
- The `run` command anonymizes one or more files.
49
-
50
- ```bash
51
- pdf-anonymizer run FILE_PATH [FILE_PATH ...] \
52
- [--characters-to-anonymize INTEGER] \
53
- [--prompt-name {simple|detailed}] \
54
- [--model-name TEXT] \
55
- [--anonymized-entities PATH]
56
- ```
57
-
58
- **Arguments**:
59
- - `FILE_PATH`: Path to one or several PDF, Markdown, or text files for anonymization.
60
-
61
- **Options**:
62
- - `--characters-to-anonymize INTEGER`: Number of characters to process in each chunk (default: `100000`).
63
- - `--prompt-name [simple|detailed]`: The prompt template to use (default: `detailed`).
64
- - `--model-name TEXT`: The language model to use.
65
- - `--anonymized-entities PATH`: Path to a file with a list of entities to anonymize.
66
-
67
- **Models**:
68
- - **Google**: `gemini-2.5-pro`, `gemini-2.5-flash` (default), `gemini-2.5-flash-lite`.
69
- - **Ollama**: `gemma:7b`, `phi4-mini`.
70
- - **Hugging Face**: `openai/gpt-oss-20b`, `mistralai/Mistral-7B-Instruct-v0.1`, `HuggingFaceH4/zephyr-7b-beta`.
71
-
72
- ### Examples
73
-
74
- **Basic anonymization**:
75
- ```bash
76
- pdf-anonymizer run document.pdf
77
- ```
78
-
79
- **Custom model and prompt**:
80
- ```bash
81
- pdf-anonymizer run notes.md --model-name phi4-mini --prompt-name simple
82
- ```
83
-
84
- ### Deanonymize
85
-
86
- The `deanonymize` command reverts anonymization using a mapping file.
87
-
88
- ```bash
89
- pdf-anonymizer deanonymize ANONYMIZED_FILE MAPPING_FILE
90
- ```
91
-
92
- **Arguments**:
93
- - `ANONYMIZED_FILE`: Path to the anonymized text file.
94
- - `MAPPING_FILE`: Path to the JSON mapping file.
95
-
96
- **Example**:
97
- ```bash
98
- pdf-anonymizer deanonymize \
99
- data/anonymized/document.anonymized.md \
100
- data/mappings/document.mapping.json
101
- ```
102
-
103
- This will create a deanonymized version of the file at `data/deanonymized/document.deanonymized.md`.