redprobe 0.1.1__tar.gz → 0.1.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. {redprobe-0.1.1 → redprobe-0.1.3}/PKG-INFO +22 -22
  2. {redprobe-0.1.1 → redprobe-0.1.3}/README.md +21 -21
  3. {redprobe-0.1.1 → redprobe-0.1.3}/pyproject.toml +1 -1
  4. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/consent.py +11 -7
  5. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/PKG-INFO +22 -22
  6. {redprobe-0.1.1 → redprobe-0.1.3}/CONTRIBUTING.md +0 -0
  7. {redprobe-0.1.1 → redprobe-0.1.3}/HISTORY.md +0 -0
  8. {redprobe-0.1.1 → redprobe-0.1.3}/LICENSE +0 -0
  9. {redprobe-0.1.1 → redprobe-0.1.3}/MANIFEST.in +0 -0
  10. {redprobe-0.1.1 → redprobe-0.1.3}/docs/index.md +0 -0
  11. {redprobe-0.1.1 → redprobe-0.1.3}/docs/installation.md +0 -0
  12. {redprobe-0.1.1 → redprobe-0.1.3}/docs/usage.md +0 -0
  13. {redprobe-0.1.1 → redprobe-0.1.3}/setup.cfg +0 -0
  14. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/__init__.py +0 -0
  15. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/__main__.py +0 -0
  16. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/cli.py +0 -0
  17. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/client.py +0 -0
  18. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/probes.py +0 -0
  19. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/redprobe.py +0 -0
  20. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/reporter.py +0 -0
  21. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/runner.py +0 -0
  22. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe/utils.py +0 -0
  23. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/SOURCES.txt +0 -0
  24. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/dependency_links.txt +0 -0
  25. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/entry_points.txt +0 -0
  26. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/requires.txt +0 -0
  27. {redprobe-0.1.1 → redprobe-0.1.3}/src/redprobe.egg-info/top_level.txt +0 -0
  28. {redprobe-0.1.1 → redprobe-0.1.3}/tests/__init__.py +0 -0
  29. {redprobe-0.1.1 → redprobe-0.1.3}/tests/test_cli.py +0 -0
  30. {redprobe-0.1.1 → redprobe-0.1.3}/tests/test_probes.py +0 -0
  31. {redprobe-0.1.1 → redprobe-0.1.3}/tests/test_redprobe.py +0 -0
  32. {redprobe-0.1.1 → redprobe-0.1.3}/tests/test_runner.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: redprobe
3
- Version: 0.1.1
3
+ Version: 0.1.3
4
4
  Summary: A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint.
5
5
  Author-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
6
6
  Maintainer-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
@@ -29,19 +29,19 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
29
29
  ## Quick Start
30
30
 
31
31
  ```bash
32
- # Run with uv (recommended)
33
- uvx redprobe
34
-
35
32
  # Generate sample probes
36
- redprobe init
33
+ uvx redprobe init
37
34
 
38
35
  # Run probes against a model
39
- redprobe run probes/
36
+ uvx redprobe run probes/
37
+
38
+ # Upgrade to latest version
39
+ uvx redprobe@latest
40
40
  ```
41
41
 
42
42
  ## Prerequisites
43
43
 
44
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
44
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
45
45
 
46
46
  ### Setting up LM Studio
47
47
 
@@ -72,12 +72,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
72
72
 
73
73
  ## Commands
74
74
 
75
- ### `redprobe init`
75
+ ### `uvx redprobe init`
76
76
 
77
77
  Create sample probe files to get started:
78
78
 
79
79
  ```bash
80
- $ redprobe init
80
+ $ uvx redprobe init
81
81
  Created sample probes in probes/
82
82
 
83
83
  Files created:
@@ -86,21 +86,21 @@ Files created:
86
86
  - harmful.yaml (3 probes)
87
87
 
88
88
  Run probes with:
89
- redprobe run probes
89
+ uvx redprobe run probes
90
90
  ```
91
91
 
92
92
  You can specify a different directory:
93
93
 
94
94
  ```bash
95
- redprobe init my-probes/
95
+ uvx redprobe init my-probes/
96
96
  ```
97
97
 
98
- ### `redprobe list-probes`
98
+ ### `uvx redprobe list-probes`
99
99
 
100
100
  Preview what probes will run without actually calling the API:
101
101
 
102
102
  ```bash
103
- $ redprobe list-probes probes/
103
+ $ uvx redprobe list-probes probes/
104
104
  Bias Detection
105
105
  Probes for demographic and social biases in model responses
106
106
 
@@ -127,12 +127,12 @@ Jailbreak Attempts
127
127
  MEDIUM jailbreak-003: Roleplay bypass
128
128
  ```
129
129
 
130
- ### `redprobe run`
130
+ ### `uvx redprobe run`
131
131
 
132
132
  Run probes against a model:
133
133
 
134
134
  ```bash
135
- $ redprobe run probes/
135
+ $ uvx redprobe run probes/
136
136
  Loaded 9 probes from 3 suite(s)
137
137
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
138
138
 
@@ -171,7 +171,7 @@ Running suite: Jailbreak Attempts
171
171
  #### Command Options
172
172
 
173
173
  ```bash
174
- redprobe run probes/ \
174
+ uvx redprobe run probes/ \
175
175
  --base-url http://localhost:1234/v1 \ # API endpoint
176
176
  --model openai/gpt-oss-20b \ # Model to test
177
177
  --api-key not-needed \ # API key (if required)
@@ -275,7 +275,7 @@ cases:
275
275
 
276
276
  ### Organizing Probes
277
277
 
278
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
278
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
279
279
 
280
280
  ```
281
281
  probes/
@@ -289,12 +289,12 @@ probes/
289
289
 
290
290
  Run a specific file:
291
291
  ```bash
292
- redprobe run probes/jailbreaks.yaml
292
+ uvx redprobe run probes/jailbreaks.yaml
293
293
  ```
294
294
 
295
295
  Run all probes in a directory:
296
296
  ```bash
297
- redprobe run probes/
297
+ uvx redprobe run probes/
298
298
  ```
299
299
 
300
300
  ## Understanding Results
@@ -318,7 +318,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
318
318
  Use `--output` to export full results including model responses:
319
319
 
320
320
  ```bash
321
- redprobe run probes/ --output results.json
321
+ uvx redprobe run probes/ --output results.json
322
322
  ```
323
323
 
324
324
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -331,7 +331,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
331
331
  # Start Ollama with a model
332
332
  ollama serve
333
333
 
334
- redprobe run probes/ \
334
+ uvx redprobe run probes/ \
335
335
  --base-url http://localhost:11434/v1 \
336
336
  --model llama2
337
337
  ```
@@ -339,7 +339,7 @@ redprobe run probes/ \
339
339
  ### OpenAI
340
340
 
341
341
  ```bash
342
- redprobe run probes/ \
342
+ uvx redprobe run probes/ \
343
343
  --base-url https://api.openai.com/v1 \
344
344
  --model gpt-4o-mini \
345
345
  --api-key $OPENAI_API_KEY
@@ -7,19 +7,19 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
7
7
  ## Quick Start
8
8
 
9
9
  ```bash
10
- # Run with uv (recommended)
11
- uvx redprobe
12
-
13
10
  # Generate sample probes
14
- redprobe init
11
+ uvx redprobe init
15
12
 
16
13
  # Run probes against a model
17
- redprobe run probes/
14
+ uvx redprobe run probes/
15
+
16
+ # Upgrade to latest version
17
+ uvx redprobe@latest
18
18
  ```
19
19
 
20
20
  ## Prerequisites
21
21
 
22
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
22
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
23
23
 
24
24
  ### Setting up LM Studio
25
25
 
@@ -50,12 +50,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
50
50
 
51
51
  ## Commands
52
52
 
53
- ### `redprobe init`
53
+ ### `uvx redprobe init`
54
54
 
55
55
  Create sample probe files to get started:
56
56
 
57
57
  ```bash
58
- $ redprobe init
58
+ $ uvx redprobe init
59
59
  Created sample probes in probes/
60
60
 
61
61
  Files created:
@@ -64,21 +64,21 @@ Files created:
64
64
  - harmful.yaml (3 probes)
65
65
 
66
66
  Run probes with:
67
- redprobe run probes
67
+ uvx redprobe run probes
68
68
  ```
69
69
 
70
70
  You can specify a different directory:
71
71
 
72
72
  ```bash
73
- redprobe init my-probes/
73
+ uvx redprobe init my-probes/
74
74
  ```
75
75
 
76
- ### `redprobe list-probes`
76
+ ### `uvx redprobe list-probes`
77
77
 
78
78
  Preview what probes will run without actually calling the API:
79
79
 
80
80
  ```bash
81
- $ redprobe list-probes probes/
81
+ $ uvx redprobe list-probes probes/
82
82
  Bias Detection
83
83
  Probes for demographic and social biases in model responses
84
84
 
@@ -105,12 +105,12 @@ Jailbreak Attempts
105
105
  MEDIUM jailbreak-003: Roleplay bypass
106
106
  ```
107
107
 
108
- ### `redprobe run`
108
+ ### `uvx redprobe run`
109
109
 
110
110
  Run probes against a model:
111
111
 
112
112
  ```bash
113
- $ redprobe run probes/
113
+ $ uvx redprobe run probes/
114
114
  Loaded 9 probes from 3 suite(s)
115
115
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
116
116
 
@@ -149,7 +149,7 @@ Running suite: Jailbreak Attempts
149
149
  #### Command Options
150
150
 
151
151
  ```bash
152
- redprobe run probes/ \
152
+ uvx redprobe run probes/ \
153
153
  --base-url http://localhost:1234/v1 \ # API endpoint
154
154
  --model openai/gpt-oss-20b \ # Model to test
155
155
  --api-key not-needed \ # API key (if required)
@@ -253,7 +253,7 @@ cases:
253
253
 
254
254
  ### Organizing Probes
255
255
 
256
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
256
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
257
257
 
258
258
  ```
259
259
  probes/
@@ -267,12 +267,12 @@ probes/
267
267
 
268
268
  Run a specific file:
269
269
  ```bash
270
- redprobe run probes/jailbreaks.yaml
270
+ uvx redprobe run probes/jailbreaks.yaml
271
271
  ```
272
272
 
273
273
  Run all probes in a directory:
274
274
  ```bash
275
- redprobe run probes/
275
+ uvx redprobe run probes/
276
276
  ```
277
277
 
278
278
  ## Understanding Results
@@ -296,7 +296,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
296
296
  Use `--output` to export full results including model responses:
297
297
 
298
298
  ```bash
299
- redprobe run probes/ --output results.json
299
+ uvx redprobe run probes/ --output results.json
300
300
  ```
301
301
 
302
302
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -309,7 +309,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
309
309
  # Start Ollama with a model
310
310
  ollama serve
311
311
 
312
- redprobe run probes/ \
312
+ uvx redprobe run probes/ \
313
313
  --base-url http://localhost:11434/v1 \
314
314
  --model llama2
315
315
  ```
@@ -317,7 +317,7 @@ redprobe run probes/ \
317
317
  ### OpenAI
318
318
 
319
319
  ```bash
320
- redprobe run probes/ \
320
+ uvx redprobe run probes/ \
321
321
  --base-url https://api.openai.com/v1 \
322
322
  --model gpt-4o-mini \
323
323
  --api-key $OPENAI_API_KEY
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "redprobe"
3
- version = "0.1.1"
3
+ version = "0.1.3"
4
4
  description = "A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint."
5
5
  readme = "README.md"
6
6
  authors = [
@@ -12,18 +12,22 @@ CONFIG_DIR = Path.home() / ".config" / "redprobe"
12
12
  CONSENT_FILE = CONFIG_DIR / "consent.json"
13
13
 
14
14
  # Version of the consent terms - bump this to require re-acknowledgment
15
- CONSENT_VERSION = "1.0"
15
+ CONSENT_VERSION = "1.1"
16
16
 
17
17
  TERMS_TEXT = """\
18
- [bold]RedProbe[/bold] is a security testing tool for AI systems.
18
+ [bold]RedProbe[/bold] is a defensive security tool for hardening AI systems.
19
19
 
20
- [bold yellow]Before using this tool, you acknowledge that:[/bold yellow]
20
+ [bold yellow]You may only use this tool for:[/bold yellow]
21
+ • Systems you own or operate
22
+ • Systems you have written permission to test
23
+ • Research/educational contexts with appropriate oversight
21
24
 
22
- 1. You will only test systems you own or have written permission to test
23
- 2. You accept responsibility for how you use this tool
24
- 3. You have read the responsible use guidelines
25
+ [bold yellow]You may NOT use this tool to:[/bold yellow]
26
+ Test systems without authorization
27
+ Extract private data from systems you don't own
28
+ • Generate harmful content for distribution
25
29
 
26
- See: https://github.com/audreyfeldroy/redprobe/blob/main/RESPONSIBLE_USE.md
30
+ By proceeding, you accept responsibility for your use of this tool.
27
31
  """
28
32
 
29
33
 
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: redprobe
3
- Version: 0.1.1
3
+ Version: 0.1.3
4
4
  Summary: A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint.
5
5
  Author-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
6
6
  Maintainer-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
@@ -29,19 +29,19 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
29
29
  ## Quick Start
30
30
 
31
31
  ```bash
32
- # Run with uv (recommended)
33
- uvx redprobe
34
-
35
32
  # Generate sample probes
36
- redprobe init
33
+ uvx redprobe init
37
34
 
38
35
  # Run probes against a model
39
- redprobe run probes/
36
+ uvx redprobe run probes/
37
+
38
+ # Upgrade to latest version
39
+ uvx redprobe@latest
40
40
  ```
41
41
 
42
42
  ## Prerequisites
43
43
 
44
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
44
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
45
45
 
46
46
  ### Setting up LM Studio
47
47
 
@@ -72,12 +72,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
72
72
 
73
73
  ## Commands
74
74
 
75
- ### `redprobe init`
75
+ ### `uvx redprobe init`
76
76
 
77
77
  Create sample probe files to get started:
78
78
 
79
79
  ```bash
80
- $ redprobe init
80
+ $ uvx redprobe init
81
81
  Created sample probes in probes/
82
82
 
83
83
  Files created:
@@ -86,21 +86,21 @@ Files created:
86
86
  - harmful.yaml (3 probes)
87
87
 
88
88
  Run probes with:
89
- redprobe run probes
89
+ uvx redprobe run probes
90
90
  ```
91
91
 
92
92
  You can specify a different directory:
93
93
 
94
94
  ```bash
95
- redprobe init my-probes/
95
+ uvx redprobe init my-probes/
96
96
  ```
97
97
 
98
- ### `redprobe list-probes`
98
+ ### `uvx redprobe list-probes`
99
99
 
100
100
  Preview what probes will run without actually calling the API:
101
101
 
102
102
  ```bash
103
- $ redprobe list-probes probes/
103
+ $ uvx redprobe list-probes probes/
104
104
  Bias Detection
105
105
  Probes for demographic and social biases in model responses
106
106
 
@@ -127,12 +127,12 @@ Jailbreak Attempts
127
127
  MEDIUM jailbreak-003: Roleplay bypass
128
128
  ```
129
129
 
130
- ### `redprobe run`
130
+ ### `uvx redprobe run`
131
131
 
132
132
  Run probes against a model:
133
133
 
134
134
  ```bash
135
- $ redprobe run probes/
135
+ $ uvx redprobe run probes/
136
136
  Loaded 9 probes from 3 suite(s)
137
137
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
138
138
 
@@ -171,7 +171,7 @@ Running suite: Jailbreak Attempts
171
171
  #### Command Options
172
172
 
173
173
  ```bash
174
- redprobe run probes/ \
174
+ uvx redprobe run probes/ \
175
175
  --base-url http://localhost:1234/v1 \ # API endpoint
176
176
  --model openai/gpt-oss-20b \ # Model to test
177
177
  --api-key not-needed \ # API key (if required)
@@ -275,7 +275,7 @@ cases:
275
275
 
276
276
  ### Organizing Probes
277
277
 
278
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
278
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
279
279
 
280
280
  ```
281
281
  probes/
@@ -289,12 +289,12 @@ probes/
289
289
 
290
290
  Run a specific file:
291
291
  ```bash
292
- redprobe run probes/jailbreaks.yaml
292
+ uvx redprobe run probes/jailbreaks.yaml
293
293
  ```
294
294
 
295
295
  Run all probes in a directory:
296
296
  ```bash
297
- redprobe run probes/
297
+ uvx redprobe run probes/
298
298
  ```
299
299
 
300
300
  ## Understanding Results
@@ -318,7 +318,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
318
318
  Use `--output` to export full results including model responses:
319
319
 
320
320
  ```bash
321
- redprobe run probes/ --output results.json
321
+ uvx redprobe run probes/ --output results.json
322
322
  ```
323
323
 
324
324
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -331,7 +331,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
331
331
  # Start Ollama with a model
332
332
  ollama serve
333
333
 
334
- redprobe run probes/ \
334
+ uvx redprobe run probes/ \
335
335
  --base-url http://localhost:11434/v1 \
336
336
  --model llama2
337
337
  ```
@@ -339,7 +339,7 @@ redprobe run probes/ \
339
339
  ### OpenAI
340
340
 
341
341
  ```bash
342
- redprobe run probes/ \
342
+ uvx redprobe run probes/ \
343
343
  --base-url https://api.openai.com/v1 \
344
344
  --model gpt-4o-mini \
345
345
  --api-key $OPENAI_API_KEY
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes