redprobe 0.1.0__tar.gz → 0.1.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. {redprobe-0.1.0 → redprobe-0.1.2}/PKG-INFO +19 -25
  2. {redprobe-0.1.0 → redprobe-0.1.2}/README.md +18 -21
  3. {redprobe-0.1.0 → redprobe-0.1.2}/pyproject.toml +1 -6
  4. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/consent.py +11 -7
  5. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/PKG-INFO +19 -25
  6. {redprobe-0.1.0 → redprobe-0.1.2}/CONTRIBUTING.md +0 -0
  7. {redprobe-0.1.0 → redprobe-0.1.2}/HISTORY.md +0 -0
  8. {redprobe-0.1.0 → redprobe-0.1.2}/LICENSE +0 -0
  9. {redprobe-0.1.0 → redprobe-0.1.2}/MANIFEST.in +0 -0
  10. {redprobe-0.1.0 → redprobe-0.1.2}/docs/index.md +0 -0
  11. {redprobe-0.1.0 → redprobe-0.1.2}/docs/installation.md +0 -0
  12. {redprobe-0.1.0 → redprobe-0.1.2}/docs/usage.md +0 -0
  13. {redprobe-0.1.0 → redprobe-0.1.2}/setup.cfg +0 -0
  14. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/__init__.py +0 -0
  15. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/__main__.py +0 -0
  16. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/cli.py +0 -0
  17. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/client.py +0 -0
  18. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/probes.py +0 -0
  19. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/redprobe.py +0 -0
  20. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/reporter.py +0 -0
  21. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/runner.py +0 -0
  22. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe/utils.py +0 -0
  23. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/SOURCES.txt +0 -0
  24. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/dependency_links.txt +0 -0
  25. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/entry_points.txt +0 -0
  26. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/requires.txt +0 -0
  27. {redprobe-0.1.0 → redprobe-0.1.2}/src/redprobe.egg-info/top_level.txt +0 -0
  28. {redprobe-0.1.0 → redprobe-0.1.2}/tests/__init__.py +0 -0
  29. {redprobe-0.1.0 → redprobe-0.1.2}/tests/test_cli.py +0 -0
  30. {redprobe-0.1.0 → redprobe-0.1.2}/tests/test_probes.py +0 -0
  31. {redprobe-0.1.0 → redprobe-0.1.2}/tests/test_redprobe.py +0 -0
  32. {redprobe-0.1.0 → redprobe-0.1.2}/tests/test_runner.py +0 -0
@@ -1,13 +1,10 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: redprobe
3
- Version: 0.1.0
3
+ Version: 0.1.2
4
4
  Summary: A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint.
5
5
  Author-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
6
6
  Maintainer-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
7
7
  License: BUSL 1.1
8
- Project-URL: bugs, https://github.com/audreyfeldroy/redprobe/issues
9
- Project-URL: changelog, https://github.com/audreyfeldroy/redprobe/blob/master/changelog.md
10
- Project-URL: homepage, https://github.com/audreyfeldroy/redprobe
11
8
  Requires-Python: >=3.10
12
9
  Description-Content-Type: text/markdown
13
10
  License-File: LICENSE
@@ -32,19 +29,16 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
32
29
  ## Quick Start
33
30
 
34
31
  ```bash
35
- # Run with uv (recommended)
36
- uvx redprobe
37
-
38
32
  # Generate sample probes
39
- redprobe init
33
+ uvx redprobe init
40
34
 
41
35
  # Run probes against a model
42
- redprobe run probes/
36
+ uvx redprobe run probes/
43
37
  ```
44
38
 
45
39
  ## Prerequisites
46
40
 
47
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
41
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
48
42
 
49
43
  ### Setting up LM Studio
50
44
 
@@ -75,12 +69,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
75
69
 
76
70
  ## Commands
77
71
 
78
- ### `redprobe init`
72
+ ### `uvx redprobe init`
79
73
 
80
74
  Create sample probe files to get started:
81
75
 
82
76
  ```bash
83
- $ redprobe init
77
+ $ uvx redprobe init
84
78
  Created sample probes in probes/
85
79
 
86
80
  Files created:
@@ -89,21 +83,21 @@ Files created:
89
83
  - harmful.yaml (3 probes)
90
84
 
91
85
  Run probes with:
92
- redprobe run probes
86
+ uvx redprobe run probes
93
87
  ```
94
88
 
95
89
  You can specify a different directory:
96
90
 
97
91
  ```bash
98
- redprobe init my-probes/
92
+ uvx redprobe init my-probes/
99
93
  ```
100
94
 
101
- ### `redprobe list-probes`
95
+ ### `uvx redprobe list-probes`
102
96
 
103
97
  Preview what probes will run without actually calling the API:
104
98
 
105
99
  ```bash
106
- $ redprobe list-probes probes/
100
+ $ uvx redprobe list-probes probes/
107
101
  Bias Detection
108
102
  Probes for demographic and social biases in model responses
109
103
 
@@ -130,12 +124,12 @@ Jailbreak Attempts
130
124
  MEDIUM jailbreak-003: Roleplay bypass
131
125
  ```
132
126
 
133
- ### `redprobe run`
127
+ ### `uvx redprobe run`
134
128
 
135
129
  Run probes against a model:
136
130
 
137
131
  ```bash
138
- $ redprobe run probes/
132
+ $ uvx redprobe run probes/
139
133
  Loaded 9 probes from 3 suite(s)
140
134
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
141
135
 
@@ -174,7 +168,7 @@ Running suite: Jailbreak Attempts
174
168
  #### Command Options
175
169
 
176
170
  ```bash
177
- redprobe run probes/ \
171
+ uvx redprobe run probes/ \
178
172
  --base-url http://localhost:1234/v1 \ # API endpoint
179
173
  --model openai/gpt-oss-20b \ # Model to test
180
174
  --api-key not-needed \ # API key (if required)
@@ -278,7 +272,7 @@ cases:
278
272
 
279
273
  ### Organizing Probes
280
274
 
281
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
275
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
282
276
 
283
277
  ```
284
278
  probes/
@@ -292,12 +286,12 @@ probes/
292
286
 
293
287
  Run a specific file:
294
288
  ```bash
295
- redprobe run probes/jailbreaks.yaml
289
+ uvx redprobe run probes/jailbreaks.yaml
296
290
  ```
297
291
 
298
292
  Run all probes in a directory:
299
293
  ```bash
300
- redprobe run probes/
294
+ uvx redprobe run probes/
301
295
  ```
302
296
 
303
297
  ## Understanding Results
@@ -321,7 +315,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
321
315
  Use `--output` to export full results including model responses:
322
316
 
323
317
  ```bash
324
- redprobe run probes/ --output results.json
318
+ uvx redprobe run probes/ --output results.json
325
319
  ```
326
320
 
327
321
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -334,7 +328,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
334
328
  # Start Ollama with a model
335
329
  ollama serve
336
330
 
337
- redprobe run probes/ \
331
+ uvx redprobe run probes/ \
338
332
  --base-url http://localhost:11434/v1 \
339
333
  --model llama2
340
334
  ```
@@ -342,7 +336,7 @@ redprobe run probes/ \
342
336
  ### OpenAI
343
337
 
344
338
  ```bash
345
- redprobe run probes/ \
339
+ uvx redprobe run probes/ \
346
340
  --base-url https://api.openai.com/v1 \
347
341
  --model gpt-4o-mini \
348
342
  --api-key $OPENAI_API_KEY
@@ -7,19 +7,16 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
7
7
  ## Quick Start
8
8
 
9
9
  ```bash
10
- # Run with uv (recommended)
11
- uvx redprobe
12
-
13
10
  # Generate sample probes
14
- redprobe init
11
+ uvx redprobe init
15
12
 
16
13
  # Run probes against a model
17
- redprobe run probes/
14
+ uvx redprobe run probes/
18
15
  ```
19
16
 
20
17
  ## Prerequisites
21
18
 
22
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
19
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
23
20
 
24
21
  ### Setting up LM Studio
25
22
 
@@ -50,12 +47,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
50
47
 
51
48
  ## Commands
52
49
 
53
- ### `redprobe init`
50
+ ### `uvx redprobe init`
54
51
 
55
52
  Create sample probe files to get started:
56
53
 
57
54
  ```bash
58
- $ redprobe init
55
+ $ uvx redprobe init
59
56
  Created sample probes in probes/
60
57
 
61
58
  Files created:
@@ -64,21 +61,21 @@ Files created:
64
61
  - harmful.yaml (3 probes)
65
62
 
66
63
  Run probes with:
67
- redprobe run probes
64
+ uvx redprobe run probes
68
65
  ```
69
66
 
70
67
  You can specify a different directory:
71
68
 
72
69
  ```bash
73
- redprobe init my-probes/
70
+ uvx redprobe init my-probes/
74
71
  ```
75
72
 
76
- ### `redprobe list-probes`
73
+ ### `uvx redprobe list-probes`
77
74
 
78
75
  Preview what probes will run without actually calling the API:
79
76
 
80
77
  ```bash
81
- $ redprobe list-probes probes/
78
+ $ uvx redprobe list-probes probes/
82
79
  Bias Detection
83
80
  Probes for demographic and social biases in model responses
84
81
 
@@ -105,12 +102,12 @@ Jailbreak Attempts
105
102
  MEDIUM jailbreak-003: Roleplay bypass
106
103
  ```
107
104
 
108
- ### `redprobe run`
105
+ ### `uvx redprobe run`
109
106
 
110
107
  Run probes against a model:
111
108
 
112
109
  ```bash
113
- $ redprobe run probes/
110
+ $ uvx redprobe run probes/
114
111
  Loaded 9 probes from 3 suite(s)
115
112
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
116
113
 
@@ -149,7 +146,7 @@ Running suite: Jailbreak Attempts
149
146
  #### Command Options
150
147
 
151
148
  ```bash
152
- redprobe run probes/ \
149
+ uvx redprobe run probes/ \
153
150
  --base-url http://localhost:1234/v1 \ # API endpoint
154
151
  --model openai/gpt-oss-20b \ # Model to test
155
152
  --api-key not-needed \ # API key (if required)
@@ -253,7 +250,7 @@ cases:
253
250
 
254
251
  ### Organizing Probes
255
252
 
256
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
253
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
257
254
 
258
255
  ```
259
256
  probes/
@@ -267,12 +264,12 @@ probes/
267
264
 
268
265
  Run a specific file:
269
266
  ```bash
270
- redprobe run probes/jailbreaks.yaml
267
+ uvx redprobe run probes/jailbreaks.yaml
271
268
  ```
272
269
 
273
270
  Run all probes in a directory:
274
271
  ```bash
275
- redprobe run probes/
272
+ uvx redprobe run probes/
276
273
  ```
277
274
 
278
275
  ## Understanding Results
@@ -296,7 +293,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
296
293
  Use `--output` to export full results including model responses:
297
294
 
298
295
  ```bash
299
- redprobe run probes/ --output results.json
296
+ uvx redprobe run probes/ --output results.json
300
297
  ```
301
298
 
302
299
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -309,7 +306,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
309
306
  # Start Ollama with a model
310
307
  ollama serve
311
308
 
312
- redprobe run probes/ \
309
+ uvx redprobe run probes/ \
313
310
  --base-url http://localhost:11434/v1 \
314
311
  --model llama2
315
312
  ```
@@ -317,7 +314,7 @@ redprobe run probes/ \
317
314
  ### OpenAI
318
315
 
319
316
  ```bash
320
- redprobe run probes/ \
317
+ uvx redprobe run probes/ \
321
318
  --base-url https://api.openai.com/v1 \
322
319
  --model gpt-4o-mini \
323
320
  --api-key $OPENAI_API_KEY
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "redprobe"
3
- version = "0.1.0"
3
+ version = "0.1.2"
4
4
  description = "A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint."
5
5
  readme = "README.md"
6
6
  authors = [
@@ -30,11 +30,6 @@ test = [
30
30
  "ipdb", # debugging
31
31
  ]
32
32
 
33
- [project.urls]
34
- bugs = "https://github.com/audreyfeldroy/redprobe/issues"
35
- changelog = "https://github.com/audreyfeldroy/redprobe/blob/master/changelog.md"
36
- homepage = "https://github.com/audreyfeldroy/redprobe"
37
-
38
33
  [project.scripts]
39
34
  redprobe = "redprobe.cli:app"
40
35
 
@@ -12,18 +12,22 @@ CONFIG_DIR = Path.home() / ".config" / "redprobe"
12
12
  CONSENT_FILE = CONFIG_DIR / "consent.json"
13
13
 
14
14
  # Version of the consent terms - bump this to require re-acknowledgment
15
- CONSENT_VERSION = "1.0"
15
+ CONSENT_VERSION = "1.1"
16
16
 
17
17
  TERMS_TEXT = """\
18
- [bold]RedProbe[/bold] is a security testing tool for AI systems.
18
+ [bold]RedProbe[/bold] is a defensive security tool for hardening AI systems.
19
19
 
20
- [bold yellow]Before using this tool, you acknowledge that:[/bold yellow]
20
+ [bold yellow]You may only use this tool for:[/bold yellow]
21
+ • Systems you own or operate
22
+ • Systems you have written permission to test
23
+ • Research/educational contexts with appropriate oversight
21
24
 
22
- 1. You will only test systems you own or have written permission to test
23
- 2. You accept responsibility for how you use this tool
24
- 3. You have read the responsible use guidelines
25
+ [bold yellow]You may NOT use this tool to:[/bold yellow]
26
+ Test systems without authorization
27
+ Extract private data from systems you don't own
28
+ • Generate harmful content for distribution
25
29
 
26
- See: https://github.com/audreyfeldroy/redprobe/blob/main/RESPONSIBLE_USE.md
30
+ By proceeding, you accept responsibility for your use of this tool.
27
31
  """
28
32
 
29
33
 
@@ -1,13 +1,10 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: redprobe
3
- Version: 0.1.0
3
+ Version: 0.1.2
4
4
  Summary: A defensive security tool for hardening AI systems. Define YAML-based test cases to systematically probe LLMs for jailbreaks, prompt injections, biases, harmful content generation, data leakage, and policy violations before attackers find them. Compatible with any OpenAI-style API endpoint.
5
5
  Author-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
6
6
  Maintainer-email: "Audrey M. Roy Greenfeld" <audrey@feldroy.com>
7
7
  License: BUSL 1.1
8
- Project-URL: bugs, https://github.com/audreyfeldroy/redprobe/issues
9
- Project-URL: changelog, https://github.com/audreyfeldroy/redprobe/blob/master/changelog.md
10
- Project-URL: homepage, https://github.com/audreyfeldroy/redprobe
11
8
  Requires-Python: >=3.10
12
9
  Description-Content-Type: text/markdown
13
10
  License-File: LICENSE
@@ -32,19 +29,16 @@ A defensive security tool for hardening AI systems. Define YAML-based test cases
32
29
  ## Quick Start
33
30
 
34
31
  ```bash
35
- # Run with uv (recommended)
36
- uvx redprobe
37
-
38
32
  # Generate sample probes
39
- redprobe init
33
+ uvx redprobe init
40
34
 
41
35
  # Run probes against a model
42
- redprobe run probes/
36
+ uvx redprobe run probes/
43
37
  ```
44
38
 
45
39
  ## Prerequisites
46
40
 
47
- RedProbe works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
41
+ RedProbe requires [uv](https://docs.astral.sh/uv/getting-started/installation/) and works with any OpenAI-compatible API. The default configuration targets [LM Studio](https://lmstudio.ai/) running locally.
48
42
 
49
43
  ### Setting up LM Studio
50
44
 
@@ -75,12 +69,12 @@ Do not use RedProbe to test systems without authorization, extract private data,
75
69
 
76
70
  ## Commands
77
71
 
78
- ### `redprobe init`
72
+ ### `uvx redprobe init`
79
73
 
80
74
  Create sample probe files to get started:
81
75
 
82
76
  ```bash
83
- $ redprobe init
77
+ $ uvx redprobe init
84
78
  Created sample probes in probes/
85
79
 
86
80
  Files created:
@@ -89,21 +83,21 @@ Files created:
89
83
  - harmful.yaml (3 probes)
90
84
 
91
85
  Run probes with:
92
- redprobe run probes
86
+ uvx redprobe run probes
93
87
  ```
94
88
 
95
89
  You can specify a different directory:
96
90
 
97
91
  ```bash
98
- redprobe init my-probes/
92
+ uvx redprobe init my-probes/
99
93
  ```
100
94
 
101
- ### `redprobe list-probes`
95
+ ### `uvx redprobe list-probes`
102
96
 
103
97
  Preview what probes will run without actually calling the API:
104
98
 
105
99
  ```bash
106
- $ redprobe list-probes probes/
100
+ $ uvx redprobe list-probes probes/
107
101
  Bias Detection
108
102
  Probes for demographic and social biases in model responses
109
103
 
@@ -130,12 +124,12 @@ Jailbreak Attempts
130
124
  MEDIUM jailbreak-003: Roleplay bypass
131
125
  ```
132
126
 
133
- ### `redprobe run`
127
+ ### `uvx redprobe run`
134
128
 
135
129
  Run probes against a model:
136
130
 
137
131
  ```bash
138
- $ redprobe run probes/
132
+ $ uvx redprobe run probes/
139
133
  Loaded 9 probes from 3 suite(s)
140
134
  Target: http://localhost:1234/v1 model openai/gpt-oss-20b
141
135
 
@@ -174,7 +168,7 @@ Running suite: Jailbreak Attempts
174
168
  #### Command Options
175
169
 
176
170
  ```bash
177
- redprobe run probes/ \
171
+ uvx redprobe run probes/ \
178
172
  --base-url http://localhost:1234/v1 \ # API endpoint
179
173
  --model openai/gpt-oss-20b \ # Model to test
180
174
  --api-key not-needed \ # API key (if required)
@@ -278,7 +272,7 @@ cases:
278
272
 
279
273
  ### Organizing Probes
280
274
 
281
- You can split probes across multiple YAML files. When you point `redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
275
+ You can split probes across multiple YAML files. When you point `uvx redprobe run` at a directory, it loads all `.yaml` and `.yml` files:
282
276
 
283
277
  ```
284
278
  probes/
@@ -292,12 +286,12 @@ probes/
292
286
 
293
287
  Run a specific file:
294
288
  ```bash
295
- redprobe run probes/jailbreaks.yaml
289
+ uvx redprobe run probes/jailbreaks.yaml
296
290
  ```
297
291
 
298
292
  Run all probes in a directory:
299
293
  ```bash
300
- redprobe run probes/
294
+ uvx redprobe run probes/
301
295
  ```
302
296
 
303
297
  ## Understanding Results
@@ -321,7 +315,7 @@ A triggered probe does not necessarily mean the model is unsafe. It means the re
321
315
  Use `--output` to export full results including model responses:
322
316
 
323
317
  ```bash
324
- redprobe run probes/ --output results.json
318
+ uvx redprobe run probes/ --output results.json
325
319
  ```
326
320
 
327
321
  The JSON includes timestamps, prompts, full responses, and matched patterns for each probe.
@@ -334,7 +328,7 @@ The JSON includes timestamps, prompts, full responses, and matched patterns for
334
328
  # Start Ollama with a model
335
329
  ollama serve
336
330
 
337
- redprobe run probes/ \
331
+ uvx redprobe run probes/ \
338
332
  --base-url http://localhost:11434/v1 \
339
333
  --model llama2
340
334
  ```
@@ -342,7 +336,7 @@ redprobe run probes/ \
342
336
  ### OpenAI
343
337
 
344
338
  ```bash
345
- redprobe run probes/ \
339
+ uvx redprobe run probes/ \
346
340
  --base-url https://api.openai.com/v1 \
347
341
  --model gpt-4o-mini \
348
342
  --api-key $OPENAI_API_KEY
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes