pyrit 0.2.0__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pyrit-0.2.0 → pyrit-0.3.0}/LICENSE +21 -21
- {pyrit-0.2.0/pyrit.egg-info → pyrit-0.3.0}/PKG-INFO +113 -115
- {pyrit-0.2.0 → pyrit-0.3.0}/README.md +51 -51
- {pyrit-0.2.0 → pyrit-0.3.0}/pyproject.toml +158 -160
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/__init__.py +1 -1
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/analytics/__init__.py +7 -7
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/analytics/conversation_analytics.py +89 -89
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/auth/auth_config.py +7 -6
- pyrit-0.3.0/pyrit/auth/azure_auth.py +134 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/__init__.py +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/chat_message_nop.py +13 -13
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/chat_message_normalizer.py +41 -41
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/chat_message_normalizer_chatml.py +40 -40
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/generic_system_squash.py +34 -34
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/cache_helper.py +28 -28
- pyrit-0.3.0/pyrit/common/constants.py +7 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/default_values.py +47 -47
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/inference.py +76 -76
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/logger.py +23 -23
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/net_utility.py +78 -78
- pyrit-0.3.0/pyrit/common/notebook_utils.py +18 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/path.py +35 -20
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/print.py +58 -58
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/prompt_template_generator.py +54 -54
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/singleton.py +22 -22
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/orchestrators/benchmark/question_answer.yaml +23 -23
- pyrit-0.3.0/pyrit/datasets/orchestrators/red_teaming/image_generation.yaml +28 -0
- pyrit-0.3.0/pyrit/datasets/orchestrators/red_teaming/naive_crescendo.yaml +23 -0
- pyrit-0.3.0/pyrit/datasets/orchestrators/red_teaming/test_persusion.yaml +22 -0
- pyrit-0.2.0/pyrit/datasets/orchestrators/red_teaming/end_token.yaml → pyrit-0.3.0/pyrit/datasets/orchestrators/red_teaming/text_generation.yaml +22 -25
- pyrit-0.3.0/pyrit/datasets/orchestrators/skeleton_key/skeleton_key.prompt +11 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/atbash_description.yaml +30 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/caesar_description.yaml +33 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/codechameleon_converter.yaml +97 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/expand_converter.yaml +21 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/morse_description.yaml +29 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/noise_converter.yaml +15 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/past_tense_converter.yaml +15 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/persuasion/authority_endorsement.yaml +87 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/persuasion/evidence_based.yaml +103 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/persuasion/expert_endorsement.yaml +89 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/persuasion/logical_appeal.yaml +102 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/persuasion/misrepresentation.yaml +87 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/shorten_converter.yaml +23 -0
- pyrit-0.3.0/pyrit/datasets/prompt_converters/tone_converter.yaml +13 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_converters/translation_converter.yaml +22 -22
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_converters/variation_converter.yaml +21 -21
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_converters/variation_converter_prompt_softener.yaml +23 -23
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/aim.yaml +33 -33
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/aligned.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/anti_gpt.yaml +15 -15
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/apophis.yaml +84 -84
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/axies.yaml +22 -22
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/balakula.yaml +20 -20
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/based_gpt_1.yaml +67 -67
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/based_gpt_2.yaml +46 -46
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/better_dan.yaml +29 -29
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/bh.yaml +20 -20
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/bish.yaml +42 -42
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/burple.yaml +14 -14
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/chad_gpt.yaml +22 -22
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/cipher_chat.yaml +16 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/coach_bobby_knight.yaml +55 -55
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/code_nesting.yaml +22 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/cody.yaml +25 -25
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/complex.yaml +21 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/confronting_personalities.yaml +20 -20
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/cooper.yaml +28 -28
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/cosmos_dan.yaml +45 -45
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_1.yaml +70 -70
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_11.yaml +67 -67
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_5.yaml +33 -33
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_7.yaml +90 -90
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_8.yaml +24 -24
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dan_9.yaml +54 -54
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/delta_gpt.yaml +16 -16
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dev_mode_1.yaml +59 -59
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dev_mode_2.yaml +59 -59
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dev_mode_3.yaml +59 -59
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dev_mode_compact.yaml +15 -15
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dev_mode_ranti.yaml +69 -69
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dude_1.yaml +40 -40
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dude_2.yaml +40 -40
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/dude_3.yaml +41 -41
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/eva.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/evil_chad_2.yaml +43 -43
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/evil_confidant.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/fr3d.yaml +39 -39
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/gpt_4_real.yaml +15 -15
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/gpt_4_simulator.yaml +46 -46
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/hackerman.yaml +28 -28
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/hypothetical_response.yaml +14 -14
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/instructions.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/jailbreak_1.yaml +31 -31
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/jailbreak_2.yaml +63 -63
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/jb.yaml +24 -24
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/jedi_mind_trick.yaml +24 -24
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/john.yaml +13 -13
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/kevin.yaml +19 -19
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/khajiit.yaml +71 -71
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/leo.yaml +57 -57
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/live_gpt.yaml +29 -29
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/m78.yaml +32 -32
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/man.yaml +58 -58
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/maximum.yaml +56 -56
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/meanie.yaml +31 -31
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/moralizing_rant.yaml +14 -14
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/mr_blonde.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/neco.yaml +34 -34
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/nraf.yaml +30 -30
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/omega.yaml +76 -76
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/omni.yaml +36 -36
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/oppo.yaml +24 -24
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/person_gpt.yaml +34 -34
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/prefix_injection.yaml +10 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/ranti.yaml +16 -16
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/refusal_suppression.yaml +19 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/ron.yaml +44 -44
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/security_researcher.yaml +10 -10
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/sim.yaml +35 -35
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/steve.yaml +27 -27
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/style_injection.yaml +10 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/superior_dan.yaml +73 -73
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/switch.yaml +19 -19
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/table_nesting.yaml +27 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/text_continuation.yaml +15 -15
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/text_continuation_nesting.yaml +29 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/three_liner.yaml +17 -17
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/translator_bot.yaml +23 -23
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/tuo.yaml +19 -19
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/ucar.yaml +56 -56
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/un_gpt.yaml +39 -39
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/violet.yaml +15 -15
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompt_templates/jailbreak/void.yaml +47 -47
- pyrit-0.3.0/pyrit/datasets/prompt_templates/jailbreak/wikipedia_with_title.yaml +10 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompts/gandalf.prompt +9 -9
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/prompts/illegal.prompt +8 -8
- pyrit-0.2.0/pyrit/datasets/score/scoring_instructions/classification_system_prompt.yaml → pyrit-0.3.0/pyrit/datasets/score/content_classifiers/content_classifier_system_prompt.yaml +52 -48
- pyrit-0.3.0/pyrit/datasets/score/content_classifiers/harmful_content.yaml +25 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/content_classifiers/sentiment.yaml +55 -51
- pyrit-0.3.0/pyrit/datasets/score/conversation_objective/conversation_objective_prompt.yaml +65 -0
- pyrit-0.3.0/pyrit/datasets/score/conversation_objective/eval.yaml +52 -0
- pyrit-0.3.0/pyrit/datasets/score/conversation_objective/refusal.yaml +50 -0
- pyrit-0.3.0/pyrit/datasets/score/likert_scales/cyber.yaml +12 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/likert_scales/fairness_bias.yaml +7 -6
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/likert_scales/harm.yaml +12 -12
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/likert_scales/hate_speech.yaml +7 -6
- pyrit-0.3.0/pyrit/datasets/score/likert_scales/likert_system_prompt.yaml +62 -0
- pyrit-0.2.0/pyrit/datasets/score/likert_scales/political_misinfo.yaml → pyrit-0.3.0/pyrit/datasets/score/likert_scales/misinformation.yaml +12 -11
- pyrit-0.2.0/pyrit/datasets/score/likert_scales/phish_emails.yaml → pyrit-0.3.0/pyrit/datasets/score/likert_scales/phishing.yaml +12 -11
- pyrit-0.2.0/pyrit/datasets/score/likert_scales/persuasion.yaml → pyrit-0.3.0/pyrit/datasets/score/likert_scales/self_harm.yaml +12 -11
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/likert_scales/sexual.yaml +8 -7
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/datasets/score/likert_scales/violence.yaml +7 -6
- pyrit-0.3.0/pyrit/datasets/score/meta/meta_judge.yaml +49 -0
- pyrit-0.3.0/pyrit/datasets/score/meta/meta_scorer_prompt.yaml +65 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/current_events.yaml +3 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/gandalf.yaml +5 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/grounded.yaml +18 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/prompt_injection.yaml +3 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/question_answering.yaml +3 -0
- pyrit-0.3.0/pyrit/datasets/score/true_false_question/true_false_system_prompt.yaml +60 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/embedding/_text_embedding.py +54 -54
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/embedding/azure_text_embedding.py +45 -45
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/embedding/openai_text_embedding.py +22 -22
- pyrit-0.3.0/pyrit/exceptions/__init__.py +23 -0
- pyrit-0.3.0/pyrit/exceptions/exception_classes.py +137 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/interfaces.py +106 -106
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/__init__.py +21 -12
- pyrit-0.3.0/pyrit/memory/azure_sql_memory.py +251 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/duckdb_memory.py +336 -298
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/memory_embedding.py +59 -59
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/memory_exporter.py +131 -91
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/memory_interface.py +287 -249
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/memory/memory_models.py +203 -157
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/__init__.py +46 -35
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/chat_message.py +32 -32
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/data_type_serializer.py +222 -188
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/identifiers.py +14 -14
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/literals.py +18 -18
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/models.py +313 -314
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/prompt_request_piece.py +153 -104
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/models/prompt_request_response.py +141 -110
- pyrit-0.3.0/pyrit/models/score.py +115 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/orchestrator/__init__.py +24 -24
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/orchestrator/orchestrator_class.py +80 -80
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/orchestrator/prompt_sending_orchestrator.py +95 -95
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/orchestrator/question_answer_benchmark_orchestrator.py +107 -107
- pyrit-0.3.0/pyrit/orchestrator/red_teaming_orchestrator.py +340 -0
- pyrit-0.3.0/pyrit/orchestrator/scoring_orchestrator.py +90 -0
- pyrit-0.3.0/pyrit/orchestrator/skeleton_key_orchestrator.py +169 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/orchestrator/xpia_orchestrator.py +16 -10
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/__init__.py +59 -35
- pyrit-0.3.0/pyrit/prompt_converter/add_text_image_converter.py +144 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/ascii_art_converter.py +33 -31
- pyrit-0.3.0/pyrit/prompt_converter/atbash_converter.py +68 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/azure_speech_text_to_audio_converter.py +106 -119
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/base64_converter.py +26 -24
- pyrit-0.3.0/pyrit/prompt_converter/caesar_converter.py +73 -0
- pyrit-0.3.0/pyrit/prompt_converter/codechameleon_converter.py +290 -0
- pyrit-0.3.0/pyrit/prompt_converter/expand_converter.py +113 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/leetspeak_converter.py +47 -45
- pyrit-0.3.0/pyrit/prompt_converter/morse_converter.py +155 -0
- pyrit-0.3.0/pyrit/prompt_converter/noise_converter.py +101 -0
- pyrit-0.3.0/pyrit/prompt_converter/persuasion_converter.py +111 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/prompt_converter.py +55 -55
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/random_capital_letters_converter.py +76 -75
- pyrit-0.3.0/pyrit/prompt_converter/repeat_token_converter.py +89 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/rot13_converter.py +25 -21
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/search_replace_converter.py +37 -36
- pyrit-0.3.0/pyrit/prompt_converter/shorten_converter.py +113 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/string_join_converter.py +34 -32
- pyrit-0.3.0/pyrit/prompt_converter/suffix_append_converter.py +37 -0
- pyrit-0.3.0/pyrit/prompt_converter/tone_converter.py +81 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/translation_converter.py +108 -96
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/unicode_confusable_converter.py +49 -48
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/unicode_sub_converter.py +28 -25
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_converter/variation_converter.py +98 -82
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_normalizer/__init__.py +14 -12
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_normalizer/normalizer_request.py +85 -73
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_normalizer/prompt_normalizer.py +239 -211
- pyrit-0.3.0/pyrit/prompt_normalizer/prompt_response_converter_configuration.py +21 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/__init__.py +33 -33
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/azure_blob_storage_target.py +141 -137
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/azure_openai_completion_target.py +152 -132
- pyrit-0.3.0/pyrit/prompt_target/dall_e_target.py +167 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/gandalf_target.py +106 -78
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_chat_target/azure_ml_chat_target.py +177 -238
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_chat_target/azure_openai_gptv_chat_target.py +321 -309
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_chat_target/ollama_chat_target.py +97 -130
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_chat_target/openai_chat_target.py +309 -339
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_chat_target/prompt_chat_target.py +69 -97
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/prompt_target.py +59 -60
- pyrit-0.3.0/pyrit/prompt_target/text_target.py +69 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/prompt_target/tts_target.py +118 -129
- pyrit-0.3.0/pyrit/score/__init__.py +35 -0
- pyrit-0.3.0/pyrit/score/azure_content_filter_scorer.py +170 -0
- pyrit-0.3.0/pyrit/score/human_in_the_loop_scorer.py +89 -0
- pyrit-0.3.0/pyrit/score/markdown_injection.py +63 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/score/question_answer_scorer.py +138 -136
- pyrit-0.3.0/pyrit/score/scorer.py +110 -0
- pyrit-0.3.0/pyrit/score/self_ask_category_scorer.py +159 -0
- pyrit-0.3.0/pyrit/score/self_ask_conversation_objective_scorer.py +126 -0
- pyrit-0.3.0/pyrit/score/self_ask_likert_scorer.py +153 -0
- pyrit-0.3.0/pyrit/score/self_ask_meta_scorer.py +120 -0
- pyrit-0.3.0/pyrit/score/self_ask_true_false_scorer.py +125 -0
- pyrit-0.3.0/pyrit/score/substring_scorer.py +49 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/show_versions.py +83 -83
- {pyrit-0.2.0 → pyrit-0.3.0/pyrit.egg-info}/PKG-INFO +113 -115
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit.egg-info/SOURCES.txt +73 -89
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit.egg-info/requires.txt +5 -6
- {pyrit-0.2.0 → pyrit-0.3.0}/setup.cfg +4 -4
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_attack_strategy.py +33 -38
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_azure_auth.py +45 -38
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_azure_text_embedding.py +53 -53
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_chat_message_normalizer.py +71 -71
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_chat_normalizer_chatml.py +82 -82
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_common_default.py +23 -23
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_common_net_utility.py +78 -78
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_data_type_serializer.py +142 -120
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_embedding_response.py +49 -49
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_models.py +68 -68
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_normalizer_request.py +90 -90
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_ollama_chat_target.py +102 -96
- pyrit-0.3.0/tests/test_prompt_normalizer.py +254 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_prompt_request_piece.py +233 -207
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_prompt_response.py +44 -44
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_prompt_template_generator.py +62 -62
- pyrit-0.2.0/.env_example +0 -110
- pyrit-0.2.0/.flake8 +0 -19
- pyrit-0.2.0/.github/ISSUE_TEMPLATE/blank_template.md +0 -8
- pyrit-0.2.0/.github/ISSUE_TEMPLATE/bug_report.md +0 -54
- pyrit-0.2.0/.github/ISSUE_TEMPLATE/config.yml +0 -1
- pyrit-0.2.0/.github/ISSUE_TEMPLATE/doc_improvement.md +0 -20
- pyrit-0.2.0/.github/ISSUE_TEMPLATE/feature_request.md +0 -22
- pyrit-0.2.0/.github/PULL_REQUEST_TEMPLATE.md +0 -26
- pyrit-0.2.0/.github/workflows/build_and_test.yml +0 -64
- pyrit-0.2.0/.gitignore +0 -167
- pyrit-0.2.0/.pre-commit-config.yaml +0 -42
- pyrit-0.2.0/CODE_OF_CONDUCT.md +0 -9
- pyrit-0.2.0/Makefile +0 -26
- pyrit-0.2.0/SECURITY.md +0 -41
- pyrit-0.2.0/SUPPORT.md +0 -25
- pyrit-0.2.0/assets/aml_compute_cluster.png +0 -0
- pyrit-0.2.0/assets/aml_deployment_name.png +0 -0
- pyrit-0.2.0/assets/aml_deployment_resource_not_ready_error.png +0 -0
- pyrit-0.2.0/assets/aml_endpoint_deployment.png +0 -0
- pyrit-0.2.0/assets/aml_hf_model.png +0 -0
- pyrit-0.2.0/assets/aml_managed_online_endpoint_api_key.png +0 -0
- pyrit-0.2.0/assets/aml_model_endpoint_schema.png +0 -0
- pyrit-0.2.0/assets/aml_score_key.png +0 -0
- pyrit-0.2.0/assets/aml_score_uri.png +0 -0
- pyrit-0.2.0/assets/aml_ws_model.png +0 -0
- pyrit-0.2.0/assets/gandalf-demo-setup.png +0 -0
- pyrit-0.2.0/assets/gandalf-home-level-1.png +0 -0
- pyrit-0.2.0/assets/huggingface_model_id.png +0 -0
- pyrit-0.2.0/assets/operation-setup.jpg +0 -0
- pyrit-0.2.0/assets/pyrit_architecture.png +0 -0
- pyrit-0.2.0/assets/self-ask-prompting-example.png +0 -0
- pyrit-0.2.0/component-governance.yml +0 -17
- pyrit-0.2.0/doc/README.md +0 -27
- pyrit-0.2.0/doc/about_pyrit.md +0 -53
- pyrit-0.2.0/doc/code/orchestrator.ipynb +0 -369
- pyrit-0.2.0/doc/code/orchestrator.py +0 -122
- pyrit-0.2.0/doc/code/scoring.ipynb +0 -185
- pyrit-0.2.0/doc/code/scoring.py +0 -82
- pyrit-0.2.0/doc/contributing.md +0 -306
- pyrit-0.2.0/doc/demo/1_gandalf.ipynb +0 -280
- pyrit-0.2.0/doc/demo/1_gandalf.py +0 -106
- pyrit-0.2.0/doc/demo/2_multiturn_strategies.ipynb +0 -244
- pyrit-0.2.0/doc/demo/2_multiturn_strategies.py +0 -65
- pyrit-0.2.0/doc/deployment/deploy_hf_model_aml.ipynb +0 -350
- pyrit-0.2.0/doc/deployment/deploy_hf_model_aml.py +0 -230
- pyrit-0.2.0/doc/deployment/download_and_register_hf_model_aml.ipynb +0 -511
- pyrit-0.2.0/doc/deployment/download_and_register_hf_model_aml.py +0 -373
- pyrit-0.2.0/doc/deployment/hf_aml_model_endpoint_guide.md +0 -52
- pyrit-0.2.0/doc/deployment/score_aml_endpoint.ipynb +0 -168
- pyrit-0.2.0/doc/deployment/score_aml_endpoint.py +0 -121
- pyrit-0.2.0/doc/deployment/troubleshooting_guide_hf_azureml.md +0 -40
- pyrit-0.2.0/doc/faqs.md +0 -41
- pyrit-0.2.0/doc/glossary.md +0 -7
- pyrit-0.2.0/doc/how_to_guide.ipynb +0 -746
- pyrit-0.2.0/doc/how_to_guide.py +0 -237
- pyrit-0.2.0/doc/setup/install_pyrit.md +0 -17
- pyrit-0.2.0/doc/setup/setup_azure.md +0 -17
- pyrit-0.2.0/policheck.yml +0 -34
- pyrit-0.2.0/pyrit/auth/azure_auth.py +0 -116
- pyrit-0.2.0/pyrit/common/constants.py +0 -5
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/bias.yaml +0 -11
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/current_events.yaml +0 -7
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/grounded.yaml +0 -20
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/harmful_content.yaml +0 -21
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/prompt_injection_detector.yaml +0 -5
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/question_answering.yaml +0 -5
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/refusal.yaml +0 -7
- pyrit-0.2.0/pyrit/datasets/score/content_classifiers/sexual_content.yaml +0 -9
- pyrit-0.2.0/pyrit/datasets/score/likert_scales/cyber.yaml +0 -11
- pyrit-0.2.0/pyrit/datasets/score/scoring_instructions/likert_system_prompt.yaml +0 -49
- pyrit-0.2.0/pyrit/orchestrator/end_token_red_teaming_orchestrator.py +0 -81
- pyrit-0.2.0/pyrit/orchestrator/red_teaming_orchestrator.py +0 -205
- pyrit-0.2.0/pyrit/orchestrator/scoring_red_teaming_orchestrator.py +0 -76
- pyrit-0.2.0/pyrit/prompt_target/dall_e_target.py +0 -185
- pyrit-0.2.0/pyrit/prompt_target/text_target.py +0 -39
- pyrit-0.2.0/pyrit/score/__init__.py +0 -37
- pyrit-0.2.0/pyrit/score/engine.py +0 -105
- pyrit-0.2.0/pyrit/score/gandalf_classifier.py +0 -127
- pyrit-0.2.0/pyrit/score/gpt_classifier.py +0 -162
- pyrit-0.2.0/pyrit/score/markdown_injection.py +0 -41
- pyrit-0.2.0/pyrit/score/score_class.py +0 -21
- pyrit-0.2.0/pyrit/score/substring_scorer.py +0 -21
- pyrit-0.2.0/pyrit/score/support_text_classification.py +0 -14
- pyrit-0.2.0/pyrit/score/text.py +0 -25
- pyrit-0.2.0/tests/__init__.py +0 -2
- pyrit-0.2.0/tests/data/embedding_1.json +0 -1552
- pyrit-0.2.0/tests/data/embedding_2.json +0 -1552
- pyrit-0.2.0/tests/memory/test_memory_embedding.py +0 -84
- pyrit-0.2.0/tests/memory/test_memory_interface.py +0 -187
- pyrit-0.2.0/tests/mocks.py +0 -155
- pyrit-0.2.0/tests/test_markdown_injection.py +0 -27
- pyrit-0.2.0/tests/test_prompt_converter.py +0 -203
- pyrit-0.2.0/tests/test_prompt_normalizer.py +0 -152
- pyrit-0.2.0/tests/test_scorers.py +0 -29
- {pyrit-0.2.0 → pyrit-0.3.0}/MANIFEST.in +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/NOTICE.txt +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/auth/__init__.py +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/chat_message_normalizer/chat_message_normalizer_tokenizer.py +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/common/__init__.py +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit/embedding/__init__.py +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit.egg-info/dependency_links.txt +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/pyrit.egg-info/top_level.txt +0 -0
- {pyrit-0.2.0 → pyrit-0.3.0}/tests/test_chat_normalizer_tokenizer.py +0 -0
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
Copyright (c) Microsoft Corporation.
|
|
2
|
-
|
|
3
|
-
MIT License
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
-
in the Software without restriction, including without limitation the rights
|
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
-
furnished to do so, subject to the following conditions:
|
|
11
|
-
|
|
12
|
-
The above copyright notice and this permission notice shall be included in all
|
|
13
|
-
copies or substantial portions of the Software.
|
|
14
|
-
|
|
15
|
-
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
-
SOFTWARE.
|
|
1
|
+
Copyright (c) Microsoft Corporation.
|
|
2
|
+
|
|
3
|
+
MIT License
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -1,115 +1,113 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: pyrit
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary: The Python Risk Identification Tool for LLMs (PyRIT) is a library used to assess the robustness of LLMs
|
|
5
|
-
Author: dlmgary, amandajean119, microsiska, rdheekonda, rlundeen2, romanlutz, jbolor21, nina-msft
|
|
6
|
-
Author-email: Microsoft AI Red Team <airedteam@microsoft.com>
|
|
7
|
-
License: MIT
|
|
8
|
-
Keywords: llm,ai-safety,ai-security,ai-red-team,ai-red-teaming,ai-robustness,ai-robustness-testing,ai-risk-assessment
|
|
9
|
-
Classifier: Development Status :: 3 - Alpha
|
|
10
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
12
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
13
|
-
Requires-Python: <3.12,>=3.10
|
|
14
|
-
Description-Content-Type: text/markdown
|
|
15
|
-
License-File: LICENSE
|
|
16
|
-
License-File: NOTICE.txt
|
|
17
|
-
Requires-Dist:
|
|
18
|
-
Requires-Dist:
|
|
19
|
-
Requires-Dist:
|
|
20
|
-
Requires-Dist:
|
|
21
|
-
Requires-Dist: azure-
|
|
22
|
-
Requires-Dist: azure-
|
|
23
|
-
Requires-Dist: azure-
|
|
24
|
-
Requires-Dist: azure-
|
|
25
|
-
Requires-Dist:
|
|
26
|
-
Requires-Dist:
|
|
27
|
-
Requires-Dist: duckdb==0.
|
|
28
|
-
Requires-Dist:
|
|
29
|
-
Requires-Dist:
|
|
30
|
-
Requires-Dist:
|
|
31
|
-
Requires-Dist:
|
|
32
|
-
Requires-Dist:
|
|
33
|
-
Requires-Dist:
|
|
34
|
-
Requires-Dist:
|
|
35
|
-
Requires-Dist:
|
|
36
|
-
Requires-Dist:
|
|
37
|
-
Requires-Dist:
|
|
38
|
-
Requires-Dist: python-dotenv>=1.0.1
|
|
39
|
-
Requires-Dist: scikit-learn>=1.4.2
|
|
40
|
-
Requires-Dist:
|
|
41
|
-
Requires-Dist:
|
|
42
|
-
Requires-Dist:
|
|
43
|
-
Requires-Dist:
|
|
44
|
-
Requires-Dist:
|
|
45
|
-
Requires-Dist:
|
|
46
|
-
Requires-Dist:
|
|
47
|
-
|
|
48
|
-
Requires-Dist:
|
|
49
|
-
|
|
50
|
-
Requires-Dist:
|
|
51
|
-
Requires-Dist:
|
|
52
|
-
Requires-Dist:
|
|
53
|
-
Requires-Dist:
|
|
54
|
-
Requires-Dist:
|
|
55
|
-
Requires-Dist:
|
|
56
|
-
Requires-Dist:
|
|
57
|
-
Requires-Dist: pytest>=
|
|
58
|
-
Requires-Dist: pytest-
|
|
59
|
-
Requires-Dist:
|
|
60
|
-
Requires-Dist:
|
|
61
|
-
Requires-Dist: types-PyYAML>=6.0.12.9; extra == "dev"
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
Any use of third-party trademarks or logos are subject to those third-party's
|
|
115
|
-
policies.
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: pyrit
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: The Python Risk Identification Tool for LLMs (PyRIT) is a library used to assess the robustness of LLMs
|
|
5
|
+
Author: dlmgary, amandajean119, microsiska, rdheekonda, rlundeen2, romanlutz, jbolor21, nina-msft
|
|
6
|
+
Author-email: Microsoft AI Red Team <airedteam@microsoft.com>
|
|
7
|
+
License: MIT
|
|
8
|
+
Keywords: llm,ai-safety,ai-security,ai-red-team,ai-red-teaming,ai-robustness,ai-robustness-testing,ai-risk-assessment
|
|
9
|
+
Classifier: Development Status :: 3 - Alpha
|
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
13
|
+
Requires-Python: <3.12,>=3.10
|
|
14
|
+
Description-Content-Type: text/markdown
|
|
15
|
+
License-File: LICENSE
|
|
16
|
+
License-File: NOTICE.txt
|
|
17
|
+
Requires-Dist: appdirs>=1.4.0
|
|
18
|
+
Requires-Dist: art==6.1.0
|
|
19
|
+
Requires-Dist: azure-cognitiveservices-speech>=1.36.0
|
|
20
|
+
Requires-Dist: azure-core>=1.26.1
|
|
21
|
+
Requires-Dist: azure-identity>=1.12.0
|
|
22
|
+
Requires-Dist: azure-ai-contentsafety>=1.0.0
|
|
23
|
+
Requires-Dist: azure-ai-ml==1.13.0
|
|
24
|
+
Requires-Dist: azure-storage-blob>=12.19.0
|
|
25
|
+
Requires-Dist: confusables==1.2.0
|
|
26
|
+
Requires-Dist: duckdb==0.10.0
|
|
27
|
+
Requires-Dist: duckdb-engine==0.11.2
|
|
28
|
+
Requires-Dist: jsonpickle>=3.0.4
|
|
29
|
+
Requires-Dist: jupyter>=1.0.0
|
|
30
|
+
Requires-Dist: ipykernel>=6.29.4
|
|
31
|
+
Requires-Dist: logzero>=1.7.0
|
|
32
|
+
Requires-Dist: numpy>=1.26.4
|
|
33
|
+
Requires-Dist: onnxruntime>=1.14.1
|
|
34
|
+
Requires-Dist: onnx>=1.16.0
|
|
35
|
+
Requires-Dist: openai>=1.23.2
|
|
36
|
+
Requires-Dist: pydantic>=2.7.1
|
|
37
|
+
Requires-Dist: pyodbc>=5.1.0
|
|
38
|
+
Requires-Dist: python-dotenv>=1.0.1
|
|
39
|
+
Requires-Dist: scikit-learn>=1.4.2
|
|
40
|
+
Requires-Dist: SQLAlchemy==2.0.28
|
|
41
|
+
Requires-Dist: sqlalchemy-stubs==0.4
|
|
42
|
+
Requires-Dist: termcolor>=2.4.0
|
|
43
|
+
Requires-Dist: tenacity>=8.2.3
|
|
44
|
+
Requires-Dist: tokenizers>=0.19.1
|
|
45
|
+
Requires-Dist: transformers>=4.40.0
|
|
46
|
+
Requires-Dist: types-requests>=2.31.0.20240406
|
|
47
|
+
Provides-Extra: dev
|
|
48
|
+
Requires-Dist: black>=24.4.0; extra == "dev"
|
|
49
|
+
Requires-Dist: flake8>=7.0.0; extra == "dev"
|
|
50
|
+
Requires-Dist: flake8-copyright>=0.2.4; extra == "dev"
|
|
51
|
+
Requires-Dist: jupytext>=1.16.1; extra == "dev"
|
|
52
|
+
Requires-Dist: mypy>=1.9.0; extra == "dev"
|
|
53
|
+
Requires-Dist: mock-alchemy>=0.2.6; extra == "dev"
|
|
54
|
+
Requires-Dist: pillow>=10.3.0; extra == "dev"
|
|
55
|
+
Requires-Dist: pre-commit>=3.3.3; extra == "dev"
|
|
56
|
+
Requires-Dist: pytest>=7.3.1; extra == "dev"
|
|
57
|
+
Requires-Dist: pytest-asyncio>=0.23.5; extra == "dev"
|
|
58
|
+
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
|
|
59
|
+
Requires-Dist: respx>=0.20.2; extra == "dev"
|
|
60
|
+
Requires-Dist: semantic-kernel==0.9.4b1; extra == "dev"
|
|
61
|
+
Requires-Dist: types-PyYAML>=6.0.12.9; extra == "dev"
|
|
62
|
+
|
|
63
|
+
# Python Risk Identification Tool for generative AI (PyRIT)
|
|
64
|
+
|
|
65
|
+
The Python Risk Identification Tool for generative AI (PyRIT) is an open
|
|
66
|
+
access automation framework to empower security professionals and ML
|
|
67
|
+
engineers to red team foundation models and their applications.
|
|
68
|
+
|
|
69
|
+
## Introduction
|
|
70
|
+
|
|
71
|
+
PyRIT is a library developed by the AI Red Team for researchers and engineers
|
|
72
|
+
to help them assess the robustness of their LLM endpoints against different
|
|
73
|
+
harm categories such as fabrication/ungrounded content (e.g., hallucination),
|
|
74
|
+
misuse (e.g., bias), and prohibited content (e.g., harassment).
|
|
75
|
+
|
|
76
|
+
PyRIT automates AI Red Teaming tasks to allow operators to focus on more
|
|
77
|
+
complicated and time-consuming tasks and can also identify security harms such
|
|
78
|
+
as misuse (e.g., malware generation, jailbreaking), and privacy harms
|
|
79
|
+
(e.g., identity theft).
|
|
80
|
+
|
|
81
|
+
The goal is to allow researchers to have a baseline of how well their model
|
|
82
|
+
and entire inference pipeline is doing against different harm categories and
|
|
83
|
+
to be able to compare that baseline to future iterations of their model.
|
|
84
|
+
This allows them to have empirical data on how well their model is doing
|
|
85
|
+
today, and detect any degradation of performance based on future improvements.
|
|
86
|
+
|
|
87
|
+
Additionally, this tool allows researchers to iterate and improve their
|
|
88
|
+
mitigations against different harms.
|
|
89
|
+
For example, at Microsoft we are using this tool to iterate on different
|
|
90
|
+
versions of a product (and its metaprompt) so that we can more effectively
|
|
91
|
+
protect against prompt injection attacks.
|
|
92
|
+
|
|
93
|
+

|
|
94
|
+
|
|
95
|
+
## Where can I learn more?
|
|
96
|
+
|
|
97
|
+
Microsoft Learn has a
|
|
98
|
+
[dedicated page on AI Red Teaming](https://learn.microsoft.com/en-us/security/ai-red-team).
|
|
99
|
+
|
|
100
|
+
Check out our [docs](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/README.md) for more information
|
|
101
|
+
on how to [install PyRIT](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/setup/install_pyrit.md),
|
|
102
|
+
our [How to Guide](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/how_to_guide.ipynb),
|
|
103
|
+
and more, as well as our [demos](https://github.com/Azure/PyRIT/tree/releases/v0.3.0/doc/demo) folder.
|
|
104
|
+
|
|
105
|
+
## Trademarks
|
|
106
|
+
|
|
107
|
+
This project may contain trademarks or logos for projects, products, or services.
|
|
108
|
+
Authorized use of Microsoft trademarks or logos is subject to and must follow
|
|
109
|
+
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
|
|
110
|
+
Use of Microsoft trademarks or logos in modified versions of this project must
|
|
111
|
+
not cause confusion or imply Microsoft sponsorship.
|
|
112
|
+
Any use of third-party trademarks or logos are subject to those third-party's
|
|
113
|
+
policies.
|
|
@@ -1,51 +1,51 @@
|
|
|
1
|
-
# Python Risk Identification Tool for generative AI (PyRIT)
|
|
2
|
-
|
|
3
|
-
The Python Risk Identification Tool for generative AI (PyRIT) is an open
|
|
4
|
-
access automation framework to empower security professionals and ML
|
|
5
|
-
engineers to red team foundation models and their applications.
|
|
6
|
-
|
|
7
|
-
## Introduction
|
|
8
|
-
|
|
9
|
-
PyRIT is a library developed by the AI Red Team for researchers and engineers
|
|
10
|
-
to help them assess the robustness of their LLM endpoints against different
|
|
11
|
-
harm categories such as fabrication/ungrounded content (e.g., hallucination),
|
|
12
|
-
misuse (e.g., bias), and prohibited content (e.g., harassment).
|
|
13
|
-
|
|
14
|
-
PyRIT automates AI Red Teaming tasks to allow operators to focus on more
|
|
15
|
-
complicated and time-consuming tasks and can also identify security harms such
|
|
16
|
-
as misuse (e.g., malware generation, jailbreaking), and privacy harms
|
|
17
|
-
(e.g., identity theft).
|
|
18
|
-
|
|
19
|
-
The goal is to allow researchers to have a baseline of how well their model
|
|
20
|
-
and entire inference pipeline is doing against different harm categories and
|
|
21
|
-
to be able to compare that baseline to future iterations of their model.
|
|
22
|
-
This allows them to have empirical data on how well their model is doing
|
|
23
|
-
today, and detect any degradation of performance based on future improvements.
|
|
24
|
-
|
|
25
|
-
Additionally, this tool allows researchers to iterate and improve their
|
|
26
|
-
mitigations against different harms.
|
|
27
|
-
For example, at Microsoft we are using this tool to iterate on different
|
|
28
|
-
versions of a product (and its metaprompt) so that we can more effectively
|
|
29
|
-
protect against prompt injection attacks.
|
|
30
|
-
|
|
31
|
-
.
|
|
37
|
-
|
|
38
|
-
Check out our [docs](https://
|
|
39
|
-
on how to [install PyRIT](https://
|
|
40
|
-
our [How to Guide](https://
|
|
41
|
-
and more, as well as our [demos](https://
|
|
42
|
-
|
|
43
|
-
## Trademarks
|
|
44
|
-
|
|
45
|
-
This project may contain trademarks or logos for projects, products, or services.
|
|
46
|
-
Authorized use of Microsoft trademarks or logos is subject to and must follow
|
|
47
|
-
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
|
|
48
|
-
Use of Microsoft trademarks or logos in modified versions of this project must
|
|
49
|
-
not cause confusion or imply Microsoft sponsorship.
|
|
50
|
-
Any use of third-party trademarks or logos are subject to those third-party's
|
|
51
|
-
policies.
|
|
1
|
+
# Python Risk Identification Tool for generative AI (PyRIT)
|
|
2
|
+
|
|
3
|
+
The Python Risk Identification Tool for generative AI (PyRIT) is an open
|
|
4
|
+
access automation framework to empower security professionals and ML
|
|
5
|
+
engineers to red team foundation models and their applications.
|
|
6
|
+
|
|
7
|
+
## Introduction
|
|
8
|
+
|
|
9
|
+
PyRIT is a library developed by the AI Red Team for researchers and engineers
|
|
10
|
+
to help them assess the robustness of their LLM endpoints against different
|
|
11
|
+
harm categories such as fabrication/ungrounded content (e.g., hallucination),
|
|
12
|
+
misuse (e.g., bias), and prohibited content (e.g., harassment).
|
|
13
|
+
|
|
14
|
+
PyRIT automates AI Red Teaming tasks to allow operators to focus on more
|
|
15
|
+
complicated and time-consuming tasks and can also identify security harms such
|
|
16
|
+
as misuse (e.g., malware generation, jailbreaking), and privacy harms
|
|
17
|
+
(e.g., identity theft).
|
|
18
|
+
|
|
19
|
+
The goal is to allow researchers to have a baseline of how well their model
|
|
20
|
+
and entire inference pipeline is doing against different harm categories and
|
|
21
|
+
to be able to compare that baseline to future iterations of their model.
|
|
22
|
+
This allows them to have empirical data on how well their model is doing
|
|
23
|
+
today, and detect any degradation of performance based on future improvements.
|
|
24
|
+
|
|
25
|
+
Additionally, this tool allows researchers to iterate and improve their
|
|
26
|
+
mitigations against different harms.
|
|
27
|
+
For example, at Microsoft we are using this tool to iterate on different
|
|
28
|
+
versions of a product (and its metaprompt) so that we can more effectively
|
|
29
|
+
protect against prompt injection attacks.
|
|
30
|
+
|
|
31
|
+

|
|
32
|
+
|
|
33
|
+
## Where can I learn more?
|
|
34
|
+
|
|
35
|
+
Microsoft Learn has a
|
|
36
|
+
[dedicated page on AI Red Teaming](https://learn.microsoft.com/en-us/security/ai-red-team).
|
|
37
|
+
|
|
38
|
+
Check out our [docs](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/README.md) for more information
|
|
39
|
+
on how to [install PyRIT](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/setup/install_pyrit.md),
|
|
40
|
+
our [How to Guide](https://github.com/Azure/PyRIT/releases/v0.3.0/doc/how_to_guide.ipynb),
|
|
41
|
+
and more, as well as our [demos](https://github.com/Azure/PyRIT/tree/releases/v0.3.0/doc/demo) folder.
|
|
42
|
+
|
|
43
|
+
## Trademarks
|
|
44
|
+
|
|
45
|
+
This project may contain trademarks or logos for projects, products, or services.
|
|
46
|
+
Authorized use of Microsoft trademarks or logos is subject to and must follow
|
|
47
|
+
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
|
|
48
|
+
Use of Microsoft trademarks or logos in modified versions of this project must
|
|
49
|
+
not cause confusion or imply Microsoft sponsorship.
|
|
50
|
+
Any use of third-party trademarks or logos are subject to those third-party's
|
|
51
|
+
policies.
|