aiverify-moonshot 0.6.4__py3-none-any.whl → 0.7.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: aiverify-moonshot
3
- Version: 0.6.4
3
+ Version: 0.7.1
4
4
  Summary: AI Verify advances Gen AI testing with Project Moonshot.
5
5
  Project-URL: Repository, https://github.com/aiverify-foundation/moonshot
6
6
  Project-URL: Documentation, https://aiverify-foundation.github.io/moonshot/
@@ -48,7 +48,7 @@ Description-Content-Type: text/markdown
48
48
 
49
49
  ![Moonshot Logo](https://github.com/aiverify-foundation/moonshot/raw/main/misc/aiverify-moonshot-logo.png)
50
50
 
51
- **Version 0.6.4**
51
+ **Version 0.7.1**
52
52
 
53
53
  A simple and modular tool to evaluate any LLM-based AI systems.
54
54
 
@@ -59,16 +59,15 @@ A simple and modular tool to evaluate any LLM-based AI systems.
59
59
 
60
60
  ## 🎯 Motivation
61
61
 
62
- Developed by the [AI Verify Foundation](https://aiverifyfoundation.sg/), [Moonshot](https://aiverifyfoundation.sg/project-moonshot/) is one of the first tools to bring Benchmarking and Red-Teaming together to help AI developers, compliance teams and AI system owners <b>evaluate LLMs and LLM-based AI systems</b>.
62
+ Developed by the [AI Verify Foundation](https://aiverifyfoundation.sg/), [Moonshot](https://aiverifyfoundation.sg/project-moonshot/) is a tool to bring Benchmarking and Red-Teaming together to help AI developers, compliance teams evaluate LLM-based Apps and LLMs.
63
63
 
64
64
  </br>
65
65
 
66
66
  ## 🚀 Why Moonshot
67
67
 
68
- In the rapidly evolving landscape of Generative AI, ensuring safety, reliability, and performance of AI systems is paramount. Moonshot addresses this critical need by providing a unified platform for:
69
- - <b>Benchmark Tests:</b> Systematically test LLMs across various performance metrics, and critical trust & safety dimensions using a wide array of open-source benchmarks and domain-specific tests.
70
- - <b>Red Team Attacks:</b> Proactively identify vulnerabilities and potential misuse scenarios in your AI systems through streamlined adversarial prompting.
71
- - <b>Streamline MLOps:</b> Integrate evaluation seamlessly into your development pipeline with flexible APIs.
68
+ In the rapidly evolving landscape of Generative AI, ensuring safety, reliability, and performance of LLM applications is paramount. Moonshot addresses this critical need by providing a unified platform for:
69
+ - <b>Benchmark Tests:</b> Systematically test LLM Apps or LLMs across critical trust & safety risks using a wide array of open-source benchmark dataset and metrics, including guided workflows to implement <b>IMDA's Starter Kit for LLM-based App Testing</b>.
70
+ - <b>Red Team Attacks:</b> Proactively identify vulnerabilities and potential misuse scenarios in your LLM applications through streamlined adversarial prompting.
72
71
 
73
72
  </br>
74
73
 
@@ -79,20 +78,20 @@ In the rapidly evolving landscape of Generative AI, ensuring safety, reliability
79
78
  - [View list of available datasets available](https://aiverify-foundation.github.io/moonshot/resources/datasets/)
80
79
  - Test for <b>Performance</b> (e.g., accuracy, BLEU)
81
80
  - Ensure <b>Trust & Safety</b> e.g., bias, toxicity, hallucination)
82
- - Utilize pre-built Cookbooks of tests or easily create your custom evaluations. [View available pre-built Cookbooks](https://aiverify-foundation.github.io/moonshot/resources/cookbooks/)
81
+ - Utilize built-in workflow to implement IMDA's Starter Kit for LLM-based App Testing. [View available pre-built Cookbooks](https://aiverify-foundation.github.io/moonshot/resources/cookbooks/)
83
82
  - <b>Powerful Red-Teaming:</b>
84
83
  - [View list of available attack modules](https://aiverify-foundation.github.io/moonshot/resources/attack_modules/)
85
84
  - Simplify adversarial prompt generation using algorithmic strategies or generative LLM to uncover potential misuse.
86
85
  - Leverage prompt templates, context strategies, and automated attack modules.
87
- - <b>Customizable Recipes:</b> Define your evaluation logic with custom datasets (input-target pairs), metrics, optional prompt templates, evaluation metric, and grading scales. [View available pre-built Recipes](https://aiverify-foundation.github.io/moonshot/resources/recipes/)
88
- - <b>Insightful Reporting:</b> Generate comprehensive HTML reports with interactive charts for clear visualization of test results, and detailed raw JSON results for deeper programmatic analysis.
89
- - <b>Extensible & Modular:</b> Designed for easy extension and integration with new LLMs, benchmarks, and attack techniques.
86
+ - <b>Customizable Recipes:</b> Build your own benchmark tests with custom datasets (input-target pairs), prompt templates (optional), evaluation metric, and grading scales. [View available pre-built Recipes](https://aiverify-foundation.github.io/moonshot/resources/recipes/)
87
+ - <b>Insightful Reporting:</b> Use our HTML reports with interactive charts for clear visualization of test results, and download detailed raw JSON results for deeper programmatic analysis.
88
+ - <b>Extensible & Modular:</b> Designed for easy extension and integration with new LLM applications, benchmarks, and attack techniques.
90
89
 
91
90
  </br>
92
91
 
93
92
  # Getting Started
94
93
 
95
- In this Beta version, Moonshot can be used through several interfaces:
94
+ Moonshot can be used through several interfaces:
96
95
  - User-friendly Web UI - [Web UI User Guide](https://aiverify-foundation.github.io/moonshot/user_guide/web_ui/web_ui_guide/)
97
96
  - Interactive Command Line Interface - [CLI User Guide](https://aiverify-foundation.github.io/moonshot/user_guide/cli/connecting_endpoints/)
98
97
  - Seamless Integration into your MLOps workflow via Moonshot Library APIs or Moonshot Web APIs - [Notebook Examples](https://github.com/aiverify-foundation/moonshot/tree/main/examples/jupyter-notebook), [Web API Docs](https://aiverify-foundation.github.io/moonshot/api_reference/web_api_swagger/)
@@ -34,7 +34,7 @@ moonshot/integrations/cli/utils/process_data.py,sha256=QVL5vp2_8ZgGicmCAdeYEHkeb
34
34
  moonshot/integrations/web_api/.env.dev,sha256=0z5_Ut8rF-UqFZtgjkH2qoqORhD5_nSs2w_OeX2SteI,182
35
35
  moonshot/integrations/web_api/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
36
36
  moonshot/integrations/web_api/__main__.py,sha256=MdnLi_ZF-olAAEJwTPU1iGYFYwo-fNWNT2qfchkH3y4,2050
37
- moonshot/integrations/web_api/app.py,sha256=nbAAnVyy1BEqlctGeVUjVYCdR3yD0mfyGZmwBM2zZec,3651
37
+ moonshot/integrations/web_api/app.py,sha256=ppmlbvde_68wbJ3oXuQvAdDAYDyhi0yy4vLe9H-29YQ,3651
38
38
  moonshot/integrations/web_api/container.py,sha256=DVkJG_qm7ItcG6tgMYOqIj07wpKhPWOOfy6-bEv72y4,5915
39
39
  moonshot/integrations/web_api/logging_conf.py,sha256=t3EGRV6tZhV732KXe8_Tiy0fiwVAWxZX5Tt8VTgrrfg,3388
40
40
  moonshot/integrations/web_api/log/.gitkeep,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
@@ -172,9 +172,9 @@ moonshot/src/utils/import_modules.py,sha256=T9zTN59PFnvY2rjyWhSV9KSIAHxWV1pyBemF
172
172
  moonshot/src/utils/log.py,sha256=YNgD7Eh2OT36XlmVBKCGUTAh9TRp4Akfe4kDdvHASgs,2502
173
173
  moonshot/src/utils/pagination.py,sha256=5seymyRoqyENIhKllAatr1T91kMCGFslcvRnJHyMSvc,814
174
174
  moonshot/src/utils/timeit.py,sha256=TvuF0w8KWhp0oZFY0cUU3UY0xlGKjchb0OkfYfgVTlc,866
175
- aiverify_moonshot-0.6.4.dist-info/METADATA,sha256=UKH5vf6h00IJVbN3TyjUO-xY_M2Chwqcut_ZfFkEVv4,11273
176
- aiverify_moonshot-0.6.4.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
177
- aiverify_moonshot-0.6.4.dist-info/licenses/AUTHORS.md,sha256=mmAbe3i3sT8JZHJMBhxp3i1xRehV0g7WB4T_eyIBuBs,59
178
- aiverify_moonshot-0.6.4.dist-info/licenses/LICENSE.md,sha256=53izDRmJZZCjpYGfyLqlxnGQN-aNWBxasuzuMXC5Ias,11347
179
- aiverify_moonshot-0.6.4.dist-info/licenses/NOTICES.md,sha256=3d1WLa7kMtzh79BXxXr-keSlvHo8Llb-5EkGgtRiMS0,51537
180
- aiverify_moonshot-0.6.4.dist-info/RECORD,,
175
+ aiverify_moonshot-0.7.1.dist-info/METADATA,sha256=8bfZclyjyzzKl-V0K6WoVydD-BWLzF2rBux8qR0-nmk,11170
176
+ aiverify_moonshot-0.7.1.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
177
+ aiverify_moonshot-0.7.1.dist-info/licenses/AUTHORS.md,sha256=mmAbe3i3sT8JZHJMBhxp3i1xRehV0g7WB4T_eyIBuBs,59
178
+ aiverify_moonshot-0.7.1.dist-info/licenses/LICENSE.md,sha256=53izDRmJZZCjpYGfyLqlxnGQN-aNWBxasuzuMXC5Ias,11347
179
+ aiverify_moonshot-0.7.1.dist-info/licenses/NOTICES.md,sha256=3d1WLa7kMtzh79BXxXr-keSlvHo8Llb-5EkGgtRiMS0,51537
180
+ aiverify_moonshot-0.7.1.dist-info/RECORD,,
@@ -71,7 +71,7 @@ def create_app(cfg: providers.Configuration) -> CustomFastAPI:
71
71
  }
72
72
 
73
73
  app: CustomFastAPI = CustomFastAPI(
74
- title="Project Moonshot", version="0.6.4", **app_kwargs
74
+ title="Project Moonshot", version="0.7.1", **app_kwargs
75
75
  )
76
76
 
77
77
  if cfg.cors.enabled():