@artemiskit/reports 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +104 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
package/README.md
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# @artemiskit/reports
|
|
2
|
+
|
|
3
|
+
HTML report generation for ArtemisKit LLM evaluation toolkit.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install @artemiskit/reports
|
|
9
|
+
# or
|
|
10
|
+
bun add @artemiskit/reports
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Overview
|
|
14
|
+
|
|
15
|
+
This package generates interactive HTML reports from ArtemisKit test runs:
|
|
16
|
+
|
|
17
|
+
- **Run Reports** - Scenario evaluation results with pass/fail status
|
|
18
|
+
- **Red Team Reports** - Security test results with vulnerability scoring
|
|
19
|
+
- **Stress Test Reports** - Load test metrics with latency percentiles
|
|
20
|
+
|
|
21
|
+
## Usage
|
|
22
|
+
|
|
23
|
+
Most users should use the [`@artemiskit/cli`](https://www.npmjs.com/package/@artemiskit/cli) which automatically generates reports. This package is for programmatic report generation.
|
|
24
|
+
|
|
25
|
+
```typescript
|
|
26
|
+
import { generateHTMLReport } from '@artemiskit/reports';
|
|
27
|
+
import type { RunManifest } from '@artemiskit/core';
|
|
28
|
+
|
|
29
|
+
// Generate report from a manifest
|
|
30
|
+
const manifest: RunManifest = { /* ... */ };
|
|
31
|
+
const html = await generateHTMLReport(manifest);
|
|
32
|
+
|
|
33
|
+
// Write to file
|
|
34
|
+
await writeFile('report.html', html);
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Report Types
|
|
38
|
+
|
|
39
|
+
### Run Reports
|
|
40
|
+
|
|
41
|
+
Generated from scenario evaluation results:
|
|
42
|
+
|
|
43
|
+
```typescript
|
|
44
|
+
import { generateHTMLReport } from '@artemiskit/reports';
|
|
45
|
+
|
|
46
|
+
const html = await generateHTMLReport(runManifest);
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Features:
|
|
50
|
+
- Pass/fail status for each test case
|
|
51
|
+
- Latency metrics
|
|
52
|
+
- Token usage
|
|
53
|
+
- Redaction indicators (when enabled)
|
|
54
|
+
- Expandable prompt/response details
|
|
55
|
+
|
|
56
|
+
### Red Team Reports
|
|
57
|
+
|
|
58
|
+
Generated from security test results:
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
import { generateRedTeamHTMLReport } from '@artemiskit/reports';
|
|
62
|
+
|
|
63
|
+
const html = await generateRedTeamHTMLReport(redteamManifest);
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Features:
|
|
67
|
+
- Vulnerability categories (injection, jailbreak, extraction, etc.)
|
|
68
|
+
- Severity ratings
|
|
69
|
+
- Defense success rate
|
|
70
|
+
- Attack mutation details
|
|
71
|
+
|
|
72
|
+
### Stress Test Reports
|
|
73
|
+
|
|
74
|
+
Generated from load test results:
|
|
75
|
+
|
|
76
|
+
```typescript
|
|
77
|
+
import { generateStressHTMLReport } from '@artemiskit/reports';
|
|
78
|
+
|
|
79
|
+
const html = await generateStressHTMLReport(stressManifest);
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Features:
|
|
83
|
+
- Requests per second
|
|
84
|
+
- Latency percentiles (p50, p95, p99)
|
|
85
|
+
- Success/error rates
|
|
86
|
+
- Concurrent request metrics
|
|
87
|
+
|
|
88
|
+
## Regenerating Reports
|
|
89
|
+
|
|
90
|
+
Reports can be regenerated from saved manifests:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
artemiskit report artemis-runs/my-project/abc123.json
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Related Packages
|
|
97
|
+
|
|
98
|
+
- [`@artemiskit/cli`](https://www.npmjs.com/package/@artemiskit/cli) - Command-line interface
|
|
99
|
+
- [`@artemiskit/core`](https://www.npmjs.com/package/@artemiskit/core) - Core runtime and evaluators
|
|
100
|
+
- [`@artemiskit/redteam`](https://www.npmjs.com/package/@artemiskit/redteam) - Security testing
|
|
101
|
+
|
|
102
|
+
## License
|
|
103
|
+
|
|
104
|
+
Apache-2.0
|