evalvault 1.62.1__py3-none-any.whl → 1.63.0__py3-none-any.whl

{evalvault-1.62.1.dist-info → evalvault-1.63.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: evalvault
-Version: 1.62.1
+Version: 1.63.0
 Summary: RAG evaluation system using Ragas with Phoenix/Langfuse tracing
 Project-URL: Homepage, https://github.com/ntts9990/EvalVault
 Project-URL: Documentation, https://github.com/ntts9990/EvalVault#readme
@@ -37,6 +37,7 @@ Requires-Dist: openpyxl
 Requires-Dist: pandas
 Requires-Dist: pydantic
 Requires-Dist: pydantic-settings
+Requires-Dist: pypdf>=4.3.0
 Requires-Dist: python-multipart
 Requires-Dist: ragas==0.4.2
 Requires-Dist: rich
@@ -174,11 +175,229 @@ uv run evalvault run --mode simple tests/fixtures/e2e/insurance_qa_korean.json \
   --auto-analyze
 ```
 
-- 결과는 `--db`에 저장되어 `history`, Web UI, 비교 분석에서 재사용됩니다.
+- 결과는 기본 DB(`data/db/evalvault.db`)에 저장되어 `history`, Web UI, 비교 분석에서 재사용됩니다.
+- `--db`를 생략해도 기본 경로로 저장되며, 모든 데이터가 자동으로 엑셀로 내보내집니다.
 - `--auto-analyze`는 요약 리포트 + 모듈별 아티팩트를 함께 생성합니다.
 
 ---
 
+## 프롬프트 오버라이드 (RAGAS / 시스템)
+
+**에디터 관점**: 기본 동작은 유지하고 필요한 항목만 YAML/파일로 덮어씁니다.
+**개발자 관점**: CLI 옵션 또는 Web API 필드로 주입합니다.
+
+### CLI
+- RAGAS 메트릭별 오버라이드: `--ragas-prompts`
+- 시스템 프롬프트 적용: `--system-prompt` 또는 `--system-prompt-file`
+
+```bash
+uv run evalvault run --mode full tests/fixtures/e2e/insurance_qa_korean.json \
+  --metrics faithfulness,answer_relevancy \
+  --ragas-prompts config/ragas_prompts.yaml
+
+uv run evalvault run --mode full tests/fixtures/e2e/insurance_qa_korean.json \
+  --system-prompt-file prompts/system.txt
+```
+
+`config/ragas_prompts.yaml` 예시:
+```yaml
+faithfulness: |
+  # custom prompt...
+answer_relevancy: |
+  # custom prompt...
+```
+
+### Web UI / API
+- `EvalRequest` 필드:
+  - `system_prompt`, `system_prompt_name`
+  - `ragas_prompt_overrides` (메트릭명 → 프롬프트 문자열)
+  - `prompt_set_name`, `prompt_set_description`
+
+---
+
+## 프롬프트 후보 추천 (`evalvault prompts suggest`)
+
+특정 `run_id`의 프롬프트 스냅샷을 기준으로, **자동/수동 후보 프롬프트**를 모은 뒤 **holdout 분리 데이터**에서 Ragas 메트릭을 평가하고, **가중치 합산 점수**로 Top 후보를 추천합니다.
+
+- 필수 전제: `run_id`가 `--db`에 저장되어 있고, 해당 run에 **프롬프트 스냅샷**이 연결되어 있어야 합니다 (`evalvault run` 실행 시 `--db` 사용).
+- 자동 후보: 기본 프롬프트를 바탕으로 템플릿 기반 변형을 생성합니다. (`--candidates`, `--auto/--no-auto`)
+- 수동 후보: `--prompt`(반복 가능), `--prompt-file`(반복 가능)로 후보를 추가합니다. (`--no-auto` 사용 시 수동 후보는 필수)
+- holdout 분리: `--holdout-ratio`(기본 0.2)로 dev/holdout을 나누고, **holdout 쪽 점수로 랭킹**을 계산합니다. 재현이 필요하면 `--seed`를 지정하세요.
+- 가중치: `--weights faithfulness=0.7,answer_relevancy=0.3` 형태로 입력하며, 내부에서 합이 1이 되도록 정규화합니다. 미지정 시 메트릭 균등 가중치가 적용됩니다.
+
+### 사용 예시
+
+```bash
+# 기본 사용 (자동 후보 + 수동 후보 파일)
+uv run evalvault prompts suggest <RUN_ID> --db data/db/evalvault.db \
+  --role system \
+  --metrics faithfulness,answer_relevancy \
+  --weights faithfulness=0.7,answer_relevancy=0.3 \
+  --candidates 5 \
+  --prompt-file prompts/candidates.txt
+
+# 요약 평가(다중 메트릭) + 가중치
+uv run evalvault prompts suggest <RUN_ID> --db data/db/evalvault.db \
+  --metrics summary_score,summary_faithfulness,entity_preservation \
+  --weights summary_score=0.5,summary_faithfulness=0.3,entity_preservation=0.2 \
+  --candidates 3
+
+# 샘플링 2개 중 index 선택
+uv run evalvault prompts suggest <RUN_ID> --db data/db/evalvault.db \
+  --generation-n 2 \
+  --selection-policy index \
+  --selection-index 1
+```
+
+- `--prompt-file`은 **한 줄당 후보 프롬프트 1개**를 읽습니다(빈 줄 제외).
+
+### 주요 옵션 요약
+- `--role`: 개선 대상 프롬프트 role (기본 system)
+- `--metrics`: 평가 메트릭 목록 (기본 run에서 사용한 메트릭)
+- `--weights`: 메트릭 가중치 (합이 1이 되도록 정규화)
+- `--candidates`: 자동 후보 수 (기본 5)
+- `--auto/--no-auto`: 자동 후보 생성 on/off
+- `--holdout-ratio`: dev/holdout 분리 비율 (기본 0.2)
+- `--seed`: 분리/샘플 재현성
+- `--generation-n`: 후보당 샘플 수
+- `--selection-policy`: 샘플 선택 정책 (`best`|`index`)
+- `--selection-index`: `selection-policy=index` 시 선택할 샘플 인덱스
+
+### 출력(기본 경로)
+
+- 요약 JSON: `reports/analysis/prompt_suggestions_<RUN_ID>.json`
+- 보고서(Markdown): `reports/analysis/prompt_suggestions_<RUN_ID>.md`
+- 아티팩트 디렉터리: `reports/analysis/artifacts/prompt_suggestions_<RUN_ID>/`
+  - 후보 목록: `reports/analysis/artifacts/prompt_suggestions_<RUN_ID>/candidates.json`
+  - 후보 점수/샘플 점수: `reports/analysis/artifacts/prompt_suggestions_<RUN_ID>/scores.json`
+  - 최종 랭킹: `reports/analysis/artifacts/prompt_suggestions_<RUN_ID>/ranking.json`
+  - 인덱스: `reports/analysis/artifacts/prompt_suggestions_<RUN_ID>/index.json`
+
+경로를 바꾸려면 `--analysis-dir` 또는 `--output`/`--report`를 사용합니다. 설계 배경은 `docs/guides/prompt_suggestions_design.md`를 참고하세요.
+
+### FAQ
+- Q. "프롬프트 스냅샷이 없습니다" 오류가 납니다.
+  - A. 해당 run이 `--db`로 저장되었는지 확인하고, `evalvault run` 실행 시 `--db`를 지정하세요.
+- Q. 자동 후보를 끄면 어떻게 되나요?
+  - A. `--no-auto` 사용 시 `--prompt` 또는 `--prompt-file`로 수동 후보를 반드시 넣어야 합니다.
+- Q. 점수는 어떤 기준인가요?
+  - A. holdout 데이터에서 Ragas 메트릭을 평가하고, `--weights` 가중치로 합산한 점수입니다.
+
+---
+
+## 엑셀 내보내기 (자동)
+
+**에디터 관점**: DB 저장과 동시에 Excel이 자동 생성됩니다.
+**개발자 관점**: 저장 로직에서 `export_run_to_excel`이 자동 호출됩니다.
+
+- 기본 DB 경로: `data/db/evalvault.db`
+- 엑셀 경로: `data/db/evalvault_run_<RUN_ID>.xlsx`
+
+**시트 구성(요약 → 상세)**
+- `Summary`, `Run`, `TestCases`, `MetricScores`, `MetricsSummary`
+- `RunPromptSets`, `PromptSets`, `PromptSetItems`, `Prompts`
+- `Feedback`, `ClusterMaps`, `StageEvents`, `StageMetrics`
+- `AnalysisReports`, `PipelineResults`
+- 시트별 컬럼 설명: `docs/guides/EVALVAULT_RUN_EXCEL_SHEETS.md`
+
+---
+
+## 외부 시스템 로그 연동 (의도분석/리트리브/리랭킹 등)
+
+**에디터 관점**: 표준 포맷(OTel/JSON/JSONL)으로 붙일 수 있어야 합니다.
+**개발자 관점**: OpenTelemetry + OpenInference 또는 Stage Events로 연결합니다.
+
+### 1) Open RAG Trace (권장)
+- OpenTelemetry + OpenInference 기반 표준 스키마
+- 스펙: `docs/architecture/open-rag-trace-spec.md`
+- 연동 규격: `docs/guides/EXTERNAL_TRACE_API_SPEC.md`
+- 샘플: `docs/guides/OPEN_RAG_TRACE_SAMPLES.md`
+
+**OTLP HTTP 전송(권장)**
+- 엔드포인트: `http://<host>:6006/v1/traces`
+- Collector 사용 시: `http://<collector-host>:4318/v1/traces`
+
+**OpenInference 필수 키(요약)**
+- `rag.module`, `spec.version`
+- 권장: `input.value`, `output.value`, `llm.model_name`, `retrieval.documents_json`
+
+### 2) EvalVault 직접 Ingest (Draft)
+- `POST /api/v1/ingest/otel-traces` (OTLP JSON)
+- `POST /api/v1/ingest/stage-events` (JSONL)
+- 예시: `docs/templates/otel_openinference_trace_example.json`
+
+**OTLP JSON 예시(요약)**
+```json
+{
+  "resourceSpans": [
+    {
+      "resource": {
+        "attributes": [
+          { "key": "service.name", "value": { "stringValue": "rag-service" } }
+        ]
+      },
+      "scopeSpans": [
+        {
+          "spans": [
+            {
+              "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
+              "spanId": "00f067aa0ba902b7",
+              "name": "retrieve",
+              "startTimeUnixNano": 1730000000000000000,
+              "endTimeUnixNano": 1730000000500000000,
+              "attributes": [
+                { "key": "rag.module", "value": { "stringValue": "retrieve" } },
+                { "key": "spec.version", "value": { "stringValue": "0.1" } },
+                { "key": "input.value", "value": { "stringValue": "보험금 지급 조건" } }
+              ]
+            }
+          ]
+        }
+      ]
+    }
+  ]
+}
+```
+
+**응답 예시(요약)**
+```json
+{
+  "status": "ok",
+  "ingested": 12,
+  "trace_ids": ["4bf92f3577b34da6a3ce929d0e0e4736"]
+}
+```
+
+**HTTP 상태 코드(요약)**
+- `200 OK`: 정상 수집
+- `400 Bad Request`: JSON/JSONL 파싱 실패
+- `422 Unprocessable Entity`: 필수 필드 누락/스키마 불일치
+- `500 Internal Server Error`: 저장/파이프라인 내부 오류
+
+### 3) Stage Events / Metrics 적재
+- 외부 파이프라인 로그를 JSON/JSONL로 저장 → DB ingest
+
+```bash
+uv run evalvault stage ingest path/to/stage_events.jsonl --db data/db/evalvault.db
+uv run evalvault stage summary <RUN_ID> --db data/db/evalvault.db
+```
+
+**Stage Event JSONL 예시(요약)**
+```jsonl
+{"run_id":"run_20260103_001","stage_id":"stg_sys_01","stage_type":"system_prompt","stage_name":"system_prompt_v1","duration_ms":18,"attributes":{"prompt_id":"sys-01"}}
+{"run_id":"run_20260103_001","stage_id":"stg_input_01","parent_stage_id":"stg_sys_01","stage_type":"input","stage_name":"user_query","duration_ms":6,"attributes":{"query":"보험금 지급 조건","language":"ko"}}
+```
+
+- Stage Event에는 **의도분석/리트리브/리랭킹**의 입력/출력/파라미터/결과를 넣습니다.
+- `--stage-store` 사용 시 EvalVault 내부 실행 로그도 자동 저장됩니다.
+
+### 4) 분석 전환 규칙(요약)
+- **RAGAS 형식 데이터셋**이면 `evalvault run` 기반 평가/분석
+- **OTel/OpenInference 트레이스**는 Phoenix로 트레이싱 연결
+- **비정형 로그(Stage Event)**는 `stage ingest` → `stage summary` → 분석 모듈로 전환
+
+---
+
 ## Web UI (FastAPI + React)
 
 ```bash
@@ -193,6 +412,11 @@ npm run dev
 
 브라우저에서 `http://localhost:5173` 접속 후, Evaluation Studio에서 실행/히스토리/리포트를 확인합니다.
 
+- LLM 보고서 언어: `/api/v1/runs/{run_id}/report?language=en` (기본 ko)
+  - 상세: `docs/guides/USER_GUIDE.md#보고서-언어-옵션`
+- 피드백 집계: 동일 `rater_id` + `test_case_id` 기준 최신 값만 집계, 취소 시 집계 제외
+  - 상세: `docs/guides/USER_GUIDE.md#피드백-집계-규칙`
+
 ---
 
 ## 산출물(Artifacts) 경로
@@ -256,12 +480,12 @@ npm run dev
 
 ### 2) 한국어/비영어권 대응 (프롬프트 언어 정렬)
 - **한국어 데이터셋 자동 감지** 후 `answer_relevancy`, `factual_correctness`에 한국어 프롬프트를 기본 적용합니다. (`src/evalvault/domain/services/evaluator.py`)
+- **요약/후보 평가 프롬프트 기본 한국어**: 요약 충실도 판정, 프롬프트 후보 평가, 지식그래프 관계 보강 프롬프트는 기본 `ko`로 동작합니다.
+  - 영어가 필요하면 API/SDK에서 `language="en"` 또는 `prompt_language="en"`을 지정하세요.
 - **사용자 프롬프트 오버라이드 지원**: 필요 시 YAML로 메트릭별 프롬프트를 덮어쓸 수 있습니다. (`src/evalvault/domain/services/ragas_prompt_overrides.py`)
 - **외부 근거(비영어권 이슈)**:
   - https://github.com/explodinggradients/ragas/issues/1829
   - https://github.com/explodinggradients/ragas/issues/402
-- **공식 문서(언어 이슈 직접 언급)**:
-  - https://docs.ragas.io/en/stable/howtos/customizations/metrics/_metrics_language_adaptation/
 
 **이유**: 질문 생성/판정 프롬프트가 영어에 고정될 경우, 비영어 입력에서 언어 불일치로 점수 왜곡이 발생할 수 있으므로 이를 최소화합니다.
 

{evalvault-1.62.1.dist-info → evalvault-1.63.0.dist-info}/RECORD

@@ -1,11 +1,11 @@
 evalvault/__init__.py,sha256=mOxNqokIN9NNmW_VhViOPq3yqjs2GvpYKZgjiR6jRDc,1279
-evalvault/debug_ragas.py,sha256=f-UQxxoTfR5IZ6JyghW78tfGEJQBNUTbdhYJJ52yVjc,4933
-evalvault/debug_ragas_real.py,sha256=BpZagb8f2WOKcG-Zy69FUduL8amUy7Xid-BlDUl5esU,3212
+evalvault/debug_ragas.py,sha256=7Sur2XqKrWLq6kldULiRITwwhVzHpIUcPl_yP2EuhEY,5312
+evalvault/debug_ragas_real.py,sha256=Jw-4olZ6Yt_VymVFNuU9wYhfj-vOcagMmnwIa7U87HU,3518
 evalvault/mkdocs_helpers.py,sha256=1AKVQ1W2_VO4qclhfyefyU9Dz1Hzkh1DWDwsFMe24jc,331
 evalvault/adapters/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 evalvault/adapters/inbound/__init__.py,sha256=SG1svel1PwqetnqVpKFLSv612_WwGwLTbFpYgwk6FMw,166
 evalvault/adapters/inbound/api/__init__.py,sha256=LeVVttCA3tLKoHA2PO4z3y8VkfVcf3Bq8CZSzo91lf4,34
-evalvault/adapters/inbound/api/adapter.py,sha256=_giGdt-grmd6bkWMhRb3KdloxI_2jUMknProC76KqWY,61140
+evalvault/adapters/inbound/api/adapter.py,sha256=tYkJciUUFOK80QcSwzrqkXP1G4qUFItFV7uBYbjBGqU,68473
 evalvault/adapters/inbound/api/main.py,sha256=KdlAxKn0QfGI3UuoTrBDBbUs2xCvP8lnWOY1ce3svcU,2619
 evalvault/adapters/inbound/api/routers/__init__.py,sha256=q07_YF9TnBl68bqcRCvhPU4-zRTyvmPoHVehwO6W7QM,19
 evalvault/adapters/inbound/api/routers/benchmark.py,sha256=yevntbZcNtMvbVODsITUBgR1Ka4pdFQrXBJJ4K4Jyr4,4477
@@ -13,7 +13,7 @@ evalvault/adapters/inbound/api/routers/config.py,sha256=CN-FH2cn0Ive-BD3WacWY6PF
 evalvault/adapters/inbound/api/routers/domain.py,sha256=RsR7GIFMjccDN7vpG1uDyk9n1DnCTH18JDGAX7o4Qqc,3648
 evalvault/adapters/inbound/api/routers/knowledge.py,sha256=7mgyoUM1PepFb4X8_Ntn0vd7ZZYcNbM3_9nyD10g4Aw,5307
 evalvault/adapters/inbound/api/routers/pipeline.py,sha256=8UgQzNFHcuqS61s69mOrPee4OMwfxVdvRWHJ2_qYBF0,17175
-evalvault/adapters/inbound/api/routers/runs.py,sha256=KyIar-5RJemO7i3dvRLM1IeKWVF57tZXrrixKpGOg7M,38029
+evalvault/adapters/inbound/api/routers/runs.py,sha256=rydOvwWk24QIYafu3XYS3oL_VVCE_jHDmjADhA19T1s,40059
 evalvault/adapters/inbound/cli/__init__.py,sha256=a42flC5NK-VfbdbBrE49IrUL5zAyKdXZYJVM6E3NTE0,675
 evalvault/adapters/inbound/cli/app.py,sha256=ytNgHRg9ZTAl33AkB1wIL8RKfQ_Cf8fsy0gSsLTs7Ew,1603
 evalvault/adapters/inbound/cli/commands/__init__.py,sha256=cNPPhsudTQWdlh_OJm9mU8LGBnJLGMswJBcIV9MAlkI,3530
@@ -32,12 +32,12 @@ evalvault/adapters/inbound/cli/commands/history.py,sha256=P8rK_nRJrmtG3y9obq3OSY
 evalvault/adapters/inbound/cli/commands/init.py,sha256=7q86fUeBVA08fU_N0lAV6Lakxirq4val2jIyALlDy3E,8822
 evalvault/adapters/inbound/cli/commands/kg.py,sha256=ycV9Xj6SUUJLTyTfLZcjXDVLcZqwo7Gw878ZhZAeDoc,19155
 evalvault/adapters/inbound/cli/commands/langfuse.py,sha256=aExhZ5WYT0FzJI4v1sF-a1jqy9b1BF46_HBtfiQjVGI,4085
-evalvault/adapters/inbound/cli/commands/method.py,sha256=K1UacoKwV9w8sLeQK8qHyTuZqFZrlcj6yS_y2izfRlo,18853
+evalvault/adapters/inbound/cli/commands/method.py,sha256=OWdoofhvsDJchgNKnGGjXfIsZ-IHKZEo6RlmTsZRRYM,19124
 evalvault/adapters/inbound/cli/commands/phoenix.py,sha256=LQi3KTLq1ybjjBuz92oQ6lYyBS3mHrCHk0qe-7bqB4U,15611
 evalvault/adapters/inbound/cli/commands/pipeline.py,sha256=NeqWLzO9kRDuZd0pHAIHglP3F7VzoNOU4JI0QcSZ120,7788
-evalvault/adapters/inbound/cli/commands/prompts.py,sha256=6UwQtKJf3JYhcNI4tQqjjsL-sp_cmu2VV7gETkCcmkk,5490
-evalvault/adapters/inbound/cli/commands/run.py,sha256=qC1u8zTJBFEDxRYWZQ6IvIcX5JCyd9vLpP7vGkce8zQ,115493
-evalvault/adapters/inbound/cli/commands/run_helpers.py,sha256=50nYzf4DUniJd7fQgT2cyh_FWVTWZzW0UMXCg-EHBuY,39764
+evalvault/adapters/inbound/cli/commands/prompts.py,sha256=lddde5VbjYaqN_9gHPLNu6DWpg5fE-KqZzjN-XYwvJw,27153
+evalvault/adapters/inbound/cli/commands/run.py,sha256=5rWCh8dTVqRgoiKu2Kd_53PxeIh0GRIkULl3GSpoSiU,117412
+evalvault/adapters/inbound/cli/commands/run_helpers.py,sha256=cc6oZHJSBJM9cxr928zq3sGrIh73u2vD0z2j9IzbPo4,40236
 evalvault/adapters/inbound/cli/commands/stage.py,sha256=oRC9c5CysLX90Iy5Ba1pc_00DaOBS78lcBvzkbdrGRM,17123
 evalvault/adapters/inbound/cli/utils/__init__.py,sha256=QPNKneZS-Z-tTnYYxtgJXgcJWY6puUlRQcKrn7Mlv1M,685
 evalvault/adapters/inbound/cli/utils/analysis_io.py,sha256=RHkKEq4e-PtbtRDlXAJWU80RYHNPw-O5V9_GujdaGfc,13393
@@ -50,7 +50,7 @@ evalvault/adapters/inbound/cli/utils/progress.py,sha256=2kFKKZ4AWNvo9IcSLhu8qbsl
 evalvault/adapters/inbound/cli/utils/validators.py,sha256=OnjNvi_iLtxjWBIrqtbGWAmx2d4zNm_-qi0Ce-3KHZk,1436
 evalvault/adapters/inbound/mcp/__init__.py,sha256=kctJsmaP4fY94T3WCOhgANk3TCLdfb24UFDsV82RCDU,1135
 evalvault/adapters/inbound/mcp/schemas.py,sha256=KUKm4gEc-UDyF8sUbyzAnAIzyZ6DcXsaCEIVR3oESNQ,4469
-evalvault/adapters/inbound/mcp/tools.py,sha256=YkmQexl9k0KFZIgRnE4HT7rZfc_eLqoMd2K6ELC1qpk,23956
+evalvault/adapters/inbound/mcp/tools.py,sha256=fnvkWS5p93o3FNmUSbh3EW4jCAVwtBKHX6kDuEbXkK8,24219
 evalvault/adapters/outbound/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 evalvault/adapters/outbound/analysis/__init__.py,sha256=TLuS-eKfXg97_Db5td1nTZkD3BErRLZLic1v2EAM2sA,6185
 evalvault/adapters/outbound/analysis/analysis_report_module.py,sha256=xah3wgJErHD_Hpb1YAwWRsxr8xaC8SW--CpNA7IgfxI,3957
@@ -81,7 +81,7 @@ evalvault/adapters/outbound/analysis/pattern_detector_module.py,sha256=SyCDO_VS-
 evalvault/adapters/outbound/analysis/pipeline_factory.py,sha256=XvcCbKCN_otv1pGUzk0oE76RV19yFga8r6RngBvgEFo,3691
 evalvault/adapters/outbound/analysis/pipeline_helpers.py,sha256=8E8IrYI5JvRrpnjxe0DS7srbPzB0XAxxXhLLYgfwsgU,5756
 evalvault/adapters/outbound/analysis/priority_summary_module.py,sha256=o8Y0rfHjYYE9WNTwKtpJulwfvLA3MNMhYjdSg15Vacc,10802
-evalvault/adapters/outbound/analysis/ragas_evaluator_module.py,sha256=KQd8zdyMGKklZane077RRNPDOf6kqVDbj_R9Qbq-quA,7275
+evalvault/adapters/outbound/analysis/ragas_evaluator_module.py,sha256=Cd-spGn56zMcqOdoTLUHTYVOFqHqR17tPFyJs7rmnbw,7659
 evalvault/adapters/outbound/analysis/retrieval_analyzer_module.py,sha256=D24GTaKabHacSBI-UqCd_jy61hnne8-QG1p4rqW1Bzk,5748
 evalvault/adapters/outbound/analysis/retrieval_benchmark_module.py,sha256=_duIBlYhAsFygEpC7DuwoAqfTbVG2xgp70JjW1LJAGE,9312
 evalvault/adapters/outbound/analysis/retrieval_quality_checker_module.py,sha256=K1IJn4bvvz-BfqQmhd5Ik9oATjq_-G7V1AZSW8zKtSE,3121
@@ -135,16 +135,17 @@ evalvault/adapters/outbound/kg/graph_rag_retriever.py,sha256=_6qd8p_2TpHnppv8LUQ
 evalvault/adapters/outbound/kg/networkx_adapter.py,sha256=wSZjMiMzpS9ZZLTRYjHm7X6TN9ITbuTWr10KyR6hFxg,21043
 evalvault/adapters/outbound/kg/parallel_kg_builder.py,sha256=sbR-bVpjTFhV47Ss8LYA3Ytk-THqo1HJz2YLfAelRyg,8886
 evalvault/adapters/outbound/kg/query_strategies.py,sha256=BDDft2Cc-xDvJWyIuyUTv0bPvaIiXJLd76YdjzMAZlk,19445
-evalvault/adapters/outbound/llm/__init__.py,sha256=GDllWZNBVqmQSFOXsXFlE82qxgaZWGL4TChu61bEG7U,4523
-evalvault/adapters/outbound/llm/anthropic_adapter.py,sha256=Qjpwqt6eDDvrSAGynvO0NZQIrhfEFJqLJfqfYPjcwaY,7491
+evalvault/adapters/outbound/llm/__init__.py,sha256=MZODPJbwpMuuubG7cxXEh75V3e0r65db5PMcFJMJfiI,3119
+evalvault/adapters/outbound/llm/anthropic_adapter.py,sha256=LTU3RhtgaKNLvMpwj1-GGCclsBZM2dzVJ1OIMO9BHqI,8162
 evalvault/adapters/outbound/llm/azure_adapter.py,sha256=FR9Gimy-jonzQjtILGHKXAY_ycbjGKbKLFiD-OZ8EVs,3033
 evalvault/adapters/outbound/llm/base.py,sha256=c4cJJTKwJiBlGNCV7qomq5mQCrbc48Yi7c3vjlat7Q0,7674
+evalvault/adapters/outbound/llm/factory.py,sha256=TTpbyDQH9TCr1wmajSkLgkF_fHGv6sX0hZOEqIWZudU,3509
 evalvault/adapters/outbound/llm/instructor_factory.py,sha256=6iJIjZbCJwDYEpV673R3lQOupzeWlvjCmhjYqthrueY,4497
-evalvault/adapters/outbound/llm/llm_relation_augmenter.py,sha256=jdeDumCi34KqAfRwLh9VTia9wiP5bvVwvHxJylLDOcg,3854
-evalvault/adapters/outbound/llm/ollama_adapter.py,sha256=L7MRlPUyYxTSVHhCz7LHducTqBifTzmovNBi26SWtr8,10615
-evalvault/adapters/outbound/llm/openai_adapter.py,sha256=Zv4gi0AxoQrXzf2hBrQjLaQVM4WttHuO2raTCekR_Kk,5163
-evalvault/adapters/outbound/llm/token_aware_chat.py,sha256=HLmwTZkiwmUuopjFwDGpamIRC44PWnwkRA4O-pUBt_s,11937
-evalvault/adapters/outbound/llm/vllm_adapter.py,sha256=zwH_KKydABn0J-hCs6OB-3809X_qBx8FPQ2EhO1gkcY,3897
+evalvault/adapters/outbound/llm/llm_relation_augmenter.py,sha256=B22CJB66Vfh_SK_11J8jlfIhjDmY_7FXp-bTL2YGWGw,4848
+evalvault/adapters/outbound/llm/ollama_adapter.py,sha256=HuPjX9rhBn95OzwNNXOgD-FIOKSUNuHEcNKDeHxw8kA,11481
+evalvault/adapters/outbound/llm/openai_adapter.py,sha256=X6S93HFFJDOhjPmayw0xobNiJDb0AtqG07e5wZRBRuE,6463
+evalvault/adapters/outbound/llm/token_aware_chat.py,sha256=yYmynaniNrYxtvXL6ejTE_d8x8nS3PEpgtTJ4eI2hgM,12977
+evalvault/adapters/outbound/llm/vllm_adapter.py,sha256=OKb3Nda9OLMmHdvLjvkeJcQVeXf-B8TDibmAs7PS7kg,5157
 evalvault/adapters/outbound/methods/__init__.py,sha256=3vyE9w3Ex2oMaO4ZE7Fy6xlHhJ6YQXHQNCvBiW9X2lM,345
 evalvault/adapters/outbound/methods/baseline_oracle.py,sha256=oUsF5sIiPY5vuDtrz0Ki05SnPlnVzn7APERP5v1KpPM,1308
 evalvault/adapters/outbound/methods/external_command.py,sha256=gR2mlgr-SCAO3cS3I7pYgS8hL8JE8Y-0VZIhg7USazY,5287
@@ -159,18 +160,19 @@ evalvault/adapters/outbound/nlp/korean/kiwi_tokenizer.py,sha256=EnJnlo6D7YsVX2o4
 evalvault/adapters/outbound/nlp/korean/korean_evaluation.py,sha256=Mxwu3zhtdm8Tew5o3Oxs8O8Wh-YN81Zs04tlS3vhNS8,20046
 evalvault/adapters/outbound/nlp/korean/korean_stopwords.py,sha256=UemEFCJudg2EpsHg8uU2eR-iCh34kw4ZSVCRvnEC6a4,4293
 evalvault/adapters/outbound/nlp/korean/toolkit.py,sha256=EYGpd89ilpn4Wg5t8pALYt4Qi0aDHYOfXGuYbQx7do0,4246
+evalvault/adapters/outbound/nlp/korean/toolkit_factory.py,sha256=x3v-AAkVInOabC4PtOtStsZrFnHun0IOqZDyQGaQVm8,586
 evalvault/adapters/outbound/phoenix/sync_service.py,sha256=i6gHpNiZXKQ5yzV9B2TPb-P1N45k_Ck5ruzh3oqp4d8,9122
 evalvault/adapters/outbound/report/__init__.py,sha256=8OUduTHnWkBLHYrc7mBg45DnAwz0RgvSJmz1HqxVjLY,477
 evalvault/adapters/outbound/report/dashboard_generator.py,sha256=Dcu18NTK4lS8XNKnnnquagpZkd-4TSf5Mb2isFNW5Pk,7800
-evalvault/adapters/outbound/report/llm_report_generator.py,sha256=sp2YRCmPOhn08vb8Bq_ayo-ZjgyBBxRhzRFvzlaDhsA,24063
+evalvault/adapters/outbound/report/llm_report_generator.py,sha256=HUDA_IPBbl54cyEjTTJzdKTQ6H4IoZi-1VBdVmZf0uI,26593
 evalvault/adapters/outbound/report/markdown_adapter.py,sha256=5PS72h_qe4ZtYs-umhX5TqQL2k5SuDaCUc6rRw9AKRw,16761
 evalvault/adapters/outbound/storage/__init__.py,sha256=n5R6thAPTx1leSwv6od6nBWcLWFa-UYD6cOLzN89T8I,614
-evalvault/adapters/outbound/storage/base_sql.py,sha256=Og-YRWHsCFQP2vnyvsgfWr4C2_ZE89ZmPXcPLiHeggU,21976
+evalvault/adapters/outbound/storage/base_sql.py,sha256=kWYaiUq5D35iMx34cX3_mjhRZoEXfgQR-tSk3UhbvcE,40792
 evalvault/adapters/outbound/storage/benchmark_storage_adapter.py,sha256=Qgf9xSSIkYQRpG4uLzcUdoYO9LTQDQ4tFRkkMYer-WA,9803
-evalvault/adapters/outbound/storage/postgres_adapter.py,sha256=IaijoeCIRi7JO2d5yfgfmF-ejobOnU7Izlx332mSUP8,39020
-evalvault/adapters/outbound/storage/postgres_schema.sql,sha256=aAfgwxWEqCBGGpn_QRD_BbzXR2Q-9cd9GMsCbFeohNY,7632
+evalvault/adapters/outbound/storage/postgres_adapter.py,sha256=HLaoQ3YJDFwOxeY0S92oPIqb-7EgWSasgt89RM86vr0,47148
+evalvault/adapters/outbound/storage/postgres_schema.sql,sha256=A9MfO0pjf4kjxoRj2KPI0Gg1cbX13I2YE3oieT-PGiI,8906
 evalvault/adapters/outbound/storage/schema.sql,sha256=LknvBvNVLvkW7c_hHTLHrxSf4TZApzbRyAk1ctuROUc,10608
-evalvault/adapters/outbound/storage/sqlite_adapter.py,sha256=SKZ9IZjchi7w89WNkZ6aTelAzaV0MqUC7cexrkndTNY,48555
+evalvault/adapters/outbound/storage/sqlite_adapter.py,sha256=nimaXvqhJf-90r4Qjd8tBZWU5_NUIvBlQ6Vox-xrQ_c,51223
 evalvault/adapters/outbound/tracer/__init__.py,sha256=xrvQQuAvF_UI02mKLMV7GTrG3zn836n5zwCRrrmhq_U,1054
 evalvault/adapters/outbound/tracer/open_rag_log_handler.py,sha256=aq96FIWD-bBaSkq-bygWhQArC9LWghSwi-S03Mga0mI,2827
 evalvault/adapters/outbound/tracer/open_rag_trace_adapter.py,sha256=P-4PN1UweITXu5uN3LJVCEL3wRwiExzhgs3y2GN78xM,4784
@@ -191,7 +193,7 @@ evalvault/config/phoenix_support.py,sha256=e6RPWd6Qb7KU6Q8pLaYTpJGWULtvEEU6B0xHW
 evalvault/config/settings.py,sha256=T92GShlYKDaVinwbsbWX2DmNfm91Cvcvh8Te8pNOTsw,12875
 evalvault/config/playbooks/improvement_playbook.yaml,sha256=9F9WVVCydFfz6zUuGYzZ4PKdW1LLtcBKVF36T7xT764,26965
 evalvault/domain/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-evalvault/domain/entities/__init__.py,sha256=CZU7VfTq2vart_j2pLemOX_TtSKzmpzB151pW-jSinw,3097
+evalvault/domain/entities/__init__.py,sha256=RZi_6oQcq-2-sJcydfKOSr03vFxo-mF7CGHN9Ma4Cdg,3379
 evalvault/domain/entities/analysis.py,sha256=gcMtumC66g-AIqb2LgfMpm5BMzwJhJkjg-zuybNoJCM,15208
 evalvault/domain/entities/analysis_pipeline.py,sha256=hD9rFHMa4rUq0InRkSKhh6HQ9ZeNYAHKADzs-kWRP04,16845
 evalvault/domain/entities/benchmark.py,sha256=CVbz_eW7Y9eM7wG7xA_xmldTIs72csdoTmu3E0NKoMU,18475
@@ -205,6 +207,7 @@ evalvault/domain/entities/kg.py,sha256=8awN1M4vxAGQZk_ZG8i2CXKTizQ8FA1VCLhUWHZq0
 evalvault/domain/entities/memory.py,sha256=bfS75q8K8_jNrB7IYh4mjP8Lkyj-I0TVsmHCP0FuICw,8423
 evalvault/domain/entities/method.py,sha256=a3jZi7SjcpK3HeVyVwQkUMwpnmg2RbxCnH4NqYPLCOI,1157
 evalvault/domain/entities/prompt.py,sha256=VzuUzqkqXv0FwTSNGyV5sSCft5sxTbG_Noq6Ymnke5o,2910
+evalvault/domain/entities/prompt_suggestion.py,sha256=Ep_XSjdYUj7pFSCMyeeZKs8yTnp74AVx05Zqr7829PE,1243
 evalvault/domain/entities/rag_trace.py,sha256=sZgnkG4fK6KOe3Np6TYAZ_tPnsRbOmucDSQns35U1n4,11868
 evalvault/domain/entities/result.py,sha256=OaGHMDLWMW2O4fNVuVTUvWFVBQ1iu93OD_oI3NumrCQ,10697
 evalvault/domain/entities/stage.py,sha256=dbVzhgpP_p2p2eDJBWe7mwyyl6zUTP9kEKN_YRUvufY,7183
@@ -219,7 +222,7 @@ evalvault/domain/metrics/registry.py,sha256=QKjo4RNHxCqObGg36xJP3KAHqFpHM50Jy7Ge
 evalvault/domain/metrics/retrieval_rank.py,sha256=F55ByadJBowyKHKBmKAZ0T0qN_R1_7UNu-MiLnT4Ypg,14675
 evalvault/domain/metrics/terms_dictionary.json,sha256=-ZQmpx6yMOYoAOpcLj-xK2LkAeCbAw0EUb6-syIOKS0,3801
 evalvault/domain/metrics/text_match.py,sha256=P-YTZs9ekDqEmxLNBP8eXnMRymPdC8V4dJPtwG2ajVM,10219
-evalvault/domain/services/__init__.py,sha256=a7YjsOuOzbY2yQvtDJg7W8dPJme31rI04Qc49w5iHKE,686
+evalvault/domain/services/__init__.py,sha256=X5Af1kf_vSt3S3mFwOV6OQdro-lFxwbVdNd7nJznkC8,1024
 evalvault/domain/services/analysis_service.py,sha256=oUEtfJHB3bNJ_Ksygx-pjnLm4CTk7_rDvDbqfkAfFD4,10838
 evalvault/domain/services/async_batch_executor.py,sha256=qYFRl7CGmv56XppeRhInde7Fw0GESCoZh8V-Iv_1hQQ,11140
 evalvault/domain/services/batch_executor.py,sha256=cYA_Q1es46n_PYeyyfm0iM2b7GGVtDoOGoMxexrf6tI,1243
@@ -235,23 +238,27 @@ evalvault/domain/services/document_versioning.py,sha256=M1qZaMpQ2exVT1wkVAmvEPPu
 evalvault/domain/services/domain_learning_hook.py,sha256=rhKBmdnrJyfGzFNsNxzyv8jZO26-WOosHSmBV_9qdJg,7176
 evalvault/domain/services/embedding_overlay.py,sha256=ZTNxUPXpHGbQ3Uri5DD3feTUFn7qrhuNshhyCQEvRuM,3559
 evalvault/domain/services/entity_extractor.py,sha256=f3Rf5saK8QsgetLNK1Hbxzt8PtttJZCicSR63S8DJ5k,14141
-evalvault/domain/services/evaluator.py,sha256=rxj0tVMwtDrekNFstT6baQDfemsH-c6XtHxX-9BLKmo,64476
+evalvault/domain/services/evaluator.py,sha256=YReil1mokTILyllAbG_QnFhob-15G5tNeWZZMbSc3yo,67551
 evalvault/domain/services/experiment_comparator.py,sha256=IBrxIwux-8GucwlLx6e5lUqB9miSPvBLGJK9ctoW7Y0,3299
 evalvault/domain/services/experiment_manager.py,sha256=2k-qGiAUyZuqqmcp4P-M3Z9HTXwwcqW5HQYKNkcIHuI,4863
 evalvault/domain/services/experiment_reporter.py,sha256=QYlVmCFSx8hKTPMezc7QjJE07b3MSQ82Q4QVucSHLVY,1420
 evalvault/domain/services/experiment_repository.py,sha256=1OQSvBmM7llNJFWgGHBL3XvJvn_OIQsKo7zophYtV1g,2423
 evalvault/domain/services/experiment_statistics.py,sha256=aOrqbBjB1swHPaFRziID1mrAowoV-pBS4XdLFhAhMLE,1115
+evalvault/domain/services/holdout_splitter.py,sha256=Sos61Zy_bBjStt8LPHJ3KxDNda-OmX7AVUsT24K1n6Q,1910
 evalvault/domain/services/improvement_guide_service.py,sha256=gMoVFlDsprOEEfRGKmdbk9_Due62J63Q-rL2zr65Q0s,17881
-evalvault/domain/services/intent_classifier.py,sha256=s-4uNv60G1n5W0ddLVAQtmm2uiyKCQL4viWn-SgH4hA,9817
+evalvault/domain/services/intent_classifier.py,sha256=hsWivDXqXJjCJEE-OI7eUGeYrewpYxlz67Z0TI3oskU,11707
 evalvault/domain/services/kg_generator.py,sha256=oEugjPdn8Pb2Q3r5yAZl0dZJibNUkEherlRVquknB6k,24969
 evalvault/domain/services/memory_aware_evaluator.py,sha256=vTiYoxiMfZ_CMjSBjqwkBRdpiXRwQ2zXnQ2pXzVHYts,5249
 evalvault/domain/services/memory_based_analysis.py,sha256=oh2irCy3le7fWiTtL31SMEhPyu7fyBVz-giO2hlNifE,4499
 evalvault/domain/services/method_runner.py,sha256=pABqKZeaALpWZYDfzAbd-VOZt2djQggRNIPuuPQeUSw,3571
 evalvault/domain/services/pipeline_orchestrator.py,sha256=yriVlEVZYDtt0Vwt4Ae6xyW1H6Dj4Hxdn8XQSvQNSoQ,19436
-evalvault/domain/services/pipeline_template_registry.py,sha256=j2WQwXrCvYd-dbtxOUTmgTZZAgNtu0eUvqgdryerCbc,27964
+evalvault/domain/services/pipeline_template_registry.py,sha256=aWqXLQ24grpSZo9M4tZLRo1ysD10c6hUpW3JupZH9e0,28083
+evalvault/domain/services/prompt_candidate_service.py,sha256=Ibyb5EaWK28Ju2HnTqHHGOoiA9Q-VwY3hjxVODALwGY,3997
 evalvault/domain/services/prompt_manifest.py,sha256=5s5Kd6-_Dn-xrjjlU99CVo6njsPhvE50H5m_85U-H6U,5612
-evalvault/domain/services/prompt_registry.py,sha256=81tq__u2fFxTEG8bWnyJ2Qdb9N89jcqIdSfOAKEbEvg,3029
+evalvault/domain/services/prompt_registry.py,sha256=THcNs4jqp4FTLv9uO-VVvN6XGorkGrcIUwQH1dL74o8,4166
+evalvault/domain/services/prompt_scoring_service.py,sha256=SlvfuIbhj92RJu4RQAJ1BGKhKkOAUOt3cZNH21HtsX4,9833
 evalvault/domain/services/prompt_status.py,sha256=r1dFLGz4SfRxXaxsULQsr0-HpJkG9YfZ_yLIxF1MMBo,6731
+evalvault/domain/services/prompt_suggestion_reporter.py,sha256=Fc6sCPebUMk8SZVpjoJ6bCEun0ma-YmayEQnulBVv8s,10577
 evalvault/domain/services/ragas_prompt_overrides.py,sha256=4BecYE2KrreUBbIM3ssP9WzHcK_wRc8jW7CE_k58QOU,1412
 evalvault/domain/services/retrieval_metrics.py,sha256=dtrQPLMrXSyWLcgF8EGcLNFwzwA59WDzEh41JRToHAY,2980
 evalvault/domain/services/retriever_context.py,sha256=ySQ-GuadiggS0LVAib4AxA_0JpasYz4S9hbjau0eyIA,6482
@@ -260,7 +267,7 @@ evalvault/domain/services/stage_event_builder.py,sha256=ScTgyeRiH7z_rnNI_2p-i9sz
 evalvault/domain/services/stage_metric_guide_service.py,sha256=_JdRsBRWirO24qYFlh6hG-dkoWlX6_XWEYKf_uUlKIQ,8807
 evalvault/domain/services/stage_metric_service.py,sha256=KukIWWhWVOtclrET6uyWJ17jG76LfkKiqrUrDIDJ3gw,15327
 evalvault/domain/services/stage_summary_service.py,sha256=VVtuAr4vwzvmNFn8rqURJrhKFqAMG4CaBmyGiUk_xG0,1590
-evalvault/domain/services/synthetic_qa_generator.py,sha256=v0-M9VXM5oZ12AkfDyhOJAc-Mv28kGmGD3H969X5f6M,16471
+evalvault/domain/services/synthetic_qa_generator.py,sha256=aiOTPoHZbKRTEeodABQ2I5lq8-Vs_kQtuzcGWd4MTGE,16526
 evalvault/domain/services/testset_generator.py,sha256=6IpiZ0pqhKEymo-AlUdfJjDkF2P1n8Md_QKV4nOheyg,4470
 evalvault/domain/services/threshold_profiles.py,sha256=YuOrD5CkXugAdSQYbMsFzS5VS1R201JOJtpKTs4dpXU,1296
 evalvault/domain/services/unified_report_service.py,sha256=lG3VpMLC1MTYUlcGl-MUEE4PUopkyrhcgj4_ye9c_vM,11829
@@ -269,9 +276,9 @@ evalvault/ports/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 evalvault/ports/inbound/__init__.py,sha256=2Wsc0vNzH8_ZaErk4OHxP93hRonLUkMbn3W28DtTDO0,562
 evalvault/ports/inbound/analysis_pipeline_port.py,sha256=RJfKtp22AYEqnmRk6RDawAK52rEmyAhuk0FUPJQUwQU,1758
 evalvault/ports/inbound/evaluator_port.py,sha256=rDvouIRUjBD7uICgrpeo11vNPvo27_0CdylRHPodPSE,1323
-evalvault/ports/inbound/learning_hook_port.py,sha256=ehpRyRNUY1PRtzIoaCyDM_QRxp6WjEQvwPskAxI4CPc,3109
+evalvault/ports/inbound/learning_hook_port.py,sha256=EeJeMl3chcPHlj5mkLOj6tm8s_qdDRvoCwK1-0l70tI,3297
 evalvault/ports/inbound/web_port.py,sha256=b4uMhwOMLXy3LeILc7ZK3RR-XtoW4p4NzoTpj4syptg,5578
-evalvault/ports/outbound/__init__.py,sha256=jEmLbY3lZ9osue6pG5dc345BdMikBEWq4cnX7ocEul0,3276
+evalvault/ports/outbound/__init__.py,sha256=0uPE6CXkoK6ECs3O4OZYAmVg5VAAHoF5rdb1eoj1NAc,3367
 evalvault/ports/outbound/analysis_cache_port.py,sha256=zPSdUVK_yw3PMWPII2YvS1WLmCGlg5bDScSuYINW9yc,1386
 evalvault/ports/outbound/analysis_module_port.py,sha256=QYzkvie9-BbONj8ZgiQUjm8I-bn8mgzlXTzIXMhehmQ,1881
 evalvault/ports/outbound/analysis_port.py,sha256=gE-iXToTgdQomj9JwNZJY4nwut8q0J6EurUmJNsnptQ,2127
@@ -283,21 +290,22 @@ evalvault/ports/outbound/embedding_port.py,sha256=ZHeKRMRBNjpZKWxsLKrD8jJz0M66JT
 evalvault/ports/outbound/improvement_port.py,sha256=fIXhcG4n6OJ1hdvWeqEoLBrVsCNdHZRgtEZjR8lf3qA,2325
 evalvault/ports/outbound/intent_classifier_port.py,sha256=gqMIk0rH6Z43ceuMMRX4vqXurgHZz-CJX2bR5PVAkjQ,2253
 evalvault/ports/outbound/korean_nlp_port.py,sha256=mJCnxBAkV8a5Nd_VX6QcjfDucY62er8GlaNO4HQA8q8,1572
-evalvault/ports/outbound/llm_port.py,sha256=qnVmQmkWgpeJ_eB_W1gLgXTWpowfDgYrXG5uKUhIsy8,3528
+evalvault/ports/outbound/llm_factory_port.py,sha256=lzoDJi6A6ltk-t3N4oY8DSwMBMfnvXGgSduILOpzoas,305
+evalvault/ports/outbound/llm_port.py,sha256=YAW0i-41yT8KzMuzZGEO5yPDkHN0onGxj55eL0cdPHY,4393
 evalvault/ports/outbound/method_port.py,sha256=sntcKgwagAdJGxp0dI-S_bhBQcOW9QpnND3fOjrsX9E,1377
 evalvault/ports/outbound/nlp_analysis_port.py,sha256=QDJHAsSpynTenuaKp78t1s--U036mtYeUEX0p5vQw24,3046
 evalvault/ports/outbound/relation_augmenter_port.py,sha256=cMcHQnmK111WzZr50vYr7affeHhOtpFZxPARwkg9xbk,651
 evalvault/ports/outbound/report_port.py,sha256=wgReSYL4SupXIoALFh0QFWfX2kzPftXpWTvGLCMd2B8,1315
 evalvault/ports/outbound/stage_storage_port.py,sha256=Nlf9upsXxgCABQB5cJdpLQYsoZNiGRAU5zE5D-Ptp2I,1201
-evalvault/ports/outbound/storage_port.py,sha256=d9f8bvAtPA2aytKrHvrfrWGOmaQSepLn23Bd_52QSbI,4862
+evalvault/ports/outbound/storage_port.py,sha256=mexWwP3wvIVigS_ZZP5wWAzGAozByNVMwnW9ziWv3eA,5897
 evalvault/ports/outbound/tracer_port.py,sha256=kTqJCUIJHnvvDzMxxGhHSfiz8_Q4CZ0WSPvIUVVOcyw,623
 evalvault/ports/outbound/tracker_port.py,sha256=05LA3AWnuE1XmGQC16Zle9i2sEV3q69Nt8ZUye_w1_Y,2532
 evalvault/reports/__init__.py,sha256=Bb1X4871msAN8I6PM6nKGED3psPwZt88hXZBAOdH06Y,113
 evalvault/reports/release_notes.py,sha256=pZj0PBFT-4F_Ty-Kv5P69BuoOnmTCn4kznDcORFJd0w,4011
 evalvault/scripts/__init__.py,sha256=NwEeIFQbkX4ml2R_PhtIoNtArDSX_suuoymgG_7Kwso,89
 evalvault/scripts/regression_runner.py,sha256=SxZori5BZ8jVQ057Mf5V5FPgIVDccrV5oRONmnhuk8w,8438
-evalvault-1.62.1.dist-info/METADATA,sha256=FL8WDZmjLoW_9zXFlxEtrRhi0C4qiQEkVCtep3YLNIE,14155
-evalvault-1.62.1.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
-evalvault-1.62.1.dist-info/entry_points.txt,sha256=Oj9Xc5gYcyUYYNmQfWI8NYGw7nN-3M-h2ipHIMlVn6o,65
-evalvault-1.62.1.dist-info/licenses/LICENSE.md,sha256=3RNWY4jjtrQ_yYa-D-7I3XO12Ti7YzxsLV_dpykujvo,11358
-evalvault-1.62.1.dist-info/RECORD,,
+evalvault-1.63.0.dist-info/METADATA,sha256=Kscv51ExIOOosrBnBXI5S1_3V0S2t2nCZhfyssREdg4,23879
+evalvault-1.63.0.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
+evalvault-1.63.0.dist-info/entry_points.txt,sha256=Oj9Xc5gYcyUYYNmQfWI8NYGw7nN-3M-h2ipHIMlVn6o,65
+evalvault-1.63.0.dist-info/licenses/LICENSE.md,sha256=3RNWY4jjtrQ_yYa-D-7I3XO12Ti7YzxsLV_dpykujvo,11358
+evalvault-1.63.0.dist-info/RECORD,,