evalvault 1.76.0__tar.gz → 1.77.0__tar.gz

{evalvault-1.76.0 → evalvault-1.77.0}/.env.offline.example

@@ -4,11 +4,13 @@
 # Usage:
 #   cp .env.offline.example .env.offline
 #   # Edit .env.offline with your air-gapped network settings
-#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d
+#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d --no-build --pull never
 
 # ================================================
-# Profile
+# Profile (choose one)
 # ================================================
+# Ollama-based (dev/prod): dev or prod
+# vLLM-based: vllm
 EVALVAULT_PROFILE=dev
 
 # ================================================
@@ -31,7 +33,7 @@ EVALVAULT_MEMORY_DB_PATH=data/db/evalvault_memory.db
 CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
 
 # ================================================
-# Docker Base Images (offline builds)
+# Docker Base Images (online build only)
 # ================================================
 EVALVAULT_PYTHON_IMAGE=python:3.12.6-slim
 EVALVAULT_UV_IMAGE=ghcr.io/astral-sh/uv:0.4.28

evalvault-1.77.0/.env.offline.ollama.example

@@ -0,0 +1,59 @@
+# EvalVault Offline Environment Template (Ollama)
+# Copy to .env.offline and fill required values.
+#
+# Usage:
+#   cp .env.offline.ollama.example .env.offline
+#   # Edit .env.offline with your air-gapped network settings
+#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d --no-build --pull never
+
+# ================================================
+# Profile (Ollama)
+# ================================================
+EVALVAULT_PROFILE=dev
+
+# ================================================
+# PostgreSQL (core stack)
+# ================================================
+POSTGRES_IMAGE=pgvector/pgvector:0.8.0-pg16
+POSTGRES_USER=evalvault
+POSTGRES_PASSWORD=evalvault
+POSTGRES_DB=evalvault
+
+# ================================================
+# Storage (SQLite paths for local file-based storage)
+# ================================================
+EVALVAULT_DB_PATH=data/db/evalvault.db
+EVALVAULT_MEMORY_DB_PATH=data/db/evalvault_memory.db
+
+# ================================================
+# API / CORS
+# ================================================
+CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
+
+# ================================================
+# Docker Base Images (online build only)
+# ================================================
+EVALVAULT_PYTHON_IMAGE=python:3.12.6-slim
+EVALVAULT_UV_IMAGE=ghcr.io/astral-sh/uv:0.4.28
+EVALVAULT_NODE_IMAGE=node:20.19-alpine
+EVALVAULT_NGINX_IMAGE=nginx:1.27.3-alpine
+
+# API_AUTH_TOKENS=
+# KNOWLEDGE_READ_TOKENS=
+# KNOWLEDGE_WRITE_TOKENS=
+
+# ================================================
+# Ollama server (air-gapped network)
+# ================================================
+# IMPORTANT: Model weights are NOT shipped with EvalVault.
+# You must provide the URL to your air-gapped Ollama server.
+
+# OLLAMA_BASE_URL=
+OLLAMA_TIMEOUT=120
+# OLLAMA_TOOL_MODELS=
+
+# ================================================
+# Faithfulness fallback (optional)
+# ================================================
+# FAITHFULNESS_FALLBACK_PROVIDER=
+# FAITHFULNESS_FALLBACK_MODEL=

evalvault-1.77.0/.env.offline.vllm.example

@@ -0,0 +1,83 @@
+# EvalVault Offline Environment Template (vLLM)
+# Copy to .env.offline and fill required values.
+#
+# Usage:
+#   cp .env.offline.vllm.example .env.offline
+#   # Edit .env.offline with your air-gapped network settings
+#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d --no-build --pull never
+
+# ================================================
+# Profile (vLLM)
+# ================================================
+EVALVAULT_PROFILE=vllm
+
+# ================================================
+# PostgreSQL (core stack)
+# ================================================
+POSTGRES_IMAGE=pgvector/pgvector:0.8.0-pg16
+POSTGRES_USER=evalvault
+POSTGRES_PASSWORD=evalvault
+POSTGRES_DB=evalvault
+
+# ================================================
+# Storage (SQLite paths for local file-based storage)
+# ================================================
+EVALVAULT_DB_PATH=data/db/evalvault.db
+EVALVAULT_MEMORY_DB_PATH=data/db/evalvault_memory.db
+
+# ================================================
+# API / CORS
+# ================================================
+CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
+
+# ================================================
+# Docker Base Images (online build only)
+# ================================================
+EVALVAULT_PYTHON_IMAGE=python:3.12.6-slim
+EVALVAULT_UV_IMAGE=ghcr.io/astral-sh/uv:0.4.28
+EVALVAULT_NODE_IMAGE=node:20.19-alpine
+EVALVAULT_NGINX_IMAGE=nginx:1.27.3-alpine
+
+# API_AUTH_TOKENS=
+# KNOWLEDGE_READ_TOKENS=
+# KNOWLEDGE_WRITE_TOKENS=
+
+# ================================================
+# vLLM server (air-gapped network - internal models)
+# ================================================
+# IMPORTANT: Model weights are NOT shipped with EvalVault.
+# You must provide the URL to your air-gapped vLLM server.
+
+# VLLM_BASE_URL=http://vllm-server:8000/v1
+# VLLM_API_KEY=
+# VLLM_MODEL=Qwen/Qwen2.5-7B-Instruct
+# VLLM_EMBEDDING_MODEL=BAAI/bge-m3
+# VLLM_EMBEDDING_BASE_URL=http://vllm-embedding:8000/v1
+
+# ================================================
+# Closed-source models (requires network proxy)
+# ================================================
+# If your air-gapped network has proxy access to external APIs,
+# you can use closed-source models via API keys.
+
+# OpenAI
+# OPENAI_API_KEY=sk-...
+# OPENAI_MODEL=gpt-4o-mini
+# OPENAI_BASE_URL=https://api.openai.com/v1
+
+# Anthropic
+# ANTHROPIC_API_KEY=sk-ant-...
+# ANTHROPIC_MODEL=claude-sonnet-4-20250514
+
+# Azure OpenAI
+# AZURE_OPENAI_API_KEY=
+# AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
+# AZURE_OPENAI_DEPLOYMENT=gpt-4o-mini
+# AZURE_OPENAI_API_VERSION=2024-02-15-preview
+
+# ================================================
+# Faithfulness fallback (optional)
+# ================================================
+# Use a more capable model for faithfulness metric if needed
+# FAITHFULNESS_FALLBACK_PROVIDER=openai
+# FAITHFULNESS_FALLBACK_MODEL=gpt-4o

{evalvault-1.76.0 → evalvault-1.77.0}/.gitignore

@@ -165,3 +165,9 @@ scratch/
 
 # Local artifacts
 MagicMock/
+
+# Manim renders
+media/
+
+# Local presentation workspace
+presentation_work/

{evalvault-1.76.0 → evalvault-1.77.0}/CLAUDE.md

@@ -4,11 +4,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Overview
 
-EvalVault is a RAG (Retrieval-Augmented Generation) evaluation system for Korean/English insurance documents. Built on Ragas + Langfuse for evaluation and tracking.
+EvalVault is a RAG (Retrieval-Augmented Generation) evaluation system for Korean/English insurance documents. Built on Ragas with dual-tracker support (MLflow + Phoenix) for experiment tracking and observability.
 
 **Core Flow:**
 ```
-Input (CSV/Excel/JSON) → Ragas Evaluation → Langfuse Trace/Score → Analysis
+Input (CSV/Excel/JSON) → Ragas Evaluation → MLflow + Phoenix (dual logging) → Analysis
 ```
 
 **Supported Metrics:**
@@ -53,6 +53,8 @@ src/evalvault/
 | DatasetPort | CSV/Excel/JSON Loaders | ✅ Complete |
 | TrackerPort | LangfuseAdapter | ✅ Complete |
 | TrackerPort | MLflowAdapter | ✅ Complete |
+| TrackerPort | PhoenixAdapter | ✅ Complete |
+| TrackerPort | MultiTrackerAdapter | ✅ Complete (dual-logging: MLflow + Phoenix) |
 | StoragePort | SQLiteAdapter | ✅ Complete |
 | StoragePort | PostgreSQLAdapter | ✅ Complete |
 | EvaluatorPort | RagasEvaluator | ✅ Complete |
@@ -69,6 +71,25 @@ src/evalvault/
 - **Purpose**: Trace logging, score tracking, evaluation history
 - **Credentials**: Inject via `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY`
 
+### Experiment Tracking (MLflow + Phoenix)
+
+EvalVault uses dual-tracker logging by default:
+- **MLflow**: Experiment management, metric comparison, model versioning
+- **Phoenix**: Real-time observability, trace visualization, prompt debugging
+
+```bash
+# MLflow (required)
+MLFLOW_TRACKING_URI=http://localhost:5000
+MLFLOW_EXPERIMENT_NAME=evalvault-experiments
+
+# Phoenix (required)
+PHOENIX_ENDPOINT=http://localhost:6006
+PHOENIX_PROJECT_NAME=evalvault
+
+# Tracker selection (default: mlflow+phoenix)
+TRACKER_PROVIDER=mlflow+phoenix  # Options: mlflow, phoenix, langfuse, mlflow+phoenix
+```
+
 ## Development Commands
 
 ```bash
@@ -188,14 +209,40 @@ OPENAI_MODEL=gpt-5-nano
 OPENAI_EMBEDDING_MODEL=text-embedding-3-small
 # OPENAI_BASE_URL=https://api.openai.com/v1  # optional
 
-# Langfuse (self-hosted)
+# Langfuse (optional, if using langfuse tracker)
 LANGFUSE_PUBLIC_KEY=pk-lf-...
 LANGFUSE_SECRET_KEY=sk-lf-...
 LANGFUSE_HOST=http://your-langfuse-host:port
+
+# MLflow + Phoenix (default dual-tracker)
+MLFLOW_TRACKING_URI=http://localhost:5000
+PHOENIX_ENDPOINT=http://localhost:6006
+TRACKER_PROVIDER=mlflow+phoenix
 ```
 
 **Note:** 메트릭 임계값(thresholds)은 환경변수가 아닌 **데이터셋 JSON 파일**에 정의합니다.
 
+## Offline Deployment
+
+EvalVault supports air-gapped (오프라인) deployment with bundled models.
+
+### Supported Configurations
+- **Ollama**: Lightweight local inference (`llama3.2`, `llama3.1`, etc.)
+- **vLLM**: High-performance GPU inference (recommended for production)
+
+### Quick Start
+```bash
+# Build offline bundle
+./scripts/offline/build_full_offline_bundle.sh
+
+# Restore on target machine
+./scripts/offline/import_images.sh
+./scripts/offline/restore_model_cache.sh
+docker compose -f docker-compose.offline.yml up -d
+```
+
+See [docs/guides/OFFLINE_DOCKER.md](docs/guides/OFFLINE_DOCKER.md) for detailed instructions.
+
 ## Data Format
 
 ### Input Dataset (JSON)
@@ -241,7 +288,8 @@ tc-001,"질문","답변","[""컨텍스트1"",""컨텍스트2""]","정답"
 | RagasEvaluator | ✅ Complete | 6 metrics (Ragas v1.0) |
 | LLM Adapters | ✅ Complete | OpenAI, Ollama, Azure, Anthropic |
 | Storage Adapters | ✅ Complete | SQLite, PostgreSQL |
-| Tracker Adapters | ✅ Complete | Langfuse, MLflow |
+| Tracker Adapters | ✅ Complete | Langfuse, MLflow, Phoenix (dual-tracker) |
+| Offline Bundle | ✅ Complete | Ollama/vLLM, Docker images, model cache |
 | CLI | ✅ Complete | run, metrics, config, history, compare, export, generate, pipeline, benchmark |
 | Testset Generation | ✅ Complete | Basic + Knowledge Graph |
 | Experiment Management | ✅ Complete | A/B testing, comparison |
@@ -265,6 +313,8 @@ tc-001,"질문","답변","[""컨텍스트1"",""컨텍스트2""]","정답"
 | [docs/ROADMAP.md](docs/ROADMAP.md) | 2026-2027 개발 로드맵 (Phase 15-19+) |
 | [docs/IMPROVEMENT_PLAN.md](docs/IMPROVEMENT_PLAN.md) | 코드 품질 개선 계획 (P1-P7 우선순위, 병렬 에이전트 워크플로우) |
 | [docs/KG_IMPROVEMENT_PLAN.md](docs/KG_IMPROVEMENT_PLAN.md) | Knowledge Graph 개선 계획 |
+| [docs/guides/OFFLINE_DOCKER.md](docs/guides/OFFLINE_DOCKER.md) | 오프라인 Docker 배포 가이드 |
+| [docs/guides/OFFLINE_MODELS.md](docs/guides/OFFLINE_MODELS.md) | 오프라인 모델 번들링 가이드 |
 | [agent/README.md](agent/README.md) | 자율 에이전트 시스템 사용 가이드 |
 
 ## Autonomous Agent System

{evalvault-1.76.0 → evalvault-1.77.0}/Dockerfile

@@ -18,15 +18,15 @@ WORKDIR /app
 # Copy dependency files and README (required by pyproject.toml)
 COPY pyproject.toml uv.lock README.md ./
 
-# Install dependencies
-RUN uv sync --frozen --no-dev --no-install-project
+# Install dependencies (include postgres for PostgreSQL support)
+RUN uv sync --frozen --no-dev --no-install-project --extra postgres
 
 # Copy source code
 COPY src/ ./src/
 COPY config/ ./config/
 
 # Install the project
-RUN uv sync --frozen --no-dev
+RUN uv sync --frozen --no-dev --extra postgres
 
 
 # Stage 2: Runtime stage

{evalvault-1.76.0 → evalvault-1.77.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: evalvault
-Version: 1.76.0
+Version: 1.77.0
 Summary: RAG evaluation system using Ragas with Phoenix/Langfuse tracing
 Project-URL: Homepage, https://github.com/ntts9990/EvalVault
 Project-URL: Documentation, https://github.com/ntts9990/EvalVault#readme
@@ -66,6 +66,7 @@ Requires-Dist: ijson>=3.3.0; extra == 'dev'
 Requires-Dist: kiwipiepy>=0.18.0; extra == 'dev'
 Requires-Dist: langchain-anthropic; extra == 'dev'
 Requires-Dist: lm-eval[api]>=0.4.0; extra == 'dev'
+Requires-Dist: manim>=0.18.0; extra == 'dev'
 Requires-Dist: mkdocs-material>=9.5.0; extra == 'dev'
 Requires-Dist: mkdocs>=1.5.0; extra == 'dev'
 Requires-Dist: mkdocstrings[python]>=0.24.0; extra == 'dev'

{evalvault-1.76.0 → evalvault-1.77.0}/config/models.yaml

@@ -85,6 +85,18 @@ profiles:
       provider: vllm
       model: qwen3-embedding:0.6b
 
+  # ──────────────────────────────────────────────────────────────
+  # Qwen3 14B (보고서 작성용)
+  # ──────────────────────────────────────────────────────────────
+  qwen3-14b:
+    description: "보고서 작성용 Qwen3 14B"
+    llm:
+      provider: ollama
+      model: qwen3:14b
+    embedding:
+      provider: ollama
+      model: qwen3-embedding:0.6b
+
 # ──────────────────────────────────────────────────────────────
 # 참고: 인프라 설정
 # ──────────────────────────────────────────────────────────────

evalvault-1.77.0/docker-compose.offline.build.yml

@@ -0,0 +1,28 @@
+# EvalVault Offline Build Overrides
+# Online build only. Do not use in air-gapped runtime.
+#
+# IMPORTANT: Builds linux/amd64 images for cross-platform compatibility.
+# Apple Silicon Macs will build amd64 images that run on Ubuntu/Intel.
+
+services:
+  evalvault-api:
+    platform: linux/amd64
+    build:
+      context: .
+      dockerfile: Dockerfile
+      platforms:
+        - linux/amd64
+      args:
+        PYTHON_IMAGE: ${EVALVAULT_PYTHON_IMAGE:-python:3.12.6-slim}
+        UV_IMAGE: ${EVALVAULT_UV_IMAGE:-ghcr.io/astral-sh/uv:0.4.28}
+
+  evalvault-web:
+    platform: linux/amd64
+    build:
+      context: ./frontend
+      dockerfile: Dockerfile
+      platforms:
+        - linux/amd64
+      args:
+        NODE_IMAGE: ${EVALVAULT_NODE_IMAGE:-node:20.19-alpine}
+        NGINX_IMAGE: ${EVALVAULT_NGINX_IMAGE:-nginx:1.27.3-alpine}

{evalvault-1.76.0 → evalvault-1.77.0}/docker-compose.offline.yml

@@ -1,9 +1,9 @@
 # EvalVault Offline Deployment - Docker Compose Configuration
 # Air-gapped environment: PostgreSQL + EvalVault API + Web UI
 #
-# Usage:
+# Usage (air-gapped runtime only; no build):
 #   docker compose -f docker-compose.offline.yml --env-file .env.offline.example config
-#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d
+#   docker compose -f docker-compose.offline.yml --env-file .env.offline up -d --no-build
 #
 # Prerequisites:
 #   - External LLM server (Ollama/vLLM) must be accessible within the air-gapped network
@@ -18,6 +18,7 @@ services:
   # PostgreSQL database for evaluation storage
   postgres:
     image: ${POSTGRES_IMAGE:-pgvector/pgvector:0.8.0-pg16}
+    platform: linux/amd64
     container_name: evalvault-postgres
     pull_policy: never
     restart: unless-stopped
@@ -38,12 +39,7 @@ services:
   # EvalVault API service (FastAPI backend)
   evalvault-api:
     image: evalvault-api:offline
-    build:
-      context: .
-      dockerfile: Dockerfile
-      args:
-        PYTHON_IMAGE: ${EVALVAULT_PYTHON_IMAGE:-python:3.12.6-slim}
-        UV_IMAGE: ${EVALVAULT_UV_IMAGE:-ghcr.io/astral-sh/uv:0.4.28}
+    platform: linux/amd64
     container_name: evalvault-api
     restart: unless-stopped
     pull_policy: never
@@ -75,6 +71,18 @@ services:
       # Faithfulness fallback (optional)
       FAITHFULNESS_FALLBACK_PROVIDER: ${FAITHFULNESS_FALLBACK_PROVIDER:-}
       FAITHFULNESS_FALLBACK_MODEL: ${FAITHFULNESS_FALLBACK_MODEL:-}
+      # OpenAI (closed-source, requires proxy access)
+      OPENAI_API_KEY: ${OPENAI_API_KEY:-}
+      OPENAI_MODEL: ${OPENAI_MODEL:-}
+      OPENAI_BASE_URL: ${OPENAI_BASE_URL:-}
+      # Anthropic (closed-source, requires proxy access)
+      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
+      ANTHROPIC_MODEL: ${ANTHROPIC_MODEL:-}
+      # Azure OpenAI (closed-source, requires proxy access)
+      AZURE_OPENAI_API_KEY: ${AZURE_OPENAI_API_KEY:-}
+      AZURE_OPENAI_ENDPOINT: ${AZURE_OPENAI_ENDPOINT:-}
+      AZURE_OPENAI_DEPLOYMENT: ${AZURE_OPENAI_DEPLOYMENT:-}
+      AZURE_OPENAI_API_VERSION: ${AZURE_OPENAI_API_VERSION:-}
       # API authentication (optional)
       API_AUTH_TOKENS: ${API_AUTH_TOKENS:-}
       KNOWLEDGE_READ_TOKENS: ${KNOWLEDGE_READ_TOKENS:-}
@@ -86,7 +94,7 @@ services:
     # Override entrypoint to run API server
     command: ["serve-api", "--host", "0.0.0.0", "--port", "8000"]
     healthcheck:
-      test: ["CMD-SHELL", "curl -f http://localhost:8000/health || exit 1"]
+      test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8000/health')\" || exit 1"]
       interval: 30s
       timeout: 10s
       start_period: 10s
@@ -98,12 +106,7 @@ services:
   # Nginx proxies /api/ to evalvault-api:8000
   evalvault-web:
     image: evalvault-web:offline
-    build:
-      context: ./frontend
-      dockerfile: Dockerfile
-      args:
-        NODE_IMAGE: ${EVALVAULT_NODE_IMAGE:-node:20.19-alpine}
-        NGINX_IMAGE: ${EVALVAULT_NGINX_IMAGE:-nginx:1.27.3-alpine}
+    platform: linux/amd64
     container_name: evalvault-web
     restart: unless-stopped
     pull_policy: never

evalvault-1.77.0/docs/guides/DI_EXTRACTION_NEXT_STEPS.md

@@ -0,0 +1,162 @@
+# 의존성 주입/모듈 분리 전체 계획서
+
+이 문서는 모듈 이식성을 높이기 위한 전체 계획과 상세 계획을 동시에 제공한다.
+현재 단계에서는 설계와 범위를 확정하고, 단계별 리팩터링의 기준을 문서화한다.
+
+## 목표
+- Settings 의존을 진입점(Entry point)으로만 제한
+- Storage/LLM/Tracker 생성 책임을 외부로 이동
+- 도메인/포트 계층을 별도 패키지로 분리 가능한 구조로 정리
+- 기능/동작의 하위 호환을 유지하며 점진적 전환
+
+## 성공 기준 (Definition of Done)
+- 모든 도메인 서비스가 Settings를 직접 참조하지 않음
+- 모든 inbound 계층에서 Settings 직접 생성 제거
+- 어댑터 생성은 entrypoint에서만 수행
+- 도메인/포트 패키지 분리 후에도 기존 CLI/API 동작 유지
+- 테스트/CI 정상 통과
+
+## 전체 단계 요약
+1) PR1: 컨테이너 스캐폴딩 + 주입 경로 추가 (하위 호환)
+2) PR2: CLI commands에서 Settings 직접 호출 제거
+3) PR3: API adapter에서 Settings 직접 호출 제거
+4) PR4: Storage/LLM/Tracker factory 호출 외부화
+5) PR5: 도메인/포트 분리 패키징
+
+## 상세 계획
+
+### PR1: 컨테이너 스캐폴딩 + 주입 경로 추가
+**목표**: 외부 주입 경로를 만들고 기존 동작은 유지한다.
+
+**신규 파일**
+- `src/evalvault/app/context.py`
+- `src/evalvault/app/container.py`
+
+**컨테이너 인터페이스**
+- `AppContext` (dataclass)
+  - settings: Settings
+  - storage: StoragePort
+  - llm: LLMPort
+  - tracker: TrackerPort
+  - domain_memory: DomainMemoryPort
+
+- `build_app_context(settings: Settings | None = None) -> AppContext`
+  - settings가 없으면 Settings 생성
+  - 내부 factory로 의존성 생성 (임시)
+
+**수정 대상 (주입 경로 추가)**
+- `src/evalvault/adapters/inbound/cli/commands/run.py`
+  - 기존 동작 유지, `context: AppContext | None = None` 옵션 추가
+- `src/evalvault/adapters/inbound/api/adapter.py`
+  - 기존 동작 유지, `context` 주입 경로 추가
+- `src/evalvault/adapters/inbound/mcp/tools.py`
+  - context에서 storage/llm 사용
+
+**검증**
+- 기존 CLI/API 사용 방식 그대로 동작
+- context 주입 시 동일 결과
+
+---
+
+### PR2: CLI Settings 직접 호출 제거
+**목표**: CLI 계층에서 Settings 직접 생성 제거
+
+**변경 방향**
+- CLI entrypoint에서 Settings/Context 생성
+- commands는 settings 대신 context에서 의존성 사용
+
+**주요 파일**
+- `src/evalvault/adapters/inbound/cli/commands/*.py`
+- `src/evalvault/adapters/inbound/cli/app.py`
+
+**검증**
+- CLI 기능 전체 동작 유지
+- 환경변수 변경 시 동일 결과
+
+---
+
+### PR3: API Settings 직접 호출 제거
+**목표**: API 계층에서 Settings 직접 생성 제거
+
+**변경 방향**
+- FastAPI create_app에서 Settings/Context 생성
+- routers/adapters에서 context를 통해 접근
+
+**주요 파일**
+- `src/evalvault/adapters/inbound/api/main.py`
+- `src/evalvault/adapters/inbound/api/adapter.py`
+- `src/evalvault/adapters/inbound/api/routers/*.py`
+
+**검증**
+- API 기능 전체 동작 유지
+- CORS/Rate limit/Auth 설정 유지
+
+---
+
+### PR4: Factory 호출 외부화
+**목표**: 내부 factory 의존 제거 (external wiring)
+
+**변경 방향**
+- build_storage_adapter/build_domain_memory_adapter를 entrypoint에서 호출
+- 서비스/핸들러는 포트 인터페이스만 받음
+
+**주요 파일**
+- `src/evalvault/adapters/outbound/storage/factory.py`
+- `src/evalvault/adapters/outbound/domain_memory/factory.py`
+- `src/evalvault/adapters/outbound/llm/factory.py`
+- `src/evalvault/adapters/inbound/*` (호출 제거)
+
+**검증**
+- 동일 설정으로 동일한 어댑터 생성
+- 테스트 통과
+
+---
+
+### PR5: 도메인/포트 분리 패키징
+**목표**: 모듈 분리 구조 완성
+
+**제안 구조**
+```
+evalvault-core/
+  domain/
+  ports/
+  services/
+
+evalvault-adapters/
+  outbound/
+  inbound/
+
+evalvault-app/
+  cli/
+  api/
+  wiring/
+```
+
+**패키징 원칙**
+- core는 외부 라이브러리 의존 최소화
+- adapters는 core에 의존하지만 core는 adapters를 모름
+- app은 core+adapters를 묶는 역할
+
+**검증**
+- core만 별도 설치 가능
+- adapters/app에서 기존 CLI/API 동작 유지
+
+---
+
+## 리스크 및 대응
+- 리스크: Settings 직접 접근 제거 과정에서 옵션 누락
+  - 대응: PR마다 테스트/스냅샷 비교
+- 리스크: CLI/APIs에서 초기화 순서 문제
+  - 대응: AppContext 초기화 순서 문서화
+- 리스크: 외부 의존성 분리 시 import 경로 붕괴
+  - 대응: 단계적 패키지 이동
+
+## 테스트 전략
+- PR1~PR3: 기존 테스트 그대로 통과
+- PR4: storage/llm/tracker 관련 unit/integration 테스트 추가
+- PR5: core 패키지 단독 설치 테스트
+
+## 결정 필요 항목
+- 컨테이너 네이밍: `AppContext` vs `AppContainer`
+- 외부 wiring 위치: CLI entrypoint vs API create_app
+- 분리 레포 구조(모노레포 vs 멀티레포)