slidesherlock 1.1.0__tar.gz

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/README.md

@@ -0,0 +1,239 @@
+# SlideSherlock
+
+Turn a PPTX into a narrated explainer video with visual guidance (highlight/trace/zoom), while preventing hallucinations using an Evidence Index + Verifier.
+
+## Architecture
+
+SlideSherlock is built with an **artifact-first pipeline** where every stage writes outputs to S3/MinIO with stable paths and updates Postgres status. For a full pipeline diagram, stage breakdown, and **recommended next steps** (testing, production hardening, features, deployment), see **[ARCHITECTURE.md](ARCHITECTURE.md)**.
+
+### Key Components
+
+1. **Evidence Index**: Tracks all evidence with `evidence_id` + `source_ref` (bbox / ppt shape id / page+char offsets)
+2. **Diagram Understanding**: 
+   - `G_native` from PPT objects (shapes/connectors/groups)
+   - Optional `G_vision` from PNG + OCR
+   - Merged to `G_unified` with provenance (NATIVE/VISION/BOTH) + confidence + NEEDS_REVIEW flags
+3. **Script Generation**: Produces segments with `claim_id`, `evidence_ids`, `entity_ids`
+4. **Verifier**: Enforces grounding with PASS/REWRITE/REMOVE, loops until no REWRITE remains
+5. **Timeline Builder**: Generates HIGHLIGHT/TRACE/ZOOM actions mapped to entity geometry
+6. **Renderer**: Generates overlays and composes final video with FFmpeg
+
+## Repository Structure
+
+```
+/slidesherlock
+  /apps
+    /api          - REST API for job submission and status
+    /worker       - Pipeline worker that processes jobs
+  /packages
+    /core         - Core business logic (diagram, script, verifier, timeline)
+    /providers    - LLM/TTS/OCR provider interfaces
+    /schemas      - TypeScript schemas and types
+  /infra
+    /docker       - Docker configurations
+  docker-compose.yml
+  Makefile
+  README.md
+```
+
+## Prerequisites
+
+### System Dependencies
+
+- **Python 3.11+** (3.12 recommended)
+- **Docker and Docker Compose** (for PostgreSQL, Redis, MinIO)
+- **LibreOffice** (for PPTX to PDF conversion)
+  - macOS: `brew install --cask libreoffice`
+  - Ubuntu/Debian: `sudo apt-get install libreoffice`
+  - CentOS/RHEL: `sudo yum install libreoffice`
+- **Poppler** (for PDF to PNG conversion, required by pdf2image)
+  - macOS: `brew install poppler`
+  - Ubuntu/Debian: `sudo apt-get install poppler-utils`
+  - CentOS/RHEL: `sudo yum install poppler-utils`
+- **FFmpeg** (for video composition - future stage)
+
+### Python Dependencies
+
+All Python dependencies are consolidated in `requirements.txt` and will be installed automatically with `make setup`.
+
+## Quick Start
+
+1. **Start infrastructure services:**
+   ```bash
+   make up
+   ```
+
+2. **Install dependencies:**
+   ```bash
+   make install
+   ```
+
+3. **Build all packages:**
+   ```bash
+   make build
+   ```
+
+4. **Initialize database:**
+   ```bash
+   make migrate
+   ```
+
+5. **Start the API server (in one terminal):**
+   ```bash
+   make api
+   ```
+
+6. **Start the worker (in another terminal):**
+   ```bash
+   make worker
+   ```
+
+## Usage
+
+### Submit a Job
+
+```bash
+curl -X POST http://localhost:3000/jobs \
+  -F "file=@presentation.pptx"
+```
+
+Response:
+```json
+{
+  "job_id": "uuid-here",
+  "status": "PENDING",
+  "message": "Job submitted successfully"
+}
+```
+
+### Check Job Status
+
+```bash
+curl http://localhost:3000/jobs/{job_id}
+```
+
+### Get Artifacts
+
+```bash
+# Get script
+curl http://localhost:3000/jobs/{job_id}/artifacts/script
+
+# Get verify report
+curl http://localhost:3000/jobs/{job_id}/artifacts/verify_report
+
+# Get timeline
+curl http://localhost:3000/jobs/{job_id}/artifacts/timeline
+
+# Get final video
+curl http://localhost:3000/jobs/{job_id}/artifacts/final_video -o output.mp4
+```
+
+## Configuration
+
+### Environment Variables
+
+**API Server:**
+- `PORT` - API server port (default: 3000)
+- `DATABASE_URL` - PostgreSQL connection string
+- `REDIS_URL` - Redis connection string
+- `MINIO_ENDPOINT` - MinIO endpoint URL
+- `MINIO_ACCESS_KEY` - MinIO access key
+- `MINIO_SECRET_KEY` - MinIO secret key
+- `MINIO_BUCKET` - MinIO bucket name
+
+**Worker:**
+- `DATABASE_URL` - PostgreSQL connection string
+- `REDIS_URL` - Redis connection string
+- `MINIO_ENDPOINT` - MinIO endpoint URL
+- `MINIO_ACCESS_KEY` - MinIO access key
+- `MINIO_SECRET_KEY` - MinIO secret key
+- `MINIO_BUCKET` - MinIO bucket name
+- `OPENAI_API_KEY` - OpenAI API key (optional; see [Secrets](#secrets-never-commit) below)
+- `USE_SYSTEM_TTS` - Set to "true" to use system TTS (macOS only)
+
+### Secrets (never commit)
+
+API keys and secrets must **not** be committed. Use environment variables or a local `.env` file.
+
+1. **Copy the example file** (no secrets inside):
+   ```bash
+   cp .env.example .env
+   ```
+2. **Edit `.env`** and set only the keys you need (e.g. `OPENAI_API_KEY=sk-...`).
+3. **`.env` is in `.gitignore`** – it will not be pushed to GitHub.
+
+**OpenAI vision (optional):** For real diagram/photo understanding, set `OPENAI_API_KEY` in `.env` and `VISION_PROVIDER=openai`. If the key is not set, the pipeline uses the stub vision provider (generic captions) and logs that the key is missing.
+
+- **Enable:** `VISION_PROVIDER=openai` (and optionally `VISION_EXTRACTOR_PROVIDER=openai`).
+- **Config (env):** `OPENAI_VISION_MODEL` (default `gpt-4o`), `OPENAI_VISION_TEMPERATURE` (default `0`), `OPENAI_VISION_TIMEOUT_SECONDS` (default `60`).
+- **Caching:** Results are cached in MinIO by image hash + model + lang + prompt version to avoid repeat API charges. Set `VISION_CACHE_ENABLED=false` to disable; cache path is `VISION_CACHE_PREFIX` (default `jobs/{job_id}/cache/vision/`).
+- **Costs:** Each uncached image uses one vision API call; enable caching for development and re-runs. Outputs are validated and stored as evidence (used by script/verifier in later pipeline stages).
+
+## Pipeline Stages
+
+1. **EXTRACTING**: Extract content from PPTX, convert slides to PNG
+2. **DIAGRAM_ANALYSIS**: Build evidence index and analyze diagram structure
+3. **SCRIPT_GENERATION**: Generate narrated script with evidence grounding
+4. **VERIFICATION**: Verify and rewrite script until all segments pass
+5. **TIMELINE_BUILDING**: Generate timeline with visual actions
+6. **RENDERING**: Generate overlays and compose final video
+7. **COMPLETED**: Job finished successfully
+
+## Artifact Paths
+
+All artifacts are stored in MinIO with the following structure:
+
+```
+jobs/{job_id}/
+  input.pptx
+  extracted.json
+  slides/
+    slide-0.png
+    slide-1.png
+  evidence/
+    index.json
+  graph/
+    native.json
+    vision.json (optional)
+    unified.json
+  script.json
+  verify_report.json
+  coverage.json
+  timeline/
+    timeline.json
+  overlays/
+    {action_id}.png
+  audio/
+    {segment_id}.wav
+  final.mp4
+```
+
+## Testing
+
+Run tests:
+```bash
+make test
+```
+
+## Development
+
+### No-Provider Mode
+
+By default, SlideSherlock runs in "no-provider mode" where:
+- LLM provider returns placeholder responses
+- TTS provider returns empty audio
+- OCR provider returns empty results
+
+This allows testing the pipeline structure without external API dependencies.
+
+### Adding Providers
+
+To use real providers:
+
+1. **OpenAI LLM**: Set `OPENAI_API_KEY` environment variable
+2. **System TTS**: Set `USE_SYSTEM_TTS=true` (macOS only)
+3. **OCR**: Implement a real OCR provider in `packages/providers/src/ocr.ts`
+
+## License
+
+MIT

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/env.py

@@ -0,0 +1,103 @@
+from logging.config import fileConfig
+from sqlalchemy import engine_from_config
+from sqlalchemy import pool
+from alembic import context
+import os
+import sys
+
+# Add project root to path
+project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+sys.path.insert(0, project_root)
+
+# Import using absolute imports from the apps.api package
+from apps.api.database import Base  # noqa: E402
+
+# Import models so Alembic can detect them for autogenerate
+from apps.api.models import (  # noqa: E402, F401
+    Project,
+    Job,
+    Artifact,
+    Slide,
+    Source,
+    EvidenceItem,
+    SourceRef,
+    ClaimLink,
+    EntityLink,
+)
+
+# this is the Alembic Config object, which provides
+# access to the values within the .ini file in use.
+config = context.config
+
+# Interpret the config file for Python logging.
+# This line sets up loggers basically.
+if config.config_file_name is not None:
+    fileConfig(config.config_file_name)
+
+# add your model's MetaData object here
+# for 'autogenerate' support
+target_metadata = Base.metadata
+
+# other values from the config, defined by the needs of env.py,
+# can be acquired:
+# my_important_option = config.get_main_option("my_important_option")
+# ... etc.
+
+
+def get_url():
+    return os.getenv(
+        "DATABASE_URL",
+        "postgresql://slidesherlock:slidesherlock@localhost:5433/slidesherlock",
+    )
+
+
+def run_migrations_offline() -> None:
+    """Run migrations in 'offline' mode.
+
+    This configures the context with just a URL
+    and not an Engine, though an Engine is acceptable
+    here as well.  By skipping the Engine creation
+    we don't even need a DBAPI to be available.
+
+    Calls to context.execute() here emit the given string to the
+    script output.
+
+    """
+    url = get_url()
+    context.configure(
+        url=url,
+        target_metadata=target_metadata,
+        literal_binds=True,
+        dialect_opts={"paramstyle": "named"},
+    )
+
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+def run_migrations_online() -> None:
+    """Run migrations in 'online' mode.
+
+    In this scenario we need to create an Engine
+    and associate a connection with the context.
+
+    """
+    configuration = config.get_section(config.config_ini_section)
+    configuration["sqlalchemy.url"] = get_url()
+    connectable = engine_from_config(
+        configuration,
+        prefix="sqlalchemy.",
+        poolclass=pool.NullPool,
+    )
+
+    with connectable.connect() as connection:
+        context.configure(connection=connection, target_metadata=target_metadata)
+
+        with context.begin_transaction():
+            context.run_migrations()
+
+
+if context.is_offline_mode():
+    run_migrations_offline()
+else:
+    run_migrations_online()

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/script.py.mako

@@ -0,0 +1,24 @@
+"""${message}
+
+Revision ID: ${up_revision}
+Revises: ${down_revision | comma,n}
+Create Date: ${create_date}
+
+"""
+from alembic import op
+import sqlalchemy as sa
+${imports if imports else ""}
+
+# revision identifiers, used by Alembic.
+revision = ${repr(up_revision)}
+down_revision = ${repr(down_revision)}
+branch_labels = ${repr(branch_labels)}
+depends_on = ${repr(depends_on)}
+
+
+def upgrade() -> None:
+    ${upgrades if upgrades else "pass"}
+
+
+def downgrade() -> None:
+    ${downgrades if downgrades else "pass"}

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/versions/001_initial_schema.py

@@ -0,0 +1,78 @@
+"""Initial schema: project, job, artifact
+
+Revision ID: 001
+Revises:
+Create Date: 2024-01-01 00:00:00.000000
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+# revision identifiers, used by Alembic.
+revision = "001"
+down_revision = None
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    # Create projects table
+    op.create_table(
+        "projects",
+        sa.Column("project_id", sa.String(), nullable=False),
+        sa.Column("name", sa.String(), nullable=False),
+        sa.Column("description", sa.Text(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+        sa.PrimaryKeyConstraint("project_id"),
+    )
+
+    # Create jobs table
+    op.create_table(
+        "jobs",
+        sa.Column("job_id", sa.String(), nullable=False),
+        sa.Column("project_id", sa.String(), nullable=False),
+        sa.Column(
+            "status",
+            sa.Enum("PENDING", "PROCESSING", "DONE", "FAILED", name="jobstatus"),
+            nullable=False,
+        ),
+        sa.Column("input_file_path", sa.String(), nullable=True),
+        sa.Column("error_message", sa.Text(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+        sa.ForeignKeyConstraint(
+            ["project_id"],
+            ["projects.project_id"],
+        ),
+        sa.PrimaryKeyConstraint("job_id"),
+    )
+
+    # Create artifacts table
+    op.create_table(
+        "artifacts",
+        sa.Column("artifact_id", sa.String(), nullable=False),
+        sa.Column("project_id", sa.String(), nullable=False),
+        sa.Column("job_id", sa.String(), nullable=True),
+        sa.Column("artifact_type", sa.String(), nullable=False),
+        sa.Column("storage_path", sa.String(), nullable=False),
+        sa.Column("metadata_json", sa.Text(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.ForeignKeyConstraint(
+            ["project_id"],
+            ["projects.project_id"],
+        ),
+        sa.ForeignKeyConstraint(
+            ["job_id"],
+            ["jobs.job_id"],
+        ),
+        sa.PrimaryKeyConstraint("artifact_id"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("artifacts")
+    op.drop_table("jobs")
+    op.drop_table("projects")
+    op.execute("DROP TYPE IF EXISTS jobstatus")

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/versions/002_add_job_status_and_artifact_fields.py

@@ -0,0 +1,60 @@
+"""Add QUEUED and RUNNING job status, add sha256 and size_bytes to artifacts
+
+Revision ID: 002
+Revises: 001
+Create Date: 2024-01-27 12:00:00.000000
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+# revision identifiers, used by Alembic.
+revision = "002"
+down_revision = "001"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    # Update JobStatus enum to include QUEUED and RUNNING
+    # PostgreSQL doesn't support IF NOT EXISTS for ALTER TYPE, so we use a DO block
+    op.execute(
+        """
+        DO $$ BEGIN
+            IF NOT EXISTS (
+                SELECT 1 FROM pg_enum
+                WHERE enumlabel = 'QUEUED'
+                AND enumtypid = (SELECT oid FROM pg_type WHERE typname = 'jobstatus')
+            ) THEN
+                ALTER TYPE jobstatus ADD VALUE 'QUEUED';
+            END IF;
+        END $$;
+        """
+    )
+    op.execute(
+        """
+        DO $$ BEGIN
+            IF NOT EXISTS (
+                SELECT 1 FROM pg_enum
+                WHERE enumlabel = 'RUNNING'
+                AND enumtypid = (SELECT oid FROM pg_type WHERE typname = 'jobstatus')
+            ) THEN
+                ALTER TYPE jobstatus ADD VALUE 'RUNNING';
+            END IF;
+        END $$;
+        """
+    )
+
+    # Add sha256 and size_bytes columns to artifacts table
+    op.add_column("artifacts", sa.Column("sha256", sa.String(), nullable=True))
+    op.add_column("artifacts", sa.Column("size_bytes", sa.String(), nullable=True))
+
+
+def downgrade() -> None:
+    # Remove columns from artifacts table
+    op.drop_column("artifacts", "size_bytes")
+    op.drop_column("artifacts", "sha256")
+
+    # Note: PostgreSQL doesn't support removing enum values easily
+    # The enum values QUEUED and RUNNING will remain but won't be used

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/versions/003_evidence_index_tables.py

@@ -0,0 +1,112 @@
+"""Evidence index tables: slides, sources, evidence_items, source_refs, claim_links, entity_links
+
+Revision ID: 003
+Revises: 002
+Create Date: 2024-01-28 12:00:00.000000
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+revision = "003"
+down_revision = "002"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "slides",
+        sa.Column("slide_id", sa.String(), nullable=False),
+        sa.Column("job_id", sa.String(), nullable=False),
+        sa.Column("slide_index", sa.Integer(), nullable=False),
+        sa.Column("slide_title", sa.Text(), nullable=True),
+        sa.Column("png_artifact_id", sa.String(), nullable=True),
+        sa.Column("pptx_ref", sa.String(), nullable=True),
+        sa.ForeignKeyConstraint(["job_id"], ["jobs.job_id"]),
+        sa.ForeignKeyConstraint(["png_artifact_id"], ["artifacts.artifact_id"]),
+        sa.PrimaryKeyConstraint("slide_id"),
+    )
+
+    op.create_table(
+        "sources",
+        sa.Column("source_id", sa.String(), nullable=False),
+        sa.Column("job_id", sa.String(), nullable=False),
+        sa.Column("type", sa.String(), nullable=False),
+        sa.Column("artifact_id", sa.String(), nullable=True),
+        sa.Column("slide_id", sa.String(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.ForeignKeyConstraint(["job_id"], ["jobs.job_id"]),
+        sa.ForeignKeyConstraint(["artifact_id"], ["artifacts.artifact_id"]),
+        sa.ForeignKeyConstraint(["slide_id"], ["slides.slide_id"]),
+        sa.PrimaryKeyConstraint("source_id"),
+    )
+
+    op.create_table(
+        "evidence_items",
+        sa.Column("evidence_id", sa.String(), nullable=False),
+        sa.Column("job_id", sa.String(), nullable=False),
+        sa.Column("slide_id", sa.String(), nullable=True),
+        sa.Column("source_id", sa.String(), nullable=False),
+        sa.Column("kind", sa.String(), nullable=False),
+        sa.Column("content", sa.Text(), nullable=False),
+        sa.Column("content_hash", sa.String(), nullable=True),
+        sa.Column("confidence", sa.Float(), nullable=True),
+        sa.Column("language", sa.String(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.ForeignKeyConstraint(["job_id"], ["jobs.job_id"]),
+        sa.ForeignKeyConstraint(["slide_id"], ["slides.slide_id"]),
+        sa.ForeignKeyConstraint(["source_id"], ["sources.source_id"]),
+        sa.PrimaryKeyConstraint("evidence_id"),
+    )
+
+    op.create_table(
+        "source_refs",
+        sa.Column("ref_id", sa.String(), nullable=False),
+        sa.Column("evidence_id", sa.String(), nullable=False),
+        sa.Column("ref_type", sa.String(), nullable=False),
+        sa.Column("slide_index", sa.Integer(), nullable=True),
+        sa.Column("ppt_shape_id", sa.String(), nullable=True),
+        sa.Column("ppt_paragraph_ix", sa.Integer(), nullable=True),
+        sa.Column("ppt_run_ix", sa.Integer(), nullable=True),
+        sa.Column("bbox_x", sa.Float(), nullable=True),
+        sa.Column("bbox_y", sa.Float(), nullable=True),
+        sa.Column("bbox_w", sa.Float(), nullable=True),
+        sa.Column("bbox_h", sa.Float(), nullable=True),
+        sa.Column("page_num", sa.Integer(), nullable=True),
+        sa.Column("char_start", sa.Integer(), nullable=True),
+        sa.Column("char_end", sa.Integer(), nullable=True),
+        sa.Column("url", sa.Text(), nullable=True),
+        sa.ForeignKeyConstraint(["evidence_id"], ["evidence_items.evidence_id"]),
+        sa.PrimaryKeyConstraint("ref_id"),
+    )
+
+    op.create_table(
+        "claim_links",
+        sa.Column("claim_link_id", sa.String(), nullable=False),
+        sa.Column("claim_id", sa.String(), nullable=False),
+        sa.Column("evidence_id", sa.String(), nullable=False),
+        sa.Column("weight", sa.Float(), nullable=True),
+        sa.ForeignKeyConstraint(["evidence_id"], ["evidence_items.evidence_id"]),
+        sa.PrimaryKeyConstraint("claim_link_id"),
+    )
+
+    op.create_table(
+        "entity_links",
+        sa.Column("entity_link_id", sa.String(), nullable=False),
+        sa.Column("entity_id", sa.String(), nullable=False),
+        sa.Column("evidence_id", sa.String(), nullable=False),
+        sa.Column("role", sa.String(), nullable=True),
+        sa.ForeignKeyConstraint(["evidence_id"], ["evidence_items.evidence_id"]),
+        sa.PrimaryKeyConstraint("entity_link_id"),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("entity_links")
+    op.drop_table("claim_links")
+    op.drop_table("source_refs")
+    op.drop_table("evidence_items")
+    op.drop_table("sources")
+    op.drop_table("slides")

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/versions/004_add_requested_language.py

@@ -0,0 +1,23 @@
+"""Add requested_language to jobs
+
+Revision ID: 004
+Revises: 003
+Create Date: 2024-01-31 12:00:00.000000
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+revision = "004"
+down_revision = "003"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.add_column("jobs", sa.Column("requested_language", sa.String(), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column("jobs", "requested_language")

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/alembic/versions/005_add_job_config_json.py

@@ -0,0 +1,23 @@
+"""Add config_json to jobs (vision config, etc.)
+
+Revision ID: 005
+Revises: 004
+Create Date: 2024-01-31 14:00:00.000000
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+revision = "005"
+down_revision = "004"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.add_column("jobs", sa.Column("config_json", sa.Text(), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column("jobs", "config_json")

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/apps/api/__init__.py

@@ -0,0 +1 @@
+# API package

slidesherlock-1.1.0/.claude/worktrees/agent-a53fd5f5/apps/api/database.py

@@ -0,0 +1,17 @@
+from sqlalchemy import create_engine
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy.orm import sessionmaker
+import os
+from dotenv import load_dotenv
+
+load_dotenv()
+
+DATABASE_URL = os.getenv(
+    "DATABASE_URL",
+    "postgresql://slidesherlock:slidesherlock@localhost:5433/slidesherlock",
+)
+
+engine = create_engine(DATABASE_URL)
+SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
+
+Base = declarative_base()