camera-llm 0.1.1__tar.gz

camera_llm-0.1.1/PKG-INFO

@@ -0,0 +1,145 @@
+Metadata-Version: 2.4
+Name: camera-llm
+Version: 0.1.1
+Summary: A local application to feed images/videos from a webcam into local LLMs for analysis and conversations.
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: PySide6>=6.6.0
+Requires-Dist: opencv-python>=4.9.0
+Requires-Dist: openai>=1.30.0
+Requires-Dist: numpy>=1.26.0
+Requires-Dist: pyqtdarktheme==0.1.7
+
+# Camera → Local LLM Inference App
+
+## Description
+
+A Python desktop app that captures images or short video clips from a webcam, optionally crops them, and sends them **in-memory** (no external file needed to be saved previously) to a local LLM via LM Studio for conversational analysis.
+
+---
+
+## Project Structure (Main Files)
+
+```
+Camera_LLM_Inference/
+└── camera_llm/
+       ├── __init__.py
+       ├── camera_thread.py         # QThread for OpenCV camera capture
+       ├── chat_session.py          # ChatSession dataclass + JSON serialisation
+       ├── chat_store.py            # Read/write chat sessions to chats/*.json
+       ├── cli.py                   # Entry point handling CLI and launching app (app initialization)
+       ├── llm_client.py            # OpenAI client → LM Studio, in-memory encode
+       ├── main_window.py           # Main window that aids navigation among the 7 screens
+       ├── styles.py                # Global stylesheet + design tokens     
+       └── screens/
+           ├── __init__.py
+           ├── screen1_home.py          # Dashboard + saved chats panel
+           ├── screen2_capture.py       # Live camera feed, capture/record controls
+           ├── screen3_crop.py          # Rubber-band crop with dimming overlay
+           ├── screen4_model_select.py  # LM Studio URL + model dropdown
+           ├── screen5_chat.py          # Chatbot with streaming, thumbnails, bubbles
+           ├── screen6_save.py          # Name & save the session
+           └── screen7_done.py          # Confirmation + auto-redirect home
+└── chats/                           # Auto-created at runtime for saved sessions
+├── pyproject.toml                   # Package metadata + entry points
+├── requirements.txt                 # pip dependencies
+```
+
+---
+
+## How to Run
+
+```bash
+cd "c:\Users\kurei\Documents\Machine_Deep Learning\Camera_LLM_Inference"
+pip install -e .
+camera-llm run
+```
+
+### Prerequisites
+1. **Webcam** connected or **IP camera** with reachable IP address (e.g. `http://[IP_ADDRESS]`)
+2. **LM Studio** running with a **vision model** loaded (e.g. LLaVA, Qwen-VL)
+3. LM Studio **local server started** (default: `http://localhost:1234`)
+
+---
+
+## Optional (download distribution zip file)
+
+Alternatively, you can [Download the Distribution Zip File to Run as Standalone Application](https://drive.proton.me/urls/ASD3RNEVY0#txhcUP2W8rzJ). This saves you the time of downloading the repo and installing the other dependencies, but it also takes around 2.5 GB of disk space (in addition to less frequent updates). Once downloaded, unzip the file then navigate to "\CameraLLMInference\CameraLLMInference.exe" to run the application.
+
+---
+
+## User Flow
+
+```mermaid
+graph TD;
+    S1["Screen 1: Home"] -->|Image| S2I["Screen 2: Camera (Image)"];
+    S1 -->|Video| S2V["Screen 2: Camera (Video)"];
+    S2I -->|Capture| S3["Screen 3: Crop"];
+    S2V -->|Stop recording| S3;
+    S3 -->|Use Full / Apply Crop| S4["Screen 4: Model Select"];
+    S4 -->|Analyse| S5["Screen 5: Chat"];
+    S5 -->|Finish| S6["Screen 6: Save?"];
+    S6 -->|Save / Skip| S7["Screen 7: Done"];
+    S7 -->|3s auto| S1;
+    S1 -->|Open saved chat| S5P["Screen 5: Chat (past chat)"];
+    S5P -->|Continue Chatting| S5;
+```
+
+---
+
+## Change Camera Feed Guide
+
+1. If you are using a standard USB webcam, you can just type 0, 1, 2, etc. and hit Enter. It will connect as a regular USB camera.
+2. If you want to use a smartphone, download a free app like IP Webcam (Android) or similar apps for iOS that broadcast your camera over WiFi. Start the server to use as a WiFi camera
+3. Note the URL that will be set (ex: http://192.168.1.100:8080).
+4. Paste that exact URL into the 'Camera' text box and press Enter.
+
+
+---
+
+## Key Design Decisions
+
+| Decision | Choice | Rationale |
+|---|---|---|
+| **GUI framework** | PySide6 | Robust threading (QThread), rich widget set, no licensing issues |
+| **Camera** | OpenCV in QThread | Non-blocking — GUI stays responsive at 30 fps |
+| **In-memory encoding** | `cv2.imencode` → base64 | No file ever touches disk; data goes straight to the LLM API |
+| **Video → LLM** | Sample up to 8 frames | Local VLMs don't accept video — sending evenly-spaced frames approximates it |
+| **LLM API** | `openai` client → `localhost:1234` | LM Studio is OpenAI-compatible; swapping to a cloud provider later is trivial |
+| **Chat persistence** | JSON files in `chats/` | Simple, portable, human-readable |
+
+---
+
+## Features
+
+- 7 Screens (Home, Capture, Crop, Model Select, Chat, Save, Done)
+- Image and Video Capture (With Rotate Feed Capability)
+- LLM Chat with Streaming
+- Save/Load Chat Sessions (Chats are listed with the last modified date and time)
+- Model Selection (LLM URL + Model Dropdown)
+- Delete or Rename Saved Chats
+> [!CAUTION]
+> Remember to load the same model that was used when first starting the chat (referenced in the title)
+
+> [!NOTE]
+> The video input feature works by sending multiple frames from the video to the LLM in order to process as multiple images. The multiple frames (images) are sent as a panel (chained images) so the LLM can interpret it as one image (full context). This is done primarily due to employing a local LLM.
+
+## What Was Verified
+
+- ✅ All dependencies install cleanly
+- ✅ All module imports resolve without errors
+- ✅ App launches and renders Screen 1 correctly
+- ✅ Fixed `pyqtdarktheme` API for v0.1.7 (`load_stylesheet` instead of `setup_theme`)
+
+## Physical Testing Passed
+- 📷 Camera feed with a physical webcam
+- 🎬 Video recording and frame sampling
+- 🤖 End-to-end LLM chat (requires LM Studio with a vision model running)
+- 💾 Save/load chat sessions
+- 📱 Verify smartphone can be used as webcam (App used in Testing: **IP Webcam** for Android)
+
+## ⏳ In Progress
+- More robust UI
+
+> [!IMPORTANT]
+> Some dummy chats are present already under the Saved Chats section (`chats/` folder). This was collected during testing and left to provide various samples. As such, they can be safely deleted.

camera_llm-0.1.1/README.md

@@ -0,0 +1,133 @@
+# Camera → Local LLM Inference App
+
+## Description
+
+A Python desktop app that captures images or short video clips from a webcam, optionally crops them, and sends them **in-memory** (no external file needed to be saved previously) to a local LLM via LM Studio for conversational analysis.
+
+---
+
+## Project Structure (Main Files)
+
+```
+Camera_LLM_Inference/
+└── camera_llm/
+       ├── __init__.py
+       ├── camera_thread.py         # QThread for OpenCV camera capture
+       ├── chat_session.py          # ChatSession dataclass + JSON serialisation
+       ├── chat_store.py            # Read/write chat sessions to chats/*.json
+       ├── cli.py                   # Entry point handling CLI and launching app (app initialization)
+       ├── llm_client.py            # OpenAI client → LM Studio, in-memory encode
+       ├── main_window.py           # Main window that aids navigation among the 7 screens
+       ├── styles.py                # Global stylesheet + design tokens     
+       └── screens/
+           ├── __init__.py
+           ├── screen1_home.py          # Dashboard + saved chats panel
+           ├── screen2_capture.py       # Live camera feed, capture/record controls
+           ├── screen3_crop.py          # Rubber-band crop with dimming overlay
+           ├── screen4_model_select.py  # LM Studio URL + model dropdown
+           ├── screen5_chat.py          # Chatbot with streaming, thumbnails, bubbles
+           ├── screen6_save.py          # Name & save the session
+           └── screen7_done.py          # Confirmation + auto-redirect home
+└── chats/                           # Auto-created at runtime for saved sessions
+├── pyproject.toml                   # Package metadata + entry points
+├── requirements.txt                 # pip dependencies
+```
+
+---
+
+## How to Run
+
+```bash
+cd "c:\Users\kurei\Documents\Machine_Deep Learning\Camera_LLM_Inference"
+pip install -e .
+camera-llm run
+```
+
+### Prerequisites
+1. **Webcam** connected or **IP camera** with reachable IP address (e.g. `http://[IP_ADDRESS]`)
+2. **LM Studio** running with a **vision model** loaded (e.g. LLaVA, Qwen-VL)
+3. LM Studio **local server started** (default: `http://localhost:1234`)
+
+---
+
+## Optional (download distribution zip file)
+
+Alternatively, you can [Download the Distribution Zip File to Run as Standalone Application](https://drive.proton.me/urls/ASD3RNEVY0#txhcUP2W8rzJ). This saves you the time of downloading the repo and installing the other dependencies, but it also takes around 2.5 GB of disk space (in addition to less frequent updates). Once downloaded, unzip the file then navigate to "\CameraLLMInference\CameraLLMInference.exe" to run the application.
+
+---
+
+## User Flow
+
+```mermaid
+graph TD;
+    S1["Screen 1: Home"] -->|Image| S2I["Screen 2: Camera (Image)"];
+    S1 -->|Video| S2V["Screen 2: Camera (Video)"];
+    S2I -->|Capture| S3["Screen 3: Crop"];
+    S2V -->|Stop recording| S3;
+    S3 -->|Use Full / Apply Crop| S4["Screen 4: Model Select"];
+    S4 -->|Analyse| S5["Screen 5: Chat"];
+    S5 -->|Finish| S6["Screen 6: Save?"];
+    S6 -->|Save / Skip| S7["Screen 7: Done"];
+    S7 -->|3s auto| S1;
+    S1 -->|Open saved chat| S5P["Screen 5: Chat (past chat)"];
+    S5P -->|Continue Chatting| S5;
+```
+
+---
+
+## Change Camera Feed Guide
+
+1. If you are using a standard USB webcam, you can just type 0, 1, 2, etc. and hit Enter. It will connect as a regular USB camera.
+2. If you want to use a smartphone, download a free app like IP Webcam (Android) or similar apps for iOS that broadcast your camera over WiFi. Start the server to use as a WiFi camera
+3. Note the URL that will be set (ex: http://192.168.1.100:8080).
+4. Paste that exact URL into the 'Camera' text box and press Enter.
+
+
+---
+
+## Key Design Decisions
+
+| Decision | Choice | Rationale |
+|---|---|---|
+| **GUI framework** | PySide6 | Robust threading (QThread), rich widget set, no licensing issues |
+| **Camera** | OpenCV in QThread | Non-blocking — GUI stays responsive at 30 fps |
+| **In-memory encoding** | `cv2.imencode` → base64 | No file ever touches disk; data goes straight to the LLM API |
+| **Video → LLM** | Sample up to 8 frames | Local VLMs don't accept video — sending evenly-spaced frames approximates it |
+| **LLM API** | `openai` client → `localhost:1234` | LM Studio is OpenAI-compatible; swapping to a cloud provider later is trivial |
+| **Chat persistence** | JSON files in `chats/` | Simple, portable, human-readable |
+
+---
+
+## Features
+
+- 7 Screens (Home, Capture, Crop, Model Select, Chat, Save, Done)
+- Image and Video Capture (With Rotate Feed Capability)
+- LLM Chat with Streaming
+- Save/Load Chat Sessions (Chats are listed with the last modified date and time)
+- Model Selection (LLM URL + Model Dropdown)
+- Delete or Rename Saved Chats
+> [!CAUTION]
+> Remember to load the same model that was used when first starting the chat (referenced in the title)
+
+> [!NOTE]
+> The video input feature works by sending multiple frames from the video to the LLM in order to process as multiple images. The multiple frames (images) are sent as a panel (chained images) so the LLM can interpret it as one image (full context). This is done primarily due to employing a local LLM.
+
+## What Was Verified
+
+- ✅ All dependencies install cleanly
+- ✅ All module imports resolve without errors
+- ✅ App launches and renders Screen 1 correctly
+- ✅ Fixed `pyqtdarktheme` API for v0.1.7 (`load_stylesheet` instead of `setup_theme`)
+
+## Physical Testing Passed
+- 📷 Camera feed with a physical webcam
+- 🎬 Video recording and frame sampling
+- 🤖 End-to-end LLM chat (requires LM Studio with a vision model running)
+- 💾 Save/load chat sessions
+- 📱 Verify smartphone can be used as webcam (App used in Testing: **IP Webcam** for Android)
+
+## ⏳ In Progress
+- More robust UI
+
+> [!IMPORTANT]
+> Some dummy chats are present already under the Saved Chats section (`chats/` folder). This was collected during testing and left to provide various samples. As such, they can be safely deleted.

camera_llm-0.1.1/camera_llm/__init__.py

@@ -0,0 +1 @@
+# Camera LLM Inference App

camera_llm-0.1.1/camera_llm/camera_thread.py

@@ -0,0 +1,82 @@
+"""
+CameraThread — captures frames from OpenCV VideoCapture in a background QThread
+and emits frame_ready(np.ndarray) signals to the GUI at ~30 fps.
+"""
+from __future__ import annotations
+
+import cv2
+import numpy as np
+from PySide6.QtCore import QThread, Signal
+
+
+class CameraThread(QThread):
+    """Background thread that continuously reads from a camera device."""
+
+    frame_ready = Signal(np.ndarray)
+    error       = Signal(str)
+
+    def __init__(self, camera_source: int | str = 0, parent=None):
+        super().__init__(parent)
+        self.camera_source = camera_source
+        self._running = False
+        self._cap: cv2.VideoCapture | None = None
+
+    # ── Public API ──────────────────────────────────────────────────────────
+
+    def start_capture(self, camera_source: int | str | None = None) -> None:
+        if camera_source is not None:
+            self.camera_source = camera_source
+        self._running = True
+        self.start()
+
+    def stop_capture(self) -> None:
+        self._running = False
+        self.wait(2000)  # give thread up to 2 s to finish
+
+    # ── QThread lifecycle ────────────────────────────────────────────────────
+
+    def run(self) -> None:
+        if isinstance(self.camera_source, int):
+            self._cap = cv2.VideoCapture(self.camera_source, cv2.CAP_DSHOW)
+            if not self._cap.isOpened():
+                # Try without backend hint (Linux / macOS)
+                self._cap = cv2.VideoCapture(self.camera_source)
+        else:
+            # IP camera URL
+            url = str(self.camera_source).strip()
+            # If the user enters a bare IP Webcam URL like "http://10.0.0.249:8080", 
+            # OpenCV needs the actual video stream endpoint, which is usually "/video".
+            if url.startswith("http") and url.count("/") == 2:
+                url += "/video"
+            elif url.startswith("http") and url.count("/") == 3 and url.endswith("/"):
+                url += "video"
+            self._cap = cv2.VideoCapture(url)
+
+        if not self._cap.isOpened():
+            self.error.emit(f"Cannot open camera source: {self.camera_source}")
+            return
+
+        # Prefer 720p for a good quality / performance balance
+        self._cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
+        self._cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
+        self._cap.set(cv2.CAP_PROP_FPS, 30)
+
+        while self._running:
+            ret, frame = self._cap.read()
+            if not ret:
+                self.error.emit("Failed to read frame from camera")
+                break
+            # Convert BGR → RGB for Qt display
+            frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+            self.frame_ready.emit(frame_rgb)
+            # ~30 fps → sleep ~33 ms
+            self.msleep(33)
+
+        if self._cap:
+            self._cap.release()
+            self._cap = None
+
+    def __del__(self):
+        self._running = False
+        if self._cap:
+            self._cap.release()

camera_llm-0.1.1/camera_llm/chat_session.py

@@ -0,0 +1,81 @@
+"""
+ChatSession — dataclass for a single chat session (image or video + messages).
+"""
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field
+from datetime import datetime
+from pathlib import Path
+from typing import Literal
+
+
+@dataclass
+class ChatSession:
+    name:       str
+    media_type: Literal["image", "video"]
+    # For image: single data-URL string.
+    # For video: list of frame data-URL strings.
+    media_data: str | list[str]
+    model:      str
+    messages:   list[dict] = field(default_factory=list)
+    timestamp:  str        = field(default_factory=lambda: datetime.now().isoformat())
+    saved_at:   str        = ""   # populated at load time from file mtime
+
+    # ── Serialisation ────────────────────────────────────────────────────────
+
+    def to_dict(self) -> dict:
+        return {
+            "name":       self.name,
+            "media_type": self.media_type,
+            "media_data": self.media_data,
+            "model":      self.model,
+            "messages":   self.messages,
+            "timestamp":  self.timestamp,
+        }
+
+    def to_json(self) -> str:
+        return json.dumps(self.to_dict(), ensure_ascii=False, indent=2)
+
+    @classmethod
+    def from_dict(cls, data: dict) -> "ChatSession":
+        return cls(
+            name       = data["name"],
+            media_type = data["media_type"],
+            media_data = data["media_data"],
+            model      = data["model"],
+            messages   = data.get("messages", []),
+            timestamp  = data.get("timestamp", ""),
+        )
+
+    @classmethod
+    def from_json(cls, json_str: str) -> "ChatSession":
+        return cls.from_dict(json.loads(json_str))
+
+    # ── Helpers ──────────────────────────────────────────────────────────────
+
+    @property
+    def display_timestamp(self) -> str:
+        try:
+            dt = datetime.fromisoformat(self.timestamp)
+            return dt.strftime("%b %d, %Y  %H:%M")
+        except ValueError:
+            return self.timestamp
+
+    @property
+    def display_saved_at(self) -> str:
+        """Display the file modification time, falling back to creation timestamp."""
+        if self.saved_at:
+            try:
+                dt = datetime.fromisoformat(self.saved_at)
+                return dt.strftime("%b %d, %Y  %H:%M")
+            except ValueError:
+                pass
+        return self.display_timestamp
+
+    def get_thumbnail_data_url(self) -> str:
+        """Return the first (or only) frame data-URL for thumbnail display."""
+        if self.media_type == "image":
+            return self.media_data  # type: ignore[return-value]
+        frames = self.media_data  # type: ignore[assignment]
+        return frames[0] if frames else ""

camera_llm-0.1.1/camera_llm/chat_store.py

@@ -0,0 +1,56 @@
+"""
+ChatStore — persists and loads ChatSession objects as JSON files
+in the `chats/` directory next to the project root.
+"""
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+from camera_llm.chat_session import ChatSession
+
+CHATS_DIR = Path(__file__).parent.parent / "chats"
+
+
+def _ensure_dir() -> None:
+    CHATS_DIR.mkdir(parents=True, exist_ok=True)
+
+
+def _safe_filename(name: str) -> str:
+    """Strip characters that are not safe for filenames."""
+    safe = re.sub(r'[\\/:*?"<>|]', "_", name).strip()
+    return safe or "untitled"
+
+
+def save(session: ChatSession) -> Path:
+    """Write a ChatSession to a JSON file. Returns the file path."""
+    _ensure_dir()
+    filename = _safe_filename(session.name) + ".json"
+    path     = CHATS_DIR / filename
+    path.write_text(session.to_json(), encoding="utf-8")
+    return path
+
+
+def load_all() -> list[ChatSession]:
+    """Load every saved ChatSession from the chats/ directory."""
+    _ensure_dir()
+    sessions: list[ChatSession] = []
+    for fpath in sorted(CHATS_DIR.glob("*.json"), key=lambda p: p.stat().st_mtime, reverse=True):
+        try:
+            session = ChatSession.from_json(fpath.read_text(encoding="utf-8"))
+            from datetime import datetime
+            session.saved_at = datetime.fromtimestamp(fpath.stat().st_mtime).isoformat()
+            sessions.append(session)
+        except Exception:
+            pass  # skip malformed files
+    return sessions
+
+
+def delete(name: str) -> bool:
+    """Delete a saved chat by its name. Returns True if deleted."""
+    filename = _safe_filename(name) + ".json"
+    path     = CHATS_DIR / filename
+    if path.exists():
+        path.unlink()
+        return True
+    return False

camera_llm-0.1.1/camera_llm/cli.py

@@ -0,0 +1,72 @@
+"""
+Camera → LLM Inference App
+Entry point — creates the QApplication, applies the dark theme, and shows the MainWindow.
+
+Usage:
+    python main.py
+"""
+import sys
+import os
+
+# Ensure the project root is on the path
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from PySide6.QtWidgets import QApplication
+from PySide6.QtGui     import QFont, QIcon
+from PySide6.QtCore    import Qt
+
+from camera_llm.main_window import MainWindow
+
+# def get_resource_path(relative_path):
+#     """Get path to resource, works for both dev and PyInstaller bundle."""
+#     if hasattr(sys, '_MEIPASS'):
+#         # PyInstaller extracts files to a temp folder (_MEIPASS) at runtime
+#         return os.path.join(sys._MEIPASS, relative_path)
+#     return os.path.join(os.path.dirname(os.path.abspath(__file__)), relative_path)
+
+
+def run_app():
+    # High-DPI support
+    QApplication.setHighDpiScaleFactorRoundingPolicy(
+        Qt.HighDpiScaleFactorRoundingPolicy.PassThrough
+    )
+
+    app = QApplication(sys.argv)
+    app.setApplicationName("Camera LLM Inference")
+    app.setOrganizationName("CameraLLM")
+
+    # Set default font
+    font = QFont("Segoe UI", 10)
+    font.setHintingPreference(QFont.HintingPreference.PreferNoHinting)
+    app.setFont(font)
+
+    # Set app-wide icon (affects taskbar, window title bar, etc.)
+    icon_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "icon.ico")
+
+    if os.path.exists(icon_path):
+        app.setWindowIcon(QIcon(icon_path))
+
+    # Apply dark theme via pyqtdarktheme as a base, then layer our custom stylesheet
+    try:
+        import qdarktheme
+        base_sheet = qdarktheme.load_stylesheet("dark")
+        app.setStyleSheet(base_sheet)
+    except (ImportError, Exception):
+        pass  # Our MAIN_STYLESHEET in styles.py covers everything standalone
+
+    window = MainWindow()
+    window.show()
+
+    sys.exit(app.exec())
+
+def cli():
+    if len(sys.argv) > 1 and sys.argv[1] == "run":
+        # Remove 'run' so QApplication doesn't try to parse it
+        sys.argv = [sys.argv[0]] + sys.argv[2:]
+        run_app()
+    else:
+        print("Usage: camera-llm run")
+        sys.exit(1)
+
+if __name__ == "__main__":
+    run_app()