pulselog 0.1.0__tar.gz

pulselog-0.1.0/PKG-INFO

@@ -0,0 +1,222 @@
+Metadata-Version: 2.4
+Name: pulselog
+Version: 0.1.0
+Summary: Real-time browser dashboard for Python logging — zero config, non-blocking
+License: MIT
+Keywords: logging,dashboard,websocket,real-time,monitoring
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Logging
+Classifier: Typing :: Typed
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: websockets>=11.0
+Requires-Dist: tomli>=2.0; python_version < "3.11"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
+
+# pulselog
+
+A Python logging library that streams every `logger.info()` to a live real-time dashboard in your browser — with zero configuration.
+
+```
+pip install pulselog
+```
+
+## Quick start
+
+```python
+from pulselog import Logger
+
+logger = Logger("my-app")
+
+logger.info("training started", epoch=1)
+logger.warning("learning rate too high", lr=0.1)
+logger.save("epoch-1", {"acc": 0.91, "loss": 0.23}, status="DONE", progress=33)
+logger.shutdown()
+```
+
+A browser tab opens automatically at `http://localhost:5678` showing all logs in real time.
+
+## Why pulselog?
+
+Standard logging solutions block the main thread on every log call — waiting for a file write, HTTP request, or DB insert. In tight loops (ML training, inference, data pipelines) this kills performance.
+
+pulselog uses a **non-blocking in-memory queue + daemon background worker**. The main thread never waits. Log calls cost ~2µs. 1 million calls complete in under 2 seconds.
+
+## Dashboard
+
+The dashboard is a single self-contained HTML file served over WebSocket — no build step, no CDN, no framework.
+
+**Logs tab:**
+- Colour-coded by level (DEBUG=gray, INFO=blue, WARNING=amber, ERROR/CRITICAL=red)
+- Level filter + full-text search
+- Virtual list rendering — handles 100k+ logs with zero browser lag
+- Auto-scroll with manual scroll override
+- Export all logs as JSON
+
+**Checkpoints tab:**
+- Progress bars for each checkpoint
+- Overall progress = average of all checkpoint progress values
+- Expandable JSON data viewer
+- Status icons: DONE ✅, IN_PROGRESS 🟡, FAILED 🔴, SKIPPED ⚫
+
+## API
+
+### Initialisation
+
+```python
+logger = Logger(
+    name,                          # shown in dashboard
+    host="localhost",              # dashboard bind host
+    port=5678,                     # auto-increments if port is taken
+    auto_open=True,                # open browser automatically
+    dashboard=True,                # set False for production/CI
+    checkpoint_path=".pulselog/checkpoints.db",
+    level="DEBUG",                 # minimum capture level
+    worker_interval=0.05,          # drain interval in seconds
+)
+```
+
+### Logging
+
+```python
+logger.debug(msg, **extra)
+logger.info(msg, **extra)
+logger.warning(msg, **extra)
+logger.error(msg, **extra)
+logger.critical(msg, **extra)
+
+# Extra kwargs appear as metadata in the dashboard
+logger.info("request handled", user_id=42, latency_ms=12)
+```
+
+### Checkpoints
+
+```python
+logger.save(
+    name,                # checkpoint identifier
+    data,                # any JSON-serialisable dict
+    status="DONE",       # "DONE"|"IN_PROGRESS"|"FAILED"|"SKIPPED"
+    note="",             # human-readable description
+    progress=None        # 0–100, shown as progress bar
+)
+
+result = logger.load("epoch-5")     # → dict | None (never raises)
+names  = logger.checkpoints()       # → list[str]
+logger.delete_checkpoint("epoch-3")
+```
+
+### Utilities
+
+```python
+logger.tag("phase-2")           # group following logs under a label
+logger.divider("epoch boundary") # insert visual divider in dashboard
+logger.flush()                   # force-drain queue (call before exit)
+logger.shutdown()                # graceful teardown
+stats = logger.stats()           # operational metrics dict
+```
+
+### Stats
+
+```python
+{
+    "records_logged":   int,
+    "records_dropped":  int,     # due to queue overflow
+    "queue_size":       int,
+    "checkpoints_saved": int,
+    "dashboard_clients": int,
+    "uptime_seconds":   float,
+}
+```
+
+## Configuration
+
+Priority order (highest → lowest): `Logger()` kwargs > env vars > `pulselog.toml` > defaults
+
+### Environment variables
+
+```bash
+PULSELOG_DASHBOARD=false
+PULSELOG_HOST=0.0.0.0
+PULSELOG_PORT=8080
+PULSELOG_AUTO_OPEN=false
+PULSELOG_CHECKPOINT_PATH=/data/checkpoints.db
+PULSELOG_LEVEL=INFO
+PULSELOG_WORKER_INTERVAL=0.01
+```
+
+### `pulselog.toml` (place in CWD)
+
+```toml
+[pulselog]
+host = "0.0.0.0"
+port = 8080
+auto_open = false
+level = "INFO"
+```
+
+## stdlib `logging` integration
+
+```python
+import logging
+from pulselog.handler import PulseHandler
+
+logging.getLogger().addHandler(PulseHandler("my-app"))
+logging.info("this appears in the pulselog dashboard")
+```
+
+## Production usage
+
+```python
+# In production: disable dashboard, keep checkpoints
+logger = Logger("prod", dashboard=False, checkpoint_path="/data/checkpoints.db")
+```
+
+With `dashboard=False`:
+- No threads are started
+- No port is bound
+- No browser is opened
+- Checkpoint reads/writes still work
+- Log calls return in ~100ns (level check only)
+
+## Performance
+
+| Operation | Throughput |
+|-----------|-----------|
+| `logger.info()` call | ~2µs |
+| 1 million log calls | <2s |
+| Queue `put()` | O(1), <1µs |
+| Dashboard at 100k logs | No lag (virtual list) |
+
+## Design
+
+```
+logger.info()           ← O(1), non-blocking
+     │
+     ▼
+  LogQueue              ← deque(maxlen=10_000), thread-safe
+     │
+     ▼ every 50ms
+BackgroundWorker        ← daemon thread
+     │
+     ├─▶ DashboardServer.broadcast()
+     │       └─▶ WebSocket clients (all connected browsers)
+     │
+     └─▶ (additional handlers)
+```
+
+The queue uses `collections.deque(maxlen=N)` — when full, the oldest record is silently dropped (never blocks). Dropped records are counted in `logger.stats()`.
+
+## License
+
+MIT

pulselog-0.1.0/README.md

@@ -0,0 +1,196 @@
+# pulselog
+
+A Python logging library that streams every `logger.info()` to a live real-time dashboard in your browser — with zero configuration.
+
+```
+pip install pulselog
+```
+
+## Quick start
+
+```python
+from pulselog import Logger
+
+logger = Logger("my-app")
+
+logger.info("training started", epoch=1)
+logger.warning("learning rate too high", lr=0.1)
+logger.save("epoch-1", {"acc": 0.91, "loss": 0.23}, status="DONE", progress=33)
+logger.shutdown()
+```
+
+A browser tab opens automatically at `http://localhost:5678` showing all logs in real time.
+
+## Why pulselog?
+
+Standard logging solutions block the main thread on every log call — waiting for a file write, HTTP request, or DB insert. In tight loops (ML training, inference, data pipelines) this kills performance.
+
+pulselog uses a **non-blocking in-memory queue + daemon background worker**. The main thread never waits. Log calls cost ~2µs. 1 million calls complete in under 2 seconds.
+
+## Dashboard
+
+The dashboard is a single self-contained HTML file served over WebSocket — no build step, no CDN, no framework.
+
+**Logs tab:**
+- Colour-coded by level (DEBUG=gray, INFO=blue, WARNING=amber, ERROR/CRITICAL=red)
+- Level filter + full-text search
+- Virtual list rendering — handles 100k+ logs with zero browser lag
+- Auto-scroll with manual scroll override
+- Export all logs as JSON
+
+**Checkpoints tab:**
+- Progress bars for each checkpoint
+- Overall progress = average of all checkpoint progress values
+- Expandable JSON data viewer
+- Status icons: DONE ✅, IN_PROGRESS 🟡, FAILED 🔴, SKIPPED ⚫
+
+## API
+
+### Initialisation
+
+```python
+logger = Logger(
+    name,                          # shown in dashboard
+    host="localhost",              # dashboard bind host
+    port=5678,                     # auto-increments if port is taken
+    auto_open=True,                # open browser automatically
+    dashboard=True,                # set False for production/CI
+    checkpoint_path=".pulselog/checkpoints.db",
+    level="DEBUG",                 # minimum capture level
+    worker_interval=0.05,          # drain interval in seconds
+)
+```
+
+### Logging
+
+```python
+logger.debug(msg, **extra)
+logger.info(msg, **extra)
+logger.warning(msg, **extra)
+logger.error(msg, **extra)
+logger.critical(msg, **extra)
+
+# Extra kwargs appear as metadata in the dashboard
+logger.info("request handled", user_id=42, latency_ms=12)
+```
+
+### Checkpoints
+
+```python
+logger.save(
+    name,                # checkpoint identifier
+    data,                # any JSON-serialisable dict
+    status="DONE",       # "DONE"|"IN_PROGRESS"|"FAILED"|"SKIPPED"
+    note="",             # human-readable description
+    progress=None        # 0–100, shown as progress bar
+)
+
+result = logger.load("epoch-5")     # → dict | None (never raises)
+names  = logger.checkpoints()       # → list[str]
+logger.delete_checkpoint("epoch-3")
+```
+
+### Utilities
+
+```python
+logger.tag("phase-2")           # group following logs under a label
+logger.divider("epoch boundary") # insert visual divider in dashboard
+logger.flush()                   # force-drain queue (call before exit)
+logger.shutdown()                # graceful teardown
+stats = logger.stats()           # operational metrics dict
+```
+
+### Stats
+
+```python
+{
+    "records_logged":   int,
+    "records_dropped":  int,     # due to queue overflow
+    "queue_size":       int,
+    "checkpoints_saved": int,
+    "dashboard_clients": int,
+    "uptime_seconds":   float,
+}
+```
+
+## Configuration
+
+Priority order (highest → lowest): `Logger()` kwargs > env vars > `pulselog.toml` > defaults
+
+### Environment variables
+
+```bash
+PULSELOG_DASHBOARD=false
+PULSELOG_HOST=0.0.0.0
+PULSELOG_PORT=8080
+PULSELOG_AUTO_OPEN=false
+PULSELOG_CHECKPOINT_PATH=/data/checkpoints.db
+PULSELOG_LEVEL=INFO
+PULSELOG_WORKER_INTERVAL=0.01
+```
+
+### `pulselog.toml` (place in CWD)
+
+```toml
+[pulselog]
+host = "0.0.0.0"
+port = 8080
+auto_open = false
+level = "INFO"
+```
+
+## stdlib `logging` integration
+
+```python
+import logging
+from pulselog.handler import PulseHandler
+
+logging.getLogger().addHandler(PulseHandler("my-app"))
+logging.info("this appears in the pulselog dashboard")
+```
+
+## Production usage
+
+```python
+# In production: disable dashboard, keep checkpoints
+logger = Logger("prod", dashboard=False, checkpoint_path="/data/checkpoints.db")
+```
+
+With `dashboard=False`:
+- No threads are started
+- No port is bound
+- No browser is opened
+- Checkpoint reads/writes still work
+- Log calls return in ~100ns (level check only)
+
+## Performance
+
+| Operation | Throughput |
+|-----------|-----------|
+| `logger.info()` call | ~2µs |
+| 1 million log calls | <2s |
+| Queue `put()` | O(1), <1µs |
+| Dashboard at 100k logs | No lag (virtual list) |
+
+## Design
+
+```
+logger.info()           ← O(1), non-blocking
+     │
+     ▼
+  LogQueue              ← deque(maxlen=10_000), thread-safe
+     │
+     ▼ every 50ms
+BackgroundWorker        ← daemon thread
+     │
+     ├─▶ DashboardServer.broadcast()
+     │       └─▶ WebSocket clients (all connected browsers)
+     │
+     └─▶ (additional handlers)
+```
+
+The queue uses `collections.deque(maxlen=N)` — when full, the oldest record is silently dropped (never blocks). Dropped records are counted in `logger.stats()`.
+
+## License
+
+MIT

pulselog-0.1.0/pulselog/__init__.py

@@ -0,0 +1,6 @@
+from __future__ import annotations
+
+from pulselog.logger import Logger
+
+__version__ = "0.1.0"
+__all__ = ["Logger", "__version__"]

pulselog-0.1.0/pulselog/checkpoint.py

@@ -0,0 +1,175 @@
+from __future__ import annotations
+
+import json
+import sqlite3
+import threading
+import time
+from pathlib import Path
+from typing import Dict, List, Optional, Any
+
+from pulselog.exceptions import InvalidStatusError
+
+# ---------------------------------------------------------------------------
+# Constants
+# ---------------------------------------------------------------------------
+VALID_STATUSES = {"DONE", "IN_PROGRESS", "FAILED", "SKIPPED"}
+
+_SCHEMA = """
+CREATE TABLE IF NOT EXISTS checkpoints (
+    name       TEXT PRIMARY KEY,
+    data       TEXT NOT NULL,
+    status     TEXT NOT NULL,
+    note       TEXT DEFAULT '',
+    progress   REAL,
+    saved_at   REAL NOT NULL
+)
+"""
+
+
+class CheckpointStore:
+    """Thread-safe SQLite-backed key/value store for training checkpoints.
+
+    Multiple threads may call save/load/delete concurrently; a threading.Lock
+    serialises all SQLite operations so we never hit "database is locked".
+
+    WAL mode is enabled so concurrent reads do not block writes.
+    """
+
+    def __init__(self, db_path: str = ":memory:") -> None:
+        """Open (or create) the checkpoint database.
+
+        Args:
+            db_path: Filesystem path to the SQLite file, or ":memory:".
+        """
+        if db_path != ":memory:":
+            Path(db_path).parent.mkdir(parents=True, exist_ok=True)
+
+        # check_same_thread=False is safe because we guard every access with _lock.
+        self._conn = sqlite3.connect(db_path, check_same_thread=False)  # guarded by self._lock
+        self._lock = threading.Lock()
+
+        with self._lock:
+            self._conn.execute("PRAGMA journal_mode=WAL")
+            self._conn.execute(_SCHEMA)
+            self._conn.commit()
+
+        self._saved_count = 0  # guarded by self._lock
+
+    # ------------------------------------------------------------------
+    # Public API
+    # ------------------------------------------------------------------
+
+    def save(
+        self,
+        name: str,
+        data: Dict[str, Any],
+        status: str = "DONE",
+        note: str = "",
+        progress: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        """Persist a checkpoint, overwriting any previous value for *name*.
+
+        Args:
+            name:     Unique checkpoint identifier.
+            data:     JSON-serialisable payload dict.
+            status:   One of DONE | IN_PROGRESS | FAILED | SKIPPED.
+            note:     Human-readable description.
+            progress: Optional 0–100 progress value.
+
+        Returns:
+            The full record dict including a ``_meta`` key.
+
+        Raises:
+            InvalidStatusError: If *status* is not one of the allowed values.
+        """
+        upper_status = status.upper()
+        if upper_status not in VALID_STATUSES:
+            raise InvalidStatusError(
+                f"Invalid checkpoint status {status!r}. "
+                f"Allowed values: {', '.join(sorted(VALID_STATUSES))}"
+            )
+
+        saved_at = time.time()
+        data_json = json.dumps(data)
+
+        with self._lock:
+            self._conn.execute(
+                """
+                INSERT OR REPLACE INTO checkpoints
+                    (name, data, status, note, progress, saved_at)
+                VALUES (?, ?, ?, ?, ?, ?)
+                """,
+                (name, data_json, upper_status, note, progress, saved_at),
+            )
+            self._conn.commit()
+            self._saved_count += 1
+
+        return {
+            **data,
+            "_meta": {
+                "name": name,
+                "status": upper_status,
+                "note": note,
+                "progress": progress,
+                "saved_at": saved_at,
+            },
+        }
+
+    def load(self, name: str) -> Optional[Dict[str, Any]]:
+        """Load a checkpoint by name.
+
+        Args:
+            name: Checkpoint identifier.
+
+        Returns:
+            Dict with original data keys plus ``_meta``, or None if not found.
+        """
+        with self._lock:
+            cur = self._conn.execute(
+                "SELECT data, status, note, progress, saved_at FROM checkpoints WHERE name = ?",
+                (name,),
+            )
+            row = cur.fetchone()
+
+        if row is None:
+            return None
+
+        data_json, status, note, progress, saved_at = row
+        data = json.loads(data_json)
+        return {
+            **data,
+            "_meta": {
+                "name": name,
+                "status": status,
+                "note": note,
+                "progress": progress,
+                "saved_at": saved_at,
+            },
+        }
+
+    def checkpoints(self) -> List[str]:
+        """Return all checkpoint names ordered by most-recently saved first.
+
+        Returns:
+            List of name strings.
+        """
+        with self._lock:
+            cur = self._conn.execute(
+                "SELECT name FROM checkpoints ORDER BY saved_at DESC"
+            )
+            return [row[0] for row in cur.fetchall()]
+
+    def delete_checkpoint(self, name: str) -> None:
+        """Delete a checkpoint by name.  Silent if the name doesn't exist.
+
+        Args:
+            name: Checkpoint identifier to remove.
+        """
+        with self._lock:
+            self._conn.execute("DELETE FROM checkpoints WHERE name = ?", (name,))
+            self._conn.commit()
+
+    def saved_count(self) -> int:
+        """Total number of save() calls since this store was created."""
+        with self._lock:
+            return self._saved_count