jsonl-viewer 0.0.1__tar.gz

jsonl_viewer-0.0.1/PKG-INFO

@@ -0,0 +1,90 @@
+Metadata-Version: 2.3
+Name: jsonl-viewer
+Version: 0.0.1
+Summary: A terminal viewer for JSONL files
+Author: Adam Fourney
+Author-email: Adam Fourney <adam.fourney@gmail.com>
+Requires-Dist: textual>=8.0.0
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/afourney/jlv
+Project-URL: Repository, https://github.com/afourney/jlv
+Project-URL: Issues, https://github.com/afourney/jlv/issues
+Description-Content-Type: text/markdown
+
+# jlv
+
+A terminal viewer for JSONL files (one JSON value per line).
+
+Install the `jsonl-viewer` package; the command is still `jlv`:
+
+```bash
+uv tool install jsonl-viewer
+jlv your-file.jsonl
+```
+
+Alternatively, use `pip install jsonl-viewer`. From a source checkout:
+
+```bash
+uv run jlv your-file.jsonl
+```
+
+Requires Python 3.12+ and Textual 8+. The left panel lists numbered records;
+the right panel shows the selected record as indented JSON. Select a string
+to read its decoded contents in a scrollable, word-wrapped inspector.
+
+## Screenshots
+
+Browse records alongside their formatted JSON:
+
+![JSONL records in the left panel and the selected record's formatted JSON in the right panel](https://raw.githubusercontent.com/afourney/jlv/main/docs/imgs/jlv_main_ui.png)
+
+Open a string to read its decoded, word-wrapped contents:
+
+![String inspector showing decoded multi-paragraph text in a scrollable window](https://raw.githubusercontent.com/afourney/jlv/main/docs/imgs/jlv_string_window.png)
+
+## Controls
+
+| Key / action | Effect |
+| --- | --- |
+| Up / Down | Select the previous / next record or JSON line |
+| Page Up / Page Down | Move by a page |
+| Home / End | Move to the first / last row |
+| Tab / Shift+Tab | Switch panels |
+| Left / Right | Scroll horizontally |
+| Enter or click | Inspect a selected string |
+| Mouse wheel / scrollbars | Scroll the panel under the pointer |
+| Escape | Close the string inspector |
+| q | Quit, including from the inspector |
+
+The inspector is read-only and supports text selection and Ctrl+C to copy.
+The main panels also support text selection. The left-hand preview is capped
+at 1,000 characters, but the right-hand document and decoded strings are not
+truncated.
+
+## Large files
+
+Both main panels render only visible rows, rather than creating a widget for
+every record or JSON field. The loader validates the entire file in one buffered
+pass and builds a compact byte-offset index. Records are read and parsed on
+demand; small bounded caches retain recent previews and formatted documents.
+The string inspector also renders a viewport, and defers wrapping until its
+actual display width is known.
+
+Validation still happens before the viewer opens: malformed JSON, invalid UTF-8,
+and interior blank lines are reported with a line number. Trailing blank lines
+are permitted. This is a read-only viewer, not a live file follower; do not edit
+the input file while viewing it.
+
+## Sample file
+
+The repository includes `stress-test.jsonl.gz`, a compressed synthetic dataset
+with 30,000 records. Extract it before opening it:
+
+```bash
+gzip -dk stress-test.jsonl.gz
+uv run jlv stress-test.jsonl
+```
+
+The extracted file is 300 MB and is Git-ignored. Records range from approximately
+1-100 KB and contain nested structures and strings of varying lengths. Some
+strings intentionally end mid-sentence to fit the generated record sizes.

jsonl_viewer-0.0.1/README.md

@@ -0,0 +1,77 @@
+# jlv
+
+A terminal viewer for JSONL files (one JSON value per line).
+
+Install the `jsonl-viewer` package; the command is still `jlv`:
+
+```bash
+uv tool install jsonl-viewer
+jlv your-file.jsonl
+```
+
+Alternatively, use `pip install jsonl-viewer`. From a source checkout:
+
+```bash
+uv run jlv your-file.jsonl
+```
+
+Requires Python 3.12+ and Textual 8+. The left panel lists numbered records;
+the right panel shows the selected record as indented JSON. Select a string
+to read its decoded contents in a scrollable, word-wrapped inspector.
+
+## Screenshots
+
+Browse records alongside their formatted JSON:
+
+![JSONL records in the left panel and the selected record's formatted JSON in the right panel](https://raw.githubusercontent.com/afourney/jlv/main/docs/imgs/jlv_main_ui.png)
+
+Open a string to read its decoded, word-wrapped contents:
+
+![String inspector showing decoded multi-paragraph text in a scrollable window](https://raw.githubusercontent.com/afourney/jlv/main/docs/imgs/jlv_string_window.png)
+
+## Controls
+
+| Key / action | Effect |
+| --- | --- |
+| Up / Down | Select the previous / next record or JSON line |
+| Page Up / Page Down | Move by a page |
+| Home / End | Move to the first / last row |
+| Tab / Shift+Tab | Switch panels |
+| Left / Right | Scroll horizontally |
+| Enter or click | Inspect a selected string |
+| Mouse wheel / scrollbars | Scroll the panel under the pointer |
+| Escape | Close the string inspector |
+| q | Quit, including from the inspector |
+
+The inspector is read-only and supports text selection and Ctrl+C to copy.
+The main panels also support text selection. The left-hand preview is capped
+at 1,000 characters, but the right-hand document and decoded strings are not
+truncated.
+
+## Large files
+
+Both main panels render only visible rows, rather than creating a widget for
+every record or JSON field. The loader validates the entire file in one buffered
+pass and builds a compact byte-offset index. Records are read and parsed on
+demand; small bounded caches retain recent previews and formatted documents.
+The string inspector also renders a viewport, and defers wrapping until its
+actual display width is known.
+
+Validation still happens before the viewer opens: malformed JSON, invalid UTF-8,
+and interior blank lines are reported with a line number. Trailing blank lines
+are permitted. This is a read-only viewer, not a live file follower; do not edit
+the input file while viewing it.
+
+## Sample file
+
+The repository includes `stress-test.jsonl.gz`, a compressed synthetic dataset
+with 30,000 records. Extract it before opening it:
+
+```bash
+gzip -dk stress-test.jsonl.gz
+uv run jlv stress-test.jsonl
+```
+
+The extracted file is 300 MB and is Git-ignored. Records range from approximately
+1-100 KB and contain nested structures and strings of varying lengths. Some
+strings intentionally end mid-sentence to fit the generated record sizes.

jsonl_viewer-0.0.1/pyproject.toml

@@ -0,0 +1,26 @@
+[project]
+name = "jsonl-viewer"
+version = "0.0.1"
+description = "A terminal viewer for JSONL files"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = ["textual>=8.0.0"]
+
+[[project.authors]]
+name = "Adam Fourney"
+email = "adam.fourney@gmail.com"
+
+[project.scripts]
+jlv = "jlv:main"
+
+[project.urls]
+Homepage = "https://github.com/afourney/jlv"
+Repository = "https://github.com/afourney/jlv"
+Issues = "https://github.com/afourney/jlv/issues"
+
+[build-system]
+requires = ["uv_build>=0.10.2,<0.11.0"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-name = "jlv"

jsonl_viewer-0.0.1/pyproject.toml.orig

@@ -0,0 +1,25 @@
+[project]
+name = "jsonl-viewer"
+version = "0.0.1"
+description = "A terminal viewer for JSONL files"
+readme = "README.md"
+authors = [
+    { name = "Adam Fourney", email = "adam.fourney@gmail.com" }
+]
+requires-python = ">=3.12"
+dependencies = ["textual>=8.0.0"]
+
+[project.scripts]
+jlv = "jlv:main"
+
+[project.urls]
+Homepage = "https://github.com/afourney/jlv"
+Repository = "https://github.com/afourney/jlv"
+Issues = "https://github.com/afourney/jlv/issues"
+
+[build-system]
+requires = ["uv_build>=0.10.2,<0.11.0"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-name = "jlv"

jsonl_viewer-0.0.1/src/jlv/__init__.py

@@ -0,0 +1,200 @@
+from __future__ import annotations
+
+import json
+import sys
+from collections import OrderedDict
+from dataclasses import dataclass
+
+from rich.cells import cell_len
+from textual.app import App, ComposeResult
+from textual import events
+from textual.binding import Binding
+from textual.containers import Horizontal
+from textual.screen import ModalScreen
+from textual.widgets import TextArea
+
+from jlv.rows import RowView
+from jlv.source import JsonlFile
+
+
+@dataclass
+class PrettyDocument:
+    lines: list[tuple[str, str | None]]
+    strings: dict[int, str]
+    width: int
+
+    @classmethod
+    def from_value(cls, value: object) -> PrettyDocument:
+        lines: list[tuple[str, str | None]] = []
+        strings: dict[int, str] = {}
+
+        def append(value: object, depth: int, prefix: str = "", suffix: str = "") -> None:
+            indent = "  " * depth
+            if isinstance(value, (dict, list)) and value:
+                opening, closing = ("{", "}") if isinstance(value, dict) else ("[", "]")
+                lines.append((indent + prefix + opening, None))
+                entries = value.items() if isinstance(value, dict) else enumerate(value)
+                for i, (key, child) in enumerate(entries):
+                    child_prefix = json.dumps(key, ensure_ascii=False) + ": " if isinstance(value, dict) else ""
+                    append(child, depth + 1, child_prefix, "," if i < len(value) - 1 else "")
+                lines.append((indent + closing + suffix, None))
+            else:
+                kind = None
+                if isinstance(value, str):
+                    kind = "string"
+                    strings[len(lines)] = value
+                elif value is None:
+                    kind = "null"
+                elif isinstance(value, bool):
+                    kind = "bool"
+                elif isinstance(value, (int, float)):
+                    kind = "number"
+                lines.append((indent + prefix + json.dumps(value, ensure_ascii=False) + suffix, kind))
+
+        append(value, 0)
+        return cls(
+            lines, strings,
+            max(
+                len(line) if line.isascii() and "\x7f" not in line else cell_len(line)
+                for line, _ in lines
+            ),
+        )
+
+
+class StringView(TextArea):
+    def __init__(self, value: str) -> None:
+        self._pending_value: str | None = value
+        super().__init__(
+            read_only=True,
+            show_cursor=False,
+            highlight_cursor_line=False,
+            soft_wrap=True,
+            id="modal-container",
+        )
+
+    def on_resize(self, event: events.Resize) -> None:
+        if self._pending_value is not None and self.content_size.width:
+            # Wrap once at the actual viewport width, not at each startup size.
+            value, self._pending_value = self._pending_value, None
+            self.load_text(value)
+            event.prevent_default()
+
+
+class StringModal(ModalScreen[None]):
+    """A read-only, wrapped document with viewport-based rendering."""
+
+    BINDINGS = [
+        Binding("escape", "dismiss", "Close"),
+        Binding("q", "app.quit", "Quit"),
+    ]
+    DEFAULT_CSS = """
+    StringModal {
+        align: center middle;
+    }
+    StringModal #modal-container {
+        width: 90%;
+        height: 90%;
+        border: thick $accent;
+        background: $surface;
+        padding: 1 2;
+        overflow-y: scroll;
+        scrollbar-size: 1 1;
+    }
+    """
+
+    def __init__(self, value: str) -> None:
+        super().__init__()
+        self.value = value
+
+    def compose(self) -> ComposeResult:
+        yield StringView(self.value)
+
+    def action_dismiss(self) -> None:
+        self.app.pop_screen()
+
+
+class JlvApp(App):
+    """JSONL Log Viewer."""
+
+    TITLE = "jlv"
+    BINDINGS = [
+        Binding("q", "quit", "Quit"),
+        Binding("tab", "focus_next", "Switch Panel"),
+        Binding("shift+tab", "focus_previous", "Switch Panel"),
+    ]
+    DEFAULT_CSS = """
+    Horizontal {
+        height: 1fr;
+    }
+    #lines-list {
+        width: 2fr;
+    }
+    #detail-list {
+        width: 3fr;
+        border-left: solid $accent;
+    }
+    """
+
+    def __init__(self, source: JsonlFile) -> None:
+        super().__init__()
+        self.source = source
+        self._documents: OrderedDict[int, PrettyDocument] = OrderedDict()
+        self._document: PrettyDocument | None = None
+        self._detail_idx: int | None = None
+        self._pretty_lines: list[tuple[str, str | None]] = []
+
+    def compose(self) -> ComposeResult:
+        with Horizontal():
+            yield RowView(id="lines-list")
+            yield RowView(id="detail-list")
+
+    def on_mount(self) -> None:
+        left = self.query_one("#lines-list", RowView)
+        left.set_rows(len(self.source), self.source.preview_width, self.source.preview)
+        left.focus()
+        self._update_detail(0)
+
+    def on_row_view_highlighted(self, event: RowView.Highlighted) -> None:
+        if event.view.id == "lines-list":
+            # Ignore queued highlights superseded by newer input in the same frame.
+            if event.index == event.view.index:
+                self._update_detail(event.index)
+
+    def on_row_view_selected(self, event: RowView.Selected) -> None:
+        if event.view.id == "detail-list" and self._document is not None:
+            value = self._document.strings.get(event.index)
+            if value is not None:
+                self.push_screen(StringModal(value))
+                return
+        self.bell()
+
+    def _update_detail(self, index: int) -> None:
+        if index == self._detail_idx:
+            return
+        if index in self._documents:
+            document = self._documents[index]
+            self._documents.move_to_end(index)
+        else:
+            document = PrettyDocument.from_value(self.source.value(index))
+            self._documents[index] = document
+            if len(self._documents) > 8:
+                self._documents.popitem(last=False)
+        self._document = document
+        self._detail_idx = index
+        self._pretty_lines = document.lines
+        self.query_one("#detail-list", RowView).set_rows(
+            len(document.lines), document.width, lambda row: document.lines[row][0]
+        )
+
+
+def main() -> None:
+    if len(sys.argv) < 2:
+        print("Usage: jlv <file.jsonl>", file=sys.stderr)
+        sys.exit(1)
+    try:
+        source = JsonlFile(sys.argv[1])
+    except (OSError, ValueError) as error:
+        print(f"Error: {error}", file=sys.stderr)
+        sys.exit(1)
+    with source:
+        JlvApp(source).run()

jsonl_viewer-0.0.1/src/jlv/rows.py

@@ -0,0 +1,192 @@
+from __future__ import annotations
+
+from collections import OrderedDict
+from collections.abc import Callable
+
+from rich.cells import cell_len
+from rich.segment import Segment
+from textual import events
+from textual.binding import Binding
+from textual.geometry import Offset, Region, Size
+from textual.message import Message
+from textual.reactive import reactive
+from textual.scroll_view import ScrollView
+from textual.selection import Selection
+from textual.strip import Strip
+
+
+class RowView(ScrollView, can_focus=True):
+    """Fixed-height selectable rows, rendered only for the visible viewport."""
+
+    BINDINGS = [
+        Binding("up", "cursor_up", show=False),
+        Binding("down", "cursor_down", show=False),
+        Binding("pageup", "cursor_page_up", show=False),
+        Binding("pagedown", "cursor_page_down", show=False),
+        Binding("home", "cursor_home", show=False),
+        Binding("end", "cursor_end", show=False),
+        Binding("enter", "select", show=False),
+    ]
+
+    DEFAULT_CSS = """
+    RowView {
+        height: 1fr;
+        scrollbar-size: 1 1;
+    }
+    RowView > .rowview--highlight {
+        background: $boost;
+    }
+    RowView:focus > .rowview--highlight {
+        background: $accent;
+        color: $text;
+    }
+    """
+    COMPONENT_CLASSES = {"rowview--highlight"}
+    index: reactive[int] = reactive(0, init=False)
+
+    class Highlighted(Message):
+        def __init__(self, view: RowView, index: int) -> None:
+            self.view = view
+            self.index = index
+            super().__init__()
+
+    class Selected(Message):
+        def __init__(self, view: RowView, index: int) -> None:
+            self.view = view
+            self.index = index
+            super().__init__()
+
+    def __init__(self, *, id: str) -> None:
+        super().__init__(id=id)
+        self.row_count = 0
+        self._line: Callable[[int], str] = lambda index: ""
+        self._unicode_cache: OrderedDict[tuple[int, int, int], tuple[Strip, int]] = OrderedDict()
+
+    def set_rows(self, count: int, width: int, line: Callable[[int], str]) -> None:
+        if self.is_mounted and self.text_selection is not None:
+            self.screen.selections = {
+                widget: selection for widget, selection in self.screen.selections.items()
+                if widget is not self
+            }
+        self.row_count = count
+        self._line = line
+        self._unicode_cache.clear()
+        self.virtual_size = Size(width, count)
+        self.index = 0
+        self.scroll_to(x=0, y=0, animate=False, force=True)
+        self.refresh()
+
+    def validate_index(self, index: int) -> int:
+        return max(0, min(index, self.row_count - 1))
+
+    def watch_index(self, old: int, new: int) -> None:
+        self.refresh_line(old)
+        self.refresh_line(new)
+        self.scroll_to_region(
+            Region(0, new, 1, 1), animate=False, x_axis=False, force=True
+        )
+        self.post_message(self.Highlighted(self, new))
+
+    def render_line(self, y: int) -> Strip:
+        x, offset = self.scroll_offset
+        row = y + offset
+        width = self.size.width
+        style = self.rich_style
+        if row >= self.row_count:
+            return Strip.blank(width, style)
+        if row == self.index:
+            style += self.get_component_rich_style("rowview--highlight")
+        text = self._line(row)
+        single_cell = text.isascii() and "\x7f" not in text
+        # Avoid constructing/cropping a giant Rich segment for long ASCII strings.
+        if single_cell:
+            visible = text[x:x + width].ljust(width)
+            strip = Strip([Segment(visible)], width)
+            character_offset = x
+        else:
+            key = (row, x, width)
+            if key not in self._unicode_cache:
+                strip = Strip([Segment(text)], cell_len(text))
+                _, remainder = Segment(text).split_cells(x)
+                character_offset = len(text) - len(remainder.text)
+                self._unicode_cache[key] = (
+                    strip.crop_extend(x, x + width, None), character_offset
+                )
+                if len(self._unicode_cache) > 256:
+                    self._unicode_cache.popitem(last=False)
+            self._unicode_cache.move_to_end(key)
+            strip, character_offset = self._unicode_cache[key]
+        strip = strip.apply_style(style)
+        if self.text_selection is not None:
+            span = self.text_selection.get_span(row)
+            if span is not None:
+                start, end = span
+                start = min(start, len(text)) if single_cell else cell_len(text[:start])
+                end = len(text) if end == -1 else end
+                end = min(end, len(text)) if single_cell else cell_len(text[:end])
+                start, end = max(0, start - x), min(width, end - x)
+                if start < end:
+                    before, selected, after = strip.divide([start, end, width])
+                    strip = Strip.join([
+                        before,
+                        selected.apply_style(self.screen.get_component_rich_style("screen--selection")),
+                        after,
+                    ])
+        return strip.apply_offsets(character_offset, row)
+
+    def get_selection(self, selection: Selection) -> tuple[str, str]:
+        start = selection.start or Offset(0, 0)
+        last = min(
+            self.row_count - 1,
+            selection.end.y if selection.end is not None else self.row_count - 1,
+        )
+        if last < start.y:
+            return "", "\n"
+        lines = [self._line(row) for row in range(start.y, last + 1)]
+        end_x = selection.end.x if selection.end is not None else len(lines[-1])
+        relative = Selection(Offset(start.x, 0), Offset(end_x, len(lines) - 1))
+        return relative.extract("\n".join(lines)), "\n"
+
+    def selection_updated(self, selection: Selection | None) -> None:
+        self.refresh()
+
+    def on_focus(self) -> None:
+        self.refresh_line(self.index)
+
+    def on_blur(self) -> None:
+        self.refresh_line(self.index)
+
+    def on_click(self, event: events.Click) -> None:
+        if event.widget is not self:
+            return
+        offset = event.get_content_offset(self)
+        if offset is None:
+            return
+        row = offset.y + self.scroll_offset.y
+        if 0 <= row < self.row_count:
+            self.focus()
+            self.index = row
+            self.post_message(self.Selected(self, row))
+            event.stop()
+
+    def action_cursor_up(self) -> None:
+        self.index -= 1
+
+    def action_cursor_down(self) -> None:
+        self.index += 1
+
+    def action_cursor_page_up(self) -> None:
+        self.index -= max(1, self.scrollable_content_region.height)
+
+    def action_cursor_page_down(self) -> None:
+        self.index += max(1, self.scrollable_content_region.height)
+
+    def action_cursor_home(self) -> None:
+        self.index = 0
+
+    def action_cursor_end(self) -> None:
+        self.index = self.row_count - 1
+
+    def action_select(self) -> None:
+        if self.row_count:
+            self.post_message(self.Selected(self, self.index))

jsonl_viewer-0.0.1/src/jlv/source.py

@@ -0,0 +1,110 @@
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import stat
+import tempfile
+from array import array
+from collections import OrderedDict
+from collections.abc import Iterator
+from pathlib import Path
+from typing import BinaryIO
+
+from rich.cells import cell_len
+
+
+class JsonlFile:
+    """Validated, indexed JSONL without retaining every decoded record in memory."""
+
+    def __init__(self, path: str | Path) -> None:
+        self._file: BinaryIO = open(path, "rb", buffering=1024 * 1024)
+        self._previews: OrderedDict[int, str] = OrderedDict()
+        self.offsets = array("Q", [0])
+        self.preview_width = 0
+        try:
+            if not stat.S_ISREG(os.fstat(self._file.fileno()).st_mode):
+                with self._file as stream:
+                    self._file = tempfile.TemporaryFile("w+b")
+                    shutil.copyfileobj(stream, self._file, length=1024 * 1024)
+                self._file.seek(0)
+            first_blank: int | None = None
+            offset = 0
+            decoder = json.JSONDecoder()
+            for line_number, raw in enumerate(self._lines(), 1):
+                offset += len(raw)
+                try:
+                    text = raw.decode("utf-8").strip()
+                except UnicodeDecodeError as error:
+                    raise ValueError(f"line {line_number} is not valid UTF-8: {error}") from error
+                if not text:
+                    if first_blank is None:
+                        first_blank = line_number
+                    continue
+                if first_blank is not None:
+                    raise ValueError(f"line {first_blank} is empty — not valid JSONL")
+                try:
+                    decoder.decode(text)
+                except (ValueError, RecursionError) as error:
+                    raise ValueError(f"line {line_number} is not valid JSON: {error}") from error
+                self.offsets.append(offset)
+                preview = self._preview_text(text, line_number)
+                width = len(preview) if preview.isascii() and "\x7f" not in preview else cell_len(preview)
+                self.preview_width = max(self.preview_width, width)
+            if not len(self):
+                raise ValueError("file is empty")
+        except BaseException:
+            self.close()
+            raise
+
+    @staticmethod
+    def _preview_text(text: str, line_number: int) -> str:
+        if len(text) > 1000:
+            text = text[:1000] + "..."
+        preview = f"{line_number:>5}  {text}"
+        return preview.expandtabs(4) if "\t" in preview else preview
+
+    def _lines(self) -> Iterator[bytes]:
+        for raw in self._file:
+            if b"\r" in raw:
+                yield from raw.splitlines(keepends=True)
+            else:
+                yield raw
+
+    def __len__(self) -> int:
+        return len(self.offsets) - 1
+
+    def raw(self, index: int) -> str:
+        if not 0 <= index < len(self):
+            raise IndexError(index)
+        if self._file.closed:
+            raise ValueError("file is closed")
+        start, end = self.offsets[index:index + 2]
+        self._file.seek(start)
+        raw = self._file.read(end - start)
+        if len(raw) != end - start:
+            raise ValueError("file was truncated while viewing; reopen it")
+        return raw.decode("utf-8")
+
+    def value(self, index: int) -> object:
+        return json.loads(self.raw(index).strip())
+
+    def preview(self, index: int) -> str:
+        if index in self._previews:
+            self._previews.move_to_end(index)
+            return self._previews[index]
+        text = self._preview_text(self.raw(index).strip(), index + 1)
+        self._previews[index] = text
+        if len(self._previews) > 256:
+            self._previews.popitem(last=False)
+        return text
+
+    def close(self) -> None:
+        self._file.close()
+        self._previews.clear()
+
+    def __enter__(self) -> JsonlFile:
+        return self
+
+    def __exit__(self, *args: object) -> None:
+        self.close()