bobframes 0.1.0__tar.gz → 0.2.0__tar.gz

{bobframes-0.1.0 → bobframes-0.2.0}/CHANGELOG.md

@@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-06-03
+
+De-hardcoding + a full report overhaul. No schema change (still schema 3); extraction output is stable
+where the pipeline is unchanged, so parquet digests are byte-identical to 0.1.0 on the same captures.
+
+### Added
+- Inline-SVG chart toolkit (`reports/charts.py`): a flagship chart per report (pass-GPU treemap,
+  draws-by-class donut, shader complexity-vs-cost scatter, overdraw reject-rate bars, instancing
+  wasted-index bars, per-KPI trend lines), deterministic and dependency-free (no `random`/`Date`).
+- Report design language: hero KPI strip, callouts with config-driven severity, provenance/device
+  strip, section cards, sticky section headers, copy buttons on long IDs, vendored Inter subset font
+  (offline, base64-inlined), and an auto-tint heatmap on ranked numeric columns.
+- Run model: per-run truth (each report names "run N of M"), a run picker, A/B compare, and a trend
+  table across runs; older runs are pre-rendered up to a configurable limit.
+- One unified `rdc-table` engine (progressive enhancement, two modes): server-baked `static` for reports
+  (JS-off / print / Ctrl-F / golden all hold) and windowed `virtual` for the catalog + per-drop drill.
+  Shared natural-numeric sort, type-split, heatmap, column groups, search/filter, controllable cell
+  truncation with hover-reveal, and full sort/filter a11y (aria-sort + keyboard-operable headers + a
+  labelled filter input) across both modes.
+- TOML config (`.bobframes.toml`, `[report]`/`[pipeline]` sections), `design_tokens.toml` token system,
+  and a state-capable draw classifier driven by table columns.
+- Reports cache with a SHA256 sidecar (corrupt/missing/mismatch falls back to a live scan, never
+  silent-empty); manifest schema-version guard (`ingest --force` hint on mismatch).
+
+### Changed
+- Catalog + drill readability: roomier rows, type-split mono/sans, and the heavy per-table row data moved
+  out of the HTML into `_pagedata/*.js` companions (the ~21 MB time-to-interactive fix) loaded via a
+  file://-safe `<script defer src>`.
+- External tool resolution + clearer pipeline error messages and exit codes; environment variables
+  renamed `RDC_*` -> `BOBFRAMES_*` (one-release legacy fallback with a deprecation warning).
+- Project paths, the report registry, and the drill-size budget are configurable rather than hardcoded.
+
 ## [0.1.0] - 2026-05-31
 
 First standalone release. v1 is Windows-only (the replay stage drives `qrenderdoc`).

{bobframes-0.1.0 → bobframes-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bobframes
-Version: 0.1.0
+Version: 0.2.0
 Summary: RenderDoc capture pipeline: ingest, analyze, render.
 Project-URL: Homepage, https://github.com/altpsyche/bobframes
 Project-URL: Issues, https://github.com/altpsyche/bobframes/issues
@@ -21,6 +21,7 @@ Classifier: Topic :: Multimedia :: Graphics
 Classifier: Topic :: Software Development :: Debuggers
 Requires-Python: <3.15,>=3.10
 Requires-Dist: pyarrow<22,>=17
+Requires-Dist: tomli>=2.0; python_version < '3.11'
 Provides-Extra: dev
 Requires-Dist: build; extra == 'dev'
 Requires-Dist: hatchling; extra == 'dev'
@@ -65,19 +66,42 @@ time to rebuild the HTML from existing Parquet without re-replaying captures.
 | Command | Purpose |
 |---|---|
 | `ingest [root] [--area X] [--label Y] [--capture N] [--force] [--pixel-grid 4] [--render-only]` | Full pipeline: export, parse, replay, parquetize, derive, manifest, commit, catalog, render. |
-| `render [root] [--area X] [--label Y]` | Rebuild HTML and catalog from existing Parquet. |
+| `render [root] [--area X] [--label Y] [--watch]` | Rebuild HTML and catalog from existing Parquet. `--watch` re-renders on token or chrome edits (alpha). |
 | `ab [root] --baseline-label X --compare-label Y` | All reports for one drop pair under `_reports/ab/<pair>/`. |
 | `report [root] <name>` | Build one named report (draws-by-class, trend, instancing, pass-gpu, shader, overdraw, dashboard). |
 | `catalog [root]` | Rebuild `_data/_catalog.parquet` only. |
 | `lint <file>...` | Check HTML or markdown against the banlist. |
 | `check` | Print resolved tool paths; non-zero when a tool is missing. |
 | `serve [root] [--port 8000] [--bind 127.0.0.1]` | Static preview via the stdlib HTTP server. |
+| `preview [root]` | Render the chrome gallery to `_reports/_chrome_preview.html`; no capture data needed. |
+| `export-tokens [--format toml\|json\|css]` | Print the design tokens to stdout in the chosen format. |
 | `smoke [--data DIR]` | End-to-end check; render-only against the bundled fixture when `--data` is omitted. |
 | `version` | Print `bobframes`, schema, and pyarrow versions. |
 
 `<root>` is positional and defaults to `.`. Flags are long-form only. Exit codes: `0` success,
 `1` pipeline or build failure, `2` usage error, `3` external tool missing, `4` interrupted.
 
+## Customizing reports
+
+The report palette lives in `bobframes/reports/design_tokens.toml` (colors, spacing, type, motion,
+and base layout sizes). A designer can edit values there without touching Python. The CSS variable
+name is fixed by its key: `surface_0` becomes `--surface-0`, `accent_primary` becomes
+`--accent-primary`.
+
+Workflow:
+
+```
+bobframes preview                 # writes _reports/_chrome_preview.html (every primitive, no data)
+# edit a value in design_tokens.toml
+bobframes preview                 # under a second; reload the page in a browser
+bobframes render C:\captures      # apply the change to real reports from existing Parquet
+```
+
+`bobframes render --watch` re-runs render-only whenever the token file or a chrome module changes
+(alpha; a 500ms poll, no extra dependency). `bobframes export-tokens --format css` prints the live
+`:root` block; `--format json` prints the nested table; `--format toml` prints the file itself. For
+prototyping, a browser's dev tools can live-edit the same variables before you commit a value.
+
 ## External tools
 
 The export stage runs `renderdoccmd convert`; the replay stage runs `qrenderdoc --python`. v1 looks

{bobframes-0.1.0 → bobframes-0.2.0}/README.md

@@ -35,19 +35,42 @@ time to rebuild the HTML from existing Parquet without re-replaying captures.
 | Command | Purpose |
 |---|---|
 | `ingest [root] [--area X] [--label Y] [--capture N] [--force] [--pixel-grid 4] [--render-only]` | Full pipeline: export, parse, replay, parquetize, derive, manifest, commit, catalog, render. |
-| `render [root] [--area X] [--label Y]` | Rebuild HTML and catalog from existing Parquet. |
+| `render [root] [--area X] [--label Y] [--watch]` | Rebuild HTML and catalog from existing Parquet. `--watch` re-renders on token or chrome edits (alpha). |
 | `ab [root] --baseline-label X --compare-label Y` | All reports for one drop pair under `_reports/ab/<pair>/`. |
 | `report [root] <name>` | Build one named report (draws-by-class, trend, instancing, pass-gpu, shader, overdraw, dashboard). |
 | `catalog [root]` | Rebuild `_data/_catalog.parquet` only. |
 | `lint <file>...` | Check HTML or markdown against the banlist. |
 | `check` | Print resolved tool paths; non-zero when a tool is missing. |
 | `serve [root] [--port 8000] [--bind 127.0.0.1]` | Static preview via the stdlib HTTP server. |
+| `preview [root]` | Render the chrome gallery to `_reports/_chrome_preview.html`; no capture data needed. |
+| `export-tokens [--format toml\|json\|css]` | Print the design tokens to stdout in the chosen format. |
 | `smoke [--data DIR]` | End-to-end check; render-only against the bundled fixture when `--data` is omitted. |
 | `version` | Print `bobframes`, schema, and pyarrow versions. |
 
 `<root>` is positional and defaults to `.`. Flags are long-form only. Exit codes: `0` success,
 `1` pipeline or build failure, `2` usage error, `3` external tool missing, `4` interrupted.
 
+## Customizing reports
+
+The report palette lives in `bobframes/reports/design_tokens.toml` (colors, spacing, type, motion,
+and base layout sizes). A designer can edit values there without touching Python. The CSS variable
+name is fixed by its key: `surface_0` becomes `--surface-0`, `accent_primary` becomes
+`--accent-primary`.
+
+Workflow:
+
+```
+bobframes preview                 # writes _reports/_chrome_preview.html (every primitive, no data)
+# edit a value in design_tokens.toml
+bobframes preview                 # under a second; reload the page in a browser
+bobframes render C:\captures      # apply the change to real reports from existing Parquet
+```
+
+`bobframes render --watch` re-runs render-only whenever the token file or a chrome module changes
+(alpha; a 500ms poll, no extra dependency). `bobframes export-tokens --format css` prints the live
+`:root` block; `--format json` prints the nested table; `--format toml` prints the file itself. For
+prototyping, a browser's dev tools can live-edit the same variables before you commit a value.
+
 ## External tools
 
 The export stage runs `renderdoccmd convert`; the replay stage runs `qrenderdoc --python`. v1 looks

bobframes-0.2.0/bobframes/_default_config.toml

@@ -0,0 +1,61 @@
+# bobframes bundled default config (c07). Single source of truth for de-hardcoded literals.
+# Values here reproduce today's output byte-identically (ADR-6). A user .bobframes.toml is
+# deep-merged ON TOP of this (override one key without restating the rest — ADR-25).
+#
+# Regex / format-string values MUST be TOML *literal* (single-quoted) strings: a basic
+# (double-quoted) string interprets backslash escapes and breaks on \d. Do NOT "fix" a decode
+# error by double-escaping — that silently changes the pattern (ADR-23 trap).
+
+schema_version = 1
+
+[tools]
+# renderdoccmd = "..."   # unset by default; resolve_tool falls through to PATH / known paths
+# qrenderdoc   = "..."
+
+[pipeline]
+replay_timeout_s  = 600.0    # H-12 — per-capture qrenderdoc replay budget (seconds)
+convert_timeout_s = 120.0    # H-13 — per-file renderdoccmd convert budget (seconds)
+# workers / pixel_grid / keep_stage are §6 keys but NOT wired in c07 (argparse remains the
+# source; workers default is min(4, os.cpu_count()), so wiring it would change behavior).
+
+[discovery]
+dated_re = '^(\d{4}-\d{2}-\d{2})(?:_(.*))?$'    # H-30 — dated drop-folder pattern
+
+[formatting]
+id_short_n         = 12                          # H-23 — id/hash truncation length
+text_trunc_max     = 60                          # H-23 — mid/left text truncation length
+chrome_scrub_chars = '[—–…“”‘’→←↑↓×·]'           # H-16 — chars scrubbed from rendered chrome
+
+[delta]
+fmt               = '{:+,.0f}'                   # H-22 — default signed delta format
+bar_label_min_pct = 8.0                          # H-21 — min segment % to draw an inline label
+
+[scoring.complexity]                             # H-17 / Q-3 — shader complexity_score weights
+w_texture_samples = 2.0
+w_branches        = 0.5
+w_loops           = 2.0
+w_discards        = 0.3
+w_dfdx_dfdy       = 0.5
+w_mat4            = 0.3
+src_len_divisor   = 100.0
+src_len_cap       = 50.0
+
+[lint]
+# User-appended banned tokens, same shape as lint_banlist.toml [[banned]] entries:
+#   [[lint.extra_banned]]
+#   pattern = 'foo'
+#   label   = 'why'
+#   flags   = ['I']
+extra_banned = []
+
+[classifier]                # c09 — draw-classification preset (H-1..H-5)
+preset = "ue"               # draw_classifier.toml (UE default); "unity"/"godot"/... -> presets/<name>.toml
+# custom_path = "C:/path/to/my_engine.toml"   # absolute path to a custom preset (wins over `preset`)
+
+[report]                    # c16 — thresholds that drive report callout severity. Tune per project.
+shader_complexity_high    = 60.0   # fragment shaders at/above this complexity are flagged
+overdraw_reject_warn_pct  = 40.0   # RT sample-rejection % -> warn callout
+overdraw_reject_alarm_pct = 70.0   # RT sample-rejection % -> alarm callout
+instancing_repeat_min     = 4      # min mesh repeat to count as an instancing candidate
+gpu_regression_pct        = 10.0   # latest-vs-previous GPU increase % -> regression callout
+max_prerendered_runs      = 10     # c16f — cap on pre-rendered older-run pages (run selector)

bobframes-0.2.0/bobframes/_version.py

@@ -0,0 +1 @@
+__version__ = "0.2.0"

{bobframes-0.1.0 → bobframes-0.2.0}/bobframes/catalog.py

@@ -21,22 +21,12 @@ import pyarrow as pa
 import pyarrow.csv as pacsv
 import pyarrow.parquet as papq
 
-from . import paths as _paths
+from . import manifest as _manifest, paths as _paths, schemas
 
 
-_CATALOG_TABLE_KEYS = [
-    'draws', 'events', 'shaders', 'textures',
-    'render_targets', 'buffers', 'programs',
-    'samplers', 'fbos', 'state_change_events',
-    'counters_per_event', 'descriptor_access',
-    'passes', 'frame_totals',
-    'clears', 'dispatches', 'rt_event_timeline',
-    'vertex_inputs', 'resource_creation',
-    'draw_bindings', 'program_transitions',
-    'pixel_history', 'vbo_samples', 'ibo_samples',
-    'post_vs_samples', 'texture_samples', 'indirect_args',
-    'pass_class_breakdown', 'texture_usage',
-]
+# Derived from the single registry (H-10). schemas.TABLES key order IS the catalog order, so the
+# row_count_* column order baked into the golden root index.html stays byte-identical.
+_CATALOG_TABLE_KEYS = tuple(schemas.TABLES.keys())
 
 
 def _find_manifests(root: str) -> list[tuple[str, str, dict]]:
@@ -55,7 +45,7 @@ def _find_manifests(root: str) -> list[tuple[str, str, dict]]:
             drop_dir = os.path.join(area_dir, drop_entry)
             if not os.path.isdir(drop_dir):
                 continue
-            mf = os.path.join(drop_dir, '_manifest.json')
+            mf = os.path.join(drop_dir, _paths.MANIFEST_NAME)
             if not os.path.exists(mf):
                 continue
             try:
@@ -113,6 +103,9 @@ def build_catalog(root: str) -> dict:
     manifests = _find_manifests(root)
     all_rows: list[dict] = []
     for data_dir, rel_path, m in manifests:
+        # D-7: refuse to (re)build over a drop written under a different SCHEMA_VERSION. This is the
+        # shared chokepoint for `render` (which rebuilds the catalog first) and `catalog`.
+        _manifest.check_schema_version(m, source=rel_path)
         all_rows.extend(_capture_rows(data_dir, rel_path, m))
 
     cols = [

{bobframes-0.1.0 → bobframes-0.2.0}/bobframes/cli.py

@@ -17,6 +17,8 @@ import logging
 import os
 import sys
 
+from .errors import BobFramesError
+
 # CLI report name -> reports submodule. `build(root)` on each builds one report.
 _REPORTS = {
     'draws-by-class': 'draws_by_class',
@@ -25,7 +27,7 @@ _REPORTS = {
     'pass-gpu': 'pass_gpu',
     'shader': 'shader_hotlist',
     'overdraw': 'overdraw',
-    'dashboard': '_dashboard',
+    'dashboard': 'dashboard',
 }
 
 
@@ -68,19 +70,80 @@ def _cmd_ingest(args: argparse.Namespace) -> int:
     if args.render_only:
         rargv += ['--render-only']
     rargv += ['--workers', str(args.workers), '--pixel-grid', str(args.pixel_grid)]
+    if args.replay_timeout is not None:
+        rargv += ['--replay-timeout', str(args.replay_timeout)]
+    if args.convert_timeout is not None:
+        rargv += ['--convert-timeout', str(args.convert_timeout)]
     return run.main(rargv)
 
 
 def _cmd_render(args: argparse.Namespace) -> int:
     from . import run
-    rargv = ['--root', os.path.abspath(args.root), '--render-only']
+    root = os.path.abspath(args.root)
+    rargv = ['--root', root, '--render-only']
     if args.area:
         rargv += ['--area', args.area]
     if args.label:
         rargv += ['--label', args.label]
+    if getattr(args, 'watch', False):
+        return _render_watch(rargv)
     return run.main(rargv)
 
 
+def _watch_paths() -> list[str]:
+    """Files the render --watch loop polls: the design tokens + the modules that emit chrome."""
+    from .reports import chrome as _c, delta as _d, formatters as _fm
+    tokens = os.path.join(os.path.dirname(_c.__file__), 'design_tokens.toml')
+    return [tokens, _c.__file__, _fm.__file__, _d.__file__]
+
+
+def _render_watch(rargv: list[str]) -> int:
+    """Alpha hot-reload (DESIGNER Track A): 500ms mtime poll on the token/chrome files; re-render in a
+    fresh subprocess on change so edited modules/TOML are reloaded. No watchdog dependency."""
+    import subprocess
+    import time
+    log = logging.getLogger('bobframes')
+    watched = _watch_paths()
+
+    def snapshot() -> dict:
+        return {p: (os.stat(p).st_mtime if os.path.exists(p) else 0.0) for p in watched}
+
+    cmd = [sys.executable, '-m', 'bobframes.run', *rargv]
+    rc = subprocess.run(cmd).returncode
+    log.info(f'watching {len(watched)} files; save an edit to re-render (Ctrl+C to stop)')
+    last = snapshot()
+    try:
+        while True:
+            time.sleep(0.5)
+            now = snapshot()
+            if now != last:
+                last = now
+                log.info('change detected; re-rendering')
+                rc = subprocess.run(cmd).returncode
+    except KeyboardInterrupt:
+        return 4
+    return rc
+
+
+def _cmd_preview(args: argparse.Namespace) -> int:
+    from .reports import preview
+    out = preview.build(os.path.abspath(args.root))
+    print(f'wrote {out}')
+    return 0
+
+
+def _cmd_export_tokens(args: argparse.Namespace) -> int:
+    import json
+    from .reports import _tokens, chrome
+    if args.format == 'toml':
+        sys.stdout.write(_tokens.tokens_toml_text())
+    elif args.format == 'json':
+        print(json.dumps(_tokens.load_tokens(), indent=2))
+    else:  # css
+        sys.stdout.write(chrome.design_tokens_css())
+    return 0
+
+
 def _cmd_ab(args: argparse.Namespace) -> int:
     from .reports import ab
     argv = [os.path.abspath(args.root),
@@ -130,20 +193,24 @@ def _cmd_check(args: argparse.Namespace) -> int:
         print('bobframes v1 is Windows-only (qrenderdoc replay requirement). '
               'Track GH issue for Linux/macOS support.', file=sys.stderr)
         return 3
+    from . import config
     if args.write_config:
-        print('--write-config is a v0.2 feature (config layer); not available yet.',
-              file=sys.stderr)
-        return 2
-    from . import qrd_harness, rdcmd
+        path, written = config.write_config_stub(os.path.abspath('.'))
+        print(f'wrote {path}' if written else f'{path} already exists; left unchanged')
+        return 0
+    from .errors import EXIT_TOOL_MISSING, ToolNotFound
     missing = False
-    for name, finder in (('renderdoccmd', rdcmd.find_renderdoccmd),
-                         ('qrenderdoc', qrd_harness.find_qrenderdoc)):
+    for name in ('renderdoccmd', 'qrenderdoc'):
         try:
-            print(f'{name}: {finder()}')
-        except FileNotFoundError:
-            print(f'{name}: NOT FOUND', file=sys.stderr)
+            path, source = config.resolve_tool_verbose(name)
+            # `source` is the winning step's description; for env/config/PATH it explains the
+            # precedence, but for a known-path hit it IS the path -> don't print it twice.
+            suffix = '' if source == path else f'  (via {source})'
+            print(f'{name}: {path}{suffix}')
+        except ToolNotFound as e:
+            print(str(e), file=sys.stderr)
             missing = True
-    return 3 if missing else 0
+    return EXIT_TOOL_MISSING if missing else 0
 
 
 def _cmd_serve(args: argparse.Namespace) -> int:
@@ -191,6 +258,10 @@ def _build_parser() -> argparse.ArgumentParser:
     sp.add_argument('--render-only', action='store_true')
     sp.add_argument('--workers', type=int, default=min(4, os.cpu_count() or 4))
     sp.add_argument('--pixel-grid', type=int, default=4)
+    sp.add_argument('--replay-timeout', type=float, default=None,
+                    help='per-capture qrenderdoc replay budget (s); overrides config')
+    sp.add_argument('--convert-timeout', type=float, default=None,
+                    help='per-file renderdoccmd convert budget (s); overrides config')
     sp.set_defaults(func=_cmd_ingest)
 
     sp = sub.add_parser('render', parents=[common],
@@ -198,6 +269,8 @@ def _build_parser() -> argparse.ArgumentParser:
     sp.add_argument('root', nargs='?', default='.')
     sp.add_argument('--area')
     sp.add_argument('--label')
+    sp.add_argument('--watch', action='store_true',
+                    help='re-render on design_tokens.toml / chrome edits (alpha, 500ms mtime poll)')
     sp.set_defaults(func=_cmd_render)
 
     sp = sub.add_parser('ab', parents=[common], help='all reports for one drop pair')
@@ -222,12 +295,23 @@ def _build_parser() -> argparse.ArgumentParser:
     sp.set_defaults(func=_cmd_lint)
 
     sp = sub.add_parser('check', parents=[common], help='resolve external tool paths')
-    sp.add_argument('--write-config', action='store_true', help='(v0.2)')
+    sp.add_argument('--write-config', action='store_true',
+                    help='write a starter .bobframes.toml to the current dir (skip if present)')
     sp.set_defaults(func=_cmd_check)
 
     sp = sub.add_parser('version', parents=[common], help='print version, schema, pyarrow')
     sp.set_defaults(func=_cmd_version)
 
+    sp = sub.add_parser('preview', parents=[common],
+                        help='render the chrome preview gallery (_reports/_chrome_preview.html; no data)')
+    sp.add_argument('root', nargs='?', default='.')
+    sp.set_defaults(func=_cmd_preview)
+
+    sp = sub.add_parser('export-tokens', parents=[common],
+                        help='print design tokens to stdout as toml|json|css')
+    sp.add_argument('--format', choices=['toml', 'json', 'css'], default='toml')
+    sp.set_defaults(func=_cmd_export_tokens)
+
     sp = sub.add_parser('serve', parents=[common], help='static preview server')
     sp.add_argument('root', nargs='?', default='.')
     sp.add_argument('--port', type=int, default=8000)
@@ -256,10 +340,22 @@ def main(argv: list[str] | None = None) -> int:
 
     _configure_logging(getattr(args, 'verbose', False))
     try:
+        # Load the config singleton against the verb's root so <root>/.bobframes.toml is honored
+        # (§6) for report/catalog/ab/render/ingest/serve. Verbs without a root (version/check/lint)
+        # lazily load bundled defaults (cwd) on first get_config(). ingest/render re-load via
+        # run.main (idempotent). ConfigError surfaces as a typed BobFramesError below.
+        root = getattr(args, 'root', None)
+        if root is not None:
+            from . import config
+            config.load_config(os.path.abspath(root))
         return args.func(args)
     except KeyboardInterrupt:
         print('interrupted', file=sys.stderr)
         return 4
+    except BobFramesError as e:
+        # Typed pipeline/tool errors carry their own exit code (errors.py / ARCHITECTURE §4).
+        print(str(e), file=sys.stderr)
+        return e.exit_code
 
 
 if __name__ == '__main__':