lumitrace 0.1.2 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e03b4d880369e5f87e97dc3d9512afe78f27e310efd2b960a1b2ce545e3a70d1
-  data.tar.gz: '0970a1a220cf484be1d75aa1d51f6004c08ce7bc1154857180247edc44253353'
+  metadata.gz: 8ffc13efecd8f5143486eff6929c93bf73aaa3934bf24fa57e906a2f90498e2c
+  data.tar.gz: df8ce2960f59409dc832d21f9d1df7a2b33cc431fde00c76633f1149a5da3ab3
 SHA512:
-  metadata.gz: 28a83ebc1a957b33da7a9421a78176f4d348cd3963d702f2b926eba5fa3e1b1fb7277d1a8e9635644a71af25c63e5072829ba32e7c3671636d956d266b2eb22a
-  data.tar.gz: 83956ce933a06f9cfd128e35146b7745410215a392b9bcd22008ae26b4568db8369cafdbb93d48bdb7d92785ccb7f342fe3445f4cf38fbba985c392b95637825
+  metadata.gz: 2030ec337dc9a941f1c9878b1faa74e15e605a715e1d2049675115573c7c1357d821ded3a200b348521a9593faf6f719ff93766f5ffe1c623d9a9c6dcaa663b1
+  data.tar.gz: 68564fe5bde3b1d00edd5390211c922b72f3565069446affed3dfdf042fc86491949f8170e175df8cdd99c55b1c05f030adab69fe140ce05cfd003f3f3a51719

data/README.md

@@ -2,6 +2,16 @@
 
 Lumitrace instruments Ruby source code at load time, records expression results, and renders an HTML view that overlays recorded values on your code. It is designed for quick, local “what happened here?” inspection during test runs or scripts.
 
+## Useful links
+
+- [runv/](https://ko1.github.io/lumitrace/runv/): Lumitrace demonstration Ruby playground with inlined tracing
+- [Tutorial](https://ko1.github.io/lumitrace/docs/tutorial.html)
+- [Tutorial in Japanese](https://ko1.github.io/lumitrace/docs/tutorial.ja.html)
+- [Spec](https://ko1.github.io/lumitrace/docs/spec.html)
+- [Supported Syntax](https://ko1.github.io/lumitrace/docs/supported_syntax.html)
+- [GitHub repository](https://github.com/ko1/lumitrace)
+
+
 ## How It Works
 
 Lumitrace hooks `RubyVM::InstructionSequence.translate` (when available) to rewrite files at require-time. It records expression results and renders an HTML view that shows them inline. Only the last N values per expression are kept to avoid huge output.
@@ -10,29 +20,41 @@ Lumitrace hooks `RubyVM::InstructionSequence.translate` (when available) to rewr
 
 ### CLI
 
-Run a script and emit HTML (default output: `lumitrace_recorded.html`):
+Run a script and emit text output (default):
+
+```bash
+lumitrace path/to/entry.rb
+```
+
+Run another command via exec:
+
+```bash
+lumitrace exec rake test
+```
+
+Emit HTML output:
 
 ```bash
-ruby exe/lumitrace path/to/entry.rb
+lumitrace -h path/to/entry.rb
 ```
 
 Limit the number of recorded values per expression (defaults to 3):
 
 ```bash
-LUMITRACE_VALUES_MAX=5 ruby exe/lumitrace path/to/entry.rb
+LUMITRACE_VALUES_MAX=5 lumitrace path/to/entry.rb
 ```
 
 Write JSON output explicitly:
 
 ```bash
-ruby exe/lumitrace path/to/entry.rb --json
-ruby exe/lumitrace path/to/entry.rb --json out/lumitrace_recorded.json
+lumitrace -j path/to/entry.rb
+lumitrace --json=out/lumitrace_recorded.json path/to/entry.rb
 ```
 
 Restrict to specific line ranges:
 
 ```bash
-ruby exe/lumitrace path/to/entry.rb --range path/to/entry.rb:10-20,30-35
+lumitrace --range path/to/entry.rb:10-20,30-35 path/to/entry.rb
 ```
 
 ### Library
@@ -58,18 +80,27 @@ require "lumitrace/enable"
 
 ## Output
 
-- HTML: `lumitrace_recorded.html` by default, override with `LUMITRACE_HTML_OUT`.
-- JSON: written only when `--json` (CLI) or `LUMITRACE_JSON_OUT` (library) is provided. Default filename is `lumitrace_recorded.json`.
+- Text: printed by default; use `--text=PATH` to write to a file.
+- HTML: `lumitrace_recorded.html` by default, or `--html=PATH`.
+- JSON: written only when `--json` (CLI) or `LUMITRACE_JSON` (library/CLI) is provided. Default filename is `lumitrace_recorded.json`.
+- Fork/exec: merged by default. Child processes write fragments under `LUMITRACE_RESULTS_DIR`.
 
 ## Environment Variables
 
 - `LUMITRACE_VALUES_MAX`: default max values per expression (default 3 if unset).
 - `LUMITRACE_ROOT`: root directory used to decide which files are instrumented.
-- `LUMITRACE_HTML_OUT`: override HTML output path.
-- `LUMITRACE_JSON_OUT`: if set, writes JSON to this path at exit.
+- `LUMITRACE_TEXT`: control text output. `1` forces text on, `0`/`false` disables. Any other value is treated as the text output path.
+- `LUMITRACE_HTML`: enable HTML output; `1` uses the default path, otherwise treats the value as the HTML output path. `0`/`false` disables.
+- `LUMITRACE_JSON`: enable JSON output; `1` uses the default path, otherwise treats the value as the JSON output path. `0`/`false` disables.
+- `LUMITRACE_ENABLE`: when `1`/`true`, `require "lumitrace"` will call `Lumitrace.enable!`. When set to a non-boolean string, it is parsed as CLI-style arguments and passed to `enable!`.
+- `LUMITRACE_VERBOSE`: when `1`/`true`, prints verbose logs to stderr.
+- `LUMITRACE_RANGE`: semicolon-separated range specs (e.g. `a.rb:1-3,5-6;b.rb`).
+- `LUMITRACE_RESULTS_DIR`: internal use. Shared results directory for fork/exec merge (default: `Dir.tmpdir/lumitrace_results/<user>_<parent_pid>`).
+- `LUMITRACE_RESULTS_PARENT_PID`: internal use. Parent PID for fork/exec merge (auto-set).
 - `LUMITRACE_GIT_DIFF=working|staged|base:REV|range:SPEC`: diff source for `enable_git_diff`.
 - `LUMITRACE_GIT_DIFF_CONTEXT=N`: expand diff hunks by +/-N lines (default 3).
 - `LUMITRACE_GIT_CMD`: git executable override (default `git`).
+- `LUMITRACE_GIT_DIFF_UNTRACKED`: include untracked files in git diff ranges (`1` default). Set to `0` to exclude.
 
 ## Notes And Limitations
 
@@ -88,5 +119,5 @@ bundle install
 Run the CLI locally:
 
 ```bash
-ruby exe/lumitrace path/to/entry.rb
+lumitrace path/to/entry.rb
 ```

data/docs/spec.md

@@ -0,0 +1,225 @@
+---
+---
+
+# Lumitrace Spec
+
+## Overview
+
+Lumitrace instruments Ruby source code at load time (via `RubyVM::InstructionSequence.translate` when available), records expression results, and renders text or HTML output that overlays recorded values on your code. It is designed for local “what happened here?” inspection.
+
+## Goals
+
+- Record expression results with minimal friction.
+- Limit recorded data size (keep only the last N values per expression).
+- Show recorded values inline on a per-file HTML view.
+- Support require-time instrumentation for multiple files.
+
+## Non-Goals
+
+- Production-safe tracing.
+- Perfect semantic preservation for all Ruby edge cases.
+
+## Public API
+
+### `require "lumitrace"`
+
+- Arguments: none.
+- Returns: nothing.
+- Side effects: loads core code only (no instrumentation, no `at_exit`).
+
+### Library entry points (common usage)
+
+- `require "lumitrace"` + `Lumitrace.enable!(...)`
+- `require "lumitrace/enable"` (calls `Lumitrace.enable!`)
+- `require "lumitrace/enable_git_diff"` (diff-scoped `Lumitrace.enable!`)
+- `LUMITRACE_ENABLE=1` + `require "lumitrace"` (auto-`enable!`)
+- `LUMITRACE_ENABLE="-t -h -j ..."` + `require "lumitrace"` (CLI-style options parsed and passed to `enable!`)
+
+### `Lumitrace.enable!(max_values: nil, ranges_by_file: nil, root: nil, text: nil, html: nil, json: nil, verbose: nil, at_exit: true)`
+
+- Arguments:
+  - `max_values`: integer, string, or nil.
+  - `ranges_by_file`: hash or nil. `{ "/path/to/file.rb" => [1..5, 10..12] }`.
+  - `text`: boolean or string or nil. When nil, determined from environment variables. When string, uses it as the text output path.
+  - `html`: boolean or string or nil. When nil, determined from environment variables.
+  - `json`: boolean or string or nil. When nil, determined from environment variables.
+  - `verbose`: boolean or nil. When nil, determined from `LUMITRACE_VERBOSE`.
+  - `at_exit`: boolean. When true, registers output at exit.
+- Returns: `nil`.
+- Side effects:
+  - Enables require-time instrumentation.
+  - Registers a single `at_exit` hook (if `at_exit: true`).
+  - Fixes the HTML output directory to the `Dir.pwd` at call time.
+- Root scope for instrumentation uses `root` if provided, otherwise `ENV["LUMITRACE_ROOT"]` if set, otherwise `Dir.pwd`.
+- Environment variables (resolved by `Lumitrace.enable!`):
+  - `LUMITRACE_VALUES_MAX`: default max values per expression when `max_values` is nil (default 3 if unset).
+  - `LUMITRACE_ROOT`: root directory used to decide which files are instrumented.
+  - `LUMITRACE_HTML`: enable HTML output; `1` uses the default path, otherwise treats the value as the HTML output path. `0`/`false` disables.
+  - `LUMITRACE_TEXT`: control text output. `1` forces text on, `0`/`false` disables. If unset, text is enabled only when both HTML and JSON are disabled. Any other value is treated as the text output path.
+  - `LUMITRACE_JSON`: enable JSON output; `1` uses the default path, otherwise treats the value as the JSON output path. `0`/`false` disables.
+  - `LUMITRACE_GIT_DIFF_UNTRACKED`: include untracked files in git diff ranges (`1` default). Set to `0` to exclude.
+  - `LUMITRACE_VERBOSE`: when `1`/`true`, prints verbose logs to stderr.
+  - `LUMITRACE_ENABLE`: when `1`/`true`, `require "lumitrace"` will call `Lumitrace.enable!`. When set to a non-boolean string, it is parsed as CLI-style arguments and passed to `enable!`.
+  - `LUMITRACE_RANGE`: semicolon-separated range specs, e.g. `a.rb:1-3,5-6;b.rb`.
+  - `LUMITRACE_RESULTS_DIR`: internal use. Shared results directory for fork/exec merge (default: `Dir.tmpdir/lumitrace_results/<user>_<parent_pid>`).
+  - `LUMITRACE_RESULTS_PARENT_PID`: internal use. Parent PID for fork/exec merge (auto-set).
+
+### `Lumitrace.disable!`
+
+- Arguments: none.
+- Returns: `nil`.
+- Side effects: disables instrumentation (does not clear recorded events).
+
+
+### `require "lumitrace/enable"`
+
+- Arguments: none.
+- Returns: nothing.
+- Side effects: calls `Lumitrace.enable!` with default arguments.
+
+### `require "lumitrace/enable_git_diff"`
+
+- Arguments: none.
+- Returns: nothing.
+- Side effects:
+  - Computes `ranges_by_file` from `git diff` scoped to the current program file.
+  - Calls `Lumitrace.enable!` when diff is non-empty.
+- Environment variables:
+  - `LUMITRACE_GIT_DIFF=working|staged|base:REV|range:SPEC` selects diff source.
+  - `LUMITRACE_GIT_DIFF_CONTEXT=N` expands hunks by +/-N lines (default 3; negative treated as 0).
+  - `LUMITRACE_GIT_CMD` overrides the git executable (default: `git`).
+  - `LUMITRACE_RANGE` can be used to pass explicit ranges via env.
+
+## Instrumentation
+
+### Activation
+
+- Call `Lumitrace.enable!`.
+- Hook `RubyVM::InstructionSequence.translate` to rewrite files at load time.
+- Only instrument files under the configured root directory.
+- Optional: restrict instrumentation to specific line ranges per file.
+
+### Root Scope
+
+- Root is `Dir.pwd` (or `ENV["LUMITRACE_ROOT"]` if set).
+- Files outside root are ignored.
+
+### Exclusions
+
+- Tool files are excluded to avoid self-instrumentation:
+  - `record_instrument.rb`
+  - `record_require.rb`
+  - `generate_resulted_html.rb`
+
+### Rewriting Strategy
+
+- AST is parsed with Prism.
+- For each node, if it matches “wrapable” expression classes, injects:
+  - `RecordInstrument.expr_record(file, start_line, start_col, end_line, end_col, (expr))`
+- Insertions are done by offset to preserve original formatting.
+
+### Range Filtering
+
+- `ranges_by_file` is a hash like `{ "/path/to/file.rb" => [1..5, 10..12] }`.
+- When provided, only files listed in the hash are instrumented.
+- For a listed file, only expressions whose start line falls within the listed ranges are instrumented.
+- If a listed file has `nil` or an empty array for ranges, the entire file is instrumented.
+- HTML rendering respects the same ranges and only shows files that produced events.
+
+### Wrap Targets
+
+- `CallNode` (except those with block bodies)
+- Variable reads:
+  - `LocalVariableReadNode`
+  - `ConstantReadNode`
+  - `InstanceVariableReadNode`
+  - `ClassVariableReadNode`
+  - `GlobalVariableReadNode`
+- Literal nodes are excluded (e.g. integer, string, true/false, nil, etc.)
+
+## Recording
+
+- Results are stored per expression key:
+  - `(file, start_line, start_col, end_line, end_col)`
+- Keep only the last N values (`max_values_per_expr`, default 3).
+- Track `total` count for how many times the expression executed.
+- Values are stored via `inspect` for non-primitive types.
+- String values are truncated to 1000 bytes for storage.
+
+## Fork/Exec Merge
+
+- Fork/exec results are merged by default.
+- The parent process writes final text/HTML/JSON.
+- Child processes write JSON fragments under `LUMITRACE_RESULTS_DIR` and do not write final outputs.
+- When `Process._fork` is available, Lumitrace hooks it to reset child events immediately after fork.
+- `exec` inherits `LUMITRACE_RESULTS_DIR` and `LUMITRACE_RESULTS_PARENT_PID` via the environment. `Lumitrace.enable!` also appends `-rlumitrace` to `RUBYOPT` to ensure the exec'd process loads Lumitrace.
+
+### Output JSON
+
+`lumitrace_recorded.json` contains an array of entries:
+
+```json
+{
+  "file": "/path/to/file.rb",
+  "start_line": 10,
+  "start_col": 4,
+  "end_line": 10,
+  "end_col": 20,
+  "values": ["..."],
+  "total": 123
+}
+```
+
+## CLI
+
+### `lumitrace`
+
+```
+lumitrace [options] script.rb [ruby_opt]
+lumitrace [options] exec CMD [args...]
+```
+
+- Text is rendered by default (from in-memory events; no JSON file is required).
+- `-t` enables text output to stdout. `--text=PATH` writes to a file.
+- `-h` enables HTML output (default path). `--html=PATH` writes to a file.
+- `-j` enables JSON output (default path). `--json=PATH` writes to a file.
+- `-g` enables git diff with `working` mode. `--git-diff=MODE` selects `staged|base:REV|range:SPEC`.
+- `--max` sets max values per expression.
+- `--range` restricts instrumentation per file (`FILE` or `FILE:1-5,10-12`). Can be repeated.
+- `--git-diff=MODE` restricts instrumentation to diff hunks (`staged|base:REV|range:SPEC`).
+- `--git-diff-context` expands hunks by +/-N lines.
+- `--git-cmd` overrides the git executable.
+- `--git-diff-no-untracked` excludes untracked files (untracked files are included by default).
+- `--verbose` prints verbose logs to stderr.
+- `LUMITRACE_VALUES_MAX` sets the default max values per expression.
+- The CLI launches a child process (Ruby or `exec` target) with `RUBYOPT=-rlumitrace` and `LUMITRACE_*` env vars.
+
+### Text Output (CLI)
+
+- Text output starts with a header line: `=== Lumitrace Results (text) ===`.
+- Each file is printed with a header: `### path/to/file.rb`.
+- Each line is prefixed with a line number like ` 12| `.
+- Skipped ranges are represented by a line containing `...`.
+- Only the last value is shown per expression; if an expression ran multiple times, the last value is annotated with the ordinal run (e.g., `#=> 2 (3rd run)`).
+- When `--text` is used and `--max` is not provided, `max_values` defaults to `1`.
+- When `ranges_by_file` is provided, only files present in the hash are shown in text output.
+
+## HTML Rendering
+
+- `GenerateResultedHtml.render_all` renders all files in one page.
+- Each file is shown in its own section.
+- Expressions are marked with an inline icon.
+- Hovering the icon shows recorded values.
+- Only the last 3 values are shown in the tooltip; additional values are summarized as `... (+N more)`.
+- Tooltip is scrollable horizontally for long values.
+
+### Copy/Paste Behavior
+
+- Inline icon uses a separate marker span to reduce copy/paste artifacts.
+- Lines are rendered as inline spans with explicit `\n` inserted.
+
+## Known Limitations
+
+- Requires `RubyVM::InstructionSequence.translate` support in the Ruby build.
+- Instrumentation is for debugging; semantics may change for unusual edge cases.
+- Tool does not attempt to preserve file encoding comments.

data/docs/supported_syntax.md

@@ -0,0 +1,117 @@
+---
+---
+
+# Supported Syntax
+
+Lumitrace instruments Ruby source by wrapping selected expression nodes with
+`Lumitrace::RecordInstrument.expr_record(...)`. It does **not** rewrite the
+entire AST, so coverage is best described as "expressions that are safe to wrap
+in parentheses and call-position contexts."
+
+This document lists what is supported today, and what is intentionally skipped
+to avoid breaking valid Ruby syntax.
+
+## Supported (Instrumented)
+
+The following node kinds are instrumented when they appear in normal expression
+positions:
+
+- Method calls (`Prism::CallNode`)
+  - Example:
+    ```ruby
+    foo(bar)
+    ```
+- Local variable reads (`Prism::LocalVariableReadNode`)
+  - Example:
+    ```ruby
+    x
+    ```
+- Numbered block parameter reads (`Prism::ItLocalVariableReadNode`)
+  - Example:
+    ```ruby
+    it
+    ```
+- Constant reads (`Prism::ConstantReadNode`)
+  - Example:
+    ```ruby
+    SomeConst
+    ```
+- Instance variable reads (`Prism::InstanceVariableReadNode`)
+  - Example:
+    ```ruby
+    @value
+    ```
+- Class variable reads (`Prism::ClassVariableReadNode`)
+  - Example:
+    ```ruby
+    @@count
+    ```
+- Global variable reads (`Prism::GlobalVariableReadNode`)
+  - Example:
+    ```ruby
+    $stdout
+    ```
+
+Notes:
+- Nodes that are not expression reads/calls are generally left as-is (e.g.,
+  definitions, control flow, assignment statements, etc.).
+
+## Not Supported (Skipped)
+
+These are intentionally skipped to keep output valid Ruby:
+
+- Definitions and structural nodes (the entire node is never wrapped):
+  - `def`, `class`, `module`, `if`, `case`, `while`, `begin`, `rescue`, etc.
+  - Example:
+    ```ruby
+    def foo
+      bar
+    end
+    ```
+
+- Literals (not wrapped):
+  - `1`, `"str"`, `:sym`, `true`, `false`, `nil`
+
+- Method calls that have a block with body (`do ... end` / `{ ... }`) are
+  instrumented at the call expression level. Example:
+  ```ruby
+  items.each do |x|
+    x + 1
+  end
+  ```
+
+- Alias statements (both aliasing globals and methods):
+  - `alias $ERROR_INFO $!`
+  - `alias old_name new_name`
+
+- The receiver part of a singleton method definition:
+  - Example:
+    ```ruby
+    def Foo.bar
+      1
+    end
+    ```
+  - `Foo` is **not** instrumented here.
+
+- Embedded variable nodes inside interpolated strings:
+  - Example:
+    ```ruby
+    "#@path?#@query"
+    ```
+  - `@path` and `@query` inside the interpolation are **not** instrumented.
+
+- Implicit keyword argument values (`token:` style):
+  - Example:
+    ```ruby
+    ec2_metadata_request(EC2_IAM_INFO, token:)
+    ```
+  - The implicit `token` read is **not** instrumented.
+
+## Rationale
+
+All skips above correspond to syntactic positions where wrapping the token with
+`expr_record(...)` would change the Ruby grammar (e.g., alias operands, method
+name positions, or implicit keyword arguments).
+
+If you want additional coverage, we can add more targeted rewrites, but they
+must preserve valid syntax in those special contexts.