ruby-test-ide 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 8573fe0b8988789da6eabe7b6baf637a1202836f754c3a42a2bd96382bccd9b5
+  data.tar.gz: 53184e3a3e26f9d2f68380bca1f7c4bbc61f0abdf0e7d847c541df3e519c22be
+SHA512:
+  metadata.gz: e0892bf3f0e8357194dc3642aae916be6b758ae8ef9a568a1cf2850481cc97beabc5963cdc8603696c838877dfbb41c6bc9e0424b8d92f8e1253259a7ea660c0
+  data.tar.gz: a20ca8a9aad2c16cb30d9be04c0efde597d8199c32cfeb81ecde8e5278355069c5991b80874eeab2912f94ae1486f54ed0993d20583bd9635ccd89ba11569403

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Samuel Garratt
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,218 @@
+# ruby-test-ide
+
+A browser IDE for Ruby whose intelligence comes from **running your code and
+asking the live objects what they can do** — the same trick `irb` uses —
+plus argument/return types **observed while your unit tests run**, instead of
+static analysis or type-annotation files (sorbet/RBS).
+
+```
+t = 'test'     # evaluated for real, in a sandboxed worker process
+t.st           # completion asks the actual String instance:
+               #   -> start_with?  strip  strip!   (with arity + owner)
+```
+
+## Run it
+
+```bash
+gem install ruby-test-ide
+cd your/ruby/project
+ruby-test-ide                 # prints e.g. http://localhost:3000/?token=<random> — open that
+ruby-test-ide -p 4000 -w path/to/project   # or pick port/workspace
+```
+
+The server executes arbitrary code from any request it accepts, so a random
+access token is required by default — copy the full URL it prints (including
+`?token=...`); the page then attaches that token to every API call itself,
+so you won't need to touch the URL again for that session. `--no-auth`
+disables the token for CI/automation; `--token yourvalue` pins a fixed one.
+
+From this repo (development):
+
+```bash
+ruby exe/ruby-test-ide --workspace workspace   # demo workspace (prints its own token)
+rake test        # unit tests (pure helpers, worker round-trip, auth gate)
+rake e2e         # browser suite (Playwright, isolated fixture workspace, --no-auth)
+rake build       # pkg/ruby-test-ide-x.y.z.gem
+rake install     # build + install locally
+rake release     # tag vX.Y.Z, push, publish to rubygems (maintainer only)
+```
+
+Zero runtime dependencies (TCPServer, RDoc/ri and TracePoint are stdlib);
+ruby >= 3.2, verified on 3.2.2 and 3.4.9 — syntax diagnostics understand both
+parser generations' error formats (3.4's Prism messages are richer and
+surface as better squiggles).
+
+The sidebar shows a collapsible folder tree (state persists across reloads)
+of Ruby files (`*.rb`, `Gemfile`, `Rakefile` — not a general file browser, so
+opening something never force-parses non-Ruby content as Ruby), is resizable
+by dragging its right edge, opens files, `＋` creates them, Cmd/Ctrl+S saves;
+unsaved buffers keep their edits when you switch files and are marked with a
+dot. All paths are fenced inside the workspace root (traversal rejected
+server-side). An empty directory is seeded with a two-file example on first
+boot.
+
+What you get in the editor:
+
+| Goal (CLAUDE.md) | How it works here |
+|---|---|
+| Autocomplete of all methods + parameter counts | On `.`, the buffer up to the cursor is evaluated in a worker; the receiver expression is `eval`'d against that binding and its **live methods** are listed with `Method#parameters`-derived signatures, arity and owner class. Works for chains (`t.strip.rev`), literals (`[1,2].su`), hash elements (`person[:name].cap`), user-defined methods, locals, constants, keywords. |
+| Ruby docs on hover | Hovering a method looks up the real `ri` database in-process (`RDoc::RI::Driver`, memoised, ~0.15 ms warm). Hovering a variable shows its class **and current value**. |
+| Error highlighting | Syntax errors (compile-only check, never executed) and runtime errors (from live eval, with the failing line) become Monaco squiggles. |
+| `irb`-grade experience | Better: with **Live eval** on, every edit re-runs the buffer under a `TracePoint`, so each line gets a grey debugger annotation of the locals it changed (`# t: String = "test"`), and the side panel shows the final state of every variable + captured stdout. |
+| Go to definition | Cmd/Ctrl+click or F12 on any method jumps to its `Method#source_location` — the runtime answer, so it works for methods defined dynamically. Workspace files open in place; gem/core locations are shown in the status bar. |
+| Multi-file workspace | Buffers evaluate *in place*: the worker chdirs to the file's directory and shims `require_relative`, so `main.rb` can require `lib/greeter.rb` and completion on a `Greeter` instance lists methods **defined in the other file** — signatures included, unsaved edits included. |
+| Gems via Gemfile | If the workspace has a `Gemfile`, the worker activates it (`bundler/setup`) before evaluating, so `require 'rest-client'` resolves exactly as under `bundle exec` and completion/hover work on gem objects (`RestClient.` → `get(url, headers = …, &block)`). Run `bundle install` with the same ruby the server uses; a missing gem surfaces as a friendly error in the panel. |
+| Types from any test framework or script | **Learn from tests** runs the commands from `.ruby_ide.yaml` (or auto-detected test files) with a type observer preloaded via `RUBYOPT` into every ruby process they start. Works with minitest, rspec, rake, cucumber, or a plain script — the IDE records, per method, the argument/return classes actually seen with sample values (`.ruby-ide/observations.json`, merged across commands and subprocesses). Completion falls back to this oracle when live eval can't produce the object — `w.current('x').` completes as `Hash` even though `current` sits on top of a mocked network call — and hover shows "Observed in unit tests: current(city: String) → Hash (e.g. {...}) — 3 calls". |
+| Access control | A random token (Jupyter-style) is required on every route except `/health` and the vendored static assets, since this server executes whatever code it's sent. Checked via an `X-Ruby-Test-Ide-Token` header (attached automatically by the page once loaded) or a `?token=` query param on first load; comparison is constant-time. `--no-auth` / `--token` override it. |
+
+`npx playwright test --project=chromium` runs the end-to-end suite
+(autocomplete, live state, diagnostics) against a real browser; the config
+boots its own server against an isolated fixture workspace automatically.
+
+## Architecture
+
+```
+browser (lib/ruby_test_ide/public/index.html, Monaco vendored — no internet needed)
+   │  POST /api/complete /hover /diagnostics /run   (JSON)
+   ▼
+lib/ruby_test_ide/server.rb — zero-gem TCPServer HTTP server
+   │  • splits "line before cursor" into receiver expression + prefix
+   │    (backward scanner: balanced brackets, strings, chains, ranges)
+   │  • syntax check via RubyVM::InstructionSequence.compile (never executes)
+   │  • ri doc lookup + memoised cache (RDoc::RI::Driver in-process,
+   │    `ri` CLI fallback)
+   │  spawns per request ──────────────┐
+   ▼                                   │ stdin: one JSON request
+lib/ruby_test_ide/runner.rb — worker   │ fd 3:  one JSON reply (user stdout
+   • fresh method-body binding         │        can't corrupt the protocol)
+     (self == main, no leaked locals)  │ pgroup + kill on 4s timeout
+   • eval(code_before, bind) — largest │ rlimit_cpu belt-and-braces
+     syntactically-valid prefix, keeps
+     partial state when code raises mid-way
+   • completion: eval(receiver).methods + Method#parameters
+   • run: TracePoint(:line) captures locals per line = debugger view
+```
+
+Design decisions that came out of research into irb/repl_type_completor:
+
+* **fd 3 for results, child process group, hard kill** — user code that
+  `puts`, forks, or loops forever cannot corrupt or hang the IDE.
+* **Method-body binding as sandbox** — `TOPLEVEL_BINDING.eval('binding')`
+  leaks the host script's locals into completions; a `def ...; binding; end`
+  binding is clean and keeps script semantics (`self` is `main`).
+* **`:line` TracePoint events fire *before* the line runs** — state shown
+  for line N is taken from the event for line N+1 (and from the binding
+  itself after eval returns, for the last line).
+* **C-implemented core methods report `[[:rest]]`** (`gsub(*args)`) — real
+  parameter names for those come from the ri call-seq shown on hover.
+  (`Method#parameters` is fully informative for methods written in Ruby,
+  which includes gem code.)
+* **`ENV['BUNDLER_VERSION']` is pinned before `bundler/setup`** — otherwise a
+  lockfile `BUNDLED WITH` an older installed bundler makes setup *re-exec*
+  the worker, whose stdin (the request) is already consumed.
+* **Monaco's word pattern is overridden for Ruby** — the default splits
+  `start_with?` before the `?`, so hover/definition would query a method
+  that doesn't exist. Receiver evaluation for hover/completion also carries
+  its own 2s timeout, falling back to test observations, so hovering a chain
+  built on a slow network call stays responsive.
+* **Monaco is vendored, not CDN-loaded** (`lib/ruby_test_ide/public/vendor/monaco`,
+  MIT-licensed, its own LICENSE ships alongside) — the IDE works fully offline
+  after `gem install`. Assets are served with a 1-year immutable cache, safe
+  only because `index.html` is templated at request time to cache-bust every
+  vendored URL with `?v=<gem version>` — an upgrade can't serve a stale
+  cached Monaco. A worker built from a `blob:` URL resolves `importScripts()`
+  against its own opaque origin, not the page's, so that URL must be
+  absolute (`window.location.origin`), not root-relative.
+* **The token is only ever embedded into `index.html` after that request's
+  own token already checked out** — `/` requires `?token=...` like every
+  other route, so there's no window where an unauthenticated request could
+  see it. HTTP header names are case-insensitive but not hyphen-insensitive:
+  `X-RubyTestIDE-Token` downcases to `x-rubytestide-token` (no internal
+  hyphens), which silently failed against a hand-written
+  `x-ruby-test-ide-token` check — caught by a test that reads the real
+  header name out of `index.html` and drives it through `Server#route`,
+  rather than hand-copying the same literal on both sides.
+
+## Docs prerequisite
+
+Hover docs need an ri database. rvm rubies often ship without one; generate it
+once (~20 s) with:
+
+```bash
+cd ~/.rvm/src/ruby-3.2.2 && rdoc --all --ri \
+  --op ~/.rvm/rubies/ruby-3.2.2/share/ri/3.2.0/system .
+```
+
+Gems installed with the `--no-document` default have no ri data; add
+`gem: --document=ri` to `~/.gemrc`, or backfill a gem with
+`rdoc --ri --op "$(gem spec NAME doc_dir 2>/dev/null || echo $GEM_HOME/doc/NAME-VERSION)/ri" $GEM_HOME/gems/NAME-VERSION/lib`.
+
+## Honest limitations (it's a PoC)
+
+* **It executes your code.** That is the point — but treat it like running
+  `ruby file.rb`: file/network side effects happen (worker is killed after
+  4 s, though, so infinite loops are fine). A "Live eval" toggle turns
+  continuous execution off.
+* Completion inside an unfinished `def`/block falls back to the largest
+  parseable prefix, so method-local variables aren't visible until the
+  block closes (irb has the same blind spot mid-edit).
+* Each request pays worker spawn cost (~50 ms) plus a full re-run of the
+  buffer — fine for scripts, wrong for slow/side-effectful programs.
+
+## How "learn from tests" works
+
+Configure it in `workspace/.ruby_ide.yaml` — any executable counts, because
+the observer doesn't run your tests, it *rides along* in them:
+
+```yaml
+learn:
+  - bundle exec rspec
+  - bundle exec rake test
+  - ruby scripts/exercise_everything.rb
+timeout: 60          # seconds per command
+```
+
+```
+POST /api/learn
+  server: read .ruby_ide.yaml learn: commands
+          (fallback: guess from test/**/*_test.rb and spec/)
+  each command runs from the workspace root with
+          RUBYOPT="-r .../observer.rb"  RUBY_IDE_OBS_DIR  RUBY_IDE_OBS_ROOT
+  observer (framework-agnostic, inherited by every ruby subprocess):
+          TracePoint(:call, :return) filtered to workspace paths
+          → per method: {args: {name: {Class: sample}}, returns: {Class: sample}, calls}
+          → dumped to one JSON file per process at exit
+  server: merge dumps → workspace/.ruby-ide/observations.json
+completion/hover workers read the observations file:
+  - receiver eval fails → trailing method name → most-seen return class
+    → complete on that class's instance_methods (flagged "observed in tests")
+  - hover merges live Method#parameters with observed types + sample values;
+    methods with no live object still get a full hover from observations
+```
+
+Mocks are the mechanism, not a problem: tests stub the network edge
+(`def @weather.fetch_json(_) = SAMPLE`), so everything *above* the stub runs
+for real and the recorded types are real. The observer registers its
+`at_exit` before minitest/rspec autorun hooks, so it dumps after the suite
+finishes; paths like `<internal:hash>` are excluded so core methods don't
+pollute the data. Note: `learn:` commands are arbitrary shell commands run
+with your privileges — the same trust level as the IDE evaluating your code.
+Ruby 3.4 note: minitest is a bundled gem, so it must be listed in the
+workspace Gemfile to load under bundler.
+
+## Where this goes next
+
+1. **Persist + cache context** (as CLAUDE.md suggests): keep a warm worker
+   per buffer, snapshot line-states, invalidate only below the edited line.
+2. **Wrap it as an LSP server** — the runner already produces everything
+   `textDocument/completion|hover|publishDiagnostics` need, which would make
+   the same engine drive VS Code natively as an extension.
+3. **Hybrid inference for un-run code paths**: seed types from the live
+   binding, then abstract-interpret the current expression the way
+   `repl_type_completor` (irb's opt-in type completor) does — no execution
+   of the hovered chain, RBS-aware, still runtime-grounded.
+4. **Deeper project awareness**: the file tree, per-file buffers and
+   cross-file requires exist now; next is rdoc generation for the user's own
+   lib/ (hover docs for their classes), file rename/delete, and evaluating
+   entry points other than the open buffer (e.g. run tests to learn types on
+   code paths the buffer doesn't reach).

data/exe/ruby-test-ide

@@ -0,0 +1,42 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'optparse'
+require 'securerandom'
+require_relative '../lib/ruby_test_ide/version'
+
+options = { port: 3000, workspace: Dir.pwd, token: nil, no_auth: false }
+OptionParser.new do |opts|
+  opts.banner = <<~BANNER
+    ruby-test-ide #{RubyTestIDE::VERSION} — runtime-reflection IDE for Ruby
+
+    Serves an IDE for a project directory. Code is EXECUTED in sandboxed
+    workers to power completion/hover, and "Learn from tests" observes your
+    test runs to know types on mocked/unreached paths. A random access token
+    is required by default (printed on boot) — this server will run
+    whatever it's asked to.
+
+    Usage: ruby-test-ide [options]
+  BANNER
+  opts.on('-p', '--port PORT', Integer, 'Port to listen on (default 3000)') { |v| options[:port] = v }
+  opts.on('-w', '--workspace DIR', 'Project directory to serve (default: current directory)') { |v| options[:workspace] = v }
+  opts.on('--token TOKEN', 'Use this token instead of generating a random one') { |v| options[:token] = v }
+  opts.on('--no-auth', 'Disable the access token (e.g. CI) — INSECURE: anyone reaching the port can run code as you') do
+    options[:no_auth] = true
+  end
+  opts.on('-v', '--version', 'Print version') do
+    puts RubyTestIDE::VERSION
+    exit
+  end
+end.parse!
+
+# Constants in the server derive from these — set before the require.
+ENV['RUBY_TEST_IDE_WORKSPACE'] = File.expand_path(options[:workspace])
+if options[:no_auth]
+  ENV['RUBY_TEST_IDE_NO_AUTH'] = '1'
+else
+  ENV['RUBY_TEST_IDE_TOKEN'] = options[:token] || SecureRandom.hex(24)
+end
+
+require_relative '../lib/ruby_test_ide'
+RubyTestIDE.start(port: options[:port])

data/lib/ruby_test_ide/observer.rb

@@ -0,0 +1,97 @@
+# frozen_string_literal: true
+
+# RubyIDE type observer.
+#
+# Preloaded (via RUBYOPT="-r.../observer.rb") into every ruby process a
+# "learn" command starts — rspec, minitest, rake, a plain script, and any
+# ruby subprocess those spawn (RUBYOPT is inherited). Framework-agnostic:
+# it doesn't run tests, it only watches methods defined under
+# RUBY_IDE_OBS_ROOT while the host process does whatever it does, then dumps
+# {method => {args, returns, samples, calls}} to a JSON file in
+# RUBY_IDE_OBS_DIR at exit.
+#
+# Registered before the host's own code loads, so its at_exit runs AFTER
+# minitest/rspec autorun hooks (at_exit is LIFO) — observations include the
+# full test run. Every path is rescued: this file must never be the reason
+# a user's test suite fails.
+
+begin
+  ruby_ide_root = ENV['RUBY_IDE_OBS_ROOT'].to_s
+  ruby_ide_dir = ENV['RUBY_IDE_OBS_DIR'].to_s
+  unless ruby_ide_root.empty? || ruby_ide_dir.empty?
+    require 'json'
+
+    ruby_ide_class_of = ->(obj) { ::Kernel.instance_method(:class).bind_call(obj).to_s }
+    ruby_ide_inspect = lambda do |val|
+      s = val.inspect
+      s.length > 140 ? "#{s[0, 139]}…" : s
+    rescue Exception # rubocop:disable Lint/RescueException
+      '(uninspectable)'
+    end
+
+    ruby_ide_observations = {}
+    # The entry script's path can be relative (`ruby test/foo_test.rb`);
+    # expand once per distinct path, not per event.
+    ruby_ide_path_cache = {}
+    ruby_ide_in_root = lambda do |path|
+      ruby_ide_path_cache.fetch(path) do
+        ruby_ide_path_cache[path] =
+          if path.start_with?('<', '(') # <internal:hash>, (eval), (irb)…
+            false
+          else
+            begin
+              File.expand_path(path).start_with?(ruby_ide_root)
+            rescue StandardError
+              false
+            end
+          end
+      end
+    end
+
+    ruby_ide_tp = TracePoint.new(:call, :return) do |t|
+      path = t.path.to_s
+      next unless ruby_ide_in_root.call(path)
+
+      begin
+        entry = ruby_ide_observations["#{t.defined_class}##{t.method_id}"] ||= {
+          'method' => t.method_id.to_s, 'owner' => t.defined_class.to_s,
+          'file' => path, 'line' => t.lineno, 'calls' => 0,
+          'args' => {}, 'returns' => {}
+        }
+        if t.event == :call
+          entry['calls'] += 1
+          b = t.binding
+          t.parameters.each do |_type, pname|
+            next unless pname.is_a?(Symbol)
+
+            begin
+              val = b.local_variable_get(pname)
+            rescue StandardError
+              next
+            end
+            slot = entry['args'][pname.to_s] ||= {}
+            slot[ruby_ide_class_of.call(val)] ||= ruby_ide_inspect.call(val) if slot.size < 4
+          end
+        else
+          slot = entry['returns']
+          slot[ruby_ide_class_of.call(t.return_value)] ||= ruby_ide_inspect.call(t.return_value) if slot.size < 4
+        end
+      rescue Exception # rubocop:disable Lint/RescueException
+        # observation must never break the host process
+      end
+    end
+    ruby_ide_tp.enable
+
+    at_exit do
+      ruby_ide_tp.disable
+      File.write(
+        File.join(ruby_ide_dir, "obs-#{Process.pid}-#{Time.now.to_f.to_s.tr('.', '')}.json"),
+        JSON.generate(ruby_ide_observations.first(2000).to_h)
+      )
+    rescue Exception # rubocop:disable Lint/RescueException
+      nil
+    end
+  end
+rescue Exception # rubocop:disable Lint/RescueException
+  nil
+end