woods 2.0.0.beta3 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (120) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +500 -420
  3. data/CONTRIBUTING.md +29 -17
  4. data/README.md +78 -178
  5. data/docs/AGENT_GUIDE.md +52 -11
  6. data/docs/AGENT_SETUP.md +34 -17
  7. data/docs/AUTOMATIC_MAINTENANCE.md +222 -0
  8. data/docs/BACKEND_MATRIX.md +18 -7
  9. data/docs/CLIENT_HOOKS.md +1 -1
  10. data/docs/CONFIGURATION_REFERENCE.md +105 -29
  11. data/docs/CONSOLE_MCP_SETUP.md +54 -9
  12. data/docs/DOCKER_SETUP.md +16 -1
  13. data/docs/EVALUATION.md +10 -4
  14. data/docs/EXTRACTOR_REFERENCE.md +23 -3
  15. data/docs/FAQ.md +14 -3
  16. data/docs/GETTING_STARTED.md +18 -17
  17. data/docs/INCREMENTAL_EXTRACTION.md +37 -8
  18. data/docs/INDEX_LAYOUT.md +2 -2
  19. data/docs/MCP_SERVERS.md +79 -7
  20. data/docs/MCP_TOOL_COOKBOOK.md +5 -5
  21. data/docs/MCP_WORKTREE_SETUP.md +55 -83
  22. data/docs/PUBLISHED_INDEX.md +17 -0
  23. data/docs/README.md +2 -1
  24. data/docs/RETRIEVAL_GUIDE.md +81 -13
  25. data/docs/SOURCE_FRESHNESS.md +1 -1
  26. data/docs/TOKEN_BENCHMARK.md +16 -10
  27. data/docs/TROUBLESHOOTING.md +142 -47
  28. data/docs/UPGRADING_TO_2.md +12 -6
  29. data/docs/WATCH_DAEMON.md +189 -24
  30. data/docs/WHY_WOODS.md +9 -5
  31. data/exe/woods-console +13 -11
  32. data/exe/woods-mcp-start +14 -9
  33. data/exe/woods-watch +5 -0
  34. data/lib/generators/woods/pgvector_generator.rb +8 -2
  35. data/lib/generators/woods/watch_generator.rb +53 -0
  36. data/lib/puma/plugin/woods.rb +10 -0
  37. data/lib/tasks/woods.rake +14 -0
  38. data/lib/woods/agent_configuration/applier.rb +5 -3
  39. data/lib/woods/agent_configuration/cli.rb +2 -2
  40. data/lib/woods/agent_configuration/layout.rb +13 -0
  41. data/lib/woods/cache/cache_middleware.rb +6 -0
  42. data/lib/woods/console/credential_scanner.rb +4 -3
  43. data/lib/woods/console/dispatch_pipeline.rb +7 -0
  44. data/lib/woods/console/embedded_executor.rb +31 -9
  45. data/lib/woods/console/sql_noise_stripper.rb +9 -7
  46. data/lib/woods/console/sql_table_scanner.rb +47 -7
  47. data/lib/woods/console/sql_validator.rb +49 -9
  48. data/lib/woods/console/sqlite_read_guard.rb +46 -0
  49. data/lib/woods/console/stdio_transport.rb +27 -0
  50. data/lib/woods/coordination/pipeline_lock.rb +3 -2
  51. data/lib/woods/embedding/indexer.rb +24 -14
  52. data/lib/woods/extractor.rb +70 -19
  53. data/lib/woods/extractors/declared_parent.rb +55 -0
  54. data/lib/woods/extractors/graphql_extractor.rb +2 -11
  55. data/lib/woods/extractors/lib_extractor.rb +10 -8
  56. data/lib/woods/extractors/mailer_extractor.rb +6 -10
  57. data/lib/woods/extractors/model_extractor.rb +1 -15
  58. data/lib/woods/extractors/poro_extractor.rb +10 -8
  59. data/lib/woods/extractors/shared_utility_methods.rb +22 -5
  60. data/lib/woods/git_command.rb +6 -7
  61. data/lib/woods/git_provenance.rb +4 -6
  62. data/lib/woods/mcp/bearer_auth.rb +2 -1
  63. data/lib/woods/mcp/bootstrapper.rb +20 -5
  64. data/lib/woods/mcp/config_resolver.rb +2 -1
  65. data/lib/woods/mcp/index_reader.rb +11 -2
  66. data/lib/woods/mcp/initialization_guidance.rb +1 -1
  67. data/lib/woods/mcp/renderers/markdown_renderer.rb +14 -8
  68. data/lib/woods/mcp/renderers/plain_renderer.rb +11 -7
  69. data/lib/woods/mcp/server.rb +63 -37
  70. data/lib/woods/mcp/tool_contract.rb +1 -1
  71. data/lib/woods/mcp/tool_response_renderer.rb +16 -0
  72. data/lib/woods/mcp/traversal_evidence_text.rb +1 -1
  73. data/lib/woods/mcp/traversal_response.rb +22 -0
  74. data/lib/woods/path_dispatcher.rb +6 -5
  75. data/lib/woods/published_index/typed_unit_reader.rb +40 -3
  76. data/lib/woods/published_index.rb +2 -2
  77. data/lib/woods/rake_helpers.rb +2 -12
  78. data/lib/woods/retrieval/corpus_status.rb +46 -0
  79. data/lib/woods/retrieval/lexical_assembler.rb +14 -3
  80. data/lib/woods/retrieval/lexical_index.rb +2 -1
  81. data/lib/woods/retriever.rb +19 -7
  82. data/lib/woods/session_tracer/file_store.rb +6 -1
  83. data/lib/woods/source_inputs/consumer_errors.rb +4 -0
  84. data/lib/woods/storage/local_corpus_stats.rb +32 -0
  85. data/lib/woods/storage/metadata_store.rb +20 -0
  86. data/lib/woods/storage/pgvector.rb +6 -2
  87. data/lib/woods/storage/vector_store.rb +10 -0
  88. data/lib/woods/temporal/json_snapshot_store.rb +35 -7
  89. data/lib/woods/version.rb +1 -1
  90. data/lib/woods/watch/child_environment.rb +30 -0
  91. data/lib/woods/watch/cli.rb +91 -0
  92. data/lib/woods/watch/daemon.rb +73 -11
  93. data/lib/woods/watch/event_stream.rb +70 -0
  94. data/lib/woods/watch/guardian.rb +142 -0
  95. data/lib/woods/watch/installation/layout.rb +70 -0
  96. data/lib/woods/watch/installation/options.rb +128 -0
  97. data/lib/woods/watch/installation/planner.rb +128 -0
  98. data/lib/woods/watch/installation/probe.rb +101 -0
  99. data/lib/woods/watch/installation/receipt.rb +77 -0
  100. data/lib/woods/watch/installation/recovery.rb +64 -0
  101. data/lib/woods/watch/installation/templates.rb +58 -0
  102. data/lib/woods/watch/installation.rb +56 -0
  103. data/lib/woods/watch/lifecycle.rb +182 -0
  104. data/lib/woods/watch/managed_child.rb +113 -0
  105. data/lib/woods/watch/managed_cleanup.rb +48 -0
  106. data/lib/woods/watch/managed_process.rb +144 -0
  107. data/lib/woods/watch/puma_adapter.rb +87 -0
  108. data/lib/woods/watch/puma_child.rb +66 -0
  109. data/lib/woods/watch/supervision_records.rb +95 -0
  110. data/lib/woods/watch/supervision_status.rb +104 -0
  111. data/lib/woods/watch/supervisor.rb +161 -0
  112. data/lib/woods/watch/supervisor_reporting.rb +46 -0
  113. data/plugin/.claude-plugin/plugin.json +1 -1
  114. data/plugin/hooks/woods-input-rules.sh +4 -4
  115. data/plugin/skills/woods-agent-enable/SKILL.md +7 -1
  116. data/plugin/skills/woods-diagnose/SKILL.md +134 -34
  117. data/plugin/skills/woods-investigate/SKILL.md +54 -15
  118. data/plugin/skills/woods-mcp-config/SKILL.md +38 -11
  119. data/plugin/skills/woods-setup/SKILL.md +72 -15
  120. metadata +38 -5
data/CONTRIBUTING.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # Contributing to Woods
2
2
 
3
3
  <!-- release-state:contributing-intro -->
4
- Woods welcomes bug fixes, extractor coverage, storage and retrieval improvements, MCP compatibility work, documentation, and focused performance changes. This guide covers the shared contribution contract. Coding agents working from a source checkout should also read the repository's [AGENTS.md](https://github.com/lost-in-the/woods/blob/v2.0.0.beta3/AGENTS.md).
4
+ Woods welcomes bug fixes, extractor coverage, storage and retrieval improvements, MCP compatibility work, documentation, and focused performance changes. This guide covers the shared contribution contract. Coding agents working from a source checkout should also read the repository's [AGENTS.md](https://github.com/lost-in-the/woods/blob/v2.0.0/AGENTS.md).
5
5
  <!-- release-state:end -->
6
6
 
7
7
  ## Choose the right channel
@@ -46,7 +46,7 @@ Create a branch from current `main`. Keep each pull request to one logical chang
46
46
  | `plugin/skills/` | Distributed Woods skills (setup/upgrade, MCP configuration, investigation, agent enablement, diagnosis) |
47
47
 
48
48
  <!-- release-state:contributing-architecture -->
49
- Read [CLAUDE.md](https://github.com/lost-in-the/woods/blob/v2.0.0.beta3/CLAUDE.md) for architecture and implementation gotchas before changing runtime behavior.
49
+ Read [CLAUDE.md](https://github.com/lost-in-the/woods/blob/v2.0.0/CLAUDE.md) for architecture and implementation gotchas before changing runtime behavior.
50
50
  <!-- release-state:end -->
51
51
 
52
52
  ### Agent orientation and static self-map
@@ -263,7 +263,7 @@ By contributing, you agree that your contribution is licensed under the [MIT Lic
263
263
 
264
264
  ## Release flow
265
265
 
266
- `main` is the development branch and never claims a released version. Between releases it carries the alpha development marker. Every release, including a beta or a release candidate, is an explicit commit plus a tag, cut by one rake task and published only by the guarded workflow.
266
+ `main` is the development branch. It carries an alpha marker before the first prerelease of a version line and after reopening development following a final release. During beta/RC iteration, it retains the last prepared prerelease version until the next `release:prepare`. Every published release is identified by its exact tagged commit; later commits on `main` are not that release even if `Woods::VERSION` is unchanged.
267
267
 
268
268
  | State | `Woods::VERSION` | Tagged | On RubyGems | Documentation links point at |
269
269
  |---|---|---|---|---|
@@ -274,7 +274,9 @@ By contributing, you agree that your contribution is licensed under the [MIT Lic
274
274
 
275
275
  RubyGems treats any letter in a version as a prerelease, so a `~> 1.6` or `~> 2.0` constraint never resolves a beta or a release candidate. Adopting one is explicit: `gem "woods", "2.0.0.beta1"`.
276
276
 
277
- `spec/release_v2/version_state_spec.rb` enforces this table on every commit. VERSION is either an alpha or the changelog carries its dated heading, and the four `release-state` documentation fences match the state VERSION declares.
277
+ `spec/release_v2/version_state_spec.rb` verifies that VERSION is either an alpha or the changelog carries its dated heading, and that the four `release-state` documentation fences match the state VERSION declares. Those checks do not establish that a checkout is the published release.
278
+
279
+ For Git-sourced candidates, record the locked Git revision, loaded gem path, and working-tree changes alongside `Woods::VERSION`. Version-only preflight establishes the declared version, not whether a particular post-tag fix is present. Match capability claims to the pinned commit or published tag. Generated release-state links continue to describe the prepared version; use the candidate commit when linking evidence about unreleased changes.
278
280
 
279
281
  ### During feature work
280
282
 
@@ -312,10 +314,12 @@ One command per transition. It never commits, tags, pushes, or publishes.
312
314
  | Alpha to the first beta | `bin/rake "release:prepare[2.0.0.beta1]"` |
313
315
  | Beta to the next beta or a release candidate | `bin/rake "release:prepare[2.0.0.rc1]"` |
314
316
  | Release candidate to the release | `bin/rake "release:prepare[2.0.0]"` |
315
- | After the release publishes, reopen development | `bin/rake "release:reopen[2.1.0.alpha]"` |
317
+ | After a final release publishes, reopen development | `bin/rake "release:reopen[2.1.0.alpha]"` |
316
318
 
317
319
  `release:prepare` refuses a dirty working tree, a version that moves backwards, a version whose base is not the line `main` is developing, and an alpha target. It then bumps VERSION, folds `## [Unreleased]` and optional entry files into `## [<version>] - <date>` with one block per `###` heading, restates the fences, regenerates the surface inventory, and prints the tag and dispatch commands. Every rewrite is computed before any of it is written, so a refusal leaves the working tree untouched.
318
320
 
321
+ `release:reopen` accepts only a final release and a strictly later alpha. It does not reopen a beta/RC or move the same version line backwards to alpha. Continue prerelease development with Unreleased notes or changelog fragments, then use `release:prepare` for the next forward beta, RC, or final when authorized.
322
+
319
323
  A final release also absorbs every prerelease section of its own base version. Cutting `2.0.0` folds `## [2.0.0.beta1]` and `## [2.0.0.rc1]` into `## [2.0.0] - <date>` and removes their headings, prerelease entries first and anything written after them second, so the notes a user reads for 2.0.0 are the whole story rather than three fragments. An empty `## [Unreleased]` is therefore legitimate for a final release cut straight from a release candidate. A beta or a release candidate has nothing to absorb, so an empty Unreleased section without entry files refuses: there is nothing new to publish.
320
324
 
321
325
  Review the diff and run the release contracts:
@@ -391,12 +395,12 @@ short body that links `CHANGELOG.md` at the tag itself (not at `main`) and
391
395
  anchors straight to that version's dated heading, so the note a reader lands
392
396
  on always matches the bytes RubyGems published.
393
397
 
394
- ### One-off 1.6.2 security maintenance release
398
+ ### One-off 1.6.3 security maintenance release
395
399
 
396
400
  The [security policy](SECURITY.md#supported-versions) supports 1.6.x security
397
401
  fixes until 2027-02-20. While main develops v2, the sole maintenance exception
398
- is `v1.6.2` from the short-lived `release/1.6.2` branch, descending from the
399
- immutable v1.6.1 commit `73423a42644176b09961be373e13648c94690933`.
402
+ is `v1.6.3` from the short-lived `release/1.6.3` branch, descending from the
403
+ immutable v1.6.2 commit `4b40e17fd68122a70ccf00d9d2ffb8af42171d3d`.
400
404
  This is a stable patch, separate from the next v2 prerelease; it does not declare
401
405
  v2 final or establish a general-purpose maintenance publishing path.
402
406
 
@@ -413,15 +417,15 @@ The preparation order is:
413
417
  1. Merge the main-side maintenance policy/tooling PR. Before creating the remote
414
418
  target, confirm its effective branch rules require pull requests and prevent
415
419
  force pushes and deletion; configure those rules before creating the target. The GitHub
416
- rules API can check `release/1.6.2` before the branch exists.
417
- 2. Create that target from the immutable v1.6.1 commit. Review the narrow security
418
- backport and its legacy preparation adapter against that line. Disable the
419
- inherited automatic tag-push publisher before any maintenance tag exists.
420
- 3. Use the legacy adapter's `release:reopen[1.6.2.alpha]` and
421
- `release:prepare[1.6.2]` transitions in clean, separately reviewed commits.
420
+ rules API can check `release/1.6.3` before the branch exists.
421
+ 2. Create that target from the immutable v1.6.2 commit. Review the narrow security
422
+ backport and its legacy preparation adapter against that line. Confirm the
423
+ inherited automatic tag-push publisher remains disabled before any maintenance tag exists.
424
+ 3. Use the legacy adapter's `release:reopen[1.6.3.alpha]` and
425
+ `release:prepare[1.6.3]` transitions in clean, separately reviewed commits.
422
426
  The adapter owns the legacy documentation profile; do not copy v2 fences or
423
427
  surface claims into v1, and never hand-edit VERSION.
424
- 4. Review and merge the prepared candidate into `release/1.6.2`. Require passing
428
+ 4. Review and merge the prepared candidate into `release/1.6.3`. Require passing
425
429
  unit, booted Rails, installed-package, lint, coverage, security and build
426
430
  jobs. Review the complete CI and package-test implementation at that SHA.
427
431
  Then pin that **exact final commit** in `MAINTENANCE_APPROVED_SHA` through the
@@ -431,6 +435,14 @@ The preparation order is:
431
435
  Every exact maintenance matrix row in the trusted profile must succeed;
432
436
  missing, duplicated, skipped or failed rows refuse publication.
433
437
 
438
+ [Temporary security-advisory forks](https://docs.github.com/en/code-security/tutorials/fix-reported-vulnerabilities/collaborate-in-a-fork)
439
+ do not run CI or enforce destination branch protections when the advisory is
440
+ merged. Review and test those patches privately, then require the upstream CI
441
+ matrix triggered by the push to `release/1.6.3` before pinning its prepared SHA.
442
+ A private test report cannot replace the upstream tag-push run and immutable
443
+ artifact required for publication. Keep the advisory unpublished until the fixed
444
+ gems are available.
445
+
434
446
  The validators require the approved SHA to remain reachable from the freshly
435
447
  fetched maintenance branch and to descend from the fixed legacy base. They retain
436
448
  exact tag/VERSION/changelog checks, the unpublished-version check, one immutable
@@ -446,11 +458,11 @@ RubyGems credentials; the remote tag is checked again immediately before push.
446
458
  A candidate fix or changed prepared SHA requires a new reviewed main pin and a
447
459
  fresh tag-push CI run. Updating main's tooling alone never authorizes different
448
460
  candidate bytes. Main's v2 release contract remains unchanged. Do not create or
449
- push tags, dispatch, publish, or claim 1.6.2 is available during preparation.
461
+ push tags, dispatch, publish, or claim 1.6.3 is available during preparation.
450
462
 
451
463
  ### Stable branches
452
464
 
453
- A stable branch is `N-M-stable`, cut from the release tag. Create one only when a released line needs a patch after a newer major has shipped on `main`; until then, `main` is the development branch. The explicitly approved short-lived `release/1.6.2` security exception above does not establish an `N-M-stable` branch.
465
+ A stable branch is `N-M-stable`, cut from the release tag. Create one only when a released line needs a patch after a newer major has shipped on `main`; until then, `main` is the development branch. The explicitly approved short-lived `release/1.6.3` security exception above does not establish an `N-M-stable` branch.
454
466
 
455
467
  ### What coding agents may do
456
468
 
data/README.md CHANGED
@@ -4,38 +4,23 @@
4
4
 
5
5
  # Woods
6
6
 
7
- **Give AI coding agents a runtime-accurate map of your Rails application.**
7
+ **Give coding agents the Rails context that source files alone leave out.**
8
8
 
9
9
  [![Gem Version](https://img.shields.io/gem/v/woods)](https://rubygems.org/gems/woods)
10
10
  [![CI](https://github.com/lost-in-the/woods/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/lost-in-the/woods/actions/workflows/ci.yml)
11
11
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.txt)
12
12
 
13
- <!-- release-state:version-banner -->
14
- > **This tree documents version 2.0.0.** It is a major update from 1.x: read [what changed and how to upgrade](docs/UPGRADING_TO_2.md) before updating. The full history is in the [CHANGELOG](CHANGELOG.md).
15
- >
16
- > `main` is the development branch and can run ahead of the latest published gem. The gem badge above shows the latest published version; documentation for a published version lives on its tag.
17
- >
18
- > ### Version: 2.0.0.beta3 is published as a prerelease; `main` documents 2.0.0
19
- >
20
- > | Line | Version | Documentation |
21
- > |---|---|---|
22
- > | Documented here | **2.0.0**, unreleased | this README and the [documentation index](docs/README.md) |
23
- > | Latest prerelease | **2.0.0.beta3** | [the v2.0.0.beta3 tag](https://github.com/lost-in-the/woods/tree/v2.0.0.beta3) |
24
- > | Latest published gem | **1.6.1** | [the v1.6.1 tag](https://github.com/lost-in-the/woods/tree/v1.6.1) |
25
- >
26
- > RubyGems treats 2.0.0.beta3 as a prerelease, so `gem "woods", "~> 2.0"` does not resolve it. Install it explicitly with `gem "woods", "2.0.0.beta3"`. The released constraint stays `gem "woods", "~> 1.6"`.
27
- <!-- release-state:end -->
13
+ Woods boots your Rails application, extracts its resolved structure, and publishes an index that coding agents can query through the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). It brings together database schema, associations, callbacks, concerns, routes, and source code so an agent can inspect how Rails assembles your application.
28
14
 
29
- Woods boots your Rails app, extracts the behavior Rails assembles at runtime, and serves it to AI tools through the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). Agents can inspect resolved routes, schema, associations, callbacks, included concerns, dependencies, and execution flows instead of guessing from source files alone.
15
+ Supports **Ruby 3.0+ and Rails 6.0–8.x**, using a Ruby version supported by your Rails release. The application must boot and connect to its database. Structural queries need no embedding provider or vector database.
30
16
 
31
- Woods 2.0 supports Ruby 3.0 or later and Rails 6.0 through 8.x. It connects AI coding tools and agents through MCP.
17
+ [Get started](#five-minute-setup) · [Documentation](docs/README.md) · [Agent setup](docs/AGENT_SETUP.md) · [Upgrade from 1.x](docs/UPGRADING_TO_2.md)
32
18
 
33
19
  ## What Woods adds
34
20
 
35
- A Rails model rarely lives in one file. Its real behavior can include database schema, generated methods, framework defaults, and concerns loaded from elsewhere:
21
+ Consider a model whose behavior is spread across Rails, the database, and a concern:
36
22
 
37
23
  ```ruby
38
- # app/models/order.rb
39
24
  class Order < ApplicationRecord
40
25
  include Auditable
41
26
  belongs_to :customer
@@ -43,44 +28,40 @@ class Order < ApplicationRecord
43
28
  end
44
29
  ```
45
30
 
46
- Woods turns that runtime class into one connected unit with:
31
+ Woods can give an agent one unit containing its columns and indexes, association metadata, resolved callbacks, and included concern source. Recorded relationships connect that unit to other parts of the application.
47
32
 
48
- - column types, indexes, and foreign keys from the live database;
49
- - associations, validations, scopes, enums, and resolved callbacks;
50
- - source from included concerns, kept beside the owning class;
51
- - callback side effects such as jobs, mailers, and columns written;
52
- - forward dependencies and reverse dependents;
53
- - route, controller, view, job, and service relationships.
33
+ An agent can then ask:
54
34
 
55
- The result is a codebase index an agent can query by exact name, pattern, dependency path, graph structure, or natural language.
35
+ > Find the Order model, inspect its callbacks and associations, and show its recorded dependents. Cite the indexed evidence and check source code for callers the graph may miss.
56
36
 
57
- Still weighing it? [Why Woods](docs/WHY_WOODS.md) makes the case against grep, cloud indexers, and IDE language servers.
37
+ Models are one part of the index: Woods also extracts controllers, routes, jobs, mailers, views, components, GraphQL types, service objects, tests, and more. See the [extractor reference](docs/EXTRACTOR_REFERENCE.md) for coverage and the [agent guide](docs/AGENT_GUIDE.md) for query examples.
58
38
 
59
39
  ## Five-minute setup
60
40
 
61
- The default setup provides structural code intelligence. It does not require an embedding provider, vector database, or access to live application records.
41
+ For agent-led setup, use the [agent installation option](#let-an-agent-install-it) and its runbook. For a new manual installation, follow the steps below.
62
42
 
63
- ### 1. Install Woods
43
+ **Already using Woods?** If you are upgrading from 1.x, follow the [upgrade guide](docs/UPGRADING_TO_2.md). For an existing 2.x installation, go directly to [retrieval modes](#retrieval-with-or-without-embeddings), [MCP configuration](docs/MCP_SERVERS.md), or the [configuration reference](docs/CONFIGURATION_REFERENCE.md). Preserve your initializer, index path, provider settings, and other client entries. Changing only the MCP launch configuration or retrieval mode does not require rerunning the installer or rebuilding the structural index.
64
44
 
65
- First [choose a version that is published on RubyGems](docs/GETTING_STARTED.md#1-install-the-gem).
66
- The example below requires a stable 2.x release; beta and release-candidate
67
- installations need an exact published prerelease pin from that guide.
45
+ Run installation and extraction commands from your Rails application root in its normal development environment. **Using Docker?** Follow [Docker setup](docs/DOCKER_SETUP.md) first: run those commands inside the application container and use paths visible to the process that runs MCP.
68
46
 
69
- ```ruby
70
- # Gemfile
71
- group :development do
72
- gem "woods", "~> 2.0"
73
- end
74
- ```
47
+ ### 1. Install and configure
48
+
49
+ These steps are for **Woods 2.x**. Choose a published 2.x version from [RubyGems](https://rubygems.org/gems/woods/versions). If only prereleases are available, use an exact prerelease pin; `~> 2.0` will not select one. Follow the chosen version's tag documentation rather than assuming every feature on `main` is published. If you choose 1.x, use its tag documentation instead of this quickstart.
50
+
51
+ <!-- release-state:version-banner -->
52
+ <!-- release-state:end -->
53
+
54
+ Once a stable 2.x release is published, add `gem "woods", "~> 2.0"` to your Gemfile's `:development` group. For a prerelease, use its exact published version instead. Then run:
75
55
 
76
56
  ```bash
77
57
  bundle install
58
+ bundle exec ruby -rwoods/version -e 'puts Woods::VERSION'
78
59
  bin/rails generate woods:install
79
60
  ```
80
61
 
81
- **Do not run the generated migration for a new default installation.** The generator creates an annotated `config/initializers/woods.rb` plus a legacy application migration for `woods_units`, `woods_edges`, and `woods_embeddings`. Woods 2's shipped structural index and storage backends do not use those application tables. Remove the migration before continuing; keep and run it only when deliberately preserving an older/custom integration that uses them.
62
+ **For a new default installation, remove the generated `db/migrate/*_create_woods_tables.rb` migration without running it.** Those legacy application tables are unused by the shipped index and storage backends. Keep the generated `config/initializers/woods.rb`; its defaults are sufficient. Only retain the migration for a deliberate older/custom integration. See [Getting started](docs/GETTING_STARTED.md#2-generate-and-review-configuration).
82
63
 
83
- ### 2. Extract and verify the codebase
64
+ ### 2. Extract and validate
84
65
 
85
66
  ```bash
86
67
  bin/rails woods:extract
@@ -88,11 +69,11 @@ bin/rails woods:validate
88
69
  bin/rails woods:stats
89
70
  ```
90
71
 
91
- Extraction must run where Rails can boot. The default index lives at `tmp/woods/`.
72
+ Run these where your Rails application can boot. The default output is `tmp/woods/`; keep this generated directory out of source control.
92
73
 
93
- ### 3. Connect the Index Server
74
+ ### 3. Connect your MCP client
94
75
 
95
- Add this to your MCP client's project configuration. The configuration location varies by client:
76
+ Adapt this example to your MCP client's project configuration format, using your application path and preserving other server entries. See [client configuration locations](docs/MCP_SERVERS.md#client-configuration-locations) for guidance:
96
77
 
97
78
  ```json
98
79
  {
@@ -106,175 +87,94 @@ Add this to your MCP client's project configuration. The configuration location
106
87
  }
107
88
  ```
108
89
 
109
- Restart or reconnect your MCP client, then ask it to call `woods_status`. A ready response with non-zero unit counts confirms the path from Rails extraction to the MCP client.
110
-
111
- The Index Server reads the published index from disk. It does not boot Rails or query application records.
90
+ Reconnect the client and ask it to call `woods_status`. Confirm the index path and non-zero unit counts. Then use `search` to discover a known class and `lookup` with its identifier **and type** to inspect it.
112
91
 
113
- > **Using Docker?** Run Rails commands inside the application container. If Woods is installed only there, launch the Index Server through that container too; a host-side server requires a host Ruby bundle and host-visible index. Follow [Docker setup](docs/DOCKER_SETUP.md).
92
+ The Index Server reads the published index without booting Rails or querying application records. See [MCP servers](docs/MCP_SERVERS.md) for client-specific configuration and HTTP transport.
114
93
 
115
- The complete walkthrough, including expected output and first questions to ask, is in [Getting started](docs/GETTING_STARTED.md).
116
-
117
- ## Let an agent install it
94
+ If Woods is installed only inside Docker, launch MCP through that container too. Host-side launch needs a host bundle and a host-visible index; see the [Docker process and path rule](docs/MCP_SERVERS.md#docker-process-and-path-rule).
118
95
 
119
- Woods is built to be agent-operated, and the fastest path is handing installation to the coding agent that will use it. Claude Code users can install the distributed skills once — they trigger on install, upgrade, configuration, investigation, and diagnosis on their own:
96
+ ## Retrieval: with or without embeddings
120
97
 
121
- ```bash
122
- /plugin marketplace add lost-in-the/plugins
123
- /plugin install woods-plugin@lost-in-the-plugins
124
- ```
98
+ Exact lookup, pattern search, and graph queries work immediately after extraction. For ranked retrieval through `codebase_retrieve`, choose a mode:
125
99
 
126
- With any coding agent (no plugin needed), paste this into a session opened at your Rails app's root:
127
-
128
- ```text
129
- Install or upgrade the woods gem in this Rails application by following
130
- https://github.com/lost-in-the/woods/blob/main/docs/AGENT_SETUP.md.
131
- Structural setup only: add the gem to the development group, run the
132
- installer, extract and validate the index, and register the Index MCP
133
- server for this app. Do not run the generated legacy migration, and do
134
- not add embedding providers, vector databases, Console/live-data access,
135
- or secrets without asking me first. If woods 1.x is already installed,
136
- follow the upgrade runbook in docs/UPGRADING_TO_2.md instead and plan a
137
- clean re-index. Finish by reporting the installed version, files
138
- changed, commands run, and one verified woods_status call through the
139
- registered MCP server.
140
- ```
100
+ | Mode | Setup | What it searches |
101
+ |---|---|---|
102
+ | **Lexical** | Set `WOODS_RETRIEVAL_MODE=lexical` in the MCP process environment and restart the server | Published extraction units, ranked by field-aware keyword matching; no provider or embeddings |
103
+ | **Semantic** (default mode) | Configure a local or hosted embedding provider, then run `bin/rails woods:embed` | Embedded code context, ranked by semantic similarity |
141
104
 
142
- The runbook holds the agent to the same guardrails the skills enforce: a version preflight, minimal diffs, and explicit approval before anything beyond the structural index. Prefer doing it by hand? The five-minute setup above is the same procedure as commands.
105
+ For the stdio configuration above, add `"env": {"WOODS_RETRIEVAL_MODE": "lexical"}` inside the `woods` server entry to choose lexical mode. Confirm the active retriever with `woods_status`.
143
106
 
144
- ## Choose your path
107
+ To switch back to semantic retrieval, remove the lexical environment override or set `WOODS_RETRIEVAL_MODE=semantic`, configure the provider and embedding artifacts, then restart the MCP server and verify `woods_status`. Switching to lexical does not delete existing vectors or provider configuration.
145
108
 
146
- | Goal | Start here |
147
- |---|---|
148
- | Install Woods yourself | [Getting started](docs/GETTING_STARTED.md) |
149
- | Ask a coding agent to install Woods safely | [Agent setup runbook](docs/AGENT_SETUP.md) |
150
- | Configure an MCP client, Docker, or HTTP | [MCP servers](docs/MCP_SERVERS.md) |
151
- | Teach an agent how to query Woods effectively | [Agent guide](docs/AGENT_GUIDE.md) |
152
- | Add semantic search with OpenAI or local Ollama | [Retrieval guide](docs/RETRIEVAL_GUIDE.md) |
153
- | Query live Rails data through the optional Console Server | [Console MCP setup and security](docs/CONSOLE_MCP_SETUP.md) |
154
- | Keep the index current automatically while coding | [Watch daemon](docs/WATCH_DAEMON.md) |
155
- | Upgrade an existing 1.x installation | [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md) |
156
- | Diagnose a failure | [Troubleshooting](docs/TROUBLESHOOTING.md) |
109
+ The [lexical guide](docs/RETRIEVAL_GUIDE.md#embedding-free-lexical-retrieval) and [semantic setup](docs/RETRIEVAL_GUIDE.md#configuring-retrieval) cover configuration, ranking, and response budgets. Lexical matching depends on shared vocabulary; semantic mode requires the configured provider and embedding artifacts.
157
110
 
158
- ## Upgrading from 1.x
111
+ ## Keeping the index current
159
112
 
160
- Woods 2.0 is a major release: identifiers, the on-disk layout, the MCP surface, and task failure posture all changed. [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md) holds the full what-changed table, the step-by-step runbook with backups and rollback, and an agent-operated upgrade prompt.
113
+ Run one watcher through your normal development startup. It catches up on
114
+ changes, publishes complete generations, and lets the Index Server refresh on
115
+ later tool calls. The [startup guide](docs/WATCH_DAEMON.md#managed-development-startup)
116
+ covers Puma, existing Foreman workflows, and Docker/Grove supervision. Managed
117
+ startup requires a supporting gem; **check installed capabilities**. Older packages run the raw
118
+ `bin/rails woods:watch` task under an external restart-capable supervisor.
161
119
 
162
- ## Optional Claude Code workflows
120
+ Without a watcher, run `bin/rails woods:incremental` after edits or
121
+ `bin/rails woods:extract` for a full rebuild. MCP registration alone does not
122
+ enable automatic maintenance.
163
123
 
164
- Woods itself is MCP-client and model independent. The separately packaged Woods plugin (install commands under [Let an agent install it](#let-an-agent-install-it)) gives Claude Code five guided skills: setup and upgrade, MCP configuration, index-driven investigation, repository agent enablement, and diagnosis. Other MCP clients do not need it; follow the human or agent runbooks linked above and configure either stdio or Streamable HTTP directly.
124
+ Incremental cost depends on the affected code and relationships; broad changes can cost as much as a full extraction. Semantic embeddings have a separate update step. See [Watch daemon](docs/WATCH_DAEMON.md), [incremental extraction](docs/INCREMENTAL_EXTRACTION.md), and [source freshness](docs/SOURCE_FRESHNESS.md).
165
125
 
166
126
  ## Two servers, two trust boundaries
167
127
 
168
- Woods ships two MCP servers. Most users only need the Index Server.
169
-
170
128
  | | Index Server | Console Server |
171
129
  |---|---|---|
172
- | Purpose | Query pre-extracted code context | Query live Rails models and schema |
173
- | Data source | Files under `tmp/woods/` | A booted Rails process and its database |
174
- | Default tools | 14 | 9 |
175
- | Optional tools | Semantic retrieval activates after embedding; advanced Ruby embeddings can wire more collaborators | `console_sql` and `console_query` raise the total to 11 when explicitly enabled |
176
- | Default posture | Read-only index | Disabled; live-data access requires deliberate setup |
177
-
178
- The 14 Index tools cover health, exact lookup, search, dependency traversal, flow tracing, graph analysis, framework source, change recency, and optional semantic retrieval. The Console Server exposes nine supported model/schema tools by default. Nineteen Tier 2/3 Console schemas (9 Tier 2, 10 Tier 3) and `console_eval` exist as source inventory but do not register in any supported mode.
179
-
180
- See [MCP servers](docs/MCP_SERVERS.md) for the callable tool lists and client configuration.
181
-
182
- ## Optional semantic search
183
-
184
- Exact search, lookup, graph traversal, and flow tools work after extraction alone. Natural-language retrieval through `codebase_retrieve` also needs embeddings:
185
-
186
- ```ruby
187
- # config/initializers/woods.rb
188
- Woods.configure_with_preset(:local)
189
- ```
190
-
191
- The `:local` preset uses SQLite metadata, in-memory vectors persisted under the index, and a local Ollama service. It needs the `sqlite3` gem in the application bundle plus an installed, running Ollama service, but no cloud API key. Pull the default model before the first embed:
130
+ | Purpose | Inspect extracted code context | Query live Rails models and schema |
131
+ | Reads | Published index files | A booted application and its database |
132
+ | Packaged tools | 14; retrieval usable when configured | 9; 11 with embedded read tools enabled |
133
+ | Setup | The workflow above | Optional, disabled by default |
192
134
 
193
- ```bash
194
- ollama pull nomic-embed-text
195
- ```
135
+ Extraction itself boots and eager-loads your application, so its boot-time behavior still runs. Treat the generated index as confidential application source. Enabling hosted embeddings sends the embedded content to that provider. MCP responses also contain application source, which your client may send to its model provider even when Woods uses lexical retrieval or local embeddings.
196
136
 
197
- MySQL/PostgreSQL applications that do not bundle `sqlite3` can use `:shared_filesystem` for local persisted stores instead. PostgreSQL/OpenAI, Qdrant/OpenAI, and shared-filesystem configurations are documented in the [backend matrix](docs/BACKEND_MATRIX.md) and [configuration reference](docs/CONFIGURATION_REFERENCE.md).
137
+ The optional Console Server can access live data. Review its [setup and security model](docs/CONSOLE_MCP_SETUP.md) before enabling it. Report vulnerabilities privately through [SECURITY.md](SECURITY.md).
198
138
 
199
- For dense Ruby source, add `gem "tokenizers", "~> 0.5"` for exact WordPiece token counting. Without it, Woods uses a character estimate that can over-pack some Ollama chunks.
139
+ ## What the index can and cannot establish
200
140
 
201
- ```bash
202
- bin/rails woods:embed
203
- ```
141
+ - **It is a snapshot.** Check freshness against your working tree before relying on it for a change.
142
+ - **Relationships are recorded evidence, not a complete call graph.** Arbitrary method-body constant references are not exhaustively indexed. No recorded dependents does not prove that a class has no callers or is safe to delete.
143
+ - **A traced flow is not proof of execution.** Follow the tool's evidence and limits, and verify behavior in the application when it matters.
204
144
 
205
- Reconnect the Index Server after the first embed, then check `woods_status` before using `codebase_retrieve`.
145
+ Use Woods to locate and connect evidence, then confirm the relevant source and tests. The [agent guide](docs/AGENT_GUIDE.md) describes this workflow.
206
146
 
207
- ## Keeping the index current
208
-
209
- Run a full extraction after installation or broad configuration changes:
210
-
211
- ```bash
212
- bin/rails woods:extract
213
- ```
214
-
215
- For automatic maintenance during development, run the watcher as a dedicated process:
147
+ ## Let an agent install it
216
148
 
217
- ```bash
218
- bin/rails woods:watch
219
- ```
149
+ Use the [agent setup runbook](docs/AGENT_SETUP.md) for a copyable installation prompt and verification checklist. Woods works with MCP-capable clients independently of a specific model or editor.
220
150
 
221
- Add it to your development process manager so it starts beside Rails:
151
+ Claude Code users can optionally install the companion workflows:
222
152
 
223
153
  ```text
224
- # Procfile.dev
225
- web: bin/rails server
226
- woods: bundle exec rake woods:watch
227
- ```
228
-
229
- The watcher catches up changes made while it was stopped, batches new file changes, reloads Rails code when safe, and publishes complete generations atomically. The Index Server notices a new generation on its next tool call and refreshes itself. **After the initial extraction, ordinary code changes need no manual re-extraction or MCP restart.**
230
-
231
- Changes to boot-captured state, including dependencies, initializers, database configuration, credentials, or schema, make the watcher exit with status 75 so a process supervisor can restart it cleanly. If semantic retrieval is enabled, the watcher keeps structural context current; run `bin/rails woods:embed_incremental` to update vectors.
232
-
233
- In CI on Rails 8.1, add one step to `config/ci.rb` so the index the gates read matches the commit under test:
234
-
235
- ```ruby
236
- step "Woods: refresh", "bin/rails woods:incremental"
154
+ /plugin marketplace add lost-in-the/plugins
155
+ /plugin install woods-plugin@lost-in-the-plugins
237
156
  ```
238
157
 
239
- Claude Code users with the Woods plugin can opt into the same refresh from a `PostToolUse` hook, plus a `SessionStart` warning scoped to commit timestamps (it does not see uncommitted edits or an older checkout). Both ship disabled; set `WOODS_HOOKS_ENABLED=1` to turn them on. See [Watch daemon](docs/WATCH_DAEMON.md#hooks-for-agent-sessions).
240
-
241
- Without a resident watcher, run `bin/rails woods:incremental` after changes. See [Watch daemon](docs/WATCH_DAEMON.md) for Docker polling, failure behavior, and restart triggers.
242
-
243
- ## What gets indexed
244
-
245
- Woods recognizes the Rails application as a connected system, including:
246
-
247
- - models, concerns, controllers, routes, middleware, and engines;
248
- - services, interactors, commands, jobs, mailers, and scheduled work;
249
- - ERB views, Phlex components, ViewComponents, and navigation edges;
250
- - GraphQL types, mutations, resolvers, and fields;
251
- - policies, serializers, decorators, validators, state machines, and events;
252
- - migrations, database views, factories, tests, configuration, and installed framework source.
253
-
254
- Read the [extractor reference](docs/EXTRACTOR_REFERENCE.md) for the complete per-type contract and [internals](docs/INTERNALS.md) for how extraction, storage, retrieval, and MCP fit together.
255
-
256
- ## Security boundary
257
-
258
- Woods extraction reads application code, resolved Rails configuration, and database schema. Treat the generated index as source code: do not publish it unless the source itself may be published.
259
-
260
- The optional Console Server has a larger trust boundary because it can read live application data. It is disabled by default and adds table blocking, credential scanning, column redaction, SQL validation, and rolled-back transactions when enabled. Those controls reduce risk; they do not turn production data access into a harmless default. Review [Console MCP security](docs/CONSOLE_MCP_SETUP.md#safety-model) before enabling it.
261
-
262
- Report vulnerabilities privately through [SECURITY.md](SECURITY.md).
158
+ The plugin guides installation, MCP configuration, investigation, repository agent setup, and diagnosis. It is distributed separately from the gem.
263
159
 
264
160
  ## Documentation
265
161
 
266
- Use the [documentation index](docs/README.md) to find guides by task or audience. Frequently used references include:
162
+ | Task | Guide |
163
+ |---|---|
164
+ | Install and verify | [Getting started](docs/GETTING_STARTED.md) |
165
+ | Configure clients, Docker, or HTTP | [MCP servers](docs/MCP_SERVERS.md) |
166
+ | Query effectively | [Agent guide](docs/AGENT_GUIDE.md) and [tool cookbook](docs/MCP_TOOL_COOKBOOK.md) |
167
+ | Configure Woods | [Configuration reference](docs/CONFIGURATION_REFERENCE.md) |
168
+ | Choose retrieval and storage | [Retrieval guide](docs/RETRIEVAL_GUIDE.md) and [backend matrix](docs/BACKEND_MATRIX.md) |
169
+ | Upgrade from 1.x | [Upgrade guide](docs/UPGRADING_TO_2.md) |
170
+ | Diagnose a failure | [Troubleshooting](docs/TROUBLESHOOTING.md) |
267
171
 
268
- - [Configuration reference](docs/CONFIGURATION_REFERENCE.md)
269
- - [MCP tool cookbook](docs/MCP_TOOL_COOKBOOK.md)
270
- - [FAQ](docs/FAQ.md)
271
- - [Troubleshooting](docs/TROUBLESHOOTING.md)
272
- - [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md)
172
+ See the [documentation index](docs/README.md) for all guides and canonical reference pages.
273
173
 
274
174
  ## Contributing
275
175
 
276
- Read [CONTRIBUTING.md](CONTRIBUTING.md) before opening an issue or pull request. Coding agents working in the source repository should also read [AGENTS.md](https://github.com/lost-in-the/woods/blob/main/AGENTS.md).
176
+ Use [GitHub issues](https://github.com/lost-in-the/woods/issues) for bugs and feature requests. Read [CONTRIBUTING.md](CONTRIBUTING.md) before submitting a pull request; coding agents should also read [AGENTS.md](https://github.com/lost-in-the/woods/blob/main/AGENTS.md).
277
177
 
278
178
  ## License
279
179
 
280
- Woods is available under the [MIT License](LICENSE.txt).
180
+ [MIT](LICENSE.txt).