esp-modkit 0.1.0

data/docs/walkthrough.md

@@ -0,0 +1,193 @@
+# Walkthrough: a mod from empty to built
+
+This is the end-to-end story the reference docs only show in pieces:
+authoring one real plugin from nothing, the way the toolchain wants you
+to work. If you've used the Construction Set, the mental shift is the
+whole point — so it's worth stating up front.
+
+## The shift: source is truth, the `.esp` is output
+
+The Construction Set is a direct-manipulation binary editor. The `.esp`
+*is* the document; you click forms and a 3D viewport, and the binary is
+what you save. `esp` inverts that. **You author intent as
+human-readable source, and the `.esp` is a disposable build artifact** —
+the same relationship a compiler has with its object files. You never
+hand-edit the `.esp`; you edit source and rebuild.
+
+That one inversion is what buys you `git diff`, branches, code review,
+procedural generation, lint-as-a-build-step, and AI authoring. The cost
+is real — you give up spatial placement and visual browsing — but
+everything below is what you get back.
+
+## 0. One-time setup
+
+```sh
+bundle install
+bin/esp setup                  # git diff driver + pre-commit hook
+bin/esp refs unpack            # vanilla Morrowind/Tribunal/Bloodmoon -> JSON
+bin/esp refs index             # SQLite index over ~69k records (~3s)
+```
+
+`refs unpack` + `refs index` are what let `lint` and `refs find` resolve
+foreign IDs (does `Imperial Legion` exist? what's the editor ID of that
+sign in Seyda Neen?). You only redo them when you upgrade the game.
+See [getting-started](getting-started.md) for installing tes3conv itself.
+
+## 1. Scaffold
+
+We'll build a signpost for Seyda Neen — an activator with a script
+attached. Scaffold the folder:
+
+```sh
+$ bin/esp new SeydaSign --author "Corey Ellis" \
+    --description "A welcome sign in Seyda Neen"
+created: mods/SeydaSign/SeydaSign.json
+created: mods/SeydaSign/README.md
+next: bin/esp build SeydaSign
+```
+
+`esp new` defaults to JSON; pass `--format rb|py|js|ts` to author in a
+real language instead. Everything for this mod now lives under
+`mods/SeydaSign/` — source, scripts, translations, design notes all stay
+together (see [authoring-guide](authoring-guide.md#per-mod-self-containment)).
+
+## 2. Author the records
+
+Open `mods/SeydaSign/SeydaSign.json`. The scaffold gives you just a
+`Header`; add an `Activator` that references a script:
+
+```json
+[
+  { "type": "Header", "flags": "", "file_type": "Esp", "version": 1.3,
+    "author": "Corey Ellis", "description": "A welcome sign in Seyda Neen",
+    "num_objects": 0, "masters": [["Morrowind.esm", 79837557]] },
+  { "type": "Activator", "flags": "", "id": "seyda_welcome_sign",
+    "name": "Welcome to Seyda Neen", "script": "seyda_sign_script",
+    "mesh": "f/active_sign.nif" }
+]
+```
+
+> **The `flags` gotcha.** Every TES3 record needs `"flags": ""` even when
+> it's empty — tes3conv rejects the build otherwise. If you forget, you
+> get a clean, pointed error rather than a stack trace:
+>
+> ```
+> $ bin/esp build SeydaSign
+> error: Error: Custom { kind: InvalidData, error: Error("missing field `flags`", line: 6, column: 1) }
+> hint: every record needs "flags": "" — tes3conv requires it even when empty
+> ```
+
+## 3. Lint catches the dangling reference
+
+We pointed the activator at `seyda_sign_script`, but that script doesn't
+exist yet — not in this mod, not in vanilla. `lint` knows, because it
+resolves every reference against the mod plus the indexed ESMs:
+
+```sh
+$ bin/esp lint SeydaSign
+ERROR  Activator   seyda_welcome_sign.script  ->  seyda_sign_script: unknown Script
+1 error(s), 0 warning(s)
+```
+
+It exits non-zero, so this is the line you wire into CI. (A reference
+that resolves in a vanilla ESM you *forgot to list as a master* is a
+`WARN`, not an error — OpenMW loads it but logs complaints.)
+
+## 4. Add the script, go green
+
+Inline MWScript text is unreadable inside JSON, so author it as a file
+and reference it with `text_source` — the build inlines it and
+regenerates the binary script blobs tes3conv expects:
+
+```
+mods/SeydaSign/scripts/seyda_sign_script.mwscript
+```
+
+```mwscript
+Begin seyda_sign_script
+    ; activate-only sign; nothing to do
+End
+```
+
+Add the `Script` record pointing at that file:
+
+```json
+{ "type": "Script", "flags": "", "id": "seyda_sign_script",
+  "text_source": "scripts/seyda_sign_script.mwscript" }
+```
+
+```sh
+$ bin/esp lint SeydaSign
+ok
+```
+
+Need the editor ID of an existing record to reference? That's what the
+index is for:
+
+```sh
+$ bin/esp refs find "seyda neen"
+Morrowind.esm  Activator  active_sign_seydaneen_01
+Morrowind.esm  Activator  active_sign_seydaneen_02
+...
+```
+
+## 5. Build and install
+
+```sh
+$ bin/esp build SeydaSign
+build: SeydaSign -> dist/SeydaSign.esp
+
+$ bin/esp install SeydaSign     # appends data= + content= lines to openmw.cfg
+```
+
+`bin/esp build SeydaSign --install` does both in one step (and
+`--config <path>` targets a specific `openmw.cfg`). Launch OpenMW, confirm
+`SeydaSign.esp` is enabled in the launcher. The
+`.esp` in `dist/` is throwaway — delete it and `esp build` reproduces it
+byte-for-byte from source.
+
+## 6. The payoff: review the diff, not the binary
+
+Because the source is text, the change you just made is a readable diff.
+In a real mod project you'd track `mods/` in git (it's gitignored *in
+the toolchain repo* only because that repo ships the tool, not content):
+
+```sh
+$ git add mods/SeydaSign && git diff --cached --stat
+ mods/SeydaSign/SeydaSign.json                    | 4 ++++
+ mods/SeydaSign/scripts/seyda_sign_script.mwscript | 3 +++
+```
+
+Adding the script is `+4 / +3` lines you can read, blame, branch, and
+review — not an opaque re-saved `.esp`. That is the entire reason for
+the inversion.
+
+## Same pipeline, three frontends
+
+Every step above is also an HTTP endpoint (`esp serve`) and an MCP tool
+(`esp mcp serve`), returning the identical `--json` payloads. So an AI
+agent runs exactly this loop without a human at the keyboard:
+
+```sh
+$ bin/esp lint SeydaSign --json
+{"mod":"SeydaSign","errors":0,"warnings":0,"issues":[]}
+```
+
+1. `commands` / `esp docs introspect` — discover the surface.
+2. `refs_find` — look up real vanilla IDs to reference.
+3. Write the source file.
+4. `lint --json` — validate; structured errors come back as
+   `{"error": "…"}`.
+5. `build --json` — produce the `.esp`.
+
+That agent loop driving diffable source is the backbone of **ESPresso**,
+the AI-first GUI built on this toolchain. The human's job moves from
+clicking forms to reviewing the diff in step 6.
+
+## Where to go next
+
+- [Authoring guide](authoring-guide.md) — every source format, scripts,
+  dialogue DSL, i18n, linting in full.
+- [Getting started](getting-started.md) — install, paths, wiring `esp`
+  into an AI client over MCP.
+- [Architecture](architecture.md) — why the pipeline is shaped this way.

data/exe/esp

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# The gem's executable. Unlike bin/esp (a dev-repo bash launcher that pins
+# rbenv + BUNDLE_GEMFILE to this checkout), this is plain Ruby: RubyGems has
+# already put the gem's lib + its dependencies on $LOAD_PATH, so a bare
+# `require 'esp'` loads the whole toolchain. No Bundler involved.
+require 'esp'
+
+Esp::CLI.start(ARGV)

data/lib/esp/active_project.rb

@@ -0,0 +1,71 @@
+module Esp
+  # Process-scoped state for "which project is the user currently working in"
+  # (step 23). Swapped in by ESPresso's POST /open-project; helpers that need
+  # a working-tree root (vcs_root for diff/approve/reject today; more later)
+  # consult this before falling back to Esp::ROOT.
+  #
+  # Step 22.5 slice 2: also tracks the project's game id (`mw` today, future
+  # `ob`/`sr`) so the plugin registry knows which Operations module owns each
+  # op for the current request. Missing game on a project read defaults to
+  # `mw` for backward compat (every pre-rename project is Morrowind).
+  #
+  # Stored module-globally rather than per-request because the WEBrick backend
+  # is single-tenant per-instance (one ESPresso ↔ one esp serve), so a global
+  # is the right shape; the mutex guards against the thread-per-request model
+  # racing a concurrent set + read.
+  module ActiveProject
+    @mutex = Mutex.new
+    @root = nil
+    @game = nil
+
+    class << self
+      def root
+        @mutex.synchronize { @root }
+      end
+
+      def game
+        @mutex.synchronize { @game }
+      end
+
+      # Store a project root + game id. Returns the normalized path so callers
+      # don't have to expand again. No directory check here —
+      # Operations.open_project validates before calling (separation: this
+      # module is pure state).
+      def set(path, game: nil)
+        normalized = File.expand_path(path.to_s)
+        @mutex.synchronize do
+          @root = normalized
+          @game = game
+        end
+        normalized
+      end
+
+      def clear!
+        @mutex.synchronize do
+          @root = nil
+          @game = nil
+        end
+      end
+
+      # Resolve a root for a request: explicit `root:` param wins (preserves
+      # the diff-panel pattern of "operate on a specific tree even if a
+      # project is open"), then the active project, then nil — the caller
+      # decides whether to fall back to Esp::ROOT or error.
+      def root_for(params)
+        explicit = params.is_a?(Hash) ? params['root'] : nil
+        return explicit unless explicit.nil? || explicit.to_s.empty?
+
+        root
+      end
+
+      # Same precedence as root_for, but with Esp::ROOT as a final fallback.
+      # This is what every op that *needs* a root should call — diff/approve/
+      # reject already do (via Esp::Operations#vcs_root); slice 1 of step
+      # 23.5 routes the game ops (build, lint, scaffold, …) through it too
+      # so ESPresso's "open project" actually moves where they operate.
+      def resolve(params = {})
+        root_for(params) || Esp::ROOT
+      end
+    end
+  end
+end

data/lib/esp/agent.rb

@@ -0,0 +1,104 @@
+require 'json'
+
+module Esp
+  # A hand-rolled tool-use loop. The agent authors mods by calling the same
+  # curated tool surface MCP exposes (Esp::McpServer::TOOLS), each tool
+  # dispatched to Esp::Operations.public_send(op, …) — so the in-house agent
+  # and external MCP clients share one tool definition.
+  #
+  # The LLM is reached through an injected **provider** (see Esp::Providers), so
+  # the loop is provider-agnostic and fully testable with a stub and no live
+  # key. When none is given, the default provider is built from the registry
+  # (Esp::Providers.default_id). See .claude/roadmap/19-agent-workspace.md.
+  #
+  # Transcript shape (neutral, owned by Agent; providers translate it to their
+  # native wire format):
+  #   { role: :user,      text: String }
+  #   { role: :assistant, text: String, tool_calls: [{id:, name:, input:}],
+  #                       raw: <provider-native message, opaque to Agent> }
+  #   { role: :tool,      results: [{id:, content: String, is_error: bool}] }
+  class Agent
+    MAX_TURNS = 20
+
+    SYSTEM = <<~PROMPT.freeze
+      You are the authoring agent inside ESPresso, the GUI for the `esp`
+      Morrowind modding toolchain. You build and edit mods by calling the
+      provided tools, which operate on the user's mod project as diffable
+      source. Read current state (records_read, refs_find) before writing.
+      Make the smallest change that satisfies the request, then build and
+      lint to confirm. Keep explanations brief — the human reviews the diff.
+    PROMPT
+
+    Result = Struct.new(:text, :messages, keyword_init: true)
+
+    def initialize(provider: nil, system: SYSTEM, max_turns: MAX_TURNS)
+      @provider = provider || Esp::Providers.build(Esp::Providers.default_id)
+      @system = system
+      @max_turns = max_turns
+    end
+
+    # Run the loop to completion for one user prompt. Optionally yields events
+    # as they happen ([:text, str] / [:tool_use, name, input] /
+    # [:tool_result, name, payload] / [:tool_error, name, error]) for a
+    # streaming UI to render.
+    def run(prompt, &on_event)
+      messages = [{ role: :user, text: prompt }]
+      @max_turns.times do
+        completion = @provider.complete(system: @system, tools: tool_definitions, messages: messages)
+        tool_calls = Array(completion.tool_calls)
+        messages << { role: :assistant, text: completion.text, tool_calls: tool_calls, raw: completion.raw }
+        on_event&.call([:text, completion.text]) unless completion.text.to_s.empty?
+
+        break if tool_calls.empty?
+
+        messages << { role: :tool, results: tool_calls.map { |tc| run_tool(tc, &on_event) } }
+      end
+      Result.new(text: final_text(messages), messages: messages)
+    end
+
+    private
+
+    def tool_definitions
+      Esp::McpServer::TOOLS.map do |tool|
+        { name: tool[:name], description: tool[:description], input_schema: tool[:input_schema] }
+      end
+    end
+
+    # Dispatch one normalized tool call to its Operations method, returning a
+    # neutral tool result. Caller-fault errors (and an unknown tool name) come
+    # back as is_error results the model can recover from.
+    def run_tool(tool_call, &on_event)
+      name = tool_call[:name].to_s
+      input = deep_stringify(tool_call[:input] || {})
+      on_event&.call([:tool_use, name, input])
+
+      op = Esp::McpServer::TOOLS_BY_NAME.fetch(name)[:op]
+      payload = Esp::Operations.dispatch(op, input)
+      on_event&.call([:tool_result, name, payload])
+      { id: tool_call[:id], content: JSON.generate(payload), is_error: false }
+    rescue KeyError
+      tool_error(tool_call[:id], name, 'unknown_tool', name, &on_event)
+    rescue *Esp::Operations.caller_errors => e
+      tool_error(tool_call[:id], name, Esp::Operations.error_code(e), e.message, &on_event)
+    end
+
+    def tool_error(id, name, code, message, &on_event)
+      error = { error: { code: code, message: message } }
+      on_event&.call([:tool_error, name, error])
+      { id: id, content: JSON.generate(error), is_error: true }
+    end
+
+    def final_text(messages)
+      last = messages.reverse.find { |m| m[:role] == :assistant }
+      last ? last[:text].to_s : ''
+    end
+
+    def deep_stringify(obj)
+      case obj
+      when Hash  then obj.each_with_object({}) { |(k, v), h| h[k.to_s] = deep_stringify(v) }
+      when Array then obj.map { |v| deep_stringify(v) }
+      else obj
+      end
+    end
+  end
+end

data/lib/esp/cli/docs.rb

@@ -0,0 +1,44 @@
+require 'json'
+require 'thor'
+
+module Esp
+  class CLI < Thor
+    class Docs < Thor
+      include Support
+
+      def self.exit_on_failure? = true
+
+      class_option :json, type: :boolean, desc: 'Output structured JSON instead of human text'
+
+      desc 'build', 'Regenerate docs/reference/ from the CLI tree and module headers'
+      option :out, type: :string, default: 'docs/reference',
+                   desc: 'Output directory (relative to repo root)'
+      def build
+        out_dir = File.expand_path(options[:out], Esp::ROOT)
+        result = Esp::DocsGenerator.build(output_dir: out_dir)
+        relative = ->(p) { p.sub("#{Esp::ROOT}/", '') }
+        payload = {
+          output_dir: relative.call(out_dir),
+          commands_md: relative.call(result[:commands]),
+          api_index_md: relative.call(result[:api_index]),
+          api_module_md: result[:api_modules].map(&relative)
+        }
+        respond(payload) do
+          say t('docs.wrote', path: relative.call(result[:commands]))
+          say t('docs.wrote', path: relative.call(result[:api_index]))
+          result[:api_modules].each { |p| say t('docs.wrote', path: relative.call(p)) }
+        end
+      end
+
+      desc 'introspect', 'Dump the CLI command tree + module surface as JSON (always JSON)'
+      def introspect
+        payload = {
+          version: Esp::VERSION,
+          commands: Esp::Introspection.command_tree,
+          modules: Esp::Introspection.module_docs
+        }
+        $stdout.puts(JSON.generate(payload))
+      end
+    end
+  end
+end

data/lib/esp/cli/i18n.rb

@@ -0,0 +1,67 @@
+require 'json'
+require 'thor'
+
+module Esp
+  class CLI < Thor
+    class I18nCli < Thor
+      include Support
+
+      def self.exit_on_failure? = true
+
+      class_option :json, type: :boolean, desc: 'Output structured JSON instead of human text'
+      class_option :root, type: :string,
+                          desc: 'Project root (defaults to the active project or the toolchain repo)'
+
+      desc 'check MOD', 'Report missing/orphan i18n keys per locale (vs. en)'
+      def check(mod)
+        result = Esp::Operations.dispatch(:i18n_check, params_with_root('mod' => mod))
+        respond(result) { render(result[:locales]) }
+      rescue Esp::Operations::InputError, Esp::Mw::Loader::LoadError => e
+        fail_with(e.message)
+      end
+
+      desc 'audit', 'Audit tool-UI locales: undefined/unused/orphan keys + missing translations'
+      def audit
+        report = Esp::UI.audit
+        respond(report) { render_audit(report) }
+        problems = report[:undefined].size + report[:unused].size + report[:orphans].size
+        exit(1) if problems.positive?
+      end
+
+      no_commands do
+        def render_audit(report)
+          render_key_list(t('i18n.audit.undefined'), report[:undefined])
+          render_key_list(t('i18n.audit.unused'), report[:unused])
+          report[:orphans].each { |loc, keys| render_key_list(t('i18n.audit.orphans', locale: loc), keys) }
+          report[:missing].each { |loc, keys| say t('i18n.audit.missing', locale: loc, count: keys.size) }
+          say(t('i18n.audit.clean')) if clean?(report)
+        end
+
+        def clean?(report)
+          report.values_at(:undefined, :unused, :orphans, :missing).all?(&:empty?)
+        end
+
+        def render_key_list(heading, keys)
+          return if keys.empty?
+
+          say heading
+          keys.each { |k| say t('i18n.audit.item', key: k) }
+        end
+
+        def render(report)
+          if report.empty?
+            say(t('i18n.none'))
+            return
+          end
+
+          report.each do |locale, info|
+            say t('i18n.locale', locale: locale)
+            info[:missing].each { |k| say t('i18n.missing', key: k) }
+            info[:orphan].each  { |k| say t('i18n.orphan', key: k) }
+            say(t('i18n.ok')) if info[:missing].empty? && info[:orphan].empty?
+          end
+        end
+      end
+    end
+  end
+end

data/lib/esp/cli/mcp.rb

@@ -0,0 +1,52 @@
+require 'json'
+require 'thor'
+
+module Esp
+  class CLI < Thor
+    # `esp mcp` — Model Context Protocol frontend. `serve` speaks JSON-RPC
+    # over stdio so an AI client can drive the toolchain as native tools;
+    # `install` wires that server into a client's config.
+    class Mcp < Thor
+      include Support
+
+      def self.exit_on_failure? = true
+
+      class_option :json, type: :boolean, desc: 'Output structured JSON instead of human text'
+
+      desc 'serve', 'Run the MCP server over stdio (JSON-RPC 2.0) for AI clients'
+      def serve
+        # stdout is the protocol channel — never print to it here.
+        warn(t('mcp.serve.listening'))
+        Esp::McpServer.new.start
+      end
+
+      desc 'install', 'Register `esp mcp serve` with an AI client (Claude Code / Desktop)'
+      option :client, type: :string, default: 'claude-code',
+                      enum: Esp::McpInstaller.clients,
+                      desc: 'Which MCP client to configure'
+      option :name, type: :string, default: Esp::McpInstaller::SERVER_NAME,
+                    desc: 'Name to register the server under in mcpServers'
+      def install
+        result = Esp::McpInstaller.install(options[:client], name: options[:name])
+        respond(result_payload(result)) do
+          say t('mcp.install.result', action: result.action, name: result.name, label: result.label)
+          say t('mcp.install.config', path: result.config_path)
+          say t('mcp.install.restart') if needs_restart?(result)
+        end
+      rescue ArgumentError => e
+        fail_with(e.message)
+      end
+
+      no_commands do
+        def result_payload(result)
+          { client: result.client, name: result.name, action: result.action.to_s,
+            config: result.config_path, entry: result.entry }
+        end
+
+        def needs_restart?(result)
+          result.client == 'claude-desktop' && result.action != :unchanged
+        end
+      end
+    end
+  end
+end

data/lib/esp/cli/plugins.rb

@@ -0,0 +1,42 @@
+require 'json'
+require 'thor'
+
+module Esp
+  class CLI < Thor
+    # `esp plugins` — inspect the plugins OpenMW already has installed,
+    # read from openmw.cfg's data=/content= entries.
+    class Plugins < Thor
+      include Support
+
+      def self.exit_on_failure? = true
+
+      class_option :json, type: :boolean, desc: 'Output structured JSON instead of human text'
+
+      desc 'list', 'List plugins installed in OpenMW (active flag + load order)'
+      option :config, type: :string, desc: 'Path to openmw.cfg (default: per-OS user config)'
+      def list
+        result = Esp::Operations.dispatch(:plugins_list, 'config' => options[:config])
+        respond(result) { render(result) }
+      end
+
+      no_commands do
+        def render(result)
+          say(header(result))
+          plugins = result[:plugins]
+          return say(t('plugins.list.none')) if plugins.empty?
+
+          active, inactive = plugins.partition { |p| p[:active] }
+          active.sort_by { |p| p[:load_order] }
+                .each { |p| say t('plugins.list.active', order: p[:load_order], name: p[:name]) }
+          inactive.sort_by { |p| p[:name] }
+                  .each { |p| say t('plugins.list.inactive', name: p[:name]) }
+        end
+
+        def header(result)
+          key = result[:exists] ? 'plugins.list.config' : 'plugins.list.missing'
+          t(key, path: result[:openmw_cfg])
+        end
+      end
+    end
+  end
+end