pgn2 1.4.0 → 2.0.0

data/README.md

@@ -192,8 +192,8 @@ Move pipeline — immortal game, 45 plies (`bench/profile_moves.rb`):
 
 | Metric | original `pgn` | pgn2 | Δ |
 |---|---:|---:|---:|
-| Replay allocations (objects) | 5124 | 976 | -4148 (-80.9%) |
-| Replay allocations (bytes) | 262608 | 62064 | -200544 (-76.4%) |
+| Replay allocations (objects) | 5124 | 931 | -4193 (-81.8%) |
+| Replay allocations (bytes) | 262608 | 66728 | -195880 (-74.6%) |
 | `Board#dup` x45 (objects) | 451 | 91 | -360 (-79.8%) |
 | `Board#dup` x45 (bytes) | 43096 | 3856 | -39240 (-91.1%) |
 | `Board#at(str)` x1000 (objects) | 6000 | 0 | -6000 (-100%) |
@@ -206,8 +206,8 @@ Parser — 500 immortal games (`bench/profile_parse.rb`):
 |---|---:|---:|---:|
 | Parse-only allocations (objects) | 1248065 | 288537 | -959528 (-76.9%) |
 | Parse-only allocations (bytes) | 120370470 | 15640374 | -104730096 (-87.0%) |
-| Parse + replay allocations (objects) | 3778073 | 773030 | -3005043 (-79.5%) |
-| Parse + replay allocations (bytes) | 249570048 | 45626128 | -203943920 (-81.7%) |
+| Parse + replay allocations (objects) | 3778073 | 751030 | -3027043 (-80.1%) |
+| Parse + replay allocations (bytes) | 249570048 | 44812592 | -204757456 (-82.0%) |
 | Parse-only throughput | 1461 ms/i | 203 ms/i | ~7.2x faster |
 | Parse + replay throughput | 1938 ms/i | 484 ms/i | ~4.0x faster |
 
@@ -272,9 +272,42 @@ What changed to get there:
     floor). Parse-only throughput +25% (305 → 203 ms/i), parse allocations
     −17% (347037 → 288537 objects / 17977414 → 15640374 bytes for 500 games).
     Output byte-identical.
+16. `PGN::Board#fen_board_string` — serializes the FEN board string by
+    walking the 0x88 `@cells` array directly (ranks 8→1, files a→h, empty-run
+    collapsing) instead of rebuilding the 8x8 `squares` array and
+    transposing on every `position.to_fen` / `game.fen_list`. `FEN#board_string`
+    delegates to it. FEN output byte-identical. Measured on the immortal game
+    (46 positions): FEN generation allocations −40% (3201 → 1913 objects /
+    231104 → 100464 bytes), ~1.44× throughput (1140 → 792 µs/i). Adds a new
+    `bench/baseline_moves.txt` section 5/6 for FEN allocation/throughput.
+17. `PGN::Game#each_position` / `PGN::Zobrist` — additive features, not hot-
+    path wins. `each_position` is a lazy enumerator sharing the replay loop
+    with `#positions` (one `Enumerator` per `#positions` call, hence the
+    parse+replay +500 objects / +80000 bytes in the table above vs. the prior
+    baseline). `PGN::Zobrist` provides a deterministic 64-bit hash table and
+    `Position#zobrist`/`#hash`/`#eql?`/`#==`; the hash is computed lazily and
+    cached, so the replay hot path (which never asks for it) pays nothing —
+    an incremental per-move update was prototyped and rejected because 64-bit
+    Integer XOR allocates a `Bignum` per operation (~9/move), regressing
+    replay +40% allocations / −32% throughput for a feature nothing
+    currently consumes. Replay stayed at 976 objects / 62064 bytes at that
+    pass (since refreshed by #18).
+18. `PGN::Board::KNIGHT_ATTACKS` / `KING_ATTACKS` — precomputed 128-entry
+    on-board target tables (frozen), built once at load from the existing
+    knight/king offsets. `PGN::Notation` (`#reaches?`, `#knight_attacked?`,
+    `#king_attacked?`, `#leaper_moves?`) and `PGN::MoveCalculator`
+    (`#leaper_origins`) iterate the masks instead of the per-call
+    `offsets.any? { from + off == to }` + `(t & 0x88).zero?` off-board test.
+    Same-harness A/B (pre-mask lib vs masks): SAN generation throughput
+    +8% (334 → 361 ips / 2.99 → 2.77 ms/i), allocations unchanged (1387
+    objects); replay neutral (930 → 931 objects, throughput within noise).
+    New `bench/baseline_moves.txt` sections 8 (SAN gen) and 9 (retained
+    memory: lazy `each_position` 95 objects vs eager `positions` 287 objects
+    when the `Game` is kept alive — ~3x less retained for streaming).
+    Output byte-identical.
 
 Public output (FEN, PGN) is byte-identical to the original gem; the full
-suite (201 examples) stays green. See `bench/IMPROVEMENTS.md` for the per-step
+suite (226 examples) stays green. See `bench/IMPROVEMENTS.md` for the per-step
 before/after deltas that produced these tables.
 
 ## Installation
@@ -291,6 +324,75 @@ Or install it yourself as:
 
     $ gem install pgn2
 
+The native perft backend (see [Native perft engine](#native-perft-engine))
+ships with the gem as prebuilt platform gems, so installing `pgn2` pulls a
+binary for your platform — no Rust toolchain required. When building from
+source (or a checkout), compile it with `bundle exec rake compile`.
+
+## Native perft engine
+
+`pgn2` ships a Rust bitboard engine (`PGN::Bitboard::Engine`)
+exposed through a thin Ruby API. It targets fast perft numbers via the
+[`chessie`][chessie] crate (MPL-2.0; magic-bitboard move generation with
+checkmask/pinmask legality) wrapped by a small adapter, and is fully
+separate from the pure-Ruby 0x88
+`PGN::Board`/`PGN::Notation`/`PGN::MoveCalculator` — those stay
+byte-identical and untouched. See `NOTICE.md` for the `chessie` license
+attribution; the gem's own code remains MIT.
+
+[chessie]: https://crates.io/crates/chessie
+
+```ruby
+require "pgn"
+
+engine = PGN::Bitboard::Engine.new("rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1")
+engine.perft(5)        # => 4865609
+engine.legal_moves     # => ["a2a3", "a2a4", ..., "h2h3", "h2h4"] (sorted UCI)
+engine.legal?("e2e4")  # => true
+engine.legal?("e2e5")  # => false
+```
+
+- `#perft(depth)` — full-width node count; validated against the standard
+  perft suite (startpos, Kiwipete, positions 3–6).
+- `#legal_moves` — legal moves as sorted UCI strings (e.g. `"e2e4"`,
+  `"e1g1"`, `"e7e8q"`). SAN disambiguation is left to the existing
+  pure-Ruby `PGN::Notation`.
+- `#legal?(uci)` — whether a UCI move is legal.
+
+Benchmark it with `bundle exec rake bench:perft` (after `rake compile`).
+
+The engine is also reachable directly from a `PGN::Position` via a FEN
+round-trip, so you don't have to build the `Engine` by hand:
+
+```ruby
+PGN::Position.start.perft(4)       # => 197281
+PGN::Position.start.legal_moves   # => ["a2a3", "a2a4", ..., "h2h3", "h2h4"] (sorted UCI)
+```
+
+Both delegate to `PGN::Bitboard::Engine.new(position.to_fen.to_s)` and
+require the compiled extension — they raise `NameError` if it is absent.
+`#legal_moves` is ~30 µs/call on a middlegame position (see
+`bench/legal_moves.rb`), so per-position enumeration is practical.
+
+### Distribution
+
+The native extension is distributed as **precompiled platform gems**
+(cross-compiled via `rake-compiler-dock` in CI; see
+`.github/workflows/release-gems.yml`), so end users need no Rust toolchain.
+For interim source builds (e.g. a Docker build stage before prebuilt gems
+are published), install the Rust toolchain in the build stage before
+`bundle install`:
+
+```dockerfile
+RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+ENV PATH=/usr/local/cargo/bin:$PATH
+```
+
+The final runtime image needs nothing extra. If the extension is absent,
+`PGN::Bitboard` is undefined and the pure-Ruby gem works as before; the
+`PGN::Position#perft` and `#legal_moves` delegations, however, raise
+`NameError` in that case (no Ruby fallback).
+
 ## Contributing
 
 1. Fork it

data/Rakefile

@@ -1,21 +1,46 @@
-require "bundler/gem_tasks"
-require "rubocop/rake_task"
+require 'bundler/gem_tasks'
+require 'rake/extensiontask'
+require 'rubocop/rake_task'
+
+spec = Gem::Specification.load('pgn2.gemspec')
+Rake::ExtensionTask.new('pgn2_native', spec) do |ext|
+  ext.ext_dir = 'ext/pgn2_native'
+  ext.lib_dir = 'lib/pgn2_native'
+end
 
 RuboCop::RakeTask.new
 
 namespace :bench do
-  desc "Run move/board profiling and write bench/baseline_moves.txt"
+  desc 'Run move/board profiling and write bench/baseline_moves.txt'
   task :moves do
-    sh "bundle exec ruby bench/profile_moves.rb > bench/baseline_moves.txt"
-    puts File.read("bench/baseline_moves.txt")
+    sh 'bundle exec ruby bench/profile_moves.rb > bench/baseline_moves.txt'
+    puts File.read('bench/baseline_moves.txt')
   end
 
-  desc "Run parse profiling and write bench/baseline_parse.txt"
+  desc 'Run parse profiling and write bench/baseline_parse.txt'
   task :parse do
-    sh "bundle exec ruby bench/profile_parse.rb > bench/baseline_parse.txt"
-    puts File.read("bench/baseline_parse.txt")
+    sh 'bundle exec ruby bench/profile_parse.rb > bench/baseline_parse.txt'
+    puts File.read('bench/baseline_parse.txt')
+  end
+
+  desc 'Run perft benchmark (native Rust bitboard engine)'
+  task :perft do
+    sh 'bundle exec ruby bench/perft.rb'
   end
 end
 
-desc "Run all benchmarks and (re)write bench/baseline_*.txt"
-task :bench => ["bench:moves", "bench:parse"]
+desc 'Run all benchmarks and (re)write bench/baseline_*.txt'
+task bench: ['bench:moves', 'bench:parse']
+
+# Cross-compile prebuilt native platform gems via rake-compiler-dock.
+# Produces fat-binary platform gems so end users (and the chessellence
+# Docker build) need no Rust toolchain. Requires Docker locally.
+namespace :native do
+  desc 'Cross-compile prebuilt platform gems via rake-compiler-dock'
+  task :gem do
+    require 'rake_compiler_dock'
+    RakeCompilerDock.sh <<-SH, verbose: true
+      bundle install && rake native:clean && rake cross native gem
+    SH
+  end
+end

data/TODO.md

@@ -17,31 +17,45 @@
     `#tokens` spec helper. Parse allocations −42% (603537 → 347037 / 500 games).
 - Speed up replay via a board-representation rewrite ("Approach B"): done.
   (b) ✓ (done in 1.3.0) Rewrote `Board` internals to the classic 0x88
-    representation (128-cell array indexed by `rank*16+file`) and rewrote
-    `MoveCalculator` to work entirely in single-integer square indices via
-    `Board#at_index`/`#apply!`, so the replay hot path no longer allocates
-    `[file,rank]` coordinate arrays or square-name strings. Off-board is a
-    single bitmask (`(idx & 0x88).zero?`, ~1.6x faster than a 0..7 bounds
-    check) and ray stepping is a single integer add. Algorithm unchanged, so
-    output is byte-identical. Measured (immortal game): replay 798→535 µs/i
-    (+49% throughput), allocations 1571→976 objects (−38%) / 92440→62064 bytes
-    (−33%); parse+replay +21% throughput. 182 specs green, 0 new rubocop
-    offenses vs main. The public string/coord API is preserved (additive).
+  representation (128-cell array indexed by `rank*16+file`) and rewrote
+  `MoveCalculator` to work entirely in single-integer square indices via
+  `Board#at_index`/`#apply!`, so the replay hot path no longer allocates
+  `[file,rank]` coordinate arrays or square-name strings. Off-board is a
+  single bitmask (`(idx & 0x88).zero?`, ~1.6x faster than a 0..7 bounds
+  check) and ray stepping is a single integer add. Algorithm unchanged, so
+  output is byte-identical. Measured (immortal game): replay 798→535 µs/i
+  (+49% throughput), allocations 1571→976 objects (−38%) / 92440→62064 bytes
+  (−33%); parse+replay +21% throughput. 182 specs green, 0 new rubocop
+  offenses vs main. The public string/coord API is preserved (additive).
+  (c) One related idea was left alone during cleanup rather than "fixed",
+  since fixing it would cost more than it's worth right now: `Board#squares`
+  rebuilds the full 8x8 array from `@cells` on every call (9 allocations,
+  64 reads); it's off the replay hot path by design, but `FEN#to_s`
+  round-trips through it on every position-to-FEN call, so FEN generation
+  pays that cost repeatedly. Memoizing would mean invalidating the cache
+  from `update`/`apply!`, i.e. adding a write to the actual hot path to
+  speed up a path that isn't hot -- the wrong trade; if FEN generation
+  becomes hot, have it read `@cells` directly instead. Also considered
+  and not attempted: column-granularity copy-on-write in `dup` (the pre-0x88
+  Board only duplicated touched file-columns on write); the flat 0x88 array
+  trades that away for simplicity and the +49% throughput measured above,
+  and reintroducing it would need its own A/B before it's worth the
+  complexity.
   (a) ✗ (attempted, rejected) A piece-location index (piece → 0x88 indices)
-    maintained in `update`/`apply!` and used for O(1) slider/leaper/king
-    origin lookups. Implemented on top of (b), all 182 specs green, but it
-    **regressed**: replay 526→727 µs/i (+38% slower), allocations 976→1591
-    objects (+63%). Root cause: `Board#dup` (called every move) must clone
-    the index (`transform_values(&:dup)` ≈ 12 piece arrays) — Board#dup went
-    91→676 objects — and every move pays per-update index maintenance
-    (`<<`/`delete`) that pawns (the most common move type, whose origins are
-    geometry-fixed and can't use the index) pay for no benefit. The index
-    helps sliders/leapers (minority of moves) but the dup + maintenance cost
-    is paid by every move. Conclusion: a global piece index is a loss for
-    replay (where only ONE given move is validated, so ray-scanning from the
-    destination is already cheap); it pays in move-*generation* libraries
-    (chess.js/python-chess) that enumerate ALL legal moves. Not worth a COW
-    variant either (maintenance + pawns). Reverted; (b) alone is the winner.
+  maintained in `update`/`apply!` and used for O(1) slider/leaper/king
+  origin lookups. Implemented on top of (b), all 182 specs green, but it
+  **regressed**: replay 526→727 µs/i (+38% slower), allocations 976→1591
+  objects (+63%). Root cause: `Board#dup` (called every move) must clone
+  the index (`transform_values(&:dup)` ≈ 12 piece arrays) — Board#dup went
+  91→676 objects — and every move pays per-update index maintenance
+  (`<<`/`delete`) that pawns (the most common move type, whose origins are
+  geometry-fixed and can't use the index) pay for no benefit. The index
+  helps sliders/leapers (minority of moves) but the dup + maintenance cost
+  is paid by every move. Conclusion: a global piece index is a loss for
+  replay (where only ONE given move is validated, so ray-scanning from the
+  destination is already cheap); it pays in move-_generation_ libraries
+  (chess.js/python-chess) that enumerate ALL legal moves. Not worth a COW
+  variant either (maintenance + pawns). Reverted; (b) alone is the winner.
 - Replace the right-recursive `tag_section`/`variation_list` rules in
   `pgn_parser.y` with ordinary left-recursion plus one explicit `.reverse`
   at the point each list is consumed, so the legacy whittle-order
@@ -53,10 +67,67 @@
   to reuse as-is. The brace check is a bandaid for `clean_text` not fully
   normalizing multi-line/nested comments in one pass; fixing that at the
   source would let `moves=` reuse unconditionally.
-- `MoveCalculator#destination_coords` memoizes into `@dest_coords` based on
-  `board`/`move`/`origin` never changing after `#initialize` — true today,
-  but only by convention, since `board`, `move`, `origin` are all public
-  `attr_accessor`s with no cache invalidation tied to their setters. If a
-  future caller ever mutates and reuses a `MoveCalculator` instance, this
-  memo goes stale silently. Either drop the public setters or invalidate
-  `@dest_coords` when they're used.
+
+## Roadmap ideas (from pioz/chess + python-chess review)
+
+Out of scope for now: SVG rendering, Chess960, Shredder-FEN, FRC castling.
+
+### Group 1 — PGN & Format Robustness
+
+- [ ] Streaming/lazy PGN reader: yield games from an `IO` without slurping
+      the whole file.
+- [ ] EPD read/write (`EPD#to_position`, `FEN#to_epd`).
+- [ ] Richer game-tree API: node-style mainline/variations, add/promote/
+      demote variations.
+- [ ] Nested-comment normalization + brace escaping for byte-perfect round
+      trips.
+- [ ] Tolerant parse mode: collect warnings/errors instead of failing on the
+      first bad move.
+
+### Group 2 — Position Intelligence / Game Rules
+
+- [ ] Full legal-move API (`Position#legal_moves`, `Position#legal?(san)`).
+- [ ] Check / pin / attackers helper methods (logic already exists privately
+      in `Notation`).
+- [ ] Game outcome detection: checkmate, stalemate, insufficient material,
+      50-move rule, threefold repetition.
+- [ ] Mutable push/pop history (`game.push(san)`, `game.pop`).
+- [ ] Position equality/hash based on the FEN-relevant parts.
+
+### Group 3 — Engine & Analysis Integration
+
+- [ ] Lightweight UCI/XBoard engine wrapper (`PGN::Engine`-style).
+- [ ] PGN annotation helpers: auto-generate NAGs/comments from engine info.
+- [ ] Optional Polyglot opening-book reader and/or Syzygy tablebase prober.
+- [ ] UCI-style castling normalization (see pioz/chess).
+
+### Group 4 — Performance / Internals  *(in progress on `perf/internals-plan`)*
+
+- [ ] FEN serializer that walks the 0x88 `@cells` array directly (skip
+      rebuilding `board.squares`).
+- [ ] Incremental Zobrist hashing in `Board`/`Position` for fast repetition /
+      transposition checks.
+- [ ] Lazy position iterator instead of eagerly building `Game#positions`.
+- [ ] Precomputed knight/king attack masks for SAN generation and legal-move
+      checks.
+- [x] Optional bitboard / C-extension backend for move generation / perft,
+      with a pure-Ruby fallback. *(done on `feat/rust-bitboard-perft` — see
+      `docs/superpowers/specs/2026-08-13-rust-bitboard-perft-design.md`;
+      `PGN::Bitboard::Engine#perft`/`#legal_moves`/`#legal?` via a Rust
+      `cdylib` shipped as precompiled platform gems.)*
+  - [ ] **Perft speed: pin-aware legal move generation.** Current legal perft
+        is ~20 Mnps (release, LTO); pseudo-legal perft is 76 Mnps, so the
+        per-move `in_check` (`is_attacked`) scan is ~3.7× of the runtime and
+        the whole gap to fast engines. Implementing pin-aware legality
+        (compute checkers + pinned pieces once per node, validate moves
+        without make/unmake, true bulk-count at depth 1) targets ~50–70 Mnps
+        (~2× off Stockfish). Keep the make/unmake path for the Ruby
+        `#legal_moves` API; only the `perft` hot path switches. Validated by
+        the existing perft oracle vs Stockfish (`bench/cross_check.rb`).
+  - [ ] Bulk-count at depth 1 (small, low-risk) and per-node `MoveList` buffer
+        reuse (avoid the per-node `Vec<Move>` alloc) — incremental wins below
+        the pin-aware rewrite.
+  - [ ] aarch64 (non-BMI2) falls back to the ray-walker sliders; add magic
+        bitboard tables there if its perft nps matters.
+  - [ ] Verify the `release-gems.yml` cross-compile (rake-compiler-dock) for
+        x86_64/aarch64 linux+darwin before relying on prebuilt gems.

data/bench/baseline_moves.txt

@@ -1,8 +1,8 @@
 Workload: immortal game, 45 plies
 
 === 1. Replay allocations (45 plies, no parse) ===
-total_allocated objects: 976
-total_allocated bytes:   62064
+total_allocated objects: 931
+total_allocated bytes:   66728
 
 === 2. Board#dup x45 (target of flat-board COW) ===
 total_allocated objects: 91
@@ -15,8 +15,42 @@ total_allocated bytes:   0
 === 4. Replay throughput (ips, excluding parse) ===
 ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [x86_64-linux]
 Warming up --------------------------------------
-     replay immortal   193.000 i/100ms
+     replay immortal   176.000 i/100ms
 Calculating -------------------------------------
-     replay immortal      1.894k (± 2.9%) i/s  (528.02 μs/i) -      9.650k in   5.095389s
+     replay immortal      1.752k (± 3.1%) i/s  (570.75 μs/i) -      8.800k in   5.022596s
+
+=== 5. FEN generation x46 (target of direct-0x88 FEN) ===
+total_allocated objects: 1821
+total_allocated bytes:   96784
+
+=== 6. FEN throughput (ips) ===
+ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [x86_64-linux]
+Warming up --------------------------------------
+        fen immortal   130.000 i/100ms
+Calculating -------------------------------------
+        fen immortal      1.305k (± 2.5%) i/s  (766.24 μs/i) -      6.630k in   5.080141s
+
+=== 7. Last-position-only (45 plies): lazy vs eager ===
+lazy  total_allocated objects: 1018
+lazy  total_allocated bytes:   65032
+eager total_allocated objects: 1020
+eager total_allocated bytes:   65680
+
+=== 8. SAN generation x45 (target of attack masks) ===
+total_allocated objects: 1387
+total_allocated bytes:   206216
+
+=== 8b. SAN throughput (ips) ===
+ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [x86_64-linux]
+Warming up --------------------------------------
+        san immortal    36.000 i/100ms
+Calculating -------------------------------------
+        san immortal    364.877 (± 0.8%) i/s    (2.74 ms/i) -      1.836k in   5.031829s
+
+=== 9. Retained memory (45 plies): lazy vs eager ===
+lazy  total_retained objects: 95
+lazy  total_retained bytes:   6280
+eager total_retained objects: 287
+eager total_retained bytes:   17232
 
 Done. Compare this file against bench/baseline_moves.txt after optimizations.

data/bench/baseline_parse.txt

@@ -5,21 +5,21 @@ total_allocated objects: 288537
 total_allocated bytes:   15640374
 
 === 2. Parse + replay allocations (500 games) ===
-total_allocated objects: 773030
-total_allocated bytes:   45626128
+total_allocated objects: 751030
+total_allocated bytes:   44812592
 
 === 3. Parse-only throughput (ips) ===
 ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [x86_64-linux]
 Warming up --------------------------------------
      parse 500 games     1.000 i/100ms
 Calculating -------------------------------------
-     parse 500 games      4.926 (±20.3%) i/s  (203.01 ms/i) -     25.000 in   5.075163s
+     parse 500 games      5.530 (±18.1%) i/s  (180.82 ms/i) -     28.000 in   5.063031s
 
 === 4. Parse + replay throughput (ips) ===
 ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [x86_64-linux]
 Warming up --------------------------------------
 parse+replay 500 games     1.000 i/100ms
 Calculating -------------------------------------
-parse+replay 500 games      2.065 (± 0.0%) i/s  (484.34 ms/i) -     11.000 in   5.327699s
+parse+replay 500 games      2.174 (± 0.0%) i/s  (460.08 ms/i) -     11.000 in   5.060868s
 
 Done. Compare this file against bench/baseline_parse.txt after optimizations.

data/bench/cross_check.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+# Cross-check the native Rust engine against Stockfish (independent oracle):
+# correctness (perft node counts) and timing (nps). Requires `stockfish`.
+require 'open3'
+
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+require 'pgn'
+
+unless PGN::Bitboard.const_defined?(:Engine)
+  warn 'PGN::Bitboard::Engine not compiled — run `bundle exec rake compile` first.'
+  exit 1
+end
+
+POSITIONS = {
+  'startpos' => 'rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1',
+  'kiwipete' => 'r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1',
+  'pos3'     => '8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1',
+  'pos4'     => 'r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1',
+  'pos5'     => 'rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8',
+  'pos6'     => 'r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10'
+}
+
+def monotonic
+  Process.clock_gettime(Process::CLOCK_MONOTONIC)
+end
+
+def stockfish_perft(fen, depth)
+  cmd = "position fen #{fen}\ngo perft #{depth}\nquit\n"
+  out, _ = Open3.capture2('stockfish', stdin_data: cmd)
+  m = out.match(/Nodes searched:\s+(\d+)/)
+  m && m[1].to_i
+end
+
+puts "name       depth  native           stockfish        match   native_nps   sf_nps"
+puts '-' * 80
+all_match = true
+POSITIONS.each do |name, fen|
+  e = PGN::Bitboard::Engine.new(fen)
+  depths = name == 'startpos' ? [5, 6] : [4, 5]
+  depths.each do |d|
+    nodes = e.perft(d)
+    t0 = monotonic
+    e.perft(d)
+    t = monotonic - t0
+    native_nps = (nodes.to_f / t).to_i
+
+    s0 = monotonic
+    sf_nodes = stockfish_perft(fen, d)
+    st = monotonic - s0
+    sf_nps = (sf_nodes.to_f / st).to_i
+
+    ok = (nodes == sf_nodes)
+    all_match = false unless ok
+    printf("%-10s  d%d   %-13d   %-13d   %s   %-11d  %d\n",
+           name, d, nodes, sf_nodes, ok ? 'OK' : 'DIFF', native_nps, sf_nps)
+  end
+end
+
+puts '-' * 80
+puts all_match ? 'ALL POSITIONS MATCH STOCKFISH ✅' : 'MISMATCH DETECTED ❌'
+exit(all_match ? 0 : 1)

data/bench/legal_moves.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+# Benchmark PGN::Position#legal_moves end-to-end (FEN round-trip +
+# native legal-gen + Ruby string materialization) to decide whether
+# shipping the method meets the < 1 ms middlegame gate.
+#
+# Run: bundle exec ruby bench/legal_moves.rb
+
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+require 'pgn'
+
+unless PGN::Bitboard.const_defined?(:Engine)
+  warn 'PGN::Bitboard::Engine not compiled — build with `bundle exec rake compile` first.'
+  exit 1
+end
+
+POSITIONS = {
+  'startpos' => 'rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1',
+  'middlegame' => 'r1bqkbnr/pppp1ppp/2n5/4p3/2B1P3/5N2/PPPP1PPP/RNBQK2R w KQkq - 4 4',
+  'kiwipete' => 'r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1'
+}.freeze
+
+def monotonic
+  Process.clock_gettime(Process::CLOCK_MONOTONIC)
+end
+
+POSITIONS.each do |name, fen|
+  pos = PGN::FEN.new(fen).to_position
+  # warmup
+  100.times { pos.legal_moves }
+  n = 2000
+  t0 = monotonic
+  n.times { pos.legal_moves }
+  elapsed = monotonic - t0
+  us = (elapsed / n) * 1_000_000.0
+  count = pos.legal_moves.length
+  printf("%-12s moves=%-3d  %.1f us/call  (%.3f ms)\n", name, count, us, us / 1000.0)
+end

data/bench/perft.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
+require 'pgn'
+
+POSITIONS = {
+  'startpos' => 'rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1',
+  'kiwipete' => 'r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1',
+  'pos3' => '8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1',
+  'pos5' => 'rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8'
+}.freeze
+
+unless PGN::Bitboard.const_defined?(:Engine)
+  warn 'PGN::Bitboard::Engine not compiled — build with `bundle exec rake compile` first.'
+  exit 1
+end
+
+def monotonic
+  Process.clock_gettime(Process::CLOCK_MONOTONIC)
+end
+
+POSITIONS.each do |name, fen|
+  e = PGN::Bitboard::Engine.new(fen)
+  [4, 5].each do |d|
+    nodes = e.perft(d)
+    t0 = monotonic
+    e.perft(d)
+    t = monotonic - t0
+    nps = (nodes.to_f / t).to_i
+    printf("%-10s d%d  nodes=%-12d  %.3fs  %d nps\n", name, d, nodes, t, nps)
+  end
+end

data/bench/profile_moves.rb

@@ -50,4 +50,97 @@ Benchmark.ips do |x|
   end
 end
 
+# --- 5. FEN generation (target of the direct-0x88 FEN builder) -------------
+positions = GAME.first.positions
+
+fen_report = MemoryProfiler.report do
+  positions.each { |p| p.to_fen.to_s }
+end
+
+puts "\n=== 5. FEN generation x#{positions.length} (target of direct-0x88 FEN) ==="
+puts "total_allocated objects: #{fen_report.total_allocated}"
+puts "total_allocated bytes:   #{fen_report.total_allocated_memsize}"
+
+puts "\n=== 6. FEN throughput (ips) ==="
+Benchmark.ips do |x|
+  x.config(time: 5, warmup: 1)
+  x.report('fen immortal') do
+    positions.each { |p| p.to_fen.to_s }
+  end
+end
+
+# --- 7. Last-position-only: lazy each_position vs eager positions ---------
+# Both paths build a fresh PGN::Game so the Game construction cost cancels;
+# the difference is building the full positions array (eager) vs not (lazy).
+lazy_report = MemoryProfiler.report do
+  game = PGN::Game.new(SAN, GAME.first.tags, GAME.first.result)
+  last = nil
+  game.each_position { |p| last = p }
+  last
+end
+
+eager_report = MemoryProfiler.report do
+  PGN::Game.new(SAN, GAME.first.tags, GAME.first.result).positions.last
+end
+
+puts "\n=== 7. Last-position-only (#{PLY} plies): lazy vs eager ==="
+puts "lazy  total_allocated objects: #{lazy_report.total_allocated}"
+puts "lazy  total_allocated bytes:   #{lazy_report.total_allocated_memsize}"
+puts "eager total_allocated objects: #{eager_report.total_allocated}"
+puts "eager total_allocated bytes:   #{eager_report.total_allocated_memsize}"
+
+# --- 8. SAN generation (Notation.san reconstruction from coords) -----------
+# Precompute one (position, from, to, promotion) tuple per ply outside the
+# measured block, so only Notation.san (reaches?/attacked?/disambiguation)
+# is measured — the path the knight/king attack masks target.
+san_inputs = []
+begin
+  pos = GAME.first.starting_position
+  SAN.each do |m|
+    mv = PGN::Move.new(m, pos.player)
+    calc = PGN::MoveCalculator.new(pos.board, mv)
+    san_inputs << [pos, calc.origin, mv.destination, mv.promotion]
+    pos = pos.move(m)
+  end
+end
+
+san_report = MemoryProfiler.report do
+  san_inputs.each { |pos, from, to, promo| PGN::Notation.san(pos, from, to, promo) }
+end
+
+puts "\n=== 8. SAN generation x#{san_inputs.length} (target of attack masks) ==="
+puts "total_allocated objects: #{san_report.total_allocated}"
+puts "total_allocated bytes:   #{san_report.total_allocated_memsize}"
+
+puts "\n=== 8b. SAN throughput (ips) ==="
+Benchmark.ips do |x|
+  x.config(time: 5, warmup: 1)
+  x.report('san immortal') do
+    san_inputs.each { |pos, from, to, promo| PGN::Notation.san(pos, from, to, promo) }
+  end
+end
+
+# --- 9. Retained memory: lazy each_position vs eager positions ------------
+# Objects allocated during the report that are STILL ALIVE at its end.
+# Both reports keep the Game alive in an outer var so the difference is
+# purely what the API memoizes: lazy streams without memoizing the array;
+# eager memoizes the full positions array on the Game.
+lazy_game = nil
+lazy_retained = MemoryProfiler.report do
+  lazy_game = PGN::Game.new(SAN, GAME.first.tags, GAME.first.result)
+  lazy_game.each_position { |_| } # stream; do NOT memoize the array
+end
+
+eager_game = nil
+eager_retained = MemoryProfiler.report do
+  eager_game = PGN::Game.new(SAN, GAME.first.tags, GAME.first.result)
+  eager_game.positions # memoize the full array on the Game
+end
+
+puts "\n=== 9. Retained memory (#{PLY} plies): lazy vs eager ==="
+puts "lazy  total_retained objects: #{lazy_retained.total_retained}"
+puts "lazy  total_retained bytes:   #{lazy_retained.total_retained_memsize}"
+puts "eager total_retained objects: #{eager_retained.total_retained}"
+puts "eager total_retained bytes:   #{eager_retained.total_retained_memsize}"
+
 puts "\nDone. Compare this file against bench/baseline_moves.txt after optimizations."