pgn2 0.4.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +50 -0
- data/.github/workflows/publish.yml +75 -0
- data/.github/workflows/release.yml +103 -0
- data/.gitignore +2 -1
- data/.rubocop.yml +38 -0
- data/CHANGELOG.md +84 -0
- data/README.md +122 -4
- data/Rakefile +20 -0
- data/TODO.md +9 -0
- data/bench/.keep +0 -0
- data/bench/IMPROVEMENTS.md +143 -0
- data/bench/baseline_moves.pre-optimization.txt +22 -0
- data/bench/baseline_moves.pre-quickwins.txt +23 -0
- data/bench/baseline_moves.txt +22 -0
- data/bench/baseline_parse.pre-optimization.txt +25 -0
- data/bench/baseline_parse.pre-quickwins.txt +26 -0
- data/bench/baseline_parse.racc.txt +25 -0
- data/bench/baseline_parse.txt +25 -0
- data/bench/profile_moves.rb +53 -0
- data/bench/profile_parse.rb +44 -0
- data/docs/superpowers/plans/2026-08-12-efficiency-optimizations.md +573 -0
- data/docs/superpowers/plans/2026-08-12-efficiency-tests-and-profiling.md +1091 -0
- data/docs/superpowers/plans/2026-08-12-to-pgn-serialization.md +162 -0
- data/docs/superpowers/plans/2026-08-13-whittle-to-racc-migration.md +130 -0
- data/docs/superpowers/specs/2026-08-12-to-pgn-serialization-design.md +217 -0
- data/docs/superpowers/specs/2026-08-13-pgn-performance-quick-wins-design.md +227 -0
- data/lib/pgn/board.rb +33 -15
- data/lib/pgn/fen.rb +16 -8
- data/lib/pgn/game.rb +23 -3
- data/lib/pgn/lexer.rb +223 -0
- data/lib/pgn/move.rb +12 -5
- data/lib/pgn/move_calculator.rb +27 -21
- data/lib/pgn/parser.rb +13 -203
- data/lib/pgn/pgn_parser.rb +393 -0
- data/lib/pgn/pgn_parser.y +140 -0
- data/lib/pgn/position.rb +3 -2
- data/lib/pgn/serializer.rb +141 -0
- data/lib/pgn/version.rb +1 -1
- data/lib/pgn.rb +3 -0
- data/pgn2.gemspec +12 -2
- data/spec/board_spec.rb +111 -0
- data/spec/fen_spec.rb +25 -0
- data/spec/game_spec.rb +74 -0
- data/spec/lexer_spec.rb +153 -0
- data/spec/move_calculator_spec.rb +226 -0
- data/spec/move_spec.rb +136 -0
- data/spec/parser_explicit_spec.rb +210 -0
- data/spec/parser_spec.rb +6 -23
- data/spec/pgn_files/doublequotes.pgn +21 -0
- data/spec/pgn_files/specialcharacters.pgn +79 -0
- data/spec/position_spec.rb +73 -0
- data/spec/serializer_spec.rb +89 -0
- data/spec/spec_helper.rb +0 -1
- metadata +103 -15
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# PGN2 performance quick wins — Approach A (quick, behavior-compatible micro-opts)
|
|
2
|
+
|
|
3
|
+
> "Approach A" denotes the safe, allocation-focused track. A later, larger
|
|
4
|
+
> design (Approach B) will cover architectural changes: piece-location indexes,
|
|
5
|
+
> lazy position computation, and a coordinate-only internal board
|
|
6
|
+
> representation.
|
|
7
|
+
|
|
8
|
+
## Goal
|
|
9
|
+
|
|
10
|
+
Make the gem faster on the combined **parse + replay** workload after the
|
|
11
|
+
whittle → Racc migration, with only safe, behavior-compatible micro-optimizations
|
|
12
|
+
(no public API changes, no serialized-output changes).
|
|
13
|
+
|
|
14
|
+
## Scope
|
|
15
|
+
|
|
16
|
+
This design covers the five highest-ROI quick wins identified in the profiles:
|
|
17
|
+
|
|
18
|
+
1. Remove per-token `PGN::Lexer::Token` allocation on the parser hot path.
|
|
19
|
+
2. Avoid double `MoveText` wrapping in `PGN::Game#moves=`.
|
|
20
|
+
3. Replace `PGN::Move` SAN regex with hand-rolled parsing.
|
|
21
|
+
4. Reduce small temporary allocations in `PGN::Position#move` and
|
|
22
|
+
`PGN::MoveCalculator`.
|
|
23
|
+
5. Promote the Racc parse baseline to canonical and record new baselines in
|
|
24
|
+
`bench/baseline_*.txt` after verification.
|
|
25
|
+
|
|
26
|
+
Out of scope: larger architecture changes such as piece-location indexes,
|
|
27
|
+
lazy position computation, or coordinate-only internal board representation.
|
|
28
|
+
|
|
29
|
+
## Context / baseline
|
|
30
|
+
|
|
31
|
+
Measured on the current checkout with `bundle exec rake bench` (Racc parser);
|
|
32
|
+
these are the canonical `bench/baseline_*.txt` values refreshed as the
|
|
33
|
+
prerequisite in item 5:
|
|
34
|
+
|
|
35
|
+
- **Parse-only:** ~125 k objects / 7.88 MB for 100 copies of the immortal game
|
|
36
|
+
(= 626,037 objects / 39.4 MB for the 500-game corpus in `bench/baseline_parse.txt`).
|
|
37
|
+
- **Parse + replay:** ~337 k objects / 21.6 MB for the same per-100 ratio
|
|
38
|
+
(= 1,683,087 objects / 108 MB for 500 games).
|
|
39
|
+
- **Replay only:** 2,177 objects / 142 kB for the 45-ply immortal game
|
|
40
|
+
(`bench/baseline_moves.txt`).
|
|
41
|
+
|
|
42
|
+
Historical snapshots are kept in `bench/baseline_*.pre-quickwins.txt` and
|
|
43
|
+
`bench/baseline_parse.racc.txt` for reference. The previously committed
|
|
44
|
+
`baseline_parse.txt` held stale pre-migration whittle numbers and has been
|
|
45
|
+
overwritten by the current Racc output.
|
|
46
|
+
|
|
47
|
+
Top allocation hot spots:
|
|
48
|
+
|
|
49
|
+
- `PGN::Lexer::Token` creation (parse-only).
|
|
50
|
+
- `PGN::MoveText` created twice per parsed move (parser + `Game#moves=`).
|
|
51
|
+
- `Move#initialize` SAN regex (`Regexp` + `MatchData` objects, replay).
|
|
52
|
+
- Small throw-away arrays and string conversions in `MoveCalculator` and
|
|
53
|
+
`Position#move`.
|
|
54
|
+
|
|
55
|
+
## Detailed changes
|
|
56
|
+
|
|
57
|
+
### 1. Lexer: parser hot path avoids `Token` objects
|
|
58
|
+
|
|
59
|
+
`lib/pgn/lexer.rb`
|
|
60
|
+
|
|
61
|
+
- Add a fast public method `next_token_pair` that returns `[type, value]`
|
|
62
|
+
directly, without allocating a `PGN::Lexer::Token` Struct.
|
|
63
|
+
The byte offset is **not** expensive (`@ss.pos`) and is still required, so
|
|
64
|
+
the saving is the Struct allocation, not offset computation.
|
|
65
|
+
- **Correctness requirement:** `next_token_pair` must invoke the same
|
|
66
|
+
`note_token(type, off)` and `advance_line(value)` side effects as
|
|
67
|
+
`next_token`. In particular `note_token` maintains `game_starts`, which
|
|
68
|
+
`PgnParser#assign_pgn!` relies on to slice each game's verbatim raw `pgn`
|
|
69
|
+
text — skipping it would corrupt `Game#pgn`.
|
|
70
|
+
- Keep `next_token` and `tokens` unchanged (they continue to return `Token`
|
|
71
|
+
objects and are used by specs/other callers).
|
|
72
|
+
- Implement both methods on top of a shared private scanning routine so the
|
|
73
|
+
scanning logic is not duplicated.
|
|
74
|
+
- The single-byte literal path already returns frozen strings via
|
|
75
|
+
`LITERAL_BYTES` under `# frozen_string_literal: true`; preserve that in
|
|
76
|
+
`next_token_pair` (no change needed beyond routing through the shared
|
|
77
|
+
routine).
|
|
78
|
+
|
|
79
|
+
`lib/pgn/pgn_parser.y`
|
|
80
|
+
|
|
81
|
+
- Change `PgnParser#next_token` to call `@lexer.next_token_pair` and return the
|
|
82
|
+
resulting array directly.
|
|
83
|
+
- Regenerate `lib/pgn/pgn_parser.rb` from the `.y` file with Racc.
|
|
84
|
+
- No changes to grammar or semantics.
|
|
85
|
+
|
|
86
|
+
### 2. Game#moves=: stop re-wrapping existing MoveText objects
|
|
87
|
+
|
|
88
|
+
`lib/pgn/game.rb`
|
|
89
|
+
|
|
90
|
+
- Detect when an element is already a `PGN::MoveText` instance and reuse it
|
|
91
|
+
directly instead of creating a new one.
|
|
92
|
+
- Still perform castling normalization (`0` → `O`) for raw strings.
|
|
93
|
+
- Preserve `clean_text` behavior for fresh comments only.
|
|
94
|
+
- **Reuse-safety invariant:** reusing the parser's `MoveText` shares one object
|
|
95
|
+
between the parser's move tree and `Game#moves`. This is safe only because
|
|
96
|
+
nothing mutates a `MoveText` after construction; record that invariant in a
|
|
97
|
+
comment so a future mutation doesn't silently corrupt both holders.
|
|
98
|
+
|
|
99
|
+
Expected effect: roughly halves `MoveText` allocations on the parse path while
|
|
100
|
+
keeping serialized output identical.
|
|
101
|
+
|
|
102
|
+
### 3. Move#initialize: hand-rolled SAN parser (highest-risk item — quantify first)
|
|
103
|
+
|
|
104
|
+
`lib/pgn/move.rb`
|
|
105
|
+
|
|
106
|
+
This is the highest-risk, and likely lowest-allocation-payoff, item: the SAN
|
|
107
|
+
regex produces ~1 `MatchData` per ply (~2.1% of the 2,177-object replay
|
|
108
|
+
baseline). Before committing to a full hand-roll, **measure #3 in isolation**
|
|
109
|
+
and confirm the payoff; if marginal, ship #1, #2, #4 alone and defer #3.
|
|
110
|
+
|
|
111
|
+
Trivial independent sub-win (do regardless of the full hand-roll):
|
|
112
|
+
|
|
113
|
+
- In `Move#piece=`, replace `return if san.match('O-O')` (which allocates a
|
|
114
|
+
`MatchData` on every `Move.new`, castling or not) with a non-allocating
|
|
115
|
+
check such as `san.include?('O-O')` or `san.start_with?('O')`.
|
|
116
|
+
|
|
117
|
+
Full hand-roll (if pursued):
|
|
118
|
+
|
|
119
|
+
- Replace `move.match(SAN_REGEX)` with a manual parse over the byte/string
|
|
120
|
+
representation of the SAN notation.
|
|
121
|
+
- Populate all existing attributes (`piece`, `destination`, `promotion`,
|
|
122
|
+
`check`, `capture`, `disambiguation`, `castle`) with the same values as
|
|
123
|
+
today, including the `piece=` early-return for castling.
|
|
124
|
+
- Keep setter methods (`piece=`, `promotion=`, `capture=`, `disambiguation=`,
|
|
125
|
+
`castle=`) so external callers that assign attributes manually are unaffected.
|
|
126
|
+
- Use frozen constants for frequently-checked piece sets (`pawn?` lives here,
|
|
127
|
+
in `move.rb` — see item 4).
|
|
128
|
+
- **Upfront fixtures:** add a dedicated spec pinning every attribute for a
|
|
129
|
+
comprehensive set of SAN strings: `O-O`, `O-O-O`, `O-O+`, `O-O-O#`;
|
|
130
|
+
pawn moves and captures incl. promotion (`e4`, `exd5`, `e8=Q`, `exd8=Q+`,
|
|
131
|
+
`b1=N#`); piece moves with file/rank/full disambiguation (`Nbd2`, `R1e2`,
|
|
132
|
+
`Qh4e1`); captures with check/mate; and the `--` "don't care" move. Do this
|
|
133
|
+
before implementation, not only if ambiguity surfaces.
|
|
134
|
+
|
|
135
|
+
### 4. Position / MoveCalculator: fewer throw-away allocations
|
|
136
|
+
|
|
137
|
+
`lib/pgn/position.rb`
|
|
138
|
+
|
|
139
|
+
- Inline `next_player` logic in `Position#move` (or simply rewrite its body to
|
|
140
|
+
`player == :white ? :black : :white`) to avoid the `(PLAYERS - [player])`
|
|
141
|
+
array allocation per ply.
|
|
142
|
+
- Apply `castling_restrictions` only when non-empty; avoid array subtraction
|
|
143
|
+
when there is nothing to remove. (Safe to return the existing `castling`
|
|
144
|
+
array directly because castling arrays are replaced, never mutated.)
|
|
145
|
+
|
|
146
|
+
`lib/pgn/move_calculator.rb`
|
|
147
|
+
|
|
148
|
+
- Compute and cache destination coordinates in `initialize` instead of calling
|
|
149
|
+
`board.coordinates_for(move.destination)` repeatedly (it can be invoked 2–3
|
|
150
|
+
times per move across `direction_origins` / `move_origins` / `pawn_origins`).
|
|
151
|
+
- Inline `valid_square?` boundary checks to remove the per-call method dispatch
|
|
152
|
+
(note: `(0..7)` are frozen range literals already cached by the VM, so the
|
|
153
|
+
win is call overhead, not Range allocation — don't claim the latter).
|
|
154
|
+
- Replace run-time array/hash literals with frozen constants or `case`/`when`
|
|
155
|
+
where they appear on hot paths (e.g. rook-origin lookup in
|
|
156
|
+
`castling_restrictions`, the per-call hash literals `{ 'a1' => 'Q', ... }`).
|
|
157
|
+
- Keep semantic behavior identical (same board updates, same disambiguation,
|
|
158
|
+
same `king_position` scan — Approach A deliberately defers indexing/caching
|
|
159
|
+
to a later design).
|
|
160
|
+
|
|
161
|
+
`lib/pgn/move.rb` (relocated from move_calculator)
|
|
162
|
+
|
|
163
|
+
- `Move#pawn?` uses `%w[P p].include?(piece)`, allocating an array each call;
|
|
164
|
+
it's invoked from `MoveCalculator#increment_halfmove?` and `pawn_origins`.
|
|
165
|
+
Replace with a frozen constant or `piece == 'P' || piece == 'p'`.
|
|
166
|
+
|
|
167
|
+
## Testing & behavior preservation
|
|
168
|
+
|
|
169
|
+
- All existing specs must pass without modification (`bundle exec rspec`).
|
|
170
|
+
- The `bench/baseline_moves.txt` and `bench/baseline_parse.txt` files will be
|
|
171
|
+
regenerated with `bundle exec rake bench` and committed if they show lower
|
|
172
|
+
object/byte counts and equal or better throughput.
|
|
173
|
+
- Public API remains unchanged: `PGN.parse`, `Game#moves=`, `Move.new`,
|
|
174
|
+
`Position#move`, etc. accept the same inputs and produce the same outputs and
|
|
175
|
+
side effects.
|
|
176
|
+
- No changes to PGN/FEN serialization format.
|
|
177
|
+
- **Lexer non-regression:** because `next_token_pair` shares the scanning
|
|
178
|
+
routine with `next_token`/`tokens`, the existing `tokens`/`next_token` specs
|
|
179
|
+
and the `game_starts`-driven `Game#pgn` raw-text slicing output must stay
|
|
180
|
+
byte-identical across the fixtures and inline inputs in
|
|
181
|
+
`spec/parser_explicit_spec.rb`.
|
|
182
|
+
|
|
183
|
+
## Risks & mitigations
|
|
184
|
+
|
|
185
|
+
| Risk | Mitigation |
|
|
186
|
+
|---|---|
|
|
187
|
+
| Hand-rolled SAN parser mishandles an edge case | Upfront comprehensive SAN fixture spec (castling both sides, promotion with/without check, pawn & piece captures, file/rank/full disambiguation, `--`) added before implementation; full suite must stay green. |
|
|
188
|
+
| `next_token_pair` drifts from `next_token` behavior | Both methods share the private scanning routine, the same `RULES` ordering, and the same `note_token`/`advance_line` side effects (incl. `game_starts`). |
|
|
189
|
+
| `Game#moves=` no longer applies castling normalization | Continue normalizing raw strings; only skip work when input is already `MoveText`. |
|
|
190
|
+
| Performance wins smaller than expected | Measure before/after with the committed baseline harness. |
|
|
191
|
+
|
|
192
|
+
## Acceptance criteria
|
|
193
|
+
|
|
194
|
+
1. `bundle exec rspec` passes (0 failures).
|
|
195
|
+
2. `lib/pgn/pgn_parser.rb` is regenerated and in sync with `.y`
|
|
196
|
+
(`git diff --stat` should reflect the generation, and CI racc-sync check
|
|
197
|
+
passes).
|
|
198
|
+
3. `bundle exec rake bench` shows lower allocated objects/bytes than the
|
|
199
|
+
**post-Racc** committed baselines (`baseline_moves.txt` and
|
|
200
|
+
`baseline_parse.txt`, the latter promoted from `baseline_parse.racc.txt`
|
|
201
|
+
per item 5). Comparing against the old whittle `baseline_parse.txt` would
|
|
202
|
+
not be a meaningful gate.
|
|
203
|
+
4. IPS numbers are equal or higher than current baselines.
|
|
204
|
+
5. Baseline files are updated and committed as part of the change set.
|
|
205
|
+
6. Lexer non-regression: `tokens`/`next_token` spec output and `Game#pgn`
|
|
206
|
+
raw-text slicing are byte-identical to the pre-change checkout.
|
|
207
|
+
|
|
208
|
+
## Estimated impact
|
|
209
|
+
|
|
210
|
+
Per-item expectations (to be confirmed by measurement, not assumed in
|
|
211
|
+
aggregate):
|
|
212
|
+
|
|
213
|
+
- **Items 1, 2, 4 (low-risk):** the largest reliable wins. Parse-side `Token`
|
|
214
|
+
elimination (#1) and halved `MoveText` allocation (#2) should show up directly
|
|
215
|
+
in parse-only object counts; `next_player`/`pawn?`/`castling_restrictions`
|
|
216
|
+
(#4) trim a handful of objects per ply.
|
|
217
|
+
- **Item 3 (SAN hand-roll):** ~1 `MatchData` per ply (~2.1% of the 2,177-object
|
|
218
|
+
replay baseline) plus the `piece=` guard fix — modest, and the bulk of the
|
|
219
|
+
estimate does **not** come from this item.
|
|
220
|
+
- **Replay allocations are dominated by `Position.new` + `Board#dup` +
|
|
221
|
+
`MoveCalculator` arrays**, which Approach A only trims at the edges; the big
|
|
222
|
+
replay win is deferred to Approach B.
|
|
223
|
+
|
|
224
|
+
Conservative aggregate guess once #1/#2/#4 land: parse-only allocations down
|
|
225
|
+
~10–15%, replay allocations down in the single-digit-percent range (lower than
|
|
226
|
+
a naive reading of these micro-opts might suggest). Actual numbers will come
|
|
227
|
+
from the baseline run; revise this section with measured per-item deltas.
|
data/lib/pgn/board.rb
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
module PGN
|
|
2
4
|
# {PGN::Board} represents the squares of a chess board and the pieces on
|
|
3
5
|
# each square. It is responsible for translating between a human readable
|
|
@@ -73,6 +75,7 @@ module PGN
|
|
|
73
75
|
#
|
|
74
76
|
def initialize(squares)
|
|
75
77
|
self.squares = squares
|
|
78
|
+
@owned = Array.new(8, false)
|
|
76
79
|
end
|
|
77
80
|
|
|
78
81
|
# @overload at(str)
|
|
@@ -88,13 +91,12 @@ module PGN
|
|
|
88
91
|
# board.at(4,3) #=> "P"
|
|
89
92
|
# board.at("e4") #=> "P"
|
|
90
93
|
#
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
end
|
|
94
|
+
# String squares are parsed with getbyte arithmetic (a=0x61, '1'=0x31)
|
|
95
|
+
# so the common string lookup allocates nothing.
|
|
96
|
+
def at(arg0, arg1 = nil)
|
|
97
|
+
return squares[arg0][arg1] unless arg1.nil?
|
|
98
|
+
|
|
99
|
+
squares[file_of(arg0)][rank_of(arg0)]
|
|
98
100
|
end
|
|
99
101
|
|
|
100
102
|
# @param changes [Hash<String, <String, nil>>] changes to make to the board
|
|
@@ -115,9 +117,16 @@ module PGN
|
|
|
115
117
|
# @example
|
|
116
118
|
# board.update("e4", "P")
|
|
117
119
|
#
|
|
120
|
+
# Copy-on-write: clone only the column being mutated, and only once per
|
|
121
|
+
# instance, so unchanged columns stay shared with any board this one was
|
|
122
|
+
# duped from.
|
|
118
123
|
def update(square, piece)
|
|
119
|
-
|
|
120
|
-
|
|
124
|
+
file = file_of(square)
|
|
125
|
+
unless @owned[file]
|
|
126
|
+
squares[file] = squares[file].dup
|
|
127
|
+
@owned[file] = true
|
|
128
|
+
end
|
|
129
|
+
squares[file][rank_of(square)] = piece
|
|
121
130
|
self
|
|
122
131
|
end
|
|
123
132
|
|
|
@@ -127,10 +136,7 @@ module PGN
|
|
|
127
136
|
# board.coordinates_for("e4") #=> [4, 3]
|
|
128
137
|
#
|
|
129
138
|
def coordinates_for(position)
|
|
130
|
-
|
|
131
|
-
file = FILE_TO_INDEX[file_chr]
|
|
132
|
-
rank = RANK_TO_INDEX[rank_chr]
|
|
133
|
-
[file, rank]
|
|
139
|
+
[file_of(position), rank_of(position)]
|
|
134
140
|
end
|
|
135
141
|
|
|
136
142
|
# @param coordinates [Array<Integer>] the coordinates of the square
|
|
@@ -154,10 +160,22 @@ module PGN
|
|
|
154
160
|
end.join("\n")
|
|
155
161
|
end
|
|
156
162
|
|
|
157
|
-
# @return [PGN::Board] a copy of self
|
|
163
|
+
# @return [PGN::Board] a copy of self. The outer array is copied; the
|
|
164
|
+
# 8 column arrays are shared and cloned lazily by #update on first
|
|
165
|
+
# mutation (copy-on-write).
|
|
158
166
|
#
|
|
159
167
|
def dup
|
|
160
|
-
PGN::Board.new(squares.
|
|
168
|
+
PGN::Board.new(squares.dup)
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
private
|
|
172
|
+
|
|
173
|
+
def file_of(square)
|
|
174
|
+
square.getbyte(0) - 97
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
def rank_of(square)
|
|
178
|
+
square.getbyte(1) - 49
|
|
161
179
|
end
|
|
162
180
|
end
|
|
163
181
|
end
|
data/lib/pgn/fen.rb
CHANGED
|
@@ -98,14 +98,22 @@ module PGN
|
|
|
98
98
|
# PGN::FEN.start.board_string #=> "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR"
|
|
99
99
|
#
|
|
100
100
|
def board_string
|
|
101
|
-
self.board
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
101
|
+
rows = self.board.squares.transpose.reverse
|
|
102
|
+
rows.map do |row|
|
|
103
|
+
s = +""
|
|
104
|
+
run = 0
|
|
105
|
+
row.each do |e|
|
|
106
|
+
if e.nil?
|
|
107
|
+
run += 1
|
|
108
|
+
else
|
|
109
|
+
s << run.to_s if run > 0
|
|
110
|
+
run = 0
|
|
111
|
+
s << e
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
s << run.to_s if run > 0
|
|
115
|
+
s
|
|
116
|
+
end.join("/")
|
|
109
117
|
end
|
|
110
118
|
|
|
111
119
|
# @return [PGN::Position] a {PGN::Position} representing the current
|
data/lib/pgn/game.rb
CHANGED
|
@@ -77,17 +77,37 @@ module PGN
|
|
|
77
77
|
#
|
|
78
78
|
# Standardize castling moves to use O's instead of 0's
|
|
79
79
|
#
|
|
80
|
+
# Reuse-safety invariant: when an element is already a MoveText we reuse
|
|
81
|
+
# it directly (sharing one object between the parser's move tree and
|
|
82
|
+
# Game#moves) whenever its comment is already fully cleaned. This is safe
|
|
83
|
+
# because nothing mutates a MoveText after this assignment returns. We
|
|
84
|
+
# still re-wrap when the comment carries braces: the parser's single-pass
|
|
85
|
+
# clean_text leaves braces on multi-line/nested comments, and the second
|
|
86
|
+
# clean_text that MoveText.new applies here is load-bearing for those
|
|
87
|
+
# (matches the legacy whittle byte-output). Skipping it would change
|
|
88
|
+
# serialized comments.
|
|
80
89
|
def moves=(moves)
|
|
81
90
|
@moves =
|
|
82
91
|
moves.map do |m|
|
|
83
|
-
if m.is_a?
|
|
84
|
-
MoveText.new(m.gsub('0', 'O'))
|
|
85
|
-
|
|
92
|
+
if m.is_a?(String)
|
|
93
|
+
MoveText.new(m.include?('0') ? m.gsub('0', 'O') : m)
|
|
94
|
+
elsif m.notation.include?('0')
|
|
86
95
|
MoveText.new(m.notation.gsub('0', 'O'), m.annotation, m.comment, m.variations)
|
|
96
|
+
elsif m.comment.nil? || !m.comment.include?('{')
|
|
97
|
+
m
|
|
98
|
+
else
|
|
99
|
+
MoveText.new(m.notation, m.annotation, m.comment, m.variations)
|
|
87
100
|
end
|
|
88
101
|
end
|
|
89
102
|
end
|
|
90
103
|
|
|
104
|
+
# @return [String] a canonical PGN string for this game, ending with a
|
|
105
|
+
# trailing newline.
|
|
106
|
+
#
|
|
107
|
+
def to_pgn
|
|
108
|
+
PGN::Serializer.new(self).to_s
|
|
109
|
+
end
|
|
110
|
+
|
|
91
111
|
def initial_fen
|
|
92
112
|
tags && tags['FEN']
|
|
93
113
|
end
|
data/lib/pgn/lexer.rb
ADDED
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'strscan'
|
|
4
|
+
|
|
5
|
+
module PGN
|
|
6
|
+
# {PGN::Lexer} is a {StringScanner}-based tokenizer for PGN. It reuses the
|
|
7
|
+
# same terminal patterns as the legacy whittle parser so tokenization is
|
|
8
|
+
# byte-compatible, but performs far fewer Ruby allocations because the
|
|
9
|
+
# scanning happens in the C-backed StringScanner.
|
|
10
|
+
#
|
|
11
|
+
# The lexer also records, per game, the byte offset of the game's first
|
|
12
|
+
# non-discarded token ({#game_starts}). The parser uses these offsets to
|
|
13
|
+
# slice the verbatim {PGN::Game#pgn} raw text out of the original input,
|
|
14
|
+
# reproducing the legacy accumulator's output exactly.
|
|
15
|
+
#
|
|
16
|
+
# All offsets are byte offsets (StringScanner works in bytes); slicing is
|
|
17
|
+
# done with {String#byteslice} so multibyte (UTF-8) input stays intact.
|
|
18
|
+
#
|
|
19
|
+
class Lexer
|
|
20
|
+
Token = Struct.new(:type, :value, :offset, :line, keyword_init: true) do
|
|
21
|
+
def inspect
|
|
22
|
+
"#<PGN::Lexer::Token #{type.inspect} value=#{value.inspect} @#{offset} L#{line}>"
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# Discarded: insignificant whitespace.
|
|
27
|
+
WSP = /\s+/
|
|
28
|
+
|
|
29
|
+
# Discarded: a PGN "rest of line" comment beginning with `%`.
|
|
30
|
+
PGN_COMMENT = /% .*/
|
|
31
|
+
|
|
32
|
+
# A tag value string. Allows unescaped double-quotes inside the value
|
|
33
|
+
# (a form seen in real-world PGN files) — only a bare backslash starts
|
|
34
|
+
# an escape. Matches the legacy parser's relaxed string rule.
|
|
35
|
+
STRING = /
|
|
36
|
+
" # beginning of string
|
|
37
|
+
(
|
|
38
|
+
[[:print:]&&[^\\]] | # printing characters except backslash
|
|
39
|
+
\\\\ | # escaped backslashes
|
|
40
|
+
\\" # escaped quotation marks
|
|
41
|
+
)* # zero or more of the above
|
|
42
|
+
" # end of string
|
|
43
|
+
/x
|
|
44
|
+
|
|
45
|
+
# A brace-delimited comment, with recursive nesting via \g<1>.
|
|
46
|
+
COMMENT = /
|
|
47
|
+
(
|
|
48
|
+
\{ # beginning of comment
|
|
49
|
+
(
|
|
50
|
+
[[:print:]&&[^\\{}]] | # printing characters except brace and backslash
|
|
51
|
+
\n |
|
|
52
|
+
\\\\ | # escaped backslashes
|
|
53
|
+
\\\{|\\\} | # escaped braces
|
|
54
|
+
\n | # newlines
|
|
55
|
+
\g<1> # recursive
|
|
56
|
+
)* # zero or more of the above
|
|
57
|
+
\} # end of comment
|
|
58
|
+
)
|
|
59
|
+
/x
|
|
60
|
+
|
|
61
|
+
# Game termination marker.
|
|
62
|
+
GAME_TERMINATION = %r{
|
|
63
|
+
1-0 | # white wins
|
|
64
|
+
0-1 | # black wins
|
|
65
|
+
1/2-1/2 | # draw
|
|
66
|
+
\* # ?
|
|
67
|
+
}x
|
|
68
|
+
|
|
69
|
+
# A move in standard algebraic notation (incl. castling, promotion,
|
|
70
|
+
# check/mate, the `--` "don't care" move).
|
|
71
|
+
SAN_MOVE = %r{
|
|
72
|
+
(
|
|
73
|
+
-- | # "don't care" move (used in variations)
|
|
74
|
+
[O0](-[O0]){1,2} | # castling (O-O, O-O-O)
|
|
75
|
+
[a-h][1-8] | # pawn moves (e4, d7)
|
|
76
|
+
[BKNQR][a-h1-8]?x?[a-h][1-8] | # major piece moves w/ optional specifier
|
|
77
|
+
[a-h][1-8]?x[a-h][1-8] # pawn captures
|
|
78
|
+
)
|
|
79
|
+
(
|
|
80
|
+
=[BNQR] # optional promotion (d8=Q)
|
|
81
|
+
)?
|
|
82
|
+
(
|
|
83
|
+
\+ | # check (g5+)
|
|
84
|
+
\# # checkmate (Qe7#)
|
|
85
|
+
)?
|
|
86
|
+
}x
|
|
87
|
+
|
|
88
|
+
# A move number indication, e.g. `1.`, `12.`, `1...`.
|
|
89
|
+
MOVE_NUMBER = /[[:digit:]]+\.*/
|
|
90
|
+
|
|
91
|
+
# A tag name (letters, digits, underscores).
|
|
92
|
+
TAG_NAME = /[A-Za-z0-9_]+/
|
|
93
|
+
|
|
94
|
+
# A numeric annotation glyph (`$1`) or a punctuation annotation (`?!`,
|
|
95
|
+
# `!?`, `??`, ...).
|
|
96
|
+
NAG = /
|
|
97
|
+
\$\d+ | # dollar sign followed by an integer
|
|
98
|
+
[?!][?!]? # support the most used annotations directly
|
|
99
|
+
/x
|
|
100
|
+
|
|
101
|
+
# Order matters: more specific / longer tokens are tried first so that
|
|
102
|
+
# e.g. `1-0` (termination) wins over `1` (move number), and `0-0`
|
|
103
|
+
# (castling) wins over `0` (move number). Whitespace and `%` comments
|
|
104
|
+
# are discarded (consumed but not emitted). Beyond that constraint,
|
|
105
|
+
# rules are ordered most- to least-frequent (one san_move/move_number
|
|
106
|
+
# per ply/full-move vs. a handful of comments/strings per game) so the
|
|
107
|
+
# common case fails the fewest regexes before matching.
|
|
108
|
+
RULES = [
|
|
109
|
+
[:wsp, WSP, true], # discarded
|
|
110
|
+
[:pgn_comment, PGN_COMMENT, true], # discarded
|
|
111
|
+
[:game_termination, GAME_TERMINATION, false],
|
|
112
|
+
[:san_move, SAN_MOVE, false],
|
|
113
|
+
[:move_number, MOVE_NUMBER, false],
|
|
114
|
+
[:nag, NAG, false],
|
|
115
|
+
[:comment, COMMENT, false],
|
|
116
|
+
[:string, STRING, false],
|
|
117
|
+
[:tag_name, TAG_NAME, false]
|
|
118
|
+
].freeze.each(&:freeze)
|
|
119
|
+
|
|
120
|
+
# Single-character literals, matched by their byte value: [type, frozen value].
|
|
121
|
+
LITERAL_BYTES = {
|
|
122
|
+
91 => [:lbracket, '['], # [
|
|
123
|
+
93 => [:rbracket, ']'], # ]
|
|
124
|
+
40 => [:lparen, '('], # (
|
|
125
|
+
41 => [:rparen, ')'] # )
|
|
126
|
+
}.freeze
|
|
127
|
+
|
|
128
|
+
def initialize(input)
|
|
129
|
+
@input = input
|
|
130
|
+
@ss = StringScanner.new(input)
|
|
131
|
+
@line = 1
|
|
132
|
+
@game_starts = []
|
|
133
|
+
@between_games = true # at start we are "between" games
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
attr_reader :game_starts
|
|
137
|
+
|
|
138
|
+
# The list of {Token}s for the whole input. Convenience for specs.
|
|
139
|
+
def tokens
|
|
140
|
+
result = []
|
|
141
|
+
while (t = next_token)
|
|
142
|
+
result << t
|
|
143
|
+
end
|
|
144
|
+
result
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
# Returns the next {Token}, or +nil+ at end of input.
|
|
148
|
+
def next_token
|
|
149
|
+
type, value, off = scan_next
|
|
150
|
+
return nil unless type
|
|
151
|
+
Token.new(type: type, value: value, offset: off, line: @line)
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Fast path for the parser: returns [type, value] for the next
|
|
155
|
+
# non-discarded token, or +nil+ at end of input. Does not allocate a
|
|
156
|
+
# {Token} Struct. Shares the same scanning routine and the same
|
|
157
|
+
# +note_token+ / +advance_line+ side effects as +next_token+ (so
|
|
158
|
+
# +game_starts+ tracking is preserved).
|
|
159
|
+
def next_token_pair
|
|
160
|
+
type, value, = scan_next
|
|
161
|
+
return nil unless type
|
|
162
|
+
[type, value]
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
private
|
|
166
|
+
|
|
167
|
+
# Scan to the next non-discarded token and return [type, value, offset],
|
|
168
|
+
# or nil at end of input. Performs the exact +advance_line+ and
|
|
169
|
+
# +note_token+ side effects that +next_token+ historically did, so
|
|
170
|
+
# +game_starts+ (used for verbatim +Game#pgn+ slicing) stays correct.
|
|
171
|
+
def scan_next
|
|
172
|
+
until @ss.eos?
|
|
173
|
+
off = @ss.pos
|
|
174
|
+
|
|
175
|
+
if (lit = LITERAL_BYTES[@input.getbyte(off)])
|
|
176
|
+
type, value = lit
|
|
177
|
+
@ss.pos = off + 1
|
|
178
|
+
note_token(type, off)
|
|
179
|
+
return [type, value, off]
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
type, value, discarded = scan_one
|
|
183
|
+
advance_line(value)
|
|
184
|
+
next if discarded
|
|
185
|
+
|
|
186
|
+
note_token(type, off)
|
|
187
|
+
return [type, value, off]
|
|
188
|
+
end
|
|
189
|
+
nil
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
# Try each terminal rule in order; return [type, matched_string, discarded]
|
|
193
|
+
# for the first match, or raise if nothing matches at the current position.
|
|
194
|
+
def scan_one
|
|
195
|
+
RULES.each do |(type, re, discarded)|
|
|
196
|
+
if (m = @ss.scan(re))
|
|
197
|
+
return [type, m, discarded]
|
|
198
|
+
end
|
|
199
|
+
end
|
|
200
|
+
raise UnconsumedInputError,
|
|
201
|
+
"Unmatched input #{@input.byteslice(@ss.pos..).inspect} on line #{@line}"
|
|
202
|
+
end
|
|
203
|
+
|
|
204
|
+
# Track per-game content-start offsets for verbatim pgn slicing.
|
|
205
|
+
# game_termination belongs to the current game and marks that the NEXT
|
|
206
|
+
# non-discarded token begins a new game.
|
|
207
|
+
def note_token(type, off)
|
|
208
|
+
if type == :game_termination
|
|
209
|
+
@between_games = true
|
|
210
|
+
elsif @between_games
|
|
211
|
+
@game_starts << off
|
|
212
|
+
@between_games = false
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
def advance_line(str)
|
|
217
|
+
@line += str.count("\n")
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
# Raised when the lexer cannot match the input at the current position.
|
|
222
|
+
class UnconsumedInputError < StandardError; end
|
|
223
|
+
end
|
data/lib/pgn/move.rb
CHANGED
|
@@ -95,13 +95,20 @@ module PGN
|
|
|
95
95
|
match = move.match(SAN_REGEX)
|
|
96
96
|
return if match.nil?
|
|
97
97
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
98
|
+
self.piece = match[:piece]
|
|
99
|
+
self.destination = match[:destination]
|
|
100
|
+
self.promotion = match[:promotion]
|
|
101
|
+
self.check = match[:check]
|
|
102
|
+
self.capture = match[:capture]
|
|
103
|
+
self.disambiguation = match[:disambiguation]
|
|
104
|
+
self.castle = match[:castle]
|
|
101
105
|
end
|
|
102
106
|
|
|
103
107
|
def piece=(val)
|
|
104
|
-
|
|
108
|
+
# Castling SAN (O-O / O-O-O) has no piece attribute; the castle attribute
|
|
109
|
+
# is set later in #initialize. Use a non-allocating prefix check instead
|
|
110
|
+
# of `san.match('O-O')`, which allocated a MatchData on every Move.new.
|
|
111
|
+
return if san.start_with?('O')
|
|
105
112
|
|
|
106
113
|
val ||= 'P'
|
|
107
114
|
@piece = black? ? val.downcase : val
|
|
@@ -157,7 +164,7 @@ module PGN
|
|
|
157
164
|
# @return [Boolean] whether the piece being moved is a pawn
|
|
158
165
|
#
|
|
159
166
|
def pawn?
|
|
160
|
-
|
|
167
|
+
piece == 'P' || piece == 'p'
|
|
161
168
|
end
|
|
162
169
|
end
|
|
163
170
|
end
|