featureparity 0.0.16 → 0.0.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -180,17 +180,19 @@ module Fp
180
180
  end
181
181
  end
182
182
 
183
- # The CI gate. Two jobs on purpose: reporting evidence and enforcing the gate are
184
- # separate concerns, and the gate must run on pull requests (where it blocks a
185
- # merge) while evidence reporting only makes sense on trusted pushes.
183
+ # The CI gate and its adaptable reporting recipe are separate concerns: the gate
184
+ # blocks pull requests, while reporting needs a trusted context with FP_API_KEY and
185
+ # should cover both pull requests and default-branch pushes.
186
186
  def workflow_body
187
187
  <<~YAML
188
188
  # FeatureParity — parity enforcement.
189
189
  #
190
- # Generated by `fp init`. Two jobs, deliberately separate:
190
+ # Generated by `fp init`. The gate is ready to use; adapt and uncomment the
191
+ # report recipe once this repository emits JUnit:
191
192
  #
192
- # report On pushes to the default branch, feed test results back to
193
- # FeatureParity so the parity matrix reflects real pass/fail.
193
+ # report On trusted pull requests, report candidate test revisions before
194
+ # merge. On default-branch pushes, record merged pass/fail and promote
195
+ # an approved exact test revision into the accepted baseline.
194
196
  # gate On pull requests, fail the build when the project violates its
195
197
  # enforcement level. Make this a required status check in branch
196
198
  # protection to actually block merges.
@@ -261,30 +263,44 @@ module Fp
261
263
  run: fp check
262
264
 
263
265
  # Uncomment and adapt once your suite emits JUnit XML. `fp ci-report` writes
264
- # real passing/failing evidence — without it, the matrix only ever shows the
265
- # `present` markers agents filed, and the gate has nothing green to see.
266
+ # real passing/failing evidence and one source fingerprint per bound testcase.
267
+ # Run it on trusted pull requests too: `--pr` lets FeatureParity open a changed-test
268
+ # finding before merge without replacing the accepted default-branch baseline.
269
+ # The later default-branch report records merged truth and promotes an approved exact
270
+ # fingerprint. Without reporting, the matrix only has agent-filed `present` markers
271
+ # and changed bound tests cannot be detected.
266
272
  #
267
273
  # It reads the fp:<slug> markers out of the checkout, so run it in the repo
268
274
  # and pass --base-dir when the report's test-file paths are relative to a
269
275
  # subdirectory (vitest, for instance, reports them relative to its package).
276
+ # Keep `if: always()` so red tests are reported. Fork pull requests cannot read
277
+ # FP_API_KEY and are skipped by the HAVE_FP_KEY guard.
270
278
  #
271
279
  # report:
272
280
  # name: Report evidence
273
- # if: github.event_name == 'push'
281
+ # if: github.event_name == 'push' || github.event_name == 'pull_request'
274
282
  # runs-on: ubuntu-latest
283
+ # env:
284
+ # HAVE_FP_KEY: ${{ secrets.FP_API_KEY != '' }}
275
285
  # steps:
276
286
  # - uses: actions/checkout@v4
277
287
  # - name: Run tests (emit JUnit)
278
288
  # run: echo "replace with your test command emitting junit.xml"
279
289
  # - name: Report to FeatureParity
280
- # if: always()
290
+ # if: always() && env.HAVE_FP_KEY == 'true'
281
291
  # env:
282
292
  # FP_API_KEY: ${{ secrets.FP_API_KEY }}
283
293
  # run: |
284
294
  # gem install featureparity
285
- # fp ci-report --junit junit.xml \\
286
- # --surface <surface-key> \\
295
+ # args=(
296
+ # --junit junit.xml
297
+ # --surface <surface-key>
287
298
  # --ci-url "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
299
+ # )
300
+ # if [ "${{ github.event_name }}" = "pull_request" ]; then
301
+ # args+=(--pr "${{ github.event.pull_request.html_url }}")
302
+ # fi
303
+ # fp ci-report "${args[@]}"
288
304
  YAML
289
305
  end
290
306
 
@@ -361,6 +377,30 @@ module Fp
361
377
  `fp propose-change` (or `--removal`) with the source and reason to put that decision before
362
378
  a human. If the intent is unclear, stop and ask rather than making the catalog say both things.
363
379
 
380
+ **Treat existing tests as protected product evidence.** Before editing an existing test,
381
+ determine what expectation it already records and whether CI has passing evidence for it.
382
+
383
+ - If the test backs passing evidence, do not rewrite it to make the current implementation
384
+ or a new requirement fit. Explicitly flag the intended test edit in the work item and pull
385
+ request. If its expected behaviour would change, propose the requirement change and wait
386
+ for explicit human approval before editing the test; a newly green run must not hide that
387
+ the proof itself changed.
388
+ - When FeatureParity reports a changed bound test, run `fp changed-tests` (or add `--json`)
389
+ and compare the accepted and candidate source revisions with the requirement. Record the
390
+ exact candidate through `fp changed-tests assess <id> --candidate-fingerprint <sha256>`
391
+ with a plain-language explanation and one classification: `still_proves`,
392
+ `no_longer_proves`, or `uncertain`. The first resolves only that fingerprint; the others
393
+ remain open for a human product decision. A later fingerprint needs a new assessment.
394
+ - A new requirement does not retroactively own or supersede a pre-existing failing test —
395
+ whether the requirement is draft or active. Do not change that test, or adopt it as the new
396
+ requirement's proof, merely to make the new work pass. Surface the conflict and wait for
397
+ explicit human approval to change the existing expectation.
398
+
399
+ A changed-test finding is an audit signal alongside the testcase's independent CI outcome.
400
+ It is not a failed test and is separate from `suspect`, which means requirement wording
401
+ changed and its evidence needs re-verification. Neither permits rewriting a test before
402
+ product intent is decided.
403
+
364
404
  If no requirement covers what you're building, propose one or a parent/child tree. Agents
365
405
  always create **drafts**; a human activates them:
366
406
 
@@ -55,7 +55,11 @@ module Fp
55
55
 
56
56
  result = client.propose_requirement_change(requirement['id'], params)
57
57
  unless result[:ok]
58
- output.error(result[:error], status: result[:status])
58
+ if result[:status] == 409 && result[:details].is_a?(Hash)
59
+ print_conflict(result, slug: slug)
60
+ else
61
+ output.error(result[:error], status: result[:status])
62
+ end
59
63
  exit 1
60
64
  end
61
65
 
@@ -64,6 +68,47 @@ module Fp
64
68
 
65
69
  private
66
70
 
71
+ # A 409 means the requirement already has an open proposal — the one-at-a-time rule
72
+ # (two "what it was" snapshots would make a rollback ambiguous). Rather than a dead
73
+ # end, show what the open proposal is and point at `fp comment` so the agent can
74
+ # raise its concern on the proposal a human is about to review.
75
+ def print_conflict(result, slug:)
76
+ pending = result[:details]['pending_change'] || {}
77
+
78
+ if output.json?
79
+ output.error(result[:error], status: result[:status])
80
+ return
81
+ end
82
+
83
+ warn "Error: #{result[:error]}"
84
+ warn ''
85
+ warn 'The open proposal:'
86
+ warn " Change: #{pending['id']}"
87
+ warn " Kind: #{pending['kind']}"
88
+ warn " Proposed: #{pending['actor']} (#{pending['actor_kind']})"
89
+ warn " Reason: #{pending['reason']}" if pending['reason']
90
+ print_conflict_source(pending['source'])
91
+ print_conflict_diff(pending['changes'])
92
+ warn ''
93
+ warn 'You cannot open a rival proposal, but you can comment on this one:'
94
+ warn " fp comment #{slug} --change #{pending['id']} --body \"...\" --project <project>"
95
+ warn 'Or wait for a human to review it. Run `fp history ' \
96
+ "#{slug} --project <project>` for the full diff."
97
+ end
98
+
99
+ def print_conflict_source(source)
100
+ return unless source.is_a?(Hash) && !source.empty?
101
+
102
+ warn " Source: #{source['type']}" if source['type']
103
+ warn " URL: #{source['url']}" if source['url']
104
+ end
105
+
106
+ def print_conflict_diff(changes)
107
+ return unless changes.is_a?(Hash) && !changes.empty?
108
+
109
+ warn ' Fields: ' + changes.keys.join(', ')
110
+ end
111
+
67
112
  def validate_enums!(opts)
68
113
  if opts[:category] && !VALID_CATEGORIES.include?(opts[:category])
69
114
  output.error("Invalid category '#{opts[:category]}'. Must be one of: #{VALID_CATEGORIES.join(', ')}")
@@ -115,9 +115,26 @@ module Fp
115
115
  exit 1
116
116
  end
117
117
 
118
+ # The server echoes back the stored record, whose status/source may differ
119
+ # from what we asked for: CI evidence is authoritative, so an agent's
120
+ # present/stub never downgrades a cell CI already verified. Report the truth
121
+ # the server kept, not the state we requested, so the caller knows whether
122
+ # their report actually landed or was superseded by CI.
123
+ stored = result[:data].is_a?(Hash) ? (result[:data]['evidence'] || {}) : {}
124
+ stored_status = stored['status']
125
+ stored_source = stored['source']
126
+ superseded = ci_superseded?(state, stored_status, stored_source)
127
+
118
128
  output.success(result[:data]) do
119
- puts "Evidence reported for '#{slug}' on surface '#{opts[:surface]}'."
120
- puts " State: #{state}"
129
+ if superseded
130
+ puts "CI already verified '#{slug}' on surface '#{opts[:surface]}' (status: #{stored_status})."
131
+ puts "Your '#{state}' report was recorded as metadata but did NOT change the status —"
132
+ puts 'CI evidence is authoritative and is never downgraded by an agent report.'
133
+ puts ' Stored: ' + "#{stored_status} (source: #{stored_source})"
134
+ else
135
+ puts "Evidence reported for '#{slug}' on surface '#{opts[:surface]}'."
136
+ puts " State: #{stored_status || state}"
137
+ end
121
138
  puts " Type: #{opts[:test_type]}" if opts[:test_type]
122
139
  puts " File: #{opts[:file]}#{opts[:line] ? ":#{opts[:line]}" : ''}"
123
140
  if opts[:pr]
@@ -140,6 +157,20 @@ module Fp
140
157
 
141
158
  private
142
159
 
160
+ # CI statuses an agent can never set. Mirrors Evidence::CI_STATUSES on the
161
+ # server — kept here so the CLI can recognise, from the echoed record, when its
162
+ # present/stub report was superseded by an authoritative CI verdict.
163
+ CI_STATUSES = %w[passing pr_passing failing].freeze
164
+
165
+ # True when the server kept a CI verdict instead of applying our report: the
166
+ # stored evidence is CI-sourced with a CI status, yet we asked for present/stub.
167
+ def ci_superseded?(requested_state, stored_status, stored_source)
168
+ return false if stored_status.nil?
169
+ return false unless VALID_STATES.include?(requested_state)
170
+
171
+ stored_source == 'ci' && CI_STATUSES.include?(stored_status) && stored_status != requested_state
172
+ end
173
+
143
174
  # Batch-upload evidence from a JUnit XML report.
144
175
  def run_junit(opts)
145
176
  require_project!(opts)
@@ -244,16 +275,34 @@ module Fp
244
275
  exit 1
245
276
  end
246
277
 
247
- reported << { binding: binding, state: state, recorded: true }
278
+ reported << { binding: binding, state: state, recorded: true,
279
+ stored_status: stored_status_from(report_result),
280
+ stored_source: stored_source_from(report_result) }
248
281
  end
249
282
 
250
283
  emit_junit_summary(opts, reported, skipped_unknown, unmatched, api_unavailable)
251
284
  end
252
285
 
286
+ # Pull the stored status/source out of the server's echoed evidence record so the
287
+ # batch summary can flag cells where CI superseded the agent's present/stub.
288
+ def stored_status_from(report_result)
289
+ data = report_result[:data]
290
+ return nil unless data.is_a?(Hash)
291
+
292
+ (data['evidence'] || {})['status']
293
+ end
294
+
295
+ def stored_source_from(report_result)
296
+ data = report_result[:data]
297
+ return nil unless data.is_a?(Hash)
298
+
299
+ (data['evidence'] || {})['source']
300
+ end
301
+
253
302
  def emit_junit_summary(opts, reported, skipped_unknown, unmatched, api_unavailable)
254
303
  data = {
255
304
  repo: opts[:repo],
256
- reported: reported.map { |r| { slug: r[:binding].slug, surface: r[:binding].surface, test_type: r[:binding].test_type, file: r[:binding].file, line: r[:binding].line, state: r[:state], recorded: r[:recorded] } },
305
+ reported: reported.map { |r| { slug: r[:binding].slug, surface: r[:binding].surface, test_type: r[:binding].test_type, file: r[:binding].file, line: r[:binding].line, state: r[:state], recorded: r[:recorded], stored_status: r[:stored_status], superseded_by_ci: ci_superseded?(r[:state], r[:stored_status], r[:stored_source]) } },
257
306
  skipped_unknown: skipped_unknown.map { |b| { slug: b.slug, surface: b.surface, file: b.file } },
258
307
  unmatched_testcases: unmatched.map { |tc| { name: tc.name, file: tc.file } }
259
308
  }
@@ -261,10 +310,18 @@ module Fp
261
310
  output.success(data) do
262
311
  puts "Reported evidence from #{opts[:junit]}:"
263
312
  reported.each do |r|
264
- marker = r[:recorded] ? '✓' : '•'
313
+ superseded = ci_superseded?(r[:state], r[:stored_status], r[:stored_source])
314
+ marker = if !r[:recorded]
315
+ '•'
316
+ elsif superseded
317
+ '⊘'
318
+ else
319
+ '✓'
320
+ end
265
321
  loc = r[:binding].line ? "#{r[:binding].file}:#{r[:binding].line}" : r[:binding].file
266
322
  type = r[:binding].test_type ? " [#{r[:binding].test_type}]" : ''
267
- puts " #{marker} #{r[:binding].slug} → #{r[:binding].surface}#{type} (#{r[:state]}) #{loc}"
323
+ shown_state = superseded ? "#{r[:stored_status]}, kept CI — #{r[:state]} ignored" : r[:state]
324
+ puts " #{marker} #{r[:binding].slug} → #{r[:binding].surface}#{type} (#{shown_state}) #{loc}"
268
325
  end
269
326
 
270
327
  unless skipped_unknown.empty?
data/lib/fp/commands.rb CHANGED
@@ -7,10 +7,12 @@ require_relative 'commands/list'
7
7
  require_relative 'commands/show'
8
8
  require_relative 'commands/propose'
9
9
  require_relative 'commands/propose_change'
10
+ require_relative 'commands/comment'
10
11
  require_relative 'commands/history'
11
12
  require_relative 'commands/changes'
12
13
  require_relative 'commands/report'
13
14
  require_relative 'commands/ci_report'
15
+ require_relative 'commands/changed_tests'
14
16
  require_relative 'commands/check'
15
17
  require_relative 'commands/init'
16
18
  require_relative 'commands/matrix'
data/lib/fp/junit.rb CHANGED
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'rexml/document'
4
+ require_relative 'source_fingerprint'
4
5
 
5
6
  module Fp
6
7
  # Parses JUnit XML reports and binds testcases to fp:<slug> markers found in
@@ -40,6 +41,15 @@ module Fp
40
41
  CiResult = Struct.new(:slug, :surface, :test_type, :status, :file, :line, :title, :test_count,
41
42
  :failure_count, keyword_init: true)
42
43
 
44
+ # One exact testcase source receipt. Unlike CiResult, these are never aggregated:
45
+ # two tests may back the same requirement cell and each can change independently.
46
+ TestRevision = Struct.new(
47
+ :slug, :surface, :test_type, :testcase_key, :testcase_name, :classname, :outcome,
48
+ :repo, :file, :marker_line, :start_line, :end_line, :commit_sha,
49
+ :fingerprint, :fingerprint_scope,
50
+ keyword_init: true
51
+ )
52
+
43
53
  # A failing testcase that carries no fp:<slug> marker at all — a "loose end". Not a
44
54
  # cell on the parity grid (nothing binds it to a requirement) and not an orphan
45
55
  # marker (there is no marker); it's a red test nobody is tracking against a
@@ -58,7 +68,7 @@ module Fp
58
68
  # directory the runner was invoked from (vitest reports `src/pages/Foo.test.jsx` from
59
69
  # inside `frontend/`), so reporting it verbatim produces a link to a file that doesn't
60
70
  # exist at that path in the repo.
61
- Match = Struct.new(:testcase, :marker, :precise, :repo_file, keyword_init: true)
71
+ Match = Struct.new(:testcase, :marker, :precise, :repo_file, :source_path, keyword_init: true)
62
72
 
63
73
  # Evidence discovered by binding a testcase to a marker.
64
74
  # `line` is the 1-based line number of the test in its source file: the
@@ -234,9 +244,13 @@ module Fp
234
244
  # A slug is `failing` if any test claiming it went red, `stub` if every test claiming
235
245
  # it was skipped (a skipped test is not proof of anything), and `passing` otherwise.
236
246
  #
237
- # Returns { results: [CiResult, ...], unmatched: [TestCase], missing_surface: [...] }.
238
- def ci_results(testcases, base_dir: Dir.pwd, default_surface: nil, default_test_type: nil)
247
+ # Returns { results: [CiResult, ...], test_revisions: [TestRevision, ...],
248
+ # unmatched: [TestCase], missing_surface: [...] }.
249
+ def ci_results(testcases, base_dir: Dir.pwd, default_surface: nil, default_test_type: nil,
250
+ commit_sha: nil, repo: nil)
239
251
  aggregates = {}
252
+ test_revisions = []
253
+ fingerprint_cache = {}
240
254
  matched = {}
241
255
  missing_surface = []
242
256
 
@@ -251,6 +265,8 @@ module Fp
251
265
  # The marker's own type pin wins; otherwise the --test-type default (nil → the
252
266
  # server files it as `unspecified`). One CI verdict per (slug, surface, type).
253
267
  test_type = marker[:test_type] || default_test_type
268
+ fingerprint = fingerprint_cache[[match.source_path, marker[:line]]] ||=
269
+ SourceFingerprint.build(match.source_path, marker)
254
270
 
255
271
  surfaces.each do |surface|
256
272
  if surface.nil? || surface.empty?
@@ -276,6 +292,18 @@ module Fp
276
292
  agg[:title] ||= title
277
293
  agg[:failing_title] ||= title if tc.failed?
278
294
  end
295
+
296
+ if fingerprint && commit_sha
297
+ outcome = if tc.stub? then 'stub' elsif tc.failed? then 'failing' else 'passing' end
298
+ test_revisions << TestRevision.new(
299
+ slug: marker[:slug], surface: surface, test_type: test_type,
300
+ testcase_key: testcase_key(match.repo_file, tc), testcase_name: tc.name,
301
+ classname: tc.classname, outcome: outcome, repo: repo, file: match.repo_file,
302
+ marker_line: marker[:line], start_line: fingerprint.start_line,
303
+ end_line: fingerprint.end_line, commit_sha: commit_sha,
304
+ fingerprint: fingerprint.fingerprint, fingerprint_scope: fingerprint.scope
305
+ )
306
+ end
279
307
  end
280
308
  end
281
309
 
@@ -293,7 +321,12 @@ module Fp
293
321
  end
294
322
 
295
323
  unmatched = testcases.reject { |tc| matched[tc.object_id] }
296
- { results: results, unmatched: unmatched, missing_surface: missing_surface.uniq }
324
+ {
325
+ results: results,
326
+ test_revisions: test_revisions,
327
+ unmatched: unmatched,
328
+ missing_surface: missing_surface.uniq
329
+ }
297
330
  end
298
331
 
299
332
  # The loose ends in a set of unmatched testcases: the ones that *failed* but carry no
@@ -326,11 +359,23 @@ module Fp
326
359
  marker, precise = marker_for(tc, cases, markers)
327
360
  next unless marker
328
361
 
329
- yield Match.new(testcase: tc, marker: marker, precise: precise, repo_file: repo_file)
362
+ yield Match.new(
363
+ testcase: tc, marker: marker, precise: precise, repo_file: repo_file,
364
+ source_path: File.expand_path(file, base_dir)
365
+ )
330
366
  end
331
367
  end
332
368
  end
333
369
 
370
+ # Stable across body edits and line movement. A rename or move is intentionally a
371
+ # new identity; marker syntax has no durable per-test id from which to infer that the
372
+ # renamed testcase is the old one.
373
+ def testcase_key(repo_file, testcase)
374
+ Digest::SHA256.hexdigest(
375
+ [repo_file, testcase.classname, testcase.name].map(&:to_s).join("\0")
376
+ )
377
+ end
378
+
334
379
  # The repository root containing `dir`, or nil when `dir` isn't inside one. #
335
380
  # Found by walking up for a `.git` entry rather than shelling out to git, so it also
336
381
  # works in a worktree (where `.git` is a file) and costs nothing.
@@ -0,0 +1,193 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'digest'
4
+
5
+ module Fp
6
+ # Produces a receipt for the exact test source CI executed. Isolation is deliberately
7
+ # conservative: when a complete test body cannot be identified confidently, the whole
8
+ # file is hashed and the scope says so instead of pretending a noisy file hash is an
9
+ # isolated test receipt.
10
+ module SourceFingerprint
11
+ Result = Struct.new(:fingerprint, :scope, :start_line, :end_line, keyword_init: true)
12
+
13
+ BRACE_EXTENSIONS = %w[.js .jsx .mjs .cjs .ts .tsx .kt .kts .swift .java .go .cs .php .rs .scala].freeze
14
+
15
+ module_function
16
+
17
+ def build(path, marker)
18
+ contents = File.binread(path)
19
+ lines = contents.lines
20
+ range = isolated_range(lines, marker, File.extname(path).downcase)
21
+
22
+ if range
23
+ first, last = range
24
+ source = lines[first..last].join
25
+ Result.new(
26
+ fingerprint: Digest::SHA256.hexdigest(source),
27
+ scope: 'isolated_test',
28
+ start_line: first + 1,
29
+ end_line: last + 1
30
+ )
31
+ else
32
+ Result.new(
33
+ fingerprint: Digest::SHA256.hexdigest(contents),
34
+ scope: 'whole_file',
35
+ start_line: 1,
36
+ end_line: lines.length
37
+ )
38
+ end
39
+ rescue Errno::ENOENT, Errno::EACCES
40
+ nil
41
+ end
42
+
43
+ def isolated_range(lines, marker, extension)
44
+ return nil unless marker[:title_kind] == :test
45
+
46
+ start_index = declaration_index(lines, marker)
47
+ return nil if start_index.nil? || start_index.negative? || start_index >= lines.length
48
+
49
+ case extension
50
+ when '.rb'
51
+ ruby_range(lines, start_index)
52
+ when '.py'
53
+ python_range(lines, start_index)
54
+ when *BRACE_EXTENSIONS
55
+ brace_range(lines, start_index)
56
+ end
57
+ end
58
+
59
+ def declaration_index(lines, marker)
60
+ marker_index = marker[:line].to_i - 1
61
+ ((marker_index + 1)...[marker_index + 41, lines.length].min).each do |index|
62
+ line = lines[index]
63
+ stripped = line.strip
64
+ next if stripped.empty? || stripped.match?(%r{\A(?:#|//|--|/\*|\*)})
65
+
66
+ return index
67
+ end
68
+ nil
69
+ end
70
+
71
+ def ruby_range(lines, start_index)
72
+ declaration = strip_ruby_literals(lines[start_index])
73
+ return nil if declaration.nil?
74
+ return nil unless declaration.match?(/\bdef\s+test_?\w+/i) ||
75
+ declaration.match?(/\b(?:it|specify|scenario)\b.*\bdo\b/)
76
+
77
+ depth = 0
78
+ (start_index...lines.length).each do |index|
79
+ code = strip_ruby_literals(lines[index])
80
+ return nil if code.nil?
81
+
82
+ depth += code.scan(/\bdo\b/).length
83
+ depth += 1 if code.match?(/^\s*(?:def|class|module|if|unless|case|begin|while|until|for)\b/)
84
+ depth -= code.scan(/\bend\b/).length
85
+ return [start_index, index] if depth.zero? && index >= start_index
86
+ return nil if depth.negative?
87
+ end
88
+ nil
89
+ end
90
+
91
+ def strip_ruby_literals(line)
92
+ return nil if line.match?(/<<[-~]?['"`]?\w+/)
93
+
94
+ line.gsub(/'(?:\\.|[^'\\])*'|"(?:\\.|[^"\\])*"/, '').sub(/#.*/, '')
95
+ end
96
+
97
+ def python_range(lines, start_index)
98
+ declaration = lines[start_index]
99
+ return nil unless declaration.match?(/^\s*(?:async\s+)?def\s+test_?\w+.*:\s*(?:#.*)?$/i)
100
+
101
+ indent = declaration[/\A\s*/].length
102
+ last_body = nil
103
+ ((start_index + 1)...lines.length).each do |index|
104
+ line = lines[index]
105
+ next if line.strip.empty? || line.lstrip.start_with?('#')
106
+
107
+ current_indent = line[/\A\s*/].length
108
+ break if current_indent <= indent
109
+
110
+ last_body = index
111
+ end
112
+ last_body ? [start_index, last_body] : nil
113
+ end
114
+
115
+ def brace_range(lines, start_index)
116
+ depth = 0
117
+ opened = false
118
+ in_block_comment = false
119
+
120
+ (start_index...lines.length).each do |index|
121
+ sanitized, in_block_comment = strip_brace_literals(lines[index], in_block_comment)
122
+ return nil if sanitized.nil?
123
+
124
+ sanitized.each_char do |char|
125
+ if char == '{'
126
+ depth += 1
127
+ opened = true
128
+ elsif char == '}' && opened
129
+ depth -= 1
130
+ return nil if depth.negative?
131
+ end
132
+ end
133
+
134
+ return [start_index, index] if opened && depth.zero?
135
+ return nil if !opened && index - start_index >= 10
136
+ end
137
+ nil
138
+ end
139
+
140
+ # Remove comments and quoted strings before balancing braces. Template literals are
141
+ # intentionally unsupported because interpolation can contain real code braces; those
142
+ # files use the labelled whole-file fallback rather than risk a false boundary.
143
+ def strip_brace_literals(line, in_block_comment)
144
+ return [nil, in_block_comment] if line.include?('`')
145
+
146
+ output = +''
147
+ quote = nil
148
+ escaped = false
149
+ index = 0
150
+ while index < line.length
151
+ char = line[index]
152
+ pair = line[index, 2]
153
+
154
+ if in_block_comment
155
+ if pair == '*/'
156
+ in_block_comment = false
157
+ index += 2
158
+ else
159
+ index += 1
160
+ end
161
+ next
162
+ end
163
+
164
+ if quote
165
+ if escaped
166
+ escaped = false
167
+ elsif char == '\\'
168
+ escaped = true
169
+ elsif char == quote
170
+ quote = nil
171
+ end
172
+ index += 1
173
+ next
174
+ end
175
+
176
+ break if pair == '//'
177
+ if pair == '/*'
178
+ in_block_comment = true
179
+ index += 2
180
+ next
181
+ end
182
+ if char == '"' || char == "'"
183
+ quote = char
184
+ else
185
+ output << char
186
+ end
187
+ index += 1
188
+ end
189
+
190
+ [output, in_block_comment]
191
+ end
192
+ end
193
+ end
data/lib/fp/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Fp
4
- VERSION = '0.0.16'
4
+ VERSION = '0.0.17'
5
5
  end
data/lib/fp.rb CHANGED
@@ -5,6 +5,7 @@ require_relative 'fp/config'
5
5
  require_relative 'fp/project_config'
6
6
  require_relative 'fp/parity'
7
7
  require_relative 'fp/client'
8
+ require_relative 'fp/source_fingerprint'
8
9
  require_relative 'fp/junit'
9
10
  require_relative 'fp/output'
10
11
  require_relative 'fp/commands'
data/skills/fp/SKILL.md CHANGED
@@ -86,6 +86,27 @@ requirement around it. Identify which promise matches product intent, then use
86
86
  `fp propose-change` (or `--removal`) with the source and reason to put that decision before a
87
87
  human. If the intent is unclear, stop and ask rather than making the catalog say both things.
88
88
 
89
+ **Treat existing tests as protected product evidence.** Before editing an existing test,
90
+ determine what expectation it already records and whether CI has passing evidence for it.
91
+
92
+ - If the test backs passing evidence, do not rewrite it to make the current implementation or a
93
+ new requirement fit. Explicitly flag the intended test edit in the work item and pull request.
94
+ If its expected behaviour would change, propose the requirement change and wait for explicit
95
+ human approval before editing the test; a newly green run must not hide that the proof changed.
96
+ - When FeatureParity reports a changed bound test, run `fp changed-tests` (or add `--json`) and
97
+ compare its accepted and candidate revisions with the requirement. Record a plain-language
98
+ assessment for the exact candidate fingerprint with `fp changed-tests assess`: `still_proves`
99
+ resolves only that fingerprint; `no_longer_proves` and `uncertain` remain open for a human
100
+ product decision. A later fingerprint always needs a new assessment.
101
+ - A new requirement does not retroactively own or supersede a pre-existing failing test, whether
102
+ it is draft or active. Do not change that test, or adopt it as the new requirement's proof,
103
+ merely to make new work pass. Surface the conflict and wait for explicit human approval to
104
+ change the existing expectation.
105
+
106
+ A changed-test finding is an audit signal alongside the independent CI outcome, not a failed
107
+ test. It is also separate from `suspect`, which means requirement wording changed and its
108
+ evidence needs re-verification. Neither permits rewriting a test before product intent is decided.
109
+
89
110
  ### 2. Add the fp:<slug> marker to tests
90
111
 
91
112
  Place a comment immediately before the test to bind it to a requirement:
@@ -118,6 +139,16 @@ fp report <slug> \
118
139
 
119
140
  **Agents report `present` (default) or `stub` only. Never `passing` or `failing`.**
120
141
 
142
+ **Don't re-report a cell CI already covers.** CI evidence (`passing`, `pr_passing`,
143
+ `failing`) is authoritative. If CI has already run the test for a surface, a `present`
144
+ report adds nothing — it won't upgrade the cell, and the server will keep the CI verdict
145
+ rather than downgrade it to a bare marker. Check first with `fp show <slug> --project X`
146
+ (or `fp matrix`): if the surface already shows a CI status, skip the report. Report
147
+ `present`/`stub` for surfaces CI hasn't reached yet — e.g. a brand-new test before its
148
+ first CI run, or local work where CI isn't wired up. When you do report over a CI-covered
149
+ cell anyway, `fp report` now tells you it was superseded (the status stays CI's, your
150
+ report lands as metadata only) so you can see the no-op instead of assuming it took.
151
+
121
152
  ### 4. Report evidence from JUnit XML (batch upload without CI)
122
153
 
123
154
  When CI isn't available, run the suite locally and upload the report: