@codyswann/lisa 4.4.15 → 4.4.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/lisa-owned-hash-ledger.d.ts.map +1 -1
- package/dist/core/lisa-owned-hash-ledger.js +2 -0
- package/dist/core/lisa-owned-hash-ledger.js.map +1 -1
- package/dist/core/upstream-evidence-manifest.d.ts.map +1 -1
- package/dist/core/upstream-evidence-manifest.js +18 -14
- package/dist/core/upstream-evidence-manifest.js.map +1 -1
- package/package.json +2 -1
- package/plugins/lisa/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/lisa/.codex-plugin/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/lisa/hooks/threshold-ratchet-compare.mjs +7 -0
- package/plugins/lisa/hooks/threshold-ratchet-families.mjs +53 -0
- package/plugins/lisa/rules/eager/falsifiable-checks.md +16 -1
- package/plugins/lisa/rules/reference/falsifiable-checks.md +74 -1
- package/plugins/lisa/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/lisa/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/lisa-agy/plugin.json +1 -1
- package/plugins/lisa-agy/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-agy/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/lisa-agy/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/lisa-cdk/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-agy/plugin.json +1 -1
- package/plugins/lisa-cdk-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cdk-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-copilot/hooks/threshold-ratchet-compare.mjs +7 -0
- package/plugins/lisa-copilot/hooks/threshold-ratchet-families.mjs +53 -0
- package/plugins/lisa-copilot/rules/eager/falsifiable-checks.md +16 -1
- package/plugins/lisa-copilot/rules/reference/falsifiable-checks.md +74 -1
- package/plugins/lisa-copilot/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-copilot/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/lisa-copilot/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/lisa-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-cursor/hooks/threshold-ratchet-compare.mjs +7 -0
- package/plugins/lisa-cursor/hooks/threshold-ratchet-families.mjs +53 -0
- package/plugins/lisa-cursor/rules/falsifiable-checks-reference.mdc +74 -1
- package/plugins/lisa-cursor/rules/falsifiable-checks.mdc +16 -1
- package/plugins/lisa-cursor/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-cursor/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/lisa-cursor/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/lisa-expo/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-agy/plugin.json +1 -1
- package/plugins/lisa-expo-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-expo-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-agy/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-harper-fabric-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-agy/plugin.json +1 -1
- package/plugins/lisa-nestjs-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-nestjs-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-agy/plugin.json +1 -1
- package/plugins/lisa-openclaw-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-openclaw-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-agy/plugin.json +1 -1
- package/plugins/lisa-phaser-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-phaser-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-rails/.codex-plugin/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/lisa-rails/.codex-plugin/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-rails/.codex-plugin/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/plugins/lisa-rails/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/lisa-rails/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-rails/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/plugins/lisa-rails-agy/plugin.json +1 -1
- package/plugins/lisa-rails-agy/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/lisa-rails-agy/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-rails-agy/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/plugins/lisa-rails-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-copilot/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/lisa-rails-copilot/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-rails-copilot/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/plugins/lisa-rails-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-rails-cursor/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/lisa-rails-cursor/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/lisa-rails-cursor/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/plugins/lisa-typescript/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-agy/plugin.json +1 -1
- package/plugins/lisa-typescript-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-typescript-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki/.codex-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-agy/plugin.json +1 -1
- package/plugins/lisa-wiki-copilot/.claude-plugin/plugin.json +1 -1
- package/plugins/lisa-wiki-cursor/.claude-plugin/plugin.json +1 -1
- package/plugins/src/base/hooks/threshold-ratchet-compare.mjs +7 -0
- package/plugins/src/base/hooks/threshold-ratchet-families.mjs +53 -0
- package/plugins/src/base/rules/eager/falsifiable-checks.md +16 -1
- package/plugins/src/base/rules/reference/falsifiable-checks.md +74 -1
- package/plugins/src/base/skills/lisa-improve-test-coverage/SKILL.md +4 -1
- package/plugins/src/base/skills/lisa-improve-tests/SKILL.md +4 -1
- package/plugins/src/base/skills/lisa-nightly-lower-code-complexity/SKILL.md +10 -2
- package/plugins/src/rails/skills/improve-code-complexity/SKILL.md +4 -1
- package/plugins/src/rails/skills/improve-test-coverage/SKILL.md +4 -1
- package/plugins/src/rails/skills/ops-verify-telemetry/SKILL.md +30 -1
- package/rails/copy-overwrite/scripts/threshold-ratchet-compare.mjs +7 -0
- package/rails/copy-overwrite/scripts/threshold-ratchet-families.mjs +53 -0
- package/scripts/check-pipeline-status-reads.mjs +684 -0
- package/typescript/copy-overwrite/scripts/threshold-ratchet-compare.mjs +7 -0
- package/typescript/copy-overwrite/scripts/threshold-ratchet-families.mjs +53 -0
|
@@ -17,8 +17,11 @@ Reduces the CyclomaticComplexity threshold by 2 and fixes all violations.
|
|
|
17
17
|
bundle exec rubocop --only Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity --format json 2>&1
|
|
18
18
|
```
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
status=0
|
|
21
|
+
bundle exec flog --all --group app/ >flog.log 2>&1 || status=$?
|
|
22
|
+
head -n 50 flog.log; echo "exit=$status"
|
|
21
23
|
```
|
|
24
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `head` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
22
25
|
4. **Note for each violation**:
|
|
23
26
|
- File path and line number
|
|
24
27
|
- Method name
|
|
@@ -16,8 +16,11 @@ If no argument provided, prompt the user for a target.
|
|
|
16
16
|
1. **Find coverage config** (`.simplecov` or `spec/spec_helper.rb`)
|
|
17
17
|
2. **Run test suite with coverage** to get current state:
|
|
18
18
|
```bash
|
|
19
|
-
|
|
19
|
+
status=0
|
|
20
|
+
bundle exec rspec >rspec.log 2>&1 || status=$?
|
|
21
|
+
tail -n 50 rspec.log; echo "exit=$status"
|
|
20
22
|
```
|
|
23
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `tail` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
21
24
|
3. **Check SimpleCov output** in `coverage/index.html` or console output
|
|
22
25
|
4. **Identify the 20 files with lowest coverage**, noting:
|
|
23
26
|
- File path
|
|
@@ -58,7 +58,36 @@ grep -i "otel\|opentelemetry\|trace_id" log/development.log | tail -10
|
|
|
58
58
|
### Check Collector Sidecar (if using Docker Compose)
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
|
|
61
|
+
# Capture the status BEFORE the pipe. A pipeline reports its LAST stage's exit
|
|
62
|
+
# code, so `docker compose logs ... | tail -20 || echo "No otel-collector"`
|
|
63
|
+
# never prints the fallback: `tail` succeeds even when there is no such
|
|
64
|
+
# service, and the absent sidecar reads as a silent pass.
|
|
65
|
+
#
|
|
66
|
+
# `|| status=$?`, not `; status=$?`: under `set -e` the `;` form exits before
|
|
67
|
+
# the assignment, so the branch that reports the failure never runs.
|
|
68
|
+
#
|
|
69
|
+
# Ask whether the service EXISTS separately from whether its logs could be
|
|
70
|
+
# read. `docker compose logs` also exits non-zero for a stopped daemon, a
|
|
71
|
+
# permission failure, and an unparseable compose file; answering all of those
|
|
72
|
+
# with "No otel-collector service" turns a broken telemetry stack into a
|
|
73
|
+
# reassuring sentence — the same silent-measurement failure one level up.
|
|
74
|
+
services_status=0
|
|
75
|
+
services=$(docker compose config --services 2>&1) || services_status=$?
|
|
76
|
+
if [ "$services_status" -ne 0 ]; then
|
|
77
|
+
echo "FAILED ($services_status) — docker compose config could not be read:"
|
|
78
|
+
printf '%s\n' "$services"
|
|
79
|
+
elif ! printf '%s\n' "$services" | grep -qx otel-collector; then
|
|
80
|
+
echo "No otel-collector service in Docker Compose"
|
|
81
|
+
else
|
|
82
|
+
logs_status=0
|
|
83
|
+
docker compose logs otel-collector >otel.log 2>&1 || logs_status=$?
|
|
84
|
+
if [ "$logs_status" -eq 0 ]; then
|
|
85
|
+
tail -n 20 otel.log
|
|
86
|
+
else
|
|
87
|
+
echo "FAILED ($logs_status) — otel-collector is configured but its logs could not be read:"
|
|
88
|
+
cat otel.log
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
62
91
|
```
|
|
63
92
|
|
|
64
93
|
## Remote Verification (AWS X-Ray)
|
|
@@ -17,8 +17,11 @@ Reduces the CyclomaticComplexity threshold by 2 and fixes all violations.
|
|
|
17
17
|
bundle exec rubocop --only Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity --format json 2>&1
|
|
18
18
|
```
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
status=0
|
|
21
|
+
bundle exec flog --all --group app/ >flog.log 2>&1 || status=$?
|
|
22
|
+
head -n 50 flog.log; echo "exit=$status"
|
|
21
23
|
```
|
|
24
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `head` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
22
25
|
4. **Note for each violation**:
|
|
23
26
|
- File path and line number
|
|
24
27
|
- Method name
|
|
@@ -16,8 +16,11 @@ If no argument provided, prompt the user for a target.
|
|
|
16
16
|
1. **Find coverage config** (`.simplecov` or `spec/spec_helper.rb`)
|
|
17
17
|
2. **Run test suite with coverage** to get current state:
|
|
18
18
|
```bash
|
|
19
|
-
|
|
19
|
+
status=0
|
|
20
|
+
bundle exec rspec >rspec.log 2>&1 || status=$?
|
|
21
|
+
tail -n 50 rspec.log; echo "exit=$status"
|
|
20
22
|
```
|
|
23
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `tail` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
21
24
|
3. **Check SimpleCov output** in `coverage/index.html` or console output
|
|
22
25
|
4. **Identify the 20 files with lowest coverage**, noting:
|
|
23
26
|
- File path
|
|
@@ -58,7 +58,36 @@ grep -i "otel\|opentelemetry\|trace_id" log/development.log | tail -10
|
|
|
58
58
|
### Check Collector Sidecar (if using Docker Compose)
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
|
|
61
|
+
# Capture the status BEFORE the pipe. A pipeline reports its LAST stage's exit
|
|
62
|
+
# code, so `docker compose logs ... | tail -20 || echo "No otel-collector"`
|
|
63
|
+
# never prints the fallback: `tail` succeeds even when there is no such
|
|
64
|
+
# service, and the absent sidecar reads as a silent pass.
|
|
65
|
+
#
|
|
66
|
+
# `|| status=$?`, not `; status=$?`: under `set -e` the `;` form exits before
|
|
67
|
+
# the assignment, so the branch that reports the failure never runs.
|
|
68
|
+
#
|
|
69
|
+
# Ask whether the service EXISTS separately from whether its logs could be
|
|
70
|
+
# read. `docker compose logs` also exits non-zero for a stopped daemon, a
|
|
71
|
+
# permission failure, and an unparseable compose file; answering all of those
|
|
72
|
+
# with "No otel-collector service" turns a broken telemetry stack into a
|
|
73
|
+
# reassuring sentence — the same silent-measurement failure one level up.
|
|
74
|
+
services_status=0
|
|
75
|
+
services=$(docker compose config --services 2>&1) || services_status=$?
|
|
76
|
+
if [ "$services_status" -ne 0 ]; then
|
|
77
|
+
echo "FAILED ($services_status) — docker compose config could not be read:"
|
|
78
|
+
printf '%s\n' "$services"
|
|
79
|
+
elif ! printf '%s\n' "$services" | grep -qx otel-collector; then
|
|
80
|
+
echo "No otel-collector service in Docker Compose"
|
|
81
|
+
else
|
|
82
|
+
logs_status=0
|
|
83
|
+
docker compose logs otel-collector >otel.log 2>&1 || logs_status=$?
|
|
84
|
+
if [ "$logs_status" -eq 0 ]; then
|
|
85
|
+
tail -n 20 otel.log
|
|
86
|
+
else
|
|
87
|
+
echo "FAILED ($logs_status) — otel-collector is configured but its logs could not be read:"
|
|
88
|
+
cat otel.log
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
62
91
|
```
|
|
63
92
|
|
|
64
93
|
## Remote Verification (AWS X-Ray)
|
|
@@ -17,8 +17,11 @@ Reduces the CyclomaticComplexity threshold by 2 and fixes all violations.
|
|
|
17
17
|
bundle exec rubocop --only Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity --format json 2>&1
|
|
18
18
|
```
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
status=0
|
|
21
|
+
bundle exec flog --all --group app/ >flog.log 2>&1 || status=$?
|
|
22
|
+
head -n 50 flog.log; echo "exit=$status"
|
|
21
23
|
```
|
|
24
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `head` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
22
25
|
4. **Note for each violation**:
|
|
23
26
|
- File path and line number
|
|
24
27
|
- Method name
|
|
@@ -16,8 +16,11 @@ If no argument provided, prompt the user for a target.
|
|
|
16
16
|
1. **Find coverage config** (`.simplecov` or `spec/spec_helper.rb`)
|
|
17
17
|
2. **Run test suite with coverage** to get current state:
|
|
18
18
|
```bash
|
|
19
|
-
|
|
19
|
+
status=0
|
|
20
|
+
bundle exec rspec >rspec.log 2>&1 || status=$?
|
|
21
|
+
tail -n 50 rspec.log; echo "exit=$status"
|
|
20
22
|
```
|
|
23
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `tail` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
21
24
|
3. **Check SimpleCov output** in `coverage/index.html` or console output
|
|
22
25
|
4. **Identify the 20 files with lowest coverage**, noting:
|
|
23
26
|
- File path
|
|
@@ -58,7 +58,36 @@ grep -i "otel\|opentelemetry\|trace_id" log/development.log | tail -10
|
|
|
58
58
|
### Check Collector Sidecar (if using Docker Compose)
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
|
|
61
|
+
# Capture the status BEFORE the pipe. A pipeline reports its LAST stage's exit
|
|
62
|
+
# code, so `docker compose logs ... | tail -20 || echo "No otel-collector"`
|
|
63
|
+
# never prints the fallback: `tail` succeeds even when there is no such
|
|
64
|
+
# service, and the absent sidecar reads as a silent pass.
|
|
65
|
+
#
|
|
66
|
+
# `|| status=$?`, not `; status=$?`: under `set -e` the `;` form exits before
|
|
67
|
+
# the assignment, so the branch that reports the failure never runs.
|
|
68
|
+
#
|
|
69
|
+
# Ask whether the service EXISTS separately from whether its logs could be
|
|
70
|
+
# read. `docker compose logs` also exits non-zero for a stopped daemon, a
|
|
71
|
+
# permission failure, and an unparseable compose file; answering all of those
|
|
72
|
+
# with "No otel-collector service" turns a broken telemetry stack into a
|
|
73
|
+
# reassuring sentence — the same silent-measurement failure one level up.
|
|
74
|
+
services_status=0
|
|
75
|
+
services=$(docker compose config --services 2>&1) || services_status=$?
|
|
76
|
+
if [ "$services_status" -ne 0 ]; then
|
|
77
|
+
echo "FAILED ($services_status) — docker compose config could not be read:"
|
|
78
|
+
printf '%s\n' "$services"
|
|
79
|
+
elif ! printf '%s\n' "$services" | grep -qx otel-collector; then
|
|
80
|
+
echo "No otel-collector service in Docker Compose"
|
|
81
|
+
else
|
|
82
|
+
logs_status=0
|
|
83
|
+
docker compose logs otel-collector >otel.log 2>&1 || logs_status=$?
|
|
84
|
+
if [ "$logs_status" -eq 0 ]; then
|
|
85
|
+
tail -n 20 otel.log
|
|
86
|
+
else
|
|
87
|
+
echo "FAILED ($logs_status) — otel-collector is configured but its logs could not be read:"
|
|
88
|
+
cat otel.log
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
62
91
|
```
|
|
63
92
|
|
|
64
93
|
## Remote Verification (AWS X-Ray)
|
|
@@ -17,8 +17,11 @@ Reduces the CyclomaticComplexity threshold by 2 and fixes all violations.
|
|
|
17
17
|
bundle exec rubocop --only Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity --format json 2>&1
|
|
18
18
|
```
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
status=0
|
|
21
|
+
bundle exec flog --all --group app/ >flog.log 2>&1 || status=$?
|
|
22
|
+
head -n 50 flog.log; echo "exit=$status"
|
|
21
23
|
```
|
|
24
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `head` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
22
25
|
4. **Note for each violation**:
|
|
23
26
|
- File path and line number
|
|
24
27
|
- Method name
|
|
@@ -16,8 +16,11 @@ If no argument provided, prompt the user for a target.
|
|
|
16
16
|
1. **Find coverage config** (`.simplecov` or `spec/spec_helper.rb`)
|
|
17
17
|
2. **Run test suite with coverage** to get current state:
|
|
18
18
|
```bash
|
|
19
|
-
|
|
19
|
+
status=0
|
|
20
|
+
bundle exec rspec >rspec.log 2>&1 || status=$?
|
|
21
|
+
tail -n 50 rspec.log; echo "exit=$status"
|
|
20
22
|
```
|
|
23
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `tail` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
21
24
|
3. **Check SimpleCov output** in `coverage/index.html` or console output
|
|
22
25
|
4. **Identify the 20 files with lowest coverage**, noting:
|
|
23
26
|
- File path
|
|
@@ -58,7 +58,36 @@ grep -i "otel\|opentelemetry\|trace_id" log/development.log | tail -10
|
|
|
58
58
|
### Check Collector Sidecar (if using Docker Compose)
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
|
|
61
|
+
# Capture the status BEFORE the pipe. A pipeline reports its LAST stage's exit
|
|
62
|
+
# code, so `docker compose logs ... | tail -20 || echo "No otel-collector"`
|
|
63
|
+
# never prints the fallback: `tail` succeeds even when there is no such
|
|
64
|
+
# service, and the absent sidecar reads as a silent pass.
|
|
65
|
+
#
|
|
66
|
+
# `|| status=$?`, not `; status=$?`: under `set -e` the `;` form exits before
|
|
67
|
+
# the assignment, so the branch that reports the failure never runs.
|
|
68
|
+
#
|
|
69
|
+
# Ask whether the service EXISTS separately from whether its logs could be
|
|
70
|
+
# read. `docker compose logs` also exits non-zero for a stopped daemon, a
|
|
71
|
+
# permission failure, and an unparseable compose file; answering all of those
|
|
72
|
+
# with "No otel-collector service" turns a broken telemetry stack into a
|
|
73
|
+
# reassuring sentence — the same silent-measurement failure one level up.
|
|
74
|
+
services_status=0
|
|
75
|
+
services=$(docker compose config --services 2>&1) || services_status=$?
|
|
76
|
+
if [ "$services_status" -ne 0 ]; then
|
|
77
|
+
echo "FAILED ($services_status) — docker compose config could not be read:"
|
|
78
|
+
printf '%s\n' "$services"
|
|
79
|
+
elif ! printf '%s\n' "$services" | grep -qx otel-collector; then
|
|
80
|
+
echo "No otel-collector service in Docker Compose"
|
|
81
|
+
else
|
|
82
|
+
logs_status=0
|
|
83
|
+
docker compose logs otel-collector >otel.log 2>&1 || logs_status=$?
|
|
84
|
+
if [ "$logs_status" -eq 0 ]; then
|
|
85
|
+
tail -n 20 otel.log
|
|
86
|
+
else
|
|
87
|
+
echo "FAILED ($logs_status) — otel-collector is configured but its logs could not be read:"
|
|
88
|
+
cat otel.log
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
62
91
|
```
|
|
63
92
|
|
|
64
93
|
## Remote Verification (AWS X-Ray)
|
|
@@ -17,8 +17,11 @@ Reduces the CyclomaticComplexity threshold by 2 and fixes all violations.
|
|
|
17
17
|
bundle exec rubocop --only Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity --format json 2>&1
|
|
18
18
|
```
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
status=0
|
|
21
|
+
bundle exec flog --all --group app/ >flog.log 2>&1 || status=$?
|
|
22
|
+
head -n 50 flog.log; echo "exit=$status"
|
|
21
23
|
```
|
|
24
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `head` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
22
25
|
4. **Note for each violation**:
|
|
23
26
|
- File path and line number
|
|
24
27
|
- Method name
|
|
@@ -16,8 +16,11 @@ If no argument provided, prompt the user for a target.
|
|
|
16
16
|
1. **Find coverage config** (`.simplecov` or `spec/spec_helper.rb`)
|
|
17
17
|
2. **Run test suite with coverage** to get current state:
|
|
18
18
|
```bash
|
|
19
|
-
|
|
19
|
+
status=0
|
|
20
|
+
bundle exec rspec >rspec.log 2>&1 || status=$?
|
|
21
|
+
tail -n 50 rspec.log; echo "exit=$status"
|
|
20
22
|
```
|
|
23
|
+
The status is captured before the pipe, because a pipeline reports its LAST stage's exit code — `tail` always succeeds, so a failing run reads as `exit=0` (`falsifiable-checks`, pager-shadowed status). `|| status=$?` rather than `; status=$?`: under `set -e` the `;` form exits before the assignment, so the failure is never reported at all.
|
|
21
24
|
3. **Check SimpleCov output** in `coverage/index.html` or console output
|
|
22
25
|
4. **Identify the 20 files with lowest coverage**, noting:
|
|
23
26
|
- File path
|
|
@@ -58,7 +58,36 @@ grep -i "otel\|opentelemetry\|trace_id" log/development.log | tail -10
|
|
|
58
58
|
### Check Collector Sidecar (if using Docker Compose)
|
|
59
59
|
|
|
60
60
|
```bash
|
|
61
|
-
|
|
61
|
+
# Capture the status BEFORE the pipe. A pipeline reports its LAST stage's exit
|
|
62
|
+
# code, so `docker compose logs ... | tail -20 || echo "No otel-collector"`
|
|
63
|
+
# never prints the fallback: `tail` succeeds even when there is no such
|
|
64
|
+
# service, and the absent sidecar reads as a silent pass.
|
|
65
|
+
#
|
|
66
|
+
# `|| status=$?`, not `; status=$?`: under `set -e` the `;` form exits before
|
|
67
|
+
# the assignment, so the branch that reports the failure never runs.
|
|
68
|
+
#
|
|
69
|
+
# Ask whether the service EXISTS separately from whether its logs could be
|
|
70
|
+
# read. `docker compose logs` also exits non-zero for a stopped daemon, a
|
|
71
|
+
# permission failure, and an unparseable compose file; answering all of those
|
|
72
|
+
# with "No otel-collector service" turns a broken telemetry stack into a
|
|
73
|
+
# reassuring sentence — the same silent-measurement failure one level up.
|
|
74
|
+
services_status=0
|
|
75
|
+
services=$(docker compose config --services 2>&1) || services_status=$?
|
|
76
|
+
if [ "$services_status" -ne 0 ]; then
|
|
77
|
+
echo "FAILED ($services_status) — docker compose config could not be read:"
|
|
78
|
+
printf '%s\n' "$services"
|
|
79
|
+
elif ! printf '%s\n' "$services" | grep -qx otel-collector; then
|
|
80
|
+
echo "No otel-collector service in Docker Compose"
|
|
81
|
+
else
|
|
82
|
+
logs_status=0
|
|
83
|
+
docker compose logs otel-collector >otel.log 2>&1 || logs_status=$?
|
|
84
|
+
if [ "$logs_status" -eq 0 ]; then
|
|
85
|
+
tail -n 20 otel.log
|
|
86
|
+
else
|
|
87
|
+
echo "FAILED ($logs_status) — otel-collector is configured but its logs could not be read:"
|
|
88
|
+
cat otel.log
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
62
91
|
```
|
|
63
92
|
|
|
64
93
|
## Remote Verification (AWS X-Ray)
|
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
import {
|
|
10
10
|
extractAllowEntries,
|
|
11
11
|
extractK6Constraints,
|
|
12
|
+
extractLighthouseAssertions,
|
|
12
13
|
extractNumericLeaves,
|
|
13
14
|
extractRubocopThresholds,
|
|
14
15
|
extractStrykerConstraints,
|
|
@@ -296,6 +297,12 @@ export function compareFile(relPath, baselineText, currentText) {
|
|
|
296
297
|
return compareStryker(relPath, base, current);
|
|
297
298
|
case "k6":
|
|
298
299
|
return compareK6(relPath, base, current);
|
|
300
|
+
case "lighthouse":
|
|
301
|
+
return compareConstraints(
|
|
302
|
+
relPath,
|
|
303
|
+
extractLighthouseAssertions(base),
|
|
304
|
+
extractLighthouseAssertions(current)
|
|
305
|
+
);
|
|
299
306
|
case KIND_ALLOW_LIST:
|
|
300
307
|
return compareAllowList(relPath, base, current);
|
|
301
308
|
default:
|
|
@@ -77,8 +77,61 @@ export const FAMILIES = [
|
|
|
77
77
|
match: /(^|\/)\.lisa\.config\.json$/,
|
|
78
78
|
kind: "allow-list",
|
|
79
79
|
},
|
|
80
|
+
{
|
|
81
|
+
id: "lighthouse",
|
|
82
|
+
match: /(^|\/)lighthouserc-config\.json$/,
|
|
83
|
+
kind: "lighthouse",
|
|
84
|
+
},
|
|
80
85
|
];
|
|
81
86
|
|
|
87
|
+
/**
|
|
88
|
+
* Which way each Lighthouse assertion key ratchets.
|
|
89
|
+
*
|
|
90
|
+
* Lighthouse mixes floors and ceilings in one file, so assigning one direction
|
|
91
|
+
* to the whole family would silently approve half of the possible weakenings.
|
|
92
|
+
* Unknown keys stay unwatched until their direction is explicit here.
|
|
93
|
+
* @type {Readonly<Record<string, "min"|"max">>}
|
|
94
|
+
*/
|
|
95
|
+
export const LIGHTHOUSE_ASSERTION_DIRECTIONS = {
|
|
96
|
+
minScore: "min",
|
|
97
|
+
maxNumericValue: "max",
|
|
98
|
+
maxLength: "max",
|
|
99
|
+
};
|
|
100
|
+
|
|
101
|
+
/**
|
|
102
|
+
* Extract numeric Lighthouse assertions with a direction chosen per key.
|
|
103
|
+
*
|
|
104
|
+
* A malformed array is deliberately ignored instead of being walked as an
|
|
105
|
+
* object whose numeric indices look like audit names.
|
|
106
|
+
* @param {unknown} conf Parsed lighthouserc-config.json
|
|
107
|
+
* @returns {Map<string, { value: number, direction: "min"|"max" }>} Dotted
|
|
108
|
+
* `assertion.key` path to numeric constraint
|
|
109
|
+
*/
|
|
110
|
+
export function extractLighthouseAssertions(conf) {
|
|
111
|
+
const out = new Map();
|
|
112
|
+
const assertions =
|
|
113
|
+
conf && typeof conf === "object" && !Array.isArray(conf)
|
|
114
|
+
? /** @type {Record<string, unknown>} */ (conf).assertions
|
|
115
|
+
: undefined;
|
|
116
|
+
if (
|
|
117
|
+
!assertions ||
|
|
118
|
+
typeof assertions !== "object" ||
|
|
119
|
+
Array.isArray(assertions)
|
|
120
|
+
) {
|
|
121
|
+
return out;
|
|
122
|
+
}
|
|
123
|
+
for (const [audit, spec] of Object.entries(assertions)) {
|
|
124
|
+
if (!spec || typeof spec !== "object" || Array.isArray(spec)) continue;
|
|
125
|
+
for (const [key, value] of Object.entries(spec)) {
|
|
126
|
+
const direction = LIGHTHOUSE_ASSERTION_DIRECTIONS[key];
|
|
127
|
+
if (direction && typeof value === "number" && Number.isFinite(value)) {
|
|
128
|
+
out.set(`${audit}.${key}`, { value, direction });
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
return out;
|
|
133
|
+
}
|
|
134
|
+
|
|
82
135
|
/**
|
|
83
136
|
* Find the family a repo-relative path belongs to.
|
|
84
137
|
* @param {string} relPath Repo-relative path (forward slashes)
|
|
@@ -6,7 +6,7 @@ A passing test, a clean lint run, a zero-hit sweep, a green ratchet: each is evi
|
|
|
6
6
|
|
|
7
7
|
This is the instrument-validity counterpart to the `verification` rule (which proves the *software* behaves) and `empirical-inquiry` (which proves a *fact*). All three reject "it looks correct." This one rejects "the check said so."
|
|
8
8
|
|
|
9
|
-
## The
|
|
9
|
+
## The five ways a check silently measures nothing
|
|
10
10
|
|
|
11
11
|
Each has been observed in real runs; each reported success while asserting nothing:
|
|
12
12
|
|
|
@@ -14,9 +14,24 @@ Each has been observed in real runs; each reported success while asserting nothi
|
|
|
14
14
|
2. **Fixture-validated assertion** — the assertion reads the test's own fixture rather than the artifact under test. Common when the production path consumes a *raw* input the fixture supplies directly.
|
|
15
15
|
3. **Stale-artifact pass** — the revert-to-verify step silently failed (generator errored, cache served old output, build skipped), so the check re-read unchanged input and "passed".
|
|
16
16
|
4. **Wrong-baseline sweep** — the detector ran against already-fixed state, so its zero is uninformative. Validate detectors against a ref where the defect still exists.
|
|
17
|
+
5. **Pager-shadowed status** — the check ran and FAILED, but its result was read through a pipe. A shell pipeline's exit status is its LAST stage's, so `gate | tail -4; echo $?` reports `tail`'s success. Shortening output is exactly what one does while checking whether a gate passed, and the failure line is exactly what the shortening drops.
|
|
17
18
|
|
|
18
19
|
## Mandatory
|
|
19
20
|
|
|
21
|
+
- **Read the command's own status, never a pipeline's.** When the answer decides whether you are blocked, redirect first and read the status before shortening anything — one spelling, portable to every shell including POSIX `sh`:
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
status=0
|
|
25
|
+
gate >gate.log 2>&1 || status=$? # `||`, not `;` — see below
|
|
26
|
+
tail -n 20 gate.log # shorten for reading, not for deciding
|
|
27
|
+
[ "$status" -eq 0 ] || echo "FAILED ($status) — full output in gate.log"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`|| status=$?` rather than `; status=$?`, because the `;` form is itself a silent-measurement bug under `set -e`: errexit kills the shell on the failing command and the assignment never runs, so the branch that reports the failure is unreachable in exactly the case it exists for. The left side of `||` is exempt from errexit, which is why this spelling survives both `set -e` and its absence — and why `status` is initialized, since on success nothing assigns it.
|
|
31
|
+
|
|
32
|
+
When the block's **own** status is what gets read — a hook, a CI step, a script someone runs with `&&` — end it with `exit "$status"` (or `return "$status"` in a function). Without that, the last command is the `echo` and the block exits 0 after printing `FAILED`: the same pager-shadowing one line further out.
|
|
33
|
+
|
|
34
|
+
This form is preferred over the alternatives because it is the only one that both survives `sh` and keeps the full output, which is what you need the moment the status says the gate failed. `set -o pipefail` is correct where it exists but is **not POSIX `sh`** — dash has no such option, and this repository runs hooks under `sh`. `${PIPESTATUS[0]}` is **bash-only** (zsh spells it `$pipestatus[1]`, 1-indexed) and is overwritten by the very next command, so it must be read immediately. Never report a gate as green from a status read through `| tail`, `| head`, or `| tee`.
|
|
20
35
|
- **Falsify before reporting.** No clean result is reportable until the check has been shown to fail on a deliberate break. **"Mentally reverting" does not count** — reasoning that the assertion *would* fail is precisely the step that lets a non-functional guard ship, because the author already believes it is load-bearing. Run the break.
|
|
21
36
|
- **Mutation-prove the guard, then read the cardinality.** Neuter the protection **in production code**, run the **whole suite**, count the failures and **read their names**, then revert. **Zero** ⇒ the guard is inert (one pinning a single field shipped with all 50 tests green while the regression walked through). **Many, unrelated** ⇒ over-broad; its next failure will not name the cause. **Exactly one, or several all named for the same regression** ⇒ load-bearing and correctly scoped. A **zero is robust to contention** — load only ever adds failures — so only the non-zero counts need a quiet machine.
|
|
22
37
|
- **Never scope the probe by filename, and never infer absent coverage from a grep.** Suites split at the line cap, so a protection's tests often sit in a sibling file: a filename-scoped probe reported cardinality 0 on a live security fix that a whole-suite run showed failing 4 tests, every one named for it.
|