@cratis/pi 7.101.0 → 7.103.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package/corpus/harnesses/copilot/instructions/github-actions.instructions.md +17 -0
- package/package/corpus/harnesses/copilot/instructions/profiles.instructions.md +10 -1
- package/package/corpus/harnesses/copilot/instructions/pull-requests.instructions.md +2 -0
- package/package/corpus/harnesses/copilot/instructions/secrets.instructions.md +4 -0
- package/package/corpus/hooks/README.md +11 -1
- package/package/corpus/hooks/scripts/cratis-check-pr.mjs +12 -1
- package/package/corpus/hooks/scripts/cratis-guard-secrets.sh +79 -23
- package/package/corpus/hooks/scripts/cratis-guard-writes.sh +145 -40
- package/package/corpus/rules/github-actions.md +17 -0
- package/package/corpus/rules/profiles.md +10 -1
- package/package/corpus/rules/pull-requests.md +2 -0
- package/package/corpus/rules/secrets.md +4 -0
- package/package/corpus/skills/analyzing-dotnet-performance/LICENSE +23 -0
- package/package/corpus/skills/analyzing-dotnet-performance/NOTICE +36 -0
- package/package/corpus/skills/analyzing-dotnet-performance/SKILL.md +214 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/async-patterns.md +122 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/collections-and-linq.md +220 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/critical-patterns.md +291 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/io-and-serialization.md +135 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/memory-and-strings.md +219 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/regex-patterns.md +95 -0
- package/package/corpus/skills/analyzing-dotnet-performance/references/structural-patterns.md +38 -0
- package/package/corpus/skills/binlog-generation/LICENSE +23 -0
- package/package/corpus/skills/binlog-generation/NOTICE +22 -0
- package/package/corpus/skills/binlog-generation/SKILL.md +129 -0
- package/package/corpus/skills/check-bin-obj-clash/LICENSE +23 -0
- package/package/corpus/skills/check-bin-obj-clash/NOTICE +27 -0
- package/package/corpus/skills/check-bin-obj-clash/SKILL.md +127 -0
- package/package/corpus/skills/check-bin-obj-clash/references/common-causes-and-fixes.md +206 -0
- package/package/corpus/skills/coverage-analysis/LICENSE +23 -0
- package/package/corpus/skills/coverage-analysis/NOTICE +32 -0
- package/package/corpus/skills/coverage-analysis/SKILL.md +198 -0
- package/package/corpus/skills/coverage-analysis/references/guidelines.md +68 -0
- package/package/corpus/skills/coverage-analysis/references/output-format.md +91 -0
- package/package/corpus/skills/coverage-analysis/references/report-generation.md +58 -0
- package/package/corpus/skills/coverage-analysis/references/setup-discovery.md +307 -0
- package/package/corpus/skills/coverage-analysis/scripts/Compute-CrapScores.ps1 +80 -0
- package/package/corpus/skills/coverage-analysis/scripts/CoverageData.ps1 +272 -0
- package/package/corpus/skills/coverage-analysis/scripts/Extract-MethodCoverage.ps1 +65 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/LICENSE +2 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/SKILL.md +201 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/build-once-and-shard.md +158 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/caching.md +86 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/flaky-and-hung-jobs.md +70 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/publish-pipelines.md +67 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/required-checks-and-path-filters.md +117 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/runner-sizing-and-images.md +65 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/security-and-pinning.md +73 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/sources.md +33 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/references/test-setup-cost.md +82 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/scripts/ci-timings.sh +356 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/scripts/fixtures/jobs/1003.json +79 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/scripts/fixtures/jobs/1007.json +79 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/scripts/fixtures/jobs/1010.json +79 -0
- package/package/corpus/skills/cratis-ci-pipeline-performance/scripts/fixtures/runs.json +102 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/LICENSE +2 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/SKILL.md +244 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/docker-images.md +86 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/dotnet-nuget.md +87 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/go-jvm-python-elixir.md +83 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/npm-yarn.md +78 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/other-environments.md +60 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/references/worked-examples.md +97 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/scripts/cratis-local-packages.sh +778 -0
- package/package/corpus/skills/cratis-engineering-local-cross-repo-verification/scripts/npm-restore-check.cjs +109 -0
- package/package/corpus/skills/csharp-refactoring/LICENSE +23 -0
- package/package/corpus/skills/csharp-refactoring/NOTICE +22 -0
- package/package/corpus/skills/csharp-refactoring/SKILL.md +175 -0
- package/package/corpus/skills/csharp-refactoring/references/operation-catalog.md +37 -0
- package/package/corpus/skills/directory-build-organization/LICENSE +23 -0
- package/package/corpus/skills/directory-build-organization/NOTICE +21 -0
- package/package/corpus/skills/directory-build-organization/SKILL.md +205 -0
- package/package/corpus/skills/directory-build-organization/references/common-patterns.md +56 -0
- package/package/corpus/skills/directory-build-organization/references/multi-level-examples.md +167 -0
- package/package/corpus/skills/directory-build-organization/references/targetframework-props-pitfall.md +54 -0
- package/package/corpus/skills/dotnet-aot-compat/LICENSE +23 -0
- package/package/corpus/skills/dotnet-aot-compat/NOTICE +18 -0
- package/package/corpus/skills/dotnet-aot-compat/SKILL.md +265 -0
- package/package/corpus/skills/dotnet-aot-compat/references/polyfills.md +43 -0
- package/package/corpus/skills/dotnet-trace-collect/LICENSE +23 -0
- package/package/corpus/skills/dotnet-trace-collect/NOTICE +65 -0
- package/package/corpus/skills/dotnet-trace-collect/SKILL.md +116 -0
- package/package/corpus/skills/dotnet-trace-collect/references/dotnet-monitor.md +97 -0
- package/package/corpus/skills/dotnet-trace-collect/references/dotnet-trace-collect-linux.md +84 -0
- package/package/corpus/skills/dotnet-trace-collect/references/dotnet-trace-collect.md +91 -0
- package/package/corpus/skills/dotnet-trace-collect/references/perfcollect.md +200 -0
- package/package/corpus/skills/dotnet-trace-collect/references/perfview.md +141 -0
- package/package/corpus/skills/dotnet-trace-collect/references/tool-selection.md +160 -0
- package/package/corpus/skills/dump-collect/LICENSE +23 -0
- package/package/corpus/skills/dump-collect/NOTICE +40 -0
- package/package/corpus/skills/dump-collect/SKILL.md +97 -0
- package/package/corpus/skills/dump-collect/references/container-dumps.md +412 -0
- package/package/corpus/skills/dump-collect/references/coreclr-dumps.md +108 -0
- package/package/corpus/skills/dump-collect/references/nativeaot-dumps.md +280 -0
- package/package/corpus/skills/eliminate-redundant-restores/LICENSE +23 -0
- package/package/corpus/skills/eliminate-redundant-restores/NOTICE +19 -0
- package/package/corpus/skills/eliminate-redundant-restores/SKILL.md +112 -0
- package/package/corpus/skills/extension-points/LICENSE +23 -0
- package/package/corpus/skills/extension-points/NOTICE +18 -0
- package/package/corpus/skills/extension-points/SKILL.md +248 -0
- package/package/corpus/skills/filter-syntax/LICENSE +23 -0
- package/package/corpus/skills/filter-syntax/NOTICE +17 -0
- package/package/corpus/skills/filter-syntax/SKILL.md +179 -0
- package/package/corpus/skills/including-generated-files/LICENSE +23 -0
- package/package/corpus/skills/including-generated-files/NOTICE +17 -0
- package/package/corpus/skills/including-generated-files/SKILL.md +139 -0
- package/package/corpus/skills/microbenchmarking/LICENSE +23 -0
- package/package/corpus/skills/microbenchmarking/NOTICE +36 -0
- package/package/corpus/skills/microbenchmarking/SKILL.md +131 -0
- package/package/corpus/skills/microbenchmarking/references/bdn-internals-and-tuning.md +143 -0
- package/package/corpus/skills/microbenchmarking/references/comparison-strategies.md +200 -0
- package/package/corpus/skills/microbenchmarking/references/diagnosers-and-exporters.md +150 -0
- package/package/corpus/skills/microbenchmarking/references/project-setup-and-running.md +189 -0
- package/package/corpus/skills/microbenchmarking/references/writing-benchmarks.md +310 -0
- package/package/corpus/skills/msbuild/LICENSE +23 -0
- package/package/corpus/skills/msbuild/NOTICE +55 -0
- package/package/corpus/skills/msbuild/SKILL.md +117 -0
- package/package/corpus/skills/msbuild/references/antipatterns/additional-antipatterns.md +291 -0
- package/package/corpus/skills/msbuild/references/antipatterns/incremental-build-inputs-outputs.md +57 -0
- package/package/corpus/skills/msbuild/references/antipatterns/private-assets.md +29 -0
- package/package/corpus/skills/msbuild/references/binlog-failure-analysis.md +85 -0
- package/package/corpus/skills/msbuild/references/binlog-generation.md +69 -0
- package/package/corpus/skills/msbuild/references/build-perf-baseline.md +493 -0
- package/package/corpus/skills/msbuild/references/build-perf-diagnostics.md +268 -0
- package/package/corpus/skills/msbuild/references/copy-to-output-directory.md +183 -0
- package/package/corpus/skills/msbuild/references/eval-performance.md +190 -0
- package/package/corpus/skills/msbuild/references/extension-points.md +281 -0
- package/package/corpus/skills/msbuild/references/incremental-build.md +365 -0
- package/package/corpus/skills/msbuild/references/msbuild-antipatterns.md +401 -0
- package/package/corpus/skills/orleans/LICENSE +21 -0
- package/package/corpus/skills/orleans/NOTICE +64 -0
- package/package/corpus/skills/orleans/SKILL.md +140 -0
- package/package/corpus/skills/orleans/references/anti-patterns.md +660 -0
- package/package/corpus/skills/orleans/references/configuration-api.md +933 -0
- package/package/corpus/skills/orleans/references/examples.md +66 -0
- package/package/corpus/skills/orleans/references/grain-api.md +784 -0
- package/package/corpus/skills/orleans/references/grains.md +75 -0
- package/package/corpus/skills/orleans/references/hosting.md +71 -0
- package/package/corpus/skills/orleans/references/implementation.md +591 -0
- package/package/corpus/skills/orleans/references/official-docs-index.md +250 -0
- package/package/corpus/skills/orleans/references/patterns.md +811 -0
- package/package/corpus/skills/orleans/references/persistence-api.md +495 -0
- package/package/corpus/skills/orleans/references/primitive-selection.md +35 -0
- package/package/corpus/skills/orleans/references/scheduling-and-services.md +295 -0
- package/package/corpus/skills/orleans/references/serialization-api.md +299 -0
- package/package/corpus/skills/orleans/references/streaming-api.md +396 -0
- package/package/corpus/skills/orleans/references/testing-patterns.md +210 -0
- package/package/corpus/skills/platform-detection/LICENSE +23 -0
- package/package/corpus/skills/platform-detection/NOTICE +17 -0
- package/package/corpus/skills/platform-detection/SKILL.md +230 -0
- package/package/corpus/skills/platform-detection/references/command-mode.md +27 -0
- package/package/corpus/skills/profiling/LICENSE +21 -0
- package/package/corpus/skills/profiling/NOTICE +50 -0
- package/package/corpus/skills/profiling/SKILL.md +144 -0
- package/package/corpus/skills/profiling/references/commands.md +439 -0
- package/package/corpus/skills/profiling/references/patterns.md +586 -0
- package/package/corpus/skills/profiling/references/profiling.md +190 -0
- package/package/corpus/skills/run-tests/LICENSE +23 -0
- package/package/corpus/skills/run-tests/NOTICE +19 -0
- package/package/corpus/skills/run-tests/SKILL.md +305 -0
- package/package/corpus/skills/target-authoring/LICENSE +23 -0
- package/package/corpus/skills/target-authoring/NOTICE +17 -0
- package/package/corpus/skills/target-authoring/SKILL.md +165 -0
- package/package/corpus/skills/vectorization/LICENSE +23 -0
- package/package/corpus/skills/vectorization/NOTICE +17 -0
- package/package/corpus/skills/vectorization/SKILL.md +189 -0
- package/package/profile-catalog.json +87 -1
- package/package.json +3 -2
|
@@ -0,0 +1,272 @@
|
|
|
1
|
+
# Shared, report-identity-scoped Cobertura normalization. This does not attest
|
|
2
|
+
# that a report was collected from the current checkout.
|
|
3
|
+
|
|
4
|
+
function New-CoverageMap {
|
|
5
|
+
return [System.Collections.Generic.Dictionary[string, object]]::new([StringComparer]::Ordinal)
|
|
6
|
+
}
|
|
7
|
+
|
|
8
|
+
function Add-CoverageDiagnostic($Diagnostics, [string]$Code, [string]$Message, [string]$Path) {
|
|
9
|
+
$Diagnostics.Add([pscustomobject]@{ code = $Code; severity = 'warning'; message = $Message; path = $Path })
|
|
10
|
+
}
|
|
11
|
+
|
|
12
|
+
function Read-CoverageInteger([string]$Text, [string]$Name, [long]$Minimum = 0) {
|
|
13
|
+
$value = 0L
|
|
14
|
+
if (-not [long]::TryParse($Text, [Globalization.NumberStyles]::Integer,
|
|
15
|
+
[Globalization.CultureInfo]::InvariantCulture, [ref]$value) -or $value -lt $Minimum) {
|
|
16
|
+
throw [FormatException]::new("Invalid $Name value '$Text'.")
|
|
17
|
+
}
|
|
18
|
+
return $value
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
function Add-CoverageLine($Map, $Node, [string]$Key, [string]$ReportId) {
|
|
22
|
+
$number = Read-CoverageInteger $Node.GetAttribute('number') 'line number' 1
|
|
23
|
+
$hits = Read-CoverageInteger $Node.GetAttribute('hits') 'line hits'
|
|
24
|
+
$lineKey = "$Key`n$number"
|
|
25
|
+
if (-not $Map.ContainsKey($lineKey)) {
|
|
26
|
+
$Map[$lineKey] = @{
|
|
27
|
+
Covered = $false
|
|
28
|
+
NonBranch = $false
|
|
29
|
+
ConflictingBranches = $false
|
|
30
|
+
Branches = [System.Collections.Generic.List[object]]::new()
|
|
31
|
+
Observations = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal)
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
$entry = $Map[$lineKey]
|
|
35
|
+
$entry.Covered = $entry.Covered -or $hits -gt 0
|
|
36
|
+
if (-not $entry.Observations.Add($ReportId)) { return }
|
|
37
|
+
$branch = $Node.GetAttribute('branch')
|
|
38
|
+
if ($branch -and $branch -notin @('true', 'false')) {
|
|
39
|
+
throw [FormatException]::new("Invalid branch flag '$branch'.")
|
|
40
|
+
}
|
|
41
|
+
if ($branch -ne 'true') {
|
|
42
|
+
$entry.NonBranch = $true
|
|
43
|
+
if ($entry.Branches.Count) { $entry.ConflictingBranches = $true }
|
|
44
|
+
return
|
|
45
|
+
}
|
|
46
|
+
if ($entry.NonBranch) { $entry.ConflictingBranches = $true }
|
|
47
|
+
$raw = $Node.GetAttribute('condition-coverage')
|
|
48
|
+
if ($raw -notmatch '^\s*[\d.]+%\s*\((\d+)/(\d+)\)\s*$') {
|
|
49
|
+
$entry.Branches.Add(@{ Total = $null; Covered = $null; Conditions = $null })
|
|
50
|
+
return
|
|
51
|
+
}
|
|
52
|
+
$covered = Read-CoverageInteger $Matches[1] 'covered branches'
|
|
53
|
+
$total = Read-CoverageInteger $Matches[2] 'total branches' 1
|
|
54
|
+
if ($covered -gt $total) { throw [FormatException]::new('Covered branches exceed total branches.') }
|
|
55
|
+
|
|
56
|
+
# Cobertura jump conditions describe two outcomes, not which outcome a
|
|
57
|
+
# 50% observation took. Keep that ambiguity even when condition ids match.
|
|
58
|
+
$conditions = New-CoverageMap
|
|
59
|
+
$nodes = @($Node.SelectNodes('conditions/condition'))
|
|
60
|
+
$conditionSum = 0
|
|
61
|
+
$usable = $nodes.Count -gt 0 -and $total -eq 2 * $nodes.Count
|
|
62
|
+
foreach ($condition in $nodes) {
|
|
63
|
+
$id = $condition.GetAttribute('number')
|
|
64
|
+
$coverage = $condition.GetAttribute('coverage')
|
|
65
|
+
if ($condition.GetAttribute('type') -ne 'jump' -or -not $id -or
|
|
66
|
+
$conditions.ContainsKey($id) -or $coverage -notmatch '^(0|50|100)(?:\.0+)?%$') {
|
|
67
|
+
$usable = $false
|
|
68
|
+
continue
|
|
69
|
+
}
|
|
70
|
+
$count = [int]$Matches[1] / 50
|
|
71
|
+
$conditions[$id] = $count
|
|
72
|
+
$conditionSum += $count
|
|
73
|
+
}
|
|
74
|
+
if (-not $usable -or $conditionSum -ne $covered) { $conditions = $null }
|
|
75
|
+
$entry.Branches.Add(@{ Total = $total; Covered = $covered; Conditions = $conditions })
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
function Get-CoverageMeasure($Lines) {
|
|
79
|
+
$coveredLines = @($Lines.Values | Where-Object Covered).Count
|
|
80
|
+
$linePercent = if ($Lines.Count) { [math]::Round(100.0 * $coveredLines / $Lines.Count, 1) } else { $null }
|
|
81
|
+
$lower = 0L
|
|
82
|
+
$upper = 0L
|
|
83
|
+
$total = 0L
|
|
84
|
+
$unknown = $Lines.Count -eq 0
|
|
85
|
+
foreach ($line in $Lines.Values) {
|
|
86
|
+
$observations = @($line.Branches)
|
|
87
|
+
if (-not $observations.Count) { continue }
|
|
88
|
+
if ($line.ConflictingBranches -or @($observations | Where-Object { $null -eq $_.Total }).Count) {
|
|
89
|
+
$unknown = $true
|
|
90
|
+
continue
|
|
91
|
+
}
|
|
92
|
+
$totals = @($observations | ForEach-Object { $_.Total } | Sort-Object -Unique)
|
|
93
|
+
if ($totals.Count -ne 1 -or $null -eq $totals[0]) { $unknown = $true; continue }
|
|
94
|
+
$capacity = $totals[0]
|
|
95
|
+
$total += $capacity
|
|
96
|
+
$useConditions = $true
|
|
97
|
+
$conditionIds = $null
|
|
98
|
+
foreach ($observation in $observations) {
|
|
99
|
+
if ($null -eq $observation.Conditions) { $useConditions = $false; break }
|
|
100
|
+
$ids = @($observation.Conditions.Keys | Sort-Object -CaseSensitive)
|
|
101
|
+
$identity = ConvertTo-Json -InputObject $ids -Compress
|
|
102
|
+
if ($null -ne $conditionIds -and $conditionIds -cne $identity) { $useConditions = $false; break }
|
|
103
|
+
$conditionIds = $identity
|
|
104
|
+
}
|
|
105
|
+
if ($useConditions) {
|
|
106
|
+
foreach ($id in $observations[0].Conditions.Keys) {
|
|
107
|
+
$counts = @($observations | ForEach-Object { $_.Conditions[$id] })
|
|
108
|
+
$lower += ($counts | Measure-Object -Maximum).Maximum
|
|
109
|
+
$upper += [math]::Min(2, ($counts | Measure-Object -Sum).Sum)
|
|
110
|
+
}
|
|
111
|
+
} else {
|
|
112
|
+
$counts = @($observations | ForEach-Object { $_.Covered })
|
|
113
|
+
$lower += ($counts | Measure-Object -Maximum).Maximum
|
|
114
|
+
$upper += [math]::Min($capacity, ($counts | Measure-Object -Sum).Sum)
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
$branchStatus = if ($unknown) { 'unknown' } elseif (-not $total) { 'not-applicable' } elseif ($lower -eq $upper) { 'exact' } else { 'bounded' }
|
|
118
|
+
return [pscustomobject]@{
|
|
119
|
+
TotalLines = $Lines.Count
|
|
120
|
+
CoveredLines = $coveredLines
|
|
121
|
+
LineCoverage = $linePercent
|
|
122
|
+
LineCoverageStatus = $(if ($Lines.Count) { 'exact' } else { 'unknown' })
|
|
123
|
+
TotalBranches = $(if ($unknown) { $null } else { $total })
|
|
124
|
+
CoveredBranches = $(if ($branchStatus -in @('exact', 'not-applicable')) { [long]$lower } else { $null })
|
|
125
|
+
BranchCoverage = $(if ($branchStatus -eq 'exact') { [math]::Round(100.0 * $lower / $total, 1) } else { $null })
|
|
126
|
+
BranchCoverageStatus = $branchStatus
|
|
127
|
+
CoveredBranchesLower = $(if (-not $unknown) { [long]$lower } else { $null })
|
|
128
|
+
CoveredBranchesUpper = $(if (-not $unknown) { [long]$upper } else { $null })
|
|
129
|
+
BranchCoverageLower = $(if (-not $unknown -and $total) { [math]::Round(100.0 * $lower / $total, 1) } else { $null })
|
|
130
|
+
BranchCoverageUpper = $(if (-not $unknown -and $total) { [math]::Round(100.0 * $upper / $total, 1) } else { $null })
|
|
131
|
+
}
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
function Get-CoverageData([string[]]$Paths) {
|
|
135
|
+
if (-not $Paths.Count) { throw [ArgumentException]::new('At least one Cobertura report is required.') }
|
|
136
|
+
$methods = New-CoverageMap
|
|
137
|
+
$classLines = New-CoverageMap
|
|
138
|
+
$methodLines = New-CoverageMap
|
|
139
|
+
$inputs = [System.Collections.Generic.List[object]]::new()
|
|
140
|
+
$reports = [System.Collections.Generic.List[object]]::new()
|
|
141
|
+
$diagnostics = [System.Collections.Generic.List[object]]::new()
|
|
142
|
+
$seen = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal)
|
|
143
|
+
$allClassesHaveLines = $true
|
|
144
|
+
foreach ($path in ($Paths | Sort-Object -Unique -CaseSensitive)) {
|
|
145
|
+
$resolved = (Get-Item -LiteralPath $path -ErrorAction Stop).FullName
|
|
146
|
+
$bytes = [IO.File]::ReadAllBytes($resolved)
|
|
147
|
+
$hash = [Convert]::ToHexString([Security.Cryptography.SHA256]::HashData($bytes)).ToLowerInvariant()
|
|
148
|
+
$inputs.Add([pscustomobject]@{ path = $resolved; sha256 = $hash })
|
|
149
|
+
if (-not $seen.Add($hash)) { continue }
|
|
150
|
+
$xml = [xml]::new()
|
|
151
|
+
$xml.XmlResolver = $null
|
|
152
|
+
$stream = [IO.MemoryStream]::new($bytes, $false)
|
|
153
|
+
try { $xml.Load($stream) } finally { $stream.Dispose() }
|
|
154
|
+
if ($xml.DocumentElement.Name -ne 'coverage') { throw [FormatException]::new("Not a Cobertura report: $path") }
|
|
155
|
+
$coverageRoot = $xml.DocumentElement
|
|
156
|
+
$report = @{ line = $null; branch = $null }
|
|
157
|
+
foreach ($metric in @('line', 'branch')) {
|
|
158
|
+
$prefix = if ($metric -eq 'line') { 'lines' } else { 'branches' }
|
|
159
|
+
$valid = $coverageRoot.GetAttribute("$prefix-valid")
|
|
160
|
+
$covered = $coverageRoot.GetAttribute("$prefix-covered")
|
|
161
|
+
if ($valid -or $covered) {
|
|
162
|
+
$denominator = Read-CoverageInteger $valid "$prefix-valid"
|
|
163
|
+
$numerator = Read-CoverageInteger $covered "$prefix-covered"
|
|
164
|
+
if ($numerator -gt $denominator) { throw [FormatException]::new("$prefix-covered exceeds $prefix-valid.") }
|
|
165
|
+
$report[$metric] = @{ Total = $denominator; Covered = $numerator }
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
$reports.Add($report)
|
|
169
|
+
$reportMethods = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal)
|
|
170
|
+
foreach ($class in $xml.SelectNodes('/coverage/packages/package/classes/class')) {
|
|
171
|
+
$filename = $class.GetAttribute('filename').Replace('\', '/')
|
|
172
|
+
$className = $class.GetAttribute('name')
|
|
173
|
+
$module = $class.ParentNode.ParentNode.GetAttribute('name')
|
|
174
|
+
if (-not $filename -or -not $className) { throw [FormatException]::new('Class name and filename are required.') }
|
|
175
|
+
$sourceKey = ConvertTo-Json -InputObject @($module, $filename) -Compress
|
|
176
|
+
$classNodes = @($class.SelectNodes('lines/line'))
|
|
177
|
+
if (-not $classNodes.Count) { $allClassesHaveLines = $false }
|
|
178
|
+
foreach ($line in $classNodes) { Add-CoverageLine $classLines $line $sourceKey $hash }
|
|
179
|
+
foreach ($method in $class.SelectNodes('methods/method')) {
|
|
180
|
+
$name = $method.GetAttribute('name')
|
|
181
|
+
if (-not $name) { throw [FormatException]::new('Method name is required.') }
|
|
182
|
+
$signature = $method.GetAttribute('signature')
|
|
183
|
+
$key = ConvertTo-Json -InputObject @($module, $filename, $className, $name, $signature) -Compress
|
|
184
|
+
if (-not $reportMethods.Add($key)) {
|
|
185
|
+
throw [FormatException]::new("Duplicate method identity for $className.$name; signatures or source mapping are ambiguous.")
|
|
186
|
+
}
|
|
187
|
+
if (-not $methods.ContainsKey($key)) {
|
|
188
|
+
$methods[$key] = @{
|
|
189
|
+
Module = $module; Class = $className; Method = $name; Signature = $signature; File = $filename
|
|
190
|
+
Lines = (New-CoverageMap)
|
|
191
|
+
Complexities = [System.Collections.Generic.HashSet[long]]::new()
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
$entry = $methods[$key]
|
|
195
|
+
$complexity = $method.GetAttribute('complexity')
|
|
196
|
+
if ($complexity) { [void]$entry.Complexities.Add((Read-CoverageInteger $complexity 'complexity' 1)) }
|
|
197
|
+
foreach ($line in $method.SelectNodes('lines/line')) {
|
|
198
|
+
Add-CoverageLine $entry.Lines $line '' $hash
|
|
199
|
+
Add-CoverageLine $methodLines $line $sourceKey $hash
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
$result = [System.Collections.Generic.List[object]]::new()
|
|
205
|
+
foreach ($entry in $methods.Values) {
|
|
206
|
+
$measure = Get-CoverageMeasure $entry.Lines
|
|
207
|
+
$complexity = if ($entry.Complexities.Count -eq 1) { @($entry.Complexities)[0] } else { $null }
|
|
208
|
+
$item = [ordered]@{
|
|
209
|
+
Class = $entry.Class; Method = $entry.Method; Signature = $entry.Signature; File = $entry.File
|
|
210
|
+
Module = $entry.Module; Complexity = $complexity
|
|
211
|
+
ComplexityStatus = $(if ($null -ne $complexity) { 'exact' } else { 'unknown' })
|
|
212
|
+
}
|
|
213
|
+
foreach ($property in $measure.PSObject.Properties) { $item[$property.Name] = $property.Value }
|
|
214
|
+
$item.UncoveredLines = $measure.TotalLines - $measure.CoveredLines
|
|
215
|
+
if ($null -eq $complexity) {
|
|
216
|
+
Add-CoverageDiagnostic $diagnostics 'COMPLEXITY_UNKNOWN' "Missing or conflicting complexity for $($entry.Class).$($entry.Method)." $entry.File
|
|
217
|
+
}
|
|
218
|
+
if ($measure.LineCoverageStatus -eq 'unknown') {
|
|
219
|
+
Add-CoverageDiagnostic $diagnostics 'METHOD_LINES_UNKNOWN' "No executable-line evidence for $($entry.Class).$($entry.Method)." $entry.File
|
|
220
|
+
}
|
|
221
|
+
if ($measure.BranchCoverageStatus -in @('unknown', 'bounded')) {
|
|
222
|
+
Add-CoverageDiagnostic $diagnostics 'BRANCH_UNION_UNCERTAIN' "Branch outcomes cannot be uniquely merged for $($entry.Class).$($entry.Method)." $entry.File
|
|
223
|
+
}
|
|
224
|
+
$result.Add([pscustomobject]$item)
|
|
225
|
+
}
|
|
226
|
+
$aggregate = Get-CoverageMeasure $classLines
|
|
227
|
+
$aggregateScope = 'class-lines'
|
|
228
|
+
if (-not $allClassesHaveLines -or -not $classLines.Count) {
|
|
229
|
+
$aggregateScope = if ($reports.Count -eq 1) { 'report' } else { 'method-lines-only' }
|
|
230
|
+
$aggregate = [pscustomobject]@{ LineCoverage = $null; LineCoverageStatus = 'unknown'; BranchCoverage = $null; BranchCoverageStatus = 'unknown' }
|
|
231
|
+
if ($reports.Count -eq 1) {
|
|
232
|
+
foreach ($metric in @('line', 'branch')) {
|
|
233
|
+
$counter = $reports[0][$metric]
|
|
234
|
+
$name = if ($metric -eq 'line') { 'LineCoverage' } else { 'BranchCoverage' }
|
|
235
|
+
if ($null -ne $counter) {
|
|
236
|
+
$aggregate.$name = if ($counter.Total) { [math]::Round(100.0 * $counter.Covered / $counter.Total, 1) } else { $null }
|
|
237
|
+
$aggregate."${name}Status" = if ($counter.Total) { 'exact' } else { 'not-applicable' }
|
|
238
|
+
}
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
if (-not $result.Count) {
|
|
243
|
+
Add-CoverageDiagnostic $diagnostics 'METHODS_UNAVAILABLE' 'The report has no method-level coverage evidence.' ''
|
|
244
|
+
}
|
|
245
|
+
return [pscustomobject]@{
|
|
246
|
+
Methods = @($result | Sort-Object -CaseSensitive Module, File, Class, Method, Signature)
|
|
247
|
+
Inputs = @($inputs)
|
|
248
|
+
Diagnostics = $diagnostics
|
|
249
|
+
Summary = [ordered]@{
|
|
250
|
+
overall_line_coverage = $aggregate.LineCoverage
|
|
251
|
+
overall_line_coverage_status = $aggregate.LineCoverageStatus
|
|
252
|
+
overall_branch_coverage = $aggregate.BranchCoverage
|
|
253
|
+
overall_branch_coverage_status = $aggregate.BranchCoverageStatus
|
|
254
|
+
coverage_scope = $aggregateScope
|
|
255
|
+
method_line_coverage = (Get-CoverageMeasure $methodLines).LineCoverage
|
|
256
|
+
total_methods = $result.Count
|
|
257
|
+
}
|
|
258
|
+
}
|
|
259
|
+
}
|
|
260
|
+
|
|
261
|
+
function Write-CoverageJson([string]$Tool, [string]$Status, $Scope, $Inputs, $Diagnostics, $Data) {
|
|
262
|
+
[ordered]@{
|
|
263
|
+
schema_version = '1'; tool = $Tool; tool_version = '1.0'; status = $Status
|
|
264
|
+
scope = $Scope; inputs = @($Inputs); diagnostics = @($Diagnostics); data = $Data
|
|
265
|
+
} | ConvertTo-Json -Depth 30
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
function Write-CoverageDiagnostics($Diagnostics) {
|
|
269
|
+
foreach ($diagnostic in $Diagnostics) {
|
|
270
|
+
[Console]::Error.WriteLine("$($diagnostic.code): $($diagnostic.message)")
|
|
271
|
+
}
|
|
272
|
+
}
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
<#
|
|
2
|
+
.SYNOPSIS
|
|
3
|
+
Extract Cobertura method coverage without inventing missing measurements.
|
|
4
|
+
.DESCRIPTION
|
|
5
|
+
Legacy output is a JSON array; diagnostics and summaries go to stderr.
|
|
6
|
+
Use -OutputFormat json-v1 for provenance, null/unknown metrics and branch bounds.
|
|
7
|
+
Merging is scoped to reported module/file/class/signature identities, not an
|
|
8
|
+
attestation of source revision. Duplicate report bytes are idempotent.
|
|
9
|
+
Exit codes: 0 complete, 2 invalid input, 3 incomplete/unrepresentable evidence.
|
|
10
|
+
#>
|
|
11
|
+
param(
|
|
12
|
+
[Parameter(Mandatory=$true)][string[]]$CoberturaPath,
|
|
13
|
+
[ValidateRange(0,100)][int]$CoverageThreshold = 80,
|
|
14
|
+
[ValidateRange(0,100)][int]$BranchThreshold = 70,
|
|
15
|
+
[ValidateSet('uncovered', 'below-threshold', 'all')][string]$Filter = 'all',
|
|
16
|
+
[ValidateSet('legacy', 'json-v1')][string]$OutputFormat = 'legacy'
|
|
17
|
+
)
|
|
18
|
+
|
|
19
|
+
try {
|
|
20
|
+
. "$PSScriptRoot\CoverageData.ps1"
|
|
21
|
+
} catch {
|
|
22
|
+
$diagnostic = @{ code = 'DEPENDENCY_UNAVAILABLE'; severity = 'error'; message = $_.Exception.Message; path = "$PSScriptRoot\CoverageData.ps1" }
|
|
23
|
+
[Console]::Error.WriteLine("$($diagnostic.code): $($diagnostic.message)")
|
|
24
|
+
if ($OutputFormat -eq 'json-v1') {
|
|
25
|
+
@{
|
|
26
|
+
schema_version = '1'; tool = 'Extract-MethodCoverage'; tool_version = '1.0'; status = 'error'
|
|
27
|
+
scope = @{}; inputs = @(); diagnostics = @($diagnostic); data = @{}
|
|
28
|
+
} | ConvertTo-Json -Depth 5
|
|
29
|
+
}
|
|
30
|
+
exit 2
|
|
31
|
+
}
|
|
32
|
+
$scope = @{ identity = 'reported-module-file-class-signature'; source_revision = $null; filter = $Filter }
|
|
33
|
+
try {
|
|
34
|
+
$coverage = Get-CoverageData $CoberturaPath
|
|
35
|
+
} catch {
|
|
36
|
+
$diagnostics = @([pscustomobject]@{ code = 'INVALID_INPUT'; severity = 'error'; message = $_.Exception.Message })
|
|
37
|
+
Write-CoverageDiagnostics $diagnostics
|
|
38
|
+
if ($OutputFormat -eq 'json-v1') { Write-CoverageJson 'Extract-MethodCoverage' 'error' $scope @() $diagnostics @{} }
|
|
39
|
+
exit 2
|
|
40
|
+
}
|
|
41
|
+
$partial = -not $coverage.Methods.Count -or @($coverage.Methods | Where-Object {
|
|
42
|
+
$_.LineCoverageStatus -eq 'unknown' -or $_.BranchCoverageStatus -in @('unknown', 'bounded') -or $_.ComplexityStatus -eq 'unknown'
|
|
43
|
+
}).Count -gt 0
|
|
44
|
+
$methods = @($coverage.Methods | Where-Object {
|
|
45
|
+
if ($Filter -eq 'all') { return $true }
|
|
46
|
+
if ($_.LineCoverageStatus -eq 'unknown' -or $_.BranchCoverageStatus -in @('unknown', 'bounded')) { return $true }
|
|
47
|
+
if ($Filter -eq 'uncovered') { return $_.LineCoverage -eq 0 }
|
|
48
|
+
return (100.0 * $_.CoveredLines / $_.TotalLines) -lt $CoverageThreshold -or
|
|
49
|
+
($_.TotalBranches -gt 0 -and (100.0 * $_.CoveredBranches / $_.TotalBranches) -lt $BranchThreshold)
|
|
50
|
+
} | Sort-Object -Property @{Expression='UncoveredLines';Descending=$true}, @{Expression='LineCoverage';Descending=$false}, Class, Method, File, Signature)
|
|
51
|
+
Write-CoverageDiagnostics $coverage.Diagnostics
|
|
52
|
+
if ($OutputFormat -eq 'json-v1') {
|
|
53
|
+
Write-CoverageJson 'Extract-MethodCoverage' $(if ($partial) { 'partial' } else { 'complete' }) $scope $coverage.Inputs $coverage.Diagnostics @{
|
|
54
|
+
methods = $methods; summary = $coverage.Summary
|
|
55
|
+
}
|
|
56
|
+
} elseif (-not $partial) {
|
|
57
|
+
$legacy = @($methods | Select-Object Class, Method, Signature, File, Complexity, LineCoverage,
|
|
58
|
+
@{Name='BranchCoverage';Expression={if ($_.BranchCoverageStatus -eq 'not-applicable') { 0 } else { $_.BranchCoverage }}},
|
|
59
|
+
CoveredLines, TotalLines, UncoveredLines, CoveredBranches, TotalBranches)
|
|
60
|
+
ConvertTo-Json -InputObject $legacy -Depth 5
|
|
61
|
+
[Console]::Error.WriteLine("METHODS_FILTERED:$($methods.Count)")
|
|
62
|
+
[Console]::Error.WriteLine("UNCOVERED_METHODS:$(@($methods | Where-Object { $_.LineCoverageStatus -eq 'exact' -and $_.LineCoverage -eq 0 }).Count)")
|
|
63
|
+
}
|
|
64
|
+
if ($partial) { exit 3 }
|
|
65
|
+
exit 0
|
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: cratis-ci-pipeline-performance
|
|
3
|
+
description: Make a Cratis repository's GitHub Actions CI/CD faster and cheaper by measuring workflow timings first, picking the lever that fits the measured bottleneck (build once and shard, caching, path filters with an aggregator, concurrency, runner routing, publish from artifacts) and verifying before and after. Use when a workflow is slow, queues, burns runner minutes, reruns or cancels a lot, or when editing .github/workflows to speed up CI. Not for authoring a release (use cratis-governed-release-methodology), recurring regressions on one surface (use cratis-engineering-regression-cluster), runtime performance of application code (use cratis-performance-review), or fixing a red build.
|
|
4
|
+
license: LICENSE
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Measure the workflow, then pick the lever
|
|
8
|
+
|
|
9
|
+
A workflow is slow for one of four different reasons, and each has a different
|
|
10
|
+
fix. Adding runners cures a queue and does nothing for a serial build; a faster
|
|
11
|
+
build does nothing for a workflow that fails a quarter of the time. Measure,
|
|
12
|
+
classify, change one lever, measure again. Never optimize from the YAML alone.
|
|
13
|
+
|
|
14
|
+
Capacity, timeout, concurrency and runner invariants live in
|
|
15
|
+
[github-actions.md](../../rules/github-actions.md); obey them and do not restate them in a change.
|
|
16
|
+
Workflow-only pull requests carry the `no-release` label per
|
|
17
|
+
[pull-requests.md](../../rules/pull-requests.md).
|
|
18
|
+
|
|
19
|
+
## 1. Measure first
|
|
20
|
+
|
|
21
|
+
Execute the bundled script (needs `gh` authenticated for the repository, and `jq`):
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
bash scripts/ci-timings.sh --repo OWNER/REPO --workflow ci.yml --limit 50 --jobs 10
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Add `--branch main` or `--event pull_request` to compare populations; PR and
|
|
28
|
+
default-branch runs behave differently. `bash scripts/ci-timings.sh --self-test`
|
|
29
|
+
proves the script itself works without network access. Exit codes: `0` report
|
|
30
|
+
printed, `2` usage error, `3` no runs or no data, `4` missing `gh`/`jq` or `gh`
|
|
31
|
+
not authenticated. Exit `3` and `4` are an unknown, not a clean workflow: say the
|
|
32
|
+
measurement was not made.
|
|
33
|
+
|
|
34
|
+
The report separates **wall clock** (what a developer waits for) from
|
|
35
|
+
**runner-minutes** (what the organization pays for or occupies). A change can
|
|
36
|
+
cut one and raise the other; report both. It prints a one-line classification:
|
|
37
|
+
|
|
38
|
+
| Class | Hint threshold | Meaning | First lever |
|
|
39
|
+
|---|---|---|---|
|
|
40
|
+
| failure-bound | failed or rerun >= 15% of completed runs | Time and minutes go into runs that are redone | Flaky and hung jobs; fail fast |
|
|
41
|
+
| cancel-bound | cancelled >= 20% | Superseded work still ran heavy legs | Cheap gate first, path filters, smaller PR matrix |
|
|
42
|
+
| queue-bound | median longest job wait >= 25% of wall | Jobs wait for runners, not for work | Fewer, bigger jobs; runner routing |
|
|
43
|
+
| execution-bound | none of the above | The critical path is serial work | Build once and shard; caching |
|
|
44
|
+
|
|
45
|
+
The thresholds are a hint, not a verdict. Several classes can hold at once; the
|
|
46
|
+
script lists extras after the primary class. Then find the **critical path over
|
|
47
|
+
the required checks**: the longest chain of `needs` from the start to the last
|
|
48
|
+
required job. Only work on that chain shortens wall clock; shaving a parallel
|
|
49
|
+
sibling does not.
|
|
50
|
+
|
|
51
|
+
Without `gh` access, read the workflow YAML, state that the result is
|
|
52
|
+
**static-only analysis**, and rank findings as hypotheses to measure later.
|
|
53
|
+
|
|
54
|
+
## 2. Guardrails before proposing a fix
|
|
55
|
+
|
|
56
|
+
- **Never hide required validation.** Release, schema, migration, analyzer and
|
|
57
|
+
shared-library checks stay on the path that gates a merge or a release. Moving
|
|
58
|
+
them to a schedule needs an explicit owner decision, not an optimization.
|
|
59
|
+
- **Say what less parallelism costs.** Merging jobs lowers queue and setup cost
|
|
60
|
+
but lengthens the critical path when the merged work was parallel. State the
|
|
61
|
+
expected wall change and runner-minute change together.
|
|
62
|
+
- **Separate repository changes from organization settings.** Workflow YAML is a
|
|
63
|
+
repository change. Runner capacity, scale sets and plan limits are not; report
|
|
64
|
+
them as a request to the owner and do not work around them in YAML.
|
|
65
|
+
- **One lever per change.** Two levers at once make the measured difference
|
|
66
|
+
unattributable.
|
|
67
|
+
- **Rank by what it saves.** Estimate minutes per day as (saving per run) x
|
|
68
|
+
(runs per day) from the report, and take the top one to three items.
|
|
69
|
+
|
|
70
|
+
## 3. Lever catalogue
|
|
71
|
+
|
|
72
|
+
Each lever lists when it fits, the Cratis default and the main pitfall. Read the
|
|
73
|
+
named reference before applying one.
|
|
74
|
+
|
|
75
|
+
**Build once, shard the tests.** Fits execution-bound workflows where build and
|
|
76
|
+
tests run serially or several jobs each rebuild. Default: one job builds in
|
|
77
|
+
`Release`, uploads the output with `actions/upload-artifact`, and a matrix of shard jobs downloads it and runs
|
|
78
|
+
`dotnet test --no-build`. Pitfall: a shard whose filter matches nothing still
|
|
79
|
+
passes, so assert that every shard ran tests. Read
|
|
80
|
+
`references/build-once-and-shard.md` when applying or reviewing this.
|
|
81
|
+
|
|
82
|
+
**Fewer, bigger jobs.** Fits queue-bound workflows: every job pays a scheduler
|
|
83
|
+
start, checkout and toolchain setup. Default: merge small sibling jobs that share
|
|
84
|
+
a setup; keep shards only where the work is long enough to repay a start. Pitfall:
|
|
85
|
+
merging the long pole into a serial chain. Read `references/build-once-and-shard.md`.
|
|
86
|
+
|
|
87
|
+
**Dependency caching.** Fits any job with a measurable restore or install step.
|
|
88
|
+
Default: the shared `Cratis/Workflows` composite actions `dotnet-cache` and
|
|
89
|
+
`yarn-cache` (restore on pull requests, save on the default branch), NuGet lock files and
|
|
90
|
+
`yarn install --immutable`. Pitfall: a cache key that changes every run, or a
|
|
91
|
+
different path list per workflow, which creates separate caches. Read
|
|
92
|
+
`references/caching.md`.
|
|
93
|
+
|
|
94
|
+
**Docker layer cache.** Fits image builds that repeat unchanged layers. Default:
|
|
95
|
+
`docker/build-push-action` with `cache-from`/`cache-to` of type `gha` and a custom
|
|
96
|
+
`scope` per image and platform. Pitfall: the default scope is shared, so builds
|
|
97
|
+
overwrite each other's cache. Read `references/caching.md`.
|
|
98
|
+
|
|
99
|
+
**Path filters with an always-run aggregator.** Fits cancel-bound workflows and
|
|
100
|
+
documentation-only changes. Default: filter heavy jobs by changed paths, and make
|
|
101
|
+
a single aggregator job with `if: always()` check every `needs.*.result`; require
|
|
102
|
+
only the aggregator. Pitfall: a required check skipped by `paths:` never reports
|
|
103
|
+
and the pull request hangs. Read `references/required-checks-and-path-filters.md`.
|
|
104
|
+
|
|
105
|
+
**Concurrency.** Pull-request verification uses `cancel-in-progress: true` keyed
|
|
106
|
+
by workflow and ref. Publish never cancels; where releases share one group, `queue:
|
|
107
|
+
max` keeps up to 100 pending runs in order instead of replacing the single pending
|
|
108
|
+
one. Pitfall: leaving the single pending slot where a replaced merge's run alone
|
|
109
|
+
would have released something (merges into the default branch share a group, also
|
|
110
|
+
when keyed on `github.ref` for merged `pull_request` `closed` events). A workflow
|
|
111
|
+
whose release tooling aggregates since the last release may keep one pending slot,
|
|
112
|
+
documented. Read `references/publish-pipelines.md`.
|
|
113
|
+
|
|
114
|
+
**Runner routing.** Fits queue-bound workflows. Default: private-repository jobs
|
|
115
|
+
on the organization scale set `cratis-arc` or a repository variable that falls
|
|
116
|
+
back to a hosted label, hosted runners for public repositories. Pitfall: never
|
|
117
|
+
route `pull_request` jobs of a public repository to self-hosted runners; that is
|
|
118
|
+
fork code on our infrastructure. The rule owns the details.
|
|
119
|
+
|
|
120
|
+
**Publish from tested artifacts.** Fits release workflows that rebuild what the
|
|
121
|
+
pull-request run already built and tested. Default: build and pack once, then
|
|
122
|
+
publish exactly those packages. Pitfall: a second build can differ from the
|
|
123
|
+
tested one. Read `references/publish-pipelines.md`.
|
|
124
|
+
|
|
125
|
+
**Fail fast on cheap gates.** Fits failure-bound and cancel-bound workflows.
|
|
126
|
+
Default: formatting, lint and unit-level checks run first, and expensive legs
|
|
127
|
+
depend on them with `needs`. Pitfall: serializing checks that were parallel; keep
|
|
128
|
+
the cheap gate short enough to repay its place on the critical path.
|
|
129
|
+
|
|
130
|
+
**Expensive matrices off the PR path.** Fits cancel-bound workflows with a wide
|
|
131
|
+
OS or framework matrix. Default: full matrix on `schedule` and
|
|
132
|
+
`workflow_dispatch`, a Linux-only representative smoke on pull requests (the
|
|
133
|
+
rule states this). Pitfall: the full matrix needs an owner who reads its failures.
|
|
134
|
+
|
|
135
|
+
**Flaky and hung jobs.** Fits failure-bound workflows. Default: a
|
|
136
|
+
`timeout-minutes` on every job, `--blame-hang`, retry only a diagnosed flake, and
|
|
137
|
+
quarantine a test with an issue. Pitfall: a blanket rerun hides the defect and
|
|
138
|
+
doubles minutes. Read `references/flaky-and-hung-jobs.md`.
|
|
139
|
+
|
|
140
|
+
**Merge queue.** Fits repositories where passing pull requests break each other
|
|
141
|
+
after merge. Default: add the `merge_group` trigger, and make the aggregator
|
|
142
|
+
handle it. Pitfall: without `merge_group` the required check never reports in the
|
|
143
|
+
queue. Read `references/required-checks-and-path-filters.md`.
|
|
144
|
+
|
|
145
|
+
**Compile once, analyze once.** Fits any workflow where one push compiles the same
|
|
146
|
+
code several times. Default: build once and reuse the output; run analyzers in one
|
|
147
|
+
parallel lint job instead of in every build, and let that job fail early on compile
|
|
148
|
+
and analyzer errors before long test legs. Pitfall: dropping analyzers from the only
|
|
149
|
+
build that consumers enforce. Read `references/test-setup-cost.md`.
|
|
150
|
+
|
|
151
|
+
**Cores per job before job count.** Fits uniformly slow jobs. Default: check `nproc`
|
|
152
|
+
and the runner's resources before sharding; undersized runners make every job slow.
|
|
153
|
+
Pitfall: adding jobs to an undersized pool. Read `references/runner-sizing-and-images.md`.
|
|
154
|
+
|
|
155
|
+
**Hidden test setup.** Fits test lanes whose step time far exceeds the runner's
|
|
156
|
+
reported test time. Default: profile the test host and fix per-class setup upstream
|
|
157
|
+
rather than sharding it. Pitfall: sharding that divides the waste. Read
|
|
158
|
+
`references/test-setup-cost.md`.
|
|
159
|
+
|
|
160
|
+
**Test a guarantee where it can break.** Fits suites that re-test a dependency.
|
|
161
|
+
Default: run them upstream, or on a dependency pin change or nightly; use snapshots
|
|
162
|
+
for shape and post-deploy synthetic checks where possible. Pitfall: relocating the
|
|
163
|
+
only check of a required guarantee. Read `references/test-setup-cost.md`.
|
|
164
|
+
|
|
165
|
+
**Images and runner contents.** Fits jobs that pull unused images or install tools per
|
|
166
|
+
run. Default: pull only what the job uses, in parallel, and bake tools into the runner
|
|
167
|
+
image; content-addressed caches before warm volumes, publish stays cold. Read
|
|
168
|
+
`references/runner-sizing-and-images.md`.
|
|
169
|
+
|
|
170
|
+
Speed work must not weaken supply-chain hygiene. Read
|
|
171
|
+
`references/security-and-pinning.md` when a change adds or updates an action,
|
|
172
|
+
touches `permissions:`, uses `pull_request_target`, or adds a cache.
|
|
173
|
+
|
|
174
|
+
## 4. Verify
|
|
175
|
+
|
|
176
|
+
1. Re-run the same `ci-timings.sh` command on the same workflow after enough new
|
|
177
|
+
runs exist (at least the sample size of the baseline; say so when fewer).
|
|
178
|
+
2. Report expected against measured savings, wall and runner-minutes separately,
|
|
179
|
+
and whether the classification moved. A saving that is not measured is labeled
|
|
180
|
+
as expected.
|
|
181
|
+
3. Run the linters locally on every edited workflow when installed: `actionlint`
|
|
182
|
+
(syntax, expressions, embedded shell) and `zizmor` (security audits such as
|
|
183
|
+
`unpinned-uses` and `excessive-permissions`). Pass means exit `0` with no
|
|
184
|
+
findings; a tool that is not installed is reported as not run, not as clean.
|
|
185
|
+
Do not assume CI runs either.
|
|
186
|
+
4. Compare required check names before and after; a renamed job silently removes
|
|
187
|
+
a required check. See `references/required-checks-and-path-filters.md`.
|
|
188
|
+
5. Confirm the workflow change ships as `no-release`.
|
|
189
|
+
|
|
190
|
+
## References
|
|
191
|
+
|
|
192
|
+
- Read `references/build-once-and-shard.md` when changing how a build is shared across jobs or how tests are split.
|
|
193
|
+
- Read `references/caching.md` when adding or changing a dependency, Docker or build cache.
|
|
194
|
+
- Read `references/required-checks-and-path-filters.md` when adding path filters, an aggregator, or `merge_group`.
|
|
195
|
+
- Read `references/publish-pipelines.md` when a release or deploy workflow queues, drops runs or rebuilds.
|
|
196
|
+
- Read `references/flaky-and-hung-jobs.md` when runs fail, hang or are rerun.
|
|
197
|
+
- Read `references/test-setup-cost.md` when test steps outlast reported test time, tests duplicate a dependency's suite, or compiles repeat per push.
|
|
198
|
+
- Read `references/runner-sizing-and-images.md` when jobs are uniformly slow, pull images, install tools or reuse workspaces.
|
|
199
|
+
- Read `references/security-and-pinning.md` when a change touches actions, permissions, triggers or caches.
|
|
200
|
+
- Read `references/sources.md` when attributing or extending the method.
|
|
201
|
+
- Execute `scripts/ci-timings.sh` for every measurement; read it only to change it. Its self-test reads the synthetic inputs `scripts/fixtures/runs.json` and `scripts/fixtures/jobs/1003.json`, `scripts/fixtures/jobs/1007.json`, `scripts/fixtures/jobs/1010.json`; edit them only together with the expected lines in the script.
|