trust-path 0.2.0__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {trust_path-0.2.0 → trust_path-0.3.0}/MANIFEST.in +2 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/PKG-INFO +6 -3
- {trust_path-0.2.0 → trust_path-0.3.0}/README.md +5 -2
- trust_path-0.3.0/benchmark/corpus.json +10 -0
- trust_path-0.3.0/benchmark/expected.json +17 -0
- trust_path-0.3.0/benchmark/run.py +147 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/pyproject.toml +1 -1
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/test_analyze.py +73 -16
- trust_path-0.3.0/tests/test_benchmark.py +98 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path/__init__.py +1 -1
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path/analyze.py +36 -6
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path.egg-info/PKG-INFO +6 -3
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path.egg-info/SOURCES.txt +4 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/.github/workflows/release.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/.github/workflows/test.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/LICENSE +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/setup.cfg +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/__init__.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/agent/.github/workflows/triage.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/head_checkout/.github/workflows/build.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/missing_call/.github/workflows/call.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/pr_title/.github/workflows/greet.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/reusable/.github/workflows/on-comment.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/reusable/.github/workflows/reply.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/safe/.github/workflows/echo-title.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/safe/.github/workflows/label.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/safe/.github/workflows/release.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/safe/.github/workflows/test.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/tags/.github/workflows/a.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/workflow_run_chain/.github/workflows/ci.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/workflow_run_chain/.github/workflows/publish.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/workflow_run_chain/.github/workflows/stage.yml +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/test_cli.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/tests/test_yamlsubset.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path/__main__.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path/cli.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path/yamlsubset.py +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path.egg-info/dependency_links.txt +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path.egg-info/entry_points.txt +0 -0
- {trust_path-0.2.0 → trust_path-0.3.0}/trust_path.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: trust-path
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: List the paths by which untrusted input can reach secrets and publishing credentials in GitHub Actions workflows.
|
|
5
5
|
License-Expression: Apache-2.0
|
|
6
6
|
Project-URL: Homepage, https://github.com/jacob-adm/trust-path
|
|
@@ -71,8 +71,9 @@ pip installs `trust-path` as a console script. Without installing, it runs from
|
|
|
71
71
|
workflow or after a `pull_request` workflow (followed through chains), or when it is a local
|
|
72
72
|
reusable workflow called from such a run.
|
|
73
73
|
3. **Find where input enters each job.** A step enters untrusted input when it uses a context such
|
|
74
|
-
as `github.event.pull_request.title` in `run
|
|
75
|
-
request's head, downloads an artifact from the upstream run,
|
|
74
|
+
as `github.event.pull_request.title` in `run:` or `with:` (directly, or through
|
|
75
|
+
`${{ env.NAME }}`), checks out or fetches the pull request's head, downloads an artifact from the upstream run, expands the changed file names of a pull
|
|
76
|
+
request, or is an AI agent action.
|
|
76
77
|
4. **Collect what it reaches.** The entry step and every later step of the same job are reached.
|
|
77
78
|
trust-path collects the secrets they use and the job's write permissions, `id-token: write`,
|
|
78
79
|
environment and AI agent actions holding a credential.
|
|
@@ -123,6 +124,8 @@ key and can write text into issues that other workflows and agents read. Run `tr
|
|
|
123
124
|
list and the YAML subset.
|
|
124
125
|
- [Command](https://github.com/jacob-adm/trust-path/blob/main/docs/commands.md): output fields, JSON, exit status and an example across workflows.
|
|
125
126
|
- [Limitations and security](https://github.com/jacob-adm/trust-path/blob/main/docs/limitations-and-security.md): what trust-path does not see, read or do.
|
|
127
|
+
- [Benchmark](https://github.com/jacob-adm/trust-path/blob/main/docs/benchmark.md): a repeatable run on pinned public workflows, with a
|
|
128
|
+
per-file comparison against zizmor
|
|
126
129
|
- [Using it with other tools](https://github.com/jacob-adm/trust-path/blob/main/docs/other-tools.md): a tested stack with git surface
|
|
127
130
|
and git-overlap, and what zizmor and poutine cover and what trust-path adds.
|
|
128
131
|
- [Contributing](https://github.com/jacob-adm/trust-path/blob/main/CONTRIBUTING.md): running the tests and contributing a change.
|
|
@@ -50,8 +50,9 @@ pip installs `trust-path` as a console script. Without installing, it runs from
|
|
|
50
50
|
workflow or after a `pull_request` workflow (followed through chains), or when it is a local
|
|
51
51
|
reusable workflow called from such a run.
|
|
52
52
|
3. **Find where input enters each job.** A step enters untrusted input when it uses a context such
|
|
53
|
-
as `github.event.pull_request.title` in `run
|
|
54
|
-
request's head, downloads an artifact from the upstream run,
|
|
53
|
+
as `github.event.pull_request.title` in `run:` or `with:` (directly, or through
|
|
54
|
+
`${{ env.NAME }}`), checks out or fetches the pull request's head, downloads an artifact from the upstream run, expands the changed file names of a pull
|
|
55
|
+
request, or is an AI agent action.
|
|
55
56
|
4. **Collect what it reaches.** The entry step and every later step of the same job are reached.
|
|
56
57
|
trust-path collects the secrets they use and the job's write permissions, `id-token: write`,
|
|
57
58
|
environment and AI agent actions holding a credential.
|
|
@@ -102,6 +103,8 @@ key and can write text into issues that other workflows and agents read. Run `tr
|
|
|
102
103
|
list and the YAML subset.
|
|
103
104
|
- [Command](https://github.com/jacob-adm/trust-path/blob/main/docs/commands.md): output fields, JSON, exit status and an example across workflows.
|
|
104
105
|
- [Limitations and security](https://github.com/jacob-adm/trust-path/blob/main/docs/limitations-and-security.md): what trust-path does not see, read or do.
|
|
106
|
+
- [Benchmark](https://github.com/jacob-adm/trust-path/blob/main/docs/benchmark.md): a repeatable run on pinned public workflows, with a
|
|
107
|
+
per-file comparison against zizmor
|
|
105
108
|
- [Using it with other tools](https://github.com/jacob-adm/trust-path/blob/main/docs/other-tools.md): a tested stack with git surface
|
|
106
109
|
and git-overlap, and what zizmor and poutine cover and what trust-path adds.
|
|
107
110
|
- [Contributing](https://github.com/jacob-adm/trust-path/blob/main/CONTRIBUTING.md): running the tests and contributing a change.
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
2
|
+
"""Run trust-path, and zizmor when it is installed, on pinned commits of
|
|
3
|
+
public repositories and compare what they report per workflow file.
|
|
4
|
+
|
|
5
|
+
python3 benchmark/run.py [--corpus FILE] [--expected FILE] [--cache DIR] [--update]
|
|
6
|
+
|
|
7
|
+
Exit status: 0 the trust-path counts match the expected file, 1 they differ,
|
|
8
|
+
2 a repository could not be fetched or a tool failed.
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
import argparse
|
|
12
|
+
import collections
|
|
13
|
+
import json
|
|
14
|
+
import os
|
|
15
|
+
import re
|
|
16
|
+
import shutil
|
|
17
|
+
import subprocess
|
|
18
|
+
import sys
|
|
19
|
+
import tempfile
|
|
20
|
+
|
|
21
|
+
HERE = os.path.dirname(os.path.abspath(__file__))
|
|
22
|
+
ZIZMOR = ["zizmor", "--offline", "--no-config", "--persona", "auditor", "--format", "json"]
|
|
23
|
+
_WORKFLOW = re.compile(r"workflows/([\w.-]+\.ya?ml)")
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
def category(source):
|
|
27
|
+
"""The kind of entry a trust-path source field names."""
|
|
28
|
+
what = source.rsplit(":", 1)[-1]
|
|
29
|
+
if what == "event text read by agent":
|
|
30
|
+
return "agent"
|
|
31
|
+
if what.endswith("of pull request head"):
|
|
32
|
+
return "head-code"
|
|
33
|
+
if what == "artifact from upstream run":
|
|
34
|
+
return "artifact"
|
|
35
|
+
return "expression"
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def summarise(records, refused):
|
|
39
|
+
"""The counts kept in the expected file for one repository."""
|
|
40
|
+
by = collections.Counter(category(r["source"]) for r in records)
|
|
41
|
+
return {"records": len(records), "refused": refused, "by_category": dict(sorted(by.items()))}
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def injection_files(records):
|
|
45
|
+
"""Workflow files where trust-path reports pasted text or fork code."""
|
|
46
|
+
return set(os.path.basename(r["workflow"]) for r in records
|
|
47
|
+
if category(r["source"]) in ("expression", "head-code"))
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def agent_files(records):
|
|
51
|
+
return set(os.path.basename(r["workflow"]) for r in records
|
|
52
|
+
if category(r["source"]) == "agent")
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
def zizmor_files(findings, ident="template-injection"):
|
|
56
|
+
"""Workflow files where zizmor reports a finding of kind `ident`."""
|
|
57
|
+
out = set()
|
|
58
|
+
for f in findings:
|
|
59
|
+
if f.get("ident") == ident:
|
|
60
|
+
out.update(_WORKFLOW.findall(json.dumps(f.get("locations", []))))
|
|
61
|
+
return out
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def compare(ours, theirs):
|
|
65
|
+
return {"both": len(ours & theirs), "trust-path only": len(ours - theirs),
|
|
66
|
+
"zizmor only": len(theirs - ours)}
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def _run(cmd, cwd=None, ok=(0,)):
|
|
70
|
+
p = subprocess.run(cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
|
71
|
+
universal_newlines=True)
|
|
72
|
+
if p.returncode not in ok:
|
|
73
|
+
raise RuntimeError("%s: exit %d: %s" % (" ".join(cmd[:2]), p.returncode,
|
|
74
|
+
p.stderr.strip()[:200]))
|
|
75
|
+
return p
|
|
76
|
+
|
|
77
|
+
|
|
78
|
+
def fetch(url, commit, dest):
|
|
79
|
+
"""Check out .github/workflows of `url` at `commit` into `dest`."""
|
|
80
|
+
if os.path.isdir(os.path.join(dest, ".github", "workflows")):
|
|
81
|
+
return
|
|
82
|
+
os.makedirs(dest)
|
|
83
|
+
_run(["git", "init", "-q"], dest)
|
|
84
|
+
_run(["git", "fetch", "-q", "--depth", "1", "--filter=blob:none", url, commit], dest)
|
|
85
|
+
_run(["git", "checkout", "-q", "FETCH_HEAD", "--", ".github/workflows"], dest)
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def main(argv=None):
|
|
89
|
+
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
|
90
|
+
parser.add_argument("--corpus", default=os.path.join(HERE, "corpus.json"))
|
|
91
|
+
parser.add_argument("--expected", default=os.path.join(HERE, "expected.json"))
|
|
92
|
+
parser.add_argument("--cache", help="directory to keep the fetched workflows in")
|
|
93
|
+
parser.add_argument("--update", action="store_true", help="rewrite the expected file")
|
|
94
|
+
args = parser.parse_args(argv)
|
|
95
|
+
with open(args.corpus) as f:
|
|
96
|
+
corpus = json.load(f)
|
|
97
|
+
cache = args.cache or tempfile.mkdtemp()
|
|
98
|
+
have_zizmor = shutil.which("zizmor") is not None
|
|
99
|
+
actual = {}
|
|
100
|
+
try:
|
|
101
|
+
if have_zizmor:
|
|
102
|
+
print("zizmor: %s" % _run(["zizmor", "--version"]).stdout.strip())
|
|
103
|
+
else:
|
|
104
|
+
print("zizmor: not installed, comparison skipped")
|
|
105
|
+
for repo in corpus:
|
|
106
|
+
name = repo["url"].rstrip("/").split("/")[-1]
|
|
107
|
+
dest = os.path.join(cache, name + "-" + repo["commit"][:12])
|
|
108
|
+
fetch(repo["url"], repo["commit"], dest)
|
|
109
|
+
p = _run([sys.executable, "-m", "trust_path", "--json", dest],
|
|
110
|
+
cwd=os.path.dirname(HERE), ok=(0, 1, 2))
|
|
111
|
+
records = json.loads(p.stdout)
|
|
112
|
+
refused = len(p.stderr.splitlines())
|
|
113
|
+
actual[name] = summarise(records, refused)
|
|
114
|
+
files = os.listdir(os.path.join(dest, ".github", "workflows"))
|
|
115
|
+
print("\n%s @ %s (%d workflow files)" % (name, repo["commit"][:12], len(files)))
|
|
116
|
+
print(" trust-path: %(records)d records, %(refused)d files refused, "
|
|
117
|
+
"%(by_category)s" % actual[name])
|
|
118
|
+
print(" trust-path: %d files with an agent path" % len(agent_files(records)))
|
|
119
|
+
if have_zizmor:
|
|
120
|
+
findings = json.loads(_run(ZIZMOR + [dest], ok=tuple(range(15))).stdout or "[]")
|
|
121
|
+
print(" injection, by workflow file: %s" % compare(
|
|
122
|
+
injection_files(records), zizmor_files(findings)))
|
|
123
|
+
except (RuntimeError, OSError, ValueError) as e:
|
|
124
|
+
sys.stderr.write("benchmark: %s\n" % e)
|
|
125
|
+
return 2
|
|
126
|
+
finally:
|
|
127
|
+
if not args.cache:
|
|
128
|
+
shutil.rmtree(cache, ignore_errors=True)
|
|
129
|
+
if args.update:
|
|
130
|
+
with open(args.expected, "w") as f:
|
|
131
|
+
json.dump(actual, f, indent=2, sort_keys=True)
|
|
132
|
+
f.write("\n")
|
|
133
|
+
return 0
|
|
134
|
+
with open(args.expected) as f:
|
|
135
|
+
expected = json.load(f)
|
|
136
|
+
if actual != expected:
|
|
137
|
+
print("\nDIFFERENT from %s:" % args.expected)
|
|
138
|
+
for name in sorted(set(actual) | set(expected)):
|
|
139
|
+
if actual.get(name) != expected.get(name):
|
|
140
|
+
print(" %s: expected %s, got %s" % (name, expected.get(name), actual.get(name)))
|
|
141
|
+
return 1
|
|
142
|
+
print("\ntrust-path counts match %s" % os.path.basename(args.expected))
|
|
143
|
+
return 0
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
if __name__ == "__main__":
|
|
147
|
+
sys.exit(main())
|
|
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
|
|
5
5
|
|
|
6
6
|
[project]
|
|
7
7
|
name = "trust-path"
|
|
8
|
-
version = "0.
|
|
8
|
+
version = "0.3.0"
|
|
9
9
|
description = "List the paths by which untrusted input can reach secrets and publishing credentials in GitHub Actions workflows."
|
|
10
10
|
readme = "README.md"
|
|
11
11
|
requires-python = ">=3.9"
|
|
@@ -166,25 +166,82 @@ class SourceTests(unittest.TestCase):
|
|
|
166
166
|
""")
|
|
167
167
|
self.assertEqual(recs, [])
|
|
168
168
|
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
169
|
+
ENV = """
|
|
170
|
+
on: issues
|
|
171
|
+
permissions: {}
|
|
172
|
+
jobs:
|
|
173
|
+
j:
|
|
174
|
+
runs-on: ubuntu-latest
|
|
175
|
+
env:
|
|
176
|
+
TITLE: ${{ github.event.issue.title }}
|
|
177
|
+
steps:
|
|
178
|
+
- name: first
|
|
179
|
+
run: %s
|
|
180
|
+
- name: second
|
|
181
|
+
run: %s
|
|
176
182
|
env:
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
183
|
+
T: ${{ secrets.T }}%s
|
|
184
|
+
"""
|
|
185
|
+
|
|
186
|
+
def test_env_variable_alone_is_not_an_entry(self):
|
|
187
|
+
# The recommended pattern: the text reaches the script as data.
|
|
188
|
+
self.assertEqual(analyse(w=self.ENV % ("true", 'echo "$TITLE"', "")), [])
|
|
189
|
+
self.assertEqual(analyse(w=self.ENV % (
|
|
190
|
+
"true", 'echo "$BODY"', "\n BODY: ${{ github.event.issue.body }}")), [])
|
|
191
|
+
|
|
192
|
+
def test_env_expression_is_an_entry_where_it_is_expanded(self):
|
|
193
|
+
recs = analyse(w=self.ENV % ("true", "echo ${{ env.TITLE }}", ""))
|
|
186
194
|
self.assertEqual([(r[0], r[3], r[4]) for r in recs],
|
|
187
195
|
[("issues:github.event.issue.title", "second", "secrets.T")])
|
|
196
|
+
recs = analyse(w=self.ENV % ("echo ${{ env.title }}", "./x", ""))
|
|
197
|
+
self.assertEqual([(r[0], r[3], r[4]) for r in recs],
|
|
198
|
+
[("issues:github.event.issue.title", "second", "secrets.T")])
|
|
199
|
+
|
|
200
|
+
def test_step_env_expression_and_override(self):
|
|
201
|
+
recs = analyse(w=self.ENV % (
|
|
202
|
+
"true", "echo ${{ env.BODY }}",
|
|
203
|
+
"\n BODY: ${{ github.event.issue.body }}"))
|
|
204
|
+
self.assertEqual([(r[0], r[3], r[4]) for r in recs],
|
|
205
|
+
[("issues:github.event.issue.body", "second", "secrets.T")])
|
|
206
|
+
# A step that sets the variable to a fixed value no longer carries the title.
|
|
207
|
+
self.assertEqual(analyse(w=self.ENV % (
|
|
208
|
+
"true", "echo ${{ env.TITLE }}", "\n TITLE: fixed")), [])
|
|
209
|
+
|
|
210
|
+
FILES = """
|
|
211
|
+
on: %s
|
|
212
|
+
jobs:
|
|
213
|
+
j:
|
|
214
|
+
runs-on: ubuntu-latest
|
|
215
|
+
steps:
|
|
216
|
+
- id: Changed
|
|
217
|
+
uses: %s
|
|
218
|
+
- name: list
|
|
219
|
+
run: for f in ${{ steps.changed.outputs.%s }}; do echo "$f"; done
|
|
220
|
+
env:
|
|
221
|
+
T: ${{ secrets.T }}%s
|
|
222
|
+
"""
|
|
223
|
+
|
|
224
|
+
def test_changed_file_names_are_an_entry(self):
|
|
225
|
+
for action in ("tj-actions/changed-files@v40", "Ana06/get-changed-files@v2",
|
|
226
|
+
"jitterbit/get-changed-files@v1"):
|
|
227
|
+
recs = analyse(w=self.FILES % ("pull_request_target", action, "all_changed_files", ""))
|
|
228
|
+
self.assertEqual([(r[0], r[3], r[4]) for r in recs], [
|
|
229
|
+
("pull_request_target:changed file names", "list", "permissions:default"),
|
|
230
|
+
("pull_request_target:changed file names", "list", "secrets.T")])
|
|
231
|
+
|
|
232
|
+
def test_changed_file_counts_and_other_steps_are_not(self):
|
|
233
|
+
for output in ("all_changed_files_count", "any_changed", "only_modified"):
|
|
234
|
+
self.assertEqual(analyse(w=self.FILES % (
|
|
235
|
+
"pull_request_target", "tj-actions/changed-files@v40", output, "")), [])
|
|
236
|
+
self.assertEqual(analyse(w=self.FILES % (
|
|
237
|
+
"pull_request_target", "actions/setup-python@v5", "all_changed_files", "")), [])
|
|
238
|
+
|
|
239
|
+
def test_changed_file_names_through_env_are_data(self):
|
|
240
|
+
text = (self.FILES % ("pull_request_target", "tj-actions/changed-files@v40", "x", "")
|
|
241
|
+
).replace("${{ steps.changed.outputs.x }}", '"$FILES"').replace(
|
|
242
|
+
"T: ${{ secrets.T }}",
|
|
243
|
+
"T: ${{ secrets.T }}\n FILES: ${{ steps.changed.outputs.all }}")
|
|
244
|
+
self.assertEqual(analyse(w=text), [])
|
|
188
245
|
|
|
189
246
|
def test_secret_before_entry_is_not_reached(self):
|
|
190
247
|
recs = analyse(w="""
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
import contextlib
|
|
2
|
+
import importlib.util
|
|
3
|
+
import io
|
|
4
|
+
import json
|
|
5
|
+
import os
|
|
6
|
+
import shutil
|
|
7
|
+
import tempfile
|
|
8
|
+
import unittest
|
|
9
|
+
|
|
10
|
+
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
11
|
+
spec = importlib.util.spec_from_file_location(
|
|
12
|
+
"benchmark_run", os.path.join(ROOT, "benchmark", "run.py"))
|
|
13
|
+
bench = importlib.util.module_from_spec(spec)
|
|
14
|
+
spec.loader.exec_module(bench)
|
|
15
|
+
|
|
16
|
+
WORKFLOW = """\
|
|
17
|
+
on: pull_request_target
|
|
18
|
+
jobs:
|
|
19
|
+
j:
|
|
20
|
+
runs-on: ubuntu-latest
|
|
21
|
+
steps:
|
|
22
|
+
- run: echo "${{ github.event.pull_request.title }}"
|
|
23
|
+
env:
|
|
24
|
+
T: ${{ secrets.T }}
|
|
25
|
+
"""
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def rec(source, workflow=".github/workflows/a.yml"):
|
|
29
|
+
return {"source": source, "workflow": workflow}
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
class PureTests(unittest.TestCase):
|
|
33
|
+
def test_category(self):
|
|
34
|
+
self.assertEqual(bench.category("issues:event text read by agent"), "agent")
|
|
35
|
+
self.assertEqual(bench.category("pull_request_target:checkout of pull request head"),
|
|
36
|
+
"head-code")
|
|
37
|
+
self.assertEqual(bench.category("pull_request_target:fetch of pull request head"),
|
|
38
|
+
"head-code")
|
|
39
|
+
self.assertEqual(bench.category("a@b>workflow_run:artifact from upstream run"), "artifact")
|
|
40
|
+
self.assertEqual(bench.category("issues:github.event.issue.title"), "expression")
|
|
41
|
+
|
|
42
|
+
def test_summarise(self):
|
|
43
|
+
records = [rec("issues:github.event.issue.title"), rec("issues:event text read by agent")]
|
|
44
|
+
self.assertEqual(bench.summarise(records, 1), {
|
|
45
|
+
"records": 2, "refused": 1, "by_category": {"agent": 1, "expression": 1}})
|
|
46
|
+
|
|
47
|
+
def test_files_and_compare(self):
|
|
48
|
+
records = [rec("issues:github.event.issue.title", ".github/workflows/a.yml"),
|
|
49
|
+
rec("issues:event text read by agent", ".github/workflows/b.yml")]
|
|
50
|
+
self.assertEqual(bench.injection_files(records), {"a.yml"})
|
|
51
|
+
self.assertEqual(bench.agent_files(records), {"b.yml"})
|
|
52
|
+
findings = [
|
|
53
|
+
{"ident": "template-injection",
|
|
54
|
+
"locations": [{"symbolic": {"key": {"Local": {"given_path": "x/.github/workflows/c.yaml"}}}}]},
|
|
55
|
+
{"ident": "unpinned-uses",
|
|
56
|
+
"locations": [{"symbolic": {"key": {"Local": {"given_path": "x/.github/workflows/a.yml"}}}}]},
|
|
57
|
+
]
|
|
58
|
+
self.assertEqual(bench.zizmor_files(findings), {"c.yaml"})
|
|
59
|
+
self.assertEqual(bench.compare({"a.yml", "b.yml"}, {"b.yml", "c.yml"}),
|
|
60
|
+
{"both": 1, "trust-path only": 1, "zizmor only": 1})
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
class RunTests(unittest.TestCase):
|
|
64
|
+
"""main() on a workflow already in the cache, so nothing is fetched."""
|
|
65
|
+
|
|
66
|
+
def setUp(self):
|
|
67
|
+
self.tmp = tempfile.mkdtemp()
|
|
68
|
+
self.addCleanup(shutil.rmtree, self.tmp)
|
|
69
|
+
commit = "0123456789abcdef0123"
|
|
70
|
+
wdir = os.path.join(self.tmp, "cache", "repo-" + commit[:12], ".github", "workflows")
|
|
71
|
+
os.makedirs(wdir)
|
|
72
|
+
with open(os.path.join(wdir, "w.yml"), "w") as f:
|
|
73
|
+
f.write(WORKFLOW)
|
|
74
|
+
self.corpus = os.path.join(self.tmp, "corpus.json")
|
|
75
|
+
with open(self.corpus, "w") as f:
|
|
76
|
+
json.dump([{"url": "https://example.invalid/owner/repo", "commit": commit}], f)
|
|
77
|
+
self.expected = os.path.join(self.tmp, "expected.json")
|
|
78
|
+
|
|
79
|
+
def run_main(self, *extra):
|
|
80
|
+
out = io.StringIO()
|
|
81
|
+
with contextlib.redirect_stdout(out):
|
|
82
|
+
code = bench.main(["--corpus", self.corpus, "--expected", self.expected,
|
|
83
|
+
"--cache", os.path.join(self.tmp, "cache")] + list(extra))
|
|
84
|
+
return code, out.getvalue()
|
|
85
|
+
|
|
86
|
+
def test_update_then_match_then_differ(self):
|
|
87
|
+
self.assertEqual(self.run_main("--update")[0], 0)
|
|
88
|
+
with open(self.expected) as f:
|
|
89
|
+
self.assertEqual(json.load(f), {"repo": {
|
|
90
|
+
"records": 2, "refused": 0, "by_category": {"expression": 2}}})
|
|
91
|
+
code, out = self.run_main()
|
|
92
|
+
self.assertEqual(code, 0)
|
|
93
|
+
self.assertIn("trust-path counts match", out)
|
|
94
|
+
with open(self.expected, "w") as f:
|
|
95
|
+
json.dump({"repo": {"records": 3, "refused": 0, "by_category": {"expression": 3}}}, f)
|
|
96
|
+
code, out = self.run_main()
|
|
97
|
+
self.assertEqual(code, 1)
|
|
98
|
+
self.assertIn("DIFFERENT", out)
|
|
@@ -90,6 +90,14 @@ _CLAUDE_FLAGS = re.compile(
|
|
|
90
90
|
r"(?:(?:\s+|=)(\"[^\"]*\"|'[^']*'|[^\s-]\S*))?", re.I)
|
|
91
91
|
_CLAUDE_NAMES = {"allowedtools": "allowedTools", "disallowedtools": "disallowedTools"}
|
|
92
92
|
|
|
93
|
+
# Actions whose outputs list the names of the files a pull request changes.
|
|
94
|
+
# The author of the pull request chooses those names.
|
|
95
|
+
FILENAME_ACTIONS = (
|
|
96
|
+
"ana06/get-changed-files",
|
|
97
|
+
"jitterbit/get-changed-files",
|
|
98
|
+
"tj-actions/changed-files",
|
|
99
|
+
)
|
|
100
|
+
|
|
93
101
|
# Actions that download artifacts from another workflow run.
|
|
94
102
|
ARTIFACT_ACTIONS = ("actions/download-artifact", "dawidd6/action-download-artifact")
|
|
95
103
|
|
|
@@ -101,7 +109,7 @@ WRITE_SCOPES = ("actions", "contents", "discussions", "id-token", "issues", "pac
|
|
|
101
109
|
|
|
102
110
|
_EXPR = re.compile(r"\$\{\{(.*?)\}\}", re.S)
|
|
103
111
|
_CONTEXT = re.compile(
|
|
104
|
-
r"(?<![\w.-])(github|inputs|secrets)((?:\.[A-Za-z_*][\w*-]*|\[[^\]]*\])*)")
|
|
112
|
+
r"(?<![\w.-])(github|inputs|secrets|env|steps)((?:\.[A-Za-z_*][\w*-]*|\[[^\]]*\])*)")
|
|
105
113
|
|
|
106
114
|
|
|
107
115
|
def _norm(name):
|
|
@@ -416,16 +424,38 @@ class Repo(object):
|
|
|
416
424
|
job_sinks.add("environment:" + env)
|
|
417
425
|
job_sinks |= allowed(secret_names(shared_env))
|
|
418
426
|
|
|
427
|
+
def tainted(env):
|
|
428
|
+
"""Environment variable name -> the untrusted contexts its value
|
|
429
|
+
expands. A variable only carries the text as data; it becomes an
|
|
430
|
+
entry where a ${{ env.NAME }} expression pastes it into a step."""
|
|
431
|
+
out = {}
|
|
432
|
+
for key in sorted(env) if isinstance(env, dict) else ():
|
|
433
|
+
out[key.upper()] = untrusted(contexts(env[key]), ctx.trigger, ctx.tainted_inputs)
|
|
434
|
+
return out
|
|
435
|
+
|
|
436
|
+
shared = {}
|
|
437
|
+
for env in shared_env:
|
|
438
|
+
shared.update(tainted(env))
|
|
439
|
+
|
|
419
440
|
# Where untrusted input enters the job: (step index, source).
|
|
420
441
|
entries = set()
|
|
421
442
|
details = {}
|
|
422
|
-
if
|
|
423
|
-
|
|
424
|
-
entries.add((0, name))
|
|
443
|
+
listers = set(s["id"].lower() for s in steps if isinstance(s.get("id"), str)
|
|
444
|
+
and _action(s.get("uses")) in FILENAME_ACTIONS)
|
|
425
445
|
for i, step in enumerate(steps):
|
|
426
|
-
|
|
427
|
-
|
|
446
|
+
variables = dict(shared, **tainted(step.get("env")))
|
|
447
|
+
used = contexts([step.get("run"), step.get("with")])
|
|
448
|
+
for name in untrusted(used, ctx.trigger, ctx.tainted_inputs):
|
|
428
449
|
entries.add((i, name))
|
|
450
|
+
for name in used:
|
|
451
|
+
if name.startswith("env."):
|
|
452
|
+
for origin in variables.get(name[4:].upper(), ()):
|
|
453
|
+
entries.add((i, origin))
|
|
454
|
+
part = name.lower().split(".")
|
|
455
|
+
if part[0] == "steps" and len(part) > 3 and part[1] in listers and \
|
|
456
|
+
part[2] == "outputs" and not part[3].endswith("_count") and \
|
|
457
|
+
not part[3].startswith(("any_", "only_")):
|
|
458
|
+
entries.add((i, "changed file names"))
|
|
429
459
|
action = _action(step.get("uses"))
|
|
430
460
|
with_ = step.get("with") if isinstance(step.get("with"), dict) else {}
|
|
431
461
|
head = ctx.trigger not in _REVIEW
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: trust-path
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: List the paths by which untrusted input can reach secrets and publishing credentials in GitHub Actions workflows.
|
|
5
5
|
License-Expression: Apache-2.0
|
|
6
6
|
Project-URL: Homepage, https://github.com/jacob-adm/trust-path
|
|
@@ -71,8 +71,9 @@ pip installs `trust-path` as a console script. Without installing, it runs from
|
|
|
71
71
|
workflow or after a `pull_request` workflow (followed through chains), or when it is a local
|
|
72
72
|
reusable workflow called from such a run.
|
|
73
73
|
3. **Find where input enters each job.** A step enters untrusted input when it uses a context such
|
|
74
|
-
as `github.event.pull_request.title` in `run
|
|
75
|
-
request's head, downloads an artifact from the upstream run,
|
|
74
|
+
as `github.event.pull_request.title` in `run:` or `with:` (directly, or through
|
|
75
|
+
`${{ env.NAME }}`), checks out or fetches the pull request's head, downloads an artifact from the upstream run, expands the changed file names of a pull
|
|
76
|
+
request, or is an AI agent action.
|
|
76
77
|
4. **Collect what it reaches.** The entry step and every later step of the same job are reached.
|
|
77
78
|
trust-path collects the secrets they use and the job's write permissions, `id-token: write`,
|
|
78
79
|
environment and AI agent actions holding a credential.
|
|
@@ -123,6 +124,8 @@ key and can write text into issues that other workflows and agents read. Run `tr
|
|
|
123
124
|
list and the YAML subset.
|
|
124
125
|
- [Command](https://github.com/jacob-adm/trust-path/blob/main/docs/commands.md): output fields, JSON, exit status and an example across workflows.
|
|
125
126
|
- [Limitations and security](https://github.com/jacob-adm/trust-path/blob/main/docs/limitations-and-security.md): what trust-path does not see, read or do.
|
|
127
|
+
- [Benchmark](https://github.com/jacob-adm/trust-path/blob/main/docs/benchmark.md): a repeatable run on pinned public workflows, with a
|
|
128
|
+
per-file comparison against zizmor
|
|
126
129
|
- [Using it with other tools](https://github.com/jacob-adm/trust-path/blob/main/docs/other-tools.md): a tested stack with git surface
|
|
127
130
|
and git-overlap, and what zizmor and poutine cover and what trust-path adds.
|
|
128
131
|
- [Contributing](https://github.com/jacob-adm/trust-path/blob/main/CONTRIBUTING.md): running the tests and contributing a change.
|
|
@@ -4,8 +4,12 @@ README.md
|
|
|
4
4
|
pyproject.toml
|
|
5
5
|
.github/workflows/release.yml
|
|
6
6
|
.github/workflows/test.yml
|
|
7
|
+
benchmark/corpus.json
|
|
8
|
+
benchmark/expected.json
|
|
9
|
+
benchmark/run.py
|
|
7
10
|
tests/__init__.py
|
|
8
11
|
tests/test_analyze.py
|
|
12
|
+
tests/test_benchmark.py
|
|
9
13
|
tests/test_cli.py
|
|
10
14
|
tests/test_yamlsubset.py
|
|
11
15
|
tests/fixtures/agent/.github/workflows/triage.yml
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/head_checkout/.github/workflows/build.yml
RENAMED
|
File without changes
|
{trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/missing_call/.github/workflows/call.yml
RENAMED
|
File without changes
|
|
File without changes
|
{trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/reusable/.github/workflows/on-comment.yml
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/workflow_run_chain/.github/workflows/ci.yml
RENAMED
|
File without changes
|
|
File without changes
|
{trust_path-0.2.0 → trust_path-0.3.0}/tests/fixtures/workflow_run_chain/.github/workflows/stage.yml
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|