@prism-d1/cli 1.0.79 → 1.0.81

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/dist/assets/github-workflows/prism-ai-metrics.yml +83 -122
  2. package/dist/assets/github-workflows/prism-eval-gate-kiro.yml +66 -0
  3. package/dist/assets/github-workflows/prism-eval-gate.yml +1 -1
  4. package/dist/assets/gitlab-workflows/.gitlab-ci.yml +0 -3
  5. package/dist/assets/gitlab-workflows/prism-ai-metrics.yml +53 -56
  6. package/dist/assets/infra/bin/app.ts +1 -0
  7. package/dist/assets/infra/lib/constructs/otel-collector-construct.ts +4 -0
  8. package/dist/assets/infra/lib/dashboard-stack.ts +378 -1235
  9. package/dist/assets/infra/lib/lambda/event-schema.ts +144 -0
  10. package/dist/assets/infra/lib/lambda/metrics-processor.ts +124 -36
  11. package/dist/assets/infra/lib/lambda/otel-receiver.ts +249 -35
  12. package/dist/assets/infra/lib/lambda/security-remediation-tracker.ts +17 -2
  13. package/dist/assets/infra/lib/lambda/velocity-widget.ts +1483 -0
  14. package/dist/assets/infra/lib/metrics-pipeline-stack.ts +111 -119
  15. package/dist/assets/infra/package.json +2 -1
  16. package/dist/assets/infra/tsconfig.json +9 -2
  17. package/dist/src/commands/assessment/web.d.ts.map +1 -1
  18. package/dist/src/commands/assessment/web.js +25 -2
  19. package/dist/src/commands/assessment/web.js.map +1 -1
  20. package/dist/src/commands/bootstrapper/install-eval-harness.d.ts +8 -2
  21. package/dist/src/commands/bootstrapper/install-eval-harness.d.ts.map +1 -1
  22. package/dist/src/commands/bootstrapper/install-eval-harness.js +46 -36
  23. package/dist/src/commands/bootstrapper/install-eval-harness.js.map +1 -1
  24. package/dist/src/commands/bootstrapper/install-git-hooks.d.ts.map +1 -1
  25. package/dist/src/commands/bootstrapper/install-git-hooks.js +10 -14
  26. package/dist/src/commands/bootstrapper/install-git-hooks.js.map +1 -1
  27. package/dist/src/commands/bootstrapper/install-github-workflows.d.ts.map +1 -1
  28. package/dist/src/commands/bootstrapper/install-github-workflows.js +7 -2
  29. package/dist/src/commands/bootstrapper/install-github-workflows.js.map +1 -1
  30. package/dist/src/commands/bootstrapper/install-gitlab-workflows.d.ts.map +1 -1
  31. package/dist/src/commands/bootstrapper/install-gitlab-workflows.js +7 -1
  32. package/dist/src/commands/bootstrapper/install-gitlab-workflows.js.map +1 -1
  33. package/dist/src/commands/bootstrapper/setup-github-oidc.d.ts.map +1 -1
  34. package/dist/src/commands/bootstrapper/setup-github-oidc.js +50 -27
  35. package/dist/src/commands/bootstrapper/setup-github-oidc.js.map +1 -1
  36. package/dist/src/commands/bootstrapper/setup-gitlab-oidc.d.ts.map +1 -1
  37. package/dist/src/commands/bootstrapper/setup-gitlab-oidc.js +69 -25
  38. package/dist/src/commands/bootstrapper/setup-gitlab-oidc.js.map +1 -1
  39. package/dist/src/commands/bootstrapper/setup-otel-sync.d.ts.map +1 -1
  40. package/dist/src/commands/bootstrapper/setup-otel-sync.js +145 -53
  41. package/dist/src/commands/bootstrapper/setup-otel-sync.js.map +1 -1
  42. package/dist/src/commands/workshop/generate-demo-data.d.ts +41 -2
  43. package/dist/src/commands/workshop/generate-demo-data.d.ts.map +1 -1
  44. package/dist/src/commands/workshop/generate-demo-data.js +391 -167
  45. package/dist/src/commands/workshop/generate-demo-data.js.map +1 -1
  46. package/dist/src/commands/workshop/verify-setup.d.ts.map +1 -1
  47. package/dist/src/commands/workshop/verify-setup.js +75 -14
  48. package/dist/src/commands/workshop/verify-setup.js.map +1 -1
  49. package/dist/src/utils/exec.d.ts +56 -0
  50. package/dist/src/utils/exec.d.ts.map +1 -0
  51. package/dist/src/utils/exec.js +72 -0
  52. package/dist/src/utils/exec.js.map +1 -0
  53. package/dist/src/utils/region.d.ts +50 -0
  54. package/dist/src/utils/region.d.ts.map +1 -0
  55. package/dist/src/utils/region.js +62 -0
  56. package/dist/src/utils/region.js.map +1 -0
  57. package/dist/src/utils/tempfile.d.ts +19 -0
  58. package/dist/src/utils/tempfile.d.ts.map +1 -0
  59. package/dist/src/utils/tempfile.js +32 -0
  60. package/dist/src/utils/tempfile.js.map +1 -0
  61. package/dist/src/utils/validate.d.ts +54 -0
  62. package/dist/src/utils/validate.d.ts.map +1 -0
  63. package/dist/src/utils/validate.js +143 -0
  64. package/dist/src/utils/validate.js.map +1 -0
  65. package/package.json +2 -2
  66. package/dist/assets/eval-harness/README.md +0 -144
  67. package/dist/assets/github-workflows/README.md +0 -110
  68. package/dist/assets/github-workflows/prism-dora-weekly.yml +0 -334
  69. package/dist/assets/gitlab-workflows/prism-dora-weekly.yml +0 -181
  70. package/dist/assets/infra/lib/lambda/defect-correlator.ts +0 -142
  71. package/dist/assets/infra/lib/lambda/spec-to-code-calculator.ts +0 -123
  72. package/dist/assets/metric-hooks/README.md +0 -107
@@ -0,0 +1 @@
1
+ {"version":3,"file":"tempfile.d.ts","sourceRoot":"","sources":["../../../src/utils/tempfile.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;GAgBG;AAMH,wBAAgB,eAAe,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,CAAC,GAAG,CAAC,CAS7F"}
@@ -0,0 +1,32 @@
1
+ /**
2
+ * Private temporary files for handing payloads to external commands.
3
+ *
4
+ * Two reasons this exists rather than writing to a fixed path under /tmp:
5
+ *
6
+ * `writeFileSync` follows symlinks. A predictable name like
7
+ * `/tmp/prism-request.json` can be pre-created by any local user as a symlink
8
+ * to a file the caller owns, and the write lands there instead. The sticky bit
9
+ * does not help — the attacker owns the symlink and we follow it. A fresh
10
+ * `mkdtemp` directory has an unpredictable name and mode 0700, so nothing can
11
+ * be planted inside it.
12
+ *
13
+ * The callback shape exists so the directory is reclaimed on every path. The
14
+ * first version of this helper returned the path and left callers to unlink
15
+ * only the file, which leaked one empty directory per invocation, and unlinked
16
+ * before throwing on failure so error paths leaked the file as well.
17
+ */
18
+ import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
19
+ import { join } from 'node:path';
20
+ import { tmpdir } from 'node:os';
21
+ export function withPrivateTemp(name, content, use) {
22
+ const dir = mkdtempSync(join(tmpdir(), 'prism-'));
23
+ try {
24
+ const file = join(dir, name);
25
+ writeFileSync(file, content, { mode: 0o600 });
26
+ return use(file);
27
+ }
28
+ finally {
29
+ rmSync(dir, { recursive: true, force: true });
30
+ }
31
+ }
32
+ //# sourceMappingURL=tempfile.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"tempfile.js","sourceRoot":"","sources":["../../../src/utils/tempfile.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;GAgBG;AAEH,OAAO,EAAE,WAAW,EAAE,aAAa,EAAE,MAAM,EAAE,MAAM,SAAS,CAAC;AAC7D,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,MAAM,EAAE,MAAM,SAAS,CAAC;AAEjC,MAAM,UAAU,eAAe,CAAI,IAAY,EAAE,OAAe,EAAE,GAAwB;IACxF,MAAM,GAAG,GAAG,WAAW,CAAC,IAAI,CAAC,MAAM,EAAE,EAAE,QAAQ,CAAC,CAAC,CAAC;IAClD,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;QAC7B,aAAa,CAAC,IAAI,EAAE,OAAO,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QAC9C,OAAO,GAAG,CAAC,IAAI,CAAC,CAAC;IACnB,CAAC;YAAS,CAAC;QACT,MAAM,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;IAChD,CAAC;AACH,CAAC"}
@@ -0,0 +1,54 @@
1
+ /**
2
+ * Validation for provider identity values collected at the prompt.
3
+ *
4
+ * These are NOT what prevents shell injection -- run() in ./exec.ts does that by
5
+ * never invoking a shell. Validation is here for two other reasons:
6
+ *
7
+ * 1. IAM role names are built from these values and accept only [\w+=,.@-].
8
+ * A value containing a slash or a space otherwise reaches AWS and comes back
9
+ * as a ValidationException that says nothing about which input was wrong.
10
+ * 2. Several of these values land inside an IAM trust policy condition. A
11
+ * malformed one does not fail loudly -- it produces a policy whose sub claim
12
+ * never matches (nothing can assume the role) or, worse, matches more
13
+ * broadly than intended.
14
+ *
15
+ * Each helper exits the process with a message naming the offending value rather
16
+ * than throwing, matching how the bootstrapper commands report other input
17
+ * errors.
18
+ */
19
+ export declare function validateGithubOwner(value: string, label?: string): string;
20
+ export declare function validateGithubRepo(value: string, label?: string): string;
21
+ /** A single GitLab group or username -- one segment, no slash. */
22
+ export declare function validateGitlabNamespace(value: string, label?: string): string;
23
+ /**
24
+ * A full GitLab project path, which may be nested: `group/subgroup/project`.
25
+ *
26
+ * Slashes are legal here, unlike a GitHub repo name, so this cannot reuse the
27
+ * GitHub validator. Every segment is checked individually; empty segments (from
28
+ * a leading, trailing, or doubled slash) are rejected because they silently
29
+ * change the project_path sub claim.
30
+ */
31
+ export declare function validateGitlabProjectPath(value: string, label?: string): string;
32
+ /**
33
+ * Normalizes a GitLab instance URL and returns the URL and its bare hostname.
34
+ *
35
+ * The hostname is used two ways that both break silently on a malformed value:
36
+ * as the last element of the OIDC provider ARN, and as the prefix of the
37
+ * `<host>:aud` / `<host>:sub` trust policy condition keys. Deriving it with a
38
+ * bare `.replace(/^https?:\/\//, '')` leaves any trailing slash or path in
39
+ * place, so `https://gitlab.com/` yields the condition key `gitlab.com/:sub`,
40
+ * which no token will ever match -- the role simply becomes unassumable, with
41
+ * no error at creation time.
42
+ */
43
+ export declare function normalizeGitlabUrl(value: string, label?: string): {
44
+ url: string;
45
+ host: string;
46
+ };
47
+ /**
48
+ * A provider-assigned numeric id (GitHub owner/repo id, GitLab project id).
49
+ *
50
+ * These reach an IAM trust policy sub claim. A non-numeric value cannot be a
51
+ * real id, and permitting one risks widening the claim rather than failing.
52
+ */
53
+ export declare function validateNumericId(value: string, label?: string): string;
54
+ //# sourceMappingURL=validate.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"validate.d.ts","sourceRoot":"","sources":["../../../src/utils/validate.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAuBH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAAwB,GAAG,MAAM,CAQxF;AAED,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAAoB,GAAG,MAAM,CAQnF;AAgBD,kEAAkE;AAClE,wBAAgB,uBAAuB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAA0B,GAAG,MAAM,CAS9F;AAED;;;;;;;GAOG;AACH,wBAAgB,yBAAyB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAAwB,GAAG,MAAM,CAU9F;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAAwB,GAAG;IAAE,GAAG,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAqB9G;AAID;;;;;GAKG;AACH,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,SAAO,GAAG,MAAM,CAKrE"}
@@ -0,0 +1,143 @@
1
+ /**
2
+ * Validation for provider identity values collected at the prompt.
3
+ *
4
+ * These are NOT what prevents shell injection -- run() in ./exec.ts does that by
5
+ * never invoking a shell. Validation is here for two other reasons:
6
+ *
7
+ * 1. IAM role names are built from these values and accept only [\w+=,.@-].
8
+ * A value containing a slash or a space otherwise reaches AWS and comes back
9
+ * as a ValidationException that says nothing about which input was wrong.
10
+ * 2. Several of these values land inside an IAM trust policy condition. A
11
+ * malformed one does not fail loudly -- it produces a policy whose sub claim
12
+ * never matches (nothing can assume the role) or, worse, matches more
13
+ * broadly than intended.
14
+ *
15
+ * Each helper exits the process with a message naming the offending value rather
16
+ * than throwing, matching how the bootstrapper commands report other input
17
+ * errors.
18
+ */
19
+ function fail(lines) {
20
+ for (const line of lines)
21
+ console.error(line);
22
+ process.exit(1);
23
+ }
24
+ // ---- GitHub ----
25
+ /**
26
+ * GitHub owner (user or org): alphanumerics and hyphens, no leading or trailing
27
+ * hyphen, 39 characters max.
28
+ *
29
+ * Deliberately looser than GitHub's current signup rule, which forbids
30
+ * consecutive hyphens. That rule postdates a lot of accounts, so grandfathered
31
+ * owners containing `--` still exist and IAM accepts them fine. Rejecting one
32
+ * would block a legitimate user for no security gain.
33
+ */
34
+ const GITHUB_OWNER = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?$/;
35
+ /** GitHub repository: alphanumerics plus `.`, `_`, `-`, 100 characters max. */
36
+ const GITHUB_REPO = /^[A-Za-z0-9._-]{1,100}$/;
37
+ export function validateGithubOwner(value, label = 'GitHub username/org') {
38
+ if (!GITHUB_OWNER.test(value)) {
39
+ fail([
40
+ `Error: ${label} "${value}" is not a valid GitHub owner name.`,
41
+ ' Expected letters, digits, and hyphens (max 39 characters), not starting or ending with a hyphen.',
42
+ ]);
43
+ }
44
+ return value;
45
+ }
46
+ export function validateGithubRepo(value, label = 'Repository name') {
47
+ if (!GITHUB_REPO.test(value) || value === '.' || value === '..') {
48
+ fail([
49
+ `Error: ${label} "${value}" is not a valid GitHub repository name.`,
50
+ ' Expected letters, digits, dots, underscores, and hyphens (max 100 characters).',
51
+ ]);
52
+ }
53
+ return value;
54
+ }
55
+ // ---- GitLab ----
56
+ /**
57
+ * One segment of a GitLab namespace or project path.
58
+ *
59
+ * GitLab permits letters, digits, `_`, `.`, `-`, requires the first character to
60
+ * be alphanumeric, and reserves the `.git` and `.atom` suffixes.
61
+ */
62
+ const GITLAB_SEGMENT = /^[A-Za-z0-9][A-Za-z0-9._-]{0,254}$/;
63
+ function validGitlabSegment(seg) {
64
+ return GITLAB_SEGMENT.test(seg) && !/\.(git|atom)$/i.test(seg);
65
+ }
66
+ /** A single GitLab group or username -- one segment, no slash. */
67
+ export function validateGitlabNamespace(value, label = 'GitLab group/username') {
68
+ if (!validGitlabSegment(value)) {
69
+ fail([
70
+ `Error: ${label} "${value}" is not a valid GitLab namespace.`,
71
+ ' Expected a single path segment of letters, digits, dots, underscores, and hyphens.',
72
+ ' Use --global with just the group name, not a full project path.',
73
+ ]);
74
+ }
75
+ return value;
76
+ }
77
+ /**
78
+ * A full GitLab project path, which may be nested: `group/subgroup/project`.
79
+ *
80
+ * Slashes are legal here, unlike a GitHub repo name, so this cannot reuse the
81
+ * GitHub validator. Every segment is checked individually; empty segments (from
82
+ * a leading, trailing, or doubled slash) are rejected because they silently
83
+ * change the project_path sub claim.
84
+ */
85
+ export function validateGitlabProjectPath(value, label = 'GitLab project path') {
86
+ const segments = value.split('/');
87
+ if (segments.length < 2 || !segments.every(validGitlabSegment)) {
88
+ fail([
89
+ `Error: ${label} "${value}" is not a valid GitLab project path.`,
90
+ ' Expected group/project, optionally nested as group/subgroup/project.',
91
+ ' Each segment may contain letters, digits, dots, underscores, and hyphens.',
92
+ ]);
93
+ }
94
+ return value;
95
+ }
96
+ /**
97
+ * Normalizes a GitLab instance URL and returns the URL and its bare hostname.
98
+ *
99
+ * The hostname is used two ways that both break silently on a malformed value:
100
+ * as the last element of the OIDC provider ARN, and as the prefix of the
101
+ * `<host>:aud` / `<host>:sub` trust policy condition keys. Deriving it with a
102
+ * bare `.replace(/^https?:\/\//, '')` leaves any trailing slash or path in
103
+ * place, so `https://gitlab.com/` yields the condition key `gitlab.com/:sub`,
104
+ * which no token will ever match -- the role simply becomes unassumable, with
105
+ * no error at creation time.
106
+ */
107
+ export function normalizeGitlabUrl(value, label = 'GitLab instance URL') {
108
+ let parsed;
109
+ try {
110
+ parsed = new URL(value);
111
+ }
112
+ catch {
113
+ return fail([
114
+ `Error: ${label} "${value}" is not a valid URL.`,
115
+ ' Expected something like https://gitlab.com or https://gitlab.example.com.',
116
+ ]);
117
+ }
118
+ if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') {
119
+ fail([`Error: ${label} must use http or https, got "${parsed.protocol}".`]);
120
+ }
121
+ if (parsed.pathname !== '/' && parsed.pathname !== '') {
122
+ fail([
123
+ `Error: ${label} "${value}" must not include a path.`,
124
+ ` Use the instance root, e.g. ${parsed.protocol}//${parsed.host}`,
125
+ ]);
126
+ }
127
+ // GitLab's aud claim is the instance URL with no trailing slash.
128
+ return { url: `${parsed.protocol}//${parsed.host}`, host: parsed.host };
129
+ }
130
+ // ---- Shared ----
131
+ /**
132
+ * A provider-assigned numeric id (GitHub owner/repo id, GitLab project id).
133
+ *
134
+ * These reach an IAM trust policy sub claim. A non-numeric value cannot be a
135
+ * real id, and permitting one risks widening the claim rather than failing.
136
+ */
137
+ export function validateNumericId(value, label = 'ID') {
138
+ if (!/^\d+$/.test(value)) {
139
+ fail([`Error: ${label} "${value}" is not numeric.`]);
140
+ }
141
+ return value;
142
+ }
143
+ //# sourceMappingURL=validate.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"validate.js","sourceRoot":"","sources":["../../../src/utils/validate.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAEH,SAAS,IAAI,CAAC,KAAe;IAC3B,KAAK,MAAM,IAAI,IAAI,KAAK;QAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC9C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC;AAED,mBAAmB;AAEnB;;;;;;;;GAQG;AACH,MAAM,YAAY,GAAG,iDAAiD,CAAC;AAEvE,+EAA+E;AAC/E,MAAM,WAAW,GAAG,yBAAyB,CAAC;AAE9C,MAAM,UAAU,mBAAmB,CAAC,KAAa,EAAE,KAAK,GAAG,qBAAqB;IAC9E,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;QAC9B,IAAI,CAAC;YACH,UAAU,KAAK,KAAK,KAAK,qCAAqC;YAC9D,oGAAoG;SACrG,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,MAAM,UAAU,kBAAkB,CAAC,KAAa,EAAE,KAAK,GAAG,iBAAiB;IACzE,IAAI,CAAC,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,KAAK,KAAK,GAAG,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;QAChE,IAAI,CAAC;YACH,UAAU,KAAK,KAAK,KAAK,0CAA0C;YACnE,kFAAkF;SACnF,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,mBAAmB;AAEnB;;;;;GAKG;AACH,MAAM,cAAc,GAAG,oCAAoC,CAAC;AAE5D,SAAS,kBAAkB,CAAC,GAAW;IACrC,OAAO,cAAc,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AACjE,CAAC;AAED,kEAAkE;AAClE,MAAM,UAAU,uBAAuB,CAAC,KAAa,EAAE,KAAK,GAAG,uBAAuB;IACpF,IAAI,CAAC,kBAAkB,CAAC,KAAK,CAAC,EAAE,CAAC;QAC/B,IAAI,CAAC;YACH,UAAU,KAAK,KAAK,KAAK,oCAAoC;YAC7D,sFAAsF;YACtF,mEAAmE;SACpE,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,yBAAyB,CAAC,KAAa,EAAE,KAAK,GAAG,qBAAqB;IACpF,MAAM,QAAQ,GAAG,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,kBAAkB,CAAC,EAAE,CAAC;QAC/D,IAAI,CAAC;YACH,UAAU,KAAK,KAAK,KAAK,uCAAuC;YAChE,wEAAwE;YACxE,6EAA6E;SAC9E,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,kBAAkB,CAAC,KAAa,EAAE,KAAK,GAAG,qBAAqB;IAC7E,IAAI,MAAW,CAAC;IAChB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC;IAC1B,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;YACV,UAAU,KAAK,KAAK,KAAK,uBAAuB;YAChD,6EAA6E;SAC9E,CAAC,CAAC;IACL,CAAC;IACD,IAAI,MAAM,CAAC,QAAQ,KAAK,QAAQ,IAAI,MAAM,CAAC,QAAQ,KAAK,OAAO,EAAE,CAAC;QAChE,IAAI,CAAC,CAAC,UAAU,KAAK,iCAAiC,MAAM,CAAC,QAAQ,IAAI,CAAC,CAAC,CAAC;IAC9E,CAAC;IACD,IAAI,MAAM,CAAC,QAAQ,KAAK,GAAG,IAAI,MAAM,CAAC,QAAQ,KAAK,EAAE,EAAE,CAAC;QACtD,IAAI,CAAC;YACH,UAAU,KAAK,KAAK,KAAK,4BAA4B;YACrD,iCAAiC,MAAM,CAAC,QAAQ,KAAK,MAAM,CAAC,IAAI,EAAE;SACnE,CAAC,CAAC;IACL,CAAC;IACD,iEAAiE;IACjE,OAAO,EAAE,GAAG,EAAE,GAAG,MAAM,CAAC,QAAQ,KAAK,MAAM,CAAC,IAAI,EAAE,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC;AAC1E,CAAC;AAED,mBAAmB;AAEnB;;;;;GAKG;AACH,MAAM,UAAU,iBAAiB,CAAC,KAAa,EAAE,KAAK,GAAG,IAAI;IAC3D,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;QACzB,IAAI,CAAC,CAAC,UAAU,KAAK,KAAK,KAAK,mBAAmB,CAAC,CAAC,CAAC;IACvD,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@prism-d1/cli",
3
- "version": "1.0.79",
3
+ "version": "1.0.81",
4
4
  "description": "PRISM D1 Velocity CLI — AI development lifecycle assessment tool",
5
5
  "author": "andklee",
6
6
  "engines": {
@@ -22,7 +22,7 @@
22
22
  "access": "public"
23
23
  },
24
24
  "scripts": {
25
- "build": "npx tsc && mkdir -p dist/assets/infra && cp -r ../bootstrapper/metric-hooks ../bootstrapper/eval-harness ../bootstrapper/github-workflows ../bootstrapper/gitlab-workflows dist/assets/ && cp -r ../infra/bin ../infra/lib ../infra/package.json ../infra/tsconfig.json ../infra/cdk.json dist/assets/infra/",
25
+ "build": "npx tsc && rm -rf dist/assets && mkdir -p dist/assets/infra && cp -r ../bootstrapper/metric-hooks ../bootstrapper/eval-harness ../bootstrapper/github-workflows ../bootstrapper/gitlab-workflows dist/assets/ && cp -r ../infra/bin ../infra/lib ../infra/package.json ../infra/tsconfig.json ../infra/cdk.json dist/assets/infra/",
26
26
  "prepublishOnly": "npm run build",
27
27
  "start": "tsx bin/prism-cli.ts",
28
28
  "dev": "tsx bin/prism-cli.ts"
@@ -1,144 +0,0 @@
1
- # Eval Harness
2
-
3
- Evaluate code quality in CI using either **kiro-cli headless mode** (recommended) or Amazon Bedrock rubrics.
4
-
5
- ## Modes
6
-
7
- | Mode | Install Command | How It Works |
8
- |---|---|---|
9
- | **Kiro** (recommended) | `prism-cli bootstrapper install-eval-harness --mode kiro` | kiro-cli reads files, follows `.kiro/steering/code-review.md`, outputs structured JSON findings |
10
- | **Bedrock** (legacy) | `prism-cli bootstrapper install-eval-harness --with-rubrics` | Per-file rubric scoring via Bedrock API, weighted average client-side |
11
-
12
- ## Kiro Mode
13
-
14
- ```bash
15
- prism-cli bootstrapper install-eval-harness --mode kiro
16
- ```
17
-
18
- This installs:
19
- - `.kiro/steering/code-review.md` — review rules (plain English)
20
- - `.github/workflows/prism-eval-gate.yml` — kiro-cli headless CI workflow
21
-
22
- **Requirements:**
23
- - `KIRO_API_KEY` repository secret (generate at https://app.kiro.dev → Settings → API Keys)
24
- - (Optional) `PRISM_METRICS_ROLE_ARN` for EventBridge metrics + AWS Continuum security scanning
25
-
26
- **How it works:**
27
- 1. PR opened → workflow triggers
28
- 2. kiro-cli reads changed files + the diff via `--trust-all-tools`
29
- 3. Outputs structured JSON: findings with file/line, severity, score
30
- 4. Gate fails if any high-severity finding or score < 0.82
31
- 5. PR comment posted with findings table
32
-
33
- ## Bedrock Mode (Legacy)
34
-
35
- ## Bedrock Mode (Legacy) — Install
36
-
37
- ```bash
38
- # Workshop mode — empty rubrics, create your own
39
- bash prism-cli bootstrapper install-eval-harness
40
-
41
- # Production mode — includes all 5 rubrics
42
- bash prism-cli bootstrapper install-eval-harness --with-rubrics
43
-
44
- # Non-interactive
45
- bash prism-cli bootstrapper install-eval-harness --model us.anthropic.claude-haiku-4-5-20251001-v1:0 --threshold 0.82 --with-rubrics
46
- ```
47
-
48
- This installs into your repo:
49
- - `.prism/.prism/eval-harness/run-eval.sh` — evaluation script
50
- - `.prism/.prism/eval-harness/eval-config.json` — model, threshold, region
51
- - `.prism/.prism/eval-harness/rubrics/` — rubric JSON files
52
- - `.github/workflows/prism-eval-gate.yml` — CI workflow
53
-
54
- ## Usage
55
-
56
- ```bash
57
- # Evaluate a single file
58
- ./.prism/.prism/eval-harness/run-eval.sh .prism/.prism/eval-harness/rubrics/code-quality.json src/handler.ts
59
-
60
- # With a spec file (for spec-compliance rubric)
61
- ./.prism/.prism/eval-harness/run-eval.sh .prism/.prism/eval-harness/rubrics/spec-compliance.json src/api.ts --spec specs/api.md
62
- ```
63
-
64
- ### Output
65
-
66
- ```
67
- correctness: 0.9 — Handles all inputs correctly including edge cases
68
- readability: 0.85 — Clear naming, minor style inconsistency in helper
69
- ...
70
-
71
- Score: 0.8720
72
- Result: PASS
73
- Hallucinations: 0
74
- ```
75
-
76
- Exit codes: `0` = pass, `1` = fail, `2` = error.
77
-
78
- ## Configuration
79
-
80
- `eval-config.json`:
81
-
82
- | Field | Description | Default |
83
- |---|---|---|
84
- | `pass_threshold` | Minimum score to pass (0-1) | `0.82` |
85
- | `eval_model_id` | Bedrock model for evaluation | `us.anthropic.claude-haiku-4-5-20251001-v1:0` |
86
- | `aws_region` | AWS region | `us-west-2` |
87
- | `event_bus` | EventBridge bus name | `prism-d1-metrics` |
88
- | `emit_to_eventbridge` | Emit events (workflow handles this) | `true` |
89
-
90
- ## Rubrics
91
-
92
- Five production rubrics are available:
93
-
94
- | Rubric | Auto-selected when file path matches |
95
- |---|---|
96
- | `code-quality.json` | Default fallback |
97
- | `api-response-quality.json` | `api`, `handler`, `route`, `controller` |
98
- | `agent-quality.json` | `agent`, `assistant`, `orchestrat`, `workflow`, `chain` |
99
- | `security-compliance.json` | `auth`, `security`, `guard`, `policy`, `iam`, `crypto` |
100
- | `spec-compliance.json` | Used when commit has `Spec-Ref:` trailer |
101
-
102
- ### Creating a Custom Rubric
103
-
104
- ```json
105
- {
106
- "rubric_name": "my-rubric",
107
- "criteria": [
108
- {
109
- "name": "criterion_name",
110
- "weight": 0.30,
111
- "description": "What this measures",
112
- "scoring": "How to score 0.0-1.0"
113
- }
114
- ]
115
- }
116
- ```
117
-
118
- Weights must sum to 1.0. The script calculates the weighted average client-side (does not trust the LLM to do math).
119
-
120
- ## CI Workflow
121
-
122
- The `prism-eval-gate.yml` workflow:
123
-
124
- 1. Detects commits with `AI-Origin:` trailers
125
- 2. Identifies changed source files from those commits
126
- 3. Auto-selects a rubric per file based on path
127
- 4. Runs `run-eval.sh` per file
128
- 5. Posts a PR comment with per-file scores
129
- 6. Waits for AWS Security Agent review (if installed)
130
- 7. Emits `prism.d1.eval` event to EventBridge
131
- 8. Fails the check if any file scores below threshold or Security Agent finds issues
132
-
133
- ### Requirements
134
-
135
- - OIDC provider configured for GitHub Actions
136
- - IAM role with `bedrock:InvokeModel` + `events:PutEvents`
137
- - Repository secret `PRISM_METRICS_ROLE_ARN`
138
- - `.prism/config.json` with `team_id`
139
-
140
- ## Uninstall
141
-
142
- ```bash
143
- bash prism-cli bootstrapper install-eval-harness --uninstall
144
- ```
@@ -1,110 +0,0 @@
1
- # GitHub Actions Workflows
2
-
3
- Reusable GitHub Actions workflows for PRISM D1 Velocity metric collection.
4
-
5
- ## Workflows
6
-
7
- | Workflow | Trigger | Purpose |
8
- |---|---|---|
9
- | `prism-ai-metrics.yml` | PR merge to main | Calculates AI-to-merge ratio, token usage, lead time. Emits `prism.d1.pr` + `prism.d1.deploy` events |
10
- | `prism-eval-gate.yml` | PR open/update | Evaluates AI-generated code per-file with auto-selected rubrics, waits for Security Agent, blocks merge on failure |
11
- | `prism-agent-eval.yml` | PR modifying agent code | Runs agent in mock mode, evaluates output with agent-quality rubric |
12
- | `prism-dora-weekly.yml` | Weekly (Monday 09:00 UTC) | Calculates DORA + AI-DORA metrics, emits to EventBridge + CloudWatch |
13
-
14
- ## Setup
15
-
16
- ### 1. Configure AWS OIDC
17
-
18
- ```bash
19
- bash prism-cli bootstrapper setup-github-oidc
20
- ```
21
-
22
- This interactively creates:
23
- - OIDC identity provider for `token.actions.githubusercontent.com`
24
- - IAM role `GitHubActions-<repo>` with trust policy scoped to your repo
25
- - Inline policy with `events:PutEvents` and `bedrock:InvokeModel`
26
-
27
- For the weekly workflow, manually add `cloudwatch:PutMetricData` to the role policy.
28
-
29
- ### 2. Set Repository Secret
30
-
31
- In GitHub: Settings → Secrets and variables → Actions → New repository secret:
32
-
33
- | Secret | Value |
34
- |---|---|
35
- | `PRISM_METRICS_ROLE_ARN` | ARN printed by `setup-github-oidc` |
36
-
37
- ### 3. Install Git Hooks + Config
38
-
39
- ```bash
40
- bash prism-cli bootstrapper install-git-hooks
41
- ```
42
-
43
- Creates `.prism/config.json` with your team ID (read by all workflows).
44
-
45
- ### 4. Install Eval Harness
46
-
47
- ```bash
48
- # Workshop mode — bring your own rubric
49
- bash prism-cli bootstrapper install-eval-harness
50
-
51
- # Production mode — includes all 5 rubrics
52
- bash prism-cli bootstrapper install-eval-harness --with-rubrics
53
- ```
54
-
55
- This copies `.prism/eval-harness/`, `eval-config.json`, rubrics, and the `prism-eval-gate.yml` workflow.
56
-
57
- ### 5. Copy Remaining Workflows
58
-
59
- ```bash
60
- mkdir -p .github/workflows
61
- cp bootstrapper/github-workflows/prism-ai-metrics.yml .github/workflows/
62
- cp bootstrapper/github-workflows/prism-dora-weekly.yml .github/workflows/
63
- # Optional — only if you have agents with --mock support:
64
- cp bootstrapper/github-workflows/prism-agent-eval.yml .github/workflows/
65
- ```
66
-
67
- ## IAM Permissions
68
-
69
- The OIDC role needs:
70
-
71
- | Permission | Used by |
72
- |---|---|
73
- | `events:PutEvents` | All workflows |
74
- | `bedrock:InvokeModel` | eval-gate, agent-eval |
75
- | `cloudwatch:PutMetricData` | dora-weekly |
76
-
77
- ## Customization
78
-
79
- | Setting | How |
80
- |---|---|
81
- | Branch | Edit `branches` in each workflow |
82
- | AWS region | Edit `aws-region` field + EventBridge commands |
83
- | Eval threshold | Edit `.prism/.prism/eval-harness/eval-config.json` → `pass_threshold` |
84
- | Eval model | Edit `.prism/.prism/eval-harness/eval-config.json` → `eval_model_id` |
85
- | Weekly schedule | Edit cron in `prism-dora-weekly.yml` (default: `0 9 * * 1`) |
86
-
87
- ## Events Emitted
88
-
89
- | Detail Type | Source Workflow | Destination |
90
- |---|---|---|
91
- | `prism.d1.pr` | ai-metrics | EventBridge |
92
- | `prism.d1.deploy` | ai-metrics | EventBridge |
93
- | `prism.d1.eval` | eval-gate | EventBridge |
94
- | `prism.d1.agent.eval` | agent-eval | EventBridge |
95
- | `prism.d1.assessment` | dora-weekly | EventBridge |
96
- | `prism.d1.security.code_review` | eval-gate (Security Agent) | EventBridge |
97
- | `AIAdoptionRate`, `SpecCoverage`, tool counts | dora-weekly | CloudWatch |
98
-
99
- All EventBridge events use source `prism.d1.velocity` and bus `prism-d1-metrics`.
100
-
101
- ## Troubleshooting
102
-
103
- | Issue | Solution |
104
- |---|---|
105
- | OIDC auth fails | Verify trust policy `sub` matches `repo:org/repo:*` |
106
- | EventBridge put fails | Check `events:PutEvents` on bus ARN |
107
- | Eval gate always skips | Ensure commits have `AI-Origin:` trailers (install git hooks) |
108
- | Weekly not running | Workflow must exist on default branch; test with `workflow_dispatch` |
109
- | Agent eval skips | No `agent/main.py` found — add `--mock` support to your agent |
110
- | Security Agent timeout | Agent takes 2+ min to start; workflow waits up to 12 min total |