raynsis-inspector 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- raynsis_inspector-0.2.0/LICENSE +202 -0
- raynsis_inspector-0.2.0/NOTICE +4 -0
- raynsis_inspector-0.2.0/PKG-INFO +332 -0
- raynsis_inspector-0.2.0/README.md +302 -0
- raynsis_inspector-0.2.0/pyproject.toml +49 -0
- raynsis_inspector-0.2.0/raynsis_inspector/__init__.py +6 -0
- raynsis_inspector-0.2.0/raynsis_inspector/__main__.py +5 -0
- raynsis_inspector-0.2.0/raynsis_inspector/audit.py +169 -0
- raynsis_inspector-0.2.0/raynsis_inspector/cli.py +423 -0
- raynsis_inspector-0.2.0/raynsis_inspector/engine.py +475 -0
- raynsis_inspector-0.2.0/raynsis_inspector/rules/core.json +2207 -0
- raynsis_inspector-0.2.0/raynsis_inspector/rules/tr.json +181 -0
- raynsis_inspector-0.2.0/raynsis_inspector/transcript.py +186 -0
- raynsis_inspector-0.2.0/raynsis_inspector.egg-info/PKG-INFO +332 -0
- raynsis_inspector-0.2.0/raynsis_inspector.egg-info/SOURCES.txt +20 -0
- raynsis_inspector-0.2.0/raynsis_inspector.egg-info/dependency_links.txt +1 -0
- raynsis_inspector-0.2.0/raynsis_inspector.egg-info/entry_points.txt +2 -0
- raynsis_inspector-0.2.0/raynsis_inspector.egg-info/top_level.txt +1 -0
- raynsis_inspector-0.2.0/setup.cfg +4 -0
- raynsis_inspector-0.2.0/tests/test_engine.py +274 -0
- raynsis_inspector-0.2.0/tests/test_features.py +288 -0
- raynsis_inspector-0.2.0/tests/test_security.py +415 -0
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,332 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: raynsis-inspector
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Evidence gates for coding agents: block dangerous tool calls and send back claims the transcript cannot prove.
|
|
5
|
+
Author: RAYNSIS AI Workforce
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/RAYNSIS/inspector
|
|
8
|
+
Project-URL: Issues, https://github.com/RAYNSIS/inspector/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/RAYNSIS/inspector/blob/main/CHANGELOG.md
|
|
10
|
+
Keywords: claude-code,hooks,guardrails,agent-governance,audit,coding-agents,llm
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
23
|
+
Classifier: Topic :: Security
|
|
24
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
25
|
+
Requires-Python: >=3.9
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
License-File: NOTICE
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# RAYNSIS Inspector
|
|
32
|
+
|
|
33
|
+
**Evidence gates for coding agents.**
|
|
34
|
+
|
|
35
|
+
*Published as `kanit` 0.1.0 (PyPI `kanit-gates`) before the rename to RAYNSIS Inspector.*
|
|
36
|
+
|
|
37
|
+
Coding agents say "all tests pass" without running them, cite links they never opened,
|
|
38
|
+
report "deployed" in a turn where no tool ran, and now and then try to edit their own guardrails.
|
|
39
|
+
RAYNSIS Inspector is a small Claude Code plugin that checks two moments:
|
|
40
|
+
|
|
41
|
+
- **Before a tool call** (PreToolUse): a dangerous call is denied, with a reason the agent can act on.
|
|
42
|
+
- **Before a reply is final** (Stop, and SubagentStop for subagents): a reply that claims work the transcript cannot prove is sent back once, with what is missing.
|
|
43
|
+
|
|
44
|
+
Every rule ships with examples it must fire on and examples it must stay quiet on.
|
|
45
|
+
Every check is logged, not only the hits, so you can see how often each rule fires per evaluation.
|
|
46
|
+
|
|
47
|
+
Python 3.9+, standard library only, no network calls, no telemetry.
|
|
48
|
+
|
|
49
|
+
> Status: 0.2.0, alpha. Rule ids and the rules file format may still change before 1.0.
|
|
50
|
+
|
|
51
|
+
## Install
|
|
52
|
+
|
|
53
|
+
In Claude Code (2.1.275 or later):
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
/plugin install raynsis-inspector --marketplace RAYNSIS/inspector
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
On older versions, or from a terminal:
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
claude plugin marketplace add RAYNSIS/inspector
|
|
63
|
+
claude plugin install raynsis-inspector@raynsis-inspector
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
The plugin needs `python3` on your `PATH`. Nothing else is installed.
|
|
67
|
+
|
|
68
|
+
To use the `raynsis-inspector` command outside Claude Code (rule checks, stats):
|
|
69
|
+
|
|
70
|
+
```
|
|
71
|
+
pip install raynsis-inspector
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
The package on PyPI and the command are `raynsis-inspector`; the Python module is `raynsis_inspector`.
|
|
75
|
+
From a clone of this repository, `pip install .` installs the same thing.
|
|
76
|
+
|
|
77
|
+
## What it does
|
|
78
|
+
|
|
79
|
+
A blocked tool call, as the agent sees it:
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
PreToolUse:Bash hook error: RAYNSIS Inspector blocked this call:
|
|
83
|
+
- [tool.no-secret-files] Secret files stay out of the agent's context. Ask the human for the one value you need. (matched: " .env")
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
A reply sent back at Stop, and the agent's next answer (Claude Code 2.1.327, recorded during release testing):
|
|
87
|
+
|
|
88
|
+
```
|
|
89
|
+
Agent: All tests pass.
|
|
90
|
+
RAYNSIS Inspector: this reply claims something the session cannot prove. Fix it before finishing:
|
|
91
|
+
- [reply.tests-pass-needs-run] You said tests pass, but no test command ran
|
|
92
|
+
in this turn. Run them and show the result, or say they were not run.
|
|
93
|
+
(matched: "All tests pass")
|
|
94
|
+
Agent: Tests were not run in this turn, so I can't claim they pass. My earlier
|
|
95
|
+
"All tests pass" reply was unsupported.
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
You can try any rule without Claude Code:
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
$ raynsis-inspector check reply 'Fixed the parser. All tests pass.' --turn 'Edit: {"file_path": "parser.py"}'
|
|
102
|
+
DENY reply.tests-pass-needs-run
|
|
103
|
+
You said tests pass, but no test command ran in this turn. Run them and show the result, or say they were not run.
|
|
104
|
+
matched: "All tests pass"
|
|
105
|
+
|
|
106
|
+
$ raynsis-inspector check reply 'Fixed the parser. All tests pass.' --turn 'Edit: {"file_path": "parser.py"}' --turn 'Bash: {"command": "pytest -q"}'
|
|
107
|
+
pass: no rule fired (18 rules checked)
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
`raynsis-inspector check` exits 1 when a deny rule fires, so it also works in scripts.
|
|
111
|
+
|
|
112
|
+
## Rules in this release
|
|
113
|
+
|
|
114
|
+
| id | on | action | what it stops |
|
|
115
|
+
|---|---|---|---|
|
|
116
|
+
| `tool.no-recursive-delete-home` | tool | deny | This deletes the home or root directory. |
|
|
117
|
+
| `tool.no-disk-wipe` | tool | deny | This writes over a disk or partition and cannot be undone. |
|
|
118
|
+
| `tool.no-force-push` | tool | deny | Force push rewrites shared history. |
|
|
119
|
+
| `tool.no-hook-bypass` | tool | deny | This skips the repository's own checks (git hooks). |
|
|
120
|
+
| `tool.no-package-publish` | tool | deny | Publishing a package, image or release is visible to everyone and cannot be fully taken back. |
|
|
121
|
+
| `tool.no-curl-pipe-shell` | tool | deny | Piping a download straight into a shell or interpreter runs unreviewed code. |
|
|
122
|
+
| `tool.no-secret-files` | tool | deny | Secret files stay out of the agent's context. |
|
|
123
|
+
| `tool.no-secret-in-command` | tool | deny | A live-looking secret is written into this command or file, where shell history, logs or git will keep it. |
|
|
124
|
+
| `tool.guardrails-read-only` | tool | deny | An agent does not edit its own guardrails: hook settings, hook scripts, installed plugins or RAYNSIS Inspector's rules and logs. |
|
|
125
|
+
| `tool.guardrails-read-only-shell` | tool | deny | Writing to guardrail files (hook settings, hook scripts, installed plugins, .raynsis-inspector/) from the shell is blocked. |
|
|
126
|
+
| `tool.guardrails-read-only-mcp` | tool | deny | An agent does not edit its own guardrails through an MCP file tool either. |
|
|
127
|
+
| `tool.no-test-skip` | tool | warn | This edit marks an existing test as skipped or expected to fail. |
|
|
128
|
+
| `reply.tests-pass-needs-run` | reply | deny | You said tests pass, but no test command ran in this turn. |
|
|
129
|
+
| `reply.build-passes-needs-run` | reply | deny | You said the build, type check or lint passes, but none of them ran in this turn. |
|
|
130
|
+
| `reply.done-needs-action` | reply | deny | You claimed finished work, but no tool ran in this turn. |
|
|
131
|
+
| `reply.unopened-link` | reply | deny | This link never appeared in anything you opened or were given this session. |
|
|
132
|
+
| `reply.tr.tests-pass-needs-run` | reply | deny | Testlerin geçtiğini yazdın ama bu turda test komutu çalışmadı. |
|
|
133
|
+
| `reply.tr.done-needs-action` | reply | deny | Bitmiş iş bildirdin ama bu turda hiçbir araç çalışmadı. |
|
|
134
|
+
|
|
135
|
+
The full patterns and their examples are in [`raynsis_inspector/rules/core.json`](raynsis_inspector/rules/core.json) and
|
|
136
|
+
[`raynsis_inspector/rules/tr.json`](raynsis_inspector/rules/tr.json). `raynsis-inspector rules` lists what is active in your project.
|
|
137
|
+
|
|
138
|
+
## How it decides
|
|
139
|
+
|
|
140
|
+
- **Tool rules** match a regex against named fields of the tool input (`command`, `file_path`, `content`, ...).
|
|
141
|
+
An `unless` regex can exempt a match. `"only_added": true` fires only when an edit adds a match
|
|
142
|
+
that the replaced text did not have, so an existing skipped test does not trip `tool.no-test-skip`.
|
|
143
|
+
- **Reply rules** match a claim in the final reply, then look for evidence among the tool calls made
|
|
144
|
+
in this turn (since the last human message). "Tests pass" needs a test command run through a shell
|
|
145
|
+
or a subagent; "the build passes" needs a build, type check or lint command; "I've deployed" needs at
|
|
146
|
+
least one tool call. Only the command counts, not its description, and reading is not running:
|
|
147
|
+
`cat tests/test_api.py`, `ls tests/` or a Read of the file is not a test run, and neither is installing
|
|
148
|
+
or locating a runner (`pip install pytest`, `which pytest`).
|
|
149
|
+
- **What is not a claim.** A question ("Do the tests pass?"), a condition ("If the tests pass, merge it",
|
|
150
|
+
"When CI is green and all tests pass, the bot merges"), an instruction ("Make sure all tests pass"), a
|
|
151
|
+
hedge ("Ideally all tests pass"), a negation ("Not all tests pass yet"), someone else's words ("The
|
|
152
|
+
README says all tests pass") and a phrase in quotation marks (my earlier "All tests pass" was wrong) are
|
|
153
|
+
not checked. A statement stays a claim: "I can confirm all tests pass", "I re-ran pytest and confirm all
|
|
154
|
+
tests pass", "This change ensures that all tests pass" and a claim under a heading line such as
|
|
155
|
+
"## Verify" are checked, and so is a past event: "Once I pushed the fix, CI went green". A question *after* the claim does not excuse it: "All tests pass, want me to
|
|
156
|
+
commit?" and "All tests pass (shall I merge?)" are checked. An `unless` phrase ("not verified") only
|
|
157
|
+
excuses the sentence it is in.
|
|
158
|
+
- **Links** in a reply must have appeared in a human message, a tool result, or the input of a
|
|
159
|
+
read-only tool, anywhere in the session (a link not found in the last 4 MB is looked up in the whole
|
|
160
|
+
transcript). A link the agent only wrote into a file does not count.
|
|
161
|
+
- **A reply is sent back once.** If the next attempt still matches (`stop_hook_active`), RAYNSIS Inspector warns the
|
|
162
|
+
human instead of blocking again, so a session can never loop.
|
|
163
|
+
- **Unknown is not missing.** After a context compaction the start of the turn may be in another file.
|
|
164
|
+
Then the turn's evidence is unknown, and the evidence rules are skipped instead of guessing.
|
|
165
|
+
`raynsis-inspector stats` and `raynsis-inspector doctor` show how often that happened.
|
|
166
|
+
- **Warn rules** do not block. Their message goes to the human, not the agent.
|
|
167
|
+
- **A slow check never becomes a silent pass.** Claude Code lets a call through when a hook times out,
|
|
168
|
+
so RAYNSIS Inspector stops itself first (6 s for a tool call, 14 s for a reply): an unfinished tool check denies
|
|
169
|
+
the call and says why, an unfinished reply check warns the human. Inputs are bounded (rules file 1 MB,
|
|
170
|
+
prior file content 2 MB) and special files (FIFOs, devices) are never read. The packaged patterns are
|
|
171
|
+
linear with no length limit: a scan stops only where its own start repeats, so the worst 60 KB command
|
|
172
|
+
we could build takes under 0.3 s per rule on our test machine, and a long argument list does not hide
|
|
173
|
+
the end of a command. The budget is for project rules that are not linear. The timer needs a POSIX system; on Windows the hook timeout is the only limit.
|
|
174
|
+
|
|
175
|
+
## Configure
|
|
176
|
+
|
|
177
|
+
Project rules live in `.raynsis-inspector/rules.json` at the project root (Claude Code's `CLAUDE_PROJECT_DIR`, so a
|
|
178
|
+
`cd` inside the session does not leave them behind). A rule with a new `id` is added. A rule with the `id` of an
|
|
179
|
+
existing rule changes only the fields you write. `"<field>+"` adds an alternative to a regex instead of
|
|
180
|
+
replacing it, `null` removes a field, and `"disable"` turns rules off.
|
|
181
|
+
|
|
182
|
+
Teach RAYNSIS Inspector your project's test command:
|
|
183
|
+
|
|
184
|
+
```json
|
|
185
|
+
{
|
|
186
|
+
"rules": [
|
|
187
|
+
{"id": "reply.tests-pass-needs-run", "needs_turn+": "make verify|scripts/ci\\.sh"}
|
|
188
|
+
],
|
|
189
|
+
"disable": ["tool.no-package-publish"]
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
A new rule:
|
|
194
|
+
|
|
195
|
+
```json
|
|
196
|
+
{
|
|
197
|
+
"rules": [
|
|
198
|
+
{
|
|
199
|
+
"id": "tool.no-prod-db",
|
|
200
|
+
"event": "tool",
|
|
201
|
+
"action": "deny",
|
|
202
|
+
"tools": ["Bash"],
|
|
203
|
+
"field": "command",
|
|
204
|
+
"pattern": "psql\\s+.*prod",
|
|
205
|
+
"message": "Production data is read through the replica.",
|
|
206
|
+
"examples": {
|
|
207
|
+
"hit": [{"tool": "Bash", "input": {"command": "psql -h prod-db -c 'delete from users'"}}],
|
|
208
|
+
"miss": [{"tool": "Bash", "input": {"command": "psql -h replica -c 'select 1'"}}]
|
|
209
|
+
}
|
|
210
|
+
}
|
|
211
|
+
]
|
|
212
|
+
}
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
| field | meaning |
|
|
216
|
+
|---|---|
|
|
217
|
+
| `id`, `message` | required; the message is what the agent (deny) or the human (warn) reads |
|
|
218
|
+
| `event` | `tool` or `reply` |
|
|
219
|
+
| `action` | `deny` or `warn` |
|
|
220
|
+
| `tools` | tool names the rule applies to (tool rules; default all) |
|
|
221
|
+
| `tool_pattern` | regex on the tool name, for tool families such as MCP servers (`^mcp__`) |
|
|
222
|
+
| `field` | input field or list of fields to match (tool rules; default the whole input as JSON) |
|
|
223
|
+
| `pattern`, `unless` | Python regexes |
|
|
224
|
+
| `needs_turn` | reply rules: the rule fires unless a tool call in this turn matches this regex |
|
|
225
|
+
| `needs_turn_tools` | reply rules: only calls of these tools count as evidence (default all) |
|
|
226
|
+
| `needs_turn_field` | reply rules: only these input fields of those calls are matched (default the whole input); the packaged test and build rules use `command` and `prompt` |
|
|
227
|
+
| `check` | `urls_seen` for the link rule; `unless_url` exempts links |
|
|
228
|
+
| `only_added` | tool rules: fire only when an edit adds a match |
|
|
229
|
+
| `examples` | `hit` and `miss` lists; `raynsis-inspector test-rules` fails a rule without a `hit` example |
|
|
230
|
+
|
|
231
|
+
Run `raynsis-inspector test-rules` after every change. An invalid rule is reported and skipped; the others keep working.
|
|
232
|
+
|
|
233
|
+
A project file can turn a packaged rule off or change it. When it does, RAYNSIS Inspector tells the human once per
|
|
234
|
+
session which rules were weakened, so a cloned repository cannot quietly disarm the gates.
|
|
235
|
+
|
|
236
|
+
Environment variables, set by the human for the Claude Code process:
|
|
237
|
+
|
|
238
|
+
| variable | effect |
|
|
239
|
+
|---|---|
|
|
240
|
+
| `RAYNSIS_INSPECTOR_SHADOW=1` | evaluate and log everything, print and block nothing |
|
|
241
|
+
| `RAYNSIS_INSPECTOR_DISABLE=rule.a,rule.b` | turn rules off for this process |
|
|
242
|
+
| `RAYNSIS_INSPECTOR_FAIL_CLOSED=1` | deny tool calls while the engine is broken (default: allow, with a warning) |
|
|
243
|
+
| `RAYNSIS_INSPECTOR_HOME=/path` | where logs go (default `.raynsis-inspector/` in the project) |
|
|
244
|
+
| `RAYNSIS_INSPECTOR_RULES=/a.json:/b.json` | extra rule files, loaded after the project file |
|
|
245
|
+
|
|
246
|
+
The agent cannot set these variables. The guardrail rules deny its common ways of changing the rest:
|
|
247
|
+
editing, moving or deleting `.raynsis-inspector/`, hook settings, hook scripts or installed plugins, restoring them
|
|
248
|
+
with git, and `claude plugin disable` or `uninstall`. They read command text, so an interpreter
|
|
249
|
+
one-liner that writes the file is not caught (see Known gaps).
|
|
250
|
+
|
|
251
|
+
## Audit and stats
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
.raynsis-inspector/audit.jsonl one line per hit, engine error or invalid rule
|
|
255
|
+
.raynsis-inspector/evals.jsonl one line per evaluation: which rules were checked, which fired
|
|
256
|
+
raynsis-inspector stats checked / raw / unique hits per rule (--window 600)
|
|
257
|
+
raynsis-inspector doctor install, rules and log health
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
`.raynsis-inspector/` gets its own `.gitignore` for the logs; `rules.json` stays committable.
|
|
261
|
+
|
|
262
|
+
**Shadow mode first.** Run `RAYNSIS_INSPECTOR_SHADOW=1` for a week next to your current setup, read `raynsis-inspector stats`,
|
|
263
|
+
then turn enforcement on.
|
|
264
|
+
|
|
265
|
+
## Where it comes from
|
|
266
|
+
|
|
267
|
+
RAYNSIS Inspector is extracted from the governance layer we run on our own agent fleet. Between 19 September
|
|
268
|
+
and 17 November 2026 that engine recorded, across 1,690 agent sessions, **4,298 unique rule events**
|
|
269
|
+
(same session and rule within 10 minutes counted once; 3,888 to 4,505 depending on the window):
|
|
270
|
+
**958 tool calls stopped** and **3,340 replies sent back for correction**. 345 of them were risky
|
|
271
|
+
attempts (spending credits, deleting, reading secrets, sending outside, touching live systems),
|
|
272
|
+
1,139 were accuracy problems, 2,814 were format.
|
|
273
|
+
|
|
274
|
+
Before release we measured false alarms on real traffic:
|
|
275
|
+
|
|
276
|
+
- **Replay.** The rules, replayed over the last 30 days of our sessions (399 transcripts). The first
|
|
277
|
+
replay (17 rules; 4,962 shell commands, 615 file edits, 502 reply endings) gave 17 hits. Reading each
|
|
278
|
+
one: five were the first reply after a context compaction, judged without the evidence from before it;
|
|
279
|
+
one was a conditional `skipTest` in a brand-new test file; the rest were rule test text inside commands.
|
|
280
|
+
After the fixes above ("unknown is not missing", `only_added`) and four rounds of an independent
|
|
281
|
+
pre-release audit, the final replay of all 18 rules over the 383 latest transcripts (our own deliberate
|
|
282
|
+
test sessions left out; 2,319 shell commands, 192 file edits, 410 reply endings) gave 2 hits. One was
|
|
283
|
+
right: a link in a reply that never appeared in its session. One was a false alarm: a heredoc that only
|
|
284
|
+
mentioned a `*.env` file name (the mention gap under Known gaps). The audit fixes changed no result on
|
|
285
|
+
this traffic: the rules before and after them gave the same two hits.
|
|
286
|
+
- **Shadow run.** 24 hours on live sessions, 6,735 evaluations, two hits, both false alarms, both fixed:
|
|
287
|
+
a `>` inside a quoted grep pattern read as a redirect, and a project test script not recognised as a
|
|
288
|
+
test command (now configurable with `needs_turn+`).
|
|
289
|
+
- **End to end.** Claude Code 2.1.327 with the plugin loaded: a `.env` read denied, an unproven
|
|
290
|
+
"All tests pass." sent back and corrected, the same claim from a subagent sent back and corrected,
|
|
291
|
+
a skip added to a test warned.
|
|
292
|
+
|
|
293
|
+
Our fleet already runs about 228 stricter house rules, so few violations reach these 18. Read the
|
|
294
|
+
numbers as a false-alarm measurement, not a catch rate.
|
|
295
|
+
|
|
296
|
+
Two lessons shaped the code:
|
|
297
|
+
|
|
298
|
+
1. **A gate that fails silently is worse than no gate.** On 4 November a rule missing one field crashed
|
|
299
|
+
our engine about 900 times in nine hours; it failed open and nobody saw it. RAYNSIS Inspector skips an invalid
|
|
300
|
+
rule instead of dying, and an engine error is printed on screen (at most every 10 minutes) and logged.
|
|
301
|
+
2. **Hits without a denominator prove nothing.** Our old log only wrote hits, so it cannot show a
|
|
302
|
+
before/after rate. RAYNSIS Inspector writes every evaluation to `evals.jsonl`, so `raynsis-inspector stats` can.
|
|
303
|
+
|
|
304
|
+
## Known gaps
|
|
305
|
+
|
|
306
|
+
- Rules are regexes, not understanding. The reply rules know common English and Turkish phrasings, not
|
|
307
|
+
all of them ("Everything passes now", "I ran the tests and they all pass" and "No type errors." are
|
|
308
|
+
not recognised). Quoted speech after a colon ("You wrote: All tests pass") and a sentence that defines
|
|
309
|
+
a result ("A green badge means all tests pass") are read as claims.
|
|
310
|
+
- Tool rules read the command text. A command that only mentions a dangerous string can trip them
|
|
311
|
+
(a heredoc of notes that says "never run rm -rf ~"), and a determined agent can hide one (`eval`,
|
|
312
|
+
encoding, a script file, a glob or a variable such as `cat .en?` or `rm -rf $d`,
|
|
313
|
+
`python -c "open('.claude/settings.json', 'w')"`). Publishing through a wrapper we do not list (`npx np`) is not caught,
|
|
314
|
+
and a command that deletes a whole tree by design (`git clean -fdx`) also deletes `.raynsis-inspector/` logs.
|
|
315
|
+
The rules catch mistakes; they are not a sandbox. Run untrusted code in a real sandbox.
|
|
316
|
+
- Evidence means a matching tool call ran in this turn, not that it succeeded: a failing `pytest` run
|
|
317
|
+
still counts as "ran". The agent still has to report the result honestly. A subagent whose prompt
|
|
318
|
+
mentions tests counts as a test run, and so does an inline script that only names a test file
|
|
319
|
+
(`python3 -c "print(open('tests/a.py').read())"`); RAYNSIS Inspector does not read what the subagent did.
|
|
320
|
+
- A link the agent typed into a shell command counts as seen, and so does a link it tried to fetch
|
|
321
|
+
even if the fetch failed.
|
|
322
|
+
- Windows: the tests pass in CI on Python 3.9 to 3.14, but RAYNSIS Inspector has not been tried in a live Claude Code
|
|
323
|
+
session on Windows. Hooks run through Git Bash or PowerShell and need `python3` on `PATH`.
|
|
324
|
+
|
|
325
|
+
## Contributing
|
|
326
|
+
|
|
327
|
+
Bug reports with the exact input that misfired are the most useful thing you can send: see
|
|
328
|
+
[CONTRIBUTING.md](CONTRIBUTING.md). Security issues: [SECURITY.md](SECURITY.md).
|
|
329
|
+
|
|
330
|
+
## License
|
|
331
|
+
|
|
332
|
+
Apache-2.0, see [LICENSE](LICENSE) and [NOTICE](NOTICE). Made by RAYNSIS AI Workforce.
|