catchbench 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- catchbench-0.1.0/LICENSE +21 -0
- catchbench-0.1.0/NOTICE +86 -0
- catchbench-0.1.0/PKG-INFO +657 -0
- catchbench-0.1.0/README.md +623 -0
- catchbench-0.1.0/pyproject.toml +38 -0
- catchbench-0.1.0/setup.cfg +4 -0
- catchbench-0.1.0/src/catchbench/__init__.py +12 -0
- catchbench-0.1.0/src/catchbench/_reuse.py +109 -0
- catchbench-0.1.0/src/catchbench/agent_detectors.py +180 -0
- catchbench-0.1.0/src/catchbench/core.py +95 -0
- catchbench-0.1.0/src/catchbench/corpora.py +158 -0
- catchbench-0.1.0/src/catchbench/detection.py +197 -0
- catchbench-0.1.0/src/catchbench/gold.py +676 -0
- catchbench-0.1.0/src/catchbench/graph_ad.py +157 -0
- catchbench-0.1.0/src/catchbench/live.py +426 -0
- catchbench-0.1.0/src/catchbench/llm_judge.py +518 -0
- catchbench-0.1.0/src/catchbench/namedvalue.py +436 -0
- catchbench-0.1.0/src/catchbench/post.py +197 -0
- catchbench-0.1.0/src/catchbench/pre.py +272 -0
- catchbench-0.1.0/src/catchbench/pre_baselines.py +108 -0
- catchbench-0.1.0/src/catchbench/pre_static_scanner.py +489 -0
- catchbench-0.1.0/src/catchbench/pygod_extra.py +182 -0
- catchbench-0.1.0/src/catchbench/pyod_extra.py +113 -0
- catchbench-0.1.0/src/catchbench.egg-info/PKG-INFO +657 -0
- catchbench-0.1.0/src/catchbench.egg-info/SOURCES.txt +42 -0
- catchbench-0.1.0/src/catchbench.egg-info/dependency_links.txt +1 -0
- catchbench-0.1.0/src/catchbench.egg-info/requires.txt +27 -0
- catchbench-0.1.0/src/catchbench.egg-info/top_level.txt +1 -0
- catchbench-0.1.0/tests/test_check_board.py +865 -0
- catchbench-0.1.0/tests/test_ci_report_check.py +445 -0
- catchbench-0.1.0/tests/test_emit_asset_manifest.py +162 -0
- catchbench-0.1.0/tests/test_emit_boards_table.py +328 -0
- catchbench-0.1.0/tests/test_emit_stats_table.py +548 -0
- catchbench-0.1.0/tests/test_emit_transfer_table.py +231 -0
- catchbench-0.1.0/tests/test_gold_ties.py +79 -0
- catchbench-0.1.0/tests/test_graph_ad_scores_every_node.py +112 -0
- catchbench-0.1.0/tests/test_namedvalue.py +272 -0
- catchbench-0.1.0/tests/test_namedvalue_ties.py +48 -0
- catchbench-0.1.0/tests/test_post_step_texts.py +182 -0
- catchbench-0.1.0/tests/test_pre_static_scanner.py +207 -0
- catchbench-0.1.0/tests/test_statistical_tests.py +44 -0
- catchbench-0.1.0/tests/test_third_party_notices.py +334 -0
- catchbench-0.1.0/tests/test_tools_are_not_stdlib_shadows.py +49 -0
- catchbench-0.1.0/tests/test_workflow_pins.py +80 -0
catchbench-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Yue Zhao
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
catchbench-0.1.0/NOTICE
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
CatchBench
|
|
2
|
+
Copyright (c) 2026 Yue Zhao
|
|
3
|
+
|
|
4
|
+
This distribution contains normalized material derived from third-party sources and retained model
|
|
5
|
+
output generated from those sources. Exact files, upstream projects, source identifiers, licence
|
|
6
|
+
status, and unresolved blocks are recorded in THIRD_PARTY_LICENSES.md and ASSET_MANIFEST.json.
|
|
7
|
+
Each source identifier is typed there as an immutable revision or as a mutable identifier, because
|
|
8
|
+
an n8n template ID and a SWE-bench submission label name a location whose content can change rather
|
|
9
|
+
than fixed bytes.
|
|
10
|
+
|
|
11
|
+
The redistributed licence texts travel with this repository rather than by link. Apache-2.0,
|
|
12
|
+
GPL-3.0, and CC-BY-4.0 read the same for every project declaring them, so one copy of each covers
|
|
13
|
+
every record: third_party/licenses/Apache-2.0.txt, GPL-3.0.txt, and CC-BY-4.0.txt, indexed with
|
|
14
|
+
their SHA-256 digests and the projects they cover in third_party/licenses/CANONICAL-TEXTS.md. MIT
|
|
15
|
+
names the copyright holder, so its notices are per project: the CrewAI and MCP notices are under
|
|
16
|
+
third_party/licenses/, and the InjecAgent and Who&When code-project notices are reproduced inline in
|
|
17
|
+
THIRD_PARTY_LICENSES.md. Records marked NOASSERTION, Other, or unverified declare no licence at all
|
|
18
|
+
and remain unfinished. Five further projects declare MIT or Apache-2.0, but their pinned licence
|
|
19
|
+
file could not be retrieved; each is marked UNRESOLVED where its notice would be, and the 6 records
|
|
20
|
+
derived from them remain unfinished. CatchBench's MIT licence does not override any source terms.
|
|
21
|
+
|
|
22
|
+
InjecAgent
|
|
23
|
+
Copyright (c) 2023 Qiusi Zhan
|
|
24
|
+
Licensed under the MIT License. The full notice is reproduced in THIRD_PARTY_LICENSES.md.
|
|
25
|
+
|
|
26
|
+
Who&When associated code project
|
|
27
|
+
Copyright (c) 2025 Ming Yin
|
|
28
|
+
Licensed under the MIT License. This notice applies to the associated code project; the licence of
|
|
29
|
+
the separately hosted Who&When dataset remains unresolved. See THIRD_PARTY_LICENSES.md.
|
|
30
|
+
|
|
31
|
+
The following upstream NOTICE text accompanies MCP-derived records. It is reproduced verbatim;
|
|
32
|
+
references to LICENSE within these blocks mean the pinned upstream LICENSE files identified in
|
|
33
|
+
THIRD_PARTY_LICENSES.md, not CatchBench's root LICENSE.
|
|
34
|
+
|
|
35
|
+
Helixar Security MCP Connector
|
|
36
|
+
Copyright 2026 Helixar Limited
|
|
37
|
+
|
|
38
|
+
This product includes software developed at Helixar Limited (https://helixar.ai).
|
|
39
|
+
|
|
40
|
+
This product references the following external specifications and identifiers:
|
|
41
|
+
|
|
42
|
+
- HDP — Human Delegation Provenance Protocol
|
|
43
|
+
IETF draft: draft-helixar-hdp-agentic-delegation-00
|
|
44
|
+
Subject to IETF Trust Legal Provisions (https://trustee.ietf.org/license-info).
|
|
45
|
+
|
|
46
|
+
- Companion paper: "HDP: A Lightweight Cryptographic Protocol for Human
|
|
47
|
+
Delegation Provenance in Agentic AI Systems" — arXiv:2604.04522,
|
|
48
|
+
Asiri Dalugoda (2026), licensed CC BY 4.0.
|
|
49
|
+
|
|
50
|
+
- Reference implementation snapshot: Zenodo DOI 10.5281/zenodo.19332023
|
|
51
|
+
(the licence of any individual snapshot is recorded in its Zenodo metadata
|
|
52
|
+
and may differ from this product's licence).
|
|
53
|
+
|
|
54
|
+
This product is licensed under the Apache License, Version 2.0 (see LICENSE).
|
|
55
|
+
|
|
56
|
+
Vacation Rental Protocol — reference implementation (HemmaBo MCP server)
|
|
57
|
+
Copyright 2026 HemmaBo / Rouiada Abbas
|
|
58
|
+
|
|
59
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
60
|
+
you may not use this file except in compliance with the License.
|
|
61
|
+
You may obtain a copy of the License in the LICENSE file or at
|
|
62
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
63
|
+
|
|
64
|
+
NOTICE — Evlek MCP Server
|
|
65
|
+
|
|
66
|
+
The MIT license in the LICENSE file applies ONLY to the contents of this public
|
|
67
|
+
repository: the MCP Registry manifest (server.json), documentation (README.md,
|
|
68
|
+
TOOLS.md), configuration examples, and reference clients.
|
|
69
|
+
|
|
70
|
+
SCOPE OF THE MIT LICENSE
|
|
71
|
+
The MIT license does NOT cover the hosted Evlek service. The following remain
|
|
72
|
+
proprietary and are NOT licensed under MIT:
|
|
73
|
+
- the evlek.app web application and its source code
|
|
74
|
+
- the Evlek iOS and Android applications
|
|
75
|
+
- the listing database and its contents
|
|
76
|
+
- AI prompts, prompt engineering, and model configurations
|
|
77
|
+
- business logic, ranking, and data pipelines
|
|
78
|
+
- brand assets, design system, and trade dress
|
|
79
|
+
|
|
80
|
+
TRADEMARK
|
|
81
|
+
"Evlek" is a trademark of Onur Dokuzoğlu. The MIT license grants no rights to
|
|
82
|
+
use the "Evlek" name, logo, branding, or trade dress. For brand or trademark
|
|
83
|
+
usage requests, contact: hello@evlek.app
|
|
84
|
+
|
|
85
|
+
CONTACT
|
|
86
|
+
hello@evlek.app https://evlek.app
|