agentskills-testing 0.7.0__tar.gz → 0.8.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: agentskills-testing
3
- Version: 0.7.0
3
+ Version: 0.8.0
4
4
  Summary: Conformance suite and test doubles for Agent Skills providers (https://agentskills.io)
5
5
  License: MIT
6
6
  Author: Pratik Panda
@@ -16,7 +16,7 @@ Classifier: Programming Language :: Python :: 3.14
16
16
  Classifier: Programming Language :: Python :: 3.15
17
17
  Classifier: Topic :: Software Development :: Libraries
18
18
  Classifier: Topic :: Software Development :: Testing
19
- Requires-Dist: agentskills-core (>=0.7.0,<1.0)
19
+ Requires-Dist: agentskills-core (>=0.8.0,<1.0)
20
20
  Requires-Dist: pytest (>=8.0)
21
21
  Requires-Dist: pytest-asyncio (>=0.24)
22
22
  Requires-Dist: pyyaml (>=6.0,<7.0)
@@ -24,7 +24,10 @@ Project-URL: Homepage, https://agentskills.io
24
24
  Project-URL: Repository, https://github.com/pratikxpanda/agentskills-sdk
25
25
  Description-Content-Type: text/markdown
26
26
 
27
- # agentskills-testing
27
+ ---
28
+ title: agentskills-testing
29
+ description: Provider conformance suites, in-memory test doubles, and pytest fixtures.
30
+ ---
28
31
 
29
32
  Conformance suite and test doubles for [Agent Skills](https://agentskills.io) providers.
30
33
 
@@ -42,22 +45,41 @@ pip install agentskills-testing
42
45
  ## Conformance suite
43
46
 
44
47
  Subclass `ProviderConformanceSuite`, supply a `provider` fixture, and pytest
45
- collects the entire provider contract against your implementation:
48
+ collects the provider contract. Save this runnable example as `tests/test_provider.py`:
46
49
 
47
50
  ```python
48
51
  import pytest
49
52
 
50
- from agentskills_testing import ProviderConformanceSuite
51
-
52
- from my_package import MyProvider
53
+ from agentskills_testing import (
54
+ ASSET_BYTES,
55
+ ASSET_NAME,
56
+ REFERENCE_BYTES,
57
+ REFERENCE_NAME,
58
+ SCRIPT_BYTES,
59
+ SCRIPT_NAME,
60
+ SKILL_ID,
61
+ InMemorySkillProvider,
62
+ ProviderConformanceSuite,
63
+ build_skill,
64
+ )
53
65
 
54
66
 
55
- class TestMyProvider(ProviderConformanceSuite):
67
+ class TestProvider(ProviderConformanceSuite):
56
68
  @pytest.fixture
57
- def provider(self):
58
- return MyProvider(...)
69
+ def provider(self) -> InMemorySkillProvider:
70
+ skill = build_skill(
71
+ SKILL_ID,
72
+ references={REFERENCE_NAME: REFERENCE_BYTES},
73
+ scripts={SCRIPT_NAME: SCRIPT_BYTES},
74
+ assets={ASSET_NAME: ASSET_BYTES},
75
+ )
76
+ return InMemorySkillProvider({SKILL_ID: skill})
59
77
  ```
60
78
 
79
+ Run `python -m pytest tests/test_provider.py -q`. For your own backend, populate it with the
80
+ same names and bytes below, then replace the fixture's provider construction. Use a yield
81
+ fixture and close clients after the yield when your provider owns network resources.
82
+
61
83
  ### Fixture contract
62
84
 
63
85
  The `provider` fixture must expose exactly one skill:
@@ -82,7 +104,7 @@ a string, and is what the default fixture prints when you forget to override it.
82
104
  ### What it checks
83
105
 
84
106
  - Metadata carries `name` and a non-empty `description`, and does **not** carry
85
- the body — a metadata call that includes the body has already spent the tokens
107
+ the body: a metadata call that includes the body has already spent the tokens
86
108
  progressive disclosure exists to save.
87
109
  - Metadata is not shared mutable state: one caller mutating the returned dict
88
110
  must not affect the next.
@@ -98,7 +120,7 @@ a string, and is what the default fixture prints when you forget to override it.
98
120
  - `discover()` agrees with `supports_discovery`, and everything it reports can
99
121
  actually be read. `register_all()` validates the whole list, so one phantom ID
100
122
  fails the entire registration.
101
- - **Traversal identifiers are refused** — parent traversal, absolute paths,
123
+ - **Traversal identifiers are refused**: parent traversal, absolute paths,
102
124
  Windows separators, percent-encoded traversal, and embedded NUL bytes, as both
103
125
  skill IDs and resource names. These are not opt-out.
104
126
  - Concurrent reads through a single instance return consistent content, which
@@ -107,19 +129,49 @@ a string, and is what the default fixture prints when you forget to override it.
107
129
  ### Size limits
108
130
 
109
131
  `ContentLimitConformanceSuite` is separate and opt-in. A size limit is not part
110
- of the universal contract — an in-memory provider has no external source to
132
+ of the universal contract: an in-memory provider has no external source to
111
133
  bound, and demanding one would assert a filesystem's constraints against a dict.
112
134
  It **is** required of any provider that reads bytes it did not author: from
113
135
  disk, from a network, from anywhere a caller can grow without asking.
114
136
 
137
+ For a concrete bounded-provider example, also install `agentskills-fs`:
138
+
115
139
  ```python
116
- from agentskills_testing import ContentLimitConformanceSuite
140
+ from pathlib import Path
117
141
 
142
+ import pytest
118
143
 
119
- class TestMyProviderLimits(ContentLimitConformanceSuite):
120
- @pytest.fixture
121
- def limited_provider(self):
122
- return MyProvider(..., max_bytes=8)
144
+ from agentskills_fs import LocalFileSystemSkillProvider
145
+ from agentskills_testing import (
146
+ ASSET_BYTES,
147
+ ASSET_NAME,
148
+ REFERENCE_BYTES,
149
+ REFERENCE_NAME,
150
+ SCRIPT_BYTES,
151
+ SCRIPT_NAME,
152
+ SKILL_ID,
153
+ ContentLimitConformanceSuite,
154
+ build_skill,
155
+ render_skill_md,
156
+ )
157
+
158
+
159
+ class TestProviderLimits(ContentLimitConformanceSuite):
160
+ @pytest.fixture
161
+ def limited_provider(self, tmp_path: Path) -> LocalFileSystemSkillProvider:
162
+ directory = tmp_path / SKILL_ID
163
+ directory.mkdir()
164
+ (directory / "SKILL.md").write_text(
165
+ render_skill_md(build_skill(SKILL_ID)), encoding="utf-8"
166
+ )
167
+ for kind, name, data in (
168
+ ("references", REFERENCE_NAME, REFERENCE_BYTES),
169
+ ("scripts", SCRIPT_NAME, SCRIPT_BYTES),
170
+ ("assets", ASSET_NAME, ASSET_BYTES),
171
+ ):
172
+ (directory / kind).mkdir()
173
+ (directory / kind / name).write_bytes(data)
174
+ return LocalFileSystemSkillProvider(tmp_path, max_file_bytes=8)
123
175
  ```
124
176
 
125
177
  The fixture holds the same skill, with a limit small enough that the skill
@@ -127,27 +179,30 @@ exceeds it.
127
179
 
128
180
  ## Test doubles
129
181
 
130
- `InMemorySkillProvider` is a real, spec-compliant provider backed by a dict — it
182
+ `InMemorySkillProvider` is a real, spec-compliant provider backed by a dict. It
131
183
  passes the conformance suite above. Prefer it to an `AsyncMock`: a mock agrees
132
184
  with whatever the test asserts, including the assertions that are wrong.
133
185
 
134
186
  ```python
187
+ import asyncio
188
+
135
189
  from agentskills_core import SkillRegistry
136
190
  from agentskills_testing import InMemorySkillProvider, build_skill
137
191
 
138
- provider = InMemorySkillProvider(
139
- {
140
- "incident-response": build_skill(
141
- "incident-response",
142
- description="Diagnose and mitigate a production incident.",
143
- body="# Incident Response\n\nPage the on-call engineer.\n",
144
- references={"severity-levels.md": b"SEV1 is customer-visible.\n"},
145
- )
146
- }
147
- )
148
192
 
149
- registry = SkillRegistry()
150
- await registry.register_all(provider)
193
+ async def main() -> None:
194
+ provider = InMemorySkillProvider({
195
+ "incident-response": build_skill(
196
+ "incident-response",
197
+ references={"severity-levels.md": b"SEV1 is customer-visible.\n"},
198
+ )
199
+ })
200
+ registry = SkillRegistry()
201
+ await registry.register_all(provider)
202
+ print(await registry.get_skill("incident-response").get_body())
203
+
204
+
205
+ asyncio.run(main())
151
206
  ```
152
207
 
153
208
  A string value is taken as the body, for the common case where the content does
@@ -163,8 +218,8 @@ that does not care about metadata does not have to invent any.
163
218
  `render_skill_md(skill)` renders one back to `SKILL.md` text, which is how you
164
219
  populate a temporary directory for the filesystem provider.
165
220
 
166
- To emulate a backend that cannot enumerate — a static HTTP host without a
167
- manifest, for instance — pass `supports_resource_listing=False`, or
221
+ To emulate a backend that cannot enumerate (a static HTTP host without a
222
+ manifest, for instance), pass `supports_resource_listing=False`, or
168
223
  `supports_discovery=False`, or both.
169
224
 
170
225
  ## Fixtures
@@ -179,6 +234,10 @@ import and no `conftest.py` entry:
179
234
  | `skill_registry` | A `SkillRegistry` with that skill already registered |
180
235
 
181
236
  ```python
237
+ import pytest
238
+
239
+
240
+ @pytest.mark.asyncio
182
241
  async def test_my_agent(skill_registry):
183
242
  catalog = await skill_registry.get_skills_catalog()
184
243
  assert "incident-response" in catalog
@@ -186,7 +245,7 @@ async def test_my_agent(skill_registry):
186
245
 
187
246
  ## License
188
247
 
189
- MIT — see [LICENSE](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE).
248
+ MIT. See [LICENSE](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE).
190
249
 
191
250
  Part of the [Agent Skills SDK](https://github.com/pratikxpanda/agentskills-sdk).
192
251
 
@@ -1,4 +1,7 @@
1
- # agentskills-testing
1
+ ---
2
+ title: agentskills-testing
3
+ description: Provider conformance suites, in-memory test doubles, and pytest fixtures.
4
+ ---
2
5
 
3
6
  Conformance suite and test doubles for [Agent Skills](https://agentskills.io) providers.
4
7
 
@@ -16,22 +19,41 @@ pip install agentskills-testing
16
19
  ## Conformance suite
17
20
 
18
21
  Subclass `ProviderConformanceSuite`, supply a `provider` fixture, and pytest
19
- collects the entire provider contract against your implementation:
22
+ collects the provider contract. Save this runnable example as `tests/test_provider.py`:
20
23
 
21
24
  ```python
22
25
  import pytest
23
26
 
24
- from agentskills_testing import ProviderConformanceSuite
25
-
26
- from my_package import MyProvider
27
+ from agentskills_testing import (
28
+ ASSET_BYTES,
29
+ ASSET_NAME,
30
+ REFERENCE_BYTES,
31
+ REFERENCE_NAME,
32
+ SCRIPT_BYTES,
33
+ SCRIPT_NAME,
34
+ SKILL_ID,
35
+ InMemorySkillProvider,
36
+ ProviderConformanceSuite,
37
+ build_skill,
38
+ )
27
39
 
28
40
 
29
- class TestMyProvider(ProviderConformanceSuite):
41
+ class TestProvider(ProviderConformanceSuite):
30
42
  @pytest.fixture
31
- def provider(self):
32
- return MyProvider(...)
43
+ def provider(self) -> InMemorySkillProvider:
44
+ skill = build_skill(
45
+ SKILL_ID,
46
+ references={REFERENCE_NAME: REFERENCE_BYTES},
47
+ scripts={SCRIPT_NAME: SCRIPT_BYTES},
48
+ assets={ASSET_NAME: ASSET_BYTES},
49
+ )
50
+ return InMemorySkillProvider({SKILL_ID: skill})
33
51
  ```
34
52
 
53
+ Run `python -m pytest tests/test_provider.py -q`. For your own backend, populate it with the
54
+ same names and bytes below, then replace the fixture's provider construction. Use a yield
55
+ fixture and close clients after the yield when your provider owns network resources.
56
+
35
57
  ### Fixture contract
36
58
 
37
59
  The `provider` fixture must expose exactly one skill:
@@ -56,7 +78,7 @@ a string, and is what the default fixture prints when you forget to override it.
56
78
  ### What it checks
57
79
 
58
80
  - Metadata carries `name` and a non-empty `description`, and does **not** carry
59
- the body — a metadata call that includes the body has already spent the tokens
81
+ the body: a metadata call that includes the body has already spent the tokens
60
82
  progressive disclosure exists to save.
61
83
  - Metadata is not shared mutable state: one caller mutating the returned dict
62
84
  must not affect the next.
@@ -72,7 +94,7 @@ a string, and is what the default fixture prints when you forget to override it.
72
94
  - `discover()` agrees with `supports_discovery`, and everything it reports can
73
95
  actually be read. `register_all()` validates the whole list, so one phantom ID
74
96
  fails the entire registration.
75
- - **Traversal identifiers are refused** — parent traversal, absolute paths,
97
+ - **Traversal identifiers are refused**: parent traversal, absolute paths,
76
98
  Windows separators, percent-encoded traversal, and embedded NUL bytes, as both
77
99
  skill IDs and resource names. These are not opt-out.
78
100
  - Concurrent reads through a single instance return consistent content, which
@@ -81,19 +103,49 @@ a string, and is what the default fixture prints when you forget to override it.
81
103
  ### Size limits
82
104
 
83
105
  `ContentLimitConformanceSuite` is separate and opt-in. A size limit is not part
84
- of the universal contract — an in-memory provider has no external source to
106
+ of the universal contract: an in-memory provider has no external source to
85
107
  bound, and demanding one would assert a filesystem's constraints against a dict.
86
108
  It **is** required of any provider that reads bytes it did not author: from
87
109
  disk, from a network, from anywhere a caller can grow without asking.
88
110
 
111
+ For a concrete bounded-provider example, also install `agentskills-fs`:
112
+
89
113
  ```python
90
- from agentskills_testing import ContentLimitConformanceSuite
114
+ from pathlib import Path
91
115
 
116
+ import pytest
92
117
 
93
- class TestMyProviderLimits(ContentLimitConformanceSuite):
94
- @pytest.fixture
95
- def limited_provider(self):
96
- return MyProvider(..., max_bytes=8)
118
+ from agentskills_fs import LocalFileSystemSkillProvider
119
+ from agentskills_testing import (
120
+ ASSET_BYTES,
121
+ ASSET_NAME,
122
+ REFERENCE_BYTES,
123
+ REFERENCE_NAME,
124
+ SCRIPT_BYTES,
125
+ SCRIPT_NAME,
126
+ SKILL_ID,
127
+ ContentLimitConformanceSuite,
128
+ build_skill,
129
+ render_skill_md,
130
+ )
131
+
132
+
133
+ class TestProviderLimits(ContentLimitConformanceSuite):
134
+ @pytest.fixture
135
+ def limited_provider(self, tmp_path: Path) -> LocalFileSystemSkillProvider:
136
+ directory = tmp_path / SKILL_ID
137
+ directory.mkdir()
138
+ (directory / "SKILL.md").write_text(
139
+ render_skill_md(build_skill(SKILL_ID)), encoding="utf-8"
140
+ )
141
+ for kind, name, data in (
142
+ ("references", REFERENCE_NAME, REFERENCE_BYTES),
143
+ ("scripts", SCRIPT_NAME, SCRIPT_BYTES),
144
+ ("assets", ASSET_NAME, ASSET_BYTES),
145
+ ):
146
+ (directory / kind).mkdir()
147
+ (directory / kind / name).write_bytes(data)
148
+ return LocalFileSystemSkillProvider(tmp_path, max_file_bytes=8)
97
149
  ```
98
150
 
99
151
  The fixture holds the same skill, with a limit small enough that the skill
@@ -101,27 +153,30 @@ exceeds it.
101
153
 
102
154
  ## Test doubles
103
155
 
104
- `InMemorySkillProvider` is a real, spec-compliant provider backed by a dict — it
156
+ `InMemorySkillProvider` is a real, spec-compliant provider backed by a dict. It
105
157
  passes the conformance suite above. Prefer it to an `AsyncMock`: a mock agrees
106
158
  with whatever the test asserts, including the assertions that are wrong.
107
159
 
108
160
  ```python
161
+ import asyncio
162
+
109
163
  from agentskills_core import SkillRegistry
110
164
  from agentskills_testing import InMemorySkillProvider, build_skill
111
165
 
112
- provider = InMemorySkillProvider(
113
- {
114
- "incident-response": build_skill(
115
- "incident-response",
116
- description="Diagnose and mitigate a production incident.",
117
- body="# Incident Response\n\nPage the on-call engineer.\n",
118
- references={"severity-levels.md": b"SEV1 is customer-visible.\n"},
119
- )
120
- }
121
- )
122
166
 
123
- registry = SkillRegistry()
124
- await registry.register_all(provider)
167
+ async def main() -> None:
168
+ provider = InMemorySkillProvider({
169
+ "incident-response": build_skill(
170
+ "incident-response",
171
+ references={"severity-levels.md": b"SEV1 is customer-visible.\n"},
172
+ )
173
+ })
174
+ registry = SkillRegistry()
175
+ await registry.register_all(provider)
176
+ print(await registry.get_skill("incident-response").get_body())
177
+
178
+
179
+ asyncio.run(main())
125
180
  ```
126
181
 
127
182
  A string value is taken as the body, for the common case where the content does
@@ -137,8 +192,8 @@ that does not care about metadata does not have to invent any.
137
192
  `render_skill_md(skill)` renders one back to `SKILL.md` text, which is how you
138
193
  populate a temporary directory for the filesystem provider.
139
194
 
140
- To emulate a backend that cannot enumerate — a static HTTP host without a
141
- manifest, for instance — pass `supports_resource_listing=False`, or
195
+ To emulate a backend that cannot enumerate (a static HTTP host without a
196
+ manifest, for instance), pass `supports_resource_listing=False`, or
142
197
  `supports_discovery=False`, or both.
143
198
 
144
199
  ## Fixtures
@@ -153,6 +208,10 @@ import and no `conftest.py` entry:
153
208
  | `skill_registry` | A `SkillRegistry` with that skill already registered |
154
209
 
155
210
  ```python
211
+ import pytest
212
+
213
+
214
+ @pytest.mark.asyncio
156
215
  async def test_my_agent(skill_registry):
157
216
  catalog = await skill_registry.get_skills_catalog()
158
217
  assert "incident-response" in catalog
@@ -160,6 +219,6 @@ async def test_my_agent(skill_registry):
160
219
 
161
220
  ## License
162
221
 
163
- MIT — see [LICENSE](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE).
222
+ MIT. See [LICENSE](https://github.com/pratikxpanda/agentskills-sdk/blob/main/LICENSE).
164
223
 
165
224
  Part of the [Agent Skills SDK](https://github.com/pratikxpanda/agentskills-sdk).
@@ -1,6 +1,6 @@
1
1
  [tool.poetry]
2
2
  name = "agentskills-testing"
3
- version = "0.7.0"
3
+ version = "0.8.0"
4
4
  description = "Conformance suite and test doubles for Agent Skills providers (https://agentskills.io)"
5
5
  license = "MIT"
6
6
  authors = ["Pratik Panda"]
@@ -21,7 +21,7 @@ classifiers = [
21
21
 
22
22
  [tool.poetry.dependencies]
23
23
  python = ">=3.12,<4.0"
24
- agentskills-core = ">=0.7.0,<1.0"
24
+ agentskills-core = ">=0.8.0,<1.0"
25
25
  pytest = ">=8.0"
26
26
  pytest-asyncio = ">=0.24"
27
27
  pyyaml = ">=6.0,<7.0"