testerkit 0.4.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (521) hide show
  1. testerkit/__init__.py +68 -0
  2. testerkit/_docs/_assets/operator-ui/channels/table.png +0 -0
  3. testerkit/_docs/_assets/operator-ui/dashboard/runs.png +0 -0
  4. testerkit/_docs/_assets/operator-ui/dashboard/stations.png +0 -0
  5. testerkit/_docs/_assets/operator-ui/events/filters.png +0 -0
  6. testerkit/_docs/_assets/operator-ui/events/table.png +0 -0
  7. testerkit/_docs/_assets/operator-ui/explore/chart.png +0 -0
  8. testerkit/_docs/_assets/operator-ui/explore/filters.png +0 -0
  9. testerkit/_docs/_assets/operator-ui/explore/plot-controls.png +0 -0
  10. testerkit/_docs/_assets/operator-ui/files/table.png +0 -0
  11. testerkit/_docs/_assets/operator-ui/launch/form.png +0 -0
  12. testerkit/_docs/_assets/operator-ui/metrics/filters.png +0 -0
  13. testerkit/_docs/_assets/operator-ui/metrics/pareto.png +0 -0
  14. testerkit/_docs/_assets/operator-ui/metrics/ppk.png +0 -0
  15. testerkit/_docs/_assets/operator-ui/metrics/yield.png +0 -0
  16. testerkit/_docs/_assets/operator-ui/results/detail-header.png +0 -0
  17. testerkit/_docs/_assets/operator-ui/results/detail-measurements.png +0 -0
  18. testerkit/_docs/_assets/operator-ui/results/detail-overview.png +0 -0
  19. testerkit/_docs/_assets/operator-ui/results/detail-steps.png +0 -0
  20. testerkit/_docs/_assets/operator-ui/results/stats.png +0 -0
  21. testerkit/_docs/_assets/operator-ui/results/table.png +0 -0
  22. testerkit/_docs/_assets/operator-ui/tour/designer.png +0 -0
  23. testerkit/_docs/_assets/operator-ui/tour/docs.png +0 -0
  24. testerkit/_docs/_assets/operator-ui/tour/fixtures.png +0 -0
  25. testerkit/_docs/_assets/operator-ui/tour/instruments.png +0 -0
  26. testerkit/_docs/_assets/operator-ui/tour/parts.png +0 -0
  27. testerkit/_docs/_assets/operator-ui/tour/profiles.png +0 -0
  28. testerkit/_docs/_assets/operator-ui/tour/stations.png +0 -0
  29. testerkit/_docs/_assets/operator-ui/tour/tests.png +0 -0
  30. testerkit/_docs/_assets/operator-ui/tour/uuts.png +0 -0
  31. testerkit/_docs/concepts/configuration/capabilities.md +447 -0
  32. testerkit/_docs/concepts/configuration/fixtures.md +335 -0
  33. testerkit/_docs/concepts/configuration/index.md +14 -0
  34. testerkit/_docs/concepts/configuration/parts.md +276 -0
  35. testerkit/_docs/concepts/configuration/stations.md +214 -0
  36. testerkit/_docs/concepts/data/data-stores.md +199 -0
  37. testerkit/_docs/concepts/data/event-log.md +185 -0
  38. testerkit/_docs/concepts/data/event-sourcing.md +76 -0
  39. testerkit/_docs/concepts/data/flight-streaming.md +58 -0
  40. testerkit/_docs/concepts/data/index.md +17 -0
  41. testerkit/_docs/concepts/data/sessions.md +78 -0
  42. testerkit/_docs/concepts/data/three-verbs.md +260 -0
  43. testerkit/_docs/concepts/execution/index.md +13 -0
  44. testerkit/_docs/concepts/execution/outcomes.md +199 -0
  45. testerkit/_docs/concepts/execution/step-hierarchy.md +170 -0
  46. testerkit/_docs/concepts/execution/step-manifest.md +115 -0
  47. testerkit/_docs/concepts/index.md +40 -0
  48. testerkit/_docs/concepts/overview/ai-integration.md +57 -0
  49. testerkit/_docs/concepts/overview/architecture.md +357 -0
  50. testerkit/_docs/concepts/overview/index.md +14 -0
  51. testerkit/_docs/concepts/overview/platform-vs-framework.md +147 -0
  52. testerkit/_docs/concepts/overview/pytest.md +53 -0
  53. testerkit/_docs/concepts/overview/tiers.md +46 -0
  54. testerkit/_docs/how-to/catalog/datasheet-to-test.md +169 -0
  55. testerkit/_docs/how-to/catalog/index.md +11 -0
  56. testerkit/_docs/how-to/configuration/configuring-stations.md +364 -0
  57. testerkit/_docs/how-to/configuration/custom-drivers.md +484 -0
  58. testerkit/_docs/how-to/configuration/index.md +13 -0
  59. testerkit/_docs/how-to/configuration/mock-mode.md +342 -0
  60. testerkit/_docs/how-to/data/benchmarking.md +120 -0
  61. testerkit/_docs/how-to/data/capture-an-artifact.md +84 -0
  62. testerkit/_docs/how-to/data/capture-waveform.md +110 -0
  63. testerkit/_docs/how-to/data/choosing-a-channel-verb.md +108 -0
  64. testerkit/_docs/how-to/data/compare-runs.md +120 -0
  65. testerkit/_docs/how-to/data/export-results.md +104 -0
  66. testerkit/_docs/how-to/data/find-flaky-tests.md +108 -0
  67. testerkit/_docs/how-to/data/grafana-dashboards.md +148 -0
  68. testerkit/_docs/how-to/data/index.md +23 -0
  69. testerkit/_docs/how-to/data/mcp-debug-failures.md +125 -0
  70. testerkit/_docs/how-to/data/mcp-query-runs.md +120 -0
  71. testerkit/_docs/how-to/data/querying-channels.md +117 -0
  72. testerkit/_docs/how-to/data/querying-events.md +85 -0
  73. testerkit/_docs/how-to/data/stream-live-channel.md +134 -0
  74. testerkit/_docs/how-to/execution/custom-operator-ui.md +136 -0
  75. testerkit/_docs/how-to/execution/index.md +21 -0
  76. testerkit/_docs/how-to/execution/limits.md +193 -0
  77. testerkit/_docs/how-to/execution/managing-sessions.md +100 -0
  78. testerkit/_docs/how-to/execution/multi-uut-testing.md +211 -0
  79. testerkit/_docs/how-to/execution/operator-prompts.md +151 -0
  80. testerkit/_docs/how-to/execution/profiles.md +304 -0
  81. testerkit/_docs/how-to/execution/spec-driven-testing.md +169 -0
  82. testerkit/_docs/how-to/execution/test-context.md +212 -0
  83. testerkit/_docs/how-to/execution/traceability.md +252 -0
  84. testerkit/_docs/how-to/execution/vector-expansion.md +394 -0
  85. testerkit/_docs/how-to/execution/writing-tests.md +363 -0
  86. testerkit/_docs/how-to/index.md +59 -0
  87. testerkit/_docs/how-to/overview/index.md +11 -0
  88. testerkit/_docs/how-to/overview/mcp-integration.md +168 -0
  89. testerkit/_docs/how-to/overview/operator-ui-tour.md +234 -0
  90. testerkit/_docs/integration/data/grafana.md +162 -0
  91. testerkit/_docs/integration/data/index.md +15 -0
  92. testerkit/_docs/integration/data/lakehouse-import.md +226 -0
  93. testerkit/_docs/integration/data/logging.md +131 -0
  94. testerkit/_docs/integration/data/results-api.md +51 -0
  95. testerkit/_docs/integration/index.md +20 -0
  96. testerkit/_docs/integration/runtime/harness.md +281 -0
  97. testerkit/_docs/integration/runtime/index.md +13 -0
  98. testerkit/_docs/integration/runtime/instruments.md +284 -0
  99. testerkit/_docs/integration/runtime/pytest-existing.md +315 -0
  100. testerkit/_docs/reference/catalog/cookbook.md +389 -0
  101. testerkit/_docs/reference/catalog/index.md +11 -0
  102. testerkit/_docs/reference/catalog/schema.md +375 -0
  103. testerkit/_docs/reference/cli.md +660 -0
  104. testerkit/_docs/reference/configuration.md +406 -0
  105. testerkit/_docs/reference/data/channels-schema.md +145 -0
  106. testerkit/_docs/reference/data/event-types.md +554 -0
  107. testerkit/_docs/reference/data/events-schema.md +120 -0
  108. testerkit/_docs/reference/data/files-schema.md +141 -0
  109. testerkit/_docs/reference/data/index.md +18 -0
  110. testerkit/_docs/reference/data/models.md +2229 -0
  111. testerkit/_docs/reference/data/outputs.md +82 -0
  112. testerkit/_docs/reference/data/parquet-schema.md +529 -0
  113. testerkit/_docs/reference/data/performance-limits.md +224 -0
  114. testerkit/_docs/reference/data/query-api.md +247 -0
  115. testerkit/_docs/reference/index.md +69 -0
  116. testerkit/_docs/reference/operator-ui/channels/detail.md +82 -0
  117. testerkit/_docs/reference/operator-ui/channels/list.md +70 -0
  118. testerkit/_docs/reference/operator-ui/dashboard.md +87 -0
  119. testerkit/_docs/reference/operator-ui/designer.md +167 -0
  120. testerkit/_docs/reference/operator-ui/events.md +89 -0
  121. testerkit/_docs/reference/operator-ui/files.md +74 -0
  122. testerkit/_docs/reference/operator-ui/fixtures.md +100 -0
  123. testerkit/_docs/reference/operator-ui/instruments.md +126 -0
  124. testerkit/_docs/reference/operator-ui/launch.md +79 -0
  125. testerkit/_docs/reference/operator-ui/live.md +102 -0
  126. testerkit/_docs/reference/operator-ui/measurements.md +159 -0
  127. testerkit/_docs/reference/operator-ui/metrics.md +217 -0
  128. testerkit/_docs/reference/operator-ui/parts.md +98 -0
  129. testerkit/_docs/reference/operator-ui/profiles.md +103 -0
  130. testerkit/_docs/reference/operator-ui/results/detail.md +214 -0
  131. testerkit/_docs/reference/operator-ui/results/list.md +106 -0
  132. testerkit/_docs/reference/operator-ui/stations.md +98 -0
  133. testerkit/_docs/reference/operator-ui/tests.md +173 -0
  134. testerkit/_docs/reference/operator-ui/uuts.md +67 -0
  135. testerkit/_docs/reference/overview/index.md +11 -0
  136. testerkit/_docs/reference/overview/pytest-native.md +142 -0
  137. testerkit/_docs/reference/overview/skills.md +177 -0
  138. testerkit/_docs/reference/pytest/fixtures.md +342 -0
  139. testerkit/_docs/reference/pytest/index.md +12 -0
  140. testerkit/_docs/reference/pytest/markers.md +257 -0
  141. testerkit/_docs/reference/runtime/api.md +246 -0
  142. testerkit/_docs/reference/runtime/client.md +280 -0
  143. testerkit/_docs/reference/runtime/connect.md +140 -0
  144. testerkit/_docs/reference/runtime/index.md +13 -0
  145. testerkit/_docs/tutorial/01-first-test.md +113 -0
  146. testerkit/_docs/tutorial/02-mock-instruments.md +88 -0
  147. testerkit/_docs/tutorial/03-fixtures.md +169 -0
  148. testerkit/_docs/tutorial/04-limits.md +139 -0
  149. testerkit/_docs/tutorial/05-configuration.md +194 -0
  150. testerkit/_docs/tutorial/06-specifications.md +273 -0
  151. testerkit/_docs/tutorial/07-real-instruments.md +245 -0
  152. testerkit/_docs/tutorial/08-capabilities.md +223 -0
  153. testerkit/_docs/tutorial/09-production.md +374 -0
  154. testerkit/_docs/tutorial/10-live-monitoring.md +156 -0
  155. testerkit/_docs/tutorial/11-waveforms-and-evidence.md +169 -0
  156. testerkit/_docs/tutorial/12-continuous-monitoring.md +145 -0
  157. testerkit/_docs/tutorial/13-parallel-testing.md +233 -0
  158. testerkit/_docs/tutorial/index.md +61 -0
  159. testerkit/_docs/tutorial/quickstart.md +282 -0
  160. testerkit/analysis/__init__.py +9 -0
  161. testerkit/analysis/_common.py +24 -0
  162. testerkit/analysis/measurement_facets.py +524 -0
  163. testerkit/analysis/measurements_query.py +1458 -0
  164. testerkit/analysis/metrics.py +389 -0
  165. testerkit/analysis/runs_query.py +434 -0
  166. testerkit/analysis/steps_query.py +413 -0
  167. testerkit/api/__init__.py +6 -0
  168. testerkit/api/_mime.py +71 -0
  169. testerkit/api/app.py +1171 -0
  170. testerkit/api/dialogs/__init__.py +36 -0
  171. testerkit/api/dialogs/manager.py +617 -0
  172. testerkit/api/dialogs/models.py +79 -0
  173. testerkit/api/models.py +80 -0
  174. testerkit/api/responses.py +205 -0
  175. testerkit/api/runner.py +222 -0
  176. testerkit/api/schemas.py +311 -0
  177. testerkit/benchmark/__init__.py +34 -0
  178. testerkit/benchmark/concurrency.py +248 -0
  179. testerkit/benchmark/core.py +232 -0
  180. testerkit/benchmark/runner.py +628 -0
  181. testerkit/benchmark/scenario.py +313 -0
  182. testerkit/benchmark/system.py +191 -0
  183. testerkit/benchmark/workloads.py +507 -0
  184. testerkit/catalog/__init__.py +6 -0
  185. testerkit/catalog/generic/generic_dmm.yaml +61 -0
  186. testerkit/catalog/generic/generic_eload.yaml +31 -0
  187. testerkit/catalog/generic/generic_fgen.yaml +56 -0
  188. testerkit/catalog/generic/generic_oscilloscope.yaml +45 -0
  189. testerkit/catalog/generic/generic_psu.yaml +77 -0
  190. testerkit/catalog/generic/generic_smu.yaml +73 -0
  191. testerkit/channels.py +538 -0
  192. testerkit/cli/__init__.py +32 -0
  193. testerkit/cli/_common.py +113 -0
  194. testerkit/cli/_time.py +179 -0
  195. testerkit/cli/benchmark_cmd.py +56 -0
  196. testerkit/cli/catalog_cmd.py +39 -0
  197. testerkit/cli/daemon.py +157 -0
  198. testerkit/cli/data_cmd.py +1040 -0
  199. testerkit/cli/discover_cmd.py +94 -0
  200. testerkit/cli/docs_cmd.py +100 -0
  201. testerkit/cli/instrument.py +172 -0
  202. testerkit/cli/mcp_cmd.py +37 -0
  203. testerkit/cli/metrics.py +366 -0
  204. testerkit/cli/project.py +312 -0
  205. testerkit/cli/root.py +14 -0
  206. testerkit/cli/runs.py +412 -0
  207. testerkit/cli/schema_cmd.py +87 -0
  208. testerkit/cli/serve_cmd.py +50 -0
  209. testerkit/cli/setup_cmd.py +604 -0
  210. testerkit/cli/station.py +294 -0
  211. testerkit/cli/validate.py +112 -0
  212. testerkit/client.py +463 -0
  213. testerkit/connect.py +635 -0
  214. testerkit/data/__init__.py +8 -0
  215. testerkit/data/_accumulator_pool.py +455 -0
  216. testerkit/data/_atomic.py +59 -0
  217. testerkit/data/_collection_indices.py +67 -0
  218. testerkit/data/_daemon_lifecycle.py +535 -0
  219. testerkit/data/_duckdb_daemon.py +695 -0
  220. testerkit/data/_duckdb_flight_server.py +813 -0
  221. testerkit/data/_event_filters.py +15 -0
  222. testerkit/data/_event_reader.py +101 -0
  223. testerkit/data/_flight_errors.py +199 -0
  224. testerkit/data/_flight_query.py +202 -0
  225. testerkit/data/_flight_retry.py +148 -0
  226. testerkit/data/_flight_subscribe.py +67 -0
  227. testerkit/data/_index_epoch.py +342 -0
  228. testerkit/data/_ipc_writer.py +185 -0
  229. testerkit/data/_json_safe.py +63 -0
  230. testerkit/data/_process.py +18 -0
  231. testerkit/data/_push_relay.py +124 -0
  232. testerkit/data/_runs_duckdb_daemon.py +2704 -0
  233. testerkit/data/_session_reaper.py +110 -0
  234. testerkit/data/_sql_helpers.py +30 -0
  235. testerkit/data/_store.py +24 -0
  236. testerkit/data/backends/__init__.py +7 -0
  237. testerkit/data/backends/_event_accumulator.py +957 -0
  238. testerkit/data/backends/_row_helpers.py +1172 -0
  239. testerkit/data/backends/parquet.py +1143 -0
  240. testerkit/data/backends/protocol.py +35 -0
  241. testerkit/data/channels/__init__.py +8 -0
  242. testerkit/data/channels/_flight_daemon.py +75 -0
  243. testerkit/data/channels/client.py +242 -0
  244. testerkit/data/channels/flight_manager.py +88 -0
  245. testerkit/data/channels/index.py +664 -0
  246. testerkit/data/channels/models.py +418 -0
  247. testerkit/data/channels/server.py +162 -0
  248. testerkit/data/channels/store.py +1481 -0
  249. testerkit/data/data_dir.py +61 -0
  250. testerkit/data/duckdb_manager.py +79 -0
  251. testerkit/data/event_log.py +384 -0
  252. testerkit/data/event_store.py +768 -0
  253. testerkit/data/events.py +1025 -0
  254. testerkit/data/exporters/__init__.py +6 -0
  255. testerkit/data/exporters/_helpers.py +40 -0
  256. testerkit/data/exporters/csv_exporter.py +168 -0
  257. testerkit/data/exporters/hdf5.py +234 -0
  258. testerkit/data/exporters/json_exporter.py +229 -0
  259. testerkit/data/exporters/mdf4.py +201 -0
  260. testerkit/data/exporters/stdf.py +255 -0
  261. testerkit/data/exporters/tdms.py +292 -0
  262. testerkit/data/files/__init__.py +71 -0
  263. testerkit/data/files/_backend.py +207 -0
  264. testerkit/data/files/_catalog_daemon.py +84 -0
  265. testerkit/data/files/catalog.py +218 -0
  266. testerkit/data/files/catalog_manager.py +325 -0
  267. testerkit/data/files/models.py +60 -0
  268. testerkit/data/files/serializers.py +414 -0
  269. testerkit/data/files/store.py +463 -0
  270. testerkit/data/files/streaming.py +783 -0
  271. testerkit/data/models.py +604 -0
  272. testerkit/data/ref.py +152 -0
  273. testerkit/data/retention.py +297 -0
  274. testerkit/data/run_store.py +367 -0
  275. testerkit/data/runs_duckdb_manager.py +103 -0
  276. testerkit/data/schema_dispatch.py +191 -0
  277. testerkit/data/schema_migrate.py +76 -0
  278. testerkit/data/schema_versions.py +85 -0
  279. testerkit/data/schemas.py +288 -0
  280. testerkit/data/subscribers/__init__.py +48 -0
  281. testerkit/data/subscribers/_base.py +20 -0
  282. testerkit/data/subscribers/_output_file.py +25 -0
  283. testerkit/data/subscribers/replay.py +55 -0
  284. testerkit/environment.py +84 -0
  285. testerkit/execution/__init__.py +11 -0
  286. testerkit/execution/_git.py +190 -0
  287. testerkit/execution/_state.py +740 -0
  288. testerkit/execution/accessors.py +54 -0
  289. testerkit/execution/audit.py +51 -0
  290. testerkit/execution/cascade.py +85 -0
  291. testerkit/execution/connections.py +491 -0
  292. testerkit/execution/harness.py +1931 -0
  293. testerkit/execution/instrument_events.py +42 -0
  294. testerkit/execution/limits.py +154 -0
  295. testerkit/execution/metadata.py +102 -0
  296. testerkit/execution/mocks.py +63 -0
  297. testerkit/execution/profiles.py +616 -0
  298. testerkit/execution/run_scope.py +1331 -0
  299. testerkit/execution/session_scope.py +196 -0
  300. testerkit/execution/sidecar.py +280 -0
  301. testerkit/execution/site_runner.py +705 -0
  302. testerkit/execution/sites.py +183 -0
  303. testerkit/execution/sync.py +294 -0
  304. testerkit/execution/uut_provider.py +231 -0
  305. testerkit/execution/vectors.py +205 -0
  306. testerkit/execution/verify.py +396 -0
  307. testerkit/expand.py +75 -0
  308. testerkit/files.py +235 -0
  309. testerkit/fixtures/__init__.py +6 -0
  310. testerkit/fixtures/manager.py +357 -0
  311. testerkit/grafana/__init__.py +1 -0
  312. testerkit/grafana/bootstrap.py +61 -0
  313. testerkit/grafana/cli.py +314 -0
  314. testerkit/grafana/dashboards/asset_utilization.json +92 -0
  315. testerkit/grafana/dashboards/channel_explorer.json +77 -0
  316. testerkit/grafana/dashboards/event_log.json +92 -0
  317. testerkit/grafana/dashboards/failure_pareto.json +104 -0
  318. testerkit/grafana/dashboards/measurement_distribution.json +97 -0
  319. testerkit/grafana/dashboards/measurement_trend.json +161 -0
  320. testerkit/grafana/dashboards/station_comparison.json +95 -0
  321. testerkit/grafana/dashboards/test_duration.json +104 -0
  322. testerkit/grafana/dashboards/unit_traceability.json +73 -0
  323. testerkit/grafana/dashboards/yield_overview.json +189 -0
  324. testerkit/grafana/provisioning/dashboards.yaml.j2 +12 -0
  325. testerkit/grafana/provisioning/datasources.yaml.j2 +17 -0
  326. testerkit/grafana/server.py +282 -0
  327. testerkit/init.py +724 -0
  328. testerkit/instruments/__init__.py +10 -0
  329. testerkit/instruments/base.py +82 -0
  330. testerkit/instruments/discovery/__init__.py +62 -0
  331. testerkit/instruments/discovery/_base.py +174 -0
  332. testerkit/instruments/discovery/lxi.py +152 -0
  333. testerkit/instruments/discovery/ni.py +95 -0
  334. testerkit/instruments/discovery/serial.py +73 -0
  335. testerkit/instruments/discovery/visa.py +98 -0
  336. testerkit/instruments/lifecycle.py +213 -0
  337. testerkit/instruments/loader.py +79 -0
  338. testerkit/instruments/locks.py +178 -0
  339. testerkit/instruments/mocks.py +230 -0
  340. testerkit/instruments/observer.py +270 -0
  341. testerkit/instruments/observers/__init__.py +36 -0
  342. testerkit/instruments/observers/_base.py +68 -0
  343. testerkit/instruments/observers/daqmx.py +70 -0
  344. testerkit/instruments/observers/descriptor.py +69 -0
  345. testerkit/instruments/observers/generic.py +92 -0
  346. testerkit/instruments/observers/lantz.py +31 -0
  347. testerkit/instruments/observers/modbus.py +96 -0
  348. testerkit/instruments/observers/motion.py +68 -0
  349. testerkit/instruments/observers/ni_modular.py +73 -0
  350. testerkit/instruments/observers/ophyd.py +80 -0
  351. testerkit/instruments/observers/pymeasure.py +120 -0
  352. testerkit/instruments/observers/qcodes.py +66 -0
  353. testerkit/instruments/observers/scpi.py +88 -0
  354. testerkit/instruments/observers/tektronix.py +57 -0
  355. testerkit/instruments/observers/visa.py +103 -0
  356. testerkit/instruments/pool.py +459 -0
  357. testerkit/instruments/proxy.py +59 -0
  358. testerkit/instruments/route_manager.py +329 -0
  359. testerkit/instruments/routed_proxy.py +88 -0
  360. testerkit/instruments/server.py +508 -0
  361. testerkit/instruments/switch.py +43 -0
  362. testerkit/instruments/visa.py +294 -0
  363. testerkit/matching/__init__.py +6 -0
  364. testerkit/matching/service.py +969 -0
  365. testerkit/mcp/__init__.py +6 -0
  366. testerkit/mcp/server.py +681 -0
  367. testerkit/mcp/tools.py +1868 -0
  368. testerkit/models/__init__.py +18 -0
  369. testerkit/models/capability.py +598 -0
  370. testerkit/models/catalog.py +76 -0
  371. testerkit/models/data_options.py +153 -0
  372. testerkit/models/enums.py +334 -0
  373. testerkit/models/instrument.py +152 -0
  374. testerkit/models/instrument_asset.py +25 -0
  375. testerkit/models/part.py +271 -0
  376. testerkit/models/part_manifest.py +96 -0
  377. testerkit/models/project.py +114 -0
  378. testerkit/models/station.py +140 -0
  379. testerkit/models/test_config.py +748 -0
  380. testerkit/ontology/__init__.py +39 -0
  381. testerkit/ontology/schema.py +188 -0
  382. testerkit/ontology/testerkit.yaml +1420 -0
  383. testerkit/parts/__init__.py +8 -0
  384. testerkit/parts/context.py +181 -0
  385. testerkit/parts/folder.py +251 -0
  386. testerkit/parts/loader.py +27 -0
  387. testerkit/prompts/__init__.py +17 -0
  388. testerkit/prompts/core.py +133 -0
  389. testerkit/pytest_plugin/__init__.py +1287 -0
  390. testerkit/pytest_plugin/autouse.py +369 -0
  391. testerkit/pytest_plugin/helpers.py +354 -0
  392. testerkit/pytest_plugin/hooks.py +1982 -0
  393. testerkit/pytest_plugin/markers.py +203 -0
  394. testerkit/pytest_plugin/retry.py +37 -0
  395. testerkit/pytest_plugin/sweeps.py +117 -0
  396. testerkit/queries.py +78 -0
  397. testerkit/reports/__init__.py +6 -0
  398. testerkit/reports/core.py +492 -0
  399. testerkit/reports/datasheet.py +831 -0
  400. testerkit/reports/templates/datasheet.html +383 -0
  401. testerkit/reports/templates/default.html +153 -0
  402. testerkit/sbom.py +142 -0
  403. testerkit/schema_export.py +101 -0
  404. testerkit/signals.py +63 -0
  405. testerkit/skills/README.md +58 -0
  406. testerkit/skills/templates/project-instructions.md +125 -0
  407. testerkit/skills/testerkit-analysis/SKILL.md +87 -0
  408. testerkit/skills/testerkit-capture/SKILL.md +114 -0
  409. testerkit/skills/testerkit-data/SKILL.md +138 -0
  410. testerkit/skills/testerkit-datasheets/SKILL.md +100 -0
  411. testerkit/skills/testerkit-datasheets/agents/scaffold-writer.md +115 -0
  412. testerkit/skills/testerkit-datasheets/agents/section-extractor.md +98 -0
  413. testerkit/skills/testerkit-datasheets/agents/section-reviewer.md +153 -0
  414. testerkit/skills/testerkit-datasheets/agents/section-splitter.md +89 -0
  415. testerkit/skills/testerkit-datasheets/agents/section-writer.md +226 -0
  416. testerkit/skills/testerkit-datasheets/references/catalog-pipeline.md +271 -0
  417. testerkit/skills/testerkit-datasheets/references/process-queue.md +42 -0
  418. testerkit/skills/testerkit-datasheets/references/scaffold.md +156 -0
  419. testerkit/skills/testerkit-datasheets/references/test-pipeline.md +312 -0
  420. testerkit/skills/testerkit-debug/SKILL.md +127 -0
  421. testerkit/skills/testerkit-interactive/SKILL.md +188 -0
  422. testerkit/skills/testerkit-interactive/references/live-ui-patterns.md +102 -0
  423. testerkit/skills/testerkit-mocks/SKILL.md +128 -0
  424. testerkit/skills/testerkit-parts/SKILL.md +126 -0
  425. testerkit/skills/testerkit-profiles/SKILL.md +139 -0
  426. testerkit/skills/testerkit-sites/SKILL.md +140 -0
  427. testerkit/skills/testerkit-stations/SKILL.md +146 -0
  428. testerkit/skills/testerkit-tests/SKILL.md +136 -0
  429. testerkit/store.py +1639 -0
  430. testerkit/ui/__init__.py +71 -0
  431. testerkit/ui/_asgi.py +162 -0
  432. testerkit/ui/app.py +32 -0
  433. testerkit/ui/components/__init__.py +13 -0
  434. testerkit/ui/components/artifact_viewer.py +250 -0
  435. testerkit/ui/components/channel_values.py +159 -0
  436. testerkit/ui/components/event_timeline.py +222 -0
  437. testerkit/ui/components/execution_gantt.py +210 -0
  438. testerkit/ui/components/file_streams.py +147 -0
  439. testerkit/ui/components/instrument_activity.py +146 -0
  440. testerkit/ui/components/session_table.py +118 -0
  441. testerkit/ui/pages/__init__.py +25 -0
  442. testerkit/ui/pages/channels/__init__.py +4 -0
  443. testerkit/ui/pages/channels/detail.py +932 -0
  444. testerkit/ui/pages/channels/list.py +515 -0
  445. testerkit/ui/pages/dashboard.py +236 -0
  446. testerkit/ui/pages/designer/__init__.py +5 -0
  447. testerkit/ui/pages/designer/graph.py +635 -0
  448. testerkit/ui/pages/designer/matching.py +663 -0
  449. testerkit/ui/pages/designer/page.py +593 -0
  450. testerkit/ui/pages/designer/properties.py +591 -0
  451. testerkit/ui/pages/designer/state.py +468 -0
  452. testerkit/ui/pages/docs/__init__.py +6 -0
  453. testerkit/ui/pages/docs/index.py +95 -0
  454. testerkit/ui/pages/docs/page.py +698 -0
  455. testerkit/ui/pages/events/__init__.py +3 -0
  456. testerkit/ui/pages/events/list.py +263 -0
  457. testerkit/ui/pages/explore.py +1108 -0
  458. testerkit/ui/pages/files/__init__.py +4 -0
  459. testerkit/ui/pages/files/detail.py +426 -0
  460. testerkit/ui/pages/files/list.py +447 -0
  461. testerkit/ui/pages/fixtures/__init__.py +8 -0
  462. testerkit/ui/pages/fixtures/detail.py +280 -0
  463. testerkit/ui/pages/fixtures/edit.py +370 -0
  464. testerkit/ui/pages/fixtures/list.py +180 -0
  465. testerkit/ui/pages/fixtures/new.py +195 -0
  466. testerkit/ui/pages/instruments/__init__.py +4 -0
  467. testerkit/ui/pages/instruments/detail.py +320 -0
  468. testerkit/ui/pages/instruments/edit.py +482 -0
  469. testerkit/ui/pages/instruments/list.py +249 -0
  470. testerkit/ui/pages/instruments/new.py +200 -0
  471. testerkit/ui/pages/launch.py +261 -0
  472. testerkit/ui/pages/live.py +124 -0
  473. testerkit/ui/pages/metrics_page.py +1229 -0
  474. testerkit/ui/pages/parts/__init__.py +11 -0
  475. testerkit/ui/pages/parts/detail.py +202 -0
  476. testerkit/ui/pages/parts/edit.py +310 -0
  477. testerkit/ui/pages/parts/list.py +160 -0
  478. testerkit/ui/pages/parts/new.py +170 -0
  479. testerkit/ui/pages/parts/requirements.py +153 -0
  480. testerkit/ui/pages/parts/stations.py +199 -0
  481. testerkit/ui/pages/profiles/__init__.py +4 -0
  482. testerkit/ui/pages/profiles/detail.py +113 -0
  483. testerkit/ui/pages/profiles/list.py +86 -0
  484. testerkit/ui/pages/results/__init__.py +4 -0
  485. testerkit/ui/pages/results/detail.py +631 -0
  486. testerkit/ui/pages/results/list.py +230 -0
  487. testerkit/ui/pages/stations/__init__.py +4 -0
  488. testerkit/ui/pages/stations/detail.py +256 -0
  489. testerkit/ui/pages/stations/edit.py +273 -0
  490. testerkit/ui/pages/stations/list.py +180 -0
  491. testerkit/ui/pages/stations/new.py +311 -0
  492. testerkit/ui/pages/tests/__init__.py +4 -0
  493. testerkit/ui/pages/tests/detail.py +152 -0
  494. testerkit/ui/pages/tests/list.py +194 -0
  495. testerkit/ui/pages/uuts/__init__.py +4 -0
  496. testerkit/ui/pages/uuts/list.py +97 -0
  497. testerkit/ui/shared/__init__.py +8 -0
  498. testerkit/ui/shared/components.py +1826 -0
  499. testerkit/ui/shared/dialogs.py +105 -0
  500. testerkit/ui/shared/event_binding.py +194 -0
  501. testerkit/ui/shared/layout.py +314 -0
  502. testerkit/ui/shared/services.py +1719 -0
  503. testerkit/ui/shared/timestamps.py +21 -0
  504. testerkit/ui/static/branding/favicon-16.png +0 -0
  505. testerkit/ui/static/branding/favicon-32.png +0 -0
  506. testerkit/ui/static/branding/favicon-48.png +0 -0
  507. testerkit/ui/static/branding/testerkit-icon-256.png +0 -0
  508. testerkit/ui/static/branding/testerkit-mark.svg +1 -0
  509. testerkit/ui/static/branding/testerkit-wordmark.svg +1 -0
  510. testerkit/ui/static/global.css +465 -0
  511. testerkit/utils/__init__.py +17 -0
  512. testerkit/utils/enum_meta.py +800 -0
  513. testerkit/utils/paths.py +75 -0
  514. testerkit/utils/ranges.py +257 -0
  515. testerkit/validation.py +82 -0
  516. testerkit/verbs.py +142 -0
  517. testerkit-0.4.0.dist-info/METADATA +250 -0
  518. testerkit-0.4.0.dist-info/RECORD +521 -0
  519. testerkit-0.4.0.dist-info/WHEEL +4 -0
  520. testerkit-0.4.0.dist-info/entry_points.txt +5 -0
  521. testerkit-0.4.0.dist-info/licenses/LICENSE +191 -0
@@ -0,0 +1,173 @@
1
+ # Tests
2
+
3
+ **URLs:** `/tests` (list), `/tests/{path}` (detail)
4
+
5
+ A test in TesterKit is a `def test_*` function that pytest discovers — either a
6
+ top-level function or a method inside a `class Test*`. The Tests pages give you
7
+ a file-level inventory of every `test_*.py` under `tests/`, cross-referenced
8
+ against run history, without importing or executing any test code.
9
+
10
+ ## List — `/tests`
11
+
12
+ Files are grouped by directory. Each directory renders a folder-icon header
13
+ followed by one row per `test_*.py` file under it.
14
+
15
+ ### File rows
16
+
17
+ Each row shows:
18
+
19
+ | Field | What it shows |
20
+ |---|---|
21
+ | Filename | The `test_*.py` filename, in monospace |
22
+ | Test count | Number of `def test_*` functions found in the file (top-level and inside `class Test*`) |
23
+ | Class count | Number of `class Test*` definitions, shown inline when non-zero |
24
+ | Vectors | A `~N vectors` badge, present when the file has parametrize, `testerkit_sweeps`, or similar list-valued markers. The count is a rough upper-bound estimate from the decorator arguments — not an exact cross-product |
25
+ | Markers | Up to four marker chips (decorator names, without the `pytest.mark.` prefix), then `+N` when there are more |
26
+ | Runs | Count of distinct `run_id` values in history where any step from this file executed; omitted when zero |
27
+ | Failed | Count of those runs with at least one failed step; omitted when there are no runs or no failures |
28
+ | Sidecar | A `sidecar` badge when a `.yaml` file with the same stem exists alongside the `.py`; a dash otherwise |
29
+
30
+ Clicking any row navigates to `/tests/{path}` for that file.
31
+
32
+ When a file cannot be parsed (syntax error or I/O problem), the row renders in
33
+ red with a warning icon and a truncated error message. The file is still
34
+ clickable — the detail page shows the full parse error.
35
+
36
+ ### Badge
37
+
38
+ A `N files` badge in the page header shows the total number of `test_*.py`
39
+ files found.
40
+
41
+ ### Observed in history (no matching source)
42
+
43
+ Below the file list, when run history contains step paths that do not match
44
+ any function in the current source, an **Observed in history (no matching
45
+ source)** section appears. This surfaces tests that were renamed, deleted, or
46
+ run from a code state that isn't checked out. The section shows the count of
47
+ orphaned paths as a warning badge.
48
+
49
+ The orphan table has these columns:
50
+
51
+ | Column | What it shows |
52
+ |---|---|
53
+ | Step Path | The `step_path` value from the steps table — bare function name or `Class/method` |
54
+ | Runs | Distinct run count |
55
+ | Passed | Runs with outcome `passed` |
56
+ | Failed | Runs with outcome `failed` |
57
+ | Last Run | Most recent step start timestamp, browser-local time |
58
+
59
+ Rows are sorted by Last Run descending so the most recently orphaned paths appear first.
60
+
61
+ ### Empty state
62
+
63
+ When no `test_*.py` files are found and there are no orphan step paths, the
64
+ file list is replaced with a card:
65
+
66
+ > No test files found.
67
+ >
68
+ > Add test_*.py files under tests/.
69
+
70
+ ### Data-testid anchors
71
+
72
+ | Element | `data-testid` |
73
+ |---|---|
74
+ | Directory-grouped file list container | `tests-table` |
75
+ | Orphaned step paths table | `tests-orphans-table` |
76
+
77
+ ## Detail — `/tests/{path}`
78
+
79
+ The `{path}` segment carries the full relative path from the project root,
80
+ including the `tests/` prefix and the `.py` extension — for example,
81
+ `/tests/tests/test_power_rail.py`. The path is resolved against the server's
82
+ current working directory; only files inside the working tree are served.
83
+
84
+ ### Header
85
+
86
+ The page header shows the file path, a `N tests` badge, a `sidecar` badge
87
+ when a sidecar exists, and a **Launch Test** button (top right) that navigates
88
+ to `/launch?test={path}`.
89
+
90
+ A note below the header reads:
91
+
92
+ > What actually runs depends on the active profile — sidecar < profile
93
+ > (last-wins). See Profiles.
94
+
95
+ "Profiles" is a link to `/profiles`. For the cascade order, see
96
+ [Profiles](../configuration.md#profile-blocks-under-profiles).
97
+
98
+ ### Test functions table
99
+
100
+ A table with one row per `def test_*` function found in the file. Columns:
101
+
102
+ | Column | What it shows |
103
+ |---|---|
104
+ | Test | Bare function name (`test_foo`) |
105
+ | Class | Parent `class Test*` name, or `—` for top-level functions |
106
+ | Markers | Comma-separated decorator names (without `pytest.mark.` prefix), or `—` |
107
+ | Vectors | Estimated parametrize / `testerkit_sweeps` vector count, or `—` when zero |
108
+ | Sidecar | `✓` when this function (or its parent class) has an entry in the sidecar `tests:` block; `—` otherwise |
109
+ | Runs | Distinct run count from history for this step path; `0` when never run |
110
+ | Passed | Runs with outcome `passed` |
111
+ | Failed | Runs with outcome `failed` |
112
+ | Last Run | Most recent step start timestamp, browser-local time; `—` when never run |
113
+
114
+ The step path used to look up run history is `Class/method` when the function
115
+ is inside a class, or the bare function name for top-level functions. This
116
+ matches the `step_path` value recorded in run history.
117
+
118
+ The test functions table is omitted when the file has a parse error.
119
+
120
+ Data-testid anchor: `test-functions-table`.
121
+
122
+ ### Tabs
123
+
124
+ Below the test functions table, two tabs are available:
125
+
126
+ | Tab | Content |
127
+ |---|---|
128
+ | Code | The `.py` file rendered as a read-only Python code block |
129
+ | Sidecar YAML | The colocated `.yaml` file rendered as a read-only YAML code block; present only when a sidecar exists |
130
+
131
+ When no sidecar exists, only the **Code** tab appears.
132
+
133
+ ### Parse error state
134
+
135
+ When the file cannot be parsed, a red card appears at the top of the content
136
+ area with the full error message. The Code tab still renders the raw file
137
+ contents.
138
+
139
+ ### Not-found state
140
+
141
+ When `{path}` does not resolve to an existing `.py` file inside the working
142
+ tree, the page shows a card:
143
+
144
+ > Test file '{path}' not found.
145
+
146
+ with a "← Back to Tests" link.
147
+
148
+ ## Underlying data
149
+
150
+ The file list is built by reading every `test_*.py` under `tests/` directly —
151
+ without importing or running any test code — to extract its function names,
152
+ class names, and decorators. Files that fail to parse are still listed,
153
+ flagged with the parse error.
154
+
155
+ Run history (Runs / Passed / Failed / Last Run) is tallied from run history
156
+ for each test, by step path. For the full step record schema, see
157
+ [Parquet schema](../data/parquet-schema.md).
158
+
159
+ A test shows the `sidecar` badge when a `.yaml` file with the same stem sits
160
+ alongside the `.py`. On the detail page, a function gets the `✓` when it (or
161
+ its class) has an entry in the sidecar's `tests:` block.
162
+
163
+ ## See also
164
+
165
+ - [How-to → Writing Tests](../../how-to/execution/writing-tests.md) — pytest
166
+ classes, sidecar YAML, the `verify` pattern
167
+ - [pytest markers reference](../pytest/markers.md) — every `@testerkit.mark.*`
168
+ decorator that appears in the Markers column
169
+ - [Profiles](profiles.md) — how profiles override sidecar config at session
170
+ start (the cascade note on the detail page)
171
+ - [Launch Test](launch.md) — start a run targeting one of these test files
172
+ - [Results](results/list.md) — per-run view; the step-level detail that feeds
173
+ the Runs / Passed / Failed counts on this page
@@ -0,0 +1,67 @@
1
+ # UUTs
2
+
3
+ **URLs:** `/uuts`
4
+
5
+ Every distinct UUT serial that has appeared in run history gets one row
6
+ here. A UUT isn't declared in YAML — it's identified at run time by the
7
+ serial number the operator (or automation) supplies when starting a run.
8
+
9
+ ## List — `/uuts`
10
+
11
+ A table with one row per distinct UUT serial observed in run history.
12
+ Rows are ordered by most-recent run first.
13
+
14
+ | Column | What it shows |
15
+ |---|---|
16
+ | Serial | The `uut_serial_number` value recorded on the run |
17
+ | Part Number | The `uut_part_number` recorded on the most recent run for this serial |
18
+ | Lot | The `uut_lot_number` recorded on the most recent run for this serial |
19
+ | Runs | Total run count across all outcomes for this serial |
20
+ | Passed | Run count with outcome `passed` |
21
+ | Failed | Run count with outcome `failed` |
22
+ | Last Run | Most recent run start timestamp, browser-local time |
23
+
24
+ A badge in the page header shows the total count of observed UUTs.
25
+
26
+ The table has no filter row. All rows are observed-only — there is no
27
+ Configured / Observed distinction because UUTs have no backing YAML.
28
+
29
+ There is no detail page at `/uuts/{serial}`. The list is the only view.
30
+
31
+ ### Empty state
32
+
33
+ When no runs with a non-empty `uut_serial_number` are present, the table is
34
+ replaced with a card:
35
+
36
+ > No UUTs observed yet.
37
+ >
38
+ > Run a test against a station to populate this list. Every distinct UUT
39
+ > serial that appears in run history shows up here.
40
+
41
+ ### Outcome columns
42
+
43
+ **Passed** and **Failed** count only runs whose outcome is exactly
44
+ `passed` or `failed`. Runs with other outcomes (for example `aborted`)
45
+ contribute to the **Runs** total but are not reflected in either column.
46
+
47
+ ## Underlying data
48
+
49
+ Each column maps to a field on the run:
50
+
51
+ | Column | Run field |
52
+ |---|---|
53
+ | Serial | `uut_serial_number` |
54
+ | Part Number | `uut_part_number` |
55
+ | Lot | `uut_lot_number` |
56
+
57
+ For the full set of run fields, see
58
+ [Parquet schema](../data/parquet-schema.md).
59
+
60
+ ## See also
61
+
62
+ - [Parquet schema](../data/parquet-schema.md) — the `uut_serial_number`,
63
+ `uut_part_number`, and `uut_lot_number` run columns
64
+ - [Results](results/list.md) — per-run view; its Serial column shows
65
+ which runs belong to a UUT
66
+ - [Launch Test](launch.md) — where the operator supplies the serial
67
+ number that populates this page
@@ -0,0 +1,11 @@
1
+ # Reference — Overview
2
+
3
+ The foundational reference pages: pytest mechanics every TesterKit test uses, and the AI / MCP skills the platform ships.
4
+
5
+ - [pytest-native](pytest-native.md) — TesterKit tests against pytest's own collection / fixtures / markers / `conftest.py` / CLI flags. The baseline every other reference page builds on. Generated from source.
6
+ - [Skills](skills.md) — the 11 Agent Skills that ship with TesterKit for AI-assisted authoring, and how `testerkit setup <tool>` installs them into each AI tool.
7
+
8
+ ## See also
9
+
10
+ - [Concepts → Overview](../../concepts/overview/index.md) — the platform-vs-framework framing
11
+ - [pytest plugin reference](../pytest/) — the fixtures and markers the plugin adds on top of pytest-native
@@ -0,0 +1,142 @@
1
+ # pytest-native reference
2
+
3
+ TesterKit is a hardware test **platform**; pytest is its primary test-runner integration. The bundled pytest plugin slots into a stock pytest install with zero configuration — every pytest concept (collection, fixtures, markers, plugins, `conftest.py`, command-line flags) works unchanged. This page is the map of what pytest gives you natively and what the plugin layers on top. For the plugin's own surface see [TesterKit fixtures](../pytest/fixtures.md) and [TesterKit markers](../pytest/markers.md). Other runner integrations (OpenHTF, LabVIEW / TestStand via the results API) live under [Integrations](../../integration/).
4
+
5
+ ## Collection
6
+
7
+ The bundled pytest plugin uses pytest's default collection — no custom file collectors.
8
+
9
+ | Convention | Default |
10
+ |---|---|
11
+ | Test files | `tests/test_*.py` or `tests/*_test.py` |
12
+ | Test classes | classes named `Test*` (no `__init__`) |
13
+ | Test functions / methods | functions named `test_*` |
14
+ | Override | standard pytest `python_files` / `python_classes` / `python_functions` in `pyproject.toml` |
15
+
16
+ What TesterKit adds at collection time:
17
+
18
+ - A `pytest_collection_modifyitems` hook merges per-test sidecar YAML (`tests/test_<module>.yaml`) into each item's marker set. This expands `testerkit_sweeps` into one pytest case per row exactly as if you had written `@pytest.mark.parametrize` — pytest still owns the case multiplication.
19
+ - Profiles (`--test-profile=<name>`) add `pytest.mark.skip` to items they exclude. The selection is visible in `pytest --collect-only -q`.
20
+
21
+ The sidecar is recursive: top-level keys apply to every test in the file; `tests: { ClassName: { ... } }` scopes per class; `tests: { ClassName: { tests: { test_method: { ... } } } }` scopes per method. See [Sidecar configuration](../configuration.md#sidecar-yaml).
22
+
23
+ ## Fixtures
24
+
25
+ pytest's fixture model is unchanged.
26
+
27
+ - **All four scopes work.** `function`, `class`, `module`, `session` — choose the one that matches your resource cost.
28
+ - **Resolution by name.** Take a fixture in the test signature; pytest resolves it from the nearest `conftest.py` upward, then from registered plugins (TesterKit included).
29
+ - **Yield fixtures, finalizers, request injection** all work as pytest documents them.
30
+ - **`autouse=True`** works. TesterKit uses an autouse-session fixture internally so every test records measurements without taking a recording fixture as an argument.
31
+
32
+ You can write your own fixtures in `conftest.py` alongside TesterKit's. A common pattern is a project-local `uut` factory wrapping the TesterKit `uut` session fixture, or a per-class hardware-setup fixture that takes `instruments` as a dependency.
33
+
34
+ The fixtures the TesterKit plugin contributes are documented in [TesterKit fixtures](../pytest/fixtures.md).
35
+
36
+ ## Markers
37
+
38
+ pytest's marker mechanism is unchanged. All of these work on TesterKit tests as documented in the pytest manual:
39
+
40
+ | Marker | Use |
41
+ |---|---|
42
+ | `@pytest.mark.parametrize` | Generate one pytest case per row. Stack to cross-product. |
43
+ | `@pytest.mark.skip` / `@pytest.mark.skipif` | Skip the test (with optional condition). |
44
+ | `@pytest.mark.xfail` | Mark as expected-to-fail; surfaces as `XFAIL` / `XPASS`. |
45
+ | `@pytest.mark.usefixtures("a", "b")` | Require fixtures without taking them in the signature. |
46
+ | `@pytest.mark.filterwarnings` | Per-test warning filters. |
47
+ | Custom markers via `pytest.ini` / `pyproject.toml` | Register and filter with `-m`. |
48
+
49
+ The seven `@pytest.mark.testerkit_*` markers TesterKit adds are registered the same way as native markers and stack with them (with one constraint — see the [no-stacking rule](../pytest/markers.md#no-stacking-rule) for TesterKit markers). See [TesterKit markers](../pytest/markers.md) for full details.
50
+
51
+ `@pytest.mark.parametrize` and `@pytest.mark.testerkit_sweeps` interoperate: both feed the same `inputs` lane on the vector row (query with `FieldRef.input(name)`).
52
+
53
+ ## conftest.py
54
+
55
+ Works as pytest documents it. Place a `conftest.py` in `tests/` (or a subdirectory) for fixtures and hooks that apply to that scope. Common uses with TesterKit:
56
+
57
+ - Project-local fixtures that wrap or extend TesterKit's (a typed `uut` accessor, a per-class measurement helper).
58
+ - `pytest_addoption` for project-specific CLI flags.
59
+ - `pytest_collection_modifyitems` for project-specific item filtering. (TesterKit's hook is in the plugin and runs alongside, not instead of, yours.)
60
+ - `pytest_runtest_setup` / `_teardown` for per-test setup beyond what fixtures express.
61
+
62
+ The TesterKit plugin loads via the standard pytest entry-point mechanism — no `conftest.py` manipulation needed.
63
+
64
+ ## Command-line flags
65
+
66
+ All pytest flags work. The ones that matter most for hardware test work:
67
+
68
+ | Flag | Purpose |
69
+ |---|---|
70
+ | `-k "expr"` | Run tests whose nodeid matches the substring expression. |
71
+ | `-m "marker"` | Run tests whose markers match the boolean expression. |
72
+ | `-x` / `--maxfail=N` | Stop after the first failure / Nth failure. |
73
+ | `--lf` / `--ff` | Run last-failed / failed-first (uses pytest's cache). |
74
+ | `--collect-only -q` | Show what would run without running. |
75
+ | `-v` / `-q` | Verbose / quiet. |
76
+ | `--tb=short` / `--tb=line` / `--tb=no` | Traceback style. |
77
+ | `-p no:plugin` | Disable a specific plugin. |
78
+ | `--co` | Alias for `--collect-only`. |
79
+
80
+ TesterKit adds the flags below.
81
+
82
+ <!-- Generated from pytest_addoption; regenerate with
83
+ `uv run python scripts/generate_reference_docs.py pytest-native` after editing the hook. -->
84
+
85
+ <!-- GENERATED:pytest-plugin-flags:start -->
86
+ | Flag | Type | Default | Description |
87
+ |---|---|---|---|
88
+ | `--uut-serial` | `text` | `'UUT001'` | UUT serial number |
89
+ | `--uut-serials` | `text` | | Per-site UUT serials: 0=SN1,1=SN2 (by index) or left=SN1,right=SN2 (by name) |
90
+ | `--site` | `text` | | Physical fixture site for this single-process run (e.g. ``0``, ``1``, or a named site like ``left``). Use this when running a single UUT against a specific position in a multi-site fixture so the run records which site was exercised. Multi-site orchestration uses ``--uut-serials`` instead — supplying both is an error. |
91
+ | `--uut-part-number` | `text` | | UUT part number |
92
+ | `--uut-revision` | `text` | | UUT revision |
93
+ | `--uut-lot-number` | `text` | | UUT lot/batch number (mirrors TESTERKIT_UUT_LOT_NUMBER env var) |
94
+ | `--station` | `text` | | Station ID or YAML path. Bare id looks up ``stations/<id>.yaml``; a value with ``/`` or ``.yaml``/``.yml`` is used as an explicit path. When unset, the resolver tries hostname auto-match against stations/*.yaml ``hostname:`` fields, then falls back to ``ProjectConfig.default_station``. |
95
+ | `--operator` | `text` | | Operator name |
96
+ | `--data-dir` | `text` | *resolved at runtime* | Directory for Parquet results (default: platform data dir) |
97
+ | `--part` | `text` | | Part ID or YAML path. Bare id looks up ``parts/<id>.yaml``; a value with ``/`` or ``.yaml``/``.yml`` is used as an explicit path. |
98
+ | `--guardband` | `text` | `'0'` | Default guardband percentage |
99
+ | `--mock-instruments` | `flag` | `False` | Use mock instruments instead of real hardware. Resolution: this flag > TESTERKIT_MOCK_INSTRUMENTS env var > testerkit.yaml `mock_instruments:` > false. |
100
+ | `--no-mock-instruments` | `flag` (inverse) | `True` | Use real hardware (overrides TESTERKIT_MOCK_INSTRUMENTS env and testerkit.yaml `mock_instruments: true`). |
101
+ | `--fixture` | `text` | | Fixture ID or YAML path. Bare id looks up ``fixtures/<id>.yaml``; a value with ``/`` or ``.yaml``/``.yml`` is used as an explicit path. When unset, the resolver tries the active profile's ``fixture:`` field, then ``ProjectConfig.default_fixture``, then the single-file fallback in ``fixtures/``. |
102
+ | `--test-phase` | `text` | | Test phase (development, validation, characterization, production). Defaults to development; a non-development phase is demoted to development on a dirty git tree or under --mock-instruments. |
103
+ | `--strict-traceability` | `flag` | `False` | Fail tests whose measurements lack required traceability fields (run_id, step_name, and spec_ref/uut_pin when a spec is active). |
104
+ | `--test-profile` | `text` | *resolved at runtime* | Named profile from testerkit.yaml `profiles:` (overrides vectors, limits, markers, and filter for the session). |
105
+ | `--no-test-profile` | `flag` | `False` | Skip profile resolution. Use when profiles are declared but you want to run with bare project defaults (ad-hoc runs). |
106
+
107
+ Plus dynamic flags generated from `testerkit.yaml`: every `profiles[*].facets:` key becomes `--<facet-key>` and every `required_inputs:` key becomes `--<required-input-key>`. See [how-to/profiles.md](../../how-to/execution/profiles.md) for the resolution chain (CLI flag → env var → profile binding → `default_*`).
108
+ <!-- GENERATED:pytest-plugin-flags:end -->
109
+
110
+ ## Plugins that interact with TesterKit
111
+
112
+ | Plugin | Notes |
113
+ |---|---|
114
+ | **pytest-rerunfailures** | Powers `@pytest.mark.testerkit_retry`. Install it if you use the marker; the translation happens in the plugin. |
115
+ | **pytest-xdist** | Parallel execution. Generally **not** appropriate for hardware tests on a single bench (instruments aren't reentrant). Fine for mock-only suites and CI lint passes. |
116
+ | **pytest-cov** | Code coverage. Works unchanged on test files; collects coverage on the test code, not on instrument drivers behind hardware. |
117
+ | **pytest-html** / **pytest-json-report** | Independent of TesterKit's own event log + parquet output. Run them alongside if you want pytest-flavored reports too. |
118
+
119
+ ## Discovery vs activation
120
+
121
+ Two things to keep separate:
122
+
123
+ - **Discovery** is pytest's: what `tests/test_*.py` files match, what items they expose. TesterKit has no opinion here.
124
+ - **Activation** is TesterKit's: which station, part, fixture, profile is loaded for the session. Driven by the CLI flags above (or by `default_station:` / `default_profile:` in `testerkit.yaml`).
125
+
126
+ A test that runs on a bringup tier with no station YAML and a test that runs on a factory tier with full traceability are **collected identically**. The activation context decides what fixtures resolve to and what limits `verify` finds.
127
+
128
+ ## See also
129
+
130
+ - [TesterKit fixtures](../pytest/fixtures.md) — all the fixtures the plugin contributes
131
+ - [TesterKit markers](../pytest/markers.md) — the seven `testerkit_*` markers and their sidecar equivalents
132
+ - [Sidecar configuration](../configuration.md#sidecar-yaml) — sidecar YAML merge semantics
133
+ - [CLI reference](../cli.md) — full flag list, including the non-pytest commands (`testerkit serve`, `testerkit runs`, etc.)
134
+ - [pytest documentation](https://docs.pytest.org/en/stable/) — canonical reference for everything in the "pytest-native" half of this page
135
+
136
+
137
+ **Related quadrants:**
138
+
139
+ - [Concepts → Overview](../../concepts/overview/index.md) — concepts entry point for this category
140
+ - [How-to → Overview](../../how-to/overview/index.md) — how-to entry point for this category
141
+ - [Integration](../../integration/index.md) — integration entry point for this category
142
+ - [Tutorial](../../tutorial/index.md) — tutorial entry point for this category
@@ -0,0 +1,177 @@
1
+ # Skills reference
2
+
3
+ TesterKit ships 12 **Agent Skills** — one directory per skill under
4
+ [`src/testerkit/skills/`](https://github.com/pragmatest-dev/testerkit/tree/main/src/testerkit/skills),
5
+ each a `SKILL.md` written to the
6
+ [Agent Skills open standard](https://github.blog/changelog/2025-12-18-github-copilot-now-supports-agent-skills/):
7
+ YAML frontmatter (`name`, `description`) an agent matches against the user's
8
+ request, then a body the agent reads once matched. This page is the
9
+ inventory — what each skill covers, how the depth behind it is sourced, and
10
+ how `testerkit setup <tool>` gets the files in front of your agent.
11
+
12
+ For motivation (why AI integration at all), see
13
+ [concepts/why-ai-integration](../../concepts/overview/ai-integration.md). For
14
+ registering the MCP server and instructions file per tool, see
15
+ [how-to/mcp-integration](../../how-to/overview/mcp-integration.md).
16
+
17
+ ## The 12 skills
18
+
19
+ | Skill | Use when |
20
+ |---|---|
21
+ | `testerkit-tests` | Testing, measuring, or logging any hardware value — the front door. Starts at zero config, grows only as the request demands. |
22
+ | `testerkit-mocks` | Running tests without real hardware — dev-machine smoke tests, CI, a bench that's tied up, or pinning one instrument call's return value. |
23
+ | `testerkit-stations` | Setting up the bench — instruments, roles, a bring-your-own driver, discovery, fixture pin routing. |
24
+ | `testerkit-parts` | Specifying the DUT — documented characteristics, pin map, or a datasheet limit as reusable part YAML. |
25
+ | `testerkit-profiles` | Different limits, sweeps, mocks, or wiring per test phase or part variant, selected with a CLI flag at run time. |
26
+ | `testerkit-sites` | Testing multiple UUTs at once on one fixture — multi-site / multi-socket parallel production testing. |
27
+ | `testerkit-capture` | Capturing non-tabular evidence during a test — a waveform, a live sensor feed, a photo, a vendor capture file, a log. |
28
+ | `testerkit-data` | Reading, querying, or exporting existing test data — runs, steps, measurements, channels, files, events — via the models, CLI, Query API, or MCP. |
29
+ | `testerkit-analysis` | Computing a statistic across runs — yield, Pareto, Ppk, a trend, a retest rate, time-loss. |
30
+ | `testerkit-debug` | Triaging why a run failed, errored, looks wrong, or is missing. |
31
+ | `testerkit-interactive` | Pausing a test for operator input, building a custom live operator screen, or driving a station interactively outside pytest. |
32
+ | `testerkit-datasheets` | Importing an instrument or part datasheet PDF into catalog or part config. |
33
+
34
+ Each row above is a condensed reading of that skill's own `description`
35
+ frontmatter — the exact text an agent matches against. `testerkit-tests` is
36
+ the deliberate front door: it starts at a bare `def test_x(verify): ...`
37
+ with no station, no YAML, and only routes to a sibling skill (mocks,
38
+ stations, parts, profiles, sites, capture, data, analysis, debug,
39
+ interactive) once the request needs that layer.
40
+
41
+ (`testerkit-capture` writes non-tabular evidence, `testerkit-data` reads any store
42
+ back, `testerkit-analysis` computes statistics over the tabular data — three
43
+ distinct jobs on the data axis, no overlap.)
44
+
45
+ ## Single-source design
46
+
47
+ A `SKILL.md` is not a standalone essay. It carries the **action
48
+ playbook** — the decision the agent has to make and the judgment calls
49
+ around it (which verb, which config rung, what to warn about) — in a body
50
+ short enough to read in full every time the skill matches. Anything that's
51
+ already documented at length lives in the shipped docs, and the skill
52
+ tells the agent to read it there instead of re-stating it:
53
+
54
+ ```
55
+ ## Deeper
56
+ Read the docs:
57
+ testerkit docs show concepts/overview/tiers
58
+ testerkit docs show how-to/execution/writing-tests
59
+ ```
60
+
61
+ This keeps exactly one copy of any factual claim — the shipped `docs/`
62
+ tree — instead of a frozen paraphrase inside the skill that drifts the
63
+ next time the underlying behavior changes. `testerkit docs show <path>`
64
+ streams that same tree; see [The `testerkit docs` CLI](#the-testerkit-docs-cli)
65
+ below.
66
+
67
+ Two skills carry a `references/` subdirectory for content that has no
68
+ shipped-doc home:
69
+
70
+ - **`testerkit-datasheets`** — `references/catalog-pipeline.md`,
71
+ `test-pipeline.md`, `scaffold.md`, `process-queue.md` are the full
72
+ phase-by-phase orchestration contracts for its four import pipelines
73
+ (too long and too procedural for a `docs/` page). It also carries
74
+ `agents/` — five sub-agent prompts (`section-splitter`,
75
+ `scaffold-writer`, `section-extractor`, `section-writer`,
76
+ `section-reviewer`) the catalog pipeline spawns in sequence; each is
77
+ single-responsibility and not invoked directly by a user.
78
+ - **`testerkit-interactive`** — `references/live-ui-patterns.md` is the
79
+ decision table + anti-pattern catalog for building a custom NiceGUI
80
+ page against live channel data (subscribe-vs-raw-thread, when to
81
+ stop mutating from a callback).
82
+
83
+ Every other skill's `SKILL.md` is self-contained plus links out to `testerkit
84
+ docs show` — no `references/` directory.
85
+
86
+ ## Install / cross-tool
87
+
88
+ `testerkit setup <tool>` copies the packaged skill directories into the
89
+ tool's **native** Agent-Skills path — the same `SKILL.md` file, read
90
+ directly, no per-tool adapter or reformatting:
91
+
92
+ | Tool | Setup command | Skills copied to | Instructions file | MCP registration |
93
+ |---|---|---|---|---|
94
+ | Claude Code | `testerkit setup claude-code` | `.claude/skills/` | `CLAUDE.md` | `claude mcp add testerkit -- <bin> mcp serve` |
95
+ | OpenAI Codex | `testerkit setup codex` | `.agents/skills/` | `AGENTS.md` | printed for `~/.codex/config.toml` (Codex's home config; TesterKit doesn't write another tool's home config) |
96
+ | Cursor | `testerkit setup cursor` | `.cursor/skills/` | `AGENTS.md` | `.cursor/mcp.json` |
97
+ | GitHub Copilot | `testerkit setup copilot` | `.github/skills/` | `.github/copilot-instructions.md` + `AGENTS.md` | `.vscode/mcp.json` |
98
+
99
+ All four read `SKILL.md` natively — Claude Code and Codex from the start;
100
+ Cursor since 2.4 (shipped 2026-01-22); GitHub Copilot since its Agent
101
+ Skills support shipped 2025-12-18 (stable in VS Code early January 2026),
102
+ across VS Code/JetBrains agent mode, Copilot CLI, and the coding agent.
103
+ There is no Cursor `.mdc` rules file and no Copilot `.prompt.md` wrapper
104
+ generated — the single `SKILL.md` is the whole artifact.
105
+
106
+ The instructions file (`CLAUDE.md` / `AGENTS.md` /
107
+ `.github/copilot-instructions.md`) is marker-managed: `testerkit setup`
108
+ writes the TesterKit section between `<!-- testerkit:start -->` /
109
+ `<!-- testerkit:end -->` markers, creating the file if it doesn't exist or
110
+ replacing only that section if it does, leaving the rest of the file
111
+ alone.
112
+
113
+ `testerkit setup claude-desktop` and `testerkit setup cline` are MCP-only —
114
+ neither projects a native skills directory. `claude-desktop` bundles the
115
+ skill files inside the `.mcpb` extension as reference material rather
116
+ than a loose, natively-read directory; `cline` writes only
117
+ `cline_mcp_settings.json`.
118
+
119
+ Every `testerkit setup <tool>` command accepts `--print-only` to preview all
120
+ of its side effects — skills copied, instructions file created/updated,
121
+ MCP config — without writing anything.
122
+
123
+ ## The `testerkit docs` CLI
124
+
125
+ ```cli
126
+ $ testerkit docs list reference
127
+ reference/catalog/schema
128
+ reference/cli
129
+ reference/overview/pytest-native
130
+ reference/overview/skills
131
+ ...
132
+
133
+ $ testerkit docs show concepts/data/three-verbs
134
+ # Three verbs: configure, observe, verify
135
+ ...
136
+ ```
137
+
138
+ `testerkit docs list [section]` enumerates shipped doc pages (optionally
139
+ scoped to `concepts`, `how-to`, `reference`, `tutorial`, or `integration`);
140
+ `testerkit docs show <path>` prints one page to stdout, resolved with or
141
+ without a trailing `.md`. Both read from the same `docs/` tree that ships
142
+ in the installed package (a bundled copy in a wheel install, the repo's
143
+ `docs/` directory in an editable/source checkout) — the environment-stable
144
+ way for an agent (or a human) to read TesterKit documentation without baking
145
+ an absolute path into project config. This is the single source every
146
+ skill's "Deeper" section points at, and it replaces the removed `testerkit
147
+ refs` command — there is no separate curated reference corpus to keep in
148
+ sync.
149
+
150
+ ## MCP vs skills
151
+
152
+ Two different jobs, not overlapping ones:
153
+
154
+ - **MCP** is live execution and introspection — `testerkit_project`,
155
+ `testerkit_run`, `testerkit_match`, `testerkit_discover`, `testerkit_runs`,
156
+ `testerkit_metrics`, and the rest of the tool surface the `testerkit mcp
157
+ serve` server exposes. Calling one of these does something or reads
158
+ something right now. Full per-tool detail: [API reference →
159
+ MCP tools](../runtime/api.md#tools).
160
+ - **Skills** are judgment and procedure — which verb to reach for, which
161
+ config rung a request actually needs, what to check before declaring a
162
+ pipeline done. A skill matching doesn't execute anything by itself; it
163
+ tells the agent which MCP tools or CLI commands to call and in what
164
+ order, and where to read further before it does.
165
+
166
+ A skill's playbook routes into MCP tools and CLI commands as the steps
167
+ require — `testerkit-tests` step 6 runs plain `pytest`; `testerkit-datasheets`
168
+ saves through `testerkit_project(action="save", ...)`; `testerkit-debug` reads
169
+ back through `testerkit_runs` / `testerkit_steps`. Neither layer substitutes for
170
+ the other.
171
+
172
+ ## See also
173
+
174
+ - [Concepts: why AI integration](../../concepts/overview/ai-integration.md) — motivation
175
+ - [How-to: MCP integration](../../how-to/overview/mcp-integration.md) — registering the server and instructions file with each AI tool
176
+ - [Reference: MCP tools](../runtime/api.md#tools) — per-tool parameter detail
177
+ - [Reference: CLI](../cli.md) — `testerkit docs`, `testerkit setup`, and every other command