ragops 2.0.0__tar.gz → 2.0.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (408) hide show
  1. ragops-2.0.2/.claude-plugin/plugin.json +13 -0
  2. ragops-2.0.2/.codex-plugin/plugin.json +32 -0
  3. ragops-2.0.2/.github/workflows/ci.yml +72 -0
  4. ragops-2.0.2/.github/workflows/codeql.yml +29 -0
  5. {ragops-2.0.0 → ragops-2.0.2}/.github/workflows/publish-pypi.yml +13 -5
  6. ragops-2.0.2/.gitignore +172 -0
  7. ragops-2.0.2/.kimi-plugin/plugin.json +20 -0
  8. ragops-2.0.2/CHANGELOG.md +110 -0
  9. ragops-2.0.2/CONTRIBUTING.md +5 -0
  10. ragops-2.0.2/PKG-INFO +99 -0
  11. ragops-2.0.2/PRIVACY.md +13 -0
  12. ragops-2.0.2/README.md +63 -0
  13. ragops-2.0.2/SECURITY.md +5 -0
  14. ragops-2.0.2/SUPPORT.md +3 -0
  15. ragops-2.0.2/TERMS.md +9 -0
  16. ragops-2.0.2/apps/api/main.py +13 -0
  17. {ragops-2.0.0 → ragops-2.0.2}/apps/github_pr_comment.py +20 -6
  18. ragops-2.0.2/apps/web/index.html +19 -0
  19. ragops-2.0.2/assets/plugin-icon.png +0 -0
  20. ragops-2.0.2/assets/plugin-icon.svg +8 -0
  21. ragops-2.0.2/docs/ARCHITECTURE.md +30 -0
  22. ragops-2.0.2/docs/CONTRACTS.md +14 -0
  23. ragops-2.0.2/docs/OPERATIONS.md +15 -0
  24. ragops-2.0.2/docs/adapters.md +16 -0
  25. ragops-2.0.2/docs/demo/infographics/evidence-stack.svg +70 -0
  26. ragops-2.0.2/docs/demo/infographics/release-gate-flow.svg +72 -0
  27. ragops-2.0.2/docs/demo/ragops-demo-screenshot.png +0 -0
  28. ragops-2.0.2/docs/demo/ragops-demo.gif +0 -0
  29. ragops-2.0.2/docs/demo/ragops-pr-comment.png +0 -0
  30. ragops-2.0.2/docs/demo/social-preview.png +0 -0
  31. ragops-2.0.2/docs/examples/github-actions-ragops.yml +25 -0
  32. ragops-2.0.2/docs/examples/github-pr-comment.yml +30 -0
  33. {ragops-2.0.0 → ragops-2.0.2}/docs/examples/gitlab-ci-ragops.yml +11 -1
  34. ragops-2.0.2/docs/governance.md +16 -0
  35. ragops-2.0.2/docs/gtm/production-pilot.md +36 -0
  36. ragops-2.0.2/docs/releases/v1.0.0.md +13 -0
  37. ragops-2.0.2/docs/releases/v1.2.0.md +13 -0
  38. ragops-2.0.2/docs/releases/v2.0.0.md +15 -0
  39. ragops-2.0.2/docs/releases/v2.0.1.md +15 -0
  40. ragops-2.0.2/docs/releases/v2.0.2.md +20 -0
  41. ragops-2.0.2/docs/submission/DIRECTORY_SUBMISSION.md +76 -0
  42. ragops-2.0.2/docs/superpowers/plans/2026-08-23-ragops-2-implementation.md +470 -0
  43. ragops-2.0.2/docs/superpowers/plans/2026-08-23-ragops-post-release-hardening.md +180 -0
  44. ragops-2.0.2/docs/superpowers/plans/2026-08-24-ragops-2.0.2-release.md +212 -0
  45. ragops-2.0.2/docs/superpowers/specs/2026-08-23-ragops-production-flagship-adoption-design.md +296 -0
  46. ragops-2.0.2/examples/customer_support_demo/knowledge_base.json +74 -0
  47. ragops-2.0.2/examples/customer_support_demo/output/baseline.json +74 -0
  48. ragops-2.0.2/examples/customer_support_demo/output/candidate.json +70 -0
  49. ragops-2.0.2/examples/customer_support_demo/output/release-report.html +31 -0
  50. ragops-2.0.0/examples/japanese_troubleshooting_agent/results/comparison.md → ragops-2.0.2/examples/customer_support_demo/output/release-report.md +6 -5
  51. ragops-2.0.2/examples/customer_support_demo/response_generator.py +198 -0
  52. ragops-2.0.2/examples/customer_support_demo/run_demo.py +183 -0
  53. ragops-2.0.2/examples/customer_support_demo/scenario.json +116 -0
  54. ragops-2.0.2/examples/phoenix_recorded_scores/README.md +36 -0
  55. ragops-2.0.2/examples/phoenix_recorded_scores/__init__.py +1 -0
  56. ragops-2.0.2/examples/phoenix_recorded_scores/adapter.py +210 -0
  57. ragops-2.0.2/examples/phoenix_recorded_scores/annotations.json +43 -0
  58. ragops-2.0.2/examples/phoenix_recorded_scores/case_ids.json +4 -0
  59. ragops-2.0.2/examples/phoenix_recorded_scores/expected-envelope.json +18 -0
  60. {ragops-2.0.0 → ragops-2.0.2}/pyproject.toml +15 -4
  61. ragops-2.0.2/scenarios/japanese_troubleshooting/external-metric-policy.toml +5 -0
  62. ragops-2.0.2/scenarios/japanese_troubleshooting/external-metrics-baseline.json +8 -0
  63. ragops-2.0.2/scenarios/japanese_troubleshooting/external-metrics-candidate.json +8 -0
  64. ragops-2.0.2/scenarios/japanese_troubleshooting/failure-zoo-candidate.json +13 -0
  65. ragops-2.0.2/scenarios/japanese_troubleshooting/failure-zoo-policy.toml +8 -0
  66. ragops-2.0.2/scenarios/japanese_troubleshooting/retrieval-poisoning-candidate.json +12 -0
  67. ragops-2.0.2/scenarios/statistical_gate/baseline.json +22 -0
  68. ragops-2.0.2/scenarios/statistical_gate/candidate-block.json +22 -0
  69. ragops-2.0.2/scenarios/statistical_gate/candidate-pass.json +22 -0
  70. ragops-2.0.2/scenarios/statistical_gate/drift-policy.toml +8 -0
  71. ragops-2.0.2/scenarios/statistical_gate/policy.toml +10 -0
  72. ragops-2.0.2/scenarios/statistical_gate/run-plan.json +16 -0
  73. ragops-2.0.2/scenarios/statistical_gate/sequential-policy.toml +13 -0
  74. ragops-2.0.2/schemas/baseline-manifest-0.1.schema.json +56 -0
  75. ragops-2.0.2/schemas/calibration-report-1.0.schema.json +23 -0
  76. ragops-2.0.2/schemas/calibration-set-1.0.schema.json +14 -0
  77. ragops-2.0.2/schemas/dataset-manifest-1.0.schema.json +18 -0
  78. ragops-2.0.2/schemas/evaluator-drift-report-0.1.schema.json +65 -0
  79. ragops-2.0.2/schemas/evidence-bundle-1.0.schema.json +32 -0
  80. ragops-2.0.2/schemas/external-metrics-0.1.schema.json +30 -0
  81. ragops-2.0.2/schemas/local-usage-event-0.1.schema.json +15 -0
  82. ragops-2.0.2/schemas/provenance-diagnosis-0.1.schema.json +26 -0
  83. ragops-2.0.2/schemas/release-decision-1.0.schema.json +15 -0
  84. ragops-2.0.2/schemas/release-policy-2.0.schema.json +16 -0
  85. ragops-2.0.2/schemas/repeated-run-plan-0.1.schema.json +39 -0
  86. ragops-2.0.2/schemas/replay-bundle-0.1.schema.json +48 -0
  87. ragops-2.0.2/schemas/sequential-report-0.1.schema.json +72 -0
  88. ragops-2.0.2/schemas/statistical-report-0.1.schema.json +79 -0
  89. ragops-2.0.2/schemas/trace-graph-1.0.schema.json +14 -0
  90. ragops-2.0.2/scripts/local_release.py +308 -0
  91. ragops-2.0.2/scripts/record_demo.sh +41 -0
  92. ragops-2.0.2/scripts/render_demo_assets.py +170 -0
  93. ragops-2.0.2/skills/evaluate-ai-release/SKILL.md +30 -0
  94. ragops-2.0.2/skills/evaluate-ai-release/agents/openai.yaml +4 -0
  95. ragops-2.0.2/skills/evaluate-ai-release/references/commands.md +12 -0
  96. ragops-2.0.2/skills/evaluate-ai-release/scripts/run_ragops.py +19 -0
  97. ragops-2.0.2/skills/ragops-feature/SKILL.md +22 -0
  98. ragops-2.0.2/skills/ragops-presentation/SKILL.md +22 -0
  99. ragops-2.0.2/skills/ragops-release/SKILL.md +21 -0
  100. ragops-2.0.2/src/ragops/__init__.py +33 -0
  101. ragops-2.0.2/src/ragops/adapter_sdk.py +68 -0
  102. ragops-2.0.2/src/ragops/adapters/__init__.py +16 -0
  103. ragops-2.0.2/src/ragops/adapters/external_metrics.py +136 -0
  104. ragops-2.0.2/src/ragops/adapters/import_profiles.py +146 -0
  105. ragops-2.0.2/src/ragops/adapters/otel_genai.py +112 -0
  106. ragops-2.0.2/src/ragops/adapters/provider_metrics.py +149 -0
  107. ragops-2.0.2/src/ragops/adapters/repeated_runs.py +283 -0
  108. ragops-2.0.2/src/ragops/adapters/signing.py +123 -0
  109. ragops-2.0.2/src/ragops/api/__init__.py +1 -0
  110. ragops-2.0.2/src/ragops/api/main.py +403 -0
  111. ragops-2.0.2/src/ragops/baseline.py +249 -0
  112. ragops-2.0.2/src/ragops/calibration.py +256 -0
  113. ragops-2.0.2/src/ragops/ci.py +116 -0
  114. ragops-2.0.2/src/ragops/cli.py +1139 -0
  115. ragops-2.0.2/src/ragops/config.py +290 -0
  116. ragops-2.0.2/src/ragops/contracts.py +236 -0
  117. ragops-2.0.2/src/ragops/datasets.py +363 -0
  118. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/demo.py +161 -4
  119. ragops-2.0.2/src/ragops/distribution.py +103 -0
  120. ragops-2.0.2/src/ragops/drift.py +157 -0
  121. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/engine.py +15 -2
  122. ragops-2.0.2/src/ragops/evidence.py +298 -0
  123. ragops-2.0.2/src/ragops/explain.py +73 -0
  124. ragops-2.0.2/src/ragops/governance.py +314 -0
  125. ragops-2.0.2/src/ragops/models.py +366 -0
  126. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/plugins.py +70 -0
  127. ragops-2.0.2/src/ragops/policy_v2.py +621 -0
  128. ragops-2.0.2/src/ragops/provenance.py +65 -0
  129. ragops-2.0.2/src/ragops/reporters.py +318 -0
  130. ragops-2.0.2/src/ragops/sequential.py +286 -0
  131. ragops-2.0.2/src/ragops/slices.py +88 -0
  132. ragops-2.0.2/src/ragops/statistical.py +435 -0
  133. ragops-2.0.2/src/ragops/trace_graph.py +323 -0
  134. ragops-2.0.2/src/ragops/usage.py +134 -0
  135. ragops-2.0.2/src/ragops/web/__init__.py +1 -0
  136. ragops-2.0.2/src/ragops/web/app.js +14 -0
  137. ragops-2.0.2/src/ragops/web/index.html +19 -0
  138. ragops-2.0.2/src/ragops/web/styles.css +1 -0
  139. ragops-2.0.2/tests/test_adapter_sdk.py +85 -0
  140. ragops-2.0.2/tests/test_adoption_recipes.py +45 -0
  141. ragops-2.0.2/tests/test_api_contract.py +165 -0
  142. ragops-2.0.2/tests/test_baseline_manifest.py +127 -0
  143. ragops-2.0.2/tests/test_calibration.py +134 -0
  144. ragops-2.0.2/tests/test_contracts_v2.py +137 -0
  145. ragops-2.0.2/tests/test_datasets.py +171 -0
  146. {ragops-2.0.0 → ragops-2.0.2}/tests/test_evaluator_depth.py +46 -2
  147. ragops-2.0.2/tests/test_evaluator_drift.py +119 -0
  148. ragops-2.0.2/tests/test_evidence_bundle.py +155 -0
  149. ragops-2.0.2/tests/test_explain_ci.py +131 -0
  150. ragops-2.0.2/tests/test_external_metrics.py +192 -0
  151. ragops-2.0.2/tests/test_flagship_demo.py +67 -0
  152. ragops-2.0.2/tests/test_github_workflows.py +68 -0
  153. ragops-2.0.2/tests/test_governance.py +133 -0
  154. ragops-2.0.2/tests/test_local_release.py +177 -0
  155. {ragops-2.0.0 → ragops-2.0.2}/tests/test_opentelemetry_adapter.py +0 -1
  156. ragops-2.0.2/tests/test_packaged_api.py +58 -0
  157. ragops-2.0.2/tests/test_phoenix_recorded_scores.py +142 -0
  158. {ragops-2.0.0 → ragops-2.0.2}/tests/test_pilot.py +1 -1
  159. ragops-2.0.2/tests/test_plugin_release.py +52 -0
  160. ragops-2.0.2/tests/test_policy_v2.py +251 -0
  161. {ragops-2.0.0 → ragops-2.0.2}/tests/test_pr_comment_publisher.py +103 -2
  162. ragops-2.0.2/tests/test_provenance_diagnosis.py +58 -0
  163. ragops-2.0.2/tests/test_provider_metric_adapters.py +106 -0
  164. {ragops-2.0.0 → ragops-2.0.2}/tests/test_public_api.py +12 -1
  165. ragops-2.0.2/tests/test_readme_visuals.py +11 -0
  166. ragops-2.0.2/tests/test_reference_benchmark.py +80 -0
  167. ragops-2.0.2/tests/test_release_version.py +28 -0
  168. ragops-2.0.2/tests/test_repeated_runs.py +205 -0
  169. {ragops-2.0.0 → ragops-2.0.2}/tests/test_schema_files.py +8 -0
  170. ragops-2.0.2/tests/test_sequential_gate.py +146 -0
  171. ragops-2.0.2/tests/test_slice_distribution.py +125 -0
  172. ragops-2.0.2/tests/test_statistical_gate.py +276 -0
  173. ragops-2.0.2/tests/test_statistical_schema_instances.py +71 -0
  174. ragops-2.0.2/tests/test_trace_graph.py +170 -0
  175. ragops-2.0.2/tests/test_usage.py +142 -0
  176. ragops-2.0.2/tests/test_workbench.py +19 -0
  177. ragops-2.0.0/.github/ISSUE_TEMPLATE/bug.yml +0 -41
  178. ragops-2.0.0/.github/ISSUE_TEMPLATE/config.yml +0 -8
  179. ragops-2.0.0/.github/ISSUE_TEMPLATE/design-partner.yml +0 -48
  180. ragops-2.0.0/.github/ISSUE_TEMPLATE/evaluator.yml +0 -40
  181. ragops-2.0.0/.github/ISSUE_TEMPLATE/feature.yml +0 -38
  182. ragops-2.0.0/.github/pull_request_template.md +0 -26
  183. ragops-2.0.0/.github/workflows/ci.yml +0 -73
  184. ragops-2.0.0/.github/workflows/nightly.yml +0 -39
  185. ragops-2.0.0/.github/workflows/pages.yml +0 -31
  186. ragops-2.0.0/.github/workflows/ragops-gate-smoke.yml +0 -24
  187. ragops-2.0.0/.github/workflows/ragops-gate.yml +0 -133
  188. ragops-2.0.0/.github/workflows/ragops-pr-comment.yml +0 -27
  189. ragops-2.0.0/.github/workflows/release.yml +0 -54
  190. ragops-2.0.0/.gitignore +0 -13
  191. ragops-2.0.0/AGENTS.md +0 -36
  192. ragops-2.0.0/CHANGELOG.md +0 -168
  193. ragops-2.0.0/CODE_OF_CONDUCT.md +0 -32
  194. ragops-2.0.0/CONTRIBUTING.md +0 -71
  195. ragops-2.0.0/PKG-INFO +0 -289
  196. ragops-2.0.0/README.md +0 -258
  197. ragops-2.0.0/SECURITY.md +0 -32
  198. ragops-2.0.0/SUPPORT.md +0 -20
  199. ragops-2.0.0/apps/api/main.py +0 -214
  200. ragops-2.0.0/apps/web/index.html +0 -27
  201. ragops-2.0.0/design-qa.md +0 -74
  202. ragops-2.0.0/docs/architecture/adr/0001-local-first-core.md +0 -19
  203. ragops-2.0.0/docs/architecture/adr/0002-baseline-regression-gates.md +0 -23
  204. ragops-2.0.0/docs/architecture/adr/0003-reference-app-outside-core.md +0 -21
  205. ragops-2.0.0/docs/architecture/adr/0004-open-core-boundary.md +0 -22
  206. ragops-2.0.0/docs/architecture/adr/0005-scenario-0.2-metadata-and-citation-precision.md +0 -23
  207. ragops-2.0.0/docs/architecture/adr/0006-response-fixture-overlays.md +0 -22
  208. ragops-2.0.0/docs/architecture/adr/0007-offline-reference-graphrag-agent.md +0 -23
  209. ragops-2.0.0/docs/architecture/adr/0008-local-team-review-workflow.md +0 -22
  210. ragops-2.0.0/docs/architecture/adr/0009-adopt-mit-license.md +0 -38
  211. ragops-2.0.0/docs/architecture/adr/0010-repair-trace-04-schema-json.md +0 -32
  212. ragops-2.0.0/docs/architecture/adr/0011-reusable-pr-gate-is-an-adapter.md +0 -35
  213. ragops-2.0.0/docs/architecture/adr/0012-opentelemetry-export-is-an-example-adapter.md +0 -34
  214. ragops-2.0.0/docs/architecture/adr/0013-additive-demo-catalog-and-codepoint-budget.md +0 -37
  215. ragops-2.0.0/docs/architecture/adr/0014-separate-pr-evaluation-from-comment-publication.md +0 -40
  216. ragops-2.0.0/docs/architecture/adr/0015-portable-design-partner-measurement.md +0 -37
  217. ragops-2.0.0/docs/architecture/adr/0016-first-class-evaluator-gates.md +0 -51
  218. ragops-2.0.0/docs/architecture/adr/0017-api-authentication-and-input-bounds.md +0 -40
  219. ragops-2.0.0/docs/architecture/component-diagram.md +0 -22
  220. ragops-2.0.0/docs/architecture/context-diagram.md +0 -18
  221. ragops-2.0.0/docs/architecture/control-plane-alpha.md +0 -34
  222. ragops-2.0.0/docs/architecture/data-model.md +0 -33
  223. ragops-2.0.0/docs/architecture/pr-comment-publishing.md +0 -65
  224. ragops-2.0.0/docs/architecture/system-overview.md +0 -23
  225. ragops-2.0.0/docs/architecture/threat-model.md +0 -36
  226. ragops-2.0.0/docs/demo/cv-linkedin-copy.md +0 -29
  227. ragops-2.0.0/docs/demo/design-audit/01-current-desktop.jpg +0 -0
  228. ragops-2.0.0/docs/demo/design-audit/02-implementation-desktop.jpg +0 -0
  229. ragops-2.0.0/docs/demo/design-audit/04-implementation-mobile.jpg +0 -0
  230. ragops-2.0.0/docs/demo/design-audit/06-desktop-comparison.jpg +0 -0
  231. ragops-2.0.0/docs/demo/design-audit/09-limitations-section.jpg +0 -0
  232. ragops-2.0.0/docs/demo/design-audit/selected-open-source-adoption.png +0 -0
  233. ragops-2.0.0/docs/demo/infographics/evidence-stack.svg +0 -54
  234. ragops-2.0.0/docs/demo/infographics/release-gate-flow.svg +0 -54
  235. ragops-2.0.0/docs/demo/portfolio-case-study.md +0 -28
  236. ragops-2.0.0/docs/demo/portfolio-handoff.md +0 -30
  237. ragops-2.0.0/docs/demo/presentation-outline.md +0 -36
  238. ragops-2.0.0/docs/demo/screenshots/ragops-limitations-screen.jpg +0 -0
  239. ragops-2.0.0/docs/demo/screenshots/ragops-mobile-screen.jpg +0 -0
  240. ragops-2.0.0/docs/demo/screenshots/ragops-portfolio-desktop.jpg +0 -0
  241. ragops-2.0.0/docs/demo/screenshots/ragops-portfolio-mobile.jpg +0 -0
  242. ragops-2.0.0/docs/demo/screenshots/ragops-release-screen.jpg +0 -0
  243. ragops-2.0.0/docs/demo/screenshots/ragops-showcase-hero.jpg +0 -0
  244. ragops-2.0.0/docs/demo/social-preview.png +0 -0
  245. ragops-2.0.0/docs/demo/video-script.md +0 -32
  246. ragops-2.0.0/docs/demo/walkthrough.md +0 -45
  247. ragops-2.0.0/docs/design-audit/v1.9/01-start.png +0 -0
  248. ragops-2.0.0/docs/design-audit/v1.9/02-pass-decision.png +0 -0
  249. ragops-2.0.0/docs/design-audit/v1.9/03-invalid-json-stale-decision.png +0 -0
  250. ragops-2.0.0/docs/design-audit/v1.9/04-fixed-pass-decision.png +0 -0
  251. ragops-2.0.0/docs/design-audit/v1.9/05-fixed-invalid-json.png +0 -0
  252. ragops-2.0.0/docs/design-audit/v1.9/workbench-audit.md +0 -88
  253. ragops-2.0.0/docs/engineering/export-your-first-trace.md +0 -56
  254. ragops-2.0.0/docs/engineering/github-pr-gate.md +0 -84
  255. ragops-2.0.0/docs/engineering/gitlab-ci-gate.md +0 -40
  256. ragops-2.0.0/docs/engineering/handbook.md +0 -31
  257. ragops-2.0.0/docs/engineering/observability.md +0 -28
  258. ragops-2.0.0/docs/engineering/provider-adapters.md +0 -32
  259. ragops-2.0.0/docs/engineering/pypi-publishing.md +0 -49
  260. ragops-2.0.0/docs/engineering/testing-strategy.md +0 -24
  261. ragops-2.0.0/docs/evaluation/answer-length-budget.md +0 -43
  262. ragops-2.0.0/docs/evaluation/benchmark-report-v0.2.md +0 -69
  263. ragops-2.0.0/docs/evaluation/evaluator-gates.md +0 -69
  264. ragops-2.0.0/docs/evaluation/red-team-plan.md +0 -28
  265. ragops-2.0.0/docs/evaluation/scenario-taxonomy.md +0 -26
  266. ragops-2.0.0/docs/evaluation/strategy.md +0 -40
  267. ragops-2.0.0/docs/getting-started.md +0 -89
  268. ragops-2.0.0/docs/gtm/design-partner-outreach.md +0 -39
  269. ragops-2.0.0/docs/gtm/design-partner-pilot-runbook.md +0 -49
  270. ragops-2.0.0/docs/gtm/github-launch-checklist.md +0 -38
  271. ragops-2.0.0/docs/gtm/pilot-measurement-plan.md +0 -83
  272. ragops-2.0.0/docs/gtm/synthetic-pilot-report.md +0 -55
  273. ragops-2.0.0/docs/product/commercialization.md +0 -36
  274. ragops-2.0.0/docs/product/competitive-analysis.md +0 -25
  275. ragops-2.0.0/docs/product/jobs-to-be-done.md +0 -28
  276. ragops-2.0.0/docs/product/oss-launch-plan.md +0 -58
  277. ragops-2.0.0/docs/product/personas.md +0 -29
  278. ragops-2.0.0/docs/product/product_thesis.md +0 -41
  279. ragops-2.0.0/docs/product/requirements-v0.1.md +0 -40
  280. ragops-2.0.0/docs/product/requirements-v0.2.md +0 -38
  281. ragops-2.0.0/docs/product/requirements-v1.5.md +0 -45
  282. ragops-2.0.0/docs/product/requirements-v1.6.md +0 -39
  283. ragops-2.0.0/docs/product/requirements-v1.7.md +0 -58
  284. ragops-2.0.0/docs/product/requirements-v1.8.md +0 -54
  285. ragops-2.0.0/docs/product/requirements-v2.0.md +0 -61
  286. ragops-2.0.0/docs/product/roadmap.md +0 -88
  287. ragops-2.0.0/docs/product/success-metrics.md +0 -35
  288. ragops-2.0.0/docs/project/backlog.md +0 -74
  289. ragops-2.0.0/docs/project/charter.md +0 -32
  290. ragops-2.0.0/docs/project/decision-log.md +0 -17
  291. ragops-2.0.0/docs/project/m1-acceptance.md +0 -40
  292. ragops-2.0.0/docs/project/m2-acceptance.md +0 -24
  293. ragops-2.0.0/docs/project/m3-acceptance.md +0 -24
  294. ragops-2.0.0/docs/project/m4-acceptance.md +0 -22
  295. ragops-2.0.0/docs/project/m5-acceptance.md +0 -26
  296. ragops-2.0.0/docs/project/milestones.md +0 -14
  297. ragops-2.0.0/docs/project/program-completion-audit.md +0 -65
  298. ragops-2.0.0/docs/project/release-plan.md +0 -26
  299. ragops-2.0.0/docs/project/risk-register.md +0 -16
  300. ragops-2.0.0/docs/project/v1-acceptance.md +0 -29
  301. ragops-2.0.0/docs/project/v1.5-acceptance.md +0 -60
  302. ragops-2.0.0/docs/project/v1.6-acceptance.md +0 -38
  303. ragops-2.0.0/docs/project/v1.7-acceptance.md +0 -48
  304. ragops-2.0.0/docs/project/v1.8-acceptance.md +0 -47
  305. ragops-2.0.0/docs/project/v2.0-acceptance.md +0 -50
  306. ragops-2.0.0/docs/project/work-breakdown.md +0 -34
  307. ragops-2.0.0/docs/releases/v1.5.0.md +0 -36
  308. ragops-2.0.0/docs/releases/v1.6.0.md +0 -32
  309. ragops-2.0.0/docs/releases/v1.7.0.md +0 -36
  310. ragops-2.0.0/docs/releases/v1.8.0.md +0 -33
  311. ragops-2.0.0/docs/releases/v2.0.0.md +0 -55
  312. ragops-2.0.0/examples/japanese_troubleshooting_agent/README.md +0 -57
  313. ragops-2.0.0/examples/japanese_troubleshooting_agent/architecture.md +0 -26
  314. ragops-2.0.0/examples/japanese_troubleshooting_agent/discovery.md +0 -30
  315. ragops-2.0.0/examples/japanese_troubleshooting_agent/executive-report.md +0 -26
  316. ragops-2.0.0/examples/japanese_troubleshooting_agent/rollout-plan.md +0 -31
  317. ragops-2.0.0/examples/opentelemetry_trace_adapter/README.md +0 -43
  318. ragops-2.0.0/site/apple-touch-icon.png +0 -0
  319. ragops-2.0.0/site/favicon.png +0 -0
  320. ragops-2.0.0/site/index.html +0 -216
  321. ragops-2.0.0/site/styles.css +0 -1
  322. ragops-2.0.0/skills/ragops-feature/SKILL.md +0 -18
  323. ragops-2.0.0/skills/ragops-presentation/SKILL.md +0 -19
  324. ragops-2.0.0/skills/ragops-release/SKILL.md +0 -18
  325. ragops-2.0.0/src/ragops/__init__.py +0 -16
  326. ragops-2.0.0/src/ragops/adapters/__init__.py +0 -5
  327. ragops-2.0.0/src/ragops/cli.py +0 -305
  328. ragops-2.0.0/src/ragops/config.py +0 -70
  329. ragops-2.0.0/src/ragops/models.py +0 -143
  330. ragops-2.0.0/src/ragops/reporters.py +0 -124
  331. ragops-2.0.0/tests/test_adoption_recipes.py +0 -34
  332. ragops-2.0.0/tests/test_api_contract.py +0 -69
  333. ragops-2.0.0/tests/test_github_workflows.py +0 -88
  334. ragops-2.0.0/tests/test_readme_visuals.py +0 -32
  335. ragops-2.0.0/tests/test_reference_benchmark.py +0 -25
  336. ragops-2.0.0/tests/test_release_version.py +0 -15
  337. ragops-2.0.0/tests/test_showcase_site.py +0 -72
  338. ragops-2.0.0/tests/test_workbench.py +0 -18
  339. {ragops-2.0.0 → ragops-2.0.2}/.dockerignore +0 -0
  340. {ragops-2.0.0 → ragops-2.0.2}/Dockerfile +0 -0
  341. {ragops-2.0.0 → ragops-2.0.2}/LICENSE +0 -0
  342. {ragops-2.0.0 → ragops-2.0.2}/apps/__init__.py +0 -0
  343. {ragops-2.0.0 → ragops-2.0.2}/apps/api/__init__.py +0 -0
  344. {ragops-2.0.0 → ragops-2.0.2}/compose.yaml +0 -0
  345. {ragops-2.0.0 → ragops-2.0.2}/docs/gtm/pilot-fixtures/synthetic-economics.json +0 -0
  346. {ragops-2.0.0 → ragops-2.0.2}/docs/gtm/pilot-fixtures/synthetic-manifest.json +0 -0
  347. {ragops-2.0.0 → ragops-2.0.2}/docs/gtm/pilot-fixtures/synthetic-observations.jsonl +0 -0
  348. {ragops-2.0.0 → ragops-2.0.2}/examples/__init__.py +0 -0
  349. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/__init__.py +0 -0
  350. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/agent.py +0 -0
  351. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/cli.py +0 -0
  352. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/data/documents.json +0 -0
  353. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/data/graph.json +0 -0
  354. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/results/graph-traces.jsonl +0 -0
  355. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/results/lexical-traces.jsonl +0 -0
  356. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/scenario.json +0 -0
  357. {ragops-2.0.0 → ragops-2.0.2}/examples/japanese_troubleshooting_agent/suite.json +0 -0
  358. {ragops-2.0.0 → ragops-2.0.2}/examples/opentelemetry_trace_adapter/__init__.py +0 -0
  359. {ragops-2.0.0 → ragops-2.0.2}/examples/opentelemetry_trace_adapter/adapter.py +0 -0
  360. {ragops-2.0.0 → ragops-2.0.2}/examples/opentelemetry_trace_adapter/spans.jsonl +0 -0
  361. {ragops-2.0.0 → ragops-2.0.2}/ragops.toml +0 -0
  362. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/attack-pack.json +0 -0
  363. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/benchmark-adversarial.json +0 -0
  364. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/benchmark-baseline.json +0 -0
  365. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/benchmark-regressed.json +0 -0
  366. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/benchmark-v0.2.json +0 -0
  367. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/evaluation-policy.toml +0 -0
  368. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/regressed_responses.json +0 -0
  369. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/sample_responses.json +0 -0
  370. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/sample_traces.jsonl +0 -0
  371. {ragops-2.0.0 → ragops-2.0.2}/scenarios/japanese_troubleshooting/scenario.json +0 -0
  372. {ragops-2.0.0 → ragops-2.0.2}/schemas/attack-pack-0.1.schema.json +0 -0
  373. {ragops-2.0.0 → ragops-2.0.2}/schemas/pilot-economics-0.1.schema.json +0 -0
  374. {ragops-2.0.0 → ragops-2.0.2}/schemas/pilot-manifest-0.1.schema.json +0 -0
  375. {ragops-2.0.0 → ragops-2.0.2}/schemas/pilot-observation-0.1.schema.json +0 -0
  376. {ragops-2.0.0 → ragops-2.0.2}/schemas/report-0.2.schema.json +0 -0
  377. {ragops-2.0.0 → ragops-2.0.2}/schemas/response-fixture-0.2.schema.json +0 -0
  378. {ragops-2.0.0 → ragops-2.0.2}/schemas/responses-0.1.schema.json +0 -0
  379. {ragops-2.0.0 → ragops-2.0.2}/schemas/scenario-0.1.schema.json +0 -0
  380. {ragops-2.0.0 → ragops-2.0.2}/schemas/scenario-0.2.schema.json +0 -0
  381. {ragops-2.0.0 → ragops-2.0.2}/schemas/trace-0.3.schema.json +0 -0
  382. {ragops-2.0.0 → ragops-2.0.2}/schemas/trace-0.4.schema.json +0 -0
  383. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/adapters/http.py +0 -0
  384. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/benchmarks.py +0 -0
  385. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/control_plane.py +0 -0
  386. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/evaluators.py +0 -0
  387. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/loader.py +0 -0
  388. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/pilot.py +0 -0
  389. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/providers/__init__.py +0 -0
  390. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/providers/openai_responses.py +0 -0
  391. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/store.py +0 -0
  392. {ragops-2.0.0 → ragops-2.0.2}/src/ragops/traces.py +0 -0
  393. {ragops-2.0.0 → ragops-2.0.2}/tests/test_answer_length_cli.py +0 -0
  394. {ragops-2.0.0 → ragops-2.0.2}/tests/test_benchmarks.py +0 -0
  395. {ragops-2.0.0 → ragops-2.0.2}/tests/test_comparison.py +0 -0
  396. {ragops-2.0.0 → ragops-2.0.2}/tests/test_config.py +0 -0
  397. {ragops-2.0.0 → ragops-2.0.2}/tests/test_control_plane.py +0 -0
  398. {ragops-2.0.0 → ragops-2.0.2}/tests/test_demo.py +0 -0
  399. {ragops-2.0.0 → ragops-2.0.2}/tests/test_engine.py +0 -0
  400. {ragops-2.0.0 → ragops-2.0.2}/tests/test_http_adapter.py +0 -0
  401. {ragops-2.0.0 → ragops-2.0.2}/tests/test_loader.py +0 -0
  402. {ragops-2.0.0 → ragops-2.0.2}/tests/test_openai_provider.py +0 -0
  403. {ragops-2.0.0 → ragops-2.0.2}/tests/test_plugins_and_traces.py +0 -0
  404. {ragops-2.0.0 → ragops-2.0.2}/tests/test_reference_agent.py +0 -0
  405. {ragops-2.0.0 → ragops-2.0.2}/tests/test_response_fixtures.py +0 -0
  406. {ragops-2.0.0 → ragops-2.0.2}/tests/test_scenario_v02_and_attacks.py +0 -0
  407. {ragops-2.0.0 → ragops-2.0.2}/tests/test_store_and_html.py +0 -0
  408. {ragops-2.0.0 → ragops-2.0.2}/tests/test_team_workflow.py +0 -0
@@ -0,0 +1,13 @@
1
+ {
2
+ "name": "ragops",
3
+ "version": "2.0.2",
4
+ "description": "Explainable regression tests and release gates for RAG systems and AI agents.",
5
+ "author": {
6
+ "name": "Thang Luu",
7
+ "url": "https://github.com/thangldw"
8
+ },
9
+ "homepage": "https://github.com/thangldw/ragops",
10
+ "repository": "https://github.com/thangldw/ragops",
11
+ "license": "MIT",
12
+ "keywords": ["rag", "ai-agents", "evaluation", "release-gates"]
13
+ }
@@ -0,0 +1,32 @@
1
+ {
2
+ "name": "ragops",
3
+ "version": "2.0.2",
4
+ "description": "Offline, explainable release gates and evidence bundles for RAG systems and AI agents.",
5
+ "author": {
6
+ "name": "Thang Luu",
7
+ "url": "https://github.com/thangldw"
8
+ },
9
+ "homepage": "https://github.com/thangldw/ragops",
10
+ "repository": "https://github.com/thangldw/ragops",
11
+ "license": "MIT",
12
+ "keywords": ["rag", "ai-agents", "evaluation", "release-gates", "regression-testing"],
13
+ "skills": "./skills/",
14
+ "interface": {
15
+ "displayName": "RAGOps",
16
+ "shortDescription": "Gate AI releases with evidence",
17
+ "longDescription": "Compare recorded candidate behavior with an accepted baseline, localize regressions by policy and case, and produce verifiable release evidence without requiring a model provider.",
18
+ "developerName": "DUC THANG LUU",
19
+ "category": "Productivity",
20
+ "capabilities": ["Read"],
21
+ "websiteURL": "https://github.com/thangldw/ragops",
22
+ "privacyPolicyURL": "https://github.com/thangldw/ragops/blob/main/PRIVACY.md",
23
+ "termsOfServiceURL": "https://github.com/thangldw/ragops/blob/main/TERMS.md",
24
+ "defaultPrompt": [
25
+ "Compare this candidate run with the accepted baseline.",
26
+ "Explain why this RAG release is blocked.",
27
+ "Create a review-ready release gate report."
28
+ ],
29
+ "logo": "./assets/plugin-icon.png",
30
+ "composerIcon": "./assets/plugin-icon.png"
31
+ }
32
+ }
@@ -0,0 +1,72 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+
8
+ permissions:
9
+ contents: read
10
+
11
+ jobs:
12
+ test:
13
+ runs-on: ubuntu-latest
14
+ strategy:
15
+ fail-fast: false
16
+ matrix:
17
+ python-version: ["3.11", "3.12", "3.13"]
18
+ steps:
19
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
20
+ with:
21
+ persist-credentials: false
22
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
23
+ with:
24
+ python-version: ${{ matrix.python-version }}
25
+ cache: pip
26
+ - name: Install test dependencies
27
+ run: python -m pip install --disable-pip-version-check -e '.[dev]' build
28
+ - name: Lint
29
+ run: ruff check .
30
+ - name: Test
31
+ env:
32
+ PYTEST_DISABLE_PLUGIN_AUTOLOAD: "1"
33
+ run: pytest -q
34
+ - name: Build and inspect distributions
35
+ run: |
36
+ python -m build
37
+ python -m zipfile --list dist/ragops-*.whl | grep 'ragops/web/index.html'
38
+ python -m zipfile --list dist/ragops-*.whl | grep 'ragops/api/main.py'
39
+ - name: Run credential-free release gate
40
+ run: |
41
+ ragops demo --profile engineer --output demo-output
42
+ ragops evidence verify --bundle demo-output/evidence
43
+
44
+ windows-no-clone:
45
+ runs-on: windows-latest
46
+ steps:
47
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
48
+ with:
49
+ python-version: "3.12"
50
+ - uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
51
+ with:
52
+ version: "0.12.5"
53
+ enable-cache: false
54
+ ignore-empty-workdir: true
55
+ - name: Run published credential-free demo without checkout
56
+ shell: pwsh
57
+ run: |
58
+ $demoJson = (& uvx --from "ragops==2.0.1" ragops demo --profile engineer --output demo-output | Out-String)
59
+ if ($LASTEXITCODE -ne 0) {
60
+ throw "ragops demo returned $LASTEXITCODE"
61
+ }
62
+ $summary = $demoJson | ConvertFrom-Json
63
+ if ($summary.candidate_decision -ne "BLOCK") {
64
+ throw "expected BLOCK, got $($summary.candidate_decision)"
65
+ }
66
+ if (-not (Test-Path "demo-output/release-report.html" -PathType Leaf)) {
67
+ throw "release-report.html was not generated"
68
+ }
69
+ uvx --from "ragops==2.0.1" ragops evidence verify --bundle demo-output/evidence
70
+ if ($LASTEXITCODE -ne 0) {
71
+ throw "evidence verification returned $LASTEXITCODE"
72
+ }
@@ -0,0 +1,29 @@
1
+ name: CodeQL
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+ schedule:
8
+ - cron: "17 3 * * 1"
9
+
10
+ permissions:
11
+ contents: read
12
+
13
+ jobs:
14
+ analyze:
15
+ runs-on: ubuntu-latest
16
+ permissions:
17
+ contents: read
18
+ security-events: write
19
+ steps:
20
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
21
+ with:
22
+ persist-credentials: false
23
+ - uses: github/codeql-action/init@db488ddef3bf6cb639b32c2e9a7c0a7ea8271d28 # v4
24
+ with:
25
+ languages: python
26
+ build-mode: none
27
+ - uses: github/codeql-action/analyze@db488ddef3bf6cb639b32c2e9a7c0a7ea8271d28 # v4
28
+ with:
29
+ category: "/language:python"
@@ -16,10 +16,10 @@ jobs:
16
16
  contents: read
17
17
  id-token: write
18
18
  steps:
19
- - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
19
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
20
20
  with:
21
21
  ref: ${{ inputs.tag }}
22
- - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
22
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
23
23
  with:
24
24
  python-version: "3.12"
25
25
  - name: Verify tag and package version
@@ -33,8 +33,14 @@ jobs:
33
33
 
34
34
  tag = sys.argv[1]
35
35
  version = tomllib.loads(Path("pyproject.toml").read_text())["project"]["version"]
36
- if tag != f"v{version}":
37
- raise SystemExit(f"tag {tag!r} does not match package version v{version}")
36
+ parts = version.split(".")
37
+ if len(parts) != 3 or not all(part.isdigit() for part in parts):
38
+ raise SystemExit("package version must use numeric MAJOR.MINOR.PATCH")
39
+ expected_tag = f"v{version}"
40
+ if tag != expected_tag:
41
+ raise SystemExit(
42
+ f"tag {tag!r} does not match package version {version!r}"
43
+ )
38
44
  print(f"publishing ragops {version}")
39
45
  PY
40
46
  - name: Download promoted GitHub Release artifacts
@@ -47,6 +53,7 @@ jobs:
47
53
  --pattern 'ragops-*.whl' \
48
54
  --pattern 'ragops-*.tar.gz' \
49
55
  --pattern 'ragops-*.cdx.json' \
56
+ --pattern 'ragops-chatgpt-plugin-*.zip' \
50
57
  --pattern 'SHA256SUMS' \
51
58
  --dir release-assets
52
59
  (cd release-assets && sha256sum --check SHA256SUMS)
@@ -54,4 +61,5 @@ jobs:
54
61
  test "$(find release-assets -maxdepth 1 -name 'ragops-*.tar.gz' | wc -l)" -eq 1
55
62
  cp release-assets/ragops-*.whl release-assets/ragops-*.tar.gz dist/
56
63
  - name: Publish distributions with Trusted Publishing
57
- uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
64
+ # Upstream container images are published for exact release tags, not commit-SHA refs.
65
+ uses: pypa/gh-action-pypi-publish@v1.14.2
@@ -0,0 +1,172 @@
1
+ # Canonical ignore rules for repositories owned by thangldw.
2
+ # Keep this file identical across repositories.
3
+
4
+ # Operating systems and file managers
5
+ .DS_Store
6
+ .AppleDouble
7
+ .LSOverride
8
+ ._*
9
+ .DocumentRevisions-V100
10
+ .Spotlight-V100
11
+ .TemporaryItems
12
+ .Trashes
13
+ .VolumeIcon.icns
14
+ .fseventsd
15
+ .localized
16
+ __MACOSX/
17
+ Thumbs.db
18
+ Thumbs.db:encryptable
19
+ ehthumbs.db
20
+ ehthumbs_vista.db
21
+ [Dd]esktop.ini
22
+ $RECYCLE.BIN/
23
+ *.lnk
24
+
25
+ # Editors and local assistants
26
+ .idea/
27
+ .vscode/
28
+ *.code-workspace
29
+ *.sublime-project
30
+ *.sublime-workspace
31
+ *.swp
32
+ *.swo
33
+ *~
34
+ .history/
35
+ .claude/
36
+ .codex/
37
+
38
+ # Secrets and local configuration
39
+ .env
40
+ .env.*
41
+ !.env.example
42
+ !.env.*.example
43
+ *.local
44
+ *.secret
45
+ *.secrets
46
+ *.key
47
+ *.pem
48
+ *.p12
49
+ *.pfx
50
+ credentials.json
51
+ service-account*.json
52
+
53
+ # Logs, temporary files, backups and local databases
54
+ *.log
55
+ *.tmp
56
+ *.temp
57
+ *.bak
58
+ *.backup
59
+ *.orig
60
+ *.rej
61
+ *.stackdump
62
+ tmp/
63
+ temp/
64
+ logs/
65
+ backups/
66
+ *.db
67
+ *.db-journal
68
+ *.db-shm
69
+ *.db-wal
70
+ *.sqlite
71
+ *.sqlite3
72
+
73
+ # JavaScript and TypeScript
74
+ node_modules/
75
+ .npm/
76
+ .npm-cache/
77
+ .yarn/
78
+ .pnpm-store/
79
+ .vite/
80
+ .next/
81
+ .nuxt/
82
+ .svelte-kit/
83
+ dist/
84
+ build/
85
+ generated/
86
+ coverage/
87
+ *.tsbuildinfo
88
+ playwright-report/
89
+ test-results/
90
+ blob-report/
91
+ src/generated/
92
+ supabase/.temp/
93
+
94
+ # Python
95
+ __pycache__/
96
+ *.py[cod]
97
+ *$py.class
98
+ .venv/
99
+ .venv*/
100
+ venv/
101
+ env/
102
+ ENV/
103
+ .python-version
104
+ .pytest_cache/
105
+ .ruff_cache/
106
+ .mypy_cache/
107
+ .pyre/
108
+ .pytype/
109
+ .hypothesis/
110
+ .tox/
111
+ .nox/
112
+ .coverage
113
+ .coverage.*
114
+ htmlcov/
115
+ *.egg-info/
116
+ .eggs/
117
+ wheels/
118
+ pip-wheel-metadata/
119
+ reports/
120
+ .ipynb_checkpoints/
121
+ **/.ipynb_checkpoints/
122
+
123
+ # Rust, Go, Java and .NET
124
+ **/target/
125
+ **/gen/schemas/
126
+ vendor/
127
+ bin/
128
+ obj/
129
+ *.class
130
+ *.jar
131
+ *.war
132
+ *.nupkg
133
+
134
+ # Swift and Apple build artifacts
135
+ **/.build/
136
+ DerivedData/
137
+ *.xcuserstate
138
+ *.xcworkspace/xcuserdata/
139
+ *.xcodeproj/xcuserdata/
140
+ *.app/
141
+ *.dmg
142
+ **/release/
143
+
144
+ # Windows installers and packaged artifacts
145
+ *.cab
146
+ *.msi
147
+ *.msix
148
+ *.msm
149
+ *.msp
150
+ *.exe
151
+ *.zip
152
+ release-artifacts/
153
+ artifacts/
154
+
155
+ # QA and generated visual output
156
+ qa/
157
+ qa-*.png
158
+ qa-*.jpg
159
+ qa-*.jpeg
160
+ audit-*.png
161
+ prototype-*.png
162
+ gameplay-*.png
163
+ design-qa.md
164
+
165
+ # Project-specific generated resources
166
+ public/ocr/paddleocr/
167
+ src-tauri/generated-resources/ocr/*
168
+ !src-tauri/generated-resources/ocr/.gitkeep
169
+ analysis/outputs/
170
+ analysis/*-artifact.json
171
+ analysis/*.html
172
+ NamiQuant-Claude-Design.html
@@ -0,0 +1,20 @@
1
+ {
2
+ "name": "ragops",
3
+ "version": "2.0.2",
4
+ "description": "Explainable regression tests and release gates for RAG systems and AI agents.",
5
+ "author": {
6
+ "name": "Thang Luu",
7
+ "url": "https://github.com/thangldw"
8
+ },
9
+ "homepage": "https://github.com/thangldw/ragops",
10
+ "license": "MIT",
11
+ "keywords": ["rag", "ai-agents", "evaluation", "release-gates"],
12
+ "skills": "./skills/",
13
+ "interface": {
14
+ "displayName": "RAGOps",
15
+ "shortDescription": "Gate RAG and agent releases with evidence",
16
+ "longDescription": "Compare candidate AI behavior with an accepted baseline, explain failed gates, and produce reviewable release evidence without requiring a model provider.",
17
+ "developerName": "Thang Luu",
18
+ "websiteURL": "https://github.com/thangldw/ragops"
19
+ }
20
+ }
@@ -0,0 +1,110 @@
1
+ # RAGOps changelog / Lịch sử thay đổi / 変更履歴
2
+
3
+ ## Unreleased
4
+
5
+ ## [2.0.2] - 2026-08-24
6
+
7
+ ### English
8
+
9
+ - Added a deterministic synthetic retrieval-poisoning Failure Zoo specimen with explicit unsupported-citation, unsupported-claim, and forbidden-output findings.
10
+ - Added checkout-free Windows verification of the published package, least-privilege CodeQL scanning, and protected-branch release gates.
11
+ - Added disabled-by-default local usage evidence that records only command, exit code, version, and UTC timestamp without arguments or paths.
12
+ - Generate the CycloneDX SBOM from the built wheel in a dependency-free environment so its root version is exact and no local build path is disclosed.
13
+
14
+ ### Tiếng Việt
15
+
16
+ - Bổ sung mẫu synthetic retrieval poisoning xác định với các finding rõ ràng về citation không được hỗ trợ, claim không có căn cứ và output bị cấm.
17
+ - Bổ sung kiểm tra Windows không checkout từ package đã public, CodeQL theo least privilege và release gate bảo vệ nhánh chính.
18
+ - Bổ sung bằng chứng sử dụng local mặc định tắt, chỉ ghi command, exit code, version và UTC timestamp, không ghi arguments hoặc paths.
19
+ - Tạo CycloneDX SBOM từ wheel đã build trong môi trường không dependency để version gốc chính xác và không lộ local build path.
20
+
21
+ ### 日本語
22
+
23
+ - unsupported citation、unsupported claim、forbidden output を明示する決定的な synthetic retrieval-poisoning Failure Zoo specimen を追加しました。
24
+ - 公開 package の checkout-free Windows 検証、least-privilege CodeQL、保護 branch release gate を追加しました。
25
+ - デフォルト無効で、引数や path を記録せず command、exit code、version、UTC timestamp のみを残す local usage evidence を追加しました。
26
+ - build 済み wheel を dependency-free environment で読み取り、正確な root version と local build path を含まない CycloneDX SBOM を生成します。
27
+
28
+ ## [2.0.1] - 2026-08-23
29
+
30
+ ### English
31
+
32
+ - Republished the unchanged RAGOps 2.0 feature set under a new patch version because PyPI permanently reserves filenames from deleted releases.
33
+ - Corrected the Trusted Publishing action reference so GitHub Actions uses the published v1.14.2 container with Core Metadata 2.5 support.
34
+
35
+ ### Tiếng Việt
36
+
37
+ - Phát hành lại toàn bộ tính năng RAGOps 2.0 dưới patch version mới vì PyPI khóa vĩnh viễn filename của release đã bị xóa.
38
+ - Sửa tham chiếu Trusted Publishing để GitHub Actions dùng container v1.14.2 đã phát hành và hỗ trợ Core Metadata 2.5.
39
+
40
+ ### 日本語
41
+
42
+ - 削除済みリリースのファイル名を PyPI が恒久的に予約するため、RAGOps 2.0 の機能を新しいパッチ版で再公開しました。
43
+ - Trusted Publishing の参照を修正し、Core Metadata 2.5 対応の公開済み v1.14.2 コンテナを使用します。
44
+
45
+ ## [2.0.0] - 2026-08-23
46
+
47
+ ### English
48
+
49
+ - Added versioned contract migration, content-addressed evidence bundles, slice/distribution gates, evaluator calibration, trace-graph evaluation, dataset leakage controls, CI renderers, portable vendor adapters, and audited blind review.
50
+ - Packaged the optional FastAPI workbench with CSP-safe DOM rendering and added Python 3.11-3.13 CI.
51
+ - Upgraded the synthetic flagship demo to generate canonical JSON, Markdown, HTML, JUnit, SARIF, GitHub Summary, and verified evidence for executive, engineer, and auditor profiles.
52
+ - Added a deterministic ChatGPT skills-only plugin bundle and v2 release evidence.
53
+
54
+ ### Tiếng Việt
55
+
56
+ - Bổ sung contract có version, evidence bundle content-addressed, slice/distribution gate, calibration, trace graph, chống dataset leakage, CI renderer, vendor adapter và blind review có audit.
57
+ - Đóng gói FastAPI workbench tùy chọn với CSP/DOM an toàn và CI Python 3.11-3.13.
58
+ - Nâng demo synthetic để xuất cùng một quyết định qua JSON, Markdown, HTML, JUnit, SARIF, GitHub Summary và evidence đã xác minh.
59
+
60
+ ### 日本語
61
+
62
+ - versioned contract、content-addressed evidence、slice/distribution gate、calibration、trace graph、dataset leakage check、CI renderer、vendor adapter、blind review audit を追加しました。
63
+ - CSP-safe な任意 FastAPI workbench と Python 3.11-3.13 CI を同梱しました。
64
+ - synthetic flagship demo は同一判定を JSON、Markdown、HTML、JUnit、SARIF、GitHub Summary、検証済み evidence に出力します。
65
+
66
+ ## [1.2.0] - 2026-08-02
67
+
68
+ ### English
69
+
70
+ - Added skills-only plugin packages and directory metadata for ChatGPT, Codex, Claude Code, Cowork and Kimi Code.
71
+ - Added the credential-free customer-support release-gate demonstration and its recorded evidence.
72
+ - Restored a release-only GitHub Actions workflow for PyPI Trusted Publishing; runtime integrations remain local CLI/API workflows.
73
+
74
+ ### Tiếng Việt
75
+
76
+ - Bổ sung package plugin dạng skills-only và metadata directory cho ChatGPT, Codex, Claude Code, Cowork và Kimi Code.
77
+ - Bổ sung demo release gate cho customer support không cần credential cùng bằng chứng đã ghi lại.
78
+ - Khôi phục GitHub Actions chỉ dùng để phát hành PyPI qua Trusted Publishing; tích hợp runtime vẫn chạy qua CLI/API cục bộ.
79
+
80
+ ### 日本語
81
+
82
+ - ChatGPT、Codex、Claude Code、Cowork、Kimi Code 向けの skills-only plugin package と directory metadata を追加しました。
83
+ - 認証情報不要の customer-support release-gate demo と記録済み証拠を追加しました。
84
+ - PyPI Trusted Publishing 専用の GitHub Actions workflow を復元しました。runtime integration は引き続きローカル CLI/API を使用します。
85
+
86
+ ## [1.0.0] - 2026-07-26
87
+
88
+ ### English
89
+
90
+ - Consolidated deterministic and statistical evaluation, policy comparison, provenance diagnosis, offline evidence and API/CLI adapters into one stable baseline.
91
+ - Removed all repository-owned GitHub Actions; release gates remain available through the local CLI and API.
92
+ - Standardized the public tag and package version at `v1.0.0`.
93
+
94
+ ### Tiếng Việt
95
+
96
+ - Hợp nhất đánh giá xác định/thống kê, so sánh policy, chẩn đoán provenance, bằng chứng offline và adapter API/CLI vào một baseline ổn định.
97
+ - Xóa toàn bộ GitHub Actions thuộc repo; release gate tiếp tục hoạt động qua CLI và API cục bộ.
98
+ - Chuẩn hóa tag public và package version thành `v1.0.0`.
99
+
100
+ ### 日本語
101
+
102
+ - 決定的・統計的評価、ポリシー比較、来歴診断、オフライン証拠、API/CLI アダプターを1つの安定版へ統合しました。
103
+ - リポジトリ所有の GitHub Actions をすべて削除し、リリースゲートはローカル CLI と API で引き続き利用できます。
104
+ - 公開タグとパッケージ版を `v1.0.0` に統一しました。
105
+
106
+ [2.0.2]: https://github.com/thangldw/ragops/releases/tag/v2.0.2
107
+ [2.0.1]: https://github.com/thangldw/ragops/releases/tag/v2.0.1
108
+ [2.0.0]: https://github.com/thangldw/ragops/releases/tag/v2.0.0
109
+ [1.2.0]: https://github.com/thangldw/ragops/releases/tag/v1.2.0
110
+ [1.0.0]: https://github.com/thangldw/ragops/releases/tag/v1.0.0
@@ -0,0 +1,5 @@
1
+ # Contributing / Đóng góp / コントリビューション
2
+
3
+ - **English:** Preserve offline core semantics, version public schemas, add evidence fixtures and run `ruff check .` plus `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q`.
4
+ - **Tiếng Việt:** Giữ semantics offline của core, version public schema, thêm evidence fixture và chạy lint/test đầy đủ.
5
+ - **日本語:** オフラインコアの意味を維持し、公開 schema を version 管理し、証拠 fixture を追加して lint/test を実行してください。
ragops-2.0.2/PKG-INFO ADDED
@@ -0,0 +1,99 @@
1
+ Metadata-Version: 2.5
2
+ Name: ragops
3
+ Version: 2.0.2
4
+ Summary: Regression tests and explainable release gates for RAG and AI agents
5
+ Project-URL: Homepage, https://github.com/thangldw/ragops
6
+ Project-URL: Documentation, https://github.com/thangldw/ragops#readme
7
+ Project-URL: Issues, https://github.com/thangldw/ragops/issues
8
+ Project-URL: Changelog, https://github.com/thangldw/ragops/blob/main/CHANGELOG.md
9
+ Author: Thang Luu
10
+ License: MIT
11
+ License-File: LICENSE
12
+ Keywords: ai-agents,evaluation,rag,regression-testing,release-gates
13
+ Classifier: Development Status :: 4 - Beta
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: License :: OSI Approved :: MIT License
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Software Development :: Quality Assurance
22
+ Requires-Python: >=3.11
23
+ Provides-Extra: api
24
+ Requires-Dist: fastapi>=0.115; extra == 'api'
25
+ Requires-Dist: uvicorn>=0.30; extra == 'api'
26
+ Provides-Extra: dev
27
+ Requires-Dist: fastapi>=0.115; extra == 'dev'
28
+ Requires-Dist: jsonschema>=4.23; extra == 'dev'
29
+ Requires-Dist: pytest>=8.0; extra == 'dev'
30
+ Requires-Dist: ruff>=0.6; extra == 'dev'
31
+ Provides-Extra: release
32
+ Requires-Dist: build>=1.2; extra == 'release'
33
+ Requires-Dist: cyclonedx-bom>=7.3; extra == 'release'
34
+ Requires-Dist: twine>=5.1; extra == 'release'
35
+ Description-Content-Type: text/markdown
36
+
37
+ # RAGOps
38
+
39
+ [English](#english) · [Tiếng Việt](#tiếng-việt) · [日本語](#日本語)
40
+
41
+ Offline, reproducible release gates for RAG systems and AI agents.
42
+
43
+ ```mermaid
44
+ %%{init: {"theme":"base","themeVariables":{"background":"#FFFFFF","fontFamily":"Arial, sans-serif","lineColor":"#667085","primaryTextColor":"#172B4D"}}}%%
45
+ flowchart LR
46
+ S["Scenario<br/>Kịch bản / シナリオ"]:::yellow
47
+ B["Accepted baseline<br/>Mốc chuẩn / 基準"]:::blue
48
+ C["Candidate traces<br/>Ứng viên / 候補"]:::purple
49
+ E["Evaluators & policy<br/>Đánh giá / 評価"]:::green
50
+ G{"PASS / WARN / BLOCK"}:::pink
51
+ S --> E
52
+ B --> E
53
+ C --> E --> G
54
+ classDef yellow fill:#FFF4A3,stroke:#C9A227,stroke-width:2px,color:#172B4D
55
+ classDef blue fill:#D9EAFD,stroke:#4C78A8,stroke-width:2px,color:#172B4D
56
+ classDef purple fill:#E9DDF7,stroke:#8064A2,stroke-width:2px,color:#172B4D
57
+ classDef green fill:#DDF5E3,stroke:#4F9D69,stroke-width:2px,color:#172B4D
58
+ classDef pink fill:#FFE1E6,stroke:#C96A7B,stroke-width:2px,color:#172B4D
59
+ ```
60
+
61
+ ## English
62
+
63
+ RAGOps compares recorded candidate behavior with an accepted baseline, applies a versioned release policy, and emits one canonical `PASS`, `WARN`, or `BLOCK` decision across JSON, Markdown, HTML, JUnit, SARIF, and GitHub Summary. The dependency-free core performs no network calls. Optional adapters import vendor exports; the API/workbench ships in the wheel through the `api` extra.
64
+
65
+ Requirements: Python 3.11+.
66
+
67
+ Current technical references: [architecture](docs/ARCHITECTURE.md), [contracts](docs/CONTRACTS.md), [governance](docs/governance.md), [operations](docs/OPERATIONS.md), [security](SECURITY.md), and [v2.0.2 release notes](docs/releases/v2.0.2.md).
68
+
69
+ The repository is also a skills-only plugin for ChatGPT, Codex, Claude Code and Cowork. It does not include a hosted MCP connector. See the [directory submission package](docs/submission/DIRECTORY_SUBMISSION.md), [privacy policy](PRIVACY.md), [terms](TERMS.md), and [support guidance](SUPPORT.md).
70
+
71
+ ```bash
72
+ pip install ragops
73
+ ragops demo --profile engineer --output ./demo-output
74
+ ragops evidence verify --bundle ./demo-output/evidence
75
+ open ./demo-output/release-report.html
76
+ ```
77
+
78
+ On Windows PowerShell, replace the final command with
79
+ `Start-Process ./demo-output/release-report.html`.
80
+
81
+ The credential-free demo intentionally returns `BLOCK`: citation coverage `1.0 -> 0.5`, citation precision `1.0 -> 0.5`, and lexical groundedness `1.0 -> 0.6`. It is synthetic benchmark evidence, not production adoption evidence.
82
+
83
+ Install `ragops[api]` and run `ragops serve` for the local authenticated API/workbench. Use `ragops adapter list` for Phoenix, Ragas, DeepEval, LangSmith, MLflow, Promptfoo, custom JSON, and installed entry-point adapters.
84
+
85
+ Local CLI usage evidence is disabled by default. Set `RAGOPS_USAGE_LOG` to an explicit JSONL path to record only the UTC timestamp, RAGOps version, top-level command, and exit code; arguments and file paths are never recorded. Summarize the file with `ragops usage-report --events usage.jsonl`. Use the consent-aware `pilot-report` contract and the [production pilot runbook](docs/gtm/production-pilot.md) for production adoption evidence.
86
+
87
+ ## Tiếng Việt
88
+
89
+ RAGOps so sánh trace ứng viên với baseline được chấp nhận, áp dụng release policy có version và xuất cùng một quyết định `PASS`, `WARN` hoặc `BLOCK` sang JSON/Markdown/HTML/JUnit/SARIF. Core không dependency, chạy offline; API/workbench là extra tùy chọn. Demo synthetic cố ý tạo `BLOCK`, không phải bằng chứng production adoption.
90
+
91
+ Yêu cầu: Python 3.11+. Dùng các lệnh ở phần English để cài đặt, kiểm tra scenario, lint và test.
92
+
93
+ ## 日本語
94
+
95
+ RAGOps は記録済み候補を承認済みベースラインと比較し、バージョン管理された方針から一つの `PASS`・`WARN`・`BLOCK` 判定を JSON、Markdown、HTML、JUnit、SARIF に出力します。依存関係のない core はオフラインで動作し、API/workbench は任意の extra です。デモは synthetic evidence であり、本番導入実績ではありません。
96
+
97
+ 必要環境は Python 3.11 以上です。セットアップ、lint、テストには English セクションのコマンドを使用してください。
98
+
99
+ Released under the [MIT License](LICENSE).
@@ -0,0 +1,13 @@
1
+ # Privacy Policy
2
+
3
+ Last updated: 2026-08-02
4
+
5
+ RAGOps and its bundled plugin run locally by default. The project does not operate a hosted RAGOps service, create user accounts, collect telemetry, or transmit repository files, scenarios, traces, reports, or credentials to the project author.
6
+
7
+ RAGOps reads only the files and command arguments selected by the user. It may write reports, replay bundles, baselines, or local experiment records only when the user requests an operation that creates those artifacts. Those files remain in user-controlled storage.
8
+
9
+ Users may explicitly set `RAGOPS_USAGE_LOG` to write local usage events containing only a UTC timestamp, RAGOps version, top-level command name, and exit code. This is disabled by default, excludes command arguments and file paths, remains in user-controlled storage, and is never transmitted to the project author.
10
+
11
+ Optional provider or HTTP adapters can send data to an endpoint explicitly configured by the user. In that case, the endpoint operator's privacy terms apply. RAGOps does not enable a provider automatically and does not embed provider credentials.
12
+
13
+ Users control retention by deleting locally generated artifacts. For privacy questions, open an issue at https://github.com/thangldw/ragops/issues.