@kevinrabun/judges 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +174 -0
- package/dist/evaluators/accessibility.d.ts +3 -0
- package/dist/evaluators/accessibility.d.ts.map +1 -0
- package/dist/evaluators/accessibility.js +306 -0
- package/dist/evaluators/accessibility.js.map +1 -0
- package/dist/evaluators/api-design.d.ts +3 -0
- package/dist/evaluators/api-design.d.ts.map +1 -0
- package/dist/evaluators/api-design.js +224 -0
- package/dist/evaluators/api-design.js.map +1 -0
- package/dist/evaluators/cloud-readiness.d.ts +3 -0
- package/dist/evaluators/cloud-readiness.d.ts.map +1 -0
- package/dist/evaluators/cloud-readiness.js +181 -0
- package/dist/evaluators/cloud-readiness.js.map +1 -0
- package/dist/evaluators/compliance.d.ts +3 -0
- package/dist/evaluators/compliance.d.ts.map +1 -0
- package/dist/evaluators/compliance.js +213 -0
- package/dist/evaluators/compliance.js.map +1 -0
- package/dist/evaluators/concurrency.d.ts +3 -0
- package/dist/evaluators/concurrency.d.ts.map +1 -0
- package/dist/evaluators/concurrency.js +220 -0
- package/dist/evaluators/concurrency.js.map +1 -0
- package/dist/evaluators/cost-effectiveness.d.ts +3 -0
- package/dist/evaluators/cost-effectiveness.d.ts.map +1 -0
- package/dist/evaluators/cost-effectiveness.js +206 -0
- package/dist/evaluators/cost-effectiveness.js.map +1 -0
- package/dist/evaluators/cybersecurity.d.ts +3 -0
- package/dist/evaluators/cybersecurity.d.ts.map +1 -0
- package/dist/evaluators/cybersecurity.js +282 -0
- package/dist/evaluators/cybersecurity.js.map +1 -0
- package/dist/evaluators/data-security.d.ts +3 -0
- package/dist/evaluators/data-security.d.ts.map +1 -0
- package/dist/evaluators/data-security.js +286 -0
- package/dist/evaluators/data-security.js.map +1 -0
- package/dist/evaluators/dependency-health.d.ts +3 -0
- package/dist/evaluators/dependency-health.d.ts.map +1 -0
- package/dist/evaluators/dependency-health.js +197 -0
- package/dist/evaluators/dependency-health.js.map +1 -0
- package/dist/evaluators/documentation.d.ts +3 -0
- package/dist/evaluators/documentation.d.ts.map +1 -0
- package/dist/evaluators/documentation.js +216 -0
- package/dist/evaluators/documentation.js.map +1 -0
- package/dist/evaluators/ethics-bias.d.ts +3 -0
- package/dist/evaluators/ethics-bias.d.ts.map +1 -0
- package/dist/evaluators/ethics-bias.js +205 -0
- package/dist/evaluators/ethics-bias.js.map +1 -0
- package/dist/evaluators/index.d.ts +12 -0
- package/dist/evaluators/index.d.ts.map +1 -0
- package/dist/evaluators/index.js +127 -0
- package/dist/evaluators/index.js.map +1 -0
- package/dist/evaluators/internationalization.d.ts +3 -0
- package/dist/evaluators/internationalization.d.ts.map +1 -0
- package/dist/evaluators/internationalization.js +176 -0
- package/dist/evaluators/internationalization.js.map +1 -0
- package/dist/evaluators/observability.d.ts +3 -0
- package/dist/evaluators/observability.d.ts.map +1 -0
- package/dist/evaluators/observability.js +171 -0
- package/dist/evaluators/observability.js.map +1 -0
- package/dist/evaluators/performance.d.ts +3 -0
- package/dist/evaluators/performance.d.ts.map +1 -0
- package/dist/evaluators/performance.js +306 -0
- package/dist/evaluators/performance.js.map +1 -0
- package/dist/evaluators/reliability.d.ts +3 -0
- package/dist/evaluators/reliability.d.ts.map +1 -0
- package/dist/evaluators/reliability.js +215 -0
- package/dist/evaluators/reliability.js.map +1 -0
- package/dist/evaluators/scalability.d.ts +3 -0
- package/dist/evaluators/scalability.d.ts.map +1 -0
- package/dist/evaluators/scalability.js +171 -0
- package/dist/evaluators/scalability.js.map +1 -0
- package/dist/evaluators/shared.d.ts +18 -0
- package/dist/evaluators/shared.d.ts.map +1 -0
- package/dist/evaluators/shared.js +147 -0
- package/dist/evaluators/shared.js.map +1 -0
- package/dist/evaluators/software-practices.d.ts +3 -0
- package/dist/evaluators/software-practices.d.ts.map +1 -0
- package/dist/evaluators/software-practices.js +272 -0
- package/dist/evaluators/software-practices.js.map +1 -0
- package/dist/evaluators/testing.d.ts +3 -0
- package/dist/evaluators/testing.d.ts.map +1 -0
- package/dist/evaluators/testing.js +185 -0
- package/dist/evaluators/testing.js.map +1 -0
- package/dist/index.d.ts +16 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +238 -0
- package/dist/index.js.map +1 -0
- package/dist/judges/accessibility.d.ts +3 -0
- package/dist/judges/accessibility.d.ts.map +1 -0
- package/dist/judges/accessibility.js +28 -0
- package/dist/judges/accessibility.js.map +1 -0
- package/dist/judges/api-design.d.ts +3 -0
- package/dist/judges/api-design.d.ts.map +1 -0
- package/dist/judges/api-design.js +30 -0
- package/dist/judges/api-design.js.map +1 -0
- package/dist/judges/cloud-readiness.d.ts +3 -0
- package/dist/judges/cloud-readiness.d.ts.map +1 -0
- package/dist/judges/cloud-readiness.js +28 -0
- package/dist/judges/cloud-readiness.js.map +1 -0
- package/dist/judges/compliance.d.ts +3 -0
- package/dist/judges/compliance.d.ts.map +1 -0
- package/dist/judges/compliance.js +28 -0
- package/dist/judges/compliance.js.map +1 -0
- package/dist/judges/concurrency.d.ts +3 -0
- package/dist/judges/concurrency.d.ts.map +1 -0
- package/dist/judges/concurrency.js +30 -0
- package/dist/judges/concurrency.js.map +1 -0
- package/dist/judges/cost-effectiveness.d.ts +3 -0
- package/dist/judges/cost-effectiveness.d.ts.map +1 -0
- package/dist/judges/cost-effectiveness.js +27 -0
- package/dist/judges/cost-effectiveness.js.map +1 -0
- package/dist/judges/cybersecurity.d.ts +3 -0
- package/dist/judges/cybersecurity.d.ts.map +1 -0
- package/dist/judges/cybersecurity.js +27 -0
- package/dist/judges/cybersecurity.js.map +1 -0
- package/dist/judges/data-security.d.ts +3 -0
- package/dist/judges/data-security.d.ts.map +1 -0
- package/dist/judges/data-security.js +25 -0
- package/dist/judges/data-security.js.map +1 -0
- package/dist/judges/dependency-health.d.ts +3 -0
- package/dist/judges/dependency-health.d.ts.map +1 -0
- package/dist/judges/dependency-health.js +30 -0
- package/dist/judges/dependency-health.js.map +1 -0
- package/dist/judges/documentation.d.ts +3 -0
- package/dist/judges/documentation.d.ts.map +1 -0
- package/dist/judges/documentation.js +30 -0
- package/dist/judges/documentation.js.map +1 -0
- package/dist/judges/ethics-bias.d.ts +3 -0
- package/dist/judges/ethics-bias.d.ts.map +1 -0
- package/dist/judges/ethics-bias.js +30 -0
- package/dist/judges/ethics-bias.js.map +1 -0
- package/dist/judges/index.d.ts +23 -0
- package/dist/judges/index.d.ts.map +1 -0
- package/dist/judges/index.js +63 -0
- package/dist/judges/index.js.map +1 -0
- package/dist/judges/internationalization.d.ts +3 -0
- package/dist/judges/internationalization.d.ts.map +1 -0
- package/dist/judges/internationalization.js +28 -0
- package/dist/judges/internationalization.js.map +1 -0
- package/dist/judges/observability.d.ts +3 -0
- package/dist/judges/observability.d.ts.map +1 -0
- package/dist/judges/observability.js +28 -0
- package/dist/judges/observability.js.map +1 -0
- package/dist/judges/performance.d.ts +3 -0
- package/dist/judges/performance.d.ts.map +1 -0
- package/dist/judges/performance.js +30 -0
- package/dist/judges/performance.js.map +1 -0
- package/dist/judges/reliability.d.ts +3 -0
- package/dist/judges/reliability.d.ts.map +1 -0
- package/dist/judges/reliability.js +30 -0
- package/dist/judges/reliability.js.map +1 -0
- package/dist/judges/scalability.d.ts +3 -0
- package/dist/judges/scalability.d.ts.map +1 -0
- package/dist/judges/scalability.js +28 -0
- package/dist/judges/scalability.js.map +1 -0
- package/dist/judges/software-practices.d.ts +3 -0
- package/dist/judges/software-practices.d.ts.map +1 -0
- package/dist/judges/software-practices.js +30 -0
- package/dist/judges/software-practices.js.map +1 -0
- package/dist/judges/testing.d.ts +3 -0
- package/dist/judges/testing.d.ts.map +1 -0
- package/dist/judges/testing.js +30 -0
- package/dist/judges/testing.js.map +1 -0
- package/dist/types.d.ts +80 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +2 -0
- package/dist/types.js.map +1 -0
- package/package.json +53 -0
- package/server.json +21 -0
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"software-practices.js","sourceRoot":"","sources":["../../src/evaluators/software-practices.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAE7C,MAAM,UAAU,wBAAwB,CAAC,IAAY,EAAE,QAAgB;IACrE,MAAM,QAAQ,GAAc,EAAE,CAAC;IAC/B,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,MAAM,MAAM,GAAG,OAAO,CAAC;IAEvB,8BAA8B;IAC9B,MAAM,cAAc,GAAG,8BAA8B,CAAC;IACtD,MAAM,QAAQ,GAAG,cAAc,CAAC,IAAI,EAAE,cAAc,CAAC,CAAC;IACtD,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACxB,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,6BAA6B;YACpC,WAAW,EAAE,4HAA4H;YACzI,WAAW,EAAE,QAAQ;YACrB,cAAc,EAAE,gJAAgJ;YAChK,SAAS,EAAE,wCAAwC;SACpD,CAAC,CAAC;IACL,CAAC;IAED,mBAAmB;IACnB,MAAM,eAAe,GAAG,0BAA0B,CAAC;IACnD,MAAM,aAAa,GAAG,cAAc,CAAC,IAAI,EAAE,eAAe,CAAC,CAAC;IAC5D,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,8BAA8B;YACrC,WAAW,EAAE,4GAA4G;YACzH,WAAW,EAAE,aAAa;YAC1B,cAAc,EAAE,iJAAiJ;YACjK,SAAS,EAAE,uCAAuC;SACnD,CAAC,CAAC;IACL,CAAC;IAED,gBAAgB;IAChB,MAAM,kBAAkB,GAAG,0HAA0H,CAAC;IACtJ,MAAM,UAAU,GAAG,cAAc,CAAC,IAAI,EAAE,kBAAkB,CAAC,CAAC;IAC5D,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC1B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,KAAK;YACf,KAAK,EAAE,wBAAwB;YAC/B,WAAW,EAAE,6HAA6H;YAC1I,WAAW,EAAE,UAAU;YACvB,cAAc,EAAE,oIAAoI;YACpJ,SAAS,EAAE,4CAA4C;SACxD,CAAC,CAAC;IACL,CAAC;IAED,kCAAkC;IAClC,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IACnC,IAAI,SAAS,GAAG,CAAC,CAAC,CAAC;IACnB,IAAI,UAAU,GAAG,CAAC,CAAC;IACnB,MAAM,aAAa,GAAa,EAAE,CAAC;IACnC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAC1C,IAAI,oGAAoG,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;YAC5H,IAAI,UAAU,KAAK,CAAC;gBAAE,SAAS,GAAG,CAAC,CAAC;QACtC,CAAC;QACD,UAAU,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QACvD,UAAU,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QACvD,IAAI,UAAU,KAAK,CAAC,IAAI,SAAS,IAAI,CAAC,EAAE,CAAC;YACvC,IAAI,CAAC,GAAG,SAAS,GAAG,EAAE,EAAE,CAAC;gBACvB,aAAa,CAAC,IAAI,CAAC,SAAS,GAAG,CAAC,CAAC,CAAC;YACpC,CAAC;YACD,SAAS,GAAG,CAAC,CAAC,CAAC;QACjB,CAAC;IACH,CAAC;IACD,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,KAAK;YACf,KAAK,EAAE,oCAAoC;YAC3C,WAAW,EAAE,4KAA4K;YACzL,WAAW,EAAE,aAAa;YAC1B,cAAc,EAAE,iHAAiH;YACjI,SAAS,EAAE,8CAA8C;SAC1D,CAAC,CAAC;IACL,CAAC;IAED,2BAA2B;IAC3B,MAAM,WAAW,GAAG,oDAAoD,CAAC;IACzE,MAAM,SAAS,GAAG,cAAc,CAAC,IAAI,EAAE,WAAW,CAAC,CAAC;IACpD,IAAI,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACzB,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,MAAM;YAChB,KAAK,EAAE,gCAAgC;YACvC,WAAW,EAAE,0IAA0I;YACvJ,WAAW,EAAE,SAAS;YACtB,cAAc,EAAE,wJAAwJ;YACxK,SAAS,EAAE,qCAAqC;SACjD,CAAC,CAAC;IACL,CAAC;IAED,qBAAqB;IACrB,MAAM,iBAAiB,GAAG,+BAA+B,CAAC;IAC1D,MAAM,eAAe,GAAG,cAAc,CAAC,IAAI,EAAE,iBAAiB,CAAC,CAAC;IAChE,IAAI,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC/B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,MAAM;YAChB,KAAK,EAAE,+CAA+C;YACtD,WAAW,EAAE,4HAA4H;YACzI,WAAW,EAAE,eAAe;YAC5B,cAAc,EAAE,8IAA8I;YAC9J,SAAS,EAAE,uCAAuC;SACnD,CAAC,CAAC;IACL,CAAC;IAED,sBAAsB;IACtB,MAAM,aAAa,GAAG,4FAA4F,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC9H,MAAM,YAAY,GAAG,6EAA6E,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC9G,IAAI,YAAY,IAAI,CAAC,aAAa,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,MAAM;YAChB,KAAK,EAAE,8BAA8B;YACrC,WAAW,EAAE,6KAA6K;YAC1L,cAAc,EAAE,mKAAmK;YACnL,SAAS,EAAE,2CAA2C;SACvD,CAAC,CAAC;IACL,CAAC;IAED,yCAAyC;IACzC,MAAM,eAAe,GAAG,sEAAsE,CAAC;IAC/F,MAAM,UAAU,GAAG,cAAc,CAAC,IAAI,EAAE,eAAe,CAAC,CAAC;IACzD,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC1B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,KAAK;YACf,KAAK,EAAE,2CAA2C;YAClD,WAAW,EAAE,mIAAmI;YAChJ,WAAW,EAAE,UAAU;YACvB,cAAc,EAAE,mHAAmH;YACnI,SAAS,EAAE,4BAA4B;SACxC,CAAC,CAAC;IACL,CAAC;IAED,2BAA2B;IAC3B,MAAM,aAAa,GAAa,EAAE,CAAC;IACnC,SAAS,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;QAC5B,MAAM,aAAa,GAAG,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACxC,IAAI,aAAa,IAAI,EAAE,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YAC3E,aAAa,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QAC5B,CAAC;IACH,CAAC,CAAC,CAAC;IACH,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,gCAAgC;YACvC,WAAW,EAAE,kIAAkI;YAC/I,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;YACtC,cAAc,EAAE,4HAA4H;YAC5I,SAAS,EAAE,iDAAiD;SAC7D,CAAC,CAAC;IACL,CAAC;IAED,qCAAqC;IACrC,MAAM,UAAU,GAAG,gBAAgB,CAAC;IACpC,MAAM,QAAQ,GAAG,cAAc,CAAC,IAAI,EAAE,UAAU,CAAC,CAAC;IAClD,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,KAAK,YAAY,IAAI,QAAQ,KAAK,YAAY,IAAI,QAAQ,KAAK,KAAK,IAAI,QAAQ,KAAK,KAAK,CAAC,EAAE,CAAC;QAChI,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,yCAAyC;YAChD,WAAW,EAAE,qJAAqJ;YAClK,WAAW,EAAE,QAAQ;YACrB,cAAc,EAAE,+EAA+E;YAC/F,SAAS,EAAE,qBAAqB;SACjC,CAAC,CAAC;IACL,CAAC;IAED,qCAAqC;IACrC,MAAM,qBAAqB,GAAG,wDAAwD,CAAC;IACvF,MAAM,mBAAmB,GAAG,cAAc,CAAC,IAAI,EAAE,qBAAqB,CAAC,CAAC;IACxE,IAAI,mBAAmB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,MAAM;YAChB,KAAK,EAAE,gDAAgD;YACvD,WAAW,EAAE,4HAA4H;YACzI,WAAW,EAAE,mBAAmB;YAChC,cAAc,EAAE,iHAAiH;YACjI,SAAS,EAAE,mDAAmD;SAC/D,CAAC,CAAC;IACL,CAAC;IAED,0CAA0C;IAC1C,MAAM,iBAAiB,GAAG,yCAAyC,CAAC;IACpE,MAAM,eAAe,GAAG,cAAc,CAAC,IAAI,EAAE,iBAAiB,CAAC,CAAC;IAChE,IAAI,eAAe,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC/B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,mCAAmC;YAC1C,WAAW,EAAE,sIAAsI;YACnJ,WAAW,EAAE,eAAe;YAC5B,cAAc,EAAE,+IAA+I;YAC/J,SAAS,EAAE,mCAAmC;SAC/C,CAAC,CAAC;IACL,CAAC;IAED,sCAAsC;IACtC,MAAM,eAAe,GAAG,cAAc,CAAC,IAAI,EAAE,gBAAgB,CAAC,CAAC;IAC/D,IAAI,eAAe,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,QAAQ,KAAK,YAAY,IAAI,QAAQ,KAAK,YAAY,IAAI,QAAQ,KAAK,KAAK,IAAI,QAAQ,KAAK,KAAK,CAAC,EAAE,CAAC;QACvI,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,6CAA6C;YACpD,WAAW,EAAE,qJAAqJ;YAClK,WAAW,EAAE,eAAe,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;YACxC,cAAc,EAAE,mEAAmE;YACnF,SAAS,EAAE,gCAAgC;SAC5C,CAAC,CAAC;IACL,CAAC;IAED,kDAAkD;IAClD,MAAM,UAAU,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,gBAAgB,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAC/D,MAAM,aAAa,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,6BAA6B,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAC/E,IAAI,SAAS,CAAC,MAAM,GAAG,GAAG,IAAI,aAAa,GAAG,EAAE,EAAE,CAAC;QACjD,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,4DAA4D;YACnE,WAAW,EAAE,YAAY,SAAS,CAAC,MAAM,cAAc,aAAa,qEAAqE;YACzI,cAAc,EAAE,2HAA2H;YAC3I,SAAS,EAAE,0CAA0C;SACtD,CAAC,CAAC;IACL,CAAC;IAED,0CAA0C;IAC1C,MAAM,iBAAiB,GAAa,EAAE,CAAC;IACvC,SAAS,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;QAC5B,IAAI,yBAAyB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YACzC,MAAM,OAAO,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACjF,MAAM,eAAe,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,wBAAwB,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;YAC/E,IAAI,eAAe,IAAI,CAAC,EAAE,CAAC;gBACzB,iBAAiB,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAChC,CAAC;QACH,CAAC;IACH,CAAC,CAAC,CAAC;IACH,IAAI,iBAAiB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACjC,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,QAAQ;YAClB,KAAK,EAAE,4CAA4C;YACnD,WAAW,EAAE,gGAAgG;YAC7G,WAAW,EAAE,iBAAiB,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;YAC1C,cAAc,EAAE,2FAA2F;YAC3G,SAAS,EAAE,gCAAgC;SAC5C,CAAC,CAAC;IACL,CAAC;IAED,uBAAuB;IACvB,MAAM,eAAe,GAAG,6DAA6D,CAAC;IACtF,MAAM,aAAa,GAAG,cAAc,CAAC,IAAI,EAAE,eAAe,CAAC,CAAC;IAC5D,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,KAAK;YACf,KAAK,EAAE,4CAA4C;YACnD,WAAW,EAAE,+DAA+D;YAC5E,WAAW,EAAE,aAAa;YAC1B,cAAc,EAAE,+FAA+F;YAC/G,SAAS,EAAE,sCAAsC;SAClD,CAAC,CAAC;IACL,CAAC;IAED,+BAA+B;IAC/B,MAAM,gBAAgB,GAAG,uEAAuE,CAAC;IACjG,MAAM,cAAc,GAAG,cAAc,CAAC,IAAI,EAAE,gBAAgB,CAAC,CAAC;IAC9D,IAAI,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,QAAQ,CAAC,IAAI,CAAC;YACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;YACzD,QAAQ,EAAE,KAAK;YACf,KAAK,EAAE,8CAA8C;YACrD,WAAW,EAAE,mHAAmH;YAChI,WAAW,EAAE,cAAc;YAC3B,cAAc,EAAE,sHAAsH;YACtI,SAAS,EAAE,iCAAiC;SAC7C,CAAC,CAAC;IACL,CAAC;IAED,OAAO,QAAQ,CAAC;AAClB,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"testing.d.ts","sourceRoot":"","sources":["../../src/evaluators/testing.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,aAAa,CAAC;AAGtC,wBAAgB,cAAc,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,OAAO,EAAE,CAkMxE"}
|
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
export function analyzeTesting(code, language) {
|
|
2
|
+
const findings = [];
|
|
3
|
+
const lines = code.split("\n");
|
|
4
|
+
const prefix = "TEST";
|
|
5
|
+
let ruleNum = 1;
|
|
6
|
+
// Detect test files with no assertions
|
|
7
|
+
const hasTestStructure = /describe\s*\(|it\s*\(|test\s*\(|def\s+test_|@Test/i.test(code);
|
|
8
|
+
if (hasTestStructure) {
|
|
9
|
+
// Check for assertions
|
|
10
|
+
const assertionLines = [];
|
|
11
|
+
lines.forEach((line, i) => {
|
|
12
|
+
if (/expect\s*\(|assert|should\.|\.to\.|\.toBe|\.toEqual|\.toThrow|assertEqual|assertTrue|verify/i.test(line)) {
|
|
13
|
+
assertionLines.push(i + 1);
|
|
14
|
+
}
|
|
15
|
+
});
|
|
16
|
+
const testBlockLines = [];
|
|
17
|
+
lines.forEach((line, i) => {
|
|
18
|
+
if (/\b(?:it|test)\s*\(\s*["'`]/i.test(line)) {
|
|
19
|
+
testBlockLines.push(i + 1);
|
|
20
|
+
}
|
|
21
|
+
});
|
|
22
|
+
if (testBlockLines.length > 0 && assertionLines.length === 0) {
|
|
23
|
+
findings.push({
|
|
24
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
25
|
+
severity: "critical",
|
|
26
|
+
title: "Test cases with no assertions",
|
|
27
|
+
description: "Tests without assertions always pass and provide no verification. They give false confidence in code correctness.",
|
|
28
|
+
lineNumbers: testBlockLines,
|
|
29
|
+
recommendation: "Add meaningful assertions to every test case. Each test should verify at least one expected behavior.",
|
|
30
|
+
reference: "Unit Testing Best Practices",
|
|
31
|
+
});
|
|
32
|
+
}
|
|
33
|
+
// Detect overly broad test names
|
|
34
|
+
const vagueTestLines = [];
|
|
35
|
+
lines.forEach((line, i) => {
|
|
36
|
+
if (/(?:it|test)\s*\(\s*["'`](test\s+\w+|works|it works|should work|test \d+|basic test)["'`]/i.test(line)) {
|
|
37
|
+
vagueTestLines.push(i + 1);
|
|
38
|
+
}
|
|
39
|
+
});
|
|
40
|
+
if (vagueTestLines.length > 0) {
|
|
41
|
+
findings.push({
|
|
42
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
43
|
+
severity: "low",
|
|
44
|
+
title: "Vague test names",
|
|
45
|
+
description: "Test names like 'works' or 'test 1' don't describe what behavior is being verified, making test failures harder to diagnose.",
|
|
46
|
+
lineNumbers: vagueTestLines,
|
|
47
|
+
recommendation: "Use descriptive test names that explain the scenario and expected outcome: 'should return 404 when user not found'.",
|
|
48
|
+
reference: "Test Naming Conventions",
|
|
49
|
+
});
|
|
50
|
+
}
|
|
51
|
+
// Detect hardcoded test data that might be brittle
|
|
52
|
+
const hardcodedDateLines = [];
|
|
53
|
+
lines.forEach((line, i) => {
|
|
54
|
+
if (/["'`]20[2-3]\d-[01]\d-[0-3]\d/i.test(line) && !/mock|stub|fixture/i.test(line)) {
|
|
55
|
+
hardcodedDateLines.push(i + 1);
|
|
56
|
+
}
|
|
57
|
+
});
|
|
58
|
+
if (hardcodedDateLines.length > 0) {
|
|
59
|
+
findings.push({
|
|
60
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
61
|
+
severity: "low",
|
|
62
|
+
title: "Hardcoded dates in tests",
|
|
63
|
+
description: "Hardcoded dates in tests can become stale and cause intermittent failures as time passes.",
|
|
64
|
+
lineNumbers: hardcodedDateLines,
|
|
65
|
+
recommendation: "Use relative dates, time-freezing libraries (sinon.useFakeTimers, freezegun), or inject clock dependencies.",
|
|
66
|
+
reference: "Testing Best Practices: Time-Dependent Tests",
|
|
67
|
+
});
|
|
68
|
+
}
|
|
69
|
+
// Detect tests with external dependencies
|
|
70
|
+
const externalDepLines = [];
|
|
71
|
+
lines.forEach((line, i) => {
|
|
72
|
+
if (/fetch\s*\(|axios\.|https?:\/\/|database|redis|mongodb/i.test(line) && !/mock|stub|fake|spy|nock|msw/i.test(line)) {
|
|
73
|
+
externalDepLines.push(i + 1);
|
|
74
|
+
}
|
|
75
|
+
});
|
|
76
|
+
if (externalDepLines.length > 0) {
|
|
77
|
+
findings.push({
|
|
78
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
79
|
+
severity: "medium",
|
|
80
|
+
title: "Tests with real external dependencies",
|
|
81
|
+
description: "Tests that call real external services or databases are slow, flaky, and may fail due to network issues or service unavailability.",
|
|
82
|
+
lineNumbers: externalDepLines,
|
|
83
|
+
recommendation: "Mock external dependencies using test doubles (jest.mock, sinon, nock, msw). Use in-memory databases for integration tests.",
|
|
84
|
+
reference: "Test Doubles: Mocks, Stubs, and Fakes",
|
|
85
|
+
});
|
|
86
|
+
}
|
|
87
|
+
// Detect tests with shared mutable state
|
|
88
|
+
const sharedStateLines = [];
|
|
89
|
+
lines.forEach((line, i) => {
|
|
90
|
+
if (/(?:let|var)\s+\w+\s*=/i.test(line.trim()) && !/(?:const|it\s*\(|test\s*\(|describe\s*\()/i.test(line.trim())) {
|
|
91
|
+
const context = lines.slice(Math.max(0, i - 5), i).join("\n");
|
|
92
|
+
if (/describe\s*\(/i.test(context) && !/beforeEach|beforeAll|setUp/i.test(context)) {
|
|
93
|
+
sharedStateLines.push(i + 1);
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
});
|
|
97
|
+
if (sharedStateLines.length > 0) {
|
|
98
|
+
findings.push({
|
|
99
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
100
|
+
severity: "medium",
|
|
101
|
+
title: "Shared mutable state between tests",
|
|
102
|
+
description: "Mutable variables declared in describe blocks but not reset in beforeEach can cause test order dependencies and flaky results.",
|
|
103
|
+
lineNumbers: sharedStateLines,
|
|
104
|
+
recommendation: "Initialize mutable test state in beforeEach/setUp hooks, or use const for immutable test data.",
|
|
105
|
+
reference: "Test Isolation Best Practices",
|
|
106
|
+
});
|
|
107
|
+
}
|
|
108
|
+
// Detect tests without error case coverage
|
|
109
|
+
const happyPathOnly = /test|it\b/i.test(code) && !/error|throw|reject|fail|invalid|unauthorized|not found|exception/i.test(code);
|
|
110
|
+
if (happyPathOnly && testBlockLines.length > 0) {
|
|
111
|
+
findings.push({
|
|
112
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
113
|
+
severity: "medium",
|
|
114
|
+
title: "Tests cover only happy path",
|
|
115
|
+
description: "No error, exception, or edge case tests detected. Tests should cover both success and failure scenarios.",
|
|
116
|
+
recommendation: "Add tests for error cases, boundary conditions, invalid inputs, and edge cases. Test both what it does and what it prevents.",
|
|
117
|
+
reference: "Test Coverage: Error Paths",
|
|
118
|
+
});
|
|
119
|
+
}
|
|
120
|
+
// Detect sleep/wait in tests
|
|
121
|
+
const sleepLines = [];
|
|
122
|
+
lines.forEach((line, i) => {
|
|
123
|
+
if (/(?:sleep|setTimeout|Thread\.sleep|time\.sleep|delay)\s*\(\s*\d/i.test(line)) {
|
|
124
|
+
sleepLines.push(i + 1);
|
|
125
|
+
}
|
|
126
|
+
});
|
|
127
|
+
if (sleepLines.length > 0) {
|
|
128
|
+
findings.push({
|
|
129
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
130
|
+
severity: "medium",
|
|
131
|
+
title: "Arbitrary sleep/delay in tests",
|
|
132
|
+
description: "Using sleep/setTimeout in tests makes them slow and flaky — the timing may not be sufficient on slow CI machines.",
|
|
133
|
+
lineNumbers: sleepLines,
|
|
134
|
+
recommendation: "Use waitFor, polling, or event-based assertions instead of arbitrary delays. Use fake timers for timer-dependent logic.",
|
|
135
|
+
reference: "Testing Library waitFor / Flaky Test Prevention",
|
|
136
|
+
});
|
|
137
|
+
}
|
|
138
|
+
// Detect overly large test files
|
|
139
|
+
if (lines.length > 500) {
|
|
140
|
+
findings.push({
|
|
141
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
142
|
+
severity: "low",
|
|
143
|
+
title: "Test file is very large",
|
|
144
|
+
description: `Test file has ${lines.length} lines. Very large test files are hard to navigate and may indicate the test subject needs refactoring.`,
|
|
145
|
+
recommendation: "Split test files by feature or behavior. Consider if the production code under test should be broken into smaller modules.",
|
|
146
|
+
reference: "Test Organization Best Practices",
|
|
147
|
+
});
|
|
148
|
+
}
|
|
149
|
+
// Detect snapshot overuse
|
|
150
|
+
const snapshotLines = [];
|
|
151
|
+
lines.forEach((line, i) => {
|
|
152
|
+
if (/toMatchSnapshot|toMatchInlineSnapshot/i.test(line)) {
|
|
153
|
+
snapshotLines.push(i + 1);
|
|
154
|
+
}
|
|
155
|
+
});
|
|
156
|
+
if (snapshotLines.length > 5) {
|
|
157
|
+
findings.push({
|
|
158
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
159
|
+
severity: "low",
|
|
160
|
+
title: "Heavy reliance on snapshot testing",
|
|
161
|
+
description: "Many snapshot assertions detected. Snapshot tests are brittle, produce large diffs, and can be blindly updated without review.",
|
|
162
|
+
lineNumbers: snapshotLines.slice(0, 5),
|
|
163
|
+
recommendation: "Prefer explicit assertions for logic. Use snapshots sparingly for UI structure. Review snapshot updates carefully.",
|
|
164
|
+
reference: "Snapshot Testing Best Practices",
|
|
165
|
+
});
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
else {
|
|
169
|
+
// No test structure detected - check if this is production code without tests
|
|
170
|
+
const hasFunctions = /function\s+\w+|=>\s*\{|def\s+\w+|public\s+\w+\s+\w+\s*\(/i.test(code);
|
|
171
|
+
const isLargeFile = lines.length > 50;
|
|
172
|
+
if (hasFunctions && isLargeFile) {
|
|
173
|
+
findings.push({
|
|
174
|
+
ruleId: `${prefix}-${String(ruleNum++).padStart(3, "0")}`,
|
|
175
|
+
severity: "medium",
|
|
176
|
+
title: "No tests detected for production code",
|
|
177
|
+
description: "This file contains significant logic but no accompanying tests were detected.",
|
|
178
|
+
recommendation: "Write unit tests covering the main functions, edge cases, and error paths. Aim for meaningful coverage of critical paths.",
|
|
179
|
+
reference: "Test-Driven Development / Testing Pyramid",
|
|
180
|
+
});
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
return findings;
|
|
184
|
+
}
|
|
185
|
+
//# sourceMappingURL=testing.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"testing.js","sourceRoot":"","sources":["../../src/evaluators/testing.ts"],"names":[],"mappings":"AAGA,MAAM,UAAU,cAAc,CAAC,IAAY,EAAE,QAAgB;IAC3D,MAAM,QAAQ,GAAc,EAAE,CAAC;IAC/B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC/B,MAAM,MAAM,GAAG,MAAM,CAAC;IACtB,IAAI,OAAO,GAAG,CAAC,CAAC;IAEhB,uCAAuC;IACvC,MAAM,gBAAgB,GAAG,oDAAoD,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACzF,IAAI,gBAAgB,EAAE,CAAC;QACrB,uBAAuB;QACvB,MAAM,cAAc,GAAa,EAAE,CAAC;QACpC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,8FAA8F,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC9G,cAAc,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC7B,CAAC;QACH,CAAC,CAAC,CAAC;QAEH,MAAM,cAAc,GAAa,EAAE,CAAC;QACpC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,6BAA6B,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC7C,cAAc,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC7B,CAAC;QACH,CAAC,CAAC,CAAC;QAEH,IAAI,cAAc,CAAC,MAAM,GAAG,CAAC,IAAI,cAAc,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC7D,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,UAAU;gBACpB,KAAK,EAAE,+BAA+B;gBACtC,WAAW,EAAE,mHAAmH;gBAChI,WAAW,EAAE,cAAc;gBAC3B,cAAc,EAAE,uGAAuG;gBACvH,SAAS,EAAE,6BAA6B;aACzC,CAAC,CAAC;QACL,CAAC;QAED,iCAAiC;QACjC,MAAM,cAAc,GAAa,EAAE,CAAC;QACpC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,2FAA2F,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC3G,cAAc,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC7B,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC9B,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,KAAK;gBACf,KAAK,EAAE,kBAAkB;gBACzB,WAAW,EAAE,8HAA8H;gBAC3I,WAAW,EAAE,cAAc;gBAC3B,cAAc,EAAE,qHAAqH;gBACrI,SAAS,EAAE,yBAAyB;aACrC,CAAC,CAAC;QACL,CAAC;QAED,mDAAmD;QACnD,MAAM,kBAAkB,GAAa,EAAE,CAAC;QACxC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,gCAAgC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,oBAAoB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACpF,kBAAkB,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACjC,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,kBAAkB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAClC,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,KAAK;gBACf,KAAK,EAAE,0BAA0B;gBACjC,WAAW,EAAE,2FAA2F;gBACxG,WAAW,EAAE,kBAAkB;gBAC/B,cAAc,EAAE,6GAA6G;gBAC7H,SAAS,EAAE,8CAA8C;aAC1D,CAAC,CAAC;QACL,CAAC;QAED,0CAA0C;QAC1C,MAAM,gBAAgB,GAAa,EAAE,CAAC;QACtC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,wDAAwD,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,8BAA8B,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACtH,gBAAgB,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC/B,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,gBAAgB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAChC,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,QAAQ;gBAClB,KAAK,EAAE,uCAAuC;gBAC9C,WAAW,EAAE,oIAAoI;gBACjJ,WAAW,EAAE,gBAAgB;gBAC7B,cAAc,EAAE,6HAA6H;gBAC7I,SAAS,EAAE,uCAAuC;aACnD,CAAC,CAAC;QACL,CAAC;QAED,yCAAyC;QACzC,MAAM,gBAAgB,GAAa,EAAE,CAAC;QACtC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,wBAAwB,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,4CAA4C,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;gBAClH,MAAM,OAAO,GAAG,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC9D,IAAI,gBAAgB,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,6BAA6B,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;oBACnF,gBAAgB,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;gBAC/B,CAAC;YACH,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,gBAAgB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAChC,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,QAAQ;gBAClB,KAAK,EAAE,oCAAoC;gBAC3C,WAAW,EAAE,gIAAgI;gBAC7I,WAAW,EAAE,gBAAgB;gBAC7B,cAAc,EAAE,gGAAgG;gBAChH,SAAS,EAAE,+BAA+B;aAC3C,CAAC,CAAC;QACL,CAAC;QAED,2CAA2C;QAC3C,MAAM,aAAa,GAAG,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,mEAAmE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACjI,IAAI,aAAa,IAAI,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC/C,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,QAAQ;gBAClB,KAAK,EAAE,6BAA6B;gBACpC,WAAW,EAAE,0GAA0G;gBACvH,cAAc,EAAE,8HAA8H;gBAC9I,SAAS,EAAE,4BAA4B;aACxC,CAAC,CAAC;QACL,CAAC;QAED,6BAA6B;QAC7B,MAAM,UAAU,GAAa,EAAE,CAAC;QAChC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,iEAAiE,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACjF,UAAU,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC1B,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,QAAQ;gBAClB,KAAK,EAAE,gCAAgC;gBACvC,WAAW,EAAE,mHAAmH;gBAChI,WAAW,EAAE,UAAU;gBACvB,cAAc,EAAE,yHAAyH;gBACzI,SAAS,EAAE,iDAAiD;aAC7D,CAAC,CAAC;QACL,CAAC;QAED,iCAAiC;QACjC,IAAI,KAAK,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;YACvB,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,KAAK;gBACf,KAAK,EAAE,yBAAyB;gBAChC,WAAW,EAAE,iBAAiB,KAAK,CAAC,MAAM,yGAAyG;gBACnJ,cAAc,EAAE,4HAA4H;gBAC5I,SAAS,EAAE,kCAAkC;aAC9C,CAAC,CAAC;QACL,CAAC;QAED,0BAA0B;QAC1B,MAAM,aAAa,GAAa,EAAE,CAAC;QACnC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,wCAAwC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACxD,aAAa,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAC5B,CAAC;QACH,CAAC,CAAC,CAAC;QACH,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC7B,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,KAAK;gBACf,KAAK,EAAE,oCAAoC;gBAC3C,WAAW,EAAE,gIAAgI;gBAC7I,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;gBACtC,cAAc,EAAE,oHAAoH;gBACpI,SAAS,EAAE,iCAAiC;aAC7C,CAAC,CAAC;QACL,CAAC;IACH,CAAC;SAAM,CAAC;QACN,8EAA8E;QAC9E,MAAM,YAAY,GAAG,2DAA2D,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5F,MAAM,WAAW,GAAG,KAAK,CAAC,MAAM,GAAG,EAAE,CAAC;QACtC,IAAI,YAAY,IAAI,WAAW,EAAE,CAAC;YAChC,QAAQ,CAAC,IAAI,CAAC;gBACZ,MAAM,EAAE,GAAG,MAAM,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACzD,QAAQ,EAAE,QAAQ;gBAClB,KAAK,EAAE,uCAAuC;gBAC9C,WAAW,EAAE,+EAA+E;gBAC5F,cAAc,EAAE,2HAA2H;gBAC3I,SAAS,EAAE,2CAA2C;aACvD,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,OAAO,QAAQ,CAAC;AAClB,CAAC"}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Judges Panel — MCP Server
|
|
4
|
+
*
|
|
5
|
+
* An MCP server that provides a panel of specialized judges to evaluate
|
|
6
|
+
* AI-generated code. Each tool returns both automated pattern-detection
|
|
7
|
+
* findings AND the judge's deep-review criteria, enabling the calling LLM
|
|
8
|
+
* to perform thorough contextual analysis beyond what static patterns catch.
|
|
9
|
+
*
|
|
10
|
+
* Tools exposed:
|
|
11
|
+
* - evaluate_code: Full panel review (all 18 judges)
|
|
12
|
+
* - evaluate_code_single_judge: Review by a specific judge
|
|
13
|
+
* - get_judges: List all available judges
|
|
14
|
+
*/
|
|
15
|
+
export {};
|
|
16
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA;;;;;;;;;;;;GAYG"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,238 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Judges Panel — MCP Server
|
|
4
|
+
*
|
|
5
|
+
* An MCP server that provides a panel of specialized judges to evaluate
|
|
6
|
+
* AI-generated code. Each tool returns both automated pattern-detection
|
|
7
|
+
* findings AND the judge's deep-review criteria, enabling the calling LLM
|
|
8
|
+
* to perform thorough contextual analysis beyond what static patterns catch.
|
|
9
|
+
*
|
|
10
|
+
* Tools exposed:
|
|
11
|
+
* - evaluate_code: Full panel review (all 18 judges)
|
|
12
|
+
* - evaluate_code_single_judge: Review by a specific judge
|
|
13
|
+
* - get_judges: List all available judges
|
|
14
|
+
*/
|
|
15
|
+
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
|
16
|
+
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
|
17
|
+
import { z } from "zod";
|
|
18
|
+
import { JUDGES, getJudge, getJudgeSummaries } from "./judges/index.js";
|
|
19
|
+
import { evaluateWithJudge, evaluateWithTribunal, formatVerdictAsMarkdown, formatEvaluationAsMarkdown, } from "./evaluators/index.js";
|
|
20
|
+
// ─── Create MCP Server ──────────────────────────────────────────────────────
|
|
21
|
+
const server = new McpServer({
|
|
22
|
+
name: "judges",
|
|
23
|
+
version: "1.0.0",
|
|
24
|
+
});
|
|
25
|
+
// ─── Tool: get_judges ────────────────────────────────────────────────────────
|
|
26
|
+
server.tool("get_judges", "List all available judges on the Agent Tribunal panel, including their areas of expertise and what they evaluate.", {}, async () => {
|
|
27
|
+
const judges = getJudgeSummaries();
|
|
28
|
+
const text = judges
|
|
29
|
+
.map((j) => `**${j.name}** (id: \`${j.id}\`)\n Domain: ${j.domain}\n ${j.description}`)
|
|
30
|
+
.join("\n\n");
|
|
31
|
+
return {
|
|
32
|
+
content: [
|
|
33
|
+
{
|
|
34
|
+
type: "text",
|
|
35
|
+
text: `# Judges Panel\n\n${text}`,
|
|
36
|
+
},
|
|
37
|
+
],
|
|
38
|
+
};
|
|
39
|
+
});
|
|
40
|
+
// ─── Tool: evaluate_code ─────────────────────────────────────────────────────
|
|
41
|
+
server.tool("evaluate_code", `Submit code to the full Judges Panel for evaluation. All 18 judges will independently review the code using both automated pattern detection and deep contextual analysis criteria. Returns a combined verdict with scores, findings, and expert review guidance for thorough evaluation.`, {
|
|
42
|
+
code: z
|
|
43
|
+
.string()
|
|
44
|
+
.describe("The source code to evaluate. Include the full file content for best results."),
|
|
45
|
+
language: z
|
|
46
|
+
.string()
|
|
47
|
+
.describe("The programming language of the code (e.g., 'typescript', 'python', 'javascript', 'csharp', 'java')."),
|
|
48
|
+
context: z
|
|
49
|
+
.string()
|
|
50
|
+
.optional()
|
|
51
|
+
.describe("Optional additional context about the code — e.g., what the code does, which framework it uses, or the deployment target."),
|
|
52
|
+
}, async ({ code, language, context }) => {
|
|
53
|
+
const verdict = evaluateWithTribunal(code, language, context);
|
|
54
|
+
const patternResults = formatVerdictAsMarkdown(verdict);
|
|
55
|
+
const deepReview = buildTribunalDeepReviewSection(JUDGES, language, context);
|
|
56
|
+
return {
|
|
57
|
+
content: [
|
|
58
|
+
{
|
|
59
|
+
type: "text",
|
|
60
|
+
text: patternResults + deepReview,
|
|
61
|
+
},
|
|
62
|
+
],
|
|
63
|
+
};
|
|
64
|
+
});
|
|
65
|
+
// ─── Tool: evaluate_code_single_judge ────────────────────────────────────────
|
|
66
|
+
const judgeIds = JUDGES.map((j) => j.id);
|
|
67
|
+
server.tool("evaluate_code_single_judge", `Submit code to a specific judge on the Judges Panel. Use get_judges to see available judges. Available judge IDs: ${judgeIds.join(", ")}`, {
|
|
68
|
+
code: z
|
|
69
|
+
.string()
|
|
70
|
+
.describe("The source code to evaluate. Include the full file content for best results."),
|
|
71
|
+
language: z
|
|
72
|
+
.string()
|
|
73
|
+
.describe("The programming language of the code (e.g., 'typescript', 'python', 'javascript', 'csharp', 'java')."),
|
|
74
|
+
judgeId: z
|
|
75
|
+
.string()
|
|
76
|
+
.describe(`The ID of the judge to use. One of: ${judgeIds.join(", ")}`),
|
|
77
|
+
context: z
|
|
78
|
+
.string()
|
|
79
|
+
.optional()
|
|
80
|
+
.describe("Optional additional context about the code — e.g., what the code does, which framework it uses, or the deployment target."),
|
|
81
|
+
}, async ({ code, language, judgeId, context }) => {
|
|
82
|
+
const judge = getJudge(judgeId);
|
|
83
|
+
if (!judge) {
|
|
84
|
+
return {
|
|
85
|
+
content: [
|
|
86
|
+
{
|
|
87
|
+
type: "text",
|
|
88
|
+
text: `Error: Unknown judge ID "${judgeId}". Available judges: ${judgeIds.join(", ")}`,
|
|
89
|
+
},
|
|
90
|
+
],
|
|
91
|
+
isError: true,
|
|
92
|
+
};
|
|
93
|
+
}
|
|
94
|
+
const evaluation = evaluateWithJudge(judge, code, language, context);
|
|
95
|
+
const patternResults = formatEvaluationAsMarkdown(evaluation);
|
|
96
|
+
const deepReview = buildSingleJudgeDeepReviewSection(judge, language, context);
|
|
97
|
+
return {
|
|
98
|
+
content: [
|
|
99
|
+
{
|
|
100
|
+
type: "text",
|
|
101
|
+
text: patternResults + deepReview,
|
|
102
|
+
},
|
|
103
|
+
],
|
|
104
|
+
};
|
|
105
|
+
});
|
|
106
|
+
// ─── Deep Review Builders ────────────────────────────────────────────────────
|
|
107
|
+
// These functions append each judge's expert criteria to the tool output,
|
|
108
|
+
// turning pattern-only results into hybrid evaluations. The calling LLM
|
|
109
|
+
// uses these criteria to perform deep contextual analysis that regex can't.
|
|
110
|
+
// ──────────────────────────────────────────────────────────────────────────────
|
|
111
|
+
function buildSingleJudgeDeepReviewSection(judge, language, context) {
|
|
112
|
+
let md = `\n\n---\n\n`;
|
|
113
|
+
md += `## 🔍 Deep Contextual Review Required\n\n`;
|
|
114
|
+
md += `> **The pattern-based findings above are a starting point only.** `;
|
|
115
|
+
md += `Static pattern matching cannot evaluate semantic correctness, architectural coherence, `;
|
|
116
|
+
md += `business logic validity, cross-module data flow, or context-dependent implications.\n>\n`;
|
|
117
|
+
md += `> **You MUST now perform a thorough contextual review** of the ${language} code using the expert criteria below. `;
|
|
118
|
+
md += `Identify issues that pattern matching cannot catch. Incorporate both the pattern findings above AND your own deep analysis into a final, unified evaluation.\n\n`;
|
|
119
|
+
if (context) {
|
|
120
|
+
md += `**Context provided:** ${context}\n\n`;
|
|
121
|
+
}
|
|
122
|
+
md += `### ${judge.name} — ${judge.domain}\n\n`;
|
|
123
|
+
md += `${judge.systemPrompt}\n\n`;
|
|
124
|
+
md += `### Response Format\n\n`;
|
|
125
|
+
md += `Provide your deep review as additional findings using the same format:\n`;
|
|
126
|
+
md += `- Rule ID prefix: \`${judge.rulePrefix}-\`\n`;
|
|
127
|
+
md += `- Severity levels: critical / high / medium / low / info\n`;
|
|
128
|
+
md += `- Include: title, description, affected lines, recommendation, and reference\n`;
|
|
129
|
+
md += `- After all findings, provide an updated score (0-100) and final verdict (PASS/WARNING/FAIL)\n`;
|
|
130
|
+
md += `- The final verdict must account for BOTH the pattern findings AND your contextual findings\n`;
|
|
131
|
+
return md;
|
|
132
|
+
}
|
|
133
|
+
function buildTribunalDeepReviewSection(judges, language, context) {
|
|
134
|
+
let md = `\n\n---\n\n`;
|
|
135
|
+
md += `## 🔍 Deep Contextual Review Required\n\n`;
|
|
136
|
+
md += `> **The pattern-based tribunal findings above are a starting point only.** `;
|
|
137
|
+
md += `Static pattern matching cannot evaluate semantic correctness, architectural coherence, `;
|
|
138
|
+
md += `business logic validity, cross-module data flow, or context-dependent implications.\n>\n`;
|
|
139
|
+
md += `> **You MUST now perform a thorough contextual review** of the ${language} code from the perspective of ALL ${judges.length} judges below. `;
|
|
140
|
+
md += `Identify issues that pattern matching cannot catch. Incorporate both the pattern findings above AND your own deep analysis into a final, unified tribunal verdict.\n\n`;
|
|
141
|
+
if (context) {
|
|
142
|
+
md += `**Context provided:** ${context}\n\n`;
|
|
143
|
+
}
|
|
144
|
+
for (const judge of judges) {
|
|
145
|
+
md += `### ${judge.name} — ${judge.domain}\n\n`;
|
|
146
|
+
md += `${judge.systemPrompt}\n\n`;
|
|
147
|
+
md += `---\n\n`;
|
|
148
|
+
}
|
|
149
|
+
md += `### Response Format\n\n`;
|
|
150
|
+
md += `For each judge, provide any additional findings your contextual analysis uncovers using:\n`;
|
|
151
|
+
md += `- The judge's rule ID prefix\n`;
|
|
152
|
+
md += `- Severity levels: critical / high / medium / low / info\n`;
|
|
153
|
+
md += `- Include: title, description, affected lines, recommendation, and reference\n\n`;
|
|
154
|
+
md += `Then provide an **OVERALL UPDATED TRIBUNAL VERDICT** that accounts for BOTH the pattern findings AND your contextual findings:\n`;
|
|
155
|
+
md += `- Per-judge scores (0-100) and verdicts\n`;
|
|
156
|
+
md += `- Overall score and verdict (PASS/WARNING/FAIL)\n`;
|
|
157
|
+
md += `- Executive summary of the most critical issues\n`;
|
|
158
|
+
return md;
|
|
159
|
+
}
|
|
160
|
+
// ─── Prompts ─────────────────────────────────────────────────────────────────
|
|
161
|
+
// Expose the judges' system prompts as MCP prompts so that an LLM-based
|
|
162
|
+
// client can use them for deeper, AI-powered analysis beyond pattern matching.
|
|
163
|
+
for (const judge of JUDGES) {
|
|
164
|
+
server.prompt(`judge-${judge.id}`, `Use the ${judge.name} persona to perform a deep ${judge.domain} review of code. This prompt provides the judge's expert criteria for LLM-powered analysis that goes beyond pattern matching.`, {
|
|
165
|
+
code: z
|
|
166
|
+
.string()
|
|
167
|
+
.describe("The source code to evaluate"),
|
|
168
|
+
language: z
|
|
169
|
+
.string()
|
|
170
|
+
.describe("The programming language"),
|
|
171
|
+
context: z
|
|
172
|
+
.string()
|
|
173
|
+
.optional()
|
|
174
|
+
.describe("Additional context about the code"),
|
|
175
|
+
}, async ({ code, language, context }) => {
|
|
176
|
+
const userMessage = `Please evaluate the following ${language} code:\n\n\`\`\`${language}\n${code}\n\`\`\`` +
|
|
177
|
+
(context ? `\n\nAdditional context: ${context}` : "") +
|
|
178
|
+
`\n\nProvide your evaluation as structured findings with rule IDs (prefix: ${judge.rulePrefix}-), severity levels (critical/high/medium/low/info), descriptions, and actionable recommendations. End with an overall score (0-100) and verdict (pass/warning/fail).`;
|
|
179
|
+
return {
|
|
180
|
+
messages: [
|
|
181
|
+
{
|
|
182
|
+
role: "user",
|
|
183
|
+
content: {
|
|
184
|
+
type: "text",
|
|
185
|
+
text: userMessage,
|
|
186
|
+
},
|
|
187
|
+
},
|
|
188
|
+
],
|
|
189
|
+
};
|
|
190
|
+
});
|
|
191
|
+
}
|
|
192
|
+
// Full tribunal prompt
|
|
193
|
+
server.prompt("full-tribunal", "Convene the full Judges Panel — all 18 judges evaluate the code in their respective domains and produce a combined verdict.", {
|
|
194
|
+
code: z
|
|
195
|
+
.string()
|
|
196
|
+
.describe("The source code to evaluate"),
|
|
197
|
+
language: z
|
|
198
|
+
.string()
|
|
199
|
+
.describe("The programming language"),
|
|
200
|
+
context: z
|
|
201
|
+
.string()
|
|
202
|
+
.optional()
|
|
203
|
+
.describe("Additional context about the code"),
|
|
204
|
+
}, async ({ code, language, context }) => {
|
|
205
|
+
const judgeInstructions = JUDGES.map((j) => `### ${j.name} — ${j.domain}\n${j.systemPrompt}`).join("\n\n---\n\n");
|
|
206
|
+
const userMessage = `You are the Judges Panel — a panel of ${JUDGES.length} expert judges who independently evaluate code for quality, security, and operational readiness.\n\n` +
|
|
207
|
+
`Evaluate the following ${language} code from the perspective of ALL ${JUDGES.length} judges below. For each judge, provide:\n` +
|
|
208
|
+
`1. Judge name and domain\n` +
|
|
209
|
+
`2. Verdict (PASS / WARNING / FAIL)\n` +
|
|
210
|
+
`3. Score (0-100)\n` +
|
|
211
|
+
`4. Specific findings with rule IDs, severity, and recommendations\n\n` +
|
|
212
|
+
`Then provide an OVERALL TRIBUNAL VERDICT that synthesizes all judges' input.\n\n` +
|
|
213
|
+
`## The Judges\n\n${judgeInstructions}\n\n` +
|
|
214
|
+
`## Code to Evaluate\n\n\`\`\`${language}\n${code}\n\`\`\`` +
|
|
215
|
+
(context ? `\n\n## Additional Context\n${context}` : "");
|
|
216
|
+
return {
|
|
217
|
+
messages: [
|
|
218
|
+
{
|
|
219
|
+
role: "user",
|
|
220
|
+
content: {
|
|
221
|
+
type: "text",
|
|
222
|
+
text: userMessage,
|
|
223
|
+
},
|
|
224
|
+
},
|
|
225
|
+
],
|
|
226
|
+
};
|
|
227
|
+
});
|
|
228
|
+
// ─── Start Server ────────────────────────────────────────────────────────────
|
|
229
|
+
async function main() {
|
|
230
|
+
const transport = new StdioServerTransport();
|
|
231
|
+
await server.connect(transport);
|
|
232
|
+
console.error("Judges Panel MCP server running on stdio");
|
|
233
|
+
}
|
|
234
|
+
main().catch((err) => {
|
|
235
|
+
console.error("Failed to start Judges Panel:", err);
|
|
236
|
+
process.exit(1);
|
|
237
|
+
});
|
|
238
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA;;;;;;;;;;;;GAYG;AAEH,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AACpE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,iBAAiB,EAAE,MAAM,mBAAmB,CAAC;AACxE,OAAO,EACL,iBAAiB,EACjB,oBAAoB,EACpB,uBAAuB,EACvB,0BAA0B,GAC3B,MAAM,uBAAuB,CAAC;AAG/B,+EAA+E;AAE/E,MAAM,MAAM,GAAG,IAAI,SAAS,CAAC;IAC3B,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,OAAO;CACjB,CAAC,CAAC;AAEH,gFAAgF;AAEhF,MAAM,CAAC,IAAI,CACT,YAAY,EACZ,mHAAmH,EACnH,EAAE,EACF,KAAK,IAAI,EAAE;IACT,MAAM,MAAM,GAAG,iBAAiB,EAAE,CAAC;IACnC,MAAM,IAAI,GAAG,MAAM;SAChB,GAAG,CACF,CAAC,CAAC,EAAE,EAAE,CACJ,KAAK,CAAC,CAAC,IAAI,aAAa,CAAC,CAAC,EAAE,kBAAkB,CAAC,CAAC,MAAM,OAAO,CAAC,CAAC,WAAW,EAAE,CAC/E;SACA,IAAI,CAAC,MAAM,CAAC,CAAC;IAEhB,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAe;gBACrB,IAAI,EAAE,qBAAqB,IAAI,EAAE;aAClC;SACF;KACF,CAAC;AACJ,CAAC,CACF,CAAC;AAEF,gFAAgF;AAEhF,MAAM,CAAC,IAAI,CACT,eAAe,EACf,2RAA2R,EAC3R;IACE,IAAI,EAAE,CAAC;SACJ,MAAM,EAAE;SACR,QAAQ,CACP,8EAA8E,CAC/E;IACH,QAAQ,EAAE,CAAC;SACR,MAAM,EAAE;SACR,QAAQ,CACP,sGAAsG,CACvG;IACH,OAAO,EAAE,CAAC;SACP,MAAM,EAAE;SACR,QAAQ,EAAE;SACV,QAAQ,CACP,2HAA2H,CAC5H;CACJ,EACD,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,EAAE;IACpC,MAAM,OAAO,GAAG,oBAAoB,CAAC,IAAI,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IAC9D,MAAM,cAAc,GAAG,uBAAuB,CAAC,OAAO,CAAC,CAAC;IACxD,MAAM,UAAU,GAAG,8BAA8B,CAAC,MAAM,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IAE7E,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAe;gBACrB,IAAI,EAAE,cAAc,GAAG,UAAU;aAClC;SACF;KACF,CAAC;AACJ,CAAC,CACF,CAAC;AAEF,gFAAgF;AAEhF,MAAM,QAAQ,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;AAEzC,MAAM,CAAC,IAAI,CACT,4BAA4B,EAC5B,qHAAqH,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,EAC1I;IACE,IAAI,EAAE,CAAC;SACJ,MAAM,EAAE;SACR,QAAQ,CACP,8EAA8E,CAC/E;IACH,QAAQ,EAAE,CAAC;SACR,MAAM,EAAE;SACR,QAAQ,CACP,sGAAsG,CACvG;IACH,OAAO,EAAE,CAAC;SACP,MAAM,EAAE;SACR,QAAQ,CACP,uCAAuC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAC7D;IACH,OAAO,EAAE,CAAC;SACP,MAAM,EAAE;SACR,QAAQ,EAAE;SACV,QAAQ,CACP,2HAA2H,CAC5H;CACJ,EACD,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,OAAO,EAAE,EAAE,EAAE;IAC7C,MAAM,KAAK,GAAG,QAAQ,CAAC,OAAO,CAAC,CAAC;IAChC,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO;YACL,OAAO,EAAE;gBACP;oBACE,IAAI,EAAE,MAAe;oBACrB,IAAI,EAAE,4BAA4B,OAAO,wBAAwB,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;iBACvF;aACF;YACD,OAAO,EAAE,IAAI;SACd,CAAC;IACJ,CAAC;IAED,MAAM,UAAU,GAAG,iBAAiB,CAAC,KAAK,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IACrE,MAAM,cAAc,GAAG,0BAA0B,CAAC,UAAU,CAAC,CAAC;IAC9D,MAAM,UAAU,GAAG,iCAAiC,CAAC,KAAK,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IAE/E,OAAO;QACL,OAAO,EAAE;YACP;gBACE,IAAI,EAAE,MAAe;gBACrB,IAAI,EAAE,cAAc,GAAG,UAAU;aAClC;SACF;KACF,CAAC;AACJ,CAAC,CACF,CAAC;AAEF,gFAAgF;AAChF,0EAA0E;AAC1E,wEAAwE;AACxE,4EAA4E;AAC5E,iFAAiF;AAEjF,SAAS,iCAAiC,CACxC,KAAsB,EACtB,QAAgB,EAChB,OAAgB;IAEhB,IAAI,EAAE,GAAG,aAAa,CAAC;IACvB,EAAE,IAAI,2CAA2C,CAAC;IAClD,EAAE,IAAI,oEAAoE,CAAC;IAC3E,EAAE,IAAI,yFAAyF,CAAC;IAChG,EAAE,IAAI,0FAA0F,CAAC;IACjG,EAAE,IAAI,kEAAkE,QAAQ,yCAAyC,CAAC;IAC1H,EAAE,IAAI,kKAAkK,CAAC;IAEzK,IAAI,OAAO,EAAE,CAAC;QACZ,EAAE,IAAI,yBAAyB,OAAO,MAAM,CAAC;IAC/C,CAAC;IAED,EAAE,IAAI,OAAO,KAAK,CAAC,IAAI,MAAM,KAAK,CAAC,MAAM,MAAM,CAAC;IAChD,EAAE,IAAI,GAAG,KAAK,CAAC,YAAY,MAAM,CAAC;IAElC,EAAE,IAAI,yBAAyB,CAAC;IAChC,EAAE,IAAI,0EAA0E,CAAC;IACjF,EAAE,IAAI,uBAAuB,KAAK,CAAC,UAAU,OAAO,CAAC;IACrD,EAAE,IAAI,4DAA4D,CAAC;IACnE,EAAE,IAAI,gFAAgF,CAAC;IACvF,EAAE,IAAI,gGAAgG,CAAC;IACvG,EAAE,IAAI,+FAA+F,CAAC;IAEtG,OAAO,EAAE,CAAC;AACZ,CAAC;AAED,SAAS,8BAA8B,CACrC,MAAyB,EACzB,QAAgB,EAChB,OAAgB;IAEhB,IAAI,EAAE,GAAG,aAAa,CAAC;IACvB,EAAE,IAAI,2CAA2C,CAAC;IAClD,EAAE,IAAI,6EAA6E,CAAC;IACpF,EAAE,IAAI,yFAAyF,CAAC;IAChG,EAAE,IAAI,0FAA0F,CAAC;IACjG,EAAE,IAAI,kEAAkE,QAAQ,qCAAqC,MAAM,CAAC,MAAM,iBAAiB,CAAC;IACpJ,EAAE,IAAI,wKAAwK,CAAC;IAE/K,IAAI,OAAO,EAAE,CAAC;QACZ,EAAE,IAAI,yBAAyB,OAAO,MAAM,CAAC;IAC/C,CAAC;IAED,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;QAC3B,EAAE,IAAI,OAAO,KAAK,CAAC,IAAI,MAAM,KAAK,CAAC,MAAM,MAAM,CAAC;QAChD,EAAE,IAAI,GAAG,KAAK,CAAC,YAAY,MAAM,CAAC;QAClC,EAAE,IAAI,SAAS,CAAC;IAClB,CAAC;IAED,EAAE,IAAI,yBAAyB,CAAC;IAChC,EAAE,IAAI,4FAA4F,CAAC;IACnG,EAAE,IAAI,gCAAgC,CAAC;IACvC,EAAE,IAAI,4DAA4D,CAAC;IACnE,EAAE,IAAI,kFAAkF,CAAC;IACzF,EAAE,IAAI,kIAAkI,CAAC;IACzI,EAAE,IAAI,2CAA2C,CAAC;IAClD,EAAE,IAAI,mDAAmD,CAAC;IAC1D,EAAE,IAAI,mDAAmD,CAAC;IAE1D,OAAO,EAAE,CAAC;AACZ,CAAC;AAED,gFAAgF;AAChF,wEAAwE;AACxE,+EAA+E;AAE/E,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;IAC3B,MAAM,CAAC,MAAM,CACX,SAAS,KAAK,CAAC,EAAE,EAAE,EACnB,WAAW,KAAK,CAAC,IAAI,8BAA8B,KAAK,CAAC,MAAM,+HAA+H,EAC9L;QACE,IAAI,EAAE,CAAC;aACJ,MAAM,EAAE;aACR,QAAQ,CAAC,6BAA6B,CAAC;QAC1C,QAAQ,EAAE,CAAC;aACR,MAAM,EAAE;aACR,QAAQ,CAAC,0BAA0B,CAAC;QACvC,OAAO,EAAE,CAAC;aACP,MAAM,EAAE;aACR,QAAQ,EAAE;aACV,QAAQ,CAAC,mCAAmC,CAAC;KACjD,EACD,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,EAAE;QACpC,MAAM,WAAW,GACf,iCAAiC,QAAQ,mBAAmB,QAAQ,KAAK,IAAI,UAAU;YACvF,CAAC,OAAO,CAAC,CAAC,CAAC,2BAA2B,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YACrD,6EAA6E,KAAK,CAAC,UAAU,uKAAuK,CAAC;QAEvQ,OAAO;YACL,QAAQ,EAAE;gBACR;oBACE,IAAI,EAAE,MAAe;oBACrB,OAAO,EAAE;wBACP,IAAI,EAAE,MAAe;wBACrB,IAAI,EAAE,WAAW;qBAClB;iBACF;aACF;SACF,CAAC;IACJ,CAAC,CACF,CAAC;AACJ,CAAC;AAED,uBAAuB;AACvB,MAAM,CAAC,MAAM,CACX,eAAe,EACf,6HAA6H,EAC7H;IACE,IAAI,EAAE,CAAC;SACJ,MAAM,EAAE;SACR,QAAQ,CAAC,6BAA6B,CAAC;IAC1C,QAAQ,EAAE,CAAC;SACR,MAAM,EAAE;SACR,QAAQ,CAAC,0BAA0B,CAAC;IACvC,OAAO,EAAE,CAAC;SACP,MAAM,EAAE;SACR,QAAQ,EAAE;SACV,QAAQ,CAAC,mCAAmC,CAAC;CACjD,EACD,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,EAAE;IACpC,MAAM,iBAAiB,GAAG,MAAM,CAAC,GAAG,CAClC,CAAC,CAAC,EAAE,EAAE,CACJ,OAAO,CAAC,CAAC,IAAI,MAAM,CAAC,CAAC,MAAM,KAAK,CAAC,CAAC,YAAY,EAAE,CACnD,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC;IAEtB,MAAM,WAAW,GACf,yCAAyC,MAAM,CAAC,MAAM,sGAAsG;QAC5J,0BAA0B,QAAQ,qCAAqC,MAAM,CAAC,MAAM,2CAA2C;QAC/H,4BAA4B;QAC5B,sCAAsC;QACtC,oBAAoB;QACpB,uEAAuE;QACvE,kFAAkF;QAClF,oBAAoB,iBAAiB,MAAM;QAC3C,gCAAgC,QAAQ,KAAK,IAAI,UAAU;QAC3D,CAAC,OAAO,CAAC,CAAC,CAAC,8BAA8B,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;IAE3D,OAAO;QACL,QAAQ,EAAE;YACR;gBACE,IAAI,EAAE,MAAe;gBACrB,OAAO,EAAE;oBACP,IAAI,EAAE,MAAe;oBACrB,IAAI,EAAE,WAAW;iBAClB;aACF;SACF;KACF,CAAC;AACJ,CAAC,CACF,CAAC;AAEF,gFAAgF;AAEhF,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAChC,OAAO,CAAC,KAAK,CAAC,0CAA0C,CAAC,CAAC;AAC5D,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,+BAA+B,EAAE,GAAG,CAAC,CAAC;IACpD,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"accessibility.d.ts","sourceRoot":"","sources":["../../src/judges/accessibility.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAE9C,eAAO,MAAM,kBAAkB,EAAE,eA2BhC,CAAC"}
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
export const accessibilityJudge = {
|
|
2
|
+
id: "accessibility",
|
|
3
|
+
name: "Judge Accessibility",
|
|
4
|
+
domain: "Accessibility (a11y)",
|
|
5
|
+
description: "Evaluates code for WCAG compliance, ARIA attributes, keyboard navigation, screen reader support, color contrast, semantic HTML, and inclusive design patterns.",
|
|
6
|
+
rulePrefix: "A11Y",
|
|
7
|
+
systemPrompt: `You are Judge Accessibility — a certified accessibility specialist (IAAP CPWA) with 15+ years building inclusive digital experiences, deep expertise in WCAG 2.2, WAI-ARIA, and assistive technology compatibility.
|
|
8
|
+
|
|
9
|
+
YOUR EVALUATION CRITERIA:
|
|
10
|
+
1. **Semantic HTML**: Are semantic elements used (nav, main, article, section, header, footer) instead of generic divs/spans? Are headings properly hierarchical (h1→h2→h3)?
|
|
11
|
+
2. **ARIA Attributes**: Are ARIA roles, states, and properties used correctly? Are they unnecessary where native HTML semantics suffice? Are live regions used for dynamic content?
|
|
12
|
+
3. **Keyboard Navigation**: Can all interactive elements be reached and operated via keyboard? Is focus management correct (tab order, focus trapping in modals, visible focus indicators)?
|
|
13
|
+
4. **Screen Reader Support**: Are images given meaningful alt text? Are form inputs labeled? Are decorative elements hidden from assistive technology?
|
|
14
|
+
5. **Color & Contrast**: Does the design rely solely on color to convey information? Are contrast ratios sufficient (4.5:1 for normal text, 3:1 for large text per WCAG AA)?
|
|
15
|
+
6. **Forms & Inputs**: Are error messages associated with their fields? Are required fields indicated programmatically? Is autocomplete used where appropriate?
|
|
16
|
+
7. **Responsive & Touch**: Is the interface usable at 200% zoom? Are touch targets at least 44x44px? Is content reflow handled without horizontal scrolling?
|
|
17
|
+
8. **Motion & Animation**: Is there a prefers-reduced-motion check? Can animations be paused? Are auto-playing media controllable?
|
|
18
|
+
9. **Dynamic Content**: Are AJAX-loaded updates announced to screen readers? Are loading states communicated? Are route changes announced in SPAs?
|
|
19
|
+
10. **Document Structure**: Is there a skip navigation link? Is the page language set? Are landmarks used appropriately?
|
|
20
|
+
|
|
21
|
+
RULES FOR YOUR EVALUATION:
|
|
22
|
+
- Assign rule IDs with prefix "A11Y-" (e.g. A11Y-001).
|
|
23
|
+
- Reference specific WCAG 2.2 success criteria (e.g., "1.1.1 Non-text Content", "2.1.1 Keyboard").
|
|
24
|
+
- Indicate the WCAG conformance level impacted (A, AA, or AAA).
|
|
25
|
+
- Recommend fixes with code examples using proper ARIA patterns.
|
|
26
|
+
- Score from 0-100 where 100 means fully WCAG 2.2 AA compliant.`,
|
|
27
|
+
};
|
|
28
|
+
//# sourceMappingURL=accessibility.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"accessibility.js","sourceRoot":"","sources":["../../src/judges/accessibility.ts"],"names":[],"mappings":"AAEA,MAAM,CAAC,MAAM,kBAAkB,GAAoB;IACjD,EAAE,EAAE,eAAe;IACnB,IAAI,EAAE,qBAAqB;IAC3B,MAAM,EAAE,sBAAsB;IAC9B,WAAW,EACT,gKAAgK;IAClK,UAAU,EAAE,MAAM;IAClB,YAAY,EAAE;;;;;;;;;;;;;;;;;;;gEAmBgD;CAC/D,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"api-design.d.ts","sourceRoot":"","sources":["../../src/judges/api-design.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAE9C,eAAO,MAAM,cAAc,EAAE,eA6B5B,CAAC"}
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
export const apiDesignJudge = {
|
|
2
|
+
id: "api-design",
|
|
3
|
+
name: "Judge API Design",
|
|
4
|
+
domain: "API Design & Contracts",
|
|
5
|
+
description: "Evaluates API design for RESTful conventions, naming consistency, proper HTTP status codes, versioning, pagination, error contract consistency, and backward compatibility.",
|
|
6
|
+
rulePrefix: "API",
|
|
7
|
+
systemPrompt: `You are Judge API Design — a senior API architect who has designed and governed public APIs used by millions of developers, with deep expertise in REST, GraphQL, gRPC, and API governance.
|
|
8
|
+
|
|
9
|
+
YOUR EVALUATION CRITERIA:
|
|
10
|
+
1. **RESTful Conventions**: Are resources named as nouns (plural)? Are HTTP methods used correctly (GET=read, POST=create, PUT=replace, PATCH=update, DELETE=remove)?
|
|
11
|
+
2. **URL Structure**: Are URLs clean, hierarchical, and consistent? Are query parameters used for filtering/sorting/pagination? Is nesting appropriate (max 2 levels)?
|
|
12
|
+
3. **HTTP Status Codes**: Are correct status codes returned (201 Created, 204 No Content, 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict, 422 Unprocessable, 429 Too Many Requests)?
|
|
13
|
+
4. **Error Responses**: Is there a consistent error response schema (error code, message, details, request ID)? Are errors actionable and developer-friendly?
|
|
14
|
+
5. **Versioning**: Is the API versioned (URL path, header, or query parameter)? Is there a strategy for deprecation and sunset?
|
|
15
|
+
6. **Pagination**: Are list endpoints paginated? Is cursor-based or offset pagination used consistently? Are total counts and next/prev links provided?
|
|
16
|
+
7. **Filtering & Sorting**: Are query parameters standardized for filtering and sorting? Are field names consistent with the response schema?
|
|
17
|
+
8. **Request/Response Schemas**: Are request and response bodies well-structured with consistent naming (camelCase or snake_case, not mixed)? Are nullable fields explicit?
|
|
18
|
+
9. **HATEOAS & Discoverability**: Are hypermedia links provided for related resources? Is the API self-documenting?
|
|
19
|
+
10. **Backward Compatibility**: Do changes break existing clients? Are new fields additive (not removing/renaming existing ones)?
|
|
20
|
+
11. **Rate Limiting Headers**: Are X-RateLimit-Limit, X-RateLimit-Remaining, and Retry-After headers included?
|
|
21
|
+
12. **OpenAPI / Documentation**: Is there an OpenAPI/Swagger specification? Are examples provided for each endpoint?
|
|
22
|
+
|
|
23
|
+
RULES FOR YOUR EVALUATION:
|
|
24
|
+
- Assign rule IDs with prefix "API-" (e.g. API-001).
|
|
25
|
+
- Reference REST API design guides (Google, Microsoft, Zalando API guidelines).
|
|
26
|
+
- Show corrected URL structures and response schemas in examples.
|
|
27
|
+
- Consider both API producer and consumer perspectives.
|
|
28
|
+
- Score from 0-100 where 100 means exemplary API design.`,
|
|
29
|
+
};
|
|
30
|
+
//# sourceMappingURL=api-design.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"api-design.js","sourceRoot":"","sources":["../../src/judges/api-design.ts"],"names":[],"mappings":"AAEA,MAAM,CAAC,MAAM,cAAc,GAAoB;IAC7C,EAAE,EAAE,YAAY;IAChB,IAAI,EAAE,kBAAkB;IACxB,MAAM,EAAE,wBAAwB;IAChC,WAAW,EACT,6KAA6K;IAC/K,UAAU,EAAE,KAAK;IACjB,YAAY,EAAE;;;;;;;;;;;;;;;;;;;;;yDAqByC;CACxD,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cloud-readiness.d.ts","sourceRoot":"","sources":["../../src/judges/cloud-readiness.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAE9C,eAAO,MAAM,mBAAmB,EAAE,eA2BjC,CAAC"}
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
export const cloudReadinessJudge = {
|
|
2
|
+
id: "cloud-readiness",
|
|
3
|
+
name: "Judge Cloud Readiness",
|
|
4
|
+
domain: "Cloud-Native Architecture & DevOps",
|
|
5
|
+
description: "Evaluates code for cloud-native patterns, 12-factor app compliance, containerization readiness, infrastructure as code, observability, and CI/CD maturity.",
|
|
6
|
+
rulePrefix: "CLOUD",
|
|
7
|
+
systemPrompt: `You are Judge Cloud Readiness — a cloud-native architect and DevOps practitioner certified across AWS, Azure, and GCP with deep expertise in platform engineering and SRE.
|
|
8
|
+
|
|
9
|
+
YOUR EVALUATION CRITERIA:
|
|
10
|
+
1. **12-Factor App Compliance**: Are configuration values externalized via environment variables? Are dependencies explicitly declared? Is the codebase suitable for stateless, disposable processes?
|
|
11
|
+
2. **Containerization**: Is the application container-friendly? Are there hardcoded paths, ports, or host dependencies? Would a Dockerfile be straightforward?
|
|
12
|
+
3. **Infrastructure as Code**: Are infrastructure dependencies defined as code (Terraform, Pulumi, CloudFormation, Bicep)? Or are there manual provisioning assumptions?
|
|
13
|
+
4. **Observability**: Is there structured logging? Are metrics exposed (Prometheus, OpenTelemetry)? Is distributed tracing implemented? Are health check endpoints provided?
|
|
14
|
+
5. **CI/CD Readiness**: Is the code testable? Are there clear build, test, and deploy stages? Are feature flags used for progressive rollout?
|
|
15
|
+
6. **Service Discovery & Configuration**: Are service URLs hardcoded or dynamically resolved? Is there support for configuration management systems?
|
|
16
|
+
7. **Resilience Patterns**: Are circuit breakers, retries with backoff, timeouts, and bulkheads implemented? Is the application designed to handle transient cloud failures?
|
|
17
|
+
8. **Multi-Cloud / Vendor Lock-In**: Is the code tightly coupled to a specific cloud provider? Are there abstraction layers for cloud-specific services?
|
|
18
|
+
9. **Security in the Cloud**: Are IAM roles used instead of long-lived credentials? Is network segmentation considered? Are secure defaults applied?
|
|
19
|
+
10. **Graceful Shutdown**: Does the application handle SIGTERM gracefully? Are in-flight requests completed before shutdown?
|
|
20
|
+
|
|
21
|
+
RULES FOR YOUR EVALUATION:
|
|
22
|
+
- Assign rule IDs with prefix "CLOUD-" (e.g. CLOUD-001).
|
|
23
|
+
- Reference the 12-Factor App methodology, CNCF patterns, and Well-Architected Framework principles.
|
|
24
|
+
- Distinguish between "can run in the cloud" and "cloud-native."
|
|
25
|
+
- Recommend specific services or patterns (e.g., "Use Azure Key Vault instead of .env files in production").
|
|
26
|
+
- Score from 0-100 where 100 means fully cloud-native.`,
|
|
27
|
+
};
|
|
28
|
+
//# sourceMappingURL=cloud-readiness.js.map
|