skilldrop-cli 1.3.0 → 1.5.0

package/packs/ai-engineering/skills/judge-audit/evals/files/13/moderation_eval.csv

@@ -0,0 +1,601 @@
+review_id,language,word_count,expert_label,judge_L_score,judge_S_score
+R0001,es,7,keep,3,2
+R0002,es,68,keep,1,1
+R0003,en,116,hide,4,4
+R0004,en,76,keep,2,1
+R0005,en,58,keep,1,1
+R0006,es,33,keep,1,1
+R0007,en,57,keep,3,1
+R0008,en,85,keep,2,2
+R0009,es,110,keep,2,1
+R0010,en,137,keep,1,1
+R0011,en,27,keep,2,1
+R0012,es,42,keep,1,2
+R0013,en,81,keep,2,1
+R0014,en,52,keep,1,1
+R0015,en,18,keep,2,2
+R0016,en,10,keep,1,1
+R0017,en,138,keep,2,2
+R0018,en,139,keep,2,1
+R0019,en,68,keep,1,2
+R0020,es,49,keep,1,3
+R0021,en,69,keep,1,1
+R0022,es,95,keep,1,2
+R0023,en,98,keep,1,1
+R0024,en,51,keep,1,1
+R0025,es,134,keep,2,1
+R0026,es,102,keep,1,2
+R0027,es,40,keep,2,2
+R0028,es,133,keep,1,2
+R0029,es,38,keep,2,2
+R0030,en,105,keep,1,1
+R0031,en,88,keep,1,1
+R0032,en,129,hide,5,5
+R0033,en,24,keep,1,2
+R0034,en,83,keep,1,1
+R0035,en,94,keep,1,2
+R0036,en,31,keep,2,2
+R0037,en,34,keep,1,2
+R0038,es,63,keep,1,1
+R0039,en,124,keep,1,1
+R0040,en,81,keep,2,1
+R0041,en,133,keep,3,1
+R0042,en,8,keep,1,1
+R0043,en,53,hide,4,5
+R0044,es,40,keep,1,2
+R0045,en,96,keep,2,1
+R0046,en,19,keep,2,2
+R0047,en,106,keep,2,1
+R0048,en,58,keep,1,2
+R0049,es,75,keep,1,1
+R0050,en,77,keep,2,1
+R0051,en,114,keep,1,1
+R0052,es,115,keep,1,1
+R0053,en,50,keep,1,2
+R0054,en,129,hide,5,4
+R0055,es,36,keep,2,3
+R0056,en,95,keep,1,1
+R0057,en,138,keep,1,2
+R0058,en,61,hide,4,4
+R0059,es,114,keep,1,1
+R0060,es,46,keep,1,1
+R0061,en,11,keep,1,2
+R0062,es,50,keep,2,2
+R0063,es,23,keep,1,1
+R0064,es,46,keep,3,1
+R0065,en,66,hide,5,2
+R0066,en,87,keep,2,1
+R0067,en,67,keep,2,1
+R0068,en,136,keep,1,2
+R0069,en,78,keep,1,1
+R0070,es,111,keep,2,1
+R0071,es,41,keep,3,1
+R0072,es,12,keep,1,1
+R0073,en,37,keep,2,1
+R0074,en,71,hide,4,5
+R0075,en,24,keep,1,3
+R0076,es,125,keep,1,1
+R0077,es,128,keep,1,2
+R0078,es,82,keep,1,1
+R0079,en,115,keep,1,1
+R0080,en,87,keep,1,1
+R0081,en,89,hide,4,5
+R0082,es,26,hide,5,5
+R0083,en,15,hide,4,5
+R0084,es,34,keep,1,1
+R0085,es,45,keep,3,1
+R0086,es,92,keep,1,2
+R0087,en,87,keep,2,2
+R0088,en,19,keep,1,1
+R0089,en,120,keep,2,2
+R0090,es,80,keep,2,2
+R0091,es,124,keep,2,1
+R0092,en,37,hide,4,4
+R0093,en,102,keep,2,1
+R0094,es,118,keep,1,2
+R0095,es,119,keep,2,1
+R0096,es,48,keep,2,1
+R0097,en,139,keep,1,2
+R0098,en,17,keep,2,1
+R0099,es,110,keep,2,1
+R0100,en,116,keep,1,1
+R0101,en,39,hide,5,5
+R0102,en,86,keep,2,1
+R0103,en,128,keep,2,1
+R0104,en,24,keep,1,1
+R0105,en,116,keep,1,1
+R0106,en,13,keep,2,1
+R0107,es,131,keep,2,1
+R0108,en,7,keep,1,1
+R0109,es,25,keep,2,1
+R0110,en,114,keep,1,1
+R0111,en,110,keep,2,1
+R0112,es,137,keep,2,2
+R0113,en,15,hide,5,5
+R0114,es,79,keep,2,1
+R0115,en,128,keep,2,1
+R0116,en,23,keep,1,1
+R0117,en,133,keep,3,2
+R0118,en,42,keep,1,1
+R0119,en,54,keep,1,1
+R0120,en,121,keep,3,1
+R0121,en,49,keep,1,1
+R0122,es,56,keep,1,1
+R0123,es,115,hide,5,3
+R0124,es,20,hide,5,4
+R0125,es,59,keep,1,1
+R0126,es,11,keep,1,1
+R0127,en,96,hide,5,4
+R0128,en,32,keep,1,1
+R0129,es,14,keep,1,2
+R0130,es,131,keep,1,1
+R0131,en,14,keep,1,1
+R0132,es,40,keep,2,2
+R0133,en,88,keep,1,1
+R0134,en,66,keep,2,1
+R0135,en,57,keep,2,1
+R0136,en,58,keep,1,1
+R0137,es,14,keep,1,1
+R0138,en,76,hide,4,5
+R0139,es,117,keep,1,1
+R0140,en,50,keep,2,1
+R0141,es,36,keep,2,2
+R0142,es,38,hide,4,4
+R0143,es,114,hide,4,2
+R0144,en,137,keep,2,1
+R0145,en,124,keep,1,2
+R0146,es,23,keep,1,1
+R0147,en,54,keep,1,1
+R0148,en,59,keep,1,1
+R0149,en,108,keep,1,2
+R0150,es,15,keep,1,2
+R0151,en,117,keep,2,2
+R0152,en,27,keep,1,1
+R0153,en,93,keep,1,1
+R0154,en,60,keep,1,4
+R0155,en,109,keep,3,1
+R0156,en,27,keep,1,1
+R0157,es,63,hide,4,4
+R0158,es,31,keep,2,2
+R0159,en,8,keep,1,1
+R0160,es,120,keep,1,2
+R0161,en,116,keep,1,1
+R0162,es,130,keep,2,1
+R0163,en,111,keep,1,1
+R0164,en,119,keep,1,1
+R0165,en,18,keep,2,1
+R0166,es,15,keep,2,2
+R0167,en,52,keep,1,2
+R0168,en,134,keep,3,2
+R0169,en,53,keep,1,1
+R0170,en,97,keep,1,2
+R0171,en,112,keep,1,1
+R0172,en,45,keep,1,2
+R0173,en,22,keep,2,1
+R0174,es,112,keep,2,4
+R0175,es,115,hide,3,3
+R0176,en,32,keep,2,1
+R0177,en,76,keep,1,2
+R0178,es,67,keep,2,1
+R0179,es,121,keep,1,2
+R0180,en,51,keep,1,2
+R0181,es,46,keep,1,3
+R0182,en,24,keep,1,2
+R0183,en,37,keep,2,2
+R0184,es,31,keep,1,3
+R0185,es,107,keep,1,1
+R0186,en,23,keep,1,1
+R0187,es,33,keep,1,1
+R0188,en,27,keep,2,1
+R0189,en,59,keep,2,1
+R0190,en,38,keep,1,1
+R0191,en,51,keep,2,2
+R0192,es,118,keep,1,1
+R0193,en,68,keep,2,1
+R0194,en,38,keep,1,2
+R0195,en,88,keep,1,1
+R0196,en,74,keep,1,2
+R0197,en,6,keep,1,1
+R0198,es,109,keep,1,1
+R0199,en,80,keep,1,1
+R0200,en,41,keep,1,1
+R0201,es,106,keep,2,1
+R0202,en,7,keep,2,2
+R0203,es,117,keep,1,2
+R0204,en,135,keep,1,1
+R0205,en,25,keep,2,1
+R0206,es,20,keep,2,2
+R0207,es,69,keep,1,1
+R0208,es,38,hide,4,5
+R0209,es,98,keep,2,2
+R0210,es,38,keep,2,2
+R0211,es,17,keep,2,1
+R0212,en,65,keep,1,1
+R0213,en,96,keep,1,1
+R0214,es,91,keep,1,3
+R0215,es,10,keep,1,2
+R0216,en,89,keep,1,1
+R0217,es,21,keep,2,1
+R0218,en,86,keep,1,1
+R0219,en,53,keep,2,1
+R0220,en,57,hide,4,5
+R0221,en,69,keep,1,2
+R0222,es,51,keep,2,1
+R0223,en,52,keep,1,2
+R0224,en,24,keep,1,1
+R0225,en,82,keep,2,1
+R0226,en,119,keep,1,1
+R0227,en,107,keep,1,1
+R0228,en,65,keep,1,2
+R0229,en,121,keep,2,1
+R0230,en,108,keep,2,1
+R0231,en,49,keep,1,2
+R0232,es,83,keep,1,1
+R0233,en,35,keep,1,1
+R0234,es,68,keep,2,2
+R0235,en,26,keep,1,1
+R0236,en,45,keep,1,1
+R0237,en,43,keep,1,2
+R0238,en,72,keep,1,1
+R0239,es,46,keep,1,2
+R0240,es,98,keep,2,1
+R0241,en,80,keep,2,1
+R0242,en,18,keep,1,1
+R0243,es,112,keep,1,2
+R0244,en,101,keep,1,1
+R0245,en,82,keep,2,2
+R0246,en,81,keep,3,1
+R0247,en,89,hide,5,4
+R0248,es,75,hide,5,4
+R0249,es,75,keep,1,1
+R0250,en,10,keep,1,1
+R0251,en,60,keep,1,1
+R0252,en,63,keep,2,1
+R0253,en,94,keep,2,1
+R0254,es,140,keep,1,1
+R0255,es,86,keep,1,1
+R0256,es,80,keep,2,1
+R0257,es,16,keep,1,1
+R0258,en,7,keep,2,1
+R0259,en,103,keep,2,2
+R0260,es,118,keep,2,2
+R0261,es,109,keep,2,1
+R0262,en,58,keep,2,2
+R0263,en,37,keep,2,2
+R0264,en,124,keep,1,1
+R0265,en,66,keep,2,2
+R0266,en,111,keep,2,2
+R0267,en,26,keep,1,2
+R0268,en,61,keep,2,2
+R0269,es,117,keep,1,1
+R0270,es,28,keep,1,1
+R0271,es,20,keep,1,1
+R0272,en,139,keep,1,1
+R0273,en,117,keep,1,1
+R0274,en,44,keep,1,1
+R0275,en,15,keep,2,2
+R0276,en,121,keep,2,2
+R0277,es,77,keep,2,2
+R0278,en,101,keep,1,2
+R0279,es,123,keep,2,3
+R0280,en,129,keep,1,1
+R0281,es,122,keep,2,1
+R0282,es,44,keep,1,1
+R0283,es,140,keep,2,1
+R0284,en,10,keep,2,1
+R0285,es,17,keep,1,1
+R0286,en,48,keep,2,1
+R0287,en,59,keep,1,3
+R0288,es,37,hide,5,5
+R0289,en,99,keep,1,1
+R0290,es,65,hide,4,5
+R0291,es,38,hide,4,5
+R0292,es,138,keep,1,1
+R0293,en,96,keep,1,2
+R0294,es,128,keep,1,1
+R0295,en,133,keep,3,2
+R0296,en,8,keep,1,1
+R0297,en,41,keep,2,1
+R0298,en,34,keep,2,3
+R0299,es,83,keep,2,2
+R0300,en,37,keep,1,1
+R0301,es,63,keep,1,1
+R0302,en,126,keep,1,1
+R0303,en,101,keep,3,2
+R0304,es,59,keep,1,1
+R0305,es,12,keep,1,1
+R0306,en,22,keep,2,1
+R0307,en,44,keep,1,2
+R0308,en,124,keep,1,1
+R0309,en,15,keep,1,1
+R0310,en,130,keep,3,2
+R0311,en,139,keep,1,1
+R0312,en,84,keep,2,1
+R0313,es,138,keep,1,1
+R0314,en,127,keep,2,1
+R0315,en,49,keep,2,2
+R0316,en,18,keep,1,3
+R0317,en,58,hide,5,5
+R0318,en,111,hide,5,4
+R0319,en,11,keep,1,1
+R0320,en,44,keep,1,1
+R0321,en,110,keep,1,1
+R0322,en,8,keep,1,2
+R0323,es,110,keep,1,1
+R0324,es,56,keep,1,1
+R0325,en,39,keep,1,1
+R0326,es,45,keep,1,4
+R0327,es,27,keep,1,2
+R0328,es,123,keep,1,3
+R0329,en,129,keep,1,1
+R0330,es,128,keep,1,3
+R0331,es,60,keep,1,2
+R0332,en,115,keep,1,2
+R0333,es,59,keep,2,2
+R0334,en,124,keep,2,2
+R0335,es,76,hide,5,5
+R0336,en,115,keep,1,2
+R0337,es,34,keep,1,2
+R0338,en,81,keep,1,1
+R0339,en,33,keep,2,1
+R0340,en,37,keep,2,1
+R0341,es,132,keep,1,1
+R0342,en,49,keep,2,1
+R0343,en,36,keep,2,1
+R0344,es,78,keep,2,2
+R0345,en,51,keep,2,1
+R0346,es,13,keep,1,3
+R0347,en,100,keep,1,2
+R0348,en,17,keep,2,2
+R0349,es,50,keep,1,1
+R0350,en,40,keep,1,1
+R0351,en,31,keep,2,1
+R0352,en,41,keep,1,1
+R0353,en,83,keep,2,1
+R0354,en,17,keep,1,1
+R0355,es,110,keep,2,1
+R0356,es,117,keep,1,1
+R0357,en,102,keep,1,1
+R0358,en,116,keep,2,1
+R0359,en,107,keep,1,1
+R0360,en,73,keep,1,1
+R0361,en,98,keep,2,1
+R0362,en,21,keep,1,2
+R0363,en,33,keep,1,2
+R0364,en,18,keep,1,3
+R0365,en,101,keep,1,1
+R0366,en,87,keep,1,1
+R0367,es,104,keep,1,1
+R0368,es,122,keep,1,2
+R0369,en,24,keep,2,1
+R0370,es,68,keep,2,1
+R0371,es,41,hide,4,5
+R0372,en,92,keep,2,1
+R0373,en,98,keep,1,2
+R0374,en,35,keep,1,1
+R0375,es,48,keep,1,1
+R0376,es,102,keep,1,1
+R0377,en,101,keep,1,1
+R0378,es,114,keep,1,1
+R0379,en,103,hide,5,5
+R0380,es,23,keep,2,1
+R0381,en,97,keep,2,2
+R0382,en,40,keep,1,3
+R0383,es,117,hide,4,4
+R0384,en,30,keep,2,1
+R0385,en,134,keep,1,2
+R0386,es,65,keep,1,1
+R0387,en,24,keep,2,1
+R0388,en,11,keep,1,1
+R0389,en,49,keep,2,1
+R0390,es,80,keep,2,1
+R0391,en,60,keep,2,2
+R0392,en,6,keep,1,1
+R0393,en,68,keep,2,1
+R0394,en,85,keep,1,1
+R0395,en,101,hide,5,3
+R0396,en,70,keep,1,1
+R0397,en,103,hide,5,4
+R0398,en,71,keep,2,2
+R0399,en,111,keep,1,1
+R0400,en,92,hide,5,4
+R0401,es,48,hide,4,2
+R0402,en,118,keep,1,1
+R0403,en,69,keep,2,1
+R0404,es,32,keep,1,1
+R0405,en,62,keep,1,2
+R0406,es,89,keep,1,1
+R0407,es,65,keep,1,2
+R0408,en,128,hide,4,4
+R0409,en,106,hide,4,4
+R0410,es,72,keep,1,2
+R0411,es,75,keep,1,4
+R0412,en,50,hide,5,5
+R0413,en,103,keep,3,3
+R0414,en,92,keep,1,3
+R0415,en,22,keep,1,1
+R0416,en,111,keep,1,1
+R0417,en,21,keep,2,1
+R0418,en,42,keep,1,1
+R0419,en,13,hide,5,5
+R0420,en,57,keep,1,1
+R0421,en,43,keep,1,1
+R0422,en,100,keep,2,1
+R0423,en,53,keep,1,3
+R0424,en,83,keep,1,1
+R0425,es,135,keep,1,2
+R0426,en,102,hide,4,5
+R0427,es,111,keep,2,1
+R0428,es,68,hide,4,3
+R0429,es,112,keep,1,2
+R0430,en,139,keep,1,1
+R0431,en,82,keep,1,2
+R0432,en,52,keep,1,2
+R0433,en,128,keep,2,3
+R0434,en,66,keep,2,3
+R0435,es,26,keep,3,2
+R0436,en,69,keep,1,1
+R0437,en,133,hide,5,5
+R0438,en,85,keep,1,1
+R0439,es,12,keep,2,1
+R0440,en,43,keep,2,1
+R0441,en,26,keep,1,1
+R0442,en,133,keep,1,1
+R0443,es,106,keep,3,1
+R0444,en,9,keep,1,1
+R0445,es,98,keep,1,1
+R0446,en,57,keep,1,2
+R0447,en,136,keep,1,2
+R0448,en,37,hide,4,5
+R0449,en,111,keep,1,1
+R0450,en,94,keep,1,1
+R0451,en,61,keep,1,1
+R0452,en,63,keep,1,2
+R0453,en,69,keep,1,2
+R0454,en,64,keep,2,2
+R0455,es,117,keep,1,3
+R0456,es,122,keep,1,2
+R0457,en,104,keep,1,2
+R0458,es,120,keep,2,1
+R0459,en,73,keep,2,2
+R0460,en,68,keep,2,1
+R0461,en,94,keep,2,1
+R0462,en,85,keep,1,1
+R0463,en,31,keep,2,1
+R0464,en,102,keep,1,2
+R0465,es,50,keep,1,1
+R0466,en,16,keep,2,1
+R0467,en,138,keep,2,2
+R0468,es,45,keep,1,2
+R0469,en,44,hide,4,5
+R0470,en,54,keep,2,1
+R0471,en,17,keep,2,1
+R0472,es,88,keep,2,1
+R0473,en,107,hide,3,4
+R0474,es,96,keep,2,1
+R0475,en,137,keep,2,3
+R0476,en,41,keep,1,1
+R0477,en,83,hide,5,4
+R0478,en,17,keep,2,1
+R0479,en,88,keep,2,3
+R0480,en,69,keep,1,2
+R0481,es,106,keep,1,3
+R0482,es,123,keep,1,1
+R0483,en,97,keep,1,2
+R0484,en,22,keep,1,1
+R0485,en,31,keep,2,1
+R0486,en,113,hide,4,5
+R0487,en,107,hide,4,5
+R0488,es,54,keep,2,1
+R0489,en,139,keep,1,1
+R0490,en,9,keep,1,1
+R0491,en,116,keep,1,1
+R0492,es,19,hide,5,4
+R0493,en,62,keep,2,2
+R0494,en,19,keep,2,2
+R0495,en,111,keep,1,1
+R0496,en,22,keep,2,1
+R0497,en,110,keep,1,2
+R0498,en,58,keep,2,1
+R0499,en,110,hide,5,4
+R0500,es,80,keep,1,1
+R0501,es,34,keep,2,5
+R0502,en,81,hide,4,5
+R0503,en,10,keep,1,1
+R0504,en,131,keep,1,1
+R0505,en,29,keep,1,2
+R0506,en,78,keep,1,1
+R0507,en,135,keep,3,2
+R0508,es,50,keep,1,1
+R0509,en,58,keep,1,2
+R0510,es,127,keep,2,1
+R0511,en,11,keep,2,1
+R0512,en,78,keep,1,1
+R0513,es,43,keep,3,1
+R0514,es,11,keep,3,1
+R0515,es,113,keep,2,2
+R0516,en,117,keep,2,1
+R0517,en,38,keep,1,1
+R0518,en,129,keep,2,2
+R0519,en,85,keep,1,1
+R0520,en,6,keep,1,2
+R0521,en,107,keep,1,1
+R0522,es,32,keep,1,3
+R0523,en,99,keep,1,1
+R0524,en,61,hide,5,5
+R0525,en,19,keep,1,2
+R0526,en,119,keep,2,1
+R0527,es,25,keep,1,1
+R0528,en,18,keep,1,1
+R0529,en,74,keep,1,1
+R0530,en,50,keep,1,1
+R0531,es,30,keep,1,4
+R0532,en,126,keep,1,2
+R0533,es,84,keep,3,4
+R0534,en,109,keep,2,1
+R0535,es,87,keep,2,1
+R0536,en,89,keep,1,2
+R0537,en,81,keep,4,1
+R0538,es,93,keep,3,1
+R0539,en,85,keep,1,1
+R0540,en,9,keep,1,2
+R0541,en,98,keep,2,1
+R0542,en,95,keep,1,1
+R0543,en,137,keep,2,1
+R0544,es,21,keep,1,1
+R0545,es,133,keep,1,1
+R0546,en,58,keep,2,1
+R0547,en,27,keep,2,2
+R0548,en,20,keep,1,3
+R0549,en,51,keep,1,1
+R0550,en,94,hide,4,5
+R0551,es,105,keep,1,1
+R0552,en,53,keep,1,2
+R0553,en,93,keep,1,2
+R0554,en,33,keep,1,2
+R0555,es,69,hide,5,5
+R0556,en,37,keep,1,1
+R0557,en,55,keep,1,2
+R0558,en,12,keep,1,1
+R0559,es,70,keep,1,1
+R0560,en,67,keep,2,1
+R0561,en,139,keep,1,1
+R0562,es,140,keep,1,2
+R0563,en,79,keep,2,2
+R0564,en,61,keep,2,2
+R0565,en,123,keep,1,1
+R0566,en,69,keep,1,2
+R0567,en,31,keep,1,1
+R0568,es,115,keep,1,1
+R0569,es,11,keep,1,1
+R0570,en,122,keep,1,2
+R0571,en,22,keep,1,2
+R0572,en,66,keep,2,1
+R0573,en,122,keep,1,1
+R0574,es,67,keep,1,2
+R0575,en,62,keep,2,2
+R0576,es,41,keep,1,2
+R0577,en,135,keep,2,1
+R0578,es,127,keep,1,2
+R0579,es,39,hide,3,3
+R0580,en,48,keep,1,1
+R0581,en,66,keep,1,2
+R0582,en,47,keep,1,1
+R0583,en,67,hide,4,5
+R0584,es,73,keep,1,2
+R0585,en,12,keep,1,1
+R0586,en,10,keep,1,2
+R0587,en,73,keep,1,1
+R0588,es,54,keep,1,3
+R0589,es,136,keep,2,1
+R0590,es,34,keep,2,1
+R0591,en,105,keep,2,1
+R0592,es,118,keep,2,1
+R0593,en,86,hide,5,5
+R0594,en,42,keep,2,1
+R0595,en,45,keep,2,1
+R0596,en,75,keep,1,2
+R0597,en,9,keep,1,1
+R0598,en,90,keep,1,2
+R0599,es,74,keep,1,2
+R0600,es,33,keep,2,1

package/packs/ai-engineering/skills/judge-audit/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "judge-audit",
-  "version": "0.3.1",
+  "version": "0.4.0",
   "description": "Audit whether an LLM-as-judge, or a panel of judges, can be trusted for the decision its scores drive, before anyone acts on them — above all whether the judge errs differently on the variants being compared (prompt v1 vs v2, model A vs B, short vs long inputs), plus chance-corrected and weighted agreement with expert labels, precision and recall on a rare class, each run reported separately with the items that flipped, position and length bias, and confidence intervals, checked against the user's own gate or criterion and ending in a direct answer (ship or not, which judge to use). Use when the user asks \"can we trust our LLM judge\", \"is our grader reliable enough to gate releases\", \"the grader says v2 is better, can we ship it\", \"which is the cheapest judge we can switch to\", \"our two judges agree 90%, is that good\", \"can we turn on auto-remove\", \"check our judge for position bias\", or has a file of judge scores and wants to know whether to act on them.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -12,14 +12,13 @@
     "optional": []
   },
   "related": [
-    "ai-usage-report",
     "eval-harness-generator",
     "llm-eval-harness"
   ],
   "handoff": [
     {
       "to": "llm-eval-harness",
-      "when": "the verdict is DO NOT RELY ON IT and the judge, its rubric or its labelled set has to be rebuilt",
+      "when": "the verdict is don't rely on it and the judge, its rubric or its labelled set has to be rebuilt",
       "purpose": "Design the labelled set, the split criteria and the pass bar the rebuilt judge will be validated against.",
       "fallback": "If llm-eval-harness is absent, list inline the criteria to split the rubric into, the number of expert-labelled items to collect (including failures), and the bar each must clear for the decision."
     }

package/packs/ai-engineering/skills/llm-eval-harness/SKILL.md

@@ -5,7 +5,7 @@ description: Design an evaluation harness for an LLM-powered feature — a versi
 
 # llm-eval-harness
 
-Builds the measurement that turns "the new prompt feels better" into "the new prompt scores 0.91 vs 0.84 baseline, with zero regressions on the critical subset." Without it, every prompt or model change is a vibe with a deploy button. Provider-neutral by design — the harness shape is the same whether the feature runs on Claude, GPT, Gemini, or a local model; only the runner differs. Distinct from `ai-usage-report` (telemetry after the fact) and `success-metrics` (product outcomes) — this is the **dev-loop quality gate**.
+Builds the measurement that turns "the new prompt feels better" into "the new prompt scores 0.91 vs 0.84 baseline, with zero regressions on the critical subset." Without it, every prompt or model change is a vibe with a deploy button. Provider-neutral by design — the harness shape is the same whether the feature runs on Claude, GPT, Gemini, or a local model; only the runner differs. Distinct from `success-metrics` (product outcomes) — this is the **dev-loop quality gate**.
 
 ## How to respond
 
@@ -64,7 +64,6 @@ Builds the measurement that turns "the new prompt feels better" into "the new pr
 
 ## When NOT to use this skill
 
-- ❌ Tabulating AI usage/adoption telemetry — that's `ai-usage-report`
 - ❌ Product-outcome metrics for a feature (adoption, handle time) — that's `success-metrics`
 - ❌ Picking which model/vendor to buy on price, terms or support — this measures quality on your task, which is one input to that decision
 - ❌ The provider-specific API mechanics (token counting, streaming, tool-call format) — consult the provider's own reference; this skill is the eval methodology

package/packs/ai-engineering/skills/llm-eval-harness/manifest.json

@@ -1,11 +1,11 @@
 {
   "name": "llm-eval-harness",
-  "version": "0.1.2",
+  "version": "0.1.4",
   "description": "Design an evaluation harness for an LLM-powered feature — a versioned golden set (representative + adversarial + regression cases), the cheapest adequate grading method per case, a metric with a pre-set pass bar and regression gate, and a failure taxonomy that targets iteration. Use when the user is building or tuning an LLM feature (prompt, RAG, agent, classifier) and needs evals, a way to test prompt/model changes, or to stop shipping quality regressions on vibes.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
   "env": { "required": [], "optional": [] },
-  "related": ["ai-usage-report", "success-metrics"],
+  "related": ["success-metrics"],
   "tags": ["llm", "evals", "ai-engineering", "testing", "llm-as-judge", "regression-gate"],
-  "model": { "tier": "standard", "rationale": "Methodology synthesis against a fixed decision tree and pitfall catalog. Grading-method and metric choices need judgment but are bounded; escalate via ambiguous-input on novel task types." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 71% and 71% of checks in two tier trials, against standard's 71% and heavy's 71% and 71%, within 5 points of the best both times." }
 }

package/packs/ai-engineering/skills/mcp-pairing/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-pairing",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Recommend which MCP servers to connect for a loop, workflow or task, stage by stage, with the narrowest access that works — what outside context or action each stage needs, the server that provides it (only ones the user named or the vendor officially publishes), the scopes to grant (read-only by default, never write on a review or gate stage), what it must not be able to do, the export-file fallback that keeps the stage running without it, and which stages need no server at all. Use when the user asks \"which MCP servers should we hook up for this loop\", \"what should our agent connect to\", \"what token scopes does this stage need\", \"is this MCP setup too broad\", or wants to wire GitHub, GitLab, Jira, Linear, Slack, Sentry, Grafana or a database into an agent workflow.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },

package/packs/ai-engineering/skills/mcp-pairing/reference.md

@@ -124,13 +124,14 @@ From each loop's `loop.json` (type in brackets). Use these when the user names t
 |---|---|
 | `build` | shape (generate) → implement (generate) → verify (verify, G2 mechanical) → decide (gate, human) |
 | `release` | plan (generate) → ready (verify) → announce (generate) → go (gate, human) |
-| `operate` | instrument (generate) → document (generate) → respond (generate) → learn (verify) |
-| `discover` | gather (generate) → structure (generate) → specify (generate) → ratify (gate) |
 | `design` | constrain (generate) → shape (generate) → threat (verify) → ratify (gate) → record (generate) |
 | `measure` | spec (generate) → review (verify) → sign-off (gate) |
 | `comply` | register → assess → evidence (generate) → check (verify) → sign-off (gate) |
 | `research` | plan (generate) → synthesise (generate) → compare (verify) |
 | `ship-a-draft` | intake (generate) → draft (generate) → critique (verify) → polish (verify) |
-| `apply` | fit (gate) → tailor → letter → form (generate) → confirm (verify) |
+| `modernize` | survey → plan → capture → rebuild (generate) → prove (verify, G10 mechanical) → cutover (gate, human) |
+| `ship-agent` | design → budget (generate) → threat (verify) → evals (generate) → launch (gate, human) |
+| `provision` | module (generate) → plan (verify) → apply (gate, human) |
+| `clarify` | page → strings (generate) → blueprint (verify) |
 
 If the user's copy of a loop has different stages, theirs wins.

package/packs/ai-engineering/skills/subagent-design/SKILL.md

@@ -39,6 +39,7 @@ Produces an **orchestration plan**: which subagents exist, what each one alone i
 ## Useful references in this skill
 
 - [`templates/orchestration-plan.md`](templates/orchestration-plan.md) — role cards, topology diagram, aggregation/verification, budget line
+- [`examples/help-center-localization.md`](examples/help-center-localization.md) — worked example: a translator grading its own work, a middle layer with no job and auto-publishing, cut to a pipeline with one justified check
 
 ## Quality bar