skilldrop-cli 1.4.0 → 1.6.0

package/packs/ai-engineering/skills/judge-audit/evals/files/13/moderation_eval.csv

@@ -0,0 +1,601 @@
+review_id,language,word_count,expert_label,judge_L_score,judge_S_score
+R0001,es,7,keep,3,2
+R0002,es,68,keep,1,1
+R0003,en,116,hide,4,4
+R0004,en,76,keep,2,1
+R0005,en,58,keep,1,1
+R0006,es,33,keep,1,1
+R0007,en,57,keep,3,1
+R0008,en,85,keep,2,2
+R0009,es,110,keep,2,1
+R0010,en,137,keep,1,1
+R0011,en,27,keep,2,1
+R0012,es,42,keep,1,2
+R0013,en,81,keep,2,1
+R0014,en,52,keep,1,1
+R0015,en,18,keep,2,2
+R0016,en,10,keep,1,1
+R0017,en,138,keep,2,2
+R0018,en,139,keep,2,1
+R0019,en,68,keep,1,2
+R0020,es,49,keep,1,3
+R0021,en,69,keep,1,1
+R0022,es,95,keep,1,2
+R0023,en,98,keep,1,1
+R0024,en,51,keep,1,1
+R0025,es,134,keep,2,1
+R0026,es,102,keep,1,2
+R0027,es,40,keep,2,2
+R0028,es,133,keep,1,2
+R0029,es,38,keep,2,2
+R0030,en,105,keep,1,1
+R0031,en,88,keep,1,1
+R0032,en,129,hide,5,5
+R0033,en,24,keep,1,2
+R0034,en,83,keep,1,1
+R0035,en,94,keep,1,2
+R0036,en,31,keep,2,2
+R0037,en,34,keep,1,2
+R0038,es,63,keep,1,1
+R0039,en,124,keep,1,1
+R0040,en,81,keep,2,1
+R0041,en,133,keep,3,1
+R0042,en,8,keep,1,1
+R0043,en,53,hide,4,5
+R0044,es,40,keep,1,2
+R0045,en,96,keep,2,1
+R0046,en,19,keep,2,2
+R0047,en,106,keep,2,1
+R0048,en,58,keep,1,2
+R0049,es,75,keep,1,1
+R0050,en,77,keep,2,1
+R0051,en,114,keep,1,1
+R0052,es,115,keep,1,1
+R0053,en,50,keep,1,2
+R0054,en,129,hide,5,4
+R0055,es,36,keep,2,3
+R0056,en,95,keep,1,1
+R0057,en,138,keep,1,2
+R0058,en,61,hide,4,4
+R0059,es,114,keep,1,1
+R0060,es,46,keep,1,1
+R0061,en,11,keep,1,2
+R0062,es,50,keep,2,2
+R0063,es,23,keep,1,1
+R0064,es,46,keep,3,1
+R0065,en,66,hide,5,2
+R0066,en,87,keep,2,1
+R0067,en,67,keep,2,1
+R0068,en,136,keep,1,2
+R0069,en,78,keep,1,1
+R0070,es,111,keep,2,1
+R0071,es,41,keep,3,1
+R0072,es,12,keep,1,1
+R0073,en,37,keep,2,1
+R0074,en,71,hide,4,5
+R0075,en,24,keep,1,3
+R0076,es,125,keep,1,1
+R0077,es,128,keep,1,2
+R0078,es,82,keep,1,1
+R0079,en,115,keep,1,1
+R0080,en,87,keep,1,1
+R0081,en,89,hide,4,5
+R0082,es,26,hide,5,5
+R0083,en,15,hide,4,5
+R0084,es,34,keep,1,1
+R0085,es,45,keep,3,1
+R0086,es,92,keep,1,2
+R0087,en,87,keep,2,2
+R0088,en,19,keep,1,1
+R0089,en,120,keep,2,2
+R0090,es,80,keep,2,2
+R0091,es,124,keep,2,1
+R0092,en,37,hide,4,4
+R0093,en,102,keep,2,1
+R0094,es,118,keep,1,2
+R0095,es,119,keep,2,1
+R0096,es,48,keep,2,1
+R0097,en,139,keep,1,2
+R0098,en,17,keep,2,1
+R0099,es,110,keep,2,1
+R0100,en,116,keep,1,1
+R0101,en,39,hide,5,5
+R0102,en,86,keep,2,1
+R0103,en,128,keep,2,1
+R0104,en,24,keep,1,1
+R0105,en,116,keep,1,1
+R0106,en,13,keep,2,1
+R0107,es,131,keep,2,1
+R0108,en,7,keep,1,1
+R0109,es,25,keep,2,1
+R0110,en,114,keep,1,1
+R0111,en,110,keep,2,1
+R0112,es,137,keep,2,2
+R0113,en,15,hide,5,5
+R0114,es,79,keep,2,1
+R0115,en,128,keep,2,1
+R0116,en,23,keep,1,1
+R0117,en,133,keep,3,2
+R0118,en,42,keep,1,1
+R0119,en,54,keep,1,1
+R0120,en,121,keep,3,1
+R0121,en,49,keep,1,1
+R0122,es,56,keep,1,1
+R0123,es,115,hide,5,3
+R0124,es,20,hide,5,4
+R0125,es,59,keep,1,1
+R0126,es,11,keep,1,1
+R0127,en,96,hide,5,4
+R0128,en,32,keep,1,1
+R0129,es,14,keep,1,2
+R0130,es,131,keep,1,1
+R0131,en,14,keep,1,1
+R0132,es,40,keep,2,2
+R0133,en,88,keep,1,1
+R0134,en,66,keep,2,1
+R0135,en,57,keep,2,1
+R0136,en,58,keep,1,1
+R0137,es,14,keep,1,1
+R0138,en,76,hide,4,5
+R0139,es,117,keep,1,1
+R0140,en,50,keep,2,1
+R0141,es,36,keep,2,2
+R0142,es,38,hide,4,4
+R0143,es,114,hide,4,2
+R0144,en,137,keep,2,1
+R0145,en,124,keep,1,2
+R0146,es,23,keep,1,1
+R0147,en,54,keep,1,1
+R0148,en,59,keep,1,1
+R0149,en,108,keep,1,2
+R0150,es,15,keep,1,2
+R0151,en,117,keep,2,2
+R0152,en,27,keep,1,1
+R0153,en,93,keep,1,1
+R0154,en,60,keep,1,4
+R0155,en,109,keep,3,1
+R0156,en,27,keep,1,1
+R0157,es,63,hide,4,4
+R0158,es,31,keep,2,2
+R0159,en,8,keep,1,1
+R0160,es,120,keep,1,2
+R0161,en,116,keep,1,1
+R0162,es,130,keep,2,1
+R0163,en,111,keep,1,1
+R0164,en,119,keep,1,1
+R0165,en,18,keep,2,1
+R0166,es,15,keep,2,2
+R0167,en,52,keep,1,2
+R0168,en,134,keep,3,2
+R0169,en,53,keep,1,1
+R0170,en,97,keep,1,2
+R0171,en,112,keep,1,1
+R0172,en,45,keep,1,2
+R0173,en,22,keep,2,1
+R0174,es,112,keep,2,4
+R0175,es,115,hide,3,3
+R0176,en,32,keep,2,1
+R0177,en,76,keep,1,2
+R0178,es,67,keep,2,1
+R0179,es,121,keep,1,2
+R0180,en,51,keep,1,2
+R0181,es,46,keep,1,3
+R0182,en,24,keep,1,2
+R0183,en,37,keep,2,2
+R0184,es,31,keep,1,3
+R0185,es,107,keep,1,1
+R0186,en,23,keep,1,1
+R0187,es,33,keep,1,1
+R0188,en,27,keep,2,1
+R0189,en,59,keep,2,1
+R0190,en,38,keep,1,1
+R0191,en,51,keep,2,2
+R0192,es,118,keep,1,1
+R0193,en,68,keep,2,1
+R0194,en,38,keep,1,2
+R0195,en,88,keep,1,1
+R0196,en,74,keep,1,2
+R0197,en,6,keep,1,1
+R0198,es,109,keep,1,1
+R0199,en,80,keep,1,1
+R0200,en,41,keep,1,1
+R0201,es,106,keep,2,1
+R0202,en,7,keep,2,2
+R0203,es,117,keep,1,2
+R0204,en,135,keep,1,1
+R0205,en,25,keep,2,1
+R0206,es,20,keep,2,2
+R0207,es,69,keep,1,1
+R0208,es,38,hide,4,5
+R0209,es,98,keep,2,2
+R0210,es,38,keep,2,2
+R0211,es,17,keep,2,1
+R0212,en,65,keep,1,1
+R0213,en,96,keep,1,1
+R0214,es,91,keep,1,3
+R0215,es,10,keep,1,2
+R0216,en,89,keep,1,1
+R0217,es,21,keep,2,1
+R0218,en,86,keep,1,1
+R0219,en,53,keep,2,1
+R0220,en,57,hide,4,5
+R0221,en,69,keep,1,2
+R0222,es,51,keep,2,1
+R0223,en,52,keep,1,2
+R0224,en,24,keep,1,1
+R0225,en,82,keep,2,1
+R0226,en,119,keep,1,1
+R0227,en,107,keep,1,1
+R0228,en,65,keep,1,2
+R0229,en,121,keep,2,1
+R0230,en,108,keep,2,1
+R0231,en,49,keep,1,2
+R0232,es,83,keep,1,1
+R0233,en,35,keep,1,1
+R0234,es,68,keep,2,2
+R0235,en,26,keep,1,1
+R0236,en,45,keep,1,1
+R0237,en,43,keep,1,2
+R0238,en,72,keep,1,1
+R0239,es,46,keep,1,2
+R0240,es,98,keep,2,1
+R0241,en,80,keep,2,1
+R0242,en,18,keep,1,1
+R0243,es,112,keep,1,2
+R0244,en,101,keep,1,1
+R0245,en,82,keep,2,2
+R0246,en,81,keep,3,1
+R0247,en,89,hide,5,4
+R0248,es,75,hide,5,4
+R0249,es,75,keep,1,1
+R0250,en,10,keep,1,1
+R0251,en,60,keep,1,1
+R0252,en,63,keep,2,1
+R0253,en,94,keep,2,1
+R0254,es,140,keep,1,1
+R0255,es,86,keep,1,1
+R0256,es,80,keep,2,1
+R0257,es,16,keep,1,1
+R0258,en,7,keep,2,1
+R0259,en,103,keep,2,2
+R0260,es,118,keep,2,2
+R0261,es,109,keep,2,1
+R0262,en,58,keep,2,2
+R0263,en,37,keep,2,2
+R0264,en,124,keep,1,1
+R0265,en,66,keep,2,2
+R0266,en,111,keep,2,2
+R0267,en,26,keep,1,2
+R0268,en,61,keep,2,2
+R0269,es,117,keep,1,1
+R0270,es,28,keep,1,1
+R0271,es,20,keep,1,1
+R0272,en,139,keep,1,1
+R0273,en,117,keep,1,1
+R0274,en,44,keep,1,1
+R0275,en,15,keep,2,2
+R0276,en,121,keep,2,2
+R0277,es,77,keep,2,2
+R0278,en,101,keep,1,2
+R0279,es,123,keep,2,3
+R0280,en,129,keep,1,1
+R0281,es,122,keep,2,1
+R0282,es,44,keep,1,1
+R0283,es,140,keep,2,1
+R0284,en,10,keep,2,1
+R0285,es,17,keep,1,1
+R0286,en,48,keep,2,1
+R0287,en,59,keep,1,3
+R0288,es,37,hide,5,5
+R0289,en,99,keep,1,1
+R0290,es,65,hide,4,5
+R0291,es,38,hide,4,5
+R0292,es,138,keep,1,1
+R0293,en,96,keep,1,2
+R0294,es,128,keep,1,1
+R0295,en,133,keep,3,2
+R0296,en,8,keep,1,1
+R0297,en,41,keep,2,1
+R0298,en,34,keep,2,3
+R0299,es,83,keep,2,2
+R0300,en,37,keep,1,1
+R0301,es,63,keep,1,1
+R0302,en,126,keep,1,1
+R0303,en,101,keep,3,2
+R0304,es,59,keep,1,1
+R0305,es,12,keep,1,1
+R0306,en,22,keep,2,1
+R0307,en,44,keep,1,2
+R0308,en,124,keep,1,1
+R0309,en,15,keep,1,1
+R0310,en,130,keep,3,2
+R0311,en,139,keep,1,1
+R0312,en,84,keep,2,1
+R0313,es,138,keep,1,1
+R0314,en,127,keep,2,1
+R0315,en,49,keep,2,2
+R0316,en,18,keep,1,3
+R0317,en,58,hide,5,5
+R0318,en,111,hide,5,4
+R0319,en,11,keep,1,1
+R0320,en,44,keep,1,1
+R0321,en,110,keep,1,1
+R0322,en,8,keep,1,2
+R0323,es,110,keep,1,1
+R0324,es,56,keep,1,1
+R0325,en,39,keep,1,1
+R0326,es,45,keep,1,4
+R0327,es,27,keep,1,2
+R0328,es,123,keep,1,3
+R0329,en,129,keep,1,1
+R0330,es,128,keep,1,3
+R0331,es,60,keep,1,2
+R0332,en,115,keep,1,2
+R0333,es,59,keep,2,2
+R0334,en,124,keep,2,2
+R0335,es,76,hide,5,5
+R0336,en,115,keep,1,2
+R0337,es,34,keep,1,2
+R0338,en,81,keep,1,1
+R0339,en,33,keep,2,1
+R0340,en,37,keep,2,1
+R0341,es,132,keep,1,1
+R0342,en,49,keep,2,1
+R0343,en,36,keep,2,1
+R0344,es,78,keep,2,2
+R0345,en,51,keep,2,1
+R0346,es,13,keep,1,3
+R0347,en,100,keep,1,2
+R0348,en,17,keep,2,2
+R0349,es,50,keep,1,1
+R0350,en,40,keep,1,1
+R0351,en,31,keep,2,1
+R0352,en,41,keep,1,1
+R0353,en,83,keep,2,1
+R0354,en,17,keep,1,1
+R0355,es,110,keep,2,1
+R0356,es,117,keep,1,1
+R0357,en,102,keep,1,1
+R0358,en,116,keep,2,1
+R0359,en,107,keep,1,1
+R0360,en,73,keep,1,1
+R0361,en,98,keep,2,1
+R0362,en,21,keep,1,2
+R0363,en,33,keep,1,2
+R0364,en,18,keep,1,3
+R0365,en,101,keep,1,1
+R0366,en,87,keep,1,1
+R0367,es,104,keep,1,1
+R0368,es,122,keep,1,2
+R0369,en,24,keep,2,1
+R0370,es,68,keep,2,1
+R0371,es,41,hide,4,5
+R0372,en,92,keep,2,1
+R0373,en,98,keep,1,2
+R0374,en,35,keep,1,1
+R0375,es,48,keep,1,1
+R0376,es,102,keep,1,1
+R0377,en,101,keep,1,1
+R0378,es,114,keep,1,1
+R0379,en,103,hide,5,5
+R0380,es,23,keep,2,1
+R0381,en,97,keep,2,2
+R0382,en,40,keep,1,3
+R0383,es,117,hide,4,4
+R0384,en,30,keep,2,1
+R0385,en,134,keep,1,2
+R0386,es,65,keep,1,1
+R0387,en,24,keep,2,1
+R0388,en,11,keep,1,1
+R0389,en,49,keep,2,1
+R0390,es,80,keep,2,1
+R0391,en,60,keep,2,2
+R0392,en,6,keep,1,1
+R0393,en,68,keep,2,1
+R0394,en,85,keep,1,1
+R0395,en,101,hide,5,3
+R0396,en,70,keep,1,1
+R0397,en,103,hide,5,4
+R0398,en,71,keep,2,2
+R0399,en,111,keep,1,1
+R0400,en,92,hide,5,4
+R0401,es,48,hide,4,2
+R0402,en,118,keep,1,1
+R0403,en,69,keep,2,1
+R0404,es,32,keep,1,1
+R0405,en,62,keep,1,2
+R0406,es,89,keep,1,1
+R0407,es,65,keep,1,2
+R0408,en,128,hide,4,4
+R0409,en,106,hide,4,4
+R0410,es,72,keep,1,2
+R0411,es,75,keep,1,4
+R0412,en,50,hide,5,5
+R0413,en,103,keep,3,3
+R0414,en,92,keep,1,3
+R0415,en,22,keep,1,1
+R0416,en,111,keep,1,1
+R0417,en,21,keep,2,1
+R0418,en,42,keep,1,1
+R0419,en,13,hide,5,5
+R0420,en,57,keep,1,1
+R0421,en,43,keep,1,1
+R0422,en,100,keep,2,1
+R0423,en,53,keep,1,3
+R0424,en,83,keep,1,1
+R0425,es,135,keep,1,2
+R0426,en,102,hide,4,5
+R0427,es,111,keep,2,1
+R0428,es,68,hide,4,3
+R0429,es,112,keep,1,2
+R0430,en,139,keep,1,1
+R0431,en,82,keep,1,2
+R0432,en,52,keep,1,2
+R0433,en,128,keep,2,3
+R0434,en,66,keep,2,3
+R0435,es,26,keep,3,2
+R0436,en,69,keep,1,1
+R0437,en,133,hide,5,5
+R0438,en,85,keep,1,1
+R0439,es,12,keep,2,1
+R0440,en,43,keep,2,1
+R0441,en,26,keep,1,1
+R0442,en,133,keep,1,1
+R0443,es,106,keep,3,1
+R0444,en,9,keep,1,1
+R0445,es,98,keep,1,1
+R0446,en,57,keep,1,2
+R0447,en,136,keep,1,2
+R0448,en,37,hide,4,5
+R0449,en,111,keep,1,1
+R0450,en,94,keep,1,1
+R0451,en,61,keep,1,1
+R0452,en,63,keep,1,2
+R0453,en,69,keep,1,2
+R0454,en,64,keep,2,2
+R0455,es,117,keep,1,3
+R0456,es,122,keep,1,2
+R0457,en,104,keep,1,2
+R0458,es,120,keep,2,1
+R0459,en,73,keep,2,2
+R0460,en,68,keep,2,1
+R0461,en,94,keep,2,1
+R0462,en,85,keep,1,1
+R0463,en,31,keep,2,1
+R0464,en,102,keep,1,2
+R0465,es,50,keep,1,1
+R0466,en,16,keep,2,1
+R0467,en,138,keep,2,2
+R0468,es,45,keep,1,2
+R0469,en,44,hide,4,5
+R0470,en,54,keep,2,1
+R0471,en,17,keep,2,1
+R0472,es,88,keep,2,1
+R0473,en,107,hide,3,4
+R0474,es,96,keep,2,1
+R0475,en,137,keep,2,3
+R0476,en,41,keep,1,1
+R0477,en,83,hide,5,4
+R0478,en,17,keep,2,1
+R0479,en,88,keep,2,3
+R0480,en,69,keep,1,2
+R0481,es,106,keep,1,3
+R0482,es,123,keep,1,1
+R0483,en,97,keep,1,2
+R0484,en,22,keep,1,1
+R0485,en,31,keep,2,1
+R0486,en,113,hide,4,5
+R0487,en,107,hide,4,5
+R0488,es,54,keep,2,1
+R0489,en,139,keep,1,1
+R0490,en,9,keep,1,1
+R0491,en,116,keep,1,1
+R0492,es,19,hide,5,4
+R0493,en,62,keep,2,2
+R0494,en,19,keep,2,2
+R0495,en,111,keep,1,1
+R0496,en,22,keep,2,1
+R0497,en,110,keep,1,2
+R0498,en,58,keep,2,1
+R0499,en,110,hide,5,4
+R0500,es,80,keep,1,1
+R0501,es,34,keep,2,5
+R0502,en,81,hide,4,5
+R0503,en,10,keep,1,1
+R0504,en,131,keep,1,1
+R0505,en,29,keep,1,2
+R0506,en,78,keep,1,1
+R0507,en,135,keep,3,2
+R0508,es,50,keep,1,1
+R0509,en,58,keep,1,2
+R0510,es,127,keep,2,1
+R0511,en,11,keep,2,1
+R0512,en,78,keep,1,1
+R0513,es,43,keep,3,1
+R0514,es,11,keep,3,1
+R0515,es,113,keep,2,2
+R0516,en,117,keep,2,1
+R0517,en,38,keep,1,1
+R0518,en,129,keep,2,2
+R0519,en,85,keep,1,1
+R0520,en,6,keep,1,2
+R0521,en,107,keep,1,1
+R0522,es,32,keep,1,3
+R0523,en,99,keep,1,1
+R0524,en,61,hide,5,5
+R0525,en,19,keep,1,2
+R0526,en,119,keep,2,1
+R0527,es,25,keep,1,1
+R0528,en,18,keep,1,1
+R0529,en,74,keep,1,1
+R0530,en,50,keep,1,1
+R0531,es,30,keep,1,4
+R0532,en,126,keep,1,2
+R0533,es,84,keep,3,4
+R0534,en,109,keep,2,1
+R0535,es,87,keep,2,1
+R0536,en,89,keep,1,2
+R0537,en,81,keep,4,1
+R0538,es,93,keep,3,1
+R0539,en,85,keep,1,1
+R0540,en,9,keep,1,2
+R0541,en,98,keep,2,1
+R0542,en,95,keep,1,1
+R0543,en,137,keep,2,1
+R0544,es,21,keep,1,1
+R0545,es,133,keep,1,1
+R0546,en,58,keep,2,1
+R0547,en,27,keep,2,2
+R0548,en,20,keep,1,3
+R0549,en,51,keep,1,1
+R0550,en,94,hide,4,5
+R0551,es,105,keep,1,1
+R0552,en,53,keep,1,2
+R0553,en,93,keep,1,2
+R0554,en,33,keep,1,2
+R0555,es,69,hide,5,5
+R0556,en,37,keep,1,1
+R0557,en,55,keep,1,2
+R0558,en,12,keep,1,1
+R0559,es,70,keep,1,1
+R0560,en,67,keep,2,1
+R0561,en,139,keep,1,1
+R0562,es,140,keep,1,2
+R0563,en,79,keep,2,2
+R0564,en,61,keep,2,2
+R0565,en,123,keep,1,1
+R0566,en,69,keep,1,2
+R0567,en,31,keep,1,1
+R0568,es,115,keep,1,1
+R0569,es,11,keep,1,1
+R0570,en,122,keep,1,2
+R0571,en,22,keep,1,2
+R0572,en,66,keep,2,1
+R0573,en,122,keep,1,1
+R0574,es,67,keep,1,2
+R0575,en,62,keep,2,2
+R0576,es,41,keep,1,2
+R0577,en,135,keep,2,1
+R0578,es,127,keep,1,2
+R0579,es,39,hide,3,3
+R0580,en,48,keep,1,1
+R0581,en,66,keep,1,2
+R0582,en,47,keep,1,1
+R0583,en,67,hide,4,5
+R0584,es,73,keep,1,2
+R0585,en,12,keep,1,1
+R0586,en,10,keep,1,2
+R0587,en,73,keep,1,1
+R0588,es,54,keep,1,3
+R0589,es,136,keep,2,1
+R0590,es,34,keep,2,1
+R0591,en,105,keep,2,1
+R0592,es,118,keep,2,1
+R0593,en,86,hide,5,5
+R0594,en,42,keep,2,1
+R0595,en,45,keep,2,1
+R0596,en,75,keep,1,2
+R0597,en,9,keep,1,1
+R0598,en,90,keep,1,2
+R0599,es,74,keep,1,2
+R0600,es,33,keep,2,1

package/packs/ai-engineering/skills/judge-audit/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "judge-audit",
-  "version": "0.3.2",
+  "version": "0.4.0",
   "description": "Audit whether an LLM-as-judge, or a panel of judges, can be trusted for the decision its scores drive, before anyone acts on them — above all whether the judge errs differently on the variants being compared (prompt v1 vs v2, model A vs B, short vs long inputs), plus chance-corrected and weighted agreement with expert labels, precision and recall on a rare class, each run reported separately with the items that flipped, position and length bias, and confidence intervals, checked against the user's own gate or criterion and ending in a direct answer (ship or not, which judge to use). Use when the user asks \"can we trust our LLM judge\", \"is our grader reliable enough to gate releases\", \"the grader says v2 is better, can we ship it\", \"which is the cheapest judge we can switch to\", \"our two judges agree 90%, is that good\", \"can we turn on auto-remove\", \"check our judge for position bias\", or has a file of judge scores and wants to know whether to act on them.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -18,7 +18,7 @@
   "handoff": [
     {
       "to": "llm-eval-harness",
-      "when": "the verdict is DO NOT RELY ON IT and the judge, its rubric or its labelled set has to be rebuilt",
+      "when": "the verdict is don't rely on it and the judge, its rubric or its labelled set has to be rebuilt",
       "purpose": "Design the labelled set, the split criteria and the pass bar the rebuilt judge will be validated against.",
       "fallback": "If llm-eval-harness is absent, list inline the criteria to split the rubric into, the number of expert-labelled items to collect (including failures), and the bar each must clear for the decision."
     }

package/packs/ai-engineering/skills/mcp-pairing/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-pairing",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Recommend which MCP servers to connect for a loop, workflow or task, stage by stage, with the narrowest access that works — what outside context or action each stage needs, the server that provides it (only ones the user named or the vendor officially publishes), the scopes to grant (read-only by default, never write on a review or gate stage), what it must not be able to do, the export-file fallback that keeps the stage running without it, and which stages need no server at all. Use when the user asks \"which MCP servers should we hook up for this loop\", \"what should our agent connect to\", \"what token scopes does this stage need\", \"is this MCP setup too broad\", or wants to wire GitHub, GitLab, Jira, Linear, Slack, Sentry, Grafana or a database into an agent workflow.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },

package/packs/ai-engineering/skills/mcp-pairing/reference.md

@@ -124,13 +124,14 @@ From each loop's `loop.json` (type in brackets). Use these when the user names t
 |---|---|
 | `build` | shape (generate) → implement (generate) → verify (verify, G2 mechanical) → decide (gate, human) |
 | `release` | plan (generate) → ready (verify) → announce (generate) → go (gate, human) |
-| `operate` | instrument (generate) → document (generate) → respond (generate) → learn (verify) |
-| `discover` | gather (generate) → structure (generate) → specify (generate) → ratify (gate) |
 | `design` | constrain (generate) → shape (generate) → threat (verify) → ratify (gate) → record (generate) |
 | `measure` | spec (generate) → review (verify) → sign-off (gate) |
 | `comply` | register → assess → evidence (generate) → check (verify) → sign-off (gate) |
 | `research` | plan (generate) → synthesise (generate) → compare (verify) |
 | `ship-a-draft` | intake (generate) → draft (generate) → critique (verify) → polish (verify) |
-| `apply` | fit (gate) → tailor → letter → form (generate) → confirm (verify) |
+| `modernize` | survey → plan → capture → rebuild (generate) → prove (verify, G10 mechanical) → cutover (gate, human) |
+| `ship-agent` | design → budget (generate) → threat (verify) → evals (generate) → launch (gate, human) |
+| `provision` | module (generate) → plan (verify) → apply (gate, human) |
+| `clarify` | page → strings (generate) → blueprint (verify) |
 
 If the user's copy of a loop has different stages, theirs wins.

package/packs/ai-engineering/skills/subagent-design/SKILL.md

@@ -39,6 +39,7 @@ Produces an **orchestration plan**: which subagents exist, what each one alone i
 ## Useful references in this skill
 
 - [`templates/orchestration-plan.md`](templates/orchestration-plan.md) — role cards, topology diagram, aggregation/verification, budget line
+- [`examples/help-center-localization.md`](examples/help-center-localization.md) — worked example: a translator grading its own work, a middle layer with no job and auto-publishing, cut to a pipeline with one justified check
 
 ## Quality bar