promptfoo 0.120.25 → 0.120.27
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -22
- package/dist/drizzle/0023_wooden_mandrill.sql +2 -0
- package/dist/drizzle/meta/0023_snapshot.json +1496 -0
- package/dist/drizzle/meta/_journal.json +7 -0
- package/dist/src/{ListApp-CYTAaS6A.js → ListApp-8WOe2nT6.js} +1 -1
- package/dist/src/{accounts-BSwMxhY3.js → accounts-CPDRAMND.js} +28 -8
- package/dist/src/{accounts-Bzx8GnL6.js → accounts-DVINui-2.js} +29 -9
- package/dist/src/{accounts-CwqY22lU.cjs → accounts-Fl2J3_Fu.cjs} +28 -8
- package/dist/src/{agentic-utils-5K7LzooV.js → agentic-utils-D922n6mm.js} +3 -3
- package/dist/src/{agents-KZxXwlyn.cjs → agents-BO2n8Z0d.cjs} +5 -5
- package/dist/src/{agents-Cnph5GLD.js → agents-BXLmVsxR.js} +2 -2
- package/dist/src/{agents-v4cW_ZgC.js → agents-BcsN_BgB.js} +5 -12
- package/dist/src/{agents-CkA9l77-.js → agents-BdUTAwi-.js} +6 -6
- package/dist/src/{agents-zHyWnnQ2.js → agents-DNvSH78i.js} +11 -18
- package/dist/src/{agents-B8RWrA68.cjs → agents-DgJf2-ez.cjs} +10 -17
- package/dist/src/{agents-B20gl7hH.js → agents-hqgSV-3o.js} +5 -5
- package/dist/src/{agents-DleC7rb4.js → agents-pMfppv9Z.js} +10 -17
- package/dist/src/{aimlapi-DIdWBHtH.cjs → aimlapi-BE_Tg9Fl.cjs} +14 -14
- package/dist/src/{aimlapi-DILJovhN.js → aimlapi-DOib86oE.js} +15 -15
- package/dist/src/{aimlapi-DtSf1ykJ.js → aimlapi-DTPACCB1.js} +2 -2
- package/dist/src/{aimlapi-DgbFGPTX.js → aimlapi-DtgPI0nE.js} +13 -13
- package/dist/src/app/assets/index-B2D0bCSI.js +439 -0
- package/dist/src/app/assets/index-NCn4eVBv.css +1 -0
- package/dist/src/app/assets/{vendor-charts-DnwydkkY.js → vendor-charts-CCl15Imd.js} +2 -2
- package/dist/src/app/assets/{vendor-markdown-DjKAU15k.js → vendor-markdown-0tekx3KX.js} +2 -2
- package/dist/src/app/assets/vendor-react-AtKqiNEf.js +4 -0
- package/dist/src/app/assets/{vendor-syntax-Dc86c4Bu.js → vendor-syntax-D06x6TQF.js} +1 -1
- package/dist/src/app/index.html +6 -36
- package/dist/src/app/tsconfig.app.tsbuildinfo +1 -1
- package/dist/src/{audio-BDrAkpVc.js → audio-BRYU0BFo.js} +5 -5
- package/dist/src/{audio-U580w8jM.js → audio-BnRUGAm_.js} +2 -2
- package/dist/src/{audio-BoR9KZV8.cjs → audio-Cwo68yZS.cjs} +4 -4
- package/dist/src/{audio-nUSOpXY7.js → audio-MSRki4JU.js} +4 -4
- package/dist/src/base-XB2tDJrB.js +196 -0
- package/dist/src/base-h961VXYk.js +195 -0
- package/dist/src/base-pGVmXNl4.cjs +224 -0
- package/dist/src/{blobs-_kNRY6vU.cjs → blobs-B-KQAFhX.cjs} +3 -3
- package/dist/src/{blobs-DlB5fwyF.js → blobs-BM_e6hCa.js} +3 -3
- package/dist/src/{blobs-Dnt2rxpc.js → blobs-CR5C4Ihh.js} +3 -3
- package/dist/src/{cache-S9MAoF9H.cjs → cache-BTVYfbka.cjs} +35 -8
- package/dist/src/{cache-C4A4H4Xb.js → cache-CIpsoBZR.js} +24 -9
- package/dist/src/{cache-B2oIvf92.js → cache-ChPcurj7.js} +3 -3
- package/dist/src/{cache-DofebEnH.js → cache-VVu_W-yg.js} +4 -4
- package/dist/src/cache-YLNCFEM2.cjs +6 -0
- package/dist/src/{cache-Bg1Wpvzq.js → cache-jsiwsAJv.js} +5 -22
- package/dist/src/{chat-XPN9YHhr.js → chat-B84t99NW.js} +2 -2
- package/dist/src/{chat-DTCPbuld.cjs → chat-BE44YOc6.cjs} +18 -15
- package/dist/src/{chat-D_cro0wd.js → chat-BcPjZXIp.js} +19 -16
- package/dist/src/{chat-noXtEAh5.js → chat-CcUCysjU.js} +17 -14
- package/dist/src/{chat-BR2x1dG_.cjs → chat-D31K7C4u.cjs} +13 -13
- package/dist/src/{chat-DQ6Z5Cbn.js → chat-DZM2GUHO.js} +14 -14
- package/dist/src/{chat-BiImdLJf.js → chat-DwWifjxi.js} +12 -12
- package/dist/src/{chatkit-TnC1YoVi.js → chatkit-Biqb_wsD.js} +4 -4
- package/dist/src/{chatkit-Dpxrq4eD.js → chatkit-D67HS_0b.js} +2 -2
- package/dist/src/{chatkit-DIrJX8xk.js → chatkit-DAB_qfzI.js} +4 -4
- package/dist/src/{chatkit-e25Ziu17.cjs → chatkit-PGG4ZYIn.cjs} +4 -4
- package/dist/src/{claude-agent-sdk-CBpAZV5h.js → claude-agent-sdk-C-IOTPfo.js} +60 -19
- package/dist/src/{claude-agent-sdk-uHLgAUZj.cjs → claude-agent-sdk-C9SiaQub.cjs} +60 -19
- package/dist/src/{claude-agent-sdk-DyPvrj-1.js → claude-agent-sdk-CiluSyW1.js} +52 -11
- package/dist/src/{claude-agent-sdk-qxE5mGVi.js → claude-agent-sdk-SVM6AdBu.js} +61 -20
- package/dist/src/cloud-BndfXy4H.js +5 -0
- package/dist/src/{cloud-lRosK6Nk.js → cloud-CZ-q9Ier.js} +28 -6
- package/dist/src/{cloudflare-ai-BprwSD6d.js → cloudflare-ai-BahKHyhh.js} +13 -13
- package/dist/src/{cloudflare-ai-D0OtFY6g.cjs → cloudflare-ai-Dfahv5SY.cjs} +14 -14
- package/dist/src/{cloudflare-ai-Z9X219gp.js → cloudflare-ai-Dxyt50Nl.js} +2 -2
- package/dist/src/{cloudflare-ai-CB7OWRAP.js → cloudflare-ai-v_qZD6_q.js} +15 -15
- package/dist/src/{cloudflare-gateway-C29gWj5O.cjs → cloudflare-gateway-BPWoZIzJ.cjs} +15 -15
- package/dist/src/{cloudflare-gateway-BjoxwqUw.js → cloudflare-gateway-Bi_FpOFy.js} +14 -14
- package/dist/src/{cloudflare-gateway-Djf3F3_H.js → cloudflare-gateway-C0guUNwk.js} +2 -2
- package/dist/src/{cloudflare-gateway-BaVwoBjE.js → cloudflare-gateway-btS7h1OZ.js} +16 -16
- package/dist/src/{codex-sdk-wTEpMM_X.cjs → codex-sdk-DFKMtAyf.cjs} +25 -7
- package/dist/src/{codex-sdk-ibXwdglL.js → codex-sdk-DSxAnbfT.js} +24 -6
- package/dist/src/{codex-sdk-DNkCbb7c.js → codex-sdk-DulY0ZRq.js} +25 -7
- package/dist/src/{codex-sdk-BASDNkIl.js → codex-sdk-IYVi9fuM.js} +23 -5
- package/dist/src/{cometapi-xiTQZp8R.js → cometapi-C9EEpJzT.js} +14 -14
- package/dist/src/{cometapi-DalITP94.cjs → cometapi-DIO64tf4.cjs} +15 -15
- package/dist/src/{cometapi-BGxGZ7VA.js → cometapi-DkNBMk0G.js} +16 -16
- package/dist/src/{cometapi-DHUAH6nK.js → cometapi-DzrR3SR_.js} +2 -2
- package/dist/src/{completion-BTbouQDz.js → completion-Bgf1VJoq.js} +6 -6
- package/dist/src/{completion-DV0Ys99K.cjs → completion-CCRT4kX1.cjs} +6 -6
- package/dist/src/{completion-HYrX5gEL.js → completion-CG29bfKX.js} +6 -6
- package/dist/src/createHash-CJcfskIZ.js +17 -0
- package/dist/src/createHash-CYQy4YeL.cjs +28 -0
- package/dist/src/createHash-Dw_iLu31.js +16 -0
- package/dist/src/{docker-Dx_FN1ag.js → docker-B81N0t4e.js} +15 -15
- package/dist/src/{docker-BAYnlbAW.js → docker-D-ayp2FW.js} +13 -13
- package/dist/src/{docker-zUAnbYTM.cjs → docker-DNcLR4Ig.cjs} +14 -14
- package/dist/src/{docker-vnOg96gi.js → docker-egERKxCF.js} +2 -2
- package/dist/src/errors-DnGCbnx8.js +36 -0
- package/dist/src/{esm-79BpbQzg.js → esm-B9dPm_BF.js} +2 -2
- package/dist/src/{esm-BQkx5roy.cjs → esm-Ct-Joyue.cjs} +2 -2
- package/dist/src/{esm-rDtG_2rg.js → esm-D2pZ87fL.js} +2 -2
- package/dist/src/eval-BhHvMY82.js +17 -0
- package/dist/src/{eval-Cxb9alW8.js → eval-C-Nr6wX_.js} +48 -15
- package/dist/src/{evalResult-CJNSpa3q.js → evalResult-4BzI2tmj.js} +7 -7
- package/dist/src/{evalResult-hI1mhi_g.cjs → evalResult-CX8wQecI.cjs} +7 -7
- package/dist/src/{evalResult-B8LbgWMA.js → evalResult-DXMWJ3sx.js} +7 -7
- package/dist/src/evalResult-Dq2gFNQY.js +12 -0
- package/dist/src/evalResult-nmcP5VKH.cjs +12 -0
- package/dist/src/evalResult-trqZjVYh.js +14 -0
- package/dist/src/{evaluator-1Tkdu8MK.js → evaluator-8aGyV12L.js} +84 -52
- package/dist/src/evaluator-CnfPstzT.js +39 -0
- package/dist/src/{extractor-BPBlLyOn.cjs → extractor-C031XmTA.cjs} +23 -31
- package/dist/src/{extractor-BFoThUOa.js → extractor-CD5yKL-G.js} +24 -20
- package/dist/src/{extractor-CuUWqwrE.js → extractor-V5x_m1i0.js} +24 -20
- package/dist/src/{fetch-Dy2YML9G.cjs → fetch-BmbD-v1L.cjs} +29 -7
- package/dist/src/{fetch-BuP-gm0E.js → fetch-CXZI9RRr.js} +24 -6
- package/dist/src/{fetch-zJuFPxN1.js → fetch-D3OHf-lV.js} +29 -7
- package/dist/src/fetch-IDPDue6F.cjs +4 -0
- package/dist/src/fetch-hKJ-It8q.js +6 -0
- package/dist/src/{fetch-BdiCecAv.js → fetch-ouKnrWK-.js} +2 -2
- package/dist/src/{fileExtensions-CzcPkiKZ.js → fileExtensions-BpuMmaFL.js} +1 -1
- package/dist/src/{formatDuration-CGa0ZDQ8.js → formatDuration-CdevI3An.js} +1 -1
- package/dist/src/{genaiTracer-foKtbVa2.js → genaiTracer-CqNnnXrE.js} +1 -1
- package/dist/src/{genaiTracer-CQlpZkrp.cjs → genaiTracer-Dres3qrN.cjs} +2 -2
- package/dist/src/{graders-CtvuaC8D.js → graders--1y2u9HO.js} +246 -110
- package/dist/src/graders-CQn7WUsd.cjs +34 -0
- package/dist/src/graders-DC6QAbpW.js +35 -0
- package/dist/src/{graders-DMAzEgEF.js → graders-DTeBrzWp.js} +248 -112
- package/dist/src/graders-DUWz3Y7j.js +37 -0
- package/dist/src/{graders-NBsuYeMx.cjs → graders-DohM2dir.cjs} +247 -111
- package/dist/src/{image-DIn_AW1Z.js → image-B0U4Hqll.js} +5 -5
- package/dist/src/{image-LhLO5KCW.js → image-C3wHC9_h.js} +55 -15
- package/dist/src/{image-CaVWfZub.cjs → image-CuKHuccK.cjs} +60 -14
- package/dist/src/{image-Bf8RiJ2g.js → image-DNEIf_aI.js} +4 -4
- package/dist/src/{image-COCWy5dX.js → image-DmE-niFE.js} +2 -2
- package/dist/src/{image-BnF25Osq.cjs → image-DpKl2F15.cjs} +4 -4
- package/dist/src/{image-DFwp1aAV.js → image-O1u4bCFg.js} +55 -15
- package/dist/src/index.cjs +262 -92
- package/dist/src/index.d.cts +122 -6
- package/dist/src/index.d.ts +122 -6
- package/dist/src/index.js +257 -87
- package/dist/src/{interactiveCheck-Dmm9vjpx.js → interactiveCheck-Bxj1Swex.js} +2 -2
- package/dist/src/{knowledgeBase-DotRBzUE.js → knowledgeBase-BZ41IFwq.js} +14 -27
- package/dist/src/{knowledgeBase-CfOvyF2s.js → knowledgeBase-Be_zyW4L.js} +10 -32
- package/dist/src/{knowledgeBase-DZw2L9kf.js → knowledgeBase-CEzQobWX.js} +5 -36
- package/dist/src/{knowledgeBase-Bf57PI7g.cjs → knowledgeBase-D-5BMXlr.cjs} +10 -32
- package/dist/src/{litellm-Ds8YmO1b.js → litellm-9vR8zpfU.js} +3 -3
- package/dist/src/{litellm-DexU8FFp.js → litellm-CRDqPhNI.js} +14 -14
- package/dist/src/{litellm-fU-xV6T4.js → litellm-DnbRJ2if.js} +16 -16
- package/dist/src/{litellm-ByNRN2Vh.cjs → litellm-hUSNM_M2.cjs} +15 -15
- package/dist/src/{logger-BdZ-IqBc.cjs → logger-B7sBeGa0.cjs} +15 -13
- package/dist/src/{logger-w8Ozp0Td.js → logger-CG1uZPbQ.js} +15 -13
- package/dist/src/{logger-BotXmWKW.js → logger-LSBxlt7a.js} +15 -13
- package/dist/src/{luma-ray-y26EHFO8.js → luma-ray-4blv9iZ2.js} +10 -10
- package/dist/src/{luma-ray-vvE-O9EA.js → luma-ray-B2__8lYH.js} +11 -11
- package/dist/src/{luma-ray-BIe5ctI4.cjs → luma-ray-Hm3d6VJE.cjs} +10 -10
- package/dist/src/{luma-ray-C-w6EsJm.js → luma-ray-drvgdpP9.js} +2 -2
- package/dist/src/main.js +1854 -1166
- package/dist/src/{messages-DpoCLD_8.js → messages-CGPPidQr.js} +13 -10
- package/dist/src/{messages-Bg2ekFPc.js → messages-Uee41Mj5.js} +11 -8
- package/dist/src/{messages-16jb9LP7.cjs → messages-XhiwCbi4.cjs} +12 -9
- package/dist/src/{meteor-DLCdreag.js → meteor-BYykdXrV.js} +1 -1
- package/dist/src/{meteor-Dj8cTkU_.js → meteor-C8lGP6P4.js} +1 -1
- package/dist/src/{meteor-SLNTgmXm.js → meteor-CsopaHrH.js} +2 -2
- package/dist/src/{meteor-odmwVbyG.cjs → meteor-e-E-2vVl.cjs} +1 -1
- package/dist/src/modelslab-ClBkr8_9.js +168 -0
- package/dist/src/modelslab-E9gO-bYd.js +170 -0
- package/dist/src/modelslab-lUVW0cmB.cjs +167 -0
- package/dist/src/modelslab-yKz-ZNB4.js +159 -0
- package/dist/src/{nova-reel-CvXCVn3f.cjs → nova-reel-D8CuO6QH.cjs} +10 -10
- package/dist/src/{nova-reel-C2LFfVTf.js → nova-reel-Dk8jNpId.js} +2 -2
- package/dist/src/{nova-reel-BM0snD2m.js → nova-reel-P9bwvtYX.js} +11 -11
- package/dist/src/{nova-reel-CJLpBKSI.js → nova-reel-u2eF2Cxm.js} +10 -10
- package/dist/src/{nova-sonic-Cjh_qopr.js → nova-sonic-BaqWlkds.js} +6 -6
- package/dist/src/{nova-sonic-D_qERM-K.js → nova-sonic-CK2rAiKi.js} +2 -2
- package/dist/src/{nova-sonic-DEx_FdFW.cjs → nova-sonic-Ds1C-dpm.cjs} +6 -6
- package/dist/src/{nova-sonic-CklvraBd.js → nova-sonic-yZapPLv7.js} +8 -8
- package/dist/src/{openai-jNIREoAW.js → openai-CcN1B8Sb.js} +2 -2
- package/dist/src/{openai-Bigwjgo1.cjs → openai-DUFopMrH.cjs} +2 -2
- package/dist/src/{openai-Dz3surb_.js → openai-PblZ3jUE.js} +2 -2
- package/dist/src/{openclaw-dHLcXUWZ.js → openclaw-A-3_loM7.js} +2 -2
- package/dist/src/{openclaw-CbHtt3B8.cjs → openclaw-B6qqDr_u.cjs} +14 -14
- package/dist/src/{openclaw-DBSSY4SP.js → openclaw-COn6QzDi.js} +13 -13
- package/dist/src/{openclaw-CnAwBfPO.js → openclaw-a3lylB-V.js} +15 -15
- package/dist/src/{opencode-sdk-CqxTHrk0.js → opencode-sdk-4bL9n-Gk.js} +7 -7
- package/dist/src/{opencode-sdk-DhcfRbBH.js → opencode-sdk-BfC2zWcR.js} +3 -3
- package/dist/src/{opencode-sdk-xUrRk5Jx.js → opencode-sdk-DMJyuwMg.js} +6 -6
- package/dist/src/{opencode-sdk-DdmtRB6D.cjs → opencode-sdk-Da-9adza.cjs} +6 -6
- package/dist/src/{otlpReceiver-uytrRKdo.js → otlpReceiver-94URx7UW.js} +6 -6
- package/dist/src/{otlpReceiver-C-1cdVDr.js → otlpReceiver-BmmTiMjA.js} +6 -6
- package/dist/src/{otlpReceiver-th2zHwjZ.cjs → otlpReceiver-lXsYVbpj.cjs} +6 -6
- package/dist/src/{otlpReceiver-DmRb0NBj.js → otlpReceiver-oyf5wLGC.js} +2 -2
- package/dist/src/{providerRegistry-DAZoFmGv.js → providerRegistry-Cq_JK_CJ.js} +2 -2
- package/dist/src/{providerRegistry-lc7a7utN.cjs → providerRegistry-CvHEVJad.cjs} +2 -2
- package/dist/src/{providerRegistry-wCWd7sKQ.js → providerRegistry-DSSHjMKf.js} +2 -2
- package/dist/src/{providers-D3lk_R7Q.js → providers-BnFpbY_s.js} +321 -210
- package/dist/src/providers-CsXB2Ix-.js +35 -0
- package/dist/src/{providers-CpUrPa8s.cjs → providers-DHbjzW2e.cjs} +330 -219
- package/dist/src/providers-DO8ltjLC.js +33 -0
- package/dist/src/providers-Dtq-xnXd.cjs +33 -0
- package/dist/src/{providers-BOvl9qv5.js → providers-Iil64vk9.js} +326 -214
- package/dist/src/{pythonUtils-CUPlf1H0.js → pythonUtils-CcT5LH1M.js} +3 -3
- package/dist/src/{pythonUtils-DZ6EbdY4.cjs → pythonUtils-DBbuI3QJ.cjs} +3 -3
- package/dist/src/{pythonUtils-vMlk9Qp5.js → pythonUtils-hZ8LeQLv.js} +3 -3
- package/dist/src/quiverai-BuI0tE39.js +212 -0
- package/dist/src/quiverai-DCGSZt4U.js +215 -0
- package/dist/src/quiverai-DiMVJQDz.cjs +214 -0
- package/dist/src/quiverai-fQNkExW4.js +217 -0
- package/dist/src/{render-kFatbqG6.js → render-Dj1smHEb.js} +4 -4
- package/dist/src/{responses-B85halqh.js → responses-CxzoQoBe.js} +21 -11
- package/dist/src/{responses-BtU_i-R0.js → responses-DOAFFENS.js} +20 -10
- package/dist/src/{responses-CXJACENh.cjs → responses-ghR3IOfy.cjs} +20 -10
- package/dist/src/rubyUtils-BUbcND2f.js +6 -0
- package/dist/src/{rubyUtils-DCVaJ3mc.cjs → rubyUtils-C8MhKGHb.cjs} +3 -3
- package/dist/src/rubyUtils-Cr55X_KE.js +5 -0
- package/dist/src/{rubyUtils-Cs35SDYa.js → rubyUtils-CwbGmgYN.js} +3 -3
- package/dist/src/rubyUtils-DlIiqoYo.cjs +5 -0
- package/dist/src/{rubyUtils-DYKoa7Ze.js → rubyUtils-DudlFZed.js} +3 -3
- package/dist/src/{sagemaker-BUagsrJ-.cjs → sagemaker-77zbJ2Q2.cjs} +17 -17
- package/dist/src/{sagemaker-hRS_Jj3-.js → sagemaker-CcxhlOAR.js} +18 -18
- package/dist/src/{sagemaker-DI6IOTjT.js → sagemaker-DuM71dVU.js} +17 -17
- package/dist/src/{sagemaker-Du4LIR97.js → sagemaker-gmskuyre.js} +2 -2
- package/dist/src/{scanner-C6QXA9Pn.js → scanner-DJYiSXQj.js} +61 -13
- package/dist/src/server/index.js +2635 -1516
- package/dist/src/{server-C_O6-VE4.cjs → server-B5v33lvE.cjs} +4 -4
- package/dist/src/{server-HQtz_gUy.js → server-BJ4m4f1D.js} +5 -5
- package/dist/src/server-C2eQH4Gu.js +6 -0
- package/dist/src/server-CXWycu7H.cjs +6 -0
- package/dist/src/server-Q6OGlxxT.js +8 -0
- package/dist/src/{server-BPC37wlX.js → server-RV_i_YX5.js} +4 -4
- package/dist/src/{signal-D4VhSJOY.js → signal-BW33JuId.js} +3 -3
- package/dist/src/{slack-DTUSadx4.js → slack-2pRrhhgJ.js} +2 -2
- package/dist/src/{slack-DPqj42Ts.js → slack-BB6yuZzp.js} +2 -2
- package/dist/src/{slack-OZYxoVON.js → slack-BQYeW9L3.js} +2 -2
- package/dist/src/{slack-BtMkB6xP.cjs → slack-DEURelTy.cjs} +2 -2
- package/dist/src/store-B3EDO9Q3.js +7 -0
- package/dist/src/{store-F-KTiqfb.cjs → store-D7CgQzAR.cjs} +3 -3
- package/dist/src/{store-BTdkdmK9.js → store-DJNsD1iC.js} +3 -3
- package/dist/src/store-Dl9F8aw5.js +6 -0
- package/dist/src/store-SnrGrlt9.cjs +6 -0
- package/dist/src/{store-rueRdzcm.js → store-s3SftUwF.js} +3 -3
- package/dist/src/{tables-w-r-abWq.cjs → tables-BKTmd6u7.cjs} +7 -5
- package/dist/src/{tables-ChjAuVAR.js → tables-DMegD0Xf.js} +7 -5
- package/dist/src/{tables-dNaVPyDr.js → tables-DfTsNN7X.js} +7 -5
- package/dist/src/{telemetry-wXyCQ1Nz.js → telemetry--WAdAfVi.js} +4 -4
- package/dist/src/telemetry-BGhiPZtl.js +8 -0
- package/dist/src/{telemetry-Dai7PdlR.js → telemetry-BedSm-bZ.js} +4 -4
- package/dist/src/telemetry-CFfiYan6.cjs +6 -0
- package/dist/src/telemetry-DHzEduxX.js +6 -0
- package/dist/src/{telemetry-Y1esJM0v.cjs → telemetry-DQgVBCAb.cjs} +4 -4
- package/dist/src/{text-C1cXk9ij.js → text-B_IrO4GZ.js} +1 -1
- package/dist/src/{text-DF2hMKdg.cjs → text-oKzCBnK6.cjs} +1 -1
- package/dist/src/{text-Dm78AVGG.js → text-oiSbwSOI.js} +1 -1
- package/dist/src/{tokenUsageUtils-D2ko2eFQ.js → tokenUsageUtils-DmZSD2eU.js} +1 -1
- package/dist/src/{transcription-C-K2hDJX.cjs → transcription-BO1AHegO.cjs} +10 -10
- package/dist/src/{transcription-i6EfSMie.js → transcription-X2-B4vkX.js} +10 -10
- package/dist/src/{transcription-CcYTL4wi.js → transcription-lzBLiTFJ.js} +11 -11
- package/dist/src/{transcription-C-M81iDA.js → transcription-mYS9vd5v.js} +2 -2
- package/dist/src/{transform-Kd6u-oNm.cjs → transform-B1Hi5lWS.cjs} +4 -4
- package/dist/src/{transform-B86_0IIb.js → transform-BEgStbHK.js} +5 -5
- package/dist/src/transform-C1x1ZlMQ.cjs +6 -0
- package/dist/src/{transform-D8dILpfZ.js → transform-CYDILYDe.js} +4 -4
- package/dist/src/{transform-CCSNqTI-.cjs → transform-D5PjiWiZ.cjs} +110 -23
- package/dist/src/transform-DYHjFmQu.js +8 -0
- package/dist/src/{transform-gpzlPrwG.js → transform-DeGlxb0D.js} +99 -24
- package/dist/src/{transform-DosdxVdh.js → transform-Dfl89yi4.js} +99 -24
- package/dist/src/transform-rmwJT5JQ.js +7 -0
- package/dist/src/{transformersAvailability-DM54tWzt.js → transformersAvailability-CjeFXhuJ.js} +1 -1
- package/dist/src/{transformersAvailability-Bkep3ka7.cjs → transformersAvailability-D-glmEy7.cjs} +1 -1
- package/dist/src/{transformersAvailability-DEU2naS1.js → transformersAvailability-SZnTS3pJ.js} +1 -1
- package/dist/src/{transformersAvailability-DkAWaK5B.js → transformersAvailability-eJooj0gX.js} +1 -1
- package/dist/src/{types-CslP2TyG.js → types-C5hEkb-x.js} +86 -3
- package/dist/src/{types-CMDs7Kzu.js → types-CXQduE9o.js} +89 -4
- package/dist/src/{types-Brw5kggg.cjs → types-DWNf48sT.cjs} +87 -4
- package/dist/src/{util-D8z5QOy8.cjs → util-1wWM599Z.cjs} +43 -15
- package/dist/src/{util-CttPoePn.js → util-Bm_-UMD_.js} +43 -15
- package/dist/src/{util-Ba18wQ8e.js → util-CoQjmE3u.js} +4 -4
- package/dist/src/{util-CBAXTxfy.cjs → util-CyUdMzV0.cjs} +4 -4
- package/dist/src/{util-CfPdWQJX.js → util-D9eLdGfa.js} +43 -15
- package/dist/src/{util-CY_XguB9.js → util-DQ984syk.js} +71 -11
- package/dist/src/{util-Dft1RxsL.js → util-Du96oyYS.js} +4 -4
- package/dist/src/{util-BECSJ0T_.js → util-_h4pVqrz.js} +72 -12
- package/dist/src/{util-Dab5xp2E.cjs → util-aLhtl3fe.cjs} +76 -10
- package/dist/src/{utils-Db5KwiXp.cjs → utils-BjLy-Q72.cjs} +6 -6
- package/dist/src/{utils-BTeLHwiH.js → utils-CFMn2yHW.js} +3 -3
- package/dist/src/{utils-ICP7kLcY.js → utils-DvWMzuMx.js} +3 -3
- package/dist/tsconfig.tsbuildinfo +1 -1
- package/package.json +54 -47
- package/dist/src/app/assets/index-COIipVf1.css +0 -1
- package/dist/src/app/assets/index-ClD64gUa.js +0 -428
- package/dist/src/app/assets/vendor-react-DHoaNLma.js +0 -4
- package/dist/src/base-CwCV0t8r.js +0 -106
- package/dist/src/base-D8VQYhNQ.cjs +0 -111
- package/dist/src/base-DumUXbOH.js +0 -107
- package/dist/src/cache-BJ31KhNT.cjs +0 -6
- package/dist/src/cloud-DHWE-9nj.js +0 -5
- package/dist/src/eval-DgiEH2xe.js +0 -16
- package/dist/src/evalResult-Boel8xoS.js +0 -13
- package/dist/src/evalResult-DGx1-Ela.js +0 -11
- package/dist/src/evalResult-DzRxYLDH.cjs +0 -11
- package/dist/src/evaluator-B4qKt437.js +0 -38
- package/dist/src/fetch-BQgz6x7S.js +0 -6
- package/dist/src/fetch-DwhKew9e.cjs +0 -4
- package/dist/src/graders-BpEqZT2Q.js +0 -34
- package/dist/src/graders-DTGI-hHE.cjs +0 -33
- package/dist/src/graders-DgMr-E4I.js +0 -36
- package/dist/src/providers-DVsoSup2.js +0 -34
- package/dist/src/providers-D_tJatIk.js +0 -32
- package/dist/src/providers-wpMPsMZ7.cjs +0 -32
- package/dist/src/quiverai-BNGJOQym.cjs +0 -54
- package/dist/src/quiverai-CedIP0PJ.js +0 -43
- package/dist/src/quiverai-DthRqZs9.js +0 -55
- package/dist/src/quiverai-Y7Is4wah.js +0 -57
- package/dist/src/rubyUtils-BA4D4gMX.js +0 -6
- package/dist/src/rubyUtils-DRRiMFV2.js +0 -5
- package/dist/src/rubyUtils-vb8OYFC-.cjs +0 -5
- package/dist/src/server-BeqUuSvj.js +0 -6
- package/dist/src/server-CwQsQ6dl.cjs +0 -6
- package/dist/src/server-Dhbq83-d.js +0 -8
- package/dist/src/store-C7U17qcu.js +0 -6
- package/dist/src/store-CuDU_jDN.js +0 -7
- package/dist/src/store-awdDxuiD.cjs +0 -6
- package/dist/src/telemetry-ClcYTCOG.js +0 -8
- package/dist/src/telemetry-DEWuLu_s.js +0 -6
- package/dist/src/telemetry-tADmeX6E.cjs +0 -6
- package/dist/src/transform-8eGmaH-7.js +0 -7
- package/dist/src/transform-BRVvWaG4.cjs +0 -6
- package/dist/src/transform-DwsntMEQ.js +0 -8
package/README.md
CHANGED
|
@@ -5,11 +5,11 @@
|
|
|
5
5
|
<a href="https://npmjs.com/package/promptfoo"><img src="https://img.shields.io/npm/dm/promptfoo" alt="npm"></a>
|
|
6
6
|
<a href="https://github.com/promptfoo/promptfoo/actions/workflows/main.yml"><img src="https://img.shields.io/github/actions/workflow/status/promptfoo/promptfoo/main.yml" alt="GitHub Workflow Status"></a>
|
|
7
7
|
<a href="https://github.com/promptfoo/promptfoo/blob/main/LICENSE"><img src="https://img.shields.io/github/license/promptfoo/promptfoo" alt="MIT license"></a>
|
|
8
|
-
<a href="https://discord.gg/promptfoo"><img src="https://
|
|
8
|
+
<a href="https://discord.gg/promptfoo"><img src="https://img.shields.io/discord/1146610656779440188?logo=discord&label=promptfoo" alt="Discord"></a>
|
|
9
9
|
</p>
|
|
10
10
|
|
|
11
11
|
<p align="center">
|
|
12
|
-
<code>promptfoo</code> is a
|
|
12
|
+
<code>promptfoo</code> is a CLI and library for evaluating and red-teaming LLM apps. Stop the trial-and-error approach - start shipping secure, reliable AI apps.
|
|
13
13
|
</p>
|
|
14
14
|
|
|
15
15
|
<p align="center">
|
|
@@ -23,11 +23,24 @@
|
|
|
23
23
|
## Quick Start
|
|
24
24
|
|
|
25
25
|
```sh
|
|
26
|
-
|
|
27
|
-
|
|
26
|
+
npm install -g promptfoo
|
|
27
|
+
promptfoo init --example getting-started
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Also available via `brew install promptfoo` and `pip install promptfoo`. You can also use `npx promptfoo@latest` to run any command without installing.
|
|
31
|
+
|
|
32
|
+
Most LLM providers require an API key. Set yours as an environment variable:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
export OPENAI_API_KEY=sk-abc123
|
|
36
|
+
```
|
|
28
37
|
|
|
29
|
-
|
|
30
|
-
|
|
38
|
+
Once you're in the example directory, run an eval and view results:
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
cd getting-started
|
|
42
|
+
promptfoo eval
|
|
43
|
+
promptfoo view
|
|
31
44
|
```
|
|
32
45
|
|
|
33
46
|
See [Getting Started](https://www.promptfoo.dev/docs/getting-started/) (evals) or [Red Teaming](https://www.promptfoo.dev/docs/red-team/) (vulnerability scanning) for more.
|
|
@@ -43,34 +56,34 @@ See [Getting Started](https://www.promptfoo.dev/docs/getting-started/) (evals) o
|
|
|
43
56
|
|
|
44
57
|
Here's what it looks like in action:
|
|
45
58
|
|
|
46
|
-
|
|
59
|
+
<img src="site/static/img/claude-vs-gpt-example@2x.png" alt="prompt evaluation matrix - web viewer" width="700">
|
|
47
60
|
|
|
48
61
|
It works on the command line too:
|
|
49
62
|
|
|
50
|
-
|
|
63
|
+
<img src="https://www.promptfoo.dev/img/docs/self-grading.gif" alt="promptfoo command line" width="700">
|
|
51
64
|
|
|
52
65
|
It also can generate [security vulnerability reports](https://www.promptfoo.dev/docs/red-team/):
|
|
53
66
|
|
|
54
|
-
|
|
67
|
+
<img src="https://www.promptfoo.dev/img/riskreport-1@2x.png" alt="gen ai red team" width="700">
|
|
55
68
|
|
|
56
69
|
## Why Promptfoo?
|
|
57
70
|
|
|
58
|
-
-
|
|
59
|
-
-
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
63
|
-
-
|
|
71
|
+
- **Developer-first**: Fast, with features like live reload and caching
|
|
72
|
+
- **Private**: LLM evals run 100% locally - your prompts never leave your machine
|
|
73
|
+
- **Flexible**: Works with any LLM API or programming language
|
|
74
|
+
- **Battle-tested**: Powers LLM apps serving 10M+ users in production
|
|
75
|
+
- **Data-driven**: Make decisions based on metrics, not gut feel
|
|
76
|
+
- **Open source**: MIT licensed, with an active community
|
|
64
77
|
|
|
65
78
|
## Learn More
|
|
66
79
|
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
-
|
|
71
|
-
-
|
|
72
|
-
-
|
|
73
|
-
-
|
|
80
|
+
- [Getting Started](https://www.promptfoo.dev/docs/getting-started/)
|
|
81
|
+
- [Full Documentation](https://www.promptfoo.dev/docs/intro/)
|
|
82
|
+
- [Red Teaming Guide](https://www.promptfoo.dev/docs/red-team/)
|
|
83
|
+
- [CLI Usage](https://www.promptfoo.dev/docs/usage/command-line/)
|
|
84
|
+
- [Node.js Package](https://www.promptfoo.dev/docs/usage/node-package/)
|
|
85
|
+
- [Supported Models](https://www.promptfoo.dev/docs/providers/)
|
|
86
|
+
- [Code Scanning Guide](https://www.promptfoo.dev/docs/code-scanning/)
|
|
74
87
|
|
|
75
88
|
## Contributing
|
|
76
89
|
|