btzsc 0.1.1__tar.gz

btzsc-0.1.1/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.3
+Name: btzsc
+Version: 0.1.1
+Summary: BTZSC: A Benchmark for Zero-Shot Text Classification across Cross-Encoders, Embedding Models, Rerankers and LLMs
+Keywords: nlp,zero-shot,text-classification,benchmark,transformers
+Author: Ilias Aarab
+Author-email: Ilias Aarab <ilias.aarab@ecb.europa.eu>
+License: Apache-2.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Dist: torch>=2.0.0
+Requires-Dist: transformers>=4.40.0
+Requires-Dist: datasets>=2.18.0
+Requires-Dist: sentence-transformers>=3.0.0
+Requires-Dist: scikit-learn>=1.4.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: tqdm>=4.66.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: tabulate>=0.9.0
+Requires-Python: >=3.10
+Project-URL: Homepage, https://github.com/aarabil/btzsc
+Project-URL: Documentation, https://github.com/aarabil/btzsc
+Project-URL: Repository, https://github.com/aarabil/btzsc
+Project-URL: Bug Tracker, https://github.com/aarabil/btzsc/issues
+Project-URL: Paper, https://openreview.net/forum?id=TODO
+Project-URL: Dataset, https://huggingface.co/datasets/btzsc/btzsc
+Description-Content-Type: text/markdown
+
+<p align="center">
+	<img src="./docs/" align="center" width="30%" alt="BTZSC banner">
+</p>
+<h1 align="center">BTZSC</h1>
+<p align="center">
+	<em>A unified benchmark for zero-shot text classification across modern model families.</em>
+</p>
+<p align="center">
+	<img src="https://img.shields.io/github/last-commit/IliasAarab/btzsc?style=default&logo=git&logoColor=white&color=0080ff" alt="last-commit">
+	<img src="https://img.shields.io/github/languages/top/IliasAarab/btzsc?style=default&color=0080ff" alt="repo-top-language">
+</p>
+<br>
+
+<details><summary>Table of Contents</summary>
+
+- [Overview](#overview)
+- [Dataset](#dataset)
+- [Install](#install)
+- [Quick start (Python)](#quick-start-python)
+- [Quick start (CLI)](#quick-start-cli)
+- [Custom models](#custom-models)
+- [Packaging & release](#packaging--release)
+- [Notes](#notes)
+
+</details>
+<hr>
+
+## Overview
+
+BTZSC is a benchmark package for zero-shot text classification across embedding models, NLI cross-encoders, rerankers, and instruction-tuned LLMs.
+
+## Dataset
+
+https://huggingface.co/datasets/btzsc/btzsc
+
+### Install
+
+```bash
+uv sync
+```
+
+### Quick start (Python)
+
+```python
+from btzsc import BTZSCBenchmark
+
+benchmark = BTZSCBenchmark(tasks=["sentiment", "topic"])
+results = benchmark.evaluate(
+	model="intfloat/e5-base-v2",
+	model_type="embedding",
+	batch_size=64,
+)
+
+print(results.summary())
+print(results.per_dataset())
+```
+
+### Quick start (CLI)
+
+```bash
+btzsc evaluate --model intfloat/e5-base-v2 --type embedding --tasks sentiment,topic
+btzsc baselines --metric f1 --top 10
+btzsc list-datasets
+```
+
+### Custom models
+
+Subclass `BaseModel` and implement:
+- `predict_scores(texts, labels, batch_size)`
+- `predict(texts, labels, batch_size)`
+
+Then pass your instance to `BTZSCBenchmark.evaluate()`.
+
+### Packaging & release
+
+Build locally:
+
+```bash
+uv build
+```
+
+Publish from GitHub using tags (workflow: `.github/workflows/publish.yml`):
+
+1. Bump `version` in `pyproject.toml`.
+2. Commit and push to `master`.
+3. Create and push a version tag:
+
+```bash
+git tag v0.1.0
+git push origin v0.1.0
+```
+
+GitHub Actions will build with `uv` and publish to PyPI via trusted publishing.
+
+For first-time setup on PyPI:
+
+- Create the project on PyPI.
+- Add a trusted publisher for this repository.
+- Set workflow path to `.github/workflows/publish.yml`.
+
+### Notes
+
+- The BTZSC dataset stores rows as `(text, hypothesis, labels)` where `labels` is binary entailment.
+- The package reconstructs grouped multiclass samples internally.
+- Baseline tables are bundled from the published BTZSC paper runs.

btzsc-0.1.1/README.md

@@ -0,0 +1,105 @@
+<p align="center">
+	<img src="./docs/" align="center" width="30%" alt="BTZSC banner">
+</p>
+<h1 align="center">BTZSC</h1>
+<p align="center">
+	<em>A unified benchmark for zero-shot text classification across modern model families.</em>
+</p>
+<p align="center">
+	<img src="https://img.shields.io/github/last-commit/IliasAarab/btzsc?style=default&logo=git&logoColor=white&color=0080ff" alt="last-commit">
+	<img src="https://img.shields.io/github/languages/top/IliasAarab/btzsc?style=default&color=0080ff" alt="repo-top-language">
+</p>
+<br>
+
+<details><summary>Table of Contents</summary>
+
+- [Overview](#overview)
+- [Dataset](#dataset)
+- [Install](#install)
+- [Quick start (Python)](#quick-start-python)
+- [Quick start (CLI)](#quick-start-cli)
+- [Custom models](#custom-models)
+- [Packaging & release](#packaging--release)
+- [Notes](#notes)
+
+</details>
+<hr>
+
+## Overview
+
+BTZSC is a benchmark package for zero-shot text classification across embedding models, NLI cross-encoders, rerankers, and instruction-tuned LLMs.
+
+## Dataset
+
+https://huggingface.co/datasets/btzsc/btzsc
+
+### Install
+
+```bash
+uv sync
+```
+
+### Quick start (Python)
+
+```python
+from btzsc import BTZSCBenchmark
+
+benchmark = BTZSCBenchmark(tasks=["sentiment", "topic"])
+results = benchmark.evaluate(
+	model="intfloat/e5-base-v2",
+	model_type="embedding",
+	batch_size=64,
+)
+
+print(results.summary())
+print(results.per_dataset())
+```
+
+### Quick start (CLI)
+
+```bash
+btzsc evaluate --model intfloat/e5-base-v2 --type embedding --tasks sentiment,topic
+btzsc baselines --metric f1 --top 10
+btzsc list-datasets
+```
+
+### Custom models
+
+Subclass `BaseModel` and implement:
+- `predict_scores(texts, labels, batch_size)`
+- `predict(texts, labels, batch_size)`
+
+Then pass your instance to `BTZSCBenchmark.evaluate()`.
+
+### Packaging & release
+
+Build locally:
+
+```bash
+uv build
+```
+
+Publish from GitHub using tags (workflow: `.github/workflows/publish.yml`):
+
+1. Bump `version` in `pyproject.toml`.
+2. Commit and push to `master`.
+3. Create and push a version tag:
+
+```bash
+git tag v0.1.0
+git push origin v0.1.0
+```
+
+GitHub Actions will build with `uv` and publish to PyPI via trusted publishing.
+
+For first-time setup on PyPI:
+
+- Create the project on PyPI.
+- Add a trusted publisher for this repository.
+- Set workflow path to `.github/workflows/publish.yml`.
+
+### Notes
+
+- The BTZSC dataset stores rows as `(text, hypothesis, labels)` where `labels` is binary entailment.
+- The package reconstructs grouped multiclass samples internally.
+- Baseline tables are bundled from the published BTZSC paper runs.

btzsc-0.1.1/pyproject.toml

@@ -0,0 +1,62 @@
+[project]
+name = "btzsc"
+version = "0.1.1"
+description = "BTZSC: A Benchmark for Zero-Shot Text Classification across Cross-Encoders, Embedding Models, Rerankers and LLMs"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+authors = [{ name = "Ilias Aarab", email = "ilias.aarab@ecb.europa.eu" }]
+requires-python = ">=3.10"
+keywords = [
+    "nlp",
+    "zero-shot",
+    "text-classification",
+    "benchmark",
+    "transformers",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "torch>=2.0.0",
+    "transformers>=4.40.0",
+    "datasets>=2.18.0",
+    "sentence-transformers>=3.0.0",
+    "scikit-learn>=1.4.0",
+    "numpy>=1.24.0",
+    "pandas>=2.0.0",
+    "tqdm>=4.66.0",
+    "click>=8.1.0",
+    "pyyaml>=6.0",
+    "tabulate>=0.9.0",
+]
+
+[project.scripts]
+btzsc = "btzsc.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/aarabil/btzsc"
+Documentation = "https://github.com/aarabil/btzsc"
+Repository = "https://github.com/aarabil/btzsc"
+"Bug Tracker" = "https://github.com/aarabil/btzsc/issues"
+Paper = "https://openreview.net/forum?id=TODO"
+Dataset = "https://huggingface.co/datasets/btzsc/btzsc"
+
+[build-system]
+requires = ["uv_build>=0.10.1,<0.11.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "pyright[nodejs]>=1.1.408",
+    "ruff>=0.13.0",
+    "pytest>=8.0.0",
+    "pytest-cov>=5.0.0",
+]
+
+[tool.pyright]
+typeCheckingMode = "basic"
+include = ["src"]

btzsc-0.1.1/src/btzsc/__init__.py

@@ -0,0 +1,10 @@
+"""BTZSC public package API."""
+
+from btzsc.benchmark import BTZSCBenchmark, BTZSCResults
+from btzsc.models.base import BaseModel
+
+__all__ = [
+    "BTZSCBenchmark",
+    "BTZSCResults",
+    "BaseModel",
+]

btzsc-0.1.1/src/btzsc/_baselines/acc_scores.csv

@@ -0,0 +1,39 @@
+mdl,agnews,amazon_counterfactual,amazon_massive_scenario,amazonpolarity,appreviews,banking77,biasframes_intent,biasframes_offensive,biasframes_sex,capsotu,emotiondair,empathetic,financialphrasebank,imdb,manifesto,massive,mtop_domain,rottentomatoes,toxic_conversations_50k,trueteacher,tweet_sentiment_extraction,wikitoxic_insult,wikitoxic_obscene,wikitoxic_threat,wikitoxic_toxicaggregated,yahootopics,yelpreviews
+bert-large-uncased,0.26499998569488525,0.2507462799549103,0.05000000074505806,0.4790000021457672,0.5120000243186951,0.03099999949336052,0.4399999976158142,0.5720000267028809,0.9139999747276306,0.0949999988079071,0.19699999690055847,0.028999999165534973,0.5623188614845276,0.5199999809265137,0.017000000923871994,0.009999999776482582,0.1509999930858612,0.5187617540359497,0.5335000157356262,0.49399998784065247,0.41499999165534973,0.5759999752044678,0.5839999914169312,0.7760000228881836,0.5540000200271606,0.12999999523162842,0.5180000066757202
+deberta-v3-large,0.2759999930858612,0.6626865863800049,0.029999999329447746,0.4869999885559082,0.5120000243186951,0.027000000700354576,0.4749999940395355,0.5789999961853027,0.125,0.014999999664723873,0.19699999690055847,0.032999999821186066,0.3971014618873596,0.4869999885559082,0.03700000047683716,0.0020000000949949026,0.09399999678134918,0.501876175403595,0.5049999952316284,0.5059999823570251,0.31700000166893005,0.6489999890327454,0.6399999856948853,0.8560000061988831,0.5789999961853027,0.10700000077486038,0.5199999809265137
+ModernBERT-large,0.24199999868869781,0.8268656730651855,0.08900000154972076,0.4860000014305115,0.49900001287460327,0.032999999821186066,0.48399999737739563,0.5329999923706055,0.722000002861023,0.027000000700354576,0.1420000046491623,0.020999999716877937,0.5695652365684509,0.5130000114440918,0.009999999776482582,0.020999999716877937,0.12600000202655792,0.5,0.4964999854564667,0.5090000033378601,0.328000009059906,0.460999995470047,0.5899999737739563,0.13600000739097595,0.6039999723434448,0.15199999511241913,0.5139999985694885
+bart-large-mnli,0.7269999980926514,0.17611940205097198,0.515999972820282,0.9330000281333923,0.9210000038146973,0.28700000047683716,0.6320000290870667,0.5690000057220459,0.07500000298023224,0.39399999380111694,0.5049999952316284,0.3889999985694885,0.41884058713912964,0.9290000200271606,0.09000000357627869,0.4300000071525574,0.8399999737739563,0.8330206274986267,0.5734999775886536,0.5149999856948853,0.6230000257492065,0.41999998688697815,0.6930000185966492,0.08399999886751175,0.6069999933242798,0.30300000309944153,0.9589999914169312
+nli-roberta-base,0.7049999833106995,0.22686567902565002,0.2770000100135803,0.8870000243186951,0.8939999938011169,0.07500000298023224,0.5149999856948853,0.5830000042915344,0.1860000044107437,0.23000000417232513,0.33899998664855957,0.3240000009536743,0.45072463154792786,0.8320000171661377,0.020999999716877937,0.3569999933242798,0.43299999833106995,0.7983114719390869,0.6554999947547913,0.4830000102519989,0.6119999885559082,0.5630000233650208,0.7129999995231628,0.1379999965429306,0.7020000219345093,0.4059999883174896,0.8859999775886536
+bert-base-uncased-nli,0.6850000023841858,0.21194030344486237,0.21400000154972076,0.8629999756813049,0.8379999995231628,0.019999999552965164,0.5630000233650208,0.5669999718666077,0.5260000228881836,0.019999999552965164,0.3880000114440918,0.15199999511241913,0.4000000059604645,0.7940000295639038,0.017000000923871994,0.2849999964237213,0.6169999837875366,0.7570356726646423,0.5795000195503235,0.492000013589859,0.6370000243186951,0.7609999775886536,0.7680000066757202,0.2930000126361847,0.6470000147819519,0.382999986410141,0.8870000243186951
+bert-large-uncased-nli,0.7480000257492065,0.4298507571220398,0.38499999046325684,0.8399999737739563,0.8550000190734863,0.08100000023841858,0.6069999933242798,0.5989999771118164,0.7559999823570251,0.21799999475479126,0.5479999780654907,0.12600000202655792,0.647826075553894,0.8100000023841858,0.01600000075995922,0.3490000069141388,0.4050000011920929,0.7176360487937927,0.5529999732971191,0.49399998784065247,0.6520000100135803,0.8149999976158142,0.7940000295639038,0.7540000081062317,0.6740000247955322,0.2549999952316284,0.8970000147819519
+bert-large-uncased-nli-triplet,0.7319999933242798,0.38805970549583435,0.29600000381469727,0.8399999737739563,0.8500000238418579,0.0689999982714653,0.6039999723434448,0.6690000295639038,0.5270000100135803,0.24300000071525574,0.296999990940094,0.2070000022649765,0.5855072736740112,0.7879999876022339,0.024000000208616257,0.3720000088214874,0.460999995470047,0.7260788083076477,0.5874999761581421,0.49000000953674316,0.6510000228881836,0.7269999980926514,0.8349999785423279,0.4950000047683716,0.777999997138977,0.4169999957084656,0.8970000147819519
+deberta-v3-base-nli,0.7639999985694885,0.17313432693481445,0.49399998784065247,0.8980000019073486,0.9100000262260437,0.12700000405311584,0.5139999985694885,0.6000000238418579,0.5529999732971191,0.23399999737739563,0.3790000081062317,0.3179999887943268,0.6710144877433777,0.9139999747276306,0.05299999937415123,0.32899999618530273,0.652999997138977,0.8236397504806519,0.6915000081062317,0.4909999966621399,0.6290000081062317,0.6809999942779541,0.8320000171661377,0.4569999873638153,0.7839999794960022,0.5289999842643738,0.9300000071525574
+deberta-v3-large-nli,0.8140000104904175,0.18208955228328705,0.5440000295639038,0.9229999780654907,0.9279999732971191,0.3610000014305115,0.6710000038146973,0.6349999904632568,0.2980000078678131,0.2280000001192093,0.4909999966621399,0.4580000042915344,0.8246376514434814,0.902999997138977,0.10899999737739563,0.43799999356269836,0.7549999952316284,0.8489680886268616,0.7289999723434448,0.49799999594688416,0.6669999957084656,0.6129999756813049,0.8190000057220459,0.3700000047683716,0.7990000247955322,0.6019999980926514,0.9829999804496765
+deberta-v3-large-nli-triplet,0.8320000171661377,0.34328359365463257,0.5709999799728394,0.9290000200271606,0.9300000071525574,0.24400000274181366,0.6990000009536743,0.6669999957084656,0.296999990940094,0.27300000190734863,0.4390000104904175,0.4230000078678131,0.8188405632972717,0.9269999861717224,0.09300000220537186,0.4230000078678131,0.5099999904632568,0.8377110958099365,0.718500018119812,0.5130000114440918,0.6439999938011169,0.6940000057220459,0.8429999947547913,0.5659999847412109,0.8230000138282776,0.37299999594688416,0.9810000061988831
+modernbert-base-nli,0.7509999871253967,0.6805970072746277,0.3619999885559082,0.9110000133514404,0.9079999923706055,0.08799999952316284,0.5189999938011169,0.6209999918937683,0.8360000252723694,0.18199999630451202,0.28299999237060547,0.29499998688697815,0.6072463989257812,0.890999972820282,0.02800000086426735,0.2770000100135803,0.46700000762939453,0.7514071464538574,0.6840000152587891,0.5,0.6169999837875366,0.75,0.7099999785423279,0.32600000500679016,0.75,0.4440000057220459,0.9599999785423279
+modernbert-large-nli,0.7580000162124634,0.4298507571220398,0.39500001072883606,0.9330000281333923,0.9200000166893005,0.23999999463558197,0.6320000290870667,0.5860000252723694,0.5479999780654907,0.08799999952316284,0.28299999237060547,0.31299999356269836,0.695652186870575,0.9139999747276306,0.09799999743700027,0.3880000114440918,0.5889999866485596,0.8649155497550964,0.6934999823570251,0.5180000066757202,0.6589999794960022,0.6069999933242798,0.8119999766349792,0.5889999866485596,0.734000027179718,0.5440000295639038,0.9789999723434448
+modernbert-large-nli-triplet,0.7160000205039978,0.39701491594314575,0.47600001096725464,0.9290000200271606,0.9200000166893005,0.21799999475479126,0.6019999980926514,0.5979999899864197,0.3970000147819519,0.12800000607967377,0.36800000071525574,0.335999995470047,0.7318840622901917,0.9110000133514404,0.07699999958276749,0.4440000057220459,0.6740000247955322,0.8714821934700012,0.699999988079071,0.5109999775886536,0.6809999942779541,0.6729999780654907,0.8040000200271606,0.5429999828338623,0.7440000176429749,0.28600001335144043,0.9789999723434448
+ms-marco-MiniLM-L6-v2,0.4180000126361847,0.7552238702774048,0.08299999684095383,0.6880000233650208,0.7120000123977661,0.2199999988079071,0.6169999837875366,0.6169999837875366,0.7459999918937683,0.3230000138282776,0.20399999618530273,0.1809999942779541,0.2869565188884735,0.625,0.14000000059604645,0.039000000804662704,0.2290000021457672,0.5787992477416992,0.5260000228881836,0.5180000066757202,0.41999998688697815,0.5270000100135803,0.5509999990463257,0.6539999842643738,0.5019999742507935,0.3269999921321869,0.6539999842643738
+gte-reranker-modernbert-base,0.7120000123977661,0.6716417670249939,0.5440000295639038,0.9120000004768372,0.9169999957084656,0.6349999904632568,0.4699999988079071,0.5239999890327454,0.8489999771118164,0.47999998927116394,0.5210000276565552,0.39399999380111694,0.4739130437374115,0.8410000205039978,0.25600001215934753,0.5659999847412109,0.597000002861023,0.8020637631416321,0.5885000228881836,0.5979999899864197,0.6179999709129333,0.5479999780654907,0.5659999847412109,0.6010000109672546,0.546999990940094,0.3880000114440918,0.9629999995231628
+bge-reranker-base,0.6549999713897705,0.420895516872406,0.460999995470047,0.6639999747276306,0.7910000085830688,0.4860000014305115,0.5019999742507935,0.4790000021457672,0.34599998593330383,0.43799999356269836,0.35499998927116394,0.2759999930858612,0.35217392444610596,0.6949999928474426,0.2720000147819519,0.44600000977516174,0.7739999890327454,0.5759850144386292,0.5690000057220459,0.4729999899864197,0.5860000252723694,0.46000000834465027,0.550000011920929,0.3109999895095825,0.5730000138282776,0.47200000286102295,0.6510000228881836
+bge-reranker-large,0.7419999837875366,0.5044776201248169,0.4970000088214874,0.8700000047683716,0.890999972820282,0.5569999814033508,0.597000002861023,0.5789999961853027,0.2070000022649765,0.5170000195503235,0.41499999165534973,0.3400000035762787,0.48260870575904846,0.8040000200271606,0.3199999928474426,0.4749999940395355,0.8220000267028809,0.7560975551605225,0.6389999985694885,0.4490000009536743,0.609000027179718,0.4449999928474426,0.5559999942779541,0.15800000727176666,0.6150000095367432,0.593999981880188,0.8790000081062317
+Qwen3-Reranker-0.6B,0.7960000038146973,0.2626865804195404,0.6629999876022339,0.9120000004768372,0.8870000243186951,0.625,0.5649999976158142,0.6439999938011169,0.07900000363588333,0.5899999737739563,0.5460000038146973,0.4189999997615814,0.37826088070869446,0.8840000033378601,0.4009999930858612,0.5249999761581421,0.7950000166893005,0.7833020687103271,0.6489999890327454,0.5049999952316284,0.6159999966621399,0.7480000257492065,0.8059999942779541,0.6959999799728394,0.7919999957084656,0.6079999804496765,0.9459999799728394
+Qwen3-Reranker-8B,0.8019999861717224,0.47164177894592285,0.753000020980835,0.9570000171661377,0.9290000200271606,0.6740000247955322,0.7549999952316284,0.7760000228881836,0.8240000009536743,0.6809999942779541,0.5609999895095825,0.48899999260902405,0.8449275493621826,0.949999988079071,0.4560000002384186,0.7160000205039978,0.8100000023841858,0.903377115726471,0.7400000095367432,0.5170000195503235,0.6539999842643738,0.8220000267028809,0.8790000081062317,0.7799999713897705,0.8569999933242798,0.6710000038146973,0.9819999933242798
+all-MiniLM-L6-v2,0.5,0.17313432693481445,0.47099998593330383,0.48899999260902405,0.5320000052452087,0.4410000145435333,0.5289999842643738,0.5049999952316284,0.7730000019073486,0.5289999842643738,0.1080000028014183,0.17299999296665192,0.4231884181499481,0.49300000071525574,0.3019999861717224,0.33799999952316284,0.6579999923706055,0.4990619122982025,0.4909999966621399,0.49000000953674316,0.39800000190734863,0.4099999964237213,0.503000020980835,0.3009999990463257,0.5329999923706055,0.38199999928474426,0.4869999885559082
+e5-base-v2,0.7680000066757202,0.3283582031726837,0.5509999990463257,0.9259999990463257,0.9269999861717224,0.6200000047683716,0.6190000176429749,0.6169999837875366,0.2070000022649765,0.5889999866485596,0.5059999823570251,0.4099999964237213,0.4275362193584442,0.8980000019073486,0.3569999933242798,0.4860000014305115,0.8360000252723694,0.8367729783058167,0.6504999995231628,0.4869999885559082,0.597000002861023,0.7429999709129333,0.6460000276565552,0.3610000014305115,0.6790000200271606,0.6179999709129333,0.9509999752044678
+e5-large-v2,0.7929999828338623,0.6626865863800049,0.5849999785423279,0.9430000185966492,0.9100000262260437,0.5630000233650208,0.5440000295639038,0.5210000276565552,0.49799999594688416,0.5479999780654907,0.49000000953674316,0.4320000112056732,0.4492753744125366,0.9279999732971191,0.33899998664855957,0.5099999904632568,0.8349999785423279,0.8545966148376465,0.6100000143051147,0.47999998927116394,0.5920000076293945,0.5350000262260437,0.6639999747276306,0.3330000042915344,0.6949999928474426,0.5619999766349792,0.9760000109672546
+e5-mistral-7b-instruct,0.7799999713897705,0.23582088947296143,0.7059999704360962,0.9399999976158142,0.9319999814033508,0.6499999761581421,0.6779999732971191,0.6019999980926514,0.08100000023841858,0.6190000176429749,0.5450000166893005,0.5389999747276306,0.573913037776947,0.9150000214576721,0.5,0.6679999828338623,0.8809999823570251,0.8442776799201965,0.6589999794960022,0.48100000619888306,0.6430000066757202,0.4620000123977661,0.43299999833106995,0.0949999988079071,0.5059999823570251,0.7120000123977661,0.9829999804496765
+bge-base-en-v1.5,0.6489999890327454,0.5134328603744507,0.5419999957084656,0.9300000071525574,0.9039999842643738,0.6449999809265137,0.5820000171661377,0.6039999723434448,0.1679999977350235,0.5789999961853027,0.5249999761581421,0.3490000069141388,0.3942028880119324,0.8970000147819519,0.33500000834465027,0.5619999766349792,0.8320000171661377,0.8142589330673218,0.640999972820282,0.4819999933242798,0.6119999885559082,0.49900001287460327,0.6819999814033508,0.17399999499320984,0.7260000109672546,0.5929999947547913,0.9440000057220459
+bge-large-en-v1.5,0.7710000276565552,0.3910447657108307,0.5809999704360962,0.9480000138282776,0.9190000295639038,0.6769999861717224,0.5789999961853027,0.5669999718666077,0.12800000607967377,0.6050000190734863,0.5450000166893005,0.38600000739097595,0.4072463810443878,0.9350000023841858,0.42500001192092896,0.5249999761581421,0.8759999871253967,0.8245778679847717,0.6934999823570251,0.4729999899864197,0.6240000128746033,0.4880000054836273,0.4959999918937683,0.11100000143051147,0.5659999847412109,0.6349999904632568,0.9509999752044678
+gte-base-en-v1.5,0.7580000162124634,0.4298507571220398,0.6179999709129333,0.8989999890327454,0.9290000200271606,0.6489999890327454,0.597000002861023,0.6399999856948853,0.10100000351667404,0.5899999737739563,0.49900001287460327,0.35899999737739563,0.4101449251174927,0.8500000238418579,0.3580000102519989,0.5410000085830688,0.8870000243186951,0.8433395624160767,0.7070000171661377,0.4830000102519989,0.6269999742507935,0.6209999918937683,0.7419999837875366,0.23499999940395355,0.6710000038146973,0.6200000047683716,0.972000002861023
+gte-large-en-v1.5,0.75,0.34029850363731384,0.6330000162124634,0.9520000219345093,0.9070000052452087,0.6320000290870667,0.5649999976158142,0.47699999809265137,0.2160000056028366,0.5899999737739563,0.453000009059906,0.3610000014305115,0.47101449966430664,0.9419999718666077,0.3889999985694885,0.5809999704360962,0.8560000061988831,0.8733583688735962,0.8115000128746033,0.44999998807907104,0.6620000004768372,0.7519999742507935,0.8159999847412109,0.4690000116825104,0.8240000009536743,0.6150000095367432,0.9309999942779541
+gte-modernbert-base,0.7580000162124634,0.32238805294036865,0.6119999885559082,0.9520000219345093,0.9229999780654907,0.6320000290870667,0.6259999871253967,0.6320000290870667,0.13199999928474426,0.5180000066757202,0.5,0.39800000190734863,0.6376811861991882,0.9089999794960022,0.38499999046325684,0.6489999890327454,0.8790000081062317,0.8189493417739868,0.7404999732971191,0.4749999940395355,0.6679999828338623,0.6349999904632568,0.546999990940094,0.15700000524520874,0.5519999861717224,0.6029999852180481,0.9589999914169312
+Qwen3-Embedding-0.6B,0.6800000071525574,0.5373134613037109,0.597000002861023,0.902999997138977,0.8690000176429749,0.6389999985694885,0.527999997138977,0.5630000233650208,0.3779999911785126,0.546999990940094,0.527999997138977,0.4020000100135803,0.4463768005371094,0.8809999823570251,0.40400001406669617,0.6299999952316284,0.8690000176429749,0.7607879638671875,0.6909999847412109,0.4729999899864197,0.5830000042915344,0.5460000038146973,0.5509999990463257,0.382999986410141,0.6740000247955322,0.6010000109672546,0.9620000123977661
+Qwen3-Embedding-8B,0.7839999794960022,0.23582088947296143,0.6949999928474426,0.9409999847412109,0.9229999780654907,0.699999988079071,0.5569999814033508,0.6389999985694885,0.30300000309944153,0.5249999761581421,0.5910000205039978,0.49900001287460327,0.6884058117866516,0.9470000267028809,0.46799999475479126,0.7120000123977661,0.9139999747276306,0.8611631989479065,0.7319999933242798,0.5090000033378601,0.5260000228881836,0.4099999964237213,0.46799999475479126,0.3019999861717224,0.5580000281333923,0.640999972820282,0.9599999785423279
+gemma-3-270m-it,0.23800000548362732,0.34328359365463257,0.0689999982714653,0.5080000162124634,0.5149999856948853,0.010999999940395355,0.4099999964237213,0.5860000252723694,0.28700000047683716,0.08900000154972076,0.10700000077486038,0.03799999877810478,0.269565224647522,0.5239999890327454,0.003000000026077032,0.008999999612569809,0.0729999989271164,0.5037523508071899,0.5009999871253967,0.5049999952316284,0.31700000166893005,0.4869999885559082,0.5120000243186951,0.22100000083446503,0.5419999957084656,0.07500000298023224,0.4740000069141388
+gemma-3-1b-it,0.3930000066757202,0.3253731429576874,0.19699999690055847,0.5929999947547913,0.6909999847412109,0.14399999380111694,0.5569999814033508,0.5249999761581421,0.5789999961853027,0.19599999487400055,0.1860000044107437,0.2150000035762787,0.2507246434688568,0.609000027179718,0.04399999976158142,0.08900000154972076,0.2720000147819519,0.6013132929801941,0.5350000262260437,0.46399998664855957,0.4309999942779541,0.5220000147819519,0.5680000185966492,0.2240000069141388,0.5619999766349792,0.1940000057220459,0.6029999852180481
+Llama-3.2-3B-Instruct,0.6769999861717224,0.35223880410194397,0.5379999876022339,0.5419999957084656,0.5410000085830688,0.40799999237060547,0.4699999988079071,0.6029999852180481,0.34599998593330383,0.4970000088214874,0.46299999952316284,0.39800000190734863,0.30000001192092896,0.5370000004768372,0.16899999976158142,0.36800000071525574,0.5479999780654907,0.5112570524215698,0.5180000066757202,0.4970000088214874,0.43799999356269836,0.5040000081062317,0.5299999713897705,0.29100000858306885,0.5550000071525574,0.5040000081062317,0.47699999809265137
+Qwen3-4B,0.824999988079071,0.8716418147087097,0.6510000228881836,0.9449999928474426,0.9169999957084656,0.3959999978542328,0.5759999752044678,0.7139999866485596,0.828000009059906,0.5950000286102295,0.5440000295639038,0.33899998664855957,0.6159420013427734,0.9240000247955322,0.2939999997615814,0.4000000059604645,0.7850000262260437,0.8677298426628113,0.5625,0.5090000033378601,0.656000018119812,0.8550000190734863,0.8949999809265137,0.8820000290870667,0.8700000047683716,0.5580000281333923,0.984000027179718
+Phi-4-mini-instruct,0.5619999766349792,0.35223880410194397,0.5080000162124634,0.6150000095367432,0.6029999852180481,0.42500001192092896,0.43700000643730164,0.640999972820282,0.2840000092983246,0.45399999618530273,0.33899998664855957,0.3140000104904175,0.22028985619544983,0.6129999756813049,0.2980000078678131,0.31200000643730164,0.5220000147819519,0.5834896564483643,0.5034999847412109,0.503000020980835,0.4359999895095825,0.5540000200271606,0.5509999990463257,0.28200000524520874,0.6010000109672546,0.4519999921321869,0.597000002861023
+Qwen3-8B,0.8529999852180481,0.7104477882385254,0.652999997138977,0.9449999928474426,0.9179999828338623,0.39500001072883606,0.6959999799728394,0.746999979019165,0.8550000190734863,0.5690000057220459,0.5360000133514404,0.25999999046325684,0.7550724744796753,0.9380000233650208,0.289000004529953,0.414000004529953,0.8240000009536743,0.8592870831489563,0.5605000257492065,0.5059999823570251,0.6660000085830688,0.8410000205039978,0.890999972820282,0.9139999747276306,0.8600000143051147,0.5770000219345093,0.9829999804496765
+Mistral-Nemo-Instruct-2407,0.8360000252723694,0.8537313342094421,0.6629999876022339,0.9440000057220459,0.8989999890327454,0.3880000114440918,0.6930000185966492,0.7879999876022339,0.9549999833106995,0.6010000109672546,0.5529999732971191,0.3179999887943268,0.7115942239761353,0.9490000009536743,0.2630000114440918,0.3880000114440918,0.7570000290870667,0.6078799366950989,0.5764999985694885,0.5370000004768372,0.6460000276565552,0.8859999775886536,0.9129999876022339,0.9629999995231628,0.8479999899864197,0.6549999713897705,0.9679999947547913

btzsc-0.1.1/src/btzsc/_baselines/f1_scores.csv

@@ -0,0 +1,39 @@
+mdl,agnews,amazon_counterfactual,amazon_massive_scenario,amazonpolarity,appreviews,banking77,biasframes_intent,biasframes_offensive,biasframes_sex,capsotu,emotiondair,empathetic,financialphrasebank,imdb,manifesto,massive,mtop_domain,rottentomatoes,toxic_conversations_50k,trueteacher,tweet_sentiment_extraction,wikitoxic_insult,wikitoxic_obscene,wikitoxic_threat,wikitoxic_toxicaggregated,yahootopics,yelpreviews
+bert-large-uncased,0.1733286827802658,0.24903321266174316,0.03296012431383133,0.4174703061580658,0.3561461567878723,0.018178250640630722,0.4225161075592041,0.48243170976638794,0.4775339663028717,0.029290905222296715,0.15561828017234802,0.004449524451047182,0.28309521079063416,0.3809039890766144,0.010123563930392265,0.00435608671978116,0.07880379259586334,0.470112681388855,0.44223901629447937,0.330655962228775,0.40998488664627075,0.513670027256012,0.5768487453460693,0.48799997568130493,0.5338317155838013,0.07909613102674484,0.3607291579246521
+deberta-v3-large,0.17695730924606323,0.5076156854629517,0.022733645513653755,0.32750505208969116,0.3674790859222412,0.013612989336252213,0.4524354040622711,0.49342963099479675,0.12474705278873444,0.011082805693149567,0.09953004121780396,0.001996611710637808,0.30083930492401123,0.32750505208969116,0.014533760026097298,0.00026939655072055757,0.06766538321971893,0.348673939704895,0.48917675018310547,0.335989385843277,0.31185320019721985,0.5521674156188965,0.5574002265930176,0.5052158236503601,0.5285928845405579,0.025485241785645485,0.3683379292488098
+ModernBERT-large,0.0974235087633133,0.45261436700820923,0.07167492061853409,0.327052503824234,0.4414460361003876,0.024128366261720657,0.38095057010650635,0.5311391353607178,0.46369731426239014,0.010429216548800468,0.05838332325220108,0.00730061624199152,0.3026798963546753,0.3390614688396454,0.006052697543054819,0.01675228402018547,0.10530678927898407,0.35531824827194214,0.427348792552948,0.3628002405166626,0.16465863585472107,0.4587130546569824,0.5835974216461182,0.13308370113372803,0.6016994714736938,0.0799928605556488,0.4339780807495117
+bart-large-mnli,0.7090643048286438,0.15155631303787231,0.5405699014663696,0.9329510927200317,0.9209651350975037,0.2827707529067993,0.6016557216644287,0.3668980002403259,0.07344245910644531,0.33400794863700867,0.43902286887168884,0.384743332862854,0.46504926681518555,0.9289840459823608,0.08716775476932526,0.411138653755188,0.8368948698043823,0.8316553235054016,0.49905434250831604,0.5149956345558167,0.537959098815918,0.3302880525588989,0.689961314201355,0.0839853435754776,0.5524932146072388,0.27278462052345276,0.9589999914169312
+nli-roberta-base,0.693808913230896,0.21881893277168274,0.24948441982269287,0.886974573135376,0.8939982652664185,0.06683805584907532,0.5059967637062073,0.46795955300331116,0.18175822496414185,0.13537120819091797,0.3415570557117462,0.31791791319847107,0.503407895565033,0.8319670557975769,0.017918219789862633,0.32371756434440613,0.410012423992157,0.798117995262146,0.6550894975662231,0.45594465732574463,0.5189624428749084,0.5442176461219788,0.7121915817260742,0.13583959639072418,0.6956897974014282,0.34989988803863525,0.8856909275054932
+bert-base-uncased-nli,0.6833652257919312,0.19995658099651337,0.23329594731330872,0.862488329410553,0.8378125429153442,0.011304003186523914,0.5621905326843262,0.5668436288833618,0.4279535114765167,0.011852134019136429,0.36164164543151855,0.15012234449386597,0.4313048720359802,0.7927390336990356,0.001060847076587379,0.32235872745513916,0.6408779621124268,0.754468560218811,0.5232468247413635,0.3365929126739502,0.5859357118606567,0.7606723308563232,0.7538200616836548,0.26134955883026123,0.611831545829773,0.33737125992774963,0.8869591951370239
+bert-large-uncased-nli,0.7425510883331299,0.4157741665840149,0.4016559422016144,0.83831787109375,0.8546218872070312,0.07846608012914658,0.6058075428009033,0.5404279828071594,0.5755754113197327,0.1265260875225067,0.4244973361492157,0.12168917059898376,0.6403998136520386,0.8048792481422424,0.01616549678146839,0.375482976436615,0.4192730784416199,0.7067925333976746,0.46734923124313354,0.39294764399528503,0.6102933287620544,0.814115047454834,0.7814091444015503,0.5536200404167175,0.6412836313247681,0.21358641982078552,0.8967098593711853
+bert-large-uncased-nli-triplet,0.7285709381103516,0.384902685880661,0.3063317835330963,0.8373348712921143,0.8499616384506226,0.062161438167095184,0.6038716435432434,0.6369782090187073,0.4285886883735657,0.13261830806732178,0.2828684151172638,0.1955133080482483,0.5776402950286865,0.7798857092857361,0.018068579956889153,0.36915647983551025,0.49423226714134216,0.7154526710510254,0.5297268033027649,0.36477410793304443,0.6139540672302246,0.7261716723442078,0.8325483202934265,0.39473915100097656,0.7741836905479431,0.3332909643650055,0.8968430757522583
+deberta-v3-base-nli,0.7646851539611816,0.14758269488811493,0.4771553575992584,0.89799964427948,0.9099708199501038,0.12482473254203796,0.4608410596847534,0.5039682388305664,0.4438576102256775,0.16506196558475494,0.3779505789279938,0.28930890560150146,0.67685467004776,0.9139721393585205,0.03519533574581146,0.3414219319820404,0.675718367099762,0.8234999775886536,0.6914591789245605,0.3327486515045166,0.5394366979598999,0.6774438619613647,0.8274722099304199,0.37141063809394836,0.7829363942146301,0.45720019936561584,0.9299719333648682
+deberta-v3-large-nli,0.814592182636261,0.15943224728107452,0.5097894072532654,0.9227490425109863,0.9279815554618835,0.34563589096069336,0.6648926734924316,0.5448735952377319,0.27395033836364746,0.2141467034816742,0.440132737159729,0.43911874294281006,0.8026540279388428,0.9023271799087524,0.05605190992355347,0.4177260398864746,0.7432827353477478,0.8481759428977966,0.7266356945037842,0.3410828411579132,0.6207084059715271,0.6007736921310425,0.8187779784202576,0.31557148694992065,0.7974669933319092,0.5326335430145264,0.9829909801483154
+deberta-v3-large-nli-triplet,0.8319669365882874,0.3432309031486511,0.5623114109039307,0.9288427829742432,0.929977297782898,0.23813845217227936,0.6989635825157166,0.6438407301902771,0.2731889486312866,0.24564850330352783,0.4229159951210022,0.41471126675605774,0.7889612913131714,0.9268383979797363,0.059760164469480515,0.4035292863845825,0.4949427843093872,0.8374935388565063,0.7164756059646606,0.4128504991531372,0.5758254528045654,0.6911536455154419,0.8422045707702637,0.4374711513519287,0.8225740194320679,0.2758510410785675,0.980995774269104
+modernbert-base-nli,0.7388247847557068,0.6130094528198242,0.3452880382537842,0.9107679128646851,0.907821536064148,0.10325706750154495,0.4637435972690582,0.5483788847923279,0.6421589255332947,0.11951573193073273,0.2848603427410126,0.29507842659950256,0.6012195348739624,0.890692949295044,0.015092020854353905,0.2466864287853241,0.45006126165390015,0.7408660054206848,0.6833003759384155,0.4156147837638855,0.5385338068008423,0.7499749660491943,0.6825619339942932,0.2855295240879059,0.7495993375778198,0.4015524387359619,0.9599224328994751
+modernbert-large-nli,0.7555487155914307,0.4157741665840149,0.41822999715805054,0.9326210021972656,0.9196913242340088,0.21216264367103577,0.6310776472091675,0.4208526611328125,0.4431055784225464,0.03901638463139534,0.2996448278427124,0.30141282081604004,0.5391452312469482,0.9136237502098083,0.0924706980586052,0.3598971664905548,0.592676043510437,0.8638569712638855,0.6931955814361572,0.3737006187438965,0.6355735063552856,0.5945841073989868,0.8074331283569336,0.4512676000595093,0.7248871326446533,0.4827136993408203,0.9789742231369019
+modernbert-large-nli-triplet,0.706705629825592,0.3895684480667114,0.5005460977554321,0.9287685751914978,0.9197109937667847,0.2112514078617096,0.5957847833633423,0.4505764842033386,0.344501793384552,0.05233711004257202,0.3704449534416199,0.31423282623291016,0.6493753790855408,0.9105490446090698,0.05937906354665756,0.421375036239624,0.6449679732322693,0.8706203699111938,0.6995192170143127,0.40925976634025574,0.6632700562477112,0.6689207553863525,0.8030894994735718,0.4223605990409851,0.7348260283470154,0.24883794784545898,0.9789847135543823
+ms-marco-MiniLM-L6-v2,0.39791375398635864,0.5258561372756958,0.08180268853902817,0.6817625761032104,0.7086011171340942,0.23708733916282654,0.6136585474014282,0.6154738664627075,0.46941402554512024,0.29766881465911865,0.20086175203323364,0.1828111708164215,0.27647721767425537,0.6169463396072388,0.07393017411231995,0.05297502502799034,0.18642884492874146,0.5772273540496826,0.525679349899292,0.3914633095264435,0.3213481307029724,0.5008226633071899,0.5130991339683533,0.45319265127182007,0.4693637490272522,0.298456072807312,0.6500681638717651
+gte-reranker-modernbert-base,0.6849942207336426,0.5595477223396301,0.5342063307762146,0.9113131761550903,0.916504979133606,0.6466980576515198,0.35648372769355774,0.5220422744750977,0.5966007709503174,0.4171936810016632,0.45084241032600403,0.3935551643371582,0.4956844449043274,0.8373899459838867,0.17288586497306824,0.5390387177467346,0.5900932550430298,0.7976751327514648,0.5407333970069885,0.5798214077949524,0.5292630791664124,0.5348722338676453,0.5600804090499878,0.4324122369289398,0.5468690395355225,0.3708208203315735,0.9629937410354614
+bge-reranker-base,0.632666826248169,0.3936595022678375,0.438944011926651,0.6631579399108887,0.7903658151626587,0.47857925295829773,0.4995477795600891,0.4740513563156128,0.3038731813430786,0.39848554134368896,0.2955600917339325,0.2894975244998932,0.35110652446746826,0.6949630975723267,0.1635625809431076,0.42827051877975464,0.7298668622970581,0.5743533372879028,0.5649909377098083,0.4728476405143738,0.530536413192749,0.4096657335758209,0.54896080493927,0.2721618413925171,0.5650734901428223,0.40837764739990234,0.6496093273162842
+bge-reranker-large,0.726889431476593,0.4591308832168579,0.4808361530303955,0.8699994683265686,0.8909868001937866,0.5650694370269775,0.5915988683700562,0.529961347579956,0.20037832856178284,0.47877487540245056,0.3911106586456299,0.34434404969215393,0.48737820982933044,0.8030894994735718,0.16162124276161194,0.4536345899105072,0.8051106333732605,0.7559291124343872,0.6296368837356567,0.4441928267478943,0.5212429761886597,0.37854886054992676,0.5302799940109253,0.1538538932800293,0.6028369069099426,0.5268266797065735,0.8786844611167908
+Qwen3-Reranker-0.6B,0.7880467772483826,0.25821834802627563,0.6567176580429077,0.9119647741317749,0.8867498636245728,0.6274953484535217,0.49556970596313477,0.5712165832519531,0.07787039130926132,0.528754472732544,0.4866149127483368,0.4076511263847351,0.4138122797012329,0.8839995265007019,0.2676747143268585,0.5314709544181824,0.7903094291687012,0.7832179069519043,0.6423856019973755,0.33554816246032715,0.5129673480987549,0.7408590912818909,0.7990478277206421,0.5021486282348633,0.7912979125976562,0.5461462736129761,0.9459965229034424
+Qwen3-Reranker-8B,0.7875484228134155,0.46236705780029297,0.7662825584411621,0.9569586515426636,0.9287347793579102,0.6905889511108398,0.737960934638977,0.7650375962257385,0.6409658193588257,0.6593928337097168,0.48749345541000366,0.4841960072517395,0.8174570798873901,0.9499607682228088,0.3251327574253082,0.6729446649551392,0.8012257814407349,0.903188943862915,0.7396240234375,0.3602437376976013,0.619925320148468,0.8217689990997314,0.8774905204772949,0.5778453350067139,0.856735110282898,0.610032320022583,0.9819982051849365
+all-MiniLM-L6-v2,0.4948541224002838,0.14758269488811493,0.47943219542503357,0.35260650515556335,0.4131661653518677,0.43356677889823914,0.46536508202552795,0.47554171085357666,0.5124979019165039,0.479082316160202,0.11101524531841278,0.1542579084634781,0.31196534633636475,0.3403925895690918,0.1498693972826004,0.33350270986557007,0.6331771016120911,0.3378419876098633,0.41617462038993835,0.395149827003479,0.39578700065612793,0.32215073704719543,0.5017039179801941,0.26442036032676697,0.5129282474517822,0.36257949471473694,0.32750505208969116
+e5-base-v2,0.7608605623245239,0.32826244831085205,0.5467093586921692,0.9257502555847168,0.926961362361908,0.6155748963356018,0.6012471318244934,0.5871238708496094,0.19746507704257965,0.5291017293930054,0.4324241578578949,0.3714253604412079,0.46327078342437744,0.8977016806602478,0.20859912037849426,0.4725361466407776,0.8123351335525513,0.8367706537246704,0.6366112232208252,0.43589115142822266,0.5437307357788086,0.7393083572387695,0.6416641473770142,0.30706092715263367,0.671910285949707,0.5515064597129822,0.9509889483451843
+e5-large-v2,0.7865350246429443,0.5692862272262573,0.5771253705024719,0.9428738355636597,0.9099823236465454,0.583658754825592,0.5438522100448608,0.5198471546173096,0.38923200964927673,0.504774808883667,0.44059354066848755,0.3835121989250183,0.5209692120552063,0.9277734756469727,0.21668392419815063,0.5100078582763672,0.801220178604126,0.85456782579422,0.5744583010673523,0.46867427229881287,0.5123862028121948,0.5073270797729492,0.6598845720291138,0.28966307640075684,0.6889909505844116,0.5175836682319641,0.9759995937347412
+e5-mistral-7b-instruct,0.7702938318252563,0.22738739848136902,0.6966646313667297,0.9398496150970459,0.931954026222229,0.6527521014213562,0.6684172749519348,0.46933332085609436,0.07987284660339355,0.6181511282920837,0.49798208475112915,0.5011402368545532,0.6228639483451843,0.9142584204673767,0.30002036690711975,0.6269175410270691,0.8699702024459839,0.8442332744598389,0.6374130249023438,0.4197064936161041,0.59898841381073,0.39535343647003174,0.3256414532661438,0.09495565295219421,0.3689675033092499,0.6368180513381958,0.9829979538917542
+bge-base-en-v1.5,0.6345731616020203,0.46789640188217163,0.538567304611206,0.9296748638153076,0.9033816456794739,0.6375364065170288,0.5726391077041626,0.5659722089767456,0.16457810997962952,0.5365762710571289,0.42592093348503113,0.29656127095222473,0.4313793182373047,0.8964173793792725,0.20281097292900085,0.5291436910629272,0.8235862851142883,0.8140226006507874,0.6097448468208313,0.4814004898071289,0.5794270038604736,0.4553821086883545,0.6794031858444214,0.16755354404449463,0.7208366394042969,0.5134701132774353,0.9439979791641235
+bge-large-en-v1.5,0.7664446234703064,0.38907960057258606,0.5628271698951721,0.9479948282241821,0.9189493656158447,0.6825442910194397,0.5388701558113098,0.46127596497535706,0.12777671217918396,0.5617057085037231,0.4404083490371704,0.34858500957489014,0.4633880853652954,0.9349526166915894,0.259449303150177,0.52516108751297,0.8636317253112793,0.8233464956283569,0.6934938430786133,0.3989500403404236,0.5672269463539124,0.43661725521087646,0.43688660860061646,0.11035144329071045,0.49887073040008545,0.5731387138366699,0.9509975910186768
+gte-base-en-v1.5,0.7503072619438171,0.4252414107322693,0.6141042709350586,0.8979170322418213,0.9289840459823608,0.6550149321556091,0.596967339515686,0.6292864084243774,0.10084804892539978,0.5663331747055054,0.42110711336135864,0.322726309299469,0.46603572368621826,0.8459572792053223,0.21299287676811218,0.5118401050567627,0.8833382725715637,0.8423840999603271,0.6947916746139526,0.34951454401016235,0.5641905665397644,0.6119153499603271,0.7413533926010132,0.21763062477111816,0.6507130861282349,0.559667706489563,0.9719838500022888
+gte-large-en-v1.5,0.7396283149719238,0.34029850363731384,0.6212527751922607,0.9519808292388916,0.9069663882255554,0.6315518617630005,0.5647697448730469,0.46699607372283936,0.20775750279426575,0.5492952466011047,0.4007812738418579,0.34420278668403625,0.4902401566505432,0.9419607520103455,0.2757111191749573,0.5730605721473694,0.8483951687812805,0.873044490814209,0.8113773465156555,0.40464848279953003,0.6052703857421875,0.7517458200454712,0.8159528970718384,0.3788304030895233,0.8219228982925415,0.5612474679946899,0.9308948516845703
+gte-modernbert-base,0.7550400495529175,0.3220013976097107,0.5985389947891235,0.9518982172012329,0.9229937791824341,0.6365822553634644,0.6256166696548462,0.5928753018379211,0.1315796822309494,0.4834834933280945,0.4404637813568115,0.3896840214729309,0.6445218324661255,0.9087033867835999,0.24006304144859314,0.6110821962356567,0.8603191375732422,0.8174377679824829,0.7400736808776855,0.47455811500549316,0.6525988578796387,0.6288232803344727,0.5205207467079163,0.15296727418899536,0.4754565358161926,0.5372447967529297,0.9589908123016357
+Qwen3-Embedding-0.6B,0.6595149636268616,0.49957114458084106,0.5716717839241028,0.9028522372245789,0.8688739538192749,0.6418111324310303,0.4549930691719055,0.524954080581665,0.32174259424209595,0.5322523713111877,0.47812420129776,0.38395485281944275,0.488581120967865,0.8802140951156616,0.24359840154647827,0.5916567444801331,0.8522663116455078,0.7592573165893555,0.6852639317512512,0.47297418117523193,0.5540332794189453,0.5197007060050964,0.5426650643348694,0.32418739795684814,0.6551154851913452,0.5451348423957825,0.9619925618171692
+Qwen3-Embedding-8B,0.7707853317260742,0.226383775472641,0.6991231441497803,0.9409787058830261,0.922999918460846,0.71482253074646,0.398776650428772,0.5426437258720398,0.26442575454711914,0.5475385785102844,0.5401405692100525,0.473588228225708,0.7176883816719055,0.9469107389450073,0.3240036964416504,0.6541801691055298,0.9084205627441406,0.8608060479164124,0.72392737865448,0.4126618504524231,0.5045839548110962,0.30670127272605896,0.38571250438690186,0.2685002386569977,0.46657270193099976,0.5887834429740906,0.9599921703338623
+gemma-3-270m-it,0.17617017030715942,0.33761143684387207,0.02690325677394867,0.46282345056533813,0.48126956820487976,0.002360794460400939,0.3849015235900879,0.5426101684570312,0.2550809681415558,0.0343693271279335,0.06215536221861839,0.012273683212697506,0.2062343955039978,0.4741121828556061,0.00011458253720775247,0.0016255928203463554,0.049714524298906326,0.4736323356628418,0.46600407361984253,0.4440944492816925,0.24197393655776978,0.4711727499961853,0.4925040304660797,0.20182383060455322,0.5071262121200562,0.03481292724609375,0.42642542719841003
+gemma-3-1b-it,0.3620033264160156,0.32488587498664856,0.17791514098644257,0.5177582502365112,0.6798030138015747,0.10407334566116333,0.5240811705589294,0.480392187833786,0.4423656463623047,0.16142979264259338,0.130890890955925,0.15641769766807556,0.24874034523963928,0.5621848702430725,0.007554742507636547,0.07866360247135162,0.22999358177185059,0.5737664699554443,0.5196077823638916,0.3920586407184601,0.36968058347702026,0.4981500804424286,0.5521719455718994,0.20803073048591614,0.515433132648468,0.17090842127799988,0.5339144468307495
+Llama-3.2-3B-Instruct,0.6745220422744751,0.34627386927604675,0.5272970795631409,0.5109429359436035,0.5057665705680847,0.39238879084587097,0.46464645862579346,0.5662714242935181,0.29700398445129395,0.4401363432407379,0.32957401871681213,0.36628174781799316,0.2830798029899597,0.5104212760925293,0.10245990008115768,0.3679518401622772,0.5484113693237305,0.4867360293865204,0.5020450353622437,0.4631797969341278,0.36277955770492554,0.49625441431999207,0.5200477242469788,0.2524811029434204,0.5338513255119324,0.4614385962486267,0.4381364583969116
+Qwen3-4B,0.822451114654541,0.8027279376983643,0.6516287326812744,0.9448078870773315,0.9166899919509888,0.37111204862594604,0.4490644335746765,0.6946063041687012,0.6385217905044556,0.5471009016036987,0.4271164834499359,0.3184030055999756,0.6628990769386292,0.9235104322433472,0.16602693498134613,0.38182488083839417,0.7752480506896973,0.8675339221954346,0.4646639823913574,0.34269624948501587,0.611158013343811,0.8536794781684875,0.8922058343887329,0.6722221970558167,0.8698828816413879,0.5222913026809692,0.9839959144592285
+Phi-4-mini-instruct,0.5731089115142822,0.3494330644607544,0.5251995325088501,0.5550732612609863,0.5450122356414795,0.4160025417804718,0.4069782495498657,0.5785272121429443,0.2553952634334564,0.4624970257282257,0.2960258722305298,0.30447065830230713,0.22743332386016846,0.5613990426063538,0.14805404841899872,0.2843197286128998,0.5550087690353394,0.5285159349441528,0.46997469663619995,0.4678521752357483,0.3526347577571869,0.5313156843185425,0.5347299575805664,0.2527029812335968,0.5594816207885742,0.4667598307132721,0.5233877301216125
+Qwen3-8B,0.8517557382583618,0.6572293639183044,0.6630200147628784,0.9448207020759583,0.9175726771354675,0.38651391863822937,0.672095775604248,0.7154461145401001,0.6741390228271484,0.5280041694641113,0.4043771028518677,0.2438962459564209,0.7512251138687134,0.9378049373626709,0.18233036994934082,0.38596004247665405,0.8239142298698425,0.8576008677482605,0.4610864818096161,0.335989385843277,0.6238663196563721,0.8404061794281006,0.8883872628211975,0.7205759882926941,0.8597756624221802,0.5463981628417969,0.9829938411712646
+Mistral-Nemo-Instruct-2407,0.8372315168380737,0.7275111675262451,0.6434688568115234,0.9440000057220459,0.8988128900527954,0.3519081473350525,0.6547112464904785,0.7859442234039307,0.8135557174682617,0.5104762315750122,0.4354870319366455,0.2901332378387451,0.748801052570343,0.9489974975585938,0.1788283735513687,0.35859915614128113,0.7514673471450806,0.539020836353302,0.49413973093032837,0.409269243478775,0.6139224767684937,0.881150484085083,0.911335825920105,0.8293161392211914,0.8477563858032227,0.5907819867134094,0.9679994583129883

btzsc-0.1.1/src/btzsc/_baselines/precision_scores.csv

@@ -0,0 +1,39 @@
+mdl,agnews,amazon_counterfactual,amazon_massive_scenario,amazonpolarity,appreviews,banking77,biasframes_intent,biasframes_offensive,biasframes_sex,capsotu,emotiondair,empathetic,financialphrasebank,imdb,manifesto,massive,mtop_domain,rottentomatoes,toxic_conversations_50k,trueteacher,tweet_sentiment_extraction,wikitoxic_insult,wikitoxic_obscene,wikitoxic_threat,wikitoxic_toxicaggregated,yahootopics,yelpreviews
+bert-large-uncased,0.18101249635219574,0.4900861382484436,0.03858058899641037,0.45232418179512024,0.6707152128219604,0.024572160094976425,0.44649267196655273,0.5492086410522461,0.47016459703445435,0.03952052816748619,0.19275575876235962,0.017377490177750587,0.3822284936904907,0.5534566044807434,0.027149589732289314,0.0035814912989735603,0.11032882332801819,0.5296506881713867,0.5969555974006653,0.24699999392032623,0.46136003732681274,0.532223105430603,0.576836109161377,0.5120319724082947,0.5577073097229004,0.10962779074907303,0.5814134478569031
+deberta-v3-large,0.3849303424358368,0.512172281742096,0.07809656858444214,0.2434999942779541,0.5810298919677734,0.017147816717624664,0.49129319190979004,0.5624772310256958,0.5025225281715393,0.008429675363004208,0.09975472092628479,0.0010312499944120646,0.3182830214500427,0.2434999942779541,0.016682803630828857,0.0001436781749362126,0.07087355852127075,0.5317261815071106,0.5057071447372437,0.2529999911785126,0.3195223808288574,0.6584779024124146,0.661960780620575,0.509821355342865,0.6169651746749878,0.037696730345487595,0.5845311880111694
+ModernBERT-large,0.060499999672174454,0.4134328365325928,0.07754817605018616,0.2432432472705841,0.4940873980522156,0.047780826687812805,0.587581217288971,0.5540754795074463,0.49897047877311707,0.015122896060347557,0.06987837702035904,0.0055073220282793045,0.29453587532043457,0.2565000057220459,0.021486753597855568,0.04419608414173126,0.22332754731178284,0.5,0.49322986602783203,0.5316318273544312,0.109333336353302,0.4683747887611389,0.583702564239502,0.49720510840415955,0.6097927093505859,0.06552055478096008,0.5610920190811157
+bart-large-mnli,0.7674412727355957,0.5868263244628906,0.6679467558860779,0.9329754114151001,0.9213510751724243,0.47028210759162903,0.6472827196121216,0.7840681076049805,0.5295014977455139,0.5878989696502686,0.4808715879917145,0.4803498387336731,0.6390920877456665,0.9289548397064209,0.217774897813797,0.5630237460136414,0.8441347479820251,0.8441861867904663,0.6812313795089722,0.5150365233421326,0.6437610983848572,0.6248708963394165,0.7562661170959473,0.5229166746139526,0.7446534633636475,0.5333313345909119,0.9597244262695312
+nli-roberta-base,0.7239819169044495,0.5259244441986084,0.43887677788734436,0.8889477252960205,0.8941649198532104,0.13073302805423737,0.531976044178009,0.5809751749038696,0.5289124250411987,0.25667986273765564,0.4731428325176239,0.4316464066505432,0.6112631559371948,0.8319753408432007,0.0878804549574852,0.4719012379646301,0.4628889560699463,0.7994593381881714,0.6562439203262329,0.4758465588092804,0.6056304574012756,0.6994266510009766,0.7184909582138062,0.5242825746536255,0.7286233305931091,0.4634127914905548,0.8943545818328857
+bert-base-uncased-nli,0.7304670810699463,0.5292267799377441,0.34321093559265137,0.8651169538497925,0.8389803171157837,0.03405291587114334,0.5710198879241943,0.5809178352355957,0.5509116649627686,0.06236717849969864,0.44819334149360657,0.3597811758518219,0.6084808111190796,0.7973349690437317,0.009742955677211285,0.5441672801971436,0.7575291395187378,0.7682543992996216,0.6505593657493591,0.4464646577835083,0.6742497086524963,0.7789746522903442,0.7689436674118042,0.5267128944396973,0.7087802886962891,0.45285505056381226,0.8868997097015381
+bert-large-uncased-nli,0.7552778124809265,0.5476767420768738,0.6135662198066711,0.8487353324890137,0.8577853441238403,0.166195347905159,0.6058303117752075,0.5906268954277039,0.585429847240448,0.32240763306617737,0.45579445362091064,0.33087220788002014,0.630742609500885,0.8359606266021729,0.07381048053503036,0.5340158343315125,0.652542769908905,0.7554200887680054,0.6485443115234375,0.4967177212238312,0.663260281085968,0.8231396079063416,0.7978278994560242,0.5720260143280029,0.7500104904174805,0.4941094219684601,0.8985104560852051
+bert-large-uncased-nli-triplet,0.7486485242843628,0.5742552876472473,0.5694993734359741,0.8558336496353149,0.8501056432723999,0.10687083750963211,0.6096311807632446,0.6752663850784302,0.5510167479515076,0.17876844108104706,0.39974313974380493,0.34211501479148865,0.6094277501106262,0.824999213218689,0.06057813763618469,0.4527166485786438,0.7017204165458679,0.7657798528671265,0.6720412373542786,0.47578948736190796,0.6625604033470154,0.7873606085777283,0.8306154608726501,0.5400728583335876,0.7908857464790344,0.4121311902999878,0.8973985910415649
+deberta-v3-base-nli,0.7633499503135681,0.08656716346740723,0.6209088563919067,0.8987488746643066,0.9109716415405273,0.23179928958415985,0.5807955861091614,0.6127842664718628,0.5521162152290344,0.37529000639915466,0.4566940665245056,0.4639076590538025,0.6758581399917603,0.9139114022254944,0.13150320947170258,0.49014008045196533,0.7852582931518555,0.8246683478355408,0.6916013956069946,0.3890807032585144,0.6463808417320251,0.7694025039672852,0.8272578120231628,0.5374786853790283,0.7942668199539185,0.5088838934898376,0.9327430725097656
+deberta-v3-large-nli,0.8132357597351074,0.5873494148254395,0.6070021986961365,0.9251483678817749,0.9281506538391113,0.4481724798679352,0.7091915607452393,0.6968613862991333,0.5381578803062439,0.41052955389022827,0.4692642092704773,0.5569407939910889,0.7762570381164551,0.9092003107070923,0.10709811747074127,0.5689337849617004,0.7898792028427124,0.8564063906669617,0.7372063994407654,0.6646546125411987,0.69362473487854,0.7445706129074097,0.8296204805374146,0.5326409339904785,0.8141666650772095,0.5630514621734619,0.9829331636428833
+deberta-v3-large-nli-triplet,0.8317348957061768,0.5857963562011719,0.6466480493545532,0.9301048517227173,0.9302252531051636,0.34609121084213257,0.7016400098800659,0.6651572585105896,0.5381077527999878,0.47797539830207825,0.49999648332595825,0.5419734716415405,0.7792794108390808,0.9280955195426941,0.18072746694087982,0.5456680059432983,0.6444740295410156,0.8395289182662964,0.7249240875244141,0.5605462193489075,0.6842126846313477,0.7779843807220459,0.8449709415435791,0.5460250973701477,0.8296811580657959,0.3875195384025574,0.9809557199478149
+modernbert-base-nli,0.7723070979118347,0.6252304315567017,0.3678560256958008,0.9123939871788025,0.9103059768676758,0.19969211518764496,0.5955341458320618,0.637680172920227,0.6163214445114136,0.2925623059272766,0.4357683062553406,0.43872055411338806,0.6271319389343262,0.8924717903137207,0.07944910228252411,0.3880825638771057,0.634975790977478,0.8002640008926392,0.6856403350830078,0.5114648938179016,0.6429415345191956,0.7781862616539001,0.7136391997337341,0.5306406617164612,0.7542345523834229,0.518562912940979,0.9609659910202026
+modernbert-large-nli,0.7711875438690186,0.5476767420768738,0.6381858587265015,0.9381246566772461,0.9252582788467407,0.28705841302871704,0.6431213617324829,0.6834946870803833,0.555086076259613,0.17529267072677612,0.4735690951347351,0.46538588404655457,0.7574691772460938,0.9173109531402588,0.14003317058086395,0.4664466083049774,0.6707661747932434,0.8766298294067383,0.6942710876464844,0.6215447783470154,0.6813621520996094,0.7359136939048767,0.8065485954284668,0.5483516454696655,0.7819554209709167,0.5348893404006958,0.979356050491333
+modernbert-large-nli-triplet,0.7674962282180786,0.5473684072494507,0.6421945691108704,0.9311951398849487,0.9248816967010498,0.3550959825515747,0.6277194619178772,0.7015386819839478,0.5417810678482056,0.17092861235141754,0.49263378977775574,0.45187288522720337,0.7207602262496948,0.915077269077301,0.20576725900173187,0.5262409448623657,0.7289433479309082,0.8816505074501038,0.7012882232666016,0.5184117555618286,0.6878244876861572,0.7644418478012085,0.8065363168716431,0.5420881509780884,0.7960731983184814,0.4079374372959137,0.9790115356445312
+ms-marco-MiniLM-L6-v2,0.5021059513092041,0.5311621427536011,0.2082698941230774,0.7118111848831177,0.7207171320915222,0.4417795240879059,0.6142018437385559,0.6441656351089478,0.4977855384349823,0.36105501651763916,0.325905442237854,0.34913337230682373,0.4312140643596649,0.6434303522109985,0.1313643604516983,0.14113575220108032,0.4953211843967438,0.5799888372421265,0.5260704755783081,0.566315770149231,0.47734230756759644,0.5011672973632812,0.5221269130706787,0.5189011096954346,0.49472469091415405,0.3344421088695526,0.666526734828949
+gte-reranker-modernbert-base,0.7636871337890625,0.5635503530502319,0.5851699113845825,0.9194884896278381,0.9254856705665588,0.676284670829773,0.5298737287521362,0.5244028568267822,0.5789476037025452,0.45853084325790405,0.4515022337436676,0.4932851195335388,0.6179410815238953,0.8644256591796875,0.23649853467941284,0.5803945064544678,0.724173367023468,0.8307623863220215,0.6515475511550903,0.6142182350158691,0.6358391046524048,0.5351483821868896,0.5607047080993652,0.5204809308052063,0.5479526519775391,0.5611310005187988,0.9629999995231628
+bge-reranker-base,0.6737851500511169,0.49580416083335876,0.439884215593338,0.6639357209205627,0.7936471104621887,0.5719695091247559,0.4995661973953247,0.4748896062374115,0.5227616429328918,0.40748095512390137,0.3168407380580902,0.3962348699569702,0.44167008996009827,0.6950551271438599,0.18218491971492767,0.44151943922042847,0.7379050254821777,0.5771682262420654,0.5716409683227539,0.4728548526763916,0.5594310760498047,0.6065172553062439,0.5764907598495483,0.5203135013580322,0.5842276811599731,0.4528435468673706,0.6512120962142944
+bge-reranker-large,0.770831823348999,0.529032289981842,0.503018856048584,0.8704652786254883,0.891020655632019,0.6598912477493286,0.5933622121810913,0.5605754852294922,0.534077525138855,0.507485032081604,0.43648621439933777,0.4934283196926117,0.509984016418457,0.8143559694290161,0.2439526617527008,0.4407878816127777,0.8228057026863098,0.7568062543869019,0.6546375751495361,0.44633111357688904,0.583641529083252,0.6232432126998901,0.6795343160629272,0.52483069896698,0.6396280527114868,0.5396367311477661,0.8870046734809875
+Qwen3-Reranker-0.6B,0.8210300207138062,0.578434407711029,0.6666264533996582,0.9118987321853638,0.8895370364189148,0.6613081097602844,0.5713315010070801,0.6866195797920227,0.5074318051338196,0.5465952157974243,0.49488916993141174,0.47439178824424744,0.6295381188392639,0.8847362399101257,0.3079465925693512,0.5602739453315735,0.8217553496360779,0.7837425470352173,0.6609043478965759,0.25275275111198425,0.6699082851409912,0.7388573884963989,0.802010178565979,0.5482288599014282,0.7932049632072449,0.5763124227523804,0.9473351240158081
+Qwen3-Reranker-8B,0.8350103497505188,0.6174156069755554,0.7919961214065552,0.9571244120597839,0.9341866970062256,0.7354130744934082,0.7938579320907593,0.7809867858886719,0.6186710596084595,0.7135266661643982,0.49956265091896057,0.5800796151161194,0.8454323410987854,0.9500150084495544,0.398116797208786,0.7083504796028137,0.8437800407409668,0.9065382480621338,0.7413942813873291,0.7558139562606812,0.6521735191345215,0.8414127826690674,0.8755072355270386,0.5833333134651184,0.8630095720291138,0.6458627581596375,0.9820953607559204
+all-MiniLM-L6-v2,0.6265597939491272,0.08656716346740723,0.5516502857208252,0.5116842985153198,0.7029896974563599,0.5618711113929749,0.6406142711639404,0.5725581049919128,0.5263156294822693,0.5245693922042847,0.46109339594841003,0.3555065989494324,0.35282549262046814,0.7449698448181152,0.18951760232448578,0.4232776463031769,0.6903243660926819,0.46404963731765747,0.4815325438976288,0.4625127911567688,0.44766688346862793,0.5290513038635254,0.5081151723861694,0.5172396898269653,0.5474761724472046,0.5528976917266846,0.2434999942779541
+e5-base-v2,0.7769278883934021,0.5567470192909241,0.5916169285774231,0.9282991886138916,0.9274652600288391,0.6523025631904602,0.6202475428581238,0.6074733734130859,0.5024656653404236,0.5093950629234314,0.45224061608314514,0.4998090863227844,0.5302709341049194,0.8996210098266602,0.24746529757976532,0.49540361762046814,0.8202486634254456,0.8367919921875,0.6776610016822815,0.4850466251373291,0.5884302854537964,0.7398092150688171,0.7081863284111023,0.5256366729736328,0.7040552496910095,0.5705708861351013,0.9531490802764893
+e5-large-v2,0.801010012626648,0.5779524445533752,0.6131265759468079,0.9441698789596558,0.9106762409210205,0.6998542547225952,0.5488690137863159,0.5234423875808716,0.5157051086425781,0.5179288387298584,0.4662151336669922,0.48358288407325745,0.6469757556915283,0.9300627708435059,0.27807387709617615,0.5467942953109741,0.8201464414596558,0.8548777103424072,0.6651859879493713,0.4799731969833374,0.5705078840255737,0.6927686929702759,0.7292996048927307,0.5286061763763428,0.7192105650901794,0.5682903528213501,0.9763883352279663
+e5-mistral-7b-instruct,0.8165492415428162,0.5696070194244385,0.7199374437332153,0.9414554238319397,0.9326616525650024,0.6966351866722107,0.6805517673492432,0.675881028175354,0.5296826958656311,0.6203116774559021,0.5303964614868164,0.5932239294052124,0.671596109867096,0.9237684607505798,0.3562106490135193,0.6466506719589233,0.869695246219635,0.8446707725524902,0.7087008953094482,0.4615013599395752,0.6376277208328247,0.7049510478973389,0.6593969464302063,0.5231822729110718,0.724402904510498,0.6586915254592896,0.9830597639083862
+bge-base-en-v1.5,0.6807701587677002,0.5360442996025085,0.597657322883606,0.9338191747665405,0.9129323959350586,0.682538628578186,0.577385425567627,0.5927748084068298,0.5050225853919983,0.5299004316329956,0.502555787563324,0.39534541964530945,0.6032983064651489,0.9015624523162842,0.24518485367298126,0.539627730846405,0.8319767117500305,0.8158644437789917,0.7074615359306335,0.4823671579360962,0.5900554656982422,0.6904691457748413,0.7384219169616699,0.5209817290306091,0.7522772550582886,0.5593596696853638,0.9451367855072021
+bge-large-en-v1.5,0.7729312181472778,0.5955401062965393,0.6163952350616455,0.9480941295623779,0.919572114944458,0.7381986379623413,0.5808204412460327,0.538241982460022,0.5167240500450134,0.5457377433776855,0.5091676115989685,0.478109210729599,0.6112039089202881,0.9349770545959473,0.2871650159358978,0.5365509986877441,0.8657565116882324,0.8338873386383057,0.6935156583786011,0.45366013050079346,0.6130553483963013,0.7028484344482422,0.6837161183357239,0.5085204243659973,0.6782971024513245,0.6037620306015015,0.9512209892272949
+gte-base-en-v1.5,0.7593144774436951,0.6097028255462646,0.6364694237709045,0.9099421501159668,0.9291197061538696,0.6832579374313354,0.601468563079834,0.6317906379699707,0.5187318325042725,0.5791536569595337,0.4316229522228241,0.36737167835235596,0.6041796803474426,0.8798589706420898,0.2409083992242813,0.5409731268882751,0.8898540735244751,0.8518723249435425,0.7464286088943481,0.4269310235977173,0.6314809918403625,0.7338423728942871,0.7833998203277588,0.5240511894226074,0.7393791675567627,0.5773304104804993,0.9719424247741699
+gte-large-en-v1.5,0.7764260768890381,0.5942674279212952,0.6572992205619812,0.9519065618515015,0.9080474376678467,0.6847038269042969,0.566199541091919,0.5052019357681274,0.5308035612106323,0.5632104873657227,0.4739866852760315,0.43467679619789124,0.5734497308731079,0.9419607520103455,0.35540515184402466,0.5871738195419312,0.862524688243866,0.877086877822876,0.8123123645782471,0.4316384792327881,0.6724035739898682,0.8009515404701233,0.8326195478439331,0.5382608771324158,0.8474968075752258,0.5916916131973267,0.9370794296264648
+gte-modernbert-base,0.76431804895401,0.5323307514190674,0.596172034740448,0.9531158208847046,0.923002302646637,0.6876528263092041,0.6264742016792297,0.6301878690719604,0.511141300201416,0.49493494629859924,0.4772695004940033,0.450422465801239,0.6730180978775024,0.9110391139984131,0.2887086868286133,0.6217591762542725,0.8648708462715149,0.8298747539520264,0.7420883178710938,0.4752989411354065,0.6603310108184814,0.73154616355896,0.6655195951461792,0.5248026847839355,0.6656557321548462,0.5605465769767761,0.9611742496490479
+Qwen3-Embedding-0.6B,0.6822134852409363,0.5722522735595703,0.5863082408905029,0.9034148454666138,0.8710983991622925,0.6807952523231506,0.5053926706314087,0.5418699383735657,0.5156175494194031,0.5408480167388916,0.484048992395401,0.4315657615661621,0.6033790707588196,0.8863784074783325,0.27557799220085144,0.5929194092750549,0.8534343242645264,0.767593502998352,0.7060188055038452,0.4729878902435303,0.5996844172477722,0.7088633179664612,0.6066822409629822,0.5332828760147095,0.7386647462844849,0.5679214596748352,0.9619758129119873
+Qwen3-Embedding-8B,0.8135340809822083,0.5923566818237305,0.7168856859207153,0.9409074783325195,0.9230391979217529,0.7595980763435364,0.7143589854240417,0.7244020700454712,0.48936134576797485,0.5921947360038757,0.5484007000923157,0.5494474172592163,0.72420334815979,0.9476759433746338,0.35499346256256104,0.6516250967979431,0.9034488201141357,0.8649088144302368,0.7627299427986145,0.5115970373153687,0.7363701462745667,0.7014169692993164,0.710321307182312,0.5296496152877808,0.7552865743637085,0.6388435959815979,0.9620493650436401
+gemma-3-270m-it,0.246751606464386,0.49547821283340454,0.0242428220808506,0.5011513829231262,0.5173690319061279,0.002926028799265623,0.40675675868988037,0.5699592232704163,0.4932486414909363,0.04259488359093666,0.04352770000696182,0.008854017592966557,0.3605327308177948,0.5250498056411743,5.746576061937958e-05,0.0009695125627331436,0.04178503155708313,0.5048661231994629,0.5013552904129028,0.5019047260284424,0.2091023027896881,0.5617921352386475,0.5833622813224792,0.492961049079895,0.5715240240097046,0.12142021954059601,0.45247310400009155
+gemma-3-1b-it,0.49714550375938416,0.5335820913314819,0.2548656761646271,0.699536919593811,0.7188526391983032,0.17591653764247894,0.5500313639640808,0.4933592677116394,0.5325133204460144,0.291238009929657,0.19724853336811066,0.24062468111515045,0.4271034598350525,0.666501522064209,0.019994866102933884,0.1974775344133377,0.40277931094169617,0.6366355419158936,0.5401451587677002,0.4262353181838989,0.40653783082962036,0.6478319764137268,0.6584980487823486,0.5171197652816772,0.6208167672157288,0.30175936222076416,0.7090381383895874
+Llama-3.2-3B-Instruct,0.7280982732772827,0.5044217705726624,0.5568433403968811,0.5463839769363403,0.5537739992141724,0.5394270420074463,0.47887909412384033,0.5913103818893433,0.5038384795188904,0.4854896664619446,0.4165148138999939,0.47685420513153076,0.3607962727546692,0.538789689540863,0.23749299347400665,0.45810678601264954,0.5855910181999207,0.5139164328575134,0.5206460952758789,0.49218320846557617,0.4320540130138397,0.5629687905311584,0.5842834711074829,0.4998076260089874,0.5765066146850586,0.511243462562561,0.4606666564941406
+Qwen3-4B,0.8497323393821716,0.775983989238739,0.6652383804321289,0.9475924372673035,0.922028660774231,0.4828844964504242,0.6940479278564453,0.7192214131355286,0.6154297590255737,0.62078857421875,0.5664623975753784,0.4028189182281494,0.7119026184082031,0.9299787282943726,0.2201460599899292,0.4583800435066223,0.7998732328414917,0.869918704032898,0.7323635816574097,0.7537612915039062,0.6702562570571899,0.8564978837966919,0.8918255567550659,0.6296330094337463,0.8699856996536255,0.6449465155601501,0.9839394092559814
+Phi-4-mini-instruct,0.7408709526062012,0.5316176414489746,0.6193781495094299,0.7121282815933228,0.6997272968292236,0.5339058637619019,0.44053563475608826,0.6652180552482605,0.5019315481185913,0.5515788793563843,0.389321893453598,0.40536338090896606,0.3777104318141937,0.6850106120109558,0.23859953880310059,0.395669162273407,0.6520477533340454,0.6564617156982422,0.5046854615211487,0.4999839663505554,0.49097007513046265,0.7045025825500488,0.6338613629341125,0.5235722661018372,0.6851626038551331,0.5803982615470886,0.706250011920929
+Qwen3-8B,0.8622773885726929,0.6696007251739502,0.6806145310401917,0.9473200440406799,0.9253227710723877,0.5226321816444397,0.7273061275482178,0.794788122177124,0.6377258896827698,0.6109967827796936,0.5580412149429321,0.4182721972465515,0.72970050573349,0.9401372671127319,0.23162594437599182,0.4751225709915161,0.8295106291770935,0.8771510124206543,0.7308112382888794,0.2529999911785126,0.7027848362922668,0.8517290353775024,0.8871627449989319,0.6636102795600891,0.8654845952987671,0.6372009515762329,0.9829134941101074
+Mistral-Nemo-Instruct-2407,0.8508498668670654,0.7452191114425659,0.6901076436042786,0.9446386098861694,0.9029363393783569,0.44835615158081055,0.761665940284729,0.7849413156509399,0.787163257598877,0.6091029644012451,0.5459252595901489,0.40234678983688354,0.7902305722236633,0.9502403736114502,0.24893233180046082,0.47370296716690063,0.792164146900177,0.76802659034729,0.7193542122840881,0.7002339959144592,0.64515221118927,0.8808333873748779,0.9089280366897583,0.7719568014144897,0.8483136892318726,0.6396505236625671,0.9689841270446777

btzsc-0.1.1/src/btzsc/_baselines/recall_scores.csv

@@ -0,0 +1,39 @@
+mdl,agnews,amazon_counterfactual,amazon_massive_scenario,amazonpolarity,appreviews,banking77,biasframes_intent,biasframes_offensive,biasframes_sex,capsotu,emotiondair,empathetic,financialphrasebank,imdb,manifesto,massive,mtop_domain,rottentomatoes,toxic_conversations_50k,trueteacher,tweet_sentiment_extraction,wikitoxic_insult,wikitoxic_obscene,wikitoxic_threat,wikitoxic_toxicaggregated,yahootopics,yelpreviews
+bert-large-uncased,0.2608878016471863,0.49240630865097046,0.06362501531839371,0.4708682894706726,0.5080965161323547,0.03117235191166401,0.4558570384979248,0.5256953835487366,0.4851379990577698,0.09962859004735947,0.17798647284507751,0.03202961012721062,0.3277882933616638,0.5080194473266602,0.02425803802907467,0.03465794026851654,0.1149892508983612,0.5187616944313049,0.5335000157356262,0.5,0.41673779487609863,0.524808406829834,0.5782321691513062,0.546785831451416,0.5489631295204163,0.11928775906562805,0.505445659160614
+deberta-v3-large,0.2724846303462982,0.5165878534317017,0.046076931059360504,0.5,0.5082085132598877,0.026604287326335907,0.4932771921157837,0.5335056185722351,0.5032030344009399,0.037681616842746735,0.17542590200901031,0.03125,0.39009273052215576,0.5,0.02116798236966133,0.0021551724057644606,0.0755637064576149,0.501876175403595,0.5049999952316284,0.5,0.3184184730052948,0.578202486038208,0.5848830342292786,0.5235831141471863,0.5708839297294617,0.09236700087785721,0.5076031684875488
+ModernBERT-large,0.25,0.5,0.09776502847671509,0.4989733099937439,0.4964637756347656,0.03836919739842415,0.5179999470710754,0.5513418912887573,0.4964858293533325,0.06778620928525925,0.17683719098567963,0.021197250112891197,0.33451467752456665,0.5,0.013843538239598274,0.028600864112377167,0.15450258553028107,0.5,0.4964999854564667,0.5033244490623474,0.3333333432674408,0.4671822488307953,0.5854091644287109,0.49391403794288635,0.6063859462738037,0.14147253334522247,0.5241302847862244
+bart-large-mnli,0.7252845168113708,0.5018050670623779,0.5454501509666443,0.9329286813735962,0.9208909273147583,0.3000609278678894,0.6145737767219543,0.5023094415664673,0.5090233683586121,0.36520224809646606,0.4806637763977051,0.38309159874916077,0.6726571917533875,0.9292382001876831,0.09726245701313019,0.42766883969306946,0.8422161340713501,0.8330206274986267,0.5734999775886536,0.5150381326675415,0.6185846328735352,0.5161438584327698,0.7288388609886169,0.5209205150604248,0.6164907217025757,0.27680033445358276,0.9596747159957886
+nli-roberta-base,0.7066395878791809,0.5120440721511841,0.279588907957077,0.8879902362823486,0.8940892219543457,0.07116952538490295,0.5283114910125732,0.5302104949951172,0.5598506331443787,0.17624379694461823,0.4559451937675476,0.3178861141204834,0.6504240036010742,0.8321985602378845,0.01680239662528038,0.33095434308052063,0.4383202791213989,0.7983114719390869,0.6554999947547913,0.4803931713104248,0.6077786684036255,0.630657434463501,0.7237746715545654,0.5491631627082825,0.7062214016914368,0.35932984948158264,0.8879522681236267
+bert-base-uncased-nli,0.6929008364677429,0.5098344087600708,0.2859208881855011,0.8619967103004456,0.8377816081047058,0.014650114811956882,0.569557785987854,0.5799597501754761,0.7322278022766113,0.05334913730621338,0.4212839901447296,0.14671137928962708,0.6046509742736816,0.7925077676773071,0.016121715307235718,0.30588337779045105,0.6160512566566467,0.7570356726646423,0.5794999599456787,0.49787968397140503,0.6324023008346558,0.7861692309379578,0.7487791776657104,0.619389533996582,0.6396063566207886,0.34131163358688354,0.8871057033538818
+bert-large-uncased-nli,0.748619794845581,0.573447048664093,0.39285600185394287,0.8379144668579102,0.8546466827392578,0.08436316251754761,0.6063221096992493,0.5612396001815796,0.8219488859176636,0.17148847877979279,0.4311284124851227,0.12993068993091583,0.7565159797668457,0.806333065032959,0.04045149311423302,0.3592279553413391,0.42847660183906555,0.717635989189148,0.5529999732971191,0.49896785616874695,0.6482795476913452,0.8357025980949402,0.7754977941513062,0.8496576547622681,0.6665986180305481,0.22243475914001465,0.8962278366088867
+bert-large-uncased-nli-triplet,0.7356671690940857,0.595885694026947,0.3278299570083618,0.8372379541397095,0.8499263525009155,0.06681493669748306,0.6088849306106567,0.6407065391540527,0.732758641242981,0.19008205831050873,0.3465455174446106,0.19607101380825043,0.71299809217453,0.7835376262664795,0.04148751124739647,0.4299509525299072,0.4751039743423462,0.7260788083076477,0.5875000357627869,0.4953993558883667,0.6477671265602112,0.7702758312225342,0.8376864194869995,0.7358786463737488,0.7751439809799194,0.37010493874549866,0.8965920805931091
+deberta-v3-base-nli,0.7681264877319336,0.5,0.49574947357177734,0.8986594676971436,0.9102022647857666,0.13334208726882935,0.5409887433052063,0.5509325265884399,0.738469123840332,0.19229184091091156,0.4229639172554016,0.3092634081840515,0.7995125651359558,0.9141499996185303,0.082496777176857,0.34249347448349,0.6700924634933472,0.8236397504806519,0.6915000081062317,0.4969155490398407,0.6244850158691406,0.7329212427139282,0.8276936411857605,0.7160041928291321,0.7863515615463257,0.4725315272808075,0.9311494827270508
+deberta-v3-large-nli,0.8169254064559937,0.505415141582489,0.5653939843177795,0.9221413731575012,0.9279314279556274,0.38250041007995605,0.6852177977561951,0.5858902335166931,0.6273885369300842,0.22876231372356415,0.4845179617404938,0.4474509358406067,0.8421474099159241,0.9014514088630676,0.11071651428937912,0.44142600893974304,0.7655045986175537,0.8489681482315063,0.7289999723434448,0.503928542137146,0.663076639175415,0.6768685579299927,0.8357380032539368,0.6705020666122437,0.8018040060997009,0.5421027541160583,0.9830665588378906
+deberta-v3-large-nli-triplet,0.8341231346130371,0.5892568230628967,0.6176847219467163,0.9284056425094604,0.9299155473709106,0.2633965313434601,0.7022200226783752,0.6444008350372314,0.6268577575683594,0.2954987585544586,0.47307032346725464,0.4202551543712616,0.8013922572135925,0.9264042377471924,0.09268350154161453,0.44312942028045654,0.5210655331611633,0.8377110958099365,0.718500018119812,0.51803058385849,0.6397720575332642,0.7445700168609619,0.8543341755867004,0.7730125188827515,0.8248316049575806,0.3296804428100586,0.9812732934951782
+modernbert-base-nli,0.7503945827484131,0.7114403247833252,0.43642690777778625,0.9102893471717834,0.907706081867218,0.09043437242507935,0.5465528964996338,0.5776380896568298,0.8482319116592407,0.19477753341197968,0.3360637426376343,0.284860223531723,0.6360207200050354,0.8902238011360168,0.0514046847820282,0.2550359070301056,0.46609121561050415,0.7514071464538574,0.6840000152587891,0.5046326518058777,0.6124007701873779,0.7804387211799622,0.6800752878189087,0.6474895477294922,0.7515480518341064,0.4001066982746124,0.9595046043395996
+modernbert-large-nli,0.7599613070487976,0.573447048664093,0.4358295798301697,0.9316798448562622,0.919562816619873,0.24262915551662445,0.6394937038421631,0.5238502621650696,0.7519950270652771,0.07798612862825394,0.3506722152233124,0.3033400774002075,0.5089853405952454,0.9129011034965515,0.1358354538679123,0.3800908029079437,0.5630275011062622,0.8649156093597412,0.6935000419616699,0.5123137831687927,0.6551805734634399,0.6706080436706543,0.8084891438484192,0.7850418090820312,0.7392317056655884,0.48952868580818176,0.9787516593933105
+modernbert-large-nli-triplet,0.7170899510383606,0.5672227144241333,0.5210382342338562,0.9281454086303711,0.9195788502693176,0.22298629581928253,0.6149761080741882,0.5382528901100159,0.6718463897705078,0.0844113677740097,0.4243590533733368,0.3219197690486908,0.6131308674812317,0.9097689986228943,0.08327395468950272,0.45177799463272095,0.6414315104484558,0.8714821338653564,0.7000000476837158,0.5060928463935852,0.6779158711433411,0.7258509993553162,0.8147618770599365,0.7501426935195923,0.7493425607681274,0.25621747970581055,0.9789597988128662
+ms-marco-MiniLM-L6-v2,0.4154464900493622,0.5248350501060486,0.13485178351402283,0.6921078562736511,0.7111815214157104,0.21929438412189484,0.6134754419326782,0.6368777751922607,0.4930448830127716,0.3598368167877197,0.2706259787082672,0.18157251179218292,0.4887731969356537,0.6291953325271606,0.09652630239725113,0.06176289916038513,0.22644147276878357,0.5787992477416992,0.5260000228881836,0.5126017928123474,0.41333243250846863,0.5011394023895264,0.5192086100578308,0.6022251844406128,0.49588721990585327,0.2964455485343933,0.6572002172470093
+gte-reranker-modernbert-base,0.7090103626251221,0.5969749689102173,0.6065269708633423,0.9103273749351501,0.9164426326751709,0.6737946271896362,0.5048359632492065,0.5248481631278992,0.6852442026138306,0.4752907454967499,0.48167750239372253,0.38813093304634094,0.6491255760192871,0.8376922607421875,0.2129371166229248,0.5720893144607544,0.603864312171936,0.8020638227462769,0.5885000228881836,0.5955897569656372,0.6136220097541809,0.5360356569290161,0.5620940923690796,0.6178680062294006,0.5478123426437378,0.3892977833747864,0.9633132219314575
+bge-reranker-base,0.6561883687973022,0.4930598735809326,0.5443271398544312,0.6631482839584351,0.7906106114387512,0.5103347301483154,0.4995654821395874,0.47452253103256226,0.588146984577179,0.521611750125885,0.33640772104263306,0.26934754848480225,0.5032211542129517,0.6951839327812195,0.19312702119350433,0.5303319692611694,0.7473637461662292,0.5759849548339844,0.5690000057220459,0.4728640913963318,0.5825268030166626,0.5421292781829834,0.5732912421226501,0.596281886100769,0.5769000053405762,0.4204748272895813,0.6498012542724609
+bge-reranker-large,0.7403790950775146,0.5504170060157776,0.5826599597930908,0.8705364465713501,0.8909690380096436,0.584199845790863,0.5916775465011597,0.5455132126808167,0.5790870189666748,0.5725051164627075,0.47783219814300537,0.3266182839870453,0.5833433270454407,0.8063130378723145,0.17478394508361816,0.5665310025215149,0.8170444369316101,0.7560975551605225,0.6389999985694885,0.44794851541519165,0.6050503849983215,0.5339279174804688,0.6108042001724243,0.5596234202384949,0.6199691295623779,0.5347358584403992,0.880921483039856
+Qwen3-Reranker-0.6B,0.794529914855957,0.5473359823226929,0.7087323665618896,0.9120965600013733,0.8866807222366333,0.6616645455360413,0.5399225950241089,0.599557638168335,0.5030565857887268,0.5923603177070618,0.5231236219406128,0.42157384753227234,0.619957685470581,0.8846499919891357,0.314266175031662,0.613998532295227,0.8025047183036804,0.7833020687103271,0.6490000486373901,0.4990118443965912,0.6112505197525024,0.7451459765434265,0.796930193901062,0.7542791962623596,0.7910267114639282,0.5440412759780884,0.9468480944633484
+Qwen3-Reranker-8B,0.7995575666427612,0.6736897826194763,0.80195552110672,0.9568408727645874,0.9285714626312256,0.7193227410316467,0.7398936152458191,0.7607153654098511,0.8823120594024658,0.694545328617096,0.5077645778656006,0.49000710248947144,0.7951804995536804,0.9499140977859497,0.34560656547546387,0.7392416000366211,0.824242889881134,0.903377115726471,0.7400000095367432,0.51113361120224,0.6507819890975952,0.8500308990478516,0.8845878839492798,0.8849372267723083,0.8586984872817993,0.6013916730880737,0.9824041128158569
+all-MiniLM-L6-v2,0.5069217085838318,0.5,0.5811715126037598,0.5012728571891785,0.5356342792510986,0.4374663829803467,0.5584455728530884,0.5465722680091858,0.5801851749420166,0.5023189783096313,0.22658133506774902,0.16362914443016052,0.32211822271347046,0.5058479309082031,0.1652449667453766,0.39252763986587524,0.6577910780906677,0.4990619122982025,0.4909999966621399,0.4853178858757019,0.3997218608856201,0.5044324398040771,0.5083074569702148,0.5802111029624939,0.5386421084403992,0.34622490406036377,0.5
+e5-base-v2,0.7691781520843506,0.5597846508026123,0.6322030425071716,0.9251173734664917,0.9268753528594971,0.6465913653373718,0.6064669489860535,0.591576337814331,0.5062779188156128,0.6365930438041687,0.45467713475227356,0.40761905908584595,0.6048827171325684,0.8972024917602539,0.24918732047080994,0.5804132223129272,0.8310351371765137,0.8367729783058167,0.6504999995231628,0.4906826615333557,0.5944191217422485,0.7499332427978516,0.6823936700820923,0.6332731246948242,0.6832839250564575,0.5562304258346558,0.952033519744873
+e5-large-v2,0.7923803925514221,0.6256380081176758,0.6555721759796143,0.9424151182174683,0.9101702570915222,0.594748318195343,0.5485363602638245,0.523840069770813,0.5717475414276123,0.5643118619918823,0.4994733929634094,0.4221965968608856,0.6750686168670654,0.9271707534790039,0.24636535346508026,0.5962241888046265,0.812618613243103,0.8545966148376465,0.6100000143051147,0.48182135820388794,0.5881868600845337,0.6086787581443787,0.7008911967277527,0.6403100490570068,0.699086606502533,0.5555293560028076,0.9765561819076538
+e5-mistral-7b-instruct,0.7775763273239136,0.531090497970581,0.7543610334396362,0.9393349885940552,0.9318516254425049,0.686503529548645,0.6689438223838806,0.5447821021080017,0.5122080445289612,0.6929008960723877,0.5377311110496521,0.5399547219276428,0.7472397685050964,0.9131993055343628,0.3193519115447998,0.7001461982727051,0.8841664791107178,0.8442776203155518,0.6589999794960022,0.4771687090396881,0.6403813362121582,0.5527304410934448,0.5122124552726746,0.5266736149787903,0.5184664726257324,0.6391779184341431,0.9833787679672241
+bge-base-en-v1.5,0.652498722076416,0.562647819519043,0.6418625712394714,0.9288598895072937,0.9034178256988525,0.6667591333389282,0.5740678310394287,0.5746545195579529,0.5098469853401184,0.6417205333709717,0.46722355484962463,0.33924028277397156,0.6518524885177612,0.8956555128097534,0.22956110537052155,0.6366740465164185,0.8345134258270264,0.8142589330673218,0.6410000324249268,0.48247748613357544,0.6098494529724121,0.5809069871902466,0.71601802110672,0.5571510195732117,0.7300294637680054,0.5850311517715454,0.9447946548461914
+bge-large-en-v1.5,0.7728698253631592,0.6181376576423645,0.6621493697166443,0.9483811259269714,0.9188587665557861,0.704035758972168,0.5598134994506836,0.5169197916984558,0.5209751725196838,0.6577588319778442,0.480523020029068,0.3890635371208191,0.6388155817985535,0.9349300265312195,0.3101934492588043,0.6150498986244202,0.8744300603866577,0.824577808380127,0.6935000419616699,0.47728073596954346,0.6208119988441467,0.5734736919403076,0.5643137693405151,0.5133605599403381,0.5759053230285645,0.5753264427185059,0.9514611959457397
+gte-base-en-v1.5,0.7591022253036499,0.6484190225601196,0.6845355033874512,0.8969283103942871,0.9289394617080688,0.6764678359031677,0.6011562347412109,0.6285054683685303,0.5147339105606079,0.6441802382469177,0.4499507546424866,0.3631083071231842,0.6296189427375793,0.8463601469993591,0.2498115450143814,0.6075910925865173,0.8944736123085022,0.8433395624160767,0.7070000171661377,0.48850634694099426,0.6224585175514221,0.6805330514907837,0.77093505859375,0.5890547633171082,0.6777241230010986,0.558665931224823,0.9720330834388733
+gte-large-en-v1.5,0.7503991723060608,0.5942674279212952,0.710762083530426,0.9521236419677734,0.9072100520133972,0.6546837687492371,0.5665242671966553,0.5044152736663818,0.5757741928100586,0.6364038586616516,0.45103153586387634,0.36367812752723694,0.6876605749130249,0.9419607520103455,0.3327782452106476,0.6551083922386169,0.8634548187255859,0.8733583688735962,0.8115000128746033,0.45337727665901184,0.6576856374740601,0.7914656400680542,0.8358572125434875,0.7222803235054016,0.8273673057556152,0.5538436770439148,0.9326444864273071
+gte-modernbert-base,0.7619649171829224,0.5357276201248169,0.6931670904159546,0.9514471292495728,0.9229871034622192,0.661003589630127,0.6271906495094299,0.6012561321258545,0.5150084495544434,0.6106833815574646,0.49262484908103943,0.4010213613510132,0.6627617478370667,0.9081318974494934,0.287731796503067,0.690264105796814,0.8759384155273438,0.8189493417739868,0.7404999732971191,0.47541648149490356,0.6663501262664795,0.6900323629379272,0.6017239689826965,0.5591003894805908,0.562312126159668,0.5449396967887878,0.9600390195846558
+Qwen3-Embedding-0.6B,0.6807345151901245,0.6247977018356323,0.683259904384613,0.9025961756706238,0.8693036437034607,0.6659894585609436,0.5030898451805115,0.5346786975860596,0.56468266248703,0.6490484476089478,0.546855092048645,0.4114857316017151,0.6478113532066345,0.8794885277748108,0.2821764349937439,0.6708149313926697,0.8623234033584595,0.7607879638671875,0.6909999847412109,0.4729841351509094,0.5823470950126648,0.6195178627967834,0.5889812707901001,0.677301287651062,0.6805440783500671,0.5439555644989014,0.9622864723205566
+Qwen3-Embedding-8B,0.7818300127983093,0.5379061102867126,0.7766295671463013,0.941142201423645,0.923051118850708,0.7408914566040039,0.5210214257240295,0.5880531072616577,0.4601544737815857,0.6487294435501099,0.597652018070221,0.5039448738098145,0.7841603755950928,0.9465738534927368,0.40162768959999084,0.7239789962768555,0.9187555909156799,0.8611632585525513,0.7319999933242798,0.5042126178741455,0.5303038954734802,0.5099667906761169,0.5426182746887207,0.6349372267723083,0.5691487789154053,0.5808680057525635,0.961013674736023
+gemma-3-270m-it,0.23454733192920685,0.4938378930091858,0.06512986868619919,0.5007985234260559,0.5129928588867188,0.01021870318800211,0.42782309651374817,0.5546879768371582,0.47593164443969727,0.04955240711569786,0.14155223965644836,0.0364009328186512,0.33413049578666687,0.5163410305976868,0.01886792480945587,0.007502308581024408,0.07872911542654037,0.5037523508071899,0.5010000467300415,0.5011001825332642,0.31279417872428894,0.5449715256690979,0.5591838359832764,0.47332635521888733,0.5492873191833496,0.07161495834589005,0.4668275713920593
+gemma-3-1b-it,0.384908527135849,0.5375326871871948,0.1943022459745407,0.5831242203712463,0.689548134803772,0.14734937250614166,0.5401318073272705,0.49461936950683594,0.6471008062362671,0.1541706770658493,0.2009340524673462,0.20882539451122284,0.46473199129104614,0.6009042263031006,0.027067728340625763,0.11287318170070648,0.2695980966091156,0.6013133525848389,0.5349999666213989,0.4599382281303406,0.43013113737106323,0.5910699367523193,0.6153833866119385,0.5616204142570496,0.5704455971717834,0.22860918939113617,0.5933911204338074
+Llama-3.2-3B-Instruct,0.6772974729537964,0.5060687065124512,0.5797895789146423,0.5358102321624756,0.5388984680175781,0.4355233311653137,0.48037463426589966,0.5743184685707092,0.5153378844261169,0.4804648756980896,0.3467283248901367,0.39415672421455383,0.37093403935432434,0.5313011407852173,0.11978432536125183,0.4001505970954895,0.5766440629959106,0.5112570524215698,0.5180000066757202,0.49405914545059204,0.4376610517501831,0.5522796511650085,0.5689176321029663,0.4990966320037842,0.5609171986579895,0.45707938075065613,0.4704800248146057
+Qwen3-4B,0.8233746290206909,0.8474106788635254,0.6912088394165039,0.9441041946411133,0.9165706634521484,0.42354440689086914,0.5426543951034546,0.6926696300506592,0.8601654767990112,0.5773991346359253,0.40451374650001526,0.34169602394104004,0.7370214462280273,0.9225436449050903,0.20944738388061523,0.4270361065864563,0.7954626679420471,0.8677297830581665,0.5625,0.5030364394187927,0.6524324417114258,0.8719052076339722,0.8926034569740295,0.9057626724243164,0.8698079586029053,0.5097914338111877,0.9842493534088135
+Phi-4-mini-instruct,0.5561092495918274,0.5401468873023987,0.5442071557044983,0.6058675646781921,0.6001824140548706,0.4476704001426697,0.45673811435699463,0.6001869440078735,0.5066988468170166,0.47562360763549805,0.3211808204650879,0.3165235221385956,0.4090906083583832,0.6044906377792358,0.17197829484939575,0.2905403673648834,0.5209943652153015,0.5834896564483643,0.5034999847412109,0.49998798966407776,0.4292381703853607,0.6248852014541626,0.5977449417114258,0.6027957201004028,0.6093939542770386,0.45940494537353516,0.5871789455413818
+Qwen3-8B,0.8527622222900391,0.7908315658569336,0.6931420564651489,0.944156289100647,0.9174827337265015,0.4338746964931488,0.6794042587280273,0.7152205109596252,0.8987663984298706,0.5517712831497192,0.38969686627388,0.2636229991912842,0.8444997072219849,0.9371775388717651,0.21952621638774872,0.42630308866500854,0.8283316493034363,0.8592870235443115,0.5605000257492065,0.5,0.6619555950164795,0.8645344972610474,0.8898411989212036,0.9224990606307983,0.8616224527359009,0.5259219408035278,0.9831706285476685
+Mistral-Nemo-Instruct-2407,0.8379960060119629,0.7138988971710205,0.6534603834152222,0.9446386098861694,0.8994015455245972,0.4029290974140167,0.6715710163116455,0.7890359163284302,0.8466761708259583,0.5213670134544373,0.4336623549461365,0.3043708801269531,0.7896091938018799,0.9498240947723389,0.20043033361434937,0.3954383432865143,0.770418643951416,0.6078799366950989,0.5764999985694885,0.531496524810791,0.6439123749732971,0.8814754486083984,0.915145754814148,0.9264454245567322,0.8475329279899597,0.5971412062644958,0.9687588810920715