@rsrsai/win-arm64 1.0.10-dev.2 → 1.0.10-dev.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/bin/core-notices/CORE-SDK-LICENSE.txt +68 -18
  2. package/bin/core-notices/models/sources.json +0 -12
  3. package/bin/core-runtime.json +3 -11
  4. package/bin/rsrs.exe +2 -2
  5. package/package.json +1 -1
  6. package/bin/core-notices/models/bge-base-zh-v1.5-MODEL-CARD.md +0 -416
  7. package/bin/core-notices/models/bge-reranker-base-MODEL-CARD.md +0 -529
package/bin/core-notices/CORE-SDK-LICENSE.txt CHANGED
@@ -1,24 +1,74 @@
1
- Respire Core Binary SDK License
1
+ Respire Core Binary SDK Noncommercial License 1.0
2
2
 
3
- Copyright (c) 2026 Asura and Respire contributors.
3
+ Copyright (c) 2026 Respire contributors.
4
4
 
5
- For an SDK release explicitly distributed under these terms, the copyright
6
- holders grant a worldwide, royalty-free, non-exclusive permission to download,
7
- use, copy and link the Respire Core binary SDK into applications, and distribute
8
- the original SDK binary and required runtime files as part of those applications
9
- or their build dependencies, including commercial applications.
5
+ 1. Scope and acceptance
10
6
 
11
- This permission applies only to first-party compiled SDK files. It grants no
12
- right to receive, publish or redistribute proprietary implementation source,
13
- and does not change the licenses of previously distributed source material.
14
- The public Rust binding and protocol have their own licenses.
7
+ These terms apply only to the first-party compiled Respire Core SDK files in a
8
+ new SDK release expressly distributed with these terms. Subject to rights that
9
+ applicable law does not allow the parties to exclude, exercising the permissions
10
+ below constitutes acceptance. These terms do not apply retroactively to an older
11
+ release distributed under different terms.
15
12
 
16
- Distributions must preserve this notice and the accompanying third-party
17
- copyright, license and attribution notices. Third-party libraries and model
18
- weights remain governed by their respective licenses, not by this grant.
13
+ 2. Noncommercial permission
14
+
15
+ The copyright holders grant a worldwide, non-exclusive, royalty-free permission
16
+ to obtain, install, use, copy, and link the unmodified SDK binary and its required
17
+ runtime files into applications solely for Noncommercial Purposes. You may
18
+ redistribute those unmodified files with an application or build dependency
19
+ solely for Noncommercial Purposes if you retain this complete license and all
20
+ required third-party notices. The recipient obtains only the permissions in
21
+ these terms. Do not represent that your application grants broader SDK rights.
22
+
23
+ 3. Commercial authorization
24
+
25
+ "Noncommercial Purposes" means purposes that do not involve, support, or
26
+ anticipate a Commercial Purpose. "Commercial Purpose" includes revenue,
27
+ compensation, profit, business advantage, commercial research or development,
28
+ commercial integration, resale, paid or business-operated hosted services, and
29
+ the operations of a commercial business. Internal deployment for a business's
30
+ operations is commercial even when no software fee is charged and there are no
31
+ external users. Use on behalf of a business is not personal noncommercial use.
32
+
33
+ All Commercial Purposes require a separate, prior written license from the
34
+ copyright holders. Downloading, linking, repository access, or receipt of a
35
+ public binding does not provide that commercial authorization. Contact Asura
36
+ at luodaoyi@gmail.com.
37
+
38
+ 4. Proprietary implementation and reverse engineering
39
+
40
+ No implementation source rights are granted. You may not publish or distribute
41
+ proprietary implementation source. Except to the extent applicable law expressly
42
+ permits the activity despite this restriction, you may not reverse engineer,
43
+ decompile, disassemble, translate, or otherwise attempt to reconstruct the
44
+ first-party implementation source, internal algorithms, or proprietary internal
45
+ structures from the SDK binary. You may not modify or create derivative works
46
+ of the first-party SDK binary, except as expressly authorized in writing.
47
+ Ordinary linking through its documented public ABI is permitted under section 2.
48
+
49
+ 5. Third-party and independent material
50
+
51
+ Restrictions in these terms apply only to the first-party proprietary SDK
52
+ material. Separately licensed libraries, models, public protocol definitions,
53
+ and thin bindings retain their own licenses. These terms do not restrict rights
54
+ granted by their respective licensors or give you rights to otherwise restricted
55
+ models. Preserve all bundled copyright, attribution, and license notices.
19
56
  No trademark rights or endorsement are granted.
20
57
 
21
- THE SDK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
22
- INCLUDING MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
23
- TO THE MAXIMUM EXTENT PERMITTED BY LAW, THE COPYRIGHT HOLDERS ARE NOT LIABLE FOR
24
- CLAIMS OR DAMAGES ARISING FROM USE OR DISTRIBUTION OF THE SDK.
58
+ 6. Existing grants and mandatory rights
59
+
60
+ These terms do not revoke a previous license grant. They do not restrict fair
61
+ use, interoperability, security research, observation, testing, or other rights
62
+ to the extent applicable law makes those rights non-excludable. A separate
63
+ written agreement controls only the permissions it expressly grants. If a
64
+ restriction is unenforceable, the other terms remain effective to the extent
65
+ allowed by applicable law.
66
+
67
+ 7. Warranty and liability
68
+
69
+ THE SDK IS PROVIDED WITHOUT WARRANTY OF ANY KIND, TO THE MAXIMUM EXTENT PERMITTED
70
+ BY LAW, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
71
+ PURPOSE, TITLE, AND NON-INFRINGEMENT. TO THE MAXIMUM EXTENT PERMITTED BY LAW, THE
72
+ COPYRIGHT HOLDERS AND CONTRIBUTORS ARE NOT LIABLE FOR CLAIMS OR DAMAGES ARISING
73
+ FROM THE SDK, ITS USE, ITS DISTRIBUTION, OR THESE TERMS, WHETHER ARISING IN CONTRACT,
74
+ TORT, OR OTHERWISE.
package/bin/core-notices/models/sources.json CHANGED
@@ -1,20 +1,8 @@
1
1
  [
2
- {
3
- "model": "bge-base-zh-v1.5",
4
- "revision": "f03589ceff5aac7111bd60cfc7d497ca17ecac65",
5
- "license": "mit",
6
- "files": []
7
- },
8
2
  {
9
3
  "model": "bge-m3",
10
4
  "revision": "5617a9f61b028005a4858fdac845db406aefb181",
11
5
  "license": "mit",
12
6
  "files": []
13
- },
14
- {
15
- "model": "bge-reranker-base",
16
- "revision": "2cfc18c9415c912f9d8155881c133215df768a70",
17
- "license": "mit",
18
- "files": []
19
7
  }
20
8
  ]
package/bin/core-runtime.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "schema_version": 1,
3
3
  "target": "aarch64-pc-windows-msvc",
4
- "sdk_version": "1.0.0",
4
+ "sdk_version": "1.0.1",
5
5
  "redistribution": "permitted-under-included-license",
6
6
  "files": [
7
7
  {
@@ -1366,27 +1366,19 @@
1366
1366
  },
1367
1367
  {
1368
1368
  "path": "core-notices/CORE-SDK-LICENSE.txt",
1369
- "sha256": "b8e51bc80c6636e053ef0c95f1231edabc8dc0f9cb319396c3d09e2dee3cef7d"
1370
- },
1371
- {
1372
- "path": "core-notices/models/bge-base-zh-v1.5-MODEL-CARD.md",
1373
- "sha256": "f41b2870f37f7ae024a580bf4a90628a6543d4c2201ef4cbc6f5eeb5ed48b5b7"
1369
+ "sha256": "89643fcd1007476800c726e1822b4b9ccfed8c297a8222b5452fd819a4b7cf7c"
1374
1370
  },
1375
1371
  {
1376
1372
  "path": "core-notices/models/bge-m3-MODEL-CARD.md",
1377
1373
  "sha256": "d4bd0fd5f778d2577e185348d1ac9de5bff5dadd9a93df60d64342f9675f506a"
1378
1374
  },
1379
- {
1380
- "path": "core-notices/models/bge-reranker-base-MODEL-CARD.md",
1381
- "sha256": "a1abe0a4e230a4f5805a843adee9cbf8807ac61e5fc114689267656964e4ecd5"
1382
- },
1383
1375
  {
1384
1376
  "path": "core-notices/models/FlagEmbedding-LICENSE.txt",
1385
1377
  "sha256": "d2066728cf4edc752cb7542fb20b97a3cde20fdb26e8d0f0e0900f0fa0ac43ef"
1386
1378
  },
1387
1379
  {
1388
1380
  "path": "core-notices/models/sources.json",
1389
- "sha256": "5baa64f06ebf804cb492d1846d1e639a3d5b36934156e534e65165aebbdd405d"
1381
+ "sha256": "c3f958d9245781235e040846d16bb867ba644895d62ba866d7678dc761ce8836"
1390
1382
  },
1391
1383
  {
1392
1384
  "path": "core-notices/consumer/dbus-MIT.txt",
package/bin/rsrs.exe CHANGED
@@ -1,4 +1,4 @@
1
1
  [diffend] Oversized file quarantined before diffing.
2
2
  name: package/bin/rsrs.exe
3
- size: 37154816 bytes
4
- sha256: d819105ca3a0bda17a2fa1c2d31a883f01241f1974ff460bf9590214d25b38d5
3
+ size: 37001216 bytes
4
+ sha256: 9a7ec2104608eb3c03d792d2b153dae1a59b50af38f71d5368b350827b31f9d3
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@rsrsai/win-arm64",
3
- "version": "1.0.10-dev.2",
3
+ "version": "1.0.10-dev.3",
4
4
  "description": "rsrs CLI platform binary",
5
5
  "repository": {
6
6
  "type": "git",
package/bin/core-notices/models/bge-base-zh-v1.5-MODEL-CARD.md DELETED
@@ -1,416 +0,0 @@
1
- ---
2
- tags:
3
- - sentence-transformers
4
- - feature-extraction
5
- - sentence-similarity
6
- - transformers
7
- license: mit
8
- language:
9
- - zh
10
- ---
11
-
12
-
13
- <h1 align="center">FlagEmbedding</h1>
14
-
15
-
16
- <h4 align="center">
17
- <p>
18
- <a href=#model-list>Model List</a> |
19
- <a href=#frequently-asked-questions>FAQ</a> |
20
- <a href=#usage>Usage</a> |
21
- <a href="#evaluation">Evaluation</a> |
22
- <a href="#train">Train</a> |
23
- <a href="#contact">Contact</a> |
24
- <a href="#citation">Citation</a> |
25
- <a href="#license">License</a>
26
- <p>
27
- </h4>
28
-
29
- More details please refer to our Github: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).
30
-
31
-
32
- [English](README.md) | [中文](https://github.com/FlagOpen/FlagEmbedding/blob/master/README_zh.md)
33
-
34
- FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.
35
- And it also can be used in vector databases for LLMs.
36
-
37
- ************* 🌟**Updates**🌟 *************
38
- - 10/12/2023: Release [LLM-Embedder](./FlagEmbedding/llm_embedder/README.md), a unified embedding model to support diverse retrieval augmentation needs for LLMs. [Paper](https://arxiv.org/pdf/2310.07554.pdf) :fire:
39
- - 09/15/2023: The [technical report](https://arxiv.org/pdf/2309.07597.pdf) of BGE has been released
40
- - 09/15/2023: The [masive training data](https://data.baai.ac.cn/details/BAAI-MTP) of BGE has been released
41
- - 09/12/2023: New models:
42
- - **New reranker model**: release cross-encoder models `BAAI/bge-reranker-base` and `BAAI/bge-reranker-large`, which are more powerful than embedding model. We recommend to use/fine-tune them to re-rank top-k documents returned by embedding models.
43
- - **update embedding model**: release `bge-*-v1.5` embedding model to alleviate the issue of the similarity distribution, and enhance its retrieval ability without instruction.
44
-
45
-
46
- <details>
47
- <summary>More</summary>
48
- <!-- ### More -->
49
-
50
- - 09/07/2023: Update [fine-tune code](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md): Add script to mine hard negatives and support adding instruction during fine-tuning.
51
- - 08/09/2023: BGE Models are integrated into **Langchain**, you can use it like [this](#using-langchain); C-MTEB **leaderboard** is [available](https://huggingface.co/spaces/mteb/leaderboard).
52
- - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
53
- - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!** :tada: :tada:
54
- - 08/01/2023: We release the [Chinese Massive Text Embedding Benchmark](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB) (**C-MTEB**), consisting of 31 test dataset.
55
-
56
- </details>
57
-
58
-
59
- ## Model List
60
-
61
- `bge` is short for `BAAI general embedding`.
62
-
63
- | Model | Language | | Description | query instruction for retrieval [1] |
64
- |:-------------------------------|:--------:| :--------:| :--------:|:--------:|
65
- | [BAAI/llm-embedder](https://huggingface.co/BAAI/llm-embedder) | English | [Inference](./FlagEmbedding/llm_embedder/README.md) [Fine-tune](./FlagEmbedding/llm_embedder/README.md) | a unified embedding model to support diverse retrieval augmentation needs for LLMs | See [README](./FlagEmbedding/llm_embedder/README.md) |
66
- | [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | Chinese and English | [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) | a cross-encoder model which is more accurate but less efficient [2] | |
67
- | [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | Chinese and English | [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) | a cross-encoder model which is more accurate but less efficient [2] | |
68
- | [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
69
- | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
70
- | [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
71
- | [BAAI/bge-large-zh-v1.5](https://huggingface.co/BAAI/bge-large-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
72
- | [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
73
- | [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
74
- | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | :trophy: rank **1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
75
- | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a base-scale model but with similar ability to `bge-large-en` | `Represent this sentence for searching relevant passages: ` |
76
- | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) |a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
77
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | :trophy: rank **1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | `为这个句子生成表示以用于检索相关文章：` |
78
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a base-scale model but with similar ability to `bge-large-zh` | `为这个句子生成表示以用于检索相关文章：` |
79
- | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章：` |
80
-
81
-
82
- [1\]: If you need to search the relevant passages to a query, we suggest to add the instruction to the query; in other cases, no instruction is needed, just use the original query directly. In all cases, **no instruction** needs to be added to passages.
83
-
84
- [2\]: Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. To balance the accuracy and time cost, cross-encoder is widely used to re-rank top-k documents retrieved by other simple models.
85
- For examples, use bge embedding model to retrieve top 100 relevant documents, and then use bge reranker to re-rank the top 100 document to get the final top-3 results.
86
-
87
- All models have been uploaded to Huggingface Hub, and you can see them at https://huggingface.co/BAAI.
88
- If you cannot open the Huggingface Hub, you also can download the models at https://model.baai.ac.cn/models .
89
-
90
-
91
- ## Frequently asked questions
92
-
93
- <details>
94
- <summary>1. How to fine-tune bge embedding model?</summary>
95
-
96
- <!-- ### How to fine-tune bge embedding model? -->
97
- Following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) to prepare data and fine-tune your model.
98
- Some suggestions:
99
- - Mine hard negatives following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune#hard-negatives), which can improve the retrieval performance.
100
- - If you pre-train bge on your data, the pre-trained model cannot be directly used to calculate similarity, and it must be fine-tuned with contrastive learning before computing similarity.
101
- - If the accuracy of the fine-tuned model is still not high, it is recommended to use/fine-tune the cross-encoder model (bge-reranker) to re-rank top-k results. Hard negatives also are needed to fine-tune reranker.
102
-
103
-
104
- </details>
105
-
106
- <details>
107
- <summary>2. The similarity score between two dissimilar sentences is higher than 0.5</summary>
108
-
109
- <!-- ### The similarity score between two dissimilar sentences is higher than 0.5 -->
110
- **Suggest to use bge v1.5, which alleviates the issue of the similarity distribution.**
111
-
112
- Since we finetune the models by contrastive learning with a temperature of 0.01,
113
- the similarity distribution of the current BGE model is about in the interval \[0.6, 1\].
114
- So a similarity score greater than 0.5 does not indicate that the two sentences are similar.
115
-
116
- For downstream tasks, such as passage retrieval or semantic similarity,
117
- **what matters is the relative order of the scores, not the absolute value.**
118
- If you need to filter similar sentences based on a similarity threshold,
119
- please select an appropriate similarity threshold based on the similarity distribution on your data (such as 0.8, 0.85, or even 0.9).
120
-
121
- </details>
122
-
123
- <details>
124
- <summary>3. When does the query instruction need to be used</summary>
125
-
126
- <!-- ### When does the query instruction need to be used -->
127
-
128
- For the `bge-*-v1.5`, we improve its retrieval ability when not using instruction.
129
- No instruction only has a slight degradation in retrieval performance compared with using instruction.
130
- So you can generate embedding without instruction in all cases for convenience.
131
-
132
- For a retrieval task that uses short queries to find long related documents,
133
- it is recommended to add instructions for these short queries.
134
- **The best method to decide whether to add instructions for queries is choosing the setting that achieves better performance on your task.**
135
- In all cases, the documents/passages do not need to add the instruction.
136
-
137
- </details>
138
-
139
-
140
- ## Usage
141
-
142
- ### Usage for Embedding Model
143
-
144
- Here are some examples for using `bge` models with
145
- [FlagEmbedding](#using-flagembedding), [Sentence-Transformers](#using-sentence-transformers), [Langchain](#using-langchain), or [Huggingface Transformers](#using-huggingface-transformers).
146
-
147
- #### Using FlagEmbedding
148
- ```
149
- pip install -U FlagEmbedding
150
- ```
151
- If it doesn't work for you, you can see [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md) for more methods to install FlagEmbedding.
152
-
153
- ```python
154
- from FlagEmbedding import FlagModel
155
- sentences_1 = ["样例数据-1", "样例数据-2"]
156
- sentences_2 = ["样例数据-3", "样例数据-4"]
157
- model = FlagModel('BAAI/bge-large-zh-v1.5',
158
- query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章：",
159
- use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
160
- embeddings_1 = model.encode(sentences_1)
161
- embeddings_2 = model.encode(sentences_2)
162
- similarity = embeddings_1 @ embeddings_2.T
163
- print(similarity)
164
-
165
- # for s2p(short query to long passage) retrieval task, suggest to use encode_queries() which will automatically add the instruction to each query
166
- # corpus in retrieval task can still use encode() or encode_corpus(), since they don't need instruction
167
- queries = ['query_1', 'query_2']
168
- passages = ["样例文档-1", "样例文档-2"]
169
- q_embeddings = model.encode_queries(queries)
170
- p_embeddings = model.encode(passages)
171
- scores = q_embeddings @ p_embeddings.T
172
- ```
173
- For the value of the argument `query_instruction_for_retrieval`, see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list).
174
-
175
- By default, FlagModel will use all available GPUs when encoding. Please set `os.environ["CUDA_VISIBLE_DEVICES"]` to select specific GPUs.
176
- You also can set `os.environ["CUDA_VISIBLE_DEVICES"]=""` to make all GPUs unavailable.
177
-
178
-
179
- #### Using Sentence-Transformers
180
-
181
- You can also use the `bge` models with [sentence-transformers](https://www.SBERT.net):
182
-
183
- ```
184
- pip install -U sentence-transformers
185
- ```
186
- ```python
187
- from sentence_transformers import SentenceTransformer
188
- sentences_1 = ["样例数据-1", "样例数据-2"]
189
- sentences_2 = ["样例数据-3", "样例数据-4"]
190
- model = SentenceTransformer('BAAI/bge-large-zh-v1.5')
191
- embeddings_1 = model.encode(sentences_1, normalize_embeddings=True)
192
- embeddings_2 = model.encode(sentences_2, normalize_embeddings=True)
193
- similarity = embeddings_1 @ embeddings_2.T
194
- print(similarity)
195
- ```
196
- For s2p(short query to long passage) retrieval task,
197
- each short query should start with an instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
198
- But the instruction is not needed for passages.
199
- ```python
200
- from sentence_transformers import SentenceTransformer
201
- queries = ['query_1', 'query_2']
202
- passages = ["样例文档-1", "样例文档-2"]
203
- instruction = "为这个句子生成表示以用于检索相关文章："
204
-
205
- model = SentenceTransformer('BAAI/bge-large-zh-v1.5')
206
- q_embeddings = model.encode([instruction+q for q in queries], normalize_embeddings=True)
207
- p_embeddings = model.encode(passages, normalize_embeddings=True)
208
- scores = q_embeddings @ p_embeddings.T
209
- ```
210
-
211
- #### Using Langchain
212
-
213
- You can use `bge` in langchain like this:
214
- ```python
215
- from langchain.embeddings import HuggingFaceBgeEmbeddings
216
- model_name = "BAAI/bge-large-en-v1.5"
217
- model_kwargs = {'device': 'cuda'}
218
- encode_kwargs = {'normalize_embeddings': True} # set True to compute cosine similarity
219
- model = HuggingFaceBgeEmbeddings(
220
- model_name=model_name,
221
- model_kwargs=model_kwargs,
222
- encode_kwargs=encode_kwargs,
223
- query_instruction="为这个句子生成表示以用于检索相关文章："
224
- )
225
- model.query_instruction = "为这个句子生成表示以用于检索相关文章："
226
- ```
227
-
228
-
229
- #### Using HuggingFace Transformers
230
-
231
- With the transformers package, you can use the model like this: First, you pass your input through the transformer model, then you select the last hidden state of the first token (i.e., [CLS]) as the sentence embedding.
232
-
233
- ```python
234
- from transformers import AutoTokenizer, AutoModel
235
- import torch
236
- # Sentences we want sentence embeddings for
237
- sentences = ["样例数据-1", "样例数据-2"]
238
-
239
- # Load model from HuggingFace Hub
240
- tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-large-zh-v1.5')
241
- model = AutoModel.from_pretrained('BAAI/bge-large-zh-v1.5')
242
- model.eval()
243
-
244
- # Tokenize sentences
245
- encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
246
- # for s2p(short query to long passage) retrieval task, add an instruction to query (not add instruction for passages)
247
- # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
248
-
249
- # Compute token embeddings
250
- with torch.no_grad():
251
- model_output = model(**encoded_input)
252
- # Perform pooling. In this case, cls pooling.
253
- sentence_embeddings = model_output[0][:, 0]
254
- # normalize embeddings
255
- sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
256
- print("Sentence embeddings:", sentence_embeddings)
257
- ```
258
-
259
- ### Usage for Reranker
260
-
261
- Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
262
- You can get a relevance score by inputting query and passage to the reranker.
263
- The reranker is optimized based cross-entropy loss, so the relevance score is not bounded to a specific range.
264
-
265
-
266
- #### Using FlagEmbedding
267
- ```
268
- pip install -U FlagEmbedding
269
- ```
270
-
271
- Get relevance scores (higher scores indicate more relevance):
272
- ```python
273
- from FlagEmbedding import FlagReranker
274
- reranker = FlagReranker('BAAI/bge-reranker-large', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
275
-
276
- score = reranker.compute_score(['query', 'passage'])
277
- print(score)
278
-
279
- scores = reranker.compute_score([['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']])
280
- print(scores)
281
- ```
282
-
283
-
284
- #### Using Huggingface transformers
285
-
286
- ```python
287
- import torch
288
- from transformers import AutoModelForSequenceClassification, AutoTokenizer
289
-
290
- tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-large')
291
- model = AutoModelForSequenceClassification.from_pretrained('BAAI/bge-reranker-large')
292
- model.eval()
293
-
294
- pairs = [['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']]
295
- with torch.no_grad():
296
- inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)
297
- scores = model(**inputs, return_dict=True).logits.view(-1, ).float()
298
- print(scores)
299
- ```
300
-
301
- ## Evaluation
302
-
303
- `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
304
- For more details and evaluation tools see our [scripts](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md).
305
-
306
- - **MTEB**:
307
-
308
- | Model Name | Dimension | Sequence Length | Average (56) | Retrieval (15) |Clustering (11) | Pair Classification (3) | Reranking (4) | STS (10) | Summarization (1) | Classification (12) |
309
- |:----:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
310
- | [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) | 1024 | 512 | **64.23** | **54.29** | 46.08 | 87.12 | 60.03 | 83.11 | 31.61 | 75.97 |
311
- | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) | 768 | 512 | 63.55 | 53.25 | 45.77 | 86.55 | 58.86 | 82.4 | 31.07 | 75.53 |
312
- | [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) | 384 | 512 | 62.17 |51.68 | 43.82 | 84.92 | 58.36 | 81.59 | 30.12 | 74.14 |
313
- | [bge-large-en](https://huggingface.co/BAAI/bge-large-en) | 1024 | 512 | 63.98 | 53.9 | 46.98 | 85.8 | 59.48 | 81.56 | 32.06 | 76.21 |
314
- | [bge-base-en](https://huggingface.co/BAAI/bge-base-en) | 768 | 512 | 63.36 | 53.0 | 46.32 | 85.86 | 58.7 | 81.84 | 29.27 | 75.27 |
315
- | [gte-large](https://huggingface.co/thenlper/gte-large) | 1024 | 512 | 63.13 | 52.22 | 46.84 | 85.00 | 59.13 | 83.35 | 31.66 | 73.33 |
316
- | [gte-base](https://huggingface.co/thenlper/gte-base) | 768 | 512 | 62.39 | 51.14 | 46.2 | 84.57 | 58.61 | 82.3 | 31.17 | 73.01 |
317
- | [e5-large-v2](https://huggingface.co/intfloat/e5-large-v2) | 1024| 512 | 62.25 | 50.56 | 44.49 | 86.03 | 56.61 | 82.05 | 30.19 | 75.24 |
318
- | [bge-small-en](https://huggingface.co/BAAI/bge-small-en) | 384 | 512 | 62.11 | 51.82 | 44.31 | 83.78 | 57.97 | 80.72 | 30.53 | 74.37 |
319
- | [instructor-xl](https://huggingface.co/hkunlp/instructor-xl) | 768 | 512 | 61.79 | 49.26 | 44.74 | 86.62 | 57.29 | 83.06 | 32.32 | 61.79 |
320
- | [e5-base-v2](https://huggingface.co/intfloat/e5-base-v2) | 768 | 512 | 61.5 | 50.29 | 43.80 | 85.73 | 55.91 | 81.05 | 30.28 | 73.84 |
321
- | [gte-small](https://huggingface.co/thenlper/gte-small) | 384 | 512 | 61.36 | 49.46 | 44.89 | 83.54 | 57.7 | 82.07 | 30.42 | 72.31 |
322
- | [text-embedding-ada-002](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 | 60.99 | 49.25 | 45.9 | 84.89 | 56.32 | 80.97 | 30.8 | 70.93 |
323
- | [e5-small-v2](https://huggingface.co/intfloat/e5-base-v2) | 384 | 512 | 59.93 | 49.04 | 39.92 | 84.67 | 54.32 | 80.39 | 31.16 | 72.94 |
324
- | [sentence-t5-xxl](https://huggingface.co/sentence-transformers/sentence-t5-xxl) | 768 | 512 | 59.51 | 42.24 | 43.72 | 85.06 | 56.42 | 82.63 | 30.08 | 73.42 |
325
- | [all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) | 768 | 514 | 57.78 | 43.81 | 43.69 | 83.04 | 59.36 | 80.28 | 27.49 | 65.07 |
326
- | [sgpt-bloom-7b1-msmarco](https://huggingface.co/bigscience/sgpt-bloom-7b1-msmarco) | 4096 | 2048 | 57.59 | 48.22 | 38.93 | 81.9 | 55.65 | 77.74 | 33.6 | 66.19 |
327
-
328
-
329
-
330
- - **C-MTEB**:
331
- We create the benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
332
- Please refer to [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md) for a detailed introduction.
333
-
334
- | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
335
- |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
336
- | [**BAAI/bge-large-zh-v1.5**](https://huggingface.co/BAAI/bge-large-zh-v1.5) | 1024 | **64.53** | 70.46 | 56.25 | 81.6 | 69.13 | 65.84 | 48.99 |
337
- | [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) | 768 | 63.13 | 69.49 | 53.72 | 79.75 | 68.07 | 65.39 | 47.53 |
338
- | [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) | 512 | 57.82 | 61.77 | 49.11 | 70.41 | 63.96 | 60.92 | 44.18 |
339
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | 1024 | 64.20 | 71.53 | 54.98 | 78.94 | 68.32 | 65.11 | 48.39 |
340
- | [bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | 1024 | 63.53 | 70.55 | 53 | 76.77 | 68.58 | 64.91 | 50.01 |
341
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | 768 | 62.96 | 69.53 | 54.12 | 77.5 | 67.07 | 64.91 | 47.63 |
342
- | [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 1024 | 58.79 | 63.66 | 48.44 | 69.89 | 67.34 | 56.00 | 48.23 |
343
- | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | 512 | 58.27 | 63.07 | 49.45 | 70.35 | 63.64 | 61.48 | 45.09 |
344
- | [m3e-base](https://huggingface.co/moka-ai/m3e-base) | 768 | 57.10 | 56.91 | 50.47 | 63.99 | 67.52 | 59.34 | 47.68 |
345
- | [m3e-large](https://huggingface.co/moka-ai/m3e-large) | 1024 | 57.05 | 54.75 | 50.42 | 64.3 | 68.2 | 59.66 | 48.88 |
346
- | [multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 768 | 55.48 | 61.63 | 46.49 | 67.07 | 65.35 | 54.35 | 40.68 |
347
- | [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) | 384 | 55.38 | 59.95 | 45.27 | 66.45 | 65.85 | 53.86 | 45.26 |
348
- | [text-embedding-ada-002(OpenAI)](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) | 1536 | 53.02 | 52.0 | 43.35 | 69.56 | 64.31 | 54.28 | 45.68 |
349
- | [luotuo](https://huggingface.co/silk-road/luotuo-bert-medium) | 1024 | 49.37 | 44.4 | 42.78 | 66.62 | 61 | 49.25 | 44.39 |
350
- | [text2vec-base](https://huggingface.co/shibing624/text2vec-base-chinese) | 768 | 47.63 | 38.79 | 43.41 | 67.41 | 62.19 | 49.45 | 37.66 |
351
- | [text2vec-large](https://huggingface.co/GanymedeNil/text2vec-large-chinese) | 1024 | 47.36 | 41.94 | 44.97 | 70.86 | 60.66 | 49.16 | 30.02 |
352
-
353
-
354
- - **Reranking**:
355
- See [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/) for evaluation script.
356
-
357
- | Model | T2Reranking | T2RerankingZh2En\* | T2RerankingEn2Zh\* | MMarcoReranking | CMedQAv1 | CMedQAv2 | Avg |
358
- |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
359
- | text2vec-base-multilingual | 64.66 | 62.94 | 62.51 | 14.37 | 48.46 | 48.6 | 50.26 |
360
- | multilingual-e5-small | 65.62 | 60.94 | 56.41 | 29.91 | 67.26 | 66.54 | 57.78 |
361
- | multilingual-e5-large | 64.55 | 61.61 | 54.28 | 28.6 | 67.42 | 67.92 | 57.4 |
362
- | multilingual-e5-base | 64.21 | 62.13 | 54.68 | 29.5 | 66.23 | 66.98 | 57.29 |
363
- | m3e-base | 66.03 | 62.74 | 56.07 | 17.51 | 77.05 | 76.76 | 59.36 |
364
- | m3e-large | 66.13 | 62.72 | 56.1 | 16.46 | 77.76 | 78.27 | 59.57 |
365
- | bge-base-zh-v1.5 | 66.49 | 63.25 | 57.02 | 29.74 | 80.47 | 84.88 | 63.64 |
366
- | bge-large-zh-v1.5 | 65.74 | 63.39 | 57.03 | 28.74 | 83.45 | 85.44 | 63.97 |
367
- | [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | 67.28 | 63.95 | 60.45 | 35.46 | 81.26 | 84.1 | 65.42 |
368
- | [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | 67.6 | 64.03 | 61.44 | 37.16 | 82.15 | 84.18 | 66.09 |
369
-
370
- \* : T2RerankingZh2En and T2RerankingEn2Zh are cross-language retrieval tasks
371
-
372
- ## Train
373
-
374
- ### BAAI Embedding
375
-
376
- We pre-train the models using [retromae](https://github.com/staoxiao/RetroMAE) and train them on large-scale pairs data using contrastive learning.
377
- **You can fine-tune the embedding model on your data following our [examples](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).**
378
- We also provide a [pre-train example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain).
379
- Note that the goal of pre-training is to reconstruct the text, and the pre-trained model cannot be used for similarity calculation directly, it needs to be fine-tuned.
380
- More training details for bge see [baai_general_embedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md).
381
-
382
-
383
-
384
- ### BGE Reranker
385
-
386
- Cross-encoder will perform full-attention over the input pair,
387
- which is more accurate than embedding model (i.e., bi-encoder) but more time-consuming than embedding model.
388
- Therefore, it can be used to re-rank the top-k documents returned by embedding model.
389
- We train the cross-encoder on a multilingual pair data,
390
- The data format is the same as embedding model, so you can fine-tune it easily following our [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker).
391
- More details please refer to [./FlagEmbedding/reranker/README.md](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker)
392
-
393
-
394
- ## Contact
395
- If you have any question or suggestion related to this project, feel free to open an issue or pull request.
396
- You also can email Shitao Xiao(stxiao@baai.ac.cn) and Zheng Liu(liuzheng@baai.ac.cn).
397
-
398
-
399
- ## Citation
400
-
401
- If you find this repository useful, please consider giving a star :star: and citation
402
-
403
- ```
404
- @misc{bge_embedding,
405
- title={C-Pack: Packaged Resources To Advance General Chinese Embedding},
406
- author={Shitao Xiao and Zheng Liu and Peitian Zhang and Niklas Muennighoff},
407
- year={2023},
408
- eprint={2309.07597},
409
- archivePrefix={arXiv},
410
- primaryClass={cs.CL}
411
- }
412
- ```
413
-
414
- ## License
415
- FlagEmbedding is licensed under the [MIT License](https://github.com/FlagOpen/FlagEmbedding/blob/master/LICENSE). The released models can be used for commercial purposes free of charge.
416
-
package/bin/core-notices/models/bge-reranker-base-MODEL-CARD.md DELETED
@@ -1,529 +0,0 @@
1
- ---
2
- license: mit
3
- language:
4
- - en
5
- - zh
6
- tags:
7
- - mteb
8
- - text-embeddings-inference
9
- model-index:
10
- - name: bge-reranker-base
11
- results:
12
- - task:
13
- type: Reranking
14
- dataset:
15
- type: C-MTEB/CMedQAv1-reranking
16
- name: MTEB CMedQAv1
17
- config: default
18
- split: test
19
- revision: None
20
- metrics:
21
- - type: map
22
- value: 81.27206722525007
23
- - type: mrr
24
- value: 84.14238095238095
25
- - task:
26
- type: Reranking
27
- dataset:
28
- type: C-MTEB/CMedQAv2-reranking
29
- name: MTEB CMedQAv2
30
- config: default
31
- split: test
32
- revision: None
33
- metrics:
34
- - type: map
35
- value: 84.10369934291236
36
- - type: mrr
37
- value: 86.79376984126984
38
- - task:
39
- type: Reranking
40
- dataset:
41
- type: C-MTEB/Mmarco-reranking
42
- name: MTEB MMarcoReranking
43
- config: default
44
- split: dev
45
- revision: None
46
- metrics:
47
- - type: map
48
- value: 35.4600511272538
49
- - type: mrr
50
- value: 34.60238095238095
51
- - task:
52
- type: Reranking
53
- dataset:
54
- type: C-MTEB/T2Reranking
55
- name: MTEB T2Reranking
56
- config: default
57
- split: dev
58
- revision: None
59
- metrics:
60
- - type: map
61
- value: 67.27728847727172
62
- - type: mrr
63
- value: 77.1315192743764
64
- pipeline_tag: text-classification
65
- library_name: sentence-transformers
66
- ---
67
-
68
- **We have updated the [new reranker](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker), supporting larger lengths, more languages, and achieving better performance.**
69
-
70
- <h1 align="center">FlagEmbedding</h1>
71
-
72
-
73
- <h4 align="center">
74
- <p>
75
- <a href=#model-list>Model List</a> |
76
- <a href=#frequently-asked-questions>FAQ</a> |
77
- <a href=#usage>Usage</a> |
78
- <a href="#evaluation">Evaluation</a> |
79
- <a href="#train">Train</a> |
80
- <a href="#citation">Citation</a> |
81
- <a href="#license">License</a>
82
- <p>
83
- </h4>
84
-
85
- **More details please refer to our Github: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).**
86
-
87
-
88
- [English](README.md) | [中文](https://github.com/FlagOpen/FlagEmbedding/blob/master/README_zh.md)
89
-
90
-
91
- FlagEmbedding focuses on retrieval-augmented LLMs, consisting of the following projects currently:
92
-
93
- - **Long-Context LLM**: [Activation Beacon](https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon)
94
- - **Fine-tuning of LM** : [LM-Cocktail](https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail)
95
- - **Embedding Model**: [Visualized-BGE](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/visual), [BGE-M3](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/BGE_M3), [LLM Embedder](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_embedder), [BGE Embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/baai_general_embedding)
96
- - **Reranker Model**: [llm rerankers](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker), [BGE Reranker](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker)
97
- - **Benchmark**: [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB)
98
-
99
- ## News
100
- - 3/18/2024: Release new [rerankers](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker), built upon powerful M3 and LLM (GEMMA and MiniCPM, not so large actually) backbones, supporitng multi-lingual processing and larger inputs, massive improvements of ranking performances on BEIR, C-MTEB/Retrieval, MIRACL, LlamaIndex Evaluation.
101
- - 3/18/2024: Release [Visualized-BGE](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/visual), equipping BGE with visual capabilities. Visualized-BGE can be utilized to generate embeddings for hybrid image-text data.
102
- - 1/30/2024: Release **BGE-M3**, a new member to BGE model series! M3 stands for **M**ulti-linguality (100+ languages), **M**ulti-granularities (input length up to 8192), **M**ulti-Functionality (unification of dense, lexical, multi-vec/colbert retrieval).
103
- It is the first embedding model which supports all three retrieval methods, achieving new SOTA on multi-lingual (MIRACL) and cross-lingual (MKQA) benchmarks.
104
- [Technical Report](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/BGE_M3/BGE_M3.pdf) and [Code](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/BGE_M3). :fire:
105
- - 1/9/2024: Release [Activation-Beacon](https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon), an effective, efficient, compatible, and low-cost (training) method to extend the context length of LLM. [Technical Report](https://arxiv.org/abs/2401.03462) :fire:
106
- - 12/24/2023: Release **LLaRA**, a LLaMA-7B based dense retriever, leading to state-of-the-art performances on MS MARCO and BEIR. Model and code will be open-sourced. Please stay tuned. [Technical Report](https://arxiv.org/abs/2312.15503)
107
- - 11/23/2023: Release [LM-Cocktail](https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail), a method to maintain general capabilities during fine-tuning by merging multiple language models. [Technical Report](https://arxiv.org/abs/2311.13534) :fire:
108
- - 10/12/2023: Release [LLM-Embedder](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_embedder), a unified embedding model to support diverse retrieval augmentation needs for LLMs. [Technical Report](https://arxiv.org/pdf/2310.07554.pdf)
109
- - 09/15/2023: The [technical report](https://arxiv.org/pdf/2309.07597.pdf) of BGE has been released
110
- - 09/15/2023: The [massive training data](https://data.baai.ac.cn/details/BAAI-MTP) of BGE has been released
111
- - 09/12/2023: New models:
112
- - **New reranker model**: release cross-encoder models `BAAI/bge-reranker-base` and `BAAI/bge-reranker-large`, which are more powerful than embedding model. We recommend to use/fine-tune them to re-rank top-k documents returned by embedding models.
113
- - **update embedding model**: release `bge-*-v1.5` embedding model to alleviate the issue of the similarity distribution, and enhance its retrieval ability without instruction.
114
-
115
-
116
- <details>
117
- <summary>More</summary>
118
- <!-- ### More -->
119
-
120
- - 09/07/2023: Update [fine-tune code](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md): Add script to mine hard negatives and support adding instruction during fine-tuning.
121
- - 08/09/2023: BGE Models are integrated into **Langchain**, you can use it like [this](#using-langchain); C-MTEB **leaderboard** is [available](https://huggingface.co/spaces/mteb/leaderboard).
122
- - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
123
- - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!** :tada: :tada:
124
- - 08/01/2023: We release the [Chinese Massive Text Embedding Benchmark](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB) (**C-MTEB**), consisting of 31 test dataset.
125
-
126
- </details>
127
-
128
-
129
- ## Model List
130
-
131
- `bge` is short for `BAAI general embedding`.
132
-
133
- | Model | Language | | Description | query instruction for retrieval [1] |
134
- |:-------------------------------|:--------:| :--------:| :--------:|:--------:|
135
- | [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) | Multilingual | [Inference](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/BGE_M3#usage) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/BGE_M3) | Multi-Functionality(dense retrieval, sparse retrieval, multi-vector(colbert)), Multi-Linguality, and Multi-Granularity(8192 tokens) | |
136
- | [BAAI/llm-embedder](https://huggingface.co/BAAI/llm-embedder) | English | [Inference](./FlagEmbedding/llm_embedder/README.md) [Fine-tune](./FlagEmbedding/llm_embedder/README.md) | a unified embedding model to support diverse retrieval augmentation needs for LLMs | See [README](./FlagEmbedding/llm_embedder/README.md) |
137
- | [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | Chinese and English | [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) | a cross-encoder model which is more accurate but less efficient [2] | |
138
- | [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | Chinese and English | [Inference](#usage-for-reranker) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) | a cross-encoder model which is more accurate but less efficient [2] | |
139
- | [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
140
- | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
141
- | [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `Represent this sentence for searching relevant passages: ` |
142
- | [BAAI/bge-large-zh-v1.5](https://huggingface.co/BAAI/bge-large-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
143
- | [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
144
- | [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | version 1.5 with more reasonable similarity distribution | `为这个句子生成表示以用于检索相关文章：` |
145
- | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | :trophy: rank **1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
146
- | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a base-scale model but with similar ability to `bge-large-en` | `Represent this sentence for searching relevant passages: ` |
147
- | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) |a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
148
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | :trophy: rank **1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | `为这个句子生成表示以用于检索相关文章：` |
149
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a base-scale model but with similar ability to `bge-large-zh` | `为这个句子生成表示以用于检索相关文章：` |
150
- | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | [Inference](#usage-for-embedding-model) [Fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章：` |
151
-
152
-
153
- [1\]: If you need to search the relevant passages to a query, we suggest to add the instruction to the query; in other cases, no instruction is needed, just use the original query directly. In all cases, **no instruction** needs to be added to passages.
154
-
155
- [2\]: Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. To balance the accuracy and time cost, cross-encoder is widely used to re-rank top-k documents retrieved by other simple models.
156
- For examples, use bge embedding model to retrieve top 100 relevant documents, and then use bge reranker to re-rank the top 100 document to get the final top-3 results.
157
-
158
- All models have been uploaded to Huggingface Hub, and you can see them at https://huggingface.co/BAAI.
159
- If you cannot open the Huggingface Hub, you also can download the models at https://model.baai.ac.cn/models .
160
-
161
-
162
- ## Frequently asked questions
163
-
164
- <details>
165
- <summary>1. How to fine-tune bge embedding model?</summary>
166
-
167
- <!-- ### How to fine-tune bge embedding model? -->
168
- Following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune) to prepare data and fine-tune your model.
169
- Some suggestions:
170
- - Mine hard negatives following this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune#hard-negatives), which can improve the retrieval performance.
171
- - If you pre-train bge on your data, the pre-trained model cannot be directly used to calculate similarity, and it must be fine-tuned with contrastive learning before computing similarity.
172
- - If the accuracy of the fine-tuned model is still not high, it is recommended to use/fine-tune the cross-encoder model (bge-reranker) to re-rank top-k results.
173
- Hard negatives also are needed to fine-tune reranker. Refer to this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker) for the fine-tuning for reranker
174
-
175
-
176
- </details>
177
-
178
- <details>
179
- <summary>2. The similarity score between two dissimilar sentences is higher than 0.5</summary>
180
-
181
- <!-- ### The similarity score between two dissimilar sentences is higher than 0.5 -->
182
- **Suggest to use bge v1.5, which alleviates the issue of the similarity distribution.**
183
-
184
- Since we finetune the models by contrastive learning with a temperature of 0.01,
185
- the similarity distribution of the current BGE model is about in the interval \[0.6, 1\].
186
- So a similarity score greater than 0.5 does not indicate that the two sentences are similar.
187
-
188
- For downstream tasks, such as passage retrieval or semantic similarity,
189
- **what matters is the relative order of the scores, not the absolute value.**
190
- If you need to filter similar sentences based on a similarity threshold,
191
- please select an appropriate similarity threshold based on the similarity distribution on your data (such as 0.8, 0.85, or even 0.9).
192
-
193
- </details>
194
-
195
- <details>
196
- <summary>3. When does the query instruction need to be used</summary>
197
-
198
- <!-- ### When does the query instruction need to be used -->
199
-
200
- For the `bge-*-v1.5`, we improve its retrieval ability when not using instruction.
201
- No instruction only has a slight degradation in retrieval performance compared with using instruction.
202
- So you can generate embedding without instruction in all cases for convenience.
203
-
204
- For a retrieval task that uses short queries to find long related documents,
205
- it is recommended to add instructions for these short queries.
206
- **The best method to decide whether to add instructions for queries is choosing the setting that achieves better performance on your task.**
207
- In all cases, the documents/passages do not need to add the instruction.
208
-
209
- </details>
210
-
211
-
212
- ## Usage
213
-
214
- ### Usage for Embedding Model
215
-
216
- Here are some examples for using `bge` models with
217
- [FlagEmbedding](#using-flagembedding), [Sentence-Transformers](#using-sentence-transformers), [Langchain](#using-langchain), or [Huggingface Transformers](#using-huggingface-transformers).
218
-
219
- #### Using FlagEmbedding
220
- ```
221
- pip install -U FlagEmbedding
222
- ```
223
- If it doesn't work for you, you can see [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md) for more methods to install FlagEmbedding.
224
-
225
- ```python
226
- from FlagEmbedding import FlagModel
227
- sentences_1 = ["样例数据-1", "样例数据-2"]
228
- sentences_2 = ["样例数据-3", "样例数据-4"]
229
- model = FlagModel('BAAI/bge-large-zh-v1.5',
230
- query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章：",
231
- use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
232
- embeddings_1 = model.encode(sentences_1)
233
- embeddings_2 = model.encode(sentences_2)
234
- similarity = embeddings_1 @ embeddings_2.T
235
- print(similarity)
236
-
237
- # for s2p(short query to long passage) retrieval task, suggest to use encode_queries() which will automatically add the instruction to each query
238
- # corpus in retrieval task can still use encode() or encode_corpus(), since they don't need instruction
239
- queries = ['query_1', 'query_2']
240
- passages = ["样例文档-1", "样例文档-2"]
241
- q_embeddings = model.encode_queries(queries)
242
- p_embeddings = model.encode(passages)
243
- scores = q_embeddings @ p_embeddings.T
244
- ```
245
- For the value of the argument `query_instruction_for_retrieval`, see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list).
246
-
247
- By default, FlagModel will use all available GPUs when encoding. Please set `os.environ["CUDA_VISIBLE_DEVICES"]` to select specific GPUs.
248
- You also can set `os.environ["CUDA_VISIBLE_DEVICES"]=""` to make all GPUs unavailable.
249
-
250
-
251
- #### Using Sentence-Transformers
252
-
253
- You can also use the `bge` models with [sentence-transformers](https://www.SBERT.net):
254
-
255
- ```
256
- pip install -U sentence-transformers
257
- ```
258
- ```python
259
- from sentence_transformers import SentenceTransformer
260
- sentences_1 = ["样例数据-1", "样例数据-2"]
261
- sentences_2 = ["样例数据-3", "样例数据-4"]
262
- model = SentenceTransformer('BAAI/bge-large-zh-v1.5')
263
- embeddings_1 = model.encode(sentences_1, normalize_embeddings=True)
264
- embeddings_2 = model.encode(sentences_2, normalize_embeddings=True)
265
- similarity = embeddings_1 @ embeddings_2.T
266
- print(similarity)
267
- ```
268
- For s2p(short query to long passage) retrieval task,
269
- each short query should start with an instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
270
- But the instruction is not needed for passages.
271
- ```python
272
- from sentence_transformers import SentenceTransformer
273
- queries = ['query_1', 'query_2']
274
- passages = ["样例文档-1", "样例文档-2"]
275
- instruction = "为这个句子生成表示以用于检索相关文章："
276
-
277
- model = SentenceTransformer('BAAI/bge-large-zh-v1.5')
278
- q_embeddings = model.encode([instruction+q for q in queries], normalize_embeddings=True)
279
- p_embeddings = model.encode(passages, normalize_embeddings=True)
280
- scores = q_embeddings @ p_embeddings.T
281
- ```
282
-
283
- #### Using Langchain
284
-
285
- You can use `bge` in langchain like this:
286
- ```python
287
- from langchain.embeddings import HuggingFaceBgeEmbeddings
288
- model_name = "BAAI/bge-large-en-v1.5"
289
- model_kwargs = {'device': 'cuda'}
290
- encode_kwargs = {'normalize_embeddings': True} # set True to compute cosine similarity
291
- model = HuggingFaceBgeEmbeddings(
292
- model_name=model_name,
293
- model_kwargs=model_kwargs,
294
- encode_kwargs=encode_kwargs,
295
- query_instruction="为这个句子生成表示以用于检索相关文章："
296
- )
297
- model.query_instruction = "为这个句子生成表示以用于检索相关文章："
298
- ```
299
-
300
-
301
- #### Using HuggingFace Transformers
302
-
303
- With the transformers package, you can use the model like this: First, you pass your input through the transformer model, then you select the last hidden state of the first token (i.e., [CLS]) as the sentence embedding.
304
-
305
- ```python
306
- from transformers import AutoTokenizer, AutoModel
307
- import torch
308
- # Sentences we want sentence embeddings for
309
- sentences = ["样例数据-1", "样例数据-2"]
310
-
311
- # Load model from HuggingFace Hub
312
- tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-large-zh-v1.5')
313
- model = AutoModel.from_pretrained('BAAI/bge-large-zh-v1.5')
314
- model.eval()
315
-
316
- # Tokenize sentences
317
- encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
318
- # for s2p(short query to long passage) retrieval task, add an instruction to query (not add instruction for passages)
319
- # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
320
-
321
- # Compute token embeddings
322
- with torch.no_grad():
323
- model_output = model(**encoded_input)
324
- # Perform pooling. In this case, cls pooling.
325
- sentence_embeddings = model_output[0][:, 0]
326
- # normalize embeddings
327
- sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
328
- print("Sentence embeddings:", sentence_embeddings)
329
- ```
330
-
331
- ### Usage for Reranker
332
-
333
- Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
334
- You can get a relevance score by inputting query and passage to the reranker.
335
- The reranker is optimized based cross-entropy loss, so the relevance score is not bounded to a specific range.
336
-
337
-
338
- #### Using FlagEmbedding
339
- ```
340
- pip install -U FlagEmbedding
341
- ```
342
-
343
- Get relevance scores (higher scores indicate more relevance):
344
- ```python
345
- from FlagEmbedding import FlagReranker
346
- reranker = FlagReranker('BAAI/bge-reranker-large', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
347
-
348
- score = reranker.compute_score(['query', 'passage'])
349
- print(score)
350
-
351
- scores = reranker.compute_score([['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']])
352
- print(scores)
353
- ```
354
-
355
-
356
- #### Using Huggingface transformers
357
-
358
- ```python
359
- import torch
360
- from transformers import AutoModelForSequenceClassification, AutoTokenizer
361
-
362
- tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-large')
363
- model = AutoModelForSequenceClassification.from_pretrained('BAAI/bge-reranker-large')
364
- model.eval()
365
-
366
- pairs = [['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']]
367
- with torch.no_grad():
368
- inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)
369
- scores = model(**inputs, return_dict=True).logits.view(-1, ).float()
370
- print(scores)
371
- ```
372
-
373
- #### Usage reranker with the ONNX files
374
-
375
- ```python
376
- from optimum.onnxruntime import ORTModelForSequenceClassification # type: ignore
377
-
378
- import torch
379
- from transformers import AutoModelForSequenceClassification, AutoTokenizer
380
-
381
- tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-large')
382
- model = AutoModelForSequenceClassification.from_pretrained('BAAI/bge-reranker-base')
383
- model_ort = ORTModelForSequenceClassification.from_pretrained('BAAI/bge-reranker-base', file_name="onnx/model.onnx")
384
-
385
- # Sentences we want sentence embeddings for
386
- pairs = [['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']]
387
-
388
- # Tokenize sentences
389
- encoded_input = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt')
390
-
391
- scores_ort = model_ort(**encoded_input, return_dict=True).logits.view(-1, ).float()
392
- # Compute token embeddings
393
- with torch.inference_mode():
394
- scores = model_ort(**encoded_input, return_dict=True).logits.view(-1, ).float()
395
-
396
- # scores and scores_ort are identical
397
- ```
398
- #### Usage reranker with infinity
399
-
400
- Its also possible to deploy the onnx/torch files with the [infinity_emb](https://github.com/michaelfeil/infinity) pip package.
401
- ```python
402
- import asyncio
403
- from infinity_emb import AsyncEmbeddingEngine, EngineArgs
404
-
405
- query='what is a panda?'
406
- docs = ['The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear', "Paris is in France."]
407
-
408
- engine = AsyncEmbeddingEngine.from_args(
409
- EngineArgs(model_name_or_path = "BAAI/bge-reranker-base", device="cpu", engine="torch" # or engine="optimum" for onnx
410
- ))
411
-
412
- async def main():
413
- async with engine:
414
- ranking, usage = await engine.rerank(query=query, docs=docs)
415
- print(list(zip(ranking, docs)))
416
- asyncio.run(main())
417
- ```
418
-
419
- ## Evaluation
420
-
421
- `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
422
- For more details and evaluation tools see our [scripts](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md).
423
-
424
- - **MTEB**:
425
-
426
- | Model Name | Dimension | Sequence Length | Average (56) | Retrieval (15) |Clustering (11) | Pair Classification (3) | Reranking (4) | STS (10) | Summarization (1) | Classification (12) |
427
- |:----:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
428
- | [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) | 1024 | 512 | **64.23** | **54.29** | 46.08 | 87.12 | 60.03 | 83.11 | 31.61 | 75.97 |
429
- | [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) | 768 | 512 | 63.55 | 53.25 | 45.77 | 86.55 | 58.86 | 82.4 | 31.07 | 75.53 |
430
- | [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) | 384 | 512 | 62.17 |51.68 | 43.82 | 84.92 | 58.36 | 81.59 | 30.12 | 74.14 |
431
- | [bge-large-en](https://huggingface.co/BAAI/bge-large-en) | 1024 | 512 | 63.98 | 53.9 | 46.98 | 85.8 | 59.48 | 81.56 | 32.06 | 76.21 |
432
- | [bge-base-en](https://huggingface.co/BAAI/bge-base-en) | 768 | 512 | 63.36 | 53.0 | 46.32 | 85.86 | 58.7 | 81.84 | 29.27 | 75.27 |
433
- | [gte-large](https://huggingface.co/thenlper/gte-large) | 1024 | 512 | 63.13 | 52.22 | 46.84 | 85.00 | 59.13 | 83.35 | 31.66 | 73.33 |
434
- | [gte-base](https://huggingface.co/thenlper/gte-base) | 768 | 512 | 62.39 | 51.14 | 46.2 | 84.57 | 58.61 | 82.3 | 31.17 | 73.01 |
435
- | [e5-large-v2](https://huggingface.co/intfloat/e5-large-v2) | 1024| 512 | 62.25 | 50.56 | 44.49 | 86.03 | 56.61 | 82.05 | 30.19 | 75.24 |
436
- | [bge-small-en](https://huggingface.co/BAAI/bge-small-en) | 384 | 512 | 62.11 | 51.82 | 44.31 | 83.78 | 57.97 | 80.72 | 30.53 | 74.37 |
437
- | [instructor-xl](https://huggingface.co/hkunlp/instructor-xl) | 768 | 512 | 61.79 | 49.26 | 44.74 | 86.62 | 57.29 | 83.06 | 32.32 | 61.79 |
438
- | [e5-base-v2](https://huggingface.co/intfloat/e5-base-v2) | 768 | 512 | 61.5 | 50.29 | 43.80 | 85.73 | 55.91 | 81.05 | 30.28 | 73.84 |
439
- | [gte-small](https://huggingface.co/thenlper/gte-small) | 384 | 512 | 61.36 | 49.46 | 44.89 | 83.54 | 57.7 | 82.07 | 30.42 | 72.31 |
440
- | [text-embedding-ada-002](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 | 60.99 | 49.25 | 45.9 | 84.89 | 56.32 | 80.97 | 30.8 | 70.93 |
441
- | [e5-small-v2](https://huggingface.co/intfloat/e5-base-v2) | 384 | 512 | 59.93 | 49.04 | 39.92 | 84.67 | 54.32 | 80.39 | 31.16 | 72.94 |
442
- | [sentence-t5-xxl](https://huggingface.co/sentence-transformers/sentence-t5-xxl) | 768 | 512 | 59.51 | 42.24 | 43.72 | 85.06 | 56.42 | 82.63 | 30.08 | 73.42 |
443
- | [all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) | 768 | 514 | 57.78 | 43.81 | 43.69 | 83.04 | 59.36 | 80.28 | 27.49 | 65.07 |
444
- | [sgpt-bloom-7b1-msmarco](https://huggingface.co/bigscience/sgpt-bloom-7b1-msmarco) | 4096 | 2048 | 57.59 | 48.22 | 38.93 | 81.9 | 55.65 | 77.74 | 33.6 | 66.19 |
445
-
446
-
447
-
448
- - **C-MTEB**:
449
- We create the benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
450
- Please refer to [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md) for a detailed introduction.
451
-
452
- | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
453
- |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
454
- | [**BAAI/bge-large-zh-v1.5**](https://huggingface.co/BAAI/bge-large-zh-v1.5) | 1024 | **64.53** | 70.46 | 56.25 | 81.6 | 69.13 | 65.84 | 48.99 |
455
- | [BAAI/bge-base-zh-v1.5](https://huggingface.co/BAAI/bge-base-zh-v1.5) | 768 | 63.13 | 69.49 | 53.72 | 79.75 | 68.07 | 65.39 | 47.53 |
456
- | [BAAI/bge-small-zh-v1.5](https://huggingface.co/BAAI/bge-small-zh-v1.5) | 512 | 57.82 | 61.77 | 49.11 | 70.41 | 63.96 | 60.92 | 44.18 |
457
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | 1024 | 64.20 | 71.53 | 54.98 | 78.94 | 68.32 | 65.11 | 48.39 |
458
- | [bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | 1024 | 63.53 | 70.55 | 53 | 76.77 | 68.58 | 64.91 | 50.01 |
459
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | 768 | 62.96 | 69.53 | 54.12 | 77.5 | 67.07 | 64.91 | 47.63 |
460
- | [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 1024 | 58.79 | 63.66 | 48.44 | 69.89 | 67.34 | 56.00 | 48.23 |
461
- | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | 512 | 58.27 | 63.07 | 49.45 | 70.35 | 63.64 | 61.48 | 45.09 |
462
- | [m3e-base](https://huggingface.co/moka-ai/m3e-base) | 768 | 57.10 | 56.91 | 50.47 | 63.99 | 67.52 | 59.34 | 47.68 |
463
- | [m3e-large](https://huggingface.co/moka-ai/m3e-large) | 1024 | 57.05 | 54.75 | 50.42 | 64.3 | 68.2 | 59.66 | 48.88 |
464
- | [multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 768 | 55.48 | 61.63 | 46.49 | 67.07 | 65.35 | 54.35 | 40.68 |
465
- | [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) | 384 | 55.38 | 59.95 | 45.27 | 66.45 | 65.85 | 53.86 | 45.26 |
466
- | [text-embedding-ada-002(OpenAI)](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) | 1536 | 53.02 | 52.0 | 43.35 | 69.56 | 64.31 | 54.28 | 45.68 |
467
- | [luotuo](https://huggingface.co/silk-road/luotuo-bert-medium) | 1024 | 49.37 | 44.4 | 42.78 | 66.62 | 61 | 49.25 | 44.39 |
468
- | [text2vec-base](https://huggingface.co/shibing624/text2vec-base-chinese) | 768 | 47.63 | 38.79 | 43.41 | 67.41 | 62.19 | 49.45 | 37.66 |
469
- | [text2vec-large](https://huggingface.co/GanymedeNil/text2vec-large-chinese) | 1024 | 47.36 | 41.94 | 44.97 | 70.86 | 60.66 | 49.16 | 30.02 |
470
-
471
-
472
- - **Reranking**:
473
- See [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/) for evaluation script.
474
-
475
- | Model | T2Reranking | T2RerankingZh2En\* | T2RerankingEn2Zh\* | MMarcoReranking | CMedQAv1 | CMedQAv2 | Avg |
476
- |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
477
- | text2vec-base-multilingual | 64.66 | 62.94 | 62.51 | 14.37 | 48.46 | 48.6 | 50.26 |
478
- | multilingual-e5-small | 65.62 | 60.94 | 56.41 | 29.91 | 67.26 | 66.54 | 57.78 |
479
- | multilingual-e5-large | 64.55 | 61.61 | 54.28 | 28.6 | 67.42 | 67.92 | 57.4 |
480
- | multilingual-e5-base | 64.21 | 62.13 | 54.68 | 29.5 | 66.23 | 66.98 | 57.29 |
481
- | m3e-base | 66.03 | 62.74 | 56.07 | 17.51 | 77.05 | 76.76 | 59.36 |
482
- | m3e-large | 66.13 | 62.72 | 56.1 | 16.46 | 77.76 | 78.27 | 59.57 |
483
- | bge-base-zh-v1.5 | 66.49 | 63.25 | 57.02 | 29.74 | 80.47 | 84.88 | 63.64 |
484
- | bge-large-zh-v1.5 | 65.74 | 63.39 | 57.03 | 28.74 | 83.45 | 85.44 | 63.97 |
485
- | [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | 67.28 | 63.95 | 60.45 | 35.46 | 81.26 | 84.1 | 65.42 |
486
- | [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | 67.6 | 64.03 | 61.44 | 37.16 | 82.15 | 84.18 | 66.09 |
487
-
488
- \* : T2RerankingZh2En and T2RerankingEn2Zh are cross-language retrieval tasks
489
-
490
- ## Train
491
-
492
- ### BAAI Embedding
493
-
494
- We pre-train the models using [retromae](https://github.com/staoxiao/RetroMAE) and train them on large-scale pairs data using contrastive learning.
495
- **You can fine-tune the embedding model on your data following our [examples](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).**
496
- We also provide a [pre-train example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain).
497
- Note that the goal of pre-training is to reconstruct the text, and the pre-trained model cannot be used for similarity calculation directly, it needs to be fine-tuned.
498
- More training details for bge see [baai_general_embedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md).
499
-
500
-
501
-
502
- ### BGE Reranker
503
-
504
- Cross-encoder will perform full-attention over the input pair,
505
- which is more accurate than embedding model (i.e., bi-encoder) but more time-consuming than embedding model.
506
- Therefore, it can be used to re-rank the top-k documents returned by embedding model.
507
- We train the cross-encoder on a multilingual pair data,
508
- The data format is the same as embedding model, so you can fine-tune it easily following our [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker).
509
- More details please refer to [./FlagEmbedding/reranker/README.md](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker)
510
-
511
-
512
-
513
- ## Citation
514
-
515
- If you find this repository useful, please consider giving a star :star: and citation
516
-
517
- ```
518
- @misc{bge_embedding,
519
- title={C-Pack: Packaged Resources To Advance General Chinese Embedding},
520
- author={Shitao Xiao and Zheng Liu and Peitian Zhang and Niklas Muennighoff},
521
- year={2023},
522
- eprint={2309.07597},
523
- archivePrefix={arXiv},
524
- primaryClass={cs.CL}
525
- }
526
- ```
527
-
528
- ## License
529
- FlagEmbedding is licensed under the [MIT License](https://github.com/FlagOpen/FlagEmbedding/blob/master/LICENSE). The released models can be used for commercial purposes free of charge.