redisbench-admin 0.11.55__py3-none-any.whl → 0.11.57__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (137) hide show
  1. redisbench_admin/environments/oss_cluster.py +9 -1
  2. redisbench_admin/run/asm.py +1 -1
  3. redisbench_admin/run_remote/remote_helpers.py +41 -11
  4. redisbench_admin/run_remote/standalone.py +2 -3
  5. redisbench_admin/utils/remote.py +2 -0
  6. {redisbench_admin-0.11.55.dist-info → redisbench_admin-0.11.57.dist-info}/METADATA +7 -4
  7. redisbench_admin-0.11.57.dist-info/RECORD +117 -0
  8. {redisbench_admin-0.11.55.dist-info → redisbench_admin-0.11.57.dist-info}/WHEEL +1 -1
  9. redisbench_admin/run/ann/pkg/.dockerignore +0 -2
  10. redisbench_admin/run/ann/pkg/.git +0 -1
  11. redisbench_admin/run/ann/pkg/.github/workflows/benchmarks.yml +0 -100
  12. redisbench_admin/run/ann/pkg/.gitignore +0 -21
  13. redisbench_admin/run/ann/pkg/LICENSE +0 -21
  14. redisbench_admin/run/ann/pkg/README.md +0 -157
  15. redisbench_admin/run/ann/pkg/algos.yaml +0 -1294
  16. redisbench_admin/run/ann/pkg/algosP.yaml +0 -67
  17. redisbench_admin/run/ann/pkg/ann_benchmarks/__init__.py +0 -2
  18. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/__init__.py +0 -0
  19. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/annoy.py +0 -26
  20. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/balltree.py +0 -22
  21. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/base.py +0 -36
  22. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/bruteforce.py +0 -110
  23. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/ckdtree.py +0 -17
  24. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/datasketch.py +0 -29
  25. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/definitions.py +0 -187
  26. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/diskann.py +0 -190
  27. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/dolphinnpy.py +0 -31
  28. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/dummy_algo.py +0 -25
  29. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/elasticsearch.py +0 -107
  30. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/elastiknn.py +0 -124
  31. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/faiss.py +0 -124
  32. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/faiss_gpu.py +0 -61
  33. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/faiss_hnsw.py +0 -39
  34. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/flann.py +0 -27
  35. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/hnswlib.py +0 -36
  36. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/kdtree.py +0 -22
  37. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/kgraph.py +0 -39
  38. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/lshf.py +0 -25
  39. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/milvus.py +0 -99
  40. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/mrpt.py +0 -41
  41. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/n2.py +0 -28
  42. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/nearpy.py +0 -48
  43. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/nmslib.py +0 -74
  44. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/onng_ngt.py +0 -100
  45. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/opensearchknn.py +0 -107
  46. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/panng_ngt.py +0 -79
  47. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/pinecone.py +0 -39
  48. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/puffinn.py +0 -45
  49. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/pynndescent.py +0 -115
  50. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/qg_ngt.py +0 -102
  51. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/redisearch.py +0 -90
  52. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/rpforest.py +0 -20
  53. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/scann.py +0 -34
  54. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/sptag.py +0 -28
  55. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/subprocess.py +0 -246
  56. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/vald.py +0 -149
  57. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/vecsim-hnsw.py +0 -43
  58. redisbench_admin/run/ann/pkg/ann_benchmarks/algorithms/vespa.py +0 -47
  59. redisbench_admin/run/ann/pkg/ann_benchmarks/constants.py +0 -1
  60. redisbench_admin/run/ann/pkg/ann_benchmarks/data.py +0 -48
  61. redisbench_admin/run/ann/pkg/ann_benchmarks/datasets.py +0 -620
  62. redisbench_admin/run/ann/pkg/ann_benchmarks/distance.py +0 -53
  63. redisbench_admin/run/ann/pkg/ann_benchmarks/main.py +0 -325
  64. redisbench_admin/run/ann/pkg/ann_benchmarks/plotting/__init__.py +0 -2
  65. redisbench_admin/run/ann/pkg/ann_benchmarks/plotting/metrics.py +0 -183
  66. redisbench_admin/run/ann/pkg/ann_benchmarks/plotting/plot_variants.py +0 -17
  67. redisbench_admin/run/ann/pkg/ann_benchmarks/plotting/utils.py +0 -165
  68. redisbench_admin/run/ann/pkg/ann_benchmarks/results.py +0 -71
  69. redisbench_admin/run/ann/pkg/ann_benchmarks/runner.py +0 -333
  70. redisbench_admin/run/ann/pkg/create_dataset.py +0 -12
  71. redisbench_admin/run/ann/pkg/create_hybrid_dataset.py +0 -147
  72. redisbench_admin/run/ann/pkg/create_text_to_image_ds.py +0 -117
  73. redisbench_admin/run/ann/pkg/create_website.py +0 -272
  74. redisbench_admin/run/ann/pkg/install/Dockerfile +0 -11
  75. redisbench_admin/run/ann/pkg/install/Dockerfile.annoy +0 -5
  76. redisbench_admin/run/ann/pkg/install/Dockerfile.datasketch +0 -4
  77. redisbench_admin/run/ann/pkg/install/Dockerfile.diskann +0 -29
  78. redisbench_admin/run/ann/pkg/install/Dockerfile.diskann_pq +0 -31
  79. redisbench_admin/run/ann/pkg/install/Dockerfile.dolphinn +0 -5
  80. redisbench_admin/run/ann/pkg/install/Dockerfile.elasticsearch +0 -45
  81. redisbench_admin/run/ann/pkg/install/Dockerfile.elastiknn +0 -61
  82. redisbench_admin/run/ann/pkg/install/Dockerfile.faiss +0 -18
  83. redisbench_admin/run/ann/pkg/install/Dockerfile.flann +0 -10
  84. redisbench_admin/run/ann/pkg/install/Dockerfile.hnswlib +0 -10
  85. redisbench_admin/run/ann/pkg/install/Dockerfile.kgraph +0 -6
  86. redisbench_admin/run/ann/pkg/install/Dockerfile.mih +0 -4
  87. redisbench_admin/run/ann/pkg/install/Dockerfile.milvus +0 -27
  88. redisbench_admin/run/ann/pkg/install/Dockerfile.mrpt +0 -4
  89. redisbench_admin/run/ann/pkg/install/Dockerfile.n2 +0 -5
  90. redisbench_admin/run/ann/pkg/install/Dockerfile.nearpy +0 -5
  91. redisbench_admin/run/ann/pkg/install/Dockerfile.ngt +0 -13
  92. redisbench_admin/run/ann/pkg/install/Dockerfile.nmslib +0 -10
  93. redisbench_admin/run/ann/pkg/install/Dockerfile.opensearchknn +0 -43
  94. redisbench_admin/run/ann/pkg/install/Dockerfile.puffinn +0 -6
  95. redisbench_admin/run/ann/pkg/install/Dockerfile.pynndescent +0 -4
  96. redisbench_admin/run/ann/pkg/install/Dockerfile.redisearch +0 -18
  97. redisbench_admin/run/ann/pkg/install/Dockerfile.rpforest +0 -5
  98. redisbench_admin/run/ann/pkg/install/Dockerfile.scann +0 -5
  99. redisbench_admin/run/ann/pkg/install/Dockerfile.scipy +0 -4
  100. redisbench_admin/run/ann/pkg/install/Dockerfile.sklearn +0 -4
  101. redisbench_admin/run/ann/pkg/install/Dockerfile.sptag +0 -30
  102. redisbench_admin/run/ann/pkg/install/Dockerfile.vald +0 -8
  103. redisbench_admin/run/ann/pkg/install/Dockerfile.vespa +0 -17
  104. redisbench_admin/run/ann/pkg/install.py +0 -70
  105. redisbench_admin/run/ann/pkg/logging.conf +0 -34
  106. redisbench_admin/run/ann/pkg/multirun.py +0 -298
  107. redisbench_admin/run/ann/pkg/plot.py +0 -159
  108. redisbench_admin/run/ann/pkg/protocol/bf-runner +0 -10
  109. redisbench_admin/run/ann/pkg/protocol/bf-runner.py +0 -204
  110. redisbench_admin/run/ann/pkg/protocol/ext-add-query-metric.md +0 -51
  111. redisbench_admin/run/ann/pkg/protocol/ext-batch-queries.md +0 -77
  112. redisbench_admin/run/ann/pkg/protocol/ext-prepared-queries.md +0 -77
  113. redisbench_admin/run/ann/pkg/protocol/ext-query-parameters.md +0 -47
  114. redisbench_admin/run/ann/pkg/protocol/specification.md +0 -194
  115. redisbench_admin/run/ann/pkg/requirements.txt +0 -14
  116. redisbench_admin/run/ann/pkg/requirements_py38.txt +0 -11
  117. redisbench_admin/run/ann/pkg/results/fashion-mnist-784-euclidean.png +0 -0
  118. redisbench_admin/run/ann/pkg/results/gist-960-euclidean.png +0 -0
  119. redisbench_admin/run/ann/pkg/results/glove-100-angular.png +0 -0
  120. redisbench_admin/run/ann/pkg/results/glove-25-angular.png +0 -0
  121. redisbench_admin/run/ann/pkg/results/lastfm-64-dot.png +0 -0
  122. redisbench_admin/run/ann/pkg/results/mnist-784-euclidean.png +0 -0
  123. redisbench_admin/run/ann/pkg/results/nytimes-256-angular.png +0 -0
  124. redisbench_admin/run/ann/pkg/results/sift-128-euclidean.png +0 -0
  125. redisbench_admin/run/ann/pkg/run.py +0 -12
  126. redisbench_admin/run/ann/pkg/run_algorithm.py +0 -3
  127. redisbench_admin/run/ann/pkg/templates/chartjs.template +0 -102
  128. redisbench_admin/run/ann/pkg/templates/detail_page.html +0 -23
  129. redisbench_admin/run/ann/pkg/templates/general.html +0 -58
  130. redisbench_admin/run/ann/pkg/templates/latex.template +0 -30
  131. redisbench_admin/run/ann/pkg/templates/summary.html +0 -60
  132. redisbench_admin/run/ann/pkg/test/__init__.py +0 -0
  133. redisbench_admin/run/ann/pkg/test/test-jaccard.py +0 -19
  134. redisbench_admin/run/ann/pkg/test/test-metrics.py +0 -99
  135. redisbench_admin-0.11.55.dist-info/RECORD +0 -243
  136. {redisbench_admin-0.11.55.dist-info → redisbench_admin-0.11.57.dist-info}/entry_points.txt +0 -0
  137. {redisbench_admin-0.11.55.dist-info → redisbench_admin-0.11.57.dist-info/licenses}/LICENSE +0 -0
@@ -201,7 +201,15 @@ def generate_cluster_redis_server_args(
201
201
  dbfilename = get_cluster_dbfilename(port)
202
202
 
203
203
  command = generate_common_server_args(
204
- binary, daemonize, dbdir, dbfilename, enable_debug_command, ip, logfile, port
204
+ binary,
205
+ daemonize,
206
+ dbdir,
207
+ dbfilename,
208
+ enable_debug_command,
209
+ ip,
210
+ logfile,
211
+ port,
212
+ enable_redis_7_config_directives,
205
213
  )
206
214
  command.extend(
207
215
  [
@@ -145,7 +145,7 @@ class ShardSlotInfo:
145
145
  """
146
146
  if shard_index < 0 or shard_index >= len(self.shards):
147
147
  raise IndexError(
148
- f"Shard index {shard_index} out of range (0..{len(self.shards)-1})"
148
+ f"Shard index {shard_index} out of range (0..{len(self.shards) - 1})"
149
149
  )
150
150
 
151
151
  shard = self.shards[shard_index]
@@ -205,8 +205,7 @@ def remote_tool_pre_bench_step(
205
205
  )
206
206
  logging.info("Finished up remote tool {} requirements".format(benchmark_tool))
207
207
 
208
-
209
- def setup_remote_benchmark_tool_requirements_ftsb(
208
+ def _setup_remote_benchmark_tool_requirements(
210
209
  client_public_ip,
211
210
  username,
212
211
  private_key,
@@ -218,14 +217,43 @@ def setup_remote_benchmark_tool_requirements_ftsb(
218
217
  ):
219
218
  commands = [
220
219
  "wget {} -q -O {}".format(tool_link, remote_tool_link),
221
- "wget {} -q -O {}".format(queries_file_link, remote_input_file),
222
- "chmod 755 {}".format(remote_tool_link),
220
+ "chmod 755 {}".format(remote_tool_link)
223
221
  ]
222
+
223
+ # detect if queries_file_link is a s3 URI or http one and act accordingly (use aws cli or wget)
224
+ if queries_file_link.startswith("s3://"):
225
+ commands.append(
226
+ "aws s3 cp {} {} --no-sign-request".format(queries_file_link, remote_input_file)
227
+ )
228
+ else:
229
+ commands.append("wget {} -q -O {}".format(queries_file_link, remote_input_file))
224
230
  execute_remote_commands(
225
231
  client_public_ip, username, private_key, commands, client_ssh_port
226
232
  )
227
233
 
228
234
 
235
+ def setup_remote_benchmark_tool_requirements_ftsb(
236
+ client_public_ip,
237
+ username,
238
+ private_key,
239
+ tool_link,
240
+ queries_file_link,
241
+ remote_tool_link,
242
+ remote_input_file,
243
+ client_ssh_port,
244
+ ):
245
+ _setup_remote_benchmark_tool_requirements(
246
+ client_public_ip,
247
+ username,
248
+ private_key,
249
+ tool_link,
250
+ queries_file_link,
251
+ remote_tool_link,
252
+ remote_input_file,
253
+ client_ssh_port,
254
+ )
255
+
256
+
229
257
  def setup_remote_benchmark_tool_requirements_tsbs(
230
258
  client_public_ip,
231
259
  username,
@@ -236,13 +264,15 @@ def setup_remote_benchmark_tool_requirements_tsbs(
236
264
  remote_input_file,
237
265
  client_ssh_port,
238
266
  ):
239
- commands = [
240
- "wget {} -q -O {}".format(tool_link, remote_tool_link),
241
- "wget {} -q -O {}".format(queries_file_link, remote_input_file),
242
- "chmod 755 {}".format(remote_tool_link),
243
- ]
244
- execute_remote_commands(
245
- client_public_ip, username, private_key, commands, client_ssh_port
267
+ _setup_remote_benchmark_tool_requirements(
268
+ client_public_ip,
269
+ username,
270
+ private_key,
271
+ tool_link,
272
+ queries_file_link,
273
+ remote_tool_link,
274
+ remote_input_file,
275
+ client_ssh_port,
246
276
  )
247
277
 
248
278
 
@@ -13,7 +13,6 @@ from redisbench_admin.utils.remote import (
13
13
  )
14
14
  from redisbench_admin.utils.ssh import SSHSession
15
15
  from redisbench_admin.utils.utils import redis_server_config_module_part
16
- import tempfile
17
16
 
18
17
 
19
18
  def ensure_redis_server_available(server_public_ip, username, private_key, port=22):
@@ -516,7 +515,7 @@ def spin_test_standalone_redis(
516
515
  return False
517
516
 
518
517
  remote_redis_conf_path = f"{temporary_dir}/redis.conf"
519
- logging.info(f"📁 Copying custom redis.conf to remote host...")
518
+ logging.info("📁 Copying custom redis.conf to remote host...")
520
519
 
521
520
  copy_result = copy_file_to_remote_setup(
522
521
  server_public_ip,
@@ -541,7 +540,7 @@ def spin_test_standalone_redis(
541
540
  return False
542
541
 
543
542
  remote_redis_server_path = f"{temporary_dir}/redis-server"
544
- logging.info(f"📁 Copying custom redis-server binary to remote host...")
543
+ logging.info("📁 Copying custom redis-server binary to remote host...")
545
544
 
546
545
  copy_result = copy_file_to_remote_setup(
547
546
  server_public_ip,
@@ -178,6 +178,8 @@ def connect_remote_ssh(port, private_key, server_public_ip, username):
178
178
  c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
179
179
  logging.info("Connecting to remote server {}".format(server_public_ip))
180
180
  c.connect(hostname=server_public_ip, port=port, username=username, pkey=k)
181
+ transport = c.get_transport()
182
+ transport.set_keepalive(10) # Send keepalive every 10 seconds
181
183
  logging.info("Connected to remote server {}".format(server_public_ip))
182
184
  return c
183
185
 
@@ -1,7 +1,8 @@
1
- Metadata-Version: 2.1
1
+ Metadata-Version: 2.4
2
2
  Name: redisbench-admin
3
- Version: 0.11.55
3
+ Version: 0.11.57
4
4
  Summary: Redis benchmark run helper. A wrapper around Redis and Redis Modules benchmark tools ( ftsb_redisearch, memtier_benchmark, redis-benchmark, aibench, etc... ).
5
+ License-File: LICENSE
5
6
  Author: filipecosta90
6
7
  Author-email: filipecosta.90@gmail.com
7
8
  Requires-Python: >=3.10.0,<4.0.0
@@ -9,18 +10,20 @@ Classifier: Programming Language :: Python :: 3
9
10
  Classifier: Programming Language :: Python :: 3.10
10
11
  Classifier: Programming Language :: Python :: 3.11
11
12
  Classifier: Programming Language :: Python :: 3.12
13
+ Classifier: Programming Language :: Python :: 3.13
14
+ Classifier: Programming Language :: Python :: 3.14
12
15
  Requires-Dist: Flask (>=2.0.1,<3.0.0)
13
16
  Requires-Dist: Flask-HTTPAuth (>=4.4.0,<5.0.0)
14
17
  Requires-Dist: GitPython (>=3.1.12,<4.0.0)
15
18
  Requires-Dist: Jinja2 (>=3.0.3,<4.0.0)
16
- Requires-Dist: PyYAML (>=6.0,<7.0)
19
+ Requires-Dist: PyYAML (>=6.0.1,<7.0.0)
17
20
  Requires-Dist: boto3 (>=1.13.24,<2.0.0)
18
21
  Requires-Dist: certifi (>=2021.10.8,<2025.0.0)
19
22
  Requires-Dist: daemonize (>=2.5.0,<3.0.0)
20
23
  Requires-Dist: flask-restx (>=0.5.1,<0.6.0)
21
24
  Requires-Dist: humanize (>=2.4.0,<3.0.0)
22
25
  Requires-Dist: jsonpath_ng (>=1.5.2,<2.0.0)
23
- Requires-Dist: matplotlib (>=3.1.2,<4.0.0)
26
+ Requires-Dist: matplotlib (>=3.9.0,<4.0.0)
24
27
  Requires-Dist: numpy (>=2.0.0,<3.0.0)
25
28
  Requires-Dist: pandas (>=2.1.2,<3.0.0)
26
29
  Requires-Dist: paramiko (>=2.7.2,<4.0.0)
@@ -0,0 +1,117 @@
1
+ redisbench_admin/__init__.py,sha256=kLgzVDw4-frGXtbJH_WKiwWcXaUZYisYBKYYOeSJWvU,473
2
+ redisbench_admin/cli.py,sha256=Kljn7e6zV8kCGRGSfZCf1SdHdvtW8d3kA5LwMba68Zw,8073
3
+ redisbench_admin/commands/__init__.py,sha256=mzVrEtqefFdopyzR-W6xx3How95dyZfToGKm1-_YzeY,95
4
+ redisbench_admin/commands/commands.json.py,sha256=mzVrEtqefFdopyzR-W6xx3How95dyZfToGKm1-_YzeY,95
5
+ redisbench_admin/compare/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
6
+ redisbench_admin/compare/args.py,sha256=rOP8td_GYPYsZiYN4olyYmUsku-oizggEZNRTB_fVro,6336
7
+ redisbench_admin/compare/compare.py,sha256=NvC7RbQUJUKlPdJQj9TjUmk2k8UtiA9wnZWNDV7IM_Q,103315
8
+ redisbench_admin/deploy/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
9
+ redisbench_admin/deploy/args.py,sha256=neLUcQqI__HkJItkQg2C293hl5g3yHG40t171r7-E5Y,1732
10
+ redisbench_admin/deploy/deploy.py,sha256=MtfJbsL97DLrbBYut6zRCzyEMebX4xWoZE-m4-JDRB8,3885
11
+ redisbench_admin/environments/__init__.py,sha256=cD7zfXt0VEmy0b7452HvcAxX_9kVj6Vm213yNdUHP20,95
12
+ redisbench_admin/environments/oss_cluster.py,sha256=qiHjhmkQ6cqvsD-4pLxXX1S1Hb2LarSg5V-XuXCoTZo,8241
13
+ redisbench_admin/environments/oss_standalone.py,sha256=Sl38rUpwJ3wNOl9zn38iK8q2iJi2pRFmaJAZJbuT_SQ,2474
14
+ redisbench_admin/export/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
15
+ redisbench_admin/export/args.py,sha256=v_WjJCNz_LeIFMNwSN6XwRmvSx1K2ys8XS1gK50EM_4,3508
16
+ redisbench_admin/export/common/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
17
+ redisbench_admin/export/common/common.py,sha256=LnvXjMLlJRzMTxiFIjrfRFfDx9JJm88OZHu7lnTOpFA,4331
18
+ redisbench_admin/export/export.py,sha256=u00NjaCbWhCJ319leVlP4ZkqiqZt5FN4Gbag4Poo23M,11274
19
+ redisbench_admin/export/google_benchmark/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
20
+ redisbench_admin/export/google_benchmark/google_benchmark_json_format.py,sha256=OuMaMmmma5VvXA0rcLIQSMxIq81oa5I3xYDFhbWj-IA,1804
21
+ redisbench_admin/export/memtier_benchmark/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
22
+ redisbench_admin/export/memtier_benchmark/memtier_benchmark_json_format.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
23
+ redisbench_admin/export/pyperf/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
24
+ redisbench_admin/export/pyperf/pyperf_json_format.py,sha256=g3S68cRAMJiogpUP31-hVltw04xN1LxHszC3fBshO3U,1677
25
+ redisbench_admin/export/redis_benchmark/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
26
+ redisbench_admin/export/redis_benchmark/metrics_definition.py,sha256=d_w37WSLtp0JZ0u-Ge1R3Hv43_bycLRCXBZ9zDR5-uE,2522
27
+ redisbench_admin/export/redis_benchmark/redis_benchmark_csv_format.py,sha256=a0_aqO4NN54FCE-TIDJ2Yqf5aFU4tlC0fHZTcQnDvEE,5780
28
+ redisbench_admin/extract/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
29
+ redisbench_admin/extract/args.py,sha256=OrEwGOd58fu1F4moUrzFq1bUb69mpiMSGeh9tGthq0o,1073
30
+ redisbench_admin/extract/extract.py,sha256=V1rAFwbOkIFfcKLH_82K4zNAcJtaQn3wz60ddzRzxBc,770
31
+ redisbench_admin/grafana_api/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
32
+ redisbench_admin/grafana_api/app.py,sha256=F4bw7ToO9bHB6mj6y9EhiTBxSNzFBHJfWJT7AcZBpl8,3416
33
+ redisbench_admin/grafana_api/args.py,sha256=7chFoUcTASHsSYLgzGWH6VEP_gtdnvK3AgLC_sR4D6o,1212
34
+ redisbench_admin/grafana_api/grafana_api.py,sha256=dG17GCYmWRILmy7h3-OiBeGzuNGnRGUv-jqR-7MRiws,1613
35
+ redisbench_admin/profilers/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
36
+ redisbench_admin/profilers/daemon.py,sha256=xm1P5c0Lv91kllT4DRLDJH4azGuPHtnpgNeGjC6ecpc,12975
37
+ redisbench_admin/profilers/flamegraph.pl,sha256=Za5XE-1gb_U-nzqwoyRwfe1TB182c64gITa-2klWTTA,35898
38
+ redisbench_admin/profilers/perf.py,sha256=HtzzMVsXEJa1H7tOAfKlbFYDn2KnxAG_IU9yKPKZB7w,27772
39
+ redisbench_admin/profilers/perf_daemon_caller.py,sha256=nD97cXmX3JytyafvNMmhUBq40uYrf6vtjdJ1TXZbvVY,4948
40
+ redisbench_admin/profilers/pprof.py,sha256=g7oNC3AtNDTUOBIh_mIi5bFl_b0mL8tqBu6qKvAOrKw,3949
41
+ redisbench_admin/profilers/profilers.py,sha256=4C1xaPyLoPydJ3eBAxW7IlSHG-3qj3A3BAKejiZXEK0,510
42
+ redisbench_admin/profilers/profilers_local.py,sha256=etU6f5EdEq2a6JrakwPVfaoHlBSb0sycTEYg5ugvjVw,12220
43
+ redisbench_admin/profilers/profilers_schema.py,sha256=XvKURihYh9Qu8lB7HXNHqgcb_G6eBznC2J70VxU3c5E,2744
44
+ redisbench_admin/profilers/stackcollapse-perf.pl,sha256=XStXxBheOVjJYdo9Gha0rf5PZb4o1J1IJ-pQY77FV08,10307
45
+ redisbench_admin/profilers/vtune.py,sha256=OMx5ABr5OtWeiqK0dTspdyeAfKlDEzhdXDKocDSa9xM,4618
46
+ redisbench_admin/run/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
47
+ redisbench_admin/run/aibench_run_inference_redisai_vision/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
48
+ redisbench_admin/run/aibench_run_inference_redisai_vision/aibench_run_inference_redisai_vision.py,sha256=03GDbZZSo21f9iBTcNTdBKQAfq_Ghd0zxr-KYe1QIMU,1917
49
+ redisbench_admin/run/ann/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
50
+ redisbench_admin/run/ann/ann.py,sha256=h8VE7fouyNkpMSjNhgjkI_R-aWb-lKIKxvNZPOO8pFY,1288
51
+ redisbench_admin/run/args.py,sha256=4OJIj19fieEjvz7Xnk1rIjaa50R5Kv1IlyMyECHtj0Y,8219
52
+ redisbench_admin/run/asm.py,sha256=Hybna8Xs7SnsP2IBwg6kMsiGPoCK0TlvhyKX94XmOEs,16738
53
+ redisbench_admin/run/cluster.py,sha256=_Y6a8Dbu1cJ7OxhgymKQSZcCmV8cZ3UpGEWL6b6O84Y,6363
54
+ redisbench_admin/run/common.py,sha256=1zJPQtIk3tjvLTjNUgxvDdcviN73psiaDBQ8cRJMIl8,29341
55
+ redisbench_admin/run/ftsb/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
56
+ redisbench_admin/run/ftsb/ftsb.py,sha256=MnviWzIxMPxAw__R1lQT6tTsH7nP7t5ejCXrALm2MUw,2580
57
+ redisbench_admin/run/git.py,sha256=6UYGcTN0MPzf4QDVoJnFkou0yZasLF6jLG7f0zoySq8,3064
58
+ redisbench_admin/run/grafana.py,sha256=iMDgMyJKinpZMTD43rZ1IcRGkadjFjCxaB48mYWkvG4,9421
59
+ redisbench_admin/run/memtier_benchmark/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
60
+ redisbench_admin/run/memtier_benchmark/memtier_benchmark.py,sha256=wTd2olovvFBZ98mOSr6DM5BJsdaiuPteEZzBqeSgbkE,4246
61
+ redisbench_admin/run/metrics.py,sha256=8EQdcZbCiFB_kIR1WtUQNOPV8y74bZ8Dj51Cv0aR4nk,7556
62
+ redisbench_admin/run/modules.py,sha256=9To85oDw2tmUNmTDxOgvKls_46oZRcd2cCt6xNjIWiA,1691
63
+ redisbench_admin/run/redis_benchmark/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
64
+ redisbench_admin/run/redis_benchmark/redis_benchmark.py,sha256=e-Az2uTlt3z2W4uzlUsdxeT8GITpxpGb-Mjb6JxrSWc,6848
65
+ redisbench_admin/run/redisgraph_benchmark_go/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
66
+ redisbench_admin/run/redisgraph_benchmark_go/redisgraph_benchmark_go.py,sha256=DijIoWr73e2t7Zy-UmODzC51IjUh6hH5I0LaPRJW4Vk,2241
67
+ redisbench_admin/run/redistimeseries.py,sha256=x3PA7QoHXu53zs5v0ekK2sVmUnA9_ZF2JxgCDf1Mui4,21331
68
+ redisbench_admin/run/run.py,sha256=AwE4_qkejAXOcDVFmwHjVqYjqi1BFZKdGNrCKBHkAHI,6030
69
+ redisbench_admin/run/s3.py,sha256=pXQXZ1rrwDCWeBegGR4aKzbKqWWwMrmqvIjFxEB3bh4,442
70
+ redisbench_admin/run/ssh.py,sha256=gRW6ROoTKlaxLKhS5tM-Ejjd6zk2iO1KN9dzBzw7GOk,2835
71
+ redisbench_admin/run/tsbs_run_queries_redistimeseries/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
72
+ redisbench_admin/run/tsbs_run_queries_redistimeseries/tsbs_run_queries_redistimeseries.py,sha256=CfXcoLS25pEkhGtka82JwgDZVPV4w2lU_VhI-reH5h4,2514
73
+ redisbench_admin/run/ycsb/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
74
+ redisbench_admin/run/ycsb/ycsb.py,sha256=cs5saVH7C4YpDvzhoa15PwEho59qTVR1E90v_FYjMVw,6873
75
+ redisbench_admin/run_async/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
76
+ redisbench_admin/run_async/async_env.py,sha256=tE1turaaZNHfOaSpGxh62EJWp88zoQFUf3sMbaS7JRA,2408
77
+ redisbench_admin/run_async/async_terraform.py,sha256=ngOQnECUuC20pZwiJItaiBnzlwT2DiKciPTHtqLURe4,11299
78
+ redisbench_admin/run_async/benchmark.py,sha256=S-dsaWGjgsPQxj8sXAACnbtNw5zlJnRFoo53ULbrMEY,1630
79
+ redisbench_admin/run_async/log.py,sha256=cD7zfXt0VEmy0b7452HvcAxX_9kVj6Vm213yNdUHP20,95
80
+ redisbench_admin/run_async/render_files.py,sha256=NMagmx-2hsMET_XN8tkmQz55g-azqW7SjAqaq4GL8F0,2676
81
+ redisbench_admin/run_async/run_async.py,sha256=g2ZOQqj9vXZYaRyNpJZtgfYyY9tMuRmEv3Hh3qWOUs8,14525
82
+ redisbench_admin/run_local/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
83
+ redisbench_admin/run_local/args.py,sha256=LPpqtx1cH1dkkeHjYlaFnAp_TijxnzPZFO2CmYD9ikU,1906
84
+ redisbench_admin/run_local/local_client.py,sha256=gwawMDOBrf7m--uyxu8kMZC5LBiLjbUBSKvzVOdOAas,124
85
+ redisbench_admin/run_local/local_db.py,sha256=9vINqKOs-wDMFEuEHT0I8KO9YnEo_h4NWNk5da3LwSY,7518
86
+ redisbench_admin/run_local/local_helpers.py,sha256=JyqLW2-Sbm35BXjxxfOB1yK7ADdLfcVrq08NLNdIwac,7026
87
+ redisbench_admin/run_local/run_local.py,sha256=TIGhSp4UaVZtoUxKMwS4BMeE7dNQJm9ODBtsDjcGrbI,35333
88
+ redisbench_admin/run_remote/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
89
+ redisbench_admin/run_remote/args.py,sha256=Ef32mg1yNYYHL5g59SzIWZqFB__RNLLriPqiucVyoNg,4826
90
+ redisbench_admin/run_remote/consts.py,sha256=bCMkwyeBD-EmOpoHKni7LjWy5WuaxGJhGhqpi4AL0RQ,386
91
+ redisbench_admin/run_remote/log.py,sha256=cD7zfXt0VEmy0b7452HvcAxX_9kVj6Vm213yNdUHP20,95
92
+ redisbench_admin/run_remote/notifications.py,sha256=-W9fLaftEFNfplBl2clHk37jbYxliDbHftQ62khN31k,2157
93
+ redisbench_admin/run_remote/remote_client.py,sha256=rRmDro1weto01wzqYpId8NMPoizEzSyudXBCjYrBVMs,14128
94
+ redisbench_admin/run_remote/remote_db.py,sha256=EEDeiOZk-godr5EINscEkOJLGWUN3gFfH6RaBzAKbak,14566
95
+ redisbench_admin/run_remote/remote_env.py,sha256=Ux_0QT1unNRlKl3cakzjG5Px1uuxOOfBoF_pnalx_T8,4936
96
+ redisbench_admin/run_remote/remote_failures.py,sha256=IOo6DyxarcwwMPCeN4gWB2JrhuC9iBLwq0nCROqr5ak,1567
97
+ redisbench_admin/run_remote/remote_helpers.py,sha256=OEFjbdqG_FJ5KY6B63jYr21vPomtM3ld_F9MsGRTaCU,11271
98
+ redisbench_admin/run_remote/run_remote.py,sha256=tZqCu1fTfB5gWooVIEsSDoaVfnVRfxeCpn-RLmYI3IM,75476
99
+ redisbench_admin/run_remote/standalone.py,sha256=WRgiY8oMJwDUPht1LAPG9tnOLvhJmhRvVOi3GBnMoX4,27952
100
+ redisbench_admin/run_remote/terraform.py,sha256=vV3eWXNwj7vsnFNqUgCir5ueZS4VYopEyzWiTtoSq0Q,4018
101
+ redisbench_admin/utils/__init__.py,sha256=DtBXRp0Q01XgCFmY-1OIePMyyYihVNAjZ1Y8zwqSDN0,101
102
+ redisbench_admin/utils/benchmark_config.py,sha256=bC2C6rnj89wkkSlOXyyfe0N15unn_M1t1zfskfVkb98,21387
103
+ redisbench_admin/utils/local.py,sha256=zUvyVI9LZMT3qyxs1pO3mXL6Bt_1z9EZUGppaRcWNRA,3890
104
+ redisbench_admin/utils/redisearch.py,sha256=lchUEzpt0zB1rHwlDlw9LLifAnxFWcLP-PePw7TjL-0,1602
105
+ redisbench_admin/utils/redisgraph_benchmark_go.py,sha256=os7EJt6kBxsFJLKkSoANbjMT7-cEq4-Ns-49alk2Tf8,2048
106
+ redisbench_admin/utils/remote.py,sha256=MsRL6x-eekzJOVE6Ho05igZzUX54m-4fRVCQlSYxyw0,42306
107
+ redisbench_admin/utils/results.py,sha256=uKk3uNJ--bSXlUj_HGQ2OaV6MVqmXJVM8xTzFV6EOw4,3267
108
+ redisbench_admin/utils/ssh.py,sha256=QW4AwlocMHJt05QMdN_4f8WeDmxiEwR80ny8VBThq6k,6533
109
+ redisbench_admin/utils/utils.py,sha256=XVSvo1_DdcYwk2jOxL3VPVPbnDnhGYt8ieYfANo6rTo,15085
110
+ redisbench_admin/watchdog/__init__.py,sha256=cD7zfXt0VEmy0b7452HvcAxX_9kVj6Vm213yNdUHP20,95
111
+ redisbench_admin/watchdog/args.py,sha256=nKsG1G6ATOZlAMHMtT9u3kXxduKCbejSZ5x8oB_ynZ8,1312
112
+ redisbench_admin/watchdog/watchdog.py,sha256=0wWYge3x_OMxWrzazNhJif2NK4tKsI963HVZqjczRag,6189
113
+ redisbench_admin-0.11.57.dist-info/METADATA,sha256=2T9Tvwug0CQlh1ueoikzvMk4sBlqjL2JP9argPeR25A,5724
114
+ redisbench_admin-0.11.57.dist-info/WHEEL,sha256=zp0Cn7JsFoX2ATtOhtaFYIiE2rmFAD4OcMhtUki8W3U,88
115
+ redisbench_admin-0.11.57.dist-info/entry_points.txt,sha256=UUawXk_AS-PlieKJ1QxPQXGsRLb6OW_F0MtmA1W0KE8,113
116
+ redisbench_admin-0.11.57.dist-info/licenses/LICENSE,sha256=AAMtfs82zOOvmG68vILivm6lxi2rcOlGObmA8jzxQvw,10768
117
+ redisbench_admin-0.11.57.dist-info/RECORD,,
@@ -1,4 +1,4 @@
1
1
  Wheel-Version: 1.0
2
- Generator: poetry-core 1.9.0
2
+ Generator: poetry-core 2.2.1
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
@@ -1,2 +0,0 @@
1
- data
2
- results
@@ -1 +0,0 @@
1
- gitdir: ../../../../.git/modules/redisbench_admin/run/ann/pkg
@@ -1,100 +0,0 @@
1
- name: ANN benchmarks
2
-
3
- on: [push, pull_request]
4
-
5
- jobs:
6
- build:
7
-
8
- runs-on: ubuntu-20.04
9
- strategy:
10
- matrix:
11
- include:
12
- - library: annoy
13
- dataset: random-xs-20-angular
14
- - library: dolphinn
15
- dataset: random-xs-20-angular
16
- - library: faiss
17
- dataset: random-xs-20-angular
18
- - library: flann
19
- dataset: random-xs-20-angular
20
- - library: kgraph
21
- dataset: random-xs-20-angular
22
- - library: milvus
23
- dataset: random-xs-20-angular
24
- - library: mrpt
25
- dataset: random-xs-20-angular
26
- - library: n2
27
- dataset: random-xs-20-angular
28
- - library: nearpy
29
- dataset: random-xs-20-angular
30
- - library: ngt
31
- dataset: random-xs-20-angular
32
- - library: nmslib
33
- dataset: random-xs-20-angular
34
- - library: hnswlib
35
- dataset: random-xs-20-angular
36
- - library: puffinn
37
- dataset: random-xs-20-angular
38
- - library: pynndescent
39
- dataset: random-xs-20-angular
40
- - library: rpforest
41
- dataset: random-xs-20-angular
42
- - library: sklearn
43
- dataset: random-xs-20-angular
44
- - library: sptag
45
- dataset: random-xs-20-angular
46
- - library: mih
47
- dataset: random-xs-16-hamming
48
- - library: datasketch
49
- dataset: random-s-jaccard
50
- - library: scann
51
- dataset: random-xs-20-angular
52
- - library: elasticsearch
53
- dataset: random-xs-20-angular
54
- - library: elastiknn
55
- dataset: random-xs-20-angular
56
- - library: opensearchknn
57
- dataset: random-xs-20-angular
58
- - library: diskann
59
- dataset: random-xs-20-angular
60
- - library: puffinn
61
- dataset: random-s-jaccard
62
- - library: pynndescent
63
- dataset: random-s-jaccard
64
- - library: vespa
65
- dataset: random-xs-20-angular
66
- - library: scipy
67
- dataset: random-xs-20-angular
68
- - library: vald
69
- dataset: random-xs-20-angular
70
- fail-fast: false
71
-
72
- steps:
73
- - uses: actions/checkout@v2 # Pull the repository
74
-
75
- - name: Install various apt packages
76
- run: sudo apt-get install -y libhdf5-dev python3-numpy python3-scipy python3-matplotlib python3-sklearn
77
-
78
- - name: Install dependencies
79
- run: |
80
- pip3 install -r requirements.txt
81
- python3 install.py
82
-
83
- env:
84
- LIBRARY: ${{ matrix.library }}
85
- DATASET: ${{ matrix.dataset }}
86
-
87
- - name: Run the benchmark
88
- run: |
89
- python3 run.py --docker-tag ann-benchmarks-${LIBRARY} --max-n-algorithms 5 --dataset $DATASET --run-disabled --timeout 300
90
- python3 run.py --docker-tag ann-benchmarks-${LIBRARY} --max-n-algorithms 5 --dataset $DATASET --run-disabled --batch --timeout 300
91
- sudo chmod -R 777 results/
92
- python3 plot.py --dataset $DATASET --output plot.png
93
- python3 plot.py --dataset $DATASET --output plot-batch.png --batch
94
- python3 -m unittest test/test-metrics.py
95
- python3 -m unittest test/test-jaccard.py
96
- python3 create_website.py --outputdir . --scatter --latex
97
-
98
- env:
99
- LIBRARY: ${{ matrix.library }}
100
- DATASET: ${{ matrix.dataset }}
@@ -1,21 +0,0 @@
1
- .DS_Store
2
- *.pyc
3
- *.o
4
- protocol/c/fr-*
5
-
6
- install/*.txt
7
- install/*.yaml
8
- install/lib-*/
9
- data/*
10
- *.class
11
-
12
- *.log
13
-
14
- results/*
15
- !results/*.png
16
- website
17
-
18
- venv
19
-
20
- .idea
21
- .vscode
@@ -1,21 +0,0 @@
1
- MIT License
2
-
3
- Copyright (c) 2018 Erik Bernhardsson
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
@@ -1,157 +0,0 @@
1
- Benchmarking nearest neighbors
2
- ==============================
3
-
4
- [![Build Status](https://img.shields.io/github/workflow/status/erikbern/ann-benchmarks/ANN%20benchmarks?style=flat-square)](https://github.com/erikbern/ann-benchmarks/actions?query=workflow:benchmarks)
5
-
6
- Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem, but so far there has not been a lot of empirical attempts at comparing approaches in an objective way.
7
-
8
- This project contains some tools to benchmark various implementations of approximate nearest neighbor (ANN) search for different metrics. We have pregenerated datasets (in HDF5) formats and we also have Docker containers for each algorithm. There's a [test suite](https://travis-ci.org/erikbern/ann-benchmarks) that makes sure every algorithm works.
9
-
10
- Evaluated
11
- =========
12
-
13
- * [Annoy](https://github.com/spotify/annoy)
14
- * [FLANN](http://www.cs.ubc.ca/research/flann/)
15
- * [scikit-learn](http://scikit-learn.org/stable/modules/neighbors.html): LSHForest, KDTree, BallTree
16
- * [PANNS](https://github.com/ryanrhymes/panns)
17
- * [NearPy](http://pixelogik.github.io/NearPy/)
18
- * [KGraph](https://github.com/aaalgo/kgraph)
19
- * [NMSLIB (Non-Metric Space Library)](https://github.com/nmslib/nmslib): SWGraph, HNSW, BallTree, MPLSH
20
- * [hnswlib (a part of nmslib project)](https://github.com/nmslib/hnsw)
21
- * [RPForest](https://github.com/lyst/rpforest)
22
- * [FAISS](https://github.com/facebookresearch/faiss.git)
23
- * [DolphinnPy](https://github.com/ipsarros/DolphinnPy)
24
- * [Datasketch](https://github.com/ekzhu/datasketch)
25
- * [PyNNDescent](https://github.com/lmcinnes/pynndescent)
26
- * [MRPT](https://github.com/teemupitkanen/mrpt)
27
- * [NGT](https://github.com/yahoojapan/NGT): ONNG, PANNG, QG
28
- * [SPTAG](https://github.com/microsoft/SPTAG)
29
- * [PUFFINN](https://github.com/puffinn/puffinn)
30
- * [N2](https://github.com/kakao/n2)
31
- * [ScaNN](https://github.com/google-research/google-research/tree/master/scann)
32
- * [Elastiknn](https://github.com/alexklibisz/elastiknn)
33
- * [OpenSearch KNN](https://github.com/opensearch-project/k-NN)
34
- * [DiskANN](https://github.com/microsoft/diskann): Vamana, Vamana-PQ
35
- * [Vespa](https://github.com/vespa-engine/vespa)
36
- * [scipy](https://docs.scipy.org/doc/scipy/reference/spatial.html): cKDTree
37
- * [vald](https://github.com/vdaas/vald)
38
-
39
- Data sets
40
- =========
41
-
42
- We have a number of precomputed data sets for this. All data sets are pre-split into train/test and come with ground truth data in the form of the top 100 neighbors. We store them in a HDF5 format:
43
-
44
- | Dataset | Dimensions | Train size | Test size | Neighbors | Distance | Download |
45
- | ----------------------------------------------------------------- | ---------: | ---------: | --------: | --------: | --------- | -------------------------------------------------------------------------- |
46
- | [DEEP1B](http://sites.skoltech.ru/compvision/noimi/) | 96 | 9,990,000 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/deep-image-96-angular.hdf5) (3.6GB)
47
- | [Fashion-MNIST](https://github.com/zalandoresearch/fashion-mnist) | 784 | 60,000 | 10,000 | 100 | Euclidean | [HDF5](http://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5) (217MB) |
48
- | [GIST](http://corpus-texmex.irisa.fr/) | 960 | 1,000,000 | 1,000 | 100 | Euclidean | [HDF5](http://ann-benchmarks.com/gist-960-euclidean.hdf5) (3.6GB) |
49
- | [GloVe](http://nlp.stanford.edu/projects/glove/) | 25 | 1,183,514 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/glove-25-angular.hdf5) (121MB) |
50
- | GloVe | 50 | 1,183,514 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/glove-50-angular.hdf5) (235MB) |
51
- | GloVe | 100 | 1,183,514 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/glove-100-angular.hdf5) (463MB) |
52
- | GloVe | 200 | 1,183,514 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/glove-200-angular.hdf5) (918MB) |
53
- | [Kosarak](http://fimi.uantwerpen.be/data/) | 27983 | 74,962 | 500 | 100 | Jaccard | [HDF5](http://ann-benchmarks.com/kosarak-jaccard.hdf5) (2.0GB) |
54
- | [MNIST](http://yann.lecun.com/exdb/mnist/) | 784 | 60,000 | 10,000 | 100 | Euclidean | [HDF5](http://ann-benchmarks.com/mnist-784-euclidean.hdf5) (217MB) |
55
- | [NYTimes](https://archive.ics.uci.edu/ml/datasets/bag+of+words) | 256 | 290,000 | 10,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/nytimes-256-angular.hdf5) (301MB) |
56
- | [SIFT](http://corpus-texmex.irisa.fr/) | 128 | 1,000,000 | 10,000 | 100 | Euclidean | [HDF5](http://ann-benchmarks.com/sift-128-euclidean.hdf5) (501MB) |
57
- | [Last.fm](https://github.com/erikbern/ann-benchmarks/pull/91) | 65 | 292,385 | 50,000 | 100 | Angular | [HDF5](http://ann-benchmarks.com/lastfm-64-dot.hdf5) (135MB) |
58
-
59
- Results
60
- =======
61
-
62
- Interactive plots can be found at <http://ann-benchmarks.com>. These are all as of December 2021, running all benchmarks on a r5.4xlarge machine on AWS with `--parallelism 7`:
63
-
64
- glove-100-angular
65
- -----------------
66
-
67
- ![glove-100-angular](https://raw.github.com/erikbern/ann-benchmarks/master/results/glove-100-angular.png)
68
-
69
- sift-128-euclidean
70
- ------------------
71
-
72
- ![glove-100-angular](https://raw.github.com/erikbern/ann-benchmarks/master/results/sift-128-euclidean.png)
73
-
74
- fashion-mnist-784-euclidean
75
- ---------------------------
76
-
77
- ![fashion-mnist-784-euclidean](https://raw.github.com/erikbern/ann-benchmarks/master/results/fashion-mnist-784-euclidean.png)
78
-
79
- lastfm-64-dot
80
- ------------------
81
-
82
- ![lastfm-64-dot](https://raw.github.com/erikbern/ann-benchmarks/master/results/lastfm-64-dot.png)
83
-
84
- nytimes-256-angular
85
- -------------------
86
-
87
- ![nytimes-256-angular](https://raw.github.com/erikbern/ann-benchmarks/master/results/nytimes-256-angular.png)
88
-
89
- glove-25-angular
90
- ----------------
91
-
92
- ![glove-25-angular](https://raw.github.com/erikbern/ann-benchmarks/master/results/glove-25-angular.png)
93
-
94
- Install
95
- =======
96
-
97
- The only prerequisite is Python (tested with 3.6) and Docker.
98
-
99
- 1. Clone the repo.
100
- 2. Run `pip install -r requirements.txt`.
101
- 3. Run `python install.py` to build all the libraries inside Docker containers (this can take a while, like 10-30 minutes).
102
-
103
- Running
104
- =======
105
-
106
- 1. Run `python run.py` (this can take an extremely long time, potentially days)
107
- 2. Run `python plot.py` or `python create_website.py` to plot results.
108
-
109
- You can customize the algorithms and datasets if you want to:
110
-
111
- * Check that `algos.yaml` contains the parameter settings that you want to test
112
- * To run experiments on SIFT, invoke `python run.py --dataset glove-100-angular`. See `python run.py --help` for more information on possible settings. Note that experiments can take a long time.
113
- * To process the results, either use `python plot.py --dataset glove-100-angular` or `python create_website.py`. An example call: `python create_website.py --plottype recall/time --latex --scatter --outputdir website/`.
114
-
115
- Including your algorithm
116
- ========================
117
-
118
- 1. Add your algorithm into `ann_benchmarks/algorithms` by providing a small Python wrapper.
119
- 2. Add a Dockerfile in `install/` for it
120
- 3. Add it to `algos.yaml`
121
- 4. Add it to `.github/workflows/benchmarks.yml`
122
-
123
- Principles
124
- ==========
125
-
126
- * Everyone is welcome to submit pull requests with tweaks and changes to how each library is being used.
127
- * In particular: if you are the author of any of these libraries, and you think the benchmark can be improved, consider making the improvement and submitting a pull request.
128
- * This is meant to be an ongoing project and represent the current state.
129
- * Make everything easy to replicate, including installing and preparing the datasets.
130
- * Try many different values of parameters for each library and ignore the points that are not on the precision-performance frontier.
131
- * High-dimensional datasets with approximately 100-1000 dimensions. This is challenging but also realistic. Not more than 1000 dimensions because those problems should probably be solved by doing dimensionality reduction separately.
132
- * Single queries are used by default. ANN-Benchmarks enforces that only one CPU is saturated during experimentation, i.e., no multi-threading. A batch mode is available that provides all queries to the implementations at once. Add the flag `--batch` to `run.py` and `plot.py` to enable batch mode.
133
- * Avoid extremely costly index building (more than several hours).
134
- * Focus on datasets that fit in RAM. For billion-scale benchmarks, see the related [big-ann-benchmarks](https://github.com/harsha-simhadri/big-ann-benchmarks) project.
135
- * We mainly support CPU-based ANN algorithms. GPU support exists for FAISS, but it has to be compiled with GPU support locally and experiments must be run using the flags `--local --batch`.
136
- * Do proper train/test set of index data and query points.
137
- * Note that we consider that set similarity datasets are sparse and thus we pass a **sorted** array of integers to algorithms to represent the set of each user.
138
-
139
-
140
- Authors
141
- =======
142
-
143
- Built by [Erik Bernhardsson](https://erikbern.com) with significant contributions from [Martin Aumüller](http://itu.dk/people/maau/) and [Alexander Faithfull](https://github.com/ale-f).
144
-
145
- Related Publication
146
- ==================
147
-
148
- The following publication details design principles behind the benchmarking framework:
149
-
150
- - M. Aumüller, E. Bernhardsson, A. Faithfull:
151
- [ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms](https://arxiv.org/abs/1807.05614). Information Systems 2019. DOI: [10.1016/j.is.2019.02.006](https://doi.org/10.1016/j.is.2019.02.006)
152
-
153
- Related Projects
154
- ================
155
-
156
- - [big-ann-benchmarks](https://github.com/harsha-simhadri/big-ann-benchmarks) is a benchmarking effort for billion-scale approximate nearest neighbor search as part of the [NeurIPS'21 Competition track](https://neurips.cc/Conferences/2021/CompetitionTrack).
157
-