red-candle 0.0.3 → 0.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: aced5f3b8f49c525244d1f464af0063d71785b883f54b4a2fe4996c1d6f0f8ea
4
- data.tar.gz: 934333c1d6fd74aca84a36086a4dcd157722af1be127494da225446a2fd0a193
3
+ metadata.gz: 5ec591f2ace2a1706864c5ee80d9a55b2832e493459180eaaa00a18b892d2276
4
+ data.tar.gz: 8807b8b426f6778e71876f3662a4881ae429e170551f2f2cae573feebd623a11
5
5
  SHA512:
6
- metadata.gz: 9779d1e2c244c477eae0a706f1f89f98cd6d95714a46792ac43395a4e7c5db19fa005bea072011ae15e573435288eb161c236459378faf0765293373ec12adf2
7
- data.tar.gz: 9687c8596dca420c96fee69c0614dc275d31aa8d898868e74407798ca8eb0b343dcd9d3936116f68768fb21da3071b6975789efe531876fd663af873723dd60f
6
+ metadata.gz: 3358efa6942ee8e0fca86051f3469878ecce217b3e5d6fd72c26f63cba3eb90d4b6a4bd19a06c1d83736b883a6465a95fe002b289b7ff5ad3f7eb0685c5f6342
7
+ data.tar.gz: 3abf5cf27f34b72c5690cfc0b8dd8314964c4b8bf62cd190fd90bbb28eae8b9b2a4a86f4604a8f9d188221d9cc29c6bcd8951e74fa9500978af2132d7d58acac
data/README.md CHANGED
@@ -18,6 +18,50 @@ x = x.reshape([3, 2])
18
18
  # Tensor[[3, 2], f32]
19
19
  ```
20
20
 
21
+ ```ruby
22
+ require 'candle'
23
+ model = Candle::Model.new
24
+ embedding = model.embedding("Hi there!")
25
+ ```
26
+
27
+ ## A note on memory usage
28
+ The `Candle::Model` defaults to the `jinaai/jina-embeddings-v2-base-en` model with the `sentence-transformers/all-MiniLM-L6-v2` tokenizer (both from [HuggingFace](https://huggingface.co)). With this configuration the model takes a little more than 3GB of memory running on my Mac. The memory stays with the instantiated `Candle::Model` class, if you instantiate more than one, you'll use more memory. Likewise, if you let it go out of scope and call the garbage collector, you'll free the memory. For example:
29
+
30
+ ```ruby
31
+ > require 'candle'
32
+ # Ruby memory = 25.9 MB
33
+ > model = Candle::Model.new
34
+ # Ruby memory = 3.50 GB
35
+ > model2 = Candle::Model.new
36
+ # Ruby memory = 7.04 GB
37
+ > model2 = nil
38
+ > GC.start
39
+ # Ruby memory = 3.56 GB
40
+ > model = nil
41
+ > GC.start
42
+ # Ruby memory = 55.2 MB
43
+ ```
44
+
45
+ ## A note on returned embeddings
46
+
47
+ The code should match the same embeddings when generated from the python `transformers` library. For instance, locally I was able to generate the same embedding for the text "Hi there!" using the python code:
48
+
49
+ ```python
50
+ from transformers import AutoModel
51
+ model = AutoModel.from_pretrained('jinaai/jina-embeddings-v2-base-en', trust_remote_code=True)
52
+ sentence = ['Hi there!']
53
+ embedding = model.encode(sentence)
54
+ print(embedding)
55
+ ```
56
+
57
+ And the following ruby:
58
+
59
+ ```ruby
60
+ require 'candle'
61
+ model = Candle::Model.new
62
+ embedding = model.embedding("Hi there!")
63
+ ```
64
+
21
65
  ## Development
22
66
 
23
67
  FORK IT!
@@ -29,6 +73,7 @@ bundle
29
73
  bundle exec rake compile
30
74
  ```
31
75
 
76
+
32
77
  Implemented with [Magnus](https://github.com/matsadler/magnus), with reference to [Polars Ruby](https://github.com/ankane/polars-ruby)
33
78
 
34
79
  Policies
@@ -7,7 +7,13 @@ edition = "2021"
7
7
  crate-type = ["cdylib"]
8
8
 
9
9
  [dependencies]
10
- candle-core = "0.2"
11
- candle-nn = "0.2"
10
+ candle-core = "0.4.1"
11
+ candle-nn = "0.4.1"
12
+ candle-transformers = "0.4.1"
13
+ tokenizers = { version = "0.15.0", default-features = true, features = ["fancy-regex"], exclude = ["onig"] }
14
+ hf-hub = "0.3.0"
12
15
  half = "2"
13
16
  magnus = "0.6"
17
+
18
+ [profile.test]
19
+ opt-level = 3