tokenizers 0.5.0-x86_64-linux-musl → 0.5.1-x86_64-linux-musl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (10) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +5 -0
  3. data/Cargo.lock +162 -175
  4. data/LICENSE-THIRD-PARTY.txt +1677 -1607
  5. data/lib/tokenizers/3.1/tokenizers.so +0 -0
  6. data/lib/tokenizers/3.2/tokenizers.so +0 -0
  7. data/lib/tokenizers/3.3/tokenizers.so +0 -0
  8. data/lib/tokenizers/from_pretrained.rb +2 -2
  9. data/lib/tokenizers/version.rb +1 -1
  10. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 11b3f1cc55046f5bbc2f1629a78255ad11f58eadbcd425cc011fc3d2edb572fe
4
- data.tar.gz: 519bff10644092768540f7c7d90bbf3720ca059dba07f9a860b559d43fe6b8bc
3
+ metadata.gz: 33811f5d5774e9715d7730def78a304df68801d3dcb044831943db2708c33c5a
4
+ data.tar.gz: cec6f7c1bdf1ea43014284e1716cd5d51fb4474270782706ebdb43938748df07
5
5
  SHA512:
6
- metadata.gz: fc29ffc0083fb6f9b7d3eff86e633871a433a27781659e11433ccc86f58973ed8997e9f64b477bf6aa6960529bc0daea0abb2ab14d94520bc58ee9d573ea05d7
7
- data.tar.gz: ac546d3e97aa4030ddd0569ca3322fcba7b1a8d9d8f2e8e8afa4f25e6929cc8e85348ebe2d7dc07c256daeb2401ec0c00b6aa847b7b64ec5a03b669ea3036f2d
6
+ metadata.gz: a68caf15b7de717546fbedd09921d5d80357dad6354a785fa2a729cdcc78f1ea3001f001947688f4669b0e85a7db07ee147714e6c5ce2d3af103f17ddf256335
7
+ data.tar.gz: 8f37e091a9e7d816b949d53da9c3b7711d9e1122f380daed0f0a2fa33551e0d46d78cb7e5437e0039e5e0fc3e7834440c8460e9e600a3b1bd71a69e45490afb2
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 0.5.1 (2024-08-13)
2
+
3
+ - Updated Tokenizers to 0.20.0
4
+ - Added precompiled gem for Linux ARM MUSL
5
+
1
6
  ## 0.5.0 (2024-05-21)
2
7
 
3
8
  - Updated Tokenizers to 0.19.1