tokenizers 0.5.0-x86_64-darwin → 0.5.1-x86_64-darwin

Sign up to get free protection for your applications and to get access to all the features.
Files changed (10) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +5 -0
  3. data/Cargo.lock +162 -175
  4. data/LICENSE-THIRD-PARTY.txt +1677 -1607
  5. data/lib/tokenizers/3.1/tokenizers.bundle +0 -0
  6. data/lib/tokenizers/3.2/tokenizers.bundle +0 -0
  7. data/lib/tokenizers/3.3/tokenizers.bundle +0 -0
  8. data/lib/tokenizers/from_pretrained.rb +2 -2
  9. data/lib/tokenizers/version.rb +1 -1
  10. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 534c9fd9e97bd10ec85240b21b6984ecb17a8ad944a1ff7370a50455f813fb2a
4
- data.tar.gz: f1d8487ff50334a6628d783b90df820742f662b2a2413ac0ab0c52e824515a8d
3
+ metadata.gz: a8025300a0bf7e671371ed11fd5fcb34a4bd347efc8377cf3cc7638af6f343ff
4
+ data.tar.gz: dd6855cfec3d0b767976ccd41a6a9df188a535be8e3243db475f8dda38eb0f61
5
5
  SHA512:
6
- metadata.gz: 949cea618e3171a5e93470d1188a370612e42d1e2db832a43be714996e05a2eab33ed7d756680ccfbbca5cc256b2bac0ae320f9fc42cf42b138f2eb62105506e
7
- data.tar.gz: 2115e676b536b29e5b0a5653930ae6c793bc4b81ad53a11c9bf47d5d5dd91bb97ec8bc1867d142781730e6a886505f0fab203157a7b12b8a409c034fedb741e5
6
+ metadata.gz: 8772c50ab806cd763241177d3fb8463fc98bfd9ae79cf54df03faf7b57b9b407aff86ee20cadeb8bc867d1ed33a76c820fca9bd9c2c4281ffaadb3a64cec6426
7
+ data.tar.gz: b808ea1fa7bc0ddc2b09245a96988691a67d3e96c9a04e97ba92170f50bacb3362743b6fa990b48060abc4757e26b220b3db877f00ee7cc6222a3006521d8ae1
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 0.5.1 (2024-08-13)
2
+
3
+ - Updated Tokenizers to 0.20.0
4
+ - Added precompiled gem for Linux ARM MUSL
5
+
1
6
  ## 0.5.0 (2024-05-21)
2
7
 
3
8
  - Updated Tokenizers to 0.19.1