tokenizers 0.5.0-x86_64-linux → 0.5.1-x86_64-linux

Sign up to get free protection for your applications and to get access to all the features.
Files changed (10) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +5 -0
  3. data/Cargo.lock +162 -175
  4. data/LICENSE-THIRD-PARTY.txt +1677 -1607
  5. data/lib/tokenizers/3.1/tokenizers.so +0 -0
  6. data/lib/tokenizers/3.2/tokenizers.so +0 -0
  7. data/lib/tokenizers/3.3/tokenizers.so +0 -0
  8. data/lib/tokenizers/from_pretrained.rb +2 -2
  9. data/lib/tokenizers/version.rb +1 -1
  10. metadata +2 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 73c92c500a274ba952cfd529fc69f96613e19f2dc2acb0e608eba84e6477442f
4
- data.tar.gz: da18d3aa30c01ae75b8b50c1fa675fd6a32acabe3f14c5a30802b6078c9801e4
3
+ metadata.gz: e1ac97056f64913b16bd3d63a97b0dfa11daad03adabf7bc09fc1f1f59c3459d
4
+ data.tar.gz: 022cb74e0108763c23f26715accff37148809f564813409db81b9c85a8273fd4
5
5
  SHA512:
6
- metadata.gz: 544f43731aba4981487a1e14a13b76dd627ec60e6c68f808d47b4b721a0153fe427402e845e7595ca729ab877008b6a452b32b95a851bfb1236b273c3e64d10b
7
- data.tar.gz: 38ca9c1e9dd511f0a875de4ba83df3015186c9e609d740525da9ee7660db70e90e1f539c20e38c43ae44fbe7df09e8ffc0781de8e7826fbe0f7a8f5c86bd6125
6
+ metadata.gz: f00e700813f6dffb830497943538d4d9d45fe4679b38a5132dd18564088fd644340b7c7f937a0ce9904f001d2914f3898cd63435ec3750c6b1b51198a5e0e207
7
+ data.tar.gz: 0c4b13af17bc407d788d5b263a024ccbd92130d1174ac1aa78412d2458923c9c6e32bcc562a8f68188867631321b6a33399fb594211e7f55f3964386d8f95700
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 0.5.1 (2024-08-13)
2
+
3
+ - Updated Tokenizers to 0.20.0
4
+ - Added precompiled gem for Linux ARM MUSL
5
+
1
6
  ## 0.5.0 (2024-05-21)
2
7
 
3
8
  - Updated Tokenizers to 0.19.1