whatlanguage 2.0.0 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0939c1109ab44fd32e9e611c20696ee315a980877c0f04d5990fe42f51c17201'
-  data.tar.gz: e5002ebde89b8c6acfa313355d2aa588b31cbb7db27870690bbeae761ecaeb2c
+  metadata.gz: 53fea55810c7a8eb102970d6d8c3fe039b031680ff0f23eb6a1c7fec6ad1a602
+  data.tar.gz: f960efb55e0d04b5e457ef53f6c382434a334eea9d44fab2065e7ce9d4376747
 SHA512:
-  metadata.gz: 2a06fa18163cee6de9278291856c1b04c633880be1a2816476ce4b47b03c374242c9eb0ea012e1fb04322de753d45492a9ca96856ec70582ea1287fb19034c2e
-  data.tar.gz: e57243d9fccc9b5c43e5da591db26085d2eedd7776a4bc3e6dfd9b8ddcfcc59112de6309f63a41f0b2f38bc171bce2d80e638b2c3a8645230657aeb643f852ef
+  metadata.gz: 99e1be592f692ba51e3a973a489703f955238da71985feb91b424e9203b8ae29d1ff635d5479ee6cea0226a06cd74c10c4f859a722c76914f277df3d41c3c779
+  data.tar.gz: e13bf56aa67c969d86213e9f1bd34bb2f0aa4ab0ab5db4c1b735ef84c9c3d2ad8e7891f0104a0881833fb5f54c42235132cf48f7a75af38f1cf5f71280529a31

data/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 2.0.1 / 2026-07-11
+
+- Added a `whatlanguage` command-line executable that reads from files given as
+  arguments (or stdin) and prints the ISO 639 code of the detected language, or
+  `und` if undetermined. (#50, suggested by Keith Bennett)
+
 ## 2.0.0 / 2026-06-10
 
 - Rewrote the detection engine from scratch. Bloom-filter dictionary lookups are

data/README.md

@@ -13,8 +13,11 @@ WhatLanguage.language("Que linguagem é essa? É uma pergunta sobre a língua po
 - No runtime dependencies.
 - Supports 20+ writing systems.
 - Ships a compact ~220 KB trigram model.
-- Requires Ruby 3.0+
-- Best on sentence-length text or longer.
+- Requires Ruby 3.0+ (JRuby and TruffleRuby also good)
+- **Best on sentence-length text or longer.** Short texts can have ambiguous results.
+
+> [!IMPORTANT]
+> v2.0 has many breaking changes as the entire library has been reimplemented, though the core `WhatLanguage.language` API remains similar. Versions 1.0.6 and earlier (so the 2007-2025 run of the library) used a Bloom-filter technique and had 5MB of binary files to handle ~20 languages. Version 2.0 is more accurate, faster, and supports more languages from a single 220KB JSON file :-)
 
 ## How it works
 
@@ -22,8 +25,6 @@ Detection is in two stages. First, the dominant Unicode script is detected; scri
 
 The trigram profiles are vendored from [whatlang](https://github.com/greyblake/whatlang-rs), a port of [Franc](https://github.com/wooorm/franc), whose models are built from the public-domain UDHR corpus (see Credits). The model is a ~220 KB JSON file.
 
-> [!IMPORTANT]
-> v2.0 has many breaking changes as the entire library has been rewritten, though the core `WhatLanguage.language` API remains similar. Versions 1.0.6 and earlier (so the 2007-2025 run of the library) used a Bloom-filter technique and had 5MB of binary files to handle ~20 languages. Version 2.0 is more accurate, faster, and supports more languages from a single 220KB JSON file :-)
 
 ## Usage
 
@@ -58,9 +59,20 @@ Short Latin-script fragments are ignored by default because there is not enough
 wl = WhatLanguage.new(min_chars: 0)
 ```
 
+## Command line
+
+Installing the gem also installs a `whatlanguage` executable that reads from files given as arguments (or stdin) and prints the ISO 639 code of the detected language, or `und` if undetermined:
+
+```bash
+$ whatlanguage README.md
+en
+$ echo "Wie geht es dir heute?" | whatlanguage
+de
+```
+
 ## Known limitations
 
-- Short fragments are unreliable. For languages resolved by statistical comparison, fewer than 10 significant characters returns `nil` by default.
+- Short fragments are unreliable. For languages resolved by statistical comparison, fewer than 10 significant characters returns `nil` by default. **This is the biggest thing I want to fix.** I think there should be a mechanism for "weighting" languages so that less likely languages don't frequently appear for small English samples, say.
 - Scores are relative ranking values, not probabilities. Use `#ranked` or `#detect.ranked` when close runners-up matter.
 - Closely related written languages can be hard to separate, especially Norwegian Bokmål/Danish, Hebrew/Yiddish, and similar language pairs.
 - Kanji-only Japanese text can classify as Chinese because Han characters alone do not identify the language.
@@ -68,6 +80,6 @@ wl = WhatLanguage.new(min_chars: 0)
 
 ## Credits
 
-Contributions from Konrad Reiche, Salimane Adjao Moustapha, Andrew Cone, Lasse Skindstad Ebert, Henrik Nyh, Daniel Sandbecker, Michael Hartl, Pedro Lambert, Tobias Preuss, Pepijn Looije, and others appreciated.
+Contributions from Konrad Reiche, Salimane Adjao Moustapha, Andrew Cone, Lasse Skindstad Ebert, Henrik Nyh, Daniel Sandbecker, Michael Hartl, Pedro Lambert, Tobias Preuss, Pepijn Looije, Keith Bennett, and others appreciated.
 
 The trigram language profiles in `lib/whatlanguage/trigrams.json` are taken from [whatlang](https://github.com/greyblake/whatlang-rs) (MIT, © Sergey Potapov), itself a derivative of [Franc](https://github.com/wooorm/franc) (MIT, © Titus Wormer). Those profiles are derived from the public-domain Universal Declaration of Human Rights translations.

data/exe/whatlanguage

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'whatlanguage'
+
+# Reads from files given as arguments, or stdin otherwise. Prints the ISO 639
+# code of the detected language, or "und" (undetermined) if detection fails.
+puts WhatLanguage.language_iso(ARGF.read) || 'und'

data/lib/whatlanguage/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 class WhatLanguage
-  VERSION = '2.0.0'
+  VERSION = '2.0.1'
 end

data/whatlanguage.gemspec

@@ -16,6 +16,7 @@ Gem::Specification.new do |gem|
   gem.required_ruby_version = '>= 3.0'
 
   gem.files = Dir['lib/**/*'] + [
+    'exe/whatlanguage',
     'README.md',
     'CHANGELOG.md',
     'LICENSE.txt',
@@ -23,6 +24,8 @@ Gem::Specification.new do |gem|
     'Rakefile',
     'whatlanguage.gemspec'
   ]
+  gem.bindir        = 'exe'
+  gem.executables   = ['whatlanguage']
   gem.require_paths = ['lib']
 
   gem.add_development_dependency 'minitest', '~> 5.0'

metadata

@@ -1,11 +1,11 @@
 --- !ruby/object:Gem::Specification
 name: whatlanguage
 version: !ruby/object:Gem::Version
-  version: 2.0.0
+  version: 2.0.1
 platform: ruby
 authors:
 - Peter Cooper
-bindir: bin
+bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
@@ -40,7 +40,8 @@ dependencies:
 description: WhatLanguage rapidly detects the language of a sample of text
 email:
 - git@peterc.org
-executables: []
+executables:
+- whatlanguage
 extensions: []
 extra_rdoc_files: []
 files:
@@ -49,6 +50,7 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
+- exe/whatlanguage
 - lib/whatlanguage.rb
 - lib/whatlanguage/languages.rb
 - lib/whatlanguage/trigrams.json