engtagger 0.4.3 → 0.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitattributes +4 -0
- data/.gitignore +3 -0
- data/.rubocop.yml +1 -1
- data/CHANGELOG.md +52 -0
- data/Gemfile +1 -0
- data/LICENSE +623 -311
- data/README.md +8 -1
- data/engtagger.gemspec +17 -4
- data/lib/engtagger/version.rb +1 -1
- data/lib/engtagger.rb +26 -24
- metadata +16 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: be5e5dcb477d743356b79d58eadb358a4bd1a47e2b318011313ea46a577402f9
|
|
4
|
+
data.tar.gz: c8c0fa1024e6d2ad2bb3d619eb92b259f7fe25076ad82119345c5ceab192af45
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2cd4aa320c5c01fa830b68163c8a0b88acc5a87eb91e6ccc2880adb033d6d291be75542867efc7fe8610257cb4cafd61714abaff59f1cb674e07ae5e38820be3
|
|
7
|
+
data.tar.gz: 5ad319a0decea08e49b1b68022d86d0b9e856524bbaa684fe518199139f7cfd7e708d218910780ec052a633b685f8a9bb68876b7b3c27f87da446d5d06f3b434
|
data/.gitattributes
ADDED
data/.gitignore
CHANGED
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,57 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.5 (2026-08-14)
|
|
4
|
+
|
|
5
|
+
License metadata correction; no code changes. This library is a port (2009) of
|
|
6
|
+
Perl Lingua::EN::Tagger, which has been licensed GPL-3.0-only since its
|
|
7
|
+
version 0.14 (2008). As a derivative work EngTagger is therefore GPL-3.0-only:
|
|
8
|
+
the bundled LICENSE is now the GPLv3 text and the gemspec declares
|
|
9
|
+
`GPL-3.0-only`. Earlier releases carried a GPLv2 license text, and 0.4.4
|
|
10
|
+
declared `GPL-2.0-or-later`; both were inaccurate.
|
|
11
|
+
|
|
12
|
+
## 0.4.4 (2026-08-14)
|
|
13
|
+
|
|
14
|
+
This is the final planned release of the 0.4 line. Version 2.0 replaces the
|
|
15
|
+
HMM engine with an averaged perceptron and does not ship the Penn
|
|
16
|
+
Treebank-derived probability tables, so its tagging output differs. Pin
|
|
17
|
+
`gem "engtagger", "~> 0.4"` to stay on this line; it will receive critical
|
|
18
|
+
fixes only.
|
|
19
|
+
|
|
20
|
+
- Fix the test suite silently rewriting the bundled lexicon files
|
|
21
|
+
(`lib/engtagger/*.hash`) on every run: a path-join bug in the test setup made
|
|
22
|
+
the existence check always fail, so `install` ran before each test. This was
|
|
23
|
+
also the enabler of the 0600-permission contamination behind #20. Tests are
|
|
24
|
+
~6x faster as a result.
|
|
25
|
+
- `get_sentences` now applies its intended punctuation cleanup. Four of its
|
|
26
|
+
five reattachment rules had discarded their results since the original port
|
|
27
|
+
(only the leading-backquote rule was ever applied), and their regexes were
|
|
28
|
+
corrupted by string escaping (`\W` matched a literal "W"). Sentences returned
|
|
29
|
+
by this method now read naturally (`"Lisa Raines, a lawyer,"`,`"$3"`).
|
|
30
|
+
Known limitation carried over from the tokenizer: contractions other than
|
|
31
|
+
possessives remain space-separated (`"I 'm"`, `"is n't"`).
|
|
32
|
+
- Fix the public `WORD` constant, whose regex was corrupted by the same
|
|
33
|
+
escaping problem and matched only `<w>`-style tags.
|
|
34
|
+
- Tag regexes produced by `EngTagger.get_ext` (NUM, NN, SEN, etc.) now end with
|
|
35
|
+
a real `\s*`, so they consume trailing whitespace including newlines instead
|
|
36
|
+
of spaces only.
|
|
37
|
+
- `assign_tag` no longer raises `NoMethodError` for words absent from the
|
|
38
|
+
lexicon; it returns `""` so callers fall back to `"nn"` as designed.
|
|
39
|
+
- `EngTagger::VERSION` is now defined at runtime (`engtagger/version` was not
|
|
40
|
+
required by the library itself).
|
|
41
|
+
- Read and write the Marshal lexicon files in binary mode to avoid corruption
|
|
42
|
+
on Windows.
|
|
43
|
+
- Remove the stray shebang and executable bit from `lib/engtagger.rb`, and the
|
|
44
|
+
obsolete `require "rubygems"`.
|
|
45
|
+
- Clarify licensing: the LICENSE file had a stray MIT license appended after
|
|
46
|
+
the GPLv2 text. The library is licensed as GPL-2.0-or-later; the MIT tail has
|
|
47
|
+
been removed and the gemspec now declares the SPDX identifier.
|
|
48
|
+
- Gemspec: use the https homepage URL, add `metadata` URIs, and drop the
|
|
49
|
+
obsolete `test_files`.
|
|
50
|
+
- Fix a dead test (`text_get_ext` typo) that had never run and would have
|
|
51
|
+
failed if enabled.
|
|
52
|
+
- Note in the README that 0.4.x carries Penn Treebank-derived probability
|
|
53
|
+
tables; 2.x is trained on openly licensed data instead.
|
|
54
|
+
|
|
3
55
|
## 0.4.3
|
|
4
56
|
|
|
5
57
|
- Fix gem packaging that produced files with owner-only (0600) permissions,
|