tugatagger 0.0.1__tar.gz

tugatagger-0.0.1/PKG-INFO

@@ -0,0 +1,24 @@
+Metadata-Version: 2.4
+Name: tugatagger
+Version: 0.0.1
+Summary: A unified wrapper for Portuguese POS tagging and benchmarking.
+Home-page: https://github.com/TigreGotico/tugatagger
+Author: JarbasAI
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Natural Language :: Portuguese
+Requires-Python: >=3.7
+Provides-Extra: spacy
+Requires-Dist: spacy; extra == "spacy"
+Provides-Extra: lexicon
+Requires-Dist: tugalex; extra == "lexicon"
+Provides-Extra: brill
+Requires-Dist: brill_postagger; extra == "brill"
+Dynamic: author
+Dynamic: classifier
+Dynamic: home-page
+Dynamic: provides-extra
+Dynamic: requires-python
+Dynamic: summary

tugatagger-0.0.1/README.md

@@ -0,0 +1,86 @@
+# TugaTagger 🇵🇹
+
+**TugaTagger** is a unified, lightweight wrapper for Portuguese Part-of-Speech (POS) tagging. It provides a standardized interface to swap between different NLP backends, making it ideal for benchmarking different approaches or maintaining consistency across multiple microservices and repositories.
+
+---
+
+## 🚀 Key Features
+
+* **Unified API:** Use the same `tag()` method regardless of the underlying engine.
+* **Multiple Backends:** Supports **spaCy**, **Brill-style** taggers, and **Lexicon-based** lookups.
+* **Robust Fallback ("Auto" Mode):** Automatically tries the best available engine, falling back to heuristic-based "guessing" if dependencies are missing.
+* **Zero-Dependency Mode:** Includes a built-in rule-based tagger for environments where installing heavy NLP models isn't feasible.
+
+---
+
+## 📦 Installation
+
+*(Note: Install the backends you intend to use)*
+
+```bash
+pip install tugatagger[brill]
+
+# To use spaCy
+pip install tugatagger[spacy]
+python -m spacy download pt_core_news_lg
+```
+
+---
+
+## 🛠 Usage
+
+### Quick Start
+
+The `auto` engine is the default. It attempts to use spaCy or Brill first and falls back to a heuristic "dummy" tagger if they aren't installed.
+
+```python
+from tugatagger import TugaTagger
+
+tagger = TugaTagger(engine="auto")
+text = "O gato preto pulou o muro."
+
+tags = tagger.tag(text)
+for word, pos in tags:
+    print(f"{word} -> {pos}")
+
+```
+
+### Choosing a Specific Engine
+
+You can force a specific backend for benchmarking or production stability.
+
+| Engine | Description                                    | Best For...                                |
+| --- |------------------------------------------------|--------------------------------------------|
+| `spacy` | Uses `pt_core_news_lg` (or your choice).       | High accuracy & context awareness.         |
+| `brill` | Transformation-based learning tagger.          | Fast performance with good accuracy.       |
+| `lexicon` | Dictionary lookup from `tugalex`.              | word-lookup tagging.                       |
+| `dummy` | Heuristics based on suffixes and common words. | Low-resource / No-dependency environments. |
+
+```python
+# Force spaCy with a specific model
+tagger = TugaTagger(engine="spacy", spacy_model="pt_core_news_sm")
+
+```
+
+---
+
+## 🧠 How the Heuristic Tagger Works
+
+When using `engine="dummy"` or as a final fallback, TugaTagger uses a multi-stage guessing logic:
+
+1. **Punctuation/Numbers:** Identifies `PUNCT` and `NUM`.
+2. **Closed-class Lookups:** Identifies common Portuguese functional words (e.g., "o", "de", "com", "mas").
+3. **Suffix Morphology:** Analyzes word endings (e.g., `-mente` → `ADV`, `-ar/-er/-ir` → `VERB`, `-ção` → `NOUN`).
+4. **Capitalization:** Heuristic for `PROPN` (Proper Nouns).
+5. **Default:** Falls back to `NOUN`.
+
+---
+
+## 🤝 Contributing
+
+If you'd like to add a new engine (e.g., Stanza or NLTK):
+
+1. Add a `tag_newengine` method to the `TugaTagger` class.
+2. Update the `engines` dictionary in the `tag()` method.
+3. Add the corresponding loading logic in `__init__`.
+

tugatagger-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

tugatagger-0.0.1/setup.py

@@ -0,0 +1,55 @@
+import os
+
+from setuptools import setup, find_packages
+
+BASEDIR = os.path.abspath(os.path.dirname(__file__))
+
+
+def get_version():
+    """ Find the version of the package"""
+    version_file = os.path.join(BASEDIR, 'tugatagger', 'version.py')
+    major, minor, build, alpha = (None, None, None, None)
+    with open(version_file) as f:
+        for line in f:
+            if 'VERSION_MAJOR' in line:
+                major = line.split('=')[1].strip()
+            elif 'VERSION_MINOR' in line:
+                minor = line.split('=')[1].strip()
+            elif 'VERSION_BUILD' in line:
+                build = line.split('=')[1].strip()
+            elif 'VERSION_ALPHA' in line:
+                alpha = line.split('=')[1].strip()
+
+            if ((major and minor and build and alpha) or
+                    '# END_VERSION_BLOCK' in line):
+                break
+    version = f"{major}.{minor}.{build}"
+    if alpha and int(alpha) > 0:
+        version += f"a{alpha}"
+    return version
+
+
+setup(
+    name="tugatagger",
+    version=get_version(),
+    description="A unified wrapper for Portuguese POS tagging and benchmarking.",
+    # long_description=open("README.md").read(),
+    # long_description_content_content_type="text/markdown",
+    author="JarbasAI",
+    url="https://github.com/TigreGotico/tugatagger",
+    packages=find_packages(),
+    install_package_data=True,
+    extras_require={
+        "spacy": ["spacy"],
+        "lexicon": ["tugalex"],
+        "brill": ["brill_postagger"],
+    },
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+        "Topic :: Scientific/Engineering :: Artificial Intelligence",
+        "Natural Language :: Portuguese",
+    ],
+    python_requires='>=3.7',
+)

tugatagger-0.0.1/tugatagger/__init__.py

@@ -0,0 +1,253 @@
+from typing import List, Tuple
+
+
+class TugaTagger:
+    """
+    A unified interface for Portuguese Part-of-Speech (POS) tagging.
+
+    Supports multiple backends including spaCy and Brill-style taggers.
+    The 'auto' mode provides a fallback mechanism to ensure tagging works
+    even if specific dependencies are missing.
+    """
+
+    def __init__(self, engine: str = "auto", spacy_model: str = "pt_core_news_lg"):
+        """
+        Create a TugaTagger configured for the chosen tagging engine and optionally preload backend models.
+
+        Parameters:
+            engine (str): Tagging engine to use: "spacy", "brill", "dummy", or "auto".
+                - "spacy": preload the spaCy model in strict mode.
+                - "brill": preload the Brill tagger in strict mode.
+                - "auto": attempt to preload both spaCy and Brill (not strict).
+            spacy_model (str): Name of the spaCy Portuguese model to load when using spaCy (default "pt_core_news_lg").
+
+        Notes:
+            Load failures for a backend will be propagated when that backend is loaded in strict mode.
+        """
+        self.engine = engine
+        self._spacy = self._brill = self._lexicon = None
+
+        if engine in ["spacy", "auto"]:
+            self.load_spacy(spacy_model, strict=(engine == "spacy"))
+        if engine in ["brill", "auto"]:
+            self.load_brill(strict=(engine == "brill"))
+        if engine in ["lexicon", "auto"]:
+            self.load_lexicon(strict=(engine == "lexicon"))
+
+    def load_lexicon(self, strict: bool = True):
+        try:
+            from tugalex import TugaLexicon
+            self._lexicon = TugaLexicon()
+        except Exception as e:
+            if strict:
+                raise e
+
+    def load_spacy(self, spacy_model: str = "pt_core_news_lg", strict: bool = True):
+        """
+        Load and cache a spaCy Portuguese NLP model on the instance.
+
+        Parameters:
+            spacy_model (str): Name of the spaCy model to load (e.g., "pt_core_news_lg").
+            strict (bool): If True, re-raise any exception encountered while loading; if False, suppress the exception and leave `self._spacy` as None.
+
+        Side effects:
+            Assigns the loaded spaCy model to `self._spacy`.
+        """
+        try:
+            import spacy
+            self._spacy = spacy.load(spacy_model, disable=["ner", "parser"])
+        except Exception as e:
+            if strict:
+                raise e
+
+    def load_brill(self, strict: bool = True):
+        """
+        Load and cache a Brill-style Portuguese POS tagger on the instance.
+
+        Parameters:
+            strict (bool): If True, re-raise any exception encountered while importing or loading the tagger;
+                if False, suppress load errors and leave `self._brill` unset when loading fails.
+        """
+        try:
+            from brill_postaggers import BrillPostagger
+            self._brill = BrillPostagger.from_pretrained("pt")
+        except Exception as e:
+            if strict:
+                raise e
+
+    def tag(self, sentence: str) -> List[Tuple[str, str]]:
+        """
+        Tags a sentence using the configured engine.
+
+        Args:
+            sentence (str): The Portuguese text to tag.
+
+        Returns:
+            List[Tuple[str, str]]: A list of (word, tag) tuples.
+        """
+        engines = {
+            "auto": self.tag_auto,
+            "dummy": self.tag_dummy,
+            "brill": self.tag_brill,
+            "spacy": self.tag_spacy,
+            "lexicon": self.tag_lexicon
+        }
+        handler = engines.get(self.engine)
+        if not handler:
+            raise ValueError(f"Invalid engine: '{self.engine}'")
+        return handler(sentence)
+
+    def tag_auto(self, sentence: str) -> List[Tuple[str, str]]:
+        """
+        Tag a sentence by trying spaCy then Brill and using the dummy tagger if both fail.
+
+        Returns:
+            List[Tuple[str, str]]: A list of (word, POS-tag) tuples produced by the first successful tagger; if both spaCy and Brill raise exceptions, returns the dummy tagger's output.
+        """
+        for method in [self.tag_brill, self.tag_spacy, self.tag_lexicon]:
+            try:
+                return method(sentence)
+            except:
+                continue
+        return self.tag_dummy(sentence)
+
+    def tag_brill(self, sentence: str) -> List[Tuple[str, str]]:
+        """
+        Tag a sentence using the Brill-style POS tagger.
+
+        If the Brill tagger is not yet loaded, it will be initialized automatically.
+
+        Parameters:
+            sentence (str): Raw text sentence to be tagged.
+
+        Returns:
+            List[Tuple[str, str]]: List of (token, POS-tag) pairs for the input sentence.
+        """
+        if self._brill is None:
+            self.load_brill(strict=True)
+        return self._brill.tag(sentence)
+
+    def tag_spacy(self, sentence: str) -> List[Tuple[str, str]]:
+        """
+        Tag a sentence using the configured spaCy Portuguese model.
+
+        Returns:
+            List[Tuple[str, str]]: A list of (token_text, POS_tag) tuples, one per token. `POS_tag` is spaCy's coarse-grained part-of-speech label.
+        """
+        if self._spacy is None:
+            self.load_spacy(strict=True)
+        doc = self._spacy(sentence)
+        return [(tok.text, tok.pos_) for tok in doc]
+
+    def tag_lexicon(self, sentence: str) -> List[Tuple[str, str]]:
+        tagged = []
+
+        # TODO: improve this
+        tokenize = lambda k: k.lower().replace(".", " .").replace("!", " !").replace("?", " ?").split()
+
+        for word in tokenize(sentence):
+            if word in self._lexicon.possible_postags:
+                possibilities = self._lexicon.possible_postags[word]
+                # TODO: how to choose postag? use surrounding context
+                tagged.append((word, possibilities[0]))
+            else:
+                tagged.append((word, self._guess_pos(word)))
+        return tagged
+
+    @classmethod
+    def tag_dummy(cls, sentence: str) -> List[Tuple[str, str]]:
+        """
+        Split the sentence on whitespace and tag each token as the noun "NOUN".
+
+        Returns:
+            List[Tuple[str, str]]: A list of (word, tag) tuples where each whitespace-separated token from the input is paired with the tag "NOUN".
+        """
+        return [(word, cls._guess_pos(word)) for word in sentence.split()]
+
+    @staticmethod
+    def _guess_pos(word: str) -> str:
+        """Applies heuristic rules to guess the POS tag."""
+        lower_word = word.lower()
+
+        # Rule 0: Punctuation
+        if not word.isalnum():
+            return "PUNCT"
+
+        # Rule 0.5: Numbers
+        if word.isdigit():
+            return "NUM"
+
+        # 1. Closed-class words (Functional words that rarely change category)
+        COMMON_WORDS = {
+            # Articles
+            "o": "DET", "a": "DET", "os": "DET", "as": "DET", "um": "DET", "uma": "DET",
+            # Prepositions
+            "de": "ADP", "do": "ADP", "da": "ADP", "em": "ADP", "no": "ADP", "na": "ADP",
+            "por": "ADP", "para": "ADP", "com": "ADP", "sem": "ADP", #"a": "ADP",
+            # Conjunctions
+            "e": "CCONJ", "mas": "CCONJ", "ou": "CCONJ", "que": "SCONJ", "se": "SCONJ",
+            # Pronouns
+            "eu": "PRON", "ele": "PRON", "ela": "PRON", "nós": "PRON", "eles": "PRON",
+            "isso": "PRON", "aquilo": "PRON",
+            # Common Verbs (Auxiliary/Copula)
+            "é": "AUX", "foi": "AUX", "são": "AUX", "está": "AUX", "ser": "AUX", "ter": "AUX",
+            # Adverbs
+            "não": "ADV", "sim": "ADV", "muito": "ADV", "mais": "ADV"
+        }
+
+        # 2. Suffix Rules (Order matters: check longer suffixes first)
+        SUFFIX_RULES = [
+            ("mente", "ADV"),   # rapidamente
+            ("ando", "VERB"),   # cantando
+            ("endo", "VERB"),   # correndo
+            ("indo", "VERB"),   # partindo
+            ("aram", "VERB"),   # cantaram
+            ("eram", "VERB"),   # correram
+            ("iram", "VERB"),   # partiram
+            ("ava", "VERB"),    # cantava
+            ("ria", "VERB"),    # cantaria
+            ("dor", "NOUN"),    # jogador
+            ("ção", "NOUN"),    # ação
+            ("são", "NOUN"),    # tensão
+            ("dade", "NOUN"),   # cidade
+            ("ismo", "NOUN"),   # realismo
+            ("ista", "NOUN"),   # realista
+            ("oso", "ADJ"),     # formoso
+            ("osa", "ADJ"),     # formosa
+            ("vel", "ADJ"),     # amável
+            ("al", "ADJ"),      # nacional
+            ("ar", "VERB"),     # amar
+            ("er", "VERB"),     # comer
+            ("ir", "VERB"),     # partir (careful with 'ir' the verb itself)
+        ]
+
+        # Rule 1: Dictionary Lookup (O(1) speed)
+        if lower_word in COMMON_WORDS:
+            return COMMON_WORDS[lower_word]
+
+        # Rule 2: Suffix Morphology
+        for suffix, tag in SUFFIX_RULES:
+            if lower_word.endswith(suffix):
+                # Exception: prevent very short words triggering rules (e.g. "lar" ending in "ar")
+                if len(lower_word) > len(suffix):
+                    return tag
+
+        # Rule 3: Capitalization (Proper Noun heuristic)
+        # If it's not the start of the sentence (hard to know context-free here) but matches Title case
+        if word[0].isupper() and word[1:].islower():
+            return "PROPN"
+
+        # Rule 4: Fallback
+        return "NOUN"
+
+if __name__ == "__main__":
+    # Initialize with 'auto' to use the best available engine
+    tagger = TugaTagger(engine="auto")
+
+    text = "O gato preto pulou o muro."
+
+    print(f"Using engine: {tagger.engine}")
+    tags = tagger.tag(text)
+
+    for word, pos in tags:
+        print(f"{word:10} -> {pos}")

tugatagger-0.0.1/tugatagger/version.py

@@ -0,0 +1,10 @@
+# START_VERSION_BLOCK
+VERSION_MAJOR = 0
+VERSION_MINOR = 0
+VERSION_BUILD = 1
+VERSION_ALPHA = 0
+# END_VERSION_BLOCK
+
+VERSION_STR = f"{VERSION_MAJOR}.{VERSION_MINOR}.{VERSION_BUILD}"
+if VERSION_ALPHA:
+    VERSION_STR += f"a{VERSION_ALPHA}"

tugatagger-0.0.1/tugatagger.egg-info/PKG-INFO

@@ -0,0 +1,24 @@
+Metadata-Version: 2.4
+Name: tugatagger
+Version: 0.0.1
+Summary: A unified wrapper for Portuguese POS tagging and benchmarking.
+Home-page: https://github.com/TigreGotico/tugatagger
+Author: JarbasAI
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Natural Language :: Portuguese
+Requires-Python: >=3.7
+Provides-Extra: spacy
+Requires-Dist: spacy; extra == "spacy"
+Provides-Extra: lexicon
+Requires-Dist: tugalex; extra == "lexicon"
+Provides-Extra: brill
+Requires-Dist: brill_postagger; extra == "brill"
+Dynamic: author
+Dynamic: classifier
+Dynamic: home-page
+Dynamic: provides-extra
+Dynamic: requires-python
+Dynamic: summary

tugatagger-0.0.1/tugatagger.egg-info/SOURCES.txt

@@ -0,0 +1,9 @@
+README.md
+setup.py
+tugatagger/__init__.py
+tugatagger/version.py
+tugatagger.egg-info/PKG-INFO
+tugatagger.egg-info/SOURCES.txt
+tugatagger.egg-info/dependency_links.txt
+tugatagger.egg-info/requires.txt
+tugatagger.egg-info/top_level.txt

tugatagger-0.0.1/tugatagger.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

tugatagger-0.0.1/tugatagger.egg-info/requires.txt

@@ -0,0 +1,9 @@
+
+[brill]
+brill_postagger
+
+[lexicon]
+tugalex
+
+[spacy]
+spacy

tugatagger-0.0.1/tugatagger.egg-info/top_level.txt

@@ -0,0 +1 @@
+tugatagger