gngram-lookup 0.2.0__tar.gz → 0.2.2__tar.gz

{gngram_lookup-0.2.0 → gngram_lookup-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: gngram-lookup
-Version: 0.2.0
+Version: 0.2.2
 Summary: Static Hash-Based Lookup for Google Ngram Frequencies
 Home-page: https://github.com/craigtrim/gngram-lookup
 License: Proprietary
@@ -30,6 +30,9 @@ Description-Content-Type: text/markdown
 # gngram-lookup
 
 [![PyPI version](https://badge.fury.io/py/gngram-lookup.svg)](https://badge.fury.io/py/gngram-lookup)
+[![Downloads](https://pepy.tech/badge/gngram-lookup)](https://pepy.tech/project/gngram-lookup)
+[![Downloads/Month](https://pepy.tech/badge/gngram-lookup/month)](https://pepy.tech/project/gngram-lookup)
+[![Tests](https://img.shields.io/badge/tests-131-brightgreen)](https://github.com/craigtrim/gngram-lookup/tree/main/tests)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
 
 Word frequency from 500 years of books. O(1) lookup. 5 million words.
@@ -71,11 +74,16 @@ gngram-freq computer
 
 ## Docs
 
-- [API Reference](docs/api.md)
-- [CLI Reference](docs/cli.md)
-- [Data Format](docs/data-format.md)
-- [Use Cases](docs/use-cases.md)
-- [Development](docs/development.md)
+- [API Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/api.md)
+- [CLI Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/cli.md)
+- [Data Format](https://github.com/craigtrim/gngram-lookup/blob/main/docs/data-format.md)
+- [Use Cases](https://github.com/craigtrim/gngram-lookup/blob/main/docs/use-cases.md)
+- [Development](https://github.com/craigtrim/gngram-lookup/blob/main/docs/development.md)
+
+## See Also
+
+- [bnc-lookup](https://pypi.org/project/bnc-lookup/) - O(1) lookup for British National Corpus
+- [wordnet-lookup](https://pypi.org/project/wordnet-lookup/) - O(1) lookup for WordNet
 
 ## Attribution
 
@@ -83,5 +91,5 @@ Data derived from the [Google Books Ngram](https://books.google.com/ngrams) data
 
 ## License
 
-Proprietary. See [LICENSE](LICENSE).
+Proprietary. See [LICENSE](https://github.com/craigtrim/gngram-lookup/blob/main/LICENSE).
 

{gngram_lookup-0.2.0 → gngram_lookup-0.2.2}/README.md

@@ -1,6 +1,9 @@
 # gngram-lookup
 
 [![PyPI version](https://badge.fury.io/py/gngram-lookup.svg)](https://badge.fury.io/py/gngram-lookup)
+[![Downloads](https://pepy.tech/badge/gngram-lookup)](https://pepy.tech/project/gngram-lookup)
+[![Downloads/Month](https://pepy.tech/badge/gngram-lookup/month)](https://pepy.tech/project/gngram-lookup)
+[![Tests](https://img.shields.io/badge/tests-131-brightgreen)](https://github.com/craigtrim/gngram-lookup/tree/main/tests)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
 
 Word frequency from 500 years of books. O(1) lookup. 5 million words.
@@ -42,11 +45,16 @@ gngram-freq computer
 
 ## Docs
 
-- [API Reference](docs/api.md)
-- [CLI Reference](docs/cli.md)
-- [Data Format](docs/data-format.md)
-- [Use Cases](docs/use-cases.md)
-- [Development](docs/development.md)
+- [API Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/api.md)
+- [CLI Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/cli.md)
+- [Data Format](https://github.com/craigtrim/gngram-lookup/blob/main/docs/data-format.md)
+- [Use Cases](https://github.com/craigtrim/gngram-lookup/blob/main/docs/use-cases.md)
+- [Development](https://github.com/craigtrim/gngram-lookup/blob/main/docs/development.md)
+
+## See Also
+
+- [bnc-lookup](https://pypi.org/project/bnc-lookup/) - O(1) lookup for British National Corpus
+- [wordnet-lookup](https://pypi.org/project/wordnet-lookup/) - O(1) lookup for WordNet
 
 ## Attribution
 
@@ -54,4 +62,4 @@ Data derived from the [Google Books Ngram](https://books.google.com/ngrams) data
 
 ## License
 
-Proprietary. See [LICENSE](LICENSE).
+Proprietary. See [LICENSE](https://github.com/craigtrim/gngram-lookup/blob/main/LICENSE).

gngram_lookup-0.2.2/gngram_counter/lookup.py

@@ -0,0 +1,244 @@
+"""
+High-level lookup API for gngram-counter.
+
+Provides simple functions for word frequency lookups similar to bnc-lookup.
+
+Includes contraction fallback: if a contraction like "don't" is not found
+directly, the stem ("do") is looked up instead. The ngram corpus only
+contains pure alphabetic words, so contractions and their suffix parts
+(n't, 'll, etc.) are absent — but the stems are present.
+"""
+
+import hashlib
+from functools import lru_cache
+from typing import TypedDict
+
+import polars as pl
+
+from gngram_counter.data import get_hash_file, is_data_installed
+from gngram_counter.normalize import normalize
+
+
+class FrequencyData(TypedDict):
+    """Frequency data for a word."""
+
+    peak_tf: int  # Decade with highest term frequency
+    peak_df: int  # Decade with highest document frequency
+    sum_tf: int  # Total term frequency across all decades
+    sum_df: int  # Total document frequency across all decades
+
+
+# Contraction suffixes stored as separate tokens in the ngram corpus
+# Order matters: longer suffixes must be checked before shorter ones
+CONTRACTION_SUFFIXES = ("n't", "'ll", "'re", "'ve", "'m", "'d")
+
+# Specific stems that form 's contractions (where 's = "is" or "has").
+# NOT generalized — 's is ambiguous with possessive, so only known
+# contraction stems are listed here. Ported from bnc-lookup.
+S_CONTRACTION_STEMS = frozenset({
+    # Pronouns (unambiguously 's = "is" or "has", never possessive)
+    'it', 'he', 'she', 'that', 'what', 'who',
+    # Adverbs / demonstratives
+    'where', 'how', 'here', 'there',
+    # "let's" = "let us"
+    'let',
+    # Indefinite pronouns
+    'somebody', 'everybody', 'everyone', 'nobody',
+    'anywhere', 'nowhere',
+})
+
+
+@lru_cache(maxsize=256)
+def _load_bucket(prefix: str) -> pl.DataFrame:
+    """Load and cache a parquet bucket file."""
+    return pl.read_parquet(get_hash_file(prefix))
+
+
+def _hash_word(word: str) -> tuple[str, str]:
+    """Hash a word and return (prefix, suffix)."""
+    h = hashlib.md5(normalize(word).encode("utf-8")).hexdigest()
+    return h[:2], h[2:]
+
+
+def _lookup_frequency(word: str) -> FrequencyData | None:
+    """Look up frequency data for a single word form (no fallbacks)."""
+    if not word:
+        return None
+    prefix, suffix = _hash_word(word)
+    try:
+        df = _load_bucket(prefix)
+    except FileNotFoundError:
+        return None
+    row = df.filter(pl.col("hash") == suffix)
+    if len(row) == 0:
+        return None
+    return FrequencyData(
+        peak_tf=row["peak_tf"][0],
+        peak_df=row["peak_df"][0],
+        sum_tf=row["sum_tf"][0],
+        sum_df=row["sum_df"][0],
+    )
+
+
+def _split_contraction(word: str) -> tuple[str, str] | None:
+    """Split a contraction into its component parts if possible.
+
+    The ngram corpus tokenizes contractions separately (e.g., "we'll" -> "we" + "'ll").
+    This function reverses that split for fallback lookup.
+
+    Returns:
+        Tuple of (stem, suffix) if the word matches a contraction pattern,
+        or None if no contraction pattern matches.
+    """
+    for suffix in CONTRACTION_SUFFIXES:
+        if word.endswith(suffix):
+            stem = word[:-len(suffix)]
+            if stem:
+                return (stem, suffix)
+
+    # Specific 's contractions from curated allowlist (not possessives)
+    if word.endswith("'s"):
+        stem = word[:-2]
+        if stem in S_CONTRACTION_STEMS:
+            return (stem, "'s")
+
+    return None
+
+
+def exists(word: str) -> bool:
+    """Check if a word exists in the ngram data.
+
+    Performs case-insensitive lookup with automatic fallbacks:
+    1. Direct lookup of the normalized word
+    2. Contraction fallback: if word is a contraction, check if both
+       components exist (e.g., "don't" -> "do" + "n't")
+
+    Args:
+        word: The word to check (case-insensitive)
+
+    Returns:
+        True if the word exists, False otherwise
+
+    Raises:
+        FileNotFoundError: If data files are not installed
+    """
+    if not is_data_installed():
+        raise FileNotFoundError(
+            "Data files not installed. Run: python -m gngram_counter.download_data"
+        )
+
+    word = normalize(word)
+
+    if _lookup_frequency(word) is not None:
+        return True
+
+    # Contraction fallback: check if the stem exists
+    parts = _split_contraction(word)
+    if parts:
+        stem, _ = parts
+        if _lookup_frequency(stem) is not None:
+            return True
+
+    return False
+
+
+def frequency(word: str) -> FrequencyData | None:
+    """Get frequency data for a word.
+
+    Performs case-insensitive lookup with contraction fallback.
+    For contractions, returns the stem's frequency data.
+
+    Args:
+        word: The word to look up (case-insensitive)
+
+    Returns:
+        FrequencyData dict with peak_tf, peak_df, sum_tf, sum_df, or None if not found
+
+    Raises:
+        FileNotFoundError: If data files are not installed
+    """
+    if not is_data_installed():
+        raise FileNotFoundError(
+            "Data files not installed. Run: python -m gngram_counter.download_data"
+        )
+
+    word = normalize(word)
+
+    result = _lookup_frequency(word)
+    if result is not None:
+        return result
+
+    # Contraction fallback: return the stem's frequency
+    parts = _split_contraction(word)
+    if parts:
+        stem, _ = parts
+        stem_freq = _lookup_frequency(stem)
+        if stem_freq is not None:
+            return stem_freq
+
+    return None
+
+
+def batch_frequency(words: list[str]) -> dict[str, FrequencyData | None]:
+    """Get frequency data for multiple words.
+
+    Args:
+        words: List of words to look up (case-insensitive)
+
+    Returns:
+        Dict mapping each word to its FrequencyData or None if not found
+
+    Raises:
+        FileNotFoundError: If data files are not installed
+    """
+    if not is_data_installed():
+        raise FileNotFoundError(
+            "Data files not installed. Run: python -m gngram_counter.download_data"
+        )
+
+    # Group words by bucket prefix for efficient batch lookups
+    by_prefix: dict[str, list[tuple[str, str, str]]] = {}
+    contraction_words: list[str] = []
+
+    for word in words:
+        normalized = normalize(word)
+        prefix, suffix = _hash_word(normalized)
+        if prefix not in by_prefix:
+            by_prefix[prefix] = []
+        by_prefix[prefix].append((word, normalized, suffix))
+
+    results: dict[str, FrequencyData | None] = {}
+
+    for prefix, entries in by_prefix.items():
+        df = _load_bucket(prefix)
+        suffixes = [s for _, _, s in entries]
+
+        # Filter to all matching suffixes at once
+        matches = df.filter(pl.col("hash").is_in(suffixes))
+        match_dict = {row["hash"]: row for row in matches.iter_rows(named=True)}
+
+        for word, normalized, suffix in entries:
+            if suffix in match_dict:
+                row = match_dict[suffix]
+                results[word] = FrequencyData(
+                    peak_tf=row["peak_tf"],
+                    peak_df=row["peak_df"],
+                    sum_tf=row["sum_tf"],
+                    sum_df=row["sum_df"],
+                )
+            else:
+                # Mark for contraction fallback
+                results[word] = None
+                contraction_words.append(word)
+
+    # Contraction fallback for words not found directly
+    for word in contraction_words:
+        normalized = normalize(word)
+        parts = _split_contraction(normalized)
+        if parts:
+            stem, _ = parts
+            stem_freq = _lookup_frequency(stem)
+            if stem_freq is not None:
+                results[word] = stem_freq
+
+    return results

gngram_lookup-0.2.2/gngram_counter/normalize.py

@@ -0,0 +1,48 @@
+"""Text normalization utilities for gngram-counter.
+
+Handles normalization of Unicode apostrophe variants and other text
+transformations to ensure consistent matching against the ngram corpus.
+
+Ported from bnc-lookup normalize.py.
+"""
+
+# Unicode characters that should normalize to ASCII apostrophe (U+0027)
+# Ordered by likelihood of occurrence in English text
+APOSTROPHE_VARIANTS = (
+    '\u2019'  # RIGHT SINGLE QUOTATION MARK (most common smart quote)
+    '\u2018'  # LEFT SINGLE QUOTATION MARK
+    '\u0060'  # GRAVE ACCENT
+    '\u00B4'  # ACUTE ACCENT
+    '\u201B'  # SINGLE HIGH-REVERSED-9 QUOTATION MARK
+    '\u2032'  # PRIME
+    '\u2035'  # REVERSED PRIME
+    '\u02B9'  # MODIFIER LETTER PRIME
+    '\u02BC'  # MODIFIER LETTER APOSTROPHE
+    '\u02C8'  # MODIFIER LETTER VERTICAL LINE
+    '\u0313'  # COMBINING COMMA ABOVE
+    '\u0315'  # COMBINING COMMA ABOVE RIGHT
+    '\u055A'  # ARMENIAN APOSTROPHE
+    '\u05F3'  # HEBREW PUNCTUATION GERESH
+    '\u07F4'  # NKO HIGH TONE APOSTROPHE
+    '\u07F5'  # NKO LOW TONE APOSTROPHE
+    '\uFF07'  # FULLWIDTH APOSTROPHE
+    '\u1FBF'  # GREEK PSILI
+    '\u1FBD'  # GREEK KORONIS
+    '\uA78C'  # LATIN SMALL LETTER SALTILLO
+)
+
+# Pre-compiled translation table for fast apostrophe normalization
+_APOSTROPHE_TABLE = str.maketrans({char: "'" for char in APOSTROPHE_VARIANTS})
+
+
+def normalize_apostrophes(text: str) -> str:
+    """Normalize Unicode apostrophe variants to ASCII apostrophe."""
+    return text.translate(_APOSTROPHE_TABLE)
+
+
+def normalize(text: str) -> str:
+    """Normalize text for ngram lookup.
+
+    Applies: apostrophe variant conversion, lowercase, strip whitespace.
+    """
+    return normalize_apostrophes(text).lower().strip()

{gngram_lookup-0.2.0 → gngram_lookup-0.2.2}/pyproject.toml

@@ -1,7 +1,7 @@
 [tool.poetry]
 name = "gngram-lookup"
 packages = [{include = "gngram_counter"}]
-version = "0.2.0"
+version = "0.2.2"
 description = "Static Hash-Based Lookup for Google Ngram Frequencies"
 authors = ["Craig Trim <craigtrim@gmail.com>"]
 maintainers = ["Craig Trim <craigtrim@gmail.com>"]

gngram_lookup-0.2.2/setup.py

@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+from setuptools import setup
+
+packages = \
+['gngram_counter']
+
+package_data = \
+{'': ['*']}
+
+install_requires = \
+['polars>=1.0,<2.0', 'pyarrow>=18.0,<19.0']
+
+entry_points = \
+{'console_scripts': ['gngram-exists = gngram_counter.cli:gngram_exists',
+                     'gngram-freq = gngram_counter.cli:gngram_freq']}
+
+setup_kwargs = {
+    'name': 'gngram-lookup',
+    'version': '0.2.2',
+    'description': 'Static Hash-Based Lookup for Google Ngram Frequencies',
+    'long_description': "# gngram-lookup\n\n[![PyPI version](https://badge.fury.io/py/gngram-lookup.svg)](https://badge.fury.io/py/gngram-lookup)\n[![Downloads](https://pepy.tech/badge/gngram-lookup)](https://pepy.tech/project/gngram-lookup)\n[![Downloads/Month](https://pepy.tech/badge/gngram-lookup/month)](https://pepy.tech/project/gngram-lookup)\n[![Tests](https://img.shields.io/badge/tests-131-brightgreen)](https://github.com/craigtrim/gngram-lookup/tree/main/tests)\n[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)\n\nWord frequency from 500 years of books. O(1) lookup. 5 million words.\n\n## Install\n\n```bash\npip install gngram-lookup\npython -m gngram_lookup.download_data\n```\n\n## Python\n\n```python\nimport gngram_lookup as ng\n\nng.exists('computer')       # True\nng.exists('xyznotaword')    # False\n\nng.frequency('computer')\n# {'peak_tf': 2000, 'peak_df': 2000, 'sum_tf': 892451, 'sum_df': 312876}\n\nng.batch_frequency(['the', 'algorithm', 'xyznotaword'])\n# {'the': {...}, 'algorithm': {...}, 'xyznotaword': None}\n```\n\n## CLI\n\n```bash\ngngram-exists computer    # True, exit 0\ngngram-exists xyznotaword # False, exit 1\n\ngngram-freq computer\n# peak_tf_decade: 2000\n# peak_df_decade: 2000\n# sum_tf: 892451\n# sum_df: 312876\n```\n\n## Docs\n\n- [API Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/api.md)\n- [CLI Reference](https://github.com/craigtrim/gngram-lookup/blob/main/docs/cli.md)\n- [Data Format](https://github.com/craigtrim/gngram-lookup/blob/main/docs/data-format.md)\n- [Use Cases](https://github.com/craigtrim/gngram-lookup/blob/main/docs/use-cases.md)\n- [Development](https://github.com/craigtrim/gngram-lookup/blob/main/docs/development.md)\n\n## See Also\n\n- [bnc-lookup](https://pypi.org/project/bnc-lookup/) - O(1) lookup for British National Corpus\n- [wordnet-lookup](https://pypi.org/project/wordnet-lookup/) - O(1) lookup for WordNet\n\n## Attribution\n\nData derived from the [Google Books Ngram](https://books.google.com/ngrams) dataset.\n\n## License\n\nProprietary. See [LICENSE](https://github.com/craigtrim/gngram-lookup/blob/main/LICENSE).\n",
+    'author': 'Craig Trim',
+    'author_email': 'craigtrim@gmail.com',
+    'maintainer': 'Craig Trim',
+    'maintainer_email': 'craigtrim@gmail.com',
+    'url': 'https://github.com/craigtrim/gngram-lookup',
+    'packages': packages,
+    'package_data': package_data,
+    'install_requires': install_requires,
+    'entry_points': entry_points,
+    'python_requires': '>=3.11,<4.0',
+}
+
+
+setup(**setup_kwargs)

gngram_lookup-0.2.0/gngram_counter/lookup.py

@@ -1,138 +0,0 @@
-"""
-High-level lookup API for gngram-counter.
-
-Provides simple functions for word frequency lookups similar to bnc-lookup.
-"""
-
-import hashlib
-from functools import lru_cache
-from typing import TypedDict
-
-import polars as pl
-
-from gngram_counter.data import get_hash_file, is_data_installed
-
-
-class FrequencyData(TypedDict):
-    """Frequency data for a word."""
-
-    peak_tf: int  # Decade with highest term frequency
-    peak_df: int  # Decade with highest document frequency
-    sum_tf: int  # Total term frequency across all decades
-    sum_df: int  # Total document frequency across all decades
-
-
-@lru_cache(maxsize=256)
-def _load_bucket(prefix: str) -> pl.DataFrame:
-    """Load and cache a parquet bucket file."""
-    return pl.read_parquet(get_hash_file(prefix))
-
-
-def _hash_word(word: str) -> tuple[str, str]:
-    """Hash a word and return (prefix, suffix)."""
-    h = hashlib.md5(word.lower().encode("utf-8")).hexdigest()
-    return h[:2], h[2:]
-
-
-def exists(word: str) -> bool:
-    """Check if a word exists in the ngram data.
-
-    Args:
-        word: The word to check (case-insensitive)
-
-    Returns:
-        True if the word exists, False otherwise
-
-    Raises:
-        FileNotFoundError: If data files are not installed
-    """
-    if not is_data_installed():
-        raise FileNotFoundError(
-            "Data files not installed. Run: python -m gngram_counter.download_data"
-        )
-
-    prefix, suffix = _hash_word(word)
-    df = _load_bucket(prefix)
-    return len(df.filter(pl.col("hash") == suffix)) > 0
-
-
-def frequency(word: str) -> FrequencyData | None:
-    """Get frequency data for a word.
-
-    Args:
-        word: The word to look up (case-insensitive)
-
-    Returns:
-        FrequencyData dict with peak_tf, peak_df, sum_tf, sum_df, or None if not found
-
-    Raises:
-        FileNotFoundError: If data files are not installed
-    """
-    if not is_data_installed():
-        raise FileNotFoundError(
-            "Data files not installed. Run: python -m gngram_counter.download_data"
-        )
-
-    prefix, suffix = _hash_word(word)
-    df = _load_bucket(prefix)
-    row = df.filter(pl.col("hash") == suffix)
-
-    if len(row) == 0:
-        return None
-
-    return FrequencyData(
-        peak_tf=row["peak_tf"][0],
-        peak_df=row["peak_df"][0],
-        sum_tf=row["sum_tf"][0],
-        sum_df=row["sum_df"][0],
-    )
-
-
-def batch_frequency(words: list[str]) -> dict[str, FrequencyData | None]:
-    """Get frequency data for multiple words.
-
-    Args:
-        words: List of words to look up (case-insensitive)
-
-    Returns:
-        Dict mapping each word to its FrequencyData or None if not found
-
-    Raises:
-        FileNotFoundError: If data files are not installed
-    """
-    if not is_data_installed():
-        raise FileNotFoundError(
-            "Data files not installed. Run: python -m gngram_counter.download_data"
-        )
-
-    # Group words by bucket prefix for efficient batch lookups
-    by_prefix: dict[str, list[tuple[str, str]]] = {}
-    for word in words:
-        prefix, suffix = _hash_word(word)
-        if prefix not in by_prefix:
-            by_prefix[prefix] = []
-        by_prefix[prefix].append((word, suffix))
-
-    results: dict[str, FrequencyData | None] = {}
-
-    for prefix, word_suffix_pairs in by_prefix.items():
-        df = _load_bucket(prefix)
-        suffixes = [s for _, s in word_suffix_pairs]
-
-        # Filter to all matching suffixes at once
-        matches = df.filter(pl.col("hash").is_in(suffixes))
-        match_dict = {row["hash"]: row for row in matches.iter_rows(named=True)}
-
-        for word, suffix in word_suffix_pairs:
-            if suffix in match_dict:
-                row = match_dict[suffix]
-                results[word] = FrequencyData(
-                    peak_tf=row["peak_tf"],
-                    peak_df=row["peak_df"],
-                    sum_tf=row["sum_tf"],
-                    sum_df=row["sum_df"],
-                )
-            else:
-                results[word] = None
-
-    return results

gngram_lookup-0.2.0/setup.py

@@ -1,35 +0,0 @@
-# -*- coding: utf-8 -*-
-from setuptools import setup
-
-packages = \
-['gngram_counter']
-
-package_data = \
-{'': ['*']}
-
-install_requires = \
-['polars>=1.0,<2.0', 'pyarrow>=18.0,<19.0']
-
-entry_points = \
-{'console_scripts': ['gngram-exists = gngram_counter.cli:gngram_exists',
-                     'gngram-freq = gngram_counter.cli:gngram_freq']}
-
-setup_kwargs = {
-    'name': 'gngram-lookup',
-    'version': '0.2.0',
-    'description': 'Static Hash-Based Lookup for Google Ngram Frequencies',
-    'long_description': "# gngram-lookup\n\n[![PyPI version](https://badge.fury.io/py/gngram-lookup.svg)](https://badge.fury.io/py/gngram-lookup)\n[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)\n\nWord frequency from 500 years of books. O(1) lookup. 5 million words.\n\n## Install\n\n```bash\npip install gngram-lookup\npython -m gngram_lookup.download_data\n```\n\n## Python\n\n```python\nimport gngram_lookup as ng\n\nng.exists('computer')       # True\nng.exists('xyznotaword')    # False\n\nng.frequency('computer')\n# {'peak_tf': 2000, 'peak_df': 2000, 'sum_tf': 892451, 'sum_df': 312876}\n\nng.batch_frequency(['the', 'algorithm', 'xyznotaword'])\n# {'the': {...}, 'algorithm': {...}, 'xyznotaword': None}\n```\n\n## CLI\n\n```bash\ngngram-exists computer    # True, exit 0\ngngram-exists xyznotaword # False, exit 1\n\ngngram-freq computer\n# peak_tf_decade: 2000\n# peak_df_decade: 2000\n# sum_tf: 892451\n# sum_df: 312876\n```\n\n## Docs\n\n- [API Reference](docs/api.md)\n- [CLI Reference](docs/cli.md)\n- [Data Format](docs/data-format.md)\n- [Use Cases](docs/use-cases.md)\n- [Development](docs/development.md)\n\n## Attribution\n\nData derived from the [Google Books Ngram](https://books.google.com/ngrams) dataset.\n\n## License\n\nProprietary. See [LICENSE](LICENSE).\n",
-    'author': 'Craig Trim',
-    'author_email': 'craigtrim@gmail.com',
-    'maintainer': 'Craig Trim',
-    'maintainer_email': 'craigtrim@gmail.com',
-    'url': 'https://github.com/craigtrim/gngram-lookup',
-    'packages': packages,
-    'package_data': package_data,
-    'install_requires': install_requires,
-    'entry_points': entry_points,
-    'python_requires': '>=3.11,<4.0',
-}
-
-
-setup(**setup_kwargs)