pdacek 0.1.0__tar.gz

pdacek-0.1.0/.gitignore

@@ -0,0 +1,26 @@
+# build artefacts
+build/
+dist/
+*.egg-info/
+.eggs/
+
+# environments
+.venv/
+venv/
+__pycache__/
+*.py[cod]
+
+# tooling
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# secrets — never commit a PyPI token
+.pypirc
+.pypi-token
+
+# OS noise
+.DS_Store
+Thumbs.db

pdacek-0.1.0/CHANGELOG.md

@@ -0,0 +1,18 @@
+# Changelog
+
+All notable changes to this project are documented here.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
+and the project uses [Semantic Versioning](https://semver.org/).
+
+## [0.1.0] - 2026-11-15
+
+### Added
+
+- Five experiments: preprocessing pipeline, n-gram probabilities, minimum edit
+  distance, CFG parsing (top-down and bottom-up), add-one Naive Bayes.
+- `pdacek.p1()` … `pdacek.p5()` shorthands for notebook use.
+- `source(n)` / `save(n)` for standalone listings.
+- `pdacek` console script and `python -m pdacek` entry point.
+- Regex tokenizer and built-in stop list as offline fallbacks for missing NLTK
+  corpora.
+- Test suite covering every experiment, the public API and the CLI.

pdacek-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sohel Havaldar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pdacek-0.1.0/PKG-INFO

@@ -0,0 +1,19 @@
+Metadata-Version: 2.5
+Name: pdacek
+Version: 0.1.0
+License: MIT
+License-File: LICENSE
+Requires-Python: >=3.9
+Requires-Dist: nltk>=3.8
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: twine>=5.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+import pdacek
+pdacek.p1()          # preprocessing pipeline
+pdacek.p2()          # n-gram probabilities
+pdacek.p3()          # minimum edit distance
+pdacek.p4()          # top-down / bottom-up parsing
+pdacek.p5()          # Naive Bayes

pdacek-0.1.0/PUBLISH.md

@@ -0,0 +1,103 @@
+# Publishing checklist
+
+Everything below is local until the upload step — nothing is sent anywhere
+without a PyPI token.
+
+## 1. The name
+
+`pdacek` — checked free on PyPI on 2026-11-15 (a 404 from the JSON API). Kept short
+on purpose: `import pdacek`, `pdacek.p1()`, `pdacek 3`. It is a valid identifier, not a
+Python keyword, and does not shadow a standard-library module.
+
+83 other two-letter names are also free; the full scan of all 676 combinations
+is in `../two-letter-names/pypi-two-letter-names.csv`. If you ever want to
+switch, one command rewrites `pyproject.toml`, the source directory, imports,
+docs and tests:
+
+```bash
+python tools/rename.py <new-name>
+```
+
+Then rebuild.
+
+## 2. Check the name is still free
+
+```bash
+curl -s -o /dev/null -w "%{http_code}\n" https://pypi.org/pypi/<name>/json
+# 404 = free, 200 = taken
+```
+
+PyPI also rejects names that only differ from an existing project by `-`, `_`
+or `.`, so pick something that normalises differently from `plnlab`.
+
+## 3. Build
+
+```bash
+python -m pip install -U build twine
+python -m build
+python -m twine check dist/*
+```
+
+`dist/` should contain one `.whl` and one `.tar.gz`, both `PASSED`.
+
+## 4. Rehearse on TestPyPI (recommended)
+
+```bash
+python -m twine upload --repository testpypi dist/*
+python -m pip install --index-url https://test.pypi.org/simple/ \
+    --extra-index-url https://pypi.org/simple/ pdacek
+```
+
+A TestPyPI rehearsal catches metadata and filename problems without burning a
+version number on the real index.
+
+## 5. Publish for real
+
+Create an API token at <https://pypi.org/manage/account/token/> (scope it to the
+project after the first upload), then:
+
+```bash
+export TWINE_USERNAME=__token__
+export TWINE_PASSWORD=pypi-AgEIcHlwaS5vcmc...   # paste the token
+python -m twine upload dist/*
+```
+
+Or pass it inline without leaving it in shell history:
+
+```bash
+TWINE_USERNAME=__token__ TWINE_PASSWORD="$(cat ~/.pypi-token)" python -m twine upload dist/*
+```
+
+Never commit the token; `~/.pypirc` with `chmod 600` is the usual alternative.
+
+## 6. After the upload
+
+```bash
+pip install pdacek          # or the name you chose
+python -c "import pdacek; pdacek.p1()"
+```
+
+The project page appears at `https://pypi.org/project/<name>/` within a minute
+or two; the JSON API updates immediately.
+
+## 7. Releasing again
+
+1. Bump `version` in `pyproject.toml` and `__version__` in
+   `src/<name>/__init__.py` (keep them in sync — the test suite checks it).
+2. Add a `CHANGELOG.md` entry.
+3. `python -m build && python -m twine upload dist/*`.
+
+Versions are immutable on PyPI: a published version can be yanked, not
+replaced, so bump before every upload.
+
+## Notes on what this package does at runtime
+
+- No network calls, no telemetry, no file writes outside the paths you pass to
+  `save()`.
+- `nltk.download(...)` is only attempted when the local corpus is missing, and
+  a failure silently falls back to the built-in regex tokenizer.
+- `save(n)` writes `<slug>.py` in the directory you name (default: current).
+
+Kept deliberately boring so the package stays a normal, inspectable library:
+anyone running `pip download` and reading `src/` sees exactly the five lab
+programs and nothing else.

pdacek-0.1.0/README.md

@@ -0,0 +1,6 @@
+import pdacek
+pdacek.p1()          # preprocessing pipeline
+pdacek.p2()          # n-gram probabilities
+pdacek.p3()          # minimum edit distance
+pdacek.p4()          # top-down / bottom-up parsing
+pdacek.p5()          # Naive Bayes

pdacek-0.1.0/conftest.py

@@ -0,0 +1,8 @@
+"""Make ``pytest`` work from a fresh clone without installing the package."""
+
+import sys
+from pathlib import Path
+
+SRC = Path(__file__).resolve().parent / "src"
+if SRC.is_dir() and str(SRC) not in sys.path:
+    sys.path.insert(0, str(SRC))

pdacek-0.1.0/examples/notebook_cells.py

@@ -0,0 +1,44 @@
+"""What a Colab/Jupyter session looks like, cell by cell.
+
+Cell 1
+------
+%pip install -q pdacek
+
+Cell 2
+------
+import pdacek
+pdacek.programs()
+
+Cell 3 - run one experiment inline
+----------------------------------
+pdacek.p3()
+
+Cell 4 - or print the listing and keep editing it below
+--------------------------------------------------------
+pdacek.source(3)
+
+Cell 5 - use the functions directly, no printing
+------------------------------------------------
+from pdacek import NaiveBayes
+
+model = NaiveBayes().fit([
+    ("fun, couple, love, love", "comedy"),
+    ("fast, furious, shoot", "action"),
+])
+model.predict(["love", "fun"])
+"""
+
+from pdacek import NaiveBayes, Parser, min_edit_distance, preprocess
+
+# Functions instead of printed output: handy when you want to grade results.
+print(min_edit_distance("kitten", "sitting"))
+
+parser = Parser([("S", ["NP", "VP"]), ("NP", ["Det", "N"]), ("VP", ["V"])])
+print(parser.top_down_parse(["the", "cat", "chases"]))
+
+print(preprocess()["stemmed"][:5])
+
+model = NaiveBayes().fit(
+    [("fun, couple, love, love", "comedy"), ("fast, furious, shoot", "action")]
+)
+print(model.predict(["love", "fun"]))

pdacek-0.1.0/examples/run_all.py

@@ -0,0 +1,15 @@
+"""Run every experiment in order: python examples/run_all.py [number ...]."""
+
+import sys
+
+import pdacek
+
+
+def main(numbers):
+    for number in numbers or pdacek.programs():
+        print(f"\n{'=' * 70}\nProgram {number}: {pdacek.programs()[number]}\n{'=' * 70}")
+        pdacek.run(number)
+
+
+if __name__ == "__main__":
+    main([int(arg) for arg in sys.argv[1:]])

pdacek-0.1.0/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["hatchling>=1.24"]
+build-backend = "hatchling.build"
+
+[project]
+name = "pdacek"
+version = "0.1.0"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+dependencies = ["nltk>=3.8"]
+
+[project.optional-dependencies]
+dev = ["pytest>=8.0", "build>=1.2", "twine>=5.0"]
+
+[project.scripts]
+pdacek = "pdacek.cli:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/pdacek"]
+
+[tool.hatch.build.targets.sdist]
+include = [
+  "src/pdacek",
+  "tests",
+  "examples",
+  "tools",
+  "README.md",
+  "CHANGELOG.md",
+  "PUBLISH.md",
+  "LICENSE",
+  "conftest.py",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

pdacek-0.1.0/src/pdacek/__init__.py

@@ -0,0 +1,116 @@
+"""pdacek - a plain, notebook-friendly NLP lab toolkit.
+
+Five self-contained experiments, each available three ways:
+
+* run it inline  ->  ``pdacek.p3()``
+* read the code   ->  ``print(pdacek.source(3))``
+* write it to disk -> ``pdacek.save(3)``
+
+The functions are also importable individually, e.g.
+``from pdacek import min_edit_distance``.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Dict, List, Optional
+
+from .programs import PROGRAMS, numbers, run, source, titles
+from .programs.pl01_preprocessing import (
+    filtration,
+    preprocess,
+    stopword_removal,
+    stemming,
+    tokenization,
+    validation,
+)
+from .programs.pl02_ngrams import build_corpus, ngram_counts, ngram_probabilities
+from .programs.pl03_edit_distance import edit_table, med, min_edit_distance, trace_operations
+from .programs.pl04_parsing import Parser
+from .programs.pl05_naive_bayes import NaiveBayes
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "__version__",
+    "programs",
+    "run",
+    "source",
+    "save",
+    "p1",
+    "p2",
+    "p3",
+    "p4",
+    "p5",
+    # helpers
+    "tokenization",
+    "filtration",
+    "validation",
+    "stopword_removal",
+    "stemming",
+    "preprocess",
+    "build_corpus",
+    "ngram_counts",
+    "ngram_probabilities",
+    "edit_table",
+    "med",
+    "min_edit_distance",
+    "trace_operations",
+    "Parser",
+    "NaiveBayes",
+]
+
+
+def programs() -> Dict[int, str]:
+    """Return ``{number: title}`` for every bundled experiment."""
+    return titles()
+
+
+def save(number: int, directory: "str | Path" = ".") -> Path:
+    """Write a program listing to ``directory`` and return its path."""
+    program = PROGRAMS[int(number)]
+    target = Path(directory) / program.script_name
+    target.parent.mkdir(parents=True, exist_ok=True)
+    target.write_text(source(number), encoding="utf-8")
+    return target
+
+
+def _run_all() -> List[int]:
+    for number in numbers():
+        run(number)
+        print()
+    return numbers()
+
+
+def p1() -> None:
+    """Program 1 - text preprocessing pipeline."""
+    run(1)
+
+
+def p2() -> None:
+    """Program 2 - unigram, bigram, trigram probabilities."""
+    run(2)
+
+
+def p3() -> None:
+    """Program 3 - minimum edit distance."""
+    run(3)
+
+
+def p4() -> None:
+    """Program 4 - top-down and bottom-up parsing."""
+    run(4)
+
+
+def p5() -> None:
+    """Program 5 - add-one smoothed Naive Bayes."""
+    run(5)
+
+
+def __getattr__(name: str) -> Optional[object]:  # pragma: no cover - tiny shim
+    """Allow ``pdacek.p6`` style access to fail with a clear message."""
+    if name.startswith("p") and name[1:].isdigit():
+        raise AttributeError(
+            f"no program {name[1:]} in this release; available: {numbers()}"
+        )
+    raise AttributeError(f"module 'pdacek' has no attribute {name!r}")

pdacek-0.1.0/src/pdacek/__main__.py

@@ -0,0 +1,6 @@
+"""Support ``python -m pdacek``."""
+
+from .cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pdacek-0.1.0/src/pdacek/cli.py

@@ -0,0 +1,98 @@
+"""Command line entry point: ``pdacek 3``, ``pdacek --list``, ..."""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from typing import List, Optional
+
+from . import __version__, numbers, programs, run, save, source
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="pdacek",
+        description="Run the bundled NLP lab experiments without opening an editor.",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=(
+            "examples:\n"
+            "  pdacek            run every experiment\n"
+            "  pdacek 3          run experiment 3\n"
+            "  pdacek p3         same thing\n"
+            "  pdacek 3 --source print the listing instead of running it\n"
+            "  pdacek 3 --save   write the listing to the current directory\n"
+        ),
+    )
+    parser.add_argument(
+        "program",
+        nargs="?",
+        help="experiment number (1-5) or name such as p3",
+    )
+    parser.add_argument("--list", "-l", action="store_true", help="list experiments and exit")
+    parser.add_argument("--source", "-s", action="store_true", help="print the listing")
+    parser.add_argument(
+        "--save",
+        nargs="?",
+        const=".",
+        metavar="DIR",
+        help="write the listing to DIR (default: current directory)",
+    )
+    parser.add_argument("--version", "-V", action="version", version=f"pdacek {__version__}")
+    return parser
+
+
+def _parse_number(raw: Optional[str]) -> Optional[int]:
+    if raw is None:
+        return None
+    cleaned = raw.strip().lower().lstrip("p").replace("program", "").strip(" _-")
+    try:
+        return int(cleaned)
+    except ValueError:
+        return None
+
+
+def list_programs() -> None:
+    print(f"pdacek {__version__} - bundled experiments\n")
+    for number, title in programs().items():
+        print(f"  {number}.  {title}")
+    print("\nRun one with: pdacek <number>")
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    args = build_parser().parse_args(argv)
+
+    if args.list:
+        list_programs()
+        return 0
+
+    number = _parse_number(args.program)
+    if args.program is not None and number is None:
+        print(f"error: could not read a program number from {args.program!r}", file=sys.stderr)
+        print(f"available: {', '.join(str(n) for n in numbers())}", file=sys.stderr)
+        return 2
+
+    try:
+        if number is None:
+            for n in numbers():
+                run(n)
+                print()
+            return 0
+
+        if args.save is not None:
+            path = save(number, args.save)
+            print(f"wrote {path}")
+            return 0
+
+        if args.source:
+            print(source(number))
+            return 0
+
+        run(number)
+        return 0
+    except KeyError as exc:
+        print(f"error: {exc.args[0]}", file=sys.stderr)
+        return 2
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

pdacek-0.1.0/src/pdacek/compat.py

@@ -0,0 +1,129 @@
+"""Shared helpers: tokenisation and NLTK corpus access with safe fallbacks.
+
+The lab machines this toolkit targets are often offline, so every helper here
+degrades to a pure-``re`` implementation instead of raising when an NLTK
+resource (``punkt``, ``stopwords``) has not been downloaded yet.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import List
+
+try:  # NLTK is a declared dependency, but keep the package importable without it.
+    import nltk
+except ImportError:  # pragma: no cover - defensive
+    nltk = None  # type: ignore[assignment]
+
+__all__ = ["tokenize", "stopwords_en", "stemmer", "ensure_punkt", "nltk_available"]
+
+_TOKEN_RE = re.compile(r"[A-Za-z]+(?:'[A-Za-z]+)?|\d+|[^\sA-Za-z\d]")
+
+# Small built-in list, used only when the NLTK corpus is unavailable.
+_FALLBACK_STOPWORDS = frozenset(
+    """
+    a about above after again against all am an and any are as at be because been
+    before being below between both but by can did do does doing down during each
+    few for from further had has have having he her here hers herself him himself
+    his how i if in into is it its itself just me more most my myself no nor not
+    now of off on once only or other our ours ourselves out over own same she
+    should so some such than that the their theirs them themselves then there
+    these they this those through to too under until up very was we were what
+    when where which while who whom why will with you your yours yourself
+    yourselves
+    """.split()
+)
+
+
+def nltk_available() -> bool:
+    """Return ``True`` when NLTK can be imported."""
+    return nltk is not None
+
+
+def _resource_present(resource: str, package: str) -> bool:
+    if nltk is None:
+        return False
+    try:
+        nltk.data.find(f"{package}/{resource}")
+        return True
+    except LookupError:
+        return False
+
+
+def ensure_punkt() -> bool:
+    """Make sure a sentence/word tokenizer model exists; return availability.
+
+    NLTK 3.9 renamed the tokenizer data to ``punkt_tab``; both names are tried
+    and any download failure (offline machine) is swallowed on purpose.
+    """
+    if nltk is None:
+        return False
+    for resource in ("punkt", "punkt_tab"):
+        if _resource_present(resource, "tokenizers"):
+            return True
+        try:
+            nltk.download(resource, quiet=True)
+        except Exception:  # pragma: no cover - network/permission problems
+            pass
+        if _resource_present(resource, "tokenizers"):
+            return True
+    return False
+
+
+def tokenize(text: str) -> List[str]:
+    """Split ``text`` into word and punctuation tokens.
+
+    Uses NLTK's ``word_tokenize`` when the model is available, otherwise a
+    regex tokenizer that keeps punctuation as its own token, which is what the
+    filtration and script-validation steps in Program 1 need.
+    """
+    if ensure_punkt():
+        try:
+            return nltk.word_tokenize(text)  # type: ignore[union-attr]
+        except LookupError:  # pragma: no cover - stale resource index
+            pass
+    return _TOKEN_RE.findall(text)
+
+
+def stopwords_en() -> frozenset:
+    """Return the English stop-word set (NLTK corpus when available)."""
+    if nltk is not None:
+        try:
+            return frozenset(nltk.corpus.stopwords.words("english"))  # type: ignore[union-attr]
+        except LookupError:
+            try:
+                nltk.download("stopwords", quiet=True)  # type: ignore[union-attr]
+                return frozenset(nltk.corpus.stopwords.words("english"))  # type: ignore[union-attr]
+            except Exception:
+                pass
+    return _FALLBACK_STOPWORDS
+
+
+class _SimpleStemmer:
+    """Very small suffix stripper used when NLTK is not importable."""
+
+    _SUFFIXES = (
+        "ational", "ization", "iveness", "ingly", "edly", "ments",
+        "ing", "ies", "ied", "ed", "es", "s",
+    )
+
+    def stem(self, word: str) -> str:
+        low = word.lower()
+        for suffix in self._SUFFIXES:
+            if len(low) > len(suffix) + 2 and low.endswith(suffix):
+                if suffix == "ies":
+                    return low[: -len(suffix)] + "y"
+                return low[: -len(suffix)]
+        return low
+
+
+def stemmer():
+    """Return a Porter stemmer (NLTK) or a minimal fallback stemmer."""
+    if nltk is not None:
+        try:
+            from nltk.stem import PorterStemmer
+
+            return PorterStemmer()
+        except Exception:  # pragma: no cover
+            pass
+    return _SimpleStemmer()