pyspamcop 0.1.0__tar.gz

pyspamcop-0.1.0/.claude/settings.local.json

@@ -0,0 +1,7 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(grep -E \"\\\\.\\(html|yaml\\)$\")"
+    ]
+  }
+}

pyspamcop-0.1.0/.gitignore

@@ -0,0 +1,173 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# PyPI configuration file
+.pypirc
+*.pm
+

pyspamcop-0.1.0/.python-version

@@ -0,0 +1 @@
+3.12

pyspamcop-0.1.0/CLAUDE.md

@@ -0,0 +1,73 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project overview
+
+pyspamcop is a Python web crawler that automates finishing [SpamCop.net](https://www.spamcop.net) spam reports. It logs into the SpamCop website, fetches pending reports, parses the HTML, and submits confirmations — sequentially, with forced delays to be polite.
+
+The project is a rewrite of the Perl-based [App-SpamcupNG](https://github.com/glasswalk3r/App-SpamcupNG). The original Perl code is kept under `legacy/` for reference.
+
+## Environment setup
+
+Uses `uv` for dependency management and Python 3.12.
+
+```bash
+make init      # create venv and install all deps (uv venv && uv sync)
+```
+
+## Commands
+
+```bash
+make lint           # ruff check --fix && ruff format && mypy
+make unit           # pytest (excluding integration tests)
+make integration    # pytest -m integration
+make coverage       # pytest --cov=pyspamcop tests/
+make clean          # remove build/test/cache artifacts
+```
+
+Run a single test file:
+```bash
+python -m pytest tests/test_html_parser.py -v
+```
+
+Run a single test by name:
+```bash
+python -m pytest tests/test_html_parser.py -k "test_find_errors" -v
+```
+
+## Architecture
+
+```
+src/pyspamcop/
+  main.py          # CLI entry point (argparse); entrypoint is pyspamcop:main; log_config() sets up logging
+  runner.py        # main_loop()/run_account(): the login -> analyse -> confirm -> record cycle
+  db.py            # Recorder: SQLite persistence of Summary objects (see Key design patterns)
+  config.py        # YAML config loading; Configuration + EmailAccount dataclasses
+  domain.py        # All domain models: Message hierarchy, Receiver, EmailHeader, MessageAge, Summary
+  html.py          # BeautifulSoup HTML parsers: find_errors, find_warnings, find_receivers,
+                   # find_next_id, find_header, find_best_contacts, report_form
+  exception.py     # BaseExceptionError and UnknownReceiverFormat
+  http/client.py   # HTTPClient: concrete httpx-based implementation of ClientBase
+  spamcop/client.py # ClientBase: abstract base class for the SpamCop HTTP session
+```
+
+### Key design patterns
+
+**Message hierarchy** (`domain.py`): `Message` is an abstract base class with two branches — `UnrecoverableSpamReportMessage` (skip this report, move on) and `WarningMessage` (report can still complete). Concrete types include `MailHostMessage`, `EmailAddressBounceMessage`, `SpamHeaderMessage`, `MailhostForgeryMessage`, and `FreshSpamMessage`. Each implements `is_related(text)`, `extract(tag)`, and `complete_message()`.
+
+**HTML parsing** (`html.py`): All SpamCop page parsing lives here as standalone functions taking a `BeautifulSoup` object. Tests use HTML fixtures from `tests/fixtures/` rather than live HTTP calls. `find_best_contacts` identifies reporting candidates on the analysis preview page; `find_receivers` parses the post-submission confirmation page.
+
+**Client abstraction** (`spamcop/client.py` + `http/client.py`): `ClientBase` defines the abstract interface (`login`, `is_authenticated`, `spam_report`, `confirm_report`, `last_response`). `HTTPClient` extends it using `httpx`. Several methods on `HTTPClient` are still stubs (`spam_report`, `confirm_report`, `last_response`).
+
+**Lookup-table registry** (`db.py`): `Recorder` persists a `Summary` to SQLite, normalizing repeated string values (email content type, spam age unit, charset, mailer, receiver address) into small reference tables via a get-or-create pattern (`_upsert_lookup`, using `INSERT OR IGNORE` + `SELECT` against each table's `UNIQUE` column). The table/column pairs are declared exactly once, in the `_LOOKUP_TABLES` tuple of `LookupTable(name, column)` — both the `CREATE TABLE` DDL (`_SCHEMA`) and the runtime column lookup (`_LOOKUP_COLUMN`) are derived from it, so schema and code can't drift apart. Add a new lookup table by adding one `LookupTable` entry, not by editing DDL and code separately.
+
+### Configuration file
+
+Default path: `~/.pyspamcop.yaml` (override with `--config`). Schema documented in `README.md`. Parsed by `read_config()` in `config.py`.
+
+### Test conventions
+
+- Unit tests use HTML fixtures in `tests/fixtures/` loaded via `read_fixture()` helper.
+- Integration tests are marked `@pytest.mark.integration` and excluded from `make unit`.
+- Tests run from the project root; fixture paths are relative (`tests/fixtures/...`).