mdfetch 0.4.0__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- mdfetch-0.5.0/.gitattributes +2 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions.yml +16 -16
- mdfetch-0.5.0/.specify/feature.json +3 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/memory/changelog.md +23 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/memory/plan.md +29 -22
- mdfetch-0.5.0/.vscode/settings.json +10 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/CLAUDE.md +6 -4
- {mdfetch-0.4.0 → mdfetch-0.5.0}/PKG-INFO +16 -2
- {mdfetch-0.4.0 → mdfetch-0.5.0}/README.md +13 -1
- {mdfetch-0.4.0 → mdfetch-0.5.0}/pyproject.toml +6 -1
- mdfetch-0.5.0/specs/008-mdfetch-cli/checklists/requirements.md +35 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/contracts/cli.md +32 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/data-model.md +9 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/plan.md +96 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/quickstart.md +13 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/research.md +15 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/spec.md +101 -0
- mdfetch-0.5.0/specs/008-mdfetch-cli/tasks.md +148 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/__init__.py +2 -1
- mdfetch-0.5.0/src/mdfetch/base.py +164 -0
- mdfetch-0.5.0/src/mdfetch/cli.py +66 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/devto.py +2 -23
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/dzone.py +4 -21
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/medium.py +1 -14
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/substack.py +2 -23
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/thenewstack.py +2 -24
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/router.py +5 -0
- mdfetch-0.5.0/tests/conftest.py +37 -0
- mdfetch-0.5.0/tests/integration/test_cli_integration.py +51 -0
- mdfetch-0.5.0/tests/unit/test_cli.py +43 -0
- mdfetch-0.5.0/tests/unit/test_fetch_errors.py +155 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_medium_extractor.py +4 -4
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_router.py +31 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_silent.py +6 -24
- {mdfetch-0.4.0 → mdfetch-0.5.0}/uv.lock +29 -1
- mdfetch-0.4.0/.specify/feature.json +0 -3
- mdfetch-0.4.0/src/mdfetch/base.py +0 -108
- mdfetch-0.4.0/tests/conftest.py +0 -0
- mdfetch-0.4.0/tests/unit/test_fetch_errors.py +0 -131
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-analyze/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-archive-run/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-checklist/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-clarify/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-constitution/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-git-commit/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-git-feature/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-git-initialize/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-git-remote/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-git-validate/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-implement/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-plan/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-reconcile-run/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-specify/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-tasks/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.claude/skills/speckit-taskstoissues/SKILL.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.analyze.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.archive.run.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.checklist.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.clarify.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.constitution.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.implement.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.plan.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.reconcile.run.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.specify.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.tasks.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gemini/commands/speckit.taskstoissues.toml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.github/copilot-instructions.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.github/workflows/ci.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.github/workflows/integration.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.github/workflows/publish.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.gitignore +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.python-version +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/.registry +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/archive/LICENSE +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/archive/README.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/archive/commands/archive.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/archive/extension.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/README.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/commands/speckit.git.commit.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/commands/speckit.git.feature.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/commands/speckit.git.initialize.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/commands/speckit.git.remote.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/commands/speckit.git.validate.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/config-template.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/extension.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/git-config.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/bash/auto-commit.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/bash/create-new-feature.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/bash/git-common.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/bash/initialize-repo.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/powershell/auto-commit.ps1 +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/powershell/create-new-feature.ps1 +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/powershell/git-common.ps1 +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/git/scripts/powershell/initialize-repo.ps1 +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/reconcile/LICENSE +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/reconcile/README.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/reconcile/commands/reconcile.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/extensions/reconcile/extension.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/init-options.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/integration.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/integrations/claude.manifest.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/integrations/gemini.manifest.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/integrations/speckit.manifest.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/memory/constitution.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/memory/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/scripts/bash/check-prerequisites.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/scripts/bash/common.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/scripts/bash/create-new-feature.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/scripts/bash/setup-plan.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/scripts/bash/setup-tasks.sh +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/templates/checklist-template.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/templates/constitution-template.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/templates/plan-template.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/templates/spec-template.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/templates/tasks-template.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/workflows/speckit/workflow.yml +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/.specify/workflows/workflow-registry.json +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/GEMINI.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/LICENSE +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/Makefile +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/contracts/api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/data-model.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/quickstart.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/001-mdfetch-medium-extractor/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/contracts/public-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/data-model.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/quickstart.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/002-devto-provider/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/contracts/extract-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/003-medium-freedium-fallback/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/004-remove-backoff/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/004-remove-backoff/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/004-remove-backoff/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/004-remove-backoff/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/004-remove-backoff/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/contracts/extractor-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/data-model.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/quickstart.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/005-substack-provider/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/contracts/public-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/data-model.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/quickstart.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/006-thenewstack-provider/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/checklists/requirements.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/contracts/public-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/data-model.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/plan.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/quickstart.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/research.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/spec.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/specs/007-dzone-provider/tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/exceptions.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/src/mdfetch/providers/__init__.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/__init__.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/__init__.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/conftest.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/architecting-the-asynchronous-agent.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/devto-integration-digest-december-2025.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/devto-integration-digest-july-2025.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/devto-integration-digest-march-2026.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/dzone-image-classification-pipeline-camel-djl.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/dzone-integration-patterns-fail-production.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/dzone-kiro-feature-to-requirements-design-tasks.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/from-drift-to-parity.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/integration-digest-december-2025.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/substack-api-trends-2025.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/substack-kafka-topic-types.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/thenewstack-api-mcp-agent.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/thenewstack-async-apis.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/thenewstack-developer-portal-api.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/thenewstack-json-schema-ai.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/snapshots/thenewstack-mcp-api-governance.md +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/test_devto_integration.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/test_dzone_integration.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/test_medium_integration.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/test_substack_integration.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/integration/test_thenewstack_integration.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/__init__.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_devto_extractor.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_dzone_extractor.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_substack_extractor.py +0 -0
- {mdfetch-0.4.0 → mdfetch-0.5.0}/tests/unit/test_thenewstack_extractor.py +0 -0
|
@@ -22,7 +22,7 @@ hooks:
|
|
|
22
22
|
- extension: git
|
|
23
23
|
command: speckit.git.commit
|
|
24
24
|
enabled: true
|
|
25
|
-
optional:
|
|
25
|
+
optional: false
|
|
26
26
|
prompt: Commit outstanding changes before clarification?
|
|
27
27
|
description: Auto-commit before spec clarification
|
|
28
28
|
condition: null
|
|
@@ -30,7 +30,7 @@ hooks:
|
|
|
30
30
|
- extension: git
|
|
31
31
|
command: speckit.git.commit
|
|
32
32
|
enabled: true
|
|
33
|
-
optional:
|
|
33
|
+
optional: false
|
|
34
34
|
prompt: Commit outstanding changes before planning?
|
|
35
35
|
description: Auto-commit before implementation planning
|
|
36
36
|
condition: null
|
|
@@ -38,7 +38,7 @@ hooks:
|
|
|
38
38
|
- extension: git
|
|
39
39
|
command: speckit.git.commit
|
|
40
40
|
enabled: true
|
|
41
|
-
optional:
|
|
41
|
+
optional: false
|
|
42
42
|
prompt: Commit outstanding changes before task generation?
|
|
43
43
|
description: Auto-commit before task generation
|
|
44
44
|
condition: null
|
|
@@ -46,7 +46,7 @@ hooks:
|
|
|
46
46
|
- extension: git
|
|
47
47
|
command: speckit.git.commit
|
|
48
48
|
enabled: true
|
|
49
|
-
optional:
|
|
49
|
+
optional: false
|
|
50
50
|
prompt: Commit outstanding changes before implementation?
|
|
51
51
|
description: Auto-commit before implementation
|
|
52
52
|
condition: null
|
|
@@ -54,7 +54,7 @@ hooks:
|
|
|
54
54
|
- extension: git
|
|
55
55
|
command: speckit.git.commit
|
|
56
56
|
enabled: true
|
|
57
|
-
optional:
|
|
57
|
+
optional: false
|
|
58
58
|
prompt: Commit outstanding changes before checklist?
|
|
59
59
|
description: Auto-commit before checklist generation
|
|
60
60
|
condition: null
|
|
@@ -62,7 +62,7 @@ hooks:
|
|
|
62
62
|
- extension: git
|
|
63
63
|
command: speckit.git.commit
|
|
64
64
|
enabled: true
|
|
65
|
-
optional:
|
|
65
|
+
optional: false
|
|
66
66
|
prompt: Commit outstanding changes before analysis?
|
|
67
67
|
description: Auto-commit before analysis
|
|
68
68
|
condition: null
|
|
@@ -70,7 +70,7 @@ hooks:
|
|
|
70
70
|
- extension: git
|
|
71
71
|
command: speckit.git.commit
|
|
72
72
|
enabled: true
|
|
73
|
-
optional:
|
|
73
|
+
optional: false
|
|
74
74
|
prompt: Commit outstanding changes before issue sync?
|
|
75
75
|
description: Auto-commit before tasks-to-issues conversion
|
|
76
76
|
condition: null
|
|
@@ -78,7 +78,7 @@ hooks:
|
|
|
78
78
|
- extension: git
|
|
79
79
|
command: speckit.git.commit
|
|
80
80
|
enabled: true
|
|
81
|
-
optional:
|
|
81
|
+
optional: false
|
|
82
82
|
prompt: Commit constitution changes?
|
|
83
83
|
description: Auto-commit after constitution update
|
|
84
84
|
condition: null
|
|
@@ -86,7 +86,7 @@ hooks:
|
|
|
86
86
|
- extension: git
|
|
87
87
|
command: speckit.git.commit
|
|
88
88
|
enabled: true
|
|
89
|
-
optional:
|
|
89
|
+
optional: false
|
|
90
90
|
prompt: Commit specification changes?
|
|
91
91
|
description: Auto-commit after specification
|
|
92
92
|
condition: null
|
|
@@ -94,7 +94,7 @@ hooks:
|
|
|
94
94
|
- extension: git
|
|
95
95
|
command: speckit.git.commit
|
|
96
96
|
enabled: true
|
|
97
|
-
optional:
|
|
97
|
+
optional: false
|
|
98
98
|
prompt: Commit clarification changes?
|
|
99
99
|
description: Auto-commit after spec clarification
|
|
100
100
|
condition: null
|
|
@@ -102,7 +102,7 @@ hooks:
|
|
|
102
102
|
- extension: git
|
|
103
103
|
command: speckit.git.commit
|
|
104
104
|
enabled: true
|
|
105
|
-
optional:
|
|
105
|
+
optional: false
|
|
106
106
|
prompt: Commit plan changes?
|
|
107
107
|
description: Auto-commit after implementation planning
|
|
108
108
|
condition: null
|
|
@@ -110,7 +110,7 @@ hooks:
|
|
|
110
110
|
- extension: git
|
|
111
111
|
command: speckit.git.commit
|
|
112
112
|
enabled: true
|
|
113
|
-
optional:
|
|
113
|
+
optional: false
|
|
114
114
|
prompt: Commit task changes?
|
|
115
115
|
description: Auto-commit after task generation
|
|
116
116
|
condition: null
|
|
@@ -118,7 +118,7 @@ hooks:
|
|
|
118
118
|
- extension: git
|
|
119
119
|
command: speckit.git.commit
|
|
120
120
|
enabled: true
|
|
121
|
-
optional:
|
|
121
|
+
optional: false
|
|
122
122
|
prompt: Commit implementation changes?
|
|
123
123
|
description: Auto-commit after implementation
|
|
124
124
|
condition: null
|
|
@@ -126,7 +126,7 @@ hooks:
|
|
|
126
126
|
- extension: git
|
|
127
127
|
command: speckit.git.commit
|
|
128
128
|
enabled: true
|
|
129
|
-
optional:
|
|
129
|
+
optional: false
|
|
130
130
|
prompt: Commit checklist changes?
|
|
131
131
|
description: Auto-commit after checklist generation
|
|
132
132
|
condition: null
|
|
@@ -134,7 +134,7 @@ hooks:
|
|
|
134
134
|
- extension: git
|
|
135
135
|
command: speckit.git.commit
|
|
136
136
|
enabled: true
|
|
137
|
-
optional:
|
|
137
|
+
optional: false
|
|
138
138
|
prompt: Commit analysis results?
|
|
139
139
|
description: Auto-commit after analysis
|
|
140
140
|
condition: null
|
|
@@ -142,7 +142,7 @@ hooks:
|
|
|
142
142
|
- extension: git
|
|
143
143
|
command: speckit.git.commit
|
|
144
144
|
enabled: true
|
|
145
|
-
optional:
|
|
145
|
+
optional: false
|
|
146
146
|
prompt: Commit after syncing issues?
|
|
147
147
|
description: Auto-commit after tasks-to-issues conversion
|
|
148
148
|
condition: null
|
|
@@ -1,3 +1,26 @@
|
|
|
1
|
+
### mdfetch — Architecture Refactoring & CLI Improvements — 2026-05-16
|
|
2
|
+
|
|
3
|
+
**Branch**: `refactor/reduce-duplication-and-robustness`
|
|
4
|
+
|
|
5
|
+
**What was added**:
|
|
6
|
+
- Centralized duplicated markdown conversion and iframe stripping into `BaseExtractor`, utilizing a `_markdownify_kwargs()` hook.
|
|
7
|
+
- Added strict `Content-Type` header validation in `_do_fetch` to eagerly reject non-HTML responses (e.g., `application/json`, `application/pdf`).
|
|
8
|
+
- Implemented `httpx.Client` connection pooling across retry attempts.
|
|
9
|
+
- Exported new public API `mdfetch.supported_domains()` which returns a `frozenset[str]` of all registered domains.
|
|
10
|
+
- Expanded the CLI (`mdfetch.cli:main`) with configurable options: `--retries`, `--retry-delay` (with click `IntRange`/`FloatRange` validation), and `--version`.
|
|
11
|
+
- Removed control-flow assertions in favor of explicit `RuntimeError` guards.
|
|
12
|
+
- Consolidated test helpers (`make_stream_mock`) into `tests/conftest.py`.
|
|
13
|
+
- Bumped package version to `0.5.0`.
|
|
14
|
+
|
|
15
|
+
**New Components**:
|
|
16
|
+
- Updates across `base.py`, `router.py`, `cli.py`, and `__init__.py`.
|
|
17
|
+
- Refactored providers (`devto.py`, `substack.py`, `thenewstack.py`, `dzone.py`) to leverage `BaseExtractor` defaults.
|
|
18
|
+
- Additional unit and integration test coverage for CLI flags and content types.
|
|
19
|
+
|
|
20
|
+
**Tasks Completed**: Refactoring and PR Review updates
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
1
24
|
# Merged Features Log
|
|
2
25
|
|
|
3
26
|
---
|
|
@@ -44,7 +44,9 @@ BaseExtractor (ABC) — src/mdfetch/base.py
|
|
|
44
44
|
│ fixed delay of retry_delay seconds between attempts (not exponential);
|
|
45
45
|
│ codes in _no_retry_codes (or class attribute) raise immediately
|
|
46
46
|
├── clean_html(soup) → Tag — abstract: platform-specific HTML isolation
|
|
47
|
-
├──
|
|
47
|
+
├── _markdownify_kwargs() → dict — hook: platform-specific markdownify options (defaults to ATX + empty language)
|
|
48
|
+
├── _replace_iframes_with_links(container, soup) — helper: converts <iframe> embeds to plain anchors
|
|
49
|
+
├── convert_to_markdown(tag) → str— concrete: base implementation using markdownify and 3+ newlines collapse
|
|
48
50
|
└── extract(url) → str — concrete template method (orchestrates the above)
|
|
49
51
|
|
|
50
52
|
MediumExtractor(BaseExtractor) — src/mdfetch/providers/medium.py
|
|
@@ -59,33 +61,35 @@ MediumExtractor(BaseExtractor) — src/mdfetch/providers/medium.py
|
|
|
59
61
|
|
|
60
62
|
DevToExtractor(BaseExtractor) — src/mdfetch/providers/devto.py
|
|
61
63
|
├── DOMAINS = frozenset({"dev.to"})
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
└── convert_to_markdown() → markdownify with ATX headings; collapses 3+ newlines to 2
|
|
64
|
+
└── clean_html() → locates div#article-body; raises UnsupportedContentTypeError if absent;
|
|
65
|
+
replaces iframes and ltag embeds with anchor links (via helper); strips empty anchor-name
|
|
66
|
+
elements; prepends h1 + cover image from crayons-article__header
|
|
66
67
|
|
|
67
68
|
SubstackExtractor(BaseExtractor) — src/mdfetch/providers/substack.py [005-substack-provider]
|
|
68
69
|
├── DOMAINS = frozenset({"substack.com"}) — also matches *.substack.com via suffix routing
|
|
69
70
|
├── _no_retry_status_codes = frozenset() — HTTP 429 retried (no Freedium-style fallback)
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
└── convert_to_markdown() → markdownify with ATX headings; collapses 3+ newlines to 2; raises EmptyContentError if empty
|
|
71
|
+
└── clean_html() → locates div.body.markup; raises UnsupportedContentTypeError if absent;
|
|
72
|
+
strips div.subscription-widget-wrap (inline CTAs + paywall terminal);
|
|
73
|
+
replaces iframes with anchor links (via helper);
|
|
74
|
+
replaces div[data-component-name] (except SubscribeWidget, Image2ToDOM) with anchor links;
|
|
75
|
+
prepends h3.subtitle from div.post-header (if present);
|
|
76
|
+
prepends h1.post-title from div.post-header (unconditional — structurally outside body)
|
|
77
77
|
|
|
78
78
|
TheNewStackExtractor(BaseExtractor) — src/mdfetch/providers/thenewstack.py [006-thenewstack-provider]
|
|
79
79
|
├── DOMAINS = frozenset({"thenewstack.io"}) — no subdomain routing needed (single-site WordPress)
|
|
80
80
|
├── _no_retry_status_codes = frozenset() — inherits base class default (no overrides needed)
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
81
|
+
└── clean_html() → locates div#tns-post-body-content; raises UnsupportedContentTypeError if absent;
|
|
82
|
+
decomposes 4 sponsored-content selectors: div.sponsored-post-disclosure,
|
|
83
|
+
div.tns-sponsored-post-disclosure, div.sponsor-disclosure, div.tns-sponsor-note;
|
|
84
|
+
replaces iframes with anchor links (via helper);
|
|
85
|
+
prepends div.post-excerpt text as new <p> tag (deck) from div#tns-post-headline (if present);
|
|
86
|
+
prepends copy.copy(h1.title) from div#tns-post-headline (if present)
|
|
87
|
+
Note: VoxPop polls (div.tns-voxpop-screen) are page-level modals outside body — no stripping needed
|
|
88
|
+
|
|
89
|
+
DZoneExtractor(BaseExtractor) — src/mdfetch/providers/dzone.py
|
|
90
|
+
├── DOMAINS = frozenset({"dzone.com"})
|
|
91
|
+
├── _markdownify_kwargs() → overrides to add code_language_callback
|
|
92
|
+
└── clean_html() → locates div.content-html; raises UnsupportedContentTypeError if absent;
|
|
89
93
|
```
|
|
90
94
|
|
|
91
95
|
### Router / Auto-Discovery
|
|
@@ -95,6 +99,7 @@ router.py
|
|
|
95
99
|
├── _REGISTRY: dict[str, type[BaseExtractor]]
|
|
96
100
|
├── @register decorator — self-enrolls provider at class definition time
|
|
97
101
|
├── _autodiscover_providers() — pkgutil.iter_modules scans mdfetch/providers/ at import
|
|
102
|
+
├── supported_domains() → frozenset[str] — returns all registered domain strings
|
|
98
103
|
├── route(url) → BaseExtractor — exact hostname match, then longest-suffix fallback
|
|
99
104
|
└── Duplicate domain registration raises ValueError
|
|
100
105
|
```
|
|
@@ -118,13 +123,15 @@ src/
|
|
|
118
123
|
├── __init__.py # Public surface: exposes extract(), exception re-exports
|
|
119
124
|
├── exceptions.py # MdfetchError hierarchy (6 exception classes)
|
|
120
125
|
├── router.py # @register, _autodiscover_providers(), route()
|
|
121
|
-
├── base.py # BaseExtractor ABC + fetch_html() + extract() template
|
|
126
|
+
├── base.py # BaseExtractor ABC + fetch_html() + convert_to_markdown() + extract() template
|
|
127
|
+
├── cli.py # click-based command line interface
|
|
122
128
|
└── providers/
|
|
123
129
|
├── __init__.py # Empty — auto-discovery handles registration
|
|
124
130
|
├── medium.py # MediumExtractor
|
|
125
131
|
├── devto.py # DevToExtractor [002-devto-provider]
|
|
126
132
|
├── substack.py # SubstackExtractor [005-substack-provider]
|
|
127
|
-
|
|
133
|
+
├── thenewstack.py # TheNewStackExtractor [006-thenewstack-provider]
|
|
134
|
+
└── dzone.py # DZoneExtractor
|
|
128
135
|
|
|
129
136
|
tests/
|
|
130
137
|
├── unit/
|
|
@@ -7,6 +7,7 @@ Python library that extracts article content from web platforms and returns it a
|
|
|
7
7
|
```
|
|
8
8
|
src/mdfetch/
|
|
9
9
|
├── __init__.py # extract() public API + exception re-exports
|
|
10
|
+
├── cli.py # Command-line interface and argument parsing
|
|
10
11
|
├── exceptions.py # MdfetchError hierarchy (7 exception classes)
|
|
11
12
|
├── base.py # BaseExtractor ABC
|
|
12
13
|
├── router.py # Domain-to-provider routing
|
|
@@ -15,7 +16,8 @@ src/mdfetch/
|
|
|
15
16
|
├── medium.py # MediumExtractor (medium.com + *.medium.com)
|
|
16
17
|
├── devto.py # DevToExtractor (dev.to)
|
|
17
18
|
├── substack.py # SubstackExtractor (substack.com + *.substack.com)
|
|
18
|
-
|
|
19
|
+
├── thenewstack.py # TheNewStackExtractor (thenewstack.io)
|
|
20
|
+
└── dzone.py # DZoneExtractor (dzone.com)
|
|
19
21
|
|
|
20
22
|
tests/
|
|
21
23
|
├── unit/ # pytest unit tests (no network)
|
|
@@ -34,7 +36,7 @@ Makefile # setup / test / lint / format / build / upgrade-deps / cl
|
|
|
34
36
|
## Key conventions
|
|
35
37
|
|
|
36
38
|
- **Package manager**: `uv` for all dev workflows — never use `pip`, `venv`, or `pip-tools` directly
|
|
37
|
-
- **Provider pattern**: new platforms = one new file under `src/mdfetch/providers
|
|
39
|
+
- **Provider pattern**: new platforms = one new file under `src/mdfetch/providers/` subclassing `BaseExtractor` and utilizing its shared conversion methods.
|
|
38
40
|
- **No logging**: all failures communicated via typed exceptions only (FR-013)
|
|
39
41
|
- **Type hints**: strict (`mypy src/` must pass with zero errors)
|
|
40
42
|
- **Linter/formatter**: `ruff` (`make lint` / `make format`)
|
|
@@ -49,13 +51,13 @@ make format # ruff format
|
|
|
49
51
|
make build # uv build (wheel + sdist)
|
|
50
52
|
make upgrade-deps # uv sync --all-extras --upgrade
|
|
51
53
|
make integration # integration tests (network required)
|
|
52
|
-
|
|
54
|
+
make typecheck # type check
|
|
53
55
|
```
|
|
54
56
|
|
|
55
57
|
<!-- SPECKIT START -->
|
|
56
58
|
## Active Feature Plan
|
|
57
59
|
|
|
58
|
-
Active feature: `
|
|
60
|
+
Active feature: `008-mdfetch-cli`. Implementation plan: [`specs/008-mdfetch-cli/plan.md`](specs/008-mdfetch-cli/plan.md).
|
|
59
61
|
|
|
60
62
|
## Known Issues & Gotchas
|
|
61
63
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: mdfetch
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.5.0
|
|
4
4
|
Summary: Extract article content from web platforms and return it as clean Markdown.
|
|
5
5
|
Project-URL: Homepage, https://github.com/stn1slv/md-fetch
|
|
6
6
|
Project-URL: Source, https://github.com/stn1slv/md-fetch
|
|
@@ -20,11 +20,13 @@ Classifier: Topic :: Internet :: WWW/HTTP
|
|
|
20
20
|
Classifier: Topic :: Text Processing :: Markup :: Markdown
|
|
21
21
|
Requires-Python: >=3.12
|
|
22
22
|
Requires-Dist: beautifulsoup4>=4.12
|
|
23
|
+
Requires-Dist: click>=8.1
|
|
23
24
|
Requires-Dist: httpx>=0.27
|
|
24
25
|
Requires-Dist: lxml>=5.0
|
|
25
26
|
Requires-Dist: markdownify>=0.13
|
|
26
27
|
Provides-Extra: dev
|
|
27
28
|
Requires-Dist: mypy>=1.9; extra == 'dev'
|
|
29
|
+
Requires-Dist: pytest-mock>=3.14.0; extra == 'dev'
|
|
28
30
|
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
29
31
|
Requires-Dist: ruff>=0.4; extra == 'dev'
|
|
30
32
|
Description-Content-Type: text/markdown
|
|
@@ -39,7 +41,19 @@ A Python library that extracts article content from web platforms and returns it
|
|
|
39
41
|
pip install mdfetch
|
|
40
42
|
```
|
|
41
43
|
|
|
42
|
-
## Usage
|
|
44
|
+
## CLI Usage
|
|
45
|
+
|
|
46
|
+
You can use the built-in `md-fetch` command directly from your terminal:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
# Fetch and print Markdown to standard output
|
|
50
|
+
md-fetch https://medium.com/example/article
|
|
51
|
+
|
|
52
|
+
# Fetch and save Markdown to a file
|
|
53
|
+
md-fetch https://dev.to/example/article --output article.md
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Python Usage
|
|
43
57
|
|
|
44
58
|
```python
|
|
45
59
|
from mdfetch import extract
|
|
@@ -8,7 +8,19 @@ A Python library that extracts article content from web platforms and returns it
|
|
|
8
8
|
pip install mdfetch
|
|
9
9
|
```
|
|
10
10
|
|
|
11
|
-
## Usage
|
|
11
|
+
## CLI Usage
|
|
12
|
+
|
|
13
|
+
You can use the built-in `md-fetch` command directly from your terminal:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
# Fetch and print Markdown to standard output
|
|
17
|
+
md-fetch https://medium.com/example/article
|
|
18
|
+
|
|
19
|
+
# Fetch and save Markdown to a file
|
|
20
|
+
md-fetch https://dev.to/example/article --output article.md
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Python Usage
|
|
12
24
|
|
|
13
25
|
```python
|
|
14
26
|
from mdfetch import extract
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "mdfetch"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.5.0"
|
|
8
8
|
description = "Extract article content from web platforms and return it as clean Markdown."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = { text = "MIT" }
|
|
@@ -27,11 +27,16 @@ dependencies = [
|
|
|
27
27
|
"beautifulsoup4>=4.12",
|
|
28
28
|
"lxml>=5.0",
|
|
29
29
|
"markdownify>=0.13",
|
|
30
|
+
"click>=8.1",
|
|
30
31
|
]
|
|
31
32
|
|
|
33
|
+
[project.scripts]
|
|
34
|
+
md-fetch = "mdfetch.cli:main"
|
|
35
|
+
|
|
32
36
|
[project.optional-dependencies]
|
|
33
37
|
dev = [
|
|
34
38
|
"pytest>=8.0",
|
|
39
|
+
"pytest-mock>=3.14.0",
|
|
35
40
|
"ruff>=0.4",
|
|
36
41
|
"mypy>=1.9",
|
|
37
42
|
]
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Specification Quality Checklist: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
|
4
|
+
**Created**: Saturday, May 16, 2026
|
|
5
|
+
**Feature**: [spec.md](../spec.md)
|
|
6
|
+
|
|
7
|
+
## Content Quality
|
|
8
|
+
|
|
9
|
+
- [x] No implementation details (languages, frameworks, APIs)
|
|
10
|
+
- [x] Focused on user value and business needs
|
|
11
|
+
- [x] Written for non-technical stakeholders
|
|
12
|
+
- [x] All mandatory sections completed
|
|
13
|
+
|
|
14
|
+
## Requirement Completeness
|
|
15
|
+
|
|
16
|
+
- [x] No [NEEDS CLARIFICATION] markers remain
|
|
17
|
+
- [x] Requirements are testable and unambiguous
|
|
18
|
+
- [x] Success criteria are measurable
|
|
19
|
+
- [x] Success criteria are technology-agnostic (no implementation details)
|
|
20
|
+
- [x] All acceptance scenarios are defined
|
|
21
|
+
- [x] Edge cases are identified
|
|
22
|
+
- [x] Scope is clearly bounded
|
|
23
|
+
- [x] Dependencies and assumptions identified
|
|
24
|
+
|
|
25
|
+
## Feature Readiness
|
|
26
|
+
|
|
27
|
+
- [x] All functional requirements have clear acceptance criteria
|
|
28
|
+
- [x] User scenarios cover primary flows
|
|
29
|
+
- [x] Feature meets measurable outcomes defined in Success Criteria
|
|
30
|
+
- [x] No implementation details leak into specification
|
|
31
|
+
|
|
32
|
+
## Notes
|
|
33
|
+
|
|
34
|
+
- All checklist items pass. The specification is complete and valid.
|
|
35
|
+
- The feature is straightforward and maps clearly to standard CLI conventions.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# CLI Contract: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
## Command Signature
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
md-fetch [OPTIONS] URL
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
### Positional Arguments
|
|
10
|
+
- `URL`: The web article URL to extract.
|
|
11
|
+
|
|
12
|
+
### Options
|
|
13
|
+
- `-o, --output PATH`: Write the Markdown output to the specified file path instead of standard output.
|
|
14
|
+
- `--retries INTEGER`: Total number of fetch attempts on transient errors (default: 3, minimum: 1).
|
|
15
|
+
- `--retry-delay FLOAT`: Seconds to wait between retry attempts (default: 2.0, minimum: 0).
|
|
16
|
+
- `--version`: Show the installed version and exit.
|
|
17
|
+
- `--help`: Show the usage instructions and exit.
|
|
18
|
+
|
|
19
|
+
## Exit Codes
|
|
20
|
+
- `0`: Success. The Markdown was printed to stdout or successfully written to the output file.
|
|
21
|
+
- `1` (or other non-zero): Failure. A network error occurred, the domain is unsupported, or no extractable content was found. An error message is printed to `stderr`.
|
|
22
|
+
|
|
23
|
+
## Unsupported Domain Error Format
|
|
24
|
+
|
|
25
|
+
When the URL's domain has no registered provider, the error is printed to `stderr` in this format:
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
Error: '<domain>' is not a supported platform.
|
|
29
|
+
Supported domains: dev.to, dzone.com, medium.com, substack.com, thenewstack.io
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
The supported-domain list is sorted alphabetically and reflects all registered providers at runtime.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# Data Model: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
No new database or internal domain entities are introduced.
|
|
4
|
+
|
|
5
|
+
## CLI Execution State
|
|
6
|
+
|
|
7
|
+
- **Input URL** (`str`): The target URL to fetch.
|
|
8
|
+
- **Output File** (`str | None`): An optional file path provided via `--output`.
|
|
9
|
+
- **Exit Code** (`int`): `0` for success, non-zero (`1`) for failure.
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Implementation Plan: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
**Branch**: `008-mdfetch-cli` | **Date**: 2026-05-16 | **Spec**: [spec.md](./spec.md)
|
|
4
|
+
|
|
5
|
+
**Input**: Feature specification from `/specs/008-mdfetch-cli/spec.md`
|
|
6
|
+
|
|
7
|
+
## Summary
|
|
8
|
+
|
|
9
|
+
Add a CLI entry point named `md-fetch` using the `click` framework. This CLI will accept a single URL, fetch the Markdown utilizing the existing library, and print it to standard output or save it to a specified file while providing graceful error handling.
|
|
10
|
+
|
|
11
|
+
## Technical Context
|
|
12
|
+
|
|
13
|
+
**Language/Version**: Python 3.12+
|
|
14
|
+
|
|
15
|
+
**Primary Dependencies**: `httpx`, `BeautifulSoup`, `markdownify`, `pytest`, `click` (new dependency for CLI)
|
|
16
|
+
|
|
17
|
+
**Target Platform**: PyPI library (cross-platform CLI tool)
|
|
18
|
+
|
|
19
|
+
**Project Type**: Library + CLI
|
|
20
|
+
|
|
21
|
+
**Performance Goals**: Inherits base class 30-second fetch timeout.
|
|
22
|
+
|
|
23
|
+
**Constraints**: Must use `click` framework. Handle `MdfetchError` subclasses explicitly without tracebacks.
|
|
24
|
+
|
|
25
|
+
## Constitution Check
|
|
26
|
+
|
|
27
|
+
*GATE: Must pass before implementation. Re-check after design phase.*
|
|
28
|
+
|
|
29
|
+
- [x] Validates Provider Pattern Architecture (N/A - does not create a new provider, but wraps existing logic cleanly)
|
|
30
|
+
- [x] Confirms Technology Stack (Using `click` as per application blueprint for CLI frameworks)
|
|
31
|
+
- [x] Adheres to Coding Standards (PEP 8, Type Hinting, Clear Vocabulary)
|
|
32
|
+
- [x] Incorporates Integration Testing (Requires CLI invocation tests, though it's mainly wrapping tested logic)
|
|
33
|
+
- [x] Respects Packaging and Distribution standards (`pyproject.toml` integration via `[project.scripts]`)
|
|
34
|
+
|
|
35
|
+
## Project Structure
|
|
36
|
+
|
|
37
|
+
### Documentation (this feature)
|
|
38
|
+
|
|
39
|
+
```text
|
|
40
|
+
specs/008-mdfetch-cli/
|
|
41
|
+
├── plan.md # This file
|
|
42
|
+
├── research.md # Phase 0 output
|
|
43
|
+
├── data-model.md # Phase 1 output
|
|
44
|
+
├── quickstart.md # Phase 1 output
|
|
45
|
+
├── contracts/ # Phase 1 output
|
|
46
|
+
└── tasks.md # Phase 2 output (future)
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
### Source Code
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
src/mdfetch/
|
|
53
|
+
└── cli.py # NEW: click command implementation
|
|
54
|
+
|
|
55
|
+
tests/integration/
|
|
56
|
+
└── test_cli_integration.py # NEW: testing the CLI entry point
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
**Non-runtime changes**: Update `pyproject.toml` to add `click` dependency and `project.scripts` configuration. Update `README.md` to document CLI usage.
|
|
60
|
+
|
|
61
|
+
## Extraction Algorithm (CLI Wrapper)
|
|
62
|
+
|
|
63
|
+
```python
|
|
64
|
+
import click
|
|
65
|
+
from mdfetch import extract
|
|
66
|
+
from mdfetch.exceptions import MdfetchError
|
|
67
|
+
|
|
68
|
+
@click.command()
|
|
69
|
+
@click.argument("url")
|
|
70
|
+
@click.option("-o", "--output", type=click.Path(writable=True, dir_okay=False), help="Save Markdown to this file")
|
|
71
|
+
def main(url, output):
|
|
72
|
+
try:
|
|
73
|
+
content = extract(url)
|
|
74
|
+
if output:
|
|
75
|
+
with open(output, "w", encoding="utf-8") as f:
|
|
76
|
+
f.write(content)
|
|
77
|
+
else:
|
|
78
|
+
click.echo(content)
|
|
79
|
+
except MdfetchError as e:
|
|
80
|
+
click.secho(str(e), err=True, fg="red")
|
|
81
|
+
raise click.Abort()
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Error Mapping
|
|
85
|
+
|
|
86
|
+
| Condition | Exception | CLI Behavior |
|
|
87
|
+
|-----------|-----------|--------------|
|
|
88
|
+
| Any `MdfetchError` | `MdfetchError` subclasses | Catch, print `str(e)` to stderr, exit code `1` |
|
|
89
|
+
| File write error | `IOError` / `OSError` | Propagated or caught by Click Path validation |
|
|
90
|
+
| Invalid URL option | N/A | Click usage error, exits with `2` |
|
|
91
|
+
|
|
92
|
+
## Complexity Tracking
|
|
93
|
+
|
|
94
|
+
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
|
95
|
+
|-----------|------------|--------------------------------------|
|
|
96
|
+
| None | N/A | N/A |
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Quickstart: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
Once installed or running via `uv run`, you can use the `md-fetch` command directly from your terminal.
|
|
4
|
+
|
|
5
|
+
## Print to stdout
|
|
6
|
+
```bash
|
|
7
|
+
md-fetch https://medium.com/example/article
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
## Save to a file
|
|
11
|
+
```bash
|
|
12
|
+
md-fetch https://dev.to/example/article --output my_article.md
|
|
13
|
+
```
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Research: mdfetch-cli
|
|
2
|
+
|
|
3
|
+
## CLI Framework Decision
|
|
4
|
+
|
|
5
|
+
- **Decision**: Use `click`.
|
|
6
|
+
- **Rationale**: The project's Python Application Blueprint mandates `click` or `typer`. `click` is a lightweight, widely adopted standard for CLI tools in Python. It handles argument parsing, standard help interfaces, and exit codes elegantly.
|
|
7
|
+
- **Alternatives considered**:
|
|
8
|
+
- `typer`: Adds type-hint-based parsing but introduces another dependency layer (Pydantic). `click` is simpler for a single-command tool.
|
|
9
|
+
- `argparse`: Built-in, but the project blueprint specifically prefers `click` or `typer` for consistency across tools.
|
|
10
|
+
|
|
11
|
+
## Error Handling
|
|
12
|
+
|
|
13
|
+
- **Decision**: Catch `mdfetch.exceptions.MdfetchError` (and its subclasses) in the CLI entry point, print `str(err)` to `sys.stderr` via `click.secho(err='red')`, and call `sys.exit(1)`.
|
|
14
|
+
- **Rationale**: `mdfetch` already encapsulates all expected failures in the `MdfetchError` hierarchy. We avoid exposing tracebacks to end users unless an unexpected bug occurs.
|
|
15
|
+
- **Alternatives considered**: Silently returning empty output or logging the error. However, CLI tools should clearly indicate failure via non-zero exit codes and standard error.
|