httpcheck 1.4.2__tar.gz

httpcheck-1.4.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Thomas Juul Dyhr
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

httpcheck-1.4.2/PKG-INFO

@@ -0,0 +1,469 @@
+Metadata-Version: 2.4
+Name: httpcheck
+Version: 1.4.2
+Summary: HTTP status code checker with advanced features
+Author-email: Thomas Juul Dyhr <thomas@dyhr.com>
+License: MIT
+Project-URL: Homepage, https://github.com/docdyhr/httpcheck
+Project-URL: Bug Tracker, https://github.com/docdyhr/httpcheck/issues
+Project-URL: Documentation, https://github.com/docdyhr/httpcheck/blob/master/README.md
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Monitoring
+Classifier: Topic :: System :: Networking :: Monitoring
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.32.5
+Requires-Dist: tabulate>=0.9.0
+Requires-Dist: tqdm>=4.67.0
+Requires-Dist: validators>=0.22.0
+Provides-Extra: macos
+Requires-Dist: rumps>=0.4.0; extra == "macos"
+Requires-Dist: pyobjc-framework-Cocoa>=10.0; extra == "macos"
+Requires-Dist: pyobjc-framework-Foundation>=10.0; extra == "macos"
+Requires-Dist: pyobjc-framework-AppKit>=10.0; extra == "macos"
+Requires-Dist: pyobjc-core>=10.0; extra == "macos"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: pytest-benchmark>=5.0; extra == "dev"
+Requires-Dist: pylint>=3.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Provides-Extra: build
+Requires-Dist: py2app>=0.28; extra == "build"
+Dynamic: license-file
+
+[![CI/CD Pipeline](https://github.com/docdyhr/httpcheck/actions/workflows/ci.yml/badge.svg)](https://github.com/docdyhr/httpcheck/actions/workflows/ci.yml)
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)
+![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)
+[![GitHub issues](https://img.shields.io/github/issues/docdyhr/httpcheck)](https://github.com/docdyhr/httpcheck/issues)
+![GitHub repo size](https://img.shields.io/github/repo-size/docdyhr/httpcheck)
+![GitHub](https://img.shields.io/github/license/docdyhr/httpcheck)
+
+# Check Websites HTTP Status Codes with httpcheck
+
+<img src="images/onSiteLogo.png" alt="onSite Logo" width="50%">
+
+* **Name**: httpcheck (CLI) / onSite (Menu Bar App)
+* **Current Version**: 1.4.1 (Security & Architecture Patch)
+* **Target Version**: 1.5.0 (Performance & Configuration Features)
+* **Programming Language**: Python 3.9+
+* **Author**: Thomas Juul Dyhr
+* **Purpose**: Advanced HTTP status checker with monitoring capabilities
+* **Development Status**: Active - See [ROADMAP.md](ROADMAP.md) for development plan
+
+## 🚀 Release Status
+
+**httpcheck v1.4.1 has been released! 🔒**
+
+✅ **NEW in v1.4.1:**
+- **Security Patches**: Updated dependencies (requests 2.32.5, urllib3 2.5.0)
+- **Improved Entry Point**: Refactored CLI to proper package function
+- **Mypy Support**: Added type checking configuration
+- **Pytest Fix**: Resolved asyncio deprecation warning
+
+✅ **v1.4.0 Foundation:**
+- **Modular Architecture**: 8 specialized modules
+- **Enhanced Security**: Enterprise-grade input validation
+- **Comprehensive Testing**: 88% test coverage with 190 test cases
+- **Multiple Output Formats**: JSON and CSV support
+- **Advanced Request Control**: Custom headers, SSL options
+- **Package Installation**: `pip install -e .`
+
+🚀 **Next: v1.5.0 Development Focus**
+- **Async I/O**: 2-3x performance improvement for large site lists
+- **Configuration Files**: User-defined defaults and settings
+- **Monitoring Mode**: Continuous site monitoring with notifications
+- **Enhanced UX**: Colored output and improved progress reporting
+
+See detailed plans in:
+- [DEVELOPMENT_PLAN.md](DEVELOPMENT_PLAN.md) - Executive overview and timeline
+- [DEVELOPMENT.md](DEVELOPMENT.md) - Technical implementation guide
+- [TODO.md](TODO.md) - Current prioritized task list
+- [ROADMAP.md](ROADMAP.md) - Long-term vision through v2.0.0
+
+## usage:
+
+```
+httpcheck [-h] [-t] [--disable-tld-checks] [--tld-warning-only]
+          [--update-tld-list] [--tld-cache-days TLD_CACHE_DAYS]
+          [-q | -v | -c | -f] [--timeout TIMEOUT] [--retries RETRIES]
+          [--workers WORKERS] [--file-summary]
+          [--comment-style {hash,slash,both}]
+          [--follow-redirects {always,never,http-only,https-only}]
+          [--max-redirects MAX_REDIRECTS] [--show-redirect-timing]
+          [--output {table,json,csv}] [--version] [site ...]
+```
+
+### positional arguments
+
+  site                   return http status codes for one or more websites
+
+### optional arguments
+
+  -h, --help             show this help message and exit
+  -t, --tld              check if domain is in global list of TLDs
+  --disable-tld-checks   disable TLD validation checks
+  --tld-warning-only     show warnings for invalid TLDs without failing
+  --update-tld-list      force update of the TLD list from publicsuffix.org
+  --tld-cache-days TLD_CACHE_DAYS
+                         number of days to keep the TLD cache valid (default: 30)
+  -q, --quiet            only print errors
+  -v, --verbose          increase output verbosity
+  -c, --code             only print status code
+  -f, --fast             fast check wtih threading
+  --timeout TIMEOUT      set the timeout for each request
+  --retries RETRIES      set the number of retries for each request
+  --retry-delay DELAY    delay in seconds between retry attempts (default: 1.0)
+  --workers WORKERS      set the number of worker threads
+  -H, --header HEADER    add custom HTTP header (can be used multiple times)
+  --no-verify-ssl        disable SSL certificate verification
+  --file-summary         show summary of file parsing results (valid URLs, comments, etc.)
+  --comment-style {hash,slash,both}
+                         comment style to recognize: hash (#), slash (//), or both (default: both)
+  --follow-redirects {always,never,http-only,https-only}
+                         control redirect following behavior (default: always)
+  --max-redirects MAX_REDIRECTS
+                         maximum number of redirects to follow (default: 30)
+  --show-redirect-timing show detailed timing for each redirect in the chain
+  --output {table,json,csv}
+                         output format: table (default), json, or csv
+  --version              show program's version number and exit
+
+### additional information
+
+  enter sites in url or 'no' url form: 'httpcheck duckduckgo.com'
+  read sites from a file: 'httpcheck @domains.txt'.
+
+  output formats:
+  - Table (default): Human-readable columnar output
+  - JSON: Machine-parsable format for automation
+  - CSV: Comma-separated values for spreadsheets
+
+  Example: 'httpcheck google.com github.com --output json'
+
+  [List of HTTP status codes](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)
+
+## installation
+
+clone repo and install requirements.
+
+### clone the repository
+
+```shell
+git clone https://github.com/docdyhr/httpcheck
+cd httpcheck/
+```
+
+### install requirements
+
+```shell
+python3 -m pip install . --user
+```
+
+On macOS, install terminal-notifier for notifications:
+
+```shell
+brew install terminal-notifier
+```
+
+### alternative install
+
+set up a virtual environment with venv.
+
+```shell
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e .
+```
+
+or
+
+```shell
+mv httpcheck.py ~/bin/httpcheck # any other bin folder
+chmod + ~/bin/httpcheck
+```
+
+## features
+
+* Check HTTP status codes for one or more websites
+* Support for macOS notifications on failures
+* **NEW: Menu bar app for continuous monitoring**
+  * Live status indicator in menu bar (🟢/🔴)
+  * Background checking with configurable intervals
+  * Rich native macOS notifications with actions
+  * Click notifications to open failing sites
+  * Easy site management through menu interface
+* Advanced redirect tracking and control
+  * Follow all redirects, none, or limit by protocol (HTTP/HTTPS)
+  * Configure maximum number of redirects to follow
+  * View detailed per-hop timing information in redirect chains
+* Enhanced TLD validation
+  * Caches TLD list for better performance
+  * Auto-updates from Public Suffix List
+  * Configurable cache duration
+  * Optional warning-only mode
+* Enhanced file input handling
+  * Support for comments using # or // style
+  * Handles inline comments
+  * Automatically strips whitespace
+  * Detailed file parsing statistics
+* Threading support for faster checks
+* Progress bar for multiple site checks
+* Configurable timeouts and retries
+* Pipe and file input support
+* **NEW: Multiple output formats**
+  * JSON format for easy parsing and automation
+  * CSV format for spreadsheet integration
+  * Table format (default) for human readability
+  * Verbose mode includes redirect details in all formats
+* **NEW: Request customization options**
+  * Custom HTTP headers support with -H flag
+  * Configurable retry delay between attempts
+  * SSL certificate verification control
+  * Enhanced timeout configuration
+
+## usage examples
+
+```shell
+python3 httpcheck.py https://duckduckgo.com
+```
+
+returns ex. 'duckduckgo.com 200'
+
+### python from venv
+
+```shell
+python httpcheck.py -v https://api.github.com/invalid
+```
+
+returns verbose output ex. '[-] api.github.com --> Client errors: 404 Not Found'
+
+### installed as binary
+
+```shell
+httpcheck -q @domains.txt
+```
+
+only returns http status code errors ex. 'api.github.com 404'
+
+```shell
+httpcheck -c 1.1.1.1
+```
+
+status code only ex. '200'
+
+### notifications
+
+On macOS, the tool will show notifications when sites fail checks. For sites with failures:
+
+* Shows failure count
+* Lists failed sites (when less than 10)
+* Groups notifications to avoid duplicates
+
+### advanced usage examples
+
+Control redirect behavior:
+
+```shell
+# Don't follow any redirects
+httpcheck --follow-redirects never example.com
+
+# Only follow redirects to HTTP URLs (not HTTPS)
+httpcheck --follow-redirects http-only example.com
+
+# Only follow redirects to HTTPS URLs (not HTTP)
+httpcheck --follow-redirects https-only example.com
+
+# Show detailed timing for each redirect
+httpcheck --show-redirect-timing http://example.com
+
+# Limit redirect chain length
+httpcheck --max-redirects 5 example.com
+
+# Show timing for each redirect hop
+httpcheck -v --show-redirect-timing example.com
+```
+
+### tld validation examples
+
+Control TLD validation behavior:
+
+```shell
+# Enable TLD validation
+httpcheck -t example.com
+
+# Force update of the TLD list from Public Suffix List
+httpcheck -t --update-tld-list example.com
+
+# Warn about invalid TLDs but don't fail
+httpcheck -t --tld-warning-only example.com
+
+# Disable TLD checks completely
+httpcheck --disable-tld-checks example.com
+
+# Set cache expiration to 60 days
+httpcheck -t --tld-cache-days 60 example.com
+```
+
+### output format examples
+
+Export results in different formats:
+
+```shell
+# JSON output for single site
+httpcheck google.com --output json
+
+# JSON output for multiple sites with verbose details
+httpcheck google.com github.com --output json -v
+
+# CSV output for spreadsheet import
+httpcheck @domains.txt --output csv
+
+# CSV with full details including redirects
+httpcheck @domains.txt --output csv -v
+
+# Process JSON output with jq
+httpcheck @sites.txt --output json | jq '.[] | select(.status != "200")'
+
+# Save CSV report with date
+httpcheck @sites.txt --output csv -v > report_$(date +%Y%m%d).csv
+
+# Filter errors only in JSON format
+httpcheck @sites.txt --output json -q | jq '.[] | .domain'
+```
+
+### request customization examples
+
+Add custom headers and control SSL verification:
+
+```shell
+# Add authorization header
+httpcheck -H "Authorization: Bearer token123" api.example.com
+
+# Multiple custom headers
+httpcheck -H "User-Agent: MyBot 1.0" -H "Accept: application/json" api.example.com
+
+# Disable SSL verification for self-signed certificates
+httpcheck --no-verify-ssl https://self-signed.example.com
+
+# Custom timeout and retry configuration
+httpcheck --timeout 30 --retries 5 --retry-delay 2.0 slow.example.com
+
+# Combine with other options
+httpcheck -H "API-Key: secret" --output json --no-verify-ssl https://api.example.com
+```
+
+### file input examples
+
+Create a domains file with comments and use it with httpcheck:
+
+```txt
+# domains.txt - Example domains file
+google.com  # Search engine
+facebook.com // Social media
+twitter.com
+// Commented out: example.com
+```
+
+```shell
+# Basic file usage
+httpcheck @domains.txt
+
+# Show file parsing statistics
+httpcheck --file-summary @domains.txt
+
+# Only recognize # style comments
+httpcheck --comment-style hash @domains.txt
+
+# Only recognize // style comments
+httpcheck --comment-style slash @domains.txt
+```
+
+## onSite - menu bar app (macOS)
+
+For continuous monitoring with a native macOS experience, use onSite - the menu bar app:
+
+### installation
+
+```shell
+# Install additional dependencies for menu bar app
+pip install rumps pyobjc-framework-Cocoa py2app
+
+# Run the menu bar app directly
+python3 httpcheck_menubar.py
+```
+
+### building a standalone app
+
+Create a native macOS app bundle:
+
+```shell
+# Build the app (creates dist/onSite.app)
+python setup.py py2app
+
+# Copy to Applications folder
+cp -r "dist/onSite.app" /Applications/
+
+# Run from Applications or Spotlight
+open "/Applications/onSite.app"
+```
+
+### menu bar app features
+
+* **Live Status Indicator**: Menu bar shows ⚡ (white lightning - all
+  good/checking), 🔴⚡ (red + lightning - failures)
+* **Badge Count**: Shows number of failed sites next to the icon
+* **Rich Notifications**: Native macOS notifications with:
+  * Site down alerts with status codes
+  * Recovery notifications when sites come back online
+  * Click actions to open failing sites in browser
+  * Custom sounds for different alert types
+* **Background Monitoring**: Configurable check intervals (default 15 minutes, minimum 60 seconds)
+* **Site Management**: Add, remove, and edit monitored sites through menu
+* **Settings**: Configure check intervals, clear failed sites, view logs
+* **Configuration Storage**: Sites and settings saved to `~/.httpcheck/`
+
+### menu bar app usage
+
+1. **Add Sites**: Click menu bar icon → "Add Site..." → Enter URL (automatically checks status)
+2. **Manual Check**: Click "Check Now" for immediate status check
+3. **Auto-Check**: Toggle automatic background checking on/off (default: 15 minutes)
+4. **View Status**: Click on menu to see all sites with status indicators
+5. **Settings**: Adjust check interval, clear failed sites, view logs
+
+### keyboard shortcuts and tips
+
+* Menu bar shows white lightning ⚡ for normal status, red lightning 🔴⚡ for failures
+* Failed site count appears as badge number
+* Notifications include action buttons (Check Now, View, Dismiss)
+* Clicking notifications opens the failing site in your default browser
+* Logs are stored in `~/Library/Logs/onSite/onsite.log` following macOS best practices
+* Log rotation: 5MB max size, 5 backup files maintained automatically
+* Logs viewable through Console.app or "View logs" menu item
+* Configuration files stored in `~/.httpcheck/` for easy backup
+
+## history
+
+checking one or more websites by constantly opening a browser seems a waste of
+time. Providing a cli tool that can be used in a bash shell, pipe or for
+scheduling tasks in cron, notifying the user, seems the way to go. The python
+requests library seems perfect for the job.
+
+## todo
+
+[TODO.md](https://github.com/docdyhr/httpcheck/blob/master/TODO.md)
+
+## license
+
+[MIT](https://github.com/docdyhr/httpcheck/blob/master/LICENSE)