cyguide 1.0.0__tar.gz

cyguide-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Harshwardhan Patil and Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cyguide-1.0.0/PKG-INFO

@@ -0,0 +1,166 @@
+Metadata-Version: 2.4
+Name: cyguide
+Version: 1.0.0
+Summary: A TUI-based guided cybersecurity learning platform
+Author: Harshwardhan Patil, CyGuide Contributors
+License: MIT
+Project-URL: Homepage, https://github.com/h45sh/cyguide
+Project-URL: Repository, https://github.com/h45sh/cyguide
+Project-URL: Issues, https://github.com/h45sh/cyguide/issues
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Security
+Classifier: Intended Audience :: Education
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: textual>=1.0.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: aiosqlite>=0.20.0
+Requires-Dist: pyperclip>=1.9.0
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: python-dotenv>=1.0.0
+Dynamic: license-file
+
+# CyGuide – TUI-Based Guided Cybersecurity Learning and Terminal Tool Orchestration Platform
+
+CyGuide is a Terminal User Interface (TUI) platform for cybersecurity education, designed as a pluggable ecosystem for security tools.
+
+## 🏗️ New Plugin Architecture
+
+CyGuide has been restructured into a "Smart Adapter / Stupid Framework" model:
+
+- **Core Engine (`cyguide/engine`)**: Manages a stateful entity graph and event log using SQLite.
+- **Unified Schemas (`cyguide/schemas`)**: Defines a shared language for security findings (Hosts, Services, Endpoints, etc.).
+- **Plugin System (`tools/`)**: Allows adding new tools (nmap, gobuster, etc.) without touching the core codebase.
+- **Textual TUI (`cyguide/ui`)**: A modern, async terminal interface.
+
+## 🚀 Getting Started
+
+### Prerequisites
+
+- Python 3.11+
+- (Optional) `Ollama` for local LLM explanations.
+
+To automatically install all required security tools (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`) and their system dependencies on Linux/macOS, run:
+
+```bash
+./scripts/install_deps.sh
+```
+
+### Installation
+
+```bash
+# Clone and install in editable mode
+pip install -e .
+```
+
+### Launching
+
+```bash
+# Launch with default database (data/cyguide.db)
+cyg
+
+# Launch with a custom SQLite database
+cyg --db my_project.db
+```
+
+### LLM Configuration
+
+CyGuide supports both local and cloud-based LLMs for generating explanations:
+
+#### Local (Ollama)
+Launch with the `--ollama` flag and optionally specify a model (defaults to `gemma`):
+```bash
+cyg --ollama --ollama-model llama3
+```
+
+#### Cloud APIs (OpenAI, Anthropic, OpenRouter, Grok, etc.)
+CyGuide uses environment variables to configure cloud LLM providers. You can set these in your shell or copy `.env.example` to `.env`.
+
+```bash
+# Required
+export CYGUIDE_API_KEY="your_api_key_here"
+
+# Optional overrides (defaults to OpenAI / gpt-4o)
+export CYGUIDE_API_TYPE="openai"       # Options: openai, anthropic
+export CYGUIDE_API_BASE="https://api.openai.com/v1" # E.g., https://openrouter.ai/api/v1
+export CYGUIDE_API_MODEL="gpt-4o"      # E.g., claude-3-5-sonnet-20240620
+```
+
+### 🩺 Diagnostic Health Check
+Verify your environment, database permissions, installed security tools, and AI backends:
+```bash
+cyg check
+```
+*(Use `cyg check --json` for scriptable JSON output in CI/CD pipelines).*
+
+### 💾 Session Portability & Reporting
+CyGuide provides built-in mechanisms to export, import, and generate reports from your investigations:
+
+#### 1. Machine-Readable Database Exports (`exports/`)
+Export a standalone, self-contained SQLite archive of a specific session containing its full entity graph, relationships, and raw command execution logs:
+```bash
+cyg export <session_id> exports/my_session.db
+```
+Import a previously exported session `.db` into your local workspace on any machine:
+```bash
+cyg import exports/my_session.db
+```
+* **Who are Exports for?**
+  * **Instructors & Graders**: Students can export their lab/CTF session into an archive and submit it. Instructors can run `cyg import` to review findings, topology, and execution timelines in the TUI.
+  * **Security Teams**: Analysts can share specific engagement graphs with team members for peer review or shift handoffs without exposing the entire global database.
+  * **Archiving**: Lightweight, standalone backups of completed pentest engagements or CTF challenges.
+
+#### 2. Human-Readable Markdown Reports (`reports/`)
+Generate a formatted executive and technical Markdown report summarizing all discovered hosts, services, vulnerabilities, and executed commands:
+```bash
+cyg export <session_id> exports/my_session.db --report reports/my_session_report.md
+```
+*(You can also trigger automatic export and reporting directly inside Power Mode by pressing `Ctrl+E`).*
+
+## 📜 Utility Scripts
+
+CyGuide includes dedicated helper scripts under `scripts/`:
+
+| Script | Purpose |
+| :--- | :--- |
+| [`./scripts/install_deps.sh`](scripts/install_deps.sh) | Automatically installs required system tools (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`) and Perl XML dependencies across Debian/Ubuntu, Arch, Fedora/RHEL, and macOS (Homebrew). |
+| [`./scripts/verify.sh`](scripts/verify.sh) | Runs the comprehensive verification pipeline: executes the 86-test pytest suite, validates and regenerates the 15-schema documentation, and checks integration pipelines. |
+| [`./scripts/generate_schema_registry.py`](scripts/generate_schema_registry.py) | Introspects the `cyguide.schemas` module and dynamically updates `docs/developer/SCHEMA_REGISTRY.md` with active finding schemas and Primary Identity Keys (PIKs). |
+
+## ✅ Testing & Verification
+
+To verify the codebase stability and ensure documentation is in sync with the schemas, run:
+
+```bash
+./scripts/verify.sh
+```
+
+## 🛠️ Adding a Tool
+
+CyGuide is built for extensibility. See [docs/user/CONTRIBUTING.md](docs/user/CONTRIBUTING.md) to learn how to write a new adapter.
+
+## 📚 Documentation
+
+*   **[User Manual](docs/user/USER_MANUAL.md)**: Operating guide, keyboard shortcuts, UI panel descriptions, and export/import usage.
+*   **[Architecture & Domain Context](docs/developer/ARCHITECTURE.md)**: System design details, database schemas, and facade layers.
+*   **[Database Architecture & Specification](docs/developer/DATABASE.md)**: Exhaustive database models, PIK hashing, ER diagrams, and portability.
+*   **[Schema Registry](docs/developer/SCHEMA_REGISTRY.md)**: Standard 15-schema vocabulary and Primary Identity Key (PIK) rules.
+*   **[Roadmap & Progress Tracker](docs/developer/ROADMAP.md)**: Unified project phases, task checklists, and historical walkthroughs.
+*   **[Autonomous Agent Harness Vision](docs/developer/FUTURE_POWER_MODE.md)**: Architecture for autonomous AI agents, HITL safety gates, and multi-agent roles.
+*   **[Developer Performance Notes](docs/developer/DEVELOPER_NOTES.md)**: Real-time streaming, event loops, and CPU load roadblocks.
+*   **[Contributors](docs/developer/CONTRIBUTORS.md)**: Project ownership and governance.
+
+## 📂 Project Structure & Data Isolation
+
+- `cyguide/`: The core application package (engine, schemas, modes, TUI).
+- `tools/`: Tool plugin adapters and manifests (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`).
+- `scripts/`: Development and system installation scripts.
+- `docs/`: Technical and user documentation.
+- `data/`: Default directory for local runtime SQLite databases (`cyguide.db`) and scratchpad notes (ignored from version control).
+- `exports/`: Destination for exported portable session `.db` archives (ignored from version control).
+- `reports/`: Destination for generated Markdown investigation reports (ignored from version control).
+
+---
+

cyguide-1.0.0/README.md

@@ -0,0 +1,142 @@
+# CyGuide – TUI-Based Guided Cybersecurity Learning and Terminal Tool Orchestration Platform
+
+CyGuide is a Terminal User Interface (TUI) platform for cybersecurity education, designed as a pluggable ecosystem for security tools.
+
+## 🏗️ New Plugin Architecture
+
+CyGuide has been restructured into a "Smart Adapter / Stupid Framework" model:
+
+- **Core Engine (`cyguide/engine`)**: Manages a stateful entity graph and event log using SQLite.
+- **Unified Schemas (`cyguide/schemas`)**: Defines a shared language for security findings (Hosts, Services, Endpoints, etc.).
+- **Plugin System (`tools/`)**: Allows adding new tools (nmap, gobuster, etc.) without touching the core codebase.
+- **Textual TUI (`cyguide/ui`)**: A modern, async terminal interface.
+
+## 🚀 Getting Started
+
+### Prerequisites
+
+- Python 3.11+
+- (Optional) `Ollama` for local LLM explanations.
+
+To automatically install all required security tools (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`) and their system dependencies on Linux/macOS, run:
+
+```bash
+./scripts/install_deps.sh
+```
+
+### Installation
+
+```bash
+# Clone and install in editable mode
+pip install -e .
+```
+
+### Launching
+
+```bash
+# Launch with default database (data/cyguide.db)
+cyg
+
+# Launch with a custom SQLite database
+cyg --db my_project.db
+```
+
+### LLM Configuration
+
+CyGuide supports both local and cloud-based LLMs for generating explanations:
+
+#### Local (Ollama)
+Launch with the `--ollama` flag and optionally specify a model (defaults to `gemma`):
+```bash
+cyg --ollama --ollama-model llama3
+```
+
+#### Cloud APIs (OpenAI, Anthropic, OpenRouter, Grok, etc.)
+CyGuide uses environment variables to configure cloud LLM providers. You can set these in your shell or copy `.env.example` to `.env`.
+
+```bash
+# Required
+export CYGUIDE_API_KEY="your_api_key_here"
+
+# Optional overrides (defaults to OpenAI / gpt-4o)
+export CYGUIDE_API_TYPE="openai"       # Options: openai, anthropic
+export CYGUIDE_API_BASE="https://api.openai.com/v1" # E.g., https://openrouter.ai/api/v1
+export CYGUIDE_API_MODEL="gpt-4o"      # E.g., claude-3-5-sonnet-20240620
+```
+
+### 🩺 Diagnostic Health Check
+Verify your environment, database permissions, installed security tools, and AI backends:
+```bash
+cyg check
+```
+*(Use `cyg check --json` for scriptable JSON output in CI/CD pipelines).*
+
+### 💾 Session Portability & Reporting
+CyGuide provides built-in mechanisms to export, import, and generate reports from your investigations:
+
+#### 1. Machine-Readable Database Exports (`exports/`)
+Export a standalone, self-contained SQLite archive of a specific session containing its full entity graph, relationships, and raw command execution logs:
+```bash
+cyg export <session_id> exports/my_session.db
+```
+Import a previously exported session `.db` into your local workspace on any machine:
+```bash
+cyg import exports/my_session.db
+```
+* **Who are Exports for?**
+  * **Instructors & Graders**: Students can export their lab/CTF session into an archive and submit it. Instructors can run `cyg import` to review findings, topology, and execution timelines in the TUI.
+  * **Security Teams**: Analysts can share specific engagement graphs with team members for peer review or shift handoffs without exposing the entire global database.
+  * **Archiving**: Lightweight, standalone backups of completed pentest engagements or CTF challenges.
+
+#### 2. Human-Readable Markdown Reports (`reports/`)
+Generate a formatted executive and technical Markdown report summarizing all discovered hosts, services, vulnerabilities, and executed commands:
+```bash
+cyg export <session_id> exports/my_session.db --report reports/my_session_report.md
+```
+*(You can also trigger automatic export and reporting directly inside Power Mode by pressing `Ctrl+E`).*
+
+## 📜 Utility Scripts
+
+CyGuide includes dedicated helper scripts under `scripts/`:
+
+| Script | Purpose |
+| :--- | :--- |
+| [`./scripts/install_deps.sh`](scripts/install_deps.sh) | Automatically installs required system tools (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`) and Perl XML dependencies across Debian/Ubuntu, Arch, Fedora/RHEL, and macOS (Homebrew). |
+| [`./scripts/verify.sh`](scripts/verify.sh) | Runs the comprehensive verification pipeline: executes the 86-test pytest suite, validates and regenerates the 15-schema documentation, and checks integration pipelines. |
+| [`./scripts/generate_schema_registry.py`](scripts/generate_schema_registry.py) | Introspects the `cyguide.schemas` module and dynamically updates `docs/developer/SCHEMA_REGISTRY.md` with active finding schemas and Primary Identity Keys (PIKs). |
+
+## ✅ Testing & Verification
+
+To verify the codebase stability and ensure documentation is in sync with the schemas, run:
+
+```bash
+./scripts/verify.sh
+```
+
+## 🛠️ Adding a Tool
+
+CyGuide is built for extensibility. See [docs/user/CONTRIBUTING.md](docs/user/CONTRIBUTING.md) to learn how to write a new adapter.
+
+## 📚 Documentation
+
+*   **[User Manual](docs/user/USER_MANUAL.md)**: Operating guide, keyboard shortcuts, UI panel descriptions, and export/import usage.
+*   **[Architecture & Domain Context](docs/developer/ARCHITECTURE.md)**: System design details, database schemas, and facade layers.
+*   **[Database Architecture & Specification](docs/developer/DATABASE.md)**: Exhaustive database models, PIK hashing, ER diagrams, and portability.
+*   **[Schema Registry](docs/developer/SCHEMA_REGISTRY.md)**: Standard 15-schema vocabulary and Primary Identity Key (PIK) rules.
+*   **[Roadmap & Progress Tracker](docs/developer/ROADMAP.md)**: Unified project phases, task checklists, and historical walkthroughs.
+*   **[Autonomous Agent Harness Vision](docs/developer/FUTURE_POWER_MODE.md)**: Architecture for autonomous AI agents, HITL safety gates, and multi-agent roles.
+*   **[Developer Performance Notes](docs/developer/DEVELOPER_NOTES.md)**: Real-time streaming, event loops, and CPU load roadblocks.
+*   **[Contributors](docs/developer/CONTRIBUTORS.md)**: Project ownership and governance.
+
+## 📂 Project Structure & Data Isolation
+
+- `cyguide/`: The core application package (engine, schemas, modes, TUI).
+- `tools/`: Tool plugin adapters and manifests (`nmap`, `gobuster`, `nikto`, `whois`, `subfinder`).
+- `scripts/`: Development and system installation scripts.
+- `docs/`: Technical and user documentation.
+- `data/`: Default directory for local runtime SQLite databases (`cyguide.db`) and scratchpad notes (ignored from version control).
+- `exports/`: Destination for exported portable session `.db` archives (ignored from version control).
+- `reports/`: Destination for generated Markdown investigation reports (ignored from version control).
+
+---
+

cyguide-1.0.0/cyguide/app.py

@@ -0,0 +1,154 @@
+"""Main CyGuide TUI Application."""
+
+import os
+from textual.app import App, SystemCommand
+from cyguide.ui.screens.home import DashboardScreen
+from cyguide.ui.screens.learning import LearningModeScreen
+from cyguide.ui.screens.tool_browser import ToolBrowserScreen
+from cyguide.ui.screens.power import PowerModeScreen
+
+from typing import Optional, Dict, Any, List
+import subprocess
+
+from cyguide.engine.registry import ToolRegistry
+from cyguide.engine.store import GraphStore
+from cyguide.engine.executor import Executor
+from cyguide.engine.explainer import LearningExplainer,  OllamaBackend
+from cyguide.modes.learning import LearningModeEngine
+from cyguide.modes.power import PowerModeEngine
+from cyguide.engine.power_facade import PowerWorkspaceFacade
+
+class CyGuideApp(App):
+    """The central orchestrator for the CyGuide TUI."""
+    
+    CSS_PATH = "ui/style.css"
+    TITLE = "CyGuide"
+    
+    # Register screens so they can be switched by name
+    SCREENS = {
+        "dashboard": DashboardScreen,
+        "tool_browser": ToolBrowserScreen,
+        "learning_mode": LearningModeScreen,
+        "power_mode": PowerModeScreen,
+    }
+
+    def __init__(self, db_path: str = "data/cyguide.db", use_ollama: bool = False, ollama_model: str = "gemma", tools_dir: str = None, **kwargs):
+        super().__init__(**kwargs)
+        self.db_path = db_path
+        self.use_ollama = use_ollama
+        self.ollama_model = ollama_model
+        self.tools_dir = tools_dir
+        self.selected_session_id: Optional[str] = None
+        self.selected_workspace_id: Optional[str] = None
+        self.user_env: Dict[str, str] = os.environ.copy()
+        self.user_shell: str = "/bin/bash"
+        # Ensure the directory for the database exists
+        db_dir = os.path.dirname(os.path.abspath(self.db_path))
+        os.makedirs(db_dir, exist_ok=True)
+
+    def detect_user_shell(self) -> str:
+        """Find the user's default shell."""
+        return os.environ.get("SHELL", "/bin/bash")
+
+    def get_shell_environment(self) -> Dict[str, str]:
+        """Capture the full login shell environment."""
+        shell = self.detect_user_shell()
+        try:
+            # Run a login shell to capture the fully resolved environment
+            result = subprocess.run(
+                [shell, "-l", "-c", "env"],
+                capture_output=True, text=True, timeout=5
+            )
+            env = {}
+            for line in result.stdout.splitlines():
+                if "=" in line:
+                    key, _, val = line.partition("=")
+                    env[key] = val
+            return env if env else os.environ.copy()
+        except Exception:
+            return os.environ.copy()
+
+    async def on_mount(self) -> None:
+        """Initialize the app with core components and the dashboard."""
+        self.user_shell = self.detect_user_shell()
+        self.user_env = self.get_shell_environment()
+        
+        self.learning_mode_tool = "nmap" # Default
+        # 1. Initialize Engines
+        self.registry = ToolRegistry(tools_dir=self.tools_dir)
+        self.registry.load_tools()
+        
+        self.store = GraphStore(self.db_path)
+        await self.store.initialize()
+        
+        # Ensure singleton Learning Sandbox exists
+        self.learning_ws_id, self.learning_session_id = await self.store.get_or_create_learning_sandbox()
+
+        from cyguide.engine.recommendation import RecommendationEngine
+        self.recommendation_engine = RecommendationEngine(self.store)
+        self.store.on_upsert(self.recommendation_engine.handle_upsert)
+
+        self.executor = Executor(self.store)
+        
+        # Setup Explainer
+        from dotenv import load_dotenv
+        load_dotenv()
+        if self.use_ollama:
+            self.explainer_backend = OllamaBackend(model=self.ollama_model)
+        else:
+            api_key = os.environ.get("CYGUIDE_API_KEY") or os.environ.get("GROQ_API_KEY")
+            if api_key:
+                from cyguide.engine.explainer import CloudAPIBackend
+                api_type = os.environ.get("CYGUIDE_API_TYPE", "openai")
+                if os.environ.get("GROQ_API_KEY") and not os.environ.get("CYGUIDE_API_BASE"):
+                    api_base = "https://api.groq.com/openai/v1"
+                    api_model = os.environ.get("CYGUIDE_API_MODEL", "llama-3.3-70b-versatile")
+                else:
+                    api_base = os.environ.get("CYGUIDE_API_BASE", "https://api.openai.com/v1")
+                    api_model = os.environ.get("CYGUIDE_API_MODEL", "gpt-4o")
+                self.explainer_backend = CloudAPIBackend(api_type, api_base, api_key, api_model)
+            else:
+                # Fallback to Ollama if no API key is provided and use_ollama is false
+                self.explainer_backend = OllamaBackend(model=self.ollama_model)
+            
+        self.explainer = LearningExplainer(self.explainer_backend)
+        
+        self.learning_engine = LearningModeEngine(self.registry, self.executor, self.explainer)
+        self.power_engine = PowerModeEngine(self.registry, self.executor)
+        self.power_facade = PowerWorkspaceFacade(self.store, self.executor, self.registry)
+        self.power_facade.user_env = self.user_env
+        self.power_facade.user_shell = self.user_shell
+
+        # 2. Setup UI
+        self.push_screen("dashboard")
+        self.animations_enabled = True
+
+    async def on_unmount(self) -> None:
+        """Cleanup resources before exit."""
+        if hasattr(self, "executor"):
+            await self.executor.shutdown()
+        if hasattr(self, "store"):
+            await self.store.close()
+
+    def action_toggle_animations(self) -> None:
+        """Toggle UI animations on or off via the command palette."""
+        self.animations_enabled = not self.animations_enabled
+        if self.animations_enabled:
+            self.remove_class("no-animations")
+            self.notify("Animations Enabled")
+        else:
+            self.add_class("no-animations")
+            self.notify("Animations Disabled")
+
+    def get_system_commands(self, screen):
+        """Add custom commands to the Command Palette (Ctrl+P)."""
+        yield from super().get_system_commands(screen)
+        yield SystemCommand(
+            "Toggle Animations", 
+            "Turn sidebar animations on or off", 
+            self.action_toggle_animations
+        )
+
+if __name__ == "__main__":
+    app = CyGuideApp()
+    app.run()

cyguide-1.0.0/cyguide/engine/adapter.py

@@ -0,0 +1,40 @@
+"""Abstract base class for all tool plugins."""
+
+from abc import ABC, abstractmethod
+from typing import AsyncIterator, List, Dict, Any
+from cyguide.schemas.base import BaseFinding
+
+
+class ToolAdapter(ABC):
+    """
+    Contract for tool contributors. 
+    Every tool in tools/ must implement this in adapter.py.
+    """
+
+    @abstractmethod
+    def validate_install(self) -> bool:
+        """Check if the tool binary is available on the system."""
+        pass
+
+    @abstractmethod
+    def build_command(self, target: BaseFinding, params: Dict[str, Any]) -> List[str]:
+        """
+        Build the command line arguments for the tool.
+        
+        Args:
+            target: The finding being scanned (e.g. a network.host)
+            params: User-provided flags or options from the TUI
+        """
+        pass
+
+    @abstractmethod
+    async def parse_output(self, raw_stdout: str, target: BaseFinding, context: Dict[str, Any] = None) -> AsyncIterator[BaseFinding]:
+        """
+        Parse tool output and yield standardized findings.
+        
+        Args:
+            raw_stdout: The raw output string from the tool
+            target: The original target finding (for context/parenting)
+            context: Optional dictionary for tracking state across lines (e.g. current host)
+        """
+        pass

cyguide-1.0.0/cyguide/engine/canonicalize.py

@@ -0,0 +1,56 @@
+# PROTECTED FILE — See CONTRIBUTING.md before modifying.
+# Changes to this file require team agreement.
+"""Normalization filters for finding data."""
+
+import urllib.parse
+
+
+def canonicalize(value):
+    """Apply normalization rules based on the type of value."""
+    if isinstance(value, str):
+        # General string cleaning
+        value = value.strip()
+        
+        # Lowercase hostnames (heuristic: no spaces, contains dots or is localhost)
+        if "." in value or value.lower() == "localhost":
+            if " " not in value and ":" not in value:
+                return value.lower()
+
+    return value
+
+
+def canonicalize_host(ip_or_name: str) -> str:
+    """Normalize IP addresses and hostnames."""
+    val = ip_or_name.strip().lower()
+    if val in ("localhost", "127.0.0.1", "::1"):
+        return "127.0.0.1"
+    # Basic IP normalization (lowercase IPv6, etc)
+    return val
+
+
+def canonicalize_url(url: str) -> str:
+    """Normalize a URL: lowercase scheme/host, strip default ports."""
+    try:
+        parsed = urllib.parse.urlparse(url)
+        scheme = parsed.scheme.lower()
+        netloc = parsed.netloc.lower()
+
+        # Strip default ports
+        if scheme == "http" and netloc.endswith(":80"):
+            netloc = netloc[:-3]
+        elif scheme == "https" and netloc.endswith(":443"):
+            netloc = netloc[:-4]
+
+        return urllib.parse.urlunparse(
+            (scheme, netloc, parsed.path, parsed.params, parsed.query, parsed.fragment)
+        )
+    except Exception:
+        return url
+
+
+def canonicalize_port(port) -> int:
+    """Ensure port is an integer."""
+    try:
+        return int(port)
+    except (ValueError, TypeError):
+        return 0