splunkctl 0.1.0__tar.gz → 0.3.0__tar.gz

{splunkctl-0.1.0 → splunkctl-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: splunkctl
-Version: 0.1.0
+Version: 0.3.0
 Summary: CLI tool for Splunk Enterprise SIEM operations.
 Author: dannyota
 License-Expression: Apache-2.0
@@ -19,57 +19,116 @@ Classifier: Topic :: System :: Systems Administration
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: splunk-sdk>=2.1.0
-Requires-Dist: click>=8.0
+Requires-Dist: splunk-sdk>=2.0
+Requires-Dist: click>=8.2
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: tabulate>=0.9
+Requires-Dist: requests>=2.32
+Requires-Dist: defusedxml>=0.7
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: mypy>=1.11; extra == "dev"
+Requires-Dist: types-requests; extra == "dev"
+Requires-Dist: types-PyYAML; extra == "dev"
+Requires-Dist: types-tabulate; extra == "dev"
+Requires-Dist: types-defusedxml; extra == "dev"
 Dynamic: license-file
 
 # splunkctl
 
 CLI tool for Splunk Enterprise SIEM operations.
 
-Query, inspect, and manage a Splunk Enterprise instance from the terminal.
-Built on the [splunk-sdk-python](https://github.com/dannyota/splunk-sdk-python)
+Query, inspect, and manage a remote Splunk Enterprise instance from your
+laptop. Built on the [splunk-sdk-python](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
 fork with [Click](https://click.palletsprojects.com/).
 
 ## Install
 
 ```bash
 pip install splunkctl
+pip install git+https://github.com/dannyota/splunk-sdk-python@splunkctl
 ```
 
-Requires Python 3.13+.
+Requires Python 3.13+. The second line installs the
+[forked SDK](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
+which adds dashboard, lookup, and HEC token support. Without it, core
+commands (search, rules, alerts, indexes, inputs, apps, users) still work.
+
+### Development
+
+```bash
+git clone https://github.com/dannyota/splunkctl
+cd splunkctl
+pip install -e .
+splunkctl --version
+```
 
 ## Quick start
 
 ```bash
 splunkctl config init                         # interactive setup
-splunkctl config test                         # verify connectivity
+splunkctl doctor                              # check connection, auth, permissions
 splunkctl search run 'index=main | head 10'   # run a search
 splunkctl rules list                          # list detection rules
-splunkctl dashboards list                     # list dashboards
 ```
 
 ## Commands
 
 | Group | Description |
 |---|---|
+| `doctor` | Connection, auth, health, and permissions check |
 | `config` | Setup, show config, test connectivity |
 | `info` | Server info (version, OS, license) |
-| `search` | Run, export, oneshot, job management |
-| `rules` | Detection rules (saved searches) — full CRUD |
+| `search` | Run, export, oneshot, upload, job management |
+| `rules` | Detection rules — CRUD, import/export (YAML) |
 | `alerts` | Fired alerts, alert actions, suppression |
 | `dashboards` | Dashboard CRUD (XML) |
 | `indexes` | Index management |
 | `inputs` | Data inputs (monitor, tcp, udp, script, http) |
-| `lookups` | Lookup table CRUD (CSV) |
+| `lookups` | Lookup table CRUD (CSV, mmdb) |
+| `hec` | HEC token management |
 | `parsers` | Source types and field extractions |
-| `apps` | App install, uninstall, update |
+| `apps` | App install (.spl/.tar.gz), uninstall, update |
 | `users` | User and role management |
 | `commands` | Machine-readable command tree (JSON) |
 | `skill` | Embedded agent operating guide |
 
+## Key features
+
+### Detection-as-code
+
+Export existing rules to YAML, version control them, deploy across
+instances:
+
+```bash
+splunkctl rules export --path detections.yml
+splunkctl rules import --path detections.yml        # dry-run preview
+splunkctl --yes rules import --path detections.yml  # apply
+```
+
+### Remote file operations
+
+Upload files from your laptop without SSH access to the server:
+
+```bash
+# Upload threat intel, logs, or sample data for indexing
+splunkctl --yes search upload --path threats.csv --index threat_intel --sourcetype csv
+
+# Upload lookup tables (CSV or GeoIP mmdb)
+splunkctl --yes lookups upload --name threats.csv --path threats.csv
+
+# Install apps from local .spl/.tar.gz packages
+splunkctl --yes apps install --path TA_windows.spl
+```
+
+### Diagnostics
+
+```bash
+splunkctl doctor             # check everything: connection, auth, health, permissions
+splunkctl doctor --json      # machine-readable output
+```
+
 ## Global flags
 
 ```
@@ -102,6 +161,23 @@ splunkctl rules list --fields name,cron   # project fields
 splunkctl rules list --out rules.json     # write to file
 ```
 
+## SDK fork
+
+splunkctl depends on a [fork of splunk-sdk-python](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
+that adds entity classes missing from the upstream SDK:
+
+| Entity | Service property | Purpose |
+|---|---|---|
+| `Dashboard` | `service.dashboards` | Dashboard CRUD |
+| `LookupTableFile` | `service.lookup_table_files` | Lookup table metadata + download |
+| `HECToken` | `service.hec_tokens` | HEC token management |
+
+Install the fork directly:
+
+```bash
+pip install git+https://github.com/dannyota/splunk-sdk-python@splunkctl
+```
+
 ## Agent integration
 
 splunkctl ships with an embedded operating guide for AI agents

splunkctl-0.3.0/README.md

@@ -0,0 +1,157 @@
+# splunkctl
+
+CLI tool for Splunk Enterprise SIEM operations.
+
+Query, inspect, and manage a remote Splunk Enterprise instance from your
+laptop. Built on the [splunk-sdk-python](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
+fork with [Click](https://click.palletsprojects.com/).
+
+## Install
+
+```bash
+pip install splunkctl
+pip install git+https://github.com/dannyota/splunk-sdk-python@splunkctl
+```
+
+Requires Python 3.13+. The second line installs the
+[forked SDK](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
+which adds dashboard, lookup, and HEC token support. Without it, core
+commands (search, rules, alerts, indexes, inputs, apps, users) still work.
+
+### Development
+
+```bash
+git clone https://github.com/dannyota/splunkctl
+cd splunkctl
+pip install -e .
+splunkctl --version
+```
+
+## Quick start
+
+```bash
+splunkctl config init                         # interactive setup
+splunkctl doctor                              # check connection, auth, permissions
+splunkctl search run 'index=main | head 10'   # run a search
+splunkctl rules list                          # list detection rules
+```
+
+## Commands
+
+| Group | Description |
+|---|---|
+| `doctor` | Connection, auth, health, and permissions check |
+| `config` | Setup, show config, test connectivity |
+| `info` | Server info (version, OS, license) |
+| `search` | Run, export, oneshot, upload, job management |
+| `rules` | Detection rules — CRUD, import/export (YAML) |
+| `alerts` | Fired alerts, alert actions, suppression |
+| `dashboards` | Dashboard CRUD (XML) |
+| `indexes` | Index management |
+| `inputs` | Data inputs (monitor, tcp, udp, script, http) |
+| `lookups` | Lookup table CRUD (CSV, mmdb) |
+| `hec` | HEC token management |
+| `parsers` | Source types and field extractions |
+| `apps` | App install (.spl/.tar.gz), uninstall, update |
+| `users` | User and role management |
+| `commands` | Machine-readable command tree (JSON) |
+| `skill` | Embedded agent operating guide |
+
+## Key features
+
+### Detection-as-code
+
+Export existing rules to YAML, version control them, deploy across
+instances:
+
+```bash
+splunkctl rules export --path detections.yml
+splunkctl rules import --path detections.yml        # dry-run preview
+splunkctl --yes rules import --path detections.yml  # apply
+```
+
+### Remote file operations
+
+Upload files from your laptop without SSH access to the server:
+
+```bash
+# Upload threat intel, logs, or sample data for indexing
+splunkctl --yes search upload --path threats.csv --index threat_intel --sourcetype csv
+
+# Upload lookup tables (CSV or GeoIP mmdb)
+splunkctl --yes lookups upload --name threats.csv --path threats.csv
+
+# Install apps from local .spl/.tar.gz packages
+splunkctl --yes apps install --path TA_windows.spl
+```
+
+### Diagnostics
+
+```bash
+splunkctl doctor             # check everything: connection, auth, health, permissions
+splunkctl doctor --json      # machine-readable output
+```
+
+## Global flags
+
+```
+--json              Force JSON output
+--format FMT        Output format: table, json, csv, jsonl
+--fields f1,f2      Project specific fields
+--out FILE          Write output to file
+--yes / -y          Apply mutations (skip dry-run preview)
+--timeout N         Request timeout in seconds (default 30)
+--config FILE       Config file path
+--debug             HTTP request/response logging
+```
+
+## Dry-run by default
+
+All write operations preview what would change. Pass `--yes` to apply.
+
+```bash
+splunkctl rules delete 'My Rule'          # shows preview only
+splunkctl rules delete 'My Rule' --yes    # actually deletes
+```
+
+## Output formats
+
+```bash
+splunkctl rules list                      # table (TTY) or JSON (pipe)
+splunkctl rules list --json               # force JSON
+splunkctl rules list --format csv         # CSV
+splunkctl rules list --fields name,cron   # project fields
+splunkctl rules list --out rules.json     # write to file
+```
+
+## SDK fork
+
+splunkctl depends on a [fork of splunk-sdk-python](https://github.com/dannyota/splunk-sdk-python/tree/splunkctl)
+that adds entity classes missing from the upstream SDK:
+
+| Entity | Service property | Purpose |
+|---|---|---|
+| `Dashboard` | `service.dashboards` | Dashboard CRUD |
+| `LookupTableFile` | `service.lookup_table_files` | Lookup table metadata + download |
+| `HECToken` | `service.hec_tokens` | HEC token management |
+
+Install the fork directly:
+
+```bash
+pip install git+https://github.com/dannyota/splunk-sdk-python@splunkctl
+```
+
+## Agent integration
+
+splunkctl ships with an embedded operating guide for AI agents
+(Claude Code, etc.):
+
+```bash
+splunkctl skill                           # print the guide
+splunkctl skill install                   # install to ~/.claude/skills/
+splunkctl commands                        # JSON command tree for discovery
+```
+
+## License
+
+Apache-2.0

{splunkctl-0.1.0 → splunkctl-0.3.0}/pyproject.toml

@@ -23,10 +23,23 @@ classifiers = [
     "Topic :: System :: Systems Administration",
 ]
 dependencies = [
-    "splunk-sdk>=2.1.0",
-    "click>=8.0",
+    "splunk-sdk>=2.0",
+    "click>=8.2",
     "pyyaml>=6.0",
     "tabulate>=0.9",
+    "requests>=2.32",
+    "defusedxml>=0.7",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "ruff>=0.6",
+    "mypy>=1.11",
+    "types-requests",
+    "types-PyYAML",
+    "types-tabulate",
+    "types-defusedxml",
 ]
 
 [project.scripts]

{splunkctl-0.1.0 → splunkctl-0.3.0}/splunkctl/__init__.py

@@ -1,3 +1,3 @@
 """CLI tool for Splunk Enterprise SIEM operations."""
 
-__version__ = "0.1.0"
+__version__ = "0.3.0"

splunkctl-0.3.0/splunkctl/client.py

@@ -0,0 +1,333 @@
+"""SDK wrapper — lazy connection and auth resolution.
+
+Includes Web UI upload support for operations the REST API cannot
+handle remotely (e.g. lookup file upload requires server-side staging).
+"""
+
+from __future__ import annotations
+
+import re
+import urllib.parse
+from pathlib import Path
+from typing import Any
+
+import click
+import requests
+import splunklib.client as splunk_client
+
+from splunkctl import config as cfg_mod
+
+
+class SplunkClient:
+    """Lazy-initializing Splunk SDK client.
+
+    Connection is established on first access to ``.service``.
+    Help, config, and offline commands never trigger auth.
+    """
+
+    def __init__(  # noqa: D107
+        self,
+        *,
+        config_path: Path | None = None,
+        host: str | None = None,
+        port: int | None = None,
+        username: str | None = None,
+        password: str | None = None,
+        token: str | None = None,
+        scheme: str | None = None,
+        app: str | None = None,
+        owner: str | None = None,
+        verify: bool | None = None,
+        timeout: int = 30,
+        debug: bool = False,
+    ) -> None:
+        self._config_path = config_path
+        self._overrides: dict[str, Any] = {
+            k: v
+            for k, v in {
+                "host": host,
+                "port": port,
+                "username": username,
+                "password": password,
+                "token": token,
+                "scheme": scheme,
+                "app": app,
+                "owner": owner,
+                "verify": verify,
+            }.items()
+            if v is not None
+        }
+        self._timeout = timeout
+        self._debug = debug
+        self._service: Any = None
+        self._web_session: _WebSession | None = None
+
+    @property
+    def service(self) -> Any:
+        """Connect on first access."""
+        if self._service is None:
+            if self._debug:
+                import logging
+
+                logging.basicConfig(level=logging.DEBUG)
+                logging.getLogger("splunklib").setLevel(logging.DEBUG)
+
+            cfg = cfg_mod.load(self._config_path)
+            cfg.update(self._overrides)
+
+            connect_args: dict[str, Any] = {
+                "host": cfg.get("host", "localhost"),
+                "port": int(cfg.get("port", 8089)),
+                "scheme": cfg.get("scheme", "https"),
+            }
+
+            if cfg.get("token"):
+                connect_args["splunkToken"] = cfg["token"]
+            else:
+                connect_args["username"] = cfg.get("username", "")
+                connect_args["password"] = cfg.get("password", "")
+
+            if cfg.get("app"):
+                connect_args["app"] = cfg["app"]
+            if cfg.get("owner"):
+                connect_args["owner"] = cfg["owner"]
+
+            if not cfg.get("verify", False):
+                connect_args["verify"] = False
+
+            if self._timeout:
+                connect_args["timeout"] = self._timeout
+
+            self._service = splunk_client.connect(**connect_args)
+
+        return self._service
+
+    def _ensure_web_session(self) -> _WebSession:
+        if self._web_session is None:
+            cfg = cfg_mod.load(self._config_path)
+            cfg.update(self._overrides)
+            self._web_session = _WebSession(
+                self.service,
+                verify=bool(cfg.get("verify", False)),
+                debug=self._debug,
+                timeout=self._timeout or 30,
+            )
+        return self._web_session
+
+    def upload_lookup(
+        self,
+        name: str,
+        file_path: Path,
+        *,
+        app: str = "search",
+        update: bool = False,
+    ) -> None:
+        """Upload a lookup CSV via the Splunk Web UI form handler."""
+        self._ensure_web_session().upload_lookup(
+            name, file_path, app=app, update=update
+        )
+
+    def install_app(
+        self,
+        file_path: Path,
+        *,
+        force: bool = False,
+    ) -> None:
+        """Install a .spl/.tar.gz app package via the Splunk Web UI."""
+        self._ensure_web_session().install_app(file_path, force=force)
+
+    def set_acl(self, entity: Any, *, sharing: str, owner: str | None = None) -> None:
+        """Change an entity's sharing level via its ACL endpoint.
+
+        Args:
+            entity: Any SDK entity (saved search, dashboard, conf stanza...).
+            sharing: One of user, app, global.
+            owner: Defaults to the entity's current owner, else "nobody".
+        """
+        acl: dict[str, Any] = dict(entity.access)
+        entity.acl_update(sharing=sharing, owner=owner or acl.get("owner", "nobody"))
+
+
+class _WebSession:
+    """Authenticated Splunk Web session for form-handler operations.
+
+    Uses ``requests`` deliberately: the manager form handlers answer a
+    plain urllib POST with a 303 that dead-ends in a 404, while a
+    keep-alive session receives the JSON result directly (verified
+    against Splunk 10.4). TLS verification is on unless the user's
+    config explicitly disables it for self-signed dev instances.
+    """
+
+    def __init__(
+        self,
+        service: Any,
+        *,
+        verify: bool = True,
+        debug: bool = False,
+        timeout: int = 30,
+    ) -> None:
+        self._host: str = service.host
+        self._username: str = service.username
+        self._password: str = service.password
+        if not self._username or not self._password:
+            raise RuntimeError(
+                "Lookup upload requires username/password authentication. "
+                "Token-only auth cannot authenticate to Splunk Web UI."
+            )
+
+        web_conf = service.confs["web"]["settings"]
+        self._web_port = int(web_conf["httpport"])
+        self._web_ssl = str(web_conf.content.get("enableSplunkWebSSL", "0")) == "1"
+
+        self._debug = debug
+        self._timeout = timeout
+        self._session = requests.Session()
+        self._session.verify = verify
+        self._csrf_token: str | None = None
+        self._logged_in = False
+
+    @property
+    def _base_url(self) -> str:
+        scheme = "https" if self._web_ssl else "http"
+        return f"{scheme}://{self._host}:{self._web_port}"
+
+    def _request(self, method: str, url: str, **kwargs: Any) -> requests.Response:
+        resp = self._session.request(method, url, timeout=self._timeout, **kwargs)
+        if self._debug:
+            click.echo(f"web {method} {url} -> {resp.status_code}", err=True)
+        return resp
+
+    def _login(self) -> None:
+        """Authenticate to Splunk Web and obtain session cookies."""
+        login_url = f"{self._base_url}/en-US/account/login"
+        page = self._request("GET", login_url).text
+
+        m = re.search(r'"cval"\s*:\s*(\d+)', page)
+        cval = m.group(1) if m else "0"
+
+        resp = self._request(
+            "POST",
+            login_url,
+            data={
+                "username": self._username,
+                "password": self._password,
+                "cval": cval,
+            },
+        )
+        try:
+            body: dict[str, Any] = resp.json()
+        except ValueError:
+            raise RuntimeError(
+                f"Splunk Web login failed: HTTP {resp.status_code}"
+            ) from None
+        if body.get("status") == "fail":
+            msg = body.get("msg", "unknown error")
+            raise RuntimeError(f"Splunk Web login failed: {msg}")
+
+        for cookie in self._session.cookies:
+            if cookie.name and cookie.name.startswith("splunkweb_csrf_token"):
+                self._csrf_token = cookie.value
+                break
+        if not self._csrf_token:
+            raise RuntimeError("Could not obtain CSRF token from Splunk Web")
+        self._logged_in = True
+
+    def upload_lookup(
+        self,
+        name: str,
+        file_path: Path,
+        *,
+        app: str = "search",
+        update: bool = False,
+    ) -> None:
+        """Upload a lookup file via multipart form POST."""
+        if not self._logged_in:
+            self._login()
+
+        if update:
+            url = (
+                f"{self._base_url}/en-US/manager/{app}"
+                f"/data/lookup-table-files/{urllib.parse.quote(name)}"
+            )
+            action = "edit"
+        else:
+            url = f"{self._base_url}/en-US/manager/{app}/data/lookup-table-files/_new"
+            action = "new"
+
+        data: dict[str, str] = {
+            "__action": action,
+            "__redirect": "",
+            "__ns": app,
+            "splunk_form_key": self._csrf_token or "",
+        }
+        if not update:
+            data["name"] = name
+
+        resp = self._request(
+            "POST",
+            url,
+            data=data,
+            files={"spl-ctrl_lookupfile": (name, file_path.read_bytes(), "text/csv")},
+        )
+        _expect_ok(resp, "Lookup upload")
+
+    def install_app(
+        self,
+        file_path: Path,
+        *,
+        force: bool = False,
+    ) -> None:
+        """Install a .spl/.tar.gz app via the Web UI upload handler."""
+        if not self._logged_in:
+            self._login()
+
+        upload_url = f"{self._base_url}/en-US/manager/appinstall/_upload"
+        page = self._request("GET", upload_url).text
+
+        m = re.search(r'name="state"\s+[^>]*value="([^"]*)"', page)
+        state = m.group(1) if m else ""
+
+        data: dict[str, str] = {
+            "state": state,
+            "splunk_form_key": self._csrf_token or "",
+        }
+        if force:
+            data["force"] = "1"
+
+        resp = self._request(
+            "POST",
+            upload_url,
+            data=data,
+            files={
+                "appfile": (
+                    file_path.name,
+                    file_path.read_bytes(),
+                    "application/gzip",
+                )
+            },
+        )
+        if resp.status_code >= 400:
+            raise RuntimeError(f"App install failed: HTTP {resp.status_code}")
+
+
+def _expect_ok(resp: requests.Response, what: str) -> None:
+    """Raise unless the form handler answered with JSON status OK."""
+    try:
+        result: dict[str, Any] = resp.json()
+    except ValueError:
+        raise RuntimeError(
+            f"{what} failed: HTTP {resp.status_code} — "
+            f"unexpected response: {resp.text[:200]!r}"
+        ) from None
+    if result.get("status") != "OK":
+        raise RuntimeError(f"{what} failed: {result.get('msg', 'unknown error')}")
+
+
+def get_client(ctx: click.Context) -> SplunkClient:
+    """Build a SplunkClient from Click context. Does not connect."""
+    obj: dict[str, Any] = ctx.ensure_object(dict)
+    return SplunkClient(
+        config_path=Path(obj["config"]) if obj.get("config") else None,
+        timeout=obj.get("timeout", 30),
+        debug=obj.get("debug", False),
+    )