spectrum-wrangler 0.3.0__tar.gz

spectrum_wrangler-0.3.0/AGENTS.md

@@ -0,0 +1,38 @@
+# Agent notes
+
+Spectrum Wrangler is one read-only CLI over a local SQLite build of the FCC's
+public radio licensing data (ULS). People and agents use the same commands.
+
+## Using the tool
+
+* Read [skills/spectrum-wrangler/SKILL.md](skills/spectrum-wrangler/SKILL.md)
+  before querying — it carries the six dataset traps that produce confidently
+  wrong answers (row-multiplying joins, `MM/DD/YYYY` text dates, the missing
+  organization key, and the rest).
+* `spectrum-wrangler capabilities` describes every command, argument, format,
+  and exit code as JSON in one call.
+* Run `spectrum-wrangler status` first and cite the FCC publication dates it
+  reports. An empty result means "not in the loaded snapshots", never "no such
+  authorization exists".
+* Exit codes: `0` success, `1` understood but nothing matched, `2` bad request.
+  Errors are JSON on stderr in machine formats.
+* The database resolves from `--database`, then `$SPECTRUM_WRANGLER_DB`, then
+  `./data/spectrum-wrangler.sqlite3` if already built, then the per-user data
+  directory. If it is missing it must be built with `spectrum-wrangler refresh`
+  (a 1.25 GB download; `--archive paging` is a small start).
+
+## Working on the code
+
+* Tests: `python3 -m unittest discover -s tests -v` — hermetic, no network,
+  under a second. Also run `python3 -m compileall -q spectrum_wrangler tests`.
+* Every query operation is declared once in `OPERATIONS` in
+  `spectrum_wrangler/cli.py`; the argparse tree, help text, and `capabilities`
+  manifest are generated from it. Never wire a subcommand by hand.
+* There are no third-party runtime dependencies, deliberately. Read
+  [docs/DEPENDENCIES.md](docs/DEPENDENCIES.md) before adding, updating, or even
+  temporarily executing a package.
+* `load.py` and `sample-fcc.csv` are legacy Python 2 artifacts kept for
+  historical reference. Do not run, modernize, or delete them.
+* [docs/QUERYING.md](docs/QUERYING.md) has the join semantics and worked
+  investigations; [docs/RESEARCH-2026.md](docs/RESEARCH-2026.md) is an archived
+  record of the 2026 source migration, not a live status page.

spectrum_wrangler-0.3.0/CHANGELOG.md

@@ -0,0 +1,45 @@
+# Changelog
+
+## 0.3.0 — 2026-08-30
+
+The rewrite release: same mission as 2017, current FCC publication, nothing to
+install. First release published to PyPI.
+
+### Changed
+* **Source.** FCC License View is retired; the tool now reads the FCC's ULS
+  weekly complete archives directly. See [docs/RESEARCH-2026.md](docs/RESEARCH-2026.md).
+* **Storage.** PostgreSQL and PostGIS replaced by SQLite with FTS5 and RTree —
+  Python 3.11+ standard library only, no services, no third-party packages.
+  See [docs/MIGRATION.md](docs/MIGRATION.md).
+* The database, download cache, and provenance manifest now default to a
+  per-user data directory (`~/Library/Application Support/spectrum-wrangler`
+  on macOS, `$XDG_DATA_HOME/spectrum-wrangler` on Linux,
+  `%LOCALAPPDATA%\spectrum-wrangler` on Windows), so the installed CLI works
+  from any directory. A database already built under `./data` keeps working,
+  and `$SPECTRUM_WRANGLER_DB` or `--database` overrides both.
+
+### Added
+* One CLI for people and agents: structured commands, a `capabilities`
+  manifest generated from the same declarations as the argparse tree, output
+  as table/JSON/NDJSON/CSV, distinct exit codes, and JSON errors on stderr.
+* Read-only enforcement at the SQLite connection, with bounded single-statement
+  SQL (`SELECT`/`WITH`/`EXPLAIN`, timeout, 1,000-row cap).
+* Full-fidelity import: all 89 documented ULS record layouts kept verbatim in
+  `raw_*` tables, common records normalized and indexed, publication drift
+  preserved in `extra_fields_json`.
+* Per-archive provenance: URL, FCC publication time, retrieval time, ETag,
+  SHA-256, byte size, and record counts, written to `source-manifest.json`.
+* An agent skill ([skills/spectrum-wrangler/SKILL.md](skills/spectrum-wrangler/SKILL.md))
+  carrying the dataset judgment a tool schema cannot.
+* Byte-count download progress on an interactive terminal.
+* CI across Python 3.11–3.14 on Linux, macOS, and Windows.
+
+### Removed
+* The Model Context Protocol server, in favour of the CLI plus the skill.
+* The PostgreSQL/PostGIS/Docker requirement.
+
+## 0.2.x and earlier — 2017
+
+The original License View loader: Python 2, `requests`, `psycopg2`, and a
+PostGIS-backed import of the FCC's now-retired denormalized CSV. Kept for
+reference as `load.py`.

spectrum_wrangler-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2017-2026 Marc DaCosta
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

spectrum_wrangler-0.3.0/MANIFEST.in

@@ -0,0 +1,7 @@
+# Ships the docs, the agent skill, and the hermetic test suite in the sdist so
+# a distribution packager can read and verify what they are building.
+include AGENTS.md
+include CHANGELOG.md
+recursive-include docs *.md *.png
+recursive-include skills *.md
+recursive-include tests *.py

spectrum_wrangler-0.3.0/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: spectrum-wrangler
+Version: 0.3.0
+Summary: A provenance-tracked local database and query interface for FCC spectrum licenses
+Author: Marc DaCosta
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/marcdacosta/spectrum-wrangler
+Project-URL: Documentation, https://github.com/marcdacosta/spectrum-wrangler/blob/master/docs/QUERYING.md
+Project-URL: Changelog, https://github.com/marcdacosta/spectrum-wrangler/blob/master/CHANGELOG.md
+Project-URL: Issues, https://github.com/marcdacosta/spectrum-wrangler/issues
+Keywords: fcc,uls,spectrum,radio,licensing,callsign,sqlite,investigative-journalism
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Legal Industry
+Classifier: Intended Audience :: Other Audience
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Communications :: Ham Radio
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering :: GIS
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+![](docs/img/radiowave.png)
+
+[![tests](https://github.com/marcdacosta/spectrum-wrangler/actions/workflows/tests.yml/badge.svg)](https://github.com/marcdacosta/spectrum-wrangler/actions/workflows/tests.yml)
+[![PyPI](https://img.shields.io/pypi/v/spectrum-wrangler)](https://pypi.org/project/spectrum-wrangler/)
+
+# What is Spectrum Wrangler?
+Spectrum Wrangler will download the most recent version of the FCC's public licensing data, parse it into a local database, clean up the geo columns so they can be used to query on, and produce a spatial index on them.
+
+# About the FCC licensing data
+In the United States the Federal Communication Commission is responsible for regulating the use of the radio waves. The FCC issues licenses for radio transmitters that exceed certain power levels or which operate on certain frequencies. Those licenses are public, and they say who is allowed to transmit, where, on what frequency, and with how much power. Taken together they are a map of who owns the air.
+
+The original version of this project used the [License View](http://reboot.fcc.gov/license-view/) database, which harmonized the schema of the various parts of the [Universal Licensing System](http://wireless.fcc.gov/uls/index.htm?job=transaction&page=weekly) (ULS) into one flat CSV. The FCC no longer maintains License View. Spectrum Wrangler now reads ULS directly, which is the same data with more detail and no denormalization: a license header, its licensee, its locations, its antennas, its frequency assignments, and its emissions all arrive as separate related records.
+
+The FCC snapshot published August 23, 2026 holds 5.2M licenses, 2.8M of them active, across 129 radio services — 6.6M licensee records, 3.3M transmitter locations (2.4M with usable coordinates), and 11.4M frequency assignments. Your own numbers will differ; `status` reports what you actually loaded.
+
+# How to use
+* Install it: `uv tool install spectrum-wrangler`, `pipx install spectrum-wrangler`, or `pip install spectrum-wrangler`. Python 3.11 or newer is the only requirement, and a clone of this repository works identically without installing (see below).
+* Run `spectrum-wrangler refresh` to download and index every current weekly archive.
+* Run `spectrum-wrangler refresh --archive paging` first if you want a small database to try.
+* Query it with `spectrum-wrangler <command>`, or point an AI agent at it (see below).
+* Read [the querying guide](docs/QUERYING.md) for how the records join and what to watch out for.
+
+The complete download is about 1.2 GB compressed and produces a database of roughly 23 GB once every source field is retained and indexed. Downloads are cached and content-hashed, so re-running `refresh` re-imports only what actually changed. Use `--normalized-only` for a much smaller database when you do not need every raw FCC field.
+
+The database, download cache, and provenance manifest live in a per-user data directory: `~/Library/Application Support/spectrum-wrangler` on macOS, `$XDG_DATA_HOME/spectrum-wrangler` (usually under `~/.local/share`) on Linux, `%LOCALAPPDATA%\spectrum-wrangler` on Windows. A database already built under `./data` in a repository checkout keeps being used, and `--database` or the `SPECTRUM_WRANGLER_DB` environment variable points anywhere else.
+
+Generated archives and databases stay out of Git. [data/source-manifest.json](data/source-manifest.json) is committed instead, and records each archive's official URL, FCC publication time, retrieval time, ETag, SHA-256, byte size, record counts, and any observed schema drift.
+
+# Do I need Docker?
+No. A project to Dockerize `spectrum-wrangler` can be found [here](https://github.com/brannondorsey/spectrum-wrangler-docker), and it made sense when this tool required a PostgreSQL server with the PostGIS extension installed and configured. That was the hard part, and a container solved it.
+
+Version 0.3 has no services and no dependencies, so a container would add a step rather than remove one — and the expensive artifact is the 23 GB database, which has to live on your disk either way. The Docker wrapper targets the retired PostgreSQL/License View workflow and is kept as a historical reference, not the current path.
+
+# Requirements
+* **Python 3.11 or newer, and nothing else.** SQLite, FTS5, and RTree ship with Python. PostgreSQL, PostGIS, and Docker are no longer needed.
+* **Disk: about 25 GB** for a full load — a 23 GB database plus a 1.25 GB download cache. `--normalized-only` cuts the database to roughly a third of that.
+* **Memory: under 600 MB.** The importer streams, so memory tracks the widest record rather than the size of the archive. Measured peaks: 77 MB for the smallest archive, 537 MB for the largest (35.4M records). An 8 GB laptop is comfortable.
+* **Time: roughly 15 minutes** of import for the full set on a modern laptop once downloaded, plus the download itself. Importing runs at about 130,000 records per second.
+
+Install it from PyPI:
+
+    uv tool install spectrum-wrangler        # or: pipx install spectrum-wrangler
+
+Or clone the repository and run it in place, no install step at all:
+
+    git clone https://github.com/marcdacosta/spectrum-wrangler.git
+    cd spectrum-wrangler
+    python3 -m spectrum_wrangler sources
+
+The two are the same program: `python3 -m spectrum_wrangler` from a checkout accepts every command written as `spectrum-wrangler` in the examples here.
+
+## Start small
+One archive is enough to learn the shape of the data, and the smallest is a four-second import:
+
+| Archive | Download | Records | Database |
+|---|---|---|---|
+| `--archive paging` | 6.5 MB | 750K | 142 MB |
+| `--archive amateur` | 198 MB | 10.5M | 2.3 GB (710 MB with `--normalized-only`) |
+| everything | 1.25 GB | 105.9M | 23 GB |
+
+Run `spectrum-wrangler sources` to see the full list of 14 archives.
+
+# Examples
+Example raw data extract from the original License View publication can be found in `sample-fcc.csv`. [The querying guide](docs/QUERYING.md) has worked investigations and the join semantics behind these.
+
+## Query to check what is loaded and how current it is
+
+    spectrum-wrangler status
+
+## Look up a call sign
+
+    spectrum-wrangler callsign W1AW
+
+    [
+      {
+        "callsign": "W1AW",
+        "display_name": "ARRL HQ OPERATORS CLUB",
+        "radio_service_code": "HA",
+        "state": "CT",
+        "license_status": "A",
+        "grant_date": "12/08/2020",
+        "expired_date": "02/26/2031",
+        "unique_system_id": 780866
+      }
+    ]
+
+## Ranged queries
+Query to search within 1000m of [40.7253319,-74.0076834](https://www.google.com/maps/search/40.7253319,-74.0076834?sa=X&ved=0ahUKEwjj06n16p_WAhWR8oMKHUMfA9oQ8gEIJzAA) and export the results:
+
+    spectrum-wrangler --format csv \
+      nearby 40.7253319 -74.0076834 --radius-km 1 > /tmp/antennas.csv
+
+Or in SQL, against the same database:
+
+    spectrum-wrangler sql \
+      "SELECT radio_service_code, count(*) AS licenses
+         FROM licenses GROUP BY 1 ORDER BY 2 DESC"
+
+# How agents use it
+Spectrum Wrangler is one CLI, used the same way by a person and by an agent. Every operation is declared once in the code, so the commands, the help text, and the machine-readable manifest cannot drift apart.
+
+    spectrum-wrangler capabilities
+
+`capabilities` describes every command, argument, output format, and exit code as JSON, so an agent can discover the whole surface in one call. Output adapts to the consumer — a table on a terminal, JSON when piped — and `--format table|json|ndjson|csv` always wins.
+
+    # everything about one licence, in one call
+    spectrum-wrangler license --callsign W1AW
+
+    # who else transmits from this building
+    spectrum-wrangler nearby 40.748444 -73.985694 --radius-km 0.1
+
+    # who holds a band, and who holds a name
+    spectrum-wrangler band 462 468 --group-by licensee
+    spectrum-wrangler organization --name "NEW YORK CITY POLICE"
+
+    # what expires this quarter, as CSV
+    spectrum-wrangler --format csv expirations --start 2026-10-01 --end 2026-12-31 --state NY
+
+Exit codes separate the cases a program must tell apart: `0` success, `1` understood but nothing matched, `2` bad request. Errors are JSON on stderr in machine formats. The `spectrum-wrangler-agent` entry point is the same CLI with output pinned to JSON whether or not it is attached to a terminal, and [AGENTS.md](AGENTS.md) is the in-repository briefing for agents working on or with the code.
+
+The CLI is read-only and enforces it at the SQLite connection rather than by convention. Custom SQL accepts one `SELECT`, `WITH`, or `EXPLAIN`, runs with a timeout, and returns at most 1,000 rows.
+
+## The skill
+[`skills/spectrum-wrangler/SKILL.md`](skills/spectrum-wrangler/SKILL.md) is the companion to the CLI, and it carries the part a tool schema cannot: what will make you confidently wrong about this dataset. Joining on `unique_system_id` alone turns one call sign into 1.8 million rows. Dates are `MM/DD/YYYY` text, so sorting them sorts by month. Counting what one organization holds has no exact answer. Point any agent at it.
+
+To install the skill for Claude Code, link it into your skills directory:
+
+    mkdir -p ~/.claude/skills
+    ln -s "$PWD/skills/spectrum-wrangler" ~/.claude/skills/spectrum-wrangler
+
+Earlier versions shipped a Model Context Protocol server. It was removed in favour of the CLI plus the skill: it duplicated every parameter declaration in a third place, needed a client restart to pick up changes, and had nowhere to put the judgment that actually matters here. If you need to query this from a client with no shell, that is the one thing the MCP server did better.
+
+# Current state
+Version 0.3 is a rewrite of the ingest and query layers. The purpose has not changed: get the public licensing dataset, clean its geographic fields, index it, and make it useful to query.
+
+What changed:
+
+* **Source.** FCC License View is retired; ULS weekly complete files replace it. See [the 2026 publication research](docs/RESEARCH-2026.md).
+* **Storage.** PostgreSQL and PostGIS are gone. SQLite with FTS5 and an RTree index does the same work with nothing to install. See [migration notes](docs/MIGRATION.md).
+* **Fidelity.** Every field of all 89 documented ULS record layouts is preserved in `raw_*` tables; the common ones are also normalized into typed, indexed tables (`licenses`, `entities`, `locations`, `antennas`, `frequencies`, `emissions`, `amateur`). Raw records join on `unique_system_identifier`. Undocumented trailing fields are kept in `extra_fields_json` rather than dropped, so FCC publication drift is visible instead of silent.
+* **Provenance.** Every import records where the file came from, when the FCC published it, when it was fetched, and its hash.
+
+Known limits:
+
+* Full-power AM/FM/TV station licensing moved to the FCC's separate Licensing and Management System (LMS) and is **not** in this database. ULS still carries broadcast auxiliary. LMS is deliberately not mixed in.
+* Rebuilds use the weekly complete files. Daily transaction deltas are not applied yet.
+* Coordinates come from the FCC as published; about a quarter of location records have no usable coordinate.
+* `load.py` and `sample-fcc.csv` are the original Python 2 loader and a License View sample. They are kept for reference and do not run against current data.
+
+## What this data contains
+Everything here is FCC public record, published for anyone to download, and Spectrum Wrangler queries all of it. There is no column gate: the database is a local file you can open with `sqlite3`, so a gate would restrict nothing while breaking legitimate lookups.
+
+Be aware of what that means in bulk. Entity records carry licensee contact details, and the amateur archive alone is roughly 1.7 million individual licensees with names and mailing addresses. Public record, but a bulk extract of private individuals is a different thing from a single lookup — worth a thought before you republish one.
+
+Read-only *is* enforced, at the SQLite connection rather than by convention: a write is refused by the engine.
+
+# Development
+
+    python3 -m unittest discover -s tests -v
+    python3 -m compileall -q spectrum_wrangler tests
+
+The suite is hermetic — no network, no fixtures on disk, under a second — and CI runs it on Python 3.11 through 3.14 across Linux, macOS, and Windows.
+
+The only build dependency is the pinned `setuptools==80.9.0`. The application itself has no third-party runtime dependencies, and [the dependency policy](docs/DEPENDENCIES.md) explains what it takes to add one.

spectrum_wrangler-0.3.0/README.md

@@ -0,0 +1,164 @@
+![](docs/img/radiowave.png)
+
+[![tests](https://github.com/marcdacosta/spectrum-wrangler/actions/workflows/tests.yml/badge.svg)](https://github.com/marcdacosta/spectrum-wrangler/actions/workflows/tests.yml)
+[![PyPI](https://img.shields.io/pypi/v/spectrum-wrangler)](https://pypi.org/project/spectrum-wrangler/)
+
+# What is Spectrum Wrangler?
+Spectrum Wrangler will download the most recent version of the FCC's public licensing data, parse it into a local database, clean up the geo columns so they can be used to query on, and produce a spatial index on them.
+
+# About the FCC licensing data
+In the United States the Federal Communication Commission is responsible for regulating the use of the radio waves. The FCC issues licenses for radio transmitters that exceed certain power levels or which operate on certain frequencies. Those licenses are public, and they say who is allowed to transmit, where, on what frequency, and with how much power. Taken together they are a map of who owns the air.
+
+The original version of this project used the [License View](http://reboot.fcc.gov/license-view/) database, which harmonized the schema of the various parts of the [Universal Licensing System](http://wireless.fcc.gov/uls/index.htm?job=transaction&page=weekly) (ULS) into one flat CSV. The FCC no longer maintains License View. Spectrum Wrangler now reads ULS directly, which is the same data with more detail and no denormalization: a license header, its licensee, its locations, its antennas, its frequency assignments, and its emissions all arrive as separate related records.
+
+The FCC snapshot published August 23, 2026 holds 5.2M licenses, 2.8M of them active, across 129 radio services — 6.6M licensee records, 3.3M transmitter locations (2.4M with usable coordinates), and 11.4M frequency assignments. Your own numbers will differ; `status` reports what you actually loaded.
+
+# How to use
+* Install it: `uv tool install spectrum-wrangler`, `pipx install spectrum-wrangler`, or `pip install spectrum-wrangler`. Python 3.11 or newer is the only requirement, and a clone of this repository works identically without installing (see below).
+* Run `spectrum-wrangler refresh` to download and index every current weekly archive.
+* Run `spectrum-wrangler refresh --archive paging` first if you want a small database to try.
+* Query it with `spectrum-wrangler <command>`, or point an AI agent at it (see below).
+* Read [the querying guide](docs/QUERYING.md) for how the records join and what to watch out for.
+
+The complete download is about 1.2 GB compressed and produces a database of roughly 23 GB once every source field is retained and indexed. Downloads are cached and content-hashed, so re-running `refresh` re-imports only what actually changed. Use `--normalized-only` for a much smaller database when you do not need every raw FCC field.
+
+The database, download cache, and provenance manifest live in a per-user data directory: `~/Library/Application Support/spectrum-wrangler` on macOS, `$XDG_DATA_HOME/spectrum-wrangler` (usually under `~/.local/share`) on Linux, `%LOCALAPPDATA%\spectrum-wrangler` on Windows. A database already built under `./data` in a repository checkout keeps being used, and `--database` or the `SPECTRUM_WRANGLER_DB` environment variable points anywhere else.
+
+Generated archives and databases stay out of Git. [data/source-manifest.json](data/source-manifest.json) is committed instead, and records each archive's official URL, FCC publication time, retrieval time, ETag, SHA-256, byte size, record counts, and any observed schema drift.
+
+# Do I need Docker?
+No. A project to Dockerize `spectrum-wrangler` can be found [here](https://github.com/brannondorsey/spectrum-wrangler-docker), and it made sense when this tool required a PostgreSQL server with the PostGIS extension installed and configured. That was the hard part, and a container solved it.
+
+Version 0.3 has no services and no dependencies, so a container would add a step rather than remove one — and the expensive artifact is the 23 GB database, which has to live on your disk either way. The Docker wrapper targets the retired PostgreSQL/License View workflow and is kept as a historical reference, not the current path.
+
+# Requirements
+* **Python 3.11 or newer, and nothing else.** SQLite, FTS5, and RTree ship with Python. PostgreSQL, PostGIS, and Docker are no longer needed.
+* **Disk: about 25 GB** for a full load — a 23 GB database plus a 1.25 GB download cache. `--normalized-only` cuts the database to roughly a third of that.
+* **Memory: under 600 MB.** The importer streams, so memory tracks the widest record rather than the size of the archive. Measured peaks: 77 MB for the smallest archive, 537 MB for the largest (35.4M records). An 8 GB laptop is comfortable.
+* **Time: roughly 15 minutes** of import for the full set on a modern laptop once downloaded, plus the download itself. Importing runs at about 130,000 records per second.
+
+Install it from PyPI:
+
+    uv tool install spectrum-wrangler        # or: pipx install spectrum-wrangler
+
+Or clone the repository and run it in place, no install step at all:
+
+    git clone https://github.com/marcdacosta/spectrum-wrangler.git
+    cd spectrum-wrangler
+    python3 -m spectrum_wrangler sources
+
+The two are the same program: `python3 -m spectrum_wrangler` from a checkout accepts every command written as `spectrum-wrangler` in the examples here.
+
+## Start small
+One archive is enough to learn the shape of the data, and the smallest is a four-second import:
+
+| Archive | Download | Records | Database |
+|---|---|---|---|
+| `--archive paging` | 6.5 MB | 750K | 142 MB |
+| `--archive amateur` | 198 MB | 10.5M | 2.3 GB (710 MB with `--normalized-only`) |
+| everything | 1.25 GB | 105.9M | 23 GB |
+
+Run `spectrum-wrangler sources` to see the full list of 14 archives.
+
+# Examples
+Example raw data extract from the original License View publication can be found in `sample-fcc.csv`. [The querying guide](docs/QUERYING.md) has worked investigations and the join semantics behind these.
+
+## Query to check what is loaded and how current it is
+
+    spectrum-wrangler status
+
+## Look up a call sign
+
+    spectrum-wrangler callsign W1AW
+
+    [
+      {
+        "callsign": "W1AW",
+        "display_name": "ARRL HQ OPERATORS CLUB",
+        "radio_service_code": "HA",
+        "state": "CT",
+        "license_status": "A",
+        "grant_date": "12/08/2020",
+        "expired_date": "02/26/2031",
+        "unique_system_id": 780866
+      }
+    ]
+
+## Ranged queries
+Query to search within 1000m of [40.7253319,-74.0076834](https://www.google.com/maps/search/40.7253319,-74.0076834?sa=X&ved=0ahUKEwjj06n16p_WAhWR8oMKHUMfA9oQ8gEIJzAA) and export the results:
+
+    spectrum-wrangler --format csv \
+      nearby 40.7253319 -74.0076834 --radius-km 1 > /tmp/antennas.csv
+
+Or in SQL, against the same database:
+
+    spectrum-wrangler sql \
+      "SELECT radio_service_code, count(*) AS licenses
+         FROM licenses GROUP BY 1 ORDER BY 2 DESC"
+
+# How agents use it
+Spectrum Wrangler is one CLI, used the same way by a person and by an agent. Every operation is declared once in the code, so the commands, the help text, and the machine-readable manifest cannot drift apart.
+
+    spectrum-wrangler capabilities
+
+`capabilities` describes every command, argument, output format, and exit code as JSON, so an agent can discover the whole surface in one call. Output adapts to the consumer — a table on a terminal, JSON when piped — and `--format table|json|ndjson|csv` always wins.
+
+    # everything about one licence, in one call
+    spectrum-wrangler license --callsign W1AW
+
+    # who else transmits from this building
+    spectrum-wrangler nearby 40.748444 -73.985694 --radius-km 0.1
+
+    # who holds a band, and who holds a name
+    spectrum-wrangler band 462 468 --group-by licensee
+    spectrum-wrangler organization --name "NEW YORK CITY POLICE"
+
+    # what expires this quarter, as CSV
+    spectrum-wrangler --format csv expirations --start 2026-10-01 --end 2026-12-31 --state NY
+
+Exit codes separate the cases a program must tell apart: `0` success, `1` understood but nothing matched, `2` bad request. Errors are JSON on stderr in machine formats. The `spectrum-wrangler-agent` entry point is the same CLI with output pinned to JSON whether or not it is attached to a terminal, and [AGENTS.md](AGENTS.md) is the in-repository briefing for agents working on or with the code.
+
+The CLI is read-only and enforces it at the SQLite connection rather than by convention. Custom SQL accepts one `SELECT`, `WITH`, or `EXPLAIN`, runs with a timeout, and returns at most 1,000 rows.
+
+## The skill
+[`skills/spectrum-wrangler/SKILL.md`](skills/spectrum-wrangler/SKILL.md) is the companion to the CLI, and it carries the part a tool schema cannot: what will make you confidently wrong about this dataset. Joining on `unique_system_id` alone turns one call sign into 1.8 million rows. Dates are `MM/DD/YYYY` text, so sorting them sorts by month. Counting what one organization holds has no exact answer. Point any agent at it.
+
+To install the skill for Claude Code, link it into your skills directory:
+
+    mkdir -p ~/.claude/skills
+    ln -s "$PWD/skills/spectrum-wrangler" ~/.claude/skills/spectrum-wrangler
+
+Earlier versions shipped a Model Context Protocol server. It was removed in favour of the CLI plus the skill: it duplicated every parameter declaration in a third place, needed a client restart to pick up changes, and had nowhere to put the judgment that actually matters here. If you need to query this from a client with no shell, that is the one thing the MCP server did better.
+
+# Current state
+Version 0.3 is a rewrite of the ingest and query layers. The purpose has not changed: get the public licensing dataset, clean its geographic fields, index it, and make it useful to query.
+
+What changed:
+
+* **Source.** FCC License View is retired; ULS weekly complete files replace it. See [the 2026 publication research](docs/RESEARCH-2026.md).
+* **Storage.** PostgreSQL and PostGIS are gone. SQLite with FTS5 and an RTree index does the same work with nothing to install. See [migration notes](docs/MIGRATION.md).
+* **Fidelity.** Every field of all 89 documented ULS record layouts is preserved in `raw_*` tables; the common ones are also normalized into typed, indexed tables (`licenses`, `entities`, `locations`, `antennas`, `frequencies`, `emissions`, `amateur`). Raw records join on `unique_system_identifier`. Undocumented trailing fields are kept in `extra_fields_json` rather than dropped, so FCC publication drift is visible instead of silent.
+* **Provenance.** Every import records where the file came from, when the FCC published it, when it was fetched, and its hash.
+
+Known limits:
+
+* Full-power AM/FM/TV station licensing moved to the FCC's separate Licensing and Management System (LMS) and is **not** in this database. ULS still carries broadcast auxiliary. LMS is deliberately not mixed in.
+* Rebuilds use the weekly complete files. Daily transaction deltas are not applied yet.
+* Coordinates come from the FCC as published; about a quarter of location records have no usable coordinate.
+* `load.py` and `sample-fcc.csv` are the original Python 2 loader and a License View sample. They are kept for reference and do not run against current data.
+
+## What this data contains
+Everything here is FCC public record, published for anyone to download, and Spectrum Wrangler queries all of it. There is no column gate: the database is a local file you can open with `sqlite3`, so a gate would restrict nothing while breaking legitimate lookups.
+
+Be aware of what that means in bulk. Entity records carry licensee contact details, and the amateur archive alone is roughly 1.7 million individual licensees with names and mailing addresses. Public record, but a bulk extract of private individuals is a different thing from a single lookup — worth a thought before you republish one.
+
+Read-only *is* enforced, at the SQLite connection rather than by convention: a write is refused by the engine.
+
+# Development
+
+    python3 -m unittest discover -s tests -v
+    python3 -m compileall -q spectrum_wrangler tests
+
+The suite is hermetic — no network, no fixtures on disk, under a second — and CI runs it on Python 3.11 through 3.14 across Linux, macOS, and Windows.
+
+The only build dependency is the pinned `setuptools==80.9.0`. The application itself has no third-party runtime dependencies, and [the dependency policy](docs/DEPENDENCIES.md) explains what it takes to add one.

spectrum_wrangler-0.3.0/docs/DEPENDENCIES.md

@@ -0,0 +1,23 @@
+# Dependency policy
+
+Spectrum Wrangler 0.3 has no third-party runtime dependencies. Its build backend
+is pinned to `setuptools==80.9.0`; PyPI registry metadata reports its release
+files were uploaded on 2025-05-27, well outside the repository's 14-day minimum
+age policy as checked on 2026-08-28.
+
+Adding, updating, installing, or temporarily executing a package requires first
+recording the exact version and official registry release timestamp. Existing
+lockfile installs are acceptable only when they do not change resolved versions.
+
+## Recorded release tooling
+
+The release workflow builds distributions with `build==1.5.0`; PyPI registry
+metadata reports its release files were uploaded on 2026-04-30, outside the
+14-day minimum age as checked on 2026-08-30. At that check `build==1.5.1` was
+yanked upstream and `build==1.6.0` was three days old, inside the window, so
+neither was adopted.
+
+GitHub Actions in CI are pinned by major version tag (`actions/checkout@v5`,
+`actions/setup-python@v5`, `actions/upload-artifact@v4`,
+`actions/download-artifact@v4`, `pypa/gh-action-pypi-publish@release/v1`).
+They run only in CI and never ship in a release artifact.

spectrum_wrangler-0.3.0/docs/MIGRATION.md

@@ -0,0 +1,47 @@
+# Migration from FCC License View
+
+This is for people who ran the 2017 loader. It covers what changed and what to
+expect; the evidence behind the decision — the FCC's response, the ULS
+publication format, the LMS boundary — is recorded once in
+[the 2026 research notes](RESEARCH-2026.md) rather than repeated here.
+
+The 2017 loader downloaded one `fcc_lic_vw.csv`, created a hand-written Postgres table, converted degree/minute/second coordinates in PL/pgSQL, and required PostGIS. That path cannot be repaired by changing its URL: the FCC has confirmed License View is no longer maintained.
+
+Version 0.3 therefore changes both the publication and storage model.
+
+| Legacy behavior | 2026 behavior |
+|---|---|
+| Retired License View CSV | Live FCC ULS weekly license archives |
+| One denormalized table | 89 lossless raw record tables plus normalized query tables |
+| Python 2, `requests`, `psycopg2` | Python 3.11+ standard library only |
+| Config embedded in `load.py` | CLI flags and stable defaults |
+| Required Postgres/PostGIS | Portable SQLite with FTS5 and RTree |
+| Destructive table replacement | Per-archive replacement with content-hash reuse |
+| No source provenance | URL, timestamps, ETag, SHA-256, bytes, and counts |
+| Human SQL only | Structured commands, bounded SQL, and machine-readable output |
+
+## Compatibility boundaries
+
+The normalized `licenses`, `entities`, `locations`, `antennas`, `frequencies`, and `emissions` tables cover the same practical query families as the old `fcclicenses` table. Their identifiers are intentionally closer to the official ULS names.
+
+The old flat file could multiply one license across locations, antennas, frequencies, and emissions. The new model keeps those relations separate, so queries express the intended cardinality rather than inheriting accidental row multiplication.
+
+Coordinates are converted to decimal WGS84 values during normalization and the original DMS components remain in `raw_lo`. RTree supplies the bounding-box phase of proximity queries and a Haversine calculation supplies exact great-circle distance; no spatial extension is required.
+
+## Raw fidelity and schema drift
+
+The importer keeps every field described by the FCC's published record definitions; the checked-in `uls_schema.json` defines all 89 layouts. Raw values remain text so identifiers, leading zeroes, source date formats, and precision are not altered.
+
+An FCC record with fewer fields is padded with nulls. Newly added trailing fields are serialized in `extra_fields_json`, making publication drift visible and recoverable instead of shifting columns or dropping data. Completely new record types go to `raw_unknown` until their official definitions are reviewed. [The research notes](RESEARCH-2026.md) give a worked example of drift already observed in the published archives.
+
+Some FCC free-text fields span unescaped physical lines. The logical-record parser recognizes the member's two-character record prefix, joins continuation text back into the preceding field with newlines, and skips physical blank separators. `raw_parser_version` in source provenance prevents an older line-oriented import from being mistaken for a current verified snapshot.
+
+## Broadcast limitation
+
+License View combined more than ULS. Its sample in this repository includes CDBS broadcast stations; the modern replacement for those station records is LMS, not ULS. ULS still contains broadcast auxiliary services (`l_LMbcast.zip`).
+
+If you previously queried this database for full-power AM/FM/TV stations, that is the one query family version 0.3 cannot answer. It is labelled rather than silently approximated, and LMS is not mixed in.
+
+## Legacy files
+
+`load.py` remains available for historical comparison. It is Python 2 code, points at the retired publication, and must not be used for a current refresh. `sample-fcc.csv` is likewise a legacy License View sample rather than a current test fixture.