refsource 0.1.0__tar.gz

refsource-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 referencesource.org
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

refsource-0.1.0/PKG-INFO

@@ -0,0 +1,228 @@
+Metadata-Version: 2.4
+Name: refsource
+Version: 0.1.0
+Summary: Look up reference data that carries its own source: every value comes with the URL it was read from and a verbatim quote from that page.
+Author: referencesource.org
+License: MIT
+Project-URL: Homepage, https://referencesource.org
+Project-URL: Catalogue, https://referencesource.org/catalog.json
+Project-URL: Method, https://referencesource.org/method/
+Keywords: reference-data,citations,provenance,verification,fact-check,regulations,compliance,lookup,datasets,source-quote
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# refsource
+
+**Reference data lookups where every value comes back with the URL it was read
+from and a verbatim quote from that page.**
+
+```bash
+pip install refsource
+```
+
+```python
+import refsource
+
+rows = refsource.lookup("conforming-loan-limits", state="AL",
+                        county_name="AUTAUGA COUNTY")
+
+rows[0]["limit_1_unit"]
+# '$832,750'
+
+rows[0]["limit_1_unit"].source
+# 'https://www.fhfa.gov/document/d/cll/fullcountyloanlimitlist2026_hera-based_final_flat.csv'
+
+rows[0]["limit_1_unit"].quote
+# '01,001,AUTAUGA COUNTY,AL,33860,"$832,750 ","$1,066,250 ",...'
+```
+
+A value is a plain string everywhere a string is expected, and it carries its
+own citation. That is the whole idea: an answer you can check beats an answer
+you have to trust.
+
+191 datasets, 73,380 records, from
+**[referencesource.org](https://referencesource.org)** — regulatory thresholds
+and deadlines, licensing rules by US state, version and end-of-support
+calendars, certification registers, exposure limits, insurance minimums,
+standards supersessions. Each one states its coverage, its sources and the
+date it was last checked.
+
+## Why it exists
+
+If you ask a language model for a county loan limit, a state's minimum liability
+cover, or when an API model shuts off, you usually get a confident answer with
+no way to check it. In our own measurement of 271 such questions, 19% came back
+confident and **wrong**.
+
+This package answers the same questions with the source attached, so the
+checking step is available rather than skipped — in your code, in a notebook,
+or in whatever an agent is doing on your behalf.
+
+## Usage
+
+### Find a dataset (no network — the catalogue ships with the package)
+
+```python
+refsource.datasets("loan limit")
+# [<Dataset conforming-loan-limits (3235 records, verified 2026-08-10)>]
+
+refsource.fields("auto-insurance-minimums")
+# ['state', 'bodily_injury_per_person', 'bodily_injury_per_accident', ...]
+```
+
+### Look records up
+
+```python
+refsource.lookup("auto-insurance-minimums", state="TX")
+refsource.lookup("auto-insurance-minimums", state=["TX", "NM", "AZ"])
+refsource.search("ai-model-deprecation-and-retirement", "gpt-4", limit=5)
+refsource.get("conforming-loan-limits", "01001")
+```
+
+Matching is case-insensitive and forgiving about spacing and punctuation, so
+`"Autauga County"` finds `"AUTAUGA COUNTY"`. A filter naming a field that does
+not exist raises `NoSuchField` instead of returning an empty list — an empty
+result from a typo looks exactly like an empty result from a real absence, and
+one of those two is a wrong answer.
+
+Datasets do not agree on how to spell things — one writes a state as `TX`, the
+next as `Texas`, because each says what its source says. When a filter matches
+nothing, ask what is actually there:
+
+```python
+refsource.dataset("auto-insurance-minimums").values_of("state", limit=5)
+# ['Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California']
+```
+
+### Read the provenance
+
+```python
+rec = refsource.lookup("conforming-loan-limits", fips_full="01001")[0]
+
+rec.source_url      # the page or file this record was read from
+rec.source_quote    # the passage that states it, verbatim
+rec.url             # the published record's own page
+rec.verified        # '2026-08-10'
+rec.stale_after     # '2027-08-10'
+print(rec.cite())   # a citation you can paste somewhere
+
+v = rec["fha_limit_1_unit"]
+v.source            # apps.hud.gov — a SECOND publisher's value,
+                    # kept with its own source rather than folded in
+v.confirmed         # the value was found word-for-word in the quote
+v.derived           # our own reading rather than the page's words
+v.disagreement      # other sources' versions of this same field, if any
+```
+
+Three things are deliberately visible rather than smoothed over:
+
+- **Fields from a second publisher keep that publisher's URL and quote.** The
+  FHFA conforming limit and the HUD/FHA limit sit in the same record; each cites
+  the file it came from. Attributing one to the other would be a false citation.
+- **`derived` marks our reading**, not the page's words — a state name we
+  normalised, an identifier we assembled.
+- **`disagreement` is not hidden.** Where two sources state a field differently,
+  you get both, each with its own quote, and you decide.
+
+### Staleness
+
+Every dataset carries the date by which it should be re-checked. Read the
+records of one that is past it and you get a `StaleDataWarning` naming the page
+with the current copy. Nothing is silently served as fresh.
+
+### Offline, caching, mirrors
+
+Records are fetched on first use and cached (24h by default; the catalogue
+itself needs no network at all).
+
+```python
+refsource.configure(
+    cache_ttl=86400,        # 0 = always re-fetch, negative = never expire
+    offline=True,           # cache only, never open a connection
+    strict=True,            # refuse a bundle that no longer matches its pinned hash
+    base_url="file:///path/to/site",   # a local copy or a mirror
+)
+```
+
+Every one of those has an environment variable too: `REFSOURCE_CACHE`,
+`REFSOURCE_CACHE_TTL`, `REFSOURCE_OFFLINE`, `REFSOURCE_STRICT`,
+`REFSOURCE_BASE_URL`, `REFSOURCE_TIMEOUT`.
+
+**Hash pinning.** The package holds the SHA-256 of every bundle as of the
+release. If a fetched bundle differs, the dataset was re-verified upstream since
+this version was cut — you get the live copy plus a `ChangedUpstreamWarning`, or
+an `IntegrityError` under `strict=True`. The point is that a change is visible
+rather than silent.
+
+## Command line
+
+```console
+$ refsource datasets loan limit
+$ refsource fields conforming-loan-limits
+$ refsource lookup conforming-loan-limits fips_full=01001
+$ refsource search ai-model-deprecation-and-retirement gpt-4 --limit 5
+$ refsource show conforming-loan-limits 01001 --json
+```
+
+```
+01001
+  county_name: AUTAUGA COUNTY
+  state: AL
+  limit_1_unit: $832,750
+  fha_limit_1_unit: 541,287
+      from https://apps.hud.gov/pub/chums/cy2026-forward-limits.txt
+      quoted: "3386000000MONTGOMERY, AL 203B S02200000541287069305008377001041125AL001…"
+  source: https://www.fhfa.gov/document/d/cll/fullcountyloanlimitlist2026_hera-based_final_flat.csv
+  quoted: "01,001,AUTAUGA COUNTY,AL,33860,"$832,750 ","$1,066,250 ","$1,288,800 ","$1,601,750 ""
+  page:   https://referencesource.org/conforming-loan-limits/01001/
+  verified 2026-08-10
+```
+
+## What is in the catalogue
+
+A sample of the 191 datasets:
+
+| dataset | what it answers |
+|---|---|
+| `conforming-loan-limits` | the FHFA and FHA loan limits for every US county |
+| `auto-insurance-minimums` | minimum liability cover by US state |
+| `ai-model-deprecation-and-retirement` | when an API model was deprecated and what replaces it |
+| `software-end-of-support` | end-of-support dates from each vendor's own page |
+| `iso-standard-supersessions` | what withdrew or replaced an ISO standard |
+| `workplace-exposure-limits` | OSHA and Cal/OSHA permissible exposure limits |
+| `fips-140-module-validation-status` | whether a cryptographic module's validation is still active |
+| `drinking-water-contaminant-limits` | EPA maximum contaminant levels |
+
+`refsource.datasets()` lists them all, offline.
+
+## Data, licensing and accuracy
+
+The records are facts with attribution, not reproductions. Each dataset states
+its own licence position and links the source it was read from; the package code
+is MIT. Where a source's terms forbid reuse, the dataset is not published at
+all.
+
+No value is ever supplied by this package or by a model — if a source does not
+state something, the row is omitted rather than guessed. Where you need to be
+sure, the quote and the URL are right there: check it.
+
+Found something wrong? That is the one thing worth reporting —
+https://referencesource.org/ has the contact and the method behind every
+dataset.
+
+## Related
+
+- **MCP server** — the same catalogue as tools for AI assistants:
+  `npx referencesource-mcp`, or add
+  `https://referencesource.org/mcp` to any MCP client.
+- **Bulk data** — every dataset publishes `data.json` at its own URL, and the
+  catalogue is at https://referencesource.org/catalog.json.

refsource-0.1.0/README.md

@@ -0,0 +1,206 @@
+# refsource
+
+**Reference data lookups where every value comes back with the URL it was read
+from and a verbatim quote from that page.**
+
+```bash
+pip install refsource
+```
+
+```python
+import refsource
+
+rows = refsource.lookup("conforming-loan-limits", state="AL",
+                        county_name="AUTAUGA COUNTY")
+
+rows[0]["limit_1_unit"]
+# '$832,750'
+
+rows[0]["limit_1_unit"].source
+# 'https://www.fhfa.gov/document/d/cll/fullcountyloanlimitlist2026_hera-based_final_flat.csv'
+
+rows[0]["limit_1_unit"].quote
+# '01,001,AUTAUGA COUNTY,AL,33860,"$832,750 ","$1,066,250 ",...'
+```
+
+A value is a plain string everywhere a string is expected, and it carries its
+own citation. That is the whole idea: an answer you can check beats an answer
+you have to trust.
+
+191 datasets, 73,380 records, from
+**[referencesource.org](https://referencesource.org)** — regulatory thresholds
+and deadlines, licensing rules by US state, version and end-of-support
+calendars, certification registers, exposure limits, insurance minimums,
+standards supersessions. Each one states its coverage, its sources and the
+date it was last checked.
+
+## Why it exists
+
+If you ask a language model for a county loan limit, a state's minimum liability
+cover, or when an API model shuts off, you usually get a confident answer with
+no way to check it. In our own measurement of 271 such questions, 19% came back
+confident and **wrong**.
+
+This package answers the same questions with the source attached, so the
+checking step is available rather than skipped — in your code, in a notebook,
+or in whatever an agent is doing on your behalf.
+
+## Usage
+
+### Find a dataset (no network — the catalogue ships with the package)
+
+```python
+refsource.datasets("loan limit")
+# [<Dataset conforming-loan-limits (3235 records, verified 2026-08-10)>]
+
+refsource.fields("auto-insurance-minimums")
+# ['state', 'bodily_injury_per_person', 'bodily_injury_per_accident', ...]
+```
+
+### Look records up
+
+```python
+refsource.lookup("auto-insurance-minimums", state="TX")
+refsource.lookup("auto-insurance-minimums", state=["TX", "NM", "AZ"])
+refsource.search("ai-model-deprecation-and-retirement", "gpt-4", limit=5)
+refsource.get("conforming-loan-limits", "01001")
+```
+
+Matching is case-insensitive and forgiving about spacing and punctuation, so
+`"Autauga County"` finds `"AUTAUGA COUNTY"`. A filter naming a field that does
+not exist raises `NoSuchField` instead of returning an empty list — an empty
+result from a typo looks exactly like an empty result from a real absence, and
+one of those two is a wrong answer.
+
+Datasets do not agree on how to spell things — one writes a state as `TX`, the
+next as `Texas`, because each says what its source says. When a filter matches
+nothing, ask what is actually there:
+
+```python
+refsource.dataset("auto-insurance-minimums").values_of("state", limit=5)
+# ['Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California']
+```
+
+### Read the provenance
+
+```python
+rec = refsource.lookup("conforming-loan-limits", fips_full="01001")[0]
+
+rec.source_url      # the page or file this record was read from
+rec.source_quote    # the passage that states it, verbatim
+rec.url             # the published record's own page
+rec.verified        # '2026-08-10'
+rec.stale_after     # '2027-08-10'
+print(rec.cite())   # a citation you can paste somewhere
+
+v = rec["fha_limit_1_unit"]
+v.source            # apps.hud.gov — a SECOND publisher's value,
+                    # kept with its own source rather than folded in
+v.confirmed         # the value was found word-for-word in the quote
+v.derived           # our own reading rather than the page's words
+v.disagreement      # other sources' versions of this same field, if any
+```
+
+Three things are deliberately visible rather than smoothed over:
+
+- **Fields from a second publisher keep that publisher's URL and quote.** The
+  FHFA conforming limit and the HUD/FHA limit sit in the same record; each cites
+  the file it came from. Attributing one to the other would be a false citation.
+- **`derived` marks our reading**, not the page's words — a state name we
+  normalised, an identifier we assembled.
+- **`disagreement` is not hidden.** Where two sources state a field differently,
+  you get both, each with its own quote, and you decide.
+
+### Staleness
+
+Every dataset carries the date by which it should be re-checked. Read the
+records of one that is past it and you get a `StaleDataWarning` naming the page
+with the current copy. Nothing is silently served as fresh.
+
+### Offline, caching, mirrors
+
+Records are fetched on first use and cached (24h by default; the catalogue
+itself needs no network at all).
+
+```python
+refsource.configure(
+    cache_ttl=86400,        # 0 = always re-fetch, negative = never expire
+    offline=True,           # cache only, never open a connection
+    strict=True,            # refuse a bundle that no longer matches its pinned hash
+    base_url="file:///path/to/site",   # a local copy or a mirror
+)
+```
+
+Every one of those has an environment variable too: `REFSOURCE_CACHE`,
+`REFSOURCE_CACHE_TTL`, `REFSOURCE_OFFLINE`, `REFSOURCE_STRICT`,
+`REFSOURCE_BASE_URL`, `REFSOURCE_TIMEOUT`.
+
+**Hash pinning.** The package holds the SHA-256 of every bundle as of the
+release. If a fetched bundle differs, the dataset was re-verified upstream since
+this version was cut — you get the live copy plus a `ChangedUpstreamWarning`, or
+an `IntegrityError` under `strict=True`. The point is that a change is visible
+rather than silent.
+
+## Command line
+
+```console
+$ refsource datasets loan limit
+$ refsource fields conforming-loan-limits
+$ refsource lookup conforming-loan-limits fips_full=01001
+$ refsource search ai-model-deprecation-and-retirement gpt-4 --limit 5
+$ refsource show conforming-loan-limits 01001 --json
+```
+
+```
+01001
+  county_name: AUTAUGA COUNTY
+  state: AL
+  limit_1_unit: $832,750
+  fha_limit_1_unit: 541,287
+      from https://apps.hud.gov/pub/chums/cy2026-forward-limits.txt
+      quoted: "3386000000MONTGOMERY, AL 203B S02200000541287069305008377001041125AL001…"
+  source: https://www.fhfa.gov/document/d/cll/fullcountyloanlimitlist2026_hera-based_final_flat.csv
+  quoted: "01,001,AUTAUGA COUNTY,AL,33860,"$832,750 ","$1,066,250 ","$1,288,800 ","$1,601,750 ""
+  page:   https://referencesource.org/conforming-loan-limits/01001/
+  verified 2026-08-10
+```
+
+## What is in the catalogue
+
+A sample of the 191 datasets:
+
+| dataset | what it answers |
+|---|---|
+| `conforming-loan-limits` | the FHFA and FHA loan limits for every US county |
+| `auto-insurance-minimums` | minimum liability cover by US state |
+| `ai-model-deprecation-and-retirement` | when an API model was deprecated and what replaces it |
+| `software-end-of-support` | end-of-support dates from each vendor's own page |
+| `iso-standard-supersessions` | what withdrew or replaced an ISO standard |
+| `workplace-exposure-limits` | OSHA and Cal/OSHA permissible exposure limits |
+| `fips-140-module-validation-status` | whether a cryptographic module's validation is still active |
+| `drinking-water-contaminant-limits` | EPA maximum contaminant levels |
+
+`refsource.datasets()` lists them all, offline.
+
+## Data, licensing and accuracy
+
+The records are facts with attribution, not reproductions. Each dataset states
+its own licence position and links the source it was read from; the package code
+is MIT. Where a source's terms forbid reuse, the dataset is not published at
+all.
+
+No value is ever supplied by this package or by a model — if a source does not
+state something, the row is omitted rather than guessed. Where you need to be
+sure, the quote and the URL are right there: check it.
+
+Found something wrong? That is the one thing worth reporting —
+https://referencesource.org/ has the contact and the method behind every
+dataset.
+
+## Related
+
+- **MCP server** — the same catalogue as tools for AI assistants:
+  `npx referencesource-mcp`, or add
+  `https://referencesource.org/mcp` to any MCP client.
+- **Bulk data** — every dataset publishes `data.json` at its own URL, and the
+  catalogue is at https://referencesource.org/catalog.json.

refsource-0.1.0/pyproject.toml

@@ -0,0 +1,43 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "refsource"
+version = "0.1.0"
+description = "Look up reference data that carries its own source: every value comes with the URL it was read from and a verbatim quote from that page."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+authors = [{ name = "referencesource.org" }]
+keywords = [
+    "reference-data", "citations", "provenance", "verification", "fact-check",
+    "regulations", "compliance", "lookup", "datasets", "source-quote",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Information Analysis",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Typing :: Typed",
+]
+dependencies = []
+
+[project.urls]
+Homepage = "https://referencesource.org"
+Catalogue = "https://referencesource.org/catalog.json"
+Method = "https://referencesource.org/method/"
+
+[project.scripts]
+refsource = "refsource.__main__:main"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+refsource = ["manifest.json", "py.typed"]

refsource-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+