erdos-problems 0.2.9 → 0.3.0

package/README.md

@@ -33,11 +33,24 @@ A short example from the first minute:
 
 ## What It Includes
 
-- a bundled snapshot of the Erdős problem atlas
+- a bundled external atlas import snapshot for provenance and seeding
 - local dossiers and workspace bundles
 - separate surfaces for public status, local route state, and verification records
 - pack-specific views where this repo already has enough structure to support them
 - an ORP-based workflow for claims, checkpoints, and run artifacts
+- a paper writer bundle mode for public-safe, citation-safe, agent-friendly drafting
+
+## Repo, npm, and External Imports
+
+- `SproutSeeds/erdos-problems` is the canonical public home for this project's atlas, dossiers, packs, and contribution workflow
+- the npm package is the installable CLI plus the bundled import snapshot, packaged dossiers, and packaged pack assets
+- the GitHub repo is the broader public collaboration space for dossiers, pack packets, docs, issues, discussions, PRs, and future deep research bundles
+- `.erdos/` is local workspace state created by the CLI; it is not canonical repo truth
+- `erdos import show` reports whether you are using the bundled import snapshot or a workspace-local refreshed import snapshot
+- `erdos import sync` currently refreshes an external atlas import snapshot from `teorth/erdosproblems` without mutating the canonical local dossier layer
+- `erdos import sync --write-package-snapshot` is the maintainer path for intentionally updating the bundled import snapshot in this repo
+- repo-only deep-research directories such as `research/`, `formal/`, `paper/`, `imports/`, and `analysis/` are intentionally kept out of npm by staying outside the `package.json` `files` list
+- repo-only public research can live in this GitHub repo without shipping in the npm tarball; see the contribution guide: https://github.com/SproutSeeds/erdos-problems/blob/main/CONTRIBUTING.md
 
 ## Start In 60 Seconds
 
@@ -64,6 +77,13 @@ erdos problem show 25
 erdos checkpoints sync
 ```
 
+Initialize or resume a paper bundle:
+
+```bash
+erdos paper init 857
+erdos paper show 857
+```
+
 Archive a solved problem:
 
 ```bash
@@ -75,7 +95,7 @@ erdos archive scaffold 1008 --json
 
 | Surface | Coverage |
 | --- | --- |
-| Bundled upstream atlas | `1183` problems |
+| Bundled external atlas snapshot | `1183` problems |
 | Native packaged dossiers | `18` |
 | Sunflower pack | `20`, `536`, `856`, `857` |
 | Number-theory pack | `1`, `2` |
@@ -130,7 +150,7 @@ erdos seed problem 25 --cluster number-theory
 ```
 
 What this does:
-- pulls upstream public metadata into a workspace bundle
+- pulls external public metadata into a workspace bundle
 - includes live site review and an agent websearch brief by default
 - seeds a local dossier into `.erdos/seeded-problems/<id>/`
 - syncs state, checkpoints, and ORP automatically
@@ -151,6 +171,19 @@ erdos maintainer review problem 25 --from-pull .erdos/pulls/25
 erdos maintainer seed problem 25 --from-pull .erdos/pulls/25 --cluster number-theory
 ```
 
+### 5. Initialize a paper writer bundle
+
+```bash
+erdos paper init 857
+erdos paper show 857
+```
+
+What this does:
+- creates or resumes a paper bundle for the problem
+- writes machine-readable public evidence and section indexes
+- scaffolds claim-safe section drafts, a citation ledger, and a privacy review
+- defaults to `paper/problems/<id>/` when run inside this repo and to `.erdos/papers/<id>/` elsewhere
+
 ## Packs
 
 ### Sunflower Pack
@@ -223,7 +256,7 @@ Design rule:
 
 If you are using Codex, Claude Code, or another coding agent, the package provides:
 - canonical dossier files
-- upstream provenance
+- external import provenance when available
 - pack-aware context where it exists
 - `.erdos/` workspace state
 - ORP templates and protocol guidance
@@ -237,7 +270,7 @@ Useful commands for agent workflows:
 erdos problem artifacts 857 --json
 erdos workspace show --json
 erdos pull literature 857 --include-crossref --include-openalex --json
-erdos upstream drift 857 --json
+erdos import drift 857 --json
 ```
 
 ## ORP And Truth Hygiene
@@ -250,8 +283,8 @@ erdos upstream drift 857 --json
 - `templates/FAILED_TOPIC.md`
 
 The package keeps these layers separate:
-- upstream public truth
-- local canonical dossier truth
+- canonical repo truth
+- external imported public truth
 - workspace bundle truth
 - pack-specific route / checkpoint / run truth
 
@@ -284,13 +317,14 @@ erdos maintainer review problem 25 --from-pull .erdos/pulls/25
 erdos archive scaffold 1008 --json
 ```
 
-## Canonical Sources
+## Canonical Home And External Inputs
 
+- canonical repo: <https://github.com/SproutSeeds/erdos-problems>
 - local dossier truth: `problems/<id>/problem.yaml`
-- bundled upstream snapshot: `data/upstream/erdosproblems/`
-- workspace upstream snapshot: `.erdos/upstream/erdosproblems/`
-- live upstream repo: <https://github.com/teorth/erdosproblems>
-- live public site: <https://www.erdosproblems.com/>
+- bundled external import snapshot: `data/upstream/erdosproblems/`
+- workspace external import snapshot: `.erdos/upstream/erdosproblems/`
+- external atlas import repo: <https://github.com/teorth/erdosproblems>
+- external public site: <https://www.erdosproblems.com/>
 
 ## Docs
 
@@ -311,7 +345,7 @@ If you are preparing a public post, start here:
 
 ### What is the difference between this and erdosproblems.com?
 
-`erdosproblems.com` is a public problem atlas. `erdos-problems` uses that public record, but adds local dossiers, workspace scaffolding, checkpoints, and pack-specific views for the problems that already have more structure here.
+`erdosproblems.com` is an external public problem source. `erdos-problems` imports from public sources when helpful, but keeps its canonical dossiers, packs, workflow, and contribution surface in this repo.
 
 ### Can I use this with Codex, Claude Code, or another agent?
 

package/data/upstream/erdosproblems/SYNC_MANIFEST.json

@@ -1,9 +1,9 @@
 {
-  "upstream_repo": "https://github.com/teorth/erdosproblems",
-  "source_url": "https://raw.githubusercontent.com/teorth/erdosproblems/master/data/problems.yaml",
-  "source_file": "data/problems.yaml",
+  "external_repo": "https://github.com/teorth/erdosproblems",
+  "external_source_url": "https://raw.githubusercontent.com/teorth/erdosproblems/master/data/problems.yaml",
+  "external_source_file": "data/problems.yaml",
+  "imported_commit": "cd91711d143184e090e5336e3165be50abfaaaf3",
   "fetched_at": "2026-03-25T19:24:57.377Z",
-  "upstream_commit": "cd91711d143184e090e5336e3165be50abfaaaf3",
   "raw_sha256": "e2d161ffe3866275704bb18a268f9417f32d8a92a0af19fa119a856445cd2021",
   "entry_count": 1183
 }

package/docs/CANONICAL_REPO_MIGRATION_PLAN.md

@@ -0,0 +1,279 @@
+# Canonical Repo Migration Plan
+
+Last updated: 2026-03-30
+
+## Decision
+
+`SproutSeeds/erdos-problems` becomes the canonical public home for:
+
+- the Erdős problem atlas as presented by this project
+- canonical seeded dossiers
+- pack-specific route, board, and compute packets
+- formal artifacts, papers, and deeper research files that are intentionally public
+- contribution, issue, and pull-request workflow
+
+External sources remain valuable, but they are inputs, not authorities:
+
+- `erdosproblems.com` remains a public source surface
+- `teorth/erdosproblems` remains an import and comparison source when useful
+- `sunflower-coda` is treated as a migration source and lab origin, not the long-term public home
+
+## Why
+
+The current architecture mixes two different ownership stories:
+
+- the package, CLI, and curated public harness live in `SproutSeeds/erdos-problems`
+- the bundled atlas snapshot and several provenance fields still point at `teorth/erdosproblems` as "upstream truth"
+
+That split is workable for importing data, but confusing for users. It leaves the project with two competing authority stories:
+
+- "this repo is the home"
+- "another repo is the truth"
+
+The migration resolves that by keeping one canonical owner and treating outside sources as import feeds with explicit provenance.
+
+## Product Model After Migration
+
+### Canonical public home
+
+- GitHub repo: `SproutSeeds/erdos-problems`
+- npm package: installable CLI and packaged public harness assets
+- canonical problem records: `problems/<id>/`
+- canonical pack context: `packs/<family>/`
+- canonical public docs: `docs/`
+
+### External imports
+
+- imported public site snapshots
+- imported external atlas snapshots
+- imported public literature adapters
+
+These stay visible as provenance, but never outrank the canonical local record.
+
+### Packaging split
+
+The GitHub repo may contain much more than the npm tarball.
+
+GitHub can hold:
+
+- deep research notes
+- paper drafts and public reference bundles
+- Lean and formalization source
+- board JSON exports
+- long-form analysis artifacts
+
+The npm package should ship only what is needed for:
+
+- the CLI
+- canonical dossiers
+- pack packets
+- packaged compute packets
+- essential docs and templates
+
+## Current State
+
+Today the repo already behaves partly like the canonical home:
+
+- `erdos problem list` and `erdos problem show` are driven by local seeded dossiers under `problems/`
+- the public deep harness for `20` and `857` is already expressed in local pack files under `packs/sunflower/`
+- the GitHub repo is already the collaboration surface for issues and PRs
+
+But these parts still reflect the old authority model:
+
+- `src/upstream/sync.js` fetches and labels `teorth/erdosproblems` as the upstream repo
+- bundled manifest files record that external repo as the source of truth
+- dossier `upstream.repo` fields and references point at `teorth/erdosproblems`
+- docs describe that external repo as the upstream truth layer
+- package shipping boundaries are still broad enough that repo-only deep research would accidentally ship if dropped into currently published directories
+
+## Migration Principles
+
+1. One canonical owner
+   `SproutSeeds/erdos-problems` is the only authority users should need to trust.
+
+2. Provenance without ambiguity
+   External records stay cited, but only as imported evidence.
+
+3. Fast install, deep repo
+   npm stays lean enough for CLI installs, while GitHub can hold the full public research archive.
+
+4. Public handoff first
+   A new user should be able to pick up `20` or `857` and immediately see the live route posture, active files, and next honest move.
+
+5. Deep material on demand
+   Users should be able to pull or inspect fuller problem bundles from the repo without shipping all of that by default in npm.
+
+## Rollout Phases
+
+## Phase 1: Ownership And Provenance Cleanup
+
+Goal:
+- make the docs and metadata say what the product now means
+
+Changes:
+- rewrite docs so `SproutSeeds/erdos-problems` is described as canonical
+- replace "upstream truth" wording with "external import source" wording
+- update dossier schema language to distinguish:
+  - canonical local record
+  - imported external sources
+  - public site snapshots
+- update any visible README/help text that suggests the external repo is authoritative
+
+Acceptance:
+- a user reading the README should understand there is one canonical repo
+- no user-facing docs should describe `teorth/erdosproblems` as the authoritative owner of this package's atlas
+
+## Phase 2: Package Boundary Split
+
+Goal:
+- let GitHub hold the full public archive without forcing it into npm
+
+Changes:
+- define repo-only directories for deep research
+- tighten `package.json` `files` handling or segregate content so deep research is excluded from the tarball
+- document which directories are:
+  - canonical and shipped
+  - canonical but repo-only
+  - generated workspace state
+
+Candidate repo-only directories:
+- `research/`
+- `formal/lean/`
+- `paper/`
+- `imports/`
+- `analysis/`
+
+Acceptance:
+- `npm pack --dry-run` excludes repo-only deep research
+- GitHub still exposes the full public archive
+
+## Phase 3: Repo Structure For Full Public Research
+
+Goal:
+- move the public-facing deep material into this repo in an orderly way
+
+Changes:
+- create canonical directories for:
+  - problem-deep research bundles
+  - Lean / formal source
+  - paper drafts and publication support
+  - imported external snapshots and reconciliations
+- preserve a clear distinction between:
+  - concise user-facing handoff packets
+  - deeper research inventory
+
+Suggested layout:
+
+```text
+erdos-problems/
+  problems/<id>/
+  packs/<family>/
+  research/problems/<id>/
+  formal/lean/
+  paper/
+  imports/
+```
+
+Acceptance:
+- the repo can absorb public material from `sunflower-coda` without collapsing the package structure
+
+## Phase 4: CLI Surface For Deep Problem Bundles
+
+Goal:
+- let users access the deeper repo-held material intentionally
+
+Changes:
+- add documented repo-backed bundle concepts, for example:
+  - deep research bundle
+  - formalization bundle
+  - paper/reference bundle
+- expose them either as:
+  - repo file paths in scaffold outputs
+  - new CLI commands
+  - both
+
+Possible commands:
+- `erdos problem deep 857`
+- `erdos problem formal 857`
+- `erdos pull repo-bundle 857`
+
+Acceptance:
+- a user can start with the lightweight public packet
+- then intentionally step into the deeper public archive without guessing where things live
+
+## Phase 5: External Import And Reconciliation Workflow
+
+Goal:
+- retain harmony with outside sources without giving away canonical ownership
+
+Changes:
+- rename the current "upstream" semantics toward import/reconcile language
+- preserve diff tooling, but point it at external comparisons rather than canonical truth
+- store imported manifests under an explicitly non-canonical namespace
+
+Possible wording shift:
+- current: "upstream snapshot"
+- target: "import snapshot" or "external atlas snapshot"
+
+Acceptance:
+- a mismatch between an external source and the canonical local record is legible as a reconciliation task, not a contradiction in authority
+
+## Phase 6: Flagship Problem Migration
+
+Goal:
+- make `20` and `857` the model examples of the full system
+
+Changes:
+- migrate the public `sunflower-coda` materials for those problems into this repo
+- replace `sunflower-coda/repo/...` path references with repo-local paths
+- ensure the public packet plus deep bundle plus formal bundle line up cleanly
+
+Acceptance:
+- a new user can pick up `20` or `857` from this repo alone
+- no crucial public handoff step depends on another repo existing
+
+## Risks
+
+### Risk: accidental npm bloat
+
+If deep research lands under currently shipped directories, npm installs become heavier and noisier.
+
+Mitigation:
+- separate repo-only directories
+- verify with `npm pack --dry-run`
+
+### Risk: provenance confusion
+
+If external import fields disappear entirely, users may lose track of where records originated.
+
+Mitigation:
+- keep provenance blocks, but label them as imported or external
+
+### Risk: partial migration
+
+If docs are updated before structure and commands are ready, users may expect deeper repo-backed pulls that do not exist yet.
+
+Mitigation:
+- roll out phase by phase
+- clearly document what exists now versus what is planned
+
+## Initial Implementation Slice
+
+This change series should start with the smallest durable slice:
+
+1. store this migration plan in the repo
+2. update ownership/provenance docs so the repo is described as canonical
+3. create the repo-only directory contract and npm boundary rules
+4. add minimal scaffolding docs for deep research bundles
+
+That gives the project a correct public model before the larger content migration begins.
+
+## Definition Of Done
+
+The migration is complete when:
+
+- `SproutSeeds/erdos-problems` is clearly the canonical public home in docs, metadata, and CLI wording
+- external sources are treated as imports and reconciliation inputs
+- the repo can hold public Lean, paper, and deep research artifacts without shipping all of them to npm
+- flagship problems like `20` and `857` can be followed from this repo alone
+- contributors can update dossiers, packs, and deep research in one place

package/docs/DEEP_RESEARCH_BUNDLE_SPEC.md

@@ -0,0 +1,129 @@
+# Deep Research Bundle Spec
+
+Last updated: 2026-03-30
+
+## Purpose
+
+This document defines how `SproutSeeds/erdos-problems` can hold the full public research archive without forcing all of it into the npm package.
+
+The package should keep a two-speed model:
+
+- fast install and immediate dossier/pack handoff through npm
+- deeper public research bundles through the GitHub repo
+
+## Repo-only directories
+
+These directories are intended to live in the GitHub repo but stay out of the npm tarball:
+
+- `research/`
+- `formal/`
+- `paper/`
+- `imports/`
+- `analysis/`
+
+They remain repo-only by staying outside the `package.json` `files` list.
+
+## Recommended layout
+
+```text
+research/
+  problems/<id>/
+    README.md
+    boards/
+    checkpoints/
+    notes/
+    wave-history/
+
+formal/
+  lean/
+    README.md
+    SunflowerLean/
+
+paper/
+  README.md
+  problems/<id>/
+  references/
+
+imports/
+  README.md
+  external-atlas/
+  public-site/
+
+analysis/
+  README.md
+  problem20/
+  problem857/
+```
+
+## Problem handoff model
+
+Every serious problem should eventually have three public layers:
+
+### 1. Canonical dossier
+
+Lives in `problems/<id>/`.
+
+Use for:
+- source page
+- canonical local status
+- short statement
+- references
+- evidence ledger
+- formalization posture
+
+### 2. Public pack packet
+
+Lives in `packs/<family>/problems/<id>/`.
+
+Use for:
+- active route
+- current frontier
+- ready queue
+- checkpoint/report templates
+- concise operational handoff
+
+### 3. Deep research bundle
+
+Lives in `research/problems/<id>/`.
+
+Use for:
+- long-form notes
+- board exports
+- route decompositions
+- wave histories
+- intermediate artifacts
+- deeper public context not needed in the npm install
+
+## Formal bundle
+
+Formalization source should live under `formal/lean/` or another clearly scoped formal directory inside this repo.
+
+Pack packets may point into those repo-local formal files, but should not point back out to `sunflower-coda` once the migration is complete.
+
+## Paper bundle
+
+Paper drafts, publication notes, and public reference-reading order should live under `paper/`.
+
+The npm install does not need all of this by default, but the repo should be able to present it as part of the canonical public archive.
+
+For problem-specific writing, prefer `paper/problems/<id>/` bundles created by `erdos paper init <id>`.
+
+## Import bundle
+
+External imports should live under `imports/` when they are intentionally retained in the repo.
+
+These are not canonical dossier truth. They are imported provenance and reconciliation material.
+
+## Migration order
+
+1. store this directory contract in the repo
+2. create the repo-only directories with READMEs
+3. migrate flagship public materials for `20` and `857`
+4. update pack references from `sunflower-coda/...` to repo-local paths
+5. add CLI affordances for pointing users at the deeper repo-held bundles
+
+## Success criteria
+
+- the GitHub repo can hold the full public archive
+- npm remains focused on install-critical assets
+- a user can understand the shallow-to-deep handoff without guessing

package/docs/ERDOS_PROBLEMS_PROBLEM_SCHEMA.md

@@ -1,6 +1,6 @@
 # Erdos Problems Problem Schema
 
-Last updated: 2026-03-25
+Last updated: 2026-03-30
 
 ## Purpose
 
@@ -8,9 +8,9 @@ This schema defines the canonical local dossier record for each seeded problem i
 
 Goals:
 - every Erdős problem has a consistent local home
-- local dossier truth stays separate from upstream public truth
+- local dossier truth stays separate from imported public truth
 - agents can bootstrap from canonical artifacts immediately after install
-- pulled upstream/site bundles can be promoted into canonical dossiers without losing provenance
+- pulled import/site bundles can be promoted into canonical dossiers without losing provenance
 
 ## Canonical files
 
@@ -21,7 +21,7 @@ Each seeded problem should have:
 - `problems/<id>/EVIDENCE.md`
 - `problems/<id>/FORMALIZATION.md`
 
-Bundled upstream snapshot artifacts live in:
+Bundled import snapshot artifacts live in:
 - `data/upstream/erdosproblems/problems.yaml`
 - `data/upstream/erdosproblems/PROBLEMS_INDEX.json`
 - `data/upstream/erdosproblems/SYNC_MANIFEST.json`
@@ -40,15 +40,16 @@ Workspace-generated artifacts may live in:
 
 ## Canonical truth split
 
-External public truth:
-- tracked, not rewritten
-- upstream structured repo: `teorth/erdosproblems`
-- public presentation site: `erdosproblems.com`
-
-Local atlas truth:
+Canonical repo truth:
+- `SproutSeeds/erdos-problems`
 - `problems/<id>/problem.yaml`
 - dossier markdown files beside it
 
+External imported public truth:
+- tracked, not silently rewritten
+- external structured repo: `teorth/erdosproblems`
+- public presentation site: `erdosproblems.com`
+
 Local harness truth:
 - route state
 - evidence notes
@@ -67,7 +68,7 @@ source:
   site: "erdosproblems.com"
   url: "https://www.erdosproblems.com/857"
   external_id: "857"
-upstream:
+external_source:
   repo: "https://github.com/teorth/erdosproblems"
   data_file: "data/problems.yaml"
   number: "857"
@@ -75,8 +76,8 @@ status:
   site_status: "open"
   site_badge: "OPEN"
   repo_status: "active"
-  upstream_status: "open"
-  upstream_last_update: "2025-08-31"
+  imported_status: "open"
+  imported_last_update: "2025-08-31"
 cluster: "sunflower"
 prize:
   display: "no"
@@ -97,8 +98,8 @@ evidence_path: "EVIDENCE.md"
 formalization_path: "FORMALIZATION.md"
 formalization:
   status: "active"
-  upstream_state: "no"
-  upstream_last_update: "2025-08-31"
+  imported_state: "no"
+  imported_last_update: "2025-08-31"
 research_state:
   open_problem: true
   active_route: "anchored_selector_linearization"
@@ -115,7 +116,7 @@ provenance:
   seeded_at: "2026-03-25T..."
   seeded_from:
     kind: "pull_bundle"
-    upstream_record_included: true
+    imported_record_included: true
     site_snapshot_included: false
 ```
 
@@ -125,7 +126,7 @@ provenance:
 - `display_name`
 - `title`
 - `source`
-- `upstream`
+- `external_source`
 - `status`
 - `cluster`
 - `family_tags`
@@ -171,12 +172,18 @@ Repo-local examples:
 - `site-proved-lean`
 - `unstarted`
 
+## Legacy compatibility
+
+During migration, older dossiers may still use an `upstream` block instead of `external_source`.
+
+That legacy spelling should be treated as imported external provenance, not as a statement that another repo outranks the canonical local dossier.
+
 ## Scaffold contract
 
 `erdos scaffold problem <id>` should create a workspace-ready bundle containing:
 - copied canonical dossier files
 - `problem.yaml`
-- upstream record snapshot when available
+- imported record snapshot when available
 - pack README context when available
 - per-problem pack context when available
 - compute packets when available
@@ -188,7 +195,7 @@ Repo-local examples:
 - a root pull manifest
 - an `artifacts/` lane
 - a `literature/` lane
-- upstream record snapshot when available
+- imported record snapshot when available
 - local dossier artifacts when the problem is already seeded locally
 - optional live site snapshot and extracted text when `--include-site` is used
 
@@ -201,4 +208,4 @@ Repo-local examples:
 - `EVIDENCE.md`
 - `FORMALIZATION.md`
 
-The resulting dossier should preserve upstream/site provenance and be safe for later manual curation.
+The resulting dossier should preserve imported provenance and site provenance and be safe for later manual curation.