dsh-aris-panel 0.1.0

package/skills/skills-codex/shared-references/citation-discipline.md

@@ -0,0 +1,452 @@
+# Citation Discipline and Hallucination Prevention
+
+Use this reference only when the built-in DBLP/CrossRef workflow in `paper-write` is not enough.
+
+This is an intentionally longer reference. It is not meant to replace the main workflow. Instead, it gives stricter standards for handling citations when:
+
+- the title, authors, year, or venue are ambiguous,
+- multiple papers could match the same description,
+- DBLP / CrossRef do not return a sufficiently clean result,
+- you need to verify that a specific factual claim really comes from the cited paper,
+- or you need to clean and standardize a bibliography in a more disciplined way.
+
+## Contents
+
+- [Why Citation Verification Matters](#why-citation-verification-matters)
+- [Useful Sources and APIs](#useful-sources-and-apis)
+- [Standard Verification Workflow](#standard-verification-workflow)
+- [How to Add an Entry to the Bibliography](#how-to-add-an-entry-to-the-bibliography)
+- [BibTeX Management Rules](#bibtex-management-rules)
+- [Common Entry Templates](#common-entry-templates)
+- [Common Failures and Troubleshooting](#common-failures-and-troubleshooting)
+- [Final Verification Checklist](#final-verification-checklist)
+
+## Why Citation Verification Matters
+
+### Typical Citation Hallucination Patterns
+
+The dangerous case is usually not a wildly fake citation. It is a citation that looks *plausible enough* to slip through:
+
+- real authors plus a fake title,
+- a real title plus the wrong year,
+- a real arXiv ID plus the wrong venue,
+- a real topic plus a non-existent DOI,
+- or a preprint version and published version silently merged into one entry.
+
+These mistakes are easy to miss by eye and disproportionately damaging during submission, review, and rebuttal.
+
+### Consequences
+
+If citation quality is weak, the mild consequences include:
+
+- reviewers conclude the related-work section is unreliable,
+- the bibliography looks messy and exposes a low-trust workflow,
+- claims appear unsupported because the cited source does not actually say what the paper attributes to it.
+
+The more serious consequences include:
+
+- being called out for hallucinated references,
+- surfacing compliance problems in desk checks or later review,
+- and creating an avoidable trust crisis around the paper.
+
+### Core Principle
+
+**Never generate citations from memory.**
+
+If a citation cannot be verified programmatically or from trusted project materials:
+
+- mark it explicitly as unresolved,
+- tell the user,
+- and do not fabricate a plausible-looking BibTeX entry.
+
+## Useful Sources and APIs
+
+### Primary Sources
+
+| Source | Best Use | Strengths | Caveats |
+|--------|----------|-----------|---------|
+| DBLP | CS/ML conference papers, BibTeX retrieval | Strong structure, strong venue metadata | Some preprints are better covered by arXiv |
+| CrossRef | DOI lookup, BibTeX content negotiation | Official metadata source, ideal when DOI exists | Quality depends on DOI registration |
+| Semantic Scholar | Paper search, citation graph, abstract lookup | Good discovery experience for ML literature | Free access may be rate limited |
+| arXiv | Preprint lookup | Strong coverage for ML preprints | Not a substitute for formal publication metadata |
+| OpenAlex | Open metadata graph, cross-checking | Broad coverage and useful as a second source | Structure may differ from DBLP conventions |
+
+### How to Choose
+
+Common decision logic:
+
+```text
+Need to search ML papers -> Semantic Scholar / DBLP
+Already have a DOI -> CrossRef content negotiation
+Only have an arXiv clue -> arXiv + CrossRef / Semantic Scholar for cross-checking
+Need a second verification source -> OpenAlex / Semantic Scholar / arXiv
+```
+
+### About Google Scholar
+
+Google Scholar is not the default verification backbone here. Do not treat “I found something by hand in Scholar” as the same thing as clean, trustworthy, structured metadata.
+
+## Standard Verification Workflow
+
+### Five-Step Process
+
+```text
+1. SEARCH   -> find candidate papers
+2. VERIFY   -> confirm the paper exists in at least two trustworthy sources
+3. RETRIEVE -> get BibTeX programmatically
+4. VALIDATE -> confirm the claim you cite is really supported by the paper
+5. ADD      -> add the entry to the bibliography with clean keys and formatting
+```
+
+### Step 1: Search
+
+Prefer **specific searches**, not broad topic words.
+
+Good query patterns:
+
+- `paper title + first author`
+- `method name + dataset + first author`
+- `claim keyword + author`
+- exact phrases from the title
+
+While searching, record:
+
+- title,
+- authors,
+- year,
+- DOI,
+- arXiv ID,
+- venue / journal.
+
+If multiple similar papers appear, do not add anything to `.bib` yet. First compare title details, year, and author order.
+
+### Step 2: Verify Existence
+
+In the ideal case, confirm the same paper in at least two sources:
+
+- Semantic Scholar + CrossRef,
+- DBLP + DOI,
+- arXiv + Semantic Scholar,
+- DBLP + OpenAlex.
+
+The minimum standard for “this is the same paper” is:
+
+- highly matching title,
+- matching first author,
+- matching or explainable year,
+- matching DOI and/or arXiv ID.
+
+If you only found the paper in one place, the citation is still not very stable.
+
+### Step 3: Retrieve BibTeX Programmatically
+
+Preferred order:
+
+1. direct DBLP `.bib`,
+2. DOI content negotiation,
+3. manual completion only if the metadata base is already trustworthy.
+
+DBLP pattern:
+
+```bash
+curl -s "https://dblp.org/search/publ/api?q=TITLE+AUTHOR&format=json&h=3"
+curl -s "https://dblp.org/rec/{key}.bib"
+```
+
+CrossRef / DOI pattern:
+
+```bash
+curl -sLH "Accept: application/x-bibtex" "https://doi.org/{doi}"
+```
+
+If both DBLP and DOI fail, move to explicit placeholder handling instead of guessing.
+
+### Step 4: Validate the Claim
+
+This is the step people most often skip and the step that matters the most.
+
+Before writing claims such as:
+
+- “X first proved …”
+- “Y showed that …”
+- “Prior work found …”
+- “Z outperformed B on benchmark A …”
+
+check that the cited paper really supports that statement.
+
+The minimum standard is:
+
+- the claim is directly supported by the title, abstract, or introduction,
+- or you have read the relevant section, table, theorem, or figure in the paper itself.
+
+Do **not**:
+
+- assume a paper supports your claim because the title sounds related,
+- inherit a claim from someone else’s related-work section without checking,
+- confuse “this paper is about the same topic” with “this paper establishes this exact statement.”
+
+### Step 5: Add the Entry
+
+Only after Steps 1-4 are complete should the entry go into `.bib`.
+
+Check before adding:
+
+- Is the entry type correct?
+- Are the authors complete?
+- Are the year and venue accurate?
+- Does the citation key match the project style?
+- Is this a duplicate of an existing entry?
+
+## How to Add an Entry to the Bibliography
+
+### Recommended Decision Tree
+
+```text
+Is there already a project-local .bib entry?
+  Yes -> reuse it, then verify it
+  No  -> continue
+
+Can DBLP provide a direct .bib entry?
+  Yes -> use DBLP
+  No  -> continue
+
+Do you have a DOI?
+  Yes -> use DOI content negotiation
+  No  -> continue
+
+Can you confirm the paper in two trustworthy sources with enough metadata?
+  Yes -> manually complete an entry from trusted metadata
+  No  -> use an explicit placeholder and report it to the user
+```
+
+### Placeholder Policy
+
+If verification fails, leave an explicit placeholder such as:
+
+```latex
+% TODO: verify before submission
+\cite{PLACEHOLDER_author2024_verify}
+```
+
+Or mark uncertainty inside the `.bib` file:
+
+```bibtex
+% [VERIFY] could not confirm DOI / venue / exact title
+```
+
+The principle is:
+
+- make uncertainty visible,
+- keep bad metadata out of the final draft,
+- and report unresolved citations before finalization.
+
+## BibTeX Management Rules
+
+### BibTeX vs BibLaTeX
+
+This `insleep`-derived workflow still prioritizes compatibility with existing conference templates. It does not force a move to BibLaTeX.
+
+Still, you should know the tradeoff:
+
+| Aspect | BibTeX | BibLaTeX |
+|--------|--------|----------|
+| Unicode support | Weaker | Better |
+| Entry types | Standard | Richer |
+| Backend | `bibtex` | `biber` |
+| Compatibility with older venue templates | Usually stronger | Not always safe |
+
+In this skill pack, if the template is already fixed, prefer compatibility over modernization.
+
+### Citation Key Format
+
+Prefer a stable key format:
+
+```text
+firstauthor_year_keyword
+```
+
+For example:
+
+```text
+vaswani_2017_attention
+devlin_2019_bert
+brown_2020_language
+```
+
+If the project already uses a different key style, preserve consistency rather than mixing styles.
+
+### Keep Only Cited Entries
+
+Do not let `references.bib` become a dumping ground.
+
+Rules:
+
+- keep only entries that are actually cited,
+- remove duplicates,
+- remove abandoned keys,
+- and choose explicitly between preprint and published versions when both exist.
+
+### When to Prefer the Published Version
+
+Prefer the formal published version when:
+
+- a conference or journal version clearly exists,
+- you need stable venue information for related work,
+- page / volume / publisher metadata matters for submission quality.
+
+Keeping the arXiv version can still be appropriate when:
+
+- the work is not formally published yet,
+- the community primarily cites the preprint,
+- or the specific version matters for the content you are citing.
+
+## Common Entry Templates
+
+### Conference Paper
+
+```bibtex
+@inproceedings{vaswani_2017_attention,
+  title = {Attention Is All You Need},
+  author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and
+            Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N. and
+            Kaiser, Lukasz and Polosukhin, Illia},
+  booktitle = {Advances in Neural Information Processing Systems},
+  year = {2017}
+}
+```
+
+### Journal Article
+
+```bibtex
+@article{hochreiter_1997_long,
+  title = {Long Short-Term Memory},
+  author = {Hochreiter, Sepp and Schmidhuber, J{\"u}rgen},
+  journal = {Neural Computation},
+  volume = {9},
+  number = {8},
+  pages = {1735--1780},
+  year = {1997}
+}
+```
+
+### arXiv Preprint
+
+```bibtex
+@misc{brown_2020_language,
+  title = {Language Models are Few-Shot Learners},
+  author = {Brown, Tom and Mann, Benjamin and Ryder, Nick and others},
+  year = {2020},
+  eprint = {2005.14165},
+  archiveprefix = {arXiv},
+  primaryclass = {cs.CL}
+}
+```
+
+## Common Failures and Troubleshooting
+
+### Case 1: No Results Found
+
+Check:
+
+- Was the title misspelled?
+- Was the author misspelled?
+- Is the query too broad?
+- Should the search use an exact phrase instead?
+
+Useful fixes:
+
+- add the first author,
+- add the year,
+- search with a distinctive title phrase rather than a broad topic term.
+
+### Case 2: DOI Does Not Resolve Cleanly
+
+Possible reasons:
+
+- the DOI exists but is not well connected through CrossRef,
+- the DOI is not the best lookup path for this paper,
+- the work mainly exists as an arXiv preprint.
+
+Useful fixes:
+
+- return to DBLP,
+- cross-check the DOI in Semantic Scholar / OpenAlex,
+- manually complete an entry from trusted metadata only when the metadata is already reliable.
+
+### Case 3: Several Papers Look Similar
+
+This is one of the most dangerous situations.
+
+Check:
+
+- author order,
+- year,
+- title wording,
+- abstract-level claim,
+- and, if needed, the PDF front page or the relevant section.
+
+Until those match, do not assume any of them is the correct paper.
+
+### Case 4: BibTeX Compilation Errors
+
+Common causes:
+
+- missing commas,
+- unmatched braces,
+- unescaped special characters,
+- Unicode that does not play well with the template.
+
+Check:
+
+- whether each field ends correctly,
+- whether title text needs LaTeX escaping,
+- whether accented names are encoded safely.
+
+### Case 5: The Same Paper Exists Under Two Keys
+
+Fix it by:
+
+- choosing one canonical key,
+- updating all in-text citations,
+- and deleting the duplicate entry.
+
+Do not leave two nearly identical entries in the final bibliography.
+
+## Final Verification Checklist
+
+Before treating a citation as complete, verify:
+
+- [ ] the paper was confirmed in at least two trustworthy sources,
+- [ ] the DOI or arXiv ID was checked,
+- [ ] the BibTeX was retrieved programmatically or completed from trusted metadata,
+- [ ] the entry type is correct (`@inproceedings`, `@article`, `@misc`, etc.),
+- [ ] the author list is complete and well formatted,
+- [ ] the year and venue were checked,
+- [ ] the citation key matches the project style,
+- [ ] the specific claim you cite is actually supported by the paper,
+- [ ] any unresolved uncertainty is marked with `[VERIFY]` or an explicit placeholder.
+
+## Output Discipline: Verification Lives in the Trace, Not in Prose
+
+Two rules on how verification shows up in what the reader sees (adapted from
+Anthropic's Claude Science `literature-review` skill, Apache-2.0):
+
+- **Never write a "citations verified" line in the output.** Not as an opener,
+  not as a footer, not as a subtitle — "All DOIs verified", "no retraction
+  flags", "bibliography checked" are process narration. Verification is
+  something that happens in your tool trace (the `verify_papers` run, the DBLP
+  lookups, the audit JSON); the reader infers it from citations that resolve
+  and claims that hold up. A reply or section whose content is "I verified
+  things" is not content. (ARIS-specific: the machine-readable verification
+  record belongs in the audit artifact — `CITATION_AUDIT.json`, traces — never
+  as reassurance prose in the paper or report body.)
+- **A style/consistency lint runs ONCE, as a lint — not as a loop-until-clean
+  gate.** Run it on the full draft, fix what it lists in a single editing
+  pass, and move on; do not re-run until it returns clean. A lint that becomes
+  a convergence loop burns rounds polishing cosmetics while real
+  (cross-model-review) gates wait. Quality/correctness convergence belongs to
+  the reviewer loops, not to lints.
+
+## Bottom Line
+
+**When a citation is uncertain, leave an explicit gap instead of silently inventing metadata.**

package/skills/skills-codex/shared-references/compute-env-contract.md

@@ -0,0 +1,163 @@
+# Compute Environment Contract
+
+> One declarative spec for what an environment IS; per-provider knowledge for
+> how it gets built HERE; a content-hash ledger so "did the env change?" has a
+> mechanical answer; and a three-tier validation ladder whose top tier is a
+> fresh agent following the skill's own doc verbatim. Adapted from Anthropic's
+> Claude Science `compute-env-setup` skill (Apache-2.0); de-coupled from its
+> proprietary `host.*` runtime — everything here runs on plain bash + SSH +
+> subagents.
+
+Every ARIS compute skill (`/run-experiment`, `/experiment-queue`,
+`/serverless-modal`, `/vast-gpu`, `/qzcli`) needs the same three things for a
+job: a software stack (exact versions, often with load-bearing install order),
+possibly large weights placed where the tool looks, and a resource shape. What
+varies per provider is only HOW those materialize. Without a shared contract,
+each skill re-encodes provider quirks and every "environment is ready" claim is
+vibes. The classic failure this prevents: agent says "env ready", the overnight
+run dies at `import flash_attn`, 8 GPUs idle until morning.
+
+## 1. Provider shapes — recognize, don't choose
+
+You are rarely choosing a shape; you are recognizing which one this provider
+already is. The shape determines what "build", "register", and "resolve" mean.
+
+| Shape | ARIS examples | Build = | Env name resolves to |
+|---|---|---|---|
+| **Direct SSH host** (conda/venv) | personal GPU boxes, lab servers | YOU are the renderer: `conda create -n <name> python=<X>`, then run `pip_phases` in order | the conda env name itself (`conda run -n <name> …`) |
+| **Scheduler cluster** (Slurm/PBS; Qizhi-like platforms) | `/qzcli` targets | `module load` or a container image built OFF-cluster and pulled (compute nodes often have **no internet** — pre-stage everything) | scheduler directives + container path in shared scratch (mind purge windows) |
+| **Managed API** (serverless) | `/serverless-modal`, Vast.ai templates | the provider's image definition (Modal `Image`, Vast template) — render the same spec into it | the provider's opaque image ref, recorded in the ledger |
+
+## 2. The declarative spec (write WHAT once; render per provider)
+
+```yaml
+# env-spec: one dict per environment, portable across shapes
+base:        "cuda12.8 + python3.10"      # FROM-image / conda create versions
+system_pkgs: [git, tmux]                  # apt in a container; conda-forge subset on no-root hosts
+pip_phases:                               # ORDERED list of lists — each inner list = ONE pip call
+  - [torch==2.8.0]                        #   phase 1 first, so later packages
+  - [flash-attn --no-build-isolation]     #   can't drag torch to a wrong wheel
+  - [transformers, peft, accelerate]
+env:         {HF_ENDPOINT: "...", OMP_NUM_THREADS: "<tier.cpus>"}
+run_commands: []                          # escape-hatch shell (RUN / %post / plain SSH)
+weight_dirs: {chai: {path: /scratch/weights/chai, source: "tool's own loader", gated: false}}
+smoke:                                    # probes that run INSIDE the env on every shape
+  import_names: [torch, flash_attn]
+  gpu_tests:                              # each = {cmd, expect}; expect is the witness regex
+    - cmd: "python -c 'import torch;torch.manual_seed(0);x=torch.randn(8,8,device=\"cuda\");print(\"WITNESS\", (x@x).shape, torch.cuda.get_device_name())'"
+      expect: "^WITNESS torch.Size"
+  cli_checks:   [nvidia-smi]
+```
+
+- **`pip_phases` ordering IS the fix** for every "package A drags B to the
+  wrong version" problem: each phase is its own pip invocation, and pip leaves
+  an already-satisfied requirement alone unless asked to upgrade. Pin the
+  fought-over package in an EARLIER phase than the fighter.
+- A clean spec renders unchanged through every renderer. If you find yourself
+  adding a field only one backend understands, that field belongs in the
+  provider's ledger entry, not the spec.
+- **Weights**: small (<~500 MB) and read by every job → bake into the env at
+  build time. Large with a cache env var → persistent scratch + point the var
+  there. Populate with the **tool's own loader** (hand-curled layouts miss
+  marker files), then verify from the tool's perspective: run the real
+  entrypoint once against the staged dir and `du -sh` every subdir — 0 B means
+  a swallowed download error.
+
+## 3. The environment ledger (content-hash = mechanical staleness)
+
+Per provider, keep an append-friendly `.aris/compute/<provider>.md` (or the
+project's existing server-notes file). One block per env, keyed by a content
+hash of the spec, computed over an EXACT canonical form so two agents can
+never hash the same spec differently: parse the spec file, re-serialize as
+JSON with sorted keys and no whitespace, sha256, first 8 hex chars —
+
+```bash
+# spec stored as YAML (env-spec.yaml); requires PyYAML. If PyYAML is absent,
+# store the spec as JSON instead and drop the yaml import — same pipeline.
+python3 -c 'import sys,json,hashlib,yaml; \
+s=json.dumps(yaml.safe_load(open(sys.argv[1])),sort_keys=True,separators=(",",":")); \
+print(hashlib.sha256(s.encode()).hexdigest()[:8])' env-spec.yaml
+```
+
+Key order, comments, indentation, and trailing whitespace in the source file
+do NOT affect the hash — only the parsed content does:
+
+```
+### env: dllm@a3f9c2e1
+how: conda env "dllm" on <host>            # or: modal image ref / .sif path + partition
+tier: {cpus: 8, mem_gib: 64, gpus: 1}
+weights: HF_HOME=/scratch/hf (24 GB; purge-window 30d)
+validated: 2026-07-02 (witness + agent-follows-doc clean)
+gotcha: <any diagnosis-table row hit on THIS provider>
+```
+
+Spec changed → hash changes → **cache miss**: the ledger entry no longer
+matches and the env must be rebuilt (or a new block added). Spec unchanged →
+warm-reuse without rebuilding or re-validating tier 1–2. This turns "I think
+the env is the same as last week" into a string comparison. Note `.aris/` is
+gitignored by convention — the ledger is **project-local and uncommitted** by
+default (like `.aris/traces/`). If you want committed, git-blameable history,
+keep the ledger blocks in the project's tracked server-notes file instead;
+the block format is the contract, not the path.
+
+## 4. Validation — three tiers; the gap between them is where debugging lives
+
+1. **Import works** — `python -c "import <pkg>"` exits 0. Necessary, cheap,
+   catches almost nothing interesting.
+2. **Kernel-dispatch witness** — a tiny SEEDED forward pass that prints a
+   sentinel line (output shape + device name + non-emptiness). Catches "torch
+   sees the GPU but the kernel was compiled for an older SM", "the compiled
+   extension's `.so` isn't on the loader path", "inference writes to a
+   read-only cache". Keep the witness command in the spec's `smoke.gpu_tests`
+   with an `expect:` regex so the SAME probe runs on every backend. Cheap —
+   run on every build.
+3. **Agent-follows-doc** — the validation that actually matters and the one
+   that's easy to skip. Spawn a FRESH subagent that gets ONLY: the compute
+   skill's doc, the provider's ledger entry, and the documented invocation.
+   It must run the invocation **verbatim** — no improvisation, no fixing —
+   and report every point where the doc's claim and reality diverge. This is
+   where you find the doc says `--ligand` but the flag is
+   `--ligand_description`, or the weights path exists but lacks the completion
+   marker the tool checks. The author agent cannot self-certify its own doc
+   (it walks through on hidden knowledge the doc never wrote down — same
+   principle as `acceptance-gate.md`: the writer never acquits its own
+   artifact); the fresh agent's stuck-point IS the doc's lie. Expensive —
+   reserve for the two moments doc and env can drift: **after any env rebuild
+   or doc edit, and before declaring an env ready**.
+
+## 5. Diagnosis table (symptom → layer → fix)
+
+When a documented invocation fails, don't patch reflexively — ask which LAYER
+is wrong: spec, build, weights, resolution, or doc. Grep-able rows (container
+rows apply only to container shapes):
+
+| Symptom | Layer | Fix |
+|---|---|---|
+| `no kernel image is available for execution` | build/spec | torch compiled for older SM than this GPU — record `sm_range` in the ledger and route jobs; rebuild only if no compatible hardware |
+| `ModuleNotFoundError` for a package not in the spec | spec | a `--no-deps` install skipped a runtime dep — read the package's `pyproject.toml` and add an explicit phase |
+| Wrong torch/numpy version after install | spec | a later package's pin won — add a `force-reinstall --no-deps` snap-back phase after it |
+| `ImportError: libfoo.so: cannot open shared object` | build | compiled `.so` not on loader path — `find` it, add its dir to `LD_LIBRARY_PATH` |
+| Tool re-downloads despite populated weights | weights | `du -sh $CACHE_VAR` first: 0 B = swallowed error; non-zero = tool checks a marker file, stage that too |
+| `OSError: Read-only file system` under cache var | weights (container) | tool writes locks next to weights on an RO mount — symlink blobs into writable `/tmp` cache |
+| 80-way thread storm on a 4-CPU allocation | exec | `os.cpu_count()` returns the HOST's cores — export `OMP/MKL/OPENBLAS_NUM_THREADS=<tier.cpus>` on every backend |
+| First job slow, every later job equally slow | build | expensive precompute runs at job time in a non-persistent workdir — run it once at build time |
+| Job COMPLETED but output dir empty | exec | the wrapper writing the completion marker never ran — often `#!/bin/bash` on a runtime that only ships `/bin/sh` |
+
+Hit a row on a specific provider → append symptom + fix to that provider's
+ledger `gotcha:` line, so the next agent doesn't rediscover it.
+
+## How compute skills use this
+
+- **Before building**: read the provider's ledger. The env — or a near-match
+  to extend — may already exist; an unchanged hash means skip the rebuild.
+- **When building**: write the spec first (§2), render it for the shape (§1),
+  run tier-1/2 validation (§4), append the ledger block (§3).
+- **Before declaring ready** (and after any rebuild/doc edit): run the
+  agent-follows-doc pass (§4.3).
+- **On failure**: diagnosis table (§5) before patching; record provider-true
+  gotchas in the ledger.
+
+Attribution: the spec/ledger/three-tier-validation design is adapted from
+Anthropic's Claude Science `compute-env-setup` skill (Apache-2.0, re-hosted by
+HughYau/AcademicForge); this document ports it off the proprietary `host.*`
+runtime onto plain bash + SSH + fresh reviewer subagents.