replication-radar 0.3.2__tar.gz → 0.3.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- replication_radar-0.3.3/DEMO.md +125 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/PKG-INFO +1 -1
- replication_radar-0.3.3/STORY.md +130 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/pyproject.toml +1 -1
- {replication_radar-0.3.2 → replication_radar-0.3.3}/site/app.js +91 -42
- {replication_radar-0.3.2 → replication_radar-0.3.3}/site/index.html +7 -4
- replication_radar-0.3.3/site/methodology.html +91 -0
- replication_radar-0.3.3/site/methodology.json +59 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/site/style.css +31 -18
- replication_radar-0.3.3/site/verdicts.json +394 -0
- replication_radar-0.3.3/src/replication_radar/data/verdicts.json +394 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/openaire.py +21 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/radar.py +8 -1
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/server.py +6 -3
- replication_radar-0.3.2/STORY.md +0 -111
- replication_radar-0.3.2/site/verdicts.json +0 -346
- replication_radar-0.3.2/src/replication_radar/data/verdicts.json +0 -346
- {replication_radar-0.3.2 → replication_radar-0.3.3}/.github/workflows/publish-pypi.yml +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/.gitignore +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/CLAUDE.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/LICENSE +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/README.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/demo_sdm.py +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/docs/app-ui.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/docs/link-types.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/docs/next-layers-plan.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/docs/openaire-mcp.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/docs/readiness-scoring-plan.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/netlify.toml +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/scripts/build_verdicts.py +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/site/README.md +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/site/curated.json +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/__init__.py +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/network.py +0 -0
- {replication_radar-0.3.2 → replication_radar-0.3.3}/src/replication_radar/verdicts.py +0 -0
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# Replication Radar — MCP demo runbook
|
|
2
|
+
|
|
3
|
+
A clean ~60-second screen recording showing the **verified-knowledge MCP** in an AI agent:
|
|
4
|
+
the agent gives a *cited, verified* answer about a research claim instead of a confident,
|
|
5
|
+
unchecked one. This is the "cite instead of hallucinate" moment — the AI-hackathon hook.
|
|
6
|
+
|
|
7
|
+
The MCP is **read-only**: it answers *"has this claim been independently checked, and did it
|
|
8
|
+
hold?"* It does **not** start replications (that's the FORRT template). Don't demo a "start a
|
|
9
|
+
replication" flow with it.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 1 · One-time setup (~5 minutes)
|
|
14
|
+
|
|
15
|
+
**Install the MCP in an isolated environment** (so the path is stable for the client):
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
python3 -m venv ~/.venvs/radar
|
|
19
|
+
~/.venvs/radar/bin/pip install replication-radar # pulls in the `mcp` runtime too
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
**Smoke-test it works** (should print `True` then a number):
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
~/.venvs/radar/bin/python - <<'PY'
|
|
26
|
+
from replication_radar.radar import replication_status, verified_claims
|
|
27
|
+
print("replicated:", replication_status("10.1126/science.aax8591")["replicated"])
|
|
28
|
+
print("verified claims in corpus:", verified_claims()["count"])
|
|
29
|
+
PY
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
**Register it with Claude Desktop.** Edit
|
|
33
|
+
`~/Library/Application Support/Claude/claude_desktop_config.json` (create it if missing) — use the
|
|
34
|
+
**absolute** python path (Claude Desktop does not use your shell PATH):
|
|
35
|
+
|
|
36
|
+
```json
|
|
37
|
+
{
|
|
38
|
+
"mcpServers": {
|
|
39
|
+
"replication-radar": {
|
|
40
|
+
"command": "/Users/annef/.venvs/radar/bin/python",
|
|
41
|
+
"args": ["-m", "replication_radar.server"]
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Quit and reopen Claude Desktop. Click the tools/🔨 icon — you should see **replication-radar**
|
|
48
|
+
with 4 tools: `radar`, `replication_status`, `find_independent_software`, `verified_claims`.
|
|
49
|
+
|
|
50
|
+
**Optional — add the OpenAIRE / Alien Gateway MCP** alongside it (you/Jean have the connection).
|
|
51
|
+
It makes the "two MCPs together" point explicit. If wiring it up is fiddly, **skip it** — the
|
|
52
|
+
demo lands with just `replication-radar`.
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 2 · Pre-flight (right before recording)
|
|
57
|
+
|
|
58
|
+
1. Ask: *"Which tools do you have from replication-radar?"* → it should list the 4. (Warms it up.)
|
|
59
|
+
2. Do **one off-record dry run** of Beat 1 below — it warms the network/HTTP caches so the real
|
|
60
|
+
take is fast and identical.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 3 · The recording — two beats
|
|
65
|
+
|
|
66
|
+
### Beat 1 — the money shot: verify + cite (≈35 s)
|
|
67
|
+
|
|
68
|
+
**Type this prompt:**
|
|
69
|
+
|
|
70
|
+
> I want to cite the finding from Soroye et al. 2020 (Science, DOI 10.1126/science.aax8591) —
|
|
71
|
+
> that projected per-species bumble-bee extirpation rankings are robust. Before I do: has that
|
|
72
|
+
> claim actually been independently replicated, and did it hold? Give me something citable.
|
|
73
|
+
|
|
74
|
+
**What to expect:** the agent calls **`replication_status("10.1126/science.aax8591")`** and gets
|
|
75
|
+
back `replicated: true` with **5 independent verdicts** — 4 `confirms` (Validated) and 1
|
|
76
|
+
`qualifies` (PartiallySupported) — each with a **signed Outcome nanopublication URL** and the
|
|
77
|
+
replication's deposit DOI. The answer should say roughly: *"independently replicated 5×, 4
|
|
78
|
+
confirmed, 1 qualifies it; here are the signed verdicts to cite,"* with links.
|
|
79
|
+
|
|
80
|
+
**The point to land (caption or voiceover):** OpenAIRE/citation count would call this paper
|
|
81
|
+
"settled"; the MCP shows it's been checked 5 times and hands you **signed, citable verdicts** —
|
|
82
|
+
including the one that *qualifies* it. That's the difference between paraphrasing and citing.
|
|
83
|
+
|
|
84
|
+
### Beat 2 — the discovery side (≈20 s, optional)
|
|
85
|
+
|
|
86
|
+
**Type this prompt:**
|
|
87
|
+
|
|
88
|
+
> What high-impact work on marine heatwaves and species distributions is worth replicating —
|
|
89
|
+
> and what's already been checked?
|
|
90
|
+
|
|
91
|
+
**What to expect:** the agent calls **`radar("marine heatwave species")`** and returns
|
|
92
|
+
impact-ranked papers, each flagged **OPEN** (a replication opportunity) or **VERIFIED** (already
|
|
93
|
+
checked, with the verdict).
|
|
94
|
+
|
|
95
|
+
**The point to land:** the same layer also tells an agent *where the replication gaps are*.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## 4 · Recording tips
|
|
100
|
+
|
|
101
|
+
- Resize the Claude window to a clean 1280×800-ish; hide other panels.
|
|
102
|
+
- ~45–75 seconds total; no audio needed — burn in 2-3 short captions for the "point to land" lines.
|
|
103
|
+
- Keep the tool-call expansion **visible** for a beat (it's proof the answer came from the MCP,
|
|
104
|
+
not the model's memory).
|
|
105
|
+
- Export as MP4 or GIF for the submission.
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## 5 · If the OpenAIRE MCP is set up too
|
|
110
|
+
|
|
111
|
+
Add a one-line framing before Beat 1: *"Two MCPs are connected — OpenAIRE for the structural
|
|
112
|
+
graph, replication-radar for the verification layer."* You don't need to force OpenAIRE to fire;
|
|
113
|
+
its presence in the tools list is enough to make the pairing point. The verification answer from
|
|
114
|
+
`replication-radar` is the star.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
*Tools reference — what the agent can call:*
|
|
119
|
+
|
|
120
|
+
| tool | answers |
|
|
121
|
+
|---|---|
|
|
122
|
+
| `replication_status(doi)` | Has this DOI been replicated, did it hold? Verdicts + signed nanopub links. |
|
|
123
|
+
| `verified_claims()` | The whole verified-knowledge corpus (every claim with a verdict). |
|
|
124
|
+
| `radar(topic)` | Impact-ranked replication targets in a field — OPEN vs VERIFIED. |
|
|
125
|
+
| `find_independent_software(doi, topic)` | Reusable engines *not* authored by the original team. |
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: replication-radar
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.3
|
|
4
4
|
Summary: MCP server that turns the OpenAIRE Graph into a ranked replication queue — impact-ranked targets and the Science Live verification overlay (retraction/supersession-aware).
|
|
5
5
|
Project-URL: Homepage, https://github.com/ScienceLiveHub/replication-radar
|
|
6
6
|
Project-URL: Repository, https://github.com/ScienceLiveHub/replication-radar
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
# Replication Radar — adding the signals the OpenAIRE Graph can't hold
|
|
2
|
+
|
|
3
|
+
*OpenAIRE AI Hackathon · Theme B (Build) · a Science Live contribution*
|
|
4
|
+
**Live app: https://openaire-hackathon.netlify.app · how it works: /methodology.html · `pip install replication-radar`**
|
|
5
|
+
|
|
6
|
+
## The question
|
|
7
|
+
|
|
8
|
+
The OpenAIRE Graph is a network of **structural links** between research entities — papers,
|
|
9
|
+
authors, institutions, funding. It can tell you how *visible* a paper is (citation influence,
|
|
10
|
+
popularity, the BIP! classes C1–C5), but not what it *means*: it links documents to one another
|
|
11
|
+
without representing the **claims** inside them, their level of evidence, their **epistemic
|
|
12
|
+
status** (confirmed, contested, retracted, superseded), or the **semantic relations between
|
|
13
|
+
results** — replication, contradiction, refinement, not just "cites".
|
|
14
|
+
|
|
15
|
+
That gap matters more than ever in the age of LLMs. A model fed the Graph swallows everything
|
|
16
|
+
equally: a result replicated fifty times reads the same as a single study on twelve mice or an
|
|
17
|
+
unreviewed preprint. The difference between *recognising text patterns* and *understanding* is
|
|
18
|
+
exactly this missing layer — verified, status-aware, traceable knowledge that a system can **cite
|
|
19
|
+
instead of paraphrase**. OpenAIRE is the infrastructure best placed to start closing that gap at
|
|
20
|
+
European scale.
|
|
21
|
+
|
|
22
|
+
So we asked a concrete build question toward it: **can we add the two most actionable missing
|
|
23
|
+
signals — is a claim *reliable* (independently checked, and did it hold) and is its *software*
|
|
24
|
+
reusable — live, on top of the Graph, without changing it?** A heavily-cited paper looks
|
|
25
|
+
identical, in the Graph today, to one nobody ever reproduced; a widely-used research tool has the
|
|
26
|
+
same "0 citations, class C5" as an abandoned script. Both are signals the Graph structurally
|
|
27
|
+
cannot hold.
|
|
28
|
+
|
|
29
|
+
## The journey
|
|
30
|
+
|
|
31
|
+
We started simply: rank papers by impact to find what's worth replicating. That worked, but it
|
|
32
|
+
just re-served the Graph's one signal. The turn came when we tried to answer "has this been
|
|
33
|
+
replicated?" — and realised the Graph *structurally cannot* hold that answer. A verification isn't
|
|
34
|
+
a paper, gets no citations, and has no node in the Graph.
|
|
35
|
+
|
|
36
|
+
But it does exist elsewhere. Science Live publishes replication outcomes as cryptographically
|
|
37
|
+
signed **nanopublications** (the FORRT chain: Quote → Claim → Study → Outcome → CiTO). So the
|
|
38
|
+
Radar pulls the verdict layer **live from the nanopub network** and overlays it on the Graph by
|
|
39
|
+
DOI. Several corrections shaped the design along the way:
|
|
40
|
+
|
|
41
|
+
- **Verification is author-agnostic.** We don't care *who* ran the replication, so the index is
|
|
42
|
+
built **by template, not by person** — querying every FORRT Outcome and CiTO on the network and
|
|
43
|
+
joining them on the nanopub trusty hash. Today that surfaces **31 independent, signed
|
|
44
|
+
replication outcomes across 21 papers**; as more people publish replications, they flow in
|
|
45
|
+
automatically.
|
|
46
|
+
- **Enumeration has to be the right shape.** We verified empirically that walking the nanopub
|
|
47
|
+
graph outward from a paper *bleeds* into adjacent chains (it once pulled a lizard study into a
|
|
48
|
+
bumble-bee paper's replications) and *misses* disconnected ones, so we enumerate by the
|
|
49
|
+
**CiTO→DOI verdict-citation** instead — the set that is actually correct.
|
|
50
|
+
- **Validity is part of the verdict.** A retracted or superseded outcome must not count. The
|
|
51
|
+
overlay filters any outcome retracted/invalidated/superseded **by its own signer**, via the
|
|
52
|
+
nanopub admin graph — only the original author can retract their own work.
|
|
53
|
+
- **Reproduce ≠ replicate, and agreement matters.** We surface the FORRT distinction (materials
|
|
54
|
+
available = reproducible; tested by a different route = replicated) and an **agreement pattern**
|
|
55
|
+
— robustly-validated, validated, contested, refuted — computed from the verdict spread, so
|
|
56
|
+
"five replications that all agree" reads differently from "five that disagree".
|
|
57
|
+
- **What, not just whether.** Each verdict carries the **claim it actually tested** — the atomic
|
|
58
|
+
AIDA statement, traversed Outcome → Study → Claim — so a card says not "Validated" but
|
|
59
|
+
*"Validated: ‘per-species extirpation rankings are sensitive to the grid resolution’"*.
|
|
60
|
+
|
|
61
|
+
Then the software side. The Graph makes research software *findable* but not *assessable*. We
|
|
62
|
+
first tried to *recommend* tooling and it failed badly (keyword-matching surfaced off-topic
|
|
63
|
+
repos), so we pivoted from recommendation to **assessment**, and after checking that standard
|
|
64
|
+
FAIR services (F-UJI, OSTrails) had no usable API, computed the **fair-software.eu** five
|
|
65
|
+
recommendations ourselves, live, from the GitHub and Software Heritage APIs.
|
|
66
|
+
|
|
67
|
+
Two disciplines run through all of it. **Everything is grounded** — every signal comes from a
|
|
68
|
+
named, verifiable source, and is documented, signal by signal, in a **machine- and human-readable
|
|
69
|
+
methodology page** (`methodology.json` + `/methodology.html`) that states where each label and
|
|
70
|
+
score comes from and how it is computed. And everything runs **client-side** against public,
|
|
71
|
+
CORS-enabled APIs — no backend, no keys — so the whole thing is a static site anyone can open,
|
|
72
|
+
and it ships accessible (Lighthouse accessibility 100, colour-blind-safe, keyboard-operable).
|
|
73
|
+
|
|
74
|
+
## The insight
|
|
75
|
+
|
|
76
|
+
- **Nanopublications are the substrate the Graph is missing — and the AI hook.** The verdict
|
|
77
|
+
layer isn't scraped text; it's built from claim-level, cryptographically-signed assertions that
|
|
78
|
+
already carry what the Graph lacks: the claim, its epistemic relation (`cito:confirms` /
|
|
79
|
+
`disputes` / `qualifies`), and its provenance. So the Graph gains, for a paper, not "this
|
|
80
|
+
document exists" but "*this specific claim was independently checked → validated → here is the
|
|
81
|
+
signed verdict*". That is exactly what an LLM needs to **cite rather than hallucinate**. We
|
|
82
|
+
package it as an **MCP server** that an agent runs **next to the OpenAIRE/Alien MCP**: one gives
|
|
83
|
+
the structural graph, the other answers "has this been checked, and did it hold". Together they
|
|
84
|
+
are the first bricks of a graph of **verified knowledge**.
|
|
85
|
+
- **Reliability and reusability are *different categories* of signal**, not better metrics. You
|
|
86
|
+
can't repair the citation axis into a truth axis or a reuse axis — you have to *add* them, and
|
|
87
|
+
you can add them *live*, on top of the Graph, without waiting for it to change.
|
|
88
|
+
- **Verification is author-agnostic and network-wide.** Keying it on a template rather than a
|
|
89
|
+
person turns a personal portfolio into a community trust layer.
|
|
90
|
+
- **Grounded-and-transparent is a discipline, not a nicety.** Every signal is sourced and
|
|
91
|
+
documented; the one feature we built on a guess (keyword tooling) we deleted — and the project
|
|
92
|
+
is stronger for it.
|
|
93
|
+
|
|
94
|
+
## What others can reuse
|
|
95
|
+
|
|
96
|
+
- **The live web app** — pure static, queries OpenAIRE + the nanopub network + GitHub/Software
|
|
97
|
+
Heritage from the browser. Fork it, point it elsewhere.
|
|
98
|
+
- **An MCP server** (`pip install replication-radar`) exposing the same engine to any agent, to
|
|
99
|
+
run alongside the OpenAIRE MCP — the verified-knowledge layer for agentic workflows.
|
|
100
|
+
- **A reproducible, author-agnostic, retraction-aware verdict-index method** — FORRT
|
|
101
|
+
Outcome/CiTO templates joined on the trusty hash, with the admin-graph validity guard. Any
|
|
102
|
+
replication network can be read this way.
|
|
103
|
+
- **A machine-readable provenance & methodology spec** (`methodology.json`, CC-BY) — every
|
|
104
|
+
signal's source and formula, reusable as a transparency pattern for any composite-score tool.
|
|
105
|
+
- **A grounded software-FAIR assessment** — the fair-software.eu recommendations + usage,
|
|
106
|
+
computed from GitHub + Software Heritage (no third-party scorer needed).
|
|
107
|
+
- **A feasibility map of the open-science API landscape** — what's reachable and CORS-friendly
|
|
108
|
+
(OpenAIRE Graph API, the nanopub SPARQL + admin graph, GitHub/SWH/Zenodo) and what isn't
|
|
109
|
+
(F-UJI/OSTrails assessment APIs; per-paper relations from the public Graph API) — so the next
|
|
110
|
+
builder doesn't re-discover it.
|
|
111
|
+
|
|
112
|
+
*A complementary facet by Jean Iaquinta uses the **OpenAIRE MCP's** citation-graph tools to trace
|
|
113
|
+
the relationships around a verified paper — and shows the citation graph contains everything
|
|
114
|
+
**except** the verification edge, which is exactly the gap the Radar fills.*
|
|
115
|
+
|
|
116
|
+
## Honest limits
|
|
117
|
+
|
|
118
|
+
Discovery recall is keyword-bound (OpenAIRE free-text terms are AND-ed); the verdict overlay is
|
|
119
|
+
network-wide but only covers claims with a DOI a search can reach; FAIR-software runs only where a
|
|
120
|
+
real repository resolves, and GitHub's unauthenticated rate limit caps how many it scores per hour
|
|
121
|
+
(results are cached so repeated use stays stable); OpenAIRE's own subject classification is
|
|
122
|
+
sometimes quirky and is shown faithfully, not corrected. None of this is hidden in the output. And
|
|
123
|
+
we add only two of the missing layers — reliability and reusability; the fuller graph of *verified
|
|
124
|
+
knowledge* (claim-level extraction at scale, temporal obsolescence, distinguishing hypothesis from
|
|
125
|
+
result from interpretation) is the direction this points at, not something we finished.
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
*Materials are dual-licensed: **source code under MIT**, and this write-up together with the
|
|
130
|
+
verdict index and methodology spec under **[CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)**.*
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "replication-radar"
|
|
3
|
-
version = "0.3.
|
|
3
|
+
version = "0.3.3"
|
|
4
4
|
description = "MCP server that turns the OpenAIRE Graph into a ranked replication queue — impact-ranked targets and the Science Live verification overlay (retraction/supersession-aware)."
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
requires-python = ">=3.10"
|
|
@@ -4,6 +4,10 @@
|
|
|
4
4
|
|
|
5
5
|
const API = "https://api.openaire.eu/graph/v1";
|
|
6
6
|
const CLASS_SCORE = { C1: 1, C2: 0.8, C3: 0.6, C4: 0.4, C5: 0.2 };
|
|
7
|
+
// Per-class tooltip text so EVERY class (not just C1/C5) explains itself on hover.
|
|
8
|
+
const CLS_PCT = { C1: "top 0.01%", C2: "top 0.1%", C3: "top 1%", C4: "top 10%", C5: "the rest (outside the top 10%)" };
|
|
9
|
+
const impactTip = (c) => `OpenAIRE BIP! citation-impact class — ${c} = ${CLS_PCT[c] || "—"} most-cited across all of science (C1 highest · C5 lowest)`;
|
|
10
|
+
const impulseTip = (c) => `OpenAIRE BIP! impulse class — ${c} = ${CLS_PCT[c] || "—"} by recent citation momentum (C1 highest · C5 lowest)`;
|
|
7
11
|
const EXAMPLES = ["species distribution", "marine heatwave", "bumble bee climate", "presence-only", "range maps scale"];
|
|
8
12
|
|
|
9
13
|
let VERDICTS = {}; // doi -> [verifications]
|
|
@@ -63,26 +67,42 @@ const readinessFrom = (matScore, impactScore, momentum) => {
|
|
|
63
67
|
const r = 0.45 * (matScore || 0) + 0.35 * impactScore + 0.20 * momentum;
|
|
64
68
|
return Math.round(r * 100) / 100;
|
|
65
69
|
};
|
|
66
|
-
//
|
|
70
|
+
// Inline Lucide icons (monochrome, inherit currentColor → the app's navy/pink/grey palette).
|
|
71
|
+
const svg = (p, s = 13) => `<svg class="ic" width="${s}" height="${s}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.1" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">${p}</svg>`;
|
|
72
|
+
const ICON = {
|
|
73
|
+
robust: svg('<path d="M20 13c0 5-3.5 7.5-7.66 8.95a1 1 0 0 1-.67-.01C7.5 20.5 4 18 4 13V6a1 1 0 0 1 1-1c2 0 4.5-1.2 6.24-2.72a1.17 1.17 0 0 1 1.52 0C14.51 3.81 17 5 19 5a1 1 0 0 1 1 1z"/><path d="m9 12 2 2 4-4"/>'),
|
|
74
|
+
validated: svg('<circle cx="12" cy="12" r="10"/><path d="m9 12 2 2 4-4"/>'),
|
|
75
|
+
contested: svg('<path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3"/><path d="M12 9v4"/><path d="M12 17h.01"/>'),
|
|
76
|
+
refuted: svg('<circle cx="12" cy="12" r="10"/><path d="m15 9-6 6"/><path d="m9 9 6 6"/>'),
|
|
77
|
+
reproducible: svg('<path d="m17 2 4 4-4 4"/><path d="M3 11v-1a4 4 0 0 1 4-4h14"/><path d="m7 22-4-4 4-4"/><path d="M21 13v1a4 4 0 0 1-4 4H3"/>'),
|
|
78
|
+
needs: svg('<circle cx="12" cy="12" r="10"/><path d="M9.09 9a3 3 0 0 1 5.83 1c0 2-3 3-3 3"/><path d="M12 17h.01"/>'),
|
|
79
|
+
dormant: svg('<path d="M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z"/>'),
|
|
80
|
+
check: svg('<path d="M20 6 9 17l-5-5"/>', 13),
|
|
81
|
+
x: svg('<path d="M18 6 6 18"/><path d="m6 6 12 12"/>', 13),
|
|
82
|
+
star: svg('<path d="M11.5 2.3a.5.5 0 0 1 .9 0l2.3 4.7a2.1 2.1 0 0 0 1.6 1.1l5.2.8a.5.5 0 0 1 .3.9l-3.7 3.6a2.1 2.1 0 0 0-.6 1.9l.9 5.1a.5.5 0 0 1-.8.6l-4.6-2.4a2.1 2.1 0 0 0-2 0L6.7 21.3a.5.5 0 0 1-.8-.6l.9-5.1a2.1 2.1 0 0 0-.6-1.9l-3.7-3.6a.5.5 0 0 1 .3-.9l5.2-.8a2.1 2.1 0 0 0 1.6-1.1z"/>', 13),
|
|
83
|
+
fork: svg('<circle cx="12" cy="18" r="3"/><circle cx="6" cy="6" r="3"/><circle cx="18" cy="6" r="3"/><path d="M18 9v2c0 .6-.4 1-1 1H7c-.6 0-1-.4-1-1V9"/><path d="M12 12v3"/>', 13),
|
|
84
|
+
};
|
|
85
|
+
// Status taxonomy — "not replicated" is DISAMBIGUATED, not penalised. (label = plain text; icon = SVG)
|
|
67
86
|
const STATUS = {
|
|
68
|
-
robust: { label: "
|
|
69
|
-
validated: { label: "
|
|
70
|
-
contested: { label: "
|
|
71
|
-
refuted: { label: "
|
|
72
|
-
reproducible: { label: "
|
|
73
|
-
needs: { label: "
|
|
74
|
-
dormant: { label: "
|
|
87
|
+
robust: { label: "Robustly validated", icon: ICON.robust, cls: "st-val", tip: "multiple independent replications, all confirmed — a settled, reliable result" },
|
|
88
|
+
validated: { label: "Validated", icon: ICON.validated, cls: "st-val", tip: "independently replicated and it held up" },
|
|
89
|
+
contested: { label: "Contested", icon: ICON.contested, cls: "st-con", tip: "independent replications DISAGREE (some confirm, some contradict/partial) — worth re-checking" },
|
|
90
|
+
refuted: { label: "Refuted", icon: ICON.refuted, cls: "st-ref", tip: "independent replication(s) contradicted it, none confirmed" },
|
|
91
|
+
reproducible: { label: "Reproducible", icon: ICON.reproducible, cls: "st-ready", tip: "original code/data are available, so it can be RE-RUN (reproduced). Note: replication ≠ reproduction — replication tests the same claim with DIFFERENT data/methods (FORRT)." },
|
|
92
|
+
needs: { label: "Needs check", icon: ICON.needs, cls: "st-needs", tip: "not yet replicated and OpenAIRE links no materials — unknown (not absent); resolve from the paper" },
|
|
93
|
+
dormant: { label: "Dormant", icon: ICON.dormant, cls: "st-dorm", tip: "no verdict, older, low momentum, no materials surfaced — likely dormant" },
|
|
75
94
|
};
|
|
76
|
-
// Distinct replication OUTCOMES for a paper — one
|
|
77
|
-
// (deduped by outcome URI
|
|
95
|
+
// Distinct replication OUTCOMES for a paper — one SIGNED nanopub per independent replication
|
|
96
|
+
// (deduped by outcome URI). A record with no outcome nanopub isn't a replication (e.g. a stale
|
|
97
|
+
// non-verdict citation in the bundled fallback), so it's dropped — this keeps the count
|
|
98
|
+
// (agreementOf) and the rendered chips (outcomeLinks) in agreement.
|
|
78
99
|
const outcomesFor = (doi) => {
|
|
79
100
|
const seen = new Set(), out = [];
|
|
80
|
-
(VERDICTS[doi] || [])
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
});
|
|
101
|
+
for (const v of VERDICTS[doi] || []) {
|
|
102
|
+
if (!v.outcome_np || seen.has(v.outcome_np)) continue;
|
|
103
|
+
seen.add(v.outcome_np);
|
|
104
|
+
out.push({ np: v.outcome_np, verdict: v.verdict || "" });
|
|
105
|
+
}
|
|
86
106
|
return out;
|
|
87
107
|
};
|
|
88
108
|
// Colour an outcome chip by its verdict so the agreement pattern is legible at a glance.
|
|
@@ -146,12 +166,21 @@ async function search(topic, type, size) {
|
|
|
146
166
|
|
|
147
167
|
// resolve a single record by DOI (any type) — used for the original paper AND the
|
|
148
168
|
// replication's own OpenAIRE node. OpenAIRE free-text matches the DOI string.
|
|
169
|
+
// Session caches — re-scanning reuses successful lookups instead of re-hitting the APIs.
|
|
170
|
+
// This is what makes repeated scans STABLE (and stops exhausting GitHub's 60/hour unauth
|
|
171
|
+
// limit, which used to make FAIR badges flicker). Only successes are cached, so a transient
|
|
172
|
+
// failure retries next time and the cache converges to complete.
|
|
173
|
+
const _DOI = new Map(), _FAIR = new Map();
|
|
149
174
|
async function fetchByDoi(doi) {
|
|
150
175
|
if (!doi) return null;
|
|
176
|
+
if (_DOI.has(doi)) return _DOI.get(doi);
|
|
177
|
+
let res = null;
|
|
151
178
|
try {
|
|
152
179
|
const hits = (await (await fetch(`${API}/researchProducts?search=${encodeURIComponent(doi)}&pageSize=5`)).json()).results || [];
|
|
153
|
-
|
|
154
|
-
} catch (e) {
|
|
180
|
+
res = hits.find((h) => doiOf(h) === doi.toLowerCase()) || hits[0] || null;
|
|
181
|
+
} catch (e) { res = null; }
|
|
182
|
+
if (res) _DOI.set(doi, res);
|
|
183
|
+
return res;
|
|
155
184
|
}
|
|
156
185
|
|
|
157
186
|
// OpenAIRE richness we already receive but were hiding
|
|
@@ -181,6 +210,12 @@ async function githubFromZenodo(doi) {
|
|
|
181
210
|
}
|
|
182
211
|
|
|
183
212
|
async function assessSoftware(url) {
|
|
213
|
+
if (_FAIR.has(url)) return _FAIR.get(url); // cached success → stable across scans, saves rate limit
|
|
214
|
+
const res = await _assessSoftware(url);
|
|
215
|
+
if (res) _FAIR.set(url, res);
|
|
216
|
+
return res;
|
|
217
|
+
}
|
|
218
|
+
async function _assessSoftware(url) {
|
|
184
219
|
const g = parseGitHub(url);
|
|
185
220
|
if (!g) return null;
|
|
186
221
|
const base = `https://api.github.com/repos/${g.owner}/${g.repo}`;
|
|
@@ -223,15 +258,27 @@ const aidaText = (u) => { if (!u) return ""; try { return decodeURIComponent(u.r
|
|
|
223
258
|
const claimType = (u) => (!u ? "" : u.replace(/.*\/terms\//, "").replace(/-FORRT-Claim$/, "").replace(/_/g, " "));
|
|
224
259
|
const claimFor = (outcome_np) => CLAIMS[npHash(outcome_np)] || null;
|
|
225
260
|
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
261
|
+
// The public nanopub-query endpoint 504s intermittently under load; one retry turns most of
|
|
262
|
+
// those transient failures into success, so the app stays on the LIVE index instead of dropping
|
|
263
|
+
// to the bundled snapshot (the source of the "random" differences).
|
|
264
|
+
async function sparqlCsv(query, tries = 3) {
|
|
265
|
+
let lastErr;
|
|
266
|
+
for (let i = 0; i < tries; i++) {
|
|
267
|
+
try {
|
|
268
|
+
const r = await fetch(`${NP_SPARQL}?query=${encodeURIComponent(query)}`, { headers: { Accept: "text/csv" } });
|
|
269
|
+
if (!r.ok) throw new Error(`nanopub-query ${r.status}`);
|
|
270
|
+
const lines = (await r.text()).trim().split(/\r?\n/);
|
|
271
|
+
const head = lines.shift().split(",");
|
|
272
|
+
return lines.map((line) => {
|
|
273
|
+
const cells = (line.match(/("([^"]*)"|[^,]*)(,|$)/g) || []).map((c) => c.replace(/,$/, "").replace(/^"|"$/g, ""));
|
|
274
|
+
const o = {}; head.forEach((h, i) => (o[h] = cells[i])); return o;
|
|
275
|
+
});
|
|
276
|
+
} catch (e) {
|
|
277
|
+
lastErr = e;
|
|
278
|
+
if (i + 1 < tries) await new Promise((res) => setTimeout(res, 700 * (i + 1)));
|
|
279
|
+
}
|
|
280
|
+
}
|
|
281
|
+
throw lastErr;
|
|
235
282
|
}
|
|
236
283
|
|
|
237
284
|
async function buildIndexFromNetwork() {
|
|
@@ -479,17 +526,17 @@ function targetRow(t) {
|
|
|
479
526
|
: `replication priority = 0.45·materials + 0.35·impact + 0.20·momentum — materials ${p.mat == null ? "unverified" : p.mat.toFixed(2)} · impact ${(p.impact || 0).toFixed(2)} · momentum ${(p.momentum || 0).toFixed(2)}`;
|
|
480
527
|
const score = `<div class="score" title="${esc(scoreTitle)}"><span>${t.priority != null ? t.priority.toFixed(2) : "—"}</span><small>PRIORITY</small></div>`;
|
|
481
528
|
const st = STATUS[t.statusKey] || STATUS.needs;
|
|
482
|
-
const badge = `<span class="badge ${st.cls}" title="${esc(st.tip)}">${st.label}</span>`
|
|
483
|
-
+ (t.cls ? `<span class="badge cls" title="
|
|
529
|
+
const badge = `<span class="badge ${st.cls}" title="${esc(st.tip)}">${st.icon}${st.label}</span>`
|
|
530
|
+
+ (t.cls ? `<span class="badge cls" title="${esc(impactTip(t.cls))}">${t.cls}</span>` : "");
|
|
484
531
|
// Materials badge ONLY when positively known. OpenAIRE rarely links code/data to a
|
|
485
532
|
// paper, so 'unknown' is the norm in live search and would be noise on every row —
|
|
486
533
|
// it's carried in the score breakdown tooltip, and resolved in the baked demo set.
|
|
487
|
-
const matMeta = (t.mat && t.mat.state === "rocrate") ? `<span class="badge mok" title="RO-Crate research object — code + data + provenance bundled"
|
|
488
|
-
: (t.mat && t.mat.state === "code") ? `<span class="badge mok" title="code repository linked to this paper"
|
|
534
|
+
const matMeta = (t.mat && t.mat.state === "rocrate") ? `<span class="badge mok" title="RO-Crate research object — code + data + provenance bundled">${ICON.check}RO-Crate</span>`
|
|
535
|
+
: (t.mat && t.mat.state === "code") ? `<span class="badge mok" title="code repository linked to this paper">${ICON.check}code</span>`
|
|
489
536
|
: "";
|
|
490
537
|
const meta = `<div class="t-meta">`
|
|
491
538
|
+ (t.year ? `<span class="badge yr">${t.year}</span>` : "")
|
|
492
|
-
+ (t.impl ? `<span class="badge imp" title="
|
|
539
|
+
+ (t.impl ? `<span class="badge imp" title="${esc(impulseTip(t.impl))}">impulse ${t.impl}</span>` : "")
|
|
493
540
|
+ matMeta + `</div>`;
|
|
494
541
|
const link = t.doi ? `<a href="https://doi.org/${t.doi}" target="_blank" rel="noopener">${t.doi}</a>` : "";
|
|
495
542
|
// what EXACTLY was replicated — the claim's AIDA statement (atomic sentence) + its FORRT type
|
|
@@ -498,12 +545,12 @@ function targetRow(t) {
|
|
|
498
545
|
? `<div class="tclaim"><span class="claimlbl">claim:</span> <span class="claimq">“${esc(cl.aida || cl.label)}”</span>${cl.type ? ` <span class="badge ctype" title="FORRT claim type">${esc(cl.type)}</span>` : ""}</div>` : "";
|
|
499
546
|
const outs = t.status === "VERIFIED" ? outcomesFor(t.doi).filter((o) => o.np) : [];
|
|
500
547
|
const verdictLink = (t.status === "VERIFIED")
|
|
501
|
-
? `<div class="tverdict">independently checked by Science Live — <b>${esc(agreementOf(t.doi).why)}</b>${outcomeLinks(outs)}</div>` : "";
|
|
548
|
+
? `<div class="tverdict">independently checked by <a href="https://sciencelive4all.org" target="_blank" rel="noopener">Science Live</a> — <b>${esc(agreementOf(t.doi).why)}</b>${outcomeLinks(outs)}</div>` : "";
|
|
502
549
|
const resolvedNote = (t.mat && t.mat.resolved)
|
|
503
550
|
? `<div class="tresolved">↳ materials resolved from ${esc(t.mat.source || "the paper")} (not in OpenAIRE): <a href="${esc(t.mat.code || "")}" target="_blank" rel="noopener">code repo</a>${(t.mat.data && t.mat.data.length) ? ` · data: ${t.mat.data.map(esc).join(", ")}` : ""}</div>`
|
|
504
551
|
: "";
|
|
505
552
|
const fairNote = t.fair
|
|
506
|
-
? `<div class="tfair">FAIR software <b>${t.fair.score}/5</b> ·
|
|
553
|
+
? `<div class="tfair">FAIR software <b>${t.fair.score}/5</b> · ${ICON.star}${t.fair.stars}${t.fair.swh ? " · in Software Heritage" : ""}</div>`
|
|
507
554
|
: "";
|
|
508
555
|
// OPEN targets get a next step: discovery here → the FORRT template handles the nanopub chain.
|
|
509
556
|
const replicateCTA = (t.status !== "VERIFIED")
|
|
@@ -566,16 +613,18 @@ function renderVerified(inField) {
|
|
|
566
613
|
const matchCard = (v) => {
|
|
567
614
|
const chips = [
|
|
568
615
|
v.oa ? `<span class="ochip oa" title="Open-access route (OpenAIRE) — how the paper is free to read: gold/diamond = OA journal, hybrid = OA in a subscription journal, green = self-archived copy, bronze = free on the publisher site with no open licence">${esc(v.oa)} OA</span>` : "",
|
|
569
|
-
...(v.fos || []).map((f) => `<span class="ochip">${esc(f).slice(0, 24)}</span>`),
|
|
570
|
-
...(v.sdg || []).map((s) =>
|
|
571
|
-
|
|
616
|
+
...(v.fos || []).map((f) => `<span class="ochip" title="Field of Science (OECD FOS scheme) — subject classification assigned by OpenAIRE">${esc(f).slice(0, 24)}</span>`),
|
|
617
|
+
...(v.sdg || []).map((s) => { const m = /^(\d+)\./.exec(s); return m
|
|
618
|
+
? `<a class="ochip sdg" href="https://sdgs.un.org/goals/goal${m[1]}" target="_blank" rel="noopener" title="UN Sustainable Development Goal ${m[1]} — assigned by OpenAIRE's SDG classifier (opens the UN definition)">${esc(s).slice(0, 22)}</a>`
|
|
619
|
+
: `<span class="ochip sdg" title="UN Sustainable Development Goal — assigned by OpenAIRE's SDG classifier">${esc(s).slice(0, 22)}</span>`; }),
|
|
620
|
+
`<span class="ochip cites" title="Citation count from the OpenAIRE Graph">${v.citations.toLocaleString()} cites</span>`,
|
|
572
621
|
].join("");
|
|
573
622
|
let repl = "";
|
|
574
623
|
if (v.repl) {
|
|
575
624
|
const f = v.repl.fair;
|
|
576
625
|
const fairBadge = f
|
|
577
|
-
? `<div class="fairrecs"><b>FAIR software (${f.score}/5):</b> ${Object.entries(f.recs).map(([k, ok]) => `<span class="${ok ? "rok" : "rno"}">${ok ?
|
|
578
|
-
<div class="fairline"
|
|
626
|
+
? `<div class="fairrecs"><b>FAIR software (${f.score}/5):</b> ${Object.entries(f.recs).map(([k, ok]) => `<span class="${ok ? "rok" : "rno"}">${ok ? ICON.check : ICON.x}${k}</span>`).join("")}</div>
|
|
627
|
+
<div class="fairline">${ICON.star}${f.stars} stars · ${ICON.fork}${f.forks} forks · ${f.swh ? `<span class="swhok">in Software Heritage</span>` : `<span class="swhno">not yet in Software Heritage</span>`}</div>`
|
|
579
628
|
: "";
|
|
580
629
|
const nodeHref = v.repl.code && v.repl.code.includes("github") ? v.repl.code : v.repl.url;
|
|
581
630
|
repl = `<div class="vrepl">↳ replication is an OpenAIRE node: <a href="${nodeHref}" target="_blank" rel="noopener">${esc(v.repl.title).slice(0, 44) || v.repl.doi}</a> <span class="ochip type">${esc(v.repl.type)}</span></div>${fairBadge}`;
|
|
@@ -591,14 +640,14 @@ function renderVerified(inField) {
|
|
|
591
640
|
<span class="vt">${esc(v.title).slice(0, 82)}</span>
|
|
592
641
|
<div class="ochips">${chips}</div>
|
|
593
642
|
${vClaimLine}
|
|
594
|
-
<div class="vline"><span class="vv ${partialOf(v) ? "partial" : ""}">${v.verdicts.join(", ")}</span> — independently checked by Science Live ${vouts.length ? outcomeLinks(vouts) : (v.cito_np ? `· <a href="${v.cito_np}" target="_blank" rel="noopener">verdict chain →</a>` : "")}</div>
|
|
643
|
+
<div class="vline"><span class="vv ${partialOf(v) ? "partial" : ""}">${v.verdicts.join(", ")}</span> — independently checked by <a href="https://sciencelive4all.org" target="_blank" rel="noopener">Science Live</a> ${vouts.length ? outcomeLinks(vouts) : (v.cito_np ? `· <a href="${v.cito_np}" target="_blank" rel="noopener">verdict chain →</a>` : "")}</div>
|
|
595
644
|
${repl}
|
|
596
645
|
</li>`;
|
|
597
646
|
};
|
|
598
647
|
const fieldHtml = field.length
|
|
599
648
|
? `<ul class="vlist">${field.map(matchCard).join("")}</ul>`
|
|
600
649
|
: `<p class="vnone">No Science Live verdict matching your search yet — every paper on the left is an <b>open</b> replication opportunity.</p>`;
|
|
601
|
-
const moreHtml = `<p class="vmore-stat">Verdicts are
|
|
650
|
+
const moreHtml = `<p class="vmore-stat">Verdicts are read <b>live</b> from the nanopub network — any signer, retraction- and supersession-filtered. <a href="methodology.html">How this works →</a></p>`;
|
|
602
651
|
el("verified").innerHTML = fieldHtml + moreHtml;
|
|
603
652
|
}
|
|
604
653
|
|
|
@@ -654,7 +703,7 @@ async function run(topic, isExample) {
|
|
|
654
703
|
}
|
|
655
704
|
|
|
656
705
|
// ---------- wire up ----------
|
|
657
|
-
el("chips").innerHTML = EXAMPLES.map((e) => `<
|
|
706
|
+
el("chips").innerHTML = EXAMPLES.map((e) => `<button type="button" class="chip">${e}</button>`).join("");
|
|
658
707
|
el("chips").addEventListener("click", (e) => { if (e.target.classList.contains("chip")) { el("topic").value = e.target.textContent; run(e.target.textContent); } });
|
|
659
708
|
el("go").addEventListener("click", () => run(el("topic").value));
|
|
660
709
|
el("topic").addEventListener("keydown", (e) => { if (e.key === "Enter") run(el("topic").value); });
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
<div class="wrap">
|
|
13
13
|
<h1>Replication Radar</h1>
|
|
14
14
|
<p class="tag">What's worth replicating next — read straight off the OpenAIRE Graph.</p>
|
|
15
|
-
<p class="sub">Science Live × OpenAIRE · impact-ranked targets · the claim each makes · already-checked overlay</p>
|
|
15
|
+
<p class="sub">Science Live × OpenAIRE · impact-ranked targets · the claim each makes · already-checked overlay · <a href="methodology.html" style="color:#fff;text-decoration:underline">how it works & where the data comes from</a></p>
|
|
16
16
|
</div>
|
|
17
17
|
</header>
|
|
18
18
|
|
|
@@ -23,7 +23,7 @@
|
|
|
23
23
|
<div class="chips" id="chips"></div>
|
|
24
24
|
</section>
|
|
25
25
|
|
|
26
|
-
<p id="status" class="status"></p>
|
|
26
|
+
<p id="status" class="status" role="status" aria-live="polite"></p>
|
|
27
27
|
|
|
28
28
|
<section id="results" hidden>
|
|
29
29
|
<div class="card chartcard">
|
|
@@ -48,8 +48,11 @@
|
|
|
48
48
|
|
|
49
49
|
<footer>
|
|
50
50
|
<div class="wrap">
|
|
51
|
-
<span
|
|
52
|
-
<
|
|
51
|
+
<span class="svc">Want a claim replicated but can't run it yourself? <a href="https://vitenhub.no/replication-services" target="_blank" rel="noopener">Science Live offers replication as a service →</a></span>
|
|
52
|
+
<div class="foot-links">
|
|
53
|
+
<span>Live against <a href="https://graph.openaire.eu/">OpenAIRE Graph</a> · verdicts from <a href="https://sciencelive4all.org">Science Live</a> nanopublications</span>
|
|
54
|
+
<span><a href="methodology.html">methodology & sources</a> · <a href="https://github.com/ScienceLiveHub/replication-radar">code (MIT)</a> · <a href="https://pypi.org/project/replication-radar/">PyPI</a> · <a href="https://github.com/ScienceLiveHub/replication-radar/blob/main/STORY.md">story (CC-BY)</a></span>
|
|
55
|
+
</div>
|
|
53
56
|
</div>
|
|
54
57
|
</footer>
|
|
55
58
|
|