pearmut 0.1.1__tar.gz → 0.1.3__tar.gz

{pearmut-0.1.1 → pearmut-0.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pearmut
-Version: 0.1.1
+Version: 0.1.3
 Summary: A tool for evaluation of model outputs, primarily MT.
 Author-email: Vilém Zouhar <vilem.zouhar@gmail.com>
 License: apache-2.0
@@ -23,7 +23,7 @@ Dynamic: license-file
 
 Pearmut is a **Platform for Evaluation and Reviewing of Multilingual Tasks**.
 It evaluates model outputs, primarily translation but also various other NLP tasks.
-Supports multimodality (text, video, audio, images) and a variety of annotation protocols (DA, ESA, MQM, paired ESA, etc).
+Supports multimodality (text, video, audio, images) and a variety of annotation protocols ([DA](https://aclanthology.org/N15-1124/), [ESA](https://aclanthology.org/2024.wmt-1.131/), [ESA<sup>AI</sup>](https://aclanthology.org/2025.naacl-long.255/), [MQM](https://doi.org/10.1162/tacl_a_00437), paired ESA, etc).
 
 [![PyPi version](https://badgen.net/pypi/v/pearmut/)](https://pypi.org/project/pearmut)
 &nbsp;
@@ -31,7 +31,7 @@ Supports multimodality (text, video, audio, images) and a variety of annotation
 &nbsp;
 [![PyPi license](https://badgen.net/pypi/license/pearmut/)](https://pypi.org/project/pearmut/)
 &nbsp;
-[![build status](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml)
+[![build status](https://github.com/zouharvi/pearmut/actions/workflows/test.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/test.yml)
 
 <img width="1000" alt="Screenshot of ESA/MQM interface" src="https://github.com/user-attachments/assets/f14c91a5-44d7-4248-ada9-387e95ca59d0" />
 
@@ -63,7 +63,7 @@ One of the simplest ones, where each user has a pre-defined list of tasks (`task
 ```python
 {
   "info": {
-    "type": "task-based",
+    "assignment": "task-based",
     "template": "pointwise",
     "protocol_score": true,                 # we want scores [0...100] for each segment
     "protocol_error_spans": true,           # we want error spans
@@ -115,19 +115,51 @@ For the standard ones (ESA, DA, MQM), we expect each item to be a dictionary (co
 ... # definition of another item (document)
 ```
 
-We also support a super simple allocation of annotations (`task-single`, not yet ⚠️), where you simply pass a list of all examples to be evaluated and they are processed in parallel by all annotators:
+## Pre-filled Error Spans (ESA<sup>AI</sup> Support)
+
+For workflows where you want to provide pre-filled error annotations (e.g., ESA<sup>AI</sup>), you can include an `error_spans` key in each item.
+These spans will be loaded into the interface as existing annotations that users can review, modify, or delete.
+
+```python
+{
+  "src": "The quick brown fox jumps over the lazy dog.",
+  "tgt": "Rychlá hnědá liška skáče přes líného psa.",
+  "error_spans": [
+    {
+      "start_i": 0,         # character index start (inclusive)
+      "end_i": 5,           # character index end (inclusive)
+      "severity": "minor",  # "minor", "major", "neutral", or null
+      "category": null      # MQM category string or null
+    },
+    {
+      "start_i": 27,
+      "end_i": 32,
+      "severity": "major",
+      "category": null
+    }
+  ]
+}
+```
+
+For **listwise** template, `error_spans` is a 2D array where each inner array corresponds to error spans for that candidate.
+
+See [examples/esaai_prefilled.json](examples/esaai_prefilled.json) for a complete example.
+
+## Single-stream Assignment
+
+We also support a simple allocation where all annotators draw from the same pool (`single-stream`). Items are randomly assigned to annotators from the pool of unfinished items:
 ```python
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "task-single",
+        "assignment": "single-stream",
         "template": "pointwise",
         "protocol_score": True,                # collect scores
         "protocol_error_spans": True,          # collect error spans
         "protocol_error_categories": False,    # do not collect MQM categories, so ESA
-        "users": 50,
+        "num_users": 50,                       # number of annotators
     },
-    "data": [...], # list of all items
+    "data": [...], # list of all items (shared among all annotators)
 }
 ```
 
@@ -137,10 +169,10 @@ We also support dynamic allocation of annotations (`dynamic`, not yet ⚠️), w
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "dynamic",
-        "template": "kway",
+        "assignment": "dynamic",
+        "template": "listwise",
         "protocol_k": 5,
-        "users": 50,
+        "num_users": 50,
     },
     "data": [...], # list of all items
 }
@@ -154,6 +186,25 @@ pearmut add my_campaign_4.json
 pearmut run
 ```
 
+## Campaign options
+
+In summary, you can select from the assignment types
+
+- `task-based`: each user has a predefined set of items
+- `single-stream`: all users are annotating together the same set of items
+- `dynamic`: WIP ⚠️
+
+and independently of that select your protocol template:
+
+- `pointwise`: evaluate a single output given a single output
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+- `listwise`: evaluate multiple outputs at the same time given a single output ⚠️
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+
 ## Campaign management
 
 When adding new campaigns or launching pearmut, a management link is shown that gives an overview of annotator progress but also an easy access to the annotation links or resetting the task progress (no data will be lost).
@@ -170,7 +221,7 @@ An intentionally incorrect token can be shown if the annotations don't pass qual
 
 We also support anything HTML-compatible both on the input and on the output.
 This includes embedded YouTube videos, or even simple `<video ` tags that point to some resource somewhere.
-For an example, try [examples/mock_multimodal.json](examples/mock_multimodal.json).
+For an example, try [examples/multimodal.json](examples/multimodal.json).
 Tip: make sure the elements are already appropriately styled.
 
 <img width="800" alt="Preview of multimodal elements in Pearmut" src="https://github.com/user-attachments/assets/f34a1a3e-ad95-4114-95ee-8a49e8003faf" />
@@ -185,7 +236,7 @@ To make changes locally, clone the repository and run the following, which will
 cd pearmut
 # watch the frontend for changes (in a separate terminal)
 npm install web/ --prefix web/
-npm run watch --prefix web/
+npm run build --prefix web/ # `watch` for rebuild on code change
 
 # install local package as editable
 pip3 install -e .

{pearmut-0.1.1 → pearmut-0.1.3}/README.md

@@ -2,7 +2,7 @@
 
 Pearmut is a **Platform for Evaluation and Reviewing of Multilingual Tasks**.
 It evaluates model outputs, primarily translation but also various other NLP tasks.
-Supports multimodality (text, video, audio, images) and a variety of annotation protocols (DA, ESA, MQM, paired ESA, etc).
+Supports multimodality (text, video, audio, images) and a variety of annotation protocols ([DA](https://aclanthology.org/N15-1124/), [ESA](https://aclanthology.org/2024.wmt-1.131/), [ESA<sup>AI</sup>](https://aclanthology.org/2025.naacl-long.255/), [MQM](https://doi.org/10.1162/tacl_a_00437), paired ESA, etc).
 
 [![PyPi version](https://badgen.net/pypi/v/pearmut/)](https://pypi.org/project/pearmut)
 &nbsp;
@@ -10,7 +10,7 @@ Supports multimodality (text, video, audio, images) and a variety of annotation
 &nbsp;
 [![PyPi license](https://badgen.net/pypi/license/pearmut/)](https://pypi.org/project/pearmut/)
 &nbsp;
-[![build status](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml)
+[![build status](https://github.com/zouharvi/pearmut/actions/workflows/test.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/test.yml)
 
 <img width="1000" alt="Screenshot of ESA/MQM interface" src="https://github.com/user-attachments/assets/f14c91a5-44d7-4248-ada9-387e95ca59d0" />
 
@@ -42,7 +42,7 @@ One of the simplest ones, where each user has a pre-defined list of tasks (`task
 ```python
 {
   "info": {
-    "type": "task-based",
+    "assignment": "task-based",
     "template": "pointwise",
     "protocol_score": true,                 # we want scores [0...100] for each segment
     "protocol_error_spans": true,           # we want error spans
@@ -94,19 +94,51 @@ For the standard ones (ESA, DA, MQM), we expect each item to be a dictionary (co
 ... # definition of another item (document)
 ```
 
-We also support a super simple allocation of annotations (`task-single`, not yet ⚠️), where you simply pass a list of all examples to be evaluated and they are processed in parallel by all annotators:
+## Pre-filled Error Spans (ESA<sup>AI</sup> Support)
+
+For workflows where you want to provide pre-filled error annotations (e.g., ESA<sup>AI</sup>), you can include an `error_spans` key in each item.
+These spans will be loaded into the interface as existing annotations that users can review, modify, or delete.
+
+```python
+{
+  "src": "The quick brown fox jumps over the lazy dog.",
+  "tgt": "Rychlá hnědá liška skáče přes líného psa.",
+  "error_spans": [
+    {
+      "start_i": 0,         # character index start (inclusive)
+      "end_i": 5,           # character index end (inclusive)
+      "severity": "minor",  # "minor", "major", "neutral", or null
+      "category": null      # MQM category string or null
+    },
+    {
+      "start_i": 27,
+      "end_i": 32,
+      "severity": "major",
+      "category": null
+    }
+  ]
+}
+```
+
+For **listwise** template, `error_spans` is a 2D array where each inner array corresponds to error spans for that candidate.
+
+See [examples/esaai_prefilled.json](examples/esaai_prefilled.json) for a complete example.
+
+## Single-stream Assignment
+
+We also support a simple allocation where all annotators draw from the same pool (`single-stream`). Items are randomly assigned to annotators from the pool of unfinished items:
 ```python
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "task-single",
+        "assignment": "single-stream",
         "template": "pointwise",
         "protocol_score": True,                # collect scores
         "protocol_error_spans": True,          # collect error spans
         "protocol_error_categories": False,    # do not collect MQM categories, so ESA
-        "users": 50,
+        "num_users": 50,                       # number of annotators
     },
-    "data": [...], # list of all items
+    "data": [...], # list of all items (shared among all annotators)
 }
 ```
 
@@ -116,10 +148,10 @@ We also support dynamic allocation of annotations (`dynamic`, not yet ⚠️), w
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "dynamic",
-        "template": "kway",
+        "assignment": "dynamic",
+        "template": "listwise",
         "protocol_k": 5,
-        "users": 50,
+        "num_users": 50,
     },
     "data": [...], # list of all items
 }
@@ -133,6 +165,25 @@ pearmut add my_campaign_4.json
 pearmut run
 ```
 
+## Campaign options
+
+In summary, you can select from the assignment types
+
+- `task-based`: each user has a predefined set of items
+- `single-stream`: all users are annotating together the same set of items
+- `dynamic`: WIP ⚠️
+
+and independently of that select your protocol template:
+
+- `pointwise`: evaluate a single output given a single output
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+- `listwise`: evaluate multiple outputs at the same time given a single output ⚠️
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+
 ## Campaign management
 
 When adding new campaigns or launching pearmut, a management link is shown that gives an overview of annotator progress but also an easy access to the annotation links or resetting the task progress (no data will be lost).
@@ -149,7 +200,7 @@ An intentionally incorrect token can be shown if the annotations don't pass qual
 
 We also support anything HTML-compatible both on the input and on the output.
 This includes embedded YouTube videos, or even simple `<video ` tags that point to some resource somewhere.
-For an example, try [examples/mock_multimodal.json](examples/mock_multimodal.json).
+For an example, try [examples/multimodal.json](examples/multimodal.json).
 Tip: make sure the elements are already appropriately styled.
 
 <img width="800" alt="Preview of multimodal elements in Pearmut" src="https://github.com/user-attachments/assets/f34a1a3e-ad95-4114-95ee-8a49e8003faf" />
@@ -164,7 +215,7 @@ To make changes locally, clone the repository and run the following, which will
 cd pearmut
 # watch the frontend for changes (in a separate terminal)
 npm install web/ --prefix web/
-npm run watch --prefix web/
+npm run build --prefix web/ # `watch` for rebuild on code change
 
 # install local package as editable
 pip3 install -e .

{pearmut-0.1.1 → pearmut-0.1.3}/pearmut.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pearmut
-Version: 0.1.1
+Version: 0.1.3
 Summary: A tool for evaluation of model outputs, primarily MT.
 Author-email: Vilém Zouhar <vilem.zouhar@gmail.com>
 License: apache-2.0
@@ -23,7 +23,7 @@ Dynamic: license-file
 
 Pearmut is a **Platform for Evaluation and Reviewing of Multilingual Tasks**.
 It evaluates model outputs, primarily translation but also various other NLP tasks.
-Supports multimodality (text, video, audio, images) and a variety of annotation protocols (DA, ESA, MQM, paired ESA, etc).
+Supports multimodality (text, video, audio, images) and a variety of annotation protocols ([DA](https://aclanthology.org/N15-1124/), [ESA](https://aclanthology.org/2024.wmt-1.131/), [ESA<sup>AI</sup>](https://aclanthology.org/2025.naacl-long.255/), [MQM](https://doi.org/10.1162/tacl_a_00437), paired ESA, etc).
 
 [![PyPi version](https://badgen.net/pypi/v/pearmut/)](https://pypi.org/project/pearmut)
 &nbsp;
@@ -31,7 +31,7 @@ Supports multimodality (text, video, audio, images) and a variety of annotation
 &nbsp;
 [![PyPi license](https://badgen.net/pypi/license/pearmut/)](https://pypi.org/project/pearmut/)
 &nbsp;
-[![build status](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/ci.yml)
+[![build status](https://github.com/zouharvi/pearmut/actions/workflows/test.yml/badge.svg)](https://github.com/zouharvi/pearmut/actions/workflows/test.yml)
 
 <img width="1000" alt="Screenshot of ESA/MQM interface" src="https://github.com/user-attachments/assets/f14c91a5-44d7-4248-ada9-387e95ca59d0" />
 
@@ -63,7 +63,7 @@ One of the simplest ones, where each user has a pre-defined list of tasks (`task
 ```python
 {
   "info": {
-    "type": "task-based",
+    "assignment": "task-based",
     "template": "pointwise",
     "protocol_score": true,                 # we want scores [0...100] for each segment
     "protocol_error_spans": true,           # we want error spans
@@ -115,19 +115,51 @@ For the standard ones (ESA, DA, MQM), we expect each item to be a dictionary (co
 ... # definition of another item (document)
 ```
 
-We also support a super simple allocation of annotations (`task-single`, not yet ⚠️), where you simply pass a list of all examples to be evaluated and they are processed in parallel by all annotators:
+## Pre-filled Error Spans (ESA<sup>AI</sup> Support)
+
+For workflows where you want to provide pre-filled error annotations (e.g., ESA<sup>AI</sup>), you can include an `error_spans` key in each item.
+These spans will be loaded into the interface as existing annotations that users can review, modify, or delete.
+
+```python
+{
+  "src": "The quick brown fox jumps over the lazy dog.",
+  "tgt": "Rychlá hnědá liška skáče přes líného psa.",
+  "error_spans": [
+    {
+      "start_i": 0,         # character index start (inclusive)
+      "end_i": 5,           # character index end (inclusive)
+      "severity": "minor",  # "minor", "major", "neutral", or null
+      "category": null      # MQM category string or null
+    },
+    {
+      "start_i": 27,
+      "end_i": 32,
+      "severity": "major",
+      "category": null
+    }
+  ]
+}
+```
+
+For **listwise** template, `error_spans` is a 2D array where each inner array corresponds to error spans for that candidate.
+
+See [examples/esaai_prefilled.json](examples/esaai_prefilled.json) for a complete example.
+
+## Single-stream Assignment
+
+We also support a simple allocation where all annotators draw from the same pool (`single-stream`). Items are randomly assigned to annotators from the pool of unfinished items:
 ```python
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "task-single",
+        "assignment": "single-stream",
         "template": "pointwise",
         "protocol_score": True,                # collect scores
         "protocol_error_spans": True,          # collect error spans
         "protocol_error_categories": False,    # do not collect MQM categories, so ESA
-        "users": 50,
+        "num_users": 50,                       # number of annotators
     },
-    "data": [...], # list of all items
+    "data": [...], # list of all items (shared among all annotators)
 }
 ```
 
@@ -137,10 +169,10 @@ We also support dynamic allocation of annotations (`dynamic`, not yet ⚠️), w
 {
     "campaign_id": "my campaign 6",
     "info": {
-        "type": "dynamic",
-        "template": "kway",
+        "assignment": "dynamic",
+        "template": "listwise",
         "protocol_k": 5,
-        "users": 50,
+        "num_users": 50,
     },
     "data": [...], # list of all items
 }
@@ -154,6 +186,25 @@ pearmut add my_campaign_4.json
 pearmut run
 ```
 
+## Campaign options
+
+In summary, you can select from the assignment types
+
+- `task-based`: each user has a predefined set of items
+- `single-stream`: all users are annotating together the same set of items
+- `dynamic`: WIP ⚠️
+
+and independently of that select your protocol template:
+
+- `pointwise`: evaluate a single output given a single output
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+- `listwise`: evaluate multiple outputs at the same time given a single output ⚠️
+  - `protocol_score`: ask for score 0 to 100
+  - `protocol_error_spans`: ask for highlighting error spans
+  - `protocol_error_categories`: ask for highlighting error categories
+
 ## Campaign management
 
 When adding new campaigns or launching pearmut, a management link is shown that gives an overview of annotator progress but also an easy access to the annotation links or resetting the task progress (no data will be lost).
@@ -170,7 +221,7 @@ An intentionally incorrect token can be shown if the annotations don't pass qual
 
 We also support anything HTML-compatible both on the input and on the output.
 This includes embedded YouTube videos, or even simple `<video ` tags that point to some resource somewhere.
-For an example, try [examples/mock_multimodal.json](examples/mock_multimodal.json).
+For an example, try [examples/multimodal.json](examples/multimodal.json).
 Tip: make sure the elements are already appropriately styled.
 
 <img width="800" alt="Preview of multimodal elements in Pearmut" src="https://github.com/user-attachments/assets/f34a1a3e-ad95-4114-95ee-8a49e8003faf" />
@@ -185,7 +236,7 @@ To make changes locally, clone the repository and run the following, which will
 cd pearmut
 # watch the frontend for changes (in a separate terminal)
 npm install web/ --prefix web/
-npm run watch --prefix web/
+npm run build --prefix web/ # `watch` for rebuild on code change
 
 # install local package as editable
 pip3 install -e .

{pearmut-0.1.1 → pearmut-0.1.3}/pearmut.egg-info/SOURCES.txt

@@ -8,12 +8,14 @@ pearmut.egg-info/entry_points.txt
 pearmut.egg-info/requires.txt
 pearmut.egg-info/top_level.txt
 server/app.py
+server/assignment.py
 server/cli.py
-server/protocols.py
 server/utils.py
 server/static/dashboard.bundle.js
 server/static/dashboard.html
 server/static/index.html
+server/static/listwise.bundle.js
+server/static/listwise.html
 server/static/pointwise.bundle.js
 server/static/pointwise.html
 server/static/assets/favicon.svg

{pearmut-0.1.1 → pearmut-0.1.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pearmut"
-version = "0.1.1"
+version = "0.1.3"
 description = "A tool for evaluation of model outputs, primarily MT."
 readme = "README.md"
 license = { text = "apache-2.0" }

{pearmut-0.1.1 → pearmut-0.1.3}/server/app.py

@@ -8,8 +8,8 @@ from fastapi.responses import JSONResponse
 from fastapi.staticfiles import StaticFiles
 from pydantic import BaseModel
 
-from .protocols import get_next_item, reset_task, update_progress
-from .utils import ROOT, load_progress_data, save_progress_data
+from .assignment import get_i_item, get_next_item, reset_task, update_progress
+from .utils import ROOT, load_progress_data, save_db_payload, save_progress_data
 
 os.makedirs(f"{ROOT}/data/outputs", exist_ok=True)
 
@@ -36,7 +36,7 @@ class LogResponseRequest(BaseModel):
     campaign_id: str
     user_id: str
     item_i: int
-    payload: Any
+    payload: dict[str, Any]
 
 
 @app.post("/log-response")
@@ -45,6 +45,7 @@ async def _log_response(request: LogResponseRequest):
 
     campaign_id = request.campaign_id
     user_id = request.user_id
+    item_i = request.item_i
 
     if campaign_id not in progress_data:
         return JSONResponse(content={"error": "Unknown campaign ID"}, status_code=400)
@@ -52,8 +53,7 @@ async def _log_response(request: LogResponseRequest):
         return JSONResponse(content={"error": "Unknown user ID"}, status_code=400)
 
     # append response to the output log
-    with open(f"{ROOT}/data/outputs/{campaign_id}.jsonl", "a") as log_file:
-        log_file.write(json.dumps(request.payload, ensure_ascii=False) + "\n")
+    save_db_payload(campaign_id, request.payload | {"user_id": user_id, "item_i": item_i})
 
     # if actions were submitted, we can log time data
     if "actions" in request.payload:
@@ -97,6 +97,32 @@ async def _get_next_item(request: NextItemRequest):
     )
 
 
+class GetItemRequest(BaseModel):
+    campaign_id: str
+    user_id: str
+    item_i: int
+
+
+@app.post("/get-i-item")
+async def _get_i_item(request: GetItemRequest):
+    campaign_id = request.campaign_id
+    user_id = request.user_id
+    item_i = request.item_i
+
+    if campaign_id not in progress_data:
+        return JSONResponse(content={"error": "Unknown campaign ID"}, status_code=400)
+    if user_id not in progress_data[campaign_id]:
+        return JSONResponse(content={"error": "Unknown user ID"}, status_code=400)
+
+    return get_i_item(
+        campaign_id,
+        user_id,
+        tasks_data,
+        progress_data,
+        item_i,
+    )
+
+
 class DashboardDataRequest(BaseModel):
     campaign_id: str
     token: str | None = None
@@ -111,19 +137,20 @@ async def _dashboard_data(request: DashboardDataRequest):
     if campaign_id not in progress_data:
         return JSONResponse(content={"error": "Unknown campaign ID"}, status_code=400)
 
-    progress_new = {
-        user_id: {
-            **user_val,
-            "total": len(tasks_data[campaign_id]["data"][user_id]),
-        } | (
-            # override if not privileged
-            {
-                "token_correct": None,
-                "token_incorrect": None,
-            } if not is_privileged else {}
-        )
-        for user_id, user_val in progress_data[campaign_id].items()
-    }
+    progress_new = {}
+    assignment = tasks_data[campaign_id]["info"]["assignment"]
+    if assignment not in ["task-based", "single-stream"]:
+        return JSONResponse(content={"error": "Unsupported campaign assignment type"}, status_code=400)
+
+    for user_id, user_val in progress_data[campaign_id].items():
+        # shallow copy
+        entry = dict(user_val)
+
+        if not is_privileged:
+            entry["token_correct"] = None
+            entry["token_incorrect"] = None
+
+        progress_new[user_id] = entry
 
     return JSONResponse(
         content={
@@ -190,19 +217,22 @@ async def _download_progress(
         return JSONResponse(content={"error": "Mismatched campaign_id and token count"}, status_code=400)
 
     output = {}
-    for campaign_id, campaign_id in enumerate(campaign_id):
-        if campaign_id not in progress_data:
-            return JSONResponse(content={"error": f"Unknown campaign ID {campaign_id}"}, status_code=400)
-        if token[campaign_id] != tasks_data[campaign_id]["token"]:
-            return JSONResponse(content={"error": f"Invalid token for campaign ID {campaign_id}"}, status_code=400)
+    for i, cid in enumerate(campaign_id):
+        if cid not in progress_data:
+            return JSONResponse(content={"error": f"Unknown campaign ID {cid}"}, status_code=400)
+        if token[i] != tasks_data[cid]["token"]:
+            return JSONResponse(content={"error": f"Invalid token for campaign ID {cid}"}, status_code=400)
 
-        output[campaign_id] = progress_data[campaign_id]
+        output[cid] = progress_data[cid]
 
     return JSONResponse(content=output, status_code=200)
 
+static_dir = f"{os.path.dirname(os.path.abspath(__file__))}/static/"
+if not os.path.exists(static_dir + "index.html"):
+    raise FileNotFoundError("Static directory not found. Please build the frontend first.")
 
 app.mount(
     "/",
-    StaticFiles(directory=f"{os.path.dirname(os.path.abspath(__file__))}/static/" , html=True, follow_symlink=True),
+    StaticFiles(directory=static_dir, html=True, follow_symlink=True),
     name="static",
 )