olympiad-lab 0.1.0__tar.gz

olympiad_lab-0.1.0/CHANGELOG.md

@@ -0,0 +1,93 @@
+# Changelog
+
+All notable changes to this package are listed here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the versions follow
+[Semantic Versioning](https://semver.org/). Within the `0.x` releases the HTTP API stays at
+version 1 and stays backward compatible.
+
+## [0.1.0] - 2026-11-13
+
+First release.
+
+### Added
+
+- The `ck` object with the steps `start`, `reading`, `load`, `look`, `split`, `batch`, `model`,
+  `loop`, `eval`, `submit`, `drill_start`, `drill_check`, `status`, `finish` and `export`, and
+  `ck.last` with the last answer.
+- Evidence descriptions for plain Python objects, NumPy arrays, pandas tables, torch tensors and
+  models, scikit-learn models, Keras models and Hugging Face outputs. The libraries are only used
+  when the notebook has already imported them.
+- IPython cell hooks that record one event per executed cell (source, duration, error). A cell
+  that was interrupted is recorded with the error type `KeyboardInterrupt`, also when it was
+  waiting in an `await`; an error inside a cell magic such as `%%capture` belongs to that cell.
+- Redaction on the student's machine, before anything is stored or sent, in cell source, error
+  messages, tracebacks, the source of the metric function and the notebook: the student id
+  becomes `<student_id>`; tokens of well-known shapes (Hugging Face, OpenAI, GitHub, Google,
+  Slack, AWS, Kaggle, a PEM private key), the password in an address and the token of an
+  `Authorization` header become `<redacted>`. A value given to a name that ends in key, token,
+  secret, auth or pass (in Python code, in a shell or environment line, in the query of an
+  address) is removed when it has the shape of a key: twelve or more letters and digits, mixed
+  through one another, or a UUID. A value given to a name that ends in password or passwd is
+  removed whatever it holds. Everything else stays as written, in the recorded source and in
+  the notebook alike: `label_key = "category_name"` is not a key. The home folder is written
+  as `~`.
+- A background sender for cell events: batches of at most 100 events, sent every 20 seconds, when
+  50 events wait, ahead of every step, at `finish` and at interpreter exit. A batch that failed is
+  sent again unchanged, so the server never stores it twice.
+- A write-ahead local spool (`olympiad_<lab>_<student>.jsonl`) with a fall-back to the temp
+  folder and to memory, and `ck.export()` to hand everything over in one file. The running
+  kernel finds its lines also after the file was renamed, moved, emptied or deleted; a line that
+  was cut off in the middle never swallows the record after it; a record the files hold twice is
+  exported once; and an export never overwrites the local file it is made from.
+- Bounded memory: at most 5,000 cell events wait in memory; in a longer outage the newer ones
+  wait in the spool file only and are sent from there, in order. A spool file above 200 MB takes
+  new cell records without their source and says so once. Waiting calls are bounded by count and
+  by size.
+- Offline mode: when the server cannot be reached the session keeps recording, tries to connect
+  again in the background and from later steps, and delivers what waited.
+- Time budgets for every step, one automatic retry where repeating a request is safe, and a short
+  pause after a failed connection so that an outage does not slow every cell.
+- Every check and every drill check carries a `call_id` that stays the same on every retry of it
+  (the one right after a broken connection, the one from the background queue, an import of the
+  exported file), so the server judges and counts each check once.
+- Only an answer that has the fields of the API counts as delivered. Anything else that answers at
+  the server address (a placeholder, a network sign-in page) is treated as no answer, so its
+  records stay in the spool and in the export.
+- Notebook capture at `finish` in Colab, Jupyter and VS Code, without image outputs when the
+  notebook is very large, and without the account name Colab writes into its metadata (the name
+  of the kernel stays, so the file is still a valid notebook). In Colab the wait for the notebook
+  page is said first (`Reading the notebook; this can take a few seconds.`) and ends after 20
+  seconds. Where the notebook program does not hand the notebook over, the cells the kernel ran
+  are sent in its place (`notebook_source: "history"`).
+- Audit mode (`OLYMPIAD_AUDIT_DIR`): no network, every call written to a local log. A `lab.json`
+  left in that folder (the `lab` object of a start answer) stands in for the answer that never
+  arrives: a table is then written in the lab's file format and stages are spelled as the lab
+  spells them.
+- The guarantee that no step raises into the notebook, in any state and from any thread, and that
+  a worker process stays silent, forked or spawned, even where it runs `ck.start` again.
+- One line for every situation a student can get into: a step before `ck.start`
+  (`! olympiad: run the first cell (ck.start) before this one.`), a second `ck.start` with another
+  lab or id (`! START: switched from lab ... to lab ...`; what the old session still had to send
+  goes out in the background), an answer of `ck.reading` that cannot be read, a metric function
+  that raises or returns no number.
+- No step keeps a reference to what it was given: data the student deletes is really gone.
+- Evidence for the forms a notebook naturally holds: a loader batch given as one argument or as
+  a dict (`{"image": ..., "label": ...}`), an `(X, y)` pair or a dict of splits in `ck.load`, a
+  model that returns `(logits, hidden)`. A loss history may be a list, records, a log with a
+  counter, a table or a history object; from records of two curves (`(train_loss, val_loss)` as
+  tuples, lists or rows of a table) the training loss is taken and the curves are never mixed.
+- The metric function of `ck.reading` is called with NumPy arrays first when NumPy is loaded and
+  with plain lists only when that raises; without NumPy it gets lists. A function written for
+  lists that joins its two arguments with `+` (two arrays would be added) is recognised and gets
+  lists; torch tensors and pandas Series are tried for a function that needs them.
+- The model description of `ck.model` carries `n_classes`: the number of classes a fitted
+  scikit-learn classifier knows, also through a pipeline, a search and a calibrated or voting
+  wrapper. The server needs it for models that return one value per row.
+- `ck.submit` in a lab that takes JSON lines (`submission_format: "jsonl"` in the lab info of the
+  start answer) writes a DataFrame or a list of records as one JSON object per row. A file or
+  text is sent as it is; without the field a table is written as CSV.
+- Output that every channel can carry: each block has an HTML and a plain-text form, colours with
+  a contrast of at least 4.5 on its own background, and no half characters that a kernel could
+  not send. Text from the server is shown as text, never as HTML.
+- Leaving the interpreter never takes longer than the exit budget of 3 seconds, whatever the
+  network does: the last round runs in the sender threads, all sessions at the same time.

olympiad_lab-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 The olympiad-lab authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

olympiad_lab-0.1.0/MANIFEST.in

@@ -0,0 +1,2 @@
+include CHANGELOG.md
+recursive-include tests *.py

olympiad_lab-0.1.0/PKG-INFO

@@ -0,0 +1,285 @@
+Metadata-Version: 2.4
+Name: olympiad-lab
+Version: 0.1.0
+Summary: Checkpoints for AI Olympiad training labs: sends evidence from a notebook to a scorer server and shows the verdict.
+Author: olympiad-lab maintainers
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/SattamAltwaim/olympiad-lab
+Project-URL: Repository, https://github.com/SattamAltwaim/olympiad-lab
+Project-URL: Issues, https://github.com/SattamAltwaim/olympiad-lab/issues
+Project-URL: Changelog, https://github.com/SattamAltwaim/olympiad-lab/blob/main/CHANGELOG.md
+Keywords: education,notebook,jupyter,colab,kaggle,machine-learning,olympiad
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Education
+Classifier: Framework :: IPython
+Classifier: Framework :: Jupyter
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Education
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25
+Dynamic: license-file
+
+# olympiad-lab
+
+Checkpoints for AI Olympiad training labs.
+
+A lab notebook calls a few short steps (`ck.load`, `ck.look`, `ck.split`, ...) between the
+student's own code. Each step describes what was just built and sends that description to the
+scorer server of the course. The server decides whether the step passes and answers with one
+line. The library holds no expected value, no rule and no label: the notebook sends evidence, the
+server decides.
+
+## Install
+
+```
+pip install olympiad-lab
+```
+
+The only dependency is `requests`. NumPy, pandas, torch and the other libraries of a lab are
+used when the notebook has already imported them, and never imported by this package.
+
+## What this library records
+
+Read this before you use it. From the moment `ck.start()` runs, and only in that notebook:
+
+- **Every cell you run**: the first 4000 characters of its source, how long it took, whether it
+  raised, and for an error its type, its message and the last 15 lines of the traceback.
+- **What each step describes**: types, shapes, row counts, column names, data types, small number
+  summaries (minimum, maximum, mean, count of missing values) and the first 200 characters of how
+  the object prints. For `ck.split` the row ids (or their hashes for very large sets). For
+  `ck.loop` up to 200 loss values. For `ck.eval` the score.
+- **What you hand in**: the submission file, the answers of `ck.reading` (including the source of
+  the metric function you pass), and at `ck.finish()` the notebook itself. When the notebook
+  program does not hand it over, the cells this kernel ran are sent in its place.
+- **Where it runs**: Colab, Kaggle, Jupyter, VS Code or script; the Python and IPython versions;
+  the operating system name; the GPU model.
+
+It does not record other notebooks, other files, the values of environment variables, or your
+account. Cells that ran before `ck.start()` are not recorded.
+
+Three things are taken out on your computer, before anything is stored or sent, from cell source,
+error messages, tracebacks and the notebook:
+
+- **Your student id** becomes `<student_id>`.
+- **Anything that looks like a secret** becomes `<redacted>`: access tokens of well-known shapes
+  (Hugging Face, OpenAI, GitHub, Google, Slack, AWS, Kaggle, a private key); a value given to a
+  name such as `api_key`, `HF_TOKEN` or `os.environ["KAGGLE_KEY"]`, also in a line such as
+  `%env KAGGLE_KEY=...`, when it has the shape of a key (twelve or more letters and digits, mixed
+  through one another); any value given to a name that ends in `password`; the password inside
+  an address and the token of an `Authorization` header.
+- **Names**: your home folder is written as `~`, and the account name that Colab writes into a
+  notebook is removed.
+
+Code that holds no secret stays exactly as you wrote it: `label_key = "category_name"` and
+`stop_token = "<|end_of_text|>"` are not keys, although their names end like one.
+
+This is a safety net, not a guarantee. A secret that looks like any other word cannot be told
+apart from one, so do not paste keys into a notebook: use `getpass.getpass()` or the secrets
+panel of Colab or Kaggle.
+
+The records go to the scorer server over HTTPS and are used by the instructors of the course for
+assessment. A copy is kept next to the notebook in `olympiad_<lab>_<student>.jsonl`.
+`ck.start()` says so every time:
+
+```
+This lab records cell runs, cell source and errors for assessment.
+```
+
+## Use
+
+```python
+!pip -q install olympiad-lab
+from olympiad import ck
+
+ck.start("nlp-01", "S07-K4QX")                  # the lab id and your student id
+ck.load(train=train_df, test=test_df)           # LOAD: the data is in memory
+ck.look(train_df.head(), size=len(train_df))    # LOOK: shows a sample, prints the size
+ck.split(train_ids=tr.id, val_ids=va.id)        # SPLIT: no row in both sets
+ck.batch(X_tr[:32], y_tr[:32])                  # BATCH: one batch or feature table
+ck.model(model, X_va[:32])                      # MODEL: runs the model on one batch
+ck.loop(losses)                                 # LOOP: the training loss fell
+ck.eval(val_f1, metric="macro_f1")              # EVAL: prints the validation score
+ck.submit("submission.csv")                     # SUBMIT: a file, a DataFrame or the text of a file
+ck.finish()                                     # sends what waits, prints a summary
+```
+
+Every step prints a tick or a cross, one sentence, and a strip of the lab's parts:
+
+```
+✓ BATCH: Batch shape and data type are right.
+Parts 4 of 7: LOAD ✓ | LOOK ✓ | SPLIT ✓ | BATCH ✓ | MODEL - | EVAL - | SUBMIT -
+```
+
+In a notebook the same block is shown as a small box that reads the same in a light and in a
+dark theme. A part that the lab does not use answers `! LOOP is not part of this lab.` A step
+that runs before `ck.start()` does nothing and says:
+
+```
+! olympiad: run the first cell (ck.start) before this one.
+```
+
+Other steps:
+
+| Step | What it does |
+|---|---|
+| `ck.reading(metric_fn=..., trivial={...}, rewards="b", format={...}, facts="...")` | Sends the task reading. It is scored after the lab; no score is shown. |
+| `ck.model(...)`, `ck.loop(...)`, `ck.eval(...)`, `ck.submit(...)` with `stage="frozen"` | Names the stage in labs that compare a frozen and a fine-tuned model. |
+| `ck.drill_start("id")`, `ck.drill_check("id", result)` | A timed drill: the server stamps the start and answers the check. |
+| `ck.status()` | The minute, the parts passed, the submissions used and left. |
+| `ck.export()` | One file with everything recorded locally. Returns its path. |
+| `ck.last` | The last answer as a dictionary. |
+
+Running a cell again is always safe. `ck.start()` never resets a session or its clock, a check
+may be repeated as often as you like (the first pass counts), and the same submission file sent
+twice uses one attempt. `ck.start()` with another lab id or another student id in the same
+notebook says so in one line (`! START: switched from lab nlp-01 to lab cv-02.`) and goes on
+with the new session; what the old one still had to send is sent in the background.
+
+The steps take what a notebook naturally holds:
+
+- A batch may be `(inputs, targets)` as a loader yields it, or a dict such as
+  `{"image": ..., "label": ...}`.
+- `ck.load` takes an `(X, y)` pair, a data loader, or a dict of splits such as
+  `{"train": ..., "test": ...}`.
+- A loss history may be a list, an array, one record per epoch (a dict, or a pair such as
+  `(train_loss, val_loss)`: the training loss is read, the two are never mixed), a table, or the
+  history object of Keras or of a Hugging Face trainer.
+- A model may return `(logits, hidden)`; a fitted scikit-learn classifier also reports how many
+  classes it knows.
+- The metric function of `ck.reading` is called as `metric_fn(y_true, y_pred)` with NumPy arrays
+  when the notebook has loaded NumPy, and with plain lists when that raises or when NumPy is not
+  loaded. A function written for lists that joins the two with `+` is recognised and gets lists.
+- `ck.submit` sends a file or the text of a file as it is. A DataFrame or a list of records is
+  written as CSV, or as JSON lines (one object per row) in a lab that asks for them.
+
+An answer of `ck.reading` that cannot be read is named in one line while the reading is still open.
+
+## What it never does
+
+- **It never raises into your code.** Whatever you pass and whatever happens to the network, a
+  step prints at most one line such as
+  `! olympiad: the LOAD step could not finish (TypeError); your code is not affected.`
+  Only `KeyboardInterrupt` and `SystemExit` pass through.
+- **It never blocks for long.** Each step has a time budget: 15 seconds for `start`, 6 for a
+  check, a reading, a drill or the status, 40 for `submit` and `finish`. Cell records travel in a
+  background thread. In Colab, `ck.finish()` first asks the notebook page for the notebook and
+  says so (`Reading the notebook; this can take a few seconds.`); without an answer it goes on
+  after 20 seconds with the cells the kernel ran.
+- **It never changes your objects.** The model is run on the batch you give, without training it.
+- **It never imports a heavy library.** NumPy, pandas, torch and the others are only looked at
+  when your notebook has already imported them.
+- **It stays quiet in worker processes.** A worker of `multiprocessing` or of a `DataLoader`,
+  forked or spawned, that reaches a step (even `ck.start` at the top of a script) sends nothing
+  and prints nothing.
+
+## When the server cannot be reached
+
+The library writes everything to the local file first and sends it second. So when the server
+does not answer:
+
+```
+! START: no answer from the server. Working offline: everything is saved locally and sent when the connection is back.
+! LOAD: no answer from the server yet. Saved locally; it will be sent again.
+```
+
+Keep working. Checks, readings and drills that got no answer are sent again in the background
+(a check that arrives this way is marked as late, and the server counts every check once however
+often it is sent). A submission and `ck.finish()` are not sent again by themselves: run the cell
+again when the connection is back. After a connection that failed, the steps of the next ten
+seconds do not wait for the server again; they are saved and queued at once.
+
+`ck.export()` is for the case that the connection never comes back. Run it at the end and send
+the file to your instructor:
+
+```python
+ck.export()
+```
+
+It writes `olympiad_export_<lab>_<student>_<utc>.json.gz` into the working folder (in Colab it
+also offers the download) and returns the path. The file holds every record of the session,
+including the submission files, so the lab can be scored later. The times in it are your
+computer's, not the server's.
+
+## Supported environments
+
+- Python 3.9, 3.10, 3.11, 3.12 and 3.13.
+- Google Colab, Kaggle notebooks (internet on), Jupyter (Notebook and Lab), VS Code notebooks and
+  plain Python scripts. In a script there are no cells, so only the steps are recorded.
+- IPython 7.34 and newer with ipykernel 6.17 and newer (the versions of Colab), up to the current
+  releases.
+- Evidence is built for plain Python objects, NumPy, pandas, torch, scikit-learn, Keras and
+  Hugging Face objects, whichever of them the notebook uses.
+
+## Settings
+
+| Setting | Meaning |
+|---|---|
+| `ck.start(lab, student_id, server="https://...")` | Use another scorer server. |
+| `OLYMPIAD_SERVER_URL` | The same, as an environment variable. The argument wins. |
+| `OLYMPIAD_STUDENT_ID` | The student id for `ck.start(lab)` without one (automated runs). |
+| `OLYMPIAD_AUDIT_DIR` | Audit mode, see below. |
+
+Without a setting the library talks to the server it was written for,
+`https://us-central1-ka-ai-olympiad.cloudfunctions.net/api`.
+
+## Audit mode
+
+With `OLYMPIAD_AUDIT_DIR` set to a folder, nothing touches the network. Every call is appended to
+`audit.jsonl` in that folder as `{"route", "body", "ts"}`, the body exactly as it would have
+been sent; submitted files are copied there as `submission_<n>_<stage>__<filename>` and the
+submit body names the copy under `file` instead of carrying the content. Every step prints
+`! LOAD recorded (audit mode)`. Instructors use it to run a handed-in notebook again and compare.
+
+No start answer arrives in this mode. A tool that knows the lab can leave its public info in the
+folder as `lab.json` (the `lab` object of a start answer); a table handed to `ck.submit` is then
+written in the lab's file format and stages are spelled as the lab spells them. Without the file
+a table is written as CSV.
+
+## The local file, for tool authors
+
+`olympiad_<lab>_<student>.jsonl` is append-only, one JSON object per line:
+
+```
+{"k": "event", "ts": "...Z", "sent": false, "session_id": "...", "boot": "a3f9c2d1", "event": {"seq": 12, "type": "cell_run", ...}}
+{"k": "call",  "ts": "...Z", "sent": false, "session_id": "...", "id": "a3f9c2d1-c4", "route": "/v1/check", "body": {...}, "response": null}
+{"k": "ack",   "ts": "...Z", "boot": "a3f9c2d1", "seq": [10, 12]}
+{"k": "ack",   "ts": "...Z", "id": "a3f9c2d1-c4", "sent": true, "response": {...}}
+```
+
+The body of a check carries its `call_id`, the same on every retry. An `ack` line records what
+happened later to earlier lines: a range of events the server accepted, or how a call ended.
+`ck.export()` folds them in: the exported file is gzip-compressed JSON lines with `event` and
+`call` lines only, where `sent` is true for what the server accepted and `response` holds its
+answer (or its error). A kernel that was restarted appends to the same file under a new `boot`
+id. A `finish` call is kept without the notebook while it is under way; only one that got no
+answer is kept in full, as a second line.
+
+The file may be renamed, moved or deleted while the notebook runs: the running kernel still
+finds every line for `ck.export()`. A line that was cut off (a kernel that was killed while it
+wrote) is skipped by every reader and never swallows the next one. Above 200 MB new cell records
+are kept without their source (the notebook says so once), and a submission is kept in full only
+while it has no answer.
+
+## Development
+
+```
+python -m pip install -e . pytest numpy pandas ipython ipykernel nbclient
+python -m pytest
+```
+
+The tests run against a small fake scorer on localhost (`tests/fake_server.py`) and cannot reach
+the real one: the built-in address is replaced and sockets are confined to the local machine.
+
+## Licence
+
+MIT. See `LICENSE`.

olympiad_lab-0.1.0/README.md

@@ -0,0 +1,254 @@
+# olympiad-lab
+
+Checkpoints for AI Olympiad training labs.
+
+A lab notebook calls a few short steps (`ck.load`, `ck.look`, `ck.split`, ...) between the
+student's own code. Each step describes what was just built and sends that description to the
+scorer server of the course. The server decides whether the step passes and answers with one
+line. The library holds no expected value, no rule and no label: the notebook sends evidence, the
+server decides.
+
+## Install
+
+```
+pip install olympiad-lab
+```
+
+The only dependency is `requests`. NumPy, pandas, torch and the other libraries of a lab are
+used when the notebook has already imported them, and never imported by this package.
+
+## What this library records
+
+Read this before you use it. From the moment `ck.start()` runs, and only in that notebook:
+
+- **Every cell you run**: the first 4000 characters of its source, how long it took, whether it
+  raised, and for an error its type, its message and the last 15 lines of the traceback.
+- **What each step describes**: types, shapes, row counts, column names, data types, small number
+  summaries (minimum, maximum, mean, count of missing values) and the first 200 characters of how
+  the object prints. For `ck.split` the row ids (or their hashes for very large sets). For
+  `ck.loop` up to 200 loss values. For `ck.eval` the score.
+- **What you hand in**: the submission file, the answers of `ck.reading` (including the source of
+  the metric function you pass), and at `ck.finish()` the notebook itself. When the notebook
+  program does not hand it over, the cells this kernel ran are sent in its place.
+- **Where it runs**: Colab, Kaggle, Jupyter, VS Code or script; the Python and IPython versions;
+  the operating system name; the GPU model.
+
+It does not record other notebooks, other files, the values of environment variables, or your
+account. Cells that ran before `ck.start()` are not recorded.
+
+Three things are taken out on your computer, before anything is stored or sent, from cell source,
+error messages, tracebacks and the notebook:
+
+- **Your student id** becomes `<student_id>`.
+- **Anything that looks like a secret** becomes `<redacted>`: access tokens of well-known shapes
+  (Hugging Face, OpenAI, GitHub, Google, Slack, AWS, Kaggle, a private key); a value given to a
+  name such as `api_key`, `HF_TOKEN` or `os.environ["KAGGLE_KEY"]`, also in a line such as
+  `%env KAGGLE_KEY=...`, when it has the shape of a key (twelve or more letters and digits, mixed
+  through one another); any value given to a name that ends in `password`; the password inside
+  an address and the token of an `Authorization` header.
+- **Names**: your home folder is written as `~`, and the account name that Colab writes into a
+  notebook is removed.
+
+Code that holds no secret stays exactly as you wrote it: `label_key = "category_name"` and
+`stop_token = "<|end_of_text|>"` are not keys, although their names end like one.
+
+This is a safety net, not a guarantee. A secret that looks like any other word cannot be told
+apart from one, so do not paste keys into a notebook: use `getpass.getpass()` or the secrets
+panel of Colab or Kaggle.
+
+The records go to the scorer server over HTTPS and are used by the instructors of the course for
+assessment. A copy is kept next to the notebook in `olympiad_<lab>_<student>.jsonl`.
+`ck.start()` says so every time:
+
+```
+This lab records cell runs, cell source and errors for assessment.
+```
+
+## Use
+
+```python
+!pip -q install olympiad-lab
+from olympiad import ck
+
+ck.start("nlp-01", "S07-K4QX")                  # the lab id and your student id
+ck.load(train=train_df, test=test_df)           # LOAD: the data is in memory
+ck.look(train_df.head(), size=len(train_df))    # LOOK: shows a sample, prints the size
+ck.split(train_ids=tr.id, val_ids=va.id)        # SPLIT: no row in both sets
+ck.batch(X_tr[:32], y_tr[:32])                  # BATCH: one batch or feature table
+ck.model(model, X_va[:32])                      # MODEL: runs the model on one batch
+ck.loop(losses)                                 # LOOP: the training loss fell
+ck.eval(val_f1, metric="macro_f1")              # EVAL: prints the validation score
+ck.submit("submission.csv")                     # SUBMIT: a file, a DataFrame or the text of a file
+ck.finish()                                     # sends what waits, prints a summary
+```
+
+Every step prints a tick or a cross, one sentence, and a strip of the lab's parts:
+
+```
+✓ BATCH: Batch shape and data type are right.
+Parts 4 of 7: LOAD ✓ | LOOK ✓ | SPLIT ✓ | BATCH ✓ | MODEL - | EVAL - | SUBMIT -
+```
+
+In a notebook the same block is shown as a small box that reads the same in a light and in a
+dark theme. A part that the lab does not use answers `! LOOP is not part of this lab.` A step
+that runs before `ck.start()` does nothing and says:
+
+```
+! olympiad: run the first cell (ck.start) before this one.
+```
+
+Other steps:
+
+| Step | What it does |
+|---|---|
+| `ck.reading(metric_fn=..., trivial={...}, rewards="b", format={...}, facts="...")` | Sends the task reading. It is scored after the lab; no score is shown. |
+| `ck.model(...)`, `ck.loop(...)`, `ck.eval(...)`, `ck.submit(...)` with `stage="frozen"` | Names the stage in labs that compare a frozen and a fine-tuned model. |
+| `ck.drill_start("id")`, `ck.drill_check("id", result)` | A timed drill: the server stamps the start and answers the check. |
+| `ck.status()` | The minute, the parts passed, the submissions used and left. |
+| `ck.export()` | One file with everything recorded locally. Returns its path. |
+| `ck.last` | The last answer as a dictionary. |
+
+Running a cell again is always safe. `ck.start()` never resets a session or its clock, a check
+may be repeated as often as you like (the first pass counts), and the same submission file sent
+twice uses one attempt. `ck.start()` with another lab id or another student id in the same
+notebook says so in one line (`! START: switched from lab nlp-01 to lab cv-02.`) and goes on
+with the new session; what the old one still had to send is sent in the background.
+
+The steps take what a notebook naturally holds:
+
+- A batch may be `(inputs, targets)` as a loader yields it, or a dict such as
+  `{"image": ..., "label": ...}`.
+- `ck.load` takes an `(X, y)` pair, a data loader, or a dict of splits such as
+  `{"train": ..., "test": ...}`.
+- A loss history may be a list, an array, one record per epoch (a dict, or a pair such as
+  `(train_loss, val_loss)`: the training loss is read, the two are never mixed), a table, or the
+  history object of Keras or of a Hugging Face trainer.
+- A model may return `(logits, hidden)`; a fitted scikit-learn classifier also reports how many
+  classes it knows.
+- The metric function of `ck.reading` is called as `metric_fn(y_true, y_pred)` with NumPy arrays
+  when the notebook has loaded NumPy, and with plain lists when that raises or when NumPy is not
+  loaded. A function written for lists that joins the two with `+` is recognised and gets lists.
+- `ck.submit` sends a file or the text of a file as it is. A DataFrame or a list of records is
+  written as CSV, or as JSON lines (one object per row) in a lab that asks for them.
+
+An answer of `ck.reading` that cannot be read is named in one line while the reading is still open.
+
+## What it never does
+
+- **It never raises into your code.** Whatever you pass and whatever happens to the network, a
+  step prints at most one line such as
+  `! olympiad: the LOAD step could not finish (TypeError); your code is not affected.`
+  Only `KeyboardInterrupt` and `SystemExit` pass through.
+- **It never blocks for long.** Each step has a time budget: 15 seconds for `start`, 6 for a
+  check, a reading, a drill or the status, 40 for `submit` and `finish`. Cell records travel in a
+  background thread. In Colab, `ck.finish()` first asks the notebook page for the notebook and
+  says so (`Reading the notebook; this can take a few seconds.`); without an answer it goes on
+  after 20 seconds with the cells the kernel ran.
+- **It never changes your objects.** The model is run on the batch you give, without training it.
+- **It never imports a heavy library.** NumPy, pandas, torch and the others are only looked at
+  when your notebook has already imported them.
+- **It stays quiet in worker processes.** A worker of `multiprocessing` or of a `DataLoader`,
+  forked or spawned, that reaches a step (even `ck.start` at the top of a script) sends nothing
+  and prints nothing.
+
+## When the server cannot be reached
+
+The library writes everything to the local file first and sends it second. So when the server
+does not answer:
+
+```
+! START: no answer from the server. Working offline: everything is saved locally and sent when the connection is back.
+! LOAD: no answer from the server yet. Saved locally; it will be sent again.
+```
+
+Keep working. Checks, readings and drills that got no answer are sent again in the background
+(a check that arrives this way is marked as late, and the server counts every check once however
+often it is sent). A submission and `ck.finish()` are not sent again by themselves: run the cell
+again when the connection is back. After a connection that failed, the steps of the next ten
+seconds do not wait for the server again; they are saved and queued at once.
+
+`ck.export()` is for the case that the connection never comes back. Run it at the end and send
+the file to your instructor:
+
+```python
+ck.export()
+```
+
+It writes `olympiad_export_<lab>_<student>_<utc>.json.gz` into the working folder (in Colab it
+also offers the download) and returns the path. The file holds every record of the session,
+including the submission files, so the lab can be scored later. The times in it are your
+computer's, not the server's.
+
+## Supported environments
+
+- Python 3.9, 3.10, 3.11, 3.12 and 3.13.
+- Google Colab, Kaggle notebooks (internet on), Jupyter (Notebook and Lab), VS Code notebooks and
+  plain Python scripts. In a script there are no cells, so only the steps are recorded.
+- IPython 7.34 and newer with ipykernel 6.17 and newer (the versions of Colab), up to the current
+  releases.
+- Evidence is built for plain Python objects, NumPy, pandas, torch, scikit-learn, Keras and
+  Hugging Face objects, whichever of them the notebook uses.
+
+## Settings
+
+| Setting | Meaning |
+|---|---|
+| `ck.start(lab, student_id, server="https://...")` | Use another scorer server. |
+| `OLYMPIAD_SERVER_URL` | The same, as an environment variable. The argument wins. |
+| `OLYMPIAD_STUDENT_ID` | The student id for `ck.start(lab)` without one (automated runs). |
+| `OLYMPIAD_AUDIT_DIR` | Audit mode, see below. |
+
+Without a setting the library talks to the server it was written for,
+`https://us-central1-ka-ai-olympiad.cloudfunctions.net/api`.
+
+## Audit mode
+
+With `OLYMPIAD_AUDIT_DIR` set to a folder, nothing touches the network. Every call is appended to
+`audit.jsonl` in that folder as `{"route", "body", "ts"}`, the body exactly as it would have
+been sent; submitted files are copied there as `submission_<n>_<stage>__<filename>` and the
+submit body names the copy under `file` instead of carrying the content. Every step prints
+`! LOAD recorded (audit mode)`. Instructors use it to run a handed-in notebook again and compare.
+
+No start answer arrives in this mode. A tool that knows the lab can leave its public info in the
+folder as `lab.json` (the `lab` object of a start answer); a table handed to `ck.submit` is then
+written in the lab's file format and stages are spelled as the lab spells them. Without the file
+a table is written as CSV.
+
+## The local file, for tool authors
+
+`olympiad_<lab>_<student>.jsonl` is append-only, one JSON object per line:
+
+```
+{"k": "event", "ts": "...Z", "sent": false, "session_id": "...", "boot": "a3f9c2d1", "event": {"seq": 12, "type": "cell_run", ...}}
+{"k": "call",  "ts": "...Z", "sent": false, "session_id": "...", "id": "a3f9c2d1-c4", "route": "/v1/check", "body": {...}, "response": null}
+{"k": "ack",   "ts": "...Z", "boot": "a3f9c2d1", "seq": [10, 12]}
+{"k": "ack",   "ts": "...Z", "id": "a3f9c2d1-c4", "sent": true, "response": {...}}
+```
+
+The body of a check carries its `call_id`, the same on every retry. An `ack` line records what
+happened later to earlier lines: a range of events the server accepted, or how a call ended.
+`ck.export()` folds them in: the exported file is gzip-compressed JSON lines with `event` and
+`call` lines only, where `sent` is true for what the server accepted and `response` holds its
+answer (or its error). A kernel that was restarted appends to the same file under a new `boot`
+id. A `finish` call is kept without the notebook while it is under way; only one that got no
+answer is kept in full, as a second line.
+
+The file may be renamed, moved or deleted while the notebook runs: the running kernel still
+finds every line for `ck.export()`. A line that was cut off (a kernel that was killed while it
+wrote) is skipped by every reader and never swallows the next one. Above 200 MB new cell records
+are kept without their source (the notebook says so once), and a submission is kept in full only
+while it has no answer.
+
+## Development
+
+```
+python -m pip install -e . pytest numpy pandas ipython ipykernel nbclient
+python -m pytest
+```
+
+The tests run against a small fake scorer on localhost (`tests/fake_server.py`) and cannot reach
+the real one: the built-in address is replaced and sockets are confined to the local machine.
+
+## Licence
+
+MIT. See `LICENSE`.