everestapi 0.2.6__tar.gz → 0.2.7__tar.gz

{everestapi-0.2.6/src/everestapi.egg-info → everestapi-0.2.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: everestapi
-Version: 0.2.6
+Version: 0.2.7
 Summary: Python SDK for the Everesteer prediction tournament platform
 Author-email: Everesteer <support@everesteer.ai>
 License-Expression: MIT
@@ -24,10 +24,17 @@ Requires-Dist: click>=8.0
 Provides-Extra: dev
 Requires-Dist: pytest>=8.0; extra == "dev"
 Requires-Dist: pytest-httpx>=0.34.0; extra == "dev"
+Requires-Dist: numpy>=1.26; extra == "dev"
+Requires-Dist: pandas>=2.0; extra == "dev"
+Requires-Dist: scipy>=1.10; extra == "dev"
 Provides-Extra: mcp
 Requires-Dist: mcp>=1.0; extra == "mcp"
 Provides-Extra: viz
 Requires-Dist: plotnine>=0.13; extra == "viz"
+Provides-Extra: scoring
+Requires-Dist: numpy>=1.26; extra == "scoring"
+Requires-Dist: pandas>=2.0; extra == "scoring"
+Requires-Dist: scipy>=1.10; extra == "scoring"
 Dynamic: license-file
 
 # everestapi
@@ -89,8 +96,11 @@ api.submit_futures_predictions(model_id="my-fut-model", predictions={...})
 Both Parquet and CSV are accepted. **Parquet is recommended** — float precision round-trips cleanly, files compress well, and it matches the format the SDK serves to you (`download_dataset` returns parquet).
 
 ```python
+# A model slot must exist before you can submit — the platform never auto-creates one.
+api.create_model(name="my-model")
+
 api.submit_predictions_file(
-    model_name="my-model",
+    model_id="my-model",
     file_path="predictions.parquet",  # or "predictions.csv"
     tournament="equities",
 )
@@ -168,16 +178,59 @@ api.get_stake_balance(model_id="my-model")
 api.claim_payout(model_id="my-model", round_id="42")
 ```
 
-### Local evaluation
+### Score validation predictions offline
+
+Reproduce the server's **exact** scoring — CORR20, AIMC20, FNC — *before* you
+submit, so you stop guessing the sign of your signal ("submit raw and negated, let
+the server decide"). The `everestapi.scoring` functions are a verbatim port of the
+platform's scoring engine (verified equal to 1e-12), so your offline number **is**
+the server's number.
+
+Install the optional scoring extra (keeps the base SDK light — numpy/pandas/scipy
+are only pulled in here):
+
+```bash
+pip install "everestapi[scoring]"
+```
 
 ```python
 import pandas as pd
+from everestapi import scoring
 
-val = pd.read_parquet("futures_validation.parquet")
-metrics = EverestAPI.evaluate(predictions, val, target="target_everest_20")
-print(metrics)  # {"simple_corr": 0.023, "weighted_corr": 0.019, "per_difficulty": {...}}
+val = pd.read_parquet("eiq_validation.parquet")
+preds = my_model.predict(val.filter(like="feature_"))
+
+# Score per exped (cross-section), then average — matches how the server scores.
+per_exped = [
+    scoring.corr20(preds[val.exped == e], val.loc[val.exped == e, "target"])
+    for e in val.exped.unique()
+]
+print("mean CORR20:", sum(per_exped) / len(per_exped))
+
+# Or every metric at once for one exped (ai_model = crowd consensus for that exped):
+scoring.score(preds_e, target_e, ai_model=consensus_e, features=features_e)
+# -> {"corr20", "aimc20", "payout", "fnc", "feature_exposure"}
 ```
 
+**Sanity-check your pipeline against the example predictions.** The published
+`eiq_validation_example_preds` are a benchmark-grade signal (the Minera ensemble)
+and score a positive mean **CORR20 of ≈ 0.07**. Score that file and reproduce a
+similar number — if you instead get ≈ −0.07, your sign is flipped; if you get ≈ 0,
+your ids/alignment are off:
+
+```python
+ex = pd.read_parquet("eiq_validation_example_preds.parquet")   # column: prediction
+val = pd.read_parquet("eiq_validation.parquet")
+ref = [
+    scoring.corr20(ex.loc[val.exped == e, "prediction"], val.loc[val.exped == e, "target"])
+    for e in val.exped.unique()
+]
+print(sum(ref) / len(ref))   # ~0.07  ->  pipeline + sign are correct
+```
+
+A quick convenience for a single overall correlation is also available:
+`EverestAPI.evaluate(predictions, val, target="target_everest_20")`.
+
 ### CLI
 
 ```bash
@@ -207,6 +260,7 @@ with EverestAPI(api_key="...") as api:
 
 - Python 3.10+
 - httpx >= 0.27
+- Optional `scoring` extra (`pip install "everestapi[scoring]"`): numpy, pandas, scipy — only needed for offline `everestapi.scoring`.
 
 ## Disclaimers
 

{everestapi-0.2.6 → everestapi-0.2.7}/README.md

@@ -57,8 +57,11 @@ api.submit_futures_predictions(model_id="my-fut-model", predictions={...})
 Both Parquet and CSV are accepted. **Parquet is recommended** — float precision round-trips cleanly, files compress well, and it matches the format the SDK serves to you (`download_dataset` returns parquet).
 
 ```python
+# A model slot must exist before you can submit — the platform never auto-creates one.
+api.create_model(name="my-model")
+
 api.submit_predictions_file(
-    model_name="my-model",
+    model_id="my-model",
     file_path="predictions.parquet",  # or "predictions.csv"
     tournament="equities",
 )
@@ -136,16 +139,59 @@ api.get_stake_balance(model_id="my-model")
 api.claim_payout(model_id="my-model", round_id="42")
 ```
 
-### Local evaluation
+### Score validation predictions offline
+
+Reproduce the server's **exact** scoring — CORR20, AIMC20, FNC — *before* you
+submit, so you stop guessing the sign of your signal ("submit raw and negated, let
+the server decide"). The `everestapi.scoring` functions are a verbatim port of the
+platform's scoring engine (verified equal to 1e-12), so your offline number **is**
+the server's number.
+
+Install the optional scoring extra (keeps the base SDK light — numpy/pandas/scipy
+are only pulled in here):
+
+```bash
+pip install "everestapi[scoring]"
+```
 
 ```python
 import pandas as pd
+from everestapi import scoring
 
-val = pd.read_parquet("futures_validation.parquet")
-metrics = EverestAPI.evaluate(predictions, val, target="target_everest_20")
-print(metrics)  # {"simple_corr": 0.023, "weighted_corr": 0.019, "per_difficulty": {...}}
+val = pd.read_parquet("eiq_validation.parquet")
+preds = my_model.predict(val.filter(like="feature_"))
+
+# Score per exped (cross-section), then average — matches how the server scores.
+per_exped = [
+    scoring.corr20(preds[val.exped == e], val.loc[val.exped == e, "target"])
+    for e in val.exped.unique()
+]
+print("mean CORR20:", sum(per_exped) / len(per_exped))
+
+# Or every metric at once for one exped (ai_model = crowd consensus for that exped):
+scoring.score(preds_e, target_e, ai_model=consensus_e, features=features_e)
+# -> {"corr20", "aimc20", "payout", "fnc", "feature_exposure"}
 ```
 
+**Sanity-check your pipeline against the example predictions.** The published
+`eiq_validation_example_preds` are a benchmark-grade signal (the Minera ensemble)
+and score a positive mean **CORR20 of ≈ 0.07**. Score that file and reproduce a
+similar number — if you instead get ≈ −0.07, your sign is flipped; if you get ≈ 0,
+your ids/alignment are off:
+
+```python
+ex = pd.read_parquet("eiq_validation_example_preds.parquet")   # column: prediction
+val = pd.read_parquet("eiq_validation.parquet")
+ref = [
+    scoring.corr20(ex.loc[val.exped == e, "prediction"], val.loc[val.exped == e, "target"])
+    for e in val.exped.unique()
+]
+print(sum(ref) / len(ref))   # ~0.07  ->  pipeline + sign are correct
+```
+
+A quick convenience for a single overall correlation is also available:
+`EverestAPI.evaluate(predictions, val, target="target_everest_20")`.
+
 ### CLI
 
 ```bash
@@ -175,6 +221,7 @@ with EverestAPI(api_key="...") as api:
 
 - Python 3.10+
 - httpx >= 0.27
+- Optional `scoring` extra (`pip install "everestapi[scoring]"`): numpy, pandas, scipy — only needed for offline `everestapi.scoring`.
 
 ## Disclaimers
 

{everestapi-0.2.6 → everestapi-0.2.7}/pyproject.toml

@@ -27,13 +27,16 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
-dev = ["pytest>=8.0", "pytest-httpx>=0.34.0"]
+dev = ["pytest>=8.0", "pytest-httpx>=0.34.0", "numpy>=1.26", "pandas>=2.0", "scipy>=1.10"]
 # MCP server (`python -m everestapi.mcp`). Optional — the server falls back to a
 # built-in JSON-RPC stdio loop when the official `mcp` SDK isn't installed.
 mcp = ["mcp>=1.0"]
 # Plotting helpers (`everestapi.plots`). Heavy — pulls matplotlib/pandas/numpy/
 # scipy/statsmodels/mizani. Optional: the SDK and MCP server never import it.
 viz = ["plotnine>=0.13"]
+# Local scoring (`everestapi.scoring`). Optional — keeps the base SDK light
+# (httpx + click only); pull these in to score validation preds offline.
+scoring = ["numpy>=1.26", "pandas>=2.0", "scipy>=1.10"]
 
 [project.scripts]
 everestapi = "everestapi.cli:cli"

{everestapi-0.2.6 → everestapi-0.2.7}/src/everestapi/__init__.py

@@ -1,6 +1,6 @@
 """EverestAPI — Python SDK for the Everesteer prediction tournament platform."""
 
-__version__ = "0.2.6"
+__version__ = "0.2.7"
 
 from everestapi.client import EverestAPI, EverestError
 from everestapi.types import (
@@ -29,4 +29,15 @@ __all__ = [
     "StakeResponse",
     "Submission",
     "UniverseResponse",
+    "scoring",
 ]
+
+
+def __getattr__(name):
+    # Lazy import so `import everestapi` stays light (httpx + click only) — the
+    # numpy/pandas/scipy stack is only loaded if you actually use scoring.
+    if name == "scoring":
+        import everestapi.scoring as scoring
+
+        return scoring
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

{everestapi-0.2.6 → everestapi-0.2.7}/src/everestapi/cli.py

@@ -108,7 +108,7 @@ def submit(model: str, file_path: str, tournament: str, api_key: str | None) ->
     try:
         if ext == "parquet":
             result = api.submit_predictions_file(
-                model_name=model,
+                model_id=model,
                 file_path=file_path,
                 tournament=tournament,
             )

{everestapi-0.2.6 → everestapi-0.2.7}/src/everestapi/client.py

@@ -133,9 +133,7 @@ def _json_or_raise(resp: httpx.Response) -> dict:
     except Exception:
         body = (resp.text or "").strip()
         looks_like_cf = (
-            "<html" in body[:200].lower()
-            or "cloudflare" in body.lower()
-            or "AccessDenied" in body
+            "<html" in body[:200].lower() or "cloudflare" in body.lower() or "AccessDenied" in body
         )
         hint = (
             " — this looks like a Cloudflare Access challenge; set "
@@ -145,14 +143,11 @@ def _json_or_raise(resp: httpx.Response) -> dict:
         )
         raise EverestError(
             resp.status_code,
-            f"expected a JSON response but received non-JSON content{hint}: "
-            f"{body[:200]}",
+            f"expected a JSON response but received non-JSON content{hint}: {body[:200]}",
         )
 
 
-def _check_prediction_range(
-    values: dict[str, float], *, lo: float = 0.0, hi: float = 1.0
-) -> None:
+def _check_prediction_range(values: dict[str, float], *, lo: float = 0.0, hi: float = 1.0) -> None:
     """Raise ``ValueError`` if any prediction is non-finite or outside ``[lo, hi]``.
 
     Mirrors the server-side futures bound so a bad submission fails fast
@@ -173,8 +168,7 @@ def _check_prediction_range(
         preview = ", ".join(bad[:5])
         more = f" (+{len(bad) - 5} more)" if len(bad) > 5 else ""
         raise ValueError(
-            f"{len(bad)} prediction(s) must be finite and within "
-            f"[{lo}, {hi}]: {preview}{more}"
+            f"{len(bad)} prediction(s) must be finite and within [{lo}, {hi}]: {preview}{more}"
         )
 
 
@@ -187,11 +181,7 @@ def _resolve_default_base_url() -> str:
     (CloudFront→S3) that serves no ``/api`` — defaulting there returns an S3
     ``AccessDenied`` for every call, so it is deliberately not the fallback.
     """
-    return (
-        os.getenv("EIQ_BASE_URL")
-        or os.getenv("EVEREST_API_URL")
-        or "https://app.everesteer.ai"
-    )
+    return os.getenv("EIQ_BASE_URL") or os.getenv("EVEREST_API_URL") or "https://app.everesteer.ai"
 
 
 class EverestAPI:
@@ -326,29 +316,42 @@ class EverestAPI:
 
     def submit_predictions_file(
         self,
-        model_name: str,
-        file_path: str,
+        model_id: str | None = None,
+        file_path: str | None = None,
         tournament: str | None = None,
+        *,
+        model_name: str | None = None,
     ) -> dict:
         """POST /api/v1/predictions/upload — submit predictions as a CSV or Parquet file.
 
         Parameters
         ----------
-        model_name : str
-            Model identifier.
+        model_id : str
+            Model identifier. (``model_name`` is a deprecated alias.)
         file_path : str
             Path to a ``.csv`` or ``.parquet`` file with columns ``ticker`` (str)
             and ``score`` (float in [-1, 1]).
         tournament : str, optional
             Tournament identifier. Defaults to the client's tournament.
         """
+        if model_id is None:
+            if model_name is None:
+                raise TypeError("submit_predictions_file() missing required argument: 'model_id'")
+            import warnings
+
+            warnings.warn(
+                "model_name= is deprecated; use model_id=", DeprecationWarning, stacklevel=2
+            )
+            model_id = model_name
+        if file_path is None:
+            raise TypeError("submit_predictions_file() missing required argument: 'file_path'")
         tourn = tournament or self.tournament
         with open(file_path, "rb") as f:
             fname = file_path.rsplit("/", 1)[-1].rsplit("\\", 1)[-1]
             resp = self._client.post(
                 "/api/v1/predictions/upload",
                 files={"file": (fname, f, "application/octet-stream")},
-                params={"model_id": model_name, "tournament": tourn},
+                params={"model_id": model_id, "tournament": tourn},
             )
         return _json_or_raise(resp)
 
@@ -491,9 +494,7 @@ class EverestAPI:
                 )
         return "\n".join(lines)
 
-    def get_validation_panel(
-        self, model_id: str, days: int = 365, source: str = "auto"
-    ) -> dict:
+    def get_validation_panel(self, model_id: str, days: int = 365, source: str = "auto") -> dict:
         """GET /api/v1/models/{model_id}/diagnostics/validation — full validation metrics.
 
         ``source="auto"`` (default) returns the latest completed validation-diagnostics
@@ -551,9 +552,19 @@ class EverestAPI:
             output_path,
         )
 
-    def get_scores(self, model_id: str, days: int = 30) -> dict:
-        """GET /api/v1/scores — scoring results for a model."""
-        params = {"model_id": model_id, "days": str(days)}
+    def get_scores(self, model_id: str, days: int = 30, response_format: str = "concise") -> dict:
+        """GET /api/v1/scores — scoring results for a model.
+
+        ``response_format="detailed"`` asks the platform to add a plain-language
+        ``interpretation`` field that reads the model's own latest numbers and
+        state and explains how to improve. ``"concise"`` (default) leaves the
+        response unchanged.
+        """
+        params = {
+            "model_id": model_id,
+            "days": str(days),
+            "response_format": response_format,
+        }
         return self._request("GET", "/api/v1/scores", params=params)
 
     def get_leaderboard(self, period: str = "30d") -> dict:
@@ -632,9 +643,15 @@ class EverestAPI:
             params["date"] = date
         return self._request("GET", f"/api/v1/diagnostics/{model_id}", params=params)
 
-    def get_round_diagnostics(self, model_id: str) -> dict:
-        """GET /api/v1/diagnostics/{model_id}/rounds — per-round scoring breakdown."""
-        return self._request("GET", f"/api/v1/diagnostics/{model_id}/rounds")
+    def get_round_diagnostics(self, model_id: str, response_format: str = "concise") -> dict:
+        """GET /api/v1/diagnostics/{model_id}/rounds — per-round scoring breakdown.
+
+        ``response_format="detailed"`` asks the platform to add a plain-language
+        ``interpretation`` field grounded in the model's own latest numbers and
+        state. ``"concise"`` (default) leaves the response unchanged.
+        """
+        params = {"response_format": response_format}
+        return self._request("GET", f"/api/v1/diagnostics/{model_id}/rounds", params=params)
 
     # -- futures tournament -----------------------------------------------
 
@@ -900,8 +917,8 @@ class EverestAPI:
 
     # -- rounds -----------------------------------------------------------
 
-    def get_rounds(self, tournament: str = "equities", limit: int = 25) -> dict:
-        """GET /api/v1/rounds — list tournament rounds."""
+    def get_rounds(self, tournament: str = "futures", limit: int = 25) -> dict:
+        """GET /api/v1/rounds — list tournament rounds (defaults to 'futures'; equities is unlaunched)."""
         return self._request(
             "GET",
             "/api/v1/rounds",
@@ -911,8 +928,13 @@ class EverestAPI:
             },
         )
 
-    def get_current_round(self, tournament: str = "equities") -> dict:
-        """GET /api/v1/rounds/current — current active round."""
+    def get_current_round(self, tournament: str = "futures") -> dict:
+        """GET /api/v1/rounds/current — current active round (defaults to 'futures'; equities is unlaunched).
+
+        With a hackathon-scoped key there is no live round — the response is a
+        diagnostics-mode payload (mode='diagnostics_hackathon') directing you to
+        download_dataset + submit_validation_diagnostics.
+        """
         return self._request(
             "GET",
             "/api/v1/rounds/current",
@@ -925,6 +947,35 @@ class EverestAPI:
         """GET /api/v1/schedule — round schedule for both tournaments."""
         return self._request("GET", "/api/v1/schedule")
 
+    def get_started(self) -> dict:
+        """GET /api/v1/get_started — mode-aware orientation.
+
+        Returns what to do next given this key's scope: the display-only
+        diagnostics-hackathon loop, or the live futures tournament flow.
+        """
+        return self._request("GET", "/api/v1/get_started")
+
+    def get_status(self) -> dict:
+        """GET /api/v1/status — state-aware orientation ("where am I").
+
+        Your whole situational picture in one call: the models you own, each
+        one's latest submission and score state, the open round's clock, your
+        stake, and a ``next_actions`` list — instead of stitching together
+        get_models + get_scores + get_current_round. Display-only.
+        """
+        return self._request("GET", "/api/v1/status")
+
+    def get_capabilities(self) -> dict:
+        """GET / (site root) with ``Accept: application/json`` — capability index.
+
+        A small machine-readable map of where everything lives: pointers to
+        ``llms.txt``, the OpenAPI spec, the MCP endpoint, the API base, the SDK,
+        and related discovery URLs. Hit before anything else to learn the
+        platform's shape without hardcoding paths. The index sits at the host
+        root; a leading-slash path resolves there against the API host base URL.
+        """
+        return self._request("GET", "/", headers={"Accept": "application/json"})
+
     # -- model uploads ----------------------------------------------------
 
     def upload_model(self, model_id: str, file_path: str) -> dict: