dslighting 1.3.9__py3-none-any.whl → 1.7.6__py3-none-any.whl

dslighting/registry/lmsys-chatbot-arena/description.md

@@ -0,0 +1,98 @@
+# Overview
+
+## Overview
+
+This competition challenges you to predict which responses users will prefer in a head-to-head battle between chatbots powered by large language models (LLMs). You'll be given a dataset of conversations from the [Chatbot Arena](https://chat.lmsys.org/), where different LLMs generate answers to user prompts. By developing a winning machine learning model, you'll help improve how chatbots interact with humans and ensure they better align with human preferences.
+
+## Description
+
+Large language models (LLMs) are rapidly entering our lives, but ensuring their responses resonate with users is critical for successful interaction. This competition presents a unique opportunity to tackle this challenge with real-world data and help us bridge the gap between LLM capability and human preference.
+
+We utilized a large dataset collected from Chatbot Arena, where users chat with two anonymous LLMs and choose the answer they prefer. Your task in this competition is to predict which response a user will prefer in these head-to-head battles.
+
+This challenge aligns with the concept of "reward models" or "preference models" in reinforcement learning from human feedback (RLHF). Previous research has identified limitations in directly prompting an existing LLM for preference predictions. These limitations often stem from biases such as favoring responses presented first (position bias), being overly verbose (verbosity bias), or exhibiting self-promotion (self-enhancement bias).
+
+We encourage you to explore various machine-learning techniques to build a model that can effectively predict user preferences. Your work will be instrumental in developing LLMs that can tailor responses to individual user preferences, ultimately leading to more user-friendly and widely accepted AI-powered conversation systems.
+
+## Evaluation
+
+Submissions are evaluated on the [log loss](https://www.kaggle.com/code/metric/log-loss?scriptVersionId=151169978) between the predicted probabilities and the ground truth values (with "eps=auto").
+
+### Submission File
+
+For each id in the test set, you must predict the probability for each target class. The file should contain a header and have the following format:
+
+```
+ id,winner_model_a,winner_model_b,winner_tie
+ 136060,0.33,0,33,0.33
+ 211333,0.33,0,33,0.33
+ 1233961,0.33,0,33,0.33
+ etc
+```
+
+## Timeline
+
+- **May 2, 2024** - Start Date.
+- **July 29, 2024** - Entry Deadline. You must accept the competition rules before this date in order to compete.
+- **July 29, 2024** - Team Merger Deadline. This is the last day participants may join or merge teams.
+- **August 5, 2024** - Final Submission Deadline.
+
+All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.
+
+## Prizes
+
+- 1st Place - $25,000
+- 2nd Place - $20,000
+- 3rd Place - $20,000
+- 4th Place - $20,000
+- 5th Place - $15,000
+
+## Code Requirements
+
+### This is a Code Competition
+
+Submissions to this competition must be made through Notebooks. In order for the "Submit" button to be active after a commit, the following conditions must be met:
+
+- CPU Notebook <= 9 hours run-time
+- GPU Notebook <= 9 hours run-time
+- Internet access disabled
+- Freely & publicly available external data is allowed, including pre-trained models
+- Submission file must be named `submission.csv`
+- Submission runtimes have been slightly obfuscated. If you repeat the exact same submission you will see up to 15 minutes of variance in the time before you receive your score.
+
+Please see the [Code Competition FAQ](https://www.kaggle.com/docs/competitions#notebooks-only-FAQ) for more information on how to submit. And review the [code debugging doc](https://www.kaggle.com/code-competition-debugging) if you are encountering submission errors.
+
+## Citation
+
+Wei-lin Chiang, Lianmin Zheng, Lisa Dunlap, Joseph E. Gonzalez, Ion Stoica, Paul Mooney, Sohier Dane, Addison Howard, Nate Keating. (2024). LMSYS - Chatbot Arena Human Preference Predictions. Kaggle. https://kaggle.com/competitions/lmsys-chatbot-arena
+
+# Data
+
+## Dataset Description
+
+The competition dataset consists of user interactions from the [ChatBot Arena](https://chat.lmsys.org/). In each user interaction a judge provides one or more prompts to two different large language models, and then indicates which of the models gave the more satisfactory response. The goal of the competition is to predict the preferences of the judges and determine the likelihood that a given prompt/response pair is selected as the winner.
+
+Please note that this is a [Code Competition](https://www.kaggle.com/competitions/lmsys-chatbot-arena/overview/code-requirements). When your submission is scored, this example test data will be replaced with the full test set. There are 55K rows in the training data, and you can expect roughly 25,000 rows in the test set.
+
+### Files
+
+**train.csv**
+
+- `id` - A unique identifier for the row.
+- `model_[a/b]` - The identity of model_[a/b]. Included in train.csv but not test.csv.
+- `prompt` - The prompt that was given as an input (to both models).
+- `response_[a/b]` - The response from model_[a/b] to the given prompt.
+- `winner_model_[a/b/tie]` - Binary columns marking the judge's selection. The ground truth target column.
+
+**test.csv**
+
+- `id`
+- `prompt`
+- `response_[a/b]`
+
+**sample_submission.csv** A submission file in the correct format.
+
+- `id`
+- `winner_model_[a/b/tie]` - This is what is predicted from the test set.
+
+*Note: the dataset for this competition contains text that may be considered profane, vulgar, or offensive.*

dslighting/registry/lmsys-chatbot-arena/description_obfuscated.md

@@ -0,0 +1,40 @@
+# Task
+
+Predict which chatbot response a user will prefer in a competition between two chatbots.
+
+# Metric
+
+Log loss with "eps=auto"
+
+# Submission Format
+
+For each id in the test set, you must predict the probability for each target class. The file should contain a header and have the following format:
+
+```
+ id,winner_model_a,winner_model_b,winner_tie
+ 136060,0.33,0,33,0.33
+ 211333,0.33,0,33,0.33
+ 1233961,0.33,0,33,0.33
+ etc
+```
+
+# Dataset
+
+**train.csv**
+
+- `id` - A unique identifier for the row.
+- `model_[a/b]` - The identity of model_[a/b]. Included in train.csv but not test.csv.
+- `prompt` - The prompt that was given as an input (to both models).
+- `response_[a/b]` - The response from model_[a/b] to the given prompt.
+- `winner_model_[a/b/tie]` - Binary columns marking the judge's selection. The ground truth target column.
+
+**test.csv**
+
+- `id`
+- `prompt`
+- `response_[a/b]`
+
+**sample_submission.csv** A submission file in the correct format.
+
+- `id`
+- `winner_model_[a/b/tie]` - This is what is predicted from the test set.

dslighting/registry/lmsys-chatbot-arena/grade.py

@@ -0,0 +1,63 @@
+import numpy as np
+import pandas as pd
+from sklearn.metrics import log_loss
+
+from mlebench.competitions.utils import one_hot_dfs_to_log_loss_inputs
+from mlebench.grade_helpers import InvalidSubmissionError
+
+
+def prepare_for_metric(submission: pd.DataFrame, answers: pd.DataFrame) -> dict:
+    """
+    The submission and answers are already one-hotted
+    """
+    classes = ["winner_model_a", "winner_model_b", "winner_tie"]
+    required_columns = ["id"] + classes
+
+    # Check if submission has the required columns
+    missing_columns = [col for col in required_columns if col not in submission.columns]
+    if missing_columns:
+        raise InvalidSubmissionError(
+            f"Submission DataFrame is missing required columns: {missing_columns}"
+        )
+
+    # Check if answers has the required columns
+    assert set(required_columns).issubset(
+        answers.columns
+    ), f"Answers DataFrame is missing required columns: {set(required_columns) - set(answers.columns)}"
+
+    # Check if submission has the correct number of rows
+    if len(submission) != len(answers):
+        raise InvalidSubmissionError(
+            f"Submission DataFrame must have {len(answers)} rows, but has {len(submission)} rows."
+        )
+
+    # Check if all values in submission are between 0 and 1
+    if (
+        not ((submission[classes] >= 0) & (submission[classes] <= 1)).all().all()
+    ):  # first all() checks if all rows are valid, second all() checks if all columns are valid
+        raise InvalidSubmissionError("All values in submission DataFrame must be between 0 and 1.")
+
+    # Check if each row in submission sums to 1
+    if not submission[classes].sum(axis=1).round(6).eq(1).all():
+        raise InvalidSubmissionError("Each row in submission DataFrame must sum to 1.")
+
+    # Use only the required columns for further processing
+    submission = submission[required_columns]
+    answers = answers[required_columns]
+
+    submission = submission.sort_values("id").reset_index(drop=True)
+    answers = answers.sort_values("id").reset_index(drop=True)
+
+    if (submission["id"].values != answers["id"].values).any():
+        raise InvalidSubmissionError("Submission and answer IDs do not match after sorting.")
+
+    log_loss_inputs = one_hot_dfs_to_log_loss_inputs(
+        submission, answers, id_column="id", apply_softmax=False
+    )
+
+    return log_loss_inputs
+
+
+def grade(submission: pd.DataFrame, answers: pd.DataFrame) -> float:
+    log_loss_inputs = prepare_for_metric(submission, answers)
+    return log_loss(**log_loss_inputs)

dslighting/registry/lmsys-chatbot-arena/prepare.py

@@ -0,0 +1,52 @@
+import shutil
+from pathlib import Path
+
+from sklearn.model_selection import train_test_split
+
+from mlebench.utils import extract, read_csv
+
+
+def prepare(raw: Path, public: Path, private: Path):
+    """
+    Splits the data in raw into public and private datasets with appropriate test/train splits.
+    """
+
+    # Create train, test from train split
+    old_train = read_csv(raw / "train.csv")
+    # Original train has 55k rows. Original hidden test has 25k rows. We make a new test set with 10% of the original train.
+    new_train, answers = train_test_split(old_train, test_size=0.1, random_state=0)
+    new_test = answers[["id", "prompt", "response_a", "response_b"]].copy()
+
+    sample_submission = answers[["id"]].copy()
+    sample_submission["winner_model_a"] = 0.3333333333333333
+    sample_submission["winner_model_b"] = 0.3333333333333333
+    sample_submission["winner_tie"] = 0.3333333333333333
+
+    # Checks
+    assert len(new_train) + len(new_test) == len(
+        old_train
+    ), f"New train and test should have the same number of rows as the original train"
+    assert set(new_train["id"]).isdisjoint(
+        set(new_test["id"])
+    ), f"New train and test should have no overlapping ids"
+    assert new_test.columns.tolist() == [
+        "id",
+        "prompt",
+        "response_a",
+        "response_b",
+    ], f"New test should have columns id, prompt, response_a, response_b"
+    assert sample_submission.columns.tolist() == [
+        "id",
+        "winner_model_a",
+        "winner_model_b",
+        "winner_tie",
+    ], f"Sample submission should have columns id, winner_model_a, winner_model_b, winner_tie"
+    assert (
+        new_train.columns.tolist() == old_train.columns.tolist()
+    ), f"New train should have the same columns as the original train"
+
+    # Write CSVs
+    answers.to_csv(private / "answers.csv", index=False)
+    new_train.to_csv(public / "train.csv", index=False)
+    new_test.to_csv(public / "test.csv", index=False)
+    sample_submission.to_csv(public / "sample_submission.csv", index=False)

dslighting/registry/lmsys-chatbot-arena/prepare_val.py

@@ -0,0 +1,115 @@
+import shutil
+from pathlib import Path
+
+import pandas as pd
+from sklearn.model_selection import train_test_split
+
+from mlebench.utils import extract, read_csv
+
+
+def _create_split(
+    df_in: pd.DataFrame,
+    public_path: Path,
+    private_path: Path,
+    test_size: float,
+    random_state: int,
+) -> pd.DataFrame:
+    """
+    Helper function to perform a data split and write files to specified directories.
+
+    Args:
+        df_in: The input DataFrame to split.
+        public_path: The directory for public-facing files (train set, test features).
+        private_path: The directory for private files (test answers).
+        test_size: The proportion of the dataset to allocate to the test set.
+        random_state: The seed for the random number generator.
+
+    Returns:
+        The DataFrame for the newly created training set.
+    """
+    # Create output directories if they don't exist
+    public_path.mkdir(exist_ok=True, parents=True)
+    private_path.mkdir(exist_ok=True, parents=True)
+
+    # Create train, test from the input dataframe
+    new_train, answers = train_test_split(
+        df_in, test_size=test_size, random_state=random_state
+    )
+    new_test = answers[["id", "prompt", "response_a", "response_b"]].copy()
+
+    sample_submission = answers[["id"]].copy()
+    sample_submission["winner_model_a"] = 0.3333333333333333
+    sample_submission["winner_model_b"] = 0.3333333333333333
+    sample_submission["winner_tie"] = 0.3333333333333333
+
+    # Checks
+    assert len(new_train) + len(new_test) == len(
+        df_in
+    ), f"New train and test should have the same number of rows as the original dataframe"
+    assert set(new_train["id"]).isdisjoint(
+        set(new_test["id"])
+    ), f"New train and test should have no overlapping ids"
+    assert new_test.columns.tolist() == [
+        "id",
+        "prompt",
+        "response_a",
+        "response_b",
+    ], f"New test should have columns id, prompt, response_a, response_b"
+    assert sample_submission.columns.tolist() == [
+        "id",
+        "winner_model_a",
+        "winner_model_b",
+        "winner_tie",
+    ], f"Sample submission should have columns id, winner_model_a, winner_model_b, winner_tie"
+    assert (
+        new_train.columns.tolist() == df_in.columns.tolist()
+    ), f"New train should have the same columns as the original dataframe"
+
+    # Write CSVs
+    answers.to_csv(private_path / "answers.csv", index=False)
+    new_train.to_csv(public_path / "train.csv", index=False)
+    new_test.to_csv(public_path / "test.csv", index=False)
+    sample_submission.to_csv(public_path / "sample_submission.csv", index=False)
+
+    return new_train
+
+
+def prepare(raw: Path, public: Path, private: Path):
+    """
+    Splits the data in raw into public and private datasets with appropriate test/train splits.
+    Also creates a secondary validation split (public_val, private_val) for local testing.
+    """
+
+    # --- Stage 1: Create the original competition split (train/test) ---
+    # This block generates the primary `public` and `private` directories.
+    # Its outputs MUST remain identical to the original script's outputs.
+    old_train_df = read_csv(raw / "train.csv")
+    train_for_val_split = _create_split(
+        df_in=old_train_df,
+        public_path=public,
+        private_path=private,
+        test_size=0.1,
+        random_state=0,
+    )
+
+    # --- Stage 2: Create the new validation split (train_val/test_val) ---
+    # This block takes the training set from Stage 1 and splits it again
+    # to create a new, smaller training set and a validation set.
+    # The outputs are saved to parallel `public_val` and `private_val` directories.
+    public_val = public.parent / "public_val"
+    private_val = private.parent / "private_val"
+
+    # Calculate the test_size needed to make the new validation set (`test_val`)
+    # have the same number of samples as the original test set from Stage 1.
+    # Original test size = 0.1 * total. New train size = 0.9 * total.
+    # We need a fraction `p` such that p * (0.9 * total) = 0.1 * total.
+    # p = 0.1 / 0.9 = 1/9.
+    val_test_size = 1 / 9
+
+    _create_split(
+        df_in=train_for_val_split,
+        public_path=public_val,
+        private_path=private_val,
+        test_size=val_test_size,
+        random_state=0,  # Use same random state for consistency
+    )

dslighting/registry/mcm-2025-c/config.yaml

@@ -0,0 +1,8 @@
+description: description.md
+id: mcm-2025-c
+metric: successful_file_read_rate
+name: Summer Olympics Dataset Analysis
+private_dir: mcm-2025-c/prepared/private
+public_dir: mcm-2025-c/prepared/public
+raw_dir: mcm-2025-c/raw
+task_type: data_cleaning

dslighting/registry/mcm-2025-c/description.md

@@ -0,0 +1,3 @@
+# Task Initialization: mcm-2025-c
+
+- `[08:35:31]` Upload complete. Ready for EDA.

dslighting/registry/mcm-2025-c/eda_report.md

@@ -0,0 +1,61 @@
+<UPDATE_REPORT>
+# Exploratory Data Analysis Report: Olympic Medal Prediction
+
+## 1. Data Overview
+- **Source**: Updated dataset with enhanced feature engineering
+- **Records**: 5,670 rows (increased from 4,752)
+- **Features**: 10 columns (expanded from 7)
+- **Primary Target**: `total_medals` - medals won by country in specific Olympic year
+
+## 2. Data Structure
+| Column | Type | Description |
+|--------|------|-------------|
+| id | int64 | Unique identifier for each record |
+| medals_lag_1 | float64 | Medal count from previous Olympics (1 lag) |
+| medals_lag_2 | float64 | Medal count from two Olympics prior (2 lag) |
+| medals_lag_3 | float64 | Medal count from three Olympics prior (3 lag) |
+| medals_ma_3 | float64 | 3-Olympic moving average of medal counts |
+| is_host | int64 | Binary indicator (1 if host country) |
+| country_avg_medals | float64 | Historical average medals per Olympics for country |
+| country_max_medals | float64 | Maximum medals ever won by country in single Olympics |
+| country_total_medals | float64 | Total historical medals won by country |
+| total_medals | int64 | Target: medals won by country in current Olympic year |
+
+## 3. Key Dataset Changes
+- **Enhanced Features**: Added comprehensive country-level statistics (avg, max, total)
+- **Moving Average**: New `medals_ma_3` feature captures trend momentum
+- **Record Increase**: Additional 918 records (19% increase)
+- **Identifier**: `id` column added for unique record identification
+
+## 4. Feature Engineering Assessment
+### Lag Features (medals_lag_1, medals_lag_2, medals_lag_3)
+- Maintain temporal dependency structure
+- Provide short-to-medium term performance memory
+
+### Moving Average (medals_ma_3)
+- Smooths out Olympic-cycle fluctuations
+- Captures performance trends beyond single lags
+
+### Country-Level Statistics
+- **country_avg_medals**: Baseline performance expectation
+- **country_max_medals**: Peak performance capability
+- **country_total_medals**: Overall historical success
+
+## 5. Data Quality Implications
+- **Missing Country Column**: Analysis now focuses on statistical patterns rather than country-specific narratives
+- **Enhanced Predictive Power**: Country-level features provide strong baseline predictors
+- **Comprehensive Coverage**: Expanded feature set captures multiple performance dimensions
+
+## 6. Modeling Advantages
+- **Rich Feature Set**: Multiple complementary perspectives on performance
+- **Baseline Establishment**: Country statistics provide strong priors
+- **Trend Capture**: Moving average adds momentum component
+- **Reduced Overfitting**: Multiple correlated features allow regularization benefits
+
+## 7. Analytical Considerations
+- **Feature Correlation**: Country statistics likely highly correlated with lag features
+- **Scale Differences**: Features span different measurement scales (may require normalization)
+- **Interpretability**: Country-level features provide intuitive performance benchmarks
+
+This updated dataset represents a significant enhancement with comprehensive feature engineering that captures both recent performance trends and historical country capabilities.
+</UPDATE_REPORT>

dslighting/registry/mcm-2025-c/report.md

@@ -0,0 +1,32 @@
+# Initial Data Scan: Summer Olympics Dataset Analysis
+
+
+
+--- COMPREHENSIVE DATA REPORT ---
+
+## Directory Structure (Current Working Directory)
+```text
+./
+├── ._data_dictionary.csv
+├── ._summerOly_athletes.csv
+├── ._summerOly_hosts.csv
+├── ._summerOly_medal_counts.csv
+├── ._summerOly_programs.csv
+├── data_dictionary.csv
+├── sample.csv
+├── summerOly_athletes.csv
+├── summerOly_hosts.csv
+├── summerOly_medal_counts.csv
+└── summerOly_programs.csv
+```
+
+## Data Schema Analysis
+### Analysis of `._data_dictionary.csv`
+Could not be analyzed due to error: 'utf-8' codec can't decode byte 0xa2 in position 37: invalid start byte
+
+### Analysis of `data_dictionary.csv`
+Could not be analyzed due to error: 'utf-8' codec can't decode byte 0x96 in position 1356: invalid start byte
+
+### Analysis of `sample.csv`
+Could not be analyzed due to error: 'utf-8' codec can't decode byte 0x95 in position 1030: invalid start byte
+

dslighting/registry/mcm_2024_c/config.yaml

@@ -0,0 +1,8 @@
+description: description.md
+id: mcm_2024_c
+metric: Accuracy
+name: Wimbledon Point Outcome Prediction
+private_dir: mcm_2024_c/prepared/private
+public_dir: mcm_2024_c/prepared/public
+raw_dir: mcm_2024_c/raw
+task_type: Binary Classification

dslighting/registry/mcm_2024_c/description.md

@@ -0,0 +1 @@
+Develop a model that captures the flow of play as points occur and apply it to one or more of the matches. Your model should identify which player is performing better at a given time in the match, as well as how much better they are performing. Provide a visualization based on your model to depict the match flow. Note: in tennis, the player serving has a much higher probability of winning the point/game. You may wish to factor this into your model in some way.

dslighting/registry/mcm_2024_c/report.md

@@ -0,0 +1,128 @@
+# Mathematical Modeling of Tennis Match Dynamics and Performance Evaluation
+
+## 1. Problem Analysis
+
+Tennis is a sport characterized by high-intensity intermittent efforts and distinct scoring systems. Evaluating player performance requires looking beyond simple match scores to understand the underlying dynamics of momentum, physical exertion, and technical efficiency. 
+
+Based on the provided Wimbledon dataset, we aim to address two primary objectives:
+1.  **Statistical Characterization:** Analyze the relationships between physical metrics (distance run), technical metrics (serve speed, aces), and match progression (rally count).
+2.  **Performance Flow Modeling:** Develop a quantitative model to evaluate player momentum and performance deviation, specifically adjusting for the inherent advantage of serving.
+
+The dataset `Wimbledon_featured_matches.csv` provides point-by-point data including rally counts, serve speeds, and player movements. Initial analysis reveals that most rallies are short (1-5 shots), emphasizing the importance of serve and return efficiency.
+
+## 2. Model Assumptions
+
+To simplify the complex reality of a tennis match for mathematical modeling, we make the following assumptions:
+
+1.  **Serve Win Probability ($P_{serve}$):** There exists a baseline probability that a player will win a point while serving. Based on the synthetic data generation logic, we assume a global baseline $P_{serve} \approx 0.65$, though we acknowledge this varies by player.
+2.  **Independence of Points:** While momentum exists, we assume the outcome of any specific point is primarily determined by the server/returner dynamic and immediate physical state, rather than long-term historical sequences (excluding the cumulative effect modeled).
+3.  **Data Representativeness:** The missing data in `speed_mph` (10.3%) is assumed to be missing at random and does not significantly bias the overall statistical distribution of serve speeds.
+4.  **Linear Physical Exertion:** The distance run is linearly proportional to the physical fatigue cost on the player for the duration of the match analyzed.
+
+## 3. Notation and Variable Definitions
+
+| Symbol | Description | Unit |
+| :--- | :--- | :--- |
+| $N$ | Total number of points in a match | count |
+| $R_i$ | Rally count for point $i$ | shots |
+| $V_i$ | Serve speed for point $i$ | mph |
+| $D_{1,i}, D_{2,i}$ | Distance run by Player 1 and Player 2 up to point $i$ | meters |
+| $S_i$ | Server for point $i$ ($P_1$ or $P_2$) | categorical |
+| $W_i$ | Winner of point $i$ ($P_1$ or $P_2$) | categorical |
+| $P_{serve}$ | Probability of the server winning the point | probability |
+| $\delta_i$ | Performance deviation score for point $i$ | score |
+| $CPI_k$ | Cumulative Performance Index after point $k$ | score |
+
+## 4. Model Formulation
+
+### 4.1 Statistical Analysis Model
+
+We analyze the correlation between rally length and physical exertion. Let $X$ be the vector of rally counts and $Y$ be the vector of distances run. We calculate the Pearson correlation coefficient $\rho$:
+
+$$ \rho_{X,Y} = \frac{\text{cov}(X,Y)}{\sigma_X \sigma_Y} $$
+
+Based on the EDA results, we expect $\rho \approx 0.82$, indicating a strong positive linear relationship: longer rallies necessitate significantly more movement.
+
+### 4.2 Performance Flow Model
+
+To quantify momentum and evaluate performance beyond simple win/loss records, we define a **Cumulative Performance Index (CPI)**. This model adjusts the raw point outcome by the expected outcome based on the inherent advantage of serving.
+
+Let $P_{serve}$ represent the baseline probability that the server wins the point. Based on our model assumptions, we assume $P_{serve} \approx 0.65$. The actual outcome of a point is binary: $1$ for a win and $0$ for a loss. The performance deviation $\delta_i$ for point $i$ is calculated from the perspective of a specific player (e.g., Player 1), accounting for whether they were serving or returning.
+
+The deviation $\delta_i$ is defined as follows:
+
+$$ \delta_i = \begin{cases} 
+1 - P_{serve} & \text{if Player wins on Serve (Expected win, small positive gain)} \\
+P_{serve} & \text{if Player wins on Return (Break of serve, large positive gain)} \\
+- P_{serve} & \text{if Player loses on Serve (Break against, large negative)} \\
+-(1 - P_{serve}) & \text{if Player loses on Return (Expected loss, small negative)} 
+\end{cases} $$
+
+The **Cumulative Performance Index ($CPI$)** for Player 1 after $k$ points is the summation of these deviations:
+
+$$ CPI_k = \sum_{i=1}^{k} \delta_i $$
+
+**Interpretation:**
+
+*   A **positive $CPI$** indicates that the player is outperforming expectations (e.g., successfully breaking serve or holding serve comfortably).
+*   A **negative $CPI$** indicates underperformance relative to the baseline (e.g., failing to convert break points or losing serve unexpectedly).
+
+This metric allows us to visualize the "flow" of the match, highlighting momentum shifts that standard scores might obscure.
+
+## 5. Solution Methods
+
+1.  **Data Preprocessing:**
+    *   Loaded `Wimbledon_featured_matches.csv` using Pandas.
+    *   Handled missing values in `speed_mph` (752 instances, 10.3%) by exclusion during distribution analysis to maintain accuracy of calculated statistics.
+
+2.  **Exploratory Data Analysis (EDA):**
+    *   **Visualization:** Generated box plots, distribution histograms, and correlation heatmaps using Seaborn and Matplotlib.
+    *   **Statistical Calculation:** Computed mean, median, standard deviation, and interquartile ranges (IQR) for key variables ($R_i, V_i, D$).
+
+3.  **Flow Model Simulation:**
+    *   Implemented the CPI calculation in Python (see `model_code_001.py`).
+    *   Simulated match flow to visualize momentum shifts.
+    *   Identified "Break Points" as critical inflection points where $\delta_i$ is maximized ($\delta = P_{serve} = 0.65$).
+
+## 6. Model Results
+
+### 6.1 Statistical Characteristics
+
+*   **Rally Dynamics:** The mean rally count is $3.13$ shots with a median of $2.00$. The distribution is right-skewed, confirming that professional tennis is dominated by short rallies. The Interquartile Range (IQR) is 1 to 4 shots, with 457 outliers identified (longer rallies).
+*   **Serve Speed:** The average serve speed is $112.41$ mph, ranging from $72$ to $141$ mph.
+*   **Physical Exertion:** There is a strong correlation between rally count and distance run ($\rho \approx 0.82$). Player 1 runs an average of $14.00$m per point compared to Player 2's $13.87$m.
+*   **Serve Impact:** Serve speed shows a moderate positive correlation with aces ($\rho \approx 0.16$), validating the importance of power serving.
+
+### 6.2 Performance Flow Analysis
+
+The Flow Model highlights the volatility of tennis matches:
+*   **Momentum Swings:** The CPI visualization demonstrates that matches often feature distinct "zones of dominance" where the cumulative index trends positively or negatively for extended periods.
+*   **Break Significance:** The model quantifies the high impact of breaking serve. A single break point contributes a delta of $+0.65$ to the returner's CPI, whereas holding serve contributes only $+0.35$. This mathematically explains why breaks are so crucial to match outcome.
+
+## 7. Sensitivity Analysis
+
+The Performance Flow Model is sensitive to the parameter $P_{serve}$ (the assumed probability of holding serve).
+
+*   **Scenario A ($P_{serve} = 0.60$):** The value of a break decreases ($\delta = 0.60$), and the penalty for getting broken decreases ($\delta = -0.60$). The CPI curve becomes less volatile, suggesting that breaks are less decisive.
+*   **Scenario B ($P_{serve} = 0.70$):** The value of a break increases ($\delta = 0.70$). The CPI curve becomes more volatile, with steeper drops for the player who gets broken.
+
+**Conclusion:** The model is robust in identifying momentum direction, but the *magnitude* of the swings is directly proportional to the dominance of the serve in the specific match context.
+
+## 8. Strengths and Weaknesses
+
+### Strengths
+1.  **Contextual Evaluation:** The CPI model accounts for the serve advantage, providing a more nuanced view of performance than raw point totals.
+2.  **Visual Intuition:** The cumulative plot offers an immediate visual representation of match flow and momentum shifts.
+3.  **Data-Driven:** The statistical analysis leverages actual match data (Wimbledon) to ground assumptions about rally length and physical exertion.
+
+### Weaknesses
+1.  **Static Probability:** The model assumes a constant $P_{serve}$. In reality, this probability changes based on fatigue, score pressure (e.g., 30-30 vs 40-0), and court surface.
+2.  **Simplified Scoring:** The model treats all points equally in terms of importance, whereas in tennis, break points and set points carry higher psychological weight.
+3.  **Missing Data:** The 10.3% missing serve speed data limits the precision of the correlation analysis regarding serve effectiveness.
+
+## 9. Model Improvements
+
+1.  **Dynamic Probability:** Incorporate a Markov Chain model where $P_{serve}$ updates dynamically based on the current score state (e.g., $P(\text{win} | \text{score}=30-30) < P(\text{win} | \text{score}=40-0)$).
+2.  **Fatigue Factor:** Integrate a decay function into the CPI based on cumulative distance run ($D_{total}$). As $D_{total}$ increases, the expected $P_{serve}$ could decrease, simulating physical fatigue.
+3.  **Cluster Analysis:** Apply K-Means clustering to rally types (e.g., "Serve/Volley", "Baseline Rally") to categorize points and apply different performance weights to each cluster.
+4.  **Imputation:** Use K-Nearest Neighbors (KNN) imputation to estimate missing `speed_mph` values based on the player's historical averages, improving the dataset's completeness.

dslighting/registry/mcm_2024_c_test/config.yaml

@@ -0,0 +1,16 @@
+competition_type: open_ended
+dataset:
+  answers: mcm_2024_c_test/raw/answer.csv
+  gold_submission: mcm_2024_c_test/raw/answer.csv
+  sample_submission: mcm_2024_c_test/raw/sampleSubmission.csv
+description: description.md
+grader:
+  grade_fn: file:/Users/liufan/Applications/Github/dslighting/benchmarks/mlebench/competitions/mcm_2024_c_test/grade.py:grade
+  name: LLMJudge
+id: mcm_2024_c_test
+metric: LLM Score
+name: Tennis Match Flow and Performance Analysis
+preparer: file:/Users/liufan/Applications/Github/dslighting/benchmarks/mlebench/competitions/mcm_2024_c_test/prepare.py:prepare
+private_dir: mcm_2024_c_test/prepared/private
+public_dir: mcm_2024_c_test/prepared/public
+raw_dir: mcm_2024_c_test/raw

dslighting/registry/mcm_2024_c_test/description.md

@@ -0,0 +1,3 @@
+# Task Initialization: mcm_2024_c_test
+
+- `[16:50:56]` Upload complete. Ready for EDA.