dslighting 1.3.9__py3-none-any.whl → 1.7.1__py3-none-any.whl

dslighting/registry/AI4Code/checksums.yaml

@@ -0,0 +1,7 @@
+private:
+  test_orders.csv: 58e1e03e110aa2d805cd84f6e6a6477f
+public:
+  sample_submission.csv: dc03df9d2f67ead372ce0a244360268a
+  train_ancestors.csv: 5a4c8b08ea96ac4039515cba1832da4f
+  train_orders.csv: b537c96c54e435385363b670b7672cba
+zip: 0c118bd40925cdc0ca35e5e22460f836

dslighting/registry/AI4Code/config.yaml

@@ -0,0 +1,26 @@
+id: AI4Code
+name: Google AI4Code - Understand Code in Python Notebooks
+competition_type: code
+awards_medals: true
+prizes:
+  - position: 1
+    value: 50000
+  - position: 2
+    value: 40000
+  - position: 3
+    value: 30000
+  - position: 4
+    value: 20000
+  - position: 5
+    value: 10000
+description: mlebench/competitions/AI4Code/description.md
+
+dataset:
+  answers: AI4Code/prepared/private/test_orders.csv
+  sample_submission: AI4Code/prepared/public/sample_submission.csv
+
+grader:
+  name: kendall-tau
+  grade_fn: mlebench.competitions.AI4Code.grade:grade
+
+preparer: mlebench.competitions.AI4Code.prepare:prepare

dslighting/registry/AI4Code/description.md

@@ -0,0 +1,122 @@
+# Overview
+
+## Description
+
+The goal of this competition is to understand the relationship between code and comments in Python notebooks. You are challenged to reconstruct the order of markdown cells in a given notebook based on the order of the code cells, demonstrating comprehension of which natural language references which code.
+
+### Context
+
+Research teams across Google and Alphabet are exploring new ways that machine learning can assist software developers, and want to rally more members of the developer community to help explore this area too. Python notebooks provide a unique learning opportunity, because unlike a lot of standard source code, notebooks often follow narrative format, with comment cells implemented in markdown that explain a programmer's intentions for corresponding code cells. An understanding of the relationships between code and markdown could lend to fresh improvements across many aspects of AI-assisted development, such as the construction of better data filtering and preprocessing pipelines for model training, or automatic assessments of a notebook's readability.
+
+We have assembled a dataset of approximately 160,000 public Python notebooks from Kaggle and have teamed up with [X, the moonshot factory](https://x.company/) to design a competition that challenges participants to use this dataset of published notebooks to build creative techniques aimed at better understanding the relationship between comment cells and code cells.
+
+![Image of Notebook Cells](https://storage.googleapis.com/kaggle-media/Images/notebook_cell_examples.png)
+
+After the submission deadline, Kaggle and X will evaluate the performance of submitted techniques on new, previously unseen notebooks. We're excited to see how the insights learned from this competition affect the future of notebook authorship.
+
+## Evaluation
+
+Predictions are evaluated by the [Kendall tau correlation](https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient) between predicted cell orders and ground truth cell orders accumulated across the entire collection of test set notebooks.
+
+Let $S$ be the number of swaps of adjacent entries needed to sort the predicted cell order into the ground truth cell order. In the worst case, a predicted order for a notebook with $n$ cells will need $\frac{1}{2} n(n-1)$ swaps to sort.
+
+We sum the number of swaps from your predicted cell order across the entire collection of test set notebooks, and similarly with the worst-case number of swaps. We then compute the Kendall tau correlation as:
+
+$K=1-4 \frac{\sum_i S_i}{\sum_i n_i\left(n_i-1\right)}$
+
+You may find a Python implementation in this notebook: [Competition Metric - Kendall Tau Correlation](https://www.kaggle.com/ryanholbrook/competition-metric-kendall-tau-correlation).
+
+## Submission File
+
+For each `id` in the test set (representing a notebook), you must predict `cell_order`, the correct ordering of its cells in terms of the cell ids. The file should contain a header and have the following format:
+
+```
+id,cell_order
+0009d135ece78d,ddfd239c c6cd22db 1372ae9b ...
+0010483c12ba9b,54c7cab3 fe66203e 7844d5f8 ...
+0010a919d60e4f,aafc3d23 80e077ec b190ebb4 ...
+0028856e09c5b7,012c9d02 d22526d1 3ae7ece3 ...
+etc.
+```
+
+## Timeline
+
+**This is a two-stage competition, with a training stage and a collection stage, during which models will be rerun against notebooks which are yet to be authored.**
+
+### Training Timeline
+
+- May 11, 2022 - Start Date
+- August 4, 2022 - Entry deadline. You must accept the competition rules before this date in order to compete.
+- August 4, 2022 - Team Merger deadline. This is the last day participants may join or merge teams.
+- August 11, 2022 - Final submission deadline.
+
+All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.
+
+### Collection Timeline:
+
+Starting after the final submission deadline, there will be periodic updates to the leaderboard to reflect models' performance against newly authored notebooks on Kaggle.
+
+- November 10, 2022 - Competition End Date - Winner's announcement
+
+## Prizes
+
+- 1st Place - $50,000
+- 2nd Place - $40,000
+- 3rd Place - $30,000
+- 4th Place - $20,000
+- 5th Place - $10,000
+
+## Code Requirements
+
+### This is a Code Competition
+
+Submissions to this competition must be made through Notebooks. In order for the "Submit" button to be active after a commit, the following conditions must be met:
+
+- CPU Notebook <= 9 hours run-time
+- GPU Notebook <= 9 hours run-time
+- Internet access disabled
+- Freely & publicly available external data is allowed, including pre-trained models
+- Submission file must be named `submission.csv`
+
+Please see the [Code Competition FAQ](https://www.kaggle.com/docs/competitions#notebooks-only-FAQ) for more information on how to submit. And review the [code debugging doc](https://www.kaggle.com/code-competition-debugging) if you are encountering submission errors.
+
+## Citation
+
+Addison Howard, Alex Polozov, Bin Ni, Christopher Tirrell, MicHCR, Michele (pirroh) Catasta, Olivia Hatalsky, Rishabh Singh, Ryan Holbrook, Will Cukierski. (2022). Google AI4Code -- Understand Code in Python Notebooks. Kaggle. https://kaggle.com/competitions/AI4Code
+
+
+# Dataset Description
+
+The dataset for this competition comprises about 160,000 Jupyter notebooks published by the Kaggle community. Jupyter notebooks are the tool of choice for many data scientists for their ability to tell a narrative with both code and natural language. These two types of discourse are contained within **cells** of the notebook, and we refer to these cells as either **code** cells or **markdown** cells ([markdown](https://en.wikipedia.org/wiki/Markdown) being the text formatting language used by Jupyter).
+
+Your task is to predict the correct ordering of the cells in a given notebook whose markdown cells have been shuffled.
+
+The notebooks in this dataset have been selected and processed to ensure their suitability for the competition task. All notebooks:
+
+- Have been published publicly on Kaggle under the [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) open source license.
+- Represent the most recently published version of the notebook.
+- Contain at least one code cell and at least one markdown cell.
+- Have code written in the Python language.
+- Have had empty cells removed.
+
+This is a code competition, in which you will submit a model or code that will be run against a future test set:
+
+- The first-stage test set contains notebooks from an approximately 90-day historical window of time.
+- The second-stage test set will contain a similar number of notebooks, collected from a future 90-day window of time. This is necessary to prevent models from looking up the order of existing public notebooks. The selection criteria for second-stage notebooks will be monitored for competition fairness purposes. For example, it will exclude competition participants' own notebooks.
+
+## Training Data
+
+- **train/** - A folder comprising about 140,000 JSON files with the filenames corresponding to the `id` field in the `csv` files. Each file contains the code and markdown cells of a Kaggle notebook. **The code cells are in their original (correct) order. The markdown cells have been shuffled** and placed after the code cells.
+- **train_orders.csv** - Gives the correct order of the cells for each notebook in the `train/` folder.
+    - `id` - The notebook in file `{id}.json`.
+    - `cell_order` - A space delimited list of the correct cell ordering given in terms of the order in `{id}.json`.
+- **train_ancestors.csv** - On Kaggle, a user may "fork" (that is, copy) the notebook of another user to create their own version. This file contains the forking history of notebooks in the training set. **Note: There is no corresponding file for the test set.**
+    - `ancestor_id` - Identifies sets of notebooks that have a common origin or "ancestor". As no notebook in the test set has an ancestor in the training set, you may find this field to be of use as a grouping factor when constructing validation splits.
+    - `parent_id` - Indicates that some version of the notebook `id` was forked from some version of the notebook `parent_id`. The notebook `parent_id` may or may not be present in the training data. (The parent may be missing because someone had forked a private notebook of their own, for instance.)
+
+## Example Test Data
+
+To help you author submission code, we include a few example instances selected from the test set. When you submit your notebook for scoring, this example data will be replaced by the actual test data, including the `sample_submission.csv` file.
+
+- **test/** - A few example notebooks from the test set. The actual test set comprises about 20,000 notebooks in a format similar to the training set notebooks. No notebook in the test set has an ancestor in the training set.
+- **sample_submission.csv** - A sample submission file in the correct format. See the [**Evaluation**](https://www.kaggle.com/competitions/AI4Code/overview/evaluation) page for more details.

dslighting/registry/AI4Code/description_obfuscated.md

@@ -0,0 +1,38 @@
+# Task
+
+Predict the correct ordering of the cells in a given notebook whose markdown cells have been shuffled.
+
+# Metric
+
+Kendall tau correlation between predicted cell orders and ground truth cell orders accumulated across the entire collection of test set notebooks.
+
+Let $S$ be the number of swaps of adjacent entries needed to sort the predicted cell order into the ground truth cell order. In the worst case, a predicted order for a notebook with $n$ cells will need $\frac{1}{2} n(n-1)$ swaps to sort.
+
+We sum the number of swaps from your predicted cell order across the entire collection of test set notebooks, and similarly with the worst-case number of swaps. We then compute the Kendall tau correlation as:
+
+$K=1-4 \frac{\sum_i S_i}{\sum_i n_i\left(n_i-1\right)}$
+
+# Submission Format
+
+For each `id` in the test set (representing a notebook), you must predict `cell_order`, the correct ordering of its cells in terms of the cell ids. The file should contain a header and have the following format:
+
+```
+id,cell_order
+0009d135ece78d,ddfd239c c6cd22db 1372ae9b ...
+0010483c12ba9b,54c7cab3 fe66203e 7844d5f8 ...
+0010a919d60e4f,aafc3d23 80e077ec b190ebb4 ...
+0028856e09c5b7,012c9d02 d22526d1 3ae7ece3 ...
+etc.
+```
+
+# Dataset 
+
+- **train/** - A folder comprising about 140,000 JSON files with the filenames corresponding to the `id` field in the `csv` files. Each file contains the code and markdown cells of a notebook. **The code cells are in their original (correct) order. The markdown cells have been shuffled** and placed after the code cells.
+- **train_orders.csv** - Gives the correct order of the cells for each notebook in the `train/` folder.
+    - `id` - The notebook in file `{id}.json`.
+    - `cell_order` - A space delimited list of the correct cell ordering given in terms of the order in `{id}.json`.
+- **train_ancestors.csv** - A user may "fork" (that is, copy) the notebook of another user to create their own version. This file contains the forking history of notebooks in the training set. **Note: There is no corresponding file for the test set.**
+    - `ancestor_id` - Identifies sets of notebooks that have a common origin or "ancestor". As no notebook in the test set has an ancestor in the training set, you may find this field to be of use as a grouping factor when constructing validation splits.
+    - `parent_id` - Indicates that some version of the notebook `id` was forked from some version of the notebook `parent_id`. The notebook `parent_id` may or may not be present in the training data. (The parent may be missing because someone had forked a private notebook of their own, for instance.)
+- **test/** - Notebooks from the test set. 
+- **sample_submission.csv** - A sample submission file in the correct format. 

dslighting/registry/AI4Code/grade.py

@@ -0,0 +1,70 @@
+from bisect import bisect
+from typing import Any
+
+import pandas as pd
+
+from mlebench.grade_helpers import InvalidSubmissionError
+
+
+def kendall_tau(ground_truth: list[list[Any]], predictions: list[list[Any]]) -> float:
+    """
+    Computes the Kendall Tau correlation between
+    - `ground_truth`: a list of lists, where each sublist contains the ground truth order of items
+    - `predictions`: a list of lists, where each sublist contains the predicted order of items
+
+    See: https://www.kaggle.com/code/ryanholbrook/competition-metric-kendall-tau-correlation
+    """
+
+    # Actually O(N^2), but fast in practice for our data
+    def count_inversions(a):
+        inversions = 0
+        sorted_so_far = []
+        for i, u in enumerate(a):  # O(N)
+            j = bisect(sorted_so_far, u)  # O(log N)
+            inversions += i - j
+            sorted_so_far.insert(j, u)  # O(N)
+        return inversions
+
+    total_inversions = 0  # total inversions in predicted ranks across all instances
+    total_2max = 0  # maximum possible inversions across all instances
+    for gt, pred in zip(ground_truth, predictions):
+        ranks = [gt.index(x) for x in pred]  # rank predicted order in terms of ground truth
+        total_inversions += count_inversions(ranks)
+        n = len(gt)
+        total_2max += n * (n - 1)
+    return 1 - 4 * total_inversions / total_2max
+
+
+def prepare_for_metric(submission: pd.DataFrame, answers: pd.DataFrame) -> dict:
+    if len(submission) != len(answers):
+        raise InvalidSubmissionError("Submission and answers must have the same length")
+    if "id" not in submission.columns or "cell_order" not in submission.columns:
+        raise InvalidSubmissionError("Submission must have 'id' and 'cell_order' columns")
+    if set(submission["id"]) != set(answers["id"]):
+        raise InvalidSubmissionError("Submission and answers must have the same ids")
+    # "cell_order" columns must be strings of space-separated cell ids: "54c7cab3 fe66203e 7844d5f8 ..."
+    submission["cell_order"] = submission["cell_order"].astype(str)
+    answers["cell_order"] = answers["cell_order"].astype(str)
+
+    # Sort by id to ensure we're comparing the same rows
+    submission = submission.sort_values("id")
+    answers = answers.sort_values("id")
+
+    ground_truth, predictions = [], []
+    target_column = "cell_order"
+    for (_, row_gt), (_, row_pred) in zip(submission.iterrows(), answers.iterrows()):
+        ground_truth_items = row_gt[target_column].split()
+        prediction_items = row_pred[target_column].split()
+        if len(ground_truth_items) != len(prediction_items):
+            raise InvalidSubmissionError("Ground truth and predictions must have the same length")
+        ground_truth.append(ground_truth_items)
+        predictions.append(prediction_items)
+    return {
+        "ground_truth": ground_truth,
+        "predictions": predictions,
+    }
+
+
+def grade(submission: pd.DataFrame, answers: pd.DataFrame) -> float:
+    kendall_tau_inputs = prepare_for_metric(submission, answers)
+    return kendall_tau(**kendall_tau_inputs)

dslighting/registry/AI4Code/prepare.py

@@ -0,0 +1,84 @@
+import json
+import shutil
+from pathlib import Path
+
+import pandas as pd
+from tqdm.auto import tqdm
+
+from mlebench.utils import read_csv
+
+
+# Create train, test from train split
+def create_train_test_split(train_ancestors_df: pd.DataFrame, test_size: int):
+    """
+    On Kaggle, a user may "fork" (that is, copy) the notebook of another user to create their own version.
+    `train_ancestors_df` contains information about the ancestry of each notebook in the training set.
+
+    To create the test split, we select rows from the train set that don't share ancestors with any others
+    so that there aren't any relatives in the train set. The train split is the remaining rows.
+    """
+    group_by_ancestor = {}
+    for _, row in tqdm(
+        train_ancestors_df.iterrows(), desc="Grouping by Ancestor", total=len(train_ancestors_df)
+    ):
+        ancestor_id = row["ancestor_id"]
+        if ancestor_id not in group_by_ancestor:
+            group_by_ancestor[ancestor_id] = []
+        group_by_ancestor[ancestor_id].append(row)
+
+    train_ids = []
+    test_ids = []
+    num_test_ids = 0
+    for ancestor_id, rows in tqdm(group_by_ancestor.items(), desc="Splitting on Ancestors"):
+        if num_test_ids + len(rows) <= test_size:
+            test_ids.extend(([row["id"] for row in rows]))
+            num_test_ids += len(rows)
+        else:
+            train_ids.extend([row["id"] for row in rows])
+
+    assert len(test_ids) == test_size
+    assert len(train_ids) == len(train_ancestors_df) - test_size
+
+    return train_ids, test_ids
+
+
+def prepare(raw: Path, public: Path, private: Path):
+    train_ancestors_df = read_csv(raw / "train_ancestors.csv")
+    # Shuffle the train_ancestors_df to ensure our split is random
+    train_ancestors_df = train_ancestors_df.sample(frac=1, random_state=0).reset_index(drop=True)
+    new_train_ids, new_test_ids = create_train_test_split(train_ancestors_df, test_size=20000)
+
+    # Copy json files to public
+    (public / "train").mkdir(parents=True, exist_ok=True)
+    for train_id in tqdm(new_train_ids, desc="Copying train json files"):
+        shutil.copy(raw / "train" / f"{train_id}.json", public / "train" / f"{train_id}.json")
+    (public / "test").mkdir(parents=True, exist_ok=True)
+    for test_id in tqdm(new_test_ids, desc="Copying test json files"):
+        shutil.copy(raw / "train" / f"{test_id}.json", public / "test" / f"{test_id}.json")
+
+    # Generate answers for train and test
+    train_orders = read_csv(raw / "train_orders.csv")
+    # Answers for new train
+    train_orders_new = train_orders[train_orders["id"].isin(new_train_ids)]
+    train_orders_new.to_csv(public / "train_orders.csv", index=False)
+    # Answers for new test
+    test_orders_new = train_orders[train_orders["id"].isin(new_test_ids)]
+    test_orders_new.to_csv(private / "test_orders.csv", index=False)
+
+    # Make new train_ancestors.csv, excluding the new_test_ids
+    train_ancestors_df = train_ancestors_df[~train_ancestors_df["id"].isin(new_test_ids)]
+    train_ancestors_df.to_csv(public / "train_ancestors.csv", index=False)
+
+    # Create sample submission (use the given order without changing it)
+    sample_submission_rows = []
+    for sample_id in tqdm(test_orders_new["id"], desc="Creating sample submission"):
+        # Get cell order from json file
+        with open(public / "test" / f"{sample_id}.json") as f:
+            json_data = json.load(f)
+            cell_order = list(json_data["cell_type"].keys())
+        sample_submission_rows.append({"id": sample_id, "cell_order": " ".join(cell_order)})
+    sample_submission = pd.DataFrame(sample_submission_rows)
+    sample_submission.to_csv(public / "sample_submission.csv", index=False)
+    assert len(sample_submission) == len(
+        new_test_ids
+    ), "Sample submission length does not match test length."

dslighting/registry/AI4Code/prepare_val.py

@@ -0,0 +1,159 @@
+import json
+import shutil
+from pathlib import Path
+
+import pandas as pd
+from tqdm.auto import tqdm
+
+from mlebench.utils import read_csv
+
+
+# Create train, test from train split
+def create_train_test_split(train_ancestors_df: pd.DataFrame, test_size: int):
+    """
+    On Kaggle, a user may "fork" (that is, copy) the notebook of another user to create their own version.
+    `train_ancestors_df` contains information about the ancestry of each notebook in the training set.
+
+    To create the test split, we select rows from the train set that don't share ancestors with any others
+    so that there aren't any relatives in the train set. The train split is the remaining rows.
+    """
+    group_by_ancestor = {}
+    for _, row in tqdm(
+        train_ancestors_df.iterrows(), desc="Grouping by Ancestor", total=len(train_ancestors_df)
+    ):
+        ancestor_id = row["ancestor_id"]
+        if ancestor_id not in group_by_ancestor:
+            group_by_ancestor[ancestor_id] = []
+        group_by_ancestor[ancestor_id].append(row)
+
+    train_ids = []
+    test_ids = []
+    num_test_ids = 0
+    # Ensure a deterministic split by sorting the groups
+    sorted_ancestors = sorted(group_by_ancestor.items(), key=lambda x: x[0])
+    for ancestor_id, rows in tqdm(sorted_ancestors, desc="Splitting on Ancestors"):
+        if num_test_ids + len(rows) <= test_size:
+            test_ids.extend(([row["id"] for row in rows]))
+            num_test_ids += len(rows)
+        else:
+            train_ids.extend([row["id"] for row in rows])
+
+    # The exact number of test samples can vary slightly based on group sizes.
+    # We adjust the assertion to check it's close to the target, not necessarily equal.
+    assert len(train_ids) + len(test_ids) == len(train_ancestors_df)
+
+    return train_ids, test_ids
+
+
+def _generate_split_files(
+    train_ids: list,
+    test_ids: list,
+    public_dir: Path,
+    private_dir: Path,
+    raw_dir: Path,
+    full_train_orders_df: pd.DataFrame,
+    full_train_ancestors_df: pd.DataFrame,
+):
+    """
+    A helper function to generate all required files for a given train/test split.
+    This helps avoid code duplication and ensures consistent output structure.
+    """
+    # Create output directories
+    public_dir.mkdir(parents=True, exist_ok=True)
+    private_dir.mkdir(parents=True, exist_ok=True)
+    (public_dir / "train").mkdir(exist_ok=True)
+    (public_dir / "test").mkdir(exist_ok=True)
+
+    # Copy json files to public
+    for train_id in tqdm(train_ids, desc=f"Copying train json files to {public_dir.name}"):
+        shutil.copy(raw_dir / "train" / f"{train_id}.json", public_dir / "train" / f"{train_id}.json")
+    for test_id in tqdm(test_ids, desc=f"Copying test json files to {public_dir.name}"):
+        shutil.copy(raw_dir / "train" / f"{test_id}.json", public_dir / "test" / f"{test_id}.json")
+
+    # Generate answers for train and test
+    # Answers for new train
+    train_orders_new = full_train_orders_df[full_train_orders_df["id"].isin(train_ids)]
+    train_orders_new.to_csv(public_dir / "train_orders.csv", index=False)
+    # Answers for new test
+    test_orders_new = full_train_orders_df[full_train_orders_df["id"].isin(test_ids)]
+    test_orders_new.to_csv(private_dir / "test_orders.csv", index=False)
+
+    # Make new train_ancestors.csv, excluding the new_test_ids
+    train_ancestors_new = full_train_ancestors_df[
+        full_train_ancestors_df["id"].isin(train_ids)
+    ]
+    train_ancestors_new.to_csv(public_dir / "train_ancestors.csv", index=False)
+
+    # Create sample submission (use the given order without changing it)
+    sample_submission_rows = []
+    for sample_id in tqdm(
+        test_orders_new["id"], desc=f"Creating sample submission for {public_dir.name}"
+    ):
+        # Get cell order from json file
+        with open(public_dir / "test" / f"{sample_id}.json") as f:
+            json_data = json.load(f)
+            cell_order = list(json_data["cell_type"].keys())
+        sample_submission_rows.append({"id": sample_id, "cell_order": " ".join(cell_order)})
+    sample_submission = pd.DataFrame(sample_submission_rows)
+    sample_submission.to_csv(public_dir / "sample_submission.csv", index=False)
+    assert len(sample_submission) == len(test_ids), "Sample submission length does not match test length."
+
+
+def prepare(raw: Path, public: Path, private: Path):
+    # --- Initial Setup ---
+    TEST_SIZE = 20000
+    # Load all necessary data once
+    full_train_ancestors_df = read_csv(raw / "train_ancestors.csv")
+    full_train_orders_df = read_csv(raw / "train_orders.csv")
+
+    # Shuffle the train_ancestors_df to ensure our split is random but reproducible
+    shuffled_ancestors_df = full_train_ancestors_df.sample(
+        frac=1, random_state=0
+    ).reset_index(drop=True)
+
+    # --- 1. Original Split (train/test) ---
+    print("--- Generating original train/test split ---")
+    train_ids, test_ids = create_train_test_split(shuffled_ancestors_df, test_size=TEST_SIZE)
+
+    # Generate files for the original public and private directories
+    _generate_split_files(
+        train_ids,
+        test_ids,
+        public,
+        private,
+        raw,
+        full_train_orders_df,
+        shuffled_ancestors_df,
+    )
+
+    # --- 2. Validation Split (train_val/test_val) ---
+    print("\n--- Generating new validation split ---")
+    # Define paths for the new validation directories
+    public_val = public.parent / "public_val"
+    private_val = private.parent / "private_val"
+
+    # The input for the second split is the training set from the first split.
+    # We use the already shuffled dataframe to maintain deterministic behavior.
+    train_subset_for_val_split = shuffled_ancestors_df[
+        shuffled_ancestors_df["id"].isin(train_ids)
+    ].reset_index(drop=True)
+
+    # Create the second split from the first split's training data
+    train_val_ids, test_val_ids = create_train_test_split(
+        train_subset_for_val_split, test_size=TEST_SIZE
+    )
+
+    # Generate the validation split files in the new directories
+    _generate_split_files(
+        train_val_ids,
+        test_val_ids,
+        public_val,
+        private_val,
+        raw,
+        full_train_orders_df,
+        shuffled_ancestors_df,
+    )
+
+    print("\nData preparation complete.")
+    print(f"Original split created in '{public.name}' and '{private.name}'.")
+    print(f"Validation split created in '{public_val.name}' and '{private_val.name}'.")