liger-kernel-nightly 0.5.2.dev20241211231633__tar.gz → 0.5.2.dev20241212000548__tar.gz

liger_kernel_nightly-0.5.2.dev20241212000548/.flake8

@@ -0,0 +1,10 @@
+# .flake8
+[flake8]
+max-line-length = 120
+exclude = 
+    .git,
+    __pycache__,
+    benchmark_internal/others,
+    .venv
+# E203: https://github.com/psf/black/issues/315
+extend-ignore=E501,B006,E731,A002,E203

liger_kernel_nightly-0.5.2.dev20241212000548/.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -0,0 +1,48 @@
+name: 🐛 Bug Report
+description: Create a report to help us reproduce and fix the bug
+
+body:
+- type: markdown
+  attributes:
+    value: >
+      #### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/linkedin/Liger-Kernel/issues).
+- type: textarea
+  attributes:
+    label: 🐛 Describe the bug
+    description: |
+      Please provide a clear and concise description of what the bug is.
+    placeholder: |
+      A clear and concise description of what the bug is.
+  validations:
+    required: true
+
+- type: textarea
+  attributes:
+    label: Reproduce
+    description: |
+      If applicable, add a minimal example so that we can reproduce the error by running the code.
+      The snippet needs to be as succinct (minimal) as possible, so please take time to trim down any irrelevant code to help us debug efficiently.
+      We are going to copy-paste your code and we expect to get the same result as you did: avoid any external data, and include the relevant imports, etc.
+      If the code is too long (hopefully, it isn't), feel free to put it in a public gist and link it in the issue: https://gist.github.com.
+      Please also paste or describe the results you observe instead of the expected results.
+      If you observe an error, please paste the error message including the **full** traceback of the exception.
+  validations:
+    required: false
+
+- type: textarea
+  attributes:
+    label: Versions
+    description: |
+      Please provide triton, torch, hardware, and other necessary versions to reproduce the bug.
+
+      For convenience, you can run the following command to get the versions of important software dependencies:
+      ```bash
+      python -m liger_kernel.env_report
+      ```
+  validations:
+    required: true
+
+- type: markdown
+  attributes:
+    value: >
+      Thanks for contributing 🎉!

liger_kernel_nightly-0.5.2.dev20241212000548/.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -0,0 +1,25 @@
+name: 🚀 Feature request
+description: Submit a proposal/request for a new Liger feature
+
+body:
+- type: textarea
+  attributes:
+    label: 🚀 The feature, motivation and pitch
+    description: >
+      A clear and concise description of the feature proposal. Please outline the motivation for the proposal. Is your feature request related to a specific problem? e.g., *"I'm working on X and would like Y to be possible"*. If this is related to another GitHub issue, please link here too.
+  validations:
+    required: true
+- type: textarea
+  attributes:
+    label: Alternatives
+    description: >
+      A description of any alternative solutions or features you've considered, if any.
+- type: textarea
+  attributes:
+    label: Additional context
+    description: >
+      Add any other context or screenshots about the feature request.
+- type: markdown
+  attributes:
+    value: >
+      Thanks for contributing 🎉!

liger_kernel_nightly-0.5.2.dev20241212000548/.github/pull_request_template.md

@@ -0,0 +1,22 @@
+## Summary
+<!--- This is a required section; please describe the main purpose of this proposed code change. --->
+
+<!---
+## Details
+This is an optional section; is there anything specific that reviewers should be aware of?
+--->
+
+## Testing Done
+<!--- This is a required section; please describe how this change was tested. --->
+
+<!-- 
+Replace BLANK with your device type. For example, A100-80G-PCIe
+
+Complete the following tasks before sending your PR, and replace `[ ]` with
+`[x]` to indicate you have done them. 
+-->
+
+- Hardware Type: <BLANK>
+- [ ] run `make test` to ensure correctness
+- [ ] run `make checkstyle` to ensure code style
+- [ ] run `make test-convergence` to ensure convergence

liger_kernel_nightly-0.5.2.dev20241212000548/.github/workflows/amd-ci.yml

@@ -0,0 +1,71 @@
+name: AMD GPU
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - "src/**"
+      - "test/**"
+  pull_request:
+    branches:
+      - main
+    paths:
+      - "src/**"
+      - "test/**"
+  schedule:
+    # Runs at 00:00 UTC daily
+    - cron: '0 0 * * *'
+  workflow_dispatch:  # Enables manual trigger
+
+concurrency:
+  # This causes it to cancel previous in-progress actions on the same PR / branch,
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  checkstyle:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install --no-deps .[fmt]
+
+    - name: Run checkstyle
+      run: make checkstyle
+
+  tests:
+    runs-on: linux-mi300-gpu-1
+    needs: [checkstyle]
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Setup Dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -e .[dev] --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2
+    
+    - name: List Python Environments
+      run: python -m pip list
+    
+    - name: Run Unit Tests
+      run: |
+        make test
+        make test-convergence

liger_kernel_nightly-0.5.2.dev20241212000548/.github/workflows/nvi-ci.yml

@@ -0,0 +1,96 @@
+name: NVIDIA GPU
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - "src/**"
+      - "test/**"
+  pull_request:
+    branches:
+      - main
+    paths:
+      - "src/**"
+      - "test/**"
+  schedule:
+    # Runs at 00:00 UTC daily
+    - cron: '0 0 * * *'
+  workflow_dispatch:  # Enables manual trigger
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  checkstyle:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install --no-deps .[fmt]
+
+    - name: Run checkstyle
+      run: make checkstyle
+
+  tests:
+    runs-on: ubuntu-latest
+    needs: [checkstyle]
+    env:
+      MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
+      MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}
+      REBUILD_IMAGE: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install modal
+
+    - name: Run tests
+      run: |
+        modal run dev.modal.tests
+
+  tests-bwd:
+    runs-on: ubuntu-latest
+    needs: [checkstyle]
+    env:
+      MODAL_TOKEN_ID: ${{ secrets.MODAL_TOKEN_ID }}
+      MODAL_TOKEN_SECRET: ${{ secrets.MODAL_TOKEN_SECRET }}
+      REBUILD_IMAGE: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install modal
+
+    - name: Run tests
+      run: |
+        modal run dev.modal.tests_bwd

liger_kernel_nightly-0.5.2.dev20241212000548/.github/workflows/publish-nightly.yml

@@ -0,0 +1,49 @@
+name: Publish Liger Kernel Nightly
+
+# Though it is name "nightly", we will trigger this workflow on push to the main branch for convenience.
+
+on:
+  push:
+    branches:
+      - main  # Trigger on push to the main branch
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.8'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install build twine wheel toml
+
+    - name: Update package name and version
+      run: |
+        VERSION=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['version'])")
+        DATE=$(date +%Y%m%d%H%M%S)
+        NEW_VERSION="$VERSION.dev$DATE"
+        sed -i "s/name = \"liger_kernel\"/name = \"liger_kernel_nightly\"/" pyproject.toml
+        sed -i "s/version = \"$VERSION\"/version = \"$NEW_VERSION\"/" pyproject.toml
+
+    - name: Build package
+      run: |
+        python -m build
+
+    - name: Publish package to PyPI
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_NIGHTLY_PASSWORD }}
+      run: |
+        twine upload dist/*
+
+    - name: Create release notes
+      run: |
+        echo "Nightly build published to PyPI with the name 'liger-kernel-nightly'."

liger_kernel_nightly-0.5.2.dev20241212000548/.github/workflows/publish-release.yml

@@ -0,0 +1,38 @@
+name: Publish Liger Kernel on Release
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v3
+      with:
+        python-version: '3.10'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install build twine wheel toml
+
+    - name: Build package
+      run: |
+        python -m build
+
+    - name: Publish package to PyPI
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      run: |
+        twine upload dist/*
+
+    - name: Create release notes
+      run: |
+        echo "Release published to PyPI with the name 'liger-kernel'."

liger_kernel_nightly-0.5.2.dev20241212000548/.gitignore

@@ -0,0 +1,20 @@
+__pycache__/
+*.egg-info/
+site/
+.cache/
+.venv/
+venv/
+.ipynb_checkpoints/
+
+# Misc
+.DS_Store
+
+# Build
+build/
+dist/
+
+# Lockfiles
+uv.lock
+
+# Benchmark images
+benchmark/visualizations

liger_kernel_nightly-0.5.2.dev20241212000548/.isort.cfg

@@ -0,0 +1,2 @@
+[settings]
+profile = black

liger_kernel_nightly-0.5.2.dev20241212000548/Makefile

@@ -0,0 +1,42 @@
+.PHONY: test checkstyle test-convergence all
+
+
+all: checkstyle test test-convergence
+
+# Command to run pytest for correctness tests
+test:
+	python -m pytest --disable-warnings test/ --ignore=test/convergence
+
+# Command to run flake8 (code style check), isort (import ordering), and black (code formatting)
+# Subsequent commands still run if the previous fails, but return failure at the end
+checkstyle:
+	flake8 .; flake8_status=$$?; \
+	isort .; isort_status=$$?; \
+	black .; black_status=$$?; \
+	if [ $$flake8_status -ne 0 ] || [ $$isort_status -ne 0 ] || [ $$black_status -ne 0 ]; then \
+		exit 1; \
+	fi
+
+# Command to run pytest for convergence tests
+# We have to explicitly set HF_DATASETS_OFFLINE=1, or dataset will silently try to send metrics and timeout (80s) https://github.com/huggingface/datasets/blob/37a603679f451826cfafd8aae00738b01dcb9d58/src/datasets/load.py#L286
+test-convergence:
+	HF_DATASETS_OFFLINE=1 python -m pytest --disable-warnings test/convergence/test_mini_models.py
+	HF_DATASETS_OFFLINE=1 python -m pytest --disable-warnings test/convergence/test_mini_models_multimodal.py
+	HF_DATASETS_OFFLINE=1 python -m pytest --disable-warnings test/convergence/test_mini_models_with_logits.py
+
+# Command to run all benchmark scripts and update benchmarking data file
+# By default this doesn't overwrite existing data for the same benchmark experiment
+# run with `make run-benchmarks OVERWRITE=1` to overwrite existing benchmark data
+BENCHMARK_DIR = benchmark/scripts
+BENCHMARK_SCRIPTS = $(wildcard $(BENCHMARK_DIR)/benchmark_*.py)
+OVERWRITE ?= 0
+
+run-benchmarks:
+	@for script in $(BENCHMARK_SCRIPTS); do \
+		echo "Running benchmark: $$script"; \
+		if [ $(OVERWRITE) -eq 1 ]; then \
+			python $$script --overwrite; \
+		else \
+			python $$script; \
+		fi; \
+	done

{liger_kernel_nightly-0.5.2.dev20241211231633/src/liger_kernel_nightly.egg-info → liger_kernel_nightly-0.5.2.dev20241212000548}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: liger_kernel_nightly
-Version: 0.5.2.dev20241211231633
+Version: 0.5.2.dev20241212000548
 Summary: Efficient Triton kernels for LLM Training
 License: BSD 2-CLAUSE LICENSE
         Copyright 2024 LinkedIn Corporation 
@@ -32,10 +32,6 @@ License-File: LICENSE
 License-File: NOTICE
 Requires-Dist: torch>=2.1.2
 Requires-Dist: triton>=2.3.1
-Provides-Extra: transformers
-Requires-Dist: transformers~=4.0; extra == "transformers"
-Provides-Extra: trl
-Requires-Dist: trl>=0.11.0; extra == "trl"
 Provides-Extra: dev
 Requires-Dist: transformers>=4.44.2; extra == "dev"
 Requires-Dist: matplotlib>=3.7.2; extra == "dev"
@@ -46,13 +42,11 @@ Requires-Dist: pytest>=7.1.2; extra == "dev"
 Requires-Dist: pytest-xdist; extra == "dev"
 Requires-Dist: pytest-rerunfailures; extra == "dev"
 Requires-Dist: datasets>=2.19.2; extra == "dev"
-Requires-Dist: torchvision>=0.16.2; extra == "dev"
 Requires-Dist: seaborn; extra == "dev"
-Provides-Extra: amd
-Requires-Dist: torch>=2.6.0.dev; extra == "amd"
-Requires-Dist: setuptools-scm>=8; extra == "amd"
-Requires-Dist: torchvision>=0.20.0.dev; extra == "amd"
-Requires-Dist: triton>=3.0.0; extra == "amd"
+Provides-Extra: fmt
+Requires-Dist: flake8; extra == "fmt"
+Requires-Dist: isort; extra == "fmt"
+Requires-Dist: black; extra == "fmt"
 
 <a name="readme-top"></a>
 
@@ -202,11 +196,13 @@ To install from source:
 ```bash
 git clone https://github.com/linkedin/Liger-Kernel.git
 cd Liger-Kernel
+
+# Install Default Dependencies
+# Setup.py will detect whether you are using AMD or NVIDIA
 pip install -e .
-# or if installing on amd platform
-pip install -e .[amd] --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2 # rocm6.2
-# or if using transformers
-pip install -e .[transformers]
+
+# Setup Development Dependencies
+pip install -e ".[dev]"
 ```
 
 

{liger_kernel_nightly-0.5.2.dev20241211231633 → liger_kernel_nightly-0.5.2.dev20241212000548}/README.md

@@ -146,11 +146,13 @@ To install from source:
 ```bash
 git clone https://github.com/linkedin/Liger-Kernel.git
 cd Liger-Kernel
+
+# Install Default Dependencies
+# Setup.py will detect whether you are using AMD or NVIDIA
 pip install -e .
-# or if installing on amd platform
-pip install -e .[amd] --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2 # rocm6.2
-# or if using transformers
-pip install -e .[transformers]
+
+# Setup Development Dependencies
+pip install -e ".[dev]"
 ```
 
 

liger_kernel_nightly-0.5.2.dev20241212000548/benchmark/benchmarks_visualizer.py

@@ -0,0 +1,169 @@
+import json
+import os
+from argparse import ArgumentParser
+from dataclasses import dataclass
+
+import matplotlib.pyplot as plt
+import pandas as pd
+import seaborn as sns
+
+DATA_PATH = "data/all_benchmark_data.csv"
+VISUALIZATIONS_PATH = "visualizations/"
+
+
+@dataclass
+class VisualizationsConfig:
+    """
+    Configuration for the visualizations script.
+
+    Args:
+        kernel_name (str): Kernel name to benchmark. (Will run `scripts/benchmark_{kernel_name}.py`)
+        metric_name (str): Metric name to visualize (speed/memory)
+        kernel_operation_mode (str): Kernel operation mode to visualize (forward/backward/full). Defaults to "full"
+        display (bool): Display the visualization. Defaults to False
+        overwrite (bool): Overwrite existing visualization, if none exist this flag has no effect as ones are always created and saved. Defaults to False
+
+    """
+
+    kernel_name: str
+    metric_name: str
+    kernel_operation_mode: str = "full"
+    display: bool = False
+    overwrite: bool = False
+
+
+def parse_args() -> VisualizationsConfig:
+    """Parse command line arguments into a configuration object.
+
+    Returns:
+        VisualizationsConfig: Configuration object for the visualizations script.
+    """
+    parser = ArgumentParser()
+    parser.add_argument(
+        "--kernel-name", type=str, required=True, help="Kernel name to benchmark"
+    )
+    parser.add_argument(
+        "--metric-name",
+        type=str,
+        required=True,
+        help="Metric name to visualize (speed/memory)",
+    )
+    parser.add_argument(
+        "--kernel-operation-mode",
+        type=str,
+        required=True,
+        help="Kernel operation mode to visualize (forward/backward/full)",
+    )
+    parser.add_argument(
+        "--display", action="store_true", help="Display the visualization"
+    )
+    parser.add_argument(
+        "--overwrite",
+        action="store_true",
+        help="Overwrite existing visualization, if none exist this flag has no effect as one are always created",
+    )
+
+    args = parser.parse_args()
+
+    return VisualizationsConfig(**dict(args._get_kwargs()))
+
+
+def load_data(config: VisualizationsConfig) -> pd.DataFrame:
+    """Loads the benchmark data from the CSV file and filters it based on the configuration.
+
+    Args:
+        config (VisualizationsConfig): Configuration object for the visualizations script.
+
+    Raises:
+        ValueError: If no data is found for the given filters.
+
+    Returns:
+        pd.DataFrame: Filtered benchmark dataframe.
+    """
+    df = pd.read_csv(DATA_PATH)
+    df["extra_benchmark_config"] = df["extra_benchmark_config_str"].apply(json.loads)
+
+    filtered_df = df[
+        (df["kernel_name"] == config.kernel_name)
+        & (df["metric_name"] == config.metric_name)
+        & (df["kernel_operation_mode"] == config.kernel_operation_mode)
+        # Use this to filter by extra benchmark configuration property
+        # & (data['extra_benchmark_config'].apply(lambda x: x.get('H') == 4096))
+        # FIXME: maybe add a way to filter using some configuration, except of hardcoding it
+    ]
+
+    if filtered_df.empty:
+        raise ValueError("No data found for the given filters")
+
+    return filtered_df
+
+
+def plot_data(df: pd.DataFrame, config: VisualizationsConfig):
+    """Plots the benchmark data, saving the result if needed.
+
+    Args:
+        df (pd.DataFrame): Filtered benchmark dataframe.
+        config (VisualizationsConfig): Configuration object for the visualizations script.
+    """
+    xlabel = df["x_label"].iloc[0]
+    ylabel = f"{config.metric_name} ({df['metric_unit'].iloc[0]})"
+    # Sort by "kernel_provider" to ensure consistent color assignment
+    df = df.sort_values(by="kernel_provider")
+
+    plt.figure(figsize=(10, 6))
+    sns.set(style="whitegrid")
+    ax = sns.lineplot(
+        data=df,
+        x="x_value",
+        y="y_value_50",
+        hue="kernel_provider",
+        marker="o",
+        palette="tab10",
+        errorbar=("ci", None),
+    )
+
+    # Seaborn can't plot pre-computed error bars, so we need to do it manually
+    lines = ax.get_lines()
+    colors = [line.get_color() for line in lines]
+
+    for (_, group_data), color in zip(df.groupby("kernel_provider"), colors):
+        # for i, row in group_data.iterrows():
+        y_error_lower = group_data["y_value_50"] - group_data["y_value_20"]
+        y_error_upper = group_data["y_value_80"] - group_data["y_value_50"]
+        y_error = [y_error_lower, y_error_upper]
+
+        plt.errorbar(
+            group_data["x_value"],
+            group_data["y_value_50"],
+            yerr=y_error,
+            fmt="o",
+            color=color,
+            capsize=5,
+        )
+    plt.legend(title="Kernel Provider")
+    plt.xlabel(xlabel)
+    plt.ylabel(ylabel)
+    plt.tight_layout()
+
+    out_path = os.path.join(
+        VISUALIZATIONS_PATH, f"{config.kernel_name}_{config.metric_name}.png"
+    )
+
+    if config.display:
+        plt.show()
+    if config.overwrite or not os.path.exists(
+        out_path
+    ):  # Save the plot if it doesn't exist or if we want to overwrite it
+        os.makedirs(VISUALIZATIONS_PATH, exist_ok=True)
+        plt.savefig(out_path)
+    plt.close()
+
+
+def main():
+    config = parse_args()
+    df = load_data(config)
+    plot_data(df, config)
+
+
+if __name__ == "__main__":
+    main()