pytorch-optimizer 4.0.0__tar.gz → 4.1.0__tar.gz

pytorch_optimizer-4.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,37 @@
+---
+name: Bug report
+about: Report a bug with steps to reproduce it
+title: ''
+labels: bug
+assignees: kozistr
+
+---
+
+## Describe the bug
+
+Describe the bug and its effect on your training or other use of the package.
+
+## Reproduce the bug
+
+- Operating system: Linux, Windows, or macOS, including the version.
+- PyTorch version:
+- Python version:
+- pytorch-optimizer version:
+- Code or script that reproduces the bug:
+
+Include the steps to run your code.
+If you cannot share the original code, write a minimal example that reproduces the bug.
+See this [example report](https://github.com/kozistr/pytorch_optimizer/issues/305#issue-2721453417).
+
+## Log
+
+Attach the complete log.
+You can highlight relevant text or include screenshots.
+
+## Expected behavior
+
+Describe the behavior you expected.
+
+## Additional context
+
+Add context that helps reproduce or explain the problem.

pytorch_optimizer-4.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,16 @@
+---
+name: Feature request
+about: Request a feature or an algorithm
+title: ''
+labels: feature request
+assignees: kozistr
+
+---
+
+## Feature
+
+Describe the feature and how you would use it.
+
+## Paper or code
+
+Link to the paper or reference implementation, if available.

pytorch_optimizer-4.1.0/.github/pull_request_template.md

@@ -0,0 +1,24 @@
+## Problem
+
+Describe the problem this change addresses.
+
+## Solution
+
+Describe the resulting behavior and how the implementation produces it.
+Include a paper or reference implementation for a new algorithm.
+
+## Other changes
+
+List related fixes or refactors that reviewers should check.
+
+## Notes
+
+Add context, questions, or checks you could not run.
+
+## Checklist
+
+- [ ] Run `uv run just format` before committing.
+- [ ] Run `uv run just check` and resolve errors.
+- [ ] Run `uv run just test` and check that new and existing tests pass.
+- [ ] Check 100% coverage for new and changed implementation code with `uv run coverage report -m`.
+- [ ] Update the documentation for the change.

pytorch_optimizer-4.1.0/.github/workflows/ci.yml

@@ -0,0 +1,138 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+  workflow_dispatch:
+  workflow_call:
+    inputs:
+      ref:
+        description: Commit to validate
+        type: string
+        required: false
+        default: ''
+      skip_coverage_upload:
+        description: Skip coverage uploads during release validation
+        type: boolean
+        required: false
+        default: false
+
+env:
+  OMP_NUM_THREADS: 8
+  MKL_NUM_THREADS: 8
+  PYTHONDONTWRITEBYTECODE: 1
+
+jobs:
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ inputs.ref || github.sha }}
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+        with:
+          python-version: 3.12
+          enable-cache: true
+          save-cache: ${{ github.ref == 'refs/heads/main' }}
+          prune-cache: false
+
+      - name: Install dependencies
+        run: uv sync --frozen --no-editable
+
+      - name: Run lint
+        run: |
+          uv run ruff check pytorch_optimizer examples tests scripts hubconf.py
+
+  type_check:
+    name: Static Type Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ inputs.ref || github.sha }}
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+        with:
+          python-version: 3.12
+          enable-cache: true
+          save-cache: ${{ github.ref == 'refs/heads/main' }}
+          prune-cache: false
+
+      - name: Install dependencies
+        run: uv sync --frozen --no-editable
+
+      - name: Run type check
+        run: uv run --frozen ty check
+
+  docs:
+    name: Documentation
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ inputs.ref || github.sha }}
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+        with:
+          python-version: 3.12
+          enable-cache: true
+          save-cache: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Build documentation
+        run: uv run --no-project --python 3.12 --with-requirements docs/requirements-docs.txt zensical build --strict
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.10', '3.11', '3.12', '3.13', '3.14', '3.15']
+
+    steps:
+    - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      with:
+        ref: ${{ inputs.ref || github.sha }}
+
+    - name: Setup uv
+      uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+      with:
+        python-version: ${{ matrix.python-version }}
+        enable-cache: true
+        save-cache: ${{ github.ref == 'refs/heads/main' }}
+        prune-cache: false
+
+    - name: Install dependencies
+      run: uv sync --frozen --no-editable
+
+    - name: Check test
+      run: uv run --frozen pytest
+
+    - name: Upload coverage
+      if: success() && !inputs.skip_coverage_upload
+      uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1
+      with:
+        token: ${{ secrets.CODECOV_TOKEN }}
+        directory: ./
+        files: ./coverage.xml
+        env_vars: OS,PYTHON
+        flags: ${{ matrix.python-version }}
+        fail_ci_if_error: true
+        verbose: false
+
+  ci-summary:
+    if: always()
+    needs: [lint, type_check, docs, test]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check results
+        if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
+        run: exit 1
+
+      - name: Report Summary
+        run: echo "✅ Lint, documentation, and tests completed for all targets"

pytorch_optimizer-4.1.0/.github/workflows/publish.yml

@@ -0,0 +1,233 @@
+name: Release
+run-name: Release / ${{ github.ref_name }}
+
+on:
+  push:
+    tags:
+      - 'v[0-9]+.[0-9]+.[0-9]+'
+  workflow_dispatch:
+    inputs:
+      bump:
+        description: Version component to increment
+        required: true
+        type: choice
+        default: patch
+        options: [major, minor, patch]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: publish
+  cancel-in-progress: false
+
+jobs:
+  guard:
+    name: Guard Release Ref
+    runs-on: ubuntu-latest
+    steps:
+      - name: Require main for manual releases
+        run: |
+          if [[ "$GITHUB_EVENT_NAME" == 'workflow_dispatch' && "$GITHUB_REF" != 'refs/heads/main' ]]; then
+            echo "Run manual releases from main, not $GITHUB_REF" >&2
+            exit 1
+          fi
+
+  validate:
+    name: Validate
+    needs: guard
+    uses: ./.github/workflows/ci.yml
+    with:
+      ref: ${{ github.sha }}
+      skip_coverage_upload: true
+
+  prepare:
+    name: Prepare Release
+    needs: validate
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+    outputs:
+      tag: ${{ steps.version.outputs.tag }}
+      ref: ${{ steps.commit.outputs.ref }}
+      notes: ${{ steps.notes.outputs.body }}
+    steps:
+      - name: Checkout release ref
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ github.sha }}
+          fetch-depth: 0
+          token: ${{ github.token }}
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+        with:
+          python-version: '3.12'
+          save-cache: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Validate and bump package version
+        id: version
+        env:
+          BUMP: ${{ inputs.bump }}
+        run: |
+          uv lock --check
+          current="$(uv version --short)"
+          if [[ ! "$current" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+            echo 'Package version must have the form MAJOR.MINOR.PATCH' >&2
+            exit 1
+          fi
+
+          if [[ "$GITHUB_EVENT_NAME" == 'workflow_dispatch' ]]; then
+            version="$(uv version --bump "$BUMP" --no-sync --short)"
+            tag="v$version"
+            if git rev-parse --verify --quiet "refs/tags/$tag"; then
+              echo "Tag $tag already exists" >&2
+              exit 1
+            fi
+          else
+            tag="$GITHUB_REF_NAME"
+            if [[ "$tag" != "v$current" ]]; then
+              echo "Tag $tag does not match package version $current" >&2
+              exit 1
+            fi
+          fi
+          echo "tag=$tag" >> "$GITHUB_OUTPUT"
+
+      - name: Generate release notes and update changelogs
+        id: notes
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GH_REPO: ${{ github.repository }}
+          RELEASE_TAG: ${{ steps.version.outputs.tag }}
+        run: |
+          gh api --method POST "repos/$GH_REPO/releases/generate-notes" \
+            -f tag_name="$RELEASE_TAG" -f target_commitish="$GITHUB_SHA" \
+            --jq .body > "$RUNNER_TEMP/release-notes.md"
+          uv run --no-project python scripts/update_changelog.py "$RELEASE_TAG" "$RUNNER_TEMP/release-notes.md"
+
+          delimiter="release_notes_${RANDOM}_${RANDOM}_${RANDOM}"
+          {
+            echo "body<<$delimiter"
+            cat "$RUNNER_TEMP/release-notes.md"
+            echo "$delimiter"
+          } >> "$GITHUB_OUTPUT"
+
+      - name: Create release preparation pull request
+        id: release_pr
+        uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
+        with:
+          branch: release/prepare-${{ steps.version.outputs.tag }}
+          commit-message: 'build: prepare release ${{ steps.version.outputs.tag }}'
+          title: 'build: prepare release ${{ steps.version.outputs.tag }}'
+          body: >-
+            Prepare the package version, lockfile, and generated changelogs for ${{ steps.version.outputs.tag }}.
+            This pull request must merge before package publishing and GitHub release creation.
+          base: main
+          add-paths: |
+            pyproject.toml
+            uv.lock
+            CHANGELOG.md
+            docs/changelogs/${{ steps.version.outputs.tag }}.md
+            docs/changelogs/index.md
+          delete-branch: true
+          token: ${{ github.token }}
+
+      - name: Merge release preparation pull request and resolve release commit
+        id: commit
+        env:
+          GH_TOKEN: ${{ github.token }}
+          GH_REPO: ${{ github.repository }}
+          PR_NUMBER: ${{ steps.release_pr.outputs.pull-request-number }}
+          PR_HEAD_SHA: ${{ steps.release_pr.outputs.pull-request-head-sha }}
+        run: |
+          if [[ -z "$PR_NUMBER" || -z "$PR_HEAD_SHA" ]]; then
+            echo 'No release preparation pull request was created' >&2
+            exit 1
+          fi
+          for attempt in {1..10}; do
+            mergeable="$(gh pr view "$PR_NUMBER" --json mergeable --jq .mergeable)"
+            if [[ "$mergeable" != 'UNKNOWN' ]]; then
+              break
+            fi
+            sleep 3
+          done
+          if [[ "$mergeable" == 'UNKNOWN' ]]; then
+            echo 'GitHub has not finished calculating PR mergeability' >&2
+            exit 1
+          fi
+          gh pr merge "$PR_NUMBER" --squash --delete-branch --match-head-commit "$PR_HEAD_SHA"
+          merged_ref="$(gh pr view "$PR_NUMBER" --json state,mergeCommit \
+            --jq 'if .state == "MERGED" then .mergeCommit.oid else empty end')"
+          if [[ -z "$merged_ref" ]]; then
+            echo 'The release preparation pull request must be merged before publishing' >&2
+            exit 1
+          fi
+          if [[ "$GITHUB_EVENT_NAME" == 'workflow_dispatch' ]]; then
+            ref="$merged_ref"
+          else
+            ref="$GITHUB_SHA"
+          fi
+          echo "ref=$ref" >> "$GITHUB_OUTPUT"
+
+  deploy:
+    name: Publish Package
+    needs: prepare
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      id-token: write
+    steps:
+      - name: Checkout release commit
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          ref: ${{ needs.prepare.outputs.ref }}
+          fetch-depth: 0
+
+      - name: Setup uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+        with:
+          python-version: '3.12'
+          save-cache: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Build package
+        run: uv build
+
+      - name: Create and push release tag
+        if: github.event_name == 'workflow_dispatch'
+        env:
+          RELEASE_TAG: ${{ needs.prepare.outputs.tag }}
+          RELEASE_REF: ${{ needs.prepare.outputs.ref }}
+        run: |
+          if git rev-parse --verify --quiet "refs/tags/$RELEASE_TAG"; then
+            if [[ "$(git rev-list -n 1 "$RELEASE_TAG")" != "$RELEASE_REF" ]]; then
+              echo "Tag $RELEASE_TAG points to a different commit" >&2
+              exit 1
+            fi
+          else
+            git config user.name 'github-actions[bot]'
+            git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
+            git tag -a "$RELEASE_TAG" -m "pytorch-optimizer $RELEASE_TAG"
+            git push origin "refs/tags/$RELEASE_TAG"
+          fi
+
+      - name: Publish package to PyPI
+        run: uv publish --trusted-publishing always --check-url https://pypi.org/simple
+
+  release:
+    name: Create Release
+    runs-on: ubuntu-latest
+    needs: [prepare, deploy]
+    permissions:
+      contents: write
+    steps:
+      - name: Create release with the prepared notes
+        uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3
+        with:
+          body: ${{ needs.prepare.outputs.notes }}
+          draft: false
+          prerelease: false
+          name: pytorch-optimizer ${{ needs.prepare.outputs.tag }}
+          tag_name: ${{ needs.prepare.outputs.tag }}
+          target_commitish: ${{ needs.prepare.outputs.ref }}
+          token: ${{ github.token }}

pytorch_optimizer-4.1.0/AGENTS.md

@@ -0,0 +1,30 @@
+# Agent Instructions
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidance.
+
+## Workflow
+
+- Use `uv`; prefer `uv run just <recipe>`.
+- After code changes, run `uv run just format`, `uv run just check`, and `uv run just test`.
+- Require 100% coverage for changed implementation code; check `uv run coverage report -m`.
+
+## Implementation
+
+- Follow existing code and typing style. Keep comments and abstractions minimal, and reuse helpers.
+- Implement optimizer formulas from the original papers using `BaseOptimizer`, `init_group()`, and `step()`.
+- Register and export new components; add one baseline training recipe for each new optimizer.
+- Do not add implementation details or experiment results to documentation.
+- Leave changelog updates to release automation.
+
+## Testing
+
+- Run existing tests and inspect coverage before adding cases or recipes.
+- Add only for uncovered implementation or demonstrated regressions; explain the gap and prefer extending existing cases.
+- Reuse shared fixtures, runners, and capability checks; avoid duplicate tests.
+- Test observable results with deterministic inputs. Benchmark speed and report test results and coverage.
+
+## Commits and PRs
+
+- Keep commits focused and commit subjects and PR titles concise, specific, and imperative.
+- Use `feat:`, `fix:`, `perf:`, `style:`, `refactor:`, `docs:`, `chore:`, `build:`, or `update:` for commit subjects.
+- Check the branch and working tree before pushing; use `--force-with-lease` for pushed history rewrites.