barbor 1.0.0__tar.gz

barbor-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jing Lin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

barbor-1.0.0/PKG-INFO

@@ -0,0 +1,56 @@
+Metadata-Version: 2.1
+Name: barbor
+Version: 1.0.0
+Summary: The gradient optimization library with barzilar borwein method.
+Home-page: https://github.com/linjing-lab/barbor
+Download-URL: https://github.com/linjing-lab/barbor/tags
+Author: 林景
+Author-email: linjing010729@163.com
+License: MIT
+Project-URL: Source, https://github.com/linjing-lab/barbor/tree/main/barbor/
+Project-URL: Tracker, https://github.com/linjing-lab/barbor/issues
+Platform: UNKNOWN
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# barbor
+
+The gradient optimization library with barzilar borwein method.
+
+## description
+
+This PyTorch implementation of the Barzilai-Borwein (BB) gradient descent optimizer represents a sophisticated advancement beyond standard first-order optimization methods. The core innovation lies in its adaptive step size computation that approximates second-order curvature information without explicit Hessian calculation, addressing a fundamental limitation of fixed-learning-rate gradient descent.
+
+The implementation introduces two complementary step size strategies: BB1 (α = s·s/s·y) and BB2 (α = s·y/y·y), where s represents parameter changes and y represents gradient differences between iterations. These formulas effectively capture local curvature, enabling the optimizer to automatically adjust step sizes based on problem geometry. The default alternating strategy intelligently switches between these variants, leveraging their complementary strengths—BB1 tends to be more stable while BB2 can achieve faster convergence.
+
+A key innovation is the adaptive restart mechanism that prevents divergence in non-convex landscapes. The code implements three restart conditions: gradient orthogonality (when s and y become nearly orthogonal), negative gradient correlation (when consecutive gradients point in opposite directions), or a combined approach. This system allows the optimizer to reset to initial learning rates when progress stalls, effectively escaping regions of poor curvature.
+
+The implementation also integrates momentum support (both standard and Nesterov variants) with the BB framework, creating a hybrid approach that combines momentum's acceleration with BB's curvature awareness. Comprehensive numerical safeguards—including regularization parameters, step size clamping, and division-by-zero protection—ensure robustness across diverse optimization landscapes.
+
+Beyond the core algorithm, the optimizer provides extensive diagnostic tools for monitoring convergence behavior, including real-time step size tracking, gradient correlation metrics, and convergence statistics. This transparency allows users to understand the adaptive behavior and make informed adjustments.
+
+The combination of curvature-aware step sizing, intelligent restart conditions, momentum integration, and robust numerical handling makes this implementation particularly valuable for non-convex optimization problems where traditional methods struggle with learning rate selection and convergence stability.
+
+## install barbor
+
+```python
+pip install torch==2.1.0 --index-url https://download.pytorch.org/whl/cu121
+pip install barbor
+```
+

barbor-1.0.0/README.md

@@ -0,0 +1,24 @@
+# barbor
+
+The gradient optimization library with barzilar borwein method.
+
+## description
+
+This PyTorch implementation of the Barzilai-Borwein (BB) gradient descent optimizer represents a sophisticated advancement beyond standard first-order optimization methods. The core innovation lies in its adaptive step size computation that approximates second-order curvature information without explicit Hessian calculation, addressing a fundamental limitation of fixed-learning-rate gradient descent.
+
+The implementation introduces two complementary step size strategies: BB1 (α = s·s/s·y) and BB2 (α = s·y/y·y), where s represents parameter changes and y represents gradient differences between iterations. These formulas effectively capture local curvature, enabling the optimizer to automatically adjust step sizes based on problem geometry. The default alternating strategy intelligently switches between these variants, leveraging their complementary strengths—BB1 tends to be more stable while BB2 can achieve faster convergence.
+
+A key innovation is the adaptive restart mechanism that prevents divergence in non-convex landscapes. The code implements three restart conditions: gradient orthogonality (when s and y become nearly orthogonal), negative gradient correlation (when consecutive gradients point in opposite directions), or a combined approach. This system allows the optimizer to reset to initial learning rates when progress stalls, effectively escaping regions of poor curvature.
+
+The implementation also integrates momentum support (both standard and Nesterov variants) with the BB framework, creating a hybrid approach that combines momentum's acceleration with BB's curvature awareness. Comprehensive numerical safeguards—including regularization parameters, step size clamping, and division-by-zero protection—ensure robustness across diverse optimization landscapes.
+
+Beyond the core algorithm, the optimizer provides extensive diagnostic tools for monitoring convergence behavior, including real-time step size tracking, gradient correlation metrics, and convergence statistics. This transparency allows users to understand the adaptive behavior and make informed adjustments.
+
+The combination of curvature-aware step sizing, intelligent restart conditions, momentum integration, and robust numerical handling makes this implementation particularly valuable for non-convex optimization problems where traditional methods struggle with learning rate selection and convergence stability.
+
+## install barbor
+
+```python
+pip install torch==2.1.0 --index-url https://download.pytorch.org/whl/cu121
+pip install barbor
+```

barbor-1.0.0/barbor/__init__.py

@@ -0,0 +1,2 @@
+from .optimizer import barbor
+__version__ = '1.0.0'

barbor-1.0.0/barbor/momentum.py

@@ -0,0 +1,36 @@
+import torch
+
+def apply_momentum(
+    param: torch.Tensor,
+    grad: torch.Tensor,
+    alpha: torch.Tensor,
+    state: dict,
+    momentum: float,
+    dampening: float,
+    nesterov: bool
+):
+    """Apply momentum to parameter update
+    
+    Args:
+        param: Parameter tensor to update
+        grad: Gradient of parameter
+        alpha: Step size
+        state: Optimizer state for the parameter
+        momentum: Momentum factor
+        dampening: Momentum dampening factor
+        nesterov: Whether to use Nesterov momentum
+    """
+    if 'momentum_buffer' not in state:
+        state['momentum_buffer'] = torch.zeros_like(param)
+    
+    buf = state['momentum_buffer']
+    
+    if nesterov:
+        # Nesterov momentum
+        grad_corrected = grad.add(buf, alpha=momentum)
+        param.data.add_(grad_corrected, alpha=-alpha.item() if torch.is_tensor(alpha) else -alpha)
+        buf.mul_(momentum).add_(grad, alpha=1 - dampening)
+    else:
+        # Standard momentum
+        buf.mul_(momentum).add_(grad, alpha=1 - dampening)
+        param.data.add_(buf, alpha=-alpha.item() if torch.is_tensor(alpha) else -alpha)

barbor-1.0.0/barbor/optimizer.py

@@ -0,0 +1,242 @@
+import torch
+from typing import Optional, Callable, Dict, List, Tuple
+from .stepsize import compute_step_size
+from .restart import check_restart_condition
+from .momentum import apply_momentum
+from .utils import validate_arguments, compute_dot_products
+
+class barbor(torch.optim.Optimizer):
+    """Barzilar-Borwein Gradient Descent Method
+    
+    Args:
+        params: Parameters to optimize
+        lr: Initial learning rate (default: 1.0)
+        method: Step size calculation method, options: 'bb1', 'bb2', 'alternating' (default: 'alternating')
+        gamma: Regularization parameter to prevent zero step size (default: 1e-8)
+        safe_guard: Step size safety guard factor (default: 1e-8)
+        min_step: Minimum step size (default: 1e-8)
+        max_step: Maximum step size (default: 1e8)
+        adaptive_restart: Whether to use adaptive restart (default: True)
+        restart_condition: Restart condition, options: 'gradient', 'angle', 'both' (default: 'both')
+        restart_tol: Restart condition tolerance (default: 0.9)
+        momentum: Momentum parameter (default: 0.0)
+        dampening: Momentum dampening (default: 0.0)
+        nesterov: Whether to use Nesterov momentum (default: False)
+    """
+    
+    def __init__(
+        self,
+        params,
+        lr: float = 1.0,
+        method: str = 'alternating',
+        gamma: float = 1e-8,
+        safe_guard: float = 1e-8,
+        min_step: float = 1e-8,
+        max_step: float = 1e8,
+        adaptive_restart: bool = True,
+        restart_condition: str = 'both',
+        restart_tol: float = 0.9,
+        momentum: float = 0.0,
+        dampening: float = 0.0,
+        nesterov: bool = False
+    ):
+        # Validate input arguments
+        validate_arguments(lr, method, restart_condition, momentum, dampening)
+        
+        defaults = dict(
+            lr=lr,
+            method=method,
+            gamma=gamma,
+            safe_guard=safe_guard,
+            min_step=min_step,
+            max_step=max_step,
+            adaptive_restart=adaptive_restart,
+            restart_condition=restart_condition,
+            restart_tol=restart_tol,
+            momentum=momentum,
+            dampening=dampening,
+            nesterov=nesterov
+        )
+        super().__init__(params, defaults)
+        
+        # Initialize state for each parameter group
+        self._initialize_states()
+    
+    def _initialize_states(self):
+        """Initialize optimizer states for all parameters"""
+        for group in self.param_groups:
+            for p in group['params']:
+                self._initialize_param_state(p, group['lr'], group['momentum'])
+    
+    def _initialize_param_state(self, p, lr: float, momentum: float):
+        """Initialize state for a single parameter"""
+        state = self.state[p]
+        state.setdefault('step', 0)
+        state.setdefault('prev_param', torch.zeros_like(p))
+        state.setdefault('prev_grad', torch.zeros_like(p))
+        state.setdefault('alpha', torch.tensor(lr, device=p.device))
+        state.setdefault('prev_alpha', torch.tensor(lr, device=p.device))
+        
+        if momentum > 0:
+            state.setdefault('momentum_buffer', torch.zeros_like(p))
+    
+    @torch.no_grad()
+    def step(self, closure: Optional[Callable[[], float]] = None):
+        """Perform a single optimization step
+        
+        Args:
+            closure: A callable that recomputes the loss and returns the loss
+            
+        Returns:
+            Loss value (if closure is provided)
+        """
+        loss = None
+        if closure is not None:
+            with torch.enable_grad():
+                loss = closure()
+        
+        for group in self.param_groups:
+            for p in group['params']:
+                if p.grad is None:
+                    continue
+                
+                self._update_parameter(p, group)
+        
+        return loss
+    
+    def _update_parameter(self, p, group: dict):
+        """Update a single parameter"""
+        grad = p.grad
+        if grad.is_sparse:
+            raise RuntimeError('BarzilaiBorwein does not support sparse gradients')
+        
+        state = self.state[p]
+        step = state['step']
+        
+        if step == 0:
+            # First step: use initial learning rate
+            new_alpha = torch.tensor(group['lr'], device=p.device)
+            restart = False
+        else:
+            # Compute new step size
+            s, y = self._compute_updates(p, state)
+            restart = self._should_restart(s, y, grad, state, group)
+            new_alpha = self._compute_new_step_size(s, y, state, group, restart)
+        
+        # Update parameter with new step size
+        self._apply_update(p, grad, state, new_alpha, group)
+        
+        # Save state for next iteration
+        self._save_state(p, grad, state, new_alpha)
+        state['step'] += 1
+    
+    def _compute_updates(self, p, state: dict) -> Tuple[torch.Tensor, torch.Tensor]:
+        """Compute parameter and gradient updates"""
+        s = p.data - state['prev_param']
+        y = p.grad - state['prev_grad']
+        return s, y
+    
+    def _should_restart(self, s: torch.Tensor, y: torch.Tensor, 
+                       grad: torch.Tensor, state: dict, group: dict) -> bool:
+        """Check if restart condition is met"""
+        if not group['adaptive_restart'] or state['step'] <= 1:
+            return False
+        
+        return check_restart_condition(
+            s, y, grad, state['prev_grad'],
+            group['restart_condition'], group['restart_tol']
+        )
+    
+    def _compute_new_step_size(self, s: torch.Tensor, y: torch.Tensor, 
+                              state: dict, group: dict, restart: bool) -> torch.Tensor:
+        """Compute new step size"""
+        if restart:
+            return torch.tensor(group['lr'], device=s.device)
+        
+        s_dot_s, s_dot_y, y_dot_y = compute_dot_products(s, y)
+        
+        new_alpha = compute_step_size(
+            s_dot_s, s_dot_y, y_dot_y,
+            state['step'], group['method'],
+            group['gamma'], group['safe_guard'],
+            s.device
+        )
+        
+        # Clip step size
+        return torch.clamp(new_alpha, group['min_step'], group['max_step'])
+    
+    def _apply_update(self, p, grad: torch.Tensor, state: dict, 
+                     alpha: torch.Tensor, group: dict):
+        """Apply parameter update"""
+        state['prev_alpha'] = state['alpha']
+        state['alpha'] = alpha
+        
+        if group['momentum'] > 0:
+            apply_momentum(
+                p, grad, alpha, state, 
+                group['momentum'], group['dampening'], group['nesterov']
+            )
+        else:
+            p.data.add_(grad, alpha=-alpha)
+    
+    def _save_state(self, p, grad: torch.Tensor, state: dict, alpha: torch.Tensor):
+        """Save current state for next iteration"""
+        state['prev_param'].copy_(p.data)
+        state['prev_grad'].copy_(grad)
+    
+    def get_step_sizes(self) -> List[float]:
+        """Get current step sizes for all parameters"""
+        alphas = []
+        for group in self.param_groups:
+            for p in group['params']:
+                state = self.state[p]
+                if 'alpha' in state:
+                    alpha = state['alpha']
+                    alphas.append(alpha.item() if torch.is_tensor(alpha) else alpha)
+        return alphas
+    
+    def reset_step_sizes(self, alpha: float = 1.0):
+        """Reset step sizes for all parameters"""
+        for group in self.param_groups:
+            for p in group['params']:
+                state = self.state[p]
+                device = p.device
+                state['alpha'] = torch.tensor(alpha, device=device)
+                state['prev_alpha'] = torch.tensor(alpha, device=device)
+    
+    def get_gradient_history_info(self) -> List[Tuple[float, float, float]]:
+        """Get gradient history information
+        
+        Returns:
+            List of (s·s, s·y, y·y) values for each parameter
+        """
+        info = []
+        for group in self.param_groups:
+            for p in group['params']:
+                state = self.state[p]
+                if 'prev_grad' in state and p.grad is not None:
+                    s, y = self._compute_updates(p, state)
+                    s_dot_s, s_dot_y, y_dot_y = compute_dot_products(s, y)
+                    info.append((s_dot_s.item(), s_dot_y.item(), y_dot_y.item()))
+        return info
+    
+    def get_convergence_info(self) -> Dict[str, List[float]]:
+        """Get convergence information"""
+        info = {
+            'step_sizes': [],
+            'gradient_norms': [],
+            'step_norms': []
+        }
+        for group in self.param_groups:
+            for p in group['params']:
+                state = self.state[p]
+                if 'alpha' in state:
+                    alpha = state['alpha']
+                    info['step_sizes'].append(alpha.item() if torch.is_tensor(alpha) else alpha)
+                if p.grad is not None:
+                    grad_norm = torch.norm(p.grad).item()
+                    info['gradient_norms'].append(grad_norm)
+                if 'prev_param' in state:
+                    step_norm = torch.norm(p.data - state['prev_param']).item()
+                    info['step_norms'].append(step_norm)
+        return info

barbor-1.0.0/barbor/restart.py

@@ -0,0 +1,90 @@
+import torch
+from typing import Union
+from enum import Enum
+
+class RestartCondition(Enum):
+    GRADIENT = 'gradient'
+    ANGLE = 'angle'
+    BOTH = 'both'
+
+def check_restart_condition(
+    s: torch.Tensor,
+    y: torch.Tensor,
+    grad: torch.Tensor,
+    prev_grad: torch.Tensor,
+    condition: Union[str, RestartCondition],
+    tol: float = 0.9
+) -> bool:
+    """Check if restart condition is met
+    
+    Restart conditions help prevent BB method from diverging on non-convex problems
+    
+    Args:
+        s: Parameter difference (x_k - x_{k-1})
+        y: Gradient difference (∇f_k - ∇f_{k-1})
+        grad: Current gradient
+        prev_grad: Previous gradient
+        condition: Restart condition type
+        tol: Tolerance for restart condition
+        
+    Returns:
+        True if restart condition is met
+    """
+    if isinstance(condition, str):
+        condition = RestartCondition(condition.lower())
+    
+    if condition == RestartCondition.GRADIENT:
+        return _check_gradient_condition(s, y, tol)
+    elif condition == RestartCondition.ANGLE:
+        return _check_angle_condition(grad, prev_grad, tol)
+    elif condition == RestartCondition.BOTH:
+        return _check_both_conditions(s, y, grad, prev_grad, tol)
+    else:
+        raise ValueError(f"Unsupported restart condition: {condition}")
+
+def _check_gradient_condition(s: torch.Tensor, y: torch.Tensor, tol: float) -> bool:
+    """Check gradient-based restart condition"""
+    s_norm = torch.norm(s)
+    y_norm = torch.norm(y)
+    
+    if s_norm < 1e-12 or y_norm < 1e-12:
+        return False
+    
+    cos_theta = torch.abs(torch.sum(s * y)) / (s_norm * y_norm)
+    return cos_theta < tol
+
+def _check_angle_condition(grad: torch.Tensor, prev_grad: torch.Tensor, tol: float) -> bool:
+    """Check angle-based restart condition"""
+    grad_norm = torch.norm(grad)
+    prev_grad_norm = torch.norm(prev_grad)
+    
+    if grad_norm < 1e-12 or prev_grad_norm < 1e-12:
+        return False
+    
+    cos_phi = torch.sum(grad * prev_grad) / (grad_norm * prev_grad_norm)
+    return cos_phi < -tol
+
+def _check_both_conditions(
+    s: torch.Tensor,
+    y: torch.Tensor,
+    grad: torch.Tensor,
+    prev_grad: torch.Tensor,
+    tol: float
+) -> bool:
+    """Check both restart conditions"""
+    restart1, restart2 = False, False
+    
+    s_norm = torch.norm(s)
+    y_norm = torch.norm(y)
+    grad_norm = torch.norm(grad)
+    prev_grad_norm = torch.norm(prev_grad)
+    
+    if s_norm > 1e-12 and y_norm > 1e-12:
+        cos_theta = torch.abs(torch.sum(s * y)) / (s_norm * y_norm)
+        restart1 = cos_theta < tol
+    
+    if grad_norm > 1e-12 and prev_grad_norm > 1e-12:
+        cos_phi = torch.sum(grad * prev_grad) / (grad_norm * prev_grad_norm)
+        restart2 = cos_phi < -tol
+    
+    return restart1 or restart2

barbor-1.0.0/barbor/stepsize.py

@@ -0,0 +1,89 @@
+import torch
+from typing import Union
+from enum import Enum
+
+class StepSizeMethod(Enum):
+    BB1 = 'bb1'
+    BB2 = 'bb2'
+    ALTERNATING = 'alternating'
+
+def compute_step_size(
+    s_dot_s: torch.Tensor,
+    s_dot_y: torch.Tensor,
+    y_dot_y: torch.Tensor,
+    step: int,
+    method: Union[str, StepSizeMethod],
+    gamma: float = 1e-8,
+    safe_guard: float = 1e-8,
+    device: torch.device = None
+) -> torch.Tensor:
+    """Compute step size using Barzilai-Borwein method
+    
+    Args:
+        s_dot_s: s·s where s = x_k - x_{k-1}
+        s_dot_y: s·y where y = ∇f_k - ∇f_{k-1}
+        y_dot_y: y·y
+        step: Current iteration number
+        method: Step size calculation method
+        gamma: Regularization parameter
+        safe_guard: Numerical safety parameter
+        device: Device for tensor creation
+        
+    Returns:
+        Computed step size
+    """
+    if isinstance(method, str):
+        method = StepSizeMethod(method.lower())
+    
+    if method == StepSizeMethod.BB1:
+        return _bb1_step(s_dot_s, s_dot_y, gamma, safe_guard, device)
+    elif method == StepSizeMethod.BB2:
+        return _bb2_step(s_dot_y, y_dot_y, gamma, safe_guard, device)
+    elif method == StepSizeMethod.ALTERNATING:
+        # Alternate between BB1 and BB2
+        if step % 2 == 1:
+            return _bb1_step(s_dot_s, s_dot_y, gamma, safe_guard, device)
+        else:
+            return _bb2_step(s_dot_y, y_dot_y, gamma, safe_guard, device)
+    else:
+        raise ValueError(f"Unsupported step size calculation method: {method}")
+
+def _bb1_step(
+    s_dot_s: torch.Tensor,
+    s_dot_y: torch.Tensor,
+    gamma: float,
+    safe_guard: float,
+    device: torch.device
+) -> torch.Tensor:
+    """Compute BB1 step size"""
+    if torch.abs(s_dot_y) < safe_guard:
+        if torch.abs(s_dot_s) < safe_guard:
+            return torch.tensor(1.0, device=device)
+        else:
+            return s_dot_s.clone()
+    
+    denominator = s_dot_y + gamma
+    if denominator <= 0:
+        denominator = torch.abs(denominator) + gamma
+    
+    return s_dot_s / denominator
+
+def _bb2_step(
+    s_dot_y: torch.Tensor,
+    y_dot_y: torch.Tensor,
+    gamma: float,
+    safe_guard: float,
+    device: torch.device
+) -> torch.Tensor:
+    """Compute BB2 step size"""
+    if torch.abs(y_dot_y) < safe_guard:
+        if torch.abs(s_dot_y) < safe_guard:
+            return torch.tensor(1.0, device=device)
+        else:
+            return s_dot_y.clone()
+    
+    denominator = y_dot_y + gamma
+    if denominator <= 0:
+        denominator = torch.abs(denominator) + gamma
+    
+    return s_dot_y / denominator

barbor-1.0.0/barbor/utils.py

@@ -0,0 +1,42 @@
+import torch
+
+def validate_arguments(
+    lr: float,
+    method: str,
+    restart_condition: str,
+    momentum: float,
+    dampening: float
+):
+    """Validate optimizer arguments
+    
+    Args:
+        lr: Learning rate
+        method: Step size method
+        restart_condition: Restart condition
+        momentum: Momentum parameter
+        dampening: Dampening parameter
+        
+    Raises:
+        ValueError: If any argument is invalid
+    """
+    if lr <= 0.0:
+        raise ValueError(f"Learning rate must be positive: {lr}")
+    valid_methods = ['bb1', 'bb2', 'alternating']
+    if method not in valid_methods:
+        raise ValueError(f"Unsupported step size calculation method: {method}. "
+                        f"Must be one of {valid_methods}")
+    valid_restart_conditions = ['gradient', 'angle', 'both']
+    if restart_condition not in valid_restart_conditions:
+        raise ValueError(f"Unsupported restart condition: {restart_condition}. "
+                        f"Must be one of {valid_restart_conditions}")
+    if momentum < 0.0:
+        raise ValueError(f"Momentum must be non-negative: {momentum}")
+    if dampening < 0.0:
+        raise ValueError(f"Dampening must be non-negative: {dampening}")
+
+def compute_dot_products(s: torch.Tensor, y: torch.Tensor) -> tuple:
+    """Compute dot products for BB step size calculation"""
+    s_dot_s = torch.sum(s * s)
+    s_dot_y = torch.sum(s * y)
+    y_dot_y = torch.sum(y * y)
+    return s_dot_s, s_dot_y, y_dot_y

barbor-1.0.0/barbor.egg-info/PKG-INFO

@@ -0,0 +1,56 @@
+Metadata-Version: 2.1
+Name: barbor
+Version: 1.0.0
+Summary: The gradient optimization library with barzilar borwein method.
+Home-page: https://github.com/linjing-lab/barbor
+Download-URL: https://github.com/linjing-lab/barbor/tags
+Author: 林景
+Author-email: linjing010729@163.com
+License: MIT
+Project-URL: Source, https://github.com/linjing-lab/barbor/tree/main/barbor/
+Project-URL: Tracker, https://github.com/linjing-lab/barbor/issues
+Platform: UNKNOWN
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# barbor
+
+The gradient optimization library with barzilar borwein method.
+
+## description
+
+This PyTorch implementation of the Barzilai-Borwein (BB) gradient descent optimizer represents a sophisticated advancement beyond standard first-order optimization methods. The core innovation lies in its adaptive step size computation that approximates second-order curvature information without explicit Hessian calculation, addressing a fundamental limitation of fixed-learning-rate gradient descent.
+
+The implementation introduces two complementary step size strategies: BB1 (α = s·s/s·y) and BB2 (α = s·y/y·y), where s represents parameter changes and y represents gradient differences between iterations. These formulas effectively capture local curvature, enabling the optimizer to automatically adjust step sizes based on problem geometry. The default alternating strategy intelligently switches between these variants, leveraging their complementary strengths—BB1 tends to be more stable while BB2 can achieve faster convergence.
+
+A key innovation is the adaptive restart mechanism that prevents divergence in non-convex landscapes. The code implements three restart conditions: gradient orthogonality (when s and y become nearly orthogonal), negative gradient correlation (when consecutive gradients point in opposite directions), or a combined approach. This system allows the optimizer to reset to initial learning rates when progress stalls, effectively escaping regions of poor curvature.
+
+The implementation also integrates momentum support (both standard and Nesterov variants) with the BB framework, creating a hybrid approach that combines momentum's acceleration with BB's curvature awareness. Comprehensive numerical safeguards—including regularization parameters, step size clamping, and division-by-zero protection—ensure robustness across diverse optimization landscapes.
+
+Beyond the core algorithm, the optimizer provides extensive diagnostic tools for monitoring convergence behavior, including real-time step size tracking, gradient correlation metrics, and convergence statistics. This transparency allows users to understand the adaptive behavior and make informed adjustments.
+
+The combination of curvature-aware step sizing, intelligent restart conditions, momentum integration, and robust numerical handling makes this implementation particularly valuable for non-convex optimization problems where traditional methods struggle with learning rate selection and convergence stability.
+
+## install barbor
+
+```python
+pip install torch==2.1.0 --index-url https://download.pytorch.org/whl/cu121
+pip install barbor
+```
+

barbor-1.0.0/barbor.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE
+README.md
+setup.py
+barbor/__init__.py
+barbor/momentum.py
+barbor/optimizer.py
+barbor/restart.py
+barbor/stepsize.py
+barbor/utils.py
+barbor.egg-info/PKG-INFO
+barbor.egg-info/SOURCES.txt
+barbor.egg-info/dependency_links.txt
+barbor.egg-info/not-zip-safe
+barbor.egg-info/top_level.txt

barbor-1.0.0/barbor.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

barbor-1.0.0/barbor.egg-info/not-zip-safe

@@ -0,0 +1 @@
+

barbor-1.0.0/barbor.egg-info/top_level.txt

@@ -0,0 +1 @@
+barbor

barbor-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

barbor-1.0.0/setup.py

@@ -0,0 +1,47 @@
+from setuptools import setup
+from barbor import __version__
+
+try:
+    with open('README.md', 'r', encoding='utf-8') as fp:
+        _long_description = fp.read()
+except FileNotFoundError:
+    _long_description = ''
+
+setup(
+      name='barbor',  # pkg_name
+      packages=['barbor',],
+      version=__version__,  # version number
+      description="The gradient optimization library with barzilar borwein method.",
+      author='林景',
+      author_email='linjing010729@163.com',
+      license='MIT',
+      url='https://github.com/linjing-lab/barbor',
+      download_url='https://github.com/linjing-lab/barbor/tags',
+      long_description=_long_description,
+      long_description_content_type='text/markdown',
+      include_package_data=True,
+      zip_safe=False,
+      setup_requires=['setuptools>=18.0', 'wheel'],
+      project_urls={
+            'Source': 'https://github.com/linjing-lab/barbor/tree/main/barbor/',
+            'Tracker': 'https://github.com/linjing-lab/barbor/issues',
+      },
+      classifiers=[
+            'Development Status :: 5 - Production/Stable',
+            'Intended Audience :: Developers',
+            'Intended Audience :: Information Technology',
+            'Intended Audience :: Science/Research',
+            'Programming Language :: Python :: 3.8',
+            'Programming Language :: Python :: 3.9',
+            'Programming Language :: Python :: 3.10',
+            'Programming Language :: Python :: 3.11',
+            'Programming Language :: Python :: 3.12',
+            'License :: OSI Approved :: MIT License',
+            'Topic :: Scientific/Engineering',
+            'Topic :: Scientific/Engineering :: Mathematics',
+            'Topic :: Scientific/Engineering :: Artificial Intelligence',
+            'Topic :: Software Development',
+            'Topic :: Software Development :: Libraries',
+            'Topic :: Software Development :: Libraries :: Python Modules',
+      ],
+)