socket-lm 0.1.0__py3-none-any.whl

socket_lm-0.1.0.dist-info/METADATA

@@ -0,0 +1,199 @@
+Metadata-Version: 2.4
+Name: socket-lm
+Version: 0.1.0
+Summary: A training-speed-focused LoRA/PEFT library offering four parameter-efficient fine-tuning methods (LoRA, ReFT, LiteLadder, GaLore)
+Author: Omur Bera Isik
+License: MIT License
+        
+        Copyright (c) 2026 Ömür Bera Işık
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Keywords: lora,peft,fine-tuning,pytorch,llm,deep-learning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.0
+Provides-Extra: quantization
+Requires-Dist: bitsandbytes>=0.41.0; extra == "quantization"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: bitsandbytes>=0.41.0; extra == "dev"
+Dynamic: license-file
+
+# Socket
+
+A LoRA/PEFT library focused on training speed. Offers four different parameter-efficient
+fine-tuning methods — all written from scratch with no dependency on external PEFT libraries
+(`peft` etc.).
+
+```bash
+pip install socket
+```
+
+> **Important:** PyPI package name is `socket`, but to avoid clashing with Python's built-in `socket`
+> (networking) module, **the import name is `socketimport`**:
+> ```python
+> import socketimport as sk
+> ```
+
+## Methods
+
+| Method | What it does | Status |
+|---|---|---|
+| **LoRA** (+ rsLoRA, DoRA) | Low-rank addition to weights: `W + (α/r)·BA` | Mature, tested |
+| **ReFT** (LoReFT) | Intervenes in hidden representations, not weights | Mature, tested |
+| **LiteLadder** | Trains a separate "side network" with no backprop to backbone | **Experimental** — validated at small scale, not yet tested with real language data |
+| **GaLore** | Full-parameter training with low-rank gradient projection to reduce optimizer memory | Mature, tested |
+
+When to use each method differs: LoRA/ReFT/LiteLadder reduce parameter count (adapter-based),
+while GaLore allows **full-parameter** training but only reduces optimizer memory — they are not
+interchangeable, but complementary tools.
+
+## Quick Start
+
+### LoRA
+
+```python
+import torch
+from socketimport import LoRAAdapter, LoRAConfig
+
+model = ...  # any nn.Module (e.g., a Llama model)
+config = LoRAConfig(r=16, alpha=32, use_rslora=True, dropout=0.05)
+adapter = LoRAAdapter(model, config)
+
+optimizer = torch.optim.AdamW(adapter.trainable_parameters(), lr=1e-4)
+# ... normal training loop, forward with adapter(x) ...
+
+adapter.merge()              # zero-overhead embedding for inference
+saved = adapter.adapter_state_dict()   # only LoRA weights (KB not MB)
+```
+
+Config parameters accept alternative names
+(`rank`, `lora_r`, `lora_alpha`, `dora`, `rslora` etc.) — if conflicting values
+are provided, an error is raised, not silently chosen.
+
+### ReFT
+
+```python
+from socketimport import ReFTAdapter, ReFTConfig
+
+# `layers` must be provided explicitly - Socket does not attempt to guess model architecture
+adapter = ReFTAdapter(
+    model, layers=model.model.layers, embed_dim=4096,
+    config=ReFTConfig(r=4, layers=(8, 16, 24)),
+)
+optimizer = torch.optim.AdamW(adapter.trainable_parameters(), lr=1e-3)
+```
+
+### LiteLadder (experimental)
+
+```python
+from socketimport import LiteLadderAdapter, LiteLadderConfig
+
+adapter = LiteLadderAdapter(
+    model, layers=model.model.layers, embed_dim=4096, output_dim=32000,
+    config=LiteLadderConfig(side_width=256, rank=32, n_taps=4),
+)
+```
+
+Backward never enters the backbone (showed much lower overhead than LoRA as depth increased in
+small-scale tests) — but this has only been validated on synthetic tasks with a single CPU core.
+**Not recommended** as production default; should be considered an opt-in experimental option.
+
+### GaLore
+
+```python
+from socketimport import GaLoreAdamW, GaLoreConfig, create_galore_param_groups
+
+groups = create_galore_param_groups(model, GaLoreConfig(rank=128, update_proj_gap=200))
+optimizer = GaLoreAdamW(groups, lr=1e-4)
+# Model is trained FULL-PARAMETER - no adapter, no merge, GaLore
+# only reduces optimizer memory usage
+```
+
+`GaLoreAdamW` does **not** implement `SocketAdapterBase` — it is not an adapter,
+but a standard `torch.optim.Optimizer` subclass.
+
+## Architecture
+
+```
+socketimport/
+├── core/
+│   ├── base.py          # SocketAdapterBase - shared interface for LoRA/ReFT/LiteLadder
+│   ├── lora.py
+│   ├── reft.py
+│   ├── lite_ladder.py
+│   └── galore.py
+```
+
+All adapter classes (`LoRAAdapter`, `ReFTAdapter`, `LiteLadderAdapter`)
+share the same `SocketAdapterBase` interface: `trainable_parameters()`,
+`merge()`/`unmerge()`, `adapter_state_dict()`/`load_adapter_state_dict()`.
+This allows the training infrastructure (trainer, distributed, profiler) to be
+written independently of which method is selected.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+444 tests covering all modules: mathematical correctness (e.g., zero-initialization as no-op,
+merge/unmerge being inverses), model freezing behavior, end-to-end training actually reducing loss,
+and save/load round-trips.
+
+> **Note:** `quantize_linear`/`quantize_model` and their tests require `bitsandbytes`
+> (see [Dependencies](#dependencies)). The `dev` extra installs this automatically; if not installed,
+> quantization tests will fail with `DependencyError` (not a library error).
+
+## Dependencies
+
+| Package | Required? | For |
+|---|---|---|
+| `torch>=2.0` | Yes | Entire library |
+| `bitsandbytes>=0.41.0` | No (`pip install socket[quantization]`) | Only `quantize_linear`/`quantize_model` (4-bit/8-bit weight quantization) |
+
+All other features (LoRA, ReFT, LiteLadder, GaLore, checkpointing, OOM recovery,
+VRAM guard, distributed backend selection, etc.) work with `torch` alone.
+
+## Limitations (honestly)
+
+- All tests run on 1 CPU core with small synthetic tasks — no scale testing on GPU
+  or with real language data yet.
+- LiteLadder is not a published method; Socket-specific architecture tested at small scale
+  (combination of LST + ReFT-style lightweight interventions).
+- No multi-seed statistical validation; results are single-seed.
+
+-Note: PyPI does not normally allow this package name; please use -pip install socket-lm- to install it.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

socket_lm-0.1.0.dist-info/RECORD

@@ -0,0 +1,43 @@
+socket_lm-0.1.0.dist-info/licenses/LICENSE,sha256=LZhsxZQFqBiE7rkm3FLsj2_A5npqmqDJAgE2C_YkWbU,1075
+socketimport/__init__.py,sha256=RaLnMdlLImGqagtScneXU6ZsRb5QrulHlPGEb37HABY,4210
+socketimport/config.py,sha256=qjzWuclfZYzRBLUxpLvZSDmwsA9HJ7c6glSJVb-Y9jM,4402
+socketimport/deps.py,sha256=BnVhastO_ylBTLP8V1Y_ryKwORJscW0294XHKo_JT2M,2110
+socketimport/device.py,sha256=jWOEAyuGXg-NCJmWCdB8bIO7ewiWXaQK-Lxopy9uhXs,6009
+socketimport/errors.py,sha256=pUQfu2xb0GO3uwjEJdLtfjDkRx8CgGQxrXQ-NweUj74,10169
+socketimport/utils.py,sha256=GfeLV4ynaDi2aN_hPMtuTROQDMalzzeCnOi1ufEwF-4,2445
+socketimport/core/__init__.py,sha256=MVut1OvGvKVkna2SsvFbqFni1Yegd6kzOjW1O86Y1QY,1447
+socketimport/core/accelerator.py,sha256=ruIR_7ughakBgx_xrPL_JdyTS_Wix5uCOXryS3lTRhE,3293
+socketimport/core/attention.py,sha256=5ve-tC-IMp0F44EX2wqM2C4fJ0zzmT8VI_gqD-4tNOM,1895
+socketimport/core/base.py,sha256=X82OIWAF2DiJi_-xT5bWLrCHC3wGSmo2AcvkOU0TTcI,3205
+socketimport/core/galore.py,sha256=rS8KRPMyoZ2Ga3umTNFEjFY31p-DQRBnkolUlwHtVMc,13262
+socketimport/core/lite_ladder.py,sha256=N50xaBF2QdZq8sjErXAQa8wE3_c9EhhkebmrH0sYBps,14401
+socketimport/core/lora.py,sha256=FACt7Y_NavOwHckhSVQOVBh04b3MHMM0MHqscCn7ueI,17472
+socketimport/core/model_loader.py,sha256=VKtshzBXMSCWcfh4-9Drv4D3uqLz4Btv4-1i7CQU4Dg,3115
+socketimport/core/quantization.py,sha256=CMpfO03LeIxYKiWX6yMDTNooh8BpXIX4X6joE9rCMKs,3493
+socketimport/core/reft.py,sha256=F1r0VnK0Di7w7Jxgs0w3ZkCCQk45KHbhAojqjYbso-w,12832
+socketimport/data/__init__.py,sha256=qvWb4n1wSdETMeGp7Ko1Z23baP6Rp3wp-Xig7Xo6fpY,65
+socketimport/data/loader.py,sha256=Ohd2JjSYlsFXiIip9pM_Sq-WIwS9iVSm_5lxn6MUk8U,2353
+socketimport/distributed/__init__.py,sha256=tn34fRlz9_EhQHSr3IpZfdPqqx0m6i8xWYwHex-Z9fk,5288
+socketimport/memory/__init__.py,sha256=Y2MbehihG8MOb3Rm0K98KQJCv5ZRmnKhOoX6vBR2gvE,451
+socketimport/memory/batch_finder.py,sha256=E9XFjwSQg7CDkZ2jzhv84PqebdVr7hOSq_O_JO5-H80,3640
+socketimport/memory/checkpointing.py,sha256=k2AKmPXj8ybISB0Og6qWeTEhVADXgeucXS4920rn8MY,2632
+socketimport/memory/tracker.py,sha256=Fp8mtom9XWRtCLYvDET-cS0c7CO66In4KxeYRZMdKtM,2954
+socketimport/optim/__init__.py,sha256=Sm9Qy3xsoGgfW10lfy6tZm7_qYZtBShhfamlZRJ00FY,413
+socketimport/optim/ema.py,sha256=9-BgAolCus3CNZqUZHEyvk8KrHxVoWdoFHmz3_J1t2o,2664
+socketimport/optim/fused.py,sha256=MzAIrhu9_TfKPLvqVo843vKH5VIFzkMlvQDyd29_F14,2477
+socketimport/optim/lion.py,sha256=GFSX-E3i1zD4b8O502bnfepHeshJlz5TYMsl2Csz1ks,2471
+socketimport/optim/scheduler.py,sha256=h0s7-_utpOHnu4P70QOqVFZy0ZVJ1g0HUvGmN5ZI8V8,3299
+socketimport/optim/swa.py,sha256=rEf5fBQzwUKuuzXLSb9KBeQcqSqLrlTAGwN418P_NgM,2327
+socketimport/profiler/__init__.py,sha256=Gm1ckYCWWOQCAE8PYJqO_9Lk69UHxtYY0xOkxMK090Q,172
+socketimport/profiler/benchmark.py,sha256=z38SCKEFO__ajPpHNtrHRJLnTHpD9y-ZYd8j5r0k0F0,2795
+socketimport/profiler/trace.py,sha256=Onh8o2J1NvJrbvyaKc5EmdeQSpUDGWb-V3IixPQNatA,1527
+socketimport/training/__init__.py,sha256=d8Vkm54c_SQd6aINlqrTJg5MjY_JEwd32y0Gmx8WMAM,792
+socketimport/training/callbacks.py,sha256=vlLUgbqwL2Hj1gEeHsIOY4lcSPbKa_7U8o2MngFVMPA,3751
+socketimport/training/framework_adapter.py,sha256=Q3vB5aJm6Naknkq7LJGjZACH_9SWvrBARGNOx84xNH4,3418
+socketimport/training/oom_recovery.py,sha256=J_Huj_HzLtGwn5Aj4y7O26tt47m5YJCa6CxzLmcnNek,3793
+socketimport/training/trainer.py,sha256=86_ATYVvuKCN6tEK4SroCHIT1WmfgyiARxluMOs0MVQ,7986
+socketimport/training/vram_guard.py,sha256=aWUWmfLQDujdvrl_b0rzo8hGONV5T89Z5iRMItvPlJI,5108
+socket_lm-0.1.0.dist-info/METADATA,sha256=LyOqqESP-uL0wFVisxZRfKAUhQtBcCE09cxxUxT6_Gw,8000
+socket_lm-0.1.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+socket_lm-0.1.0.dist-info/top_level.txt,sha256=gHAgDHBRkVnTlTvqydTTm5CP39pDgD9D_W23fIfDzPM,13
+socket_lm-0.1.0.dist-info/RECORD,,

socket_lm-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

socket_lm-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ömür Bera Işık
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

socket_lm-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+socketimport

socketimport/__init__.py

@@ -0,0 +1,161 @@
+"""Socket - egitim hizina odaklanan bir LoRA/PEFT kutuphanesi.
+
+PyPI paket adi 'socket'tir ama Python'un yerlesik `socket` (ag) modulu ile
+karismamasi icin import adi 'socketimport'tur:
+
+    pip install socket
+    import socketimport as sk
+"""
+
+from .core.accelerator import Accelerator
+from .core.attention import is_flash_attention_active, optimized_attention
+from .core.base import SocketAdapterBase
+from .core.galore import GaLoreAdamW, GaLoreConfig, GaLoreProjector, create_galore_param_groups
+from .core.lite_ladder import LiteLadderAdapter, LiteLadderConfig, LiteLadderSide
+from .core.lora import DEFAULT_TARGET_MODULES, LoRAAdapter, LoRAConfig, LoRALinear
+from .core.model_loader import ModelLoadConfig, ModelLoadError, SafeModelLoader
+from .core.quantization import QuantizationConfig, quantize_linear, quantize_model
+from .core.reft import LoReftIntervention, ReFTAdapter, ReFTConfig
+from .config import SocketConfig
+from .deps import get_version, is_available, require, require_any
+from .device import DeviceManager, DeviceType
+from .errors import (
+    ConfigurationError,
+    DependencyError,
+    ErrorReport,
+    ErrorSeverity,
+    GradientMonitor,
+    RateLimitedLogger,
+    SocketError,
+    auto_install,
+    classify_error,
+)
+from .memory import (
+    MemorySnapshot,
+    MemoryTracker,
+    apply_activation_checkpointing,
+    binary_search_max,
+    find_max_batch_size,
+    is_checkpointed,
+    remove_activation_checkpointing,
+)
+from .data import PrefetchLoader
+from .distributed import DistributedConfig, select_backend
+from .optim import (
+    EMA,
+    SWA,
+    Lion,
+    convert_to_fused,
+    get_constant_schedule_with_warmup,
+    get_warmup_cosine_schedule,
+    get_warmup_linear_schedule,
+)
+from .profiler import BenchmarkResult, benchmark, compare
+from .profiler import summarize as profiler_summarize
+from .profiler import trace as profiler_trace
+from .training import (
+    EarlyStopping,
+    FrameworkInfo,
+    LossSpikeDetector,
+    OOMRecovery,
+    OOMRecoveryConfig,
+    OOMRecoveryExhausted,
+    RecoveryPlan,
+    Trainer,
+    TrainerConfig,
+    VRAMGuard,
+    VRAMGuardConfig,
+    VRAMStatus,
+    detect_framework,
+    optimal_attention_backend,
+    optimal_dtype,
+)
+from .training import framework_summary
+from .utils import check_environment, format_alpaca, format_chatml
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "SocketAdapterBase",
+    "LoRAAdapter",
+    "LoRAConfig",
+    "LoRALinear",
+    "DEFAULT_TARGET_MODULES",
+    "ReFTAdapter",
+    "ReFTConfig",
+    "LoReftIntervention",
+    "LiteLadderAdapter",
+    "LiteLadderConfig",
+    "LiteLadderSide",
+    "GaLoreAdamW",
+    "GaLoreConfig",
+    "GaLoreProjector",
+    "create_galore_param_groups",
+    "SafeModelLoader",
+    "ModelLoadConfig",
+    "ModelLoadError",
+    "QuantizationConfig",
+    "quantize_linear",
+    "quantize_model",
+    "optimized_attention",
+    "is_flash_attention_active",
+    "Accelerator",
+    "DeviceManager",
+    "DeviceType",
+    "SocketError",
+    "ConfigurationError",
+    "DependencyError",
+    "ErrorReport",
+    "ErrorSeverity",
+    "GradientMonitor",
+    "RateLimitedLogger",
+    "classify_error",
+    "auto_install",
+    "is_available",
+    "get_version",
+    "require",
+    "require_any",
+    "MemoryTracker",
+    "MemorySnapshot",
+    "find_max_batch_size",
+    "binary_search_max",
+    "apply_activation_checkpointing",
+    "remove_activation_checkpointing",
+    "is_checkpointed",
+    "VRAMGuard",
+    "VRAMGuardConfig",
+    "VRAMStatus",
+    "OOMRecovery",
+    "OOMRecoveryConfig",
+    "OOMRecoveryExhausted",
+    "RecoveryPlan",
+    "LossSpikeDetector",
+    "EarlyStopping",
+    "Trainer",
+    "TrainerConfig",
+    "FrameworkInfo",
+    "detect_framework",
+    "optimal_dtype",
+    "optimal_attention_backend",
+    "framework_summary",
+    "Lion",
+    "EMA",
+    "SWA",
+    "convert_to_fused",
+    "get_warmup_linear_schedule",
+    "get_warmup_cosine_schedule",
+    "get_constant_schedule_with_warmup",
+    "PrefetchLoader",
+    "DistributedConfig",
+    "select_backend",
+    "benchmark",
+    "compare",
+    "BenchmarkResult",
+    "profiler_trace",
+    "profiler_summarize",
+    "SocketConfig",
+    "format_alpaca",
+    "format_chatml",
+    "check_environment",
+    "__version__",
+]

socketimport/config.py

@@ -0,0 +1,103 @@
+"""
+Socket Config
+==============
+Ust duzey egitim yapilandirmasi - hangi adaptasyon yontemi, temel
+egitim hiperparametreleri. LoRAConfig/ReFTConfig/vb'nin YERINE gecmez -
+onlarla BIRLIKTE kullanilir (method='lora' secildiginde ayrica bir
+LoRAConfig de verilir)."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, fields
+from typing import Any, ClassVar
+
+_VALID_METHODS = ("lora", "reft", "lite_ladder", "galore")
+_VALID_SCHEDULERS = ("linear", "cosine", "constant")
+_VALID_PRECISIONS = ("no", "fp16", "bf16")
+
+
+@dataclass
+class SocketConfig:
+    method: str = "lora"
+    output_dir: str = "./socket_output"
+    num_epochs: int = 1
+    max_steps: int | None = None
+    per_device_batch_size: int = 8
+    gradient_accumulation_steps: int = 1
+    learning_rate: float = 2e-4
+    lr_scheduler: str = "cosine"
+    warmup_ratio: float = 0.03
+    mixed_precision: str = "bf16"
+    max_grad_norm: float = 1.0
+    logging_steps: int = 10
+    save_steps: int = 100
+    seed: int = 42
+
+    _ALIASES: ClassVar[dict[str, str]] = {
+        "adaptation_method": "method",
+        "epochs": "num_epochs",
+        "batch_size": "per_device_batch_size",
+        "per_device_train_batch_size": "per_device_batch_size",
+        "grad_accum_steps": "gradient_accumulation_steps",
+        "lr": "learning_rate",
+        "scheduler": "lr_scheduler",
+        "fp16": "mixed_precision",
+    }
+
+    def __post_init__(self) -> None:
+        if self.method not in _VALID_METHODS:
+            raise ValueError(f"method {_VALID_METHODS} icinde olmali, alinan: {self.method!r}")
+        if self.num_epochs <= 0:
+            raise ValueError(f"num_epochs pozitif olmali, alinan: {self.num_epochs!r}")
+        if self.max_steps is not None and self.max_steps <= 0:
+            raise ValueError(f"max_steps verilirse pozitif olmali, alinan: {self.max_steps!r}")
+        if self.per_device_batch_size <= 0:
+            raise ValueError(
+                f"per_device_batch_size pozitif olmali, alinan: {self.per_device_batch_size!r}"
+            )
+        if self.gradient_accumulation_steps <= 0:
+            raise ValueError(
+                f"gradient_accumulation_steps pozitif olmali, alinan: "
+                f"{self.gradient_accumulation_steps!r}"
+            )
+        if self.learning_rate <= 0:
+            raise ValueError(f"learning_rate pozitif olmali, alinan: {self.learning_rate!r}")
+        if self.lr_scheduler not in _VALID_SCHEDULERS:
+            raise ValueError(
+                f"lr_scheduler {_VALID_SCHEDULERS} icinde olmali, alinan: {self.lr_scheduler!r}"
+            )
+        if not (0.0 <= self.warmup_ratio < 1.0):
+            raise ValueError(f"warmup_ratio [0,1) araliginda olmali, alinan: {self.warmup_ratio!r}")
+        if self.mixed_precision not in _VALID_PRECISIONS:
+            raise ValueError(
+                f"mixed_precision {_VALID_PRECISIONS} icinde olmali, alinan: {self.mixed_precision!r}"
+            )
+        if self.max_grad_norm <= 0:
+            raise ValueError(f"max_grad_norm pozitif olmali, alinan: {self.max_grad_norm!r}")
+        if self.logging_steps <= 0:
+            raise ValueError(f"logging_steps pozitif olmali, alinan: {self.logging_steps!r}")
+        if self.save_steps <= 0:
+            raise ValueError(f"save_steps pozitif olmali, alinan: {self.save_steps!r}")
+
+    def effective_batch_size(self, world_size: int = 1) -> int:
+        """Gradient accumulation ve dagitik egitim (varsa) sonrasi
+        GERCEK etkin batch size'i hesaplar."""
+        return self.per_device_batch_size * self.gradient_accumulation_steps * world_size
+
+    @classmethod
+    def from_kwargs(cls, **kwargs: Any) -> "SocketConfig":
+        resolved: dict[str, Any] = {}
+        for key, value in kwargs.items():
+            canonical = cls._ALIASES.get(key, key)
+            if canonical in resolved and resolved[canonical] != value:
+                raise ValueError(
+                    f"'{key}' parametresi '{canonical}' alanina celisen bir "
+                    f"deger veriyor (once {resolved[canonical]!r} verilmisti, "
+                    f"simdi {value!r})."
+                )
+            resolved[canonical] = value
+        valid_fields = {f.name for f in fields(cls)}
+        unknown = set(resolved) - valid_fields
+        if unknown:
+            raise ValueError(f"Bilinmeyen SocketConfig parametre(ler)i: {sorted(unknown)}")
+        return cls(**resolved)

socketimport/core/__init__.py

@@ -0,0 +1,42 @@
+from .accelerator import Accelerator
+from .attention import is_flash_attention_active, optimized_attention
+from .base import SocketAdapterBase
+from .galore import GaLoreAdamW, GaLoreConfig, GaLoreProjector, create_galore_param_groups
+from .lite_ladder import LiteLadderAdapter, LiteLadderConfig, LiteLadderSide
+from .lora import DEFAULT_TARGET_MODULES, LoRAAdapter, LoRAConfig, LoRALinear
+from .model_loader import ModelLoadConfig, ModelLoadError, SafeModelLoader
+from .quantization import QuantizationConfig, quantize_linear, quantize_model
+from .reft import LoReftIntervention, ReFTAdapter, ReFTConfig
+
+__all__ = [
+    "SocketAdapterBase",
+    # LoRA
+    "LoRAAdapter",
+    "LoRAConfig",
+    "LoRALinear",
+    "DEFAULT_TARGET_MODULES",
+    # ReFT
+    "ReFTAdapter",
+    "ReFTConfig",
+    "LoReftIntervention",
+    # LiteLadder (deneysel - kucuk olcekte dogrulandi, gercek dil
+    # verisiyle henuz test edilmedi, varsayilan olarak onerilmez)
+    "LiteLadderAdapter",
+    "LiteLadderConfig",
+    "LiteLadderSide",
+    # GaLore (SocketAdapterBase DEGIL - bir optimizer)
+    "GaLoreAdamW",
+    "GaLoreConfig",
+    "GaLoreProjector",
+    "create_galore_param_groups",
+    # Model yukleme / quantization / attention / accelerator
+    "SafeModelLoader",
+    "ModelLoadConfig",
+    "ModelLoadError",
+    "QuantizationConfig",
+    "quantize_linear",
+    "quantize_model",
+    "optimized_attention",
+    "is_flash_attention_active",
+    "Accelerator",
+]

socketimport/core/accelerator.py

@@ -0,0 +1,89 @@
+"""
+Socket Core - accelerator
+============================
+Karisik hassasiyet (mixed precision), gradient accumulation ve
+aktivasyon checkpointing'i tek bir arayuzde toplayan Accelerator.
+
+Onceki incelenen bir kutuphanede DEPRECATED `torch.cuda.amp.GradScaler`/
+`torch.cuda.amp.autocast` API'si kullaniliyordu. Burada GUNCEL
+`torch.amp.GradScaler(device_type)` / `torch.autocast(device_type=...)`
+API'si kullanilir - bu ayrica CPU'da da (bf16 VE fp16 icin, bu sandbox'ta
+dogrulandi) GERCEKTEN CALISIR, yani sadece CUDA'da degil, CPU'da da tam
+anlamiyla test edilebilir."""
+
+from __future__ import annotations
+
+import contextlib
+from typing import Any, Iterator
+
+import torch
+import torch.nn as nn
+
+from ..device import DeviceManager
+
+_VALID_PRECISIONS = ("no", "fp16", "bf16")
+
+
+class Accelerator:
+    def __init__(
+        self,
+        mixed_precision: str = "no",
+        gradient_accumulation_steps: int = 1,
+        device_manager: DeviceManager | None = None,
+    ):
+        if mixed_precision not in _VALID_PRECISIONS:
+            raise ValueError(
+                f"mixed_precision {_VALID_PRECISIONS} icinde olmali, alinan: {mixed_precision!r}"
+            )
+        if gradient_accumulation_steps <= 0:
+            raise ValueError(
+                f"gradient_accumulation_steps pozitif olmali, alinan: "
+                f"{gradient_accumulation_steps!r}"
+            )
+
+        self.device_manager = device_manager or DeviceManager()
+        self.mixed_precision = mixed_precision
+        self.gradient_accumulation_steps = gradient_accumulation_steps
+        self._micro_step = 0
+
+        self._autocast_device_type = (
+            self.device_manager.device_type.value
+            if self.device_manager.device_type.value in ("cuda", "cpu")
+            else "cpu"
+        )
+        self.scaler: torch.amp.GradScaler | None = None
+        if mixed_precision == "fp16":
+            self.scaler = torch.amp.GradScaler(self._autocast_device_type)
+
+    def autocast(self) -> contextlib.AbstractContextManager:
+        if self.mixed_precision == "no":
+            return contextlib.nullcontext()
+        dtype = torch.float16 if self.mixed_precision == "fp16" else torch.bfloat16
+        return torch.autocast(device_type=self._autocast_device_type, dtype=dtype)
+
+    def backward(self, loss: torch.Tensor) -> None:
+        """Loss'u gradient_accumulation_steps'e boler ve geri yayilim
+        yapar. Olceklendirme (scaler varsa) burada uygulanir."""
+        scaled_loss = loss / self.gradient_accumulation_steps
+        if self.scaler is not None:
+            self.scaler.scale(scaled_loss).backward()
+        else:
+            scaled_loss.backward()
+
+    def should_step(self) -> bool:
+        """Her mikro-adimda cagrilir; sadece gradient_accumulation_steps'in
+        katinda True doner - o zaman optimizer.step() cagirmak
+        gerektigi anlamina gelir."""
+        self._micro_step += 1
+        return self._micro_step % self.gradient_accumulation_steps == 0
+
+    def step(self, optimizer: torch.optim.Optimizer) -> None:
+        if self.scaler is not None:
+            self.scaler.step(optimizer)
+            self.scaler.update()
+        else:
+            optimizer.step()
+        optimizer.zero_grad(set_to_none=True)
+
+    def reset(self) -> None:
+        self._micro_step = 0