lightning 2.0.7__tar.gz → 2.0.8__tar.gz

{lightning-2.0.7/src/lightning.egg-info → lightning-2.0.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lightning
-Version: 2.0.7
+Version: 2.0.8
 Summary: The Deep Learning framework to train, deploy, and ship AI products Lightning fast.
 Home-page: https://github.com/Lightning-AI/lightning
 Author: Lightning AI et al.
@@ -26,17 +26,17 @@ Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
-Provides-Extra: pytorch-test
-Provides-Extra: pytorch-examples
-Provides-Extra: pytorch-strategies
-Provides-Extra: pytorch-extra
+Provides-Extra: fabric-strategies
 Provides-Extra: fabric-test
 Provides-Extra: fabric-examples
-Provides-Extra: fabric-strategies
+Provides-Extra: app-ui
 Provides-Extra: app-test
-Provides-Extra: app-cloud
 Provides-Extra: app-components
-Provides-Extra: app-ui
+Provides-Extra: app-cloud
+Provides-Extra: pytorch-extra
+Provides-Extra: pytorch-strategies
+Provides-Extra: pytorch-test
+Provides-Extra: pytorch-examples
 Provides-Extra: fabric-all
 Provides-Extra: fabric-dev
 Provides-Extra: pytorch-all
@@ -44,13 +44,13 @@ Provides-Extra: pytorch-dev
 Provides-Extra: app-extra
 Provides-Extra: app-all
 Provides-Extra: app-dev
+Provides-Extra: strategies
 Provides-Extra: test
 Provides-Extra: examples
-Provides-Extra: strategies
-Provides-Extra: extra
-Provides-Extra: cloud
-Provides-Extra: components
 Provides-Extra: ui
+Provides-Extra: components
+Provides-Extra: cloud
+Provides-Extra: extra
 Provides-Extra: all
 Provides-Extra: dev
 License-File: LICENSE
@@ -73,7 +73,7 @@ ______________________________________________________________________
   <a href="https://lightning.ai/docs/pytorch/stable/">PyTorch Lightning</a> •
   <a href="https://lightning.ai/docs/fabric/stable/">Fabric</a> •
   <a href="https://lightning.ai/docs/app/stable/">Lightning Apps</a> •
-  <a href="https://pytorch-lightning.readthedocs.io/en/2.0.7">Docs</a> •
+  <a href="https://pytorch-lightning.readthedocs.io/en/2.0.8">Docs</a> •
   <a href="#community">Community</a> •
   <a href="https://lightning.ai/docs/pytorch/stable/generated/CONTRIBUTING.html">Contribute</a> •
 </p>
@@ -82,9 +82,9 @@ ______________________________________________________________________
 
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pytorch-lightning)](https://pypi.org/project/pytorch-lightning/)
 [![PyPI Status](https://badge.fury.io/py/pytorch-lightning.svg)](https://badge.fury.io/py/pytorch-lightning)
-[![PyPI Status](https://pepy.tech/badge/pytorch-lightning)](https://pepy.tech/project/pytorch-lightning)
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/pytorch-lightning)](https://pepy.tech/project/pytorch-lightning)
 [![Conda](https://img.shields.io/conda/v/conda-forge/lightning?label=conda&color=success)](https://anaconda.org/conda-forge/lightning)
-[![codecov](https://codecov.io/gh/Lightning-AI/lightning/release/2.0.7/graph/badge.svg?token=SmzX8mnKlA)](https://codecov.io/gh/Lightning-AI/lightning)
+[![codecov](https://codecov.io/gh/Lightning-AI/lightning/release/2.0.8/graph/badge.svg?token=SmzX8mnKlA)](https://codecov.io/gh/Lightning-AI/lightning)
 
 [![Discord](https://img.shields.io/discord/1077906959069626439?style=plastic)](https://discord.gg/VptPCZkGNa)
 ![GitHub commit activity](https://img.shields.io/github/commit-activity/w/lightning-ai/lightning)
@@ -487,10 +487,10 @@ Lightning is rigorously tested across multiple CPUs, GPUs, TPUs, IPUs, and HPUs
 
 |       System / PyTorch ver.        |                                                                                              1.11                                                                                               |                                                                                                              1.12                                                                                                               | 1.13                                                                                                                                                                                                                            | 2.0  |
 | :--------------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- |
-|        Linux py3.9 \[GPUs\]        |                                                                                                -                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(GPUs)?branchName=refs%2Ftags%2F2.0.7>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=24&branchName=master) | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(GPUs)?branchName=refs%2Ftags%2F2.0.7>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=24&branchName=master) | Soon |
+|        Linux py3.9 \[GPUs\]        |                                                                                                -                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(GPUs)?branchName=refs%2Ftags%2F2.0.8>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=24&branchName=master) | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(GPUs)?branchName=refs%2Ftags%2F2.0.8>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=24&branchName=master) | Soon |
 |        Linux py3.9 \[TPUs\]        |                                                                                                -                                                                                                |                     [![Test PyTorch - TPU](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml)                     |                                                                                                                                                                                                                                 | Soon |
-|        Linux py3.8 \[IPUs\]        |                                                                                                -                                                                                                |                                                                                                                -                                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(IPUs)?branchName=refs%2Ftags%2F2.0.7>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=25&branchName=master) | Soon |
-|        Linux py3.8 \[HPUs\]        |                                                                                                -                                                                                                |                                                                                                                -                                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=refs%2Ftags%2F2.0.7>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | Soon |
+|        Linux py3.8 \[IPUs\]        |                                                                                                -                                                                                                |                                                                                                                -                                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(IPUs)?branchName=refs%2Ftags%2F2.0.8>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=25&branchName=master) | Soon |
+|        Linux py3.8 \[HPUs\]        |                                                                                                -                                                                                                |                                                                                                                -                                                                                                                | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=refs%2Ftags%2F2.0.8>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | Soon |
 |  Linux (multiple Python versions)  | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |                 [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                 | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                                 | Soon |
 |   OSX (multiple Python versions)   | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |                 [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                 | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                                 | Soon |
 | Windows (multiple Python versions) | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |                 [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                 | [![Test PyTorch](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml)                                 | Soon |

{lightning-2.0.7 → lightning-2.0.8}/README.md

@@ -25,7 +25,7 @@ ______________________________________________________________________
 
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pytorch-lightning)](https://pypi.org/project/pytorch-lightning/)
 [![PyPI Status](https://badge.fury.io/py/pytorch-lightning.svg)](https://badge.fury.io/py/pytorch-lightning)
-[![PyPI Status](https://pepy.tech/badge/pytorch-lightning)](https://pepy.tech/project/pytorch-lightning)
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/pytorch-lightning)](https://pepy.tech/project/pytorch-lightning)
 [![Conda](https://img.shields.io/conda/v/conda-forge/lightning?label=conda&color=success)](https://anaconda.org/conda-forge/lightning)
 [![codecov](https://codecov.io/gh/Lightning-AI/lightning/branch/master/graph/badge.svg?token=SmzX8mnKlA)](https://codecov.io/gh/Lightning-AI/lightning)
 

{lightning-2.0.7 → lightning-2.0.8}/requirements/app/base.txt

@@ -24,7 +24,7 @@ Jinja2 <=3.1.2
 PyYAML <=6.0.1
 requests <2.31.1
 rich >=12.3.0, <=13.4.2
-urllib3 <=2.0.2
+urllib3 <=2.0.4
 uvicorn <=0.23.2
 websocket-client <1.6.2
 websockets <=11.0.3

{lightning-2.0.7 → lightning-2.0.8}/requirements/app/test.txt

@@ -1,4 +1,4 @@
-coverage ==7.2.7
+coverage ==7.3.0
 pytest ==7.4.0
 pytest-timeout ==2.1.0
 pytest-cov ==4.1.0
@@ -11,4 +11,4 @@ trio <0.22.0  # strict https://github.com/python-trio/trio/pull/2213
 pympler
 psutil <=5.9.5
 setuptools <68.1.0
-requests-mock <=1.10.0
+requests-mock ==1.11.0

{lightning-2.0.7 → lightning-2.0.8}/requirements/fabric/base.txt

@@ -4,6 +4,6 @@
 numpy >=1.17.2, <1.25.3
 torch >=1.11.0, <2.1.0
 fsspec[http]>2021.06.0, <2023.7.0
-packaging >=17.1, <=23.0
+packaging >=17.1, <=23.1
 typing-extensions >=4.0.0, <=4.7.1
 lightning-utilities >=0.7.0, <0.10.0

{lightning-2.0.7 → lightning-2.0.8}/requirements/fabric/examples.txt

@@ -1,5 +1,5 @@
 # NOTE: the upper bound for the package version is only set for CI stability, and it is dropped while installing this package
 #  in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment
 torchvision >=0.12.0, <=0.15.2
-torchmetrics >=0.10.0, <0.12.0
+torchmetrics >=0.10.0, <1.1.0
 lightning-utilities >=0.8.0, <0.10.0

{lightning-2.0.7 → lightning-2.0.8}/requirements/fabric/test.txt

@@ -1,4 +1,4 @@
-coverage ==7.2.7
+coverage ==7.3.0
 pytest ==7.4.0
 pytest-cov ==4.1.0
 pytest-rerunfailures ==10.3

{lightning-2.0.7 → lightning-2.0.8}/requirements/pytorch/base.txt

@@ -3,7 +3,7 @@
 
 numpy >=1.17.2, <1.25.3
 torch >=1.11.0, <2.1.0
-tqdm >=4.57.0, <4.66.0
+tqdm >=4.57.0, <4.67.0
 PyYAML >=5.4, <=6.0.1
 fsspec[http] >2021.06.0, <2023.5.0
 torchmetrics >=0.7.0, <0.12.0  # needed for using fixed compare_version

{lightning-2.0.7 → lightning-2.0.8}/requirements/pytorch/examples.txt

@@ -3,5 +3,5 @@
 torchvision >=0.12.0, <=0.15.2
 gym[classic_control] >=0.17.0, <0.26.3
 ipython[all] <8.14.1
-torchmetrics >=0.10.0, <0.12.0
+torchmetrics >=0.10.0, <1.1.0
 lightning-utilities >=0.7.0, <0.10.0

{lightning-2.0.7 → lightning-2.0.8}/requirements/pytorch/test.txt

@@ -1,4 +1,4 @@
-coverage ==7.2.7
+coverage ==7.3.0
 pytest ==7.4.0
 pytest-cov ==4.1.0
 pytest-forked ==1.4.0

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/__init__.py

@@ -1,6 +1,5 @@
 """Root package info."""
 import logging
-import os
 
 # explicitly don't set root logger's propagation and leave this to subpackages to manage
 _logger = logging.getLogger(__name__)
@@ -27,9 +26,7 @@ from lightning.pytorch.callbacks import Callback  # noqa: E402
 from lightning.pytorch.core import LightningDataModule, LightningModule  # noqa: E402
 from lightning.pytorch.trainer import Trainer  # noqa: E402
 
-import lightning.app  # isort: skip # noqa: E402
-
-lightning.app._PROJECT_ROOT = os.path.dirname(lightning.app._PROJECT_ROOT)
+import lightning.app  # isort: skip # noqa: E402, F401
 
 
 __all__ = [

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/app/CHANGELOG.md

@@ -4,6 +4,17 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
+## [2.0.8] - 2023-08-29
+
+## Changed
+
+- Change top folder ([#18212](https://github.com/Lightning-AI/lightning/pull/18212))
+- Remove `_handle_is_headless` calls in app run loop ([#18362](https://github.com/Lightning-AI/lightning/pull/18362))
+
+### Fixed
+
+- refactor path to root preventing circular import ([#18357](https://github.com/Lightning-AI/lightning/pull/18357))
+
 
 ## [2.0.7] - 2023-08-14
 

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/app/__init__.py

@@ -45,5 +45,8 @@ __package_name__ = "lightning.app".split(".")[0]
 
 _PACKAGE_ROOT = os.path.dirname(__file__)
 _PROJECT_ROOT = os.path.dirname(os.path.dirname(_PACKAGE_ROOT))
+if __package_name__ == "lightning":
+    _PACKAGE_ROOT = os.path.dirname(_PACKAGE_ROOT)
+    _PROJECT_ROOT = os.path.dirname(_PROJECT_ROOT)
 
 __all__ = ["LightningApp", "LightningFlow", "LightningWork", "LightningPlugin", "BuildConfig", "CloudCompute"]

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/app/core/app.py

@@ -43,8 +43,6 @@ from lightning.app.storage.path import _storage_root_dir
 from lightning.app.utilities import frontend
 from lightning.app.utilities.app_helpers import (
     _delta_to_app_state_delta,
-    _handle_is_headless,
-    _is_headless,
     _LightningAppRef,
     _should_dispatch_app,
     Logger,
@@ -437,7 +435,6 @@ class LightningApp:
             self.backend.update_work_statuses(self.works)
 
         self._update_layout()
-        self._update_is_headless()
         self._update_status()
         self.maybe_apply_changes()
 
@@ -538,13 +535,6 @@ class LightningApp:
             layout = _collect_layout(self, component)
             component._layout = layout
 
-    def _update_is_headless(self) -> None:
-        self.is_headless = _is_headless(self)
-
-        # If `is_headless` changed, handle it.
-        # This ensures support for apps which dynamically add a UI at runtime.
-        _handle_is_headless(self)
-
     def _update_status(self) -> None:
         old_status = self.status
 

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/app/runners/cloud.py

@@ -220,7 +220,8 @@ class CloudRuntime(Runtime):
         # Resolution
         root = self._resolve_root()
         # If the root will already be there, we don't need to upload and preserve the absolute entrypoint
-        absolute_entrypoint = str(root).startswith("/project")
+        top_folder = os.getenv("FILESYSTEM_TOP_FOLDER_NAME", "project")
+        absolute_entrypoint = str(root).startswith(f"/{top_folder}")
         # If system customization files found, it will set their location path
         sys_customizations_root = self._resolve_env_root()
         repo = self._resolve_repo(

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/app/utilities/app_helpers.py

@@ -33,7 +33,6 @@ from unittest.mock import MagicMock
 
 import websockets
 from deepdiff import Delta
-from lightning_cloud.openapi import AppinstancesIdBody, Externalv1LightningappInstance, V1LightningappInstanceState
 
 import lightning.app
 from lightning.app.utilities.exceptions import LightningAppStateException
@@ -581,42 +580,3 @@ def _is_headless(app: "LightningApp") -> bool:
             if "target" in entry:
                 return False
     return True
-
-
-def _handle_is_headless(app: "LightningApp"):
-    """Utility for runtime-specific handling of changes to the ``is_headless`` property."""
-    app_id = os.getenv("LIGHTNING_CLOUD_APP_ID", None)
-    project_id = os.getenv("LIGHTNING_CLOUD_PROJECT_ID", None)
-
-    if app_id is None or project_id is None:
-        return
-
-    from lightning.app.utilities.network import LightningClient
-
-    client = LightningClient()
-    list_apps_response = client.lightningapp_instance_service_list_lightningapp_instances(project_id=project_id)
-
-    current_lightningapp_instance: Optional[Externalv1LightningappInstance] = None
-    for lightningapp_instance in list_apps_response.lightningapps:
-        if lightningapp_instance.id == app_id:
-            current_lightningapp_instance = lightningapp_instance
-            break
-
-    if not current_lightningapp_instance:
-        return
-
-    if any(
-        [
-            current_lightningapp_instance.spec.is_headless == app.is_headless,
-            current_lightningapp_instance.status.phase != V1LightningappInstanceState.RUNNING,
-        ]
-    ):
-        return
-
-    current_lightningapp_instance.spec.is_headless = app.is_headless
-
-    client.lightningapp_instance_service_update_lightningapp_instance(
-        project_id=project_id,
-        id=current_lightningapp_instance.id,
-        body=AppinstancesIdBody(name=current_lightningapp_instance.name, spec=current_lightningapp_instance.spec),
-    )

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/CHANGELOG.md

@@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
+## [2.0.8] - 2023-08-29
+
+### Changed
+
+- On XLA, avoid setting the global rank before processes have been launched as this will initialize the PJRT computation client in the main process ([#16966](https://github.com/Lightning-AI/lightning/pull/16966))
+
+### Fixed
+
+- Fixed model parameters getting shared between processes when running with `strategy="ddp_spawn"` and `accelerator="cpu"`; this has a necessary memory impact, as parameters are replicated for each process now ([#18238](https://github.com/Lightning-AI/lightning/pull/18238))
+- Removed false positive warning when using `fabric.no_backward_sync` with XLA strategies ([#17761](https://github.com/Lightning-AI/lightning/pull/17761))
+- Fixed issue where Fabric would not initialize the global rank, world size, and rank-zero-only rank after initialization and before launch ([#16966](https://github.com/Lightning-AI/lightning/pull/16966))
+- Fixed FSDP full-precision `param_dtype` training (`16-mixed`, `bf16-mixed` and `32-true` configurations) to avoid FSDP assertion errors with PyTorch < 2.0 ([#18278](https://github.com/Lightning-AI/lightning/pull/18278))
+
 
 ## [2.0.7] - 2023-08-14
 

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/connector.py

@@ -499,8 +499,8 @@ class _Connector:
                 self.strategy.parallel_devices = self._parallel_devices
         if hasattr(self.strategy, "num_nodes"):
             self.strategy._num_nodes = self._num_nodes_flag
-        if hasattr(self.strategy, "set_world_ranks"):
-            self.strategy.set_world_ranks()
+        if hasattr(self.strategy, "_set_world_ranks"):
+            self.strategy._set_world_ranks()
         self.strategy._configure_launcher()
 
         if _IS_INTERACTIVE and self.strategy.launcher and not self.strategy.launcher.is_interactive_compatible:

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/fabric.py

@@ -581,7 +581,7 @@ class Fabric:
                 "You need to set up the model first before you can call `self.no_backward_sync()`:"
                 " `model = self.setup(model, ...)`"
             )
-        if not enabled or isinstance(self._strategy, SingleDeviceStrategy):
+        if not enabled or isinstance(self._strategy, (SingleDeviceStrategy, XLAStrategy)):
             context = nullcontext()
         elif self._strategy._backward_sync_control is None:
             rank_zero_warn(
@@ -781,14 +781,28 @@ class Fabric:
             return run_function(*args, **kwargs)
 
     def _move_model_to_device(self, model: nn.Module, optimizers: List[Optimizer]) -> nn.Module:
-        initial_device = next(model.parameters(), torch.tensor(0)).device
-        if any(param.device != initial_device for param in model.parameters()):
-            rank_zero_warn(
-                "The model passed to `Fabric.setup()` has parameters on different devices. Since `move_to_device=True`,"
-                " all parameters will be moved to the new device. If this is not desired, set "
-                " `Fabric.setup(..., move_to_device=False)`.",
-                category=PossibleUserWarning,
-            )
+        try:
+            initial_name, initial_param = next(model.named_parameters())
+        except StopIteration:
+            pass
+        else:
+            initial_device = initial_param.device
+            count = 0
+            first_name, first_device = None, None
+            for name, param in model.named_parameters():
+                if param.device != initial_device:
+                    count += 1
+                    if first_name is None:
+                        first_name = name
+                        first_device = param.device
+            if count > 0:
+                rank_zero_warn(
+                    f"The model passed to `Fabric.setup()` has {count} parameters on different devices (for example"
+                    f" {first_name!r} on {first_device} and {initial_name!r} on {initial_device}). Since"
+                    " `move_to_device=True`, all parameters will be moved to the new device. If this is not"
+                    " desired, set `Fabric.setup(..., move_to_device=False)`.",
+                    category=PossibleUserWarning,
+                )
 
         if isinstance(self._strategy, XLAStrategy):
             # When the user creates the optimizer, they reference the parameters on the CPU.

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/plugins/environments/lsf.py

@@ -89,7 +89,7 @@ class LSFEnvironment(ClusterEnvironment):
         if world_size is None:
             raise ValueError(
                 "Cannot determine world size. Environment variable `JSM_NAMESPACE_SIZE` not found."
-                "Make sure you run your executable with `jsrun`."
+                " Make sure you run your executable with `jsrun`."
             )
         return int(world_size)
 
@@ -102,7 +102,7 @@ class LSFEnvironment(ClusterEnvironment):
         if global_rank is None:
             raise ValueError(
                 "Cannot determine global rank. Environment variable `JSM_NAMESPACE_RANK` not found."
-                "Make sure you run your executable with `jsrun`."
+                " Make sure you run your executable with `jsrun`."
             )
         return int(global_rank)
 
@@ -115,7 +115,7 @@ class LSFEnvironment(ClusterEnvironment):
         if local_rank is None:
             raise ValueError(
                 "Cannot determine local rank. Environment variable `JSM_NAMESPACE_LOCAL_RANK` not found."
-                "Make sure you run your executable with `jsrun`."
+                " Make sure you run your executable with `jsrun`."
             )
         return int(local_rank)
 

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/plugins/precision/fsdp.py

@@ -16,7 +16,7 @@ from typing import Literal, Optional, TYPE_CHECKING
 import torch
 
 from lightning.fabric.plugins.precision.amp import MixedPrecision
-from lightning.fabric.utilities.imports import _TORCH_GREATER_EQUAL_1_12
+from lightning.fabric.utilities.imports import _TORCH_GREATER_EQUAL_1_12, _TORCH_GREATER_EQUAL_2_0
 
 if TYPE_CHECKING:
     from torch.distributed.fsdp.fully_sharded_data_parallel import MixedPrecision as TorchMixedPrecision
@@ -48,11 +48,14 @@ class FSDPPrecision(MixedPrecision):
     def mixed_precision_config(self) -> "TorchMixedPrecision":
         from torch.distributed.fsdp.fully_sharded_data_parallel import MixedPrecision as TorchMixedPrecision
 
+        # With PyTorch < 2.0, FSDP uses the noneness of `param_dtype` as a proxy for the `_uses_param_mixed_precision`
+        # property. In order to avoid FSDP assertion failures, we therefore avoid setting `param_dtype` to
+        # `torch.float32` here with PyTorch < 2.0.
         if self.precision == "16-mixed":
-            param_dtype = torch.float32
+            param_dtype = None if not _TORCH_GREATER_EQUAL_2_0 else torch.float32
             reduce_dtype = buffer_dtype = torch.float16
         elif self.precision == "bf16-mixed":
-            param_dtype = torch.float32
+            param_dtype = None if not _TORCH_GREATER_EQUAL_2_0 else torch.float32
             reduce_dtype = buffer_dtype = torch.bfloat16
         elif self.precision == "16-true":
             param_dtype = reduce_dtype = buffer_dtype = torch.float16

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/strategies/ddp.py

@@ -186,7 +186,6 @@ class DDPStrategy(ParallelStrategy):
 
     def _setup_distributed(self) -> None:
         self._set_world_ranks()
-        rank_zero_only.rank = self.global_rank
         self._process_group_backend = self._get_process_group_backend()
         assert self.cluster_environment is not None
         _init_dist_connection(self.cluster_environment, self._process_group_backend, timeout=self._timeout)
@@ -195,11 +194,12 @@ class DDPStrategy(ParallelStrategy):
         return self._process_group_backend or _get_default_process_group_backend_for_device(self.root_device)
 
     def _set_world_ranks(self) -> None:
-        if self.cluster_environment is None:
-            return
-        self.cluster_environment.set_global_rank(self.node_rank * self.num_processes + self.local_rank)
-        self.cluster_environment.set_world_size(self.num_nodes * self.num_processes)
-        rank_zero_only.rank = self.cluster_environment.global_rank()
+        if self.cluster_environment is not None:
+            self.cluster_environment.set_global_rank(self.node_rank * self.num_processes + self.local_rank)
+            self.cluster_environment.set_world_size(self.num_nodes * self.num_processes)
+        # `LightningEnvironment.set_global_rank` will do this too, but we cannot rely on that implementation detail
+        # additionally, for some implementations, the setter is a no-op, so it's safer to access the getter
+        rank_zero_only.rank = self.global_rank
 
     def _determine_ddp_device_ids(self) -> Optional[List[int]]:
         if self.root_device.type == "cpu":

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/strategies/deepspeed.py

@@ -33,7 +33,7 @@ from lightning.fabric.strategies.ddp import DDPStrategy
 from lightning.fabric.strategies.registry import _StrategyRegistry
 from lightning.fabric.strategies.strategy import _Sharded
 from lightning.fabric.utilities.distributed import log
-from lightning.fabric.utilities.rank_zero import rank_zero_info, rank_zero_only, rank_zero_warn
+from lightning.fabric.utilities.rank_zero import rank_zero_info, rank_zero_warn
 from lightning.fabric.utilities.seed import reset_seed
 from lightning.fabric.utilities.types import _PATH
 
@@ -579,7 +579,6 @@ class DeepSpeedStrategy(DDPStrategy, _Sharded):
         _validate_device_index_selection(self.parallel_devices)
         reset_seed()
         self._set_world_ranks()
-        rank_zero_only.rank = self.global_rank
         self._init_deepspeed_distributed()
         if not self._config_initialized:
             self._format_config()

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/strategies/fsdp.py

@@ -297,7 +297,6 @@ class FSDPStrategy(ParallelStrategy, _Sharded):
     def _setup_distributed(self) -> None:
         reset_seed()
         self._set_world_ranks()
-        rank_zero_only.rank = self.global_rank
         self._process_group_backend = self._get_process_group_backend()
         assert self.cluster_environment is not None
         _init_dist_connection(self.cluster_environment, self._process_group_backend, timeout=self._timeout)
@@ -306,11 +305,12 @@ class FSDPStrategy(ParallelStrategy, _Sharded):
         return self._process_group_backend or _get_default_process_group_backend_for_device(self.root_device)
 
     def _set_world_ranks(self) -> None:
-        if self.cluster_environment is None:
-            return
-        self.cluster_environment.set_global_rank(self.node_rank * self.num_processes + self.local_rank)
-        self.cluster_environment.set_world_size(self.num_nodes * self.num_processes)
-        rank_zero_only.rank = self.cluster_environment.global_rank()
+        if self.cluster_environment is not None:
+            self.cluster_environment.set_global_rank(self.node_rank * self.num_processes + self.local_rank)
+            self.cluster_environment.set_world_size(self.num_nodes * self.num_processes)
+        # `LightningEnvironment.set_global_rank` will do this too, but we cannot rely on that implementation detail
+        # additionally, for some implementations, the setter is a no-op, so it's safer to access the getter
+        rank_zero_only.rank = self.global_rank
 
 
 def _setup_activation_checkpointing(module: "FullyShardedDataParallel", layers: List[Type[Module]]) -> None:

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/strategies/launchers/multiprocessing.py

@@ -11,6 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+import itertools
 import os
 from dataclasses import dataclass
 from multiprocessing.queues import SimpleQueue
@@ -19,7 +20,10 @@ from typing import Any, Callable, Dict, Literal, Optional, TYPE_CHECKING
 import torch
 import torch.backends.cudnn
 import torch.multiprocessing as mp
+from lightning_utilities.core.apply_func import apply_to_collection
+from torch.nn import Module
 
+from lightning.fabric.accelerators.cpu import CPUAccelerator
 from lightning.fabric.strategies.launchers.launcher import _Launcher
 from lightning.fabric.utilities.apply_func import move_data_to_device
 from lightning.fabric.utilities.imports import _IS_INTERACTIVE
@@ -122,6 +126,10 @@ class _MultiProcessingLauncher(_Launcher):
     ) -> None:
         if global_states:
             global_states.restore()
+
+        if self._start_method == "spawn" and isinstance(self._strategy.accelerator, CPUAccelerator):
+            args, kwargs = _disable_module_memory_sharing((args, kwargs))
+
         os.environ["LOCAL_RANK"] = str(process_idx)
         results = function(*args, **kwargs)
 
@@ -190,3 +198,21 @@ def _check_bad_cuda_fork() -> None:
     if _IS_INTERACTIVE:
         message += " You will have to restart the Python kernel."
     raise RuntimeError(message)
+
+
+def _disable_module_memory_sharing(data: Any) -> Any:
+    """Disables memory sharing on parameters and buffers of `nn.Module`s contained in the given collection.
+
+    Note: This is only required when running on CPU.
+    """
+    # PyTorch enables memory sharing automatically on all tensors that are passed through `mp.spawn`.
+    # For model weights and buffers, this is undesired and can lead to race conditions between processes.
+    # Hence, we copy the tensors in the entire module to ensure it doesn't share memory with other processes.
+
+    @torch.no_grad()
+    def unshare(module: Module) -> Module:
+        for tensor in itertools.chain(module.parameters(), module.buffers()):
+            tensor.data = tensor.data.clone()
+        return module
+
+    return apply_to_collection(data, function=unshare, dtype=Module)

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/fabric/strategies/xla.py

@@ -94,7 +94,6 @@ class XLAStrategy(ParallelStrategy):
 
     def setup_environment(self) -> None:
         self._launched = True
-        self._set_world_ranks()
         rank_zero_only.rank = self.global_rank
         super().setup_environment()
 
@@ -208,11 +207,6 @@ class XLAStrategy(ParallelStrategy):
     def register_strategies(cls, strategy_registry: _StrategyRegistry) -> None:
         strategy_registry.register("xla", cls, description=cls.__class__.__name__)
 
-    def _set_world_ranks(self) -> None:
-        if self.cluster_environment is None:
-            return
-        rank_zero_only.rank = self.cluster_environment.global_rank()
-
     @staticmethod
     def _validate_dataloader(dataloader: object) -> None:
         if not has_len(dataloader):

{lightning-2.0.7 → lightning-2.0.8}/src/lightning/pytorch/CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 
+## [2.0.8] - 2023-08-29
+
+### Changed
+
+- On XLA, avoid setting the global rank before processes have been launched as this will initialize the PJRT computation client in the main process ([#16966](https://github.com/Lightning-AI/lightning/pull/16966))
+- Fix inefficiency in rich progress bar ([#18369](https://github.com/Lightning-AI/lightning/pull/18369))
+
+### Fixed
+
+- Fixed FSDP full-precision `param_dtype` training (`16-mixed` and `bf16-mixed` configurations) to avoid FSDP assertion errors with PyTorch < 2.0 ([#18278](https://github.com/Lightning-AI/lightning/pull/18278))
+- Fixed an issue that prevented the use of custom logger classes without an `experiment` property defined ([#18093](https://github.com/Lightning-AI/lightning/pull/18093))
+- Fixed setting the tracking uri in `MLFlowLogger` for logging artifacts to the MLFlow server ([#18395](https://github.com/Lightning-AI/lightning/pull/18395))
+- Fixed redundant `iter()` call to dataloader when checking dataloading configuration ([#18415](https://github.com/Lightning-AI/lightning/pull/18415))
+- Fixed model parameters getting shared between processes when running with `strategy="ddp_spawn"` and `accelerator="cpu"`; this has a necessary memory impact, as parameters are replicated for each process now ([#18238](https://github.com/Lightning-AI/lightning/pull/18238))
+- Properly manage `fetcher.done` with `dataloader_iter` ([#18376](https://github.com/Lightning-AI/lightning/pull/18376))
+
+
 ## [2.0.7] - 2023-08-14
 
 ### Added