PyPI - skypilot-nightly - Versions diffs - 1.0.0.dev20250820__py3-none-any.whl → 1.0.0.dev20250822__py3-none-any.whl - Mend

skypilot-nightly 1.0.0.dev20250820py3-none-any.whl → 1.0.0.dev20250822py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.

This version of skypilot-nightly might be problematic. Click here for more details.

Files changed (64) hide show

sky/dashboard/out/workspace/new.html CHANGED Viewed

	@@ -1 +1 @@
1	- <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-~~008593a02784a2df~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspace/new-3f88a1c7e86a3f86.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspace/new","query":{},"buildId":"~~8ZscIHnvBWz3AXkxsJL6H~~","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
1	+ <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-6e76f636a048e145.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspace/new-3f88a1c7e86a3f86.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspace/new","query":{},"buildId":"WD29VpW0S7wsYey0qFBHQ","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>

sky/dashboard/out/workspaces/[name].html CHANGED Viewed

	@@ -1 +1 @@
1	- <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-~~008593a02784a2df~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/616-3d59f75e2ccf9321.js" defer=""></script><script src="/dashboard/_next/static/chunks/6130-2be46d70a38f1e82.js" defer=""></script><script src="/dashboard/_next/static/chunks/5739-d67458fcb1386c92.js" defer=""></script><script src="/dashboard/_next/static/chunks/7411-b15471acd2cba716.js" defer=""></script><script src="/dashboard/_next/static/chunks/1272-1ef0bf0237faccdb.js" defer=""></script><script src="/dashboard/_next/static/chunks/~~6633~~-~~efe924b9b8136699~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/6989-01359c57e018caa4.js" defer=""></script><script src="/dashboard/_next/static/chunks/3850-ff4a9a69d978632b.js" defer=""></script><script src="/dashboard/_next/static/chunks/8969-~~23c8fbdb8b397d59~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/6990-08b2a1cae076a943.js" defer=""></script><script src="/dashboard/_next/static/chunks/6135-4b4d5e824b7f9d3c.js" defer=""></script><script src="/dashboard/_next/static/chunks/1121-~~2edb8ab2ba080a76~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/6601-06114c982db410b6.js" defer=""></script><script src="/dashboard/_next/static/chunks/3015-~~bf218e4973bf5c8f~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/1141-~~2f60a90b7d76838e~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspaces/%5Bname%5D-~~65f72dee417237ef~~.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspaces/[name]","query":{},"buildId":"~~8ZscIHnvBWz3AXkxsJL6H~~","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
1	+ <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-6e76f636a048e145.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/616-3d59f75e2ccf9321.js" defer=""></script><script src="/dashboard/_next/static/chunks/6130-2be46d70a38f1e82.js" defer=""></script><script src="/dashboard/_next/static/chunks/5739-d67458fcb1386c92.js" defer=""></script><script src="/dashboard/_next/static/chunks/7411-b15471acd2cba716.js" defer=""></script><script src="/dashboard/_next/static/chunks/1272-1ef0bf0237faccdb.js" defer=""></script><script src="/dashboard/_next/static/chunks/7205-88191679e7988c57.js" defer=""></script><script src="/dashboard/_next/static/chunks/6989-01359c57e018caa4.js" defer=""></script><script src="/dashboard/_next/static/chunks/3850-ff4a9a69d978632b.js" defer=""></script><script src="/dashboard/_next/static/chunks/8969-4a6f1a928fb6d370.js" defer=""></script><script src="/dashboard/_next/static/chunks/6990-08b2a1cae076a943.js" defer=""></script><script src="/dashboard/_next/static/chunks/6135-4b4d5e824b7f9d3c.js" defer=""></script><script src="/dashboard/_next/static/chunks/1121-8afcf719ea87debc.js" defer=""></script><script src="/dashboard/_next/static/chunks/6601-06114c982db410b6.js" defer=""></script><script src="/dashboard/_next/static/chunks/3015-6c9c09593b1e67b6.js" defer=""></script><script src="/dashboard/_next/static/chunks/1141-943efc7aff0f0c06.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspaces/%5Bname%5D-de06e613e20bc977.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspaces/[name]","query":{},"buildId":"WD29VpW0S7wsYey0qFBHQ","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>

sky/dashboard/out/workspaces.html CHANGED Viewed

	@@ -1 +1 @@
1	- <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-~~008593a02784a2df~~.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspaces-~~338de9df523d883a~~.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/~~8ZscIHnvBWz3AXkxsJL6H~~/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspaces","query":{},"buildId":"~~8ZscIHnvBWz3AXkxsJL6H~~","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
1	+ <!DOCTYPE html><html><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="/dashboard/_next/static/css/4614e06482d7309e.css" as="style"/><link rel="stylesheet" href="/dashboard/_next/static/css/4614e06482d7309e.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="/dashboard/_next/static/chunks/polyfills-78c92fac7aa8fdd8.js"></script><script src="/dashboard/_next/static/chunks/webpack-6e76f636a048e145.js" defer=""></script><script src="/dashboard/_next/static/chunks/framework-cf60a09ccd051a10.js" defer=""></script><script src="/dashboard/_next/static/chunks/main-f15ccb73239a3bf1.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/_app-ce361c6959bc2001.js" defer=""></script><script src="/dashboard/_next/static/chunks/pages/workspaces-be35b22e2046564c.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_buildManifest.js" defer=""></script><script src="/dashboard/_next/static/WD29VpW0S7wsYey0qFBHQ/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/workspaces","query":{},"buildId":"WD29VpW0S7wsYey0qFBHQ","assetPrefix":"/dashboard","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>

sky/exceptions.py CHANGED Viewed

@@ -661,3 +661,11 @@ class RequestInterruptedError(Exception):
 class SkyletInternalError(Exception):
     """Raised when a Skylet internal error occurs."""
     pass
+class ClientError(Exception):
+    """Raised when a there is a client error occurs.
+    If a request encounters a ClientError, it will not be retried to the server.
+    """
+    pass

sky/jobs/utils.py CHANGED Viewed

@@ -1494,7 +1494,7 @@ def format_job_table(
         'JOB DURATION',
         '#RECOVERIES',
         'STATUS',
-        'WORKER_POOL',
+        'POOL',
     ]
     if show_all:
         # TODO: move SCHED. STATE to a separate flag (e.g. --debug)
@@ -1597,6 +1597,10 @@ def format_job_table(
             user_values = get_user_column_values(job_tasks[0])
+            pool = job_tasks[0].get('pool')
+            if pool is None:
+                pool = '-'
             job_id = job_hash[1] if tasks_have_k8s_user else job_hash
             job_values = [
                 job_id,
@@ -1610,7 +1614,7 @@ def format_job_table(
                 job_duration,
                 recovery_cnt,
                 status_str,
-                job_tasks[0].get('pool', '-'),
+                pool,
             ]
             if show_all:
                 details = job_tasks[current_task_id].get('details')
@@ -1637,6 +1641,9 @@ def format_job_table(
             submitted = log_utils.readable_time_duration(task['submitted_at'])
             user_values = get_user_column_values(task)
             task_workspace = '-' if len(job_tasks) > 1 else workspace
+            pool = task.get('pool')
+            if pool is None:
+                pool = '-'
             values = [
                 task['job_id'] if len(job_tasks) == 1 else ' \u21B3',
                 task['task_id'] if len(job_tasks) > 1 else '-',
@@ -1653,7 +1660,7 @@ def format_job_table(
                 job_duration,
                 task['recovery_count'],
                 task['status'].colored_str(),
-                task.get('pool', '-'),
+                pool,
             ]
             if show_all:
                 # schedule_state is only set at the job level, so if we have

sky/optimizer.py CHANGED Viewed

@@ -1262,12 +1262,13 @@ def _check_specified_clouds(dag: 'dag_lib.Dag') -> None:
 def _check_specified_regions(task: task_lib.Task) -> None:
-    """Check if specified regions (Kubernetes contexts) are enabled.
+    """Check if specified regions (Kubernetes/SSH contexts) are enabled.
     Args:
         task: The task to check.
     """
-    # Only check for Kubernetes now
+    # Only check for Kubernetes/SSH for now
+    # Below check works because SSH inherits Kubernetes cloud.
     if not all(
             isinstance(resources.cloud, clouds.Kubernetes)
             for resources in task.resources):
@@ -1276,12 +1277,21 @@ def _check_specified_regions(task: task_lib.Task) -> None:
     for resources in task.resources:
         if resources.region is None:
             continue
-        existing_contexts = clouds.Kubernetes.existing_allowed_contexts()
+        is_ssh = isinstance(resources.cloud, clouds.SSH)
+        if is_ssh:
+            existing_contexts = clouds.SSH.existing_allowed_contexts()
+        else:
+            existing_contexts = clouds.Kubernetes.existing_allowed_contexts()
         region = resources.region
         task_name = f' {task.name!r}' if task.name is not None else ''
         msg = f'Task{task_name} requires '
         if region not in existing_contexts:
-            infra_str = f'Kubernetes/{region}'
+            if is_ssh:
+                infra_str = f'SSH/{region.lstrip("ssh-")}'
+            else:
+                infra_str = f'Kubernetes/{region}'
             logger.warning(f'{infra_str} is not enabled.')
             volume_mounts_str = ''
             if task.volume_mounts:

sky/provision/docker_utils.py CHANGED Viewed

@@ -83,6 +83,21 @@ def check_docker_image(cname, docker_cmd):
     return _check_helper(cname, '.Config.Image', docker_cmd)
+def maybe_remove_container_cmds(container_name, docker_cmd):
+    """Remove the container if it exists. If not, it will be a no-op.
+    """
+    docker_rm = [
+        docker_cmd,
+        'rm',
+        '-f',
+        container_name,
+        '2>/dev/null',
+        '||',
+        'true',
+    ]
+    return ' '.join(docker_rm)
 def docker_start_cmds(
     image,
     container_name,
@@ -285,6 +300,10 @@ class DockerInitializer:
                 'sudo mv /tmp/daemon.json /etc/docker/daemon.json;'
                 'sudo systemctl restart docker; } || true')
             user_docker_run_options = self.docker_config.get('run_options', [])
+            remove_container_cmd = maybe_remove_container_cmds(
+                self.container_name,
+                self.docker_cmd,
+            )
             start_command = docker_start_cmds(
                 specific_image,
                 self.container_name,
@@ -292,7 +311,7 @@ class DockerInitializer:
                     self._auto_configure_shm(user_docker_run_options)),
                 self.docker_cmd,
             )
-            self._run(start_command)
+            self._run(f'{remove_container_cmd}; {start_command}')
         # SkyPilot: Setup Commands.
         # TODO(zhwu): the following setups should be aligned with the kubernetes

sky/provision/kubernetes/instance.py CHANGED Viewed

@@ -797,15 +797,18 @@ def _create_pods(region: str, cluster_name_on_cloud: str,
                        'For more details, refer to https://docs.skypilot.co/en/latest/reference/config.html')  # pylint: disable=line-too-long
     needs_gpus = False
+    needs_gpus_nvidia = False
     limits = pod_spec['spec']['containers'][0].get('resources',
                                                    {}).get('limits')
     if limits is not None:
         needs_gpus = limits.get(kubernetes_utils.get_gpu_resource_key(context),
                                 0) > 0
+        needs_gpus_nvidia = limits.get(
+            kubernetes_utils.SUPPORTED_GPU_RESOURCE_KEYS['nvidia'], 0) > 0
     # TPU pods provisioned on GKE use the default containerd runtime.
     # Reference: https://cloud.google.com/kubernetes-engine/docs/how-to/migrate-containerd#overview  # pylint: disable=line-too-long
-    if nvidia_runtime_exists and needs_gpus:
+    if nvidia_runtime_exists and needs_gpus_nvidia:
         pod_spec['spec']['runtimeClassName'] = 'nvidia'
     logger.debug(f'run_instances: calling create_namespaced_pod '

sky/resources.py CHANGED Viewed

@@ -1260,10 +1260,14 @@ class Resources:
     def extract_docker_image(self) -> Optional[str]:
         if self.image_id is None:
             return None
-        if len(self.image_id) == 1 and self.region in self.image_id:
-            image_id = self.image_id[self.region]
-            if image_id.startswith('docker:'):
-                return image_id[len('docker:'):]
+        # Handle dict image_id
+        if len(self.image_id) == 1:
+            # Check if the single key matches the region or is None (any region)
+            image_key = list(self.image_id.keys())[0]
+            if image_key == self.region or image_key is None:
+                image_id = self.image_id[image_key]
+                if image_id.startswith('docker:'):
+                    return image_id[len('docker:'):]
         return None
     def _try_validate_image_id(self) -> None:
@@ -1333,13 +1337,19 @@ class Resources:
                     'Kubernetes, please explicitly specify the cloud.') from e
         if self._region is not None:
-            if self._region not in self._image_id:
+            # If the image_id has None as key (region-agnostic),
+            # use it for any region
+            if None in self._image_id:
+                # Replace None key with the actual region
+                self._image_id = {self._region: self._image_id[None]}
+            elif self._region not in self._image_id:
                 with ux_utils.print_exception_no_traceback():
                     raise ValueError(
                         f'image_id {self._image_id} should contain the image '
                         f'for the specified region {self._region}.')
-            # Narrow down the image_id to the specified region.
-            self._image_id = {self._region: self._image_id[self._region]}
+            else:
+                # Narrow down the image_id to the specified region.
+                self._image_id = {self._region: self._image_id[self._region]}
         # Check the image_id's are valid.
         for region, image_id in self._image_id.items():

sky/server/requests/executor.py CHANGED Viewed

@@ -427,9 +427,9 @@ async def execute_request_coroutine(request: api_requests.Request):
     event loop. This is designed for executing tasks that are not CPU
     intensive, e.g. sky logs.
     """
+    context.initialize()
     ctx = context.get()
-    if ctx is None:
-        raise ValueError('Context is not initialized')
+    assert ctx is not None, 'Context is not initialized'
     logger.info(f'Executing request {request.request_id} in coroutine')
     func = request.entrypoint
     request_body = request.request_body

sky/server/requests/serializers/decoders.py CHANGED Viewed

@@ -203,3 +203,8 @@ def decode_job_status(
 def decode_kubernetes_node_info(
         return_value: Dict[str, Any]) -> models.KubernetesNodesInfo:
     return models.KubernetesNodesInfo.from_dict(return_value)
+@register_decoders('endpoints')
+def decode_endpoints(return_value: Dict[int, str]) -> Dict[int, str]:
+    return {int(k): v for k, v in return_value.items()}

sky/server/requests/serializers/encoders.py CHANGED Viewed

@@ -209,3 +209,8 @@ def encode_job_status(return_value: Dict[int, Any]) -> Dict[int, str]:
 def encode_kubernetes_node_info(
         return_value: 'models.KubernetesNodesInfo') -> Dict[str, Any]:
     return return_value.to_dict()
+@register_encoder('endpoints')
+def encode_endpoints(return_value: Dict[int, str]) -> Dict[str, str]:
+    return {str(k): v for k, v in return_value.items()}

sky/server/rest.py CHANGED Viewed

@@ -47,9 +47,10 @@ class RetryContext:
 @contextlib.contextmanager
 def _retry_in_context():
-    token = _RETRY_CONTEXT.set(RetryContext())
+    context = RetryContext()
+    token = _RETRY_CONTEXT.set(context)
     try:
-        yield
+        yield context
     finally:
         _RETRY_CONTEXT.reset(token)
@@ -76,6 +77,8 @@ def retry_transient_errors(max_retries: int = 3,
         if isinstance(e, requests.exceptions.HTTPError):
             # Only server error is considered as transient.
             return e.response.status_code >= 500
+        if isinstance(e, exceptions.ClientError):
+            return False
         # It is hard to enumerate all other errors that are transient, e.g.
         # broken pipe, connection refused, etc. Instead, it is safer to assume
         # all other errors might be transient since we only retry for 3 times
@@ -88,26 +91,53 @@ def retry_transient_errors(max_retries: int = 3,
         @functools.wraps(func)
         def wrapper(*args, **kwargs):
             backoff = common_utils.Backoff(initial_backoff, max_backoff_factor)
-            with _retry_in_context():
-                for retry_cnt in range(max_retries):
+            consecutive_failed_count = 0
+            with _retry_in_context() as context:
+                previous_line_processed = context.line_processed  # should be 0
+                def _handle_exception():
+                    # If the function made progress on a retry,
+                    # clears the backoff and resets the failed retry count.
+                    # Otherwise, increments the failed retry count.
+                    nonlocal backoff
+                    nonlocal consecutive_failed_count
+                    nonlocal previous_line_processed
+                    if context.line_processed > previous_line_processed:
+                        backoff = common_utils.Backoff(initial_backoff,
+                                                       max_backoff_factor)
+                        previous_line_processed = context.line_processed
+                        consecutive_failed_count = 0
+                    else:
+                        consecutive_failed_count += 1
+                while consecutive_failed_count < max_retries:
                     try:
                         return func(*args, **kwargs)
                     # Occurs when the server proactively interrupts the request
                     # during rolling update, we can retry immediately on the
                     # new replica.
                     except exceptions.RequestInterruptedError:
+                        _handle_exception()
                         logger.debug('Request interrupted. Retry immediately.')
                         continue
                     except Exception as e:  # pylint: disable=broad-except
-                        if retry_cnt >= max_retries - 1:
+                        _handle_exception()
+                        if consecutive_failed_count >= max_retries:
                             # Retries exhausted.
                             raise
                         if not is_transient_error(e):
                             # Permanent error, no need to retry.
                             raise
-                        logger.debug(f'Retry {func.__name__} due to {e}, '
-                                     f'attempt {retry_cnt + 1}/{max_retries}')
-                        time.sleep(backoff.current_backoff())
+                        logger.debug(
+                            f'Retry {func.__name__} due to {e}, '
+                            f'attempt {consecutive_failed_count}/{max_retries}')
+                        # Only sleep if this is not the first retry.
+                        # The idea is that if the function made progress on a
+                        # retry, we should try again immediately to reduce the
+                        # waiting time.
+                        if consecutive_failed_count > 0:
+                            time.sleep(backoff.current_backoff())
         return cast(F, wrapper)

sky/server/server.py CHANGED Viewed

@@ -1185,10 +1185,6 @@ async def logs(
     # TODO(zhwu): This should wait for the request on the cluster, e.g., async
     # launch, to finish, so that a user does not need to manually pull the
     # request status.
-    # Only initialize the context in logs handler to limit the scope of this
-    # experimental change.
-    # TODO(aylei): init in lifespan() to enable SkyPilot context in all APIs.
-    context.initialize()
     request_task = executor.prepare_request(
         request_id=request.state.request_id,
         request_name='logs',
@@ -1198,8 +1194,14 @@ async def logs(
     )
     task = asyncio.create_task(executor.execute_request_coroutine(request_task))
-    def cancel_task():
-        task.cancel()
+    async def cancel_task():
+        try:
+            logger.info('Client disconnected for request: '
+                        f'{request.state.request_id}')
+            task.cancel()
+            await task
+        except asyncio.CancelledError:
+            pass
     # Cancel the task after the request is done or client disconnects
     background_tasks.add_task(cancel_task)

sky/skypilot_config.py CHANGED Viewed

@@ -514,10 +514,10 @@ def parse_and_validate_config_file(config_path: str) -> config_utils.Config:
 def _parse_dotlist(dotlist: List[str]) -> config_utils.Config:
-    """Parse a comma-separated list of key-value pairs into a dictionary.
+    """Parse a single key-value pair into a dictionary.
     Args:
-        dotlist: A comma-separated list of key-value pairs.
+        dotlist: A single key-value pair.
     Returns:
         A config_utils.Config object with the parsed key-value pairs.
@@ -788,7 +788,7 @@ def _compose_cli_config(cli_config: Optional[List[str]]) -> config_utils.Config:
     """Composes the skypilot CLI config.
     CLI config can either be:
     - A path to a config file
-    - A comma-separated list of key-value pairs
+    - A single key-value pair
     """
     if not cli_config:
@@ -804,7 +804,7 @@ def _compose_cli_config(cli_config: Optional[List[str]]) -> config_utils.Config:
             config_source = maybe_config_path
             # cli_config is a path to a config file
             parsed_config = parse_and_validate_config_file(maybe_config_path)
-        else:  # cli_config is a comma-separated list of key-value pairs
+        else:  # cli_config is a single key-value pair
             parsed_config = _parse_dotlist(cli_config)
         _validate_config(parsed_config, config_source)
     except ValueError as e:

sky/users/permission.py CHANGED Viewed

@@ -46,7 +46,8 @@ class PermissionService:
                 engine = global_user_state.initialize_and_get_db()
                 db_utils.add_all_tables_to_db_sqlalchemy(
                     sqlalchemy_adapter.Base.metadata, engine)
-                adapter = sqlalchemy_adapter.Adapter(engine)
+                adapter = sqlalchemy_adapter.Adapter(
+                    engine, db_class=sqlalchemy_adapter.CasbinRule)
                 model_path = os.path.join(os.path.dirname(__file__),
                                           'model.conf')
                 enforcer = casbin.Enforcer(model_path, adapter)
@@ -67,7 +68,7 @@ class PermissionService:
                 username.encode()).hexdigest()[:common_utils.USER_HASH_LENGTH]
             user_info = global_user_state.get_user(user_hash)
             if user_info:
-                logger.info(f'Basic auth user {username} already exists')
+                logger.debug(f'Basic auth user {username} already exists')
                 return
             global_user_state.add_or_update_user(
                 models.User(id=user_hash, name=username, password=password))
@@ -168,8 +169,6 @@ class PermissionService:
         """
         user_roles = self.enforcer.get_roles_for_user(user_id)
         if not user_roles:
-            logger.info(f'User {user_id} has no roles, adding'
-                        f' default role {rbac.get_default_role()}')
             self.enforcer.add_grouping_policy(user_id, rbac.get_default_role())
             return True
         return False
@@ -183,7 +182,7 @@ class PermissionService:
             # Avoid calling get_user_roles, as it will require the lock.
             current_roles = self.enforcer.get_roles_for_user(user_id)
             if not current_roles:
-                logger.warning(f'User {user_id} has no roles')
+                logger.debug(f'User {user_id} has no roles')
                 return
             self.enforcer.remove_grouping_policy(user_id, current_roles[0])
             self.enforcer.save_policy()
@@ -197,12 +196,12 @@ class PermissionService:
             # Avoid calling get_user_roles, as it will require the lock.
             current_roles = self.enforcer.get_roles_for_user(user_id)
             if not current_roles:
-                logger.warning(f'User {user_id} has no roles')
+                logger.debug(f'User {user_id} has no roles')
             else:
                 # TODO(hailong): how to handle multiple roles?
                 current_role = current_roles[0]
                 if current_role == new_role:
-                    logger.info(f'User {user_id} already has role {new_role}')
+                    logger.debug(f'User {user_id} already has role {new_role}')
                     return
                 self.enforcer.remove_grouping_policy(user_id, current_role)

{skypilot_nightly-1.0.0.dev20250820.dist-info → skypilot_nightly-1.0.0.dev20250822.dist-info}/METADATA RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: skypilot-nightly
-Version: 1.0.0.dev20250820
+Version: 1.0.0.dev20250822
 Summary: SkyPilot: Run AI on Any Infra — Unified, Faster, Cheaper.
 Author: SkyPilot Team
 License: Apache 2.0
@@ -253,7 +253,7 @@ Dynamic: summary
 ----
-SkyPilot is a system for running, managing, and scaling AI workloads on any AI infrastructure.
+SkyPilot is a system to run, manage, and scale AI workloads on any AI infrastructure.
 SkyPilot gives **AI teams** a simple interface to run jobs on any infra.
 **Infra teams** get a unified control plane to manage any AI compute — with advanced scheduling, scaling, and orchestration.

skypilot-nightly 1.0.0.dev20250820__py3-none-any.whl → 1.0.0.dev20250822__py3-none-any.whl

Potentially problematic release.

skypilot-nightly 1.0.0.dev20250820py3-none-any.whl → 1.0.0.dev20250822py3-none-any.whl