accli 1.1.3__tar.gz → 2.0.0__tar.gz

{accli-1.1.3 → accli-2.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: accli
-Version: 1.1.3
+Version: 2.0.0
 Summary: IIASA Accelerator Client
 Author-email: Wrufesh S <wrufesh@gmail.com>
 License: The MIT License (MIT)
@@ -39,9 +39,9 @@ Requires-Dist: requests>=2.25.1
 Requires-Dist: jsonschema>=4.5.1
 Requires-Dist: pydantic>=2.0
 Requires-Dist: GitPython
+Requires-Dist: hf-xet==1.5.2
 Dynamic: license-file
 
-
 # Accelerator Terminal Client and Python API
 
 This package provides:
@@ -49,7 +49,8 @@ This package provides:
 - A **command-line client** (`accli`) for interacting with the **Accelerator**.
 - A **Python API** via the `accli` package.
 
-Both interfaces communicate with the Accelerator’s REST API, implemented in the [Control Services Backend](https://github.com/iiasa/control_services_backend).
+Both interfaces communicate with the Accelerator’s REST API, implemented in
+the [Control Services Backend](https://github.com/iiasa/control_services_backend).
 
 ---
 
@@ -57,10 +58,11 @@ Both interfaces communicate with the Accelerator’s REST API, implemented in th
 
 This client uses **device authentication** via Auth0.
 
-- **OAuth Flow**: [Device Authorization Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow)  
-- **Token Validity**: 7 days  
-- **Access Control**: Role-Based Access Control (RBAC) with stateless tokens  
-- **Grants**: Limited and scoped  
+- **OAuth Flow
+  **: [Device Authorization Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow)
+- **Token Validity**: 7 days
+- **Access Control**: Role-Based Access Control (RBAC) with stateless tokens
+- **Grants**: Limited and scoped
 
 ---
 
@@ -77,11 +79,13 @@ This client uses **device authentication** via Auth0.
 No Python install required. Run the appropriate command for your system:
 
 **Linux / macOS:**
+
 ```bash
 curl -fsSL https://raw.githubusercontent.com/iiasa/accli/master/scripts/install.sh | bash
 ```
 
 **Windows (PowerShell):**
+
 ```powershell
 irm https://raw.githubusercontent.com/iiasa/accli/master/scripts/install.ps1 | iex
 ```
@@ -132,6 +136,17 @@ Usage: accli [OPTIONS] COMMAND [ARGS]...
 
 ## 👩‍💻 Developer Guide
 
+### Handling self-signed certificates
+
+1. Copy and paste certificates from `<control_services_backend>/certs/ca.crt` as
+   `./certs/dev-backend.crt` and `<accelerator_service>/minio_certs/ca.crt` as
+   `./certs/dev-minio-ca.crt`.
+2. Run
+   ```
+   cd scripts
+   import_certs_certutil.ps1
+   ```
+
 ### 🛠 Build & Upload
 
 Follow the official Python packaging tutorial:  

{accli-1.1.3 → accli-2.0.0}/README.md

@@ -1,4 +1,3 @@
-
 # Accelerator Terminal Client and Python API
 
 This package provides:
@@ -6,7 +5,8 @@ This package provides:
 - A **command-line client** (`accli`) for interacting with the **Accelerator**.
 - A **Python API** via the `accli` package.
 
-Both interfaces communicate with the Accelerator’s REST API, implemented in the [Control Services Backend](https://github.com/iiasa/control_services_backend).
+Both interfaces communicate with the Accelerator’s REST API, implemented in
+the [Control Services Backend](https://github.com/iiasa/control_services_backend).
 
 ---
 
@@ -14,10 +14,11 @@ Both interfaces communicate with the Accelerator’s REST API, implemented in th
 
 This client uses **device authentication** via Auth0.
 
-- **OAuth Flow**: [Device Authorization Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow)  
-- **Token Validity**: 7 days  
-- **Access Control**: Role-Based Access Control (RBAC) with stateless tokens  
-- **Grants**: Limited and scoped  
+- **OAuth Flow
+  **: [Device Authorization Flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow)
+- **Token Validity**: 7 days
+- **Access Control**: Role-Based Access Control (RBAC) with stateless tokens
+- **Grants**: Limited and scoped
 
 ---
 
@@ -34,11 +35,13 @@ This client uses **device authentication** via Auth0.
 No Python install required. Run the appropriate command for your system:
 
 **Linux / macOS:**
+
 ```bash
 curl -fsSL https://raw.githubusercontent.com/iiasa/accli/master/scripts/install.sh | bash
 ```
 
 **Windows (PowerShell):**
+
 ```powershell
 irm https://raw.githubusercontent.com/iiasa/accli/master/scripts/install.ps1 | iex
 ```
@@ -89,6 +92,17 @@ Usage: accli [OPTIONS] COMMAND [ARGS]...
 
 ## 👩‍💻 Developer Guide
 
+### Handling self-signed certificates
+
+1. Copy and paste certificates from `<control_services_backend>/certs/ca.crt` as
+   `./certs/dev-backend.crt` and `<accelerator_service>/minio_certs/ca.crt` as
+   `./certs/dev-minio-ca.crt`.
+2. Run
+   ```
+   cd scripts
+   import_certs_certutil.ps1
+   ```
+
 ### 🛠 Build & Upload
 
 Follow the official Python packaging tutorial:  

{accli-1.1.3 → accli-2.0.0}/accli/AcceleratorJobProjectService.py

@@ -446,6 +446,30 @@ class AcceleratorJobProjectService:
             headers=headers
         )
 
+    def register_validation_with_filename(
+            self,
+            validated_filename: str,
+            dataset_template_id: int,
+            validated_metadata: dict,
+            validation_supporting_filenames: List[str]
+    ):
+        headers = {"Content-Type": "application/json"}
+
+        headers.update(self.common_request_headers)
+
+        res = self.http_client_request(
+            "PUT",
+            f"{self.cli_base_url}/register-validation-with-filename/",
+            json=dict(
+                validated_filename=validated_filename,
+                dataset_template_id=dataset_template_id,
+                validated_metadata=validated_metadata,
+                validation_supporting_filenames=validation_supporting_filenames
+            ),
+            headers=headers
+        )
+
+
     def get_dataset_template_details(
             self,
             dataset_template_id

{accli-1.1.3 → accli-2.0.0}/accli/AcceleratorTaskDispatcher.py

@@ -139,10 +139,10 @@ class JobDispatchModel(BaseModel):
 class WKubeTaskMeta(BaseModel):
     required_cores: float
     required_ram: float
-    required_storage_local: float
+    required_storage_local: Optional[float] = None
 
     # is ignored if it is a callback and child of non-free node jobs
-    required_storage_workflow: float
+    required_storage_workflow: Optional[float] = None
 
     job_secrets: Optional[dict] = {}
 

{accli-1.1.3 → accli-2.0.0}/accli/AcceleratorTerminalCliProjectService.py

@@ -4,6 +4,7 @@ import json
 import os
 
 import requests
+from requests.adapters import HTTPAdapter
 import urllib3
 from typing import List, Tuple
 
@@ -21,7 +22,12 @@ class AccAPIError(Exception):
         self.status_code = status_code
 
 
-retries = urllib3.util.Retry(total=10, backoff_factor=1)
+retries = urllib3.util.Retry(
+    total=10,
+    backoff_factor=1,
+    status_forcelist=[408, 429, 500, 502, 503, 504],
+    allowed_methods=["HEAD", "GET", "OPTIONS", "PUT"]
+)
 
 http_client = urllib3.poolmanager.PoolManager(
     num_pools=20, retries=retries, maxsize=2000, block=True
@@ -41,8 +47,9 @@ class AcceleratorTerminalCliProjectService:
     ):
 
         self.user_token = user_token
+        self.verify_cert = verify_cert
 
-        if verify_cert:
+        if self.verify_cert:
             self.http_client = http_client
         else:
             self.http_client = http_client_wo_cert_verification
@@ -264,16 +271,22 @@ class AcceleratorTerminalCliProjectService:
             project_slug, app_bucket_id, uniqified_filename, upload_id, part_number
         )
 
-        part_upload_response = requests.put(
-            put_presigned_url,
-            data=part_data,
-            # headers=headers,
-            verify=False,
-        )
+        session = requests.Session()
+        adapter = HTTPAdapter(max_retries=retries)
+        session.mount("https://", adapter)
+        session.verify = self.verify_cert
+
+        with session.put(
+                put_presigned_url,
+                data=part_data,
+                timeout=120
+        ) as part_upload_response:
+            part_upload_response.raise_for_status()
+
+            etag = part_upload_response.headers.get("etag").replace('"', "")
 
         progress.update(task, advance=len(part_data))
 
-        etag = part_upload_response.headers.get("etag").replace('"', "")
         return part_number, etag
 
     def upload_filestream_to_accelerator(self, project_slug, filename, file_stream, progress, task,

{accli-1.1.3 → accli-2.0.0}/accli/CsvRegionalTimeseriesValidator.py

@@ -205,8 +205,8 @@ class CsvRegionalTimeseriesValidator:
             for key in self.errors:
                 print(f"[bold red]Invalid data:[/bold red] {self.errors[key]}")
                 print(f"[bold red]Error:[/bold red] {key}")
-            print("[bold red]Data is not valid against selected template. ⚠️⚠️")
+            print("[bold red]Data is not valid against selected template.[/bold red]")
             typer.Exit(1)
         else:
-            print("[bold green]Data validated against selected template.[/bold green] 🎉🎉")
+            print("[bold green]Data validated against selected template.[/bold green]")
             typer.Exit(0)

{accli-1.1.3 → accli-2.0.0}/accli/_version.py

@@ -1,3 +1,3 @@
 # Please strictly put double quote to use this info for git tag 
 # Read a developer guide for git tag command with regex
-VERSION = "v1.1.3"
+VERSION = "v2.0.0"

accli-2.0.0/accli/cleanup-nfs.ps1

@@ -0,0 +1,57 @@
+<#
+.SYNOPSIS
+    Reverts all Windows NFS Client and Task Scheduler changes made by setup-nfs.ps1.
+.DESCRIPTION
+    This script removes the scheduled task, registry keys, ProgramData configs, and disables
+    the native Windows NFS Client features. Useful for testing first-time accli setups.
+.NOTES
+    Must be run in an elevated (Administrator) PowerShell window.
+.EXAMPLE
+    # Open an Administrator PowerShell window and execute:
+    powershell -ExecutionPolicy Bypass -File .\accli\cleanup-nfs.ps1
+#>
+
+# Ensure running as Administrator
+$isAdmin = ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
+if (-not $isAdmin) {
+    Write-Error "This script must be run as Administrator."
+    Write-Host "Please open PowerShell as Administrator and run:" -ForegroundColor Yellow
+    Write-Host "  powershell -ExecutionPolicy Bypass -File `"$PSCommandPath`"" -ForegroundColor Yellow
+    exit 1
+}
+
+Write-Host "=== Reverting accli Windows NFS Client Configuration ===" -ForegroundColor Cyan
+
+# 1. Unregister Scheduled Tasks
+Write-Host "Removing Scheduled Tasks..." -ForegroundColor Yellow
+if (Get-Command Unregister-ScheduledTask -ErrorAction SilentlyContinue) {
+    Unregister-ScheduledTask -TaskName "accli-mount-nfs" -Confirm:$false -ErrorAction SilentlyContinue | Out-Null
+    Unregister-ScheduledTask -TaskName "accli-umount-nfs" -Confirm:$false -ErrorAction SilentlyContinue | Out-Null
+} else {
+    schtasks /delete /tn "accli-mount-nfs" /f 2> $null | Out-Null
+    schtasks /delete /tn "accli-umount-nfs" /f 2> $null | Out-Null
+}
+Write-Host "[OK] Scheduled Tasks removed." -ForegroundColor Green
+
+# 2. Delete Registry Policy Keys
+Write-Host "Removing registry policies (EnableLinkedConnections, AnonymousUid, AnonymousGid)..." -ForegroundColor Yellow
+Remove-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "EnableLinkedConnections" -ErrorAction SilentlyContinue | Out-Null
+Remove-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\ClientForNFS\CurrentVersion\Default" -Name "AnonymousUid" -ErrorAction SilentlyContinue | Out-Null
+Remove-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\ClientForNFS\CurrentVersion\Default" -Name "AnonymousGid" -ErrorAction SilentlyContinue | Out-Null
+Write-Host "[OK] Registry policies removed." -ForegroundColor Green
+
+# 3. Delete Config Folder & Helpers
+Write-Host "Deleting ProgramData configurations..." -ForegroundColor Yellow
+Remove-Item -Path "C:\ProgramData\accli" -Recurse -Force -ErrorAction SilentlyContinue | Out-Null
+Write-Host "[OK] ProgramData folder removed." -ForegroundColor Green
+
+# 4. Disable Windows NFS Client features
+Write-Host "Disabling Windows Client for NFS features..." -ForegroundColor Yellow
+if (Get-Command Disable-WindowsOptionalFeature -ErrorAction SilentlyContinue) {
+    Disable-WindowsOptionalFeature -Online -FeatureName ServicesForNFS-ClientOnly,ClientForNFS-Infrastructure -NoRestart | Out-Null
+}
+Write-Host "[OK] NFS Client features disabled." -ForegroundColor Green
+
+Write-Host ""
+Write-Host "Reversion completed successfully!" -ForegroundColor Green
+Write-Host "⚠ Note: A Windows sign-out/sign-in or system restart is required for changes to take effect." -ForegroundColor Yellow