nb2slurm 0.0.1.dev1__tar.gz → 0.0.1.dev2__tar.gz

{nb2slurm-0.0.1.dev1 → nb2slurm-0.0.1.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nb2slurm
-Version: 0.0.1.dev1
+Version: 0.0.1.dev2
 Summary: Seamlessly scale a single-subject notebook workflow to many subjects on a SLURM HPC, driven entirely from notebooks.
 Project-URL: Homepage, https://github.com/eWaterCycle/nb2slurm
 Author: Mark Melotto
@@ -300,6 +300,41 @@ wf.status(ssh=cfg)                                     # parsed squeue
 wf.cancel(ssh=cfg)                                     # scancel what we submitted
 ```
 
+## Connecting over SSH
+
+`SSHConfig` is how nb2slurm reaches the cluster — paramiko for commands, `rsync`
+for file sync. You need an SSH key registered on the cluster. nb2slurm can make
+the key, but **you** register it (most HPCs disable password login, so there's no
+way for a tool to install it for you):
+
+```python
+# 1. make a keypair locally (ed25519 recommended; key_type="rsa" also supported)
+nb2slurm.generate_key(key_type="ed25519")     # -> ~/.ssh/id_ed25519 (+.pub), prints the public key
+# copy the printed PUBLIC key into your HPC's key-upload page / portal (SURF,
+# Snellius, ...), or into ~/.ssh/authorized_keys on a login node. Reprint any
+# time with nb2slurm.public_key("~/.ssh/id_ed25519").
+
+# 2. describe the connection
+cfg = nb2slurm.SSHConfig(
+    host="snellius.surf.nl", user="me",
+    remote_dir="/home/me/myproject",
+    key_filename="~/.ssh/id_ed25519",
+    # passphrase="…",   # only if the key is encrypted AND you're not using ssh-agent
+)
+
+# 3. check it before anything else
+cfg.test_connection()    # runs hostname/whoami, prints a clear OK/FAIL (fails fast, never hangs)
+```
+
+**Passphrase-protected keys.** The "password" most clusters prompt for is your
+key's *passphrase* (decrypted locally), not a server login. The clean way is to
+load the key into **ssh-agent** once — `ssh-add ~/.ssh/id_ed25519` — so both
+nb2slurm *and* `rsync` (`push`/`pull`) authenticate through it with no prompts.
+Passing `passphrase=` to `SSHConfig` also works, but it only unlocks the paramiko
+calls (`test_connection`/`submit`/`status`/`check`); `push`/`pull` go through the
+`rsync` CLI and still need the agent. Secrets (`password`, `passphrase`) are never
+written to disk by `save_config`.
+
 ## Creating the conda environment + kernel
 
 The SLURM job does `conda activate <env>` and papermill needs a registered
@@ -326,6 +361,23 @@ Passing `environment=env` to `Workflow` keeps the names in sync (it errors if
 `kernel`/`conda_env` disagree) and makes `build()` also write `environment.yml`.
 `create_environment()` uses `mamba` when available, falls back to `conda`, and
 registers the kernel via `ipykernel`. Omit `ssh=` to build the env locally instead.
+It runs **non-interactively** (it never stalls on a conda `[Y/n]` prompt over SSH)
+and **streams** conda/mamba output live, so a multi-minute solve doesn't look like
+a hang.
+
+### Rebuilding or removing an environment
+
+`create_environment` is idempotent: re-running it *updates the env in place* (no
+`Overwrite?` prompt). To recover from a half-built env, or force a clean slate:
+
+```python
+wf.create_environment(ssh=cfg, overwrite=True)  # delete, then rebuild from scratch
+wf.remove_environment(ssh=cfg)                  # delete the env AND its Jupyter kernel
+wf.environment.exists(ssh=cfg)                  # -> True/False, changes nothing
+```
+
+(`Environment` has the same `.create(overwrite=...)`, `.remove()`, and `.exists()`
+if you're driving one directly.)
 
 ### Using a cluster's existing environment (no env creation)
 
@@ -478,6 +530,11 @@ wf.submit([("NL","123","ssp126")], ssh=cfg)   # override: run an explicit subset
 wf.submit(ssh=cfg, jobs_json="rerun.json")    # override: use a different file
 ```
 
+`concurrency` caps how many jobs run at once, chained with SLURM `afterany`
+dependencies so you don't flood the queue. Set **`concurrency=0`** to submit
+everything at once with **no dependencies** — ideal for a handful of quick,
+independent jobs. You can also override it per call: `wf.submit(ssh=cfg, concurrency=0)`.
+
 Because each job's output dir is built from the JSON, your first notebook never
 builds folders — it just receives `outdir` and writes `settings.json`. The
 underlying parser is exposed as `nb2slurm.Structure` if you want it directly
@@ -501,7 +558,10 @@ The split is the safety mechanism:
   back can't overwrite a notebook you changed locally while jobs were running.
 
 So the normal loop after editing a notebook is: `push` the change, submit again
-(finished work is skipped via `done.csv`), then `pull` results when ready.
+(finished work is skipped via `done.csv`), then `pull` results when ready. `pull`
+is safe to run even before any results exist — it creates the remote `output/`
+and `done/` dirs if they're missing rather than erroring, and is fine to re-run as
+more jobs finish.
 
 ## Control notebooks
 

{nb2slurm-0.0.1.dev1 → nb2slurm-0.0.1.dev2}/README.md

@@ -70,6 +70,41 @@ wf.status(ssh=cfg)                                     # parsed squeue
 wf.cancel(ssh=cfg)                                     # scancel what we submitted
 ```
 
+## Connecting over SSH
+
+`SSHConfig` is how nb2slurm reaches the cluster — paramiko for commands, `rsync`
+for file sync. You need an SSH key registered on the cluster. nb2slurm can make
+the key, but **you** register it (most HPCs disable password login, so there's no
+way for a tool to install it for you):
+
+```python
+# 1. make a keypair locally (ed25519 recommended; key_type="rsa" also supported)
+nb2slurm.generate_key(key_type="ed25519")     # -> ~/.ssh/id_ed25519 (+.pub), prints the public key
+# copy the printed PUBLIC key into your HPC's key-upload page / portal (SURF,
+# Snellius, ...), or into ~/.ssh/authorized_keys on a login node. Reprint any
+# time with nb2slurm.public_key("~/.ssh/id_ed25519").
+
+# 2. describe the connection
+cfg = nb2slurm.SSHConfig(
+    host="snellius.surf.nl", user="me",
+    remote_dir="/home/me/myproject",
+    key_filename="~/.ssh/id_ed25519",
+    # passphrase="…",   # only if the key is encrypted AND you're not using ssh-agent
+)
+
+# 3. check it before anything else
+cfg.test_connection()    # runs hostname/whoami, prints a clear OK/FAIL (fails fast, never hangs)
+```
+
+**Passphrase-protected keys.** The "password" most clusters prompt for is your
+key's *passphrase* (decrypted locally), not a server login. The clean way is to
+load the key into **ssh-agent** once — `ssh-add ~/.ssh/id_ed25519` — so both
+nb2slurm *and* `rsync` (`push`/`pull`) authenticate through it with no prompts.
+Passing `passphrase=` to `SSHConfig` also works, but it only unlocks the paramiko
+calls (`test_connection`/`submit`/`status`/`check`); `push`/`pull` go through the
+`rsync` CLI and still need the agent. Secrets (`password`, `passphrase`) are never
+written to disk by `save_config`.
+
 ## Creating the conda environment + kernel
 
 The SLURM job does `conda activate <env>` and papermill needs a registered
@@ -96,6 +131,23 @@ Passing `environment=env` to `Workflow` keeps the names in sync (it errors if
 `kernel`/`conda_env` disagree) and makes `build()` also write `environment.yml`.
 `create_environment()` uses `mamba` when available, falls back to `conda`, and
 registers the kernel via `ipykernel`. Omit `ssh=` to build the env locally instead.
+It runs **non-interactively** (it never stalls on a conda `[Y/n]` prompt over SSH)
+and **streams** conda/mamba output live, so a multi-minute solve doesn't look like
+a hang.
+
+### Rebuilding or removing an environment
+
+`create_environment` is idempotent: re-running it *updates the env in place* (no
+`Overwrite?` prompt). To recover from a half-built env, or force a clean slate:
+
+```python
+wf.create_environment(ssh=cfg, overwrite=True)  # delete, then rebuild from scratch
+wf.remove_environment(ssh=cfg)                  # delete the env AND its Jupyter kernel
+wf.environment.exists(ssh=cfg)                  # -> True/False, changes nothing
+```
+
+(`Environment` has the same `.create(overwrite=...)`, `.remove()`, and `.exists()`
+if you're driving one directly.)
 
 ### Using a cluster's existing environment (no env creation)
 
@@ -248,6 +300,11 @@ wf.submit([("NL","123","ssp126")], ssh=cfg)   # override: run an explicit subset
 wf.submit(ssh=cfg, jobs_json="rerun.json")    # override: use a different file
 ```
 
+`concurrency` caps how many jobs run at once, chained with SLURM `afterany`
+dependencies so you don't flood the queue. Set **`concurrency=0`** to submit
+everything at once with **no dependencies** — ideal for a handful of quick,
+independent jobs. You can also override it per call: `wf.submit(ssh=cfg, concurrency=0)`.
+
 Because each job's output dir is built from the JSON, your first notebook never
 builds folders — it just receives `outdir` and writes `settings.json`. The
 underlying parser is exposed as `nb2slurm.Structure` if you want it directly
@@ -271,7 +328,10 @@ The split is the safety mechanism:
   back can't overwrite a notebook you changed locally while jobs were running.
 
 So the normal loop after editing a notebook is: `push` the change, submit again
-(finished work is skipped via `done.csv`), then `pull` results when ready.
+(finished work is skipped via `done.csv`), then `pull` results when ready. `pull`
+is safe to run even before any results exist — it creates the remote `output/`
+and `done/` dirs if they're missing rather than erroring, and is fine to re-run as
+more jobs finish.
 
 ## Control notebooks
 

{nb2slurm-0.0.1.dev1 → nb2slurm-0.0.1.dev2}/docs/control/0_config.ipynb

@@ -1,187 +1,191 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# 0. Configure your run\n",
-    "\n",
-    "This is the only notebook you normally edit. Fill in the cells below, then run it\n",
-    "top to bottom. It saves two files in the project root:\n",
-    "\n",
-    "* **`control_config.json`** — all your settings, read by `1_build`, `2_submit`\n",
-    "  and `3_sync` (so they always agree).\n",
-    "* **`jobs.json`** — the nested list of jobs to run.\n",
-    "\n",
-    "Re-run this notebook whenever you change a setting."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Project\n",
-    "\n",
-    "Name, the notebooks to run in order, and what varies per job."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "project_name = \"myproject\"\n",
-    "notebooks = [\n",
-    "    \"notebooks/0_settings.ipynb\",   # first notebook: writes settings.json\n",
-    "    \"notebooks/1_analysis.ipynb\",\n",
-    "    \"notebooks/2_more.ipynb\",\n",
-    "]\n",
-    "varying = [\"country\", \"region\", \"scenario\"]   # the levels in jobs.json (below)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": "## The jobs to run\n\nA nested dict: each root-to-leaf path is one SLURM job, and the levels line up\nwith `varying` above. nb2slurm builds the matching output folders and submits one\njob per leaf.\n\n**Format:** a *dict* nests one more level, a *list* at the bottom means several\njobs sharing that parent, and `None`/`[]` ends the path. It's plain JSON, so for\nsmall runs type it by hand (below) and for big runs generate it in Python (a\ncomprehension, a CSV, an API query, ...) — see the README for the rules. The only\nthing that matters is that the finished dict lands in `jobs.json`."
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "jobs = {\n",
-    "    \"NL\": {\"123\": [\"ssp126\", \"ssp245\"]},\n",
-    "    \"DE\": {\"789\": [\"ssp585\"]},\n",
-    "}\n",
-    "# -> jobs: (NL,123,ssp126) (NL,123,ssp245) (DE,789,ssp585)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## HPC connection\n",
-    "\n",
-    "Your cluster login details. The project dir is derived so it can't drift."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "username_on_slurm    = \"me\"\n",
-    "slurm_host           = \"spider.surf.nl\"\n",
-    "project_dir_on_slurm = f\"/home/{username_on_slurm}/{project_name}\"\n",
-    "ssh_key_file         = \"~/.ssh/id_ed25519\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Resources per job\n",
-    "\n",
-    "What each SLURM job asks for, and where outputs go."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": "time_for_job_to_run = \"04:00:00\"   # HH:MM:SS wall-clock limit\ncpus_per_job        = 2\nnodes_per_job       = 1\njobs_at_once        = 3            # max running in parallel\noutput_directory    = \"output\"    # could be \"/scratch/me/output\""
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": "## Conda environment (optional)\n\nSkip if your cluster already provides Python (set `conda_env`/`setup` instead).\nOtherwise nb2slurm builds this environment + kernel for you in `1_build`.\n\n*Need a different environment for one or two notebooks (e.g. a calibration step)?*\nAdd `kernels={\"notebooks/step_8.ipynb\": \"myenv2\"}` and\n`extra_environments=[nb2slurm.Environment(name=\"myenv2\", kernel=\"myenv2\", ...)]`\nto the `Workflow` in the **Save** cell below — see the README for details. The\nsingle-environment setup here covers the common case."
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "conda_env_name = \"myenv\"\n",
-    "kernel_name    = \"myenv\"\n",
-    "conda_packages = [\"xarray\", \"numpy\"]\n",
-    "pip_packages   = [\"nb2slurm\", \"ewatercycle\"]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Data mounts (optional)\n",
-    "\n",
-    "rclone mounts the job should set up before running."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "data_mounts = [\n",
-    "    {\"remote\": \"dcache:/climate-data/caravan\", \"mountpoint\": \"/scratch/caravan\"},\n",
-    "]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Save\n",
-    "\n",
-    "Builds the objects from your settings and writes `control_config.json` and\n",
-    "`jobs.json`. The other notebooks load these — you don't edit them by hand."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import json, pathlib, nb2slurm\n",
-    "\n",
-    "env = nb2slurm.Environment(\n",
-    "    name=conda_env_name, kernel=kernel_name,\n",
-    "    conda_packages=conda_packages, pip_packages=pip_packages,\n",
-    ")\n",
-    "wf = nb2slurm.Workflow(\n",
-    "    name=project_name, notebooks=notebooks, kernel=kernel_name,\n",
-    "    varying=varying, jobs_json=\"jobs.json\",\n",
-    "    resources=dict(nodes=nodes_per_job, cpus=cpus_per_job, time=time_for_job_to_run),\n",
-    "    mounts=data_mounts, concurrency=jobs_at_once,\n",
-    "    output_dir=output_directory, environment=env,\n",
-    ")\n",
-    "cfg = nb2slurm.SSHConfig(\n",
-    "    host=slurm_host, user=username_on_slurm,\n",
-    "    remote_dir=project_dir_on_slurm, key_filename=ssh_key_file,\n",
-    ")\n",
-    "\n",
-    "pathlib.Path(\"jobs.json\").write_text(json.dumps(jobs, indent=2))\n",
-    "nb2slurm.save_config(\"control_config.json\", workflow=wf, ssh=cfg)\n",
-    "print(\"saved control_config.json and jobs.json\")"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 0. Configure your run\n",
+    "\n",
+    "This is the only notebook you normally edit. Fill in the cells below, then run it\n",
+    "top to bottom. It saves two files in the project root:\n",
+    "\n",
+    "* **`control_config.json`** — all your settings, read by `1_build`, `2_submit`\n",
+    "  and `3_sync` (so they always agree).\n",
+    "* **`jobs.json`** — the nested list of jobs to run.\n",
+    "\n",
+    "Re-run this notebook whenever you change a setting."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Project\n",
+    "\n",
+    "Name, the notebooks to run in order, and what varies per job."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "project_name = \"myproject\"\n",
+    "notebooks = [\n",
+    "    \"notebooks/0_settings.ipynb\",   # first notebook: writes settings.json\n",
+    "    \"notebooks/1_analysis.ipynb\",\n",
+    "    \"notebooks/2_more.ipynb\",\n",
+    "]\n",
+    "varying = [\"country\", \"region\", \"scenario\"]   # the levels in jobs.json (below)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## The jobs to run\n\nA nested dict: each root-to-leaf path is one SLURM job, and the levels line up\nwith `varying` above. nb2slurm builds the matching output folders and submits one\njob per leaf.\n\n**Format:** a *dict* nests one more level, a *list* at the bottom means several\njobs sharing that parent, and `None`/`[]` ends the path. It's plain JSON, so for\nsmall runs type it by hand (below) and for big runs generate it in Python (a\ncomprehension, a CSV, an API query, ...) — see the README for the rules. The only\nthing that matters is that the finished dict lands in `jobs.json`."
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "jobs = {\n",
+    "    \"NL\": {\"123\": [\"ssp126\", \"ssp245\"]},\n",
+    "    \"DE\": {\"789\": [\"ssp585\"]},\n",
+    "}\n",
+    "# -> jobs: (NL,123,ssp126) (NL,123,ssp245) (DE,789,ssp585)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## HPC connection\n",
+    "\n",
+    "Your cluster login details. The project dir is derived so it can't drift."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "username_on_slurm    = \"me\"\n",
+    "slurm_host           = \"spider.surf.nl\"\n",
+    "project_dir_on_slurm = f\"/home/{username_on_slurm}/{project_name}\"\n",
+    "ssh_key_file         = \"~/.ssh/id_ed25519\"\n",
+    "\n",
+    "# No key yet? nb2slurm.generate_key(key_type=\"ed25519\") makes ~/.ssh/id_ed25519\n",
+    "# and prints the public key to register with your HPC. ssh-add it (or pass\n",
+    "# passphrase= to SSHConfig) if the key has a passphrase."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Resources per job\n",
+    "\n",
+    "What each SLURM job asks for, and where outputs go."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "time_for_job_to_run = \"04:00:00\"   # HH:MM:SS wall-clock limit\ncpus_per_job        = 2\nnodes_per_job       = 1\njobs_at_once        = 3            # max running in parallel\noutput_directory    = \"output\"    # could be \"/scratch/me/output\""
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Conda environment (optional)\n\nSkip if your cluster already provides Python (set `conda_env`/`setup` instead).\nOtherwise nb2slurm builds this environment + kernel for you in `1_build`.\n\n*Need a different environment for one or two notebooks (e.g. a calibration step)?*\nAdd `kernels={\"notebooks/step_8.ipynb\": \"myenv2\"}` and\n`extra_environments=[nb2slurm.Environment(name=\"myenv2\", kernel=\"myenv2\", ...)]`\nto the `Workflow` in the **Save** cell below — see the README for details. The\nsingle-environment setup here covers the common case."
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "conda_env_name = \"myenv\"\n",
+    "kernel_name    = \"myenv\"\n",
+    "conda_packages = [\"xarray\", \"numpy\"]\n",
+    "pip_packages   = [\"nb2slurm\", \"ewatercycle\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Data mounts (optional)\n",
+    "\n",
+    "rclone mounts the job should set up before running."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data_mounts = [\n",
+    "    {\"remote\": \"dcache:/climate-data/caravan\", \"mountpoint\": \"/scratch/caravan\"},\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Save\n",
+    "\n",
+    "Builds the objects from your settings and writes `control_config.json` and\n",
+    "`jobs.json`. The other notebooks load these — you don't edit them by hand."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json, pathlib, nb2slurm\n",
+    "\n",
+    "env = nb2slurm.Environment(\n",
+    "    name=conda_env_name, kernel=kernel_name,\n",
+    "    conda_packages=conda_packages, pip_packages=pip_packages,\n",
+    ")\n",
+    "wf = nb2slurm.Workflow(\n",
+    "    name=project_name, notebooks=notebooks, kernel=kernel_name,\n",
+    "    varying=varying, jobs_json=\"jobs.json\",\n",
+    "    resources=dict(nodes=nodes_per_job, cpus=cpus_per_job, time=time_for_job_to_run),\n",
+    "    mounts=data_mounts, concurrency=jobs_at_once,\n",
+    "    output_dir=output_directory, environment=env,\n",
+    ")\n",
+    "cfg = nb2slurm.SSHConfig(\n",
+    "    host=slurm_host, user=username_on_slurm,\n",
+    "    remote_dir=project_dir_on_slurm, key_filename=ssh_key_file,\n",
+    ")\n",
+    "\n",
+    "pathlib.Path(\"jobs.json\").write_text(json.dumps(jobs, indent=2))\n",
+    "nb2slurm.save_config(\"control_config.json\", workflow=wf, ssh=cfg)\n",
+    "print(\"saved control_config.json and jobs.json\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}