flux-qfw 0.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- flux_qfw-0.0.0/LICENSE +21 -0
- flux_qfw-0.0.0/NOTICE +21 -0
- flux_qfw-0.0.0/PKG-INFO +179 -0
- flux_qfw-0.0.0/README.md +167 -0
- flux_qfw-0.0.0/flux_qfw/__init__.py +10 -0
- flux_qfw-0.0.0/flux_qfw/cli/__init__.py +315 -0
- flux_qfw-0.0.0/flux_qfw/cli/__main__.py +3 -0
- flux_qfw-0.0.0/flux_qfw/cli_plugin.py +504 -0
- flux_qfw-0.0.0/flux_qfw/envelope.py +61 -0
- flux_qfw-0.0.0/flux_qfw/flux_plugins/flux-qfw.lua +283 -0
- flux_qfw-0.0.0/flux_qfw/instance.py +116 -0
- flux_qfw-0.0.0/flux_qfw/qpm/__init__.py +33 -0
- flux_qfw-0.0.0/flux_qfw/qpm/base.py +114 -0
- flux_qfw-0.0.0/flux_qfw/qpm/defw.py +159 -0
- flux_qfw-0.0.0/flux_qfw/qpm/mock.py +155 -0
- flux_qfw-0.0.0/flux_qfw/resources.py +409 -0
- flux_qfw-0.0.0/flux_qfw/scout.py +313 -0
- flux_qfw-0.0.0/flux_qfw.egg-info/PKG-INFO +179 -0
- flux_qfw-0.0.0/flux_qfw.egg-info/SOURCES.txt +26 -0
- flux_qfw-0.0.0/flux_qfw.egg-info/dependency_links.txt +1 -0
- flux_qfw-0.0.0/flux_qfw.egg-info/entry_points.txt +2 -0
- flux_qfw-0.0.0/flux_qfw.egg-info/top_level.txt +1 -0
- flux_qfw-0.0.0/pyproject.toml +26 -0
- flux_qfw-0.0.0/setup.cfg +4 -0
- flux_qfw-0.0.0/test/test_envelope.py +39 -0
- flux_qfw-0.0.0/test/test_mock_backend.py +38 -0
- flux_qfw-0.0.0/test/test_mock_session.py +88 -0
- flux_qfw-0.0.0/test/test_resources.py +267 -0
flux_qfw-0.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2022-2023 LLNS, LLC and other DevTools Developers.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
flux_qfw-0.0.0/NOTICE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
This work was produced under the auspices of the U.S. Department of
|
|
2
|
+
Energy by Lawrence Livermore National Laboratory under Contract
|
|
3
|
+
DE-AC52-07NA27344.
|
|
4
|
+
|
|
5
|
+
This work was prepared as an account of work sponsored by an agency of
|
|
6
|
+
the United States Government. Neither the United States Government nor
|
|
7
|
+
Lawrence Livermore National Security, LLC, nor any of their employees
|
|
8
|
+
makes any warranty, expressed or implied, or assumes any legal liability
|
|
9
|
+
or responsibility for the accuracy, completeness, or usefulness of any
|
|
10
|
+
information, apparatus, product, or process disclosed, or represents that
|
|
11
|
+
its use would not infringe privately owned rights.
|
|
12
|
+
|
|
13
|
+
Reference herein to any specific commercial product, process, or service
|
|
14
|
+
by trade name, trademark, manufacturer, or otherwise does not necessarily
|
|
15
|
+
constitute or imply its endorsement, recommendation, or favoring by the
|
|
16
|
+
United States Government or Lawrence Livermore National Security, LLC.
|
|
17
|
+
|
|
18
|
+
The views and opinions of authors expressed herein do not necessarily
|
|
19
|
+
state or reflect those of the United States Government or Lawrence
|
|
20
|
+
Livermore National Security, LLC, and shall not be used for advertising
|
|
21
|
+
or product endorsement purposes.
|
flux_qfw-0.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: flux-qfw
|
|
3
|
+
Version: 0.0.0
|
|
4
|
+
Summary: QFw quantum resources as Fluxion resources: a Flux CLI plugin, a job shell plugin, and the QPM reservation client that replaces the qfw-slurm layer
|
|
5
|
+
License: MIT
|
|
6
|
+
Project-URL: Homepage, https://github.com/OpenQSE/flux-qfw
|
|
7
|
+
Requires-Python: >=3.9
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
License-File: NOTICE
|
|
11
|
+
Dynamic: license-file
|
|
12
|
+
|
|
13
|
+
# flux-qfw
|
|
14
|
+
|
|
15
|
+
> QFw quantum devices as Fluxion resources.
|
|
16
|
+
|
|
17
|
+
flux-qfw is the Flux version of [qfw-slurm](https://github.com/openQSE/qfw-slurm). While the qfw-slurm reserves devices before Slurm grants the job, flux-qfw puts each QFw QPM into the Fluxion resource graph, next to the compute nodes, and lets Fluxion schedule it like a core. The flux submit (and similar) client is updated to ask for the device:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
flux submit -N2 -n8 --qfw-qpu=nwqsim ./my_hybrid_app.sh
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
What happens next depends on control, because we do not control a second vendor queue.
|
|
24
|
+
|
|
25
|
+
- **A device we own and control.** Fluxion gives the job its nodes and the device together, in one match. If the device is busy, the job waits in the queue the same way it would for a busy core.
|
|
26
|
+
- **A device behind a vendor's queue.** A grant from Fluxion does not mean the vendor is ready. So the submit turns into a pair of jobs. The user's job goes in first, held with its nodes reserved. Then a small scout job goes in that names the user's job, holds the device in the graph, and waits for the vendor. When the vendor is ready, the scout releases the user's job and both run together.
|
|
27
|
+
|
|
28
|
+
Either way the device is a vertex in the graph and Fluxion decides who has it. This integration needs flux-core and the `hold` branch of [vsoch/flux-sched](https://github.com/vsoch/flux-sched), which adds the hold, the release RPC, and the `coschedule` queue policy. The plugins are tested end to end with a mock QPM. The DEFw backend that interacts to a real QPM is a port of the qfw-slurm gateway code and has not been tested yet.
|
|
29
|
+
|
|
30
|
+
## What is in here
|
|
31
|
+
|
|
32
|
+
| Piece | Where | What it does |
|
|
33
|
+
| --- | --- | --- |
|
|
34
|
+
| CLI plugin | `cli-plugins/qfw.py`, `flux_qfw/cli_plugin.py` | Adds the `--qfw-*` options to `flux run`, `submit`, `batch` and `alloc`. Adds the device to the job's resources, or submits the scout pair for an external queue. |
|
|
35
|
+
| Shell plugin | `flux_qfw/flux_plugins/flux-qfw.lua` | Runs in every job shell. Exports what the tasks need, like `QFW_RESERVATIONS` or the simulator hosts. |
|
|
36
|
+
| Scout | `flux_qfw/scout.py` | The program the scout job runs. Opens the vendor session and releases the user's job when it is ready. |
|
|
37
|
+
| Helper | `flux-qfw` | Command line tool with `devices`, `add`, `sessions`, `scout`, `register`, `unregister`, `memo`, `granted`, `status` and `plugins`. |
|
|
38
|
+
| Backends | `flux_qfw/qpm/` | `defw` talks to a real QPM. `mock` keeps reservations and sessions in small files for testing. |
|
|
39
|
+
|
|
40
|
+
## The resource shape
|
|
41
|
+
|
|
42
|
+
Each QPM is one `qdevice_<service_id>` vertex with one or more `qpu` children, and each `qpu` has a `qubit` pool as wide as the device. It sits under the cluster root, beside the racks and not under any node.
|
|
43
|
+
|
|
44
|
+
```text
|
|
45
|
+
/cluster0/rack0/c1/core0 a compute core
|
|
46
|
+
/cluster0/qdevice_nwqsim0/qpu0 the NWQ-Sim device
|
|
47
|
+
/cluster0/qdevice_nwqsim0/qpu0/qubit0 a pool of 32 qubits
|
|
48
|
+
/cluster0/qdevice_fake-iqm0/qpu0 a vendor device with two leases
|
|
49
|
+
/cluster0/qdevice_fake-iqm0/qpu0/qubit0 of 20 qubits each
|
|
50
|
+
/cluster0/qdevice_fake-iqm0/qpu1
|
|
51
|
+
/cluster0/qdevice_fake-iqm0/qpu1/qubit0
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
The number of `qpu` vertices is how many jobs can hold the device at once. Use one for a device the site owns and should be exclusive. Use the number of concurrent leases for a vendor device. Fluxion does the admitting, so the QPM's own admission policy can be `unlimited`. A hybrid job keeps its normal classical request and gets a second top level entry for the device. The `qpu` asks for as many qubits as `--qfw-max-qubits`, which is one unless the user says more, so Fluxion only gives the job a device that wide. The plugin refuses a job wider than the device at submit.
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
[
|
|
58
|
+
{"type": "node", "count": 2, "with": [{"type": "slot", "count": 4, "label": "task",
|
|
59
|
+
"with": [{"type": "core", "count": 1}]}]},
|
|
60
|
+
{"type": "qdevice_nwqsim", "count": 1, "with": [{"type": "qpu", "count": 1, "exclusive": true,
|
|
61
|
+
"with": [{"type": "qubit", "count": 5}]}]}
|
|
62
|
+
]
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
A hybrid job needs a node count, so always pass `-N`. Without it there is no node level to put beside the device, and the plugin refuses the job at submit. I am assuming it is not possible to run quantum work without some small amount of classical resources to submit it. The plugin also reads the live graph. If it finds levels between node and core, like a socket, it adds them to the request so Fluxion can match it.
|
|
66
|
+
|
|
67
|
+
## Devices we own
|
|
68
|
+
|
|
69
|
+
These are configured in the instance's `[qfw]` table, one entry per device.
|
|
70
|
+
|
|
71
|
+
```toml
|
|
72
|
+
[qfw]
|
|
73
|
+
launch-agent = "/usr/local/libexec/qfw-flux/flux-launch-agent"
|
|
74
|
+
|
|
75
|
+
[qfw.qpm.nwqsim]
|
|
76
|
+
scope = "job"
|
|
77
|
+
|
|
78
|
+
[qfw.qpm.shim-ornl-20q]
|
|
79
|
+
scope = "site"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
There are two kinds.
|
|
83
|
+
|
|
84
|
+
**Job scope.** This is for a simulator. The job runs the simulator itself on its own nodes, so there is nothing to reserve. Every rank gets `QFW_QPU_SCOPE=job`, the job's hosts in `QFW_SIMULATOR_NODES`, and a launch agent so the MPI runtime starts its daemons with `flux exec` instead of ssh. QFw then runs exactly as it would on a workstation.
|
|
85
|
+
|
|
86
|
+
**Site scope.** This is for a QPM the site runs as a service. The job already holds the device in the graph, so the shell does not acquire anything. When the job starts it registers the job with the QPM, and when the job ends it unregisters it, even if the task fails. The QPM hands back the reservation id QFw needs. If the QPM refuses a job Fluxion already granted, the job fails with a message to set the QPM's admission policy to `unlimited`. Inside the job, `QFW_RESERVATIONS` looks like `[["shim-ornl-20q","<id>"]]`, which is the format QFw's `qfw-srun` uses.
|
|
87
|
+
|
|
88
|
+
## Devices behind a vendor queue
|
|
89
|
+
|
|
90
|
+
Add `queue = "external"` to the device's entry.
|
|
91
|
+
|
|
92
|
+
```toml
|
|
93
|
+
[qfw.qpm.fake-iqm]
|
|
94
|
+
scope = "site"
|
|
95
|
+
queue = "external"
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The user still types `flux submit --qfw-qpu=fake-iqm`. Under the hood the plugin submits two jobs.
|
|
99
|
+
|
|
100
|
+
1. The **user's job** asks for its normal nodes and no device. It is submitted first, held. Under the `coschedule` policy Fluxion keeps its nodes reserved on every scheduling loop but does not start it. Other work can still run around it.
|
|
101
|
+
2. The **scout job** is submitted as soon as the user's job id comes back, with that id as its partner. The plugin memos the scout's id onto the user's job as `qfw_scout`. The scout asks for one core and the device. It waits in the queue like any other job, so it only runs while it holds a lease. That means the number of open vendor sessions can never be more than the number of leases.
|
|
102
|
+
|
|
103
|
+
The user's reserved nodes are off limits to the scout, so the scout needs a core outside them. The plugin checks the graph at submit and refuses a job that leaves no such core, like one that fills every node it could run on. The core can be on another node, or left over on one of the user's nodes when the job does not take whole nodes.
|
|
104
|
+
|
|
105
|
+
Once the scout starts, it opens a session on the vendor's queue and waits. When the session is ready it writes the reservation onto the user's job as a memo and releases the hold. The user's job starts right away, and its shell exports the reservation as `QFW_RESERVATIONS`. The scout keeps the device and the session until the user's job ends, then closes the session and exits.
|
|
106
|
+
|
|
107
|
+
The scout gives the vendor a fixed time to get ready, set with `session-wait` in the device's table or `--qfw-session-wait`, and one hour by default. If the session is not ready by then, the scout closes it and the user's job gets an exception, so its nodes are not reserved forever. The scout's own clock starts before the user's job, so its time limit is the session wait plus the user's duration plus a minute. That way the scout, and the session it keeps, outlive the user's job.
|
|
108
|
+
|
|
109
|
+
If the user cancels their job while it is held, the scout closes the session and exits. If the scout is cancelled, the user's job gets an exception. If the vendor refuses the session, the user's job gets an exception too. We are never left waiting forever, waiting for Godot. Run `flux-qfw sessions` to see sessions.
|
|
110
|
+
|
|
111
|
+
The `mock` backend can fake a vendor queue. `FLUX_QFW_MOCK_SESSION_DELAY` sets how long the fake vendor takes, and the default is five seconds. The `defw` backend does not support sessions yet.
|
|
112
|
+
|
|
113
|
+
## Adding devices to a cluster
|
|
114
|
+
|
|
115
|
+
Make R as usual, then add the devices before the instance starts.
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
flux R encode --hosts=flux-lead,c[1-8] --cores=0-3 --property=normal:1-8 > R.json
|
|
119
|
+
flux-qfw add --r R.json --output R-q.json nwqsim:32 fake-iqm=2:20
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
A device is `NAME[=QPUS][:QUBITS]`, so `fake-iqm=2:20` is two leases of 20 qubits each. Both default to one. Point `[resource] path` at `R-q.json` and set `noverify = true`. The `--r` option takes a plain R, an R that already has a scheduling graph, or a bare graph. If you build your own graph in Python, call `flux_qfw.resources.add_devices_to_graph()`.
|
|
123
|
+
|
|
124
|
+
You can also add a device to an instance that is already running.
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
flux-qfw add fake-iqm=2:20
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
This uses Fluxion's `add_subgraph` RPC and only the instance owner can run it. The new device shows up in Fluxion's graph and in the R of any job that gets it. It does not show up in the `resource.R` that Flux stored at boot.
|
|
131
|
+
|
|
132
|
+
Check the devices either way.
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
flux-qfw devices
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## Submitting
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
export FLUX_CLI_PLUGINPATH=$(flux-qfw plugins)
|
|
142
|
+
flux run -N1 -n4 --qfw-qpu=nwqsim --qfw-max-qubits=5 --qfw-max-shots=1024 ./qfw_qiskit_simple.sh 5
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
These are the options. They match what qfw-slurm's `salloc --qpu` took.
|
|
146
|
+
|
|
147
|
+
| Option | What it means |
|
|
148
|
+
| --- | --- |
|
|
149
|
+
| `--qfw-qpu` | The device to use |
|
|
150
|
+
| `--qfw-qpus` | How many leases to take on it, default 1 |
|
|
151
|
+
| `--qfw-max-qubits` | The widest circuit, default 1. Fluxion matches the device's qubits against it, and the QPM gets it too |
|
|
152
|
+
| `--qfw-max-depth`, `--qfw-max-shots`, `--qfw-circ-count` | The size of the work, sent to the QPM when a reservation or session is made |
|
|
153
|
+
| `--qfw-one-q-gates`, `--qfw-two-q-gates`, `--qfw-measurements` | More size bounds for the QPM |
|
|
154
|
+
| `--qfw-workload-kind` | `hybrid` (default) or `quantum` |
|
|
155
|
+
| `--qfw-scope` | Override the device's scope, `job` or `site` |
|
|
156
|
+
| `--qfw-queue` | Override the device's queue, `owned` or `external` |
|
|
157
|
+
| `--qfw-session-wait` | How long the scout waits for the vendor, like `30m`. Overrides the device's `session-wait`, default `1h` |
|
|
158
|
+
| `--qfw-backend` | `defw` (default) or `mock` |
|
|
159
|
+
| `--qfw-no-reserve` | Schedule the device but do not tell the QPM |
|
|
160
|
+
|
|
161
|
+
Fluxion reads only the qubits. Its job is to decide who holds the device, and whether the device is wide enough. Whether the rest of the work fits is the QPM's call. The helper runs with the environment you submitted from. For the `defw` backend, activate QFw with `qfw-activate` first so it can find the venv and `QFW_SITE_CONFIG`.
|
|
162
|
+
|
|
163
|
+
## Tests
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
make unit # no flux needed
|
|
167
|
+
make test # also runs the t/ tests under flux start with Fluxion
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
## How this differs from qfw-slurm
|
|
171
|
+
|
|
172
|
+
- There is no gateway daemon, SPANK plugin, burst buffer or job submit Lua. The device is a resource and Fluxion allocates it.
|
|
173
|
+
- Vendor queues use a scout job and a held job, instead of reserving before the allocation is granted.
|
|
174
|
+
- There is no job identity check yet for site scope. The shell runs as the job owner and calls the QPM directly. That is fine for a device the site controls, but a multi user site will want a verified path.
|
|
175
|
+
- There are no heterogeneous jobs. A hybrid job is one job with two resource entries.
|
|
176
|
+
|
|
177
|
+
## License
|
|
178
|
+
|
|
179
|
+
MIT. See LICENSE, COPYRIGHT and NOTICE.
|
flux_qfw-0.0.0/README.md
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# flux-qfw
|
|
2
|
+
|
|
3
|
+
> QFw quantum devices as Fluxion resources.
|
|
4
|
+
|
|
5
|
+
flux-qfw is the Flux version of [qfw-slurm](https://github.com/openQSE/qfw-slurm). While the qfw-slurm reserves devices before Slurm grants the job, flux-qfw puts each QFw QPM into the Fluxion resource graph, next to the compute nodes, and lets Fluxion schedule it like a core. The flux submit (and similar) client is updated to ask for the device:
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
flux submit -N2 -n8 --qfw-qpu=nwqsim ./my_hybrid_app.sh
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
What happens next depends on control, because we do not control a second vendor queue.
|
|
12
|
+
|
|
13
|
+
- **A device we own and control.** Fluxion gives the job its nodes and the device together, in one match. If the device is busy, the job waits in the queue the same way it would for a busy core.
|
|
14
|
+
- **A device behind a vendor's queue.** A grant from Fluxion does not mean the vendor is ready. So the submit turns into a pair of jobs. The user's job goes in first, held with its nodes reserved. Then a small scout job goes in that names the user's job, holds the device in the graph, and waits for the vendor. When the vendor is ready, the scout releases the user's job and both run together.
|
|
15
|
+
|
|
16
|
+
Either way the device is a vertex in the graph and Fluxion decides who has it. This integration needs flux-core and the `hold` branch of [vsoch/flux-sched](https://github.com/vsoch/flux-sched), which adds the hold, the release RPC, and the `coschedule` queue policy. The plugins are tested end to end with a mock QPM. The DEFw backend that interacts to a real QPM is a port of the qfw-slurm gateway code and has not been tested yet.
|
|
17
|
+
|
|
18
|
+
## What is in here
|
|
19
|
+
|
|
20
|
+
| Piece | Where | What it does |
|
|
21
|
+
| --- | --- | --- |
|
|
22
|
+
| CLI plugin | `cli-plugins/qfw.py`, `flux_qfw/cli_plugin.py` | Adds the `--qfw-*` options to `flux run`, `submit`, `batch` and `alloc`. Adds the device to the job's resources, or submits the scout pair for an external queue. |
|
|
23
|
+
| Shell plugin | `flux_qfw/flux_plugins/flux-qfw.lua` | Runs in every job shell. Exports what the tasks need, like `QFW_RESERVATIONS` or the simulator hosts. |
|
|
24
|
+
| Scout | `flux_qfw/scout.py` | The program the scout job runs. Opens the vendor session and releases the user's job when it is ready. |
|
|
25
|
+
| Helper | `flux-qfw` | Command line tool with `devices`, `add`, `sessions`, `scout`, `register`, `unregister`, `memo`, `granted`, `status` and `plugins`. |
|
|
26
|
+
| Backends | `flux_qfw/qpm/` | `defw` talks to a real QPM. `mock` keeps reservations and sessions in small files for testing. |
|
|
27
|
+
|
|
28
|
+
## The resource shape
|
|
29
|
+
|
|
30
|
+
Each QPM is one `qdevice_<service_id>` vertex with one or more `qpu` children, and each `qpu` has a `qubit` pool as wide as the device. It sits under the cluster root, beside the racks and not under any node.
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
/cluster0/rack0/c1/core0 a compute core
|
|
34
|
+
/cluster0/qdevice_nwqsim0/qpu0 the NWQ-Sim device
|
|
35
|
+
/cluster0/qdevice_nwqsim0/qpu0/qubit0 a pool of 32 qubits
|
|
36
|
+
/cluster0/qdevice_fake-iqm0/qpu0 a vendor device with two leases
|
|
37
|
+
/cluster0/qdevice_fake-iqm0/qpu0/qubit0 of 20 qubits each
|
|
38
|
+
/cluster0/qdevice_fake-iqm0/qpu1
|
|
39
|
+
/cluster0/qdevice_fake-iqm0/qpu1/qubit0
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
The number of `qpu` vertices is how many jobs can hold the device at once. Use one for a device the site owns and should be exclusive. Use the number of concurrent leases for a vendor device. Fluxion does the admitting, so the QPM's own admission policy can be `unlimited`. A hybrid job keeps its normal classical request and gets a second top level entry for the device. The `qpu` asks for as many qubits as `--qfw-max-qubits`, which is one unless the user says more, so Fluxion only gives the job a device that wide. The plugin refuses a job wider than the device at submit.
|
|
43
|
+
|
|
44
|
+
```json
|
|
45
|
+
[
|
|
46
|
+
{"type": "node", "count": 2, "with": [{"type": "slot", "count": 4, "label": "task",
|
|
47
|
+
"with": [{"type": "core", "count": 1}]}]},
|
|
48
|
+
{"type": "qdevice_nwqsim", "count": 1, "with": [{"type": "qpu", "count": 1, "exclusive": true,
|
|
49
|
+
"with": [{"type": "qubit", "count": 5}]}]}
|
|
50
|
+
]
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
A hybrid job needs a node count, so always pass `-N`. Without it there is no node level to put beside the device, and the plugin refuses the job at submit. I am assuming it is not possible to run quantum work without some small amount of classical resources to submit it. The plugin also reads the live graph. If it finds levels between node and core, like a socket, it adds them to the request so Fluxion can match it.
|
|
54
|
+
|
|
55
|
+
## Devices we own
|
|
56
|
+
|
|
57
|
+
These are configured in the instance's `[qfw]` table, one entry per device.
|
|
58
|
+
|
|
59
|
+
```toml
|
|
60
|
+
[qfw]
|
|
61
|
+
launch-agent = "/usr/local/libexec/qfw-flux/flux-launch-agent"
|
|
62
|
+
|
|
63
|
+
[qfw.qpm.nwqsim]
|
|
64
|
+
scope = "job"
|
|
65
|
+
|
|
66
|
+
[qfw.qpm.shim-ornl-20q]
|
|
67
|
+
scope = "site"
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
There are two kinds.
|
|
71
|
+
|
|
72
|
+
**Job scope.** This is for a simulator. The job runs the simulator itself on its own nodes, so there is nothing to reserve. Every rank gets `QFW_QPU_SCOPE=job`, the job's hosts in `QFW_SIMULATOR_NODES`, and a launch agent so the MPI runtime starts its daemons with `flux exec` instead of ssh. QFw then runs exactly as it would on a workstation.
|
|
73
|
+
|
|
74
|
+
**Site scope.** This is for a QPM the site runs as a service. The job already holds the device in the graph, so the shell does not acquire anything. When the job starts it registers the job with the QPM, and when the job ends it unregisters it, even if the task fails. The QPM hands back the reservation id QFw needs. If the QPM refuses a job Fluxion already granted, the job fails with a message to set the QPM's admission policy to `unlimited`. Inside the job, `QFW_RESERVATIONS` looks like `[["shim-ornl-20q","<id>"]]`, which is the format QFw's `qfw-srun` uses.
|
|
75
|
+
|
|
76
|
+
## Devices behind a vendor queue
|
|
77
|
+
|
|
78
|
+
Add `queue = "external"` to the device's entry.
|
|
79
|
+
|
|
80
|
+
```toml
|
|
81
|
+
[qfw.qpm.fake-iqm]
|
|
82
|
+
scope = "site"
|
|
83
|
+
queue = "external"
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The user still types `flux submit --qfw-qpu=fake-iqm`. Under the hood the plugin submits two jobs.
|
|
87
|
+
|
|
88
|
+
1. The **user's job** asks for its normal nodes and no device. It is submitted first, held. Under the `coschedule` policy Fluxion keeps its nodes reserved on every scheduling loop but does not start it. Other work can still run around it.
|
|
89
|
+
2. The **scout job** is submitted as soon as the user's job id comes back, with that id as its partner. The plugin memos the scout's id onto the user's job as `qfw_scout`. The scout asks for one core and the device. It waits in the queue like any other job, so it only runs while it holds a lease. That means the number of open vendor sessions can never be more than the number of leases.
|
|
90
|
+
|
|
91
|
+
The user's reserved nodes are off limits to the scout, so the scout needs a core outside them. The plugin checks the graph at submit and refuses a job that leaves no such core, like one that fills every node it could run on. The core can be on another node, or left over on one of the user's nodes when the job does not take whole nodes.
|
|
92
|
+
|
|
93
|
+
Once the scout starts, it opens a session on the vendor's queue and waits. When the session is ready it writes the reservation onto the user's job as a memo and releases the hold. The user's job starts right away, and its shell exports the reservation as `QFW_RESERVATIONS`. The scout keeps the device and the session until the user's job ends, then closes the session and exits.
|
|
94
|
+
|
|
95
|
+
The scout gives the vendor a fixed time to get ready, set with `session-wait` in the device's table or `--qfw-session-wait`, and one hour by default. If the session is not ready by then, the scout closes it and the user's job gets an exception, so its nodes are not reserved forever. The scout's own clock starts before the user's job, so its time limit is the session wait plus the user's duration plus a minute. That way the scout, and the session it keeps, outlive the user's job.
|
|
96
|
+
|
|
97
|
+
If the user cancels their job while it is held, the scout closes the session and exits. If the scout is cancelled, the user's job gets an exception. If the vendor refuses the session, the user's job gets an exception too. We are never left waiting forever, waiting for Godot. Run `flux-qfw sessions` to see sessions.
|
|
98
|
+
|
|
99
|
+
The `mock` backend can fake a vendor queue. `FLUX_QFW_MOCK_SESSION_DELAY` sets how long the fake vendor takes, and the default is five seconds. The `defw` backend does not support sessions yet.
|
|
100
|
+
|
|
101
|
+
## Adding devices to a cluster
|
|
102
|
+
|
|
103
|
+
Make R as usual, then add the devices before the instance starts.
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
flux R encode --hosts=flux-lead,c[1-8] --cores=0-3 --property=normal:1-8 > R.json
|
|
107
|
+
flux-qfw add --r R.json --output R-q.json nwqsim:32 fake-iqm=2:20
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
A device is `NAME[=QPUS][:QUBITS]`, so `fake-iqm=2:20` is two leases of 20 qubits each. Both default to one. Point `[resource] path` at `R-q.json` and set `noverify = true`. The `--r` option takes a plain R, an R that already has a scheduling graph, or a bare graph. If you build your own graph in Python, call `flux_qfw.resources.add_devices_to_graph()`.
|
|
111
|
+
|
|
112
|
+
You can also add a device to an instance that is already running.
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
flux-qfw add fake-iqm=2:20
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
This uses Fluxion's `add_subgraph` RPC and only the instance owner can run it. The new device shows up in Fluxion's graph and in the R of any job that gets it. It does not show up in the `resource.R` that Flux stored at boot.
|
|
119
|
+
|
|
120
|
+
Check the devices either way.
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
flux-qfw devices
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Submitting
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
export FLUX_CLI_PLUGINPATH=$(flux-qfw plugins)
|
|
130
|
+
flux run -N1 -n4 --qfw-qpu=nwqsim --qfw-max-qubits=5 --qfw-max-shots=1024 ./qfw_qiskit_simple.sh 5
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
These are the options. They match what qfw-slurm's `salloc --qpu` took.
|
|
134
|
+
|
|
135
|
+
| Option | What it means |
|
|
136
|
+
| --- | --- |
|
|
137
|
+
| `--qfw-qpu` | The device to use |
|
|
138
|
+
| `--qfw-qpus` | How many leases to take on it, default 1 |
|
|
139
|
+
| `--qfw-max-qubits` | The widest circuit, default 1. Fluxion matches the device's qubits against it, and the QPM gets it too |
|
|
140
|
+
| `--qfw-max-depth`, `--qfw-max-shots`, `--qfw-circ-count` | The size of the work, sent to the QPM when a reservation or session is made |
|
|
141
|
+
| `--qfw-one-q-gates`, `--qfw-two-q-gates`, `--qfw-measurements` | More size bounds for the QPM |
|
|
142
|
+
| `--qfw-workload-kind` | `hybrid` (default) or `quantum` |
|
|
143
|
+
| `--qfw-scope` | Override the device's scope, `job` or `site` |
|
|
144
|
+
| `--qfw-queue` | Override the device's queue, `owned` or `external` |
|
|
145
|
+
| `--qfw-session-wait` | How long the scout waits for the vendor, like `30m`. Overrides the device's `session-wait`, default `1h` |
|
|
146
|
+
| `--qfw-backend` | `defw` (default) or `mock` |
|
|
147
|
+
| `--qfw-no-reserve` | Schedule the device but do not tell the QPM |
|
|
148
|
+
|
|
149
|
+
Fluxion reads only the qubits. Its job is to decide who holds the device, and whether the device is wide enough. Whether the rest of the work fits is the QPM's call. The helper runs with the environment you submitted from. For the `defw` backend, activate QFw with `qfw-activate` first so it can find the venv and `QFW_SITE_CONFIG`.
|
|
150
|
+
|
|
151
|
+
## Tests
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
make unit # no flux needed
|
|
155
|
+
make test # also runs the t/ tests under flux start with Fluxion
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
## How this differs from qfw-slurm
|
|
159
|
+
|
|
160
|
+
- There is no gateway daemon, SPANK plugin, burst buffer or job submit Lua. The device is a resource and Fluxion allocates it.
|
|
161
|
+
- Vendor queues use a scout job and a held job, instead of reserving before the allocation is granted.
|
|
162
|
+
- There is no job identity check yet for site scope. The shell runs as the job owner and calls the QPM directly. That is fine for a device the site controls, but a multi user site will want a verified path.
|
|
163
|
+
- There are no heterogeneous jobs. A hybrid job is one job with two resource entries.
|
|
164
|
+
|
|
165
|
+
## License
|
|
166
|
+
|
|
167
|
+
MIT. See LICENSE, COPYRIGHT and NOTICE.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
"""flux-qfw schedules QFw quantum devices with Fluxion.
|
|
2
|
+
|
|
3
|
+
Each QFw QPM is a qdevice_<service_id> vertex with qpu children in the
|
|
4
|
+
Fluxion resource graph. A hybrid job asks for its normal classical resources
|
|
5
|
+
and a device. For a device the site owns, Fluxion gives the job both at once.
|
|
6
|
+
For a device behind a vendor queue, a scout job holds the device and the
|
|
7
|
+
user's job is held until the vendor is ready.
|
|
8
|
+
"""
|
|
9
|
+
|
|
10
|
+
__version__ = "0.0.1"
|