fair-cli 0.10.2__tar.gz → 0.12.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- fair_cli-0.12.0/CHANGELOG.md +413 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/CITATION.cff +2 -2
- fair_cli-0.12.0/PKG-INFO +576 -0
- fair_cli-0.12.0/README.md +528 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/__init__.py +15 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/cli.py +439 -63
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/configuration/__init__.py +34 -16
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/exceptions.py +15 -7
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/history.py +74 -12
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/identifiers.py +53 -6
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/register.py +130 -1
- fair_cli-0.12.0/fair/registry/database.py +235 -0
- fair_cli-0.12.0/fair/registry/lifecycle.py +1060 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/requests.py +76 -7
- fair_cli-0.12.0/fair/registry/server.py +1078 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/storage.py +178 -28
- fair_cli-0.12.0/fair/registry/stores.py +391 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/sync.py +1096 -160
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/session.py +922 -185
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/staging.py +77 -7
- fair_cli-0.12.0/fair/templates/hist.jinja +3 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/user_config/__init__.py +300 -24
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/user_config/globbing.py +6 -4
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/user_config/validation.py +85 -2
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/utilities.py +34 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/pyproject.toml +2 -1
- fair_cli-0.10.2/CHANGELOG.md +0 -192
- fair_cli-0.10.2/PKG-INFO +0 -402
- fair_cli-0.10.2/README.md +0 -355
- fair_cli-0.10.2/fair/registry/server.py +0 -519
- fair_cli-0.10.2/fair/templates/hist.jinja +0 -3
- {fair_cli-0.10.2 → fair_cli-0.12.0}/LICENSE +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/common.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/configuration/validation.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/files.txt +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/logging.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/__init__.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/file_formats.json +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/file_types.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/registry/versioning.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/run.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/templates/__init__.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/templates/config.jinja +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/testing.py +0 -0
- {fair_cli-0.10.2 → fair_cli-0.12.0}/fair/virtualenv.py +0 -0
|
@@ -0,0 +1,413 @@
|
|
|
1
|
+
# 2026-11-11 [v0.12.0](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.12.0)
|
|
2
|
+
|
|
3
|
+
## Changed behaviour
|
|
4
|
+
- `fair run` refuses a configuration with no `description` in `run_metadata`, or a blank one. So do `fair run --ci`
|
|
5
|
+
and `--rehearse`. The registry needs a description of every code run, and without one the run's API failed later.
|
|
6
|
+
The refusal comes before anything is registered. `fair pull` needs no description.
|
|
7
|
+
- `fair run` and `fair pull` check each name that a `write:` or `register:` entry would register. They refuse a name
|
|
8
|
+
that breaks the naming convention before registering anything, and name the entry. A namespace holds ASCII letters,
|
|
9
|
+
digits, `_` and `-`. A data product's name is one or more such parts joined by single `/`. A part may also hold
|
|
10
|
+
`.`, but not at its start or end. A `read:` entry is not checked.
|
|
11
|
+
- `fair registry install` without `--directory` now looks for the registry in the directory the configuration names.
|
|
12
|
+
With `--force` it replaces the registry there. Before, it installed in `~/.fair/registry` whatever the
|
|
13
|
+
configuration said. When a registry is installed already, the refusal names `fair registry upgrade`.
|
|
14
|
+
- `fair init --using <file>` and `fair init --ci` keep a machine's global configuration where there is one. Before,
|
|
15
|
+
they replaced its user, namespaces, remote, local registry and data store for every project on the machine. Their
|
|
16
|
+
settings are now the project's alone, and the project uses the machine's local registry. A file naming another
|
|
17
|
+
local registry, port or data store is refused; a project's own store is `run_metadata: write_data_store`.
|
|
18
|
+
- A `read:` entry's `use: cache` names your own copy of the data product's file, such as a private one the remote
|
|
19
|
+
cannot send. `fair pull` now pulls the entry as any other read, and copies that file into the local data store in
|
|
20
|
+
place of the download. It refuses a file without the hash the remote records. Before, it pulled nothing for the
|
|
21
|
+
entry, and `fair run` let it through unregistered. A wildcard `read:` entry with `cache` is refused.
|
|
22
|
+
- A `write:` pattern's version token is now bumped from the highest version of any name the pattern matches, and
|
|
23
|
+
every name written under the pattern takes that version. Before, a name already registered was bumped from its own
|
|
24
|
+
highest version, so it could be written below another name under the pattern. A `write:` entry for one name now
|
|
25
|
+
takes the place of a pattern's entry for that name. Two `write:` patterns matching one registered name are refused.
|
|
26
|
+
- `fair add` refuses a code run that records no inputs or outputs. A run records them as it finishes, so such a run
|
|
27
|
+
is still running, or stopped before it finished.
|
|
28
|
+
|
|
29
|
+
## Added
|
|
30
|
+
- `fair add --all` stages every finished code run of this repository that the remote lacks, or holds only in part.
|
|
31
|
+
`--remote` names the remote, `origin` by default. A run has finished once it records inputs or outputs. It is
|
|
32
|
+
refused while another process holds a session on the local registry, since that may be a run in progress.
|
|
33
|
+
- `fair rm <uuid>...` deletes code runs from the local registry and from the project's staging. It refuses a run the
|
|
34
|
+
remote holds (`--local` does not ask it), and a run whose outputs another run read, unless that run is deleted too.
|
|
35
|
+
It asks first (`--yes` answers for you). What the runs wrote stays registered. Before, `fair rm` did nothing.
|
|
36
|
+
- `fair list code-runs` lists every code run, not the first ten. It shows whether each has finished, whether the
|
|
37
|
+
project has staged it, and how much of it `origin` holds. `--json` prints them as JSON, for a program to read.
|
|
38
|
+
- `fair log -n <count>` lists the newest `<count>` jobs, and `fair log --all` every one. Ten is still the default.
|
|
39
|
+
- `fair registry erase-unknown-data` erases the files in the local registry's data stores that the registry does not
|
|
40
|
+
hold. It runs only while the registry is stopped. It names each store and warns that another registry using it may
|
|
41
|
+
hold the files, then erases them only once the store's path is typed. `--dry-run` lists what it would erase. It keeps
|
|
42
|
+
any file that a run in progress could still be writing, unless given `--delete-all`. We recommend a data store for
|
|
43
|
+
each local registry.
|
|
44
|
+
- `fair registry reset` empties the local registry. It deletes every record and keeps the install. It asks first
|
|
45
|
+
(`--yes` answers for you). It registers the file types again, and restarts the registry if it was running. If it
|
|
46
|
+
fails, or is stopped by Ctrl-C, the registry keeps its records. Run in a project, it also registers the project's
|
|
47
|
+
user again and clears the project's staging. `--delete-files` also deletes the files the registry held in its data
|
|
48
|
+
stores.
|
|
49
|
+
- `fair registry upgrade` moves the local registry to another version: the newest release, or the one given by
|
|
50
|
+
`--version`. It keeps the registry's records if the new version's tables are the same. If they differ, it leaves
|
|
51
|
+
the registry as it was, as it does if it fails or is stopped by Ctrl-C. Given the installed version, it rebuilds the
|
|
52
|
+
registry. That mends one whose Python has gone.
|
|
53
|
+
- While `fair registry erase-unknown-data`, `reset`, `upgrade` or `install --force` runs, any other command that would
|
|
54
|
+
start, stop or use the registry is refused. The refusal names the command that is running, and so does
|
|
55
|
+
`fair registry status`.
|
|
56
|
+
- `fair push` and `fair pull` send the records attached to what they send. These are named components, licences,
|
|
57
|
+
keywords, key-values, quality-control records, code releases, and the issues on components. A record added after its
|
|
58
|
+
object was sent goes with the next push or pull of that object. If the other registry holds a record with other
|
|
59
|
+
values, the record there is kept as it is, with a warning.
|
|
60
|
+
- An issue is not sent again if the other registry holds it already. It is found there by its UUID, or else by its
|
|
61
|
+
severity and description on the same component.
|
|
62
|
+
- `fair push` warns of earlier code runs that the remote does not hold. Each such run wrote a pushed data product, an
|
|
63
|
+
input of a pushed run, or an input of another such run. The warning names the `fair add` command that would send it.
|
|
64
|
+
|
|
65
|
+
## Fixed
|
|
66
|
+
- `fair push` took time growing with the square of the number of data products a code run read: about 23 hours for a
|
|
67
|
+
run that read 600. A push now sends or checks each data product once, and links each code run once, after the data
|
|
68
|
+
products are sent. A push that fails links the runs of what it sent before the failure.
|
|
69
|
+
- A code run that read or wrote a named part (a component) of a data product could not be pushed. The push failed
|
|
70
|
+
with "Failed to access component ... on remote registry". A run is now recorded on the remote as reading and
|
|
71
|
+
writing the parts it did. Before, a run pushed by itself was given every part of each file it used.
|
|
72
|
+
- When two machines ran one commit of a code repository and pushed to the same registry, the registry got two objects
|
|
73
|
+
for that commit. The repository's release was on only one of them. If the machines registered the repository at
|
|
74
|
+
different paths, the push failed instead. A push or pull now gives a run the object that the other registry holds
|
|
75
|
+
for its commit. If there are several, it takes the one with a release, else the oldest, as DataPipeline.jl does
|
|
76
|
+
locally. A release registered as a data product gets the same object.
|
|
77
|
+
- A push that reached two code runs the remote held already could give the second run the first's outputs. Registry
|
|
78
|
+
v1.5.0 refuses that, so the push failed.
|
|
79
|
+
- `fair push` and `fair pull` took a data product on the other registry with the same namespace, name and version to
|
|
80
|
+
be the same one, whatever its file or source. A push then left the remote's data product in place without a word.
|
|
81
|
+
A run that used it failed with "Failed to access [] on remote registry". Both commands now check every data product
|
|
82
|
+
they would send before sending anything. They refuse one held there with other contents, and name each side's file
|
|
83
|
+
or source.
|
|
84
|
+
- With a web proxy set in the environment (`http_proxy`, `https_proxy` or `all_proxy`), every request to the local
|
|
85
|
+
registry went to the proxy. Commands then failed with an error naming the registry's token or another server.
|
|
86
|
+
Requests to this machine now go to it directly. Requests to a remote registry or an identifier service still use
|
|
87
|
+
the proxy.
|
|
88
|
+
- `fair add` failed unless the local registry was running. It now starts the registry, and stops it afterwards if it
|
|
89
|
+
started it. `fair add ""` staged an empty name, and is now refused.
|
|
90
|
+
- A code run deleted from the local registry stayed in the project's staging. `fair add` staged it again, and
|
|
91
|
+
`fair push` then failed naming it. `fair status`, `fair add` and `fair push` now remove it from staging and say so.
|
|
92
|
+
`fair add` refuses it, and a push sends the rest.
|
|
93
|
+
- Shell completion of a data product for `fair add` offered nothing: it ran `fair list` and stopped. It now offers
|
|
94
|
+
the data products the project's staging names.
|
|
95
|
+
- A registry search dropped any filter set to `false` or `0`, so it found everything.
|
|
96
|
+
- With no `git` on the path, every command failed with GitPython's own error, even `fair --version` and `--help`.
|
|
97
|
+
These two now run. A command that needs `git` says so, and names `GIT_PYTHON_GIT_EXECUTABLE`.
|
|
98
|
+
- `fair registry install --version` took a branch only as `origin/<branch>`. It now takes the branch's name. Given a
|
|
99
|
+
version that the registry's repository does not have, it stopped the running registry before refusing. It now
|
|
100
|
+
checks the version before stopping anything.
|
|
101
|
+
- `fair log` stopped with "Cannot open log for job" at the first job that another repository ran with the same data
|
|
102
|
+
store. By default every project on a machine shares one data store. `fair log` now lists this repository's jobs,
|
|
103
|
+
then says how many others it skipped and which repositories ran them. It lists at most 10 jobs, not 11. If this
|
|
104
|
+
repository has run no jobs, it says so.
|
|
105
|
+
- `fair view` stopped with a Python traceback when given a job that another repository ran. It now says it found no
|
|
106
|
+
such job, and names `fair log`.
|
|
107
|
+
- `fair log` and `fair remote` printed some text in white or yellow, which is hard or impossible to read on a light
|
|
108
|
+
background. A job's id and date, and a remote's name, are now bold in the terminal's own colour. A remote's address
|
|
109
|
+
is plain.
|
|
110
|
+
- `fair init --using` failed on a file with a top-level `description`, as earlier documented examples had. It now
|
|
111
|
+
ignores the key. A registry in the file with no `data_store` is now refused naming that key, not another.
|
|
112
|
+
- A user whose identifier, such as an ORCID, the local registry held under another spelling of the name could not
|
|
113
|
+
register: `fair init` and `fair pull` stopped with "Expected to receive a URL location from registry post". The
|
|
114
|
+
registry's author with that identifier is now used, with a warning naming both spellings.
|
|
115
|
+
|
|
116
|
+
## Development
|
|
117
|
+
- The implementations workflow no longer installs `libmemcached-dev`, which the registry has not needed since before
|
|
118
|
+
v1.0.0. So the Python, Java and Julia jobs no longer need Ubuntu's package archive, which GitHub's runners could
|
|
119
|
+
not reach for a time.
|
|
120
|
+
- The C++ job of the implementations workflow no longer upgrades the runner's own packages, which it does not use.
|
|
121
|
+
- A storage test no longer checks its DOI online. It failed when the site the DOI leads to closed the connection.
|
|
122
|
+
|
|
123
|
+
# 2026-10-19 [v0.11.0](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.11.0)
|
|
124
|
+
|
|
125
|
+
## Changed behaviour
|
|
126
|
+
- A command that starts the local registry stops it again when the last command using it finishes; it used to leave
|
|
127
|
+
it running. Each command holds a session on the registry while it runs. `fair registry start` keeps it running
|
|
128
|
+
until `fair registry stop`, and run while a command has it running keeps that registry rather than refusing.
|
|
129
|
+
- `fair registry start` with no `--port`, and a command that starts the registry, use the configured port, where
|
|
130
|
+
they used 8000 whatever the configuration said.
|
|
131
|
+
- An install's registry runs on one host at a time: where its directory is shared between machines, starting,
|
|
132
|
+
stopping or using it from another machine is refused, naming the machine running it and the file that records it.
|
|
133
|
+
- `fair purge --all`, `fair registry uninstall` and `fair registry install --force` stop the registry first, and
|
|
134
|
+
remove nothing while a command is using it or it runs on another machine.
|
|
135
|
+
- Two versions of the CLI sharing one registry install are not supported: one of an earlier version neither records
|
|
136
|
+
its sessions nor respects this one's.
|
|
137
|
+
- Records left behind when the registry or a session ends outside `fair` - a restart, a crash, a killed process - are
|
|
138
|
+
removed with a line saying what was removed and why, naming `fair registry start` where that had started the
|
|
139
|
+
registry. A record left half-written is removed the same way. One that cannot be removed - on Windows, a file
|
|
140
|
+
another program has open - is waited for briefly, then named and left, and the command carries on; one that
|
|
141
|
+
cannot be written is waited for the same way, then is an error naming it. A session whose process cannot be
|
|
142
|
+
checked is taken to be running, and says so where it stops the registry being stopped.
|
|
143
|
+
- `fair init` asks for the user's identifier before their email, and offers as the email the address the
|
|
144
|
+
identifier's record makes public - an ORCID's or a GitHub account's. Anything answering its questions in order
|
|
145
|
+
must put the email after the identifier.
|
|
146
|
+
- A `read:` wildcard's own entry is kept in the working config beside the entries it expands to, as a `write:`
|
|
147
|
+
wildcard's is, so that an API can tell a pattern that matched nothing from one never given. `fair pull` fetches
|
|
148
|
+
nothing for it. Its version - its matches' highest, the version it names, or 0.0.0 - means nothing.
|
|
149
|
+
- A `*` in a wildcard no longer matches `[`, `]`, `?` or `*`, which the registry refuses in a name, so a pattern
|
|
150
|
+
never matches itself.
|
|
151
|
+
|
|
152
|
+
## Added
|
|
153
|
+
- `fair registry session open --pid <pid> --json` and `fair registry session close <id>`, for a program using the
|
|
154
|
+
registry outside a `fair` command, such as a language API. A script run by `fair run` is told the run's session
|
|
155
|
+
in `FDP_SESSION`.
|
|
156
|
+
- `fair registry status` says who holds the registry and which processes are using it.
|
|
157
|
+
- `fair run --rehearse` prepares a job as a run would without running it - no git check, nothing staged - and prints
|
|
158
|
+
its files as one JSON object, `{"job_dir", "config", "script", "data_store", "token_file"}`, for a language API's
|
|
159
|
+
rehearsal of a run.
|
|
160
|
+
- The README says which parts of the CLI's configuration a program such as a language API may read.
|
|
161
|
+
- A `register:` entry for an external object may have `no_file: true` in place of `root`, `path` and `file_type`,
|
|
162
|
+
for a source with no file of its own, such as a data store a run queries: `fair pull` registers it as a data
|
|
163
|
+
product with no file, which a run may read. Its `version` must be given.
|
|
164
|
+
- `run_metadata.shell` may give flags for the shell's program after its name: `shell: julia --project=@.` runs a
|
|
165
|
+
Julia script in the repository's project. A shell the CLI does not know - an empty `shell:` included - is
|
|
166
|
+
refused, naming those it does, where it stopped with a `KeyError` or a `TypeError`.
|
|
167
|
+
|
|
168
|
+
## Fixed
|
|
169
|
+
- `fair purge --all` answered "no" removed the project's `.fair` folder, and with it a data store kept there; it now
|
|
170
|
+
removes nothing. `fair purge --yes` asked its question anyway, and now answers every question as the README says.
|
|
171
|
+
- `fair registry install --force` removed the old registry before it had the new one, so a `--version` that does not
|
|
172
|
+
exist left no working registry. The old one is now kept until the new one is built, and put back if the install
|
|
173
|
+
fails; a first install that fails leaves nothing behind.
|
|
174
|
+
- `fair push` and `fair pull` left out a code repository's release (`CodeRepoRelease`), so the remote's record of a
|
|
175
|
+
run or a data product lost it. The release now goes with its repository's object.
|
|
176
|
+
- A data product on a code repository's object, such as a release of the code registered with its DOI, was pushed
|
|
177
|
+
with the repository's web page as its file, and could not then be pulled. Its location is the repository at a
|
|
178
|
+
commit, and it is now pushed and pulled as that, with nothing downloaded.
|
|
179
|
+
- A `register:` entry with no version was registered again at the next patch version on every `fair pull`, whether
|
|
180
|
+
or not its file had changed, and `fair run` then read a version that did not exist (#317). The highest version is
|
|
181
|
+
now kept while the file is the same, and `fair run` reads it. `${{LATEST}}` on such an entry keeps the highest
|
|
182
|
+
version and refuses a changed file.
|
|
183
|
+
- `fair run` did not check that `fair pull` had registered what a `register:` entry names, so the model found out;
|
|
184
|
+
it now refuses to start.
|
|
185
|
+
- A ROR identifier (`https://ror.org/...`) of an author or a source was checked by fetching ROR's website, which
|
|
186
|
+
refuses automated requests that come too often (429), and was then rejected. It is now checked through ROR's API.
|
|
187
|
+
- On Windows, `fair registry start` could fail - no response, or no token - when the registry's start script gave up
|
|
188
|
+
waiting for the server, leaving the server running. The CLI now waits for the server itself, up to 30 seconds, and
|
|
189
|
+
writes the registry's token if the script did not.
|
|
190
|
+
- A command that crashed, or a reboot after `fair registry start`, left a session file behind that made every later
|
|
191
|
+
command skip starting the registry and `fair registry stop` refuse. A session now names its process, and one whose
|
|
192
|
+
process has gone is removed. This needs a new dependency, `psutil`.
|
|
193
|
+
- A failed `fair registry start` left a session file behind, and a server on the port that never answered was waited
|
|
194
|
+
on for ever; neither now happens. Another install's registry on the port is named as such.
|
|
195
|
+
- An internal error, and a registry's refusal of a request (HTTP 403), printed their message and exited 0, so a
|
|
196
|
+
script or a CI step saw success. Both now exit non-zero, the refusal shown as an error.
|
|
197
|
+
- `fair pull` of a `register:` entry at a version given outright, registered already with another file, copied the
|
|
198
|
+
new file over the registered one before the registry refused it, so the registered file no longer matched its
|
|
199
|
+
record. It is now refused before anything is copied; the same file is passed over, as before.
|
|
200
|
+
- `fair init --ci` named the remote user `FAIRDataPipeline`, which a freshly built remote registry does not have, so
|
|
201
|
+
`fair push` stopped. It now takes the user the remote's token belongs to, where the remote answers.
|
|
202
|
+
- `fair run` refused a detached checkout - a tag, a commit, a pull request on CI - unless given `--dirty`. It now runs
|
|
203
|
+
where the commit is one the remote has, on a branch or a tag, and refuses one the remote does not have.
|
|
204
|
+
`${{GIT_BRANCH}}` there is an error saying there is no branch.
|
|
205
|
+
- A `fair init` that failed - a token the remote refused, a registry that would not start, Ctrl-C other than at a
|
|
206
|
+
question, a `--using` file it refused - left `.fair` and an empty global configuration behind, so `fair init` again
|
|
207
|
+
failed too (#272). It now puts back what it changed, and `fair init` starts afresh where an earlier one left `.fair`
|
|
208
|
+
unfinished, or the global configuration empty.
|
|
209
|
+
- `fair init` failing in a subdirectory of a project could write the configuration it had refused over the project's;
|
|
210
|
+
the project's is now left as it was.
|
|
211
|
+
- `fair init` in a subdirectory of a project made the project's staging file and not its own, so a later command
|
|
212
|
+
there that staged something failed; it now makes its own.
|
|
213
|
+
- A local registry that answered every request with an internal error (HTTP 500) was taken for one that works, and a
|
|
214
|
+
command failed at its first request. Starting it, or a command that finds it running, now says so and names its
|
|
215
|
+
log; `fair registry stop` still stops it.
|
|
216
|
+
- `fair init` with an ORCID whose record keeps its holder's name private, or gives no family name, stopped with a
|
|
217
|
+
`TypeError`. It now asks for a name the record does not give, and takes a missing family name as empty. An author
|
|
218
|
+
in a `register:` entry whose ORCID keeps its name private is registered by its identifier. A one-word name - given
|
|
219
|
+
with no identifier, or to `fair config user.name` - failed the configuration's validation and broke the job logs;
|
|
220
|
+
its family name is now empty too.
|
|
221
|
+
|
|
222
|
+
# 2026-10-05 [v0.10.2](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.10.2)
|
|
223
|
+
|
|
224
|
+
## Changed behaviour
|
|
225
|
+
- `fair pull` refuses a `register:` entry for an external object whose identifier or unique name, title and
|
|
226
|
+
release version are those of a source already registered from a different file: a source is one file. Give
|
|
227
|
+
the entry a `title` of its own, or its `release_version`; the same file under another name still registers.
|
|
228
|
+
With a local registry before data-registry v1.4.0, where each data product has its own source, nothing is
|
|
229
|
+
checked.
|
|
230
|
+
- `fair init` in a repository that is already initialised registers its user in the local registry, where it
|
|
231
|
+
did nothing. A registry reinstalled since the repository was initialised has no record of the user, and a
|
|
232
|
+
run failed until the repository was purged and initialised afresh. The repository itself is left as it is.
|
|
233
|
+
|
|
234
|
+
## Added
|
|
235
|
+
- A `register:` entry for an external object may give `release_version`, the version of the source it was taken
|
|
236
|
+
from, beside `release_date`. A local registry at data-registry v1.4.0 or later records it as the external
|
|
237
|
+
object's version, 1.0.0 without it; an earlier one records the data product's version there whatever is
|
|
238
|
+
given. It is not the entry's `version`, which is the data product's.
|
|
239
|
+
|
|
240
|
+
## Fixed
|
|
241
|
+
- A GitHub author or user was refused as "not a recognised github" once the GitHub API's limit for
|
|
242
|
+
anonymous requests (60 an hour per address) was spent. GitHub lookups now send the token in `GITHUB_TOKEN`
|
|
243
|
+
or `GITHUB_PAT` if set, retry without it if GitHub refuses it (expired or revoked), and say when the rate
|
|
244
|
+
limit is the problem.
|
|
245
|
+
- The FAIR-CLI binaries could not install a local registry, so `fair init` and `fair registry install` failed:
|
|
246
|
+
the registry's virtual environment was built from the binary in place of a Python. They now use a Python
|
|
247
|
+
>= 3.10 from `FAIR_PYTHON` or the `PATH`, or have `uv` fetch one, and say so if there is none. A missing
|
|
248
|
+
Python is found before anything is installed.
|
|
249
|
+
- `fair push` failed, with "Failed to access [] on remote registry", for a data product registered from a web
|
|
250
|
+
address once a code run had used it, when the local registry was data-registry v1.4.0. The remote records
|
|
251
|
+
such a file in two places with one hash - where it is stored and where it came from - and the wrong one
|
|
252
|
+
could be taken.
|
|
253
|
+
- `fair registry install --force` refused an existing install, as it does without `--force`, instead of
|
|
254
|
+
replacing it.
|
|
255
|
+
- A file that was already registered was not registered again under a second data product name, or in a
|
|
256
|
+
second namespace: `fair pull` passed over the entry without a word, and a run that read it then failed. An
|
|
257
|
+
entry is now passed over only when that data product, in that namespace and at that version, already holds
|
|
258
|
+
the file.
|
|
259
|
+
- `fair push` and `fair pull` left out a data product that shares its external object with another, as
|
|
260
|
+
data-registry v1.4.0 allows, while reporting it synchronised: its file was moved and its record was not made.
|
|
261
|
+
- Starting the local registry reported success when another registry already held its port, and the commands
|
|
262
|
+
that followed went to that one. The start now fails, naming the address: the server that answers there
|
|
263
|
+
must accept the installed registry's token.
|
|
264
|
+
- `fair push` and `fair pull` stopped with a Python error at a data product that has no file - one whose
|
|
265
|
+
object has no storage location, as an entry for a deposit as a whole may be - and so at a code run that
|
|
266
|
+
read one. Its records are now synchronised, and a `register:` entry of the same name is told it exists.
|
|
267
|
+
- `fair push` gave a code run an input it did not read when two data products held the same file, as one
|
|
268
|
+
file registered under two names does: a run that read one of them and wrote something new arrived on the
|
|
269
|
+
remote as having read both. A run pushed before this keeps the extra input.
|
|
270
|
+
- A file registered under a second data product name, or in a second namespace, was copied into the data store
|
|
271
|
+
again, though the registry records a file once there and nothing referred to the copy. The copy is no
|
|
272
|
+
longer kept: the data product's file is the one already in the store.
|
|
273
|
+
- A data product pulled from a remote could not be read through an API. `fair pull` copied the remote's record
|
|
274
|
+
of where the file is - the remote's own store - and fetched the file to a place no record named, so a model
|
|
275
|
+
was handed an address and no file. A pulled file is now recorded at its place in the local data store, and
|
|
276
|
+
a file that two data products share is fetched once. Data products pulled with an earlier version keep
|
|
277
|
+
the old record.
|
|
278
|
+
|
|
279
|
+
## Development
|
|
280
|
+
- The test of a data product's dependencies allows for any a registry adds; data-registry v1.4.0 adds one.
|
|
281
|
+
|
|
282
|
+
# 2026-10-05 [v0.10.1](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.10.1)
|
|
283
|
+
|
|
284
|
+
## Changed behaviour
|
|
285
|
+
- A wildcard entry keeps to one namespace: the one its `use:` names, else the default for its block. It
|
|
286
|
+
used to match names in every namespace, so a `write:` pattern could write a name into a namespace
|
|
287
|
+
other than its own.
|
|
288
|
+
- A wildcard entry gives one entry for each name it matches: for a `read:`, the version its `use:` names,
|
|
289
|
+
else the highest; for a `write:`, the version to be written. It used to give one for every version, and
|
|
290
|
+
to ignore a version on a `read:`. A `read:` pattern that matches nothing is not an error.
|
|
291
|
+
- `fair run` refuses a `read:` of a data product, or of a version of one, that is not in the registry,
|
|
292
|
+
naming the namespace and registry it looked in. Such a read used to pass unchecked, or as version 0.0.0.
|
|
293
|
+
- `fair push` fails if a file cannot be uploaded. It used to warn and carry on, leaving the remote with
|
|
294
|
+
the file's records and no file. A file is now uploaded before its records are written, so a push that
|
|
295
|
+
failed can be run again.
|
|
296
|
+
- `fair pull` checks each file it fetches against the hash the remote registry holds for it, and refuses
|
|
297
|
+
one that differs.
|
|
298
|
+
|
|
299
|
+
## Added
|
|
300
|
+
- Registering an external object records where its file was fetched from: the `root` and `path` of its
|
|
301
|
+
`register:` entry, as the object's `original_store`, with the hash of the file found there. `fair push`
|
|
302
|
+
takes that record to the remote. A file registered from the machine itself has no such record.
|
|
303
|
+
- `fair push` takes a refusal of an upload address with HTTP 409 to mean that the remote's store already
|
|
304
|
+
holds the file, sends nothing, and carries on. No released registry answers so yet.
|
|
305
|
+
|
|
306
|
+
## Fixed
|
|
307
|
+
- `fair push` sent no value that was false, leaving the remote to apply its default: an external object
|
|
308
|
+
registered with `primary: false` arrived as primary, and a location with `public: false` as public.
|
|
309
|
+
- A list from a registry stopped at its first 100 rows, so wildcards, staging and version lookups went
|
|
310
|
+
wrong, with no message, in a registry holding more.
|
|
311
|
+
- `fair pull` looked for the version of a `read:` entry in the local registry only, so an entry at the
|
|
312
|
+
default version, for something not yet local, asked the remote for version 0.0.0 and failed.
|
|
313
|
+
- `fair push` held each file whole in memory, `fair pull` more than twice over, and both left a copy of
|
|
314
|
+
it in the temporary directory. Files are now sent and received a block at a time, whatever their
|
|
315
|
+
size, and nothing is left behind.
|
|
316
|
+
- An upload of 2 GiB or more over plain http failed on macOS.
|
|
317
|
+
- An external object could not be registered with a `unique_name` and `alternate_identifier_type` in
|
|
318
|
+
place of an `identifier`: `fair pull` stopped, reporting that the object already existed.
|
|
319
|
+
- `fair identify` looked at only the newest of a file's storage locations, so it could miss the data
|
|
320
|
+
products of a file the registry records in more than one place.
|
|
321
|
+
- `fair run` could not start the script when the path of the data store, which holds the job
|
|
322
|
+
directory, had a space in it.
|
|
323
|
+
|
|
324
|
+
## Development
|
|
325
|
+
- The Python job of the implementations workflow starts an object store for its remote registry, so
|
|
326
|
+
that its `fair push` uploads the files. It had none, and passed while a failed upload was a warning.
|
|
327
|
+
|
|
328
|
+
# 2026-10-03 [v0.10.0](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.10.0)
|
|
329
|
+
|
|
330
|
+
## Changed behaviour
|
|
331
|
+
- `fair remote add` takes the remote's token file with `--token FILE`, and prompts for it if omitted.
|
|
332
|
+
The command never worked before: the label was stored as the token path.
|
|
333
|
+
- `fair run --ci` prepares the job directory and prints it without running the script, as documented.
|
|
334
|
+
- `fair init --ci` leaves an existing `.fair/` alone, as `fair init` does; it used to delete it, data
|
|
335
|
+
store included. Use `fair purge` to start afresh.
|
|
336
|
+
- Wildcards: each `*` matches one segment of a name (not `/`), anchored at both ends, in `read:` and
|
|
337
|
+
`write:` alike, matching DataPipeline.jl. `data_product/*` no longer matches `data_product/1/2`.
|
|
338
|
+
- `${{GIT_TAG}}` is the nearest tag in the history of the current commit (`git describe`), not the
|
|
339
|
+
last tag by name; with no such tag it is an error rather than a crash.
|
|
340
|
+
- `run_metadata.shell` now chooses the interpreter the script runs with.
|
|
341
|
+
|
|
342
|
+
## Fixed
|
|
343
|
+
- `write:` entries with a wildcard were dropped from the working config (#140). The pattern is now kept
|
|
344
|
+
for new names, and existing matches are written as the pattern entry describes them (its `use: version`,
|
|
345
|
+
`file_type`, `description`).
|
|
346
|
+
- `fair push` from a project whose data store was not the local registry's first storage root recorded
|
|
347
|
+
its files on the remote under the pusher's `file://` path (data-registry #229).
|
|
348
|
+
- `fair pull` of a `read:` entry from a remote registry failed.
|
|
349
|
+
- A DOI whose publisher refuses automated requests (e.g. Wiley, HTTP 403) was rejected as an invalid
|
|
350
|
+
identifier. An identifier that resolves is now accepted; one that does not (404) is still refused.
|
|
351
|
+
- `${{DATETIME-<format>}}` always failed with "Failed to parse formatted datetime variable".
|
|
352
|
+
- Substituting a path with backslashes, such as `${{CONFIG_DIR}}` on Windows, failed `fair pull` and `fair run`
|
|
353
|
+
(#267). Variables are now substituted into the configuration's values, so any text is inserted as it is.
|
|
354
|
+
- `fair remote remove` crashed.
|
|
355
|
+
- A ROR or GRID ID the ROR API could not parse (e.g. one containing `!`) crashed with `KeyError`
|
|
356
|
+
instead of being reported as not found.
|
|
357
|
+
- Run from a subdirectory of a project, commands lost configuration changes and left a stale session
|
|
358
|
+
file, so the next command did not start the registry and `fair registry stop` needed `--force`.
|
|
359
|
+
- Staging ignored the configured local registry and always used port 8000.
|
|
360
|
+
- A download that failed with an HTTP error registered the error page as the data.
|
|
361
|
+
- `fair run` could crash on Windows when the model printed a character the console cannot encode.
|
|
362
|
+
- `fair run` in a git repository without the configured remote gave a bare `IndexError`; it now says
|
|
363
|
+
which remote is missing, or reads `run_metadata: remote_repo:` if given.
|
|
364
|
+
- `fair init` in a subdirectory of a FAIR repository named it as `'True'`.
|
|
365
|
+
|
|
366
|
+
## Development
|
|
367
|
+
- CI ran no tests: an unused dev dependency (`pylama`) crashed `pytest --markers`, leaving the test loop
|
|
368
|
+
empty. It is removed, and a failure to list the tests now fails the job; all 8 jobs run the full suite.
|
|
369
|
+
- numpy is chosen per Python version (>= 2.3 from 3.11), so Python 3.14 gets a wheel, not a source build.
|
|
370
|
+
- Download tests use a local HTTP server rather than github.com.
|
|
371
|
+
- `test_configuration.py::test_local_config_query` is marked `xfail`: it has never passed, and its intent
|
|
372
|
+
is unknown.
|
|
373
|
+
|
|
374
|
+
# 2026-09-22 [v0.9.9](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.9.9)
|
|
375
|
+
- Requires Python >= 3.10, in step with data-registry v1.2.2.
|
|
376
|
+
- `${{RUN_ID}}` in a `write:` name is left for the language API to fill in at `finalise`.
|
|
377
|
+
- ROR/GRID organisation lookups failed after the ROR API moved to its v2 schema.
|
|
378
|
+
|
|
379
|
+
# Between v0.2.3 and v0.9.8
|
|
380
|
+
Not recorded release by release; see the [releases](https://github.com/FAIRDataPipeline/FAIR-CLI/releases)
|
|
381
|
+
and the git log.
|
|
382
|
+
- Fix virtual environment bug in `fair` binaries where `venv` assumes the main executable is `python`.
|
|
383
|
+
- Registration of `author` and `namespace` objects.
|
|
384
|
+
- Allow specification of tag to install registry from during `fair registry install`.
|
|
385
|
+
- Wildcard '*' parsing introduced for data products.
|
|
386
|
+
- Ability to `push` to a registry added.
|
|
387
|
+
- Added `--dirty` option to `fair run` to allow running with uncommitted changes.
|
|
388
|
+
- Added `config.yaml` file validation.
|
|
389
|
+
- Added initialisation from existing registry.
|
|
390
|
+
- Switch to setting port not local URI during initialisation.
|
|
391
|
+
- Added option to specify port on `fair registry start`.
|
|
392
|
+
- Added `cli-config.yaml` file validation during initialisation.
|
|
393
|
+
- Added `fair reset` to list of commands.
|
|
394
|
+
- Add tab completion for staging data products.
|
|
395
|
+
|
|
396
|
+
# 2021-11-17 [v0.2.3](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.3)
|
|
397
|
+
- Move handling of the user `config.yaml` file to a separate class `JobConfiguration`.
|
|
398
|
+
- Added various fixes to improve functionality within Windows.
|
|
399
|
+
- Move registry installation from script execution to internal function which sets up virtual environment etc.
|
|
400
|
+
- Added a test suite to the project.
|
|
401
|
+
- Added additional recognised identifiers for author setup from an organisation GRID and ROR.
|
|
402
|
+
|
|
403
|
+
# 2021-10-06 [v0.2.2](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.2)
|
|
404
|
+
- Update to package metadata for PyPi
|
|
405
|
+
|
|
406
|
+
# 2021-10-06 [v0.2.1](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.1)
|
|
407
|
+
- Automatic starter `config.yaml` generation.
|
|
408
|
+
- Local and Global CLI configurations.
|
|
409
|
+
- Start/stop local registry either explicitly or during synchronisations.
|
|
410
|
+
- Run logs available via git-like interface.
|
|
411
|
+
- Added ability to add/remove files.
|
|
412
|
+
- Repository style handling, acts like another git-like tool per project.
|
|
413
|
+
- Creation of an interface for `fair` using `click`.
|
|
@@ -26,7 +26,7 @@ authors:
|
|
|
26
26
|
affiliation: University of Glasgow
|
|
27
27
|
orcid: https://orcid.org/0000-0002-4424-9890
|
|
28
28
|
|
|
29
|
-
date-released: 2026-
|
|
29
|
+
date-released: 2026-11-11
|
|
30
30
|
keywords:
|
|
31
31
|
- FAIR Data Pipeline
|
|
32
32
|
- FAIR
|
|
@@ -36,4 +36,4 @@ license: BSD-2-Clause
|
|
|
36
36
|
message: If you use this software, please cite it using these metadata.
|
|
37
37
|
repository-code: https://github.com/FAIRDataPipeline/FAIR-CLI/
|
|
38
38
|
title: "The FAIR Data Pipeline command line tool"
|
|
39
|
-
version: 0.
|
|
39
|
+
version: 0.12.0
|