qedro 0.2.0__tar.gz

qedro-0.2.0/CHANGELOG.md

@@ -0,0 +1,158 @@
+# Changelog
+
+Notable changes, in the words of somebody deciding whether to upgrade.
+
+`0.x` may change interfaces between releases. The entries below say which, and
+output formats and the `qedro.yaml` schema are the parts most likely to move
+before `1.0`.
+
+## 0.2.0 — 2026-09-18
+
+> **First public release**, and the first version on PyPI.
+>
+> **0.1.0 was never released.** It was tagged locally on 2026-08-22 and deliberately
+> not published, so that Qedro could first be run against real work. That run found
+> the orchestration-parent, guessed-domain, Art. 30(1), file-naming and read-only
+> changes below, and publishing 0.1.0 would have shipped a record that overstated
+> what it covers. Everything that was in 0.1.0 is included in this entry, and there
+> is no separate 0.1.0 to upgrade from.
+
+First release. Three projections over OpenLineage, four output formats, and no
+adoption of anything required beyond lineage a platform already emits.
+
+### The reader
+
+- `qedro events` reads a directory of `.json`, `.ndjson` or `.jsonl`, walking
+  nested directories, or any Marquez-compatible HTTP API. Any spelling of the
+  address works — a bare host, `.../api/v1`, or the full endpoint.
+- The window is passed to the backend **and applied again to what comes back**,
+  so a backend that ignores it produces a correct result rather than a quietly
+  wider one.
+- Reading stops after 10,000 events and says so, rather than returning a prefix
+  of the history that looks like all of it.
+- Malformed records are skipped and counted, never raised on, and the count
+  reaches the completeness decision.
+- Read-only by construction: `GET` is the only verb the code can build. A token
+  comes from `QEDRO_API_TOKEN` and never from the command line, where it would
+  land in shell history and in `ps`.
+
+### `qedro ropa` — the Art. 30 record
+
+- One activity per job, with the purpose and lawful basis that job declared and
+  **where each field came from**. A value from an emitted facet is evidence; the
+  same string from `qedro.yaml` is an assertion, and the two never merge.
+- A facet always beats the mapping file. A config that could silently rewrite
+  emitted evidence would make the record unfalsifiable.
+- `qedro.yaml` — controller, domains and job patterns — read as YAML, JSON or
+  TOML, with no behaviour depending on which.
+- The Art. 30 processing facet is published as a spec in
+  `schemas/openlineage-art30-processing-facet.json`, generated from
+  `pipeline_runtime.descriptor.Processing` rather than written beside it.
+- **Orchestration parents are not listed as activities.** A job whose run is named
+  as a parent through the standard `ParentRunFacet`, and which read and wrote nothing
+  and carries no `processing` facet, is named in the scope statement instead. The job
+  count still includes it, and it does not withhold the mark. Found against real dbt
+  lineage, where the invocation job showed up as an extra activity.
+- **Activities that read datasets and wrote none are named** on a `read only` line in
+  the Art. 30 scope statement, and in JSON as `scope.read_only`. They stay listed, and
+  the mark is not withheld for them. Found against real Spark lineage, where the
+  integration emits a job for each schema-reading action.
+- **File datasets print as a URI**, `file:///data/raw/orders.csv`, instead of
+  `file//data/raw/orders.csv`. The Airflow provider and the Spark integration emit
+  the bare namespace `file` where the OpenLineage naming conventions give
+  `file://{host}`, and an absolute path as the name; a name that starts with `/` is
+  no longer given a second one, which also covers HDFS. No other dataset key
+  changes, and `provenance --dataset` accepts the new spelling, the file name alone,
+  or the old spelling.
+- **Real Airflow and Spark lineage as test fixtures**, beside the dbt capture:
+  Airflow 3.3.1 with the OpenLineage provider 2.20.1, and Spark 4.2.0 with
+  openlineage-spark 1.53.0. The parent rule leaves out the Airflow DAG run and the
+  Spark application run. `docs/compatibility.md` now names the captures, and no
+  longer says dbt, Airflow and Spark emit `sourceCodeLocation`, which none of the
+  captures carries.
+- **The Art. 30 record states which Art. 30(1) items it has fields for**, on every
+  run including one that earns the mark: (a) the controller and (b) the purposes, and
+  none for (c) to (g). It is a scope line rather than a reason to withhold the mark,
+  because it would otherwise fire on every run. JSON carries `scope.art30` as lists of
+  covered and not covered letters. The deployer view does not repeat it.
+- The text scope statement wraps long values under the value instead of running past
+  the terminal width.
+- **A guessed domain is said to be guessed.** When `domains:` is set, the scope
+  statement names the domains guessed from the job namespace and those set by a
+  rule's `domain:`, and the silent-domain reason names the guess and the override.
+  JSON carries `domains_guessed`, `domains_mapped`, and a `domain_source` per
+  activity. Found against real dbt lineage, whose default namespace `dbt` made a
+  declared domain look silent.
+- `--activities` reads a separate document of **declared activities**, meaning
+  processing that emits no lineage. They appear beside the evidenced activities with
+  every field marked `declared` and `no lineage` where the events would have spoken,
+  are counted on their own line in the scope statement without changing the evidenced
+  counts, and withhold the mark. The out-of-view sentence is reworded so that it is
+  true whether or not anything is declared.
+
+### `qedro ropa --view deployer` — the same record, per AI use case
+
+- For each activity whose runs reported a model version: purpose and legal basis,
+  the model versions with their run counts and dates, the inputs the latest run read,
+  and the span of run records in view.
+- Built from the Art. 30 record object, so purpose and legal basis are the record's
+  own entries with their provenance, not a second reading of the events.
+- The model version is read from the standard OpenLineage `tags` run facet (key
+  `model_version`) or from `cordata_provenance.step_params`, where `pipeline-runtime`
+  puts it.
+- Art. 26 of the AI Act is cited from the consolidated text (CELEX
+  02024R1689-20260727) with its condition: Annex III high-risk systems, from
+  2 December 2027. The view does not decide whether a use case is high-risk.
+- Retention is reported as the span of run records in view, never as a retention
+  policy, and a short span does not withhold the mark.
+- Declared activities that name a `model` are listed as AI use cases, marked
+  `declared, no lineage`, and withhold the mark.
+
+### `qedro quality` — the assertion history
+
+- What was checked about each dataset and when, from the standard OpenLineage
+  `dataQualityAssertions` facet that Great Expectations already emits.
+- **Datasets with no assertions are half the artefact**, not a footnote: a
+  dataset with no failures and a dataset with no checks look identical in every
+  summary anybody writes, and they are opposites.
+- A *failing* expectation does not withhold the mark. The mark says this account
+  of what was checked is complete, not that the data is good.
+- `--domain` narrows it, repeatably; `--days` is shorthand for the window. The scope
+  statement names the filter and how many datasets it left out, in which domains, and
+  whether each domain was guessed from the job namespace. A filter that leaves out
+  every dataset says so, instead of reporting that no datasets were found.
+
+### `qedro provenance` — the chain back to a commit
+
+- Walks backwards from a dataset through the runs that produced it, the code
+  each run executed and the commit it was at, using the standard
+  `sourceCodeLocation` facet that dbt, Airflow and Spark all emit.
+- **Unknown is not unsigned.** OpenLineage has no standard place for whether a
+  commit was signed, so silence is a third state: it withholds the mark exactly
+  as a failed signature would and is never rendered as one. `json` emits `null`,
+  not `false`.
+- A chain ends for three different reasons — a genuine source, a producing run
+  outside the window, or the depth limit — and each is counted and named.
+- An ambiguous `--dataset` is answered with the candidates rather than a guess.
+
+### Output
+
+- `text`, `markdown`, `json` and `xlsx`. The format follows `--out`'s filename
+  unless `--format` says otherwise.
+- **xlsx puts the verdict above the table on the first sheet.** A caveat below
+  the data is a caveat nobody reads.
+- Every format states the scope of what was looked at, on every run including
+  one that earns the mark.
+
+### The mark
+
+- `∎` is printed only when an artefact stands on its own evidence, and the
+  reasons for withholding it are always printed with it. `--no-symbol`, or
+  `QEDRO_NO_SYMBOL=1`, for terminals with no glyph for U+220E.
+
+### Also
+
+- `demo/` — three weeks of synthetic lineage from dbt, Airflow and Spark, with
+  no AWS anywhere in it, in two estates that differ only in whether the
+  pipelines declare their Art. 30 fields.
+- Apache-2.0. Python 3.12, 3.13 and 3.14.

qedro-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,146 @@
+# Contributing
+
+Issues and pull requests are welcome. Most of this page is about scope and about
+four commitments that look like details and are not, because the quickest way to
+waste an afternoon here is to send something good that cannot be merged.
+
+## What this repo is for
+
+Producing compliance artefacts from evidence a platform already emits, and being
+honest about the parts it cannot evidence. That second half decides most of what
+follows.
+
+## Especially welcome
+
+**A case where the tombstone is printed and should not be.** The mark means *this
+artefact stands on its own evidence*. If you can construct a run that earns it
+while something is unproven, that is the most valuable bug report this project
+can receive, and a failing test is worth more than a fix.
+
+**Lineage from an emitter we have not seen.** Production events disagree with the
+spec in ways nobody predicts. A file that the reader mishandles — or handles
+while quietly skipping records — is exactly what `tests/fixtures/` is for.
+
+**A projection's output read by somebody who does the job.** A DPO, an auditor, a
+data steward saying *this column is not what I would ask for* is worth more than
+any amount of internal reasoning about it.
+
+**A source adapter.** `sources.read()` dispatches on scheme behind one entry
+point. A new source yields `Event` objects and stores nothing.
+
+## Out of scope
+
+- **Any write path into a system Qedro reads.** See below; this is not
+  negotiable.
+- **A server, a database, a UI.** Not because they are bad ideas — there is a
+  design for them — but because they are not this repository. It stays a CLI
+  that reads evidence and writes a file.
+- **Resolving a commit or a signature over the network.** `provenance` reports
+  what an emitter reported. A projection that phoned a forge would have acquired
+  a credential and a network dependency, and the tool would no longer be
+  read-only by construction.
+- **Modelling more facets as types.** Core event fields are typed; facets stay
+  raw dictionaries. See the rule below.
+
+## Four commitments that must not erode
+
+Each of these has been argued out, and each will look like an unnecessary
+constraint at some point.
+
+**Classification is source-neutral.** LakeFormation is one adapter beside GCP
+Data Catalog policy tags, Purview classifications and OpenMetadata terms — never
+the shape of the model, and never a name in the code. The acceptance test is
+that `qedro ropa` runs against a Marquez instance with no AWS anywhere
+(`tests/test_cli.py::TestRopaAgainstAnApi`).
+
+**The vocabulary is data the tool loads, never types it is compiled against.**
+`src/qedro/vocabularies/dsgvo.yaml` is a starting point, not a standard. Point
+`--vocabulary` at your own and the shipped one is replaced wholesale. Nothing in
+the code knows what a lawful basis is.
+
+**The tombstone is earned, not printed.** It is withheld whenever any part of an
+artefact rests on assertion rather than evidence, and **withholding is the
+signal** — a mark printed on every successful run says nothing. Do not make it
+unconditional, and do not add a flag that forces it.
+
+**Read-only by construction.** No write path in any handler, and no credential
+that could acquire one. This is a property of what the code is able to reach,
+not a policy it follows. The HTTP source can only build `GET`.
+
+## Ground rules
+
+**Facets stay raw dictionaries.** OpenLineage has a long tail of them and every
+emitter populates a different subset, so a projection asks for the facet it
+wants and copes with absence. Typing them means breaking whenever one moves.
+
+**Parsing never raises.** A directory of production lineage contains events that
+disagree with the spec, and a reader that dies on the first is useless. Bad
+records are skipped and **counted**, and the count reaches the completeness
+decision. That chain is relied upon, not diagnostics.
+
+**What the user wrote does raise.** A config typo silently produces an artefact
+with a hole in it, which is worse than a traceback. `errors.py` is the line
+between the two.
+
+**Every artefact states its own scope**, on every run, including one that earns
+the mark. A scope statement that appeared only when coverage was poor would
+teach a reader that its absence means full coverage. See
+[#3](https://github.com/cordata-tech/qedro/issues/3).
+
+**Two files are generated and CI checks both.** `schemas/openlineage-art30-processing-facet.json`
+comes from `pipeline_runtime.descriptor.Processing` via `tools/facet_schema.py`,
+and `demo/` comes from `tools/seed.py`. Editing either by hand looks like it
+worked until the next regeneration reverts it.
+
+## Running things
+
+```bash
+pip install -e ".[dev]"
+pytest -q
+ruff check . && ruff format --check .
+python tools/facet_schema.py --check
+python tools/seed.py --check
+```
+
+Python 3.12 is the floor; CI runs 3.12, 3.13 and 3.14.
+
+Two test modules import `pipeline_runtime`, which is not on PyPI — the Art. 30
+facet schema is generated from its `Processing` model and a test fails when the
+two diverge. Install it the way CI does, or skip those two:
+
+```bash
+pip install --no-deps "cordata-pipeline-runtime @ git+https://github.com/cordata-tech/pipeline-runtime@main"
+```
+
+The import is unconditional on purpose. A skip when the module is absent would
+mean CI silently stops checking the thing the test exists for.
+
+## Commits and pull requests
+
+**Explain the reasoning, not the diff.** The diff is in the diff. A commit
+message here is for the person who finds this code in a year and wants to know
+why it is shaped this way — especially when the answer is *we tried the obvious
+thing and it was wrong*.
+
+**Sign your commits.** `git config commit.gpgsign true` with an SSH or GPG key.
+A project about provenance that accepts unsigned commits is making a point it
+would rather not.
+
+**Tests that read as sentences.** Test names in this repository describe the
+behaviour being protected, not the function being called, and the ones guarding a
+commitment say which commitment. Follow that; it is why the suite is readable.
+
+## Releases
+
+`0.x` versions may change interfaces between releases, and the changelog says
+which. Output formats and the `qedro.yaml` schema are the parts most likely to
+move before `1.0`.
+
+Releases happen when something is worth releasing rather than on a calendar, and
+each one is a tag, a changelog entry and a PyPI artefact. `main` is expected to
+be releasable at any point: CI runs on every push, and a red `main` is a bug
+before it is anything else.
+
+Pushing a `v*` tag is what publishes. The workflow refuses to ship if the tag
+disagrees with `qedro.__version__`, or if the changelog entry for that version
+is missing or still marked unreleased.

qedro-0.2.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

qedro-0.2.0/MANIFEST.in

@@ -0,0 +1,33 @@
+# What goes in the source distribution.
+#
+# The wheel is the code. The sdist is the code plus everything needed to check
+# that the code does what it says — which for this project is not a nicety. A
+# README that argues claims should be verifiable, shipped in an artefact where
+# they cannot be, would be making the opposite point.
+#
+# So `pytest` on an unpacked sdist runs the same 401 tests CI does — verified,
+# not assumed. `demo/` is present because `tests/test_demo.py` asserts against
+# those exact bytes, and `tools/` because two test modules import it. Twenty-
+# seven of those tests also need `pipeline_runtime`, which is a dev dependency
+# installed from git; without it the other 374 still run.
+
+include CHANGELOG.md CONTRIBUTING.md SECURITY.md
+graft tests
+graft tools
+graft schemas
+graft docs
+
+# Three weeks of synthetic lineage, and the reason it is worth the megabyte:
+# without it the shipped tests cannot run and the README's demo is a promise
+# the artefact does not keep.
+graft demo
+
+# `docs/architecture.md` is a proposal, not a description, and is deliberately
+# untracked until the target architecture is agreed. It sits in the working
+# tree, which is exactly where `graft docs` would find it — so it is excluded
+# by name rather than by hoping nobody builds an sdist from a dirty checkout.
+exclude docs/architecture.md
+exclude docs/architecture.pdf
+
+global-exclude *.py[cod] __pycache__/* .DS_Store
+prune **/__pycache__