hydo-assert 0.2.0__tar.gz

hydo_assert-0.2.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

hydo_assert-0.2.0/PKG-INFO

@@ -0,0 +1,238 @@
+Metadata-Version: 2.4
+Name: hydo-assert
+Version: 0.2.0
+Summary: Assertion SDK for programs tested under hydo, a deterministic hypervisor (always/sometimes properties, BUGGIFY fault points, bounded liveness, operation histories)
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/kakoc/hydo
+Project-URL: Documentation, https://github.com/kakoc/hydo/tree/main/sdk/python#readme
+Project-URL: Repository, https://github.com/kakoc/hydo
+Project-URL: Issues, https://github.com/kakoc/hydo/issues
+Keywords: testing,assertions,deterministic,fault-injection,simulation,hydo
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Topic :: Software Development :: Testing
+Classifier: Typing :: Typed
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# hydo-assert: the Python assertion SDK
+
+A Python program running under hydo (the deterministic hypervisor of
+[github.com/kakoc/hydo](https://github.com/kakoc/hydo)) uses this package to say what must be
+true and what should happen at least once. It writes the same console records as the Go SDK
+(`sdk/go`), byte for byte, with the same deduplication rules, so `hydo run` and the explorer
+(`hydo explore`) read them unchanged. The wire format and the verdict
+rules are in `hydo/ASSERTIONS.md` of that repository; paths below are relative to it.
+
+- Distribution name `hydo-assert`, import name `hydo_assert`.
+- Standard library only; Python 3.8 or later; thread-safe and safe to call from asyncio code.
+- Install it from PyPI (`pip install hydo-assert`), or from a checkout (`pip install ./sdk/python`),
+  or put `sdk/python/src` on `PYTHONPATH`, or copy `src/hydo_assert/` into the guest image (the
+  `hydo/onboard/examples/python-kv` example does that).
+
+## Use
+
+```python
+import hydo_assert as ha
+
+# Declare at module level: the catalog is complete once the module is imported.
+ACKED = ha.always("kv.acked_durable", "every acknowledged PUT reads back with its value")
+PARSES = ha.always_or_unreachable("kv.log_parses", "every complete log line parses on replay")
+SIZE = ha.always("kv.log_size", "the log holds at most 10000 entries")
+REPLAYED = ha.sometimes("kv.recovered", "the server replayed a non-empty log at start")
+TORN = ha.reachable("kv.torn_tail", "the log ended in a partial line")
+BAD = ha.unreachable("kv.bad_command", "the server got a command it does not know")
+BACK = ha.eventually("kv.back", "the server answers within 5 s", 5.0)        # seconds or timedelta
+LOST = ha.buggify("kv.put.lost_reply", "the PUT is logged but its reply is lost")
+
+ha.catalog()                                       # once, early: the HYDO-ASSERT-DECL lines
+ACKED.check(got == want, {"key": k, "got": got})   # returns bool(cond)
+REPLAYED.check(n > 0, {"entries": n})
+TORN.hit({"bytes": len(tail)})
+BAD.hit()
+SIZE.cmp(n, "<=", 10000)                           # details: left, op, right
+if LOST.fire({"key": k}):                          # BUGGIFY fault point
+    return
+BACK.await_cond(lambda: ping() == "PONG", poll=0.2)   # blocks; one record
+w = BACK.begin(); ...; w.done()                    # or: anchor now, condition true now / w.expire()
+ha.setup_complete()                                # HYDO-SETUP-COMPLETE
+```
+
+In asyncio code use `await BACK.await_async(cond, poll)`: `cond` may be a plain function or an
+`async def`, and the wait between checks is `asyncio.sleep`. Every other call is a short write
+to the console under one process-wide lock that is never held across a blocking call or an
+`await`, so it is safe from coroutines and from threads.
+
+| Go SDK (`hydoassert`) | Python (`hydo_assert`) |
+|---|---|
+| `Always(id, msg)`, `AlwaysOrUnreachable`, `Sometimes`, `Reachable`, `Unreachable` | `always(id, msg)`, `always_or_unreachable`, `sometimes`, `reachable`, `unreachable` (return an `Assertion`) |
+| `a.Check(cond, d)`, `a.Hit(d)` | `a.check(cond, details=None)`, `a.hit(details=None)` |
+| `a.Cmp(left, op, right, d)` | `a.cmp(left, op, right, details=None)` (`op` is one of `<`, `<=`, `==`, `!=`, `>=`, `>`) |
+| `Eventually(id, msg, within)`, `l.Await(cond, poll, d)`, `l.Begin()`, `w.Done(d)`, `w.Expire(d)` | `eventually(id, msg, within)`, `l.await_cond(cond, poll, details=None)`, `await l.await_async(cond, poll, details=None)`, `l.begin()`, `w.done(details=None)`, `w.expire(details=None)` |
+| `Buggify(id, msg)`, `p.Fire(d)` | `buggify(id, msg)`, `p.fire(details=None)` |
+| `Catalog()`, `SetupComplete()` | `catalog()`, `setup_complete()` |
+| `D{...}` | any mapping (a `dict`); `hydo_assert.D` is `dict` |
+
+The naming follows the other SDKs (`sdk/js`, `sdk/rust`): the Go names in Python's snake case,
+and `await_cond` / `await_async` for Go's `Await`. That table, `Liveness`, `Watch` and `Point`
+(the classes the calls return) and the history recorder below are the whole API: a comparison
+has one form, `cmp` with its operator, and names that start with `_` (the modules `_core`,
+`_buggify`, `_eventually`, `_json`, `_sink`) are the package's own.
+
+## Records, deduplication, catalog
+
+- Records go to `/dev/ttyS0` (opened once, `O_WRONLY|O_APPEND|O_NOCTTY`, one `write` per
+  record). `HYDO_ASSERT_OUT=<path>` writes to that existing file, `stderr` to standard error
+  (plain logging outside hydo), `off` nowhere, and `logging` (Python only) logs each record line
+  at INFO on the logger `hydo_assert`. If the file cannot be opened, standard error is used.
+- A process writes each (assertion, outcome) pair once: the first pass and the first failure,
+  with the details of that evaluation. A `sometimes` that evaluated false is never written.
+  `unreachable` is written with `ok:false`. A record whose details hold a NaN or an infinity is
+  not written at all, but still counts as written (as in Go, where `json.Marshal` fails after
+  the pair was marked).
+- The catalog (`catalog()`) lists every declared assertion in declaration order: the order in
+  which the declaring statements ran, which for module-level declarations is the import order.
+  A module that is never imported contributes nothing; import it before calling `catalog()`.
+  Call `catalog()` in the process whose console output is collected for each run (for a
+  server started before the explorer's snapshot point, that is a client or the oracle, not the
+  server; see the example).
+- Details are encoded exactly as Go's `encoding/json` encodes the equivalent Go value: keys in
+  UTF-8 byte order, `<`, `>`, `&`, U+2028 and U+2029 escaped, other non-ASCII text raw, Go's
+  float format (`1e+21`, `1.5e-7`, `3` for `3.0`, `-0`). The caller's dict is never changed
+  (Go adds `left`/`op`/`right`, `hit`, `elapsed_ms` to the caller's map; Python adds them to a
+  copy, which gives the same record).
+- Python values Go has no form for: a tuple is an array; `bytes` is a base64 string (as Go
+  encodes `[]byte`); a set is an array sorted by encoded element; integer dict keys are written
+  in decimal; any other object is the string `str(obj)`. A lone surrogate (an undecodable byte
+  read with `surrogateescape`) is written as `\ufffd`, as the pinned guest Go 1.26.5 writes an
+  invalid UTF-8 byte (Go 1.27 writes a raw U+FFFD instead).
+- `details` that is not a mapping is wrapped as `{"value": details}`.
+- After `os.fork()` the child keeps the parent's deduplication state (it is one program's
+  history) and gets fresh locks.
+
+## BUGGIFY
+
+Same decisions as the Go SDK: configured by `$HYDO_BUGGIFY`, or else the file
+`$HYDO_BUGGIFY_FILE` (default `/tmp/hydo-buggify`), holding
+`<seed-hex>[,<enable-permille>[,<fire-permille>]]` (defaults 250 and 250, FoundationDB's
+values), parsed as Go's `strconv.ParseUint` does (no `0x`, no sign, no spaces inside a field;
+an unparsable seed leaves BUGGIFY off, an unparsable permille keeps its default). It is read
+once, at the first evaluation of any point, which also writes
+`HYDO-BUGGIFY {"seed":..,"enable_pm":..,"fire_pm":..,"enabled":[..]}`. A point is enabled
+for the run when `splitmix64(seed ^ fnv1a(id)) % 1000 < enable`; its k-th evaluation fires
+when `splitmix64(splitmix64(seed ^ fnv1a(id)) + k·0x9e3779b97f4a7c15) % 1000 < fire`. Each
+point is also a `sometimes` property `buggify.<id>` ("fault point fired: <msg>"), recorded at
+its first firing with `{"hit": k}`. Without a configuration every point returns False and has
+no other effect. Under hydo the schedule token `B=<seed>` (`hydo explore --buggify`) reaches the
+file through the guest init: an onboarded project gets it with `buggify: true` in its
+`hydo.yaml` (`hydo/onboard/MANIFEST.md`).
+
+## History recorder
+
+The module `hydo_assert.history` records operation histories for hydo's host-side checkers
+(linearizability of a register or compare-and-set object per key, and Elle-style list-append;
+the format and the checkers are in `hydo/HISTORIES.md`). It writes the same `HYDO-HIST` lines as
+the Go recorder (`sdk/go/hydohist`), byte for byte except the time field. It is a separate
+module: `import hydo_assert` does not load it, so the assertion records are unchanged.
+
+```python
+from hydo_assert import history as hh
+
+kv = hh.History("kv")
+op = kv.invoke(3, "write", "k1", hh.enc(v))      # process 3, before the request
+try:
+    rev = put("k1", v)
+    op.ok("k1", hh.ver(v, rev))                  # the version the store assigned, if any
+except Rejected:
+    op.fail()                                    # it definitely had no effect
+except Exception:
+    op.info()                                    # a timeout: the outcome is unknown
+kv.invoke(3, "read", "k1").ok("k1", hh.value_or_nil(get("k1")))   # None means absent
+
+la = hh.History("la")
+tx = la.invoke(4, "txn", hh.append_mop("x", "1"), hh.read_mop("y"))
+tx.ok(hh.append_mop("x", "1"), hh.read_result_mop("y", "1", "2"))
+```
+
+| Go (`hydohist`) | Python (`hydo_assert.history`) |
+|---|---|
+| `New(name)` | `History(name)` |
+| `h.Invoke(proc, f, args...)`, `h.Record(proc, typ, f, args...)` | `h.invoke(proc, f, *args)`, `h.record(proc, typ, f, *args)` |
+| `op.Ok(args...)`, `op.Fail()`, `op.Info()` | `op.ok(*args)` (no args repeats the invocation's), `op.fail()`, `op.info()` |
+| `Enc`, `Ver`, `ValOrNil(v, ok)`, `List` | `enc`, `ver`, `value_or_nil(v)` (None means absent), `list_value` |
+| `Append`, `Read`, `ReadResult`, `Nil` | `append_mop`, `read_mop`, `read_result_mop`, `NIL` |
+
+- `list_value` has a longer name than Go's `List` so that it does not hide the builtin `list`
+  for a program that uses `from hydo_assert.history import *`. The micro-op helpers have the
+  Rust SDK's names.
+- A process is an int or a token without spaces. Arguments are tokens: a `str` is written as
+  it is (encode free-form values with `enc`), an `int` is written in decimal, and `bytes` are
+  written as they are. Versions are int64 values; a value outside that range raises
+  `ValueError`, and a value that is not an int (including `bool`) raises `TypeError`.
+- Records go to `/dev/console`. `HYDO_HIST_OUT=<path>` appends to that file and creates it if
+  it is missing (mode 0644), `stderr` writes to standard error, and `off` writes nothing. If the
+  file cannot be opened, standard error is used. The setting is read once, at the first record,
+  and is separate from `HYDO_ASSERT_OUT`.
+- Each record is one `os.write` call made under a lock, so threads never interleave lines. The
+  time field is `time.monotonic_ns()` (CLOCK_MONOTONIC on Linux, deterministic under hydo). It
+  is read under the lock, so the times of successive lines never decrease; Go reads it just
+  before its lock, and the line format is the same. A forked child gets a new lock and keeps the
+  descriptor.
+- Text is written as UTF-8. A `str` is encoded with the `surrogateescape` rule, so a string
+  that came from bytes that way (`os.fsdecode`, `sys.argv`, `os.environ`, or `enc` of bytes)
+  gives back its original bytes. Any other lone surrogate is written as U+FFFD. `enc` of
+  `bytes` keeps invalid UTF-8 bytes as they are, as Go's `Enc` does, and returns them as
+  surrogateescape characters.
+- A process has at most one operation in flight: give each concurrent client (thread or task)
+  its own process number. Record the invocation before sending the request and the completion
+  after the reply, because the checkers order operations by the position of their records.
+
+## Determinism
+
+The records are a pure function of the program's evaluation sequence: no clocks (except the
+guest's monotonic clock for `eventually`, which hvx makes deterministic), no host state,
+catalog in declaration order, sorted keys. A Python program is itself deterministic under hydo:
+the guest CRNG is seeded identically every run, so even the hash seed is the same; setting
+`PYTHONHASHSEED=0` makes that independent of the CRNG. `PYTHONDONTWRITEBYTECODE=1` avoids
+writing `.pyc` files on the guest's read-only root.
+
+## Tests
+
+- `python3 -m unittest discover -s sdk/python/tests`: the encoder (Go's float and string
+  forms), deduplication, details not mutated, NaN, catalog order, 16 threads writing 2000
+  evaluations each without an interleaved line, asyncio `eventually`, fork, BUGGIFY decisions
+  and rates. Passes on Python 3.8.20 and 3.15.0.
+- `tests/test_cases.py` (run by the same `unittest` command): the shared cases (`sdk/cases`,
+  written by the reference SDK, `sdk/rust`; format and rules in `sdk/rust/README.md`, "The
+  shared cases"). For every case and run it starts itself as a child process, makes the
+  case's calls with this package and `hydo_assert.history`, and compares the bytes the sink
+  received and the calls' results with the case's files: every assertion type, comparisons,
+  details with tricky values (invalid UTF-8 as `surrogateescape` characters), the catalog,
+  `eventually`, BUGGIFY under 18 configurations, `HYDO_ASSERT_OUT`, and the
+  history records with every helper and `HYDO_HIST_OUT` setting. Cases that need a float32
+  (`f32`) are skipped and listed. To add a case, add it to the Rust SDK's
+  `tests/cases/defs.rs` and run `HYDO_BLESS=1 cargo test --test cases` there; this test then
+  runs it. A format change starts in the Rust SDK too: after it is blessed, this test fails on
+  the lines this package must change. No other language is needed to run it. Passes on Python
+  3.8.20 and 3.15.0.
+- `tests/test_history.py` (run by the same `unittest` command): the exact lines of the Go
+  recorder's own unit test, the encoding of every byte class, the argument checks, the sink
+  settings, one `os.write` per record, 8 threads writing 400 records each without an
+  interleaved line, and fork.
+- The files of `src/hydo_assert/` are copied into the python-kv example's images, so a change
+  to any of them changes those images' bytes and every python-kv fingerprint (the last such
+  change, a reworded comment in `_core.py`, re-recorded them in milestone 157;
+  docs/REBASELINE.md has the values).
+
+## Limits
+
+- No call-site form that registers itself before it runs (the Rust SDK's macros do that with a
+  linker section). A Python assertion is in the catalog once the statement that declares it has
+  run, so declare at module level.
+- `cmp` compares any two Python values that support the operator; values that cannot be
+  compared evaluate to false. Go's `Cmp` takes only `int64`.

hydo_assert-0.2.0/README.md

@@ -0,0 +1,216 @@
+# hydo-assert: the Python assertion SDK
+
+A Python program running under hydo (the deterministic hypervisor of
+[github.com/kakoc/hydo](https://github.com/kakoc/hydo)) uses this package to say what must be
+true and what should happen at least once. It writes the same console records as the Go SDK
+(`sdk/go`), byte for byte, with the same deduplication rules, so `hydo run` and the explorer
+(`hydo explore`) read them unchanged. The wire format and the verdict
+rules are in `hydo/ASSERTIONS.md` of that repository; paths below are relative to it.
+
+- Distribution name `hydo-assert`, import name `hydo_assert`.
+- Standard library only; Python 3.8 or later; thread-safe and safe to call from asyncio code.
+- Install it from PyPI (`pip install hydo-assert`), or from a checkout (`pip install ./sdk/python`),
+  or put `sdk/python/src` on `PYTHONPATH`, or copy `src/hydo_assert/` into the guest image (the
+  `hydo/onboard/examples/python-kv` example does that).
+
+## Use
+
+```python
+import hydo_assert as ha
+
+# Declare at module level: the catalog is complete once the module is imported.
+ACKED = ha.always("kv.acked_durable", "every acknowledged PUT reads back with its value")
+PARSES = ha.always_or_unreachable("kv.log_parses", "every complete log line parses on replay")
+SIZE = ha.always("kv.log_size", "the log holds at most 10000 entries")
+REPLAYED = ha.sometimes("kv.recovered", "the server replayed a non-empty log at start")
+TORN = ha.reachable("kv.torn_tail", "the log ended in a partial line")
+BAD = ha.unreachable("kv.bad_command", "the server got a command it does not know")
+BACK = ha.eventually("kv.back", "the server answers within 5 s", 5.0)        # seconds or timedelta
+LOST = ha.buggify("kv.put.lost_reply", "the PUT is logged but its reply is lost")
+
+ha.catalog()                                       # once, early: the HYDO-ASSERT-DECL lines
+ACKED.check(got == want, {"key": k, "got": got})   # returns bool(cond)
+REPLAYED.check(n > 0, {"entries": n})
+TORN.hit({"bytes": len(tail)})
+BAD.hit()
+SIZE.cmp(n, "<=", 10000)                           # details: left, op, right
+if LOST.fire({"key": k}):                          # BUGGIFY fault point
+    return
+BACK.await_cond(lambda: ping() == "PONG", poll=0.2)   # blocks; one record
+w = BACK.begin(); ...; w.done()                    # or: anchor now, condition true now / w.expire()
+ha.setup_complete()                                # HYDO-SETUP-COMPLETE
+```
+
+In asyncio code use `await BACK.await_async(cond, poll)`: `cond` may be a plain function or an
+`async def`, and the wait between checks is `asyncio.sleep`. Every other call is a short write
+to the console under one process-wide lock that is never held across a blocking call or an
+`await`, so it is safe from coroutines and from threads.
+
+| Go SDK (`hydoassert`) | Python (`hydo_assert`) |
+|---|---|
+| `Always(id, msg)`, `AlwaysOrUnreachable`, `Sometimes`, `Reachable`, `Unreachable` | `always(id, msg)`, `always_or_unreachable`, `sometimes`, `reachable`, `unreachable` (return an `Assertion`) |
+| `a.Check(cond, d)`, `a.Hit(d)` | `a.check(cond, details=None)`, `a.hit(details=None)` |
+| `a.Cmp(left, op, right, d)` | `a.cmp(left, op, right, details=None)` (`op` is one of `<`, `<=`, `==`, `!=`, `>=`, `>`) |
+| `Eventually(id, msg, within)`, `l.Await(cond, poll, d)`, `l.Begin()`, `w.Done(d)`, `w.Expire(d)` | `eventually(id, msg, within)`, `l.await_cond(cond, poll, details=None)`, `await l.await_async(cond, poll, details=None)`, `l.begin()`, `w.done(details=None)`, `w.expire(details=None)` |
+| `Buggify(id, msg)`, `p.Fire(d)` | `buggify(id, msg)`, `p.fire(details=None)` |
+| `Catalog()`, `SetupComplete()` | `catalog()`, `setup_complete()` |
+| `D{...}` | any mapping (a `dict`); `hydo_assert.D` is `dict` |
+
+The naming follows the other SDKs (`sdk/js`, `sdk/rust`): the Go names in Python's snake case,
+and `await_cond` / `await_async` for Go's `Await`. That table, `Liveness`, `Watch` and `Point`
+(the classes the calls return) and the history recorder below are the whole API: a comparison
+has one form, `cmp` with its operator, and names that start with `_` (the modules `_core`,
+`_buggify`, `_eventually`, `_json`, `_sink`) are the package's own.
+
+## Records, deduplication, catalog
+
+- Records go to `/dev/ttyS0` (opened once, `O_WRONLY|O_APPEND|O_NOCTTY`, one `write` per
+  record). `HYDO_ASSERT_OUT=<path>` writes to that existing file, `stderr` to standard error
+  (plain logging outside hydo), `off` nowhere, and `logging` (Python only) logs each record line
+  at INFO on the logger `hydo_assert`. If the file cannot be opened, standard error is used.
+- A process writes each (assertion, outcome) pair once: the first pass and the first failure,
+  with the details of that evaluation. A `sometimes` that evaluated false is never written.
+  `unreachable` is written with `ok:false`. A record whose details hold a NaN or an infinity is
+  not written at all, but still counts as written (as in Go, where `json.Marshal` fails after
+  the pair was marked).
+- The catalog (`catalog()`) lists every declared assertion in declaration order: the order in
+  which the declaring statements ran, which for module-level declarations is the import order.
+  A module that is never imported contributes nothing; import it before calling `catalog()`.
+  Call `catalog()` in the process whose console output is collected for each run (for a
+  server started before the explorer's snapshot point, that is a client or the oracle, not the
+  server; see the example).
+- Details are encoded exactly as Go's `encoding/json` encodes the equivalent Go value: keys in
+  UTF-8 byte order, `<`, `>`, `&`, U+2028 and U+2029 escaped, other non-ASCII text raw, Go's
+  float format (`1e+21`, `1.5e-7`, `3` for `3.0`, `-0`). The caller's dict is never changed
+  (Go adds `left`/`op`/`right`, `hit`, `elapsed_ms` to the caller's map; Python adds them to a
+  copy, which gives the same record).
+- Python values Go has no form for: a tuple is an array; `bytes` is a base64 string (as Go
+  encodes `[]byte`); a set is an array sorted by encoded element; integer dict keys are written
+  in decimal; any other object is the string `str(obj)`. A lone surrogate (an undecodable byte
+  read with `surrogateescape`) is written as `\ufffd`, as the pinned guest Go 1.26.5 writes an
+  invalid UTF-8 byte (Go 1.27 writes a raw U+FFFD instead).
+- `details` that is not a mapping is wrapped as `{"value": details}`.
+- After `os.fork()` the child keeps the parent's deduplication state (it is one program's
+  history) and gets fresh locks.
+
+## BUGGIFY
+
+Same decisions as the Go SDK: configured by `$HYDO_BUGGIFY`, or else the file
+`$HYDO_BUGGIFY_FILE` (default `/tmp/hydo-buggify`), holding
+`<seed-hex>[,<enable-permille>[,<fire-permille>]]` (defaults 250 and 250, FoundationDB's
+values), parsed as Go's `strconv.ParseUint` does (no `0x`, no sign, no spaces inside a field;
+an unparsable seed leaves BUGGIFY off, an unparsable permille keeps its default). It is read
+once, at the first evaluation of any point, which also writes
+`HYDO-BUGGIFY {"seed":..,"enable_pm":..,"fire_pm":..,"enabled":[..]}`. A point is enabled
+for the run when `splitmix64(seed ^ fnv1a(id)) % 1000 < enable`; its k-th evaluation fires
+when `splitmix64(splitmix64(seed ^ fnv1a(id)) + k·0x9e3779b97f4a7c15) % 1000 < fire`. Each
+point is also a `sometimes` property `buggify.<id>` ("fault point fired: <msg>"), recorded at
+its first firing with `{"hit": k}`. Without a configuration every point returns False and has
+no other effect. Under hydo the schedule token `B=<seed>` (`hydo explore --buggify`) reaches the
+file through the guest init: an onboarded project gets it with `buggify: true` in its
+`hydo.yaml` (`hydo/onboard/MANIFEST.md`).
+
+## History recorder
+
+The module `hydo_assert.history` records operation histories for hydo's host-side checkers
+(linearizability of a register or compare-and-set object per key, and Elle-style list-append;
+the format and the checkers are in `hydo/HISTORIES.md`). It writes the same `HYDO-HIST` lines as
+the Go recorder (`sdk/go/hydohist`), byte for byte except the time field. It is a separate
+module: `import hydo_assert` does not load it, so the assertion records are unchanged.
+
+```python
+from hydo_assert import history as hh
+
+kv = hh.History("kv")
+op = kv.invoke(3, "write", "k1", hh.enc(v))      # process 3, before the request
+try:
+    rev = put("k1", v)
+    op.ok("k1", hh.ver(v, rev))                  # the version the store assigned, if any
+except Rejected:
+    op.fail()                                    # it definitely had no effect
+except Exception:
+    op.info()                                    # a timeout: the outcome is unknown
+kv.invoke(3, "read", "k1").ok("k1", hh.value_or_nil(get("k1")))   # None means absent
+
+la = hh.History("la")
+tx = la.invoke(4, "txn", hh.append_mop("x", "1"), hh.read_mop("y"))
+tx.ok(hh.append_mop("x", "1"), hh.read_result_mop("y", "1", "2"))
+```
+
+| Go (`hydohist`) | Python (`hydo_assert.history`) |
+|---|---|
+| `New(name)` | `History(name)` |
+| `h.Invoke(proc, f, args...)`, `h.Record(proc, typ, f, args...)` | `h.invoke(proc, f, *args)`, `h.record(proc, typ, f, *args)` |
+| `op.Ok(args...)`, `op.Fail()`, `op.Info()` | `op.ok(*args)` (no args repeats the invocation's), `op.fail()`, `op.info()` |
+| `Enc`, `Ver`, `ValOrNil(v, ok)`, `List` | `enc`, `ver`, `value_or_nil(v)` (None means absent), `list_value` |
+| `Append`, `Read`, `ReadResult`, `Nil` | `append_mop`, `read_mop`, `read_result_mop`, `NIL` |
+
+- `list_value` has a longer name than Go's `List` so that it does not hide the builtin `list`
+  for a program that uses `from hydo_assert.history import *`. The micro-op helpers have the
+  Rust SDK's names.
+- A process is an int or a token without spaces. Arguments are tokens: a `str` is written as
+  it is (encode free-form values with `enc`), an `int` is written in decimal, and `bytes` are
+  written as they are. Versions are int64 values; a value outside that range raises
+  `ValueError`, and a value that is not an int (including `bool`) raises `TypeError`.
+- Records go to `/dev/console`. `HYDO_HIST_OUT=<path>` appends to that file and creates it if
+  it is missing (mode 0644), `stderr` writes to standard error, and `off` writes nothing. If the
+  file cannot be opened, standard error is used. The setting is read once, at the first record,
+  and is separate from `HYDO_ASSERT_OUT`.
+- Each record is one `os.write` call made under a lock, so threads never interleave lines. The
+  time field is `time.monotonic_ns()` (CLOCK_MONOTONIC on Linux, deterministic under hydo). It
+  is read under the lock, so the times of successive lines never decrease; Go reads it just
+  before its lock, and the line format is the same. A forked child gets a new lock and keeps the
+  descriptor.
+- Text is written as UTF-8. A `str` is encoded with the `surrogateescape` rule, so a string
+  that came from bytes that way (`os.fsdecode`, `sys.argv`, `os.environ`, or `enc` of bytes)
+  gives back its original bytes. Any other lone surrogate is written as U+FFFD. `enc` of
+  `bytes` keeps invalid UTF-8 bytes as they are, as Go's `Enc` does, and returns them as
+  surrogateescape characters.
+- A process has at most one operation in flight: give each concurrent client (thread or task)
+  its own process number. Record the invocation before sending the request and the completion
+  after the reply, because the checkers order operations by the position of their records.
+
+## Determinism
+
+The records are a pure function of the program's evaluation sequence: no clocks (except the
+guest's monotonic clock for `eventually`, which hvx makes deterministic), no host state,
+catalog in declaration order, sorted keys. A Python program is itself deterministic under hydo:
+the guest CRNG is seeded identically every run, so even the hash seed is the same; setting
+`PYTHONHASHSEED=0` makes that independent of the CRNG. `PYTHONDONTWRITEBYTECODE=1` avoids
+writing `.pyc` files on the guest's read-only root.
+
+## Tests
+
+- `python3 -m unittest discover -s sdk/python/tests`: the encoder (Go's float and string
+  forms), deduplication, details not mutated, NaN, catalog order, 16 threads writing 2000
+  evaluations each without an interleaved line, asyncio `eventually`, fork, BUGGIFY decisions
+  and rates. Passes on Python 3.8.20 and 3.15.0.
+- `tests/test_cases.py` (run by the same `unittest` command): the shared cases (`sdk/cases`,
+  written by the reference SDK, `sdk/rust`; format and rules in `sdk/rust/README.md`, "The
+  shared cases"). For every case and run it starts itself as a child process, makes the
+  case's calls with this package and `hydo_assert.history`, and compares the bytes the sink
+  received and the calls' results with the case's files: every assertion type, comparisons,
+  details with tricky values (invalid UTF-8 as `surrogateescape` characters), the catalog,
+  `eventually`, BUGGIFY under 18 configurations, `HYDO_ASSERT_OUT`, and the
+  history records with every helper and `HYDO_HIST_OUT` setting. Cases that need a float32
+  (`f32`) are skipped and listed. To add a case, add it to the Rust SDK's
+  `tests/cases/defs.rs` and run `HYDO_BLESS=1 cargo test --test cases` there; this test then
+  runs it. A format change starts in the Rust SDK too: after it is blessed, this test fails on
+  the lines this package must change. No other language is needed to run it. Passes on Python
+  3.8.20 and 3.15.0.
+- `tests/test_history.py` (run by the same `unittest` command): the exact lines of the Go
+  recorder's own unit test, the encoding of every byte class, the argument checks, the sink
+  settings, one `os.write` per record, 8 threads writing 400 records each without an
+  interleaved line, and fork.
+- The files of `src/hydo_assert/` are copied into the python-kv example's images, so a change
+  to any of them changes those images' bytes and every python-kv fingerprint (the last such
+  change, a reworded comment in `_core.py`, re-recorded them in milestone 157;
+  docs/REBASELINE.md has the values).
+
+## Limits
+
+- No call-site form that registers itself before it runs (the Rust SDK's macros do that with a
+  linker section). A Python assertion is in the catalog once the statement that declares it has
+  run, so declare at module level.
+- `cmp` compares any two Python values that support the operator; values that cannot be
+  compared evaluate to false. Go's `Cmp` takes only `int64`.