kisstdlib 0.0.7__tar.gz → 0.0.9__tar.gz

kisstdlib-0.0.9/PKG-INFO

@@ -0,0 +1,195 @@
+Metadata-Version: 2.1
+Name: kisstdlib
+Version: 0.0.9
+Summary: System programming with conceptually and algebraically simple primitives.
+Author-email: Jan Malakhovski <oxij@oxij.org>
+Project-URL: Homepage, https://oxij.org/software/kisstdlib/
+Project-URL: GitHub, https://github.com/oxij/kisstdlib
+Project-URL: Support Development, https://oxij.org/#support
+Keywords: sys,stdlib
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: License :: OSI Approved :: Python Software Foundation License
+Classifier: License :: OSI Approved :: BSD License
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+Requires-Dist: sortedcontainers==2.4.*
+
+# Table of Contents
+<details><summary>(Click me to see it.)</summary>
+<ul>
+<li><a href="#what-is-kisstdlib" id="toc-what-is-kisstdlib">What is <code>kisstdlib</code>?</a></li>
+<li><a href="#parts-and-pieces" id="toc-parts-and-pieces">Parts and pieces</a></li>
+<li><a href="#usage" id="toc-usage">Usage</a>
+<ul>
+<li><a href="#describe-forest" id="toc-describe-forest">describe-forest</a></li>
+</ul></li>
+<li><a href="#development-.test-example.sh---help---wine" id="toc-development-.test-example.sh---help---wine">Development: <code>./test-example.sh [--help] [--wine]</code></a></li>
+</ul>
+</details>
+
+# What is `kisstdlib`?
+
+`kisstdlib` is a set of modules for the [Python](https://www.python.org/) programming language, designed mostly for system programming with a touch of everything else, that aims to enhance the standard experience while keeping everything it does conceptually and algebraically simple.
+
+This library borrows heavily from the [Haskell](https://www.haskell.org/) programming language, where appropriate.
+(The Python's bundled modules do that too in many places, but `kisstdlib` borrows more.)
+
+In short, `kisstdlib` mostly implements useful extensions for Python's bundled modules, keeping the APIs backward compatible and naming things in the same style.
+The only exception to this is, yet unpublished, `io.loop` module, which implements Free-Monad-based IO.
+
+At the moment, `kisstdlib` is an alpha work in progress software with an unstable API.
+
+# <span id="pieces"/>Parts and pieces
+
+`kisstdlib` consists of:
+
+- a bunch of Python modules under [./kisstdlib](./kisstdlib);
+
+  they are pretty well-documented there, though there's no generated `sphinx` docs yet;
+
+- a bunch of human-readable [examples](./example) using those modules;
+
+- some useful thin-wrapper programs over `kisstdlib` functions, useful for writing whole-program and/or fixed-output/extensional-equality tests;
+
+  the documentation of which gets rendered into the following "Usage" section;
+
+- as well as some testing-related infrastructure built on top of this, in [./devscript](./devscript).
+
+# Usage
+
+## describe-forest
+
+Produce a plain-text recursive deterministic `find`/`ls`/`stat`-like description of given file and/or directory inputs.
+
+The output format is designed to be descriptive and easily `diff`able while also producing minimally dissimilar outputs for similar inputs, even when those inputs contain lots of symlinks and/or hardlinks.
+I.e., essentially, this is an alternative to `ls -lR` and/or `find . -exec ls -l {} \;` which generates outputs that change very little when files with multiple symlinks and/or hardlinks change.
+
+This is most useful for testing code that produces filesystem trees.
+
+The most verbose output format this program can produce, for a single input file
+
+```bash
+describe-forest --full path/to/README.md
+```
+
+looks as follows:
+
+```
+. reg mode 644 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+```
+
+Note how both the path to and the name of the file do not appear in the output.
+This is what you would want for doing things like
+
+```bash
+if ! diff -U 0 <(describe-forest --full v1/path/to/README.md) <(describe-forest --full v2/path/to/README.md) ; then
+    echo "output changed between versions!" >&2
+    exit 1
+fi
+```
+
+which this program is designed for.
+
+For a single input directory
+
+```bash
+describe-forest --full path/to/dir
+```
+
+the output looks similar to this:
+
+```
+. dir mode 700 mtime [2025-01-01 00:00:00]
+afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+sub dir mode 700 mtime [2025-01-01 00:03:00]
+sub/afile-hardlink.jpg ref ==> afile.jpg
+sub/afile-symlink.jpg sym mode 777 mtime [2025-01-01 00:59:59] -> ../afile.jpg
+sub/zfile-hardlink.jpg reg mode 600 mtime [2025-01-01 00:02:00] size 256 sha256 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+unix-socket ??? mode 600 mtime [2025-01-01 01:00:00] size 0
+zfile.jpg ref ==> sub/zfile-hardlink.jpg
+```
+
+Hardlinks, which are denoted by `ref`s above, are processed as follows:
+
+- each new file encountered in lexicographic walk is rendered fully,
+- files with repeated dev+inode numbers are rendered by emitting `ref ==> ` followed by the full path (or `ref => ` followed by the relative path, with `--relative-hardlink`) to the previously encountered element.
+
+This way, renaming a file in the input changes at most two lines.
+
+Symlinks are rendered by simply emitting the path they store, unless `--follow-symlinks` is given, in which case the targets they point to get rendered instead.
+
+Multiple inputs get named by numbering them starting from "0".
+Thus, for instance, running this program with the same input file given twice
+
+```bash
+describe-forest --full path/to/README.md path/to/README.md
+```
+
+produces something like:
+
+```
+0 reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+1 ref ==> 0
+```
+
+And giving the same directory with that file inside twice produces:
+
+```
+0 dir mode 700 mtime [2025-01-01 00:00:00]
+0/afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+1 dir mode 700 mtime [2025-01-01 00:00:00]
+1/afile.jpg ref ==> 0/afile.jpg
+```
+
+In its default output format, though, the program emits only `size`s and `sha256`s, when appropriate:
+
+```
+. dir
+afile.jpg reg size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+```
+
+which is what you would usually want for writing tests.
+Though, if you are testing `rsync` or some such, feel free to use other options described below.
+
+See `devscript` directory in `kisstdlib`'s repository for examples of some shell machinery that uses this to implement arbitrary-program fixed-output tests, which is a nice and simple way to test programs by testing their outputs against outputs of different versions of themselves.
+
+Also, internally, this programs is actually a thin wrapper over `describe_forest` function of `kisstdlib.fs` Python module, which can be used with `pytest` or some such.
+
+- positional arguments:
+  - `PATH`
+  : input directories
+
+- options:
+  - `--version`
+  : show program's version number and exit
+  - `-h, --help`
+  : show this help message and exit
+  - `--markdown`
+  : show `--help` formatted in Markdown
+  - `--numbers`
+  : emit number prefixes even with a single input `PATH`
+  - `--literal`
+  : emit paths without escaping them even when they contain special symbols
+  - `--modes`
+  : emit file modes
+  - `--mtimes`
+  : emit file mtimes
+  - `--no-sizes`
+  : do not emit file sizes
+  - `--full`
+  : an alias for `--mtimes --modes`
+  - `--relative, --relative-hardlinks`
+  : emit relative paths when emitting `ref`s
+  - `-L, --dereference, --follow-symlinks`
+  : follow all symbolic links; replaces all `sym` elements of the output with description of symlink targets
+  - `--time-precision INT`
+  : time precision (as a negative power of 10); default: `0`, which means seconds, set to `9` for nanosecond precision
+  - `--hash-length INT`
+  : cut hashes by taking their prefixes of this many characters; default: print them whole
+
+# Development: `./test-example.sh [--help] [--wine]`
+
+Check that all `kisstdlib/example/*.py` are fixed-output.

kisstdlib-0.0.9/README.md

@@ -0,0 +1,176 @@
+# Table of Contents
+<details><summary>(Click me to see it.)</summary>
+<ul>
+<li><a href="#what-is-kisstdlib" id="toc-what-is-kisstdlib">What is <code>kisstdlib</code>?</a></li>
+<li><a href="#parts-and-pieces" id="toc-parts-and-pieces">Parts and pieces</a></li>
+<li><a href="#usage" id="toc-usage">Usage</a>
+<ul>
+<li><a href="#describe-forest" id="toc-describe-forest">describe-forest</a></li>
+</ul></li>
+<li><a href="#development-.test-example.sh---help---wine" id="toc-development-.test-example.sh---help---wine">Development: <code>./test-example.sh [--help] [--wine]</code></a></li>
+</ul>
+</details>
+
+# What is `kisstdlib`?
+
+`kisstdlib` is a set of modules for the [Python](https://www.python.org/) programming language, designed mostly for system programming with a touch of everything else, that aims to enhance the standard experience while keeping everything it does conceptually and algebraically simple.
+
+This library borrows heavily from the [Haskell](https://www.haskell.org/) programming language, where appropriate.
+(The Python's bundled modules do that too in many places, but `kisstdlib` borrows more.)
+
+In short, `kisstdlib` mostly implements useful extensions for Python's bundled modules, keeping the APIs backward compatible and naming things in the same style.
+The only exception to this is, yet unpublished, `io.loop` module, which implements Free-Monad-based IO.
+
+At the moment, `kisstdlib` is an alpha work in progress software with an unstable API.
+
+# <span id="pieces"/>Parts and pieces
+
+`kisstdlib` consists of:
+
+- a bunch of Python modules under [./kisstdlib](./kisstdlib);
+
+  they are pretty well-documented there, though there's no generated `sphinx` docs yet;
+
+- a bunch of human-readable [examples](./example) using those modules;
+
+- some useful thin-wrapper programs over `kisstdlib` functions, useful for writing whole-program and/or fixed-output/extensional-equality tests;
+
+  the documentation of which gets rendered into the following "Usage" section;
+
+- as well as some testing-related infrastructure built on top of this, in [./devscript](./devscript).
+
+# Usage
+
+## describe-forest
+
+Produce a plain-text recursive deterministic `find`/`ls`/`stat`-like description of given file and/or directory inputs.
+
+The output format is designed to be descriptive and easily `diff`able while also producing minimally dissimilar outputs for similar inputs, even when those inputs contain lots of symlinks and/or hardlinks.
+I.e., essentially, this is an alternative to `ls -lR` and/or `find . -exec ls -l {} \;` which generates outputs that change very little when files with multiple symlinks and/or hardlinks change.
+
+This is most useful for testing code that produces filesystem trees.
+
+The most verbose output format this program can produce, for a single input file
+
+```bash
+describe-forest --full path/to/README.md
+```
+
+looks as follows:
+
+```
+. reg mode 644 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+```
+
+Note how both the path to and the name of the file do not appear in the output.
+This is what you would want for doing things like
+
+```bash
+if ! diff -U 0 <(describe-forest --full v1/path/to/README.md) <(describe-forest --full v2/path/to/README.md) ; then
+    echo "output changed between versions!" >&2
+    exit 1
+fi
+```
+
+which this program is designed for.
+
+For a single input directory
+
+```bash
+describe-forest --full path/to/dir
+```
+
+the output looks similar to this:
+
+```
+. dir mode 700 mtime [2025-01-01 00:00:00]
+afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+sub dir mode 700 mtime [2025-01-01 00:03:00]
+sub/afile-hardlink.jpg ref ==> afile.jpg
+sub/afile-symlink.jpg sym mode 777 mtime [2025-01-01 00:59:59] -> ../afile.jpg
+sub/zfile-hardlink.jpg reg mode 600 mtime [2025-01-01 00:02:00] size 256 sha256 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+unix-socket ??? mode 600 mtime [2025-01-01 01:00:00] size 0
+zfile.jpg ref ==> sub/zfile-hardlink.jpg
+```
+
+Hardlinks, which are denoted by `ref`s above, are processed as follows:
+
+- each new file encountered in lexicographic walk is rendered fully,
+- files with repeated dev+inode numbers are rendered by emitting `ref ==> ` followed by the full path (or `ref => ` followed by the relative path, with `--relative-hardlink`) to the previously encountered element.
+
+This way, renaming a file in the input changes at most two lines.
+
+Symlinks are rendered by simply emitting the path they store, unless `--follow-symlinks` is given, in which case the targets they point to get rendered instead.
+
+Multiple inputs get named by numbering them starting from "0".
+Thus, for instance, running this program with the same input file given twice
+
+```bash
+describe-forest --full path/to/README.md path/to/README.md
+```
+
+produces something like:
+
+```
+0 reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+1 ref ==> 0
+```
+
+And giving the same directory with that file inside twice produces:
+
+```
+0 dir mode 700 mtime [2025-01-01 00:00:00]
+0/afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+1 dir mode 700 mtime [2025-01-01 00:00:00]
+1/afile.jpg ref ==> 0/afile.jpg
+```
+
+In its default output format, though, the program emits only `size`s and `sha256`s, when appropriate:
+
+```
+. dir
+afile.jpg reg size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+```
+
+which is what you would usually want for writing tests.
+Though, if you are testing `rsync` or some such, feel free to use other options described below.
+
+See `devscript` directory in `kisstdlib`'s repository for examples of some shell machinery that uses this to implement arbitrary-program fixed-output tests, which is a nice and simple way to test programs by testing their outputs against outputs of different versions of themselves.
+
+Also, internally, this programs is actually a thin wrapper over `describe_forest` function of `kisstdlib.fs` Python module, which can be used with `pytest` or some such.
+
+- positional arguments:
+  - `PATH`
+  : input directories
+
+- options:
+  - `--version`
+  : show program's version number and exit
+  - `-h, --help`
+  : show this help message and exit
+  - `--markdown`
+  : show `--help` formatted in Markdown
+  - `--numbers`
+  : emit number prefixes even with a single input `PATH`
+  - `--literal`
+  : emit paths without escaping them even when they contain special symbols
+  - `--modes`
+  : emit file modes
+  - `--mtimes`
+  : emit file mtimes
+  - `--no-sizes`
+  : do not emit file sizes
+  - `--full`
+  : an alias for `--mtimes --modes`
+  - `--relative, --relative-hardlinks`
+  : emit relative paths when emitting `ref`s
+  - `-L, --dereference, --follow-symlinks`
+  : follow all symbolic links; replaces all `sym` elements of the output with description of symlink targets
+  - `--time-precision INT`
+  : time precision (as a negative power of 10); default: `0`, which means seconds, set to `9` for nanosecond precision
+  - `--hash-length INT`
+  : cut hashes by taking their prefixes of this many characters; default: print them whole
+
+# Development: `./test-example.sh [--help] [--wine]`
+
+Check that all `kisstdlib/example/*.py` are fixed-output.

kisstdlib-0.0.9/kisstdlib/__init__.py

@@ -0,0 +1,125 @@
+# Copyright (c) 2018-2025 Jan Malakhovski <oxij@oxij.org>
+#
+# This file is a part of `kisstdlib` project.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+"""Re-exports allowing to simply use `from kisstdlib import *` in most simple programs."""
+
+import os.path as _op
+import sys as _sys
+import typing as _t
+
+from gettext import gettext, ngettext
+
+from .base import *
+from .failure import *
+from .string_ext import *
+from .itertools_ext import *
+from .io.stdio import *
+from .logging_ext import *
+from .signal_ext import *
+
+
+def setup_kisstdlib(
+    prog: str | None = None,
+    do_setup_stdio: bool = True,
+    signals: list[str] | None = None,
+    **kwargs: _t.Any,
+) -> tuple[LogCounter, ANSILogHandler]:
+    """Run all `setup_*` functions of all other `kisstdlib.*` modules and setup
+    everything they configure to play well with each other.
+    """
+    if prog is None:
+        prog = _op.basename(_sys.argv[0])
+    if do_setup_stdio:
+        setup_stdio()
+    res = setup_logging(prog, **kwargs)
+    setup_delay_signal_messages(prog + ": ")
+    setup_delay_signals(signals)
+    return res
+
+
+def run_kisstdlib_main(
+    setup_logging_result: tuple[LogCounter, ANSILogHandler],
+    func: _t.Callable[AParamSpec, None],
+    /,
+    *args: AParamSpec.args,
+    **kwargs: AParamSpec.kwargs,
+) -> None:
+    """Run given function with given arguments under `with no_signals()` block while
+    properly handling all exceptions in `ExceptionGroup`s,
+    `CatastrophicFailure`s, `Exception`s, etc it raises by turning them into
+    `logging.error`s. Then, flush `stdio`, report warning and error counts, and
+    call `sys.exit` with a non-zero exit code if the function itself
+    `sys.exit`ed with a non-zero code, or if it `raise`d any exceptions, or
+    logged any errors via `logging`.
+
+    I.e., this functions abstracts away all the boilerplate code you need to
+    write to build a proper UNIXy CLI program.
+
+    `setup_logging_result` argument should be supplied from the result of
+    `setup_logging` or `setup_kisstdlib`.
+    """
+    counter, logger = setup_logging_result
+    exit_code: int = 0
+
+    try:
+        try:
+            with no_signals():
+                func(*args, **kwargs)
+        except SystemExit as exc:
+            code = exc.code
+            if code is not None:
+                if isinstance(code, str):
+                    critical("%s", code)
+                else:
+                    exit_code = code
+    except* (SignalInterrupt, KeyboardInterrupt):
+        critical(gettext("Interrupted!"))
+    except* CatastrophicFailure as excs:
+        ce: CatastrophicFailure
+        for ce in flat_exceptions(excs):
+            critical("%s", ce.get_message(gettext))
+    except* BaseException as excs:
+        be: BaseException
+        for be in flat_exceptions(excs):
+            critical(gettext("Uncaught exception: %s"), str(be), exc_info=be)
+
+    logger.flush()
+    stdout.flush()
+    stderr.flush()
+
+    num_warnings = counter.warnings
+    num_errors = counter.errors
+
+    if num_warnings > 0:
+        warning(
+            ngettext("There was %d warning!", "There were %d warnings!", num_warnings),
+            num_warnings,
+        )
+    if num_errors > 0:
+        error(
+            ngettext("There was %d error!", "There were %d errors!", num_errors),
+            num_errors,
+        )
+        if exit_code == 0:
+            exit_code = 1
+
+    _sys.exit(exit_code)