moat-kv 0.71.0__py3-none-any.whl → 0.71.7__py3-none-any.whl

docs/source/common_protocol.rst

@@ -1,47 +0,0 @@
-=================
-MoaT-KV's protocol
-=================
-
-MoaT-KV's native protocol (both client/server and server/server) is based on
-MsgPack. Strings must be valid UTF-8 and are distinct from binary buffers.
-
-++++++++++++++++++
-MsgPack Extensions
-++++++++++++++++++
-
-MoaT-KV is expected to be a transparent protocol. Unknown extensions
-must not cause the reader to crash, and should be round-trip-safe: a client
-which reads an object and modifies attribute A should not modify attribute
-B even if B contains an element with an unknown extension.
-
-This should, but currently does not, apply to extensions 2 and 3.
-
---------
-DateTime
---------
-
-A client must support MsgPack's Timestamp extension.
-
-------
-Bignum
-------
-
-Extension 2 is used for unsigned arbitrary-sized integers.
-
------
-Paths
------
-
-Extension 3 packages a MsgPack path.
-
-MoaT-KV uses Path objects to refer to its nodes. Paths are lists which may
-contain arbitrary msgpack data structures but should be limited to UTF-8
-strings and non-negative integers.
-
-A client must support both Path objects and plain lists when reading. It
-should send Path elements. Using plain lists instead is supported but
-not recommended.
-
-In this documentation's protocol examples, Paths are shown as ``P('some.string')``
-for readability.
-

docs/source/conf.py

@@ -1,201 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-#
-# Documentation build configuration file, created by
-# sphinx-quickstart on Sat Jan 21 19:11:14 2017.
-#
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
-
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
-#
-import os
-import sys
-
-# So autodoc can import our package
-sys.path.insert(0, os.path.abspath("../.."))
-
-# Warn about all references to unknown targets
-nitpicky = True
-# Except for these ones, which we expect to point to unknown targets:
-nitpick_ignore = [
-    # Format is ("sphinx reference type", "string"), e.g.:
-    ("py:obj", "bytes-like"),
-    ("py:class", "RangeSet"),
-    ("py:class", "Any"),
-]
-
-autodoc_inherit_docstrings = False
-
-# -- General configuration ------------------------------------------------
-
-# If your documentation needs a minimal Sphinx version, state it here.
-#
-# needs_sphinx = '1.0'
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
-# ones.
-extensions = [
-    "sphinx.ext.autodoc",
-    "sphinx.ext.intersphinx",
-    "sphinx.ext.coverage",
-    "sphinx.ext.napoleon",
-    "sphinxcontrib_trio",
-]
-
-intersphinx_mapping = {
-    "python": ("https://docs.python.org/3", None),
-    "trio": ("https://trio.readthedocs.io/en/stable", None),
-}
-
-autodoc_member_order = "bysource"
-
-# Add any paths that contain templates here, relative to this directory.
-templates_path = []
-
-# The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-#
-# source_suffix = ['.rst', '.md']
-source_suffix = ".rst"
-
-# The master toctree document.
-master_doc = "index"
-
-# General information about the project.
-project = "MoaT-KV"
-copyright = "The MoaT-KV authors"
-author = "The MoaT-KV authors"
-
-# The version info for the project you're documenting, acts as replacement for
-# |version| and |release|, also used in various other places throughout the
-# built documents.
-#
-# The short X.Y version.
-import pkg_resources
-
-version = pkg_resources.get_distribution("moat.kv")._version
-
-# The full version, including alpha/beta/rc tags.
-release = version
-
-# The language for content autogenerated by Sphinx. Refer to documentation
-# for a list of supported languages.
-#
-# This is also used if you do content translation via gettext catalogs.
-# Usually you set "language" from the command line for these cases.
-language = None
-
-# List of patterns, relative to source directory, that match files and
-# directories to ignore when looking for source files.
-# This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = []
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = "sphinx"
-
-# The default language for :: blocks
-highlight_language = "python3"
-
-# If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = False
-
-
-# -- Options for HTML output ----------------------------------------------
-
-# The theme to use for HTML and HTML Help pages.  See the documentation for
-# a list of builtin themes.
-#
-# html_theme = 'alabaster'
-
-# We have to set this ourselves, not only because it's useful for local
-# testing, but also because if we don't then RTD will throw away our
-# html_theme_options.
-import sphinx_rtd_theme
-
-html_theme = "sphinx_rtd_theme"
-html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
-
-# Theme options are theme-specific and customize the look and feel of a theme
-# further.  For a list of options available for each theme, see the
-# documentation.
-#
-html_theme_options = {
-    # default is 2
-    # show deeper nesting in the RTD theme's sidebar TOC
-    # https://stackoverflow.com/questions/27669376/
-    # I'm not 100% sure this actually does anything with our current
-    # versions/settings...
-    "navigation_depth": 4,
-    "logo_only": True,
-}
-
-# Add any paths that contain custom static files (such as style sheets) here,
-# relative to this directory. They are copied after the builtin static files,
-# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ["_static"]
-
-
-# -- Options for HTMLHelp output ------------------------------------------
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = "moat.kv-doc"
-
-
-# -- Options for LaTeX output ---------------------------------------------
-
-latex_elements = {
-    # The paper size ('letterpaper' or 'a4paper').
-    #
-    # 'papersize': 'letterpaper',
-    # The font size ('10pt', '11pt' or '12pt').
-    #
-    # 'pointsize': '10pt',
-    # Additional stuff for the LaTeX preamble.
-    #
-    # 'preamble': '',
-    # Latex figure (float) alignment
-    #
-    # 'figure_align': 'htbp',
-}
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title,
-#  author, documentclass [howto, manual, or own class]).
-latex_documents = [
-    (master_doc, "moat.kv.tex", "Trio Documentation", author, "manual"),
-]
-
-
-# -- Options for manual page output ---------------------------------------
-
-# One entry per manual page. List of tuples
-# (source start file, name, description, authors, manual section).
-man_pages = [(master_doc, "moat.kv", "MoaT-KV Documentation", [author], 1)]
-
-
-# -- Options for Texinfo output -------------------------------------------
-
-# Grouping the document tree into Texinfo files. List of tuples
-# (source start file, target name, title, author,
-#  dir menu entry, description, category)
-texinfo_documents = [
-    (
-        master_doc,
-        "moat.kv",
-        "MoaT-KV Documentation",
-        author,
-        "MoaT-KV",
-        "A distributed no-master key-value store",
-        "Miscellaneous",
-    ),
-]

docs/source/debugging.rst

@@ -1,70 +0,0 @@
-==============================
-Fixing MoaT-KV network problems
-==============================
-
-As the MoaT-KV network is fully asynchronous, there's no way to avoid
-getting into trouble – there's no arbitration of inconsistent data.
-
-This document explains how to get back out, if necessary.
-
-Missing data
-============
-
-See the `Server protocol <server_protocol>` for details on how MoaT-KV
-works. From that document it's obvious that when a node increments its
-``tick`` but the associated data gets lost (e.g. if the node or its Serf
-agent crashes), you have a problem.
-
-Worse: a server will not start if the "missing" list is non-empty. The
-problem is that stale data causes difficult-to-resolve inconsistencies
-when written to. TODO: allow the server to be in maintainer-only mode when
-that happens.
-
-First, run ``moat kv internal state -ndmrk``. Your output will look
-somewhat like this::
-
-    deleted:  # Ticks known to be deleted
-      test1:
-      - 12
-    known:  # Ticks known to be superseded
-      test1:
-      - 1
-      - - 3
-        - 10
-      test2:
-      - 1
-    missing:  # Ticks we need to worry about
-      test1:
-      - 2
-    node: test1  # the server we just asked
-    nodes:  # all known nodes and their ticks
-      test1: 12
-      test2: 1
-    remote_missing: {}  # used in recovery
-    tock: 82  # MoaT-KV's global event counter
-    
-This is not healthy: The ``missing`` element contains data. You can
-manually mark the offending data as stale::
-
-   one $ moat kv internal mark test1 2
-   known:
-      test1:
-      - - 1
-        - 11
-      test2:
-      - 1
-    node: test1
-    tock: 92  # If this is not higher than before, clean your glasses ;-)
-    one $
-
-This shows that the offending ``tick`` has been successfully added to the
-``known`` list. Calling ``moat kv internal state -m`` verifies that
-the list is now empty.
-
-Use the ``--broadcast`` flag to send this message to all MoaT-KV servers,
-not just the one you're a client of.
-
-This action will allow the bad record to re-surface when the node that has
-the record reconnects, assuming that there is one. You can use the ``mark``
-command's ``--deleted`` flag to ensure that it will be discarded instead.
-

docs/source/extend.rst

@@ -1,37 +0,0 @@
-================
-Extending MoaT-KV
-================
-
-MoaT-KV comes with a built-in extension mechanism for its command line,
-based on Python namespaces and import hackery.
-
-Yor extension needs to ship a ``moat.kv.NAME`` module, with a
-``_main.py`` file that exports a ``cli`` command (usually used as an
-`asyncclick.group`). This adds the subcommand ``NAME`` to ``moat kv``.
-
-Command line helper
-===================
-
-MoaT-KV commands follow a standard scheme (TODO some don't yet):
-
-* what kind of object do you want to affect
-* the name of the object to affect (or create)
-* [ maybe start over with a sub-object ]
-* the action you want to take
-* some options affecting the action, and/or
-* the generic set of parameter+value options (``-v``/``-e``/``-p``)
-
-In order to simplify implementing that, there's a couple of helper methods.
-
-`moat.kv.obj.command.std_command` takes a ``click.Group`` command and
-attaches a subgroup with standard add/set/delete commands to it. The
-new group is returned so you can attach more commands to it if you want.
-
-`moat.util.attr_args` attaches MoaT-KV's generic parameter+value options.
-
-`moat.util.process_args` takes a dict (usually) and the generic options'
-variables (``set_``, ``vars_``, ``eval_``, ``path_``, ``proxy_``)
-and applies them.
-
-It's the caller's job to verify that the result is sane. TODO: support
-using a validation library (probably jsonschema).

docs/source/history.rst

@@ -1,36 +0,0 @@
-Release history
-===============
-
-.. currentmodule:: moat.kv
-
-.. towncrier release notes start
-
-Migration from DistKV
-=====================
-
-As of 2023-05, the protocol is unchanged.
-
-To re-use the DistKV data as-is, copy ``/etc/distkv.cfg``  to
-``/etc/moat/moat.cfg``. Then edit ``moat.cfg``:
-
-* Move the ``logging:`` entry to the bottom if it isn't there already
-* Indent everything above that entry two spaces
-* Insert a line ``kv:`` as the first line of the file (not indented!)
-
-Then, add these items as appropriate::
-
-    kv:
-      inv:
-        prefix: !P :.distkv.inventory
-      config:
-        prefix: !P :.distkv.config
-      errors:
-        prefix: !P :.distkv.errors
-      codes:
-        prefix: !P :.distkv.code.proc
-      modules:
-        prefix: !P :.distkv.code.module
-      runner:
-        prefix: !P :.distkv.run
-        state: !P :.distkv.state
-    

docs/source/index.rst

@@ -1,75 +0,0 @@
-.. documentation master file, created by
-   sphinx-quickstart on Sat Jan 21 19:11:14 2017.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-
-===============================================
-MoaT-KV: A distributed no-master key-value store
-===============================================
-
-Rationale
-=========
-
-Any kind of distributed storage is subject to the CAP theorem (also called
-"Brewer's theorem"): you can't get all of (global) Consistency,
-Availability, and Partition tolerance. The problem is that you do want all
-three of these.
-
-One way around this problem is to recognize that on most KV storage
-systems, any given record is rarely (if ever) changed by more than one
-entity at the same time. Thus, a simple gossip protocol is sufficient
-for distributing data.
-
-MoaT-KV is intended to be used in a mostly-RAM architecture. There is no
-disk-based storage backend; snapshots and event logs are used to restore a
-system, if necessary.
-
-See `Protocol Overview <overview.html>` for details about MoaT-KV's
-choices.
-
-API
-===
-
-MoaT-KV offers an efficient interface to access and change data.
-
-
-Status
-======
-
-MoaT-KV is in production use as the backbone of the author's home and office
-automation setup.
-
-Note that as of MoaT-KV 0.30, multi-word paths were replaced with dotted
-strings. Some pieces of documentation might still reflect the old style.
-
-.. toctree::
-   :maxdepth: 2
-
-   overview.rst
-   tutorial.rst
-   startup.rst
-   command_line.rst
-   common_protocol.rst
-   client_protocol.rst
-   server_protocol.rst
-   auth.rst
-   acls.rst
-   code.rst
-   model.rst
-   translator.rst
-   debugging.rst
-   extend.rst
-   related.rst
-
-   TODO.rst
-   history.rst
-
-====================
- Indices and tables
-====================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
-* :ref:`glossary`

docs/source/model.rst

@@ -1,54 +0,0 @@
-==========
-Data Model
-==========
-
-This section documents some of MoaT-KV's server-internal classes.
-
-
-.. automodule:: moat.kv.model
-   :members:
-
-ACLs
-----
-
-ACL checks are performed by :class:`~moat.kv.types.ACLFinder`. This class
-collects all relevant ACL entries for any given (sub)path, sorted by
-depth-first specificty. This basically means that you collect all ACLs
-that could possibly match a path and sort them; the ``+`` and ``#``
-wildcards get sorted last. Then the system picks the first entry that
-actually has a value.
-
-This basically means that if you have a path ``a b c d e f g`` and ACLs ``a
-b # g`` and ``a # d e f g``, the first ACL will match because ``b`` is
-more specific than ``#``, even though the second ACL is longer and thus
-could be regarded as being more specific. However, the current rule is more
-stable when used with complex ACLs and thus more secure.
-
-.. autoclass:: moat.kv.types.ACLFinder
-   :members:
-
-Helper methods and classes
---------------------------
-
-.. autoclass:: moat.kv.util.MsgWriter
-   :members:
-
-.. automodule:: moat.kv.util
-   :members:
-
-.. py:data:: moat.kv.util.NotGiven
-
-   This object marks the absence of information where simply not using the
-   data element or keyword at all would be inconvenient.
-
-   For instance, in ``def fn(value=NotGiven, **kw)`` you'd need to test
-   ``'value'  in kw``, or use an exception. The problem is that this would
-   not show up in the function's signature.
-
-   With ``NotGiven`` you can simply test ``value is`` (or ``is not``) ``NotGiven``.
-
-.. automodule:: moat.kv.runner
-   :members:
-
-.. automodule:: moat.kv.actor
-   :members:

docs/source/overview.rst

@@ -1,83 +0,0 @@
-=======================
-Principles of operation
-=======================
-
-MoaT-KV relies on the fact that on most KV storage systems, any given record
-is rarely (if ever) changed by more than one entity at the same time. Thus,
-a simple gossip protocol is sufficient for distributing data.
-
-To recover from missed changes, each node in a MoaT-KV network maintains a
-change counter ("tick"). All data records (:class:`moat.kv.model.Entry`) are
-tagged with a chain of events (:class:`moat.kv.model.NodeEvent`), consisting
-of the ``n`` most recent ``(node, tick)`` values which changed this
-entry. Nodes do not appear in a chain more than once. Dropped ticks
-are added to a per-node list of "known"(-to-have-been-superseded) counter
-values.
-
-The maximum chain length is determined by the number of partitions a MoaT-KV
-network might split into. Thus the network guarantees that it is possible
-which side of a split modified a record when the split is healed.
-
-If both sides did, the conflict is resolved deterministically.
-TODO: when this happens, send a notification to clients.
-
-After a network split, a four-step protocol re-synchronizes the
-participants:
-
-* broadcast the current counters
-
-* broadcast known-value and known-deleted lists
-
-* broadcast a list of missing node events
-
-* broadcast the missed data
-
-MoaT-KV does not have a master node, much less a consensus-based election
-system (Raft, Paxos, …). Instead, MoaT-KV uses an `asyncactor.Actor` to
-compile a short list of available servers that's broadcast every few
-seconds.
-
-When a partitioned network is re-joined, the current housekeepers are
-responsible for driving and monitoring the re-sync protocol.
-
-
-Storage
-=======
-
-MoaT-KV is intended to be used in a mostly-RAM architecture. There is no
-disk-based storage backend; snapshots and event logs are used to restore a
-system, if necessary. Feeding old snapshots to a running system is mostly
-benign, but see below.
-
-MoaT-KV runs on top of MQTT.
-It supports all data types that can be transmitted by
-`MsgPack <https://github.com/msgpack/msgpack/blob/master/spec.md>`.
-
-TODO: MsgPack has extension types, so constructing Python objects is possible.
-
-Record Deletion
-===============
-
-Deleting data records is when MoaT-KV's synchronization protocol breaks
-down, because MoaT-KV can't attach chains to records which no longer exist.
-
-MoaT-KV fixes this by keeping a separate record of deleted entries, or
-rather their chain links. This works well for mostly-static storages but
-becomes a problem on more dynamic systems.
-
-Thus, periodic clean-up is required. This is achieved by creating a
-separate "Delete" Actor group which contains every system with persistent
-storage plus one system per network that's not already covered.
-
-When every node of this group is online, they periodically broadcast a
-tuple of ``tock`` values: one which signals that deletions with earlier
-``tock``\s may safely be flushed, and a high-water limit for the next
-round.
-
-A node that receives this tuple compares the received first value with the
-last transmission's second. If it's higher, deletions may have been missed,
-most likely due to a network outage between that node and the closest Delete
-member. Since the records are now gone, the node will connect to one of the
-Delete group members and send a list of each entry's last-change chain links.
-The recipient will re-broadcast any misses as "new" deletions.
-

docs/source/related.rst

@@ -1,89 +0,0 @@
-============================
-Plugins and related software
-============================
-
-Additions to this list will be appreciated!
-
-
-Home Assistant
-==============
-
-`Home Assistant`__ is a front-end for home automation. (Actually it's more
-than that, but ``moat.kv`` uses it as a front-end and prefers to do the
-automation part itself.)
-
-`moat.kv-hass <https://github.com/M-o-a-T/disthass>`__ documents how to
-connect Home Assistant to the ``moat.kv`` system and helps with creating the
-data structures that teach Home Assistant about moat.kv-controlled sensors
-and actors.
-
-
-KNX
-===
-
-KNX is a serial bus for building control.
-
-`knxd <https://github.com/knxd/knxd/>`__ is a server commonly used to talk to KNX interfaces.
-
-`xknx <https://github.com/XKNX/xknx>`__ is a Python package you can use to talk to ``knxd``.
-
-`moat.kv-knx <https://github.com/M-o-a-T/distknx>`__ connects values stored
-in moat.kv to devices on the KNX bus.
-
- 
-1wire
-=====
-
-`1wire <https://en.wikipedia.org/wiki/1-Wire>`__ is a two- or three-wire
-bus (one signal wire, somewhat-optional 5V power, ground) that is
-frequently used to connect inexpensive sensors and actors to a computer.
-
-`OWFS <https://www.owfs.org/>`__ is the server commonly used on Linux
-systems to talk to 1wire.
-
-`asyncowfs <https://github.com/M-o-a-T/asyncowfs>`__ is a Python package
-that provides a high-level object-oriented async interface to OWFS.
-
-`moat-kv-owfs <https://github.com/M-o-a-T/distknx>`__ uses ``asyncowfs`` to
-connect values stored in ``moat.kv`` to attributes if 1wire devices.
-
-
-Inventory Management
-====================
-
-`moat-kv-inv <https://github.com/M-o-a-T/moat-kv-inv>`__ is a command-line
-extension that stores of hosts, networks and cables. It contains templating
-code so you can auto-create the configuration for your router (if it's text
-instead of some binary format).
-
-
-Akumuli
-=======
-
-`Akumuli <https://akumuli.org/>`__ is a time series database.
-
-`asyncakumuli <https://github.com/M-o-a-T/asyncakumuli>`__ is a Python package
-that provides an async interface to Akumuli.
-
-`moat.kv-akumuli <https://github.com/M-o-a-T/distakumuli>`__ implements a
-background task that monitors values stored in ``moat.kv`` and mirrors them
-into Akumuli, thus saving their history.
-
-
-Binary I/O
-==========
-
-`asyncgpio <https://github.com/M-o-a-T/asyncgpio>`__ is a Python package
-that provides structured access to your computer's I/O ports.
-
-`moat.kv-gpio <https://github.com/M-o-a-T/distgpio>`__ contains code that
-mirrors a binary value stored in ``moat.kv`` to a GPIO pin and vice versa.
-
-
-Wago I/O controllers
-====================
-
-The German company `WAGO Kontakttechnik <https://www.wago.com>`__ makes the
-``750-*`` line of extensible rugged controllers with various modules.
-
-