dotted-notation 0.48.1__tar.gz → 0.49.1__tar.gz

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/CHANGELOG.md

@@ -3,6 +3,44 @@
 All notable changes to `dotted` are recorded here. Versions prior to
 the ones listed are omitted — browse git history for earlier entries.
 
+## [0.49.1]
+
+### Documentation
+- The README describes a reference, `$$(expr)`, as an inline evaluation
+  expression: the dotted expression inside, transforms included, is
+  evaluated during traversal and its value stands where it is written. In a
+  transform argument, guard or filter value that is the value itself; in an
+  access position it is the key looked up.
+
+## [0.49.0]
+
+### Added
+- Packing: a `;` just inside either paren of a group, `(a,b;)` or
+  `(;a,b)`, makes the group yield its matches as one tuple instead of
+  one by one. The group keeps its kind, `(a&b;)` packs both or nothing,
+  `(a#,b;)` what the cut lets through, `(a.*;)` a one-branch group, and
+  nothing matched packs to `()`. The tuple is one value, so a packed
+  path is not a pattern and a transform after it applies once, which is
+  how an aggregate over a pattern is written: `(a.*;)|sum` is one
+  number, `items[*].(p[*];)|sum` one per item. Guards compare the
+  tuple, `(a.*;)|sum>2`, and the path may continue into it,
+  `(a.*;)[0]`. A packed group may lead a filter key, running from each
+  item: `items[(p[*];)|sum>2]` filters by an aggregate over the item.
+  `update` and `remove` through a pack raise `ValueError`; the SQL
+  translator raises `TranslationError`.
+- `|sum` and `|mean` transforms, on any container value.
+
+### Changed
+- **Breaking**: `;` is a reserved character. Quote a key that contains
+  one: `"a;b"`.
+
+### Fixed
+- On the compiled engine, groups and filters compared by identity, so
+  `parse('(a,b)') == parse('(a, b)')` was `False` there and `True` on
+  the Python engine. A compiled class that defines `__hash__` alone
+  does not inherit `__eq__`; `OpGroup` and the filter classes now
+  spell theirs out.
+
 ## [0.48.1]
 
 ### Fixed

{dotted_notation-0.48.1/dotted_notation.egg-info → dotted_notation-0.49.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dotted_notation
-Version: 0.48.1
+Version: 0.49.1
 Summary: Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms
 Author-email: Frey Waid <logophage1@gmail.com>
 License-Expression: MIT
@@ -274,6 +274,11 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.49.0
+- **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
+  [Packing](#packing)). If you have keys containing a literal `;`, quote
+  them: `"a;b"`.
+
 ### v0.47.0
 - **`!` binds tighter than guards, filters and transforms.** `!a=5` is now
   `(!a)=5`, the values other than `a`'s that equal 5; the old meaning,
@@ -1391,7 +1396,8 @@ To chain through the items, use a pattern instead:
 ## Substitutions and References
 
 All `$`-prefixed syntax falls into two categories: **substitutions** (resolved
-at replace time) and **references** (resolved during traversal).
+at replace time) and **references**, inline expressions evaluated during
+traversal.
 
 | Syntax | Type | Resolved against |
 |---|---|---|
@@ -1497,14 +1503,22 @@ the parsed result, so a path used many times can be compiled once:
 <a id="references"></a>
 ### References
 
-A **reference** resolves a dotted path against the root object during traversal
-and uses the result as a literal key. The syntax is `$$(path)`:
+A **reference**, `$$(expr)`, is an inline evaluation expression. The dotted
+expression inside is evaluated during traversal, as `get` would evaluate it
+against the root, and its value stands where the reference is written. The
+expression is a full dotted path, so it may carry patterns and transforms:
+`$$(config.offset|mul:10)` is the value of `config.offset`, times ten.
+
+Where the reference stands decides what its value does. In a transform
+argument, a guard value or a filter value it is the value itself (see
+[References in transforms and guards](#references-in-transforms-and-guards)).
+In an access position, as a key or slot, it is the key to look up:
 
     >>> data = {'config': {'field': 'name'}, 'name': 'Alice'}
     >>> dotted.get(data, '$$(config.field)')
     'Alice'
 
-Here `$$(config.field)` resolves to `'name'`, then looks up `data['name']`.
+Here `$$(config.field)` evaluates to `'name'`, then looks up `data['name']`.
 
 References work with all access types and in any position:
 
@@ -1601,6 +1615,12 @@ and so on:
     >>> dotted.get(data, 'n|add:$$(config.offset)')
     15
 
+The expression inside may chain transforms of its own; they apply to its value
+before it is passed:
+
+    >>> dotted.get(data, 'n|add:$$(config.offset|mul:10)')
+    105
+
     >>> data = {'items': [{'price': 100, 'rate': 1.5}, {'price': 20, 'rate': 2}]}
     >>> dotted.get(data, 'items[*].price|mul:$$(^^rate)')
     (150.0, 40)
@@ -2147,6 +2167,77 @@ fallback as disjunction—first concrete path (last to first):
     >>> dotted.update({'a': {}}, 'a(.x,.y)?', 99)    # nothing matches → creates last (.y)
     {'a': {'y': 99}}
 
+#### Packing
+
+A group yields its matches one by one. A `;` just inside either paren
+**packs** them: the group yields one value, the tuple of what it would
+have yielded. `(a,b;)` and `(;a,b)` are the same pack, and the group
+keeps its own kind—`(a,b;)` packs the branches that match, `(a&b;)` packs
+both or nothing, `(a#,b;)` packs what the cut lets through, `(a,b;)?`
+packs the first match—and `(a.*;)` packs a one-branch group. Nothing
+matched packs to `()`:
+
+    >>> d = {'a': {'x': 1, 'y': 2}, 'b': 3}
+    >>> dotted.get(d, '(a.*;)')
+    (1, 2)
+    >>> dotted.get(d, '(b,zz;)')
+    (3,)
+    >>> dotted.get(d, '(b&zz;)')
+    ()
+    >>> dotted.get(d, '(zz.*;)')
+    ()
+
+The tuple is one value, so a packed path is not a pattern: `get` returns
+the value bare, and a transform after the pack applies to the tuple once.
+This is how an aggregate over a pattern is written—the pack gathers, an
+ordinary [transform](#transforms) reduces:
+
+    >>> dotted.get(d, '(a.*;)|sum')
+    3
+    >>> dotted.get(d, '(a.*;)|mean')
+    1.5
+    >>> dotted.get(d, '(a.*;)|set|len')
+    2
+    >>> dotted.get({'a': 1, 'b': 2}, '(a,b;)|sum')      # not (a|sum,b|sum)
+    3
+
+The group runs from the node the pack sits at, so a pack under a pattern
+gives one tuple per match—a group-by—while a pack around the whole path
+gathers across every level inside it:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[*].(p[*];)|sum')
+    (3, 3, 0)
+    >>> dotted.get(d, '(items[*].p[*];)|sum')
+    6
+
+The path may continue into the tuple, and a guard compares it, with
+[guard transforms](#guard-transforms) before it:
+
+    >>> d = {'a': {'x': 1, 'y': 2}}
+    >>> dotted.get(d, '(a.*;)[0]')
+    1
+    >>> dotted.get(d, '(a.*;)|sum>2')
+    (1, 2)
+
+A packed group may also lead a [filter](#filters) key, where it runs
+from each item and its tuple is the value compared—filter by an
+aggregate over the item:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[(p[*];)|sum>2]')
+    [{'p': [1, 2]}, {'p': [3]}]
+    >>> dotted.get(d, 'items[*&(p[*];)|len=0]')
+    ({'p': []},)
+
+A pack is its own path segment: `walk` and `expand` give it back as
+written, `(a.*;)|sum`. There is nothing under a tuple of matches to write
+to, so `update` and `remove` through a pack raise `ValueError`, and a pack
+cannot be negated. `;` is a reserved character; quote a key that contains
+one, `"a;b"`.
+
+`(a.*;)[*]` yields the matches again: `[*]` is the inverse of packing.
+
 #### Negation (NOT)
 
 Use `!` prefix to exclude keys matching a pattern:
@@ -2198,7 +2289,8 @@ A guard, filter or transform applies to the whole expression on its left:
 an access path (on its last segment), a `!` expression (on what it
 yields), or a parenthesized group, where it is shorthand for writing it on
 every branch: `(a,b)=5` is `(a=5,b=5)` and `(a,b)|int` is `(a|int,b|int)`,
-which is also how `assemble` writes them.
+which is also how `assemble` writes them. A [packed](#packing) group
+yields one value, so `(a,b;)|sum` applies to that tuple once.
 
     >>> d = {'a': 1, 'b': 2, 'c': 3}
     >>> dotted.get(d, '!a&b,c')       # ((!a)&b), c
@@ -3068,6 +3160,8 @@ decorator.
 | `coalesce` | fallbacks... | The first of the value and the fallbacks that is not `None`: `\|coalesce:0` |
 | `strip` | `chars`, `raises` | Strip whitespace or specified chars |
 | `len` | `default` | Get length. Optional default if not sized: `\|len:0` |
+| `sum` | `start`, `raises` | Sum a container: `\|sum`, `\|sum:10`; over a pattern, `(a.*;)\|sum` |
+| `mean` | `raises` | Arithmetic mean of a container: `(a.*;)\|mean` |
 | `lowercase` | `raises` | Convert string to lowercase |
 | `uppercase` | `raises` | Convert string to uppercase |
 | `add` | `rhs`, `raises` | Add value: `\|add:10` |

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/README.md

@@ -227,6 +227,11 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.49.0
+- **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
+  [Packing](#packing)). If you have keys containing a literal `;`, quote
+  them: `"a;b"`.
+
 ### v0.47.0
 - **`!` binds tighter than guards, filters and transforms.** `!a=5` is now
   `(!a)=5`, the values other than `a`'s that equal 5; the old meaning,
@@ -1344,7 +1349,8 @@ To chain through the items, use a pattern instead:
 ## Substitutions and References
 
 All `$`-prefixed syntax falls into two categories: **substitutions** (resolved
-at replace time) and **references** (resolved during traversal).
+at replace time) and **references**, inline expressions evaluated during
+traversal.
 
 | Syntax | Type | Resolved against |
 |---|---|---|
@@ -1450,14 +1456,22 @@ the parsed result, so a path used many times can be compiled once:
 <a id="references"></a>
 ### References
 
-A **reference** resolves a dotted path against the root object during traversal
-and uses the result as a literal key. The syntax is `$$(path)`:
+A **reference**, `$$(expr)`, is an inline evaluation expression. The dotted
+expression inside is evaluated during traversal, as `get` would evaluate it
+against the root, and its value stands where the reference is written. The
+expression is a full dotted path, so it may carry patterns and transforms:
+`$$(config.offset|mul:10)` is the value of `config.offset`, times ten.
+
+Where the reference stands decides what its value does. In a transform
+argument, a guard value or a filter value it is the value itself (see
+[References in transforms and guards](#references-in-transforms-and-guards)).
+In an access position, as a key or slot, it is the key to look up:
 
     >>> data = {'config': {'field': 'name'}, 'name': 'Alice'}
     >>> dotted.get(data, '$$(config.field)')
     'Alice'
 
-Here `$$(config.field)` resolves to `'name'`, then looks up `data['name']`.
+Here `$$(config.field)` evaluates to `'name'`, then looks up `data['name']`.
 
 References work with all access types and in any position:
 
@@ -1554,6 +1568,12 @@ and so on:
     >>> dotted.get(data, 'n|add:$$(config.offset)')
     15
 
+The expression inside may chain transforms of its own; they apply to its value
+before it is passed:
+
+    >>> dotted.get(data, 'n|add:$$(config.offset|mul:10)')
+    105
+
     >>> data = {'items': [{'price': 100, 'rate': 1.5}, {'price': 20, 'rate': 2}]}
     >>> dotted.get(data, 'items[*].price|mul:$$(^^rate)')
     (150.0, 40)
@@ -2100,6 +2120,77 @@ fallback as disjunction—first concrete path (last to first):
     >>> dotted.update({'a': {}}, 'a(.x,.y)?', 99)    # nothing matches → creates last (.y)
     {'a': {'y': 99}}
 
+#### Packing
+
+A group yields its matches one by one. A `;` just inside either paren
+**packs** them: the group yields one value, the tuple of what it would
+have yielded. `(a,b;)` and `(;a,b)` are the same pack, and the group
+keeps its own kind—`(a,b;)` packs the branches that match, `(a&b;)` packs
+both or nothing, `(a#,b;)` packs what the cut lets through, `(a,b;)?`
+packs the first match—and `(a.*;)` packs a one-branch group. Nothing
+matched packs to `()`:
+
+    >>> d = {'a': {'x': 1, 'y': 2}, 'b': 3}
+    >>> dotted.get(d, '(a.*;)')
+    (1, 2)
+    >>> dotted.get(d, '(b,zz;)')
+    (3,)
+    >>> dotted.get(d, '(b&zz;)')
+    ()
+    >>> dotted.get(d, '(zz.*;)')
+    ()
+
+The tuple is one value, so a packed path is not a pattern: `get` returns
+the value bare, and a transform after the pack applies to the tuple once.
+This is how an aggregate over a pattern is written—the pack gathers, an
+ordinary [transform](#transforms) reduces:
+
+    >>> dotted.get(d, '(a.*;)|sum')
+    3
+    >>> dotted.get(d, '(a.*;)|mean')
+    1.5
+    >>> dotted.get(d, '(a.*;)|set|len')
+    2
+    >>> dotted.get({'a': 1, 'b': 2}, '(a,b;)|sum')      # not (a|sum,b|sum)
+    3
+
+The group runs from the node the pack sits at, so a pack under a pattern
+gives one tuple per match—a group-by—while a pack around the whole path
+gathers across every level inside it:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[*].(p[*];)|sum')
+    (3, 3, 0)
+    >>> dotted.get(d, '(items[*].p[*];)|sum')
+    6
+
+The path may continue into the tuple, and a guard compares it, with
+[guard transforms](#guard-transforms) before it:
+
+    >>> d = {'a': {'x': 1, 'y': 2}}
+    >>> dotted.get(d, '(a.*;)[0]')
+    1
+    >>> dotted.get(d, '(a.*;)|sum>2')
+    (1, 2)
+
+A packed group may also lead a [filter](#filters) key, where it runs
+from each item and its tuple is the value compared—filter by an
+aggregate over the item:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[(p[*];)|sum>2]')
+    [{'p': [1, 2]}, {'p': [3]}]
+    >>> dotted.get(d, 'items[*&(p[*];)|len=0]')
+    ({'p': []},)
+
+A pack is its own path segment: `walk` and `expand` give it back as
+written, `(a.*;)|sum`. There is nothing under a tuple of matches to write
+to, so `update` and `remove` through a pack raise `ValueError`, and a pack
+cannot be negated. `;` is a reserved character; quote a key that contains
+one, `"a;b"`.
+
+`(a.*;)[*]` yields the matches again: `[*]` is the inverse of packing.
+
 #### Negation (NOT)
 
 Use `!` prefix to exclude keys matching a pattern:
@@ -2151,7 +2242,8 @@ A guard, filter or transform applies to the whole expression on its left:
 an access path (on its last segment), a `!` expression (on what it
 yields), or a parenthesized group, where it is shorthand for writing it on
 every branch: `(a,b)=5` is `(a=5,b=5)` and `(a,b)|int` is `(a|int,b|int)`,
-which is also how `assemble` writes them.
+which is also how `assemble` writes them. A [packed](#packing) group
+yields one value, so `(a,b;)|sum` applies to that tuple once.
 
     >>> d = {'a': 1, 'b': 2, 'c': 3}
     >>> dotted.get(d, '!a&b,c')       # ((!a)&b), c
@@ -3021,6 +3113,8 @@ decorator.
 | `coalesce` | fallbacks... | The first of the value and the fallbacks that is not `None`: `\|coalesce:0` |
 | `strip` | `chars`, `raises` | Strip whitespace or specified chars |
 | `len` | `default` | Get length. Optional default if not sized: `\|len:0` |
+| `sum` | `start`, `raises` | Sum a container: `\|sum`, `\|sum:10`; over a pattern, `(a.*;)\|sum` |
+| `mean` | `raises` | Arithmetic mean of a container: `(a.*;)\|mean` |
 | `lowercase` | `raises` | Convert string to lowercase |
 | `uppercase` | `raises` | Convert string to uppercase |
 | `add` | `rhs`, `raises` | Add value: `\|add:10` |

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/dotted/filters.py

@@ -52,8 +52,9 @@ class FilterKey(base.MatchOp):
             self.parts = tuple(self.args)
         # Override args with tuple of parts for proper hashing
         self.args = self.parts
-        # For simple keys (single part), expose value for backwards compat
-        self.value = self.parts[0].value if len(self.parts) == 1 else None
+        # For simple keys (single part), expose value for backwards compat;
+        # a packed group, (a,b;), has none
+        self.value = getattr(self.parts[0], 'value', None) if len(self.parts) == 1 else None
 
     def __repr__(self):
         out = []
@@ -66,9 +67,20 @@ class FilterKey(base.MatchOp):
     def __hash__(self):
         return hash(self.parts)
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.parts == other.parts
+
     def is_dotted(self):
         return len(self.parts) > 1
 
+    def is_packed(self):
+        """
+        True if the key starts with a packed group, [(p[*];)|sum>2]: the
+        group runs from the item and its tuple is the key's value.
+        """
+        return bool(self.parts) and hasattr(self.parts[0], 'gather')
+
     def is_template(self):
         """
         True if any part of the filter key contains a substitution reference.
@@ -81,24 +93,29 @@ class FilterKey(base.MatchOp):
         """
         return _any_reference(self.parts)
 
-    def get_values(self, node):
+    def get_values(self, node, settings=base.SETTINGS):
         """
         Get all values from node matching this key, traversing dotted path if needed.
-        Supports slot parts (e.g. tags[*]) to yield each element of a list for matching.
+        Supports slot parts (e.g. tags[*]) to yield each element of a list for matching,
+        and a leading packed group, (p[*];), whose tuple is the value; settings
+        are the traversal's, for references inside the group.
         Yields (value, True) for each match, or (None, False) if no matches.
         """
-        yield from self._get_values(node, list(self.parts))
+        yield from self._get_values(node, list(self.parts), settings)
 
-    def _get_values(self, node, parts):
+    def _get_values(self, node, parts, settings=base.SETTINGS):
         if not parts:
             yield node, True
             return
         part = parts[0]
         rest = parts[1:]
+        if hasattr(part, 'gather'):
+            yield from self._get_values(part.gather(node, settings), rest, settings)
+            return
         if isinstance(part, Slot):
             try:
                 for v in part.values(node):
-                    yield from self._get_values(v, rest)
+                    yield from self._get_values(v, rest, settings)
             except (TypeError, AttributeError):
                 yield None, False
             return
@@ -106,7 +123,7 @@ class FilterKey(base.MatchOp):
             try:
                 s = part.slice(node)
                 val = node[s]
-                yield from self._get_values(val, rest)
+                yield from self._get_values(val, rest, settings)
             except (TypeError, AttributeError, KeyError, IndexError):
                 yield None, False
             return
@@ -127,7 +144,7 @@ class FilterKey(base.MatchOp):
         found = False
         for km in part.matches(node.keys()):
             child = node[km]
-            yield from self._get_values(child, rest)
+            yield from self._get_values(child, rest, settings)
             found = True
             break
         if not found:
@@ -145,9 +162,9 @@ class FilterKey(base.MatchOp):
         """
         For simple keys, delegate to the inner part's matches
         """
-        if not self.is_dotted():
+        if not self.is_dotted() and not self.is_packed():
             return self.parts[0].matches(keys)
-        return ()
+        return iter(())
 
     def matchable(self, op):
         """
@@ -214,6 +231,10 @@ class FilterKeyValue(FilterOp):
     def __hash__(self):
         return hash((self._eq_str, self.key, self.val, self.guard_transforms))
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and (self._eq_str, self.key, self.val, self.guard_transforms) == (other._eq_str, other.key, other.val, other.guard_transforms)
+
     def __repr__(self):
         t_str = ''.join('|' + t.operator() for t in self.guard_transforms) if self.guard_transforms else ''
         return f'{self.key}{t_str}{self._eq_str}{self.val}'
@@ -252,11 +273,11 @@ class FilterKeyValue(FilterOp):
         self.val sees the value as ^, node as ^^ and node's tracked parents
         above that; one that does not resolve matches nothing.
         """
-        if not hasattr(node, 'keys'):
+        if not hasattr(node, 'keys') and not self.key.is_packed():
             return False
         from . import results
         parents = (node,) + (settings.parents or ())
-        for val, found in self.key.get_values(node):
+        for val, found in self.key.get_values(node, settings):
             if not found:
                 continue
             try:
@@ -292,9 +313,13 @@ class FilterKeyValue(FilterOp):
             return None
         if not self.key.matchable(op.key) or not self.val.matchable(op.val):
             return None
-        mk = next(self.key.matches((op.key.value,)), base.marker)
+        if self.key.is_packed():
+            # a packed key has no single value: it matches an equal key
+            mk = self.key.match(op.key)
+        else:
+            mk = next(self.key.matches((op.key.value,)), base.marker)
         mv = next(self.val.matches((op.val.value,)), base.marker)
-        if base.marker in (mk, mv):
+        if mk is None or base.marker in (mk, mv):
             return None
         return type(op)(op.key, op.val)
 
@@ -384,6 +409,10 @@ class FilterGroup(FilterOp):
     def __hash__(self):
         return hash(self.inner)
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.inner == other.inner
+
     def __repr__(self):
         return f'({self.inner})'
 
@@ -434,6 +463,10 @@ class FilterAnd(FilterOp):
     def __hash__(self):
         return hash(self.filters)
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.filters == other.filters
+
     def __repr__(self):
         return '&'.join(str(f) for f in self.filters)
 
@@ -492,6 +525,10 @@ class FilterOr(FilterOp):
     def __hash__(self):
         return hash(self.filters)
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.filters == other.filters
+
     def __repr__(self):
         return ','.join(str(f) for f in self.filters)
 
@@ -538,6 +575,10 @@ class FilterKeyValueFirst(FilterOp):
     def __hash__(self):
         return hash(self.inner)
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.inner == other.inner
+
     def __repr__(self):
         return f'{self.inner}?'
 
@@ -598,6 +639,10 @@ class FilterNot(FilterOp):
     def __hash__(self):
         return hash(('not', self.inner))
 
+    def __eq__(self, other):
+        # spelled out: a compiled class with __hash__ alone compares by identity
+        return self.__class__ == other.__class__ and self.inner == other.inner
+
     def __repr__(self):
         return f'!{self.inner}'
 

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/dotted/grammar.py

@@ -43,6 +43,8 @@ def inner_not_action(t):
     item = t[0]
     if isinstance(item, (list, tuple, pp.ParseResults)) and len(item) == 1:
         item = item[0]  # an op_seq of one op
+    if isinstance(item, groups.Pack):
+        raise ValueError(f'a packed group cannot be negated: !{item!r}')
     if isinstance(item, groups.OpGroupOr):
         # a parenthesized single path, !(a.b): the group adds nothing, so
         # !(a.b) is the same op as !a.b
@@ -92,8 +94,8 @@ def inner_to_opgroup(parsed_result):
     items = list(parsed_result)
     if not items:
         return groups.OpGroupOr()
-    # If there's a single OpGroup result, use it directly
-    if len(items) == 1 and isinstance(items[0], groups.OpGroup):
+    # If there's a single OpGroup (or packed group) result, use it directly
+    if len(items) == 1 and isinstance(items[0], (groups.OpGroup, groups.Pack)):
         inner = items[0]
         # Only unwrap redundant OpGroupOr wrapping a single OpGroupAnd/OpGroupNot
         if isinstance(inner, groups.OpGroupOr):
@@ -220,7 +222,7 @@ none = (pp.Literal('None') | pp.Literal('null')).set_parse_action(matchers.NoneV
 true = (pp.Literal('True') | pp.Literal('true')).set_parse_action(matchers.Boolean)
 false = (pp.Literal('False') | pp.Literal('false')).set_parse_action(matchers.Boolean)
 
-reserved = '.[]*:|+?/=,@&()!~#{}$<>' # {} for container syntax, $ for substitution, <> for comparisons
+reserved = '.[]*:|+?/=,@&()!~#{}$<>;' # {} for container syntax, $ for substitution, <> for comparisons, ; for packing
 breserved = ''.join('\\' + i for i in reserved)
 tilde = L('~')
 
@@ -601,7 +603,10 @@ key = _concat_key | _key_atom
 _filter_key_part = string | _common_pats | numeric_extended | non_integer | numeric_key | word
 filter_key_slice = (lb + slice + rb).set_parse_action(lambda t: access.Slice(*t))
 filter_key_slot = (lb + (_commons | numeric_extended | numeric_slot) + rb).set_parse_action(lambda t: access.Slot(t[0]))
-_filter_key_segment = _filter_key_part | filter_key_slice | filter_key_slot
+# A packed group may lead a filter key, [(p[*];)|sum>2]; set below, once
+# the group grammar exists.
+_filter_key_group = pp.Forward()
+_filter_key_segment = _filter_key_group | _filter_key_part | filter_key_slice | filter_key_slot
 filter_key = pp.Group(
     _filter_key_segment + ZM(filter_key_slice | filter_key_slot | (dot + _filter_key_part))
 ).set_parse_action(flt.FilterKey)
@@ -729,6 +734,10 @@ slicefilter = (lb + filters + ZM(amp + filters) + rb).set_parse_action(access.Sl
 softcut_marker = L('##')
 cut_marker = softcut_marker | L('#')
 
+# Pack marker: a ; just inside either paren, (a,b;) or (;a,b), packs the
+# group's matches into one tuple (see groups.Pack).
+pack_marker = L(';')
+
 # Slot grouping: [(*&filter, +)] for disjunction inside slots; [(*&filter#, +)] for cut
 def _slot_item_action(t):
     _, filters, _, _, args = _classify_tokens(list(t))
@@ -754,8 +763,12 @@ def _grouped_with_type_restriction(parsed_result, first=False):
     """
     Parse action for grouped expressions with optional type restriction.
     Last token may be a TypeSpec; if so, wrap the OpGroup in TypeRestriction.
+    A ; token at either end packs the group (see groups.Pack); the type
+    restriction then guards the tuple.
     """
     items = list(parsed_result)
+    packed = any(isinstance(i, str) and i == ';' for i in items)
+    items = [i for i in items if not (isinstance(i, str) and i == ';')]
     tr = None
     if items and isinstance(items[-1], utypes.TypeSpec):
         tr = items.pop()
@@ -763,32 +776,50 @@ def _grouped_with_type_restriction(parsed_result, first=False):
         grp = inner_to_opgroup_first(items)
     else:
         grp = inner_to_opgroup(items)
+    if packed:
+        grp = groups.Pack(grp)
     if tr:
         grp = tr.wrap(grp)
     return grp
 
-inner_grouped = (lparen + inner_expr + rparen).set_parse_action(
+_packable = lambda expr: Opt(pack_marker) + expr + Opt(pack_marker)
+
+
+def _filter_key_pack_action(s, loc, t):
+    """
+    Parse action for a group leading a filter key: it must be packed,
+    the tuple being the value compared; an unpacked group has no single
+    value there, so it fails to parse as it always did.
+    """
+    grp = _grouped_with_type_restriction(t)
+    if not isinstance(grp, groups.Pack):
+        raise pp.ParseException(s, loc, 'a group in a filter key must be packed, (a,b;)')
+    return grp
+
+_filter_key_group <<= (lparen + _packable(inner_expr) + rparen).set_parse_action(_filter_key_pack_action)
+
+inner_grouped = (lparen + _packable(inner_expr) + rparen).set_parse_action(
     lambda t: _grouped_with_type_restriction(t))
-inner_grouped_first = (lparen + inner_expr + rparen + S('?')).set_parse_action(
+inner_grouped_first = (lparen + _packable(inner_expr) + rparen + S('?')).set_parse_action(
     lambda t: _grouped_with_type_restriction(t, first=True))
 # Explicit-op grouped: used mid-path where bare-key inference is not allowed.
 # Uses a separate precedence tower (_explicit_inner_expr) that only allows
 # op_seqs built from _op_seq_cont (no bare keys even in first position).
 # The FollowedBy is a fast-fail optimization — if the first char after ( can't
 # start an explicit branch, don't even try the tower.
-_explicit_op_start = pp.FollowedBy(pp.Char('.@[!~('))
+_explicit_op_start = pp.FollowedBy(pp.Char('.@[!~(;'))
 _explicit_inner_expr = pp.Forward()
-_inner_grouped_explicit = (lparen + _explicit_op_start + _explicit_inner_expr + rparen).set_parse_action(
+_inner_grouped_explicit = (lparen + _explicit_op_start + _packable(_explicit_inner_expr) + rparen).set_parse_action(
     lambda t: _grouped_with_type_restriction(t))
-_inner_grouped_explicit_first = (lparen + _explicit_op_start + _explicit_inner_expr + rparen + S('?')).set_parse_action(
+_inner_grouped_explicit_first = (lparen + _explicit_op_start + _packable(_explicit_inner_expr) + rparen + S('?')).set_parse_action(
     lambda t: _grouped_with_type_restriction(t, first=True))
 # Bare-key grouped: first branch does NOT start with an access op.
 # Used with prefix shorthand (.(a,b), @(a,b)) where the prefix distributes.
 # Allows ! and ~ (modifiers) but rejects . @ [ ( (access ops / nested groups).
 _bare_key_start = ~pp.FollowedBy(pp.Char('.@[('))
-_inner_grouped_bare = (lparen + _bare_key_start + inner_expr + rparen).set_parse_action(
+_inner_grouped_bare = (lparen + _bare_key_start + _packable(inner_expr) + rparen).set_parse_action(
     lambda t: _grouped_with_type_restriction(t))
-_inner_grouped_bare_first = (lparen + _bare_key_start + inner_expr + rparen + S('?')).set_parse_action(
+_inner_grouped_bare_first = (lparen + _bare_key_start + _packable(inner_expr) + rparen + S('?')).set_parse_action(
     lambda t: _grouped_with_type_restriction(t, first=True))
 
 # Recursive operator: ** (recursive wildcard) and *pattern (recursive chain-following)

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/dotted/groups.py

@@ -92,6 +92,12 @@ class OpGroup(base.TraversalOp):
     def __hash__(self):
         return hash(self.branches)
 
+    def __eq__(self, other):
+        # spelled out, not inherited: a compiled class that defines
+        # __hash__ alone compares by identity (CPython inherits
+        # tp_richcompare only together with tp_hash)
+        return self.__class__ == other.__class__ and self.branches == other.branches
+
     def is_pattern(self):
         return True
 
@@ -841,6 +847,117 @@ class OpGroupNot(OpGroup):
 # =============================================================================
 
 
+class Pack(base.TraversalOp):
+    """
+    A packed group, (a,b;) or (;a,b): the group's matches as one tuple.
+    The group keeps its own kind, so (a,b;) packs whatever branches
+    match, (a&b;) packs both or nothing, (a#,b;) packs what the cut
+    lets through, and (a.*;) packs a one-branch group; no matches pack
+    to (). Yielding one value, it is not a pattern, and a transform
+    after it applies to the tuple once: (a.*;)|sum. The group is run
+    from the node the pack sits at, so a[*].(b.*;) packs per a. Writes
+    through a pack raise. Not a Wrap: the pack is the op, the group
+    its argument, so an unwrapping stops here.
+    """
+
+    def __init__(self, inner):
+        super().__init__(inner)
+        self.inner = inner
+
+    def __repr__(self):
+        return self.operator(top=True)
+
+    def is_template(self):
+        return self.inner.is_template()
+
+    def is_reference(self):
+        return self.inner.is_reference()
+
+    def matchable(self, op):
+        return op == self
+
+    def match(self, op, specials=False):
+        """
+        A pack denotes only itself: it matches an equal pack.
+        """
+        if not self.matchable(op):
+            return None
+        return base.MatchResult(self)
+
+    def do_match(self, rest_pats, path_ops, partial):
+        """
+        One segment, matched by equality (see match).
+        """
+        if not path_ops or not self.match(path_ops[0]):
+            return None
+        rest = base.match_ops(rest_pats, path_ops[1:], partial)
+        if rest is None:
+            return None
+        return [(self, False)] + rest
+
+    def operator(self, top=False):
+        """
+        The group's text with the marker before its closing paren:
+        (a,b;), (a,b;)?.
+        """
+        s = self.inner.operator(top)
+        i = s.rindex(')')
+        return s[:i] + ';' + s[i:]
+
+    def is_pattern(self):
+        return False
+
+    def as_attrs(self):
+        return Pack(self.inner.as_attrs())
+
+    def resolve(self, bindings, partial=False):
+        new_inner = self.inner.resolve(bindings, partial)
+        if new_inner is self.inner:
+            return self
+        return Pack(new_inner)
+
+    def concrete(self, val):
+        """
+        A pack is its own path segment: a walked path reads back as
+        written, (a.*;).
+        """
+        return self
+
+    def gather(self, node, settings=base.SETTINGS):
+        """
+        The tuple of the group's matches from node, stopping at a cut.
+        """
+        return tuple(engine.values((self.inner,), node, settings=settings))
+
+    def items(self, node, settings=base.SETTINGS, filtered=True):
+        yield None, self.gather(node, settings)
+
+    def keys(self, node, settings=base.SETTINGS):
+        yield None
+
+    def values(self, node, settings=base.SETTINGS):
+        yield self.gather(node, settings)
+
+    def is_empty(self, node):
+        return False
+
+    def push_children(self, stack, frame, paths):
+        """
+        One frame, the tuple, with this pack as its path segment; the
+        node the group ran from is its parent for ^^ references.
+        """
+        packed = self.gather(frame.node, frame.settings)
+        cp = frame.prefix + (self,) if paths else frame.prefix
+        stack.push(base.Frame(frame.ops, packed, cp, settings=frame.settings.below(frame.node)))
+        return ()
+
+    def do_update(self, ops, node, val, has_defaults, _path, nop, nop_from_unwrap=False, settings=base.SETTINGS):
+        raise ValueError(f'cannot update through a packed group: {self!r}')
+
+    def do_remove(self, ops, node, val, nop, settings=base.SETTINGS):
+        raise ValueError(f'cannot remove through a packed group: {self!r}')
+
+
 def _attr_branch(branch):
     """
     Convert leading Key to Attr in a branch tuple.

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/dotted/sql/pg.py

@@ -336,6 +336,8 @@ class PostgresMixin:
             op = op.inner
         if isinstance(op, wrappers.TypeRestriction):
             raise TranslationError('type restriction not supported')
+        if isinstance(op.most_inner, _groups.Pack):
+            raise TranslationError('packed group not supported')
         # Groups: each branch is walked independently, so dispatch first
         # (OpGroup has is_pattern=True but shouldn't take the pattern-mode
         # path wholesale).
@@ -459,6 +461,8 @@ class PostgresMixin:
             return self._op_to_jsonpath(op.inner, jvars)
         if isinstance(op, wrappers.TypeRestriction):
             raise TranslationError('type restriction not supported')
+        if isinstance(op.most_inner, _groups.Pack):
+            raise TranslationError('packed group not supported')
         # NB: Slot is a subclass of Key, so check Slot first.
         if isinstance(op, access.Slot):
             return self._key_slot_to_jsonpath(op, is_key=False), None
@@ -674,6 +678,8 @@ class PostgresMixin:
         if isinstance(key, _filters.FilterKey):
             parts = ['@']
             for p in key.parts:
+                if isinstance(p, _groups.Pack):
+                    raise TranslationError('packed group in filter key not supported')
                 if isinstance(p, access.Slot):
                     m = p.op
                     if isinstance(m, matchers.Wildcard):

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/dotted/transforms.py

@@ -3,6 +3,7 @@ Default transforms
 """
 import decimal
 import math
+import statistics
 from .results import Dotted
 
 
@@ -502,6 +503,45 @@ def transform_not_in(val, rhs, *modes):
     return val
 
 
+@transform('sum')
+def transform_sum(val, start=None, *modes):
+    """
+    Sum the elements of val
+        <dotted>|sum                sum(val)
+        <dotted>|sum:<start>        sum(val, <start>)
+        <dotted>|sum::raises        sum(val) or raises
+    >>> import dotted
+    >>> dotted.get({'a': {'x': 1, 'y': 2}}, '(a.*;)|sum')
+    3
+    """
+    try:
+        if start is None:
+            return sum(val)
+        return sum(val, start)
+    except TypeError:
+        if 'raises' in modes:
+            raise
+    return val
+
+
+@transform('mean')
+def transform_mean(val, *modes):
+    """
+    Arithmetic mean of the elements of val
+        <dotted>|mean               statistics.mean(val)
+        <dotted>|mean:raises        statistics.mean(val) or raises
+    >>> import dotted
+    >>> dotted.get({'a': {'x': 1, 'y': 2}}, '(a.*;)|mean')
+    1.5
+    """
+    try:
+        return statistics.mean(val)
+    except (TypeError, statistics.StatisticsError):
+        if 'raises' in modes:
+            raise
+    return val
+
+
 @transform('list')
 def transform_list(val, *modes):
     """

{dotted_notation-0.48.1 → dotted_notation-0.49.1/dotted_notation.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dotted_notation
-Version: 0.48.1
+Version: 0.49.1
 Summary: Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms
 Author-email: Frey Waid <logophage1@gmail.com>
 License-Expression: MIT
@@ -274,6 +274,11 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.49.0
+- **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
+  [Packing](#packing)). If you have keys containing a literal `;`, quote
+  them: `"a;b"`.
+
 ### v0.47.0
 - **`!` binds tighter than guards, filters and transforms.** `!a=5` is now
   `(!a)=5`, the values other than `a`'s that equal 5; the old meaning,
@@ -1391,7 +1396,8 @@ To chain through the items, use a pattern instead:
 ## Substitutions and References
 
 All `$`-prefixed syntax falls into two categories: **substitutions** (resolved
-at replace time) and **references** (resolved during traversal).
+at replace time) and **references**, inline expressions evaluated during
+traversal.
 
 | Syntax | Type | Resolved against |
 |---|---|---|
@@ -1497,14 +1503,22 @@ the parsed result, so a path used many times can be compiled once:
 <a id="references"></a>
 ### References
 
-A **reference** resolves a dotted path against the root object during traversal
-and uses the result as a literal key. The syntax is `$$(path)`:
+A **reference**, `$$(expr)`, is an inline evaluation expression. The dotted
+expression inside is evaluated during traversal, as `get` would evaluate it
+against the root, and its value stands where the reference is written. The
+expression is a full dotted path, so it may carry patterns and transforms:
+`$$(config.offset|mul:10)` is the value of `config.offset`, times ten.
+
+Where the reference stands decides what its value does. In a transform
+argument, a guard value or a filter value it is the value itself (see
+[References in transforms and guards](#references-in-transforms-and-guards)).
+In an access position, as a key or slot, it is the key to look up:
 
     >>> data = {'config': {'field': 'name'}, 'name': 'Alice'}
     >>> dotted.get(data, '$$(config.field)')
     'Alice'
 
-Here `$$(config.field)` resolves to `'name'`, then looks up `data['name']`.
+Here `$$(config.field)` evaluates to `'name'`, then looks up `data['name']`.
 
 References work with all access types and in any position:
 
@@ -1601,6 +1615,12 @@ and so on:
     >>> dotted.get(data, 'n|add:$$(config.offset)')
     15
 
+The expression inside may chain transforms of its own; they apply to its value
+before it is passed:
+
+    >>> dotted.get(data, 'n|add:$$(config.offset|mul:10)')
+    105
+
     >>> data = {'items': [{'price': 100, 'rate': 1.5}, {'price': 20, 'rate': 2}]}
     >>> dotted.get(data, 'items[*].price|mul:$$(^^rate)')
     (150.0, 40)
@@ -2147,6 +2167,77 @@ fallback as disjunction—first concrete path (last to first):
     >>> dotted.update({'a': {}}, 'a(.x,.y)?', 99)    # nothing matches → creates last (.y)
     {'a': {'y': 99}}
 
+#### Packing
+
+A group yields its matches one by one. A `;` just inside either paren
+**packs** them: the group yields one value, the tuple of what it would
+have yielded. `(a,b;)` and `(;a,b)` are the same pack, and the group
+keeps its own kind—`(a,b;)` packs the branches that match, `(a&b;)` packs
+both or nothing, `(a#,b;)` packs what the cut lets through, `(a,b;)?`
+packs the first match—and `(a.*;)` packs a one-branch group. Nothing
+matched packs to `()`:
+
+    >>> d = {'a': {'x': 1, 'y': 2}, 'b': 3}
+    >>> dotted.get(d, '(a.*;)')
+    (1, 2)
+    >>> dotted.get(d, '(b,zz;)')
+    (3,)
+    >>> dotted.get(d, '(b&zz;)')
+    ()
+    >>> dotted.get(d, '(zz.*;)')
+    ()
+
+The tuple is one value, so a packed path is not a pattern: `get` returns
+the value bare, and a transform after the pack applies to the tuple once.
+This is how an aggregate over a pattern is written—the pack gathers, an
+ordinary [transform](#transforms) reduces:
+
+    >>> dotted.get(d, '(a.*;)|sum')
+    3
+    >>> dotted.get(d, '(a.*;)|mean')
+    1.5
+    >>> dotted.get(d, '(a.*;)|set|len')
+    2
+    >>> dotted.get({'a': 1, 'b': 2}, '(a,b;)|sum')      # not (a|sum,b|sum)
+    3
+
+The group runs from the node the pack sits at, so a pack under a pattern
+gives one tuple per match—a group-by—while a pack around the whole path
+gathers across every level inside it:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[*].(p[*];)|sum')
+    (3, 3, 0)
+    >>> dotted.get(d, '(items[*].p[*];)|sum')
+    6
+
+The path may continue into the tuple, and a guard compares it, with
+[guard transforms](#guard-transforms) before it:
+
+    >>> d = {'a': {'x': 1, 'y': 2}}
+    >>> dotted.get(d, '(a.*;)[0]')
+    1
+    >>> dotted.get(d, '(a.*;)|sum>2')
+    (1, 2)
+
+A packed group may also lead a [filter](#filters) key, where it runs
+from each item and its tuple is the value compared—filter by an
+aggregate over the item:
+
+    >>> d = {'items': [{'p': [1, 2]}, {'p': [3]}, {'p': []}]}
+    >>> dotted.get(d, 'items[(p[*];)|sum>2]')
+    [{'p': [1, 2]}, {'p': [3]}]
+    >>> dotted.get(d, 'items[*&(p[*];)|len=0]')
+    ({'p': []},)
+
+A pack is its own path segment: `walk` and `expand` give it back as
+written, `(a.*;)|sum`. There is nothing under a tuple of matches to write
+to, so `update` and `remove` through a pack raise `ValueError`, and a pack
+cannot be negated. `;` is a reserved character; quote a key that contains
+one, `"a;b"`.
+
+`(a.*;)[*]` yields the matches again: `[*]` is the inverse of packing.
+
 #### Negation (NOT)
 
 Use `!` prefix to exclude keys matching a pattern:
@@ -2198,7 +2289,8 @@ A guard, filter or transform applies to the whole expression on its left:
 an access path (on its last segment), a `!` expression (on what it
 yields), or a parenthesized group, where it is shorthand for writing it on
 every branch: `(a,b)=5` is `(a=5,b=5)` and `(a,b)|int` is `(a|int,b|int)`,
-which is also how `assemble` writes them.
+which is also how `assemble` writes them. A [packed](#packing) group
+yields one value, so `(a,b;)|sum` applies to that tuple once.
 
     >>> d = {'a': 1, 'b': 2, 'c': 3}
     >>> dotted.get(d, '!a&b,c')       # ((!a)&b), c
@@ -3068,6 +3160,8 @@ decorator.
 | `coalesce` | fallbacks... | The first of the value and the fallbacks that is not `None`: `\|coalesce:0` |
 | `strip` | `chars`, `raises` | Strip whitespace or specified chars |
 | `len` | `default` | Get length. Optional default if not sized: `\|len:0` |
+| `sum` | `start`, `raises` | Sum a container: `\|sum`, `\|sum:10`; over a pattern, `(a.*;)\|sum` |
+| `mean` | `raises` | Arithmetic mean of a container: `(a.*;)\|mean` |
 | `lowercase` | `raises` | Convert string to lowercase |
 | `uppercase` | `raises` | Convert string to uppercase |
 | `add` | `rhs`, `raises` | Add value: `\|add:10` |

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "dotted_notation"
-version = "0.48.1"
+version = "0.49.1"
 description = "Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms"
 readme = "README.md"
 requires-python = ">=3.6"

{dotted_notation-0.48.1 → dotted_notation-0.49.1}/setup.py

@@ -11,6 +11,7 @@ same files run as Python.
 import os
 
 from setuptools import Extension, setup
+from setuptools.command.build_ext import build_ext as _build_ext
 
 # The engine and the ops it runs. grammar.py, the parser, is left out on
 # purpose: pyparsing inspects its parse actions as Python functions.
@@ -47,7 +48,20 @@ def extensions():
     return cythonize(
         [Extension('dotted.' + name, ['dotted/' + name + '.py'], optional=True) for name in MODULES],
         compiler_directives={'language_level': 3},
+        nthreads=os.cpu_count() or 1,
     )
 
 
-setup(ext_modules=extensions())
+class build_ext(_build_ext):
+    """
+    Compile the extensions one per core. setuptools builds them one after
+    another unless -j is given; the modules are independent, so a release
+    wheel compiles in a fraction of the time on a multi-core runner.
+    """
+    def finalize_options(self):
+        super().finalize_options()
+        if self.parallel is None:
+            self.parallel = os.cpu_count() or 1
+
+
+setup(ext_modules=extensions(), cmdclass={'build_ext': build_ext})