dotted-notation 0.49.4__tar.gz → 0.51.0__tar.gz

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/CHANGELOG.md

@@ -3,6 +3,36 @@
 All notable changes to `dotted` are recorded here. Versions prior to
 the ones listed are omitted — browse git history for earlier entries.
 
+## [0.51.0]
+
+### Added
+- `ReferenceResolver.lex(text, position)`, a parse-time hook for the
+  tokens dotted's grammar cannot read: a slot, `merged[k]`, a slice
+  bound, `items[:n]`, a filter value, `[*&cur=lot.cur]`, or a guard
+  value, `a=k`. A dotted name is one token. It answers with dotted text
+  read in the token's place (`'$$(k)'`), `None` to leave the
+  `ParseError`, or raises to fail the parse at the token. Tokens dotted
+  reads are never offered; `literals` remaps those at run time. `parse`
+  takes `resolver=`, and the calls taking one parse with it (#66).
+
+## [0.50.0]
+
+### Changed
+- **Breaking**: `[+]` and `[+?]` are the slot past the end, so nothing
+  is ever there to read. `get` through them is the default, not the
+  last item (use `[-1]`); `has` is `False`; `setdefault` always appends,
+  `[+?]` only when the value is absent, and returns the value it set;
+  `remove`, `pluck` and `expand` through them find nothing; a guard on
+  them, `[+]=2`, never passes.
+- In an update, a group branch holding `[+]` always applies, since a
+  write to it always does. `[(*&k=1, +)]` updates the match and appends,
+  as before, and `(a, b[+])` now appends to `b` when it is empty too;
+  with a cut, `[(*&k=1#, +)]`, the append is still the fallback.
+
+### Fixed
+- `update` through a softcut group with an append branch,
+  `[(*&k=1##, +)]`, raised `AttributeError` once the filter matched.
+
 ## [0.49.4]
 
 ### Fixed

{dotted_notation-0.49.4/dotted_notation.egg-info → dotted_notation-0.51.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dotted_notation
-Version: 0.49.4
+Version: 0.51.0
 Summary: Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms
 Author-email: Frey Waid <logophage1@gmail.com>
 License-Expression: MIT
@@ -275,6 +275,18 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.50.0
+- **Nothing is read through `[+]` or `[+?]`.** They are the slot past the
+  end, so `get(d, 'a[+]')` is the default, not the last item; use `a[-1]`.
+  `has` is `False`, `remove`, `pluck` and `expand` through them find
+  nothing, a guard on them never passes, and `setdefault` always appends
+  (`[+?]` only when the value is absent). See
+  [The append `+` operator](#the-append--operator).
+- **An append branch in a group always applies in an update.**
+  `(a, b[+])` appends to `b` whether or not it is empty; it used to append
+  only when `b` already had items. With a cut, `[(*&k=1#, +)]`, the append
+  is still the fallback.
+
 ### v0.49.0
 - **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
   [Packing](#packing)). If you have keys containing a literal `;`, quote
@@ -1757,6 +1769,29 @@ A path handed in already parsed goes through `fold()` again, so a hook must
 be content to see its own output. `Dotted.segments()` gives any parsed
 path's segment values, and a slice of a parsed path, `ops[1:]`, is a path.
 
+A host's own names may stand where dotted has no reading for a bare token:
+a slot, `merged[k]`, a slice bound, `items[:n]`, a filter value,
+`[*&cur=want]`, or a guard value, `a=k`. The parse-time hook
+`lex(text, position)` is offered each such token as written, a dotted name
+whole, with its position, one of `'slots'`, `'slices'`, `'filters'` or
+`'guards'`. It answers with dotted text to read in the token's place, `None`
+to leave the `ParseError`, or raises to fail the parse at the token. The
+answer must be one token dotted reads there, and is not offered again.
+
+    >>> class Names(dotted.ReferenceResolver):
+    ...     def lex(self, text, position):
+    ...         return f'$$({text})'
+    >>> d = {'merged': {'x': 1, 'y': 2}, 'k': 'x', 'c': {'account': 'y'}}
+    >>> dotted.get(d, 'merged[k]', resolver=Names())
+    1
+    >>> dotted.get(d, 'merged[c.account]', resolver=Names())
+    2
+
+A token dotted reads, `m[$x]` or `|sub:taken`, is never offered to `lex()`;
+a host that means something else by one remaps it at run time, through
+`literals`. `parse` takes `resolver=` too, and a path is cached per resolver
+that lexes.
+
 <a id="escaping"></a>
 ### Escaping
 
@@ -2403,6 +2438,18 @@ sequence. You may append an item or slice to the end a sequence.
     >>> dotted.update(d, '*.there[+:]', [999])
     {'hi': {'there': [1, 2, 3, 8, 999]}, 'bye': {'there': [4, 5, 6, 8, 999]}}
 
+`[+]` is the slot past the end, so nothing is ever there to read: a `get`
+through it is the default, and `[-1]` is the last item. A write to it always
+applies, so in a group an append branch is updated along with any branch
+that matches:
+
+    >>> dotted.get(d, 'hi.there[+]') is None
+    True
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}, {'k': 9}]
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1#, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}]
+
 <a id="the-append-unique--operator"></a>
 ### The append-unique `+?` operator
 

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/README.md

@@ -227,6 +227,18 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.50.0
+- **Nothing is read through `[+]` or `[+?]`.** They are the slot past the
+  end, so `get(d, 'a[+]')` is the default, not the last item; use `a[-1]`.
+  `has` is `False`, `remove`, `pluck` and `expand` through them find
+  nothing, a guard on them never passes, and `setdefault` always appends
+  (`[+?]` only when the value is absent). See
+  [The append `+` operator](#the-append--operator).
+- **An append branch in a group always applies in an update.**
+  `(a, b[+])` appends to `b` whether or not it is empty; it used to append
+  only when `b` already had items. With a cut, `[(*&k=1#, +)]`, the append
+  is still the fallback.
+
 ### v0.49.0
 - **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
   [Packing](#packing)). If you have keys containing a literal `;`, quote
@@ -1709,6 +1721,29 @@ A path handed in already parsed goes through `fold()` again, so a hook must
 be content to see its own output. `Dotted.segments()` gives any parsed
 path's segment values, and a slice of a parsed path, `ops[1:]`, is a path.
 
+A host's own names may stand where dotted has no reading for a bare token:
+a slot, `merged[k]`, a slice bound, `items[:n]`, a filter value,
+`[*&cur=want]`, or a guard value, `a=k`. The parse-time hook
+`lex(text, position)` is offered each such token as written, a dotted name
+whole, with its position, one of `'slots'`, `'slices'`, `'filters'` or
+`'guards'`. It answers with dotted text to read in the token's place, `None`
+to leave the `ParseError`, or raises to fail the parse at the token. The
+answer must be one token dotted reads there, and is not offered again.
+
+    >>> class Names(dotted.ReferenceResolver):
+    ...     def lex(self, text, position):
+    ...         return f'$$({text})'
+    >>> d = {'merged': {'x': 1, 'y': 2}, 'k': 'x', 'c': {'account': 'y'}}
+    >>> dotted.get(d, 'merged[k]', resolver=Names())
+    1
+    >>> dotted.get(d, 'merged[c.account]', resolver=Names())
+    2
+
+A token dotted reads, `m[$x]` or `|sub:taken`, is never offered to `lex()`;
+a host that means something else by one remaps it at run time, through
+`literals`. `parse` takes `resolver=` too, and a path is cached per resolver
+that lexes.
+
 <a id="escaping"></a>
 ### Escaping
 
@@ -2355,6 +2390,18 @@ sequence. You may append an item or slice to the end a sequence.
     >>> dotted.update(d, '*.there[+:]', [999])
     {'hi': {'there': [1, 2, 3, 8, 999]}, 'bye': {'there': [4, 5, 6, 8, 999]}}
 
+`[+]` is the slot past the end, so nothing is ever there to read: a `get`
+through it is the default, and `[-1]` is the last item. A write to it always
+applies, so in a group an append branch is updated along with any branch
+that matches:
+
+    >>> dotted.get(d, 'hi.there[+]') is None
+    True
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}, {'k': 9}]
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1#, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}]
+
 <a id="the-append-unique--operator"></a>
 ### The append-unique `+?` operator
 

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/dotted/access.py

@@ -816,17 +816,31 @@ class SlotSpecial(Slot):
         return []
 
     def items(self, node, settings=base.SETTINGS, filtered=True):
-        return self._last_item(node)
-
-    def _last_item(self, node):
-        try:
-            yield -1, node[-1]
-        except (TypeError, IndexError):
-            pass
+        """
+        Nothing: an append slot is the one past the end, so there is never
+        a value there to read. A write to it always applies (see update).
+        """
+        return iter(())
 
     def is_empty(self, node):
         return True
 
+    def do_update(self, ops, node, val, has_defaults, _path, nop, nop_from_unwrap=False, settings=base.SETTINGS):
+        """
+        In a default skeleton (has_defaults, see engine.build_default),
+        upsert already appended this slot's placeholder, so the rest of the
+        path is written into that last item instead of a new one.
+        """
+        if not (ops and has_defaults):
+            return super().do_update(ops, node, val, has_defaults, _path, nop, nop_from_unwrap=nop_from_unwrap, settings=settings)
+        try:
+            last = node[-1]
+        except (TypeError, IndexError):
+            return node
+        pass_nop = nop and not nop_from_unwrap
+        built = engine.updates(ops, last, val, True, _path + [(self, -1)], pass_nop, settings=settings.below(node))
+        return self.update(node, -1, built)
+
     def update(self, node, key, val):
         if not isinstance(key, str):
             return super().update(node, key, val)

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/dotted/api.py

@@ -8,6 +8,7 @@ import threading
 import pyparsing as pp
 from . import grammar
 from . import engine
+from . import groups
 from . import results
 from . import transforms
 from . import base
@@ -98,21 +99,35 @@ def set_parse_cache(size=CACHE_SIZE):
 
 
 @functools.lru_cache(CACHE_SIZE)
-def _parse(ops):
-    with _parse_lock:
+def _parse(ops, lexer=None):
+    with _parse_lock, grammar.lexing(lexer):
         try:
             parsed = grammar.template.parse_string(ops, parse_all=True)
-        except pp.ParseException as e:
+        except pp.ParseBaseException as e:
             raise ParseError(f"Invalid dotted notation: {e.msg}\n  {repr(ops)}\n  {' ' * e.loc}^") from None
     return results.Dotted({'ops': parsed['ops']})
 
 
-def parse(path, bindings=None, partial=True):
+def _lexer(resolver):
+    """
+    The resolver whose lex() a parse offers what dotted cannot read: the
+    call's own or the process default, or None when it keeps the default,
+    which lexes nothing, so the parse is cached by path alone.
+    """
+    r = base.RESOLVER if resolver is None else resolver
+    if type(r).lex is base.Resolver.lex:
+        return None
+    return r
+
+
+def parse(path, bindings=None, partial=True, resolver=None):
     """
     Parse dotted notation. Results are LRU-cached (same path string reuses cached parse).
 
     If bindings is provided, resolve substitutions against them.
     If partial is False, raise TypeError on unresolved templates.
+    A token dotted cannot read is offered to the resolver's lex(), the
+    call's own or the process default (see ReferenceResolver).
 
     >>> parse('hello.there|str:"=%s"')
     Dotted([hello, there|str:'=%s'])
@@ -122,7 +137,7 @@ def parse(path, bindings=None, partial=True):
     elif isinstance(path, tuple):
         ops = results.Dotted({'ops': path, 'transforms': ()})
     else:
-        ops = _parse(path)
+        ops = _parse(path, _lexer(resolver))
     if bindings is not None:
         ops = ops.resolve(bindings, partial=partial)
     if not partial and _is_template(ops):
@@ -366,7 +381,7 @@ def mutable(obj, path, strict=False, bindings=None, resolver=None):
     >>> mutable(((1, 2),), '[0][0]')
     False
     """
-    ops = parse(path, bindings=bindings, partial=False)
+    ops = parse(path, bindings=bindings, partial=False, resolver=resolver)
 
     # Empty path can never mutate (can't rebind caller's variable)
     if len(ops) == 1 and isinstance(ops[0], access.Empty):
@@ -427,7 +442,7 @@ def build_multi(obj, paths, strict=False, bindings=None, resolver=None):
     if obj is AUTO:
         obj = _auto_root(((p, None) for p in paths))
     for path in paths:
-        ops = _prepare(parse(path, bindings=bindings, partial=False), resolver)
+        ops = _prepare(parse(path, bindings=bindings, partial=False, resolver=resolver), resolver)
         built = engine.build(ops, obj, strict=strict, resolver=resolver)
         # walked paths are concrete: nothing for update() to resolve or check
         for found, val in _pluck_parsed(built, ops, strict=strict, resolver=resolver):
@@ -496,7 +511,7 @@ def get(obj, path, default=None, pattern_default=(), apply_transforms=True, stri
     >>> get({'a': {'b': 7, 'c': 3}}, '**=7')
     (7,)
     """
-    ops = _prepare(parse(path, bindings=bindings, partial=False), resolver)
+    ops = _prepare(parse(path, bindings=bindings, partial=False, resolver=resolver), resolver)
     chain = ops.simple_chain if _SIMPLE_FASTPATH and not _literals(resolver) else None
     if chain is not None:
         val = engine.simple_get(chain, obj, strict=strict)
@@ -550,13 +565,15 @@ def setdefault(obj, path, val, apply_transforms=True, strict=False, bindings=Non
     >>> setdefault({}, 'a.b.c', 7)
     7
     """
-    ops = _prepare(parse(path, bindings=bindings, partial=False), resolver)
+    ops = _prepare(parse(path, bindings=bindings, partial=False, resolver=resolver), resolver)
     if obj is AUTO:
         obj = _auto_root_from_path(ops)
     if has(obj, ops, strict=strict, resolver=resolver):
         return get(obj, ops, apply_transforms=apply_transforms, strict=strict, resolver=resolver)
     obj = update(obj, ops, val, apply_transforms=apply_transforms, strict=strict, resolver=resolver)
-    return get(obj, ops, apply_transforms=False, strict=strict, resolver=resolver)
+    # an append slot, [+], has nothing to read back: what was set is val
+    default = val if groups.appends(ops) else None
+    return get(obj, ops, default=default, apply_transforms=False, strict=strict, resolver=resolver)
 
 
 def setdefault_multi(obj, pathvalues, apply_transforms=True, strict=False, bindings=None, resolver=None):
@@ -590,7 +607,7 @@ def update_if(obj, path, val, pred=lambda val: val is not None, mutable=True, ap
     """
     parsed = path
     if obj is AUTO:
-        parsed = parse(path)
+        parsed = parse(path, resolver=resolver)
         obj = _auto_root_from_path(parsed)
     if not mutable and _is_mutable_container(obj):
         obj = utils.deepcopy(obj)
@@ -599,7 +616,7 @@ def update_if(obj, path, val, pred=lambda val: val is not None, mutable=True, ap
     if pred is not None and not pred(val):
         return obj
 
-    ops = _prepare(parse(parsed, bindings=bindings, partial=False), resolver)
+    ops = _prepare(parse(parsed, bindings=bindings, partial=False, resolver=resolver), resolver)
     return engine.updates(ops, obj, val, strict=strict, resolver=resolver, transforms=apply_transforms)
 
 
@@ -694,7 +711,7 @@ def remove_if(obj, path, pred=lambda path: path is not None, val=ANY, mutable=Tr
     """
     parsed = path
     if obj is AUTO:
-        parsed = parse(path)
+        parsed = parse(path, resolver=resolver)
         obj = _auto_root_from_path(parsed)
     if not mutable and _is_mutable_container(obj):
         obj = utils.deepcopy(obj)
@@ -703,7 +720,7 @@ def remove_if(obj, path, pred=lambda path: path is not None, val=ANY, mutable=Tr
     if pred is not None and not pred(path):
         return obj
 
-    ops = _prepare(parse(parsed, bindings=bindings, partial=False), resolver)
+    ops = _prepare(parse(parsed, bindings=bindings, partial=False, resolver=resolver), resolver)
     return engine.removes(ops, obj, val, strict=strict, resolver=resolver)
 
 
@@ -1071,7 +1088,7 @@ def expand_multi(obj, patterns, strict=False, bindings=None, resolver=None):
     """
     seen = {}
     for pat in patterns:
-        ops = _prepare(parse(pat, bindings=bindings, partial=False), resolver)
+        ops = _prepare(parse(pat, bindings=bindings, partial=False, resolver=resolver), resolver)
         paths = (o.assemble() for o in engine.expands(ops, obj, strict=strict, resolver=resolver))
         for found in paths:
             if found in seen:
@@ -1106,7 +1123,7 @@ def apply_multi(obj, patterns, strict=False, bindings=None, resolver=None):
     seen = {}
     _marker = object()
     for pat in patterns:
-        for ops in engine.expands(_prepare(parse(pat, bindings=bindings, partial=False), resolver), obj, strict=strict, resolver=resolver):
+        for ops in engine.expands(_prepare(parse(pat, bindings=bindings, partial=False, resolver=resolver), resolver), obj, strict=strict, resolver=resolver):
             if ops in seen:
                 continue
             seen[ops] = None
@@ -1147,7 +1164,7 @@ def _pluck_written(obj, patterns, strict=False, bindings=None, resolver=None):
     """
     seen = {}
     for pattern in patterns:
-        ops = _prepare(parse(pattern, bindings=bindings, partial=False), resolver)
+        ops = _prepare(parse(pattern, bindings=bindings, partial=False, resolver=resolver), resolver)
         for path, val in engine.walk(ops, obj, paths=True, strict=strict, resolver=resolver, transforms=False):
             if path is utypes.CUT_SENTINEL:
                 break
@@ -1167,7 +1184,7 @@ def pluck(obj, pattern, default=None, strict=False, bindings=None, resolver=None
     >>> pluck(d, 'a.b')
     ('a.b', 'seven')
     """
-    parsed = parse(pattern)
+    parsed = parse(pattern, resolver=resolver)
     out = tuple(pluck_multi(obj, (parsed,), default=default, strict=strict, resolver=resolver, bindings=bindings))
     if not out:
         return ()
@@ -1185,7 +1202,7 @@ def walk(obj, pattern, strict=False, bindings=None, resolver=None):
     >>> list(walk(d, 'a.*'))
     [('a.b', 1), ('a.c', 2)]
     """
-    ops = _prepare(parse(pattern, bindings=bindings, partial=False), resolver)
+    ops = _prepare(parse(pattern, bindings=bindings, partial=False, resolver=resolver), resolver)
     for path, val in engine.walk(ops, obj, paths=True, strict=strict, resolver=resolver, transforms=False):
         if path is utypes.CUT_SENTINEL:
             break

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/dotted/base.py

@@ -154,6 +154,10 @@ class NOP(metaclass=MetaNOP):
 # and filter values.
 LITERALS = ('segments', 'transforms', 'guards', 'filters')
 
+# The positions a resolver's lex() may be offered a token in: a slot, a
+# slice bound, a filter value and a guard value.
+LEXED = ('slots', 'slices', 'filters', 'guards')
+
 
 class Resolver:
     """
@@ -163,6 +167,7 @@ class Resolver:
     in and that node's ancestors. This default reads it as dotted does:
     a path against the root, or with ^ against the node, ^^ its parent,
     and so on. Raises UnresolvedReference when there is nothing there.
+    Its lex() is the parse-time hook for tokens dotted cannot read.
     """
 
     # True to be offered every plain token in a var-permitting position as
@@ -191,6 +196,21 @@ class Resolver:
         check_literals(literals)
         self.literals = literals
 
+    def lex(self, text, position):
+        """
+        The parse-time hook, offered a token dotted's grammar has no
+        reading for: text as written, merged[k] offering 'k', and the
+        position it stands in, one of LEXED. A dotted name is one token,
+        merged[a.b] offering 'a.b'. It answers with dotted text to read
+        in the token's place, '$$(k)' making the path merged[$$(k)], or
+        None to leave the path unparsed; raising fails the parse at the
+        token. The text is read in that position as one token, and is
+        not offered to lex() again. A token dotted reads is never
+        offered: a host that means something else by one says so at run
+        time, through literals. The default lexes nothing.
+        """
+        return None
+
     def fold(self, segments):
         """
         The path-level hook, called once with a path's segment strings

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/dotted/grammar.py

@@ -1,5 +1,6 @@
 """
 """
+import contextlib
 import decimal
 import re
 import pyparsing as pp
@@ -288,10 +289,134 @@ var = _escaped_dollar | _reference | _paren_subst | _raw_subst | _dollar_word
 # |min:$amount.value or =^lots.taken: one string, for a resolver to read
 # whole. In a segment position the . separates segments, so not there.
 _arg_sigil = pp.Regex(r'(?:\$(?![0-9($])|\^)[^' + re.escape(reserved.replace('.', '')) + r' \t\n\r]+').set_parse_action(matchers.Word)
+
+# ---------------------------------------------------------------------------
+# Lexed tokens: what dotted cannot read in a slot, a slice bound, a filter
+# value or a guard value is offered to the resolver's lex() (see
+# base.Resolver.lex), last, once dotted's own readings have failed.
+# ---------------------------------------------------------------------------
+
+# The resolver a path is being parsed for, when it lexes, and its answers
+# so far, by (text, position): reading an answer resets pyparsing's
+# packrat cache, so the parse may come back to a token it has lexed.
+LEXER = None
+_ANSWERS = {}
+
+_WORD_CHARS = frozenset(pp.alphanums + '_')
+
+
+@contextlib.contextmanager
+def lexing(lexer):
+    """
+    Parse with lexer offered what dotted cannot read (see Lexed); None
+    for no lexer. Not reentrant: api._parse holds its lock around it.
+    """
+    global LEXER
+    LEXER = lexer
+    _ANSWERS.clear()
+    try:
+        yield
+    finally:
+        LEXER = None
+        _ANSWERS.clear()
+
+
+def _token_end(s, loc, stops):
+    """
+    Where the token at loc ends: at the first of stops, whitespace or a
+    closing bracket outside the brackets and quotes it opens itself.
+    """
+    depth = 0
+    quote = None
+    i = loc
+    while i < len(s):
+        c = s[i]
+        if quote:
+            if c == '\\':
+                i += 2
+                continue
+            if c == quote:
+                quote = None
+        elif c in '"\'':
+            quote = c
+        elif c in '([{':
+            depth += 1
+        elif depth:
+            if c in ')]}':
+                depth -= 1
+        elif c in stops or c in ')]}' or c.isspace():
+            return i
+        i += 1
+    return i
+
+
+class Lexed(pp.Token):
+    """
+    A token dotted has no reading for at position, offered to LEXER. It
+    runs to the first of stops (see _token_end) and is offered only when
+    one of follow, or with end the end of the path, comes after it. The
+    text the lexer answers is read whole with reading, without LEXER.
+    """
+    def __init__(self, position, reading, stops, follow=None, end=False):
+        super().__init__()
+        self.position = position
+        self.reading = reading
+        self.stops = stops
+        self.follow = stops if follow is None else follow
+        self.end = end
+        self.mayReturnEmpty = False
+        self.mayIndexError = False
+        self.errmsg = f'Expected a token in {position}'
+
+    def _generateDefaultName(self):
+        return f'Lexed({self.position})'
+
+    def parseImpl(self, instring, loc, do_actions=True):
+        global LEXER
+        lexer = LEXER
+        if lexer is None:
+            raise pp.ParseException(instring, loc, self.errmsg, self)
+        stop = _token_end(instring, loc, self.stops)
+        if stop == loc:
+            raise pp.ParseException(instring, loc, self.errmsg, self)
+        if stop == len(instring) and not self.end:
+            raise pp.ParseException(instring, loc, self.errmsg, self)
+        if stop < len(instring) and instring[stop] not in self.follow:
+            raise pp.ParseException(instring, loc, self.errmsg, self)
+        text = instring[loc:stop]
+        key = (text, self.position)
+        if key not in _ANSWERS:
+            try:
+                _ANSWERS[key] = lexer.lex(text, self.position)
+            except Exception as e:
+                raise pp.ParseFatalException(instring, loc, f'{text!r} in {self.position}: {e}') from e
+        lexed = _ANSWERS[key]
+        if lexed is None:
+            raise pp.ParseException(instring, loc, self.errmsg, self)
+        LEXER = None
+        try:
+            tokens = self.reading.parse_string(lexed, parse_all=True)
+        except pp.ParseBaseException:
+            raise pp.ParseFatalException(
+                instring, loc, f'{text!r} lexed as {lexed!r}, not one token in {self.position}') from None
+        finally:
+            LEXER = lexer
+        return stop, list(tokens)
+
+
+# After a token dotted reads, while a lexer is in force: no word character
+# follows, so null in =nullable or 3 in [:3x] is not read as a prefix of
+# the token, which is the lexer's to read whole.
+_ends = pp.Empty().leave_whitespace().add_condition(
+    lambda s, loc, t: LEXER is None or s[loc:loc + 1] not in _WORD_CHARS, call_during_try=True)
+
 # A slice bound: an integer, + (the length), or a $$(path) reference or
-# $(name)/$N substitution standing for one.
-_slice_bound = integer | plus | _reference | _paren_subst | _raw_subst
-slice = pp.Optional(_slice_bound) + ':' + pp.Optional(_slice_bound) \
+# $(name)/$N substitution standing for one. A start bound is lexed only
+# before its :, so [k] is not offered as one.
+_slice_read = integer | plus | _reference | _paren_subst | _raw_subst
+_slice_bound = (_slice_read + _ends) | Lexed('slices', _slice_read, ':]')
+_slice_start = (_slice_read + _ends) | Lexed('slices', _slice_read, ':]', follow=':')
+slice = pp.Optional(_slice_start) + ':' + pp.Optional(_slice_bound) \
          + pp.Optional(':') + pp.Optional(_slice_bound)
 
 _common_pats = wildcard_first | wildcard | var | regex_first | regex
@@ -548,6 +673,11 @@ value = pp.Forward()
 _value_group_inner = value + OM(_ccomma + value)
 value_group = (S('(') + _value_group_inner + S(')')).set_parse_action(lambda t: [ct.ValueGroup(*t)])
 value <<= value_group | container_value | string_glob | bytes_literal | string | wildcard | _arg_sigil | var | regex | numeric_quoted | none | true | false | numeric_extended | numeric_slot
+# A filter's or guard's value, else a token for the lexer: it runs to what
+# may follow a value, a dot not among them, so =lot.x is one token.
+_value_stops = '&,)]|?#;'
+filter_value = (value + _ends) | Lexed('filters', value, _value_stops, end=True)
+guard_value = (value + _ends) | Lexed('guards', value, _value_stops, end=True)
 # ---------------------------------------------------------------------------
 # Concat: part+part for key construction using Python's native + operator.
 # Each part can carry per-part transforms: part|xform+part|xform
@@ -631,7 +761,7 @@ def _filter_single_action(t):
     cls = flt.FILTER_PRED_CLS[op_str]
     return cls(rest + [val])
 
-_filter_single_all = (filter_key + ZM(pipe + transform) + _pred_op_re + value).set_parse_action(_filter_single_action)
+_filter_single_all = (filter_key + ZM(pipe + transform) + _pred_op_re + filter_value).set_parse_action(_filter_single_action)
 
 # Recursive filter expression with grouping
 filter_expr = pp.Forward()
@@ -736,6 +866,12 @@ slicecmd = (lb + Opt(L('~')) + Opt(slice) + rb).set_parse_action(
     lambda t: wrappers.NopWrap(access.Slice(*t[1:])) if t and t[0] == '~' else access.Slice(*t))
 slicefilter = (lb + filters + ZM(amp + filters) + rb).set_parse_action(access.SliceFilter)
 
+# A slot dotted cannot read, [k] or [a.b], offered to the lexer: tried
+# after every other bracket, so [a.b=1] stays a filter.
+_slot_read = _slot_concat | _commons | numeric_extended | numeric_slot
+lexedcmd = (lb + Opt(L('~')) + Lexed('slots', _slot_read, ']&') + ZM(amp + filters) + rb
+            + Opt(_seg_mid)).set_parse_action(_slotcmd_action)
+
 # Cut markers: ## = soft cut, # = hard cut (try ## first so ## isn't parsed as # + #)
 softcut_marker = L('##')
 cut_marker = softcut_marker | L('#')
@@ -983,7 +1119,7 @@ _at_nop_grouped = _at_nop_grouped.set_parse_action(
 )
 
 # NOP (~): match but don't update
-_nop_inner = inner_grouped_first | inner_grouped | recursive_op | keycmd | attrcmd | slotgroup_first | slotgroup | slotcmd | slotspecial | slicefilter | slicecmd
+_nop_inner = inner_grouped_first | inner_grouped | recursive_op | keycmd | attrcmd | slotgroup_first | slotgroup | slotcmd | slotspecial | slicefilter | slicecmd | lexedcmd
 _nop_wrap = (tilde + _nop_inner).set_parse_action(lambda t: wrappers.NopWrap(t[1]))
 
 # Explicit NOP (~): like _nop_wrap but inner must start with an access op (./@/[]).
@@ -998,7 +1134,7 @@ _explicit_nop_inner = (
     attrcmd |
     slotgroup_first | slotgroup |
     slotcmd |
-    slotspecial | slicefilter | slicecmd
+    slotspecial | slicefilter | slicecmd | lexedcmd
 )
 _explicit_nop_wrap = (tilde + _explicit_nop_inner).set_parse_action(lambda t: wrappers.NopWrap(t[1]))
 
@@ -1015,7 +1151,7 @@ _op_seq_cont_items = (
     attrcmd |
     slotgroup_first | slotgroup |
     slotcmd |
-    slotspecial | slicefilter | slicecmd
+    slotspecial | slicefilter | slicecmd | lexedcmd
 )
 _op_seq_cont << _op_seq_cont_items
 _continues <<= pp.FollowedBy(_op_seq_cont)
@@ -1034,7 +1170,7 @@ _op_seq_first_items = (
     attrcmd |
     slotgroup_first | slotgroup |
     slotcmd |
-    slotspecial | slicefilter | slicecmd
+    slotspecial | slicefilter | slicecmd | lexedcmd
 )
 # First item also allows bare grouped expressions — (a,b) with bare keys.
 # This is valid at the start of a path (top-level inference).
@@ -1051,7 +1187,7 @@ op_seq_item << (inner_grouped_first | inner_grouped | _op_seq_first_items)
 # guard, they transform the value.
 # ---------------------------------------------------------------------------
 
-_postfix = Opt(type_restriction) + ZM(amp + filters) + ZM(pipe + transform) + Opt(_pred_op_re + value + ZM(pipe + transform))
+_postfix = Opt(type_restriction) + ZM(amp + filters) + ZM(pipe + transform) + Opt(_pred_op_re + guard_value + ZM(pipe + transform))
 
 
 def _split_postfix(tokens):
@@ -1148,7 +1284,7 @@ _explicit_op_seq_cont_items = (
     attrcmd |
     slotgroup_first | slotgroup |
     slotcmd |
-    slotspecial | slicefilter | slicecmd
+    slotspecial | slicefilter | slicecmd | lexedcmd
 )
 _explicit_op_seq = pp.Group(_explicit_op_seq_cont_items + ZM(_explicit_op_seq_cont_items))
 _explicit_inner_atom = _explicit_op_seq

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/dotted/groups.py

@@ -5,7 +5,7 @@ from . import matchers
 from . import wrappers
 from . import engine
 from . import results
-from .access import AccessOp, Key, Attr, Slot
+from .access import AccessOp, Key, Attr, Slot, SlotSpecial
 
 
 def leading_keys(path):
@@ -415,12 +415,12 @@ class OpGroupOr(OpGroup):
                 if softcut_paths and path and softcut_paths.overlaps(path):
                     continue
                 paths.append(path)
-            if not paths:
+            if not paths and not appends(item):
                 continue
             matched_any = True
             if marker is base.BRANCH_SOFTCUT:
                 softcut_paths.extend(p for p in paths if p)
-            if softcut_paths:
+            if softcut_paths and paths:
                 branch_nop = nop or any(isinstance(op, wrappers.NopWrap) for op in item)
                 for path in paths:
                     node = engine.updates(list(path), node, val, has_defaults, _path, branch_nop, settings=settings)
@@ -524,7 +524,7 @@ class OpGroupFirst(OpGroup):
             branch_ops = list(branch) + list(ops)
             if not branch_ops:
                 continue
-            if base.has_any(engine.gets(branch_ops, node, settings=settings)):
+            if appends(branch) or base.has_any(engine.gets(branch_ops, node, settings=settings)):
                 return engine.updates(branch_ops, node, val, has_defaults, _path, nop, settings=settings)
         return _disjunction_fallback(self, ops, node, val, has_defaults, _path, nop, settings=settings)
 
@@ -968,6 +968,15 @@ def _attr_branch(branch):
     return branch
 
 
+def appends(ops):
+    """
+    True if ops hold an append slot, [+] or [+?]. Nothing is ever there to
+    read, but a write always applies, so in an update a group branch that
+    holds one counts as matched.
+    """
+    return any(isinstance(op.most_inner, SlotSpecial) for op in ops)
+
+
 def _is_concrete_path(branch_ops):
     """
     Return True if branch_ops represents a concrete path (no wildcards/patterns).

{dotted_notation-0.49.4 → dotted_notation-0.51.0/dotted_notation.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dotted_notation
-Version: 0.49.4
+Version: 0.51.0
 Summary: Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms
 Author-email: Frey Waid <logophage1@gmail.com>
 License-Expression: MIT
@@ -275,6 +275,18 @@ Several Python libraries handle nested data access. Here's how dotted compares:
 <a id="breaking-changes"></a>
 ## Breaking Changes
 
+### v0.50.0
+- **Nothing is read through `[+]` or `[+?]`.** They are the slot past the
+  end, so `get(d, 'a[+]')` is the default, not the last item; use `a[-1]`.
+  `has` is `False`, `remove`, `pluck` and `expand` through them find
+  nothing, a guard on them never passes, and `setdefault` always appends
+  (`[+?]` only when the value is absent). See
+  [The append `+` operator](#the-append--operator).
+- **An append branch in a group always applies in an update.**
+  `(a, b[+])` appends to `b` whether or not it is empty; it used to append
+  only when `b` already had items. With a cut, `[(*&k=1#, +)]`, the append
+  is still the fallback.
+
 ### v0.49.0
 - **`;` is now a reserved character**: it packs a group, `(a,b;)` (see
   [Packing](#packing)). If you have keys containing a literal `;`, quote
@@ -1757,6 +1769,29 @@ A path handed in already parsed goes through `fold()` again, so a hook must
 be content to see its own output. `Dotted.segments()` gives any parsed
 path's segment values, and a slice of a parsed path, `ops[1:]`, is a path.
 
+A host's own names may stand where dotted has no reading for a bare token:
+a slot, `merged[k]`, a slice bound, `items[:n]`, a filter value,
+`[*&cur=want]`, or a guard value, `a=k`. The parse-time hook
+`lex(text, position)` is offered each such token as written, a dotted name
+whole, with its position, one of `'slots'`, `'slices'`, `'filters'` or
+`'guards'`. It answers with dotted text to read in the token's place, `None`
+to leave the `ParseError`, or raises to fail the parse at the token. The
+answer must be one token dotted reads there, and is not offered again.
+
+    >>> class Names(dotted.ReferenceResolver):
+    ...     def lex(self, text, position):
+    ...         return f'$$({text})'
+    >>> d = {'merged': {'x': 1, 'y': 2}, 'k': 'x', 'c': {'account': 'y'}}
+    >>> dotted.get(d, 'merged[k]', resolver=Names())
+    1
+    >>> dotted.get(d, 'merged[c.account]', resolver=Names())
+    2
+
+A token dotted reads, `m[$x]` or `|sub:taken`, is never offered to `lex()`;
+a host that means something else by one remaps it at run time, through
+`literals`. `parse` takes `resolver=` too, and a path is cached per resolver
+that lexes.
+
 <a id="escaping"></a>
 ### Escaping
 
@@ -2403,6 +2438,18 @@ sequence. You may append an item or slice to the end a sequence.
     >>> dotted.update(d, '*.there[+:]', [999])
     {'hi': {'there': [1, 2, 3, 8, 999]}, 'bye': {'there': [4, 5, 6, 8, 999]}}
 
+`[+]` is the slot past the end, so nothing is ever there to read: a `get`
+through it is the default, and `[-1]` is the last item. A write to it always
+applies, so in a group an append branch is updated along with any branch
+that matches:
+
+    >>> dotted.get(d, 'hi.there[+]') is None
+    True
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}, {'k': 9}]
+    >>> dotted.update([{'k': 1}, {'k': 2}], '[(*&k=1#, +)]', {'k': 9})
+    [{'k': 9}, {'k': 2}]
+
 <a id="the-append-unique--operator"></a>
 ### The append-unique `+?` operator
 

{dotted_notation-0.49.4 → dotted_notation-0.51.0}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "dotted_notation"
-version = "0.49.4"
+version = "0.51.0"
 description = "Dotted notation for safe nested data traversal with optional chaining, pattern matching, and transforms"
 readme = "README.md"
 requires-python = ">=3.6"