TSVZ 4.6__tar.gz → 4.7__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: TSVZ
3
- Version: 4.6
3
+ Version: 4.7
4
4
  Summary: Key-value store kept in a TSV / CSV / NSV / PSV file, with tsvz-spec-v1 support
5
5
  Home-page: https://github.com/yufei-pan/TSVZ
6
6
  Author: Yufei Pan
@@ -34,7 +34,7 @@ key-value store in a delimiter-separated text file**. The first column of each
34
34
  row is the key; the file behaves like an ordered dictionary that is persisted to
35
35
  disk as you change it.
36
36
 
37
- **Version 4.6**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
37
+ **Version 4.7**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
38
38
  next to your code and `import TSVZ`, or install it.
39
39
 
40
40
  ```bash
@@ -224,6 +224,7 @@ delimiter or encoding argument that conflicts with the extension.
224
224
  | L4 | New warnings on stderr (invalid encoding, lines dropped by `strict`); data unchanged. |
225
225
  | L5 | A damaged compressed file is read up to the damage instead of raising. |
226
226
  | L6 | `mapToFile` no longer glues a row it adds at the end of the file to the next one. |
227
+ | L7 | `TSVZed`'s `in` and `get` find a key as `t[key]` does: the live key equal to it, else the key right-stripped. 3.39 matched them exactly, so `'x ' in t` was false while `t['x ']` read `x`, and `t.setdefault('x ', …)` overwrote `x`. |
227
228
 
228
229
  ### `.tsvz`, `.csvz`, `.nsvz`, `.psvz`: now tsvz-spec-v1
229
230
 
@@ -243,6 +244,7 @@ delimiter or encoding argument that conflicts with the extension.
243
244
  | S12 | `TSVZedLite.clear()` and `clear_file()` raise `OSError` when the clear is refused (a part could not be read, the path names one part of a store, or the store kept changing). Memory is reloaded from the unchanged store first. |
244
245
  | S13 | Compressed parts are written in members of at most 1 MiB of records (`#_member_size_#`), Zstandard frames with a checksum; an append never reads or repairs the part. |
245
246
  | S14 | A zero byte in a value is written `<NUL>`, and a raw zero byte in a record is read as zero-fill damage (spec §4.9): under the default `recover` only the bytes after it are read. **A value that 4.2 or 4.3 wrote with a raw zero byte therefore reads differently in 4.4.** Such values are rare; before upgrading, `grep -c -P '\x00' store.tsvz` (any `.tsvz`, `.csvz` or `.psvz` part) finds them. Stop 4.2 and 4.3 handlers before upgrading: they read `<NUL>` as text, as they read `<CR>`. |
247
+ | S15 | A lookup (`t[key]`, `in`, `get`, `del`, `pop`, `setdefault`) takes the live key equal to the one given before the right-stripped one, so a key ending in whitespace (written while `#_strip_trailing_whites_#` was false) can be read and deleted by name (spec §20.2.7). Setting a key still right-strips it. |
246
248
 
247
249
  ### The `tsvz` command
248
250
 
@@ -262,6 +264,7 @@ and `-d=,`, except the ones C5 and C9 list.
262
264
  | C9 | Arguments after STORE for `read`, `clear` and `scrub` are a usage error, exit 2 (3.39 ignored them, so `clear junk` cleared the file). |
263
265
  | C10 | The table (a terminal, or `--format table`) of `get`, `keys`, `pop`, `popitem`, `setdefault`, and of `read` on a `.tsvz`-family store, shows the rows only: no row is drawn as a header, and no row is cut to the first row's or the terminal's width. `read` of a `.tsv`-family file keeps 3.39's table, whose first row is the file's header line. |
264
266
  | C11 | The table shows control characters, bidi controls and line separators as escapes (`\r`, `\n`, `\t`, `\xNN`, `\uNNNN`), so no value can move the cursor, reorder the line or restyle the terminal. It sizes columns in terminal columns, so CJK text (two columns a character) lines up. The table is for reading: a backslash in a value is shown as is, so use records for exact values. |
267
+ | C12 | While stripping is on, `get`, `has`, `pop`, `delete` and `setdefault` take the live key equal to a KEY ending in a space or tab before the stripped one (spec §20.2.7), so `tsvz keys s \| tsvz delete s -` deletes every key. `delete` of such a KEY reads the store first, and exits 1 without writing when the store cannot be read whole. |
265
268
 
266
269
  ## Spec deviations and interpretations
267
270
 
@@ -7,7 +7,7 @@ key-value store in a delimiter-separated text file**. The first column of each
7
7
  row is the key; the file behaves like an ordered dictionary that is persisted to
8
8
  disk as you change it.
9
9
 
10
- **Version 4.6**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
10
+ **Version 4.7**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
11
11
  next to your code and `import TSVZ`, or install it.
12
12
 
13
13
  ```bash
@@ -197,6 +197,7 @@ delimiter or encoding argument that conflicts with the extension.
197
197
  | L4 | New warnings on stderr (invalid encoding, lines dropped by `strict`); data unchanged. |
198
198
  | L5 | A damaged compressed file is read up to the damage instead of raising. |
199
199
  | L6 | `mapToFile` no longer glues a row it adds at the end of the file to the next one. |
200
+ | L7 | `TSVZed`'s `in` and `get` find a key as `t[key]` does: the live key equal to it, else the key right-stripped. 3.39 matched them exactly, so `'x ' in t` was false while `t['x ']` read `x`, and `t.setdefault('x ', …)` overwrote `x`. |
200
201
 
201
202
  ### `.tsvz`, `.csvz`, `.nsvz`, `.psvz`: now tsvz-spec-v1
202
203
 
@@ -216,6 +217,7 @@ delimiter or encoding argument that conflicts with the extension.
216
217
  | S12 | `TSVZedLite.clear()` and `clear_file()` raise `OSError` when the clear is refused (a part could not be read, the path names one part of a store, or the store kept changing). Memory is reloaded from the unchanged store first. |
217
218
  | S13 | Compressed parts are written in members of at most 1 MiB of records (`#_member_size_#`), Zstandard frames with a checksum; an append never reads or repairs the part. |
218
219
  | S14 | A zero byte in a value is written `<NUL>`, and a raw zero byte in a record is read as zero-fill damage (spec §4.9): under the default `recover` only the bytes after it are read. **A value that 4.2 or 4.3 wrote with a raw zero byte therefore reads differently in 4.4.** Such values are rare; before upgrading, `grep -c -P '\x00' store.tsvz` (any `.tsvz`, `.csvz` or `.psvz` part) finds them. Stop 4.2 and 4.3 handlers before upgrading: they read `<NUL>` as text, as they read `<CR>`. |
220
+ | S15 | A lookup (`t[key]`, `in`, `get`, `del`, `pop`, `setdefault`) takes the live key equal to the one given before the right-stripped one, so a key ending in whitespace (written while `#_strip_trailing_whites_#` was false) can be read and deleted by name (spec §20.2.7). Setting a key still right-strips it. |
219
221
 
220
222
  ### The `tsvz` command
221
223
 
@@ -235,6 +237,7 @@ and `-d=,`, except the ones C5 and C9 list.
235
237
  | C9 | Arguments after STORE for `read`, `clear` and `scrub` are a usage error, exit 2 (3.39 ignored them, so `clear junk` cleared the file). |
236
238
  | C10 | The table (a terminal, or `--format table`) of `get`, `keys`, `pop`, `popitem`, `setdefault`, and of `read` on a `.tsvz`-family store, shows the rows only: no row is drawn as a header, and no row is cut to the first row's or the terminal's width. `read` of a `.tsv`-family file keeps 3.39's table, whose first row is the file's header line. |
237
239
  | C11 | The table shows control characters, bidi controls and line separators as escapes (`\r`, `\n`, `\t`, `\xNN`, `\uNNNN`), so no value can move the cursor, reorder the line or restyle the terminal. It sizes columns in terminal columns, so CJK text (two columns a character) lines up. The table is for reading: a backslash in a value is shown as is, so use records for exact values. |
240
+ | C12 | While stripping is on, `get`, `has`, `pop`, `delete` and `setdefault` take the live key equal to a KEY ending in a space or tab before the stripped one (spec §20.2.7), so `tsvz keys s \| tsvz delete s -` deletes every key. `delete` of such a KEY reads the store first, and exits 1 without writing when the store cannot be read whole. |
238
241
 
239
242
  ## Spec deviations and interpretations
240
243
 
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: TSVZ
3
- Version: 4.6
3
+ Version: 4.7
4
4
  Summary: Key-value store kept in a TSV / CSV / NSV / PSV file, with tsvz-spec-v1 support
5
5
  Home-page: https://github.com/yufei-pan/TSVZ
6
6
  Author: Yufei Pan
@@ -34,7 +34,7 @@ key-value store in a delimiter-separated text file**. The first column of each
34
34
  row is the key; the file behaves like an ordered dictionary that is persisted to
35
35
  disk as you change it.
36
36
 
37
- **Version 4.6**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
37
+ **Version 4.7**, Python **3.6+**, one self-contained module (`TSVZ.py`): drop it
38
38
  next to your code and `import TSVZ`, or install it.
39
39
 
40
40
  ```bash
@@ -224,6 +224,7 @@ delimiter or encoding argument that conflicts with the extension.
224
224
  | L4 | New warnings on stderr (invalid encoding, lines dropped by `strict`); data unchanged. |
225
225
  | L5 | A damaged compressed file is read up to the damage instead of raising. |
226
226
  | L6 | `mapToFile` no longer glues a row it adds at the end of the file to the next one. |
227
+ | L7 | `TSVZed`'s `in` and `get` find a key as `t[key]` does: the live key equal to it, else the key right-stripped. 3.39 matched them exactly, so `'x ' in t` was false while `t['x ']` read `x`, and `t.setdefault('x ', …)` overwrote `x`. |
227
228
 
228
229
  ### `.tsvz`, `.csvz`, `.nsvz`, `.psvz`: now tsvz-spec-v1
229
230
 
@@ -243,6 +244,7 @@ delimiter or encoding argument that conflicts with the extension.
243
244
  | S12 | `TSVZedLite.clear()` and `clear_file()` raise `OSError` when the clear is refused (a part could not be read, the path names one part of a store, or the store kept changing). Memory is reloaded from the unchanged store first. |
244
245
  | S13 | Compressed parts are written in members of at most 1 MiB of records (`#_member_size_#`), Zstandard frames with a checksum; an append never reads or repairs the part. |
245
246
  | S14 | A zero byte in a value is written `<NUL>`, and a raw zero byte in a record is read as zero-fill damage (spec §4.9): under the default `recover` only the bytes after it are read. **A value that 4.2 or 4.3 wrote with a raw zero byte therefore reads differently in 4.4.** Such values are rare; before upgrading, `grep -c -P '\x00' store.tsvz` (any `.tsvz`, `.csvz` or `.psvz` part) finds them. Stop 4.2 and 4.3 handlers before upgrading: they read `<NUL>` as text, as they read `<CR>`. |
247
+ | S15 | A lookup (`t[key]`, `in`, `get`, `del`, `pop`, `setdefault`) takes the live key equal to the one given before the right-stripped one, so a key ending in whitespace (written while `#_strip_trailing_whites_#` was false) can be read and deleted by name (spec §20.2.7). Setting a key still right-strips it. |
246
248
 
247
249
  ### The `tsvz` command
248
250
 
@@ -262,6 +264,7 @@ and `-d=,`, except the ones C5 and C9 list.
262
264
  | C9 | Arguments after STORE for `read`, `clear` and `scrub` are a usage error, exit 2 (3.39 ignored them, so `clear junk` cleared the file). |
263
265
  | C10 | The table (a terminal, or `--format table`) of `get`, `keys`, `pop`, `popitem`, `setdefault`, and of `read` on a `.tsvz`-family store, shows the rows only: no row is drawn as a header, and no row is cut to the first row's or the terminal's width. `read` of a `.tsv`-family file keeps 3.39's table, whose first row is the file's header line. |
264
266
  | C11 | The table shows control characters, bidi controls and line separators as escapes (`\r`, `\n`, `\t`, `\xNN`, `\uNNNN`), so no value can move the cursor, reorder the line or restyle the terminal. It sizes columns in terminal columns, so CJK text (two columns a character) lines up. The table is for reading: a backslash in a value is shown as is, so use records for exact values. |
267
+ | C12 | While stripping is on, `get`, `has`, `pop`, `delete` and `setdefault` take the live key equal to a KEY ending in a space or tab before the stripped one (spec §20.2.7), so `tsvz keys s \| tsvz delete s -` deletes every key. `delete` of such a KEY reads the store first, and exits 1 without writing when the store cannot be read whole. |
265
268
 
266
269
  ## Spec deviations and interpretations
267
270
 
@@ -40,10 +40,10 @@ if os.name == 'nt':
40
40
  elif os.name == 'posix':
41
41
  import fcntl
42
42
 
43
- version = '4.6'
43
+ version = '4.7'
44
44
  __version__ = version
45
45
  author = 'pan@zopyr.us'
46
- COMMIT_DATE = '2026-10-12'
46
+ COMMIT_DATE = '2026-10-13'
47
47
 
48
48
  DEFAULT_DELIMITER = '\t'
49
49
  DEFAULTS_INDICATOR_KEY = '#_defaults_#'
@@ -1605,6 +1605,8 @@ def scrubTabularFile(fileName,teeLogger = None,header = '',createIfNotExist = Fa
1605
1605
  # ===========================================================================
1606
1606
  MAX_SPEC_VERSION = 1
1607
1607
  _MARKER_RE = re.compile(r'^#_[A-Za-z0-9_-]+_#$')
1608
+ #: What a lookup gets for a key that is not live (spec §20.2.7: then the stripped key is tried).
1609
+ _NO_ROW = object()
1608
1610
  _CHECKSUM_RE = re.compile(r'^#_checksum_([A-Za-z0-9_-]+)_#$', re.IGNORECASE)
1609
1611
  _ASCII_DIGITS_RE = re.compile(r'^[0-9]+$')
1610
1612
  _SPEC_DECODE_RE = re.compile(r'<(?:sep|LF|CR|NUL|lt|#)>')
@@ -1669,8 +1671,7 @@ def _specTombstones(keys, delimiter, strip):
1669
1671
  (spec §9.6).
1670
1672
  """
1671
1673
  rows = [([key], False) for key in keys]
1672
- if strip and any(key.endswith((' ', '\t')) and _specEncodeField(key, delimiter, isKey=True).endswith((' ', '\t'))
1673
- for key in keys):
1674
+ if strip and any(_specStrippable(key, delimiter) for key in keys):
1674
1675
  rows.insert(0, (['#_strip_trailing_whites_#', 'false'], True))
1675
1676
  rows.append((['#_strip_trailing_whites_#', 'true'], True))
1676
1677
  return rows
@@ -1687,6 +1688,55 @@ def _specTombstoneLines(key, delimiter, strip):
1687
1688
  for cells, marker in _specTombstones([key], delimiter, strip))
1688
1689
 
1689
1690
 
1691
+ def _specStrippable(key, delimiter):
1692
+ """True when a reader that strips (spec §7.7) would change ``key``: its encoded form ends in a space or tab."""
1693
+ return key.endswith((' ', '\t')) and _specEncodeField(key, delimiter, isKey=True).endswith((' ', '\t'))
1694
+
1695
+
1696
+ def _specNameKey(key, delimiter, strip, live):
1697
+ """The key that a requested ``key`` names (spec §20.2.7) in a store whose stripping is ``strip``.
1698
+
1699
+ While stripping is on, a key a reader would strip names the live key equal
1700
+ to it (``live`` holds the live keys), else the key a reader resolves it to.
1701
+ """
1702
+ if not strip or key in live or not _specStrippable(key, delimiter):
1703
+ return key
1704
+ return _specDecodeField(_specEncodeField(key, delimiter, isKey=True).rstrip(' \t'), delimiter)
1705
+
1706
+
1707
+ def _specNamesKeys(rows, delimiter):
1708
+ """True when naming the keys of ``delete`` rows (``[(cells, marker), ...]``) needs the store (spec §20.2.7)."""
1709
+ return any(not marker and _specStrippable(cells[0], delimiter) for cells, marker in rows)
1710
+
1711
+
1712
+ def _specDeleteRows(rows, delimiter, strip, live):
1713
+ """The batch ``delete`` appends for ``rows`` (``[(cells, marker), ...]``, spec §20.2, §20.3).
1714
+
1715
+ Every key names a key of the store as it was before the batch, whose
1716
+ stripping was ``strip`` and whose live keys are ``live`` (§20.2.7). The
1717
+ tombstone of a key a reader would strip is written with stripping off
1718
+ (§9.6). A reset of ``#_strip_trailing_whites_#`` in the batch turns
1719
+ stripping back on (§10.3, §12.1).
1720
+ """
1721
+ batch = []
1722
+ named = strip
1723
+ guarded = False
1724
+ for cells, marker in rows:
1725
+ if marker:
1726
+ if cells[0].lower() == '#_strip_trailing_whites_#':
1727
+ strip, guarded = True, False
1728
+ batch.append((cells, marker))
1729
+ continue
1730
+ key = _specNameKey(cells[0], delimiter, named, live)
1731
+ if strip and not guarded and _specStrippable(key, delimiter):
1732
+ batch.append((['#_strip_trailing_whites_#', 'false'], True))
1733
+ guarded = True
1734
+ batch.append(([key], False))
1735
+ if guarded:
1736
+ batch.append((['#_strip_trailing_whites_#', 'true'], True))
1737
+ return batch
1738
+
1739
+
1690
1740
  class _SpecState(object):
1691
1741
  """Reader state of spec §12, carried across records and parts."""
1692
1742
 
@@ -3628,14 +3678,37 @@ class TSVZed(OrderedDict):
3628
3678
  # else:
3629
3679
  # self.appendEvent.set()
3630
3680
 
3681
+ def _nameKey(self, key):
3682
+ """The key a requested ``key`` names: the live key equal to it, else ``key`` right-stripped (spec §20.2.7).
3683
+
3684
+ The fallback strips every trailing whitespace whatever the stripping
3685
+ marker says, as ``__setitem__`` strips every key it writes.
3686
+ """
3687
+ key = str(key)
3688
+ return key if OrderedDict.__contains__(self, key) else key.rstrip()
3689
+
3631
3690
  def __getitem__(self, key):
3632
- return super().__getitem__(str(key).rstrip())
3633
-
3691
+ key = str(key)
3692
+ row = dict.get(self, key, _NO_ROW) # one probe for the usual hit
3693
+ return row if row is not _NO_ROW else OrderedDict.__getitem__(self, key.rstrip())
3694
+
3695
+ def __contains__(self, key):
3696
+ # OrderedDict.setdefault asks this before it reads or writes, so it names keys as t[key] does.
3697
+ key = str(key)
3698
+ return dict.__contains__(self, key) or dict.__contains__(self, key.rstrip())
3699
+
3700
+ def get(self, key, default=None):
3701
+ key = str(key)
3702
+ row = dict.get(self, key, _NO_ROW)
3703
+ if row is _NO_ROW:
3704
+ row = dict.get(self, key.rstrip(), _NO_ROW)
3705
+ return default if row is _NO_ROW else row
3706
+
3634
3707
 
3635
3708
  def __delitem__(self,key):
3636
3709
  if self._spec:
3637
3710
  return self._specDelItem(key)
3638
- key = str(key).rstrip()
3711
+ key = self._nameKey(key)
3639
3712
  if key == DEFAULTS_INDICATOR_KEY:
3640
3713
  self.defaults = [DEFAULTS_INDICATOR_KEY]
3641
3714
  if self.verbose:
@@ -3739,7 +3812,7 @@ class TSVZed(OrderedDict):
3739
3812
  self.lastUpdateTime = get_time_ns()
3740
3813
 
3741
3814
  def _specDelItem(self, key):
3742
- key = str(key).rstrip()
3815
+ key = self._nameKey(key)
3743
3816
  if _MARKER_RE.match(key):
3744
3817
  # A lone marker key resets that marker (spec §12.1).
3745
3818
  self._specSetMarker(key, [key])
@@ -3914,12 +3987,14 @@ memoryOnly:{self.memoryOnly}
3914
3987
  is raised.
3915
3988
 
3916
3989
  '''
3917
- key = str(key).rstrip()
3918
- if key not in self:
3990
+ key = self._nameKey(key)
3991
+ if not OrderedDict.__contains__(self, key):
3919
3992
  if default is self.__marker:
3920
3993
  raise KeyError(key)
3921
3994
  return default
3922
- value = super().pop(key)
3995
+ # Not OrderedDict.pop: before Python 3.11 it goes through del self[key], which writes a tombstone too.
3996
+ value = OrderedDict.__getitem__(self, key)
3997
+ OrderedDict.__delitem__(self, key)
3923
3998
  if not self.memoryOnly:
3924
3999
  self.__appendEmptyLine(key)
3925
4000
  self.lastUpdateTime = get_time_ns()
@@ -3929,7 +4004,7 @@ memoryOnly:{self.memoryOnly}
3929
4004
  '''Move an existing element to the end (or beginning if last is false).
3930
4005
  Raise KeyError if the element does not exist.
3931
4006
  '''
3932
- key = str(key).rstrip()
4007
+ key = self._nameKey(key)
3933
4008
  super().move_to_end(key, last)
3934
4009
  if self._spec:
3935
4010
  _warnOnce(self, 'move_to_end', self._fileName,
@@ -4452,9 +4527,18 @@ class TSVZedLite(MutableMapping):
4452
4527
  """'tsvz' for tsvz-spec-v1 files (.tsvz/.csvz/.nsvz/.psvz), 'tsv' for 3.39-format files."""
4453
4528
  return 'tsvz' if self._spec else 'tsv'
4454
4529
 
4530
+ def _nameKey(self, key):
4531
+ """The key a requested ``key`` names: the live key equal to it, else ``key`` right-stripped (spec §20.2.7).
4532
+
4533
+ The fallback strips every trailing whitespace whatever the stripping
4534
+ marker says, as ``__setitem__`` strips every key it writes.
4535
+ """
4536
+ key = str(key)
4537
+ return key if key in self.indexes else key.rstrip()
4538
+
4455
4539
  def __contains__(self, key):
4456
4540
  if self._spec:
4457
- return str(key).rstrip() in self.indexes
4541
+ return self._nameKey(key) in self.indexes
4458
4542
  return MutableMapping.__contains__(self, key)
4459
4543
 
4460
4544
  def get(self, key, default=None):
@@ -4832,7 +4916,7 @@ class TSVZedLite(MutableMapping):
4832
4916
 
4833
4917
  # Implement basic __getitem__, __setitem__, __delitem__, __iter__, and __len__. needed for MutableMapping
4834
4918
  def __getitem__(self,key):
4835
- key = str(key).rstrip()
4919
+ key = self._nameKey(key)
4836
4920
  if key not in self.indexes:
4837
4921
  if key == DEFAULTS_INDICATOR_KEY:
4838
4922
  return self.defaults
@@ -4911,7 +4995,7 @@ class TSVZedLite(MutableMapping):
4911
4995
  self.indexes[key] = self.__writeValues(value)
4912
4996
 
4913
4997
  def __delitem__(self,key):
4914
- key = str(key).rstrip()
4998
+ key = self._nameKey(key)
4915
4999
  if self._spec:
4916
5000
  if _MARKER_RE.match(key):
4917
5001
  # A lone marker key resets that marker (spec §12.1).
@@ -4969,7 +5053,7 @@ class TSVZedLite(MutableMapping):
4969
5053
 
4970
5054
  __marker = object()
4971
5055
  def pop(self, key, default=__marker, return_pos = False):
4972
- key = str(key).rstrip()
5056
+ key = self._nameKey(key)
4973
5057
  try:
4974
5058
  pos = self.indexes.pop(key)
4975
5059
  except KeyError:
@@ -5430,9 +5514,9 @@ class _ServeStore(object):
5430
5514
  return not self.parts
5431
5515
 
5432
5516
  def resolve(self, key):
5433
- """A requested key as a reader resolves it (spec §7.7; 3.39 rstrip for loose files)."""
5517
+ """The key a requested key names (spec §20.2.7; 3.39 rstrip for loose files)."""
5434
5518
  if self.spec:
5435
- return key.rstrip(' \t') if self.state.strip else key
5519
+ return _specNameKey(key, self.delimiter, self.state.strip, self.data)
5436
5520
  return key.rstrip()
5437
5521
 
5438
5522
  def read(self, log):
@@ -5528,10 +5612,27 @@ class _ServeStore(object):
5528
5612
  self.writer.wait(queued[0])
5529
5613
 
5530
5614
  def write(self, rows, sync, log, written=False):
5531
- """Append ``rows`` (``[(cells, marker), ...]``) as one batch: ``set``, ``delete`` and bulk input."""
5615
+ """Append ``rows`` (``[(cells, marker), ...]``) as one batch: ``set`` and its bulk input."""
5616
+ with self.lock:
5617
+ queued = self._submit(rows, sync, log)
5618
+ self._ack(queued, written)
5619
+
5620
+ def delete(self, rows, sync, log, written=False):
5621
+ """Append the tombstones of ``delete`` (``[(cells, marker), ...]``) as one batch (spec §20.2.7).
5622
+
5623
+ Naming a key a reader would strip needs the store as it is now, read
5624
+ under the lock, so that delete is atomic (spec §21.8.2). Returns False,
5625
+ having written nothing, when that read could not be completed.
5626
+ """
5532
5627
  with self.lock:
5628
+ if self.spec and _specNamesKeys(rows, self.delimiter):
5629
+ self._fresh(log)
5630
+ if self.unreadable:
5631
+ return False
5632
+ rows = _specDeleteRows(rows, self.delimiter, self.state.strip, self.data)
5533
5633
  queued = self._submit(rows, sync, log)
5534
5634
  self._ack(queued, written)
5635
+ return True
5535
5636
 
5536
5637
  def pop(self, key, sync, log):
5537
5638
  """Atomically return the row of ``key`` and append its tombstone; None when it is missing."""
@@ -5781,7 +5882,10 @@ def _serveDispatch(store, line, bulk, log):
5781
5882
  rows = [([key], bool(_MARKER_RE.match(field))) for key, field in zip(args, raw)]
5782
5883
  else:
5783
5884
  rows = [(args, bool(_MARKER_RE.match(raw[0])))]
5784
- store.write(rows, sync, log, written)
5885
+ if not keysOnly:
5886
+ store.write(rows, sync, log, written)
5887
+ elif not store.delete(rows, sync, log, written):
5888
+ return 1, [], _SERVE_UNREADABLE + '; nothing was written'
5785
5889
  return 0, [], ''
5786
5890
  if operation in ('pop', 'popitem'):
5787
5891
  if operation == 'pop':
@@ -6482,7 +6586,10 @@ class TSVZClient(MutableMapping):
6482
6586
 
6483
6587
  def __delitem__(self, key):
6484
6588
  """Delete ``key``; a missing key is not an error (as in ``TSVZed``)."""
6485
- self._request(['delete', _serveEncode(str(key))], write=True)
6589
+ key = str(key)
6590
+ # Resending the delete of a key ending in whitespace could name, and delete, the stripped key (spec §20.2.7).
6591
+ retry = not (self._spec and _specStrippable(key, self.delimiter))
6592
+ self._request(['delete', _serveEncode(key)], write=True, retry=retry)
6486
6593
 
6487
6594
  _marker = object()
6488
6595
 
@@ -6508,6 +6615,11 @@ class TSVZClient(MutableMapping):
6508
6615
  cells = self._cells(key, default)
6509
6616
  if len(cells) < 2:
6510
6617
  return self.get(key)
6618
+ if str(key) != cells[0]:
6619
+ # cells[0] is stripped as set strips it; a live key equal to ``key`` wins first (spec §20.2.7).
6620
+ row = self.get(key)
6621
+ if row is not None:
6622
+ return row
6511
6623
  lines = self._request(['setdefault'] + self._fields(cells), write=True, retry=False)[0]
6512
6624
  return _serveDecodeLine(lines[0]) if lines else None
6513
6625
 
@@ -6953,7 +7065,7 @@ def _cliRead(args, delimiter, logger, stdin, stdout):
6953
7065
  def _cliGet(args, delimiter, logger, stdin, stdout):
6954
7066
  """``get STORE KEY ...`` (spec §20.2): each KEY's row in argument order; exit 3 when one is missing.
6955
7067
 
6956
- Keys resolve as a reader resolves them (§7.7). For a missing key of a
7068
+ Each KEY names a key as §20.2.7 says. For a missing key of a
6957
7069
  spec store this prints what ``TSVZed`` returns for it (§14.5): the key and
6958
7070
  the active defaults while ``#_return_defaults_when_missing_#`` is true,
6959
7071
  else nothing. A missing key of a loose file prints nothing.
@@ -6972,7 +7084,7 @@ def _cliGet(args, delimiter, logger, stdin, stdout):
6972
7084
  missing = False
6973
7085
  for key in args.args:
6974
7086
  if spec:
6975
- key = key.rstrip(' \t') if state.strip else key
7087
+ key = _specNameKey(key, delimiter, state.strip, data)
6976
7088
  else:
6977
7089
  key = key.rstrip()
6978
7090
  row = data.get(key)
@@ -7018,15 +7130,20 @@ def _cliStdinBatch(args, delimiter, logger, stdin, keysOnly):
7018
7130
  so is a line with an empty key (reported). ``keysOnly`` keeps the first
7019
7131
  field only: a tombstone, or a marker reset.
7020
7132
  """
7021
- spec = _isSpecPath(args.store)
7133
+ rows = _cliStdinRows(args, delimiter, logger, stdin, keysOnly)
7134
+ if not _isSpecPath(args.store):
7135
+ return [cells for cells, marker in rows]
7136
+ return [_specFormatRecord(cells, delimiter, marker=marker) for cells, marker in rows]
7137
+
7138
+
7139
+ def _cliStdinRows(args, delimiter, logger, stdin, keysOnly):
7140
+ """Standard input for ``set STORE -`` / ``delete STORE -`` as ``[(cells, marker), ...]`` (spec §20.3)."""
7022
7141
  source = _Reporter('<stdin>', logger)
7023
7142
  try:
7024
- rows = _parseRecordLines(_cliStdinLines(stdin, source), delimiter, spec, keysOnly, source)
7143
+ return _parseRecordLines(_cliStdinLines(stdin, source), delimiter, _isSpecPath(args.store), keysOnly,
7144
+ source)
7025
7145
  finally:
7026
7146
  source.flush()
7027
- if not spec:
7028
- return [cells for cells, marker in rows]
7029
- return [_specFormatRecord(cells, delimiter, marker=marker) for cells, marker in rows]
7030
7147
 
7031
7148
 
7032
7149
  def _parseRecordLines(texts, delimiter, spec, keysOnly, reporter):
@@ -7091,12 +7208,33 @@ def _cliDelete(args, delimiter, logger, stdin, stdout):
7091
7208
  """``delete STORE KEY [KEY ...]`` (spec §20.2): one tombstone per KEY, appended in one write.
7092
7209
 
7093
7210
  ``delete STORE -`` deletes the first field of every line on standard input (§20.3).
7211
+ A key that a reader would strip names a key of the store (§20.2.7), so the
7212
+ store is read first; when it cannot be read whole nothing is written and
7213
+ the exit status is 1.
7094
7214
  """
7095
- if args.args == ['-']:
7096
- _cliAppendBatch(args, delimiter, logger, _cliStdinBatch(args, delimiter, logger, stdin, keysOnly=True))
7215
+ if not _isSpecPath(args.store):
7216
+ if args.args == ['-']:
7217
+ _cliAppendBatch(args, delimiter, logger, _cliStdinBatch(args, delimiter, logger, stdin, keysOnly=True))
7218
+ else:
7219
+ appendLinesTabularFile(args.store, [[key] for key in args.args], teeLogger=logger, header=args.header,
7220
+ createIfNotExist=True, verbose=args.verbose, strict=args.strict,
7221
+ delimiter=delimiter)
7097
7222
  return _cliWritten(args, logger)
7098
- appendLinesTabularFile(args.store, [[key] for key in args.args], teeLogger=logger, header=args.header,
7099
- createIfNotExist=True, verbose=args.verbose, strict=args.strict, delimiter=delimiter)
7223
+ if args.args == ['-']:
7224
+ rows = _cliStdinRows(args, delimiter, logger, stdin, keysOnly=True)
7225
+ else:
7226
+ rows = [([key], bool(_MARKER_RE.match(key))) for key in args.args]
7227
+ if _specNamesKeys(rows, delimiter):
7228
+ strip, live = True, {} # a store delete creates starts with stripping on (spec §10.3)
7229
+ if _cliStoreExists(args.store):
7230
+ load, readable = _cliSpecLoad(args, delimiter, logger)
7231
+ if not readable:
7232
+ logger.teelog('tsvz: {}: could not read every part of the store; nothing was written'.format(
7233
+ args.store), 'error')
7234
+ return 1
7235
+ strip, live = load.state.strip, load.data
7236
+ rows = _specDeleteRows(rows, delimiter, strip, live)
7237
+ _cliAppendBatch(args, delimiter, logger, [_specFormatRecord(cells, delimiter, marker=marker) for cells, marker in rows])
7100
7238
  return _cliWritten(args, logger)
7101
7239
 
7102
7240
 
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes