recs 0.9.2__tar.gz → 0.10.0__tar.gz

recs-0.10.0/PKG-INFO

@@ -0,0 +1,141 @@
+Metadata-Version: 2.4
+Name: recs
+Version: 0.10.0
+Summary: 🎙 The Universal Recorder 🎙
+Author: Tom Ritchford
+Author-email: Tom Ritchford <tom@swirly.com>
+License-Expression: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: dtyper
+Requires-Dist: numpy
+Requires-Dist: pyaudio
+Requires-Dist: rich
+Requires-Dist: sounddevice
+Requires-Dist: soundfile
+Requires-Dist: threa>=1.9.0
+Requires-Dist: typer
+Requires-Dist: impall
+Requires-Dist: overrides
+Requires-Dist: coverage
+Requires-Dist: strenum
+Requires-Dist: humanfriendly
+Requires-Dist: tomli
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+
+#  🎬 recs: the Universal Recorder 🎬
+
+## Why should there be a record button at all?
+
+A long time ago, I asked myself, "Why is there a record button and the possibility
+of missing a take? Why not record everything?"
+
+I sometimes play music, and I have mixed bands live, and I wanted a program that would
+simply record everything at all times which I didn't have to stop and start, that I
+could run completely separately from my other music programs.
+
+Separately, I wanted to digitize a huge number of cassettes and LPs, so I wanted
+a program that ran in the background and recorded everything except silence, so I just
+play the music into the machine, and have it divided into pieces
+
+Nothing like that existed so I wrote it.
+
+## `recs`:  the Universal Recorder
+
+`recs` records any or every audio input on your machine, intelligently filters
+out quiet, and stores the results in named, organized files.
+
+Free, open-source, configurable, light on CPU and memory, and bulletproof
+
+### Bulletproof?
+
+It's not difficult to record some audio. Writing a program that runs continuously and
+records audio even as real-world things happen is considerably harder.
+
+It is impossible to prevent all loss, but considerable ingenuity and pulling of cables
+has been used to mitigate and minimize this through software.  See Appendix A.
+
+### Universal?
+
+It is a "Universal Recorder" because the plan to be able to record all streams of data:
+audio is simply the start.
+
+I have already [written code](https://github.com/rec/litoid) to do this for MIDI and DMX
+- it works well but it isn't productionized, and I'll be folding that in in due time,
+but most of the difficulty and most of the value in this first step is the audio, so I
+have focused on just audio for this first release!
+
+It might be that video is also incorporated in the far future, but the tooling is just
+not there for Python yet, and it would be much too heavy to sit in the background all
+the time and almost be forgotten about, so you could call it an Almost Universal
+Recorder if you liked.
+
+### Installation
+
+`recs` is a standard PyPi package - use `poetry add recs` or `pip install recs` or your
+favorite package manager.
+
+To test, type `recs --info`, which prints JSON describing the input devices
+you have. Here's a snippet from my machine:
+
+```
+[
+    {
+        "name": "FLOW 8 (Recording)",
+        "index": 1,
+        "hostapi": 0,
+        "max_input_channels": 10,
+        "max_output_channels": 4,
+        "default_low_input_latency": 0.01,
+        "default_low_output_latency": 0.004354166666666667,
+        "default_high_input_latency": 0.1,
+        "default_high_output_latency": 0.0136875,
+        "default_samplerate": 48000.0
+    },
+    {
+        "name": "USB PnP Sound Device",
+        "index": 2,
+        ...
+    },
+    ...
+]
+```
+
+### Basic Usage
+
+Pick your nicest terminal program, go to a favorite directory with some free space, and
+type:
+
+```
+recs
+```
+
+`recs` will start recording all the active audio channels into your current directory
+and display the results in the terminal.
+
+What "active"means can be customized rather a lot, but by default when a channel becomes
+too quiet for more than a short time, it stops recording, and will start a new recording
+automatically when the channel receives a signal.
+
+Some care is taken to preserve the quiet before the start or after the end of a
+recording to prevent abrupt transitions.
+
+
+#### Appendix A: Failure modes
+
+1. Hardware crash or power loss
+2. Segfault or similar C/C++ errors
+
+
+The aim is to be as bulletproof as possible. The pre-beta existing as I write this
+(2023/11/19) seems to handle harder cases like hybernation well, and can
+detect if a  device goes offline and report it.
+
+The holy grail is reconnecting to a device that comes back online: this is an
+[unsolved problem](https://github.com/spatialaudio/python-sounddevice/issues/382)
+in Python, I believe, but I am on my way to solving it.

recs-0.10.0/README.md

@@ -0,0 +1,111 @@
+#  🎬 recs: the Universal Recorder 🎬
+
+## Why should there be a record button at all?
+
+A long time ago, I asked myself, "Why is there a record button and the possibility
+of missing a take? Why not record everything?"
+
+I sometimes play music, and I have mixed bands live, and I wanted a program that would
+simply record everything at all times which I didn't have to stop and start, that I
+could run completely separately from my other music programs.
+
+Separately, I wanted to digitize a huge number of cassettes and LPs, so I wanted
+a program that ran in the background and recorded everything except silence, so I just
+play the music into the machine, and have it divided into pieces
+
+Nothing like that existed so I wrote it.
+
+## `recs`:  the Universal Recorder
+
+`recs` records any or every audio input on your machine, intelligently filters
+out quiet, and stores the results in named, organized files.
+
+Free, open-source, configurable, light on CPU and memory, and bulletproof
+
+### Bulletproof?
+
+It's not difficult to record some audio. Writing a program that runs continuously and
+records audio even as real-world things happen is considerably harder.
+
+It is impossible to prevent all loss, but considerable ingenuity and pulling of cables
+has been used to mitigate and minimize this through software.  See Appendix A.
+
+### Universal?
+
+It is a "Universal Recorder" because the plan to be able to record all streams of data:
+audio is simply the start.
+
+I have already [written code](https://github.com/rec/litoid) to do this for MIDI and DMX
+- it works well but it isn't productionized, and I'll be folding that in in due time,
+but most of the difficulty and most of the value in this first step is the audio, so I
+have focused on just audio for this first release!
+
+It might be that video is also incorporated in the far future, but the tooling is just
+not there for Python yet, and it would be much too heavy to sit in the background all
+the time and almost be forgotten about, so you could call it an Almost Universal
+Recorder if you liked.
+
+### Installation
+
+`recs` is a standard PyPi package - use `poetry add recs` or `pip install recs` or your
+favorite package manager.
+
+To test, type `recs --info`, which prints JSON describing the input devices
+you have. Here's a snippet from my machine:
+
+```
+[
+    {
+        "name": "FLOW 8 (Recording)",
+        "index": 1,
+        "hostapi": 0,
+        "max_input_channels": 10,
+        "max_output_channels": 4,
+        "default_low_input_latency": 0.01,
+        "default_low_output_latency": 0.004354166666666667,
+        "default_high_input_latency": 0.1,
+        "default_high_output_latency": 0.0136875,
+        "default_samplerate": 48000.0
+    },
+    {
+        "name": "USB PnP Sound Device",
+        "index": 2,
+        ...
+    },
+    ...
+]
+```
+
+### Basic Usage
+
+Pick your nicest terminal program, go to a favorite directory with some free space, and
+type:
+
+```
+recs
+```
+
+`recs` will start recording all the active audio channels into your current directory
+and display the results in the terminal.
+
+What "active"means can be customized rather a lot, but by default when a channel becomes
+too quiet for more than a short time, it stops recording, and will start a new recording
+automatically when the channel receives a signal.
+
+Some care is taken to preserve the quiet before the start or after the end of a
+recording to prevent abrupt transitions.
+
+
+#### Appendix A: Failure modes
+
+1. Hardware crash or power loss
+2. Segfault or similar C/C++ errors
+
+
+The aim is to be as bulletproof as possible. The pre-beta existing as I write this
+(2023/11/19) seems to handle harder cases like hybernation well, and can
+detect if a  device goes offline and report it.
+
+The holy grail is reconnecting to a device that comes back online: this is an
+[unsolved problem](https://github.com/spatialaudio/python-sounddevice/issues/382)
+in Python, I believe, but I am on my way to solving it.

recs-0.10.0/pyproject.toml

@@ -0,0 +1,67 @@
+[project]
+name = "recs"
+version = "0.10.0"
+description = "🎙 The Universal Recorder 🎙"
+authors = [{ name = "Tom Ritchford", email = "tom@swirly.com" }]
+requires-python = ">=3.10"
+readme = "README.md"
+license = "MIT"
+classifiers = ["Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14"]
+dependencies = [
+    "dtyper",
+    "numpy",
+    "pyaudio",
+    "rich",
+    "sounddevice",
+    "soundfile",
+    "threa>=1.9.0",
+    "typer",
+    "impall",
+    "overrides",
+    "coverage",
+    "strenum",
+    "humanfriendly",
+    "tomli",
+]
+
+[project.scripts]
+recs = "recs.base.cli:run"
+
+[dependency-groups]
+dev = [
+    "coverage>=7.4.1",
+    "pytest",
+    "pyupgrade>=3.21.2",
+    "ruff",
+    "tdir",
+    "ty>=0.0.14",
+]
+
+[tool.uv]
+
+[tool.uv.build-backend]
+module-root = ""
+
+[build-system]
+requires = ["uv_build>=0.9.0,<0.10.0"]
+build-backend = "uv_build"
+
+[tool.coverage.run]
+branch = true
+source = ["recs"]
+omit = [
+    "recs/__main__.py",
+    "recs/cfg/cli.py",
+    "recs/cfg/app.py",
+]
+
+[tool.coverage.report]
+fail_under = 91
+skip_covered = true
+exclude_lines = ["pragma: no cover", "if False:", "if __name__ == .__main__.:", "raise NotImplementedError"]
+
+[tool.ruff]
+line-length = 88
+
+[tool.ruff.format]
+quote-style = "single"

recs-0.10.0/recs/__main__.py

@@ -0,0 +1,29 @@
+import sys
+
+import click
+
+from recs.base import RecsError
+from recs.cfg import app, cli
+
+assert cli
+
+
+def run() -> int:
+    try:
+        app.app(prog_name='recs', standalone_mode=False)
+        return 0
+
+    except RecsError as e:
+        print('ERROR:', *e.args, file=sys.stderr)
+
+    except click.ClickException as e:
+        print(f'{e.__class__.__name__}: {e.message}', file=sys.stderr)
+
+    except click.Abort:
+        print('Interrupted', file=sys.stderr)
+
+    return -1
+
+
+if __name__ == '__main__':
+    sys.exit(run())

recs-0.10.0/recs/audio/block.py

@@ -0,0 +1,103 @@
+# mypy: disable-error-code="no-any-return, type-arg"
+
+import dataclasses as dc
+import numbers
+import typing as t
+from functools import cached_property
+
+import numpy as np
+
+from recs.cfg.source import to_matrix
+
+_EMPTY_SEEN = False
+
+
+@dc.dataclass(frozen=True)
+class Block:
+    block: np.ndarray
+
+    def __post_init__(self) -> None:
+        if not self.block.size:
+            raise ValueError('Empty block')
+        self.__dict__['block'] = to_matrix(self.block)
+
+    def __len__(self) -> int:
+        return self.block.shape[0]
+
+    def __getitem__(self, index: int | slice) -> 'Block':
+        return Block(self.block[index])
+
+    @cached_property
+    def is_float(self) -> bool:
+        return not issubclass(self.block.dtype.type, numbers.Integral)
+
+    @cached_property
+    def scale(self) -> float:
+        if self.is_float:
+            return 1
+        return float(1 << (8 * self.block.dtype.itemsize - 1))
+
+    @cached_property
+    def volume(self) -> float:
+        return sum(self.amplitude) / len(self.amplitude)
+
+    @cached_property
+    def channel_count(self) -> int:
+        return (self.block.shape + (1,))[1]
+
+    @cached_property
+    def amplitude(self) -> np.ndarray:
+        return (self.max - self.min) / (2 * self.scale)
+
+    @cached_property
+    def max(self) -> np.ndarray:
+        return self.block.max(0)
+
+    @cached_property
+    def min(self) -> np.ndarray:
+        return self.block.min(0)
+
+    @cached_property
+    def asfloat(self) -> 'Block':
+        if self.is_float:
+            return self
+        b = self.block.astype('double' if self.block.dtype.itemsize > 4 else 'float')
+        b /= self.scale
+        return Block(b)
+
+    @cached_property
+    def rms(self) -> np.ndarray:
+        b = self.asfloat.block
+        if b is self.block:
+            b = b * b
+        else:
+            b *= b
+        return np.sqrt(b.mean(0))
+
+
+@dc.dataclass
+class Blocks:
+    blocks: list[Block] = dc.field(default_factory=list)
+    duration: int = 0
+
+    def append(self, block: Block) -> None:
+        self.blocks.append(block)
+        self.duration += len(block)
+
+    def clear(self) -> None:
+        self.duration = 0
+        self.blocks.clear()
+
+    def clip(self, sample_length: int, from_start: bool) -> t.Sequence[Block]:
+        clipped = []
+        assert sample_length >= 0
+        while self.duration > sample_length:
+            clipped.append(self.blocks.pop(0 if from_start else -1))
+            self.duration -= len(clipped[-1])
+        return clipped
+
+    def __iter__(self) -> t.Iterator[Block]:
+        return iter(self.blocks)
+
+    def __getitem__(self, i: int) -> Block:
+        return self.blocks[i]

recs-0.10.0/recs/audio/channel_writer.py

@@ -0,0 +1,238 @@
+import contextlib
+import sys
+import typing as t
+from datetime import datetime
+from pathlib import Path
+from threading import Lock
+
+from numpy.typing import NDArray
+from overrides import override
+from soundfile import SoundFile
+from threa import Runnable
+
+from recs.base.state import ChannelState
+from recs.base.type_conversions import SUBTYPE_TO_SDTYPE
+from recs.base.types import SDTYPE, Active, Format, SdType
+from recs.cfg import Cfg, Track, time_settings
+from recs.misc import counter, file_list
+
+from .block import Block, Blocks
+from .file_opener import FileOpener
+from .header_size import header_size
+
+URL = 'https://github.com/rec/recs'
+
+BUFFER = 0x80
+MAX_WAV_SIZE = 0x1_0000_0000 - BUFFER
+
+ITEMSIZE = {
+    SdType.float32: 4,
+    SdType.int16: 2,
+    SdType.int32: 4,
+}
+
+BLOCK_FUZZ = 2
+
+
+class ChannelWriter(Runnable):
+    bytes_in_file: int = 0
+
+    frames_in_file: int = 0
+    frames_written: int = 0  # Used elsewhere
+
+    largest_file_size: int = 0
+    longest_file_frames: int = 0
+
+    timestamp: float = 0
+
+    _sfs: t.Sequence[SoundFile] = ()
+
+    @property
+    def active(self) -> Active:
+        return Active.active if self._sfs else Active.inactive
+
+    def __init__(
+        self, cfg: Cfg, times: time_settings.TimeSettings[int], track: Track
+    ) -> None:
+        super().__init__()
+
+        self.cfg = cfg
+        self.do_not_record = cfg.dry_run or cfg.calibrate
+        self.metadata = cfg.metadata
+        self.times = times
+        self.track = track
+
+        self._blocks = Blocks()
+        self._lock = Lock()
+
+        if track.source.format is None or cfg.cfg.formats:
+            self.formats = cfg.formats
+        else:
+            self.formats = [track.source.format]
+
+        if track.source.subtype is None or cfg.cfg.subtype:
+            subtype = cfg.subtype
+        else:
+            subtype = track.source.subtype
+
+        if track.source.subtype is None or cfg.cfg.sdtype:
+            sdtype = cfg.sdtype or SDTYPE
+        else:
+            sdtype = SUBTYPE_TO_SDTYPE[track.source.subtype]
+
+        self.files_written = file_list.FileList()
+        self.frame_size = ITEMSIZE[sdtype] * len(track.channels)
+        self.longest_file_frames = times.longest_file_time
+
+        self.openers = [
+            FileOpener(
+                channels=len(track.channels),
+                format=f,
+                samplerate=track.source.samplerate,
+                subtype=subtype,
+            )
+            for f in self.formats
+        ]
+        self._volume = counter.MovingBlock(times.moving_average_time)
+
+        def size(f: str) -> int:
+            return (
+                MAX_WAV_SIZE if f == Format.wav and not self.cfg.infinite_length else 0
+            )
+
+        self.largest_file_size = max(0, *(size(f) for f in cfg.formats))
+
+    def to_block(self, array: NDArray) -> Block:
+        return Block(array[:, self.track.slice])
+
+    def receive_update(
+        self, block: Block, timestamp: float, should_record: bool = False
+    ) -> ChannelState:
+        with self._lock:
+            should_record = should_record or self.should_record(block)
+            return self._receive_block(block, timestamp, should_record)
+
+    def should_record(self, block: Block) -> bool:
+        return (
+            self.times.record_everything
+            or block.volume >= self.times.noise_floor_amplitude
+        )
+
+    @override
+    def stop(self) -> None:
+        with self._lock:
+            self.running = False
+            self._write_and_close()
+            self.stopped = True
+
+    def _close(self) -> None:
+        sfs, self._sfs = self._sfs, ()
+        for sf in sfs:
+            if sf.frames and sf.frames >= self.times.shortest_file_time:
+                print(sf.name, file=sys.stderr)
+                sf.close()
+            else:
+                with contextlib.suppress(Exception):
+                    sf.close()
+                with contextlib.suppress(Exception):
+                    Path(sf.name).unlink()
+
+    def _open(self, offset: int) -> t.Sequence[SoundFile]:
+        timestamp = self.timestamp - offset / self.track.source.samplerate
+        date = datetime.fromtimestamp(timestamp).isoformat()
+        index = 1 + len(self.files_written)
+        metadata = {'date': date, 'software': URL, 'tracknumber': str(index)}
+        metadata |= self.metadata
+
+        self.bytes_in_file = max(header_size(metadata, f) for f in self.cfg.formats)
+        self.frames_in_file = 0
+
+        path = self.cfg.output_directory.make_path(
+            self.track, self.cfg.aliases, timestamp, index
+        )
+        sfs = [o.create(metadata, path) for o in self.openers]
+        self.files_written.extend(Path(sf.name) for sf in sfs)
+        return sfs
+
+    def _receive_block(
+        self, block: Block, timestamp: float, should_record: bool
+    ) -> ChannelState:
+        saved_state = self._state(
+            max_amp=max(block.max) / block.scale,
+            min_amp=min(block.min) / block.scale,
+        )
+
+        dt = self.timestamp - timestamp
+        self.timestamp = timestamp
+        self._volume(block)
+
+        if not self.do_not_record and (self._sfs or not self.stopped):
+            expected_dt = len(block) / self.track.source.samplerate
+
+            if dt > expected_dt * BLOCK_FUZZ:  # We were asleep, or otherwise lost time
+                self._write_and_close()
+
+            self._blocks.append(block)
+
+            if should_record:
+                if not self._sfs:  # Record some quiet before the first block
+                    length = self.times.quiet_before_start + len(self._blocks[-1])
+                    self._blocks.clip(length, from_start=True)
+
+                self._write_blocks(self._blocks)
+                self._blocks.clear()
+
+            if self.stopped or self._blocks.duration > self.times.stop_after_quiet:
+                self._write_and_close()
+
+        return self._state() - saved_state
+
+    def _state(self, **kwargs: t.Any) -> ChannelState:
+        return ChannelState(
+            file_count=len(self.files_written),
+            file_size=self.files_written.total_size,
+            is_active=bool(self._sfs),
+            recorded_time=self.frames_written / self.track.source.samplerate,
+            timestamp=self.timestamp,
+            volume=tuple(self._volume.mean()),
+            **kwargs,
+        )
+
+    def _write_and_close(self) -> None:
+        # Record some quiet after the last block
+        removed = self._blocks.clip(self.times.quiet_after_end, from_start=False)
+
+        if self._sfs and removed:
+            self._write_blocks(reversed(removed))
+
+        self._close()
+
+    def _write_blocks(self, blox: t.Iterable[Block]) -> None:
+        blocks = list(blox)
+
+        # The last block in the list ends at self.timestamp so
+        # we keep track of the sample offset before that
+        offset = -sum(len(b) for b in blocks)
+
+        for b in blocks:
+            # Check if this block will overrun the file size or length
+            remains: list[int] = []
+
+            if self.longest_file_frames:
+                remains.append(self.longest_file_frames - self.frames_in_file)
+
+            if self._sfs and self.largest_file_size:
+                file_bytes = self.largest_file_size - self.bytes_in_file
+                remains.append(file_bytes // self.frame_size)
+
+            if remains and min(remains) <= len(b):
+                self._close()
+
+            self._sfs = self._sfs or self._open(offset)
+            for sf in self._sfs:
+                sf.write(b.block)
+            offset += len(b)
+
+            self.frames_in_file += len(b)
+            self.frames_written += len(b)
+            self.bytes_in_file += len(b) * self.frame_size