sinter 1.12.1__tar.gz → 1.13.0__tar.gz

{sinter-1.12.1 → sinter-1.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: sinter
-Version: 1.12.1
+Version: 1.13.0
 Summary: Samples stim circuits and decodes them using pymatching.
 Author: Craig Gidney
 Author-email: craig.gidney@gmail.com
@@ -30,8 +30,8 @@ quantum error correction circuits.
 
 Sinter takes Stim circuits annotated with noise, detectors, and logical
 observables.
-It uses stim to sample the circuits and pymatching to predict whether the
-logical observables were flipped or not, given the detector data.
+It uses stim to sample the circuits and a decoder such as pymatching to predict
+whether the logical observables were flipped or not, given the detector data.
 It records how often this succeeds, and how often it fails (the error rate).
 
 Sinter uses python multiprocessing to do parallel sampling across multiple CPU
@@ -40,11 +40,12 @@ specified by the user (such as a target number of errors), saves the results to
 as simple CSV format, and has some basic  plotting functionality for viewing the
 results.
 
-Sinter doesn't support cloud compute, but it does scale well on
-a single machine.
+Sinter doesn't support cloud compute, but it does scale well on  a single
+machine.
 I've tested it on 2 core machines, 4 core machines, and 96 core machines.
-Although there are potential pitfalls (e.g. setting batch sizes too large causes thrashing),
-sinter generally achieves good resource utilization of the processes you assign to it.
+Although there are potential pitfalls (e.g. setting batch sizes too large causes
+thrashing), sinter generally achieves good resource utilization of the processes
+you assign to it.
 
 <a name="how_to_install"></a>
 # How to install
@@ -63,8 +64,8 @@ to use sinter's python API.
 <a name="how_to_use_python"></a>
 # How to use: Python API
 
-This example assumes you are in a python environment with sinter
-installed.
+This example assumes you are in a python environment with `sinter` and
+`pymatching` installed.
 
 ```python
 import stim
@@ -152,19 +153,8 @@ and the corresponding image saved to `plot.png`:
 ## python API utility methods
 
 Sinter's python module exposes a variety of methods that are handy for plotting
-or analyzing QEC data. These include:
-
-- `sinter.fit_binomial`: Fit a binomial hypothesis to data, including a likelihood range (the Bayesian equivalent of computing the standard deviation).
-- `sinter.fit_line_slope`: Predict slope by fitting sampled coordinates to a line.
-- `sinter.fit_line_y_at_x`: Predict Y values from X values by fitting sampled coordinates to a line
-- `sinter.comma_separated_key_values(path)`: A reasonable value to give for `--metadata_func` mapping `"folder/b=test,a=2.stim"` to `{'b': 'test', 'a': 2}`.
-- `sinter.predict_observables_bit_packed`: Runs a decoder on given detection event data, producing predicted observable flip data.
-- `sinter.predict_discards_bit_packed`: Converts detection event data into "what shots should be postselected" data according to given rules.
-- `sinter.predict_on_disk`: Converts detection event data into discard data and observable prediction data, using data from disk and writing results to disk.
-- `sinter.stats_from_csv_files`: Read saved CSV data.
-- `sinter.shot_error_rate_to_piece_error_rate`: Compute per-round error rates.
-- `sinter.better_sorted_str_terms`: A text sorting key that puts `"A100"` before `"A99"`, by noticing numbers instead of sorting purely lexicographically.
-- `sinter.group_by`: Combines items from a list into keyed groups.
+or analyzing QEC data.
+See the [sinter API reference](https://github.com/quantumlib/Stim/blob/main/doc/sinter_api.md).
 
 <a name="how_to_use_linux"></a>
 # How to use: Linux Command Line
@@ -210,9 +200,13 @@ But this is just an example, so we'll use normal surface code circuits.
 You can use sinter to collect statistics on each circuit by using the `sinter collect` command.
 This command takes options specifying how much data to collect, how to do decoding, etc.
 
+The `processes` argument decides how many workers to use. Set it to `auto` to set
+it to the number of CPUs on your machine.
+
 The `metadata_func` argument can be used to specify custom python expression that turns the `path`
 into a dictionary or other JSON object associated with the circuit.
-For convenience, sinter includes the method `sinter.comma_separated_key_values(path)` which parses
+If you set `metadata_func` to `auto` then will use the method
+`sinter.comma_separated_key_values(path)` which parses
 stim circuit paths like `folder/a=2,b=test.stim` into a dictionary like `{'a': 2, 'b': 'test'}`.
 
 By default, sinter writes the collected statistics to stdout as CSV data.
@@ -224,9 +218,9 @@ instead of overwriting it.
 
 ```bash
 sinter collect \
-    --processes 4 \
+    --processes auto \
     --circuits circuits/*.stim \
-    --metadata_func "sinter.comma_separated_key_values(path)" \
+    --metadata_func auto \
     --decoders pymatching \
     --max_shots 1_000_000 \
     --max_errors 1000 \
@@ -263,17 +257,23 @@ sinter combine stats.csv
 # plot
 
 You can use `sinter plot` to view the results you've collected.
-This command takes a CSV file, and also some command indicating how to group each case
-into single curves and also what the desired X coordinate of a case is.
-This is done in a flexible but very hacky way, by specifying a python expression using the case's filename: 
+This command takes a CSV file, an argument `--group_func` indicating how to
+group the statistics into curves, an argument `--x_func` indicating how to
+pick the X coordinate of each point, and various other arguments. Each `*_func`
+argument takes a string that will be evaluated as a python expression, with
+various useful values in scope such as a `metadata` value containing the
+json metadata for the various points being evaluated. There is also a special
+`m` value where `m.key` is shorthand for `metadata.get('key', None)`.
+
+Here is an example of a `sinter plot` command:
 
 ```bash
 sinter plot \
     --in stats.csv \
-    --group_func "'Rotated Surface Code d=' + str(metadata['d'])" \
-    --x_func "metadata['p']" \
-    --fig_size 1024 1024 \
+    --group_func "f'''Rotated Surface Code d={m.d}'''" \
+    --x_func m.p \
     --xaxis "[log]Physical Error Rate" \
+    --fig_size 1024 1024 \
     --out surface_code_figure.png \
     --show
 ```
@@ -285,7 +285,6 @@ Which will save a png image of, and also open a window showing, a plot like this
 <a name="csv_format"></a>
 # The csv format for sample statistics
 
-
 Sinter saves samples as a table using a Comma Separated Value format.
 For example:
 
@@ -321,8 +320,8 @@ shots that were from separate circuits or separate versions of a circuit.
 dictionary with helpful keys like "noise_level" or "circuit_name". The json
 value is serialized into JSON and then escaped so that it can be put into the
 CSV data (e.g. quotes get doubled up).
-- `custom_counts` (json[Dict[str, int]]): A field that can store a dictionary
-from string keys to integer counts represented in
+- `custom_counts` (json[Dict[str, int]]): An optional field that can store a
+dictionary from string keys to integer counts represented in
 [Java Script Object Notation](https://json.org).
 The counts can be a huge variety of things, ranging from per-observable error
 counts to detection event counts. In general, any value that should be added

{sinter-1.12.1 → sinter-1.13.0}/README.md

@@ -16,8 +16,8 @@ quantum error correction circuits.
 
 Sinter takes Stim circuits annotated with noise, detectors, and logical
 observables.
-It uses stim to sample the circuits and pymatching to predict whether the
-logical observables were flipped or not, given the detector data.
+It uses stim to sample the circuits and a decoder such as pymatching to predict
+whether the logical observables were flipped or not, given the detector data.
 It records how often this succeeds, and how often it fails (the error rate).
 
 Sinter uses python multiprocessing to do parallel sampling across multiple CPU
@@ -26,11 +26,12 @@ specified by the user (such as a target number of errors), saves the results to
 as simple CSV format, and has some basic  plotting functionality for viewing the
 results.
 
-Sinter doesn't support cloud compute, but it does scale well on
-a single machine.
+Sinter doesn't support cloud compute, but it does scale well on  a single
+machine.
 I've tested it on 2 core machines, 4 core machines, and 96 core machines.
-Although there are potential pitfalls (e.g. setting batch sizes too large causes thrashing),
-sinter generally achieves good resource utilization of the processes you assign to it.
+Although there are potential pitfalls (e.g. setting batch sizes too large causes
+thrashing), sinter generally achieves good resource utilization of the processes
+you assign to it.
 
 <a name="how_to_install"></a>
 # How to install
@@ -49,8 +50,8 @@ to use sinter's python API.
 <a name="how_to_use_python"></a>
 # How to use: Python API
 
-This example assumes you are in a python environment with sinter
-installed.
+This example assumes you are in a python environment with `sinter` and
+`pymatching` installed.
 
 ```python
 import stim
@@ -138,19 +139,8 @@ and the corresponding image saved to `plot.png`:
 ## python API utility methods
 
 Sinter's python module exposes a variety of methods that are handy for plotting
-or analyzing QEC data. These include:
-
-- `sinter.fit_binomial`: Fit a binomial hypothesis to data, including a likelihood range (the Bayesian equivalent of computing the standard deviation).
-- `sinter.fit_line_slope`: Predict slope by fitting sampled coordinates to a line.
-- `sinter.fit_line_y_at_x`: Predict Y values from X values by fitting sampled coordinates to a line
-- `sinter.comma_separated_key_values(path)`: A reasonable value to give for `--metadata_func` mapping `"folder/b=test,a=2.stim"` to `{'b': 'test', 'a': 2}`.
-- `sinter.predict_observables_bit_packed`: Runs a decoder on given detection event data, producing predicted observable flip data.
-- `sinter.predict_discards_bit_packed`: Converts detection event data into "what shots should be postselected" data according to given rules.
-- `sinter.predict_on_disk`: Converts detection event data into discard data and observable prediction data, using data from disk and writing results to disk.
-- `sinter.stats_from_csv_files`: Read saved CSV data.
-- `sinter.shot_error_rate_to_piece_error_rate`: Compute per-round error rates.
-- `sinter.better_sorted_str_terms`: A text sorting key that puts `"A100"` before `"A99"`, by noticing numbers instead of sorting purely lexicographically.
-- `sinter.group_by`: Combines items from a list into keyed groups.
+or analyzing QEC data.
+See the [sinter API reference](https://github.com/quantumlib/Stim/blob/main/doc/sinter_api.md).
 
 <a name="how_to_use_linux"></a>
 # How to use: Linux Command Line
@@ -196,9 +186,13 @@ But this is just an example, so we'll use normal surface code circuits.
 You can use sinter to collect statistics on each circuit by using the `sinter collect` command.
 This command takes options specifying how much data to collect, how to do decoding, etc.
 
+The `processes` argument decides how many workers to use. Set it to `auto` to set
+it to the number of CPUs on your machine.
+
 The `metadata_func` argument can be used to specify custom python expression that turns the `path`
 into a dictionary or other JSON object associated with the circuit.
-For convenience, sinter includes the method `sinter.comma_separated_key_values(path)` which parses
+If you set `metadata_func` to `auto` then will use the method
+`sinter.comma_separated_key_values(path)` which parses
 stim circuit paths like `folder/a=2,b=test.stim` into a dictionary like `{'a': 2, 'b': 'test'}`.
 
 By default, sinter writes the collected statistics to stdout as CSV data.
@@ -210,9 +204,9 @@ instead of overwriting it.
 
 ```bash
 sinter collect \
-    --processes 4 \
+    --processes auto \
     --circuits circuits/*.stim \
-    --metadata_func "sinter.comma_separated_key_values(path)" \
+    --metadata_func auto \
     --decoders pymatching \
     --max_shots 1_000_000 \
     --max_errors 1000 \
@@ -249,17 +243,23 @@ sinter combine stats.csv
 # plot
 
 You can use `sinter plot` to view the results you've collected.
-This command takes a CSV file, and also some command indicating how to group each case
-into single curves and also what the desired X coordinate of a case is.
-This is done in a flexible but very hacky way, by specifying a python expression using the case's filename: 
+This command takes a CSV file, an argument `--group_func` indicating how to
+group the statistics into curves, an argument `--x_func` indicating how to
+pick the X coordinate of each point, and various other arguments. Each `*_func`
+argument takes a string that will be evaluated as a python expression, with
+various useful values in scope such as a `metadata` value containing the
+json metadata for the various points being evaluated. There is also a special
+`m` value where `m.key` is shorthand for `metadata.get('key', None)`.
+
+Here is an example of a `sinter plot` command:
 
 ```bash
 sinter plot \
     --in stats.csv \
-    --group_func "'Rotated Surface Code d=' + str(metadata['d'])" \
-    --x_func "metadata['p']" \
-    --fig_size 1024 1024 \
+    --group_func "f'''Rotated Surface Code d={m.d}'''" \
+    --x_func m.p \
     --xaxis "[log]Physical Error Rate" \
+    --fig_size 1024 1024 \
     --out surface_code_figure.png \
     --show
 ```
@@ -271,7 +271,6 @@ Which will save a png image of, and also open a window showing, a plot like this
 <a name="csv_format"></a>
 # The csv format for sample statistics
 
-
 Sinter saves samples as a table using a Comma Separated Value format.
 For example:
 
@@ -307,8 +306,8 @@ shots that were from separate circuits or separate versions of a circuit.
 dictionary with helpful keys like "noise_level" or "circuit_name". The json
 value is serialized into JSON and then escaped so that it can be put into the
 CSV data (e.g. quotes get doubled up).
-- `custom_counts` (json[Dict[str, int]]): A field that can store a dictionary
-from string keys to integer counts represented in
+- `custom_counts` (json[Dict[str, int]]): An optional field that can store a
+dictionary from string keys to integer counts represented in
 [Java Script Object Notation](https://json.org).
 The counts can be a huge variety of things, ranging from per-observable error
 counts to detection event counts. In general, any value that should be added

{sinter-1.12.1 → sinter-1.13.0}/setup.py

@@ -19,7 +19,7 @@ with open('README.md', encoding='UTF-8') as f:
 with open('requirements.txt', encoding='UTF-8') as f:
     requirements = f.read().splitlines()
 
-__version__ = '1.12.1'
+__version__ = '1.13.0'
 
 setup(
     name='sinter',

{sinter-1.12.1 → sinter-1.13.0}/src/sinter/__init__.py

@@ -1,4 +1,4 @@
-__version__ = '1.12.1'
+__version__ = '1.13.0'
 
 from sinter._anon_task_stats import (
     AnonTaskStats,
@@ -15,6 +15,9 @@ from sinter._collection_options import (
 from sinter._csv_out import (
     CSV_HEADER,
 )
+from sinter._decoding_all_built_in_decoders import (
+    BUILT_IN_DECODERS,
+)
 from sinter._existing_data import (
     read_stats_from_csv_files,
     stats_from_csv_files,

{sinter-1.12.1 → sinter-1.13.0}/src/sinter/_decoding_fusion_blossom.py

@@ -1,19 +1,58 @@
 import math
 import pathlib
-from typing import Callable, List, TYPE_CHECKING
-from typing import Tuple
+from typing import Callable, List, TYPE_CHECKING, Tuple
 
 import numpy as np
 import stim
 
-from sinter._decoding_decoder_class import Decoder
+from sinter._decoding_decoder_class import Decoder, CompiledDecoder
 
 if TYPE_CHECKING:
     import fusion_blossom
 
 
+class FusionBlossomCompiledDecoder(CompiledDecoder):
+    def __init__(self, solver: 'fusion_blossom.SolverSerial', fault_masks: 'np.ndarray', num_dets: int, num_obs: int):
+        self.solver = solver
+        self.fault_masks = fault_masks
+        self.num_dets = num_dets
+        self.num_obs = num_obs
+
+    def decode_shots_bit_packed(
+            self,
+            *,
+            bit_packed_detection_event_data: 'np.ndarray',
+    ) -> 'np.ndarray':
+        num_shots = bit_packed_detection_event_data.shape[0]
+        predictions = np.zeros(shape=(num_shots, self.num_obs), dtype=np.uint8)
+        import fusion_blossom
+        for shot in range(num_shots):
+            dets_sparse = np.flatnonzero(np.unpackbits(bit_packed_detection_event_data[shot], count=self.num_dets, bitorder='little'))
+            syndrome = fusion_blossom.SyndromePattern(syndrome_vertices=dets_sparse)
+            self.solver.solve(syndrome)
+            prediction = int(np.bitwise_xor.reduce(self.fault_masks[self.solver.subgraph()]))
+            predictions[shot] = np.packbits(prediction, bitorder='little')
+            self.solver.clear()
+        return predictions
+
+
 class FusionBlossomDecoder(Decoder):
     """Use fusion blossom to predict observables from detection events."""
+
+    def compile_decoder_for_dem(self, *, dem: 'stim.DetectorErrorModel') -> CompiledDecoder:
+        try:
+            import fusion_blossom
+        except ImportError as ex:
+            raise ImportError(
+                "The decoder 'fusion_blossom' isn't installed\n"
+                "To fix this, install the python package 'fusion_blossom' into your environment.\n"
+                "For example, if you are using pip, run `pip install fusion_blossom`.\n"
+            ) from ex
+
+        solver, fault_masks = detector_error_model_to_fusion_blossom_solver_and_fault_masks(dem)
+        return FusionBlossomCompiledDecoder(solver, fault_masks, dem.num_detectors, dem.num_observables)
+
+
     def decode_via_files(self,
                          *,
                          num_shots: int,
@@ -36,7 +75,6 @@ class FusionBlossomDecoder(Decoder):
         error_model = stim.DetectorErrorModel.from_file(dem_path)
         solver, fault_masks = detector_error_model_to_fusion_blossom_solver_and_fault_masks(error_model)
         num_det_bytes = math.ceil(num_dets / 8)
-
         with open(dets_b8_in_path, 'rb') as dets_in_f:
             with open(obs_predictions_b8_out_path, 'wb') as obs_out_f:
                 for _ in range(num_shots):

{sinter-1.12.1 → sinter-1.13.0}/src/sinter/_main_plot.py

@@ -151,7 +151,7 @@ def parse_args(args: List[str]) -> Any:
                         )
     parser.add_argument('--plot_args_func',
                         type=str,
-                        default='''{'marker': 'ov*sp^<>8PhH+xXDd|'[index % 18]}''',
+                        default='''{'marker': 'ov*sp^<>8P+xXhHDd|'[index % 18]}''',
                         help='A python expression used to customize the look of curves.\n'
                              'Values available to the python expression:\n'
                              '    index: A unique integer identifying the curve.\n'
@@ -220,7 +220,11 @@ def parse_args(args: List[str]) -> Any:
     parser.add_argument('--ymin',
                         default=None,
                         type=float,
-                        help='Sets the minimum value of the y axis (max always 1).')
+                        help='Forces the minimum value of the y axis.')
+    parser.add_argument('--ymax',
+                        default=None,
+                        type=float,
+                        help='Forces the maximum value of the y axis.')
     parser.add_argument('--title',
                         default=None,
                         type=str,
@@ -474,6 +478,7 @@ def _plot_helper(
     xaxis: str,
     yaxis: Optional[str],
     min_y: Optional[float],
+    max_y: Optional[float],
     max_x: Optional[float],
     min_x: Optional[float],
     title: Optional[str],
@@ -536,7 +541,7 @@ def _plot_helper(
 
     x_scale_name: Optional[str] = None
     for ax in [ax_err, ax_dis, ax_cus]:
-        x_scale_name = x_scale_name or _set_axis_scale_label_ticks(
+        v = _set_axis_scale_label_ticks(
             ax=ax,
             y_not_x=False,
             axis_label=xaxis,
@@ -548,6 +553,7 @@ def _plot_helper(
             plotted_stats=plotted_stats,
             v_func=x_func,
         )
+        x_scale_name = x_scale_name or v
 
     y_scale_name: Optional[str] = None
     if ax_err is not None:
@@ -556,7 +562,7 @@ def _plot_helper(
             y_not_x=True,
             axis_label=f"Logical Error Rate (per {failure_unit})" if yaxis is None else yaxis,
             default_scale='log',
-            forced_max_v=1 if min_y is None or 1 > min_y else None,
+            forced_max_v=max_y if max_y is not None else 1 if min_y is None or 1 > min_y else None,
             default_min_v=1e-4,
             default_max_v=1,
             forced_min_v=min_y,
@@ -611,6 +617,8 @@ def _plot_helper(
             default_max_v=1,
             plotted_stats=plotted_stats,
             v_func=y_func,
+            forced_min_v=min_y,
+            forced_max_v=max_y,
         )
         plot_custom(
             ax=ax_cus,
@@ -753,6 +761,7 @@ def main_plot(*, command_line_args: List[str]):
         yaxis=args.yaxis,
         fig_size=args.fig_size,
         min_y=args.ymin,
+        max_y=args.ymax,
         max_x=args.xmax,
         min_x=args.xmin,
         highlight_max_likelihood_factor=args.highlight_max_likelihood_factor,

{sinter-1.12.1 → sinter-1.13.0}/src/sinter/_task.py

@@ -183,7 +183,7 @@ class Task:
             >>> task.strong_id_value()
             {'circuit': 'H 0', 'decoder': 'pymatching', 'decoder_error_model': '', 'postselection_mask': None, 'json_metadata': None}
         """
-        if self.decoder is None:
+        if self.circuit is None:
             raise ValueError("Can't compute strong_id until `circuit` is set.")
         if self.decoder is None:
             raise ValueError("Can't compute strong_id until `decoder` is set.")

{sinter-1.12.1 → sinter-1.13.0}/src/sinter.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: sinter
-Version: 1.12.1
+Version: 1.13.0
 Summary: Samples stim circuits and decodes them using pymatching.
 Author: Craig Gidney
 Author-email: craig.gidney@gmail.com
@@ -30,8 +30,8 @@ quantum error correction circuits.
 
 Sinter takes Stim circuits annotated with noise, detectors, and logical
 observables.
-It uses stim to sample the circuits and pymatching to predict whether the
-logical observables were flipped or not, given the detector data.
+It uses stim to sample the circuits and a decoder such as pymatching to predict
+whether the logical observables were flipped or not, given the detector data.
 It records how often this succeeds, and how often it fails (the error rate).
 
 Sinter uses python multiprocessing to do parallel sampling across multiple CPU
@@ -40,11 +40,12 @@ specified by the user (such as a target number of errors), saves the results to
 as simple CSV format, and has some basic  plotting functionality for viewing the
 results.
 
-Sinter doesn't support cloud compute, but it does scale well on
-a single machine.
+Sinter doesn't support cloud compute, but it does scale well on  a single
+machine.
 I've tested it on 2 core machines, 4 core machines, and 96 core machines.
-Although there are potential pitfalls (e.g. setting batch sizes too large causes thrashing),
-sinter generally achieves good resource utilization of the processes you assign to it.
+Although there are potential pitfalls (e.g. setting batch sizes too large causes
+thrashing), sinter generally achieves good resource utilization of the processes
+you assign to it.
 
 <a name="how_to_install"></a>
 # How to install
@@ -63,8 +64,8 @@ to use sinter's python API.
 <a name="how_to_use_python"></a>
 # How to use: Python API
 
-This example assumes you are in a python environment with sinter
-installed.
+This example assumes you are in a python environment with `sinter` and
+`pymatching` installed.
 
 ```python
 import stim
@@ -152,19 +153,8 @@ and the corresponding image saved to `plot.png`:
 ## python API utility methods
 
 Sinter's python module exposes a variety of methods that are handy for plotting
-or analyzing QEC data. These include:
-
-- `sinter.fit_binomial`: Fit a binomial hypothesis to data, including a likelihood range (the Bayesian equivalent of computing the standard deviation).
-- `sinter.fit_line_slope`: Predict slope by fitting sampled coordinates to a line.
-- `sinter.fit_line_y_at_x`: Predict Y values from X values by fitting sampled coordinates to a line
-- `sinter.comma_separated_key_values(path)`: A reasonable value to give for `--metadata_func` mapping `"folder/b=test,a=2.stim"` to `{'b': 'test', 'a': 2}`.
-- `sinter.predict_observables_bit_packed`: Runs a decoder on given detection event data, producing predicted observable flip data.
-- `sinter.predict_discards_bit_packed`: Converts detection event data into "what shots should be postselected" data according to given rules.
-- `sinter.predict_on_disk`: Converts detection event data into discard data and observable prediction data, using data from disk and writing results to disk.
-- `sinter.stats_from_csv_files`: Read saved CSV data.
-- `sinter.shot_error_rate_to_piece_error_rate`: Compute per-round error rates.
-- `sinter.better_sorted_str_terms`: A text sorting key that puts `"A100"` before `"A99"`, by noticing numbers instead of sorting purely lexicographically.
-- `sinter.group_by`: Combines items from a list into keyed groups.
+or analyzing QEC data.
+See the [sinter API reference](https://github.com/quantumlib/Stim/blob/main/doc/sinter_api.md).
 
 <a name="how_to_use_linux"></a>
 # How to use: Linux Command Line
@@ -210,9 +200,13 @@ But this is just an example, so we'll use normal surface code circuits.
 You can use sinter to collect statistics on each circuit by using the `sinter collect` command.
 This command takes options specifying how much data to collect, how to do decoding, etc.
 
+The `processes` argument decides how many workers to use. Set it to `auto` to set
+it to the number of CPUs on your machine.
+
 The `metadata_func` argument can be used to specify custom python expression that turns the `path`
 into a dictionary or other JSON object associated with the circuit.
-For convenience, sinter includes the method `sinter.comma_separated_key_values(path)` which parses
+If you set `metadata_func` to `auto` then will use the method
+`sinter.comma_separated_key_values(path)` which parses
 stim circuit paths like `folder/a=2,b=test.stim` into a dictionary like `{'a': 2, 'b': 'test'}`.
 
 By default, sinter writes the collected statistics to stdout as CSV data.
@@ -224,9 +218,9 @@ instead of overwriting it.
 
 ```bash
 sinter collect \
-    --processes 4 \
+    --processes auto \
     --circuits circuits/*.stim \
-    --metadata_func "sinter.comma_separated_key_values(path)" \
+    --metadata_func auto \
     --decoders pymatching \
     --max_shots 1_000_000 \
     --max_errors 1000 \
@@ -263,17 +257,23 @@ sinter combine stats.csv
 # plot
 
 You can use `sinter plot` to view the results you've collected.
-This command takes a CSV file, and also some command indicating how to group each case
-into single curves and also what the desired X coordinate of a case is.
-This is done in a flexible but very hacky way, by specifying a python expression using the case's filename: 
+This command takes a CSV file, an argument `--group_func` indicating how to
+group the statistics into curves, an argument `--x_func` indicating how to
+pick the X coordinate of each point, and various other arguments. Each `*_func`
+argument takes a string that will be evaluated as a python expression, with
+various useful values in scope such as a `metadata` value containing the
+json metadata for the various points being evaluated. There is also a special
+`m` value where `m.key` is shorthand for `metadata.get('key', None)`.
+
+Here is an example of a `sinter plot` command:
 
 ```bash
 sinter plot \
     --in stats.csv \
-    --group_func "'Rotated Surface Code d=' + str(metadata['d'])" \
-    --x_func "metadata['p']" \
-    --fig_size 1024 1024 \
+    --group_func "f'''Rotated Surface Code d={m.d}'''" \
+    --x_func m.p \
     --xaxis "[log]Physical Error Rate" \
+    --fig_size 1024 1024 \
     --out surface_code_figure.png \
     --show
 ```
@@ -285,7 +285,6 @@ Which will save a png image of, and also open a window showing, a plot like this
 <a name="csv_format"></a>
 # The csv format for sample statistics
 
-
 Sinter saves samples as a table using a Comma Separated Value format.
 For example:
 
@@ -321,8 +320,8 @@ shots that were from separate circuits or separate versions of a circuit.
 dictionary with helpful keys like "noise_level" or "circuit_name". The json
 value is serialized into JSON and then escaped so that it can be put into the
 CSV data (e.g. quotes get doubled up).
-- `custom_counts` (json[Dict[str, int]]): A field that can store a dictionary
-from string keys to integer counts represented in
+- `custom_counts` (json[Dict[str, int]]): An optional field that can store a
+dictionary from string keys to integer counts represented in
 [Java Script Object Notation](https://json.org).
 The counts can be a huge variety of things, ranging from per-observable error
 counts to detection event counts. In general, any value that should be added