attr-eomt 1.1.3__tar.gz → 1.2.0__tar.gz

{attr_eomt-1.1.3 → attr_eomt-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: attr-eomt
-Version: 1.1.3
+Version: 1.2.0
 Summary: Standalone EoMT (Encoder-only Mask Transformer) for instance segmentation, with DINOv2 init, COCO training/validation and inference.
 Project-URL: Homepage, https://github.com/imagra93/attr-eomt
 Project-URL: Documentation, https://imagra93.github.io/attr-eomt
@@ -72,7 +72,7 @@ It's a clean-room, Apache-2.0 reimplementation: the weights you train are yours
 from eomt import EoMT
 
 model = EoMT("l")                            # fresh large model (DINOv2 backbone)
-model.train(data="coco", epochs=50)          # COCO 2017 auto-downloads if missing
+model.train(data="coco")                     # COCO 2017 auto-downloads if missing; 12 epochs, the official recipe
 
 model = EoMT("runs/train/eomt-l")            # reload a run — size/classes/heads auto-detected
 model.predict("images/", plot=True)          # render masks/boxes + per-instance attributes
@@ -206,14 +206,22 @@ simply reuse that same query→GT assignment and read the answer off the matched
   query→GT assignment the detection loss used; the attribute is read *after* matching.
 - **Gate.** An optional IoU gate drops barely-overlapping matched pairs (common early in
   training) so attributes only learn from queries that actually localize the object.
-- **Loss.** Cross-entropy per head over matched queries, summed across heads and scaled
+- **Loss.** Cross-entropy per head over matched queries (a regression head: Huber on its
+  standardized value, see below), summed across heads and scaled
   by `aux_w` (default `1.0`), added to the detector loss. Empty-match batches contribute
   a graph-preserving zero, and missing labels use `ignore_index` and contribute nothing.
+- **Balancing.** Both losses are O(1), so equal weights are a sane start. `aux_w_per_head`
+  weights each head, `aux_class_weights=True` re-weights rare classes (rare values of a
+  regression head, by smoothed density), and `aux_balance="uncertainty"` learns one weight
+  per head (Kendall et al. 2018) with the detector loss as the fixed anchor. Each epoch
+  prints every head's pull on the shared queries (its gradient norm over the detector's,
+  `train/aux_grad/<head>`): well above 1 a head steers the encoder, far below it only reads it.
 - **Checkpoint selection is unchanged.** The attribute "rides along": its per-head
   matched-query accuracy is shown live and written to `metrics.csv`, but never drives
   `best.pt` (still `segm/mAP` or `bbox/mAP`).
 - **Inference.** Each result attaches `aux = {head: {"ids", "probs"}}` for the kept
-  detections, and `predict(plot=True)` renders each attribute next to the class label
+  detections (and `aux_values = {head: values}` in real units for the regression heads), and
+  `predict(plot=True)` renders each attribute next to the class label
   using names stored in the checkpoint.
 
 ---
@@ -282,8 +290,76 @@ label set, in one model:
 At inference a scoped head reports `ids = -1` ("not applicable") for detections whose class it
 does not cover. The scope is stored in the checkpoint, so it survives reload.
 
+**Flip-aware attributes.** A horizontal flip mirrors the photo but not the labels, so an attribute
+that encodes a side (a left door shows up as a right one) is corrupted by `flip_prob > 0`. Give
+its definition `flip_aware`, a map of category names to the ones they become on a horizontal
+flip, and the training augmentation swaps them for every instance it flips (each pair swaps both
+ways, so `{"left": "right"}` is enough; unlisted categories stay as they are):
+
+```jsonc
+{"name": "laterality", "categories": [{"id": 0, "name": "left"}, {"id": 1, "name": "center"}, {"id": 2, "name": "right"}],
+ "flip_aware": {"left": "right"}}
+```
+
+Mosaic, mixup and copy-paste flip each source image independently, and the swap follows each
+instance. Only the horizontal flip is handled: keep `vflip_prob` and `rot90_prob` off for such an
+attribute. It is categorical only. The mapping is stored in the checkpoint, and test-time augmentation with
+`flip` swaps it back in each mirrored view's prediction before the views are merged (a checkpoint saved before the
+mapping was stored takes it from the val JSON, with a warning). The training banner lists each head's swapped pairs. A
+regression head needs no mapping: its value must not change under a flip (see below).
+
+**Regression attributes.** `"type": "regression"` makes a head regress one real number per
+instance instead of choosing a category; categorical and regression heads mix freely in one
+dataset. `log` (regress `log1p` of the value, for heavy-tailed positives such as costs) and
+`range` (valid values; predictions are clipped to it) are optional, and `applies_to` works as above:
+
+```jsonc
+"attributes": [
+  {"name": "severity", "categories": [...]},
+  {"name": "repair_cost", "type": "regression", "log": true, "range": [0, 20000], "applies_to": ["bumper"]}
+]
+// per annotation:
+"attributes": {"severity": 3, "repair_cost": 812.5}
+```
+
+A missing, non-numeric or out-of-range value is ignored. The head is trained on the value
+standardized by the train split's mean and std (stored in the checkpoint), validated as the
+mean absolute error in real units (`val/aux_mae/<head>`; `metrics.png` gives each regression
+head its own MAE panel, `aux_per_class.png` its MAE per primary class), and predicted in real units
+(`aux_values`). The value must not change under the augmentations (scale, crop, flip,
+rotation): a cost or a score is fine, a pixel area or an angle is not. Tracking votes on the
+categorical heads only, and re-identification cannot gate on a regression head.
+
+A value that depends on how much of the instance is in view can say so with `crop`: the
+training augmentation measures, per instance, the fraction it keeps in view (mask mass, or box
+area, inside the crop window, after copy-paste occlusion), and the value follows it.
+`"crop": "scale"` multiplies the value by that fraction — for an extensive quantity such as the
+object's area in cm² (half the damage cropped away ⇒ half the area). `"crop": "ignore"` drops the
+value when less than `min_visible` (default `0.9`) is kept — for a property of the whole object
+that a cut view no longer shows. Validation and inference see whole images, so the head predicts
+the full value there:
+
+```jsonc
+{"name": "damage_area", "type": "regression", "log": true, "crop": "scale"},
+{"name": "part_length", "type": "regression", "crop": "ignore", "min_visible": 0.95}
+```
+
+How the fraction is measured: for masks, the share of the instance's mask mass whose source pixels
+the crop (and rotation / shear / perspective warp) lands inside the window: exact, so an instance
+the window does not cut keeps exactly 1 (a warp alone is not a cut); copy-paste occlusion counts as
+hidden, and instances brought in by mosaic, mixup or copy-paste are measured against their own crops. For boxes (detect family), the ratio of
+the box areas: exact for a crop, approximate under rotation. Each call of the built-in
+augmentation exposes it as `TrainAugment.last_info["visible"]`.
+
+Limits: the head's mean / std and its `aux_class_weights` density come from the unscaled values,
+so with `"scale"` the values trained on sit somewhat lower than those statistics. Only the visible
+fraction is tracked, not the scale jitter, so a pixel area is still not supported. A custom
+`transform=` gets no adjustment. `crop` is rejected on a categorical attribute, and it is a
+training-only setting (not stored in the checkpoint). The training banner prints each head's crop
+mode.
+
 **Sidecar format (optional).** You can keep the COCO JSON as plain, standard COCO and put the
-attributes beside it instead of inside it: an `attributes.yaml` schema in the dataset root plus
+attributes beside it instead of inside it: an `attributes.yaml` schema (same keys, regression heads included) in the dataset root plus
 `attributes/<split>.json` values keyed by annotation id (`{ann_id: {head: value}}`). If present
 (and the JSON has no embedded `attributes`), it is merged in memory at load — so a plain COCO
 dataset always works and the sidecar is picked up automatically when you add it. Embedded
@@ -406,7 +482,7 @@ size, normalization and any auxiliary heads are auto-detected from the `.pt`):
 from eomt import EoMT
 
 # Train on COCO 2017 (auto-downloaded on first run):
-EoMT("l").train(data="coco", epochs=50, batch=4)
+EoMT("l").train(data="coco", batch=4)   # 12 epochs, validated every 2: the official recipe
 
 # ...or any COCO-format dataset (point at its data.yaml):
 EoMT("s").train(data="sample_data/data.yaml", epochs=1, batch=1)
@@ -424,7 +500,7 @@ GPUs; the main process scores them and alone logs and writes checkpoints. The pr
 argument must be picklable.
 
 ```python
-EoMT("l", device="0,1").train(data="coco", epochs=50, batch=2)   # 2 per GPU x accum 4 x 2 GPUs = effective 16
+EoMT("l", device="0,1").train(data="coco", batch=2)   # 2 per GPU x accum 4 x 2 GPUs = effective 16
 ```
 
 **Test-time augmentation and tiled inference** (instance family). `predict()` and `val()` take `augment=True` (the
@@ -433,8 +509,9 @@ image at `imgsz` and zoomed to 1.5x `imgsz`) and `tiles=True` (overlapping tiles
 mask IoU >= 0.5 measured where both views can see, so an object cut by a tile border is put back together; the mask
 logits are averaged and the score is the mean over the views that could see the object (one that missed it votes 0).
 Each takes a dict of overrides; see
-[`eomt/tta.py`](eomt/tta.py). `augment={"flip": True}` also merges mirrored views: never with left / right classes or
-attributes, which a mirrored view predicts swapped.
+[`eomt/tta.py`](eomt/tta.py). `augment={"flip": True}` also merges mirrored views. Attributes marked `flip_aware` are swapped
+back in a mirrored view; left / right *primary classes* are not (a mirrored view predicts them swapped), so keep `flip` off
+for those, and for attributes that encode a side without being flip-aware.
 
 ```python
 m = EoMT("runs/train/eomt-l")
@@ -443,7 +520,8 @@ m.val(data="data.yaml", tiles=True)                                 # native til
 m.predict("images/", augment={"flip": True}, tiles=1024)            # 1x / 1.5x and mirrored; 1024 px tiles
 ```
 
-`scripts/val.py` and `scripts/predict.py` take `--augment` and `--tiles [SIZE]`. Each view is a forward pass: `augment`
+`scripts/val.py` and `scripts/predict.py` take `--augment` and `--tiles [SIZE]` (`scripts/val.py` also `--flip`; its arguments are
+in [Command-line scripts](#command-line-scripts-arguments-and-things-to-take-into-account)). Each view is a forward pass: `augment`
 costs ~3.5x one prediction (the 1.5x view has 2.25x the pixels); a 1280 x 960 photo at `imgsz` 644 makes 6 tiles + the
 whole image.
 
@@ -462,11 +540,89 @@ LR, every step of the LR schedule, and the masked-attention annealing agree exac
 | encoder LR | layer-wise decay `llrd=0.8`, `backbone_lr_mult=1.0`: the **top ViT block is at the full `lr0`** and each block below is 0.8× the one above; embeddings follow the first block, the final norm is at `lr0` |
 | schedule | polynomial decay (`poly_power=0.9`) to 0 after a two-stage warmup in optimizer steps `warmup_steps=(500, 1000)`: the head warms up while the ViT LR is held at 0, then the ViT ramps up |
 | masked attention | annealed block by block (`mask_anneal_start/end`, fractions of training: 1/6–2/6, 2/6–3/6, 3/6–4/6, 4/6–5/6, polynomial); the last sixth of training is mask-free |
+| length | `epochs=12`, validated every `val_interval=2` epochs and after the last (official COCO instance recipe, about 89k optimizer steps at batch 16) |
 
 In EoMT the last four transformer blocks are the ones that process the queries and predict the masks, so they must learn at
 close to the head's speed: scaling the whole encoder by 0.1 (as detector-style recipes do for a backbone) starves them. The
-LR decay and the annealing span `epochs`, and validation runs mask-free, so metrics logged early in a long run understate
-the model. The official COCO recipe trains for 12 epochs (about 89k optimizer steps at batch 16).
+LR decay and the annealing span `epochs`.
+
+**Validation follows the training regime**, as in the official EoMT: masked attention at the annealing probabilities of
+the epoch's end. A model that still masks is far worse without its masks (part damage, ViT-L at epoch 11 of 50: 0.034
+segm mAP mask-free against 0.114 masked), so a mask-free curve before the last sixth follows the annealing and jumps as each
+block finishes it, not the model. `best.pt` is deployed mask-free, so it is only chosen once every block's annealing is
+over (a run stopped earlier, e.g. with `stop_after_epochs`, has `last.pt` alone), and every checkpoint is saved mask-free.
+Training prints the windows in epochs when it starts.
+
+### Command-line scripts: arguments and things to take into account
+
+**`scripts/train.py`.** Every flag is a keyword of `train()`; a flag left out keeps `train()`'s default.
+
+| flag | `train()` keyword | what it does |
+|---|---|---|
+| `--data` | `data` | dataset YAML, or `coco` (auto-downloads) |
+| `--size` / `--task` | | model size `s` `b` `l`; head family `instance` (masks) or `detect` (boxes) |
+| `--epochs` | `epochs` | total epochs: the LR schedule and the masked-attention annealing span them |
+| `--batch` / `--nominal-batch` | `batch` / `nominal_batch` | micro-batch **per GPU** / global effective batch (16); the difference is gradient accumulation |
+| `--device` | `device` | `auto`, `cpu`, `cuda:N`, or `0,1` for several GPUs (one process each) |
+| `--weights` / `--resume` | `init_weights` / `resume` | warm-start a fresh run from a checkpoint / continue a run. **One of the two, not both** |
+| `--lr`, `--warmup HEAD,VIT` | `lr0`, `warmup_steps` | peak head LR (the encoder follows with layer-wise decay) and the two-stage warmup in optimizer steps |
+| `--ema-decay`, `--ema-tau` | `ema_decay`, `ema_tau` | EMA of the weights (what is validated and exported as `best.pt`) |
+| `--mask-free` | `mask_anneal_start/end` | no masked attention from step 0 (annealing windows 0 -> 0) |
+| `--aux-dropout` | `aux_head_dropout` | dropout in the attribute heads |
+| `--val-interval`, `--workers`, `--seed` | same | validate every N epochs (and after the last); data-loader workers per GPU; seed |
+| `--aug KEY=VALUE`, `--aug-preset` | `aug` | augmentation overrides (see [Augmentation](#augmentation)) |
+| `--fpn-scales`, `--box-head`, `--no-deep-supervision`, `--upscale-blocks`, `--iou-aware-cls`, `--quality-weight`, `--val-imgsz` | same | [architecture options](#architecture-options) |
+
+Things to take into account when training:
+
+- **Resume or warm start.** `--resume` continues a run exactly (epoch, optimizer, schedule, EMA, best metric) and re-reads the
+  architecture from the checkpoint: keep the other flags as they were, and the seed is ignored so the shuffle is not replayed.
+  `--weights` loads the weights only and starts a new run at epoch 0. Start from a `best.pt` (the EMA, saved mask-free); a
+  `last.pt` holds the live weights.
+- **A short fine-tune needs its own recipe.** Lower `--lr`, shorten `--warmup`, and lower `--ema-decay` (at 0.9999 the EMA
+  averages over ~10k steps and barely leaves its starting point in a few epochs). If the model is deployed mask-free, pass
+  `--mask-free`: with the default annealing a warm-started model is trained *masked* for the first sixth of the run, a regime it
+  no longer runs in.
+- **`best.pt` is only chosen mask-free.** With the default annealing that means from 5/6 of the schedule on; with `--mask-free`
+  every validation is eligible.
+- **Validated epochs only.** With `--val-interval N` the other epochs have empty validation cells in `metrics.csv`; the curves in
+  `metrics.png` join the validated points.
+- **Attribute heads follow the dataset's definitions.** Categories, `applies_to`, `flip_aware` and `crop` / zoom settings come
+  from the dataset (the training banner lists them). The flip mapping, scope and regression statistics are stored in the
+  checkpoint; `crop` and the zoom settings are training-only.
+
+**`scripts/val.py`.**
+
+| flag | what it does |
+|---|---|
+| `weights` | checkpoint `.pt` or a run / weights folder (`best.pt` preferred) |
+| `--data`, `--batch`, `--workers`, `--device` | dataset YAML (`coco`), batch, data-loader workers, device |
+| `--imgsz` | evaluation input size (default: the trained one; the positional embeddings are interpolated) |
+| `--augment` | add the image at 1.5x `imgsz` |
+| `--tiles [SIZE]` | overlapping tiles (default `imgsz` px, native resolution) plus the whole image |
+| `--flip` | also run every view mirrored (alone: `--augment` with mirrored views) |
+| `--no-attributes` | mAP only |
+| `--ema`, `--mask annealed\|free` | validate the EMA weights of a `last.pt` (see below) |
+
+Things to take into account when validating:
+
+- **What is reported.** `segm/*` and `bbox/*` mAP and, for a model with secondary heads, `aux_acc/<head>` (accuracy) and
+  `aux_mae/<head>` (regression heads, mean absolute error in real units). They are scored on the same predictions as the mAP, so
+  they also work with `--augment`, `--tiles` and `--flip`.
+- **The attribute population is the deployed one.** Each GT instance is matched to the kept detection of its class with the
+  highest mask IoU (one to one, IoU >= 0.5). The trainer's `val/aux_*` columns score matched *queries* instead, so the two differ
+  slightly: compare TTA against plain within the script's numbers, not against `metrics.csv`. A regression head's MAE is
+  dominated by its large values, so also look at a relative error before concluding.
+- **Regime.** A model that is still annealing is only meaningful with its masks: `--ema --mask annealed` reproduces the trainer's
+  validation of a `last.pt` (EMA weights, the masked-attention probabilities of that epoch, read from the run's `args.yaml`),
+  `--ema --mask free` is the deployed regime, and `best.pt` is already the EMA, mask-free.
+- **TTA is not free and not uniform.** It costs one forward pass per view and mostly helps small objects and strict-IoU mAP; the
+  classification heads often do not move. A regression head's prediction is *averaged over views at different zoom*, so a value
+  that depends on the object's apparent size can drift under TTA: check it against the plain run before relying on it.
+- **`--flip` and the heads.** Heads the checkpoint stores as flip-aware are swapped back in mirrored views; an older checkpoint
+  takes the mapping from the dataset's JSON, with a warning. Left / right *primary classes* are not handled: keep `--flip` off for
+  them. The val categories of every head must match the model's, or the run stops instead of scoring against the wrong ids.
+- **One process.** `scripts/val.py` runs on one device; the trainer's own validation uses all the GPUs.
 
 ### Architecture options
 
@@ -525,7 +681,7 @@ mass-preserving), so thin objects are not shredded into dots by nearest-neighbou
 | geometry | `flip_prob` 0.5 · `vflip_prob` 0 · `rot90_prob` 0 · LSJ `min_scale`/`max_scale` 0.1–2.0 · `rotate_prob` 0.3 (±10°) · `shear_prob` 0.2 (±5°) · `perspective_prob` 0.15 |
 | optics / sensor | colour jitter 1.0 · `gamma_prob` 0.3 · `grayscale_prob` 0.05 · `blur_prob` 0.2 · `noise_prob` 0.2 · `jpeg_prob` 0.3 · `glare_prob` 0.15 · `erasing_prob` 0.2 |
 | multi-image (instance family) | `mosaic_prob` 0 · `mixup_prob` 0 · `copy_paste_prob` 0 |
-| crop | `instance_crop_prob` 0 (instance-aware crop) · `instance_crop_empty_prob` 0 |
+| crop | `instance_crop_prob` 0 (instance-aware crop) · `instance_crop_empty_prob` 0 · `zoom_prob` 0 (close-ups, `zoom_scale` 4–5x) |
 | masks | `mask_resize="area"` (or `"nearest"`) |
 
 Override per run, per dataset, or from the CLI — precedence is explicit keywords (`flip_prob`, `min_scale`,
@@ -553,12 +709,19 @@ python scripts/train.py --data my/data.yaml --aug min_scale=0.6 --aug instance_c
 Things worth knowing:
 
 - **Orientation.** Flips, 90° turns and rotations change the pixels but not the label. If an attribute encodes
-  orientation (e.g. a `viewpoint` head) set `flip_prob=0` and `rotate_prob=0`; `vflip_prob` / `rot90_prob` stay off
-  unless your scenes have no canonical "up".
+  orientation (e.g. a `viewpoint` head) set `flip_prob=0` and `rotate_prob=0`, or, for a left / right attribute,
+  mark it `flip_aware` (see *Flip-aware attributes*) and keep the horizontal flip; `vflip_prob` / `rot90_prob` stay
+  off unless your scenes have no canonical "up".
 - **Thin or tiny objects.** Large-Scale Jitter at 0.1 shrinks a 5 px crack to under a pixel; keep the effective
   scale within ~[0.4, 1.0] (`min_scale = 0.4·long_side/imgsz`, `max_scale = 1.0·long_side/imgsz`). The instance-aware
   crop pays off when the crop is much smaller than the image (high zoom, small `imgsz`), and
   `instance_crop_rarity_attr` makes rare attribute values the crop's focus more often.
+- **Close-ups.** `zoom_prob` makes an occasional sample a close-up: scaled by `s ~ U(zoom_scale)` (default 4–5x
+  the whole-image view, past the photo's own resolution) instead of the LSJ range, with the window on one instance
+  (whole when it fits, else a window inside it). `zoom_attr` restricts it to instances with a value of that
+  attribute and to the samples that have one, e.g. `{"zoom_prob": 0.1, "zoom_attr": "area"}` so a crop-scaled area
+  head also learns that a part filling the frame keeps its real area, and that a cut part keeps the share in view.
+  It resizes the whole photo first (~1.5x the time of a normal sample at 5x), so keep it occasional.
 - **Multi-image ops** (`mosaic`, `mixup`, `copy_paste`) add instances from other samples and are *off*: they splice or
   blend long thin structures, so try them deliberately and look at the result.
 - **Negatives.** Images with no annotations are dropped from the train set by default. `train(keep_empty=True)` keeps

{attr_eomt-1.1.3 → attr_eomt-1.2.0}/README.md

@@ -32,7 +32,7 @@ It's a clean-room, Apache-2.0 reimplementation: the weights you train are yours
 from eomt import EoMT
 
 model = EoMT("l")                            # fresh large model (DINOv2 backbone)
-model.train(data="coco", epochs=50)          # COCO 2017 auto-downloads if missing
+model.train(data="coco")                     # COCO 2017 auto-downloads if missing; 12 epochs, the official recipe
 
 model = EoMT("runs/train/eomt-l")            # reload a run — size/classes/heads auto-detected
 model.predict("images/", plot=True)          # render masks/boxes + per-instance attributes
@@ -166,14 +166,22 @@ simply reuse that same query→GT assignment and read the answer off the matched
   query→GT assignment the detection loss used; the attribute is read *after* matching.
 - **Gate.** An optional IoU gate drops barely-overlapping matched pairs (common early in
   training) so attributes only learn from queries that actually localize the object.
-- **Loss.** Cross-entropy per head over matched queries, summed across heads and scaled
+- **Loss.** Cross-entropy per head over matched queries (a regression head: Huber on its
+  standardized value, see below), summed across heads and scaled
   by `aux_w` (default `1.0`), added to the detector loss. Empty-match batches contribute
   a graph-preserving zero, and missing labels use `ignore_index` and contribute nothing.
+- **Balancing.** Both losses are O(1), so equal weights are a sane start. `aux_w_per_head`
+  weights each head, `aux_class_weights=True` re-weights rare classes (rare values of a
+  regression head, by smoothed density), and `aux_balance="uncertainty"` learns one weight
+  per head (Kendall et al. 2018) with the detector loss as the fixed anchor. Each epoch
+  prints every head's pull on the shared queries (its gradient norm over the detector's,
+  `train/aux_grad/<head>`): well above 1 a head steers the encoder, far below it only reads it.
 - **Checkpoint selection is unchanged.** The attribute "rides along": its per-head
   matched-query accuracy is shown live and written to `metrics.csv`, but never drives
   `best.pt` (still `segm/mAP` or `bbox/mAP`).
 - **Inference.** Each result attaches `aux = {head: {"ids", "probs"}}` for the kept
-  detections, and `predict(plot=True)` renders each attribute next to the class label
+  detections (and `aux_values = {head: values}` in real units for the regression heads), and
+  `predict(plot=True)` renders each attribute next to the class label
   using names stored in the checkpoint.
 
 ---
@@ -242,8 +250,76 @@ label set, in one model:
 At inference a scoped head reports `ids = -1` ("not applicable") for detections whose class it
 does not cover. The scope is stored in the checkpoint, so it survives reload.
 
+**Flip-aware attributes.** A horizontal flip mirrors the photo but not the labels, so an attribute
+that encodes a side (a left door shows up as a right one) is corrupted by `flip_prob > 0`. Give
+its definition `flip_aware`, a map of category names to the ones they become on a horizontal
+flip, and the training augmentation swaps them for every instance it flips (each pair swaps both
+ways, so `{"left": "right"}` is enough; unlisted categories stay as they are):
+
+```jsonc
+{"name": "laterality", "categories": [{"id": 0, "name": "left"}, {"id": 1, "name": "center"}, {"id": 2, "name": "right"}],
+ "flip_aware": {"left": "right"}}
+```
+
+Mosaic, mixup and copy-paste flip each source image independently, and the swap follows each
+instance. Only the horizontal flip is handled: keep `vflip_prob` and `rot90_prob` off for such an
+attribute. It is categorical only. The mapping is stored in the checkpoint, and test-time augmentation with
+`flip` swaps it back in each mirrored view's prediction before the views are merged (a checkpoint saved before the
+mapping was stored takes it from the val JSON, with a warning). The training banner lists each head's swapped pairs. A
+regression head needs no mapping: its value must not change under a flip (see below).
+
+**Regression attributes.** `"type": "regression"` makes a head regress one real number per
+instance instead of choosing a category; categorical and regression heads mix freely in one
+dataset. `log` (regress `log1p` of the value, for heavy-tailed positives such as costs) and
+`range` (valid values; predictions are clipped to it) are optional, and `applies_to` works as above:
+
+```jsonc
+"attributes": [
+  {"name": "severity", "categories": [...]},
+  {"name": "repair_cost", "type": "regression", "log": true, "range": [0, 20000], "applies_to": ["bumper"]}
+]
+// per annotation:
+"attributes": {"severity": 3, "repair_cost": 812.5}
+```
+
+A missing, non-numeric or out-of-range value is ignored. The head is trained on the value
+standardized by the train split's mean and std (stored in the checkpoint), validated as the
+mean absolute error in real units (`val/aux_mae/<head>`; `metrics.png` gives each regression
+head its own MAE panel, `aux_per_class.png` its MAE per primary class), and predicted in real units
+(`aux_values`). The value must not change under the augmentations (scale, crop, flip,
+rotation): a cost or a score is fine, a pixel area or an angle is not. Tracking votes on the
+categorical heads only, and re-identification cannot gate on a regression head.
+
+A value that depends on how much of the instance is in view can say so with `crop`: the
+training augmentation measures, per instance, the fraction it keeps in view (mask mass, or box
+area, inside the crop window, after copy-paste occlusion), and the value follows it.
+`"crop": "scale"` multiplies the value by that fraction — for an extensive quantity such as the
+object's area in cm² (half the damage cropped away ⇒ half the area). `"crop": "ignore"` drops the
+value when less than `min_visible` (default `0.9`) is kept — for a property of the whole object
+that a cut view no longer shows. Validation and inference see whole images, so the head predicts
+the full value there:
+
+```jsonc
+{"name": "damage_area", "type": "regression", "log": true, "crop": "scale"},
+{"name": "part_length", "type": "regression", "crop": "ignore", "min_visible": 0.95}
+```
+
+How the fraction is measured: for masks, the share of the instance's mask mass whose source pixels
+the crop (and rotation / shear / perspective warp) lands inside the window: exact, so an instance
+the window does not cut keeps exactly 1 (a warp alone is not a cut); copy-paste occlusion counts as
+hidden, and instances brought in by mosaic, mixup or copy-paste are measured against their own crops. For boxes (detect family), the ratio of
+the box areas: exact for a crop, approximate under rotation. Each call of the built-in
+augmentation exposes it as `TrainAugment.last_info["visible"]`.
+
+Limits: the head's mean / std and its `aux_class_weights` density come from the unscaled values,
+so with `"scale"` the values trained on sit somewhat lower than those statistics. Only the visible
+fraction is tracked, not the scale jitter, so a pixel area is still not supported. A custom
+`transform=` gets no adjustment. `crop` is rejected on a categorical attribute, and it is a
+training-only setting (not stored in the checkpoint). The training banner prints each head's crop
+mode.
+
 **Sidecar format (optional).** You can keep the COCO JSON as plain, standard COCO and put the
-attributes beside it instead of inside it: an `attributes.yaml` schema in the dataset root plus
+attributes beside it instead of inside it: an `attributes.yaml` schema (same keys, regression heads included) in the dataset root plus
 `attributes/<split>.json` values keyed by annotation id (`{ann_id: {head: value}}`). If present
 (and the JSON has no embedded `attributes`), it is merged in memory at load — so a plain COCO
 dataset always works and the sidecar is picked up automatically when you add it. Embedded
@@ -366,7 +442,7 @@ size, normalization and any auxiliary heads are auto-detected from the `.pt`):
 from eomt import EoMT
 
 # Train on COCO 2017 (auto-downloaded on first run):
-EoMT("l").train(data="coco", epochs=50, batch=4)
+EoMT("l").train(data="coco", batch=4)   # 12 epochs, validated every 2: the official recipe
 
 # ...or any COCO-format dataset (point at its data.yaml):
 EoMT("s").train(data="sample_data/data.yaml", epochs=1, batch=1)
@@ -384,7 +460,7 @@ GPUs; the main process scores them and alone logs and writes checkpoints. The pr
 argument must be picklable.
 
 ```python
-EoMT("l", device="0,1").train(data="coco", epochs=50, batch=2)   # 2 per GPU x accum 4 x 2 GPUs = effective 16
+EoMT("l", device="0,1").train(data="coco", batch=2)   # 2 per GPU x accum 4 x 2 GPUs = effective 16
 ```
 
 **Test-time augmentation and tiled inference** (instance family). `predict()` and `val()` take `augment=True` (the
@@ -393,8 +469,9 @@ image at `imgsz` and zoomed to 1.5x `imgsz`) and `tiles=True` (overlapping tiles
 mask IoU >= 0.5 measured where both views can see, so an object cut by a tile border is put back together; the mask
 logits are averaged and the score is the mean over the views that could see the object (one that missed it votes 0).
 Each takes a dict of overrides; see
-[`eomt/tta.py`](eomt/tta.py). `augment={"flip": True}` also merges mirrored views: never with left / right classes or
-attributes, which a mirrored view predicts swapped.
+[`eomt/tta.py`](eomt/tta.py). `augment={"flip": True}` also merges mirrored views. Attributes marked `flip_aware` are swapped
+back in a mirrored view; left / right *primary classes* are not (a mirrored view predicts them swapped), so keep `flip` off
+for those, and for attributes that encode a side without being flip-aware.
 
 ```python
 m = EoMT("runs/train/eomt-l")
@@ -403,7 +480,8 @@ m.val(data="data.yaml", tiles=True)                                 # native til
 m.predict("images/", augment={"flip": True}, tiles=1024)            # 1x / 1.5x and mirrored; 1024 px tiles
 ```
 
-`scripts/val.py` and `scripts/predict.py` take `--augment` and `--tiles [SIZE]`. Each view is a forward pass: `augment`
+`scripts/val.py` and `scripts/predict.py` take `--augment` and `--tiles [SIZE]` (`scripts/val.py` also `--flip`; its arguments are
+in [Command-line scripts](#command-line-scripts-arguments-and-things-to-take-into-account)). Each view is a forward pass: `augment`
 costs ~3.5x one prediction (the 1.5x view has 2.25x the pixels); a 1280 x 960 photo at `imgsz` 644 makes 6 tiles + the
 whole image.
 
@@ -422,11 +500,89 @@ LR, every step of the LR schedule, and the masked-attention annealing agree exac
 | encoder LR | layer-wise decay `llrd=0.8`, `backbone_lr_mult=1.0`: the **top ViT block is at the full `lr0`** and each block below is 0.8× the one above; embeddings follow the first block, the final norm is at `lr0` |
 | schedule | polynomial decay (`poly_power=0.9`) to 0 after a two-stage warmup in optimizer steps `warmup_steps=(500, 1000)`: the head warms up while the ViT LR is held at 0, then the ViT ramps up |
 | masked attention | annealed block by block (`mask_anneal_start/end`, fractions of training: 1/6–2/6, 2/6–3/6, 3/6–4/6, 4/6–5/6, polynomial); the last sixth of training is mask-free |
+| length | `epochs=12`, validated every `val_interval=2` epochs and after the last (official COCO instance recipe, about 89k optimizer steps at batch 16) |
 
 In EoMT the last four transformer blocks are the ones that process the queries and predict the masks, so they must learn at
 close to the head's speed: scaling the whole encoder by 0.1 (as detector-style recipes do for a backbone) starves them. The
-LR decay and the annealing span `epochs`, and validation runs mask-free, so metrics logged early in a long run understate
-the model. The official COCO recipe trains for 12 epochs (about 89k optimizer steps at batch 16).
+LR decay and the annealing span `epochs`.
+
+**Validation follows the training regime**, as in the official EoMT: masked attention at the annealing probabilities of
+the epoch's end. A model that still masks is far worse without its masks (part damage, ViT-L at epoch 11 of 50: 0.034
+segm mAP mask-free against 0.114 masked), so a mask-free curve before the last sixth follows the annealing and jumps as each
+block finishes it, not the model. `best.pt` is deployed mask-free, so it is only chosen once every block's annealing is
+over (a run stopped earlier, e.g. with `stop_after_epochs`, has `last.pt` alone), and every checkpoint is saved mask-free.
+Training prints the windows in epochs when it starts.
+
+### Command-line scripts: arguments and things to take into account
+
+**`scripts/train.py`.** Every flag is a keyword of `train()`; a flag left out keeps `train()`'s default.
+
+| flag | `train()` keyword | what it does |
+|---|---|---|
+| `--data` | `data` | dataset YAML, or `coco` (auto-downloads) |
+| `--size` / `--task` | | model size `s` `b` `l`; head family `instance` (masks) or `detect` (boxes) |
+| `--epochs` | `epochs` | total epochs: the LR schedule and the masked-attention annealing span them |
+| `--batch` / `--nominal-batch` | `batch` / `nominal_batch` | micro-batch **per GPU** / global effective batch (16); the difference is gradient accumulation |
+| `--device` | `device` | `auto`, `cpu`, `cuda:N`, or `0,1` for several GPUs (one process each) |
+| `--weights` / `--resume` | `init_weights` / `resume` | warm-start a fresh run from a checkpoint / continue a run. **One of the two, not both** |
+| `--lr`, `--warmup HEAD,VIT` | `lr0`, `warmup_steps` | peak head LR (the encoder follows with layer-wise decay) and the two-stage warmup in optimizer steps |
+| `--ema-decay`, `--ema-tau` | `ema_decay`, `ema_tau` | EMA of the weights (what is validated and exported as `best.pt`) |
+| `--mask-free` | `mask_anneal_start/end` | no masked attention from step 0 (annealing windows 0 -> 0) |
+| `--aux-dropout` | `aux_head_dropout` | dropout in the attribute heads |
+| `--val-interval`, `--workers`, `--seed` | same | validate every N epochs (and after the last); data-loader workers per GPU; seed |
+| `--aug KEY=VALUE`, `--aug-preset` | `aug` | augmentation overrides (see [Augmentation](#augmentation)) |
+| `--fpn-scales`, `--box-head`, `--no-deep-supervision`, `--upscale-blocks`, `--iou-aware-cls`, `--quality-weight`, `--val-imgsz` | same | [architecture options](#architecture-options) |
+
+Things to take into account when training:
+
+- **Resume or warm start.** `--resume` continues a run exactly (epoch, optimizer, schedule, EMA, best metric) and re-reads the
+  architecture from the checkpoint: keep the other flags as they were, and the seed is ignored so the shuffle is not replayed.
+  `--weights` loads the weights only and starts a new run at epoch 0. Start from a `best.pt` (the EMA, saved mask-free); a
+  `last.pt` holds the live weights.
+- **A short fine-tune needs its own recipe.** Lower `--lr`, shorten `--warmup`, and lower `--ema-decay` (at 0.9999 the EMA
+  averages over ~10k steps and barely leaves its starting point in a few epochs). If the model is deployed mask-free, pass
+  `--mask-free`: with the default annealing a warm-started model is trained *masked* for the first sixth of the run, a regime it
+  no longer runs in.
+- **`best.pt` is only chosen mask-free.** With the default annealing that means from 5/6 of the schedule on; with `--mask-free`
+  every validation is eligible.
+- **Validated epochs only.** With `--val-interval N` the other epochs have empty validation cells in `metrics.csv`; the curves in
+  `metrics.png` join the validated points.
+- **Attribute heads follow the dataset's definitions.** Categories, `applies_to`, `flip_aware` and `crop` / zoom settings come
+  from the dataset (the training banner lists them). The flip mapping, scope and regression statistics are stored in the
+  checkpoint; `crop` and the zoom settings are training-only.
+
+**`scripts/val.py`.**
+
+| flag | what it does |
+|---|---|
+| `weights` | checkpoint `.pt` or a run / weights folder (`best.pt` preferred) |
+| `--data`, `--batch`, `--workers`, `--device` | dataset YAML (`coco`), batch, data-loader workers, device |
+| `--imgsz` | evaluation input size (default: the trained one; the positional embeddings are interpolated) |
+| `--augment` | add the image at 1.5x `imgsz` |
+| `--tiles [SIZE]` | overlapping tiles (default `imgsz` px, native resolution) plus the whole image |
+| `--flip` | also run every view mirrored (alone: `--augment` with mirrored views) |
+| `--no-attributes` | mAP only |
+| `--ema`, `--mask annealed\|free` | validate the EMA weights of a `last.pt` (see below) |
+
+Things to take into account when validating:
+
+- **What is reported.** `segm/*` and `bbox/*` mAP and, for a model with secondary heads, `aux_acc/<head>` (accuracy) and
+  `aux_mae/<head>` (regression heads, mean absolute error in real units). They are scored on the same predictions as the mAP, so
+  they also work with `--augment`, `--tiles` and `--flip`.
+- **The attribute population is the deployed one.** Each GT instance is matched to the kept detection of its class with the
+  highest mask IoU (one to one, IoU >= 0.5). The trainer's `val/aux_*` columns score matched *queries* instead, so the two differ
+  slightly: compare TTA against plain within the script's numbers, not against `metrics.csv`. A regression head's MAE is
+  dominated by its large values, so also look at a relative error before concluding.
+- **Regime.** A model that is still annealing is only meaningful with its masks: `--ema --mask annealed` reproduces the trainer's
+  validation of a `last.pt` (EMA weights, the masked-attention probabilities of that epoch, read from the run's `args.yaml`),
+  `--ema --mask free` is the deployed regime, and `best.pt` is already the EMA, mask-free.
+- **TTA is not free and not uniform.** It costs one forward pass per view and mostly helps small objects and strict-IoU mAP; the
+  classification heads often do not move. A regression head's prediction is *averaged over views at different zoom*, so a value
+  that depends on the object's apparent size can drift under TTA: check it against the plain run before relying on it.
+- **`--flip` and the heads.** Heads the checkpoint stores as flip-aware are swapped back in mirrored views; an older checkpoint
+  takes the mapping from the dataset's JSON, with a warning. Left / right *primary classes* are not handled: keep `--flip` off for
+  them. The val categories of every head must match the model's, or the run stops instead of scoring against the wrong ids.
+- **One process.** `scripts/val.py` runs on one device; the trainer's own validation uses all the GPUs.
 
 ### Architecture options
 
@@ -485,7 +641,7 @@ mass-preserving), so thin objects are not shredded into dots by nearest-neighbou
 | geometry | `flip_prob` 0.5 · `vflip_prob` 0 · `rot90_prob` 0 · LSJ `min_scale`/`max_scale` 0.1–2.0 · `rotate_prob` 0.3 (±10°) · `shear_prob` 0.2 (±5°) · `perspective_prob` 0.15 |
 | optics / sensor | colour jitter 1.0 · `gamma_prob` 0.3 · `grayscale_prob` 0.05 · `blur_prob` 0.2 · `noise_prob` 0.2 · `jpeg_prob` 0.3 · `glare_prob` 0.15 · `erasing_prob` 0.2 |
 | multi-image (instance family) | `mosaic_prob` 0 · `mixup_prob` 0 · `copy_paste_prob` 0 |
-| crop | `instance_crop_prob` 0 (instance-aware crop) · `instance_crop_empty_prob` 0 |
+| crop | `instance_crop_prob` 0 (instance-aware crop) · `instance_crop_empty_prob` 0 · `zoom_prob` 0 (close-ups, `zoom_scale` 4–5x) |
 | masks | `mask_resize="area"` (or `"nearest"`) |
 
 Override per run, per dataset, or from the CLI — precedence is explicit keywords (`flip_prob`, `min_scale`,
@@ -513,12 +669,19 @@ python scripts/train.py --data my/data.yaml --aug min_scale=0.6 --aug instance_c
 Things worth knowing:
 
 - **Orientation.** Flips, 90° turns and rotations change the pixels but not the label. If an attribute encodes
-  orientation (e.g. a `viewpoint` head) set `flip_prob=0` and `rotate_prob=0`; `vflip_prob` / `rot90_prob` stay off
-  unless your scenes have no canonical "up".
+  orientation (e.g. a `viewpoint` head) set `flip_prob=0` and `rotate_prob=0`, or, for a left / right attribute,
+  mark it `flip_aware` (see *Flip-aware attributes*) and keep the horizontal flip; `vflip_prob` / `rot90_prob` stay
+  off unless your scenes have no canonical "up".
 - **Thin or tiny objects.** Large-Scale Jitter at 0.1 shrinks a 5 px crack to under a pixel; keep the effective
   scale within ~[0.4, 1.0] (`min_scale = 0.4·long_side/imgsz`, `max_scale = 1.0·long_side/imgsz`). The instance-aware
   crop pays off when the crop is much smaller than the image (high zoom, small `imgsz`), and
   `instance_crop_rarity_attr` makes rare attribute values the crop's focus more often.
+- **Close-ups.** `zoom_prob` makes an occasional sample a close-up: scaled by `s ~ U(zoom_scale)` (default 4–5x
+  the whole-image view, past the photo's own resolution) instead of the LSJ range, with the window on one instance
+  (whole when it fits, else a window inside it). `zoom_attr` restricts it to instances with a value of that
+  attribute and to the samples that have one, e.g. `{"zoom_prob": 0.1, "zoom_attr": "area"}` so a crop-scaled area
+  head also learns that a part filling the frame keeps its real area, and that a cut part keeps the share in view.
+  It resizes the whole photo first (~1.5x the time of a normal sample at 5x), so keep it occasional.
 - **Multi-image ops** (`mosaic`, `mixup`, `copy_paste`) add instances from other samples and are *off*: they splice or
   blend long thin structures, so try them deliberately and look at the result.
 - **Negatives.** Images with no annotations are dropped from the train set by default. `train(keep_empty=True)` keeps

{attr_eomt-1.1.3 → attr_eomt-1.2.0}/eomt/__init__.py

@@ -4,7 +4,7 @@ Public API:
 
     >>> from eomt import EoMT
     >>> model = EoMT("l")                      # fresh model (DINOv2 backbone)
-    >>> model.train(data="coco", epochs=50)    # COCO auto-downloads if missing
+    >>> model.train(data="coco")               # COCO auto-downloads if missing; 12 epochs, the official recipe
     >>> model = EoMT("runs/train/eomt-l")      # reload a run (everything auto-detected)
     >>> results = model.predict("images/", plot=True)
 

{attr_eomt-1.1.3 → attr_eomt-1.2.0}/eomt/api.py

@@ -4,7 +4,7 @@
     from eomt import EoMT
 
     model = EoMT("l")                       # fresh large model (DINOv2 backbone)
-    model.train(data="coco", epochs=50)     # COCO auto-downloads if missing
+    model.train(data="coco")                # COCO auto-downloads if missing; 12 epochs, the official recipe
 
     model = EoMT("runs/train/eomt-l")       # reload a run (everything auto-detected)
     model.val(data="coco")
@@ -286,7 +286,8 @@ class EoMT:
     # -------------------------------------------------------------------- val
     def val(self, data: str | Path = "coco", *, batch: int = 4, workers: int = 4,
             conf_thres: float = 0.0, max_det: int = 100, letterbox: bool | None = None,
-            imgsz: int | None = None, **kw) -> dict:
+            imgsz: int | None = None, augment: bool | dict = False, tiles: bool | int | dict = False,
+            attributes: bool = True, **kw) -> dict:
         """Evaluate on a dataset's val split, returning COCO mAP metrics.
 
         Segmentation models report ``segm/*`` (+ ``bbox/*``); detection
@@ -294,6 +295,11 @@ class EoMT:
         evaluation image size (defaults to the trained size); the model interpolates
         its positional embeddings to the new resolution, so any multiple of the patch
         size works.
+
+        ``augment`` / ``tiles`` evaluate with test-time augmentation / tiled inference (instance family; see
+        :mod:`eomt.tta`; a head marked ``flip_aware`` is swapped back in mirrored views). For a segmentation model with
+        secondary heads, ``attributes`` (default on) adds ``aux_acc/<head>`` (accuracy) and ``aux_mae/<head>`` (regression
+        heads, real units), scored on the same predictions as the mAP; see :func:`eomt.engine.validate.evaluate`.
         """
         cfg = _resolve_data(data)
         if not (cfg["val_images"] and cfg["val_json"]):
@@ -304,12 +310,14 @@ class EoMT:
         lb = letterbox if letterbox is not None else bool(getattr(model, "preprocess_letterbox", False))
         val_ds = CocoValImages(
             cfg["val_images"], cfg["val_json"], imgsz=_resolve_imgsz(imgsz, model), letterbox=lb,
-            mean=getattr(model, "pixel_mean", None), std=getattr(model, "pixel_std", None),
+            mean=getattr(model, "pixel_mean", None), std=getattr(model, "pixel_std", None), attributes=attributes,
         )
-        eval_fn = _evaluate_detection if getattr(model, "family", "instance") == "detect" else _evaluate
+        detect = getattr(model, "family", "instance") == "detect"
+        eval_fn = _evaluate_detection if detect else _evaluate
         return eval_fn(
             model, val_ds, device=dev, batch_size=batch, num_workers=workers,
-            conf_thres=conf_thres, max_det=max_det, **kw,
+            conf_thres=conf_thres, max_det=max_det, augment=augment, tiles=tiles,
+            **({} if detect else {"attributes": attributes}), **kw,
         )
 
     # -------------------------------------------------------------- compress