pikobs 3.0.415__tar.gz → 3.0.416__tar.gz

{pikobs-3.0.415 → pikobs-3.0.416}/CHANGELOG.md

@@ -1487,3 +1487,7 @@ Please use the following tags when editing this file:
 ## [3.0.415] 2026-09-17
 ### Added
 - bugs
+
+## [3.0.416] 2026-09-17
+### Added
+- bugs

{pikobs-3.0.415/pikobs.egg-info → pikobs-3.0.416}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pikobs
-Version: 3.0.415
+Version: 3.0.416
 Summary: pikobs
 Home-page: https://gitlab.science.gc.ca/dlo001/pikobs
 Author: David Lobon

pikobs-3.0.416/VERSION

@@ -0,0 +1 @@
+3.0.416

{pikobs-3.0.415 → pikobs-3.0.416}/pikobs/scatter/scatter.py

@@ -652,7 +652,10 @@ selector of the viewer:
 
 **significance** shows how sure the test is, from 0 to 100 %. Grey boxes
 stay at or below 95 % (darker when closer), orange ones are above 95 % and
-red ones above 99 %; the thick line on the colour bar marks 95 %. The box at
+red ones above 99 %; the thick line on the colour bar marks 95 %. For
+``omp`` and ``oma``, a crossed box (``xxx``) passed both tests but its bias
+and spread moved in opposite directions, so it is not counted (see *Why omp
+and oma need both tests*). The box at
 the top right recalls which test was run and counts the significant boxes:
 
 .. image:: _static/scatter_cmp_omp_significance.png
@@ -661,7 +664,8 @@ the top right recalls which test was run and counts the significant boxes:
    :width: 100%
 
 **significant only** is the difference map with the same colours and the
-same scale, where only the boxes above 95 % are drawn. It is usually the
+same scale, where only the boxes that pass are drawn (for ``omp`` / ``oma``:
+both tests, same direction). It is usually the
 one to show in a meeting:
 
 .. image:: _static/scatter_cmp_omp_sigonly.png
@@ -777,16 +781,108 @@ the difference only carries the change of the background; that is why the
 paired test sees small bias changes that an independent test would miss.
 ``bcorr`` uses the same test on the bias correction.
 
-An ``omp`` or ``oma`` box is significant only when **both** tests are
-above 95 %, and the significance map shows the lower of the two. On boxes of
-10° or more the confidences are written in the box, e.g. ``t99 F97``. An
-experience that moves the bias without touching the spread therefore stays
-mostly non-significant on the ``omp`` map; its ``t`` value in the box shows
-it.
+An ``omp`` or ``oma`` box counts only when **both** tests are above 95 %
+**and** the bias and the spread moved in the same direction. The
+significance map shows the lower of the two confidences, and on boxes of
+10° or more both are written in the box, e.g. ``t99 F97``. The next section
+explains why both are needed.
+
+Why omp and oma need both tests
+-------------------------------
+
+Think of the observation as the centre of a target, ``[O]``, and of each
+O-B as a shot. The **t-test** looks at the aim: did the average shot move
+closer to the centre, or further? The **F-test** looks at the hand: are the
+shots more grouped, or more scattered?
+
+Both tests are two-sided. Passing one means *this changed more than chance
+allows*, in either direction; the sign of the change says which way. So a
+box needs both tests above 95 %, and both changes pointing the same way.
+
+**Case 1: better aim, shakier hand.** The experience brings the average
+onto the centre, but its shots are all over the place.
+
+.. code-block:: text
+
+    CONTROL                          EXPERIENCE
+    off-centre, grouped              centred, scattered
+
+                                            E
+                  CCC                              E
+         [O]      CCC                       [O]
+                  CCC                   E              E
+                                               E
+
+    t-test: passes, the bias is smaller      (better)
+    F-test: passes, the spread is larger     (worse)
+
+The average looks perfect, but the model lost its consistency. The two
+changes disagree, so the box is **not drawn** on the significant-only map
+and is crossed (``xxx``) on the significance map.
+
+**Case 2: steady hand, wrong aim.** The experience groups its shots
+beautifully, but the whole group has drifted away from the centre.
+
+.. code-block:: text
+
+    CONTROL                          EXPERIENCE
+    centred, scattered               grouped, off-centre
+
+         C
+     C       C                                      EEE
+        [O]                                 [O]     EEE
+     C       C                                      EEE
+         C
+
+    t-test: passes, the bias is larger       (worse)
+    F-test: passes, the spread is smaller    (better)
+
+The spread went down, but a systematic error came in. Again the changes
+disagree: **not drawn**, crossed on the significance map.
+
+**Case 3: better on both.** The experience corrects the bias and tightens
+the spread. This is the result you are looking for.
+
+.. code-block:: text
+
+    CONTROL                          EXPERIENCE
+    off-centre and scattered         centred and grouped
+
+            C
+                                               EEE
+    [O]                                       E[O]E
+                    C                          EEE
+            C
+
+    t-test: passes, the bias is smaller      (better)
+    F-test: passes, the spread is smaller    (better)
+
+The box is drawn in **red** on the significant-only map and counted as
+*better*.
+
+**Case 4: worse on both.** The mirror image of case 3: the experience moves
+away from the centre and scatters more. The box is drawn in **blue** and
+counted as *worse*, which is just as useful to know.
+
+A last case is the quiet one: the bias improves but the spread does not
+change at all. The t-test passes, the F-test does not, and the box stays
+out of the significant-only map. Its ``t`` value, written in the box on the
+significance map, still shows that something moved.
+
+In one line, for ``omp`` and ``oma``:
+
+.. code-block:: text
+
+    drawn in red   = bias better AND spread better, both above 95 %
+    drawn in blue  = bias worse  AND spread worse,  both above 95 %
+    not drawn      = anything else (crossed when both pass in opposite directions)
 
 Reading the tests
 -----------------
 
+* ``stdomp`` and ``stdoma`` only look at the spread, so the F-test alone
+  decides there, and ``bcorr`` only looks at the mean. Only ``omp`` and
+  ``oma`` combine both.
 * The F-test treats the two runs as independent samples. With common
   observations they are strongly correlated, which makes the F-test
   cautious: a real change of the spread needs more observations to pass.
@@ -2636,6 +2732,21 @@ def _paired_t_confidence(mx, my, vx, vy, cxy, N) -> np.ndarray:
     return conf
 
 
+def _both_tests(d_bias, d_spread, t_conf, f_conf):
+    """omp / oma: a box counts only when the bias and the spread both changed
+    significantly AND in the same direction (both better or both worse).
+
+    Returns (values, significant, confidences, kind); confidences also
+    carries 'mixed', the boxes where both tests pass but the bias and the
+    spread moved in opposite directions.
+    """
+    both  = _is_significant(t_conf) & _is_significant(f_conf)
+    same  = np.sign(d_bias) == np.sign(d_spread)
+    mixed = both & ~same
+    return (d_bias, both & same,
+            {'t': t_conf, 'F': f_conf, 'mixed': mixed}, 'improve')
+
+
 def _paired_metric(fonction: str, N, SX, SY, SXX, SYY, SXY
                    ) -> Tuple[np.ndarray, np.ndarray,
                               Dict[str, np.ndarray], str]:
@@ -2664,8 +2775,7 @@ def _paired_metric(fonction: str, N, SX, SY, SXX, SYY, SXY
 
         t_conf = _paired_t_confidence(mx, my, vx, vy, cxy, N)
         values = np.abs(my) - np.abs(mx)
-        return (values, _is_significant(t_conf) & _is_significant(f_conf),
-                {'t': t_conf, 'F': f_conf}, 'improve')
+        return _both_tests(values, s_exp - s_ref, t_conf, f_conf)
 
 
 def _diff_metric(fonction: str, avg1, avg2, std1, std2, bc1, bc2, n1, n2,
@@ -2703,8 +2813,7 @@ def _diff_metric(fonction: str, avg1, avg2, std1, std2, bc1, bc2, n1, n2,
 
         t_conf = _ttest_confidence(avg1, std1, n1, avg2, std2, n2)
         values = np.abs(avg2) - np.abs(avg1)
-        return (values, _is_significant(t_conf) & _is_significant(f_conf),
-                {'t': t_conf, 'F': f_conf}, 'improve')
+        return _both_tests(values, s_exp - s_ref, t_conf, f_conf)
 
 
 _CBAR_DIFF: Dict[str, str] = {
@@ -2732,8 +2841,8 @@ _CBAR_SINGLE: Dict[str, str] = {
 
 
 _TEST_TEXT = {
-    'omp':    "t-test (bias O-B) + F-test (σ O-B)\nshown: the lower confidence",
-    'oma':    "t-test (bias O-A) + F-test (σ O-A)\nshown: the lower confidence",
+    'omp':    "t-test (bias O-B) + F-test (σ O-B)\nboth > 95 %, same direction",
+    'oma':    "t-test (bias O-A) + F-test (σ O-A)\nboth > 95 %, same direction",
     'stdomp': "F-test on σ(O-B), F = σ²big / σ²small",
     'stdoma': "F-test on σ(O-A), F = σ²big / σ²small",
     'bcorr':  "Welch t-test on the mean bias corr.",
@@ -2741,10 +2850,10 @@ _TEST_TEXT = {
 
 
 _TEST_TEXT_PAIRED = {
-    'omp':    "paired t-test (bias) + F-test (σ), O-B\n"
-              "of common obs, lower confidence shown",
-    'oma':    "paired t-test (bias) + F-test (σ), O-A\n"
-              "of common obs, lower confidence shown",
+    'omp':    "paired t-test (bias) + F-test (σ)\n"
+              "on O-B of common obs, both > 95 %, same direction",
+    'oma':    "paired t-test (bias) + F-test (σ)\n"
+              "on O-A of common obs, both > 95 %, same direction",
     'stdomp': "F-test on σ(O-B) of common obs,\nF = σ²big / σ²small",
     'stdoma': "F-test on σ(O-A) of common obs,\nF = σ²big / σ²small",
     'bcorr':  "paired t-test on the bias corr., common obs",
@@ -2753,7 +2862,8 @@ _TEST_TEXT_PAIRED = {
 
 def _info_text(panel, fonction, is_diff, kind, confs, values, diff_values,
                signif, in_axes, boundaries, on_map_obs, on_map_ref, n_days,
-               avg=None, std=None, n_black=0, n_pairs=None) -> str:
+               avg=None, std=None, n_black=0, n_pairs=None,
+               n_mixed=0) -> str:
     """Text of the box at the top right of a map."""
     if not is_diff:
         if fonction in ('dens', 'nobs'):
@@ -2786,6 +2896,9 @@ def _info_text(panel, fonction, is_diff, kind, confs, values, diff_values,
         n_wor = int((box & signif & (diff_values > 0)).sum())
         counts += f'\n{n_bet} better (red), {n_wor} worse (blue)'
 
+    mixed_line = (f'\nmixed: {n_mixed} boxes, bias and σ changed\n'
+                  f'in opposite directions (not counted)'
+                  if n_mixed else '')
     black_line = (f'\nblack: {n_black} boxes not compared, no\n'
                   + ('common observation in control and experience'
                      if n_pairs is not None else
@@ -2795,17 +2908,19 @@ def _info_text(panel, fonction, is_diff, kind, confs, values, diff_values,
     if panel == PANEL_SIGONLY:
         return (f'Only boxes with confidence > {_MIN_CONFIDENCE:.0f} %\n'
                 f'({tests.get(fonction, "").splitlines()[0]})\n'
-                f'{counts}\n{nobs_line}{black_line}')
+                f'{counts}\n{nobs_line}{mixed_line}{black_line}')
 
     if panel == PANEL_CONF:
         return (f'{tests.get(fonction, "")}\n'
                 f'(1 - p) x 100, > {_MIN_CONFIDENCE:.0f} % = significant\n'
-                f'{counts}\n{nobs_line}{black_line}')
+                f'{counts}\n{nobs_line}'
+                + (mixed_line.replace('boxes,', 'boxes (xxx),')
+                   if n_mixed else '') + black_line)
 
     text = nobs_line
     if confs:
         text += f'\n{counts.replace("boxes", f"boxes at {_MIN_CONFIDENCE:.0f} %")}'
-    return text + black_line
+    return text + mixed_line + black_line
 
 
 def _render_tile_map(task: Dict[str, Any],
@@ -2957,6 +3072,10 @@ def _render_tile_map_impl(task: Dict[str, Any],
     if signif is None:
         signif = np.ones(len(values), dtype=bool)
 
+    mixed = confs.pop('mixed', None) if confs else None
+    if mixed is None:
+        mixed = np.zeros(len(values), dtype=bool)
+
     # combined confidence: the weakest test decides (omp/oma need both)
     conf = None
     if confs:
@@ -3038,6 +3157,7 @@ def _render_tile_map_impl(task: Dict[str, Any],
         on_map_ref = float(n1[in_axes].sum()) if is_diff else 0.0
 
     patches, patch_colors = [], []
+    patches_mixed = []
     for i in np.flatnonzero(in_axes):
         if panel == PANEL_SIGONLY and not signif[i]:
             continue
@@ -3058,12 +3178,21 @@ def _render_tile_map_impl(task: Dict[str, Any],
         patches.append(plt.Polygon(
             _project_polygon(proj_obj, lat[i], lon[i], deltax, deltay, pc)))
         patch_colors.append(facecolors[i])
+        if panel == PANEL_CONF and mixed[i]:
+            patches_mixed.append(plt.Polygon(
+                _project_polygon(proj_obj, lat[i], lon[i], deltax, deltay, pc)))
 
     if patches:
         ax.add_collection(PatchCollection(
             patches, facecolors=patch_colors,
             edgecolors='black', linewidths=0.2, alpha=ALPHA_TILES, zorder=4))
 
+    if patches_mixed:
+        # both tests pass, but bias and spread moved in opposite directions
+        ax.add_collection(PatchCollection(
+            patches_mixed, facecolors='none', edgecolors='#3a3a3a',
+            linewidths=0.3, hatch='xxx', zorder=5))
+
     if exclusive:
         ax.add_collection(PatchCollection(
             [plt.Polygon(_project_polygon(proj_obj, lt, ln, deltax, deltay, pc))
@@ -3132,7 +3261,8 @@ def _render_tile_map_impl(task: Dict[str, Any],
                          avg if not is_diff else None,
                          std if not is_diff else None,
                          n_black=len(exclusive),
-                         n_pairs=n_pairs_on_map)
+                         n_pairs=n_pairs_on_map,
+                         n_mixed=int((mixed & in_axes).sum()))
     # top right, next to the title block
     info_txt = ax.text(1.0, 1.02, textstr, transform=ax.transAxes,
             fontsize=11, ha='right', va='bottom',
@@ -3265,7 +3395,10 @@ _INTRO_DIFF = [
     "<b>significant only</b> is the difference map where only the boxes "
     "above 95 % are drawn. The spread is tested with the F-test "
     "F = σ²larger / σ²smaller, two-sided; the bias and bcorr with a paired "
-    "t-test; omp and oma need both, and the map shows the lower confidence. "
+    "t-test. omp and oma need both tests above 95 % and the bias and the "
+    "spread moving the same way (both better: red, both worse: blue); when "
+    "they move in opposite directions the box is crossed on the significance "
+    "map and left out. The map shows the lower confidence. "
     "obs, nobs and dens have no test; bcorr exists for radiances only.",
 
     "The scale is symmetric and adapts to each map (steps like 0.1, 0.2, 0.3, "

{pikobs-3.0.415 → pikobs-3.0.416}/pikobs/script/run_scatter.sh

@@ -46,7 +46,7 @@ CONTROL_NAME="Control"
 # Output directory. WARNING: wiped on each run to prevent ghost files.
 # Will contain one subdir per family with the .db files and .png maps,
 # plus an interactive HTML viewer at pikobs_scatter_viewer.html.
-PATHWORK="/home/dlo001/sites8/pikobs_scatter_exp"
+PATHWORK="/home/dlo001/sites8/pikobs_scatter_exp2"
 
 # ---- Time window ---------------------------------------------------------
 # Processing window in YYYYMMDDHH format (UTC). Inclusive on both ends,

{pikobs-3.0.415 → pikobs-3.0.416/pikobs.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pikobs
-Version: 3.0.415
+Version: 3.0.416
 Summary: pikobs
 Home-page: https://gitlab.science.gc.ca/dlo001/pikobs
 Author: David Lobon

pikobs-3.0.415/VERSION

@@ -1 +0,0 @@
-3.0.415