gea-program 0.32.0__tar.gz → 0.34.0__tar.gz

{gea_program-0.32.0/gea_program.egg-info → gea_program-0.34.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gea-program
-Version: 0.32.0
+Version: 0.34.0
 Summary: GEA-Program: downhole gauge monitoring - canonical measurement records, quality rules, drift evaluation with a re-fit change log, well-test validation, alarm management, model cards, store-and-forward, monthly SLA measurement, FAT/SAT protocols and a dashboard, generated from live objects in the client's reporting outline.
 Author: Daniel T. Murphy
 License: MPL-2.0
@@ -125,7 +125,7 @@ gea dashboard --catalog-well volve_f12_f14_production_excerpt:15/9-F-12:10000 --
 gea client-report --report accuracy --out client_report
 gea model-cards --out model_cards
 gea sbom --out sbom
-gea accept                     # the product gate (383 checks)
+gea accept                     # the product gate (389 checks)
 gea help drift                 # the help library, by the job (16 pages; the same text is on every dashboard page)
 gea guide                      # the click-by-click tester guide (docs/TESTER_GUIDE.md)
 gea gui                        # the desktop window (pip install "gea-program[desktop]")
@@ -648,6 +648,39 @@ against it; `gea seedlink --config ... --hello` prints what a server says of
 itself; `gea help seedlink`. A TexNet station is served by EarthScope at
 `rtserve.iris.washington.edu:18000`, network `TX`.
 
+### The site map
+
+The Map tab puts everything in the workspace that has a position in one
+WGS84 frame: the wells at their declared surface positions (a NAD27
+declaration moved by its datum shift, the shift shown), the structure layers
+- faults and folds, formation contours, areas, points - each drawn at the
+detail the view needs without a vertex moved, the World Stress Map's SHmax
+directions, the seismic stations and the Seismicity Response Areas, over the
+outline of the US states and counties. The outline ships with the program
+(US Census Bureau 2017 cartographic boundaries by way of us-atlas, with its
+provenance), so the map needs no network. A well without a declared position
+is listed as not placed and is never drawn. Drag, the wheel or the keys pan
+and zoom; click anywhere for what every layer says about that spot and the
+distance to every placed well; rings draw distance circles around the wells.
+`gea help map`.
+
+### The charts
+
+A well's page carries the Charts card: up to four channels stacked on one
+horizontal axis, a cursor that reads them all at the same moment, cut from
+every sample of the well's file. Drag across a stretch to zoom into it, the
+wheel zooms where you point, double-click or Reset shows everything; the
+keyboard does the same (arrows, + and -, Page Up and Page Down, Home). A
+month-wide view keeps a one-second spike - each bucket is the minimum, the
+maximum and the mean of its samples - and a stretch with no samples is drawn
+empty, never bridged. On a drilling recorder's export the axis switches
+between time and depth (the drilling samples by bit depth, kept off hole a
+later sidetrack abandoned), the depth events are marked where they
+happened, and each section of the recording is a button. The Wells page
+carries the drilling progress of every drilling-recorder well: hole depth
+against days, one colour per well. The refresh writes a chart cache per well
+from the read it already makes; `gea help charts`.
+
 ### The setup checklist
 
 The Setup tab (`#/setup`, `gea checklist --workspace C:\site`) says what a

{gea_program-0.32.0 → gea_program-0.34.0}/README.md

@@ -86,7 +86,7 @@ gea dashboard --catalog-well volve_f12_f14_production_excerpt:15/9-F-12:10000 --
 gea client-report --report accuracy --out client_report
 gea model-cards --out model_cards
 gea sbom --out sbom
-gea accept                     # the product gate (383 checks)
+gea accept                     # the product gate (389 checks)
 gea help drift                 # the help library, by the job (16 pages; the same text is on every dashboard page)
 gea guide                      # the click-by-click tester guide (docs/TESTER_GUIDE.md)
 gea gui                        # the desktop window (pip install "gea-program[desktop]")
@@ -609,6 +609,39 @@ against it; `gea seedlink --config ... --hello` prints what a server says of
 itself; `gea help seedlink`. A TexNet station is served by EarthScope at
 `rtserve.iris.washington.edu:18000`, network `TX`.
 
+### The site map
+
+The Map tab puts everything in the workspace that has a position in one
+WGS84 frame: the wells at their declared surface positions (a NAD27
+declaration moved by its datum shift, the shift shown), the structure layers
+- faults and folds, formation contours, areas, points - each drawn at the
+detail the view needs without a vertex moved, the World Stress Map's SHmax
+directions, the seismic stations and the Seismicity Response Areas, over the
+outline of the US states and counties. The outline ships with the program
+(US Census Bureau 2017 cartographic boundaries by way of us-atlas, with its
+provenance), so the map needs no network. A well without a declared position
+is listed as not placed and is never drawn. Drag, the wheel or the keys pan
+and zoom; click anywhere for what every layer says about that spot and the
+distance to every placed well; rings draw distance circles around the wells.
+`gea help map`.
+
+### The charts
+
+A well's page carries the Charts card: up to four channels stacked on one
+horizontal axis, a cursor that reads them all at the same moment, cut from
+every sample of the well's file. Drag across a stretch to zoom into it, the
+wheel zooms where you point, double-click or Reset shows everything; the
+keyboard does the same (arrows, + and -, Page Up and Page Down, Home). A
+month-wide view keeps a one-second spike - each bucket is the minimum, the
+maximum and the mean of its samples - and a stretch with no samples is drawn
+empty, never bridged. On a drilling recorder's export the axis switches
+between time and depth (the drilling samples by bit depth, kept off hole a
+later sidetrack abandoned), the depth events are marked where they
+happened, and each section of the recording is a button. The Wells page
+carries the drilling progress of every drilling-recorder well: hole depth
+against days, one colour per well. The refresh writes a chart cache per well
+from the read it already makes; `gea help charts`.
+
 ### The setup checklist
 
 The Setup tab (`#/setup`, `gea checklist --workspace C:\site`) says what a

{gea_program-0.32.0 → gea_program-0.34.0}/gea/__init__.py

@@ -123,7 +123,7 @@ from .profile_catalog import (
     read_pangaea_txt,
 )
 
-__version__ = "0.32.0"
+__version__ = "0.34.0"
 __all__ = [
     "aging_rate", "rate_psi_yr", "accuracy_psi",
     "Sensor", "SimulatorConfig", "DownholeEngine",

{gea_program-0.32.0 → gea_program-0.34.0}/gea/acceptance_tests.py

@@ -8170,6 +8170,284 @@ def section_ca_setup_checklist(tmp: str) -> None:
        "request; the help page and its view mapping"
        + (f" [failed: {', '.join(failed3)}]" if failed3 else ""))
 
+
+def section_cb_charts(tmp: str) -> None:
+    """Section CB - the interactive charts: a window cut from every sample through the series cache, an envelope that
+    keeps a one-second spike in a month-wide view and never bridges a gap, the depth axis for a drilling recorder from
+    the same samples as the gamma log, the drilling progress of every well, the page's zoom, pan, cursor and keys."""
+    import http.cookiejar
+    import os as _os
+    import urllib.request
+    import urllib.error
+    import numpy as _np
+    from . import edr as E
+    from . import series_cache as SCH
+    from . import backup as BK
+    from . import helplib as H
+    from .workspace import Workspace
+    from .service import Service, Users
+    pkg = Path(__file__).parent
+    d = Path(tmp, "cb"); d.mkdir()
+
+    ev = E.selftest_events(str(d))
+    st = E.read(ev["path"])
+    meta = SCH.build(st, str(d / "c"), extras=E.chart_masks(st))
+    c = SCH.load(str(d / "c"), ev["path"])
+    idx = _np.asarray(c.index); rop = _np.asarray(c.channel("Rate Of Penetration"), dtype=float)
+    w = SCH.window(c, ["Rate Of Penetration", "Hole Depth"], points=300)
+    edges = _np.linspace(idx[0], idx[-1], 301); b = _np.clip(_np.searchsorted(edges, idx, "right") - 1, 0, 299)
+    brute_ok = True
+    for k in range(300):
+        m = (b == k) & ~_np.isnan(rop)
+        got = w["channels"]["Rate Of Penetration"]
+        if m.any():
+            if got["max"][k] is None or abs(got["max"][k] - rop[m].max()) > 1e-3 or abs(got["min"][k] - rop[m].min()) > 1e-3 or abs(got["mean"][k] - rop[m].mean()) > 1e-2 or got["n"][k] != int(m.sum()):
+                brute_ok = False; break
+        elif got["max"][k] is not None:
+            brute_ok = False; break
+    gaps = sum(1 for x in w["channels"]["Rate Of Penetration"]["mean"] if x is None)
+    spike = st.channels["Rate Of Penetration"].values.copy()
+    w_small = SCH.window(c, ["Rate Of Penetration"], x0=float(idx[1300]), x1=float(idx[1300]) + 300)
+    dw = SCH.depth_window(c, ["Gamma"], points=200)
+    pr = SCH.progress(c)
+    cons = SCH.connections(c, float(idx[0]), float(idx[-1]))
+    stale_size = _os.path.getsize(ev["path"])
+    with open(ev["path"], "a", encoding="utf-8") as fh:
+        fh.write("")
+    _os.utime(ev["path"], (1, 1))
+    stale = SCH.load(str(d / "c"), ev["path"])
+    c1 = {
+        "cache": meta["n"] == 34720 and {"drilling", "keep_b", "keep_h", "excluded", "bottom", "flag_idle"} <= set(meta["extras"])
+                 and len(meta["sections"]) == 2 and _os.path.isfile(str(d / "c" / "index.npy")),
+        "envelope": w["mode"] == "envelope" and len(w["x"]) == 300 and w["samples"] == 34720 and brute_ok and gaps > 200,
+        "samples": w_small["mode"] == "samples" and len(w_small["x"]) == 301 and w_small["channels"]["Rate Of Penetration"]["value"][0] is not None,
+        "depth": dw["axis"] == "Bit Depth" and dw["extent"] == [4800.0, 5750.0] and dw["samples"] == 27420
+                 and all(v is None or abs(v - 110.0) < 1e-6 for v in dw["channels"]["Gamma"]["mean"]) and "abandoned" in dw["how"],
+        "progress": pr is not None and pr["first_utc"] == "2026-07-01T00:00:00" and max(v for v in pr["bottom_max"] if v is not None) == 5750.0
+                    and abs(pr["days"][-1] - 2.4) < 0.01,
+        "connections": len(cons) == 3,
+        "stale": stale is None and stale_size == _os.path.getsize(ev["path"]),
+    }
+    failed1 = [k for k, v in c1.items() if not v]
+    ok(not failed1,
+       "CB1 the series cache: every sample of a drilling recorder export with the depth pass's masks and the running bottom; a window of equal time "
+       "buckets whose minimum, maximum, mean and count equal the samples' own, a bucket with none left empty (a gap, never bridged), the samples "
+       "themselves when they fit; the depth axis the drilling samples by bit depth kept off the abandoned hole (4,800 to 5,750 ft, the gamma log's "
+       "range); the drilling progress the running bottom against days; a cache whose source changed is not used"
+       + (f" [failed: {', '.join(failed1)}]" if failed1 else ""))
+
+    wsp = str(d / "ws"); ws = Workspace.create(wsp, "Pad 3", actor="tester")
+    w1 = ws.add_well_file(ev["path"], display="Well B", actor="tester")
+    w2 = ws.add_well_catalog("volve_f12_f14_production_excerpt", "15/9-F-12", 10000.0, actor="tester")
+    ws.refresh_dashboard(actor="tester")
+    built = _os.path.isfile(_os.path.join(wsp, "cache", "series", w1["id"], "x_drilling.npy"))
+    Users(str(Path(wsp, "users.json"))).add("v", "viewer-pass-1", "viewer")
+    svc = Service(wsp, port=0, scheduler=False).start()
+    base_url = svc.url.rstrip("/")
+    o = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
+
+    def call(method, path, body=None):
+        req = urllib.request.Request(base_url + path, method=method, data=(json.dumps(body).encode() if body is not None else None))
+        req.add_header("Content-Type", "application/json"); req.add_header("X-GEA-Action", "1")
+        try:
+            with o.open(req, timeout=120) as rr_:
+                return rr_.status, json.loads(rr_.read())
+        except urllib.error.HTTPError as e:
+            return e.code, json.loads(e.read())
+    q = lambda s_: urllib.parse.quote(s_)
+    try:
+        call("POST", "/api/login", {"name": "v", "password": "viewer-pass-1"})
+        st_m, m1 = call("GET", f"/api/wells/{w1['id']}/chart/meta")
+        st_w, wt = call("GET", f"/api/wells/{w1['id']}/chart/window?ch={q('Rate Of Penetration')}&ch={q('Hole Depth')}&points=400")
+        st_z, wz = call("GET", f"/api/wells/{w1['id']}/chart/window?ch={q('Rate Of Penetration')}&x0=172800&x1=180000&points=400")
+        st_d, wd = call("GET", f"/api/wells/{w1['id']}/chart/window?ch=Gamma&axis=depth&points=200")
+        st_p, pp = call("GET", "/api/pad/progress")
+        st_g, mg = call("GET", f"/api/wells/{w2['id']}/chart/meta")
+        st_bad, bad = call("GET", f"/api/wells/{w1['id']}/chart/window?ch=Nothing")
+        st_none, none_ = call("GET", f"/api/wells/{w1['id']}/chart/window")
+        st_x, _ = call("GET", "/api/wells/no-such-well/chart/meta")
+    finally:
+        svc.stop()
+    c2 = {
+        "refresh_builds": built and not any(f.startswith("cache/") for f in BK.site_files(wsp)),
+        "meta": st_m == 200 and m1["axes"] == ["time", "depth"] and m1["n"] == 34720 and len(m1["events"]) == 4 and m1["events"][2]["type"] == "SIDETRACK"
+                and m1["events"][0]["x"] > 0 and len(m1["sections"]) == 2 and m1["default"] == ["Rate Of Penetration"],
+        "window": st_w == 200 and wt["mode"] == "envelope" and set(wt["channels"]) == {"Rate Of Penetration", "Hole Depth"} and st_z == 200
+                  and wz["mode"] == "envelope" and "connections" in wz and wz["x0"] == 172800.0,
+        "depth": st_d == 200 and wd["extent"] == [4800.0, 5750.0],
+        "progress": st_p == 200 and [x["well_id"] for x in pp["wells"]] == [w1["id"]] and pp["wells"][0]["display"] == "Well B",
+        "gauge_well": st_g == 200 and mg["axes"] == ["time_s"] and mg["n"] > 0 and mg["events"] == [],
+        "errors": st_bad == 400 and "no channel 'Nothing'" in bad.get("error", "") and st_none == 400 and "name one or more channels" in none_.get("error", "") and st_x == 404,
+    }
+    failed2 = [k for k, v in c2.items() if not v]
+    ok(not failed2,
+       "CB2 in a site: the refresh writes each file well's chart cache from the read it already makes, and backups leave it out; the API gives a "
+       "signed-in viewer the channels, the axes, the depth events at their times, the sections, a time window with the connections when zoomed in, "
+       "the depth window and every drilling-recorder well's progress; a gauge well's cache is built on its first view; a channel the well does not "
+       "have, no channel at all and an unknown well are named"
+       + (f" [failed: {', '.join(failed2)}]" if failed2 else ""))
+
+    page_src = (pkg / "web" / "app.html").read_text(encoding="utf-8")
+    help_md = (pkg / "help" / "charts.md").read_text(encoding="utf-8")
+    c3 = {
+        "page": all(x in page_src for x in ("async function chartPanel", "async function padProgress", "/chart/meta", "/chart/window", "/api/pad/progress",
+                                             'data-axis="depth"', "data-sec=", "PageUp", "'Home' || ev.key === 'Escape'", "ArrowLeft", "dblclick", "'wheel'",
+                                             "setPointerCapture", "ResizeObserver", "cp-tbl", 'aria-live="polite"', "role=\"application\"", "Drilling progress",
+                                             "#/help/charts", "cp-loading")),
+        "palette": page_src.count("--s1:") == 3 and page_src.count("--s3:") == 3,
+        "help": any(t["topic"] == "charts" for t in H.topics()) and not H.check("charts") and "never bridged" in help_md.split("will not call a measurement")[1],
+    }
+    failed3 = [k for k, v in c3.items() if not v]
+    ok(not failed3,
+       "CB3 the page: up to four tracks on one axis, a cursor that reads every track at the same moment, drag to zoom, the wheel, double-click and "
+       "Reset, pan, the keys (arrows, + and -, Page Up and Page Down, Home and Esc), time or depth, the section buttons, the table view, the frame "
+       "kept while a window loads, the drilling progress chart with one colour per well in a fixed order in both themes; the help page"
+       + (f" [failed: {', '.join(failed3)}]" if failed3 else ""))
+
+
+def section_cc_site_map(tmp: str) -> None:
+    """Section CC - the site map: the shipped state and county outline with its provenance, every positioned thing in one
+    WGS84 frame (a NAD27 well moved by its datum shift and saying so, a well without a position never drawn), a layer
+    thinned to the view's pixel without a vertex moved, what every layer says about a spot, the page."""
+    import hashlib as _hl
+    import http.cookiejar
+    import math as _m
+    import urllib.request
+    import urllib.error
+    from . import edr as E
+    from . import helplib as H
+    from . import shapefile as SF
+    from . import site_map as SM
+    from .workspace import Workspace
+    from .service import Service, Users
+    pkg = Path(__file__).parent
+    d = Path(tmp, "cc"); d.mkdir()
+
+    ol = SM.outline_data()
+    prov = json.loads((pkg / "reference" / "us_outline.provenance.json").read_text(encoding="utf-8"))
+    cul = [c for c in ol["counties"] if c["name"] == "Culberson"]
+    wide = SM.outline_view([-110.0, 25.0, -90.0, 37.0], 0.02)
+    near = SM.outline_view([-105.2, 31.0, -104.0, 32.0], 0.002)
+    c1 = {
+        "outline": len(ol["counties"]) == 3231 and ol["states"].get("48") == "Texas" and cul and cul[0]["fips"] == "48109"
+                   and abs(cul[0]["label"][0] + 104.65) < 0.2 and abs(cul[0]["label"][1] - 31.47) < 0.2,
+        "provenance": prov["sha256"] == _hl.sha256((pkg / "reference" / "us_outline.json").read_bytes()).hexdigest() and prov["package_license"] == "ISC"
+                      and "Copyright 2013-2019 Michael Bostock" in prov["package_license_text"] and "public domain" in prov["original_source"]
+                      and prov["counts"]["counties"] == 3231,
+        "detail": all(a["s"] == 1 for a in wide["arcs"]) and not wide["labels"] and any(a["s"] == 0 for a in near["arcs"])
+                  and "Culberson" in [x["name"] for x in near["labels"]],
+    }
+    failed1 = [k for k, v in c1.items() if not v]
+    ok(not failed1,
+       "CC1 the outline ships with the program: the US Census Bureau's 2017 cartographic counties by way of us-atlas 3.0.1, 3,231 counties, the "
+       "provenance with its hash, the package's ISC notice and the public-domain source; state lines only in a wide view, county lines and names "
+       "once the view is close (Culberson, FIPS 48109)"
+       + (f" [failed: {', '.join(failed1)}]" if failed1 else ""))
+
+    lat1, lon1 = 31.42, -104.10
+    kmlon = 1.0 / (111.195 * _m.cos(_m.radians(lat1)))
+    faults = [([[(lon1 - 0.2 + 0.01 * i, lat1 - 0.3 + 0.0002 * j) for j in range(3001)]], {"TYPE": "normal fault", "NAME": f"F{i}"}) for i in range(0, 40, 8)]
+    SF.write(str(d / "Faults"), "PolyLine", faults, [("TYPE", "C", 20, 0), ("NAME", "C", 10, 0)], SF.NAD83_GEOGCS)
+    ctr = [([[(lon1 + (i - 1.5) * 4 * kmlon + 0.001 * k, lat1 - 0.3 + 0.006 * k) for k in range(100)]], {"ElevTop_ft": -8500.0 + 500.0 * i}) for i in range(4)]
+    SF.write(str(d / "Top_A"), "PolyLine", ctr, [("ElevTop_ft", "F", 10, 1)], SF.NAD83_GEOGCS)
+    SF.write(str(d / "Picks"), "Point", [([[(lon1 + 0.0001 * i, lat1 + 0.00005 * i)]], {"NAME": f"P{i}"}) for i in range(6000)], [("NAME", "C", 10, 0)], SF.NAD83_GEOGCS)
+    with open(d / "wsm.csv", "w", encoding="utf-8") as f:
+        f.write("ID,LAT,LON,AZI,TYPE,DEPTH,QUALITY,REGIME,LOCALITY\n")
+        for i in range(40):
+            f.write(f"W{i},{lat1 - 0.1 + 0.005 * i:.4f},{lon1 - 0.05:.4f},{60 + i},BO,2.5,{'DCBA'[i % 4]},NF,test\n")
+    wsp = str(d / "ws"); ws = Workspace.create(wsp, "Pad 3", actor="tester")
+    empty_view = SM.overview(ws)["view"]
+    ws.add_layer(str(d / "Faults.shp"), "Basement faults", actor="tester", citation="test")
+    ws.add_layer(str(d / "Top_A.shp"), "Wolfcamp top", actor="tester", formation="Wolfcamp", role="top", citation="test")
+    ws.add_layer(str(d / "Picks.shp"), "Picks", actor="tester", citation="test")
+    ws.add_stress_map(str(d / "wsm.csv"), actor="tester", citation="test WSM")
+    layer_view0 = SM.overview(ws)["view"]
+    a = E.selftest(str(d)); b = E.selftest_events(str(d))
+    wa = ws.add_well_file(a["path"], display="Well A", actor="tester"); wb = ws.add_well_file(b["path"], display="Well B", actor="tester")
+    wc = ws.add_well_file(a["path"], display="Well C", actor="tester")
+    ws.set_well(wa["id"], surface={"lat": 31.41, "lon": -104.12, "datum": "NAD27"}, actor="tester")
+    ws.set_well(wb["id"], surface={"lat": 31.43, "lon": -104.08, "datum": "NAD83"}, actor="tester")
+    o = SM.overview(ws)
+    wl = {w["id"]: w for w in o["wells"]}
+    view = o["view"]["bbox"]
+    fine = SM.layer_view(ws, "Basement-faults", [-105, 30, -103, 32], 0.0)
+    coarse = SM.layer_view(ws, "Basement-faults", [-105, 30, -103, 32], 0.01)
+    raw = SM._layer(ws, "Basement-faults")
+    src_pts = {(round(float(x), 5), round(float(y), 5)) for f in raw["feats"] for pp in f["parts"] for x, y in pp}
+    moved = [p for f in coarse["features"] for pp in f["parts"] for p in pp if (p[0], p[1]) not in src_pts]
+    ends_kept = all(f["parts"][0][0] == fine["features"][k]["parts"][0][0] and f["parts"][0][-1] == fine["features"][k]["parts"][0][-1] for k, f in enumerate(coarse["features"]))
+    pts = SM.layer_view(ws, "Picks", [-105, 30, -103, 32], 0.0, max_points=1000)
+    wh = SM.what_here(ws, 31.42, -104.10)
+    sv = SM.stress_view(ws, [-104.3, 31.2, -103.9, 31.6])
+    c2 = {
+        "frame": "WGS84" in o["frame"] and o["view"]["why"] == "the placed wells" and empty_view["why"].startswith("nothing in the workspace")
+                 and layer_view0["why"].startswith("the area most of the layers cover (the median extent of 3 of the 3)"),
+        "wells": wl[wa["id"]]["placed"] and 30.0 < wl[wa["id"]]["shift_m"] < 90.0 and wl[wa["id"]]["declared"]["datum"] == "NAD27"
+                 and wl[wb["id"]]["placed"] and wl[wb["id"]]["shift_m"] == 0.0 and not wl[wc["id"]]["placed"] and "lat" not in wl[wc["id"]]
+                 and view[0] < -104.12 < view[2] and view[1] < 31.43 < view[3],
+        "layers": [(l["id"], l["kind"]) for l in o["layers"]] == [("Basement-faults", "structure"), ("Wolfcamp-top", "contours"), ("Picks", "points")]
+                  and o["layers"][0]["classes"] == ["fault"] and o["layers"][0]["n_points"] == 5 * 3001 and o["stress_map"]["n_records"] == 40,
+        "thinning": fine["vertices"] == 5 * 3001 and coarse["vertices"] < fine["vertices"] / 10 and not moved and ends_kept
+                    and all(f.get("cls") == "fault" for f in coarse["features"]),
+        "points": pts["in_view"] == 6000 and pts["shown"] == 1000 and pts["kind"] == "points",
+        "what_here": wh["answered"] >= 2 and any("Nearest mapped fault" in ln for l_ in wh["layers"] for ln in (l_["lines"] or []))
+                     and any("contours" in ln for l_ in wh["layers"] for ln in (l_["lines"] or []))
+                     and [w["id"] for w in wh["wells"]] == sorted([wa["id"], wb["id"]], key=lambda i: next(x["distance_km"] for x in wh["wells"] if x["id"] == i))
+                     and len(wh["wells"]) == 2,
+        "stress": sv["n_in_view"] == 40 and sv["records"][0]["quality"] == "A" and sv["records"][-1]["quality"] == "D" and sv["citation"] == "test WSM",
+    }
+    failed2 = [k for k, v in c2.items() if not v]
+    ok(not failed2,
+       "CC2 one frame: a well declared on NAD27 moved to WGS84 by its datum shift and the shift kept with it, a well declared on NAD83 not moved, a "
+       "well with no position never placed; the map opens on the placed wells (else the area most layers cover, else Texas, said so); a layer "
+       "thinned to the view's pixel keeps its ends and moves no vertex, points sampled evenly past the limit; each layer's own statement for a spot "
+       "and the distance to every placed well; the stress records in view, best quality first"
+       + (f" [failed: {', '.join(failed2)}]" if failed2 else ""))
+
+    Users(str(Path(wsp, "users.json"))).add("v", "viewer-pass-1", "viewer")
+    svc = Service(wsp, port=0, scheduler=False).start()
+    base_url = svc.url.rstrip("/")
+    op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
+
+    def call(path):
+        req = urllib.request.Request(base_url + path, method="GET")
+        try:
+            with op.open(req, timeout=120) as rr_:
+                return rr_.status, json.loads(rr_.read())
+        except urllib.error.HTTPError as e:
+            return e.code, json.loads(e.read())
+    try:
+        lreq = urllib.request.Request(base_url + "/api/login", method="POST", data=json.dumps({"name": "v", "password": "viewer-pass-1"}).encode())
+        lreq.add_header("Content-Type", "application/json"); lreq.add_header("X-GEA-Action", "1"); op.open(lreq, timeout=30)
+        bb = ",".join(f"{x:.5f}" for x in view)
+        st_o, ov = call("/api/map")
+        st_l, lv = call(f"/api/map/layer/Wolfcamp-top?bbox={bb}&tol=0.001")
+        st_ol, olv = call(f"/api/map/outline?bbox={bb}&tol=0.001")
+        st_s, svv = call(f"/api/map/stress?bbox={bb}")
+        st_a, at = call("/api/map/at?lat=31.42&lon=-104.10")
+        st_bad, _ = call("/api/map/layer/Wolfcamp-top?bbox=1,2,3")
+        st_nol, _ = call(f"/api/map/layer/no-such-layer?bbox={bb}")
+        st_noll, _ = call("/api/map/at?lat=x")
+    finally:
+        svc.stop()
+    page_src = (pkg / "web" / "app.html").read_text(encoding="utf-8")
+    help_md = (pkg / "help" / "map.md").read_text(encoding="utf-8")
+    c3 = {
+        "api": st_o == 200 and len(ov["wells"]) == 3 and st_l == 200 and lv["kind"] == "contours" and all("value" in f for f in lv["features"])
+               and st_ol == 200 and st_s == 200 and st_a == 200 and len(at["wells"]) == 2 and st_bad == 400 and st_nol == 404 and st_noll == 400,
+        "page": all(x in page_src for x in ("VIEWS.map", "['#/map', 'Map']", "/api/map/layer/", "/api/map/outline", "/api/map/stress", "/api/map/at",
+                                             "Not placed", "declare it", "data-fit", "mpRings", "ArrowUp", "'Home' || k === 'Escape'", "scale bar", "'wheel'",
+                                             "What is at", "setPointerCapture")),
+        "help": any(t["topic"] == "map" for t in H.topics()) and not H.check("map") and H.VIEW_TOPIC.get("map") == "map"
+                and "never moved" in help_md.split("will not call a measurement")[1],
+    }
+    failed3 = [k for k, v in c3.items() if not v]
+    ok(not failed3,
+       "CC3 the API (the contents, a layer with its contour values, the outline, the stress records, what is at a spot; a malformed box, an unknown "
+       "layer and a missing position named) and the Map tab: pan and zoom by pointer, wheel and keys, layer toggles and fit, distance rings, a scale "
+       "bar, the not-placed wells with the way to place them, what is at a click; the help page"
+       + (f" [failed: {', '.join(failed3)}]" if failed3 else ""))
+
 def main() -> int:
     print("GEA-Program - ACCEPTANCE SUITE (the product gate)")
     # a handle a job subprocess still holds at the end must not turn a finished gate into a traceback on Windows:
@@ -8252,6 +8530,8 @@ def main() -> int:
         section_by_edr(tmp)
         section_bz_tool_library(tmp)
         section_ca_setup_checklist(tmp)
+        section_cb_charts(tmp)
+        section_cc_site_map(tmp)
     if _FAILS:
         print(f"[ACCEPTANCE] {len(_FAILS)} FAILURES ({_PASS} passed):")
         for f in _FAILS:

{gea_program-0.32.0 → gea_program-0.34.0}/gea/backup.py

@@ -28,7 +28,7 @@ from datetime import datetime, timezone
 from typing import Dict, List, Optional
 
 STALE_AFTER_H = 26.0                      # a daily backup that is more than a day and change old
-EXCLUDE_TOP = ('jobs',)                   # taken selectively: schedule.json only
+EXCLUDE_TOP = ('jobs', 'cache')          # jobs taken selectively (schedule.json only); cache is rebuilt from the sources
 EXCLUDE_NAMES = ('_uploads', '__pycache__')
 LOG_NAME = 'backups.jsonl'                # records/backups.jsonl: one line per backup taken from this workspace
 

{gea_program-0.32.0 → gea_program-0.34.0}/gea/dashboard.py

@@ -347,8 +347,9 @@ def render(data: dict, out_dir: str, site_name: str = 'site', program_version: s
 def orchestrate(out_dir: str, catalog_wells: List[dict], file_wells: List[dict], td_ft: float = 10500.0,
                 gauge_spec=None, outages: Optional[List[str]] = None, month: Optional[str] = None,
                 site_name: str = 'site', criteria_path: Optional[str] = None, monitor_root: Optional[str] = None,
-                run_sat: bool = False) -> dict:
-    """catalog_wells: [{'entry', 'well', 'md_ft'}]; file_wells: [{'path', 'name'}]."""
+                run_sat: bool = False, series_cache_root: Optional[str] = None) -> dict:
+    """catalog_wells: [{'entry', 'well', 'md_ft'}]; file_wells: [{'path', 'name'}]. With `series_cache_root`, each file
+    well's stream is also cached for the charts, from the same read."""
     from . import (production_live_stream, ingest, Reconciler, SimulatorConfig, CATALOG, __version__ as _v)
     from .sample_record import TagCatalogue, records_from_stream
     from .client_reports import (gauge_drift_report, well_test_report, alarm_event_report, data_resilience_report,
@@ -401,16 +402,23 @@ def orchestrate(out_dir: str, catalog_wells: List[dict], file_wells: List[dict],
             continue
         with open(os.path.join(d, 'well.json'), 'w', encoding='utf-8') as f:
             json.dump({'display': name, 'source': fw['path'], 'index_kind': stream.index_kind, 'source_format': stream.source_format}, f)
+        cache_dir = os.path.join(series_cache_root, name) if series_cache_root else None
         if stream.source_format == 'edr':                         # a drilling recorder's export: the drilling record, not the gauge pipeline
             from .edr import write_record
             try:
-                write_record(stream, d, _v)
+                write_record(stream, d, _v, cache_dir=cache_dir)
                 made.setdefault(name, []).append('drilling_record')
             except Exception as ex:
                 with open(os.path.join(d, 'drilling_record_not_written.json'), 'w', encoding='utf-8') as f:
                     json.dump({'well': name, 'reason': f'{type(ex).__name__}: {ex}'}, f, indent=1)
                 made.setdefault(name, []).append('drilling_record_not_written')
             continue
+        if cache_dir:
+            try:
+                from . import series_cache as _SCH
+                _SCH.build(stream, cache_dir)
+            except Exception:
+                pass                                              # the cache is a convenience; the page builds it on first view
         if stream.index_kind != 'time_s':                      # a depth log is a survey input: run the strata survey, not the gauge pipeline
             try:
                 from .survey_cmd import run_survey

{gea_program-0.32.0 → gea_program-0.34.0}/gea/edr.py

@@ -322,6 +322,32 @@ def _idle_flag(drilling: np.ndarray, hole: Optional[np.ndarray], rop: Optional[n
     return idle
 
 
+def drilling_mask(stream: LiveStream, de: dict) -> Tuple[np.ndarray, np.ndarray]:
+    """The samples that are drilling - the on-bottom flag set at the bottom of the hole, not excluded, not the flag
+    left on with nothing drilled - and, separately, the flag-left-on samples. One definition for the record and the charts."""
+    hole, rop, onb = _ch(stream, 'hole'), _ch(stream, 'rop'), _ch(stream, 'onb')
+    n = len(stream.index)
+    on_flag = (onb == 1.0) if onb is not None else np.zeros(n, dtype=bool)
+    excl = de['excluded']; M = de['running_bottom']
+    if hole is not None and M is not None:
+        drilling = on_flag & ~excl & ~np.isnan(hole) & (hole >= np.nan_to_num(M, nan=np.inf) - 10.0)
+    else:
+        drilling = on_flag & ~excl
+    idle = _idle_flag(drilling, hole, rop)
+    return drilling & ~idle, idle
+
+
+def chart_masks(stream: LiveStream, de: Optional[dict] = None) -> dict:
+    """What the charts need from the depth pass, as arrays the length of the stream: drilling, the samples a depth
+    table may use (by hole depth and by bit depth), the excluded samples, and the running bottom of the hole."""
+    de = de if de is not None else depth_events(stream)
+    drilling, idle = drilling_mask(stream, de)
+    keep_h, keep_b = _keep_masks(stream, de)
+    n = len(stream.index)
+    bottom = de['running_bottom'] if de['running_bottom'] is not None else np.full(n, np.nan)
+    return {'drilling': drilling, 'flag_idle': idle, 'keep_h': keep_h, 'keep_b': keep_b, 'excluded': de['excluded'], 'bottom': bottom}
+
+
 def _keep_masks(stream: LiveStream, de: dict) -> Tuple[np.ndarray, np.ndarray]:
     """Samples a depth table may use: not excluded, and not in a borehole abandoned by a later sidetrack
     (by hole depth for the TVD table, by bit depth for the gamma log)."""
@@ -358,12 +384,7 @@ def record(stream: LiveStream, de: Optional[dict] = None) -> dict:
         out['notes'].append('channels this record needs are not in the file: ' + ', '.join(missing))
     on_flag = (onb == 1.0) if onb is not None else np.zeros(n, dtype=bool)
     excl = de['excluded']; M = de['running_bottom']; inc = de['increments']
-    if hole is not None and M is not None:
-        drilling = on_flag & ~excl & ~np.isnan(hole) & (hole >= np.nan_to_num(M, nan=np.inf) - 10.0)
-    else:
-        drilling = on_flag & ~excl
-    idle = _idle_flag(drilling, hole, rop)
-    drilling = drilling & ~idle
+    drilling, idle = drilling_mask(stream, de)
     rot = drilling & (rpm > 0) if rpm is not None else drilling
     prev_end = None
     for si, (a, b) in enumerate(sections, 1):
@@ -679,10 +700,14 @@ def document(rec: dict, program_version: str):
                     data={k: v for k, v in rec.items() if k != 'lines'} | {'_records': [], '_catalogue_obj': None})
 
 
-def write_record(stream: LiveStream, out_dir: str, program_version: str) -> dict:
-    """The record beside the well's other reports: drilling_record.json / .md / .html, and the MD-TVD table as CSV."""
+def write_record(stream: LiveStream, out_dir: str, program_version: str, cache_dir: Optional[str] = None) -> dict:
+    """The record beside the well's other reports: drilling_record.json / .md / .html, and the MD-TVD table as CSV;
+    with `cache_dir`, the charts' series cache from the same read and the same depth pass."""
     from .client_reports import write
     de = depth_events(stream)
+    if cache_dir:
+        from . import series_cache as SCH
+        SCH.build(stream, cache_dir, extras=chart_masks(stream, de))
     rec = record(stream, de=de)
     os.makedirs(out_dir, exist_ok=True)
     paths = write(document(rec, program_version), out_dir, basename='drilling_record')

gea_program-0.34.0/gea/help/charts.md

@@ -0,0 +1,11 @@
+# The charts
+
+> A well's channels, stacked on one horizontal axis with a cursor that reads them all at the same moment, cut from every sample of the well's file - not from a thinned copy. Drag across a stretch to zoom into it, wheel to zoom where you point, double-click to see everything again; on a drilling recorder's export the axis switches between time and depth, the depth events are marked where they happened, the recording's sections are a click each, and the Wells page carries the drilling progress of every well on one chart.
+
+**The command**: the Charts card on a well's page. Choose up to four channels (one per track); point at the chart for the values at that moment - each channel's mean in the bucket under the cursor, with its minimum and maximum and how many samples it holds, or the sample itself when the window is close enough to show every one. Drag across a stretch to zoom into it; the wheel zooms where you point; + and − zoom, ◀ and ▶ pan by half a window, Reset (or double-click) shows everything. The keyboard does the same once the chart has focus: ← → move the cursor (Shift for ten steps), + and − zoom, Page Up and Page Down pan, Home or Esc resets. Time or Depth switches the axis on a drilling recorder; the section buttons zoom to one stretch of recording. Table shows the window as rows. On the Wells page, Drilling progress plots each drilling-recorder well's hole depth against days since its first sample; point at it for every well's depth on that day.
+
+**What it writes**: a chart cache per well under `cache\series\` in the workspace, written by the refresh from the same read that writes the reports - the index, each channel, which samples were flagged, and for a drilling recorder the drilling samples, the samples a depth table may use, the excluded samples and the running bottom of the hole. It is a copy: it carries the source file's size and time and is rebuilt when they no longer match, the first view of a well with no cache builds it, and backups leave it out.
+
+**The number to check**: the sample count under the chart against the file's rows (`gea edr --file`); a bucket's minimum and maximum against the raw rows for a short stretch (zoom until the chart says every sample is shown); the depth axis's range against the gamma log's.
+
+**What this page will not call a measurement**: a bucket's mean - it is arithmetic on the samples in the bucket, and the band around it is their range; a line across a gap - a stretch with no samples is drawn empty, never bridged; the depth axis as the hole - it is the drilling samples by bit depth, kept off hole a later sidetrack abandoned, the same samples the gamma log uses; the drilling progress as a plan - it is the running bottom the depth pass read from the recorder, stale and re-referenced depths left out.

gea_program-0.34.0/gea/help/map.md

@@ -0,0 +1,11 @@
+# The site map
+
+> Everything in the workspace that has a position, in one frame: the wells where their surface positions were declared, the structure layers the site brought (faults and folds, formation contours, areas, points), the World Stress Map's SHmax directions, the seismic stations, the Seismicity Response Areas, and the outline of the US states and counties underneath. A well without a declared position is listed as not placed and is never drawn anywhere. Click the map for what every layer says about that spot.
+
+**The command**: the Map tab. Drag to pan, the wheel or + − to zoom, Home to return to where the map opened (the placed wells, else the response area, else the area most of the layers cover). Tick a layer to draw it, "fit" to zoom to it; Rings draws distance circles around every placed well. Point at the map for the position (WGS84) and the nearest placed well; click it - or press Enter for the centre - for each layer's own statement there: the nearest fault and its distance and bearing, the contours it lies between, the area it falls in, the nearest points, and the distance and bearing to every placed well. The keyboard does the same once the map has focus: arrows pan, + and − zoom, Home returns. A well's marker opens its page; "declare it" beside a not-placed well opens the setup checklist.
+
+**What it writes**: nothing. The map reads the workspace: each well's declared surface position, each layer's shapefile (read once per run of the program and again when it changes), the stress map, the stations and the response areas. The outline is shipped with the program (`reference/us_outline.json`, from the US Census Bureau's 2017 cartographic boundary files by way of the us-atlas package, with its provenance beside it), so the map needs no network.
+
+**The number to check**: a placed well against its survey plat - its marker is drawn at the declared latitude and longitude moved from the declared datum to WGS84, and the move is printed beside it (about fifty metres for NAD27 in West Texas); a layer's line against the map it came from, at the scale that map was made.
+
+**What this page will not call a measurement**: a well's position - it is the operator's declaration, drawn where it was declared; a line drawn at this zoom as the line in the file - a vertex that would fall in the same screen cell as the one before it is left out of the drawing (never moved), and the full line is what "what is here" measures against; the outline as a survey - it is the Census Bureau's cartographic generalisation, good to a few hundred metres; a distance to a mapped fault as a hazard - it is a fact of the map.

{gea_program-0.32.0 → gea_program-0.34.0}/gea/helplib.py

@@ -43,6 +43,8 @@ INDEX: List[tuple] = [
     ('edr', 'The drilling recorder', 'the rig\'s one-second EDR export read as delivered - sections, depths, hours on bottom, the rate of penetration, the recorder\'s MD-TVD table - and what the record will not say'),
     ('dlis', 'The DLIS log', 'a well log as it is delivered (API RP66 V1), read as the standard lays it out: the frames, the channels, what was skipped; into the strata survey'),
     ('historian', 'The historian pull', 'an operator\'s gauge history out of the PI System through PI Web API, read-only, every value with its archive time and flags'),
+    ('map', 'The site map', 'everything with a position in one frame: the placed wells, the structure layers, the stress directions, the stations and response areas over the state and county outline; what every layer says about any spot'),
+    ('charts', 'The charts', 'a well\'s channels stacked on one axis with a linked cursor, zoom and pan from every sample, time or depth for a drilling recorder, the depth events marked, and the drilling progress of every well'),
     ('setup', 'The setup checklist', 'what this workspace still needs before its reports can be handed over - each well\'s location, elevation and number, the client site, the backup, the reports - read from the workspace as it stands, each with its form or button'),
     ('tools', 'Tools and ports', 'every class of downhole tool in use across the industry with a hover description and a pop-open tutorial, your own fleet with its datasheets, the gauges this program reads with their citations and ratings, the formats and protocols that bring readings in, and the rating check on a tool string'),
     ('field-ports', 'The field ports', 'Modbus RTU on the serial line, DNP3 from the pad RTU, HART-IP from the transmitter itself - read-only, polled, with the stand-ins to rehearse against'),
@@ -69,7 +71,7 @@ REQUIRED_LINES = ('**The command**', '**What it writes**', '**The number to chec
 VIEW_TOPIC: Dict[str, str] = {
     'home': 'start', 'wells': 'data-in', 'well': 'drift', 'live': 'patches', 'patch': 'patches', 'files': 'files', 'alarms': 'alarms',
     'approvals': 'well-tests', 'config': 'site', 'reports': 'drift', 'verify': 'start', 'survey': 'data-in', 'jobs': 'site',
-    'admin': 'site', 'prefs': 'site', 'search': 'start', 'welcome': 'start', 'seismic': 'seismic', 'seismic_station': 'seismic', 'track': 'seismic', 'sar': 'seismic', 'sites': 'sites', 'setup': 'setup', 'audit': 'audit-update', 'tools': 'tools', 'catalog': 'catalog',
+    'admin': 'site', 'prefs': 'site', 'search': 'start', 'welcome': 'start', 'seismic': 'seismic', 'seismic_station': 'seismic', 'track': 'seismic', 'sar': 'seismic', 'sites': 'sites', 'setup': 'setup', 'map': 'map', 'audit': 'audit-update', 'tools': 'tools', 'catalog': 'catalog',
 }