gea-program 0.78.1__tar.gz → 0.80.0__tar.gz

{gea_program-0.78.1/gea_program.egg-info → gea_program-0.80.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gea-program
-Version: 0.78.1
+Version: 0.80.0
 Summary: GEA-Program: downhole gauge monitoring - canonical measurement records, quality rules, drift evaluation with a re-fit change log, well-test validation, alarm management, model cards, store-and-forward, monthly SLA measurement, FAT/SAT protocols and a dashboard, generated from live objects in the client's reporting outline.
 Author: Daniel T. Murphy
 License: MPL-2.0
@@ -473,10 +473,17 @@ build, drop or hold and by how much (toolface and steering ratio for a
 rotary steerable system), the weight on bit, the rotary speed, the flow
 rate, and whether to drill on, circulate, or ream and wash first - with its
 reasons and the confidence of each choice. A position filter (since
-0.78.1) places the bit in the layer the way a geosteerer does, from the
+0.78.0) places the bit in the layer the way a geosteerer does, from the
 gamma and the deep-resistivity distance against the model, and the screen
 draws the band it puts the layer in. The driller accepts, rejects or
-records what was done instead, by name; every recommendation, answer and
+records what was done instead, by name. The third dimension (since
+0.80.0): the lateral has an azimuth, the agent's sixth choice is where to
+aim it and the recommendation reads as one toolface; the layer is a surface with
+a dip across the plan line as well as along it, the hole is placed by
+minimum curvature through the surveys' azimuths, the corridor the lease
+and the offset wells leave the hole is in the twin and on the screen's
+plan view, and the config's model carries the plan's azimuth, the cross
+dip and the corridor. Every recommendation, answer and
 command is on record with its hash, and GEA pulls the records into the
 well from the **Drill string** tab's card (`gea workspace --path C:\site
 --action driller-pull --box http://192.168.1.60:8790 --well SWD-1`). The
@@ -830,7 +837,7 @@ unsigned, the key hand-overs - and shows the reports as they were signed. A
 reader made from the workspace knows the site's key fingerprints, so it
 says whether a dossier is the site's; it allows no network request, and it
 leaves the trusted time-stamp to `verify_dossier.py`. For a client who wants
-it installed, every release also carries `GEA-Reader-Setup-0.78.1.exe`: GEA
+it installed, every release also carries `GEA-Reader-Setup-0.80.0.exe`: GEA
 Reader as a Windows program of its own - its own icon and taskbar button,
 Check with GEA Reader on a dossier's right-click menu, fingerprints the client
 saves kept on that computer, nothing of GEA-Program. `gea help reader`.
@@ -1595,7 +1602,7 @@ attaches the zips to the GitHub release of that tag.
 
 ### The Windows installer
 
-Every release also carries `GEA-Program-Setup-0.78.1.exe` (for 0.78.1): the
+Every release also carries `GEA-Program-Setup-0.80.0.exe` (for 0.80.0): the
 same Windows kit installed like any Windows program, for the signed-in user
 alone and with no administrator rights (`%LOCALAPPDATA%\Programs\GEA-Program`).
 It asks for the site workspace (an existing one such as `C:\site` is used as
@@ -1614,7 +1621,7 @@ it, removes the program and leaves the workspace. Silent, for a site's
 software deployment:
 
 ```
-GEA-Program-Setup-0.78.1.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /WORKSPACE="C:\site" /SITENAME="Pad 3" /TASKS="desktopicon"
+GEA-Program-Setup-0.80.0.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /WORKSPACE="C:\site" /SITENAME="Pad 3" /TASKS="desktopicon"
 ```
 
 The setup is built by Inno Setup 6 from `tools/installer.iss` in the same

{gea_program-0.78.1 → gea_program-0.80.0}/README.md

@@ -434,10 +434,17 @@ build, drop or hold and by how much (toolface and steering ratio for a
 rotary steerable system), the weight on bit, the rotary speed, the flow
 rate, and whether to drill on, circulate, or ream and wash first - with its
 reasons and the confidence of each choice. A position filter (since
-0.78.1) places the bit in the layer the way a geosteerer does, from the
+0.78.0) places the bit in the layer the way a geosteerer does, from the
 gamma and the deep-resistivity distance against the model, and the screen
 draws the band it puts the layer in. The driller accepts, rejects or
-records what was done instead, by name; every recommendation, answer and
+records what was done instead, by name. The third dimension (since
+0.80.0): the lateral has an azimuth, the agent's sixth choice is where to
+aim it and the recommendation reads as one toolface; the layer is a surface with
+a dip across the plan line as well as along it, the hole is placed by
+minimum curvature through the surveys' azimuths, the corridor the lease
+and the offset wells leave the hole is in the twin and on the screen's
+plan view, and the config's model carries the plan's azimuth, the cross
+dip and the corridor. Every recommendation, answer and
 command is on record with its hash, and GEA pulls the records into the
 well from the **Drill string** tab's card (`gea workspace --path C:\site
 --action driller-pull --box http://192.168.1.60:8790 --well SWD-1`). The
@@ -791,7 +798,7 @@ unsigned, the key hand-overs - and shows the reports as they were signed. A
 reader made from the workspace knows the site's key fingerprints, so it
 says whether a dossier is the site's; it allows no network request, and it
 leaves the trusted time-stamp to `verify_dossier.py`. For a client who wants
-it installed, every release also carries `GEA-Reader-Setup-0.78.1.exe`: GEA
+it installed, every release also carries `GEA-Reader-Setup-0.80.0.exe`: GEA
 Reader as a Windows program of its own - its own icon and taskbar button,
 Check with GEA Reader on a dossier's right-click menu, fingerprints the client
 saves kept on that computer, nothing of GEA-Program. `gea help reader`.
@@ -1556,7 +1563,7 @@ attaches the zips to the GitHub release of that tag.
 
 ### The Windows installer
 
-Every release also carries `GEA-Program-Setup-0.78.1.exe` (for 0.78.1): the
+Every release also carries `GEA-Program-Setup-0.80.0.exe` (for 0.80.0): the
 same Windows kit installed like any Windows program, for the signed-in user
 alone and with no administrator rights (`%LOCALAPPDATA%\Programs\GEA-Program`).
 It asks for the site workspace (an existing one such as `C:\site` is used as
@@ -1575,7 +1582,7 @@ it, removes the program and leaves the workspace. Silent, for a site's
 software deployment:
 
 ```
-GEA-Program-Setup-0.78.1.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /WORKSPACE="C:\site" /SITENAME="Pad 3" /TASKS="desktopicon"
+GEA-Program-Setup-0.80.0.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /WORKSPACE="C:\site" /SITENAME="Pad 3" /TASKS="desktopicon"
 ```
 
 The setup is built by Inno Setup 6 from `tools/installer.iss` in the same

{gea_program-0.78.1 → gea_program-0.80.0}/gea/__init__.py

@@ -123,7 +123,7 @@ from .profile_catalog import (
     read_pangaea_txt,
 )
 
-__version__ = "0.78.1"
+__version__ = "0.80.0"
 __all__ = [
     "aging_rate", "rate_psi_yr", "accuracy_psi",
     "Sensor", "SimulatorConfig", "DownholeEngine",

{gea_program-0.78.1 → gea_program-0.80.0}/gea/__main__.py

@@ -370,7 +370,7 @@ def main(argv=None) -> int:
     p_ws.add_argument("--path", type=str, required=True, help="the workspace folder")
     p_ws.add_argument("--action", type=str, default="list",
                       choices=["init", "add-file", "add-catalog", "add-live", "remove", "list", "migrate", "refresh", "audit",
-                               "add-seismic", "refresh-seismic", "remove-seismic", "sar-film", "bit-image", "field-pull", "driller-pull", "refresh-all", "add-track", "refresh-track", "remove-track",
+                               "add-seismic", "refresh-seismic", "remove-seismic", "sar-film", "bit-image", "field-pull", "driller-pull", "driller-shadow", "refresh-all", "add-track", "refresh-track", "remove-track",
                                "seismic-dataset", "seismic-field", "add-site", "sites", "site-report", "remove-site", "rename",
                                "set-well", "sra-define", "sra-report", "associate", "osdu-export", "quakeml-export", "well-identity", "vibration-report",
                                "add-live-station", "live-fold", "live-run", "month-end", "add-layer", "layers", "remove-layer", "structure", "interval",
@@ -3563,6 +3563,20 @@ def main(argv=None) -> int:
                 if not a.track:
                     raise SystemExit("remove-track needs --track TRACK_ID")
                 ws.remove_track(a.track, actor=a.actor); print("removed (folder kept):", a.track); return 0
+            elif a.action == "driller-shadow":
+                from . import driller_link as DL
+                if not a.well:
+                    raise SystemExit("driller-shadow needs --well Well-30636033 (a well with a drilling-recorder export)")
+                try:
+                    r = DL.shadow(ws, a.well, actor=a.actor)
+                except (ValueError, RuntimeError) as e:
+                    raise SystemExit(f"driller-shadow: {e}")
+                sm = r['summary']
+                print(f"{r['label']} - {r['display']}: {sm['decisions']} decisions over {sm['lateral_ft']:,.0f} ft of lateral; the steer agrees {sm['steer_agree_pct']:.0f}%, opposes {sm['steer_oppose_pct']:.0f}%; "
+                      f"WOB {sm['wob_agree_pct']:.0f}%, rpm {sm['rpm_agree_pct']:.0f}%, flow {sm['flow_agree_pct']:.0f}%; out by the gamma {sm['out_by_gamma_pct']:.0f}% in {sm['exits']} stretch(es), "
+                      f"the advisor called the way back at the first reading {sm['exits_called_at_first_reading']} time(s); the model {'read off the lateral in hindsight' if r['model_derived'] else 'as given'}; "
+                      f"the report: {r['report']['html']}")
+                return 0
             elif a.action == "driller-pull":
                 from . import driller_link as DL
                 if not a.box or not a.well:

{gea_program-0.78.1 → gea_program-0.80.0}/gea/acceptance_tests.py

@@ -16698,7 +16698,7 @@ def section_dv_driller(tmp: str) -> None:
                 e_ = inf["estimate"]; truth = inf["lithology"]
                 guess = max(("above", e_["p_above"]), ("in", e_["p_in"]), ("below", e_["p_below"]), key=lambda x: x[1])[0]
                 conf[truth][guess] += 1; n_steps += 1
-                errs.append(abs(e_["d_top"] - (lt.tvd - sc.layer_top(lt.vs))))
+                errs.append(abs(e_["d_top"] - (lt.tvd - sc.layer_top(lt.vs, getattr(lt, "y", 0.0)))))
     right = sum(conf[k][k] for k in conf); total = sum(sum(v.values()) for v in conf.values())
     # a filter told nothing but the gamma of a flat layer sits where the model says; told a boundary, it moves
     f1 = PF.PositionFilter(9000.0, 0.0, 40.0, 60.0, 130.0, 115.0, seed=3)
@@ -16730,6 +16730,287 @@ def section_dv_driller(tmp: str) -> None:
        "card, the workspace command, the VPN kind, the kit and the workflow, and the help page carry it" + (f" [failed: {', '.join(failed4)}]" if failed4 else ""))
 
 
+def section_dw_shadow(tmp: str) -> None:
+    """Section DW - shadow mode: a drilled lateral replayed through the advisor against the record. A lateral drilled
+    in the twin by the agent itself, written as the driller's lateral file: the heel found below a vertical
+    approach, the model read off the gamma in hindsight close to the twin's own layer, one decision per 30 ft, the
+    agent agreeing with its own record more often than not, the exits the gamma marks called at the first reading,
+    the report in three files labelled SHADOW MODE; in GEA, a drilling-recorder well shadowed as a job from the
+    card, the lateral file written from the export beside the well, the report under the well's reports, the
+    audit line, the refusals and the help."""
+    import http.cookiejar
+    import importlib
+    import urllib.error
+    import urllib.request
+    from . import driller_link as DL
+    from . import edr as E
+    from . import helplib as H
+    from .service import Service, Users
+    from .workspace import Workspace
+    pkg = Path(__file__).parent
+    d = Path(tmp, "dw"); d.mkdir()
+    prog = DL.program()
+    if prog is None:
+        ok(True, "DW the driller's program (gea-driller) is not installed beside GEA in this install; nothing to shadow here")
+        return
+    if prog["env"] and prog["env"] not in sys.path:
+        sys.path.insert(0, prog["env"])
+    SH = importlib.import_module("gea_driller.shadow"); P = importlib.import_module("gea_driller.policy"); S = importlib.import_module("gea_driller.service")
+    np_ = importlib.import_module("numpy")
+    pol = P.Policy.load(S.SHIPPED_POLICY)
+    # -- DW1 the replay on a lateral whose layer is known ------------------------------------------------------------------
+    lat_csv = str(d / "lat.csv")
+    info = SH.twin_lateral(lat_csv, seed=7, act_fn=P.policy_act_fn(pol))
+    lat = SH.read_lateral(lat_csv)
+    heel = SH.find_heel(lat["md_ft"], lat["tvd_ft"])
+    res = SH.run(lat_csv, pol, out_dir=str(d / "out"), well="Twin 7", site="Pad 3 shadow")
+    sc = info["scene"]; m = res["model"]; sm = res["summary"]
+    bad_cols = str(d / "bad.csv")
+    Path(bad_cols).write_text("md_ft,tvd_ft\n1,2\n", encoding="utf-8")
+    try:
+        SH.read_lateral(bad_cols); cols_refused = False
+    except ValueError:
+        cols_refused = True
+    given = SH.run(lat_csv, pol, model={"layer_top_tvd_ft": sc["layer_top_heel_ft"], "thickness_ft": sc["thickness_ft"], "dip_deg": sc["dip_deg"], "gamma_in": sc["gamma_in_above_below"][0],
+                                        "gamma_above": sc["gamma_in_above_below"][1], "gamma_below": sc["gamma_in_above_below"][2], "heel_md_ft": info["heel_md"]}, well="Twin 7")
+    md_text = Path(res["paths"]["md"]).read_text(encoding="utf-8"); html_text = Path(res["paths"]["html"]).read_text(encoding="utf-8")
+    c1 = {
+        "heel": -150.0 <= lat["md_ft"][heel] - info["heel_md"] <= 30.0,          # found in the last of the build, never past the heel
+        "model": abs(m["layer_top_tvd_ft"] - sc["layer_top_heel_ft"]) <= 40.0 and 0.5 <= m["thickness_ft"] / sc["thickness_ft"] <= 1.6 and abs(m["dip_deg"] - sc["dip_deg"]) <= 1.0 and res["model_derived"],
+        "decisions": abs(sm["decisions"] - sm["lateral_ft"] / 30.0) <= 4 and sm["decisions"] >= 100,
+        "self_agreement": sm["steer_agree_pct"] >= 35.0 and sm["steer_oppose_pct"] <= 30.0 and sm["steer_agree_pct"] > sm["steer_oppose_pct"],
+        "exits": sm["exits"] >= 1 and sm["exits_called_at_first_reading"] >= 1,
+        "given_model": not given["model_derived"] and given["model"]["thickness_ft"] == sc["thickness_ft"] and given["summary"]["model_fit"] == "given" and sm["model_fit"] in ("good", "fair"),
+        "report": all(Path(res["paths"][k]).is_file() for k in ("json", "md", "html")) and "SHADOW MODE" in md_text and "SHADOW MODE" in html_text and "<svg" in html_text
+                  and "Not the pre-drill model" in md_text and "nothing here touched the well" in md_text.lower(),
+        "refusal": cols_refused,
+    }
+    failed1 = [k for k, v in c1.items() if not v]
+    ok(not failed1,
+       f"DW1 a lateral drilled in the twin by the agent, replayed: the heel found below the vertical approach, the model read off the gamma in hindsight within "
+       f"40 ft, a factor and a degree of the twin's own layer, one decision per 30 ft ({sm['decisions']}), the agent agreeing with its own record "
+       f"{sm['steer_agree_pct']:.0f}% and opposing it {sm['steer_oppose_pct']:.0f}%, the gamma's exits called at the first reading, a given model taken as given, "
+       "the report in three files labelled SHADOW MODE with the drawing, a lateral file without the columns refused" + (f" [failed: {', '.join(failed1)}]" if failed1 else ""))
+    # -- DW2 GEA: a drilling-recorder well shadowed from the card -----------------------------------------------------------
+    ws = Workspace.create(str(d / "ws"), "Pad 3 shadow", actor="tester")
+    exp = E.selftest(str(d), 3000)
+    w = ws.add_well_file(exp["path"], display="Well EDR", actor="tester")
+    w2 = ws.add_well_records("Paper well", actor="tester")
+    rec = DL.shadow(ws, w["id"], actor="tester")
+    try:
+        DL.shadow(ws, w2["id"], actor="tester"); no_edr_refused = False
+    except ValueError:
+        no_edr_refused = True
+    Users(str(Path(ws.path, "users.json"))).add("op", "op-pass-123", "operator")
+    svc = Service(ws.path, port=0, scheduler=False).start()
+    base = svc.url.rstrip("/")
+    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
+
+    def gcall(method, path, body=None):
+        rq = urllib.request.Request(base + path, method=method, data=(json.dumps(body).encode() if body is not None else None))
+        rq.add_header("Content-Type", "application/json"); rq.add_header("X-GEA-Action", "1")
+        try:
+            with opener.open(rq, timeout=60) as rr_:
+                return rr_.status, json.loads(rr_.read())
+        except urllib.error.HTTPError as e:
+            return e.code, json.loads(e.read() or b"{}")
+    try:
+        gcall("POST", "/api/login", {"name": "op", "password": "op-pass-123"})
+        s_d, dd = gcall("GET", "/api/driller")
+        s_bad, _ = gcall("POST", "/api/driller/shadow", {"well": w2["id"]})
+        s_job, job = gcall("POST", "/api/driller/shadow", {"well": w["id"]})
+    finally:
+        svc.stop()
+    app_src = (pkg / "web" / "app.html").read_text(encoding="utf-8")
+    main_src = (pkg / "__main__.py").read_text(encoding="utf-8")
+    c2 = {
+        "lateral_file": Path(ws.path, "wells", w["id"], "driller", "lateral.csv").is_file() and rec["lateral"]["rows"] >= 1000 and rec["lateral"]["gamma_rows"] > 0,
+        "report": all(Path(ws.reports_dir, rec["report"][k]).is_file() for k in ("html", "md", "json")) and rec["label"] == "SHADOW MODE" and rec["summary"]["decisions"] >= 20 and rec["model_derived"],
+        "record": DL.last_shadow(ws, w["id"]) is not None and [x["well"] for x in DL.shadow_wells(ws)] == [w["id"]],
+        "refusal": no_edr_refused,
+        "audit": any(a["action"] == "driller.shadow" and a["detail"]["label"] == "SHADOW MODE" for a in ws.audit_log()),
+        "routes": s_d == 200 and [x["well"] for x in dd["shadow"]] == [w["id"]] and dd["shadow"][0]["shadow"]["summary"]["decisions"] >= 20 and s_bad == 400 and s_job == 200 and "id" in job,
+        "card": all(t_ in app_src for t_ in ("Shadow a drilled well", "/api/driller/shadow", "Shadow mode")),
+        "cli": '"driller-shadow"' in main_src and "DL.shadow" in main_src,
+        "help": not H.check("driller") and "SHADOW MODE" in (pkg / "help" / "driller.md").read_text(encoding="utf-8"),
+    }
+    failed2 = [k for k, v in c2.items() if not v]
+    ok(not failed2,
+       "DW2 in GEA a drilling-recorder well is shadowed: the lateral file written from the export beside the well, the driller's program run on it, the report "
+       "under the well's reports labelled SHADOW MODE, the run recorded and audited; a well without an export is refused; the route runs it as a job and "
+       "refuses the wrong well; the card, the workspace command and the help page carry it" + (f" [failed: {', '.join(failed2)}]" if failed2 else ""))
+
+
+def section_dx_third_dimension(tmp: str) -> None:
+    """Section DX - the third dimension of GEA Driller: the directional arithmetic (minimum curvature, the dogleg of a
+    build and a turn, the toolface), the twin in the plan's frame (the turn, the layer as a surface, the corridor,
+    an older five-choice action), the filter's cross dip; the rig side (the bit placed from surveys with azimuths,
+    the twin as the rig publishing the azimuth, a decision with the turn and the toolface, an override with a turn,
+    the limit on both together, the config's refusals, the screen, the help); the policy with six heads holding the
+    corridor on the held-out scenes, and shadow mode judging the turn with the record's azimuth and not without.
+    The sixth choice is where to aim the azimuth (left of the plan, along it, right of it); the tool turns toward it
+    at its own rate, as a directional driller holds the azimuth he is given."""
+    import importlib
+    import math as _m
+    from . import driller_link as DL
+    from . import helplib as H
+    pkg = Path(__file__).parent
+    d = Path(tmp, "dx"); d.mkdir()
+    prog = DL.program()
+    if prog is None:
+        ok(True, "DX the driller's program (gea-driller) is not installed beside GEA in this install; nothing to check here")
+        return
+    if prog["env"] and prog["env"] not in sys.path:
+        sys.path.insert(0, prog["env"])
+    T = importlib.import_module("gea_driller.twin"); G = importlib.import_module("gea_driller.geometry"); PF = importlib.import_module("gea_driller.position")
+    P = importlib.import_module("gea_driller.policy"); A = importlib.import_module("gea_driller.advisor"); C = importlib.import_module("gea_driller.config")
+    S = importlib.import_module("gea_driller.service"); R = importlib.import_module("gea_driller.rig"); SH = importlib.import_module("gea_driller.shadow")
+    np_ = importlib.import_module("numpy")
+    # -- DX1 the arithmetic, the twin, the filter -----------------------------------------------------------------------------
+    dn, de, dv, dl = G.min_curvature(90.0, 30.0, 90.0, 30.0, 1000.0)
+    _, _, dv_b, dl_b = G.min_curvature(90.0, 0.0, 91.5, 0.0, 100.0)
+    _, _, _, dl_t = G.min_curvature(90.0, 0.0, 90.0, 2.0, 100.0)
+    _, _, _, dl_t60 = G.min_curvature(60.0, 0.0, 60.0, 2.0, 100.0)
+    lh = T.Lateral(T.Scene(3)); lh.reset(); lt = T.Lateral(T.Scene(3)); lt.reset(); l5 = T.Lateral(T.Scene(3)); l5.reset()
+    for _ in range(8):
+        lh.step(T.HOLD_ACTION); lt.step((2, 1, 1, 1, 0, 2)); l5.step((2, 1, 1, 1, 0))
+    sc = T.Scene(3)
+    cross = (sc.layer_top(500.0, 100.0) - sc.layer_top(500.0, 0.0)) - 100.0 * _m.tan(_m.radians(sc.cross_dip_deg))
+    # a hole aimed right of the plan and never brought back leaves the corridor and is lost
+    lw = T.Lateral(T.Scene(9)); lw.scene.corridor_ft = 50.0; lw.reset()
+    while not lw.done and lw.steps < 400:
+        lw.step((2, 1, 1, 1, 0, 2))
+    try:
+        T.full_action((1, 2, 3)); short_refused = False
+    except ValueError:
+        short_refused = True
+    # the filter: with a cross dip in the model, the top moves with the offset; the cross dip is read only across the plan line
+    f_along = PF.PositionFilter(9000.0, 0.0, 40.0, 60.0, 130.0, 115.0, seed=3, model_cross_dip_deg=2.0)
+    f_across = PF.PositionFilter(9000.0, 0.0, 40.0, 60.0, 130.0, 115.0, seed=3, model_cross_dip_deg=2.0)
+    for k in range(40):
+        f_along.predict(30.0, 0.0); f_along.update(9020.0, 60.0)                      # along the plan: the gamma in the layer, nothing learnt of the cross dip
+        f_across.predict(0.0, 10.0); f_across.update(9020.0 + 10.0 * (k + 1) * _m.tan(_m.radians(1.0)), 62.0, 4.0)   # across: the top tracks a real cross dip of 1 degree, the tool says so
+    e_along = f_along.estimate(9020.0); e_across = f_across.estimate(9020.0 + 400.0 * _m.tan(_m.radians(1.0)))
+    top_shift = float(f_along.model_top(0.0, 100.0) - f_along.model_top(0.0, 0.0))
+    c1 = {
+        "straight": abs(dn - 1000.0 * _m.cos(_m.radians(30.0))) < 1e-6 and abs(de - 500.0) < 1e-6 and abs(dv) < 1e-6 and abs(dl) < 1e-9,
+        "dogleg": abs(dl_b - 1.5) < 1e-9 and abs(dl_t - 2.0) < 1e-6 and abs(dl_t60 - 2.0 * _m.sin(_m.radians(60.0))) < 0.02 and abs(dv_b + 100.0 * _m.sin(_m.radians(0.75))) < 0.05,
+        "toolface": abs(G.toolface_of(1.5, 0.0, 90.0)) < 1e-9 and abs(G.toolface_of(0.0, 2.0, 90.0) - 90.0) < 1e-9 and abs(G.toolface_of(-1.5, 0.0, 90.0) - 180.0) < 1e-9
+                    and abs(G.toolface_of(0.0, -2.0, 90.0) - 270.0) < 1e-9 and abs(G.dls_of(3.0, 4.0, 90.0) - 5.0) < 1e-9,
+        "turns": lt.az > lh.az + 1.5 and lt.y > lh.y + 5.0 and 2.5 < lt.az - lh.az < 6.0 and abs(T.turn_toward(4.0, 0.0) - 2.0) < 1e-9 and abs(T.turn_toward(0.0, 0.5) + 0.75) < 1e-9,
+        "surface": abs(cross) < 1.0 + sc.und_amp and T.Scene(1).cross_dip_deg != T.Scene(2).cross_dip_deg and "cross_dip_deg" in sc.describe() and "corridor_ft" in sc.describe(),
+        "lost": lw.done and lw.ended == "outside the corridor" and lw.summary()["lost"] and lw.max_offset > lw.scene.corridor_ft + T.CORRIDOR_END_FT and any("outside the corridor" in e for e in lw.events),
+        "five_choices": abs(l5.az - lh.az) < 1e-9 and abs(l5.y - lh.y) < 1e-9 and short_refused and len(T.ACTION_SIZES) == 6 and T.ACTION_NAMES[-1] == "aim",
+        "obs": len(T.OBS_NAMES) == T.OBS_SIZE == 34 == lt.observe().shape[0] and all(np_.isfinite(lt.observe())) and "offset_right_of_plan" in T.OBS_NAMES,
+        "filter": abs(top_shift - 100.0 * _m.tan(_m.radians(2.0))) < 1e-9 and abs(e_along["cross_dip_deg"] - 2.0) < 0.3 and e_along["cross_dip_sigma"] > 0.3
+                  and abs(e_across["cross_dip_deg"] - 1.0) < 0.6 and e_across["cross_dip_sigma"] < e_along["cross_dip_sigma"],
+    }
+    failed1 = [k for k, v in c1.items() if not v]
+    ok(not failed1,
+       "DX1 minimum curvature lands a straight hole 30° off the plan where the sine says, a build's dogleg is the build and a turn's counts by the inclination's sine, "
+       "the toolface of a build is up and of a turn is right; in the twin an aim right of the plan turns the hole toward it at the tool's rate and swings it to the "
+       "right, the layer's top moves across the plan line by the cross dip, a hole aimed out of the corridor is lost and says so, a five-choice action is taken aimed "
+       "along the plan and a shorter one refused, "
+       "the observation is 34 named and finite; the filter moves the model's top by its cross dip and reads the cross dip only across the plan line"
+       + (f" [failed: {', '.join(failed1)}]" if failed1 else ""))
+    # -- DX2 the rig side --------------------------------------------------------------------------------------------------------
+    shipped = P.Policy.load(S.SHIPPED_POLICY)
+    cfg = json.loads(json.dumps(C.EXAMPLE)); cfg["output"]["dir"] = str(d / "rig"); cfg["status"]["port"] = 0; cfg["status"]["bind"] = "127.0.0.1"
+    cfg["rig"]["transport"] = "sim"; cfg["model"]["plan_azimuth_deg"] = 40.0; cfg["model"]["corridor_ft"] = 120.0; cfg["model"]["cross_dip_deg"] = 1.0
+    cfg = C.check(cfg)
+    heel_tvd0 = float(cfg["model"]["heel_tvd_ft"])                    # the twin as the rig rewrites the model block in place below
+    try:
+        C.check(json.loads(json.dumps(cfg)) | {"model": {**cfg["model"], "cross_dip_deg": 20.0}}); cross_refused = False
+    except C.ConfigError:
+        cross_refused = True
+    try:
+        C.check(json.loads(json.dumps(cfg)) | {"model": {**cfg["model"], "corridor_ft": 2.0}}); corridor_refused = False
+    except C.ConfigError:
+        corridor_refused = True
+    # the bit placed from surveys: a feed that holds 90 degrees at the plan's azimuth plus 30 for 600 ft
+    feed = R.Feed(cfg); feed.state.update({"hole_depth": 10800.0, "gamma": 60.0, "continuous_inc": 90.0, "continuous_azi": 70.0}); feed.last_utc = time.time(); feed.connected = True
+    for md_ in (10200.0, 10290.0, 10380.0, 10470.0, 10560.0, 10650.0):
+        feed.add_survey(md_, 90.0, 70.0, "test")
+    adv = A.Advisor(cfg, shipped, feed, A.Records(cfg))
+    obs_, ctx = adv.observation(feed.snapshot())
+    rec = adv.decide(feed.snapshot())
+    words_turn = adv._words(1.5, 25.0, 120.0, 500.0, "drill", {"inc": 90.0, "bed": 0.1, "balling": 0.0, "plan_az": 40.0}, {}, 2.0)
+    words_aim = adv._words(1.5, 25.0, 120.0, 500.0, "drill", {"inc": 90.0, "bed": 0.1, "balling": 0.0, "plan_az": 40.0}, {}, 2.0, 4.0)
+    words_hold = adv._words(0.0, 25.0, 120.0, 500.0, "drill", {"inc": 90.0, "bed": 0.1, "balling": 0.0, "plan_az": 40.0}, {}, 0.0, 0.0)
+    ov = adv._action_of({"turn_deg_100ft": -2.0, "build_deg_100ft": 1.5}, [2, 1, 1, 1, 0])
+    # the limit holds the build and the turn together: a 4 deg/100 ft limit on a 3 build with a 2 turn clips both
+    cfg_l = json.loads(json.dumps(cfg)); cfg_l["limits"]["max_dls_deg_100ft"] = 3.0
+    adv_l = A.Advisor(C.check(cfg_l), shipped, feed, A.Records(cfg_l))
+    rl = adv_l.decide(feed.snapshot())
+    # the twin as the rig publishes the azimuth and the plan view has what it needs
+    drl = S.Driller(cfg, policy=shipped, twin_seed=11, twin_pace_s=0.5)
+    drl.start()
+    try:
+        t_end = time.monotonic() + 20
+        while time.monotonic() < t_end and len(drl.advisor.history) < 2:
+            time.sleep(0.2)
+        live = drl.live(); snap = drl.feed.snapshot()
+        import urllib.request
+        with urllib.request.urlopen(drl.url + "/live", timeout=5) as r_:
+            screen = r_.read().decode("utf-8")
+        cur = dict(drl.advisor.history[-1]) if drl.advisor.history else None
+    finally:
+        drl.stop()
+    sv_text = Path(cfg["output"]["dir"], "surveys.csv").read_text(encoding="utf-8")
+    c2 = {
+        "placed": abs(ctx["vs"] - 600.0 * _m.cos(_m.radians(30.0))) < 0.5 and abs(ctx["y"] - 600.0 * _m.sin(_m.radians(30.0))) < 0.5 and abs(ctx["az"] - 30.0) < 1e-9
+                  and abs(ctx["tvd"] - heel_tvd0) < 0.5 and ctx["corridor"] == 120.0 and obs_.shape[0] == T.OBS_SIZE and abs(obs_[29] * 50.0 - ctx["y"]) < 1e-6,
+        "decision": rec is not None and rec["recommend"].get("aim_deg") in T.AIM_CHOICES and abs(rec["recommend"]["turn_deg_100ft"]) <= T.TURN_RATE + 1e-9
+                    and abs(rec["recommend"]["turn_deg_100ft"] - T.turn_toward(rec["recommend"]["aim_deg"], 30.0)) < 1e-6 and rec["recommend"]["steer_az"] in ("hold", "turn right", "turn left")
+                    and abs(rec["recommend"]["aim_azimuth_deg"] - (40.0 + rec["recommend"]["aim_deg"])) < 1e-9
+                    and 0.0 <= rec["recommend"]["toolface_deg"] < 360.0 and any("plan line" in r_ for r_ in rec["reasons"]) and "aim" in " ".join(rec["reasons"]) and len(rec["action"]) == 6,
+        "words": "turn right 2.0°/100 ft" in words_turn and "toolface 53°" in words_turn and "Build +1.5" in words_turn and words_hold.startswith("Hold angle and azimuth")
+                 and "aim 4° right of the plan (azimuth 44.0°), turning right 2.0°/100 ft" in words_aim,
+        "override": ov == (3, 1, 1, 1, 0, 0),
+        "limit": rl is not None and G.dls_of(rl["recommend"]["build_deg_100ft"], rl["recommend"]["turn_deg_100ft"], 90.0) <= 3.0 + 1e-9,
+        "refusals": cross_refused and corridor_refused,
+        "twin_rig": snap["state"].get("continuous_azi") is not None and live["plan_azimuth_deg"] is not None and live["model"]["corridor_ft"] > 0 and live["path"] and "y" in live["path"][-1]
+                    and live["advisor"]["position"] is not None and "true_y" in live["twin"] and cur is not None and "turn_deg_100ft" in cur["recommend"]
+                    and all(s_.get("azi") is not None for s_ in snap["surveys"]),
+        "screen": "The plan view" in screen and "drawPlan" in screen and "turn_deg_100ft" in screen and "off the plan line" in screen and 'placeholder="aim' in screen,
+        "surveys_file": "azimuth_deg" in sv_text.splitlines()[0] and len(sv_text.splitlines()) >= 2 and sv_text.splitlines()[1].split(",")[2] not in ("", "0.00"),
+        "help": not H.check("driller") and "The third dimension" in (pkg / "help" / "driller.md").read_text(encoding="utf-8") and "plan view" in (pkg / "help" / "tutorial" / "28-driller-office.md").read_text(encoding="utf-8"),
+    }
+    failed2 = [k for k, v in c2.items() if not v]
+    ok(not failed2,
+       "DX2 the advisor places the bit from surveys 30° off the plan where the arithmetic says (along, right, depth), a decision carries the aim, the turn it asks "
+       "from where the hole points and the toolface, with the azimuth and the corridor among its reasons, the words name a build and an aim as one toolface, an "
+       "override takes the aim or the turn made, the DLS limit "
+       "holds the build and the turn together, a cross dip and a corridor out of range are refused, the twin as the rig publishes the azimuth and surveys with it, "
+       "the screen carries the plan view, and the help and the tutorial say so" + (f" [failed: {', '.join(failed2)}]" if failed2 else ""))
+    # -- DX3 the policy and shadow mode -------------------------------------------------------------------------------------------
+    ev = P.evaluate(P.policy_act_fn(shipped), range(20), "agent"); tb = P.evaluate(T.textbook_action, range(20), "textbook")
+    lat_az = str(d / "lat_az.csv"); lat_no = str(d / "lat_no.csv")
+    info = SH.twin_lateral(lat_az, seed=5, act_fn=P.policy_act_fn(shipped)); SH.twin_lateral(lat_no, seed=5, act_fn=P.policy_act_fn(shipped), with_azimuth=False)
+    sc5 = info["scene"]
+    model5 = {"layer_top_tvd_ft": sc5["layer_top_heel_ft"], "thickness_ft": sc5["thickness_ft"], "dip_deg": sc5["dip_deg"], "cross_dip_deg": sc5["cross_dip_deg"],
+              "gamma_in": sc5["gamma_in_above_below"][0], "gamma_above": sc5["gamma_in_above_below"][1], "gamma_below": sc5["gamma_in_above_below"][2],
+              "heel_md_ft": info["heel_md"], "plan_azimuth_deg": sc5["plan_azimuth_deg"], "corridor_ft": sc5["corridor_ft"]}
+    r_az = SH.run(lat_az, shipped, model=model5, out_dir=str(d / "sh_az"), well="Twin 5"); r_no = SH.run(lat_no, shipped, model=model5, well="Twin 5")
+    md_az = Path(r_az["paths"]["md"]).read_text(encoding="utf-8")
+    c3 = {
+        "heads": list(shipped.to_json()["action_sizes"]) == list(T.ACTION_SIZES) and shipped.to_json()["obs_size"] == 34 and int(shipped.meta.get("trained_steps", 0)) >= 1000000,
+        "corridor": ev["corridor_pct"] >= 90.0 and ev["lost"] == 0 and ev["max_offset_ft"] < 200.0,
+        "textbook_corridor": tb["corridor_pct"] >= 95.0 and tb["lost"] == 0,
+        "layer": ev["in_layer_pct"] > tb["in_layer_pct"] and ev["reward"] > tb["reward"],
+        "shadow_turn": r_az["summary"]["azimuth_in_record"] and r_az["summary"]["turn_agree_pct"] is not None and r_az["summary"]["turn_agree_pct"] >= 30.0
+                       and (r_az["summary"]["turn_oppose_pct"] or 0.0) <= 30.0 and "The turn (right, hold or left" in md_az,
+        "shadow_without": not r_no["summary"]["azimuth_in_record"] and r_no["summary"]["turn_agree_pct"] is None and all(x["actual"]["turn"] is None for x in r_no["decisions"]),
+        "model_through": r_az["model"]["plan_azimuth_deg"] == sc5["plan_azimuth_deg"] and r_az["model"]["cross_dip_deg"] == sc5["cross_dip_deg"],
+    }
+    failed3 = [k for k, v in c3.items() if not v]
+    ok(not failed3,
+       f"DX3 the shipped policy has six heads over 34 observations; on twenty held-out scenes it keeps {ev['corridor_pct']:.0f}% of the hole in the corridor "
+       f"(none lost, {ev['in_layer_pct']:.0f}% in the layer against the textbook's {tb['in_layer_pct']:.0f}%, SIMULATION) and the textbook holds the corridor; shadow mode "
+       f"judges the turn from a record with the azimuth ({r_az['summary']['turn_agree_pct'] or 0:.0f}% agreeing) and leaves it unjudged without one, and a given model's "
+       "plan azimuth and cross dip go through" + (f" [failed: {', '.join(failed3)}]" if failed3 else ""))
+
+
 def main() -> int:
     print("GEA-Program - ACCEPTANCE SUITE (the product gate)")
     # a handle a job subprocess still holds at the end must not turn a finished gate into a traceback on Windows:
@@ -16859,6 +17140,8 @@ def main() -> int:
         section_dt_vpn(tmp)
         section_du_tutorial(tmp)
         section_dv_driller(tmp)
+        section_dw_shadow(tmp)
+        section_dx_third_dimension(tmp)
     if _FAILS:
         print(f"[ACCEPTANCE] {len(_FAILS)} FAILURES ({_PASS} passed):")
         for f in _FAILS:

{gea_program-0.78.1 → gea_program-0.80.0}/gea/driller_link.py

@@ -276,3 +276,131 @@ def wells_with_records(ws) -> List[dict]:
         if lp:
             out.append({'well': w['id'], 'display': w.get('display', w['id']), 'pulled_utc': lp['pulled_utc'], 'source': lp['source'], 'summary': lp['summary'], 'rig_well': lp.get('rig_well')})
     return out
+
+
+# -- shadow mode: a drilled well of this workspace replayed through the advisor ---------------------------------------------
+
+LATERAL_COLUMNS = ('md_ft', 'tvd_ft', 'gamma_api', 'wob_klb', 'rpm', 'flow_gpm', 'rop_ft_hr', 'spp_psi', 'hours')
+
+
+def lateral_from_edr(ws, well_id: str, out_csv: str) -> dict:
+    """The driller's lateral file from a well's drilling-recorder export: one row per foot of hole made while on
+    bottom - the first second the hole reached that foot - with the gamma as the median of the seconds since the
+    foot before, and the rig's parameters at that second. Written beside the well; the export is read, not copied."""
+    import csv as _csv
+    from . import edr as E
+    well = next((w for w in ws.wells() if w['id'] == well_id), None)
+    if well is None:
+        raise ValueError(f'no well {well_id}')
+    src = next((os.path.join(ws.path, 'wells', well_id, 'source', f) for f in well.get('files', []) if f.lower().endswith('.csv')), None)
+    if not src or not E.is_edr(src):
+        raise ValueError(f"{well_id} has no drilling-recorder export (an EDR CSV) under its sources")
+    st = E.read(src)
+    ch = {k: E._ch(st, k) for k in ('hole', 'tvd', 'gamma', 'wob', 'rpm', 'flow', 'rop', 'spp', 'onb')}
+    hole = ch['hole']
+    if hole is None or ch['tvd'] is None:
+        raise ValueError('the export carries no hole depth or no interpolated TVD; the lateral cannot be placed')
+    # the depth channel read for what it can be trusted for, as the drilling record reads it: a section that opens on
+    # another well's depth, a re-reference, a sidetrack's abandoned hole are left out
+    de = E.depth_events(st)
+    keep_h, _ = E._keep_masks(st, de)
+    secs = st.index
+    n = len(hole)
+    rows = []
+    seen = set(); g_acc = []
+    for i in range(n):
+        h = hole[i]
+        if h is None or (isinstance(h, float) and (h != h)) or not keep_h[i]:
+            continue
+        if ch['gamma'] is not None and not (ch['gamma'][i] != ch['gamma'][i]):
+            g_acc.append(float(ch['gamma'][i]))
+        foot = int(h)
+        if foot in seen:
+            continue
+        seen.add(foot)
+        tvd = ch['tvd'][i]
+        rows.append({'md_ft': float(h), 'tvd_ft': float(tvd) if not (tvd != tvd) else float('nan'),
+                     'gamma_api': float(sorted(g_acc)[len(g_acc) // 2]) if g_acc else float('nan'),
+                     'wob_klb': _v(ch['wob'], i), 'rpm': _v(ch['rpm'], i), 'flow_gpm': _v(ch['flow'], i), 'rop_ft_hr': _v(ch['rop'], i), 'spp_psi': _v(ch['spp'], i),
+                     'hours': float(secs[i]) / 3600.0})
+        g_acc = []
+    if len(rows) < 100:
+        raise ValueError(f'the export yields only {len(rows)} feet of new hole; a lateral is thousands')
+    rows.sort(key=lambda r: r['md_ft'])
+    os.makedirs(os.path.dirname(out_csv), exist_ok=True)
+    with open(out_csv, 'w', encoding='utf-8', newline='') as f:
+        w = _csv.DictWriter(f, fieldnames=list(LATERAL_COLUMNS)); w.writeheader()
+        for r in rows:
+            w.writerow({c: ('' if (isinstance(r[c], float) and r[c] != r[c]) or r[c] is None else f'{r[c]:.3f}') for c in LATERAL_COLUMNS})
+    rows.sort(key=lambda r: r['md_ft'])
+    return {'rows': len(rows), 'source': src, 'md_from': rows[0]['md_ft'], 'md_to': rows[-1]['md_ft'], 'gamma_rows': sum(1 for r in rows if r['gamma_api'] == r['gamma_api']),
+            'seconds': int(n), 'excluded_rows': int((~keep_h).sum()), 'depth_events': {k: len(v) for k, v in de.items() if isinstance(v, list)}}
+
+
+def _v(arr, i):
+    if arr is None:
+        return float('nan')
+    x = arr[i]
+    return float(x) if not (x != x) else float('nan')
+
+
+def shadow(ws, well_id: str, actor: str = 'system', model: Optional[dict] = None, timeout_s: float = 1800.0) -> dict:
+    """SHADOW MODE on a well of this workspace: the lateral file from its export, the driller's own program run
+    on it as a process of its own, the report kept under the well's reports and the summary in the driller folder."""
+    from . import __version__
+    prog = program()
+    if prog is None:
+        raise ValueError("the driller's program (gea-driller) is not installed beside GEA: the kit and the setup carry it")
+    well = next((w for w in ws.wells() if w['id'] == well_id), None)
+    if well is None:
+        raise ValueError(f'no well {well_id}')
+    d = ws.dir('wells', well_id, 'driller')
+    lat_csv = os.path.join(d, 'lateral.csv')
+    lat = lateral_from_edr(ws, well_id, lat_csv)
+    if model is None and os.path.isfile(os.path.join(d, 'model.json')):       # the site's pre-drill model, kept beside the well
+        with open(os.path.join(d, 'model.json'), encoding='utf-8') as f:
+            model = json.load(f)
+        model = model.get('model', model)
+    out = ws.dir('reports', 'wells', well_id, 'driller_shadow')
+    cmd = [sys.executable, '-m', 'gea_driller', 'shadow', '--lateral', lat_csv, '--out', out, '--well', str(well.get('display', well_id)), '--site', str(ws.manifest.get('name') or 'site'), '--json']
+    if model:
+        mp = os.path.join(d, 'model.json')
+        if not os.path.isfile(mp):
+            with open(mp, 'w', encoding='utf-8') as f:
+                json.dump({'model': model}, f, indent=1)
+        cmd += ['--model', mp]
+    env = dict(os.environ)
+    if prog['env']:
+        env['PYTHONPATH'] = prog['env'] + (os.pathsep + env['PYTHONPATH'] if env.get('PYTHONPATH') else '')
+    r = subprocess.run(cmd, capture_output=True, text=True, env=env, timeout=timeout_s)
+    if r.returncode != 0:
+        raise RuntimeError(f"the driller's shadow run failed: {(r.stderr or r.stdout).strip()[-600:]}")
+    res = json.loads(r.stdout)
+    rec = {'well': well_id, 'display': well.get('display', well_id), 'run_utc': res['run_utc'], 'by': actor, 'label': res['label'], 'driller_version': res['version'],
+           'lateral': lat, 'model': res['model'], 'model_derived': res['model_derived'], 'summary': res['summary'], 'exits': res['exits'][:50],
+           'report': {k: os.path.relpath(v, ws.reports_dir) for k, v in res['paths'].items()}, 'gea_version': __version__}
+    with open(os.path.join(d, 'shadow.json'), 'w', encoding='utf-8') as f:
+        json.dump(rec, f, indent=1)
+    ws.audit(actor, 'driller.shadow', {'well': well_id, 'decisions': res['summary']['decisions'], 'steer_agree_pct': round(res['summary']['steer_agree_pct'], 1),
+                                       'model_derived': res['model_derived'], 'label': res['label']}, inputs=[lat['source']])
+    return rec
+
+
+def last_shadow(ws, well_id: str) -> Optional[dict]:
+    p = os.path.join(ws.path, 'wells', well_id, 'driller', 'shadow.json')
+    try:
+        with open(p, encoding='utf-8') as f:
+            return json.load(f)
+    except (OSError, ValueError):
+        return None
+
+
+def shadow_wells(ws) -> List[dict]:
+    """The wells a shadow run can take (a drilling-recorder export among their sources), with the last run if any."""
+    from . import edr as E
+    out = []
+    for w in ws.wells():
+        srcs = [os.path.join(ws.path, 'wells', w['id'], 'source', f) for f in w.get('files', []) if f.lower().endswith('.csv')]
+        if any(os.path.isfile(p) and E.is_edr(p) for p in srcs):
+            out.append({'well': w['id'], 'display': w.get('display', w['id']), 'shadow': last_shadow(ws, w['id'])})
+    return out

{gea_program-0.78.1 → gea_program-0.80.0}/gea/help/IT_SECURITY.md

@@ -180,7 +180,10 @@ declared extras.
   feed, a crossed limit or the section's end; this version's only downlink
   keeps every command on record and sends none. The records carry no
   secret; GEA pulls them checked against their hashes. The twin as the rig
-  on the office PC binds to 127.0.0.1 only.
+  on the office PC binds to 127.0.0.1 only. Since 0.80.0 the records carry
+  the hole's azimuth and its offset from the plan line, as the surveys
+  give them - a position, not a coordinate: the plan's azimuth and the
+  heel are in the rig PC's config, no surface location is.
 
 ## What it records