fm35-decoder 1.1__tar.gz → 1.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {fm35_decoder-1.1 → fm35_decoder-1.2}/.gitignore +4 -1
- {fm35_decoder-1.1 → fm35_decoder-1.2}/PKG-INFO +5 -3
- {fm35_decoder-1.1 → fm35_decoder-1.2}/README.md +4 -2
- {fm35_decoder-1.1 → fm35_decoder-1.2}/pyproject.toml +1 -1
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/decoder.py +105 -42
- fm35_decoder-1.2/src/fm35_decoder/table_codes/CL_0513.csv +12 -0
- fm35_decoder-1.2/src/fm35_decoder/table_codes/Nh_2700.csv +12 -0
- fm35_decoder-1.2/src/fm35_decoder/table_codes/Sr_3849.csv +10 -0
- fm35_decoder-1.2/src/fm35_decoder/table_codes/h_1600.csv +12 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/table_codes/rara_3685.csv +47 -103
- fm35_decoder-1.2/src/fm35_decoder/table_codes/sasa_3872.csv +102 -0
- fm35_decoder-1.2/src/fm35_decoder/tables.py +383 -0
- fm35_decoder-1.1/src/fm35_decoder/table_codes/CL_0513.csv +0 -12
- fm35_decoder-1.1/src/fm35_decoder/table_codes/Nh_2700.csv +0 -12
- fm35_decoder-1.1/src/fm35_decoder/table_codes/Sr_3849.csv +0 -10
- fm35_decoder-1.1/src/fm35_decoder/table_codes/h_1600.csv +0 -12
- fm35_decoder-1.1/src/fm35_decoder/table_codes/sasa_3872.csv +0 -27
- fm35_decoder-1.1/src/fm35_decoder/tables.py +0 -291
- {fm35_decoder-1.1 → fm35_decoder-1.2}/LICENSE +0 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/__init__.py +0 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/table_codes/CH_0509.csv +0 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/table_codes/CM_0515.csv +0 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/table_codes/D_0777.csv +0 -0
- {fm35_decoder-1.1 → fm35_decoder-1.2}/src/fm35_decoder/table_codes/T_3931.csv +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: fm35_decoder
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.2
|
|
4
4
|
Summary: A Python tool to decode standard FM35 TEMP weather messages from upper-air observations.
|
|
5
5
|
Project-URL: Homepage, https://github.com/luciodaou/fm35_decoder
|
|
6
6
|
Project-URL: Bug Tracker, https://github.com/luciodaou/fm35_decoder/issues
|
|
@@ -115,6 +115,8 @@ Missing geopotential heights are computed using the Hypsometric Equation:
|
|
|
115
115
|
$$Z_2 - Z_1 = \frac{R_d \cdot \bar{T}_v}{g_0} \ln\left(\frac{P_1}{P_2}\right)$$
|
|
116
116
|
The decoder incorporates **virtual temperature ($T_v$)** when moisture data is available, accounting for air density variations in warm and humid tropical boundary layers.
|
|
117
117
|
|
|
118
|
+
A level without a temperature takes one for this calculation only: interpolated in $\ln P$ between observed temperatures, and the top temperature above the last one. Above the last temperature a height is computed only within the limits of Regulation 35.2.2.4 (25 hPa, and a quarter of the level's pressure); further up a level keeps no height unless the report gives it. Under the same rule, a standard level just above the top of the sounding that the report does not hold is added with its geopotential only (`Source` = `Extrapolated`).
|
|
119
|
+
|
|
118
120
|
---
|
|
119
121
|
|
|
120
122
|
## Performance & Architecture
|
|
@@ -149,9 +151,9 @@ The decoder incorporates **virtual temperature ($T_v$)** when moisture data is a
|
|
|
149
151
|
CL Cloud Low Type Stratus nebulosus and/or Stratus fractus TTBB
|
|
150
152
|
CM Cloud Mid Type No CM clouds TTBB
|
|
151
153
|
CH Cloud High Type No CH clouds TTBB
|
|
152
|
-
sr Solar/Inst Solar Corr
|
|
154
|
+
sr Solar/Inst Solar Corr Solar and infrared corrected automatically by radiosonde system TTAA
|
|
153
155
|
rara Solar/Inst Sonde Type Vaisala RS41/DigiCORA MW41 (Finland) TTAA
|
|
154
|
-
sasa Solar/Inst Tracking
|
|
156
|
+
sasa Solar/Inst Tracking Automatic satellite navigation TTAA
|
|
155
157
|
8GGgg Solar/Inst Time 11:31 TTAA
|
|
156
158
|
PtPtPt Tropopause Pressure 90.6hPa TTCC
|
|
157
159
|
TtTtTt Tropopause Temperature -77.1C TTCC
|
|
@@ -97,6 +97,8 @@ Missing geopotential heights are computed using the Hypsometric Equation:
|
|
|
97
97
|
$$Z_2 - Z_1 = \frac{R_d \cdot \bar{T}_v}{g_0} \ln\left(\frac{P_1}{P_2}\right)$$
|
|
98
98
|
The decoder incorporates **virtual temperature ($T_v$)** when moisture data is available, accounting for air density variations in warm and humid tropical boundary layers.
|
|
99
99
|
|
|
100
|
+
A level without a temperature takes one for this calculation only: interpolated in $\ln P$ between observed temperatures, and the top temperature above the last one. Above the last temperature a height is computed only within the limits of Regulation 35.2.2.4 (25 hPa, and a quarter of the level's pressure); further up a level keeps no height unless the report gives it. Under the same rule, a standard level just above the top of the sounding that the report does not hold is added with its geopotential only (`Source` = `Extrapolated`).
|
|
101
|
+
|
|
100
102
|
---
|
|
101
103
|
|
|
102
104
|
## Performance & Architecture
|
|
@@ -131,9 +133,9 @@ The decoder incorporates **virtual temperature ($T_v$)** when moisture data is a
|
|
|
131
133
|
CL Cloud Low Type Stratus nebulosus and/or Stratus fractus TTBB
|
|
132
134
|
CM Cloud Mid Type No CM clouds TTBB
|
|
133
135
|
CH Cloud High Type No CH clouds TTBB
|
|
134
|
-
sr Solar/Inst Solar Corr
|
|
136
|
+
sr Solar/Inst Solar Corr Solar and infrared corrected automatically by radiosonde system TTAA
|
|
135
137
|
rara Solar/Inst Sonde Type Vaisala RS41/DigiCORA MW41 (Finland) TTAA
|
|
136
|
-
sasa Solar/Inst Tracking
|
|
138
|
+
sasa Solar/Inst Tracking Automatic satellite navigation TTAA
|
|
137
139
|
8GGgg Solar/Inst Time 11:31 TTAA
|
|
138
140
|
PtPtPt Tropopause Pressure 90.6hPa TTCC
|
|
139
141
|
TtTtTt Tropopause Temperature -77.1C TTCC
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
"""
|
|
2
2
|
FM 35-XII TEMP Decoder.
|
|
3
|
-
Decodes
|
|
3
|
+
Decodes WMO FM 35 TEMP upper-air radiosonde observations from fixed land stations
|
|
4
|
+
(the ship and mobile headers of FM 36 / FM 38 are not read).
|
|
4
5
|
"""
|
|
5
6
|
|
|
6
7
|
from __future__ import annotations
|
|
@@ -79,7 +80,7 @@ def decode_temperature(ttt_str: str, tables: Optional[Dict[str, Any]] = None) ->
|
|
|
79
80
|
tt = int(ttt_str[:2])
|
|
80
81
|
ta_code = ttt_str[2]
|
|
81
82
|
|
|
82
|
-
t_table = (tables or WMO_TABLES).get("T_3931",
|
|
83
|
+
t_table = (tables or WMO_TABLES).get("T_3931", {})
|
|
83
84
|
entry = t_table.get(ta_code)
|
|
84
85
|
|
|
85
86
|
if entry:
|
|
@@ -135,7 +136,8 @@ def decode_wind(dff_str: str) -> Tuple[Optional[float], Optional[float]]:
|
|
|
135
136
|
|
|
136
137
|
Returns (direction_degrees, speed_knots).
|
|
137
138
|
For variable wind (dd=99), direction is returned as np.nan while speed is preserved.
|
|
138
|
-
For calm wind (dd=00, fff=000), returns (0.0, 0.0).
|
|
139
|
+
For calm wind (dd=00, fff=000), returns (0.0, 0.0). dd=00 with a speed (north is 36) is not a
|
|
140
|
+
direction: the speed is kept and the direction is np.nan.
|
|
139
141
|
"""
|
|
140
142
|
if not dff_str or len(dff_str) != 5 or "/////" in dff_str or "/" in dff_str:
|
|
141
143
|
return None, None
|
|
@@ -161,6 +163,8 @@ def decode_wind(dff_str: str) -> Tuple[Optional[float], Optional[float]]:
|
|
|
161
163
|
# Calm wind
|
|
162
164
|
if dd == 0 and speed == 0:
|
|
163
165
|
return 0.0, 0.0
|
|
166
|
+
if (dd == 0 and direction_unit == 0) or (dd == 36 and direction_unit == 5):
|
|
167
|
+
return np.nan, float(speed)
|
|
164
168
|
|
|
165
169
|
direction = dd * 10 + direction_unit
|
|
166
170
|
return float(direction), float(speed)
|
|
@@ -219,9 +223,14 @@ def decode_height(pressure: float, h_str: str) -> Optional[int]:
|
|
|
219
223
|
# --- Atmospheric Thermodynamics & Physics ---
|
|
220
224
|
|
|
221
225
|
|
|
222
|
-
def interpolate_data(
|
|
226
|
+
def interpolate_data(
|
|
227
|
+
df: pd.DataFrame, missing_layers: Optional[List[Tuple[str, float, float]]] = None
|
|
228
|
+
) -> pd.DataFrame:
|
|
223
229
|
"""
|
|
224
230
|
Interpolates missing Temperature, DewPoint, and Wind data vertically.
|
|
231
|
+
- missing_layers: (kind, bottom, top) layers reported as missing (nn/// /////), kind being
|
|
232
|
+
"Temperature" or "Wind" and the pressures those of the boundary levels; nothing is filled
|
|
233
|
+
strictly inside them.
|
|
225
234
|
- Temperature/DewPoint: Linear interpolation in Log-Pressure space.
|
|
226
235
|
- Physical bound: Enforces DewPoint <= Temp (no super-saturation).
|
|
227
236
|
- Wind: Vector interpolation (U/V components) in Log-Pressure space.
|
|
@@ -244,6 +253,13 @@ def interpolate_data(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
244
253
|
df["log_p"] = np.log(df["Pressure"].astype(float))
|
|
245
254
|
df = df.set_index("log_p")
|
|
246
255
|
|
|
256
|
+
def in_missing_layer(kind: str) -> pd.Series:
|
|
257
|
+
inside = pd.Series(False, index=df.index)
|
|
258
|
+
for layer_kind, bottom, top in missing_layers or ():
|
|
259
|
+
if layer_kind == kind:
|
|
260
|
+
inside |= (df["Pressure"] < bottom) & (df["Pressure"] > top)
|
|
261
|
+
return inside
|
|
262
|
+
|
|
247
263
|
# --- Temperature & Dewpoint Interpolation ---
|
|
248
264
|
for col in ["Temp", "DewPoint"]:
|
|
249
265
|
if col in df.columns:
|
|
@@ -252,6 +268,7 @@ def interpolate_data(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
252
268
|
# only between observed levels: nothing is made up above the last or below the first
|
|
253
269
|
df[col] = df[col].interpolate(method="index", limit_area="inside")
|
|
254
270
|
df.loc[mask, col] = df.loc[mask, col].round(1)
|
|
271
|
+
df.loc[mask & in_missing_layer("Temperature"), col] = np.nan
|
|
255
272
|
|
|
256
273
|
# Physical constraint: Dewpoint cannot exceed dry temperature
|
|
257
274
|
if "Temp" in df.columns and "DewPoint" in df.columns:
|
|
@@ -291,8 +308,9 @@ def interpolate_data(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
291
308
|
|
|
292
309
|
# Fill missing values in original columns; a variable wind (speed without direction) keeps
|
|
293
310
|
# its NaN direction
|
|
294
|
-
|
|
295
|
-
|
|
311
|
+
fillable = df["u"].notna() & ~in_missing_layer("Wind")
|
|
312
|
+
missing_speed = df["WindSpeed"].isna() & fillable
|
|
313
|
+
missing_dir = df["WindDir"].isna() & df["WindSpeed"].isna() & fillable
|
|
296
314
|
|
|
297
315
|
df.loc[missing_speed, "WindSpeed"] = reconstructed_speed[missing_speed].round(1)
|
|
298
316
|
df.loc[missing_dir, "WindDir"] = pd.Series(reconstructed_dir, index=df.index)[missing_dir].round(0)
|
|
@@ -308,7 +326,13 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
308
326
|
Calculates missing Geopotential Height values using the Hypsometric Equation.
|
|
309
327
|
Formula: Z2 = Z1 + (R * Tv_avg / g) * ln(P1 / P2)
|
|
310
328
|
Uses virtual temperature Tv where humidity is available to ensure physical accuracy.
|
|
311
|
-
Includes WMO Extrapolation Rule 35.2.2.4 upward to the next standard level
|
|
329
|
+
Includes WMO Extrapolation Rule 35.2.2.4 upward to the next standard level: a level the
|
|
330
|
+
report does not hold gets a row with its geopotential only (the extrapolated temperature
|
|
331
|
+
serves that calculation and is not given), and a level the report holds is left as reported.
|
|
332
|
+
A level without a temperature takes one for this calculation only: interpolated in ln(P)
|
|
333
|
+
between observed temperatures, and the top temperature above the last one, there within the
|
|
334
|
+
limits of the same rule (25 hPa and a quarter of the level's pressure); a level further up
|
|
335
|
+
keeps no height unless the report gives it.
|
|
312
336
|
"""
|
|
313
337
|
if df.empty or "Pressure" not in df.columns or "Temp" not in df.columns:
|
|
314
338
|
return df
|
|
@@ -324,12 +348,17 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
324
348
|
dewpoints = pd.to_numeric(df.get("DewPoint", pd.Series(dtype=float)), errors="coerce").to_numpy(dtype=float)
|
|
325
349
|
heights = pd.to_numeric(df.get("Height", pd.Series(dtype=float)), errors="coerce").to_numpy(dtype=float)
|
|
326
350
|
|
|
327
|
-
#
|
|
328
|
-
|
|
351
|
+
# Temperature in ln(P) between observed levels, for a continuous profile in the hypsometric
|
|
352
|
+
# calculation (the top and bottom temperatures are held beyond the observed ones)
|
|
353
|
+
valid_t = ~np.isnan(temps) & (pressures > 0)
|
|
329
354
|
if not valid_t.any():
|
|
330
355
|
return df
|
|
356
|
+
# Lowest pressure with an observed temperature: heights above it are bounded (35.2.2.4)
|
|
357
|
+
p_top_t = float(pressures[valid_t].min())
|
|
331
358
|
if not valid_t.all():
|
|
332
|
-
|
|
359
|
+
with np.errstate(divide="ignore", invalid="ignore"):
|
|
360
|
+
height_coord = -np.log(pressures)
|
|
361
|
+
temps = np.interp(height_coord, height_coord[valid_t], temps[valid_t])
|
|
333
362
|
|
|
334
363
|
# Virtual temperature: Tv = T_K * (1 + 0.608 * w)
|
|
335
364
|
temp_k = temps + 273.15
|
|
@@ -348,6 +377,9 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
348
377
|
for i in range(1, n):
|
|
349
378
|
if np.isnan(heights[i]) and not np.isnan(heights[i - 1]):
|
|
350
379
|
p1, p2 = pressures[i - 1], pressures[i]
|
|
380
|
+
beyond = p_top_t - p2
|
|
381
|
+
if beyond > 25.0 or beyond > 0.25 * p2:
|
|
382
|
+
continue
|
|
351
383
|
if p1 > 0 and p2 > 0 and p1 > p2:
|
|
352
384
|
avg_tv = (tv_k[i - 1] + tv_k[i]) / 2.0
|
|
353
385
|
dz = (R * avg_tv / g) * np.log(p1 / p2)
|
|
@@ -379,7 +411,8 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
379
411
|
1000, 925, 850, 700, 500, 400, 300, 250, 200, 150,
|
|
380
412
|
100, 70, 50, 30, 20, 10, 7, 5, 3, 2, 1,
|
|
381
413
|
]
|
|
382
|
-
|
|
414
|
+
reported = set(pd.to_numeric(df["Pressure"], errors="coerce").round(1))
|
|
415
|
+
targets = [sl for sl in standard_levels if sl < p_min and float(sl) not in reported]
|
|
383
416
|
|
|
384
417
|
new_rows = []
|
|
385
418
|
for p_target in targets:
|
|
@@ -387,8 +420,8 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
387
420
|
if delta_p <= 25 and delta_p <= 0.25 * p_target:
|
|
388
421
|
try:
|
|
389
422
|
p_base = p_min + delta_p
|
|
390
|
-
all_p =
|
|
391
|
-
all_t =
|
|
423
|
+
all_p = valid_rows["Pressure"].to_numpy(dtype=float)
|
|
424
|
+
all_t = valid_rows["Temp"].to_numpy(dtype=float) + 273.15
|
|
392
425
|
log_p = np.log(all_p)
|
|
393
426
|
log_p_base = np.log(p_base)
|
|
394
427
|
|
|
@@ -410,8 +443,6 @@ def calculate_geopotential(df: pd.DataFrame) -> pd.DataFrame:
|
|
|
410
443
|
{
|
|
411
444
|
"Pressure": int(p_target),
|
|
412
445
|
"Height": round(z_target),
|
|
413
|
-
"Temp": round(t_target - 273.15, 1),
|
|
414
|
-
"DewPoint": None,
|
|
415
446
|
"Source": "Extrapolated",
|
|
416
447
|
}
|
|
417
448
|
)
|
|
@@ -502,11 +533,12 @@ PART_NAMES = ("TTAA", "TTBB", "TTCC", "TTDD")
|
|
|
502
533
|
|
|
503
534
|
KT_PER_MS = 3600.0 / 1852.0
|
|
504
535
|
|
|
505
|
-
# Groups that cannot be a TTTDD group: "not observed" markers,
|
|
506
|
-
#
|
|
507
|
-
#
|
|
508
|
-
# in the message says so; temperatures of -66 and -77 C
|
|
509
|
-
|
|
536
|
+
# Groups that cannot be a TTTDD group: "not observed" markers, 41414 (41.4 C with a dew point of
|
|
537
|
+
# 40 C) and 55555 (depression code 55 is not used). The other markers are valid TTTDD groups
|
|
538
|
+
# (21212 = 21.2 C, 31313 = -31.3 C, 52525 = -52.5 C, 56565 = -56.5 C with a depression of 15 C,
|
|
539
|
+
# ...), read as data where the position in the message says so; temperatures of -66 and -77 C
|
|
540
|
+
# likewise start like the 66/77 indicators.
|
|
541
|
+
_NOT_TEMP = frozenset(["88999", "77999", "66999", "41414", "55555"])
|
|
510
542
|
|
|
511
543
|
# Regional (51515-59595) and national (61616-69696) sections come last in a part: no level data
|
|
512
544
|
# follows them. 55555 and 66666 are not in the list: in parts B and D 55555 is a significant level
|
|
@@ -556,16 +588,17 @@ def wind_top_pressure(id_char: str, part_c: bool) -> Optional[float]:
|
|
|
556
588
|
Id (code table 1734): pressure of the last standard level that carries a wind group, in
|
|
557
589
|
hundreds of hPa in Part A (0 = 1000, 9 = 925, 8 = 850) and tens of hPa in Part C. Levels
|
|
558
590
|
above it have no wind group (Id 2 covers 250 and 200 hPa, Id 1 covers 150 and 100 hPa).
|
|
559
|
-
"/" (no wind at any standard level) gives infinity;
|
|
591
|
+
"/" (no wind at any standard level) gives infinity; a figure the table does not define for
|
|
592
|
+
the part (6 in Part A; 0, 4, 6, 8, 9 in Part C) or anything else gives None (unknown).
|
|
560
593
|
"""
|
|
561
594
|
if id_char == "/":
|
|
562
595
|
return math.inf
|
|
563
|
-
if len(id_char) != 1 or id_char not in "0123456789":
|
|
564
|
-
return None
|
|
565
|
-
d = int(id_char)
|
|
566
596
|
if part_c:
|
|
567
|
-
return
|
|
568
|
-
return {
|
|
597
|
+
return {"7": 70.0, "5": 50.0, "3": 30.0, "2": 20.0, "1": 10.0}.get(id_char)
|
|
598
|
+
return {
|
|
599
|
+
"0": 1000.0, "9": 925.0, "8": 850.0, "7": 700.0, "5": 500.0,
|
|
600
|
+
"4": 400.0, "3": 300.0, "2": 200.0, "1": 100.0,
|
|
601
|
+
}.get(id_char)
|
|
569
602
|
|
|
570
603
|
|
|
571
604
|
def split_bulletin(text: Optional[str]) -> Dict[str, str]:
|
|
@@ -695,9 +728,9 @@ def parse_ttaa_ttcc(
|
|
|
695
728
|
continue
|
|
696
729
|
pressure = special_pressure(g[2:])
|
|
697
730
|
i += 1
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
731
|
+
# Without a readable pressure the level is not kept, but its groups are still its own
|
|
732
|
+
found: List[Dict[str, Any]] = []
|
|
733
|
+
found.append(
|
|
701
734
|
{"Symbol": "PtPtPt", "Subject": "Tropopause", "Description": "Pressure", "Value": f"{pressure}hPa"}
|
|
702
735
|
)
|
|
703
736
|
t_grp = group(i)
|
|
@@ -705,12 +738,12 @@ def parse_ttaa_ttcc(
|
|
|
705
738
|
t = decode_temperature(t_grp[:3], tables=cloud_tables)
|
|
706
739
|
d = decode_dewpoint_depression(t_grp[3:], tables=cloud_tables)
|
|
707
740
|
if t is not None:
|
|
708
|
-
|
|
741
|
+
found.append(
|
|
709
742
|
{"Symbol": "TtTtTt", "Subject": "Tropopause", "Description": "Temperature", "Value": f"{t}C"}
|
|
710
743
|
)
|
|
711
744
|
if t is not None and d is not None:
|
|
712
745
|
dw = calculate_dewpoint(t, d)
|
|
713
|
-
|
|
746
|
+
found.append(
|
|
714
747
|
{"Symbol": "DtDt", "Subject": "Tropopause", "Description": "Dewpoint", "Value": f"{dw}C"}
|
|
715
748
|
)
|
|
716
749
|
i += 1
|
|
@@ -718,7 +751,7 @@ def parse_ttaa_ttcc(
|
|
|
718
751
|
if _is_wind_group(w_grp):
|
|
719
752
|
wd, ws = decode_wind(w_grp)
|
|
720
753
|
if ws is not None:
|
|
721
|
-
|
|
754
|
+
found.append(
|
|
722
755
|
{
|
|
723
756
|
"Symbol": "dtdtftft",
|
|
724
757
|
"Subject": "Tropopause",
|
|
@@ -727,6 +760,8 @@ def parse_ttaa_ttcc(
|
|
|
727
760
|
}
|
|
728
761
|
)
|
|
729
762
|
i += 1
|
|
763
|
+
if pressure is not None:
|
|
764
|
+
special_data.extend(found)
|
|
730
765
|
continue
|
|
731
766
|
|
|
732
767
|
# Maximum Wind Level (77PmPmPm or 66PmPmPm, dmdmfmfmfm, optional 4vbvbvava)
|
|
@@ -736,16 +771,14 @@ def parse_ttaa_ttcc(
|
|
|
736
771
|
continue
|
|
737
772
|
pressure = special_pressure(g[2:])
|
|
738
773
|
i += 1
|
|
739
|
-
|
|
740
|
-
continue
|
|
741
|
-
special_data.append(
|
|
774
|
+
found = [
|
|
742
775
|
{"Symbol": "PmPmPm", "Subject": "Max Wind", "Description": "Pressure", "Value": f"{pressure}hPa"}
|
|
743
|
-
|
|
776
|
+
]
|
|
744
777
|
wind_grp = group(i)
|
|
745
778
|
if _is_wind_group(wind_grp):
|
|
746
779
|
wd, ws = decode_wind(wind_grp)
|
|
747
780
|
if ws is not None:
|
|
748
|
-
|
|
781
|
+
found.append(
|
|
749
782
|
{
|
|
750
783
|
"Symbol": "dmdmfmfmfm",
|
|
751
784
|
"Subject": "Max Wind",
|
|
@@ -758,13 +791,15 @@ def parse_ttaa_ttcc(
|
|
|
758
791
|
shear_grp = group(i)
|
|
759
792
|
if shear_grp is not None and len(shear_grp) == 5 and shear_grp.startswith("4"):
|
|
760
793
|
vb, va = shear_grp[1:3], shear_grp[3:5]
|
|
761
|
-
|
|
794
|
+
found.append(
|
|
762
795
|
{"Symbol": "vbvb", "Subject": "Max Wind", "Description": "Shear Below", "Value": f"{vb}{unit}"}
|
|
763
796
|
)
|
|
764
|
-
|
|
797
|
+
found.append(
|
|
765
798
|
{"Symbol": "vava", "Subject": "Max Wind", "Description": "Shear Above", "Value": f"{va}{unit}"}
|
|
766
799
|
)
|
|
767
800
|
i += 1
|
|
801
|
+
if pressure is not None:
|
|
802
|
+
special_data.extend(found)
|
|
768
803
|
continue
|
|
769
804
|
|
|
770
805
|
# Regional / Instrumentation group 31313
|
|
@@ -856,6 +891,8 @@ def parse_ttbb_ttdd(
|
|
|
856
891
|
mode = "TEMP"
|
|
857
892
|
is_ttdd = "TTDD" in message
|
|
858
893
|
last_p: Optional[float] = None
|
|
894
|
+
# A layer of missing data (nn/// /////) is open until the level that closes it
|
|
895
|
+
layer_open = False
|
|
859
896
|
unit = "kt"
|
|
860
897
|
|
|
861
898
|
i = 0
|
|
@@ -867,6 +904,7 @@ def parse_ttbb_ttdd(
|
|
|
867
904
|
is_ttdd = (g == "TTDD")
|
|
868
905
|
mode = "TEMP"
|
|
869
906
|
last_p = None
|
|
907
|
+
layer_open = False
|
|
870
908
|
i += 1
|
|
871
909
|
if i < len(groups) and re.match(r"^[0-9]{4}[0-9/]$", groups[i]):
|
|
872
910
|
unit = speed_unit(groups[i])
|
|
@@ -879,6 +917,7 @@ def parse_ttbb_ttdd(
|
|
|
879
917
|
if g == "21212":
|
|
880
918
|
mode = "WIND"
|
|
881
919
|
last_p = None
|
|
920
|
+
layer_open = False
|
|
882
921
|
i += 1
|
|
883
922
|
continue
|
|
884
923
|
|
|
@@ -916,6 +955,12 @@ def parse_ttbb_ttdd(
|
|
|
916
955
|
i += 1
|
|
917
956
|
continue
|
|
918
957
|
|
|
958
|
+
# Layer of missing data (35.3.1.6, 35.3.2.2): nn/// /////, between its boundary levels
|
|
959
|
+
if g[2:] == "///":
|
|
960
|
+
layer_open = True
|
|
961
|
+
i += 2 if i + 1 < len(groups) and groups[i + 1] == "/////" else 1
|
|
962
|
+
continue
|
|
963
|
+
|
|
919
964
|
try:
|
|
920
965
|
ppp_part = int(g[2:])
|
|
921
966
|
if nn == "00":
|
|
@@ -930,11 +975,24 @@ def parse_ttbb_ttdd(
|
|
|
930
975
|
if is_ttdd:
|
|
931
976
|
pressure = pressure / 10.0
|
|
932
977
|
|
|
978
|
+
if layer_open and last_p is not None and last_p > pressure:
|
|
979
|
+
special_data.append(
|
|
980
|
+
{
|
|
981
|
+
"Symbol": "nn///",
|
|
982
|
+
"Subject": "Missing Data",
|
|
983
|
+
"Description": "Temperature" if mode == "TEMP" else "Wind",
|
|
984
|
+
"Value": f"{last_p:g}-{pressure:g}hPa",
|
|
985
|
+
"Bottom": last_p,
|
|
986
|
+
"Top": pressure,
|
|
987
|
+
}
|
|
988
|
+
)
|
|
989
|
+
layer_open = False
|
|
933
990
|
last_p = pressure
|
|
934
991
|
|
|
935
992
|
if mode == "TEMP":
|
|
993
|
+
# The group after nnPPP is the temperature group, 21212 (21.2 C) included
|
|
936
994
|
t_group = groups[i + 1] if i + 1 < len(groups) else None
|
|
937
|
-
if t_group and len(t_group) == 5
|
|
995
|
+
if t_group and len(t_group) == 5:
|
|
938
996
|
t = decode_temperature(t_group[:3], tables=cloud_tables)
|
|
939
997
|
d = decode_dewpoint_depression(t_group[3:], tables=cloud_tables)
|
|
940
998
|
levels_data.append(
|
|
@@ -1015,6 +1073,7 @@ def decode_full(
|
|
|
1015
1073
|
|
|
1016
1074
|
data_frames: List[pd.DataFrame] = []
|
|
1017
1075
|
special_frames: List[pd.DataFrame] = []
|
|
1076
|
+
missing_layers: List[Tuple[str, float, float]] = []
|
|
1018
1077
|
|
|
1019
1078
|
for name, parser in [
|
|
1020
1079
|
("TTAA", parse_ttaa_ttcc),
|
|
@@ -1027,13 +1086,16 @@ def decode_full(
|
|
|
1027
1086
|
lvls, spcls = parser(msg, cloud_tables=cloud_tables)
|
|
1028
1087
|
if lvls:
|
|
1029
1088
|
data_frames.append(pd.DataFrame(lvls))
|
|
1089
|
+
missing_layers += [
|
|
1090
|
+
(s["Description"], s["Bottom"], s["Top"]) for s in spcls if s["Subject"] == "Missing Data"
|
|
1091
|
+
]
|
|
1030
1092
|
if spcls:
|
|
1031
1093
|
special_frames.append(pd.DataFrame(spcls).assign(Part=name))
|
|
1032
1094
|
|
|
1033
1095
|
df_main = merge_data(data_frames)
|
|
1034
1096
|
if not df_main.empty and df_main["Temp"].notna().sum() < 2:
|
|
1035
1097
|
raise ValueError("TEMP message without temperatures above the surface")
|
|
1036
|
-
df_main = interpolate_data(df_main)
|
|
1098
|
+
df_main = interpolate_data(df_main, missing_layers)
|
|
1037
1099
|
df_main = calculate_geopotential(df_main)
|
|
1038
1100
|
|
|
1039
1101
|
df_special = pd.concat(special_frames, ignore_index=True) if special_frames else pd.DataFrame()
|
|
@@ -1078,7 +1140,8 @@ def decode(
|
|
|
1078
1140
|
Returns:
|
|
1079
1141
|
tuple[pd.DataFrame, pd.DataFrame]:
|
|
1080
1142
|
- df_main: Vertical profile with Pressure, Height, Temp, DewPoint, WindDir, WindSpeed.
|
|
1081
|
-
- df_special: Tropopause, Maximum Wind, Cloud layers,
|
|
1143
|
+
- df_special: Tropopause, Maximum Wind, Cloud layers, layers of missing data, and
|
|
1144
|
+
Radiosonde/Solar metadata,
|
|
1082
1145
|
with the part (TTAA, TTBB, TTCC, TTDD) each entry came from in the Part column.
|
|
1083
1146
|
"""
|
|
1084
1147
|
# Defensive type validation
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
"Code","Description"
|
|
2
|
+
"0","No CL clouds"
|
|
3
|
+
"1","Cumulus humilis and/or Cumulus fractus other than of bad weather"
|
|
4
|
+
"2","Cumulus mediocris or congestus, with or without Cumulus of species in code 1"
|
|
5
|
+
"3","Cumulonimbus calvus, with or without Cumulus, Stratocumulus or Stratus"
|
|
6
|
+
"4","Stratocumulus cumulogenitus"
|
|
7
|
+
"5","Stratocumulus other than cumulogenitus"
|
|
8
|
+
"6","Stratus nebulosus and/or Stratus fractus other than of bad weather"
|
|
9
|
+
"7","Stratus fractus and/or Cumulus fractus of bad weather"
|
|
10
|
+
"8","Cumulus and Stratocumulus other than cumulogenitus, with bases at different levels"
|
|
11
|
+
"9","Cumulonimbus capillatus, with or without Cumulonimbus calvus, Cumulus, Stratocumulus, Stratus"
|
|
12
|
+
"/","CL clouds invisible owing to darkness, fog, blowing dust or sand, or other similar phenomena"
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
"Code","Description"
|
|
2
|
+
"0","No clouds"
|
|
3
|
+
"1","1 okta or less, but not zero"
|
|
4
|
+
"2","2 oktas"
|
|
5
|
+
"3","3 oktas"
|
|
6
|
+
"4","4 oktas"
|
|
7
|
+
"5","5 oktas"
|
|
8
|
+
"6","6 oktas"
|
|
9
|
+
"7","7 oktas or more, but not 8 oktas"
|
|
10
|
+
"8","8 oktas"
|
|
11
|
+
"9","Sky obscured by fog and/or other meteorological phenomena"
|
|
12
|
+
"/","Cloud cover indiscernible for reasons other than fog or other meteorological phenomena, or observation not made"
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
"Code","Description"
|
|
2
|
+
"0","No correction"
|
|
3
|
+
"1","CIMO solar corrected and CIMO infrared corrected"
|
|
4
|
+
"2","CIMO solar corrected and infrared corrected"
|
|
5
|
+
"3","CIMO solar corrected only"
|
|
6
|
+
"4","Solar and infrared corrected automatically by radiosonde system"
|
|
7
|
+
"5","Solar corrected automatically by radiosonde system"
|
|
8
|
+
"6","Solar and infrared corrected as specified by country"
|
|
9
|
+
"7","Solar corrected as specified by country"
|
|
10
|
+
"/","Missing value"
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
"Code","Description"
|
|
2
|
+
"0","< 50 m (< 150 ft)"
|
|
3
|
+
"1","50-100 m (150-300 ft)"
|
|
4
|
+
"2","100-200 m (300-600 ft)"
|
|
5
|
+
"3","200-300 m (600-1000 ft)"
|
|
6
|
+
"4","300-600 m (1000-2000 ft)"
|
|
7
|
+
"5","600-1000 m (2000-3300 ft)"
|
|
8
|
+
"6","1000-1500 m (3300-5000 ft)"
|
|
9
|
+
"7","1500-2000 m (5000-6500 ft)"
|
|
10
|
+
"8","2000-2500 m (6500-8000 ft)"
|
|
11
|
+
"9","> 2500 m (> 8000 ft), or no clouds"
|
|
12
|
+
"/","Height of base of cloud not known, or base of clouds below and tops above the station"
|