sdc-mapper 4.2.1__tar.gz → 4.3.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. {sdc_mapper-4.2.1/src/sdc_mapper.egg-info → sdc_mapper-4.3.1}/PKG-INFO +26 -6
  2. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/README.md +25 -5
  3. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/pyproject.toml +1 -1
  4. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/__init__.py +5 -3
  5. sdc_mapper-4.3.1/src/sdc_mapper/codes.py +92 -0
  6. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/mapping.py +3 -1
  7. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/placement.py +29 -7
  8. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/schemas/mapping.schema.json +31 -35
  9. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1/src/sdc_mapper.egg-info}/PKG-INFO +26 -6
  10. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper.egg-info/SOURCES.txt +2 -0
  11. sdc_mapper-4.3.1/tests/test_codes.py +107 -0
  12. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/LICENSE +0 -0
  13. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/NOTICE +0 -0
  14. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/setup.cfg +0 -0
  15. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/cli.py +0 -0
  16. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/contract.py +0 -0
  17. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/engine.py +0 -0
  18. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/gate.py +0 -0
  19. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/log.py +0 -0
  20. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/mcp_server.py +0 -0
  21. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/report.py +0 -0
  22. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/resolve.py +0 -0
  23. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/schemas/log-entry.schema.json +0 -0
  24. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/schemas/sdc4.xsd +0 -0
  25. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper/service.py +0 -0
  26. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper.egg-info/dependency_links.txt +0 -0
  27. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper.egg-info/entry_points.txt +0 -0
  28. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper.egg-info/requires.txt +0 -0
  29. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/src/sdc_mapper.egg-info/top_level.txt +0 -0
  30. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/tests/test_core.py +0 -0
  31. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/tests/test_placement.py +0 -0
  32. {sdc_mapper-4.2.1 → sdc_mapper-4.3.1}/tests/test_service.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sdc-mapper
3
- Version: 4.2.1
3
+ Version: 4.3.1
4
4
  Summary: The shared core of the Semantic Data Charter's Governed Data Record mappers: validate, resolve, map, and report every value; MCP server included.
5
5
  Author: Axius SDC, Inc.
6
6
  License-Expression: Apache-2.0
@@ -33,7 +33,8 @@ target could say it too. Every call is appended to a hash-chained log that holds
33
33
  Each target format is its own package (`sdc-gdr2ubl`, `sdc-gdr2edifact`, `sdc-gdr2omop`, ...) registered under the
34
34
  entry-point group `sdc_mapper.targets`. Outbound only. Apache 2.0.
35
35
 
36
- **Status:** 4.2.0. The first target mapper, `sdc-gdr2ubl` (OASIS UBL 2.3), is in development.
36
+ **Status:** 4.3.1. Five target mappers are released: `sdc-gdr2ubl` (OASIS UBL 2.3), `sdc-gdr2edifact` (UN/EDIFACT
37
+ D.24A), `sdc-gdr2omop` (OMOP CDM 5.4), `sdc-gdr2fhir` (HL7 FHIR R4) and `sdc-gdr2hl7v2` (HL7 v2.5.1).
37
38
 
38
39
  ## Use
39
40
 
@@ -87,10 +88,29 @@ rules:
87
88
  reason: the target profile carries no contact
88
89
  ```
89
90
 
90
- Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model binds a value to), `omit`;
91
- clusters `group` or `omit`. Steps: `..` up, `.` none, `+name` an element of the value's own, `=name` the element of that
92
- name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is reported, never dropped. `place(record, mapping)` turns a record and its
93
- mapping into placements a mapper writes from.
91
+ Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model defines an enumerated value
92
+ by), `link` (the code of the concept the component links to), `omit`; clusters `group` or `omit`. `binding` and `link`
93
+ take a `system` (`http://loinc.org`, `http://snomed.info/sct`, a code list's IRI); their text is the model's, so a mapper
94
+ reports it as a lookup (`LOOKUP_TRANSFORMS`). Steps: `..` up, `.` none, `+name` an element of the value's own, `=name`
95
+ the element of that name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is
96
+ reported, never dropped. `place(record, mapping)` turns a record and its mapping into placements a mapper writes from.
97
+
98
+ ## Codes the model names
99
+
100
+ `sdc_mapper.codes` reads the IRIs the component libraries use for a concept as a code system and a code: LOINC codes
101
+ and answers, SNOMED CT, NCI Thesaurus, and any code list whose IRI ends in `#code` (UBL's, HL7's). A code system is
102
+ named by the IRI FHIR uses for it; a mapper turns that into its own name for the system (OMOP's `vocabulary_id`, HL7
103
+ v2's coding system). `bound_code(value)` is the code an enumerated value is defined by, `linked_code(value)` the code
104
+ of the concept its component links to (`skos:exactMatch`, then `dcterms:subject`), and `model_codes(value)` every code
105
+ the model names for it: a target code that is one of these came from the model.
106
+
107
+ ## Writing to a target
108
+
109
+ `place()` gives each value its target steps. Steps written `=name` join an element made by another value, so a writer
110
+ holds those writes back until every other value is written, then writes them shallowest first: sort them by
111
+ `join_depth(steps)`. A row-oriented target (a set of tables, as OMOP's) can treat each step as a table or a column,
112
+ `+TABLE` as one new row per value, and keep the record's own row (its person, its document) under a name of its own;
113
+ `sdc-gdr2omop` does so with a `RECORD` table.
94
114
 
95
115
  ## Writing a mapper
96
116
 
@@ -11,7 +11,8 @@ target could say it too. Every call is appended to a hash-chained log that holds
11
11
  Each target format is its own package (`sdc-gdr2ubl`, `sdc-gdr2edifact`, `sdc-gdr2omop`, ...) registered under the
12
12
  entry-point group `sdc_mapper.targets`. Outbound only. Apache 2.0.
13
13
 
14
- **Status:** 4.2.0. The first target mapper, `sdc-gdr2ubl` (OASIS UBL 2.3), is in development.
14
+ **Status:** 4.3.1. Five target mappers are released: `sdc-gdr2ubl` (OASIS UBL 2.3), `sdc-gdr2edifact` (UN/EDIFACT
15
+ D.24A), `sdc-gdr2omop` (OMOP CDM 5.4), `sdc-gdr2fhir` (HL7 FHIR R4) and `sdc-gdr2hl7v2` (HL7 v2.5.1).
15
16
 
16
17
  ## Use
17
18
 
@@ -65,10 +66,29 @@ rules:
65
66
  reason: the target profile carries no contact
66
67
  ```
67
68
 
68
- Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model binds a value to), `omit`;
69
- clusters `group` or `omit`. Steps: `..` up, `.` none, `+name` an element of the value's own, `=name` the element of that
70
- name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is reported, never dropped. `place(record, mapping)` turns a record and its
71
- mapping into placements a mapper writes from.
69
+ Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model defines an enumerated value
70
+ by), `link` (the code of the concept the component links to), `omit`; clusters `group` or `omit`. `binding` and `link`
71
+ take a `system` (`http://loinc.org`, `http://snomed.info/sct`, a code list's IRI); their text is the model's, so a mapper
72
+ reports it as a lookup (`LOOKUP_TRANSFORMS`). Steps: `..` up, `.` none, `+name` an element of the value's own, `=name`
73
+ the element of that name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is
74
+ reported, never dropped. `place(record, mapping)` turns a record and its mapping into placements a mapper writes from.
75
+
76
+ ## Codes the model names
77
+
78
+ `sdc_mapper.codes` reads the IRIs the component libraries use for a concept as a code system and a code: LOINC codes
79
+ and answers, SNOMED CT, NCI Thesaurus, and any code list whose IRI ends in `#code` (UBL's, HL7's). A code system is
80
+ named by the IRI FHIR uses for it; a mapper turns that into its own name for the system (OMOP's `vocabulary_id`, HL7
81
+ v2's coding system). `bound_code(value)` is the code an enumerated value is defined by, `linked_code(value)` the code
82
+ of the concept its component links to (`skos:exactMatch`, then `dcterms:subject`), and `model_codes(value)` every code
83
+ the model names for it: a target code that is one of these came from the model.
84
+
85
+ ## Writing to a target
86
+
87
+ `place()` gives each value its target steps. Steps written `=name` join an element made by another value, so a writer
88
+ holds those writes back until every other value is written, then writes them shallowest first: sort them by
89
+ `join_depth(steps)`. A row-oriented target (a set of tables, as OMOP's) can treat each step as a table or a column,
90
+ `+TABLE` as one new row per value, and keep the record's own row (its person, its document) under a name of its own;
91
+ `sdc-gdr2omop` does so with a `RECORD` table.
72
92
 
73
93
  ## Writing a mapper
74
94
 
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "sdc-mapper"
7
- version = "4.2.1"
7
+ version = "4.3.1"
8
8
  description = "The shared core of the Semantic Data Charter's Governed Data Record mappers: validate, resolve, map, and report every value; MCP server included."
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.12"
@@ -15,14 +15,16 @@ from .engine import ERROR, CallResult, Engine
15
15
  from .gate import REFUSED, VALID, VALID_WITH_EV, Gate, GateResult, Recovery
16
16
  from .log import Log, entry_hash
17
17
  from .mapping import Mapping, MappingError, Rule, load_mapping
18
- from .placement import JOIN, ClusterPlacement, Extra, Placement, place, text_of
18
+ from .codes import CONCEPT_LINKS, EXACT_MATCH, LOINC, NCIT, SNOMED_CT, SUBJECT, CodeRef, bound_code, code_of, linked_code, model_codes
19
+ from .placement import JOIN, LOOKUP_TRANSFORMS, ClusterPlacement, Extra, Placement, join_depth, place, text_of
19
20
  from .report import CoverageReport, IncompleteMapping, build_report
20
21
  from .resolve import Exceptional, ResolvedRecord, ResolvedValue, Units, resolve
21
22
 
22
- __version__ = "4.2.1"
23
+ __version__ = "4.3.1"
23
24
  __all__ = [
24
25
  "Engine", "CallResult", "Gate", "GateResult", "Recovery", "Log", "entry_hash", "Mapping", "MappingError", "Rule",
25
- "load_mapping", "place", "Placement", "ClusterPlacement", "Extra", "JOIN", "text_of", "CoverageReport", "IncompleteMapping", "build_report", "ResolvedRecord", "ResolvedValue",
26
+ "load_mapping", "place", "Placement", "ClusterPlacement", "Extra", "JOIN", "join_depth", "LOOKUP_TRANSFORMS", "CodeRef", "code_of",
27
+ "bound_code", "linked_code", "model_codes", "CONCEPT_LINKS", "EXACT_MATCH", "SUBJECT", "LOINC", "SNOMED_CT", "NCIT", "text_of", "CoverageReport", "IncompleteMapping", "build_report", "ResolvedRecord", "ResolvedValue",
26
28
  "Exceptional", "Units", "resolve", "Mapper", "MapResult", "ValueOutcome", "EVRule", "installed_mappers",
27
29
  "VALID", "VALID_WITH_EV", "REFUSED", "ERROR", "LOOKUP", "DECLARED", "CARRIED", "NOT_REPRESENTABLE", "VALUE_CLASSES",
28
30
  "PRESERVED", "DEGRADED", "EV_CARRIED", "LOST", "EV_CLASSES", "ENTRY_POINT_GROUP", "__version__",
@@ -0,0 +1,92 @@
1
+ """Codes the model names (4.3.0): the IRIs a model uses for a concept, read as a code system and a code.
2
+
3
+ A model names concepts in two places. An enumerated value is defined by a code (`rdfs:isDefinedBy`: a LOINC answer, a
4
+ SNOMED CT concept, an entry in a UBL code list). A component links to the concept it measures or asks
5
+ (`skos:exactMatch` to a LOINC code, `dcterms:subject` to a SNOMED CT concept). Wherever such an IRI names one concept
6
+ of a code system, the code is the model's: a target that writes it is looking it up, not choosing it.
7
+
8
+ `code_of()` reads the IRI forms the component libraries use. A code system is named by the IRI FHIR uses for it, the
9
+ one most targets recognize; each mapper turns that into its own name for the system (OMOP's `vocabulary_id`, HL7 v2's
10
+ coding system table) in its mapping file or its code.
11
+ """
12
+ from __future__ import annotations
13
+
14
+ import re
15
+ from dataclasses import dataclass
16
+
17
+ LOINC = "http://loinc.org"
18
+ SNOMED_CT = "http://snomed.info/sct"
19
+ NCIT = "http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl"
20
+
21
+ EXACT_MATCH = "http://www.w3.org/2004/02/skos/core#exactMatch"
22
+ SUBJECT = "http://purl.org/dc/terms/subject"
23
+ #: The links `link` reads by default, in order: what the component is, then what it is about.
24
+ CONCEPT_LINKS = (EXACT_MATCH, SUBJECT)
25
+
26
+ _FORMS = [
27
+ # LOINC codes, answers (LA), parts (LP) and groups (LG), with or without the /rdf/ path and a trailing slash
28
+ (re.compile(r"^https?://loinc\.org/(?:rdf/)?((?:LA|LP|LG)?[0-9]+-[0-9])/?$"), LOINC),
29
+ (re.compile(r"^https?://snomed\.info/(?:id|sct)/([0-9]+)/?$"), SNOMED_CT),
30
+ (re.compile(r"^https?://purl\.obolibrary\.org/obo/NCIT_(C[0-9]+)$"), NCIT),
31
+ ]
32
+ #: An IRI whose fragment is the code: a code list or code system and one entry in it (UBL's code lists, HL7's).
33
+ _FRAGMENT = re.compile(r"^(https?://[^#\s]+)#([^#\s]+)$")
34
+
35
+
36
+ @dataclass(frozen=True)
37
+ class CodeRef:
38
+ system: str # the code system, by the IRI FHIR uses for it, or the IRI before the fragment
39
+ code: str
40
+
41
+
42
+ def code_of(iri: str | None, code_lists: bool = True) -> CodeRef | None:
43
+ """The code an IRI names, or None (an IRI that names a whole code system, or a page about a variable).
44
+
45
+ The code systems' own forms (LOINC, SNOMED CT, NCI Thesaurus) always read as codes. An IRI ending in `#code` reads
46
+ as an entry in a code list only with `code_lists`: that is how a model binds an enumerated value to a list (UBL's,
47
+ HL7's), but the same form also names a vocabulary term (`prov#Entity`) or an anchor on a page about a variable
48
+ (`DEMO_J.htm#SEQN`), which are not codes. 4.3.1: concept links read without it.
49
+ """
50
+ if not iri:
51
+ return None
52
+ iri = iri.strip()
53
+ for form, system in _FORMS:
54
+ m = form.match(iri)
55
+ if m:
56
+ return CodeRef(system, m.group(1))
57
+ if code_lists:
58
+ m = _FRAGMENT.match(iri)
59
+ if m:
60
+ return CodeRef(m.group(1), m.group(2))
61
+ return None
62
+
63
+
64
+ def _in(ref: CodeRef | None, system: str | None) -> bool:
65
+ return ref is not None and (system is None or ref.system == system.rstrip("/") or ref.system.startswith(system))
66
+
67
+
68
+ def bound_code(value, system: str | None = None) -> CodeRef | None:
69
+ """The code the model defines an enumerated value by, if it is one (in `system`, when given): a code system's
70
+ concept, or an entry in a code list."""
71
+ ref = code_of(value.code.defined_by) if getattr(value, "code", None) else None
72
+ return ref if _in(ref, system) else None
73
+
74
+
75
+ def linked_code(value, system: str | None = None, predicates=CONCEPT_LINKS) -> CodeRef | None:
76
+ """The first code the component links to (in `system`, when given), by the predicates in order: a code system's
77
+ concept only, never a `#fragment` (a link names a concept; a fragment in a link is a term or a page anchor)."""
78
+ links = getattr(value, "links", None) or {}
79
+ for predicate in predicates:
80
+ for iri in links.get(predicate, []):
81
+ ref = code_of(iri, code_lists=False)
82
+ if _in(ref, system):
83
+ return ref
84
+ return None
85
+
86
+
87
+ def model_codes(value) -> set[str]:
88
+ """Every code the model names for a value: its enumeration value's definition and the component's concept links.
89
+ A target code that is one of these came from the model."""
90
+ refs = [bound_code(value)] + [code_of(i, code_lists=False)
91
+ for p in CONCEPT_LINKS for i in (getattr(value, "links", None) or {}).get(p, [])]
92
+ return {r.code for r in refs if r}
@@ -40,7 +40,9 @@ class Rule:
40
40
  units: dict | None = None
41
41
  also: list[dict] = field(default_factory=list)
42
42
  on_miss: str = "fail"
43
- prefix: str | None = None
43
+ prefix: str | None = None # binding: the code system (the 4.1.0 name; `system` since 4.3.0)
44
+ system: str | None = None # binding, link: the code system the code must be in
45
+ predicates: list[str] = field(default_factory=list) # link: the links to read, in order
44
46
  order: int | None = None
45
47
 
46
48
  def applies_within(self, ancestors: list[str]) -> bool:
@@ -15,6 +15,7 @@ from dataclasses import dataclass, field
15
15
  from datetime import date, datetime, time
16
16
  from decimal import Decimal
17
17
 
18
+ from .codes import CONCEPT_LINKS, bound_code, linked_code
18
19
  from .mapping import Mapping, MappingError, Rule
19
20
  from .resolve import ResolvedRecord, ResolvedValue
20
21
 
@@ -146,6 +147,17 @@ def _transform(rule, value: ResolvedValue, mapping: Mapping) -> str | None:
146
147
  transform = get("transform")
147
148
  if transform == "constant":
148
149
  return get("value")
150
+ if transform == "link":
151
+ # the code of the concept the component links to: what the value is a value of, so written beside an
152
+ # Exceptional Value too
153
+ system = get("system")
154
+ ref = linked_code(value, system, tuple(get("predicates") or CONCEPT_LINKS))
155
+ if ref:
156
+ return ref.code
157
+ if (get("on_miss") or "fail") == "omit":
158
+ raise _Miss(f"{value.label} links to no code" + (f" in {system}" if system else ""))
159
+ raise MappingError(f"mapping {mapping.id}: {value.label} ({value.ct_id}) links to no code"
160
+ + (f" in {system}" if system else ""))
149
161
  if value.exceptional:
150
162
  return None # an Exceptional Value's value is never written
151
163
  if transform == "copy":
@@ -153,15 +165,15 @@ def _transform(rule, value: ResolvedValue, mapping: Mapping) -> str | None:
153
165
  if transform == "format":
154
166
  return _format(value, get("format") or "")
155
167
  if transform == "binding":
156
- # the code the model binds the value to: the fragment of the IRI its enumeration value is defined by
157
- prefix = get("prefix") or ""
158
- bound = value.code.defined_by if value.code else None
159
- if bound and bound.startswith(prefix) and "#" in bound:
160
- return bound.rsplit("#", 1)[1]
168
+ # the code the model defines the enumerated value by (any IRI form `codes` reads), in `system` when given
169
+ system = get("system") or get("prefix")
170
+ ref = bound_code(value, system)
171
+ if ref:
172
+ return ref.code
161
173
  if (get("on_miss") or "fail") == "omit":
162
- raise _Miss(f"{value.label} is not bound under {prefix}")
174
+ raise _Miss(f"{value.label} is not bound to a code" + (f" in {system}" if system else ""))
163
175
  raise MappingError(f"mapping {mapping.id}: {value.label} ({value.ct_id}) holds {text_of(value)!r}, which the model "
164
- f"does not bind to a code under {prefix}")
176
+ "does not bind to a code" + (f" in {system}" if system else ""))
165
177
  if transform == "code":
166
178
  raw = text_of(value)
167
179
  if raw is None:
@@ -223,3 +235,13 @@ def place(record: ResolvedRecord, mapping: Mapping) -> tuple[list[Placement], li
223
235
  p.units_text = symbol
224
236
  placements.append(p)
225
237
  return placements, list(clusters.values())
238
+
239
+
240
+ def join_depth(steps) -> int:
241
+ """Where a write's deepest '=name' step is (-1 for none). A writer resolves writes with '=' steps after every other
242
+ value, the shallowest first, so a join through an element another join makes finds it made."""
243
+ return max((i for i, (_, key) in enumerate(steps) if key == JOIN), default=-1)
244
+
245
+
246
+ #: Transforms whose text is the model's own (a lookup, in the report), not the mapping's choice.
247
+ LOOKUP_TRANSFORMS = frozenset({"binding", "link"})
@@ -2,7 +2,7 @@
2
2
  "$schema": "https://json-schema.org/draft/2020-12/schema",
3
3
  "$id": "https://semanticdatacharter.com/schemas/sdc-mapper/mapping-4.json",
4
4
  "title": "sdc-mapper mapping file",
5
- "description": "How one model's components reach one target where the model alone does not settle it. Bound to one model schema version and one target version; immutable once published. No expressions or code. Since 4.1.0, rules may map a cluster to a target element (with a `within` context for a cluster used in several places), target paths are relative to the enclosing mapped cluster unless they start with '/', a final '@name' step places an attribute, and a `units` sub-rule places a quantity's unit. A '..' step goes up from the enclosing mapped cluster's element, a '.' step is no step, and a step prefixed '+' makes a new element for each value rather than sharing one by name. `also` writes the same value to further places; `on_miss: omit` leaves a code-table miss unwritten instead of failing. The `binding` transform writes the code the model binds a value to (the fragment of the IRI its enumeration value is defined by, under `prefix`): the model, not the mapping, supplies the value. A step written '=name' joins the element of that name its parent already has (one a nested cluster makes), or makes one when there is none. A cluster rule's `order` ranks the elements made under it among same-named elements made under its sibling clusters.",
5
+ "description": "How one model's components reach one target where the model alone does not settle it. Bound to one model schema version and one target version; immutable once published. No expressions or code. Since 4.1.0, rules may map a cluster to a target element (with a `within` context for a cluster used in several places), target paths are relative to the enclosing mapped cluster unless they start with '/', a final '@name' step places an attribute, and a `units` sub-rule places a quantity's unit. A '..' step goes up from the enclosing mapped cluster's element, a '.' step is no step, and a step prefixed '+' makes a new element for each value rather than sharing one by name. `also` writes the same value to further places; `on_miss: omit` leaves a code-table miss unwritten instead of failing. The `binding` transform writes the code the model binds a value to (the fragment of the IRI its enumeration value is defined by, under `prefix`): the model, not the mapping, supplies the value. A step written '=name' joins the element of that name its parent already has (one a nested cluster makes), or makes one when there is none. A cluster rule's `order` ranks the elements made under it among same-named elements made under its sibling clusters. Since 4.3.0, `binding` reads every IRI form the component libraries use (LOINC, SNOMED CT, NCIt, a code list's fragment) and takes the code system as `system`; `link` writes the code of the concept the component links to (skos:exactMatch, then dcterms:subject, or `predicates`).",
6
6
  "type": "object",
7
7
  "required": [
8
8
  "mapping",
@@ -195,7 +195,8 @@
195
195
  "constant",
196
196
  "omit",
197
197
  "group",
198
- "binding"
198
+ "binding",
199
+ "link"
199
200
  ]
200
201
  },
201
202
  "table": {
@@ -281,7 +282,8 @@
281
282
  "code",
282
283
  "format",
283
284
  "constant",
284
- "binding"
285
+ "binding",
286
+ "link"
285
287
  ]
286
288
  },
287
289
  "table": {
@@ -305,6 +307,18 @@
305
307
  },
306
308
  "prefix": {
307
309
  "type": "string"
310
+ },
311
+ "system": {
312
+ "type": "string",
313
+ "description": "binding, link: the code system the code must be in, by its IRI (http://loinc.org, http://snomed.info/sct, a code list's IRI)."
314
+ },
315
+ "predicates": {
316
+ "type": "array",
317
+ "items": {
318
+ "type": "string"
319
+ },
320
+ "minItems": 1,
321
+ "description": "link: the links to read, in order (default skos:exactMatch, dcterms:subject)."
308
322
  }
309
323
  },
310
324
  "allOf": [
@@ -349,20 +363,6 @@
349
363
  "value"
350
364
  ]
351
365
  }
352
- },
353
- {
354
- "if": {
355
- "properties": {
356
- "transform": {
357
- "const": "binding"
358
- }
359
- }
360
- },
361
- "then": {
362
- "required": [
363
- "prefix"
364
- ]
365
- }
366
366
  }
367
367
  ]
368
368
  }
@@ -374,6 +374,18 @@
374
374
  "order": {
375
375
  "type": "integer",
376
376
  "description": "For a cluster rule: where the elements made under this cluster stand among elements of the same name made under its siblings (Invoice Line 3 has order 3), where the record's own order is not the target's."
377
+ },
378
+ "system": {
379
+ "type": "string",
380
+ "description": "binding, link: the code system the code must be in, by its IRI (http://loinc.org, http://snomed.info/sct, a code list's IRI)."
381
+ },
382
+ "predicates": {
383
+ "type": "array",
384
+ "items": {
385
+ "type": "string"
386
+ },
387
+ "minItems": 1,
388
+ "description": "link: the links to read, in order (default skos:exactMatch, dcterms:subject)."
377
389
  }
378
390
  },
379
391
  "allOf": [
@@ -405,7 +417,8 @@
405
417
  "format",
406
418
  "constant",
407
419
  "omit",
408
- "binding"
420
+ "binding",
421
+ "link"
409
422
  ]
410
423
  }
411
424
  }
@@ -478,23 +491,6 @@
478
491
  "reason"
479
492
  ]
480
493
  }
481
- },
482
- {
483
- "if": {
484
- "properties": {
485
- "transform": {
486
- "const": "binding"
487
- }
488
- },
489
- "required": [
490
- "transform"
491
- ]
492
- },
493
- "then": {
494
- "required": [
495
- "prefix"
496
- ]
497
- }
498
494
  }
499
495
  ]
500
496
  },
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sdc-mapper
3
- Version: 4.2.1
3
+ Version: 4.3.1
4
4
  Summary: The shared core of the Semantic Data Charter's Governed Data Record mappers: validate, resolve, map, and report every value; MCP server included.
5
5
  Author: Axius SDC, Inc.
6
6
  License-Expression: Apache-2.0
@@ -33,7 +33,8 @@ target could say it too. Every call is appended to a hash-chained log that holds
33
33
  Each target format is its own package (`sdc-gdr2ubl`, `sdc-gdr2edifact`, `sdc-gdr2omop`, ...) registered under the
34
34
  entry-point group `sdc_mapper.targets`. Outbound only. Apache 2.0.
35
35
 
36
- **Status:** 4.2.0. The first target mapper, `sdc-gdr2ubl` (OASIS UBL 2.3), is in development.
36
+ **Status:** 4.3.1. Five target mappers are released: `sdc-gdr2ubl` (OASIS UBL 2.3), `sdc-gdr2edifact` (UN/EDIFACT
37
+ D.24A), `sdc-gdr2omop` (OMOP CDM 5.4), `sdc-gdr2fhir` (HL7 FHIR R4) and `sdc-gdr2hl7v2` (HL7 v2.5.1).
37
38
 
38
39
  ## Use
39
40
 
@@ -87,10 +88,29 @@ rules:
87
88
  reason: the target profile carries no contact
88
89
  ```
89
90
 
90
- Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model binds a value to), `omit`;
91
- clusters `group` or `omit`. Steps: `..` up, `.` none, `+name` an element of the value's own, `=name` the element of that
92
- name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is reported, never dropped. `place(record, mapping)` turns a record and its
93
- mapping into placements a mapper writes from.
91
+ Transforms: `copy`, `code` (a table), `format`, `constant`, `binding` (the code the model defines an enumerated value
92
+ by), `link` (the code of the concept the component links to), `omit`; clusters `group` or `omit`. `binding` and `link`
93
+ take a `system` (`http://loinc.org`, `http://snomed.info/sct`, a code list's IRI); their text is the model's, so a mapper
94
+ reports it as a lookup (`LOOKUP_TRANSFORMS`). Steps: `..` up, `.` none, `+name` an element of the value's own, `=name`
95
+ the element of that name the parent already has. `within:` restricts a rule to a context. A value no rule reaches is
96
+ reported, never dropped. `place(record, mapping)` turns a record and its mapping into placements a mapper writes from.
97
+
98
+ ## Codes the model names
99
+
100
+ `sdc_mapper.codes` reads the IRIs the component libraries use for a concept as a code system and a code: LOINC codes
101
+ and answers, SNOMED CT, NCI Thesaurus, and any code list whose IRI ends in `#code` (UBL's, HL7's). A code system is
102
+ named by the IRI FHIR uses for it; a mapper turns that into its own name for the system (OMOP's `vocabulary_id`, HL7
103
+ v2's coding system). `bound_code(value)` is the code an enumerated value is defined by, `linked_code(value)` the code
104
+ of the concept its component links to (`skos:exactMatch`, then `dcterms:subject`), and `model_codes(value)` every code
105
+ the model names for it: a target code that is one of these came from the model.
106
+
107
+ ## Writing to a target
108
+
109
+ `place()` gives each value its target steps. Steps written `=name` join an element made by another value, so a writer
110
+ holds those writes back until every other value is written, then writes them shallowest first: sort them by
111
+ `join_depth(steps)`. A row-oriented target (a set of tables, as OMOP's) can treat each step as a table or a column,
112
+ `+TABLE` as one new row per value, and keep the record's own row (its person, its document) under a name of its own;
113
+ `sdc-gdr2omop` does so with a `RECORD` table.
94
114
 
95
115
  ## Writing a mapper
96
116
 
@@ -4,6 +4,7 @@ README.md
4
4
  pyproject.toml
5
5
  src/sdc_mapper/__init__.py
6
6
  src/sdc_mapper/cli.py
7
+ src/sdc_mapper/codes.py
7
8
  src/sdc_mapper/contract.py
8
9
  src/sdc_mapper/engine.py
9
10
  src/sdc_mapper/gate.py
@@ -23,6 +24,7 @@ src/sdc_mapper.egg-info/top_level.txt
23
24
  src/sdc_mapper/schemas/log-entry.schema.json
24
25
  src/sdc_mapper/schemas/mapping.schema.json
25
26
  src/sdc_mapper/schemas/sdc4.xsd
27
+ tests/test_codes.py
26
28
  tests/test_core.py
27
29
  tests/test_placement.py
28
30
  tests/test_service.py
@@ -0,0 +1,107 @@
1
+ """Codes the model names (4.3.0): the IRI forms, `binding` with a code system, `link`, and the join order."""
2
+ from __future__ import annotations
3
+
4
+ import dataclasses
5
+
6
+ import pytest
7
+
8
+ from sdc_mapper import (JOIN, LOINC, LOOKUP_TRANSFORMS, NCIT, SNOMED_CT, CodeRef, MappingError, code_of, join_depth,
9
+ model_codes, place)
10
+ from test_placement import mapping, resolved
11
+
12
+ UBL_LIST = "http://docs.oasis-open.org/ubl/os-UBL-2.3/cl/gc/default/PaymentMeansCode-2.3.gc"
13
+
14
+
15
+ @pytest.mark.parametrize("iri,expected", [
16
+ ("https://loinc.org/8480-6/", CodeRef(LOINC, "8480-6")),
17
+ ("http://loinc.org/rdf/8480-6", CodeRef(LOINC, "8480-6")),
18
+ ("https://loinc.org/LA32-8/", CodeRef(LOINC, "LA32-8")),
19
+ ("http://snomed.info/id/38341003", CodeRef(SNOMED_CT, "38341003")),
20
+ ("http://snomed.info/sct/38341003", CodeRef(SNOMED_CT, "38341003")),
21
+ ("http://purl.obolibrary.org/obo/NCIT_C17459", CodeRef(NCIT, "C17459")),
22
+ (f"{UBL_LIST}#2", CodeRef(UBL_LIST, "2")),
23
+ ("http://terminology.hl7.org/CodeSystem/v3-NullFlavor#ASKU",
24
+ CodeRef("http://terminology.hl7.org/CodeSystem/v3-NullFlavor", "ASKU")),
25
+ ])
26
+ def test_the_iri_forms_the_libraries_use(iri, expected):
27
+ assert code_of(iri) == expected
28
+
29
+
30
+ @pytest.mark.parametrize("iri", ["http://loinc.org", "https://www.gs1.org/standards/id-keys", "", None,
31
+ "https://wwwn.cdc.gov/Nchs/Nhanes/2017-2018/DEMO_J.htm"])
32
+ def test_a_whole_system_or_a_page_names_no_code(iri):
33
+ assert code_of(iri) is None
34
+
35
+
36
+ def _value_with(rs, **changes):
37
+ v = next(v for v in rs.values if v.kind == "string" and v.value)
38
+ return dataclasses.replace(v, **changes)
39
+
40
+
41
+ def test_link_writes_the_code_of_the_concept_the_component_links_to(tmp_path):
42
+ rs = resolved()
43
+ links = {"http://www.w3.org/2004/02/skos/core#exactMatch": ["https://loinc.org/8480-6/"],
44
+ "http://purl.org/dc/terms/subject": ["http://snomed.info/id/38341003"]}
45
+ v = _value_with(rs, links=links)
46
+ rs.values[v.position] = v
47
+ m = mapping(tmp_path, [{"component": v.ct_id, "to": "/x:Value", "transform": "copy",
48
+ "also": [{"to": "/x:Code", "transform": "link", "system": LOINC},
49
+ {"to": "/x:Subject", "transform": "link", "system": SNOMED_CT}]}])
50
+ p = next(p for p in place(rs, m)[0] if p.value.position == v.position)
51
+ assert [(e.steps[-1][0], e.text) for e in p.extras] == [("x:Code", "8480-6"), ("x:Subject", "38341003")]
52
+ assert model_codes(v) == {"8480-6", "38341003"}
53
+
54
+
55
+ def test_link_is_written_beside_an_exceptional_value_and_a_miss_fails(tmp_path):
56
+ from sdc_mapper.resolve import Exceptional
57
+ rs = resolved()
58
+ v = _value_with(rs, links={"http://www.w3.org/2004/02/skos/core#exactMatch": ["https://loinc.org/8480-6/"]},
59
+ exceptional=Exceptional(code="ASKR", name="asked and refused", origin="source"))
60
+ rs.values[v.position] = v
61
+ rule = {"component": v.ct_id, "to": "/x:Code", "transform": "link"}
62
+ p = next(p for p in place(rs, mapping(tmp_path, [rule]))[0] if p.value.position == v.position)
63
+ assert p.text == "8480-6", "the concept is known even when the value is not"
64
+ with pytest.raises(MappingError, match="links to no code in http://snomed"):
65
+ place(rs, mapping(tmp_path, [{**rule, "system": SNOMED_CT}]))
66
+
67
+
68
+ def test_binding_takes_a_code_system_and_reads_every_form(tmp_path):
69
+ rs = resolved()
70
+ bound = next(v for v in rs.values if v.code and v.code.defined_by and "#" in v.code.defined_by)
71
+ system = bound.code.defined_by.rsplit("#", 1)[0]
72
+ rule = {"component": bound.ct_id, "to": "/x:Code", "transform": "binding", "system": system}
73
+ p = next(p for p in place(rs, mapping(tmp_path, [rule]))[0] if p.value.position == bound.position)
74
+ assert p.text == bound.code.defined_by.rsplit("#", 1)[1]
75
+ loinc_answer = dataclasses.replace(bound, code=dataclasses.replace(bound.code, defined_by="https://loinc.org/LA33-6/"))
76
+ rs.values[bound.position] = loinc_answer
77
+ p = next(p for p in place(rs, mapping(tmp_path, [{**rule, "system": LOINC}]))[0] if p.value.position == bound.position)
78
+ assert p.text == "LA33-6"
79
+
80
+
81
+ def test_join_depth_orders_deferred_writes():
82
+ a, b, c = ("x:A", ()), ("x:B", JOIN), ("x:C", JOIN)
83
+ assert join_depth([a]) == -1 and join_depth([a, b]) == 1 and join_depth([b, a, c]) == 2
84
+ assert LOOKUP_TRANSFORMS == {"binding", "link"}
85
+
86
+
87
+ @pytest.mark.parametrize("iri", ["https://wwwn.cdc.gov/Nchs/Nhanes/2017-2018/DEMO_J.htm#SEQN",
88
+ "http://www.w3.org/ns/prov#Entity", "http://www.w3.org/2000/01/rdf-schema#label"])
89
+ def test_a_term_or_a_page_anchor_is_no_concept_code(iri, tmp_path):
90
+ """4.3.1: a `#fragment` in a concept link is a vocabulary term or an anchor on a page, not a code."""
91
+ assert code_of(iri, code_lists=False) is None
92
+ rs = resolved()
93
+ v = _value_with(rs, links={"http://www.w3.org/2004/02/skos/core#exactMatch": [iri, "https://loinc.org/8480-6/"]})
94
+ rs.values[v.position] = v
95
+ assert model_codes(v) == {"8480-6"}, "the link to LOINC is a code; the fragment is not"
96
+ p = next(p for p in place(rs, mapping(tmp_path, [{"component": v.ct_id, "to": "/x:Code", "transform": "link"}]))[0]
97
+ if p.value.position == v.position)
98
+ assert p.text == "8480-6"
99
+ only = _value_with(rs, links={"http://www.w3.org/2004/02/skos/core#exactMatch": [iri]})
100
+ rs.values[only.position] = only
101
+ with pytest.raises(MappingError, match="links to no code"):
102
+ place(rs, mapping(tmp_path, [{"component": only.ct_id, "to": "/x:Code", "transform": "link"}]))
103
+
104
+
105
+ def test_a_code_list_entry_still_binds_an_enumerated_value():
106
+ assert code_of(f"{UBL_LIST}#2") == CodeRef(UBL_LIST, "2")
107
+ assert code_of(f"{UBL_LIST}#2", code_lists=False) is None
File without changes
File without changes
File without changes