iev 0.4.6 → 0.4.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/scheduled_tasks.lock +1 -0
  3. data/.rubocop_todo.yml +5 -47
  4. data/CLAUDE.md +11 -1
  5. data/Gemfile +4 -2
  6. data/TODO.reconcile/01-design.md +57 -0
  7. data/TODO.reconcile/02-change-models.md +44 -0
  8. data/TODO.reconcile/02-termbase-loader.md +48 -0
  9. data/TODO.reconcile/03-live-loader.md +28 -0
  10. data/TODO.reconcile/03-status-mapper.md +24 -0
  11. data/TODO.reconcile/04-concept-merger.md +45 -0
  12. data/TODO.reconcile/04-termbase-loader.md +28 -0
  13. data/TODO.reconcile/05-content-diff.md +45 -0
  14. data/TODO.reconcile/05-live-loader.md +20 -0
  15. data/TODO.reconcile/06-content-differ.md +32 -0
  16. data/TODO.reconcile/06-v3-serializer.md +42 -0
  17. data/TODO.reconcile/07-cli-and-run.md +27 -0
  18. data/TODO.reconcile/07-concept-merger.md +32 -0
  19. data/TODO.reconcile/08-report.md +53 -0
  20. data/TODO.reconcile/09-pipeline.md +23 -0
  21. data/TODO.reconcile/10-script-and-run.md +15 -0
  22. data/data/locales/glossarist_enums.yml +705 -0
  23. data/iev.gemspec +4 -2
  24. data/lib/iev/bibliography_builder.rb +87 -0
  25. data/lib/iev/cli/command.rb +29 -0
  26. data/lib/iev/cli/command_helper.rb +1 -6
  27. data/lib/iev/config.rb +10 -1
  28. data/lib/iev/exporter.rb +37 -4
  29. data/lib/iev/figure_builder.rb +186 -0
  30. data/lib/iev/multi_doc_yaml.rb +60 -0
  31. data/lib/iev/reconciler/change.rb +29 -0
  32. data/lib/iev/reconciler/change_set.rb +48 -0
  33. data/lib/iev/reconciler/concept_merger.rb +130 -0
  34. data/lib/iev/reconciler/content_differ.rb +157 -0
  35. data/lib/iev/reconciler/live_loader.rb +163 -0
  36. data/lib/iev/reconciler/pipeline.rb +104 -0
  37. data/lib/iev/reconciler/reconciled_concept.rb +14 -0
  38. data/lib/iev/reconciler/report.rb +118 -0
  39. data/lib/iev/reconciler/status_mapper.rb +29 -0
  40. data/lib/iev/reconciler/term_marker_parser.rb +196 -0
  41. data/lib/iev/reconciler/termbase_loader.rb +141 -0
  42. data/lib/iev/reconciler.rb +17 -0
  43. data/lib/iev/scraper/browser.rb +155 -50
  44. data/lib/iev/scraper/page_parser.rb +116 -24
  45. data/lib/iev/source_parser.rb +12 -1
  46. data/lib/iev/subject_area_concepts.rb +4 -2
  47. data/lib/iev/utilities.rb +7 -5
  48. data/lib/iev/version.rb +1 -1
  49. data/lib/iev.rb +4 -0
  50. data/scripts/find_unparseable.rb +22 -0
  51. metadata +54 -6
data/TODO.reconcile/08-report.md ADDED
@@ -0,0 +1,53 @@
1
+ # 08 — Report
2
+
3
+ ## Goal
4
+
5
+ Generate dataset-level change reports from the reconciled concepts.
6
+ This answers "what changed across the entire IEV?"
7
+
8
+ ## Output files
9
+
10
+ ```
11
+ report/
12
+ summary.yaml # aggregate statistics
13
+ changes.csv # one row per field-level change
14
+ retired.yaml # concepts retired since termbase
15
+ new_concepts.yaml # concepts added since termbase
16
+ ```
17
+
18
+ ## summary.yaml
19
+
20
+ ```yaml
21
+ total_concepts: 23058
22
+ sources:
23
+ in_both: 16643
24
+ termbase_only: 5585
25
+ live_only: 830
26
+ changes:
27
+ total: 5499
28
+ by_field:
29
+ definition: 4200
30
+ designation: 350
31
+ notes: 580
32
+ examples: 120
33
+ status: 249
34
+ by_language:
35
+ eng: 5000
36
+ fra: 450
37
+ deu: 49
38
+ by_section:
39
+ "113-01": 45
40
+ "102-01": 12
41
+ ...
42
+ ```
43
+
44
+ ## changes.csv
45
+
46
+ Columns: `code, section, field, language, detected_at, old_value, new_value`
47
+
48
+ ## Checklist
49
+
50
+ - [x] Implement Report
51
+ - [x] Spec: summary counts correct
52
+ - [x] Spec: CSV format correct
53
+ - [x] Spec: retired/new lists correct
data/TODO.reconcile/09-pipeline.md ADDED
@@ -0,0 +1,23 @@
1
+ # 09 — Pipeline
2
+
3
+ ## Goal
4
+
5
+ Orchestrate the full reconciliation: load → merge → serialize → report.
6
+
7
+ ## Flow
8
+
9
+ ```
10
+ Pipeline#run
11
+ ├── TermbaseLoader.load(path) → Hash<code, ManagedConcept>
12
+ ├── LiveLoader.load(dir) → Hash<code, ManagedConcept>
13
+ ├── for each code in union:
14
+ │ ├── ConceptMerger.merge(code, tb, live) → ReconciledConcept
15
+ │ └── collect ReconciledConcept
16
+ ├── save concepts to output/concepts/*.yaml
17
+ └── Report.new(reconciled).write_to(output/report/)
18
+ ```
19
+
20
+ ## Checklist
21
+
22
+ - [x] Implement Pipeline
23
+ - [x] Spec: pipeline produces expected output with test fixtures
data/TODO.reconcile/10-script-and-run.md ADDED
@@ -0,0 +1,15 @@
1
+ # 10 — Script wrapper + end-to-end run
2
+
3
+ ## Goal
4
+
5
+ Rewrite `scripts/reconcile.rb` as a thin wrapper around the library
6
+ code. Then run the full pipeline and verify output.
7
+
8
+ ## Checklist
9
+
10
+ - [x] Rewrite scripts/reconcile.rb as thin wrapper
11
+ - [x] Run end-to-end pipeline
12
+ - [x] Verify concepts/*.yaml output
13
+ - [x] Verify report/*.yaml output
14
+ - [x] Spot-check 5 concepts (in-both, retired, new, with changes)
15
+ - [x] Spot-check report accuracy