iodef 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ec203460113e786d9571db8360e79b7514881fbf38a2af7539813cdf4608069c
+  data.tar.gz: accc3826848d943da9d860d91d7889099fdad3c3141771737d0315c6d9da98af
+SHA512:
+  metadata.gz: 775da0539aa19b31c1b914ac29e6bd7426cd594dcbcac2ab9c5b0d4a3b74681457d7d9f67cb0cf85e40fddae0121787fb1cc1ce4dd641133139e4b1dbbd42ed8
+  data.tar.gz: b770d06ee184c30f732d6ff69c9b367584915008e3d1cfba2562c8cc9a33a86cdb0a7e4190c910ec26a6c9956572bc517c63293ece3cb98226a5a3e045afefca

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0 - 2026-10-12
+
+Initial release.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Yudai Takada
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,111 @@
+# IODEF
+
+Read, validate, build, and exchange incident reports using the JSON binding of IODEF v2 ([RFC 8727](https://www.rfc-editor.org/rfc/rfc8727.html), with the information model in [RFC 7970](https://www.rfc-editor.org/rfc/rfc7970.html)).
+
+```ruby
+require "iodef"
+
+document = IODEF.parse(json)
+incident = document.incidents.first
+incident.incident_id
+incident.contacts.first.role
+incident.description.text("ja")
+
+issues = IODEF.validate(json)
+issues.each { |issue| puts "#{issue.pointer}: #{issue.rule}: #{issue.message}" }
+```
+
+Install with `gem install iodef`. Ruby 3.3 or newer, JRuby, and TruffleRuby are supported. The only required runtime gem is `json` (2.19.4 or newer, including 3.x); no other gem from this project is required. `IODEF` and `Iodef` refer to the same module.
+
+## Build and edit a report
+
+```ruby
+document = IODEF::Document.build(lang: "en") do |doc|
+  doc.add_incident(purpose: "reporting") do |incident|
+    incident.incident_id = { id: "case-24", name: "response.example" }
+    incident.generation_time = "2026-10-01T09:00:00Z"
+    incident.description = ["Unexpected traffic", { value: "不審な通信", lang: "ja" }]
+    incident.add_contact(role: "creator", type: "organization") do |contact|
+      contact.contact_name = ["Response team"]
+      contact.email = [{ "EmailTo" => "response@response.example" }]
+    end
+  end
+end
+
+json = document.to_json
+```
+
+Builders require the incident's ID, generation time, and at least one contact. They do not invent reporting identities or timestamps. Known properties use snake_case accessors; wire property names retain their RFC spelling. `Document`, `Incident`, `Contact`, `Assessment`, `EventData`, and `Indicator` have typed views. Other objects are ordinary hashes. Plural conveniences include `incidents`, `contacts`, `assessments`, and `indicators`.
+
+Typed child views update their document. `[]` exposes the underlying value, `[]=` copies the assigned value, and `to_h` returns a detached copy. Symbols in builder hashes become strings. Unknown JSON properties are preserved, including nested values and large integers. Defaults such as restriction levels are not inserted into the source data.
+
+## Validation and limits
+
+Parsing is strict by default. `IODEF.parse(json, strict: false)` returns a document with a `warnings` array of `Violation` objects. `validate: false` is a compatibility spelling for this inspection mode. `validate!` refreshes diagnostics after edits. Serialization always validates and raises `IODEF::ValidationError` with `violations` and the first JSON `pointer`.
+
+Required properties, types, nonempty collections, enum values, extension selector pairs, identifiers/references, dates, binary encodings, and information-model conditions are checked. Unknown properties remain available as extensions; their semantics are not inferred. Validation never fetches referenced URLs, schemas, or other network resources. XML, JSON, or regex text embedded in a report is not executed. JSON-valued extension strings are checked for JSON syntax.
+
+| Option | Default |
+| --- | ---: |
+| `max_input_size` | 16 MiB of input/output; also bounds copied string data |
+| `max_depth` | 32 nested levels |
+| `max_collection_size` | 10,000 members per array or object |
+| `max_nodes` | 200,000 JSON/CBOR values |
+| `max_binary_size` | 1 MiB per decoded binary value |
+| `max_violations` | 1,000 diagnostics |
+
+Pass overrides to `parse`, `validate`, `parse_cbor`, or model constructors. Limits are positive integers. Resource-limit failures raise `LimitError` even in inspection mode. Malformed/duplicate-key JSON raises `ParseError`; cycles, nonfinite numbers, invalid string encodings, and non-JSON Ruby objects are rejected. Diagnostics identify properties and rules without including incident values.
+
+## Optional CBOR
+
+Install the optional `cbor` gem (`~> 0.5.10`), then:
+
+```ruby
+require "iodef/cbor"
+bytes = document.to_cbor
+copy = IODEF.parse_cbor(bytes)
+```
+
+All 207 numeric map keys are supported. The encoder uses date tag 0, base64 byte-string tag 22, hexadecimal byte-string tag 23, and URI tag 32. Dates and URI spelling are preserved. Typed byte content becomes binary; hex output uses uppercase. The decoder also accepts contextually untagged strings/bytes, as used in the RFC's own CBOR examples. Integer bignums are preserved. IODEF real values are encoded as float32; values that would round or overflow raise `ConversionError`. Unknown JSON floats keep their original precision.
+
+Unknown CBOR tags, unassigned integer keys, and opaque byte strings without an IODEF type cannot be represented unambiguously in JSON and raise `ConversionError`. Duplicate/colliding keys, excessive nesting, large collections, and trailing data are rejected before native decoding. The optional native backend is tested on CRuby 3.3–4.0 and TruffleRuby. JRuby and CRuby 4.1 development builds support the JSON API; the current `cbor` extension uses a C API removed in CRuby 4.1 and cannot be built there.
+
+## Scope
+
+| Supported in 0.1.0 | Outside 0.1.0 |
+| --- | --- |
+| JSON binding, all RFC 8727 class/property definitions | RFC 7970 XML import/export |
+| Typed core models, multilingual text, builders | RID or other exchange transports |
+| IODEF enum extensions and preserved unknown JSON fields | Vendor extension semantics |
+| Optional CBOR, numeric keys, typed binary values | Signing, encryption, threat correlation |
+| RBS signatures and bounded validation | Authenticating senders or verifying referenced evidence |
+
+The implementation has independent, MIT-licensed property metadata in `data/`. It does not embed CDDL, JSON Schema, RFC examples, IANA source downloads, or third-party implementation code. CDDL is a development-only conformance oracle fetched into ignored `tmp/`; it is not a runtime dependency.
+
+### Specification differences
+
+RFC 8727's CDDL has a verified missing-brace correction ([erratum 7334](https://www.rfc-editor.org/errata/eid7334)). The generated test oracle applies it. Its section 4.2 JSON example needs presentation-wrapped strings joined; the original text is retained beside the corrected fixture. Both complete JSON and CBOR examples are tested.
+
+The JSON examples use version `2.0`; the XML information model uses `2.00`. Both spellings are accepted and preserved. Required nonempty child groups and conditional values follow RFC 7970: nonnumeric `Confidence` omits `value`, numeric confidence requires it; a node can include both addresses and domain data; `HashData` supports `ext-scope`. `RelatedActivity.Description` accepts multilingual strings. These constraints are not all accurately expressed in the RFC 8727 CDDL. The section 2.2.5 illustration uses a CWE `SpecID` absent from its normative enum: use `private` with `ext-SpecID` for that extension.
+
+The pinned IANA registry and RFC have differing spellings for `breach-propietary` / `breach-proprietary`, `anonymization` / `anomyzation`, and `ntp-stamp` / `ntpstamp`; both are accepted. RFC 8727 additionally defines the `json` extension data type. This library validates format and structure, not whether a reported address, software identifier, or incident fact is authentic.
+
+## Development
+
+```sh
+bundle install
+bundle exec rake test
+COVERAGE=1 bundle exec rake test
+bundle exec rubocop
+bundle exec rbs -I sig validate
+bundle exec ruby script/interop.rb
+bundle exec ruby script/benchmark.rb 5000
+```
+
+The fixture task verifies SHA-256-pinned RFC/IANA sources, retains source notices, and regenerates examples under ignored `tmp/rfc/`. Interoperability checks validate the two complete examples against downloaded CDDL and exchange CBOR with a pinned Python `cbor2` implementation, also kept under ignored `tmp/` with its license. The property table is checked against every CDDL object definition, and tests cover malformed values, enum extensions, limits, randomized round trips, and numeric precision.
+
+`script/update_registry.rb` audits the pinned IANA metadata. `--refresh` downloads the current registry, rejects unexplained differences, and updates its checksum only after the audit succeeds. Review RFC errata and registry changes before refreshing. No downloaded reference material is tracked or included in the gem.
+
+## License
+
+MIT. See [LICENSE.txt](LICENSE.txt). Optional dependencies and temporary development references retain their own licenses.

data/SECURITY.md

@@ -0,0 +1,7 @@
+# Security
+
+Report suspected vulnerabilities privately through [GitHub's vulnerability reporting page](https://github.com/ydah/iodef/security/advisories/new). Include the gem version, Ruby engine, and a small synthetic reproducer. Do not include operational incident reports, personal data, credentials, or private indicators in public issues.
+
+IODEF documents contain untrusted, potentially sensitive content. This library validates representation and selected information-model rules; it does not authenticate reports or enforce disclosure restrictions. Apply authorization, transport encryption, retention policies, and trusted source selection in the calling application. Do not execute embedded scripts, XML, regexes, or extension payloads. The library itself never fetches document-supplied URLs or schemas.
+
+Size, depth, node, collection, binary, and diagnostic limits apply by default. Increase them only for known workloads. The optional native CBOR backend should be kept updated; only the IODEF tag set is allowed before it receives input. Invalid or ambiguous CBOR values fail instead of silently dropping information.

data/data/enums.yml

@@ -0,0 +1,40 @@
+# Protocol identifiers, maintained independently from RFC source code and schemas.
+restriction: [public, partner, need-to-know, private, default, white, green, amber, red, ext-value]
+purpose: [traceback, mitigation, reporting, watch, other, ext-value]
+status: [new, in-progress, forwarded, resolved, future, ext-value]
+contact_role: [creator, reporter, admin, tech, provider, user, billing, legal, irt, abuse, cc, cc-irt, leo, vendor, vendor-support, victim, victim-notified, ext-value]
+contact_type: [person, organization, ext-value]
+registry: [internic, apnic, arin, lacnic, ripe, afrinic, local, ext-value]
+postal_type: [street, mailing, ext-value]
+email_type: [direct, hotline, ext-value]
+telephone_type: [wired, mobile, fax, hotline, ext-value]
+discovery_source: [nidps, hips, siem, av, third-party-monitoring, incident, os-log, application-log, device-log, network-flow, passive-dns, investigation, audit, internal-notification, external-notification, leo, partner, actor, unknown, ext-value]
+spec_id: ['urn:ietf:params:xml:ns:mile:mmdef:1.2', private]
+occurrence: [actual, potential]
+severity: [low, medium, high]
+completion: [failed, succeeded]
+system_impact: [takeover-account, takeover-service, takeover-system, cps-manipulation, cps-damage, availability-data, availability-account, availability-service, availability-system, damaged-system, damaged-data, breach-proprietary, breach-propietary, breach-privacy, breach-credential, breach-configuration, integrity-data, integrity-configuration, integrity-hardware, traffic-redirection, monitoring-traffic, monitoring-host, policy, unknown, ext-value]
+business_impact: [breach-proprietary, breach-privacy, breach-credential, loss-of-integrity, loss-of-service, theft-financial, theft-service, degraded-reputation, asset-damage, asset-manipulation, legal, extortion, unknown, ext-value]
+business_severity: [none, low, medium, high, unknown, ext-value]
+time_metric: [labor, elapsed, downtime, ext-value]
+duration: [second, minute, hour, day, month, quarter, year, ext-value]
+rating: [low, medium, high, numeric, unknown, ext-value]
+action: [nothing, contact-source-site, contact-target-site, contact-sender, investigate, block-host, block-network, block-port, rate-limit-host, rate-limit-network, rate-limit-port, redirect-traffic, honeypot, upgrade-software, rebuild-asset, harden-asset, remediate-other, status-triage, status-new-info, watch-and-report, training, defined-coa, other, ext-value]
+system_category: [source, target, intermediate, sensor, infrastructure, ext-value]
+yes_no_unknown: ['yes', 'no', unknown]
+ownership: [organization, personal, partner, customer, no-relationship, unknown, ext-value]
+address_category: [asn, atm, e-mail, ipv4-addr, ipv4-net, ipv4-net-masked, ipv4-net-mask, ipv6-addr, ipv6-net, ipv6-net-masked, mac, site-uri, ext-value]
+node_category: [client, client-enterprise, client-partner, client-remote, client-kiosk, client-mobile, server-internal, server-public, www, mail, webmail, messaging, streaming, voice, file, ftp, p2p, name, directory, credential, print, application, database, backup, dhcp, assessment, source-control, config-management, monitoring, infra, infra-firewall, infra-router, infra-switch, camera, proxy, remote-access, log, virtualization, pos, scada, scada-supervisory, sinkhole, honeypot, anomyzation, anonymization, c2-server, malware-distribution, drop-server, hop-point, reflector, phishing-site, spear-phishing-site, recruiting-site, fraudulent-site, ext-value]
+counter_type: [count, peak, average, ext-value]
+counter_unit: [byte, mbit, packet, flow, session, alert, message, event, host, site, organization, ext-value]
+system_status: [spoofed, fraudulent, innocent-hacked, innocent-hijacked, unknown, ext-value]
+domain_status: [reservedDelegation, assignedAndActive, assignedAndInactive, assignedAndOnHold, revoked, transferPending, registryLock, registrarLock, other, unknown, ext-value]
+pattern_type: [regex, binary, xpath, ext-value]
+offset_unit: [line, byte, ext-value]
+registry_action: [add-key, add-value, delete-key, delete-value, modify-key, modify-value, ext-value]
+hash_scope: [file-contents, file-pe-section, file-pe-iat, file-pe-resource, file-pdf-object, email-hash, email-headers-hash, email-body-hash, ext-value]
+bulk_type: [asn, atm, e-mail, ipv4-addr, ipv4-net, ipv4-net-mask, ipv6-addr, ipv6-net, ipv6-net-mask, mac, site-uri, domain-name, domain-to-ipv4, domain-to-ipv6, domain-to-ipv4-timestamp, domain-to-ipv6-timestamp, ipv4-port, ipv6-port, windows-reg-key, file-hash, email-x-mailer, email-subject, http-user-agent, http-request-uri, mutex, file-path, user-name, ext-value]
+operator: [not, and, or, xor]
+software_spec: [custom, cpe, swid, ext-value]
+software_dtype: [bytes, integer, real, string, xml, ext-value]
+dtype: [boolean, byte, bytes, character, date-time, ntpstamp, ntp-stamp, integer, portlist, real, string, file, path, frame, packet, ipv4-packet, json, ipv6-packet, url, csv, winreg, xml, ext-value]

data/data/mapkeys.yml

@@ -0,0 +1,5 @@
+# RFC 8727 section 5 wire identifiers, in integer order starting at -24.
+first: -24
+names: >-
+  version lang format-id private-enum-name private-enum-id Incident AdditionalData value translation-id name dtype ext-dtype meaning formatid restriction ext-restriction observable-id SoftwareReference URL Description spec-name ext-spec-name purpose ext-purpose
+  status ext-status IncidentID AlternativeID RelatedActivity DetectTime StartTime EndTime RecoveryTime ReportTime GenerationTime Discovery Assessment Method Contact EventData Indicator History id instance ThreatActor Campaign IndicatorID Confidence ThreatActorID CampaignID role ext-role type ext-type ContactName ContactTitle RegistryHandle PostalAddress Email Telephone Timezone handle registry ext-registry PAddress EmailTo TelephoneNumber source ext-source DetectionPattern DetectionConfiguration Application Reference AttackPattern Vulnerability Weakness SpecID ext-SpecID ContentID RawData Platform Scoring ReferenceName specIndex ID occurrence IncidentCategory Impact SystemImpact BusinessImpact TimeImpact MonetaryImpact IntendedImpact Counter MitigatingFactor Cause severity completion ext-severity metric ext-metric duration ext-duration currency rating ext-rating HistoryItem action ext-action DateTime DefinedCOA System Expectation RecordData category ext-category interface spoofed virtual ownership ext-ownership Node NodeRole Service OperatingSystem AssetID DomainData Address Location vlan-name vlan-num unit ext-unit system-status ext-system-status domain-status ext-domain-status Name DateDomainWasChecked RegistrationDate ExpirationDate RelatedDNS NameServers DomainContacts Server SameDomainContact ip-protocol ServiceName Port Portlist ProtoCode ProtoType ProtoField ApplicationHeaderField EmailData IANAService EmailFrom EmailSubject EmailX-Mailer EmailHeaderField EmailHeaders EmailBody EmailMessage HashData Signature RecordPattern RecordItem FileData WindowsRegistryKeysModified CertificateData offset offsetunit ext-offsetunit Key registryaction ext-registryaction KeyName KeyValue Certificate X509Data File FileName FileSize FileType AssociatedSoftware FileProperties scope HashTargetID Hash FuzzyHash DigestMethod DigestValue CanonicalizationMethod FuzzyHashValue AlternativeIndicatorID Observable uid-ref IndicatorExpression IndicatorReference AttackPhase BulkObservable BulkObservableFormat BulkObservableList operator ext-operator euid-ref AttackPhaseID

data/data/types.yml

@@ -0,0 +1,282 @@
+# Independently maintained implementation metadata. Sections refer to RFC 7970;
+# JSON cardinalities and property spellings follow RFC 8727 sections 2 and 3.
+# A trailing ! requires a property; [] denotes a nonempty list when present.
+groups:
+  restricted: {restriction: restriction, ext-restriction: string}
+  observable: {observable-id: id}
+  described: {Description: 'mlstring[]'}
+  extensible: {AdditionalData: 'Extension[]'}
+  temporal: {DetectTime: datetime, StartTime: datetime, EndTime: datetime, RecoveryTime: datetime, ReportTime: datetime}
+  linked: {URL: 'uri[]'}
+  structured: {SpecID: 'spec_id!', ext-SpecID: string, ContentID: string, RawData: 'binary[]', Reference: 'Reference[]'}
+types:
+  Document:
+    section: '3.1'
+    groups: [extensible]
+    fields: {version: 'version!', lang: language, format-id: string, private-enum-name: string, private-enum-id: string, Incident: 'Incident[]!'}
+  Incident:
+    section: '3.2'
+    groups: [restricted, observable, described, extensible, temporal]
+    fields: {purpose: 'purpose!', ext-purpose: string, status: status, ext-status: string, lang: language, IncidentID: 'IncidentID!', AlternativeID: AlternativeID, RelatedActivity: 'RelatedActivity[]', GenerationTime: 'datetime!', Discovery: 'Discovery[]', Assessment: 'Assessment[]', Method: 'Method[]', Contact: 'Contact[]!', EventData: 'EventData[]', Indicator: 'Indicator[]', History: History}
+  IncidentID:
+    section: '3.4'
+    groups: [restricted]
+    fields: {id: 'string!', name: 'string!', instance: string}
+  AlternativeID:
+    section: '3.5'
+    groups: [restricted]
+    fields: {IncidentID: 'IncidentID[]!'}
+  RelatedActivity:
+    section: '3.6'
+    groups: [restricted, described, extensible, linked]
+    fields: {IncidentID: 'IncidentID[]', ThreatActor: 'ThreatActor[]', Campaign: 'Campaign[]', IndicatorID: 'IndicatorID[]', Confidence: Confidence}
+    at_least_one: [IncidentID, URL, ThreatActor, Campaign, IndicatorID, Confidence, Description, AdditionalData]
+  ThreatActor:
+    section: '3.7'
+    groups: [restricted, described, extensible, linked]
+    fields: {ThreatActorID: 'string[]'}
+    at_least_one: [ThreatActorID, URL, Description, AdditionalData]
+  Campaign:
+    section: '3.8'
+    groups: [restricted, described, extensible, linked]
+    fields: {CampaignID: 'string[]'}
+    at_least_one: [CampaignID, URL, Description, AdditionalData]
+  Contact:
+    section: '3.9'
+    groups: [restricted, described, extensible]
+    fields: {role: 'contact_role!', ext-role: string, type: 'contact_type!', ext-type: string, ContactName: 'mlstring[]', ContactTitle: 'mlstring[]', RegistryHandle: 'RegistryHandle[]', PostalAddress: 'PostalAddress[]', Email: 'Email[]', Telephone: 'Telephone[]', Timezone: timezone, Contact: 'Contact[]'}
+    at_least_one: [ContactName, ContactTitle, Description, RegistryHandle, PostalAddress, Email, Telephone, Timezone, Contact, AdditionalData]
+  RegistryHandle:
+    section: '3.9.1'
+    fields: {handle: 'string!', registry: 'registry!', ext-registry: string}
+  PostalAddress:
+    section: '3.9.2'
+    groups: [described]
+    fields: {type: postal_type, ext-type: string, PAddress: 'mlstring!'}
+  Email:
+    section: '3.9.3'
+    groups: [described]
+    fields: {type: email_type, ext-type: string, EmailTo: 'string!'}
+  Telephone:
+    section: '3.9.4'
+    groups: [described]
+    fields: {type: telephone_type, ext-type: string, TelephoneNumber: 'string!'}
+  Discovery:
+    section: '3.10'
+    groups: [restricted, described]
+    fields: {source: discovery_source, ext-source: string, Contact: 'Contact[]', DetectionPattern: 'DetectionPattern[]'}
+  DetectionPattern:
+    section: '3.10.1'
+    groups: [restricted, observable, described]
+    fields: {DetectionConfiguration: 'string[]', Application: 'Software!'}
+    at_least_one: [Description, DetectionConfiguration]
+  Method:
+    section: '3.11'
+    groups: [restricted, described, extensible]
+    fields: {Reference: 'Reference[]', AttackPattern: 'StructuredInformation[]', Vulnerability: 'StructuredInformation[]', Weakness: 'StructuredInformation[]'}
+    at_least_one: [Reference, Description, AttackPattern, Vulnerability, Weakness, AdditionalData]
+  Reference:
+    section: '3.11.1'
+    groups: [observable, described, linked]
+    fields: {ReferenceName: ReferenceName}
+    at_least_one: [ReferenceName, URL, Description]
+  ReferenceName:
+    section: '3.11.1.1'
+    fields: {specIndex: 'integer!', ID: 'id!'}
+  Assessment:
+    section: '3.12'
+    groups: [restricted, observable, extensible]
+    fields: {occurrence: occurrence, IncidentCategory: 'mlstring[]', Impact: 'Impact[]!', Counter: 'Counter[]', MitigatingFactor: 'mlstring[]', Cause: 'mlstring[]', Confidence: Confidence}
+  Impact:
+    section: '3.12'
+    fields: {SystemImpact: SystemImpact, BusinessImpact: BusinessImpact, TimeImpact: TimeImpact, MonetaryImpact: MonetaryImpact, IntendedImpact: BusinessImpact}
+    exactly_one: [SystemImpact, BusinessImpact, TimeImpact, MonetaryImpact, IntendedImpact]
+  SystemImpact:
+    section: '3.12.1'
+    groups: [described]
+    fields: {type: 'system_impact!', ext-type: string, severity: severity, completion: completion}
+  BusinessImpact:
+    section: '3.12.2'
+    groups: [described]
+    fields: {type: 'business_impact!', ext-type: string, severity: business_severity, ext-severity: string}
+  TimeImpact:
+    section: '3.12.3'
+    fields: {value: 'positive!', severity: severity, metric: 'time_metric!', ext-metric: string, duration: duration, ext-duration: string}
+  MonetaryImpact:
+    section: '3.12.4'
+    fields: {value: 'positive!', severity: severity, currency: string}
+  Confidence:
+    section: '3.12.5'
+    fields: {rating: 'rating!', ext-rating: string, value: number}
+  History:
+    section: '3.13'
+    groups: [restricted]
+    fields: {HistoryItem: 'HistoryItem[]!'}
+  HistoryItem:
+    section: '3.13.1'
+    groups: [restricted, observable, described, extensible]
+    fields: {action: 'action!', ext-action: string, DateTime: 'datetime!', IncidentID: IncidentID, Contact: Contact, DefinedCOA: 'string[]'}
+  EventData:
+    section: '3.14'
+    groups: [restricted, observable, described, extensible, temporal]
+    fields: {Contact: 'Contact[]', Discovery: 'Discovery[]', Assessment: Assessment, Method: 'Method[]', System: 'System[]', Expectation: 'Expectation[]', RecordData: 'RecordData[]', EventData: 'EventData[]'}
+    at_least_one: [Description, DetectTime, StartTime, EndTime, RecoveryTime, ReportTime, Contact, Discovery, Assessment, Method, System, Expectation, RecordData, EventData, AdditionalData]
+  Expectation:
+    section: '3.15'
+    groups: [restricted, observable, described]
+    fields: {action: action, ext-action: string, severity: severity, DefinedCOA: 'string[]', StartTime: datetime, EndTime: datetime, Contact: Contact}
+  System:
+    section: '3.17'
+    groups: [restricted, observable, described, extensible]
+    fields: {category: system_category, ext-category: string, interface: string, spoofed: yes_no_unknown, virtual: yes_no_unknown, ownership: ownership, ext-ownership: string, Node: 'Node!', NodeRole: 'NodeRole[]', Service: 'Service[]', OperatingSystem: 'Software[]', Counter: 'Counter[]', AssetID: 'string[]'}
+  Node:
+    section: '3.18'
+    fields: {DomainData: 'DomainData[]', Address: 'Address[]', PostalAddress: PostalAddress, Location: 'mlstring[]', Counter: 'Counter[]'}
+    at_least_one: [DomainData, Address]
+  Address:
+    section: '3.18.1'
+    groups: [observable]
+    fields: {value: 'string!', category: 'address_category!', ext-category: string, vlan-name: string, vlan-num: integer}
+  NodeRole:
+    section: '3.18.2'
+    groups: [described]
+    fields: {category: 'node_category!', ext-category: string}
+  Counter:
+    section: '3.18.3'
+    fields: {value: 'number!', type: 'counter_type!', ext-type: string, unit: 'counter_unit!', ext-unit: string, meaning: string, duration: duration, ext-duration: string}
+  DomainData:
+    section: '3.19'
+    groups: [observable]
+    fields: {system-status: 'system_status!', ext-system-status: string, domain-status: 'domain_status!', ext-domain-status: string, Name: 'string!', DateDomainWasChecked: datetime, RegistrationDate: datetime, ExpirationDate: datetime, RelatedDNS: 'Extension[]', NameServers: 'NameServers[]', DomainContacts: DomainContacts}
+  NameServers:
+    section: '3.19.1'
+    fields: {Server: 'string!', Address: 'Address[]!'}
+  DomainContacts:
+    section: '3.19.2'
+    fields: {SameDomainContact: string, Contact: 'Contact[]'}
+    exactly_one: [SameDomainContact, Contact]
+  Service:
+    section: '3.20'
+    groups: [observable]
+    fields: {ip-protocol: integer, ServiceName: ServiceName, Port: integer, Portlist: portlist, ProtoCode: integer, ProtoType: integer, ProtoField: integer, ApplicationHeaderField: 'Extension[]', EmailData: EmailData, Application: Software}
+    at_least_one: [ServiceName, Port, Portlist, ProtoCode, ProtoType, ProtoField, ApplicationHeaderField, EmailData, Application]
+  ServiceName:
+    section: '3.20.1'
+    groups: [described, linked]
+    fields: {IANAService: string}
+    at_least_one: [IANAService, URL, Description]
+  EmailData:
+    section: '3.21'
+    groups: [observable]
+    fields: {EmailTo: 'string[]', EmailFrom: string, EmailSubject: string, EmailX-Mailer: string, EmailHeaderField: 'Extension[]', EmailHeaders: string, EmailBody: string, EmailMessage: string, HashData: 'HashData[]', Signature: 'binary[]'}
+  RecordData:
+    section: '3.22'
+    groups: [restricted, observable, described, extensible, linked]
+    fields: {DateTime: datetime, Application: Software, RecordPattern: 'RecordPattern[]', RecordItem: 'Extension[]', FileData: 'FileData[]', WindowsRegistryKeysModified: 'WindowsRegistryKeysModified[]', CertificateData: 'CertificateData[]'}
+    at_least_one: [RecordItem, FileData, WindowsRegistryKeysModified, CertificateData]
+  RecordPattern:
+    section: '3.22.1'
+    fields: {value: 'string!', type: 'pattern_type!', ext-type: string, offset: integer, offsetunit: offset_unit, ext-offsetunit: string, instance: integer}
+  WindowsRegistryKeysModified:
+    section: '3.23'
+    groups: [observable]
+    fields: {Key: 'Key[]!'}
+  Key:
+    section: '3.23.1'
+    groups: [observable]
+    fields: {registryaction: registry_action, ext-registryaction: string, KeyName: 'string!', KeyValue: string}
+  CertificateData:
+    section: '3.24'
+    groups: [restricted, observable]
+    fields: {Certificate: 'Certificate[]!'}
+  Certificate:
+    section: '3.24.1'
+    groups: [observable, described]
+    fields: {X509Data: 'binary!'}
+  FileData:
+    section: '3.25'
+    groups: [restricted, observable]
+    fields: {File: 'File[]!'}
+  File:
+    section: '3.25.1'
+    groups: [observable, linked]
+    fields: {FileName: string, FileSize: integer, FileType: string, HashData: HashData, Signature: 'binary[]', AssociatedSoftware: Software, FileProperties: 'Extension[]'}
+  HashData:
+    section: '3.26'
+    fields: {scope: 'hash_scope!', ext-scope: string, HashTargetID: string, Hash: 'Digest[]', FuzzyHash: 'FuzzyHash[]'}
+    at_least_one: [Hash, FuzzyHash]
+  Digest:
+    section: '3.26.1'
+    fields: {DigestMethod: 'binary!', DigestValue: 'binary!', CanonicalizationMethod: binary, Application: Software}
+  FuzzyHash:
+    section: '3.26.2'
+    groups: [extensible]
+    fields: {FuzzyHashValue: 'Extension[]!', Application: Software}
+  Indicator:
+    section: '3.29'
+    groups: [restricted, described, extensible]
+    fields: {IndicatorID: 'IndicatorID!', AlternativeIndicatorID: 'AlternativeIndicatorID[]', StartTime: datetime, EndTime: datetime, Confidence: Confidence, Contact: 'Contact[]', Observable: Observable, uid-ref: id, IndicatorExpression: IndicatorExpression, IndicatorReference: IndicatorReference, NodeRole: 'NodeRole[]', AttackPhase: 'AttackPhase[]', Reference: 'Reference[]'}
+    exactly_one: [Observable, uid-ref, IndicatorExpression, IndicatorReference]
+  IndicatorID:
+    section: '3.29.1'
+    fields: {id: 'id!', name: 'string!', version: 'string!'}
+  AlternativeIndicatorID:
+    section: '3.29.2'
+    groups: [restricted]
+    fields: {IndicatorID: 'IndicatorID[]!'}
+  Observable:
+    section: '3.29.3'
+    groups: [restricted]
+    fields: {System: System, Address: Address, DomainData: DomainData, EmailData: EmailData, Service: Service, WindowsRegistryKeysModified: WindowsRegistryKeysModified, FileData: FileData, CertificateData: CertificateData, RegistryHandle: RegistryHandle, RecordData: RecordData, EventData: EventData, Incident: Incident, Expectation: Expectation, Reference: Reference, Assessment: Assessment, DetectionPattern: DetectionPattern, HistoryItem: HistoryItem, BulkObservable: BulkObservable, AdditionalData: 'Extension[]'}
+    exactly_one: [System, Address, DomainData, EmailData, Service, WindowsRegistryKeysModified, FileData, CertificateData, RegistryHandle, RecordData, EventData, Incident, Expectation, Reference, Assessment, DetectionPattern, HistoryItem, BulkObservable, AdditionalData]
+  BulkObservable:
+    section: '3.29.3.1'
+    groups: [extensible]
+    fields: {type: bulk_type, ext-type: string, BulkObservableFormat: BulkObservableFormat, BulkObservableList: 'string!'}
+  BulkObservableFormat:
+    section: '3.29.3.1.1'
+    groups: [extensible]
+    fields: {Hash: Digest}
+    at_least_one: [Hash, AdditionalData]
+  IndicatorExpression:
+    section: '3.29.4'
+    groups: [extensible]
+    fields: {operator: operator, ext-operator: string, IndicatorExpression: 'IndicatorExpression[]', Observable: 'Observable[]', uid-ref: 'id[]', IndicatorReference: 'IndicatorReference[]', Confidence: Confidence}
+    at_least_one: [IndicatorExpression, Observable, uid-ref, IndicatorReference]
+  IndicatorReference:
+    section: '3.29.7'
+    fields: {uid-ref: id, euid-ref: string, version: string}
+    exactly_one: [uid-ref, euid-ref]
+  AttackPhase:
+    section: '3.29.8'
+    groups: [described, extensible, linked]
+    fields: {AttackPhaseID: 'string[]'}
+    at_least_one: [AttackPhaseID, URL, Description, AdditionalData]
+  MLString:
+    section: '2.4'
+    fields: {value: 'string!', lang: language, translation-id: string}
+  Software:
+    section: '2.15'
+    groups: [linked, described]
+    fields: {SoftwareReference: SoftwareReference}
+    at_least_one: [SoftwareReference, URL, Description]
+  SoftwareReference:
+    section: '2.15.1'
+    fields: {value: string, spec-name: 'software_spec!', ext-spec-name: string, dtype: software_dtype, ext-dtype: string}
+  StructuredInformation:
+    section: 'RFC8727 2.2.5'
+    groups: [structured]
+    fields: {Platform: 'Platform[]', Scoring: 'Scoring[]'}
+    at_most_one: [RawData, Reference]
+  Platform:
+    section: 'RFC8727 2.2.5'
+    groups: [structured]
+    fields: {}
+  Scoring:
+    section: 'RFC8727 2.2.5'
+    groups: [structured]
+    fields: {}
+  Extension:
+    section: '2.16'
+    groups: [restricted, observable]
+    fields: {value: 'string!', name: string, dtype: 'dtype!', ext-dtype: string, meaning: string, formatid: string}

data/lib/iodef/cbor.rb

@@ -0,0 +1,131 @@
+# frozen_string_literal: true
+
+require "iodef"
+require "cbor"
+require_relative "cbor_guard"
+
+module Iodef
+  module CBORCodec
+    module_function
+
+    Float32 = Struct.new(:value) do
+      def to_cbor(packer)
+        packer.buffer.write("\xfa".b + [value].pack("g"))
+        packer
+      end
+    end
+    private_constant :Float32
+
+    def encode(value, type = "Document")
+      return value.map { |item| encode(item, type.delete_suffix("[]")) } if type.end_with?("[]")
+
+      if value.is_a?(Hash)
+        fields = SCHEMA.fetch(type == "mlstring" ? "MLString" : type, {}).fetch("fields", {})
+        return value.to_h do |key, item|
+          descriptor = fields.fetch(key, "any").delete_suffix("!")
+          descriptor = content_type(value) if key == "value" && %w[Extension SoftwareReference].include?(type)
+          [MAPKEYS.fetch(key, key), encode(item, descriptor)]
+        end
+      end
+      return value.map { |item| encode(item, "any") } if value.is_a?(Array)
+      return CBOR::Tagged.new(22, value.unpack1("m0").b) if type == "binary"
+      return CBOR::Tagged.new(23, [value].pack("H*").b) if type == "hex"
+      return CBOR::Tagged.new(0, value) if type == "datetime"
+      return CBOR::Tagged.new(32, value) if type == "uri"
+
+      if %w[number positive].include?(type)
+        rounded = [value].pack("g").unpack1("g")
+        raise ConversionError, "IODEF CBOR real values must be exactly representable as float32" unless rounded.finite? && rounded == value
+
+        return Float32.new(rounded)
+      end
+      value
+    end
+
+    def decode(value, type = "Document", path = "")
+      if type.end_with?("[]") && value.is_a?(Array)
+        return value.each_with_index.map { |item, index| decode(item, type.delete_suffix("[]"), Iodef.pointer(path, index)) }
+      end
+      return decode_object(value, type, path) if value.is_a?(Hash)
+      return value.each_with_index.map { |item, index| decode(item, "any", Iodef.pointer(path, index)) } if value.is_a?(Array)
+
+      if value.is_a?(CBOR::Tagged) && [253, 254].include?(value.tag)
+        raise ConversionError, "bignum tag requires a byte string at #{path}" unless value.value.is_a?(String) && value.value.encoding == Encoding::BINARY
+
+        magnitude = value.value.empty? ? 0 : value.value.unpack1("H*").to_i(16)
+        value = value.tag == 254 ? -1 - magnitude : magnitude
+      end
+      case type
+      when "binary" then decode_tag(value, 22, path) { |bytes| [bytes].pack("m0") }
+      when "hex" then decode_tag(value, 23, path) { |bytes| bytes.unpack1("H*").upcase }
+      when "datetime" then decode_tag(value, 0, path) { |text| text }
+      when "uri" then decode_tag(value, 32, path) { |text| text }
+      else
+        if value.is_a?(CBOR::Tagged) || value.is_a?(CBOR::Simple) || (value.is_a?(String) && value.encoding == Encoding::BINARY)
+          raise ConversionError, "CBOR value has no unambiguous JSON representation at #{path}"
+        end
+
+        value
+      end
+    end
+
+    def decode_object(value, type, path)
+      fields = SCHEMA.fetch(type == "mlstring" ? "MLString" : type, {}).fetch("fields", {})
+      named = value.each_with_object({}) do |(key, item), result|
+        key = KEYNAMES.fetch(key) { raise ConversionError, "unassigned CBOR map key at #{path}" } if key.is_a?(Integer)
+        raise ConversionError, "CBOR map key must be a text string or assigned integer at #{path}" unless key.is_a?(String) && key.encoding != Encoding::BINARY
+        raise ParseError, "colliding CBOR map keys at #{path}" if result.key?(key)
+
+        result[key] = item
+      end
+      named.to_h do |key, item|
+        descriptor = fields.fetch(key, "any").delete_suffix("!")
+        descriptor = content_type(named) if key == "value" && %w[Extension SoftwareReference].include?(type)
+        [key, decode(item, descriptor, Iodef.pointer(path, key))]
+      end
+    end
+
+    def content_type(value)
+      case value["dtype"]
+      when "byte", "file" then "binary"
+      when "bytes", "frame", "packet", "ipv4-packet", "ipv6-packet" then "hex"
+      else "string"
+      end
+    end
+
+    def decode_tag(value, tag, path)
+      binary = [22, 23].include?(tag)
+      return yield(value) if value.is_a?(String) && (value.encoding == Encoding::BINARY) == binary
+
+      valid = value.is_a?(CBOR::Tagged) && value.tag == (tag == 32 ? 255 : tag) && value.value.is_a?(String)
+      valid &&= (value.value.encoding == Encoding::BINARY) == [22, 23].include?(tag)
+      raise ConversionError, "expected CBOR tag #{tag} at #{path}" unless valid
+
+      yield value.value
+    end
+  end
+
+  class << self
+    def parse_cbor(bytes, strict: true, validate: true, **limits)
+      options = self.limits(limits)
+      raise ParseError, "CBOR input must be a String" unless bytes.is_a?(String)
+      raise LimitError, "input exceeds max_input_size" if bytes.bytesize > options[:max_input_size]
+
+      guarded = CBORGuard.new(bytes, options).validate!
+      data = CBORCodec.decode(CBOR.decode(guarded))
+      Document.new(data, strict: strict && validate, **options)
+    rescue CBOR::UnpackError, CBOR::TypeError, EOFError, EncodingError
+      raise ParseError, "invalid CBOR input"
+    end
+  end
+
+  class Document
+    def to_cbor
+      validate!
+      bytes = CBOR.encode(CBORCodec.encode(to_h))
+      raise LimitError, "output exceeds max_input_size" if bytes.bytesize > limits[:max_input_size]
+
+      bytes
+    end
+  end
+end