pq_crypto 0.3.1 → 0.4.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 534c420f62323e9ddb49b48acdfa7af869731e3273bda56637d6bfee753de8d7
-  data.tar.gz: f003494b4d33702a1b718dc6ded7a9f0612b315e41f0d739a5b2fe3ebba9d1d1
+  metadata.gz: 65d8d9a5d3aeecaba257218dcae5bedb56c82283a4d4d34cdea4a5946c5d2b80
+  data.tar.gz: 210cd374801dd9f8ce7beafa4f5eb528c807bcedd3762977445849e2def7e215
 SHA512:
-  metadata.gz: 1a039325a13cf8c074741fb70edc1e66f4e4939727b4c1369665794bc6ed71ef1ed2ef50d9debd9064d850e4856fac166f325889a9df4bfd3d8849a2d7a918e9
-  data.tar.gz: 304888656181149eed84eca063e24eeb6c862819f6c54022e77b287ca4030b8602e9f61eb17809ad86ca0eea84796c6275a2afa12018ffdeaafc489c15c8f24a
+  metadata.gz: a2085b6a3b6b48389219b81fa8b7a0e656c6583587df9a52817fcf28e663210be744e9e6a121dbaccdb9d06e8ce12fc2ee7b84f581fb9ce4e4bd74273df6a4b5
+  data.tar.gz: b6e7e3737f0d052d045b6fd4c424f188f6eab01333bc86b8685c707b6a90c14f4af0ecc3cacad6e8ca36750d71694d11481e5d4e617fa41bdbfe68a19a5ea0c3

data/.github/workflows/ci.yml

@@ -35,3 +35,59 @@ jobs:
 
       - name: Run tests
         run: bundle exec rake test
+
+  interop-openssl-3-5-required:
+    name: interop-openssl-3.5-required
+    runs-on: ubuntu-24.04
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Build OpenSSL 3.5
+        run: |
+          set -euxo pipefail
+          sudo apt-get update
+          sudo apt-get install -y build-essential perl pkg-config curl ca-certificates
+          curl -fsSLO https://www.openssl.org/source/openssl-3.5.0.tar.gz
+          tar -xzf openssl-3.5.0.tar.gz
+          cd openssl-3.5.0
+          ./Configure linux-x86_64 --prefix=/opt/openssl-3.5 --openssldir=/opt/openssl-3.5 shared
+          make -s -j"$(nproc)"
+          sudo make -s install_sw
+          echo /opt/openssl-3.5/lib64 | sudo tee /etc/ld.so.conf.d/openssl-3.5.conf
+          sudo ldconfig
+          /opt/openssl-3.5/bin/openssl version
+          /opt/openssl-3.5/bin/openssl version | grep -E '^OpenSSL 3\.(5|[6-9]|[1-9][0-9])\.'
+          echo /opt/openssl-3.5/bin >> "$GITHUB_PATH"
+          {
+            echo "OPENSSL_ROOT_DIR=/opt/openssl-3.5"
+            echo "PKG_CONFIG_PATH=/opt/openssl-3.5/lib64/pkgconfig"
+            echo "LD_LIBRARY_PATH=/opt/openssl-3.5/lib64"
+          } >> "$GITHUB_ENV"
+
+      - name: Set up Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: "3.4"
+          bundler-cache: true
+
+      - name: Set up Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: "1.26.0"
+
+      - name: Compile extension
+        run: bundle exec rake compile
+
+      - name: Run tests
+        run: |
+          set -o pipefail
+          bundle exec rake test 2>&1 | tee rake_test.log
+
+      - name: Refuse skipped OpenSSL 3.5 interop tests
+        run: |
+          if grep -E "Skipped: .* (mlkem|mldsa)_spki" rake_test.log; then
+            echo "OpenSSL 3.5 interop tests must NOT skip on this matrix entry."
+            exit 1
+          fi

data/CHANGELOG.md

@@ -1,5 +1,55 @@
 # Changelog
 
+## [0.4.2] - 2026-04-29
+
+### Fixed
+- Fixed native extension build from packaged gem by keeping generated
+  `pqcrypto_version.h` available after `make clean`.
+
+## [0.4.1] — 2026-04-29
+
+### Fixed
+
+- Hardened native ML-DSA signing error handling by checking RNG failures.
+- Improved streaming ML-DSA validation and context handling without regressing throughput.
+
+## [0.4.0] — 2026-04-28
+
+### Added — standard serialization and expanded parameter sets
+
+- Added a central algorithm registry with legacy `pqc_container_*` OIDs kept separate from RFC 9935 / RFC 9881 standard OIDs.
+- Added RFC 9935 SPKI public-key and PKCS#8 private-key support for ML-KEM.
+- Added RFC 9881 SPKI public-key and expanded-key PKCS#8 support for ML-DSA.
+- Added public API support for ML-KEM-512, ML-KEM-1024, ML-DSA-44, and ML-DSA-87 alongside the existing ML-KEM-768 and ML-DSA-65 support.
+- Added OpenSSL 3.5+ interoperability tests for standard SPKI / PKCS#8 encodings where supported by the linked OpenSSL.
+- Added NIST ACVP KAT test infrastructure for ML-KEM and ML-DSA parameter sets.
+- Added opt-in ML-DSA seed/both PKCS#8 import support. The default remains off; callers must set `PQCrypto::PKCS8.allow_ml_dsa_seed_format = true`.
+
+### Changed
+
+- Bumped the gem version to `0.4.0`.
+- `pqc_container_*` remains frozen and project-local. It is still limited to the original three algorithms: `:ml_kem_768`, `:ml_dsa_65`, and `:ml_kem_768_x25519_xwing`.
+- `DETAILS[:oid]` continues to expose the legacy `pqc_container_*` OID for backward compatibility. Standard OIDs are available via `PQCrypto::AlgorithmRegistry.standard_oid`.
+- The native `PQCrypto.version` path now derives from `lib/pq_crypto/version.rb` through a generated C header, avoiding a second manually maintained version string.
+
+### Security notes
+
+- ML-DSA seed-format imports are opt-in because PQClean does not expose a public ML-DSA derandomized keypair entrypoint. The implementation reuses the thread-local seed-replay path documented in `SECURITY.md`.
+- The project remains unaudited and experimental.
+
+## [0.3.2] — 2026-04-25
+
+### Added — streaming ML-DSA for large inputs
+
+- Added `PQCrypto::Signature::SecretKey#sign_io(io, chunk_size: 1 << 20, context: "".b)`.
+- Added `PQCrypto::Signature::PublicKey#verify_io(io, signature, chunk_size: 1 << 20, context: "".b)` and `verify_io!`.
+- Implemented streaming pure ML-DSA through an internal FIPS 204 ExternalMu path. Existing one-shot `sign` / `verify` semantics are unchanged; no public `sign_mu` / `verify_mu` API is exposed.
+
+### Notes
+
+- Streaming is primarily for large IO inputs and lower peak memory pressure. It is not a HashML-DSA/prehash speed mode; CPU cost is still dominated by SHAKE/Keccak.
+- Default empty-context streaming signatures interoperate with the existing one-shot `verify(message, signature)` API. Non-empty `context:` must be supplied again during `verify_io`.
+
 ## [0.3.1] — 2026-04-24
 
 ### Fixed — X-Wing draft-10 compatibility

data/GET_STARTED.md

@@ -1,75 +1,419 @@
 # Getting started with pq_crypto
 
-## 1. Build the extension
+This guide shows the common `pq_crypto` workflows. The README is intentionally
+short; examples and practical details live here.
+
+## 1. Install
+
+Add the gem to your application:
+
+```ruby
+# Gemfile
+gem "pq_crypto"
+```
+
+Install it:
 
 ```bash
 bundle install
+```
+
+If you are working from a repository checkout, compile the native extension:
+
+```bash
 bundle exec rake compile
 ```
 
-## 2. Generate an ML-KEM-768 keypair
+Use the gem from Ruby:
+
+```ruby
+require "pq_crypto"
+```
+
+## 2. Check the available algorithms
+
+```ruby
+PQCrypto.supported_kems
+# => [:ml_kem_512, :ml_kem_768, :ml_kem_1024]
+
+PQCrypto.supported_signatures
+# => [:ml_dsa_44, :ml_dsa_65, :ml_dsa_87]
+
+PQCrypto.supported_hybrid_kems
+# => [:ml_kem_768_x25519_xwing]
+```
+
+Useful metadata:
+
+```ruby
+PQCrypto.version
+PQCrypto.backend
+
+PQCrypto::KEM.details(:ml_kem_768)
+PQCrypto::Signature.details(:ml_dsa_65)
+PQCrypto::AlgorithmRegistry.standard_oid(:ml_kem_768)
+```
+
+`DETAILS[:oid]` remains the legacy `pqc_container_*` OID for backward
+compatibility. Use `AlgorithmRegistry.standard_oid` when you need the standard
+RFC OID.
+
+## 3. ML-KEM: generate, encapsulate, decapsulate
+
+Generate a keypair:
 
 ```ruby
 keypair = PQCrypto::KEM.generate(:ml_kem_768)
+public_key = keypair.public_key
+secret_key = keypair.secret_key
+```
+
+Encapsulate with the public key:
+
+```ruby
+result = public_key.encapsulate
+ciphertext = result.ciphertext
+sender_shared_secret = result.shared_secret
+```
+
+Decapsulate with the secret key:
+
+```ruby
+receiver_shared_secret = secret_key.decapsulate(ciphertext)
+
+sender_shared_secret == receiver_shared_secret
+# => true
+```
+
+The same API shape applies to other supported ML-KEM parameter sets:
+
+```ruby
+PQCrypto::KEM.generate(:ml_kem_512)
+PQCrypto::KEM.generate(:ml_kem_1024)
+```
+
+## 4. ML-DSA: sign and verify
+
+Generate a signature keypair:
+
+```ruby
+keypair = PQCrypto::Signature.generate(:ml_dsa_65)
+public_key = keypair.public_key
+secret_key = keypair.secret_key
+```
+
+Sign and verify a message:
+
+```ruby
+message = "hello".b
+signature = secret_key.sign(message)
+
+public_key.verify(message, signature)
+# => true
+
+public_key.verify!(message, signature)
+# returns true, or raises on mismatch
+```
+
+The same API shape applies to other supported ML-DSA parameter sets:
+
+```ruby
+PQCrypto::Signature.generate(:ml_dsa_44)
+PQCrypto::Signature.generate(:ml_dsa_87)
+```
+
+## 5. ML-DSA for large files
+
+For large inputs, use the streaming helpers so the whole message does not need
+to be materialized as one Ruby string.
+
+```ruby
+keypair = PQCrypto::Signature.generate(:ml_dsa_65)
+
+signature = File.open("document.bin", "rb") do |io|
+  keypair.secret_key.sign_io(io, chunk_size: 1 << 20)
+end
+
+ok = File.open("document.bin", "rb") do |io|
+  keypair.public_key.verify_io(io, signature, chunk_size: 1 << 20)
+end
 ```
 
-## 3. Encapsulate and decapsulate
+With an optional FIPS 204 context:
 
 ```ruby
+context = "invoice-v1".b
+
+signature = File.open("document.bin", "rb") do |io|
+  keypair.secret_key.sign_io(io, context: context)
+end
+
+ok = File.open("document.bin", "rb") do |io|
+  keypair.public_key.verify_io(io, signature, context: context)
+end
+```
+
+Notes:
+
+- `context` must match during verification.
+- `context` is limited to 255 bytes by FIPS 204.
+- `chunk_size` must be positive.
+- `sign_io` / `verify_io` are pure ML-DSA streaming helpers. They are not
+  HashML-DSA/prehash shortcuts and do not expose public `sign_mu` /
+  `verify_mu` APIs.
+
+## 6. Hybrid KEM: ML-KEM-768 + X25519 X-Wing
+
+```ruby
+keypair = PQCrypto::HybridKEM.generate(:ml_kem_768_x25519_xwing)
+
 result = keypair.public_key.encapsulate
-shared_secret = keypair.secret_key.decapsulate(result.ciphertext)
+ciphertext = result.ciphertext
+sender_shared_secret = result.shared_secret
+
+receiver_shared_secret = keypair.secret_key.decapsulate(ciphertext)
+
+sender_shared_secret == receiver_shared_secret
+# => true
+```
+
+The raw X-Wing secret key exported by this API is the draft-style 32-byte
+secret seed, not the expanded ML-KEM/X25519 private material.
+
+See `SECURITY.md` for audit status and hybrid interoperability caveats.
+
+## 7. SPKI public-key serialization
+
+Use SPKI when you need standard public-key serialization for ML-KEM or ML-DSA.
+
+ML-KEM example:
+
+```ruby
+keypair = PQCrypto::KEM.generate(:ml_kem_768)
+
+pem = keypair.public_key.to_spki_pem
+imported = PQCrypto::KEM.public_key_from_spki_pem(pem)
+
+imported == keypair.public_key
+# => true
 ```
 
-## 4. Generate an ML-DSA-65 keypair
+DER form:
 
 ```ruby
-sig = PQCrypto::Signature.generate(:ml_dsa_65)
+der = keypair.public_key.to_spki_der
+imported = PQCrypto::KEM.public_key_from_spki_der(der)
 ```
 
-## 5. Sign and verify
+ML-DSA example:
 
 ```ruby
-signature = sig.secret_key.sign("message")
+keypair = PQCrypto::Signature.generate(:ml_dsa_65)
 
-sig.public_key.verify("message", signature)
-sig.public_key.verify!("message", signature)
+pem = keypair.public_key.to_spki_pem
+imported = PQCrypto::Signature.public_key_from_spki_pem(pem)
 ```
 
-## 6. Hybrid KEM (X-Wing)
+You can require an expected algorithm during import:
 
 ```ruby
-hybrid = PQCrypto::HybridKEM.generate(:ml_kem_768_x25519_xwing)
-result = hybrid.public_key.encapsulate
-shared_secret = hybrid.secret_key.decapsulate(result.ciphertext)
+PQCrypto::KEM.public_key_from_spki_pem(pem, algorithm: :ml_kem_768)
 ```
 
-The raw X-Wing secret key exported by this API is the draft-10 32-byte
-decapsulation seed, not the expanded ML-KEM/X25519 private material.
+## 8. PKCS#8 private-key serialization
+
+Use PKCS#8 when you need standard private-key serialization for ML-KEM or
+ML-DSA.
+
+### ML-KEM PKCS#8
+
+`SecretKey#to_pkcs8_pem` exports the expanded private key by default:
+
+```ruby
+keypair = PQCrypto::KEM.generate(:ml_kem_768)
+
+pem = keypair.secret_key.to_pkcs8_pem
+imported = PQCrypto::KEM.secret_key_from_pkcs8_pem(pem)
+
+imported == keypair.secret_key
+# => true
+```
 
-The hybrid mode follows `draft-connolly-cfrg-xwing-kem`. See
-`SECURITY.md` for audit status.
+DER form:
+
+```ruby
+der = keypair.secret_key.to_pkcs8_der
+imported = PQCrypto::KEM.secret_key_from_pkcs8_der(der)
+```
+
+ML-KEM PKCS#8 supports `:seed`, `:expanded`, and `:both` formats. A generated
+`SecretKey` does not retain the original seed, so exporting `:seed` or `:both`
+from `SecretKey#to_pkcs8_*` is intentionally unavailable. If you explicitly
+have the seed material, use the low-level PKCS#8 encoder.
+
+### ML-DSA PKCS#8
+
+ML-DSA private keys support expanded-key PKCS#8 by default:
+
+```ruby
+keypair = PQCrypto::Signature.generate(:ml_dsa_65)
+
+pem = keypair.secret_key.to_pkcs8_pem
+imported = PQCrypto::Signature.secret_key_from_pkcs8_pem(pem)
+```
 
-## 7. Serialize a key
+ML-DSA seed/both import is intentionally opt-in:
 
 ```ruby
+PQCrypto::PKCS8.allow_ml_dsa_seed_format = true
+imported = PQCrypto::Signature.secret_key_from_pkcs8_pem(pem)
+```
+
+Seed/both export from an existing ML-DSA `SecretKey` is intentionally not
+available because the object does not retain the original seed material. When
+you explicitly have seed material, call `PQCrypto::PKCS8.encode_der` /
+`encode_pem` directly.
+
+## 9. pq_crypto-local container serialization
+
+The `pqc_container_*` APIs are retained for backward compatibility with older
+`pq_crypto` releases.
+
+```ruby
+keypair = PQCrypto::KEM.generate(:ml_kem_768)
+
 der = keypair.public_key.to_pqc_container_der
 imported = PQCrypto::KEM.public_key_from_pqc_container_der(der)
 ```
 
-`pqc_container_*` formats are pq_crypto-specific.
+PEM form:
+
+```ruby
+pem = keypair.secret_key.to_pqc_container_pem
+imported = PQCrypto::KEM.secret_key_from_pqc_container_pem(pem)
+```
+
+Important caveats:
+
+- `pqc_container_*` is project-local.
+- It is not ASN.1 SPKI or PKCS#8.
+- It is not advertised as interoperable with external PKI tooling.
+- It remains limited to the original algorithms:
+  - `:ml_kem_768`
+  - `:ml_dsa_65`
+  - `:ml_kem_768_x25519_xwing`
+
+For external interoperability, prefer SPKI for public keys and PKCS#8 for
+private keys.
+
+## 10. Raw key import/export
+
+Raw bytes can be useful when integrating with storage or protocols that already
+handle framing.
+
+```ruby
+keypair = PQCrypto::KEM.generate(:ml_kem_768)
+
+public_bytes = keypair.public_key.to_bytes
+secret_bytes = keypair.secret_key.to_bytes
+
+public_key = PQCrypto::KEM.public_key_from_bytes(:ml_kem_768, public_bytes)
+secret_key = PQCrypto::KEM.secret_key_from_bytes(:ml_kem_768, secret_bytes)
+```
+
+Signature keys have the same shape:
 
-## 8. Inspect supported algorithms
+```ruby
+keypair = PQCrypto::Signature.generate(:ml_dsa_65)
+
+public_key = PQCrypto::Signature.public_key_from_bytes(:ml_dsa_65, keypair.public_key.to_bytes)
+secret_key = PQCrypto::Signature.secret_key_from_bytes(:ml_dsa_65, keypair.secret_key.to_bytes)
+```
+
+## 11. Secure wiping
+
+`PQCrypto.secure_wipe(str)` zeroes a mutable Ruby string in place.
 
 ```ruby
-PQCrypto.supported_kems           # => [:ml_kem_768]
-PQCrypto.supported_hybrid_kems    # => [:ml_kem_768_x25519_xwing]
-PQCrypto.supported_signatures     # => [:ml_dsa_65]
+raw = File.binread("secret-key.bin")
+key = PQCrypto::KEM.secret_key_from_bytes(:ml_kem_768, raw)
+
+PQCrypto.secure_wipe(raw)
+key.wipe!
+```
+
+`wipe!` is best-effort only. It clears the current Ruby string buffer owned by
+the key object. It cannot erase every copy that may have been created by Ruby,
+OpenSSL, serialization, logging, or the garbage collector.
+
+## 12. Equality and inspection
+
+Key equality uses constant-time comparison through OpenSSL `CRYPTO_memcmp` via
+`PQCrypto.ct_equals`.
+
+```ruby
+keypair.public_key == imported_public_key
+keypair.secret_key == imported_secret_key
+```
+
+Secret key `inspect` output is intentionally redacted, and secret key objects
+do not expose a public fingerprint method.
+
+## 13. Build-time Keccak backend
+
+The default build uses PQClean's scalar `common/fips202.c` backend:
+
+```bash
+PQCRYPTO_KECCAK_BACKEND=clean bundle exec rake compile
 ```
 
-## 9. Practical notes
+`PQCRYPTO_KECCAK_BACKEND=xkcp` is reserved for a separately vendored, reviewed,
+`fips202.h`-compatible XKCP adapter. If requested without that adapter, the
+build aborts instead of silently falling back to `clean`.
 
-- OpenSSL 3.0+ with SHA3-256 is required.
-- `PQCrypto::Testing` exposes deterministic helpers only for
-  regression tests.
-- Key equality uses constant-time comparison. `#hash` returns a
-  hash derived from a SHA-256 fingerprint, not the raw bytes.
+## 14. Async / Fiber scheduler behavior
+
+On Ruby 3.4, signing and verification use Ruby's scheduler-aware
+`rb_nogvl(..., RB_NOGVL_OFFLOAD_SAFE)` path automatically. With a scheduler
+that implements `blocking_operation_wait`, blocking native work can be moved
+off the event loop.
+
+## 15. Test-only deterministic helpers
+
+`PQCrypto::Testing` exposes deterministic helpers for regression tests:
+
+```ruby
+PQCrypto::Testing.ml_kem_keypair_from_seed(seed)       # 64-byte d||z seed
+PQCrypto::Testing.ml_kem_encapsulate_from_seed(pk, seed) # 32-byte seed
+PQCrypto::Testing.ml_dsa_keypair_from_seed(seed)       # 32-byte seed
+PQCrypto::Testing.ml_dsa_sign_from_seed(message, sk, seed)
+```
+
+These helpers are intended for tests only. They drive stock PQClean entrypoints
+and are not part of the normal application API.
+
+## 16. Development commands
+
+Run the test suite:
+
+```bash
+bundle exec rake test
+```
+
+Refresh the pinned PQClean vendor snapshot only when intentionally updating
+vendored sources:
+
+```bash
+bundle exec ruby script/vendor_libs.rb
+```
+
+To intentionally change the upstream snapshot, override all pinning inputs
+together:
+
+```bash
+PQCLEAN_VERSION=<full-git-commit> \
+PQCLEAN_URL=https://github.com/PQClean/PQClean/archive/<full-git-commit>.tar.gz \
+PQCLEAN_SHA256=<archive-sha256> \
+PQCLEAN_STRIP=PQClean-<full-git-commit> \
+  bundle exec ruby script/vendor_libs.rb
+```