baseh 1.1.0 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d62aeddba45748e9bf4c1d40935e09d8f18bbe032f7d97320e4f78b7598d33ed
-  data.tar.gz: 29a24ad4f909e9832ae668c1d0d1765d94884c4533b7fa1a98bfbc72d86e2347
+  metadata.gz: f624ebca106c2a3733ad2d655ff7f56a0b32b5ce55882e8932d107bc7561997d
+  data.tar.gz: 6bd8749f57881c40ef0b83b70d35b159fd69a2bb2fb493c0a009bb0b829be7b1
 SHA512:
-  metadata.gz: e8b586ae0ac27e61037a413032bfcff338140ffb0a9fd0f99bebe3377657244d326e8a03e53c185c58713408d0968df71a8bfaf0ed908a80faea6c29b805de98
-  data.tar.gz: a0a5578f2f70d553415396b47e28183edab7eab72e780fdd76834863a79ef8e0b3a99b7a7fb1fa55b8310ad38c7fcde2c82cc6a62cf87eb3b57d9f401a0ce97e
+  metadata.gz: b5b2361b1d7d6fc42927f9b9a9ec63d1905a514fb98d8bee73bd365b51a603b8f9a33b668c9219b39a5dc07236a2fd1a0040a3cecbc9061e2e71f001f57130c5
+  data.tar.gz: 760c74a6d57f511b4b1d24f0b87ae4e29e364dde6f15d44a7075fc00748943efc022950203051dc00c1113fa2cdcd34a0857974a5a349a28be6006af673089fe

data/README.md

@@ -1,44 +1,105 @@
 # baseh
 
 Ruby port of the baseH (Human Reference Code) codec. Encodes integer IDs as
-fixed-length, checksummed, human-friendly reference codes with an opt-in
-reversible feistel-v1 permutation and profanity safety. The normative spec
-is `spec/IMPLEMENTATION_CODEC.md` in the monorepo root.
+short, checksummed, human-friendly reference codes with a feistel-v1
+permutation on every tier and profanity safety. The normative spec is
+`spec/IMPLEMENTATION_CODEC.md` in the monorepo root.
 
 ## Install
 
+```sh
+gem install baseh
+```
+
+or in a Gemfile:
+
 ```ruby
 # Gemfile
 gem "baseh", path: "ruby"
 ```
 
-or
+or from a local build:
 
 ```sh
 gem build baseh.gemspec
-gem install ./baseh-1.0.0.gem
+gem install ./baseh-2.0.0.gem
 ```
 
 Zero runtime dependencies. Only `openssl` and `json` from the standard
 library are used.
 
-## Frozen tiers
+## Expandable mode (recommended default)
+
+Profiles carry a `mode:` field: `"expandable"` or `"fixed"`. Expandable is
+the recommended default for new users. The frozen tier
+`Baseh.baseh_expandable_v1` ships as the starting point, with a keyed
+private-mapping variant `Baseh.baseh_expandable_p_v1` that relates to it
+exactly as the other `-p` tiers relate to their plain tiers.
+
+```ruby
+require "baseh"
+
+codec = Baseh::Baseh.new(Baseh.baseh_expandable_v1)
+
+code = codec.encode(id: 123_456)   # 4 characters at this namespace size
+codec.decode(code).id              # => 123456
+```
 
-Four frozen tiers ship with the gem, built from the full alphanumeric set
-with cumulative visual and spoken strips. All four encode 6 body symbols,
-are case-insensitive and run the default profanity blocklist.
+How expandable differs from fixed:
+
+- **Codes start short and grow.** Minimum length is 4 characters
+  (`min_length`, default 4). When the id sequence climbs past a length's
+  capacity, codes simply become one character longer — transparently, no
+  migration, no re-issue. Old shorter codes keep decoding forever.
+- **Medium safety plus the zero ban.** The default expandable body alphabet
+  is the 27 symbols left after the medium visual and spoken safety strips
+  and the `0`/`O` zero ban, so an issued code never emits a visual or spoken
+  confusable. A custom alphabet containing `0`/`O` has those symbols silently
+  removed during profile preparation (the derived alphabet is always
+  displayed in tooling). This applies on top of whatever visual/spoken
+  safety levels, profanity modes, or blocklists are configured; every
+  existing profile option composes with expandable unchanged.
+- **Checksum alphabet gains `0`.** The checksum alphabet is the body
+  alphabet plus `0` (28 symbols for the default). The existing input alias
+  `O -> 0` remains, so a typed or misread `O` in a checksum position
+  resolves to `0`. There is no left-padding in expandable mode; a `0` or
+  `O` in a body position of presented input is simply an invalid character.
+- **Permutation stays on.** The Feistel permutation is applied per
+  generation (per code length), with the length mixed into the key
+  derivation alongside the profile id. Codes within each length look random
+  even though issuance is a sequential counter. Presentation only, not
+  encryption.
+- **Separators appear later.** Grouping only kicks in once codes reach
+  `separator_min_length` (the shipped tier uses 6, so no hyphen until codes
+  are 6+ characters). Below the threshold there is no separator.
+- **Decode is length-driven.** The code's length selects the generation;
+  the checksum validates exactly as in fixed mode, domain-separated by
+  profile id.
+
+The security posture is unchanged: a code is a reference alias, never an
+authorization token. Expandable codes in the smallest generations are a
+small namespace, so rate-limit public lookups and enforce authorization
+after decode.
+
+## Fixed mode (frozen tiers)
+
+Four frozen tiers ship with the gem, all `mode: "fixed"`, built from the
+full alphanumeric set with cumulative visual and spoken strips. All four encode 6 body symbols,
+are case-insensitive, hyphen-delimit at the midpoint, run the default
+profanity blocklist and permute with the published frozen key.
 
 | Tier | Helper | Body symbols | Checksum | Format | Capacity |
 | ---- | ------ | ------------ | -------- | ------ | -------- |
 | Minimum | `Baseh.baseh_minimum_v1` | 36 | none | `XXX-XXX` | 2,176,782,336 |
-| Light | `Baseh.baseh_light_v1` | 31 | 1 | plain | 887,503,681 |
-| Medium | `Baseh.baseh_medium_v1` | 28 | 1 | plain | 481,890,304 |
-| Heavy | `Baseh.baseh_heavy_v1` | 26 | 1 | plain | 308,915,776 |
+| Light | `Baseh.baseh_light_v1` | 31 | 2 | `XXXX-XXXX` | 887,503,681 |
+| Medium | `Baseh.baseh_medium_v1` | 28 | 2 | `XXXX-XXXX` | 481,890,304 |
+| Heavy | `Baseh.baseh_heavy_v1` | 26 | 2 | `XXXX-XXXX` | 308,915,776 |
 
-Medium is the default. Minimum keeps the full alphabet and uses a hyphen
-delimiter; the rest have no separator. Each tier keeps the typed O/I/L
-aliases where possible and adds spoken-confusion aliases for the stripped
-symbols.
+Medium is the default of the fixed tiers. The frozen key is public by
+design: it hides sequence, not records. It is not a secret and anyone can read it at
+`Baseh::FROZEN_KEY_BYTES`; it must never change for a live namespace. Each
+tier keeps the typed O/I/L aliases where possible and adds spoken-confusion
+aliases for the stripped symbols.
 
 Every helper returns a freshly built mutable profile hash on each call, so
 callers can load a default and modify it before constructing a codec.
@@ -48,30 +109,43 @@ callers can load a default and modify it before constructing a codec.
 ```ruby
 require "baseh"
 
-codec = Baseh::Baseh.new(Baseh.baseh_medium_v1)
+codec = Baseh::Baseh.new(Baseh.baseh_expandable_v1)
 
-code = codec.encode(id: 123_456)           # => raw fixed-width code
+code = codec.encode(id: 123_456)           # => short code, grows as ids climb
 
 result = codec.decode(code)
 result.id                                  # => 123456
 result.canonical_code                      # => canonical form
 result.corrected                           # => true when input needed correction
 
-codec.capacity                             # => 481890304
-
-check = codec.validate("0000000")
+check = codec.validate("00000000")
 check.valid                                # => false
 check.reason                               # => "INVALID_CHECKSUM"
 
+# Live as-you-type feedback for a code entry field (never raises)
+look = codec.inspect("C8XP8")
+look.state                                 # => "typing" (or "empty", "bad-char",
+                                           #     "too-long", "invalid", "valid")
+look.typed                                 # => "C8XP-8"
+look.progress                              # => 0.625
+
 # Spoken-confusion correction
-result = codec.decode("TB14QDF", try_correction: true, confusion_profile: :light)
+result = codec.decode("TB14QDFU", try_correction: true, confusion_profile: :light)
+```
+
+The same API applies to the fixed tiers (`codec.capacity` reports the fixed
+namespace size there; expandable grows instead):
+
+```ruby
+codec = Baseh::Baseh.new(Baseh.baseh_medium_v1)
+codec.capacity                             # => 481890304
 ```
 
-## Permutation (opt-in)
+## Permutation
 
-The `-p` variants opt a tier into the reversible feistel-v1 permutation.
-`key_bytes:` is required; keep the key in a secret manager and never change
-it for a live profile:
+The plain tiers permute with `Baseh::FROZEN_KEY_BYTES`, the published frozen
+key. The `-p` variants take caller-supplied key material instead; keep that
+key in a secret manager and never change it for a live profile:
 
 ```ruby
 profile = Baseh.baseh_medium_p_v1(
@@ -82,7 +156,7 @@ codec = Baseh::Baseh.new(profile)
 ```
 
 `rounds:` is also accepted (default 8). The `-p` profile is identical to
-its plain tier apart from the permutation; its profile id gains a `-p`
+its plain tier apart from the key material; its profile id gains a `-p`
 segment, for example `baseh-medium-p-v1`.
 
 ## Profanity safety (spec 18)
@@ -121,3 +195,19 @@ The vector tests load `../vectors/vectors.json` and
 `../vectors/feistel-vectors.json` from the monorepo root and assert every
 entry. Running the suite from a different directory layout requires those
 files at that relative path.
+
+The full 100,000-body single-substitution checksum sweep of the test-suite
+spec (section 6) runs under `BASEH_SOAK=1` alongside the soak suite:
+
+```sh
+BASEH_SOAK=1 rake test          # full soak + 100k checksum sweep per tier
+BASEH_SOAK=1 BASEH_SOAK_BODIES=1_000 rake test   # smoke the sweep
+```
+
+## Linting
+
+No RuboCop config ships in this gem: RuboCop is not part of the bundle, and
+adding a lint dependency was judged not worth it for a zero-dependency,
+~500-line library whose conformance is enforced by the shared cross-language
+vectors. The codebase follows the standard Ruby style (two-space indent,
+frozen string literals) by convention instead.

data/lib/baseh/baseh.rb

@@ -31,6 +31,13 @@ module Baseh
     # Result of validate, which never raises on user input.
     ValidateResult = Struct.new(:valid, :canonical_code, :reason, keyword_init: true)
 
+    # Result of inspect (spec 12.5). #state is one of "empty", "typing",
+    # "bad-char", "too-long", "invalid", "valid". Payload fields are present
+    # exactly when the state carries them: #typed and #progress for "typing",
+    # #reason for "invalid", #id and #canonical_code for "valid".
+    InspectResult = Struct.new(:state, :typed, :progress, :reason, :id, :canonical_code,
+                               keyword_init: true)
+
     attr_reader :profile
 
     # @param profile [Hash] profile definition per spec 2.1 (symbol keys)
@@ -41,19 +48,85 @@ module Baseh
     end
 
     # Spec section 4. Capacity is an arbitrary-precision Integer.
+    # Spec 12.3: fixed mode only; expandable profiles have no single
+    # capacity (use the per-generation formulas of spec 19.1).
     def capacity
+      if @profile.mode != "fixed"
+        raise BasehError.new(
+          "INVALID_PROFILE",
+          "capacity is only defined for fixed-mode profiles",
+          safe_for_customer: false
+        )
+      end
       @profile.capacity
     end
 
-    # Spec section 8, with the spec 18.2 blocklist scan over the raw code.
+    # Spec 19.1/22.3. First id of generation length: the sum of each
+    # generation's capacity A^(k - effectiveK(k)) for k from minLength
+    # through length-1. The effective checksum length is per-generation
+    # (spec 22), so the sum is not a single geometric series when the short
+    # checksum is on.
+    def generation_base(length)
+      base = 0
+      @profile.min_length.upto(length - 1) do |l|
+        base += generation_capacity(l)
+      end
+      base
+    end
+
+    # Spec 19.1/22.3. Ids held by generation length: A^(length - effectiveK).
+    def generation_capacity(length)
+      @profile.body_alphabet.length**(length - @profile.effective_checksum_length(length))
+    end
+
+    # Smallest generation whose range holds id, per spec 19.6. The loop is
+    # capped at the 32-symbol code limit (33 - minLength iterations) so an
+    # adversarial id fails fast with OUT_OF_RANGE instead of running big-integer
+    # multiplication on exponentially growing values.
+    def generation_for_id(id)
+      l = @profile.min_length
+      base = 0
+      cap = generation_capacity(l)
+      while id >= base + cap
+        if l >= 32
+          raise BasehError.new(
+            "OUT_OF_RANGE",
+            "ID requires a code longer than 32 symbols"
+          )
+        end
+        base += cap
+        l += 1
+        cap = generation_capacity(l)
+      end
+      l
+    end
+
+    # Spec 19.5. Balanced grouping: the split is a pure function of the total
+    # length — g = max(2, ceil(L / 5)) groups differing in size by at most
+    # one, larger groups to the left. There is no configurable pattern in
+    # expandable mode (grouping must be empty, section 2.2).
+    def self.expandable_grouping(length)
+      g = [2, (length + 4) / 5].max
+      base = length / g
+      return [length] if base < 1
+
+      rem = length % g
+      [base + 1] * rem + [base] * (g - rem)
+    end
+
+    # Spec section 8 (fixed mode) / 19.6 (expandable mode), with the spec
+    # 18.2 blocklist scan over the raw code.
     #
-    # @param id [Integer] 0 <= id < capacity
-    # @return [String] canonical code (grouped only when a separator is set)
+    # @param id [Integer] 0 <= id < capacity (fixed); any non-negative id
+    #   whose code fits in 32 symbols (expandable)
+    # @return [String] canonical code (grouped only when a separator applies)
     # @raise [BasehError] OUT_OF_RANGE, PERMUTATION_FAILURE, BLOCKED_CODE
     def encode(id:)
       unless id.is_a?(Integer)
         raise TypeError, "id must be an Integer"
       end
+      return encode_expandable(id) if @profile.mode == "expandable"
+
       if id.negative? || id >= @profile.capacity
         raise BasehError.new("OUT_OF_RANGE", "ID #{id} is outside the profile capacity")
       end
@@ -93,8 +166,22 @@ module Baseh
       end
 
       raw = normalize(input, accept_spaces)
-      body = raw.slice(0, @profile.body_length)
-      supplied_checksum = raw.slice(@profile.body_length..) || ""
+      # Spec 22: the generation is selected by the presented total length,
+      # so the effective checksum length is a deterministic function of it.
+      effective_k =
+        if @profile.mode == "expandable"
+          @profile.effective_checksum_length(raw.length)
+        else
+          @profile.checksum_length
+        end
+      body_length =
+        if @profile.mode == "expandable"
+          raw.length - effective_k
+        else
+          @profile.body_length
+        end
+      body = raw.slice(0, body_length)
+      supplied_checksum = raw.slice(body_length..) || ""
 
       # normalize validates every symbol against the union of the body and
       # checksum alphabets (spec 3.1 step 6). A checksum-only symbol in a
@@ -111,7 +198,7 @@ module Baseh
         )
       end
 
-      if Checksum.calculate_checksum(@profile, body, @body_index) != supplied_checksum
+      if Checksum.calculate_checksum(@profile, body, @body_index, effective_k) != supplied_checksum
         unless try_correction && max_corrections != 0
           raise BasehError.new(
             "INVALID_CHECKSUM",
@@ -132,7 +219,7 @@ module Baseh
         end
         valid = {}
         generate_candidates(body, filtered, max_corrections).each do |candidate|
-          if Checksum.calculate_checksum(@profile, candidate, @body_index) == supplied_checksum
+          if Checksum.calculate_checksum(@profile, candidate, @body_index, effective_k) == supplied_checksum
             valid[candidate] = true
           end
         end
@@ -155,7 +242,22 @@ module Baseh
 
       value = BaseN.decode_base_n(body, @profile.body_alphabet, @body_index)
       perm = @profile.permutation
-      if perm[:enabled]
+      if @profile.mode == "expandable"
+        # Spec 19.7: the offset is de-permuted within the generation's own
+        # domain (length mixed into the key derivation), then the generation
+        # base is added back.
+        l = raw.length
+        if perm[:enabled]
+          value = Feistel.inverse_permute(
+            value, generation_capacity(l),
+            profile_id: @profile.profile_id,
+            key_bytes: perm[:key_bytes],
+            rounds: perm[:rounds],
+            length: l
+          )
+        end
+        value = generation_base(l) + value
+      elsif perm[:enabled]
         value = Feistel.inverse_permute(
           value, @profile.capacity,
           profile_id: @profile.profile_id,
@@ -181,14 +283,80 @@ module Baseh
       ValidateResult.new(valid: false, reason: e.code)
     end
 
-    # Spec 3.1 normalization, steps 1-9, with the spec 3.4 re-pad. Returns
-    # the raw unformatted string.
+    # Spec 12.5. Live as-you-type inspection. Gates on the typed length before
+    # validating, so the spec 3.4 re-pad can never paint an incomplete
+    # fixed-mode code "valid" (or "invalid"): a short fixed input is "typing",
+    # never checked. Never raises on user input and never reports "valid" for
+    # an incomplete code.
+    def inspect(input)
+      p = @profile
+      # Step 1: remove every occurrence of the separator string, then drop
+      # ASCII whitespace anywhere (a paste can carry either inside the code).
+      s = p.separator.empty? ? input.dup : input.gsub(p.separator, "")
+      s = s.delete("\t\n\v\f\r ")
+      typed = s.length
+      # Step 2.
+      return InspectResult.new(state: "empty") if typed.zero?
+
+      # Step 3. Fixed: complete exactly at bodyLength + checksumLength.
+      # Expandable: complete at every length from minLength through 32 (the
+      # length selects the generation), 32 is the over-length bound.
+      fixed = p.mode == "fixed"
+      expected = fixed ? p.body_length + p.checksum_length : 32
+      return InspectResult.new(state: "too-long") if typed > expected
+
+      # Step 4: spec 3.1 steps 4-6 without the length checks — case
+      # normalization, aliases, then union membership. A symbol outside both
+      # alphabets is "bad-char"; a symbol valid only in the other region (say
+      # a checksum-only symbol typed into the body) passes here and fails
+      # later under validate as INVALID_CHARACTER.
+      s = s.upcase unless p.case_sensitive
+      raw = s.each_char.map do |ch|
+        inspect_union?(ch) ? ch : p.aliases.fetch(ch, ch)
+      end.join
+      return InspectResult.new(state: "bad-char") if raw.each_char.any? { |ch| !inspect_union?(ch) }
+
+      # Step 5.
+      complete = fixed ? typed == expected : typed >= p.min_length
+      unless complete
+        return InspectResult.new(
+          state: "typing",
+          typed: format_partial(raw),
+          progress: typed.to_f / (fixed ? expected : p.min_length)
+        )
+      end
+
+      # Step 6: judge the normalized string (no separator, no whitespace,
+      # case- and alias-normalized).
+      result = validate(raw)
+      return InspectResult.new(state: "invalid", reason: result.reason) unless result.valid
+
+      decoded = decode(raw)
+      InspectResult.new(state: "valid", id: decoded.id, canonical_code: decoded.canonical_code)
+    end
+
+    # Spec 3.1 normalization, steps 1-9, with the spec 3.4 re-pad in fixed
+    # mode only. Returns the raw unformatted string.
     def normalize(input, accept_spaces)
       s = input.gsub(ASCII_WS, "")
-      s = s.delete(@profile.separator) unless @profile.separator.empty?
+      had_separator = !@profile.separator.empty? && s.include?(@profile.separator)
+      # Literal-substring removal (gsub with a string pattern), matching the
+      # JS split-join reference: String#delete would treat the separator as a
+      # character class and corrupt multi-character separators like "..".
+      s = s.gsub(@profile.separator, "") unless @profile.separator.empty?
       s = s.delete(" ") if accept_spaces
       s = s.upcase unless @profile.case_sensitive
-      s = s.each_char.map { |ch| @profile.aliases.fetch(ch, ch) }.join
+      # Spec 3.2: an alias never maps two distinct canonical symbols into one
+      # value, so a symbol that is already canonical stays as-is and only
+      # non-canonical symbols are aliased. (In fixed tiers alias sources are
+      # never canonical, so this changes nothing there.)
+      s = s.each_char.map do |ch|
+        if @body_index.key?(ch) || @profile.checksum_alphabet.include?(ch)
+          ch
+        else
+          @profile.aliases.fetch(ch, ch)
+        end
+      end.join
 
       s.each_char do |ch|
         next if @body_index.key?(ch) || @profile.checksum_alphabet.include?(ch)
@@ -199,6 +367,32 @@ module Baseh
         )
       end
 
+      if @profile.mode == "expandable"
+        # Spec 19.2/19.7: no left-padding and no stripped-zero leniency. Input
+        # shorter than minLength or longer than 32 fails INVALID_LENGTH, and a
+        # separator below separatorMinLength is rejected (spec 19.5: the
+        # decoder expects no separators there).
+        if s.length < @profile.min_length
+          raise BasehError.new(
+            "INVALID_LENGTH",
+            "Expected at least #{@profile.min_length} symbols, got #{s.length}"
+          )
+        end
+        if s.length > 32
+          raise BasehError.new(
+            "INVALID_LENGTH",
+            "Expected at most 32 symbols, got #{s.length}"
+          )
+        end
+        if had_separator && s.length < @profile.separator_min_length
+          raise BasehError.new(
+            "INVALID_CHARACTER",
+            "Separators do not appear below #{@profile.separator_min_length} symbols"
+          )
+        end
+        return s
+      end
+
       expected = @profile.body_length + @profile.checksum_length
       # Spec 3.4: a code that lost leading zero body symbols is re-padded
       # with the body zero symbol. The checksum symbols always remain, so
@@ -242,20 +436,32 @@ module Baseh
     private
 
     # Spec 18.2: case-insensitive substring scan over the raw unformatted
-    # code. BLOCKED_CODE is an issuance decision, not an end-user condition.
+    # code, plus the spec 21.2 run scan. BLOCKED_CODE is an issuance
+    # decision, not an end-user condition.
     def check_blocklist!(raw)
-      return if @profile.blocklist.empty?
-
-      upper = raw.upcase
-      @profile.blocklist.each do |word|
-        next unless upper.include?(word)
+      unless @profile.blocklist.empty?
+        upper = raw.upcase
+        @profile.blocklist.each do |word|
+          next unless upper.include?(word)
 
-        raise BasehError.new(
-          "BLOCKED_CODE",
-          "The generated reference contains a blocked substring",
-          safe_for_customer: false
-        )
+          raise BasehError.new(
+            "BLOCKED_CODE",
+            "The generated reference contains a blocked substring",
+            safe_for_customer: false
+          )
+        end
       end
+      # Spec 21.2: a run of the same symbol at or above maxRepetition blocks
+      # the code. Runs are measured on the raw string, so a separator never
+      # breaks a run.
+      max = @profile.max_repetition
+      return unless max.positive? && /(.)\1{#{max - 1},}/.match?(raw)
+
+      raise BasehError.new(
+        "BLOCKED_CODE",
+        "The generated reference repeats a symbol beyond the profile limit",
+        safe_for_customer: false
+      )
     end
 
     def confusion_map(name)
@@ -269,12 +475,83 @@ module Baseh
       end
     end
 
+    # Spec 19.6. The id selects its generation by magnitude; the offset
+    # within the generation is permuted in that generation's own domain.
+    def encode_expandable(id)
+      if id.negative?
+        raise BasehError.new("OUT_OF_RANGE", "ID #{id} is negative")
+      end
+      l = generation_for_id(id)
+      value = id - generation_base(l)
+      domain = generation_capacity(l)
+      perm = @profile.permutation
+      effective_k = @profile.effective_checksum_length(l)
+      if perm[:enabled]
+        value = Feistel.permute(
+          value, domain,
+          profile_id: @profile.profile_id,
+          key_bytes: perm[:key_bytes],
+          rounds: perm[:rounds],
+          length: l
+        )
+      end
+      body = BaseN.encode_base_n(value, @profile.body_alphabet, l - effective_k)
+      checksum = Checksum.calculate_checksum(@profile, body, @body_index, effective_k)
+      raw = body + checksum
+      check_blocklist!(raw)
+      format_raw(raw)
+    end
+
+    # Spec 11/19.5. In expandable mode the separator applies only at or above
+    # separatorMinLength, with the balanced grouping derived from the total
+    # length.
     def format_raw(raw)
       return raw if @profile.separator.empty?
 
+      grouping =
+        if @profile.mode == "expandable"
+          return raw if raw.length < @profile.separator_min_length
+
+          self.class.expandable_grouping(raw.length)
+        else
+          @profile.grouping
+        end
       parts = []
       offset = 0
-      @profile.grouping.each do |size|
+      grouping.each do |size|
+        parts << raw.slice(offset, size)
+        offset += size
+      end
+      parts.join(@profile.separator)
+    end
+
+    # Spec 12.5 step 4: membership in the union of the body and checksum
+    # alphabets.
+    def inspect_union?(ch)
+      @body_index.key?(ch) || @profile.checksum_alphabet.include?(ch)
+    end
+
+    # Spec 12.5: separators inserted into a partially typed code, as far as
+    # the groups go. Fixed mode walks the configured grouping, emitting no
+    # separator for a group the symbols do not reach; expandable mode uses
+    # the balanced grouping rule of spec 19.5 for the typed length, bare
+    # below separatorMinLength.
+    def format_partial(raw)
+      return raw if @profile.separator.empty?
+
+      grouping =
+        if @profile.mode == "expandable"
+          return raw if raw.length < @profile.separator_min_length
+
+          self.class.expandable_grouping(raw.length)
+        else
+          @profile.grouping
+        end
+      parts = []
+      offset = 0
+      grouping.each do |size|
+        break if offset >= raw.length
+
         parts << raw.slice(offset, size)
         offset += size
       end
@@ -284,7 +561,7 @@ module Baseh
     def canonical_raw(canonical_code)
       return canonical_code if @profile.separator.empty?
 
-      canonical_code.delete(@profile.separator)
+      canonical_code.gsub(@profile.separator, "")
     end
   end
 end

data/lib/baseh/basen.rb

@@ -8,6 +8,13 @@ module Baseh
     # Spec 5.1. All arithmetic stays in Integer (arbitrary precision).
     def encode_base_n(value, alphabet, length)
       base = alphabet.length
+      capacity = base**length
+      if value.negative? || value >= capacity
+        raise BasehError.new(
+          "OUT_OF_RANGE",
+          "Value #{value} does not fit in #{length} base-#{base} symbols"
+        )
+      end
       out = Array.new(length)
       v = value
       (length - 1).downto(0) do |pos|