offline-translate 0.1.0__tar.gz

offline_translate-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+*.egg-info/
+dist/
+.pytest_cache/
+.DS_Store
+
+# downloaded language packs and the phone dump (contains personal data)
+models/
+phone_dump/
+.claude/

offline_translate-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 asimon655
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

offline_translate-0.1.0/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.5
+Name: offline-translate
+Version: 0.1.0
+Summary: Offline machine translation using Google Translate's on-device language packs
+Project-URL: Homepage, https://github.com/asimon655/offline-translate
+Project-URL: Issues, https://github.com/asimon655/offline-translate/issues
+Author: asimon655
+License-Expression: MIT
+License-File: LICENSE
+Keywords: google-translate,nmt,offline,tflite,translation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: ai-edge-litert>=1.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: sentencepiece>=0.2
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# offline-translate
+
+Offline machine translation in Python using the same on-device neural translation models that the
+Google Translate Android app downloads for offline use.
+
+- ~100 languages, each pack translates to/from English (~50 MB per language)
+- Fully offline once a pack is downloaded; no API key, no account
+- Language pairs without English (e.g. Hebrew → French) pivot through English, like the app does
+- Runs on CPU via [LiteRT](https://ai.google.dev/edge/litert) (TensorFlow Lite); ~0.3 s per sentence
+
+```bash
+pip install offline-translate
+```
+
+Supported platforms are those with `ai-edge-litert` wheels: macOS (Apple Silicon), Linux (x86_64,
+aarch64) and Windows (x64), Python 3.10+.
+
+## Usage
+
+```python
+from offline_translate import Translator
+
+t = Translator("~/.local/share/offline-translate")   # where models are stored
+
+t.download("he")                          # Hebrew <-> English, ~55 MB
+t.translate("Good morning!", target="he").text
+# 'בוקר טוב!'
+
+t.translate("בוקר טוב", target="en")      # source is auto-detected among installed languages
+# Translation(text='Good morning', source='iw', target='en', via=('iw', 'en'))
+```
+
+### Managing languages
+
+```python
+for lang in t.languages():                # full catalog (fetched once from Google, then cached)
+    print(lang.code, lang.name, lang.installed, lang.download_size, lang.update_available)
+
+t.installed_languages()                   # ['en', 'iw'] — works offline, no catalog needed
+t.download("fr", progress=lambda done, total: print(f"{done}/{total}"))
+t.remove("fr")
+
+t.update_catalog()                        # check Google for a newer catalog
+[l for l in t.languages() if l.update_available]   # then t.download(code) to update
+```
+
+Language codes follow Google's catalog (`iw` for Hebrew, `zh-Hant`, `jw`, …). Common aliases
+(`he`, `zh-TW`, `jv`) and English names (`"Hebrew"`) are accepted everywhere.
+
+English is built in: there is no separate English pack, since every pack contains both
+directions to and from English.
+
+### API
+
+| | |
+|---|---|
+| `Translator(models_dir, channel="prod", beam=4)` | `models_dir` is created if missing. `beam=1` is faster, slightly worse. |
+| `translate(text, target, source=None) -> Translation` | `Translation(text, source, target, via)` |
+| `detect(text) -> str` | Guess among installed languages |
+| `languages(installed_only=False) -> list[Language]` | `Language(code, name, installed, download_size, size, update_available, builtin, supported)` |
+| `installed_languages() -> list[str]` | |
+| `download(lang, progress=None, force=False)` | Verifies SHA-1 of every file; resumes interrupted downloads |
+| `remove(lang)` | |
+| `update_catalog() -> str` | Returns the catalog version |
+| `TranslateError` | Raised for unknown/uninstalled languages, checksum failures, … |
+
+Network errors surface as `OSError`.
+
+## Command line
+
+The repository also has a small CLI built on the library (not part of the PyPI package):
+
+```bash
+./translate get he                 # download Hebrew
+./translate "Good morning"         # English -> Hebrew by default; source auto-detected
+./translate -s he -t fr "בוקר טוב"  # any pair, via English
+./translate list --installed
+```
+
+`tools/pull_from_phone.py` copies already-downloaded packs from a rooted Android phone over adb.
+
+## How it works
+
+The Android app reads a manifest at
+`https://dl.google.com/translate/offline/manifest/gtr/prod/manifest.pb`, which lists versions of
+a *package profile* — the catalog of language packs and their files. Each translation direction is
+a TFLite encoder, an LSTM decoder run step by step with beam search, and SentencePiece tokenizers.
+
+### Limitations
+
+- 15 languages (am, az, bs, hy, kk, ky, ml, mn, ne, pa, ps, si, so, tg, uz) still ship an older
+  model format with a WordPiece tokenizer that isn't supported yet; they're listed with
+  `supported=False`.
+- Quality matches the app's offline mode, which is below online Google Translate.
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest                          # offline unit tests (fake Google server, fake model)
+pytest --network                # + checks against the real Google catalog
+pytest --models=~/models        # + end-to-end translation with installed Hebrew models
+```
+
+## Disclaimer
+
+This project is not affiliated with or endorsed by Google. It contains no Google code or models;
+language packs are downloaded from Google's public servers at runtime and are subject to Google's
+terms.

offline_translate-0.1.0/README.md

@@ -0,0 +1,109 @@
+# offline-translate
+
+Offline machine translation in Python using the same on-device neural translation models that the
+Google Translate Android app downloads for offline use.
+
+- ~100 languages, each pack translates to/from English (~50 MB per language)
+- Fully offline once a pack is downloaded; no API key, no account
+- Language pairs without English (e.g. Hebrew → French) pivot through English, like the app does
+- Runs on CPU via [LiteRT](https://ai.google.dev/edge/litert) (TensorFlow Lite); ~0.3 s per sentence
+
+```bash
+pip install offline-translate
+```
+
+Supported platforms are those with `ai-edge-litert` wheels: macOS (Apple Silicon), Linux (x86_64,
+aarch64) and Windows (x64), Python 3.10+.
+
+## Usage
+
+```python
+from offline_translate import Translator
+
+t = Translator("~/.local/share/offline-translate")   # where models are stored
+
+t.download("he")                          # Hebrew <-> English, ~55 MB
+t.translate("Good morning!", target="he").text
+# 'בוקר טוב!'
+
+t.translate("בוקר טוב", target="en")      # source is auto-detected among installed languages
+# Translation(text='Good morning', source='iw', target='en', via=('iw', 'en'))
+```
+
+### Managing languages
+
+```python
+for lang in t.languages():                # full catalog (fetched once from Google, then cached)
+    print(lang.code, lang.name, lang.installed, lang.download_size, lang.update_available)
+
+t.installed_languages()                   # ['en', 'iw'] — works offline, no catalog needed
+t.download("fr", progress=lambda done, total: print(f"{done}/{total}"))
+t.remove("fr")
+
+t.update_catalog()                        # check Google for a newer catalog
+[l for l in t.languages() if l.update_available]   # then t.download(code) to update
+```
+
+Language codes follow Google's catalog (`iw` for Hebrew, `zh-Hant`, `jw`, …). Common aliases
+(`he`, `zh-TW`, `jv`) and English names (`"Hebrew"`) are accepted everywhere.
+
+English is built in: there is no separate English pack, since every pack contains both
+directions to and from English.
+
+### API
+
+| | |
+|---|---|
+| `Translator(models_dir, channel="prod", beam=4)` | `models_dir` is created if missing. `beam=1` is faster, slightly worse. |
+| `translate(text, target, source=None) -> Translation` | `Translation(text, source, target, via)` |
+| `detect(text) -> str` | Guess among installed languages |
+| `languages(installed_only=False) -> list[Language]` | `Language(code, name, installed, download_size, size, update_available, builtin, supported)` |
+| `installed_languages() -> list[str]` | |
+| `download(lang, progress=None, force=False)` | Verifies SHA-1 of every file; resumes interrupted downloads |
+| `remove(lang)` | |
+| `update_catalog() -> str` | Returns the catalog version |
+| `TranslateError` | Raised for unknown/uninstalled languages, checksum failures, … |
+
+Network errors surface as `OSError`.
+
+## Command line
+
+The repository also has a small CLI built on the library (not part of the PyPI package):
+
+```bash
+./translate get he                 # download Hebrew
+./translate "Good morning"         # English -> Hebrew by default; source auto-detected
+./translate -s he -t fr "בוקר טוב"  # any pair, via English
+./translate list --installed
+```
+
+`tools/pull_from_phone.py` copies already-downloaded packs from a rooted Android phone over adb.
+
+## How it works
+
+The Android app reads a manifest at
+`https://dl.google.com/translate/offline/manifest/gtr/prod/manifest.pb`, which lists versions of
+a *package profile* — the catalog of language packs and their files. Each translation direction is
+a TFLite encoder, an LSTM decoder run step by step with beam search, and SentencePiece tokenizers.
+
+### Limitations
+
+- 15 languages (am, az, bs, hy, kk, ky, ml, mn, ne, pa, ps, si, so, tg, uz) still ship an older
+  model format with a WordPiece tokenizer that isn't supported yet; they're listed with
+  `supported=False`.
+- Quality matches the app's offline mode, which is below online Google Translate.
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest                          # offline unit tests (fake Google server, fake model)
+pytest --network                # + checks against the real Google catalog
+pytest --models=~/models        # + end-to-end translation with installed Hebrew models
+```
+
+## Disclaimer
+
+This project is not affiliated with or endorsed by Google. It contains no Google code or models;
+language packs are downloaded from Google's public servers at runtime and are subject to Google's
+terms.

offline_translate-0.1.0/conftest.py

@@ -0,0 +1,8 @@
+"""Root conftest: pytest only reads command-line options registered here."""
+import os
+
+
+def pytest_addoption(parser):
+    parser.addoption("--network", action="store_true", help="run tests that talk to dl.google.com")
+    parser.addoption("--models", metavar="DIR", default=os.environ.get("OFFLINE_TRANSLATE_TEST_MODELS"),
+                     help="models folder with Hebrew installed, for end-to-end translation tests")

offline_translate-0.1.0/pyproject.toml

@@ -0,0 +1,55 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "offline-translate"
+dynamic = ["version"]
+description = "Offline machine translation using Google Translate's on-device language packs"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "asimon655" }]
+keywords = ["translation", "offline", "nmt", "tflite", "google-translate"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Operating System :: MacOS",
+    "Operating System :: POSIX :: Linux",
+    "Operating System :: Microsoft :: Windows",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Topic :: Text Processing :: Linguistic",
+    "Typing :: Typed",
+]
+dependencies = [
+    "ai-edge-litert>=1.0",
+    "numpy>=1.24",
+    "sentencepiece>=0.2",
+]
+
+[project.urls]
+Homepage = "https://github.com/asimon655/offline-translate"
+Issues = "https://github.com/asimon655/offline-translate/issues"
+
+[project.optional-dependencies]
+test = ["pytest>=8"]
+
+[tool.hatch.version]
+path = "src/offline_translate/__init__.py"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/offline_translate"]
+
+# Explicit allow-list: models/, phone_dump/ etc. must never end up in an upload.
+[tool.hatch.build.targets.sdist]
+only-include = ["src", "tests", "conftest.py", "README.md", "LICENSE", "pyproject.toml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"
+markers = [
+    "network: talks to dl.google.com (enable with --network)",
+    "models: needs real installed models (enable with --models=DIR)",
+]

offline_translate-0.1.0/src/offline_translate/__init__.py

@@ -0,0 +1,12 @@
+"""Offline translation with Google Translate's on-device language packs.
+
+    from offline_translate import Translator
+    t = Translator("~/models")
+    t.download("he")
+    t.translate("Hello", target="he").text
+"""
+from .languages import NAMES, normalize
+from .translator import Language, TranslateError, Translation, Translator
+
+__version__ = "0.1.0"
+__all__ = ["Translator", "Translation", "Language", "TranslateError", "NAMES", "normalize", "__version__"]

offline_translate-0.1.0/src/offline_translate/catalog.py

@@ -0,0 +1,172 @@
+"""Google Translate offline package catalog.
+
+How the Android app finds packs (reverse engineered):
+  1. Manifest  https://dl.google.com/translate/offline/manifest/gtr/<channel>/manifest.pb
+     lists every package-profile version (+ sha1/size). The channel comes from a server flag ("prod").
+  2. Profile   https://dl.google.com/translate/offline/package_profile/<maj>/<min>/<patch>/translate_premium.pb.zlib
+     lists every language pack, its packages (nmt models, dictionary, translit) and their files.
+  3. Files     https://dl.google.com/translate/offline/nmt/<ver>/premium/<src>_<tgt>/0/<file>.zlib
+"""
+import struct
+import urllib.request
+import zlib
+from dataclasses import dataclass, field
+
+BASE = "https://dl.google.com/translate/offline"
+MANIFEST_URL = BASE + "/manifest/gtr/{channel}/manifest.pb"
+USER_AGENT = "Mozilla/5.0"
+
+PKG_NMT = 3
+# Files our inference engine knows how to run; packages with other layouts are skipped.
+NMT_FILES = {
+    "encoder_0.tflite", "decoder_init_0.tflite", "decoder_step_0.tflite",
+    "src_spm_model.model", "tgt_spm_model.model",
+}
+
+
+# --- minimal protobuf reader (Google's schema isn't public) ---
+def _varint(b, i):
+    r = s = 0
+    while True:
+        c = b[i]
+        i += 1
+        r |= (c & 0x7F) << s
+        s += 7
+        if c < 0x80:
+            return r, i
+
+
+def parse_pb(b):
+    """Parse a protobuf message into {field: [values]}; length-delimited values stay bytes."""
+    out, i = {}, 0
+    while i < len(b):
+        key, i = _varint(b, i)
+        f, t = key >> 3, key & 7
+        if t == 0:
+            v, i = _varint(b, i)
+        elif t == 2:
+            n, i = _varint(b, i)
+            v, i = b[i:i + n], i + n
+        elif t == 5:
+            v, i = struct.unpack("<f", b[i:i + 4])[0], i + 4
+        elif t == 1:
+            v, i = struct.unpack("<d", b[i:i + 8])[0], i + 8
+        else:
+            raise ValueError(f"unsupported protobuf wire type {t}")
+        out.setdefault(f, []).append(v)
+    return out
+
+
+def _str(m, f):
+    return m[f][0].decode()
+
+
+def _hash(m):
+    """{1: {1: algo, 2: hex}, 2: size} -> (hex, size)."""
+    return _str(parse_pb(m[1][0]), 2), m[2][0]
+
+
+def fetch(url, timeout=60):
+    req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return r.read()
+
+
+def open_url(url, timeout=60):
+    return urllib.request.urlopen(urllib.request.Request(url, headers={"User-Agent": USER_AGENT}), timeout=timeout)
+
+
+@dataclass(frozen=True)
+class ProfileRef:
+    version: tuple
+    url: str
+    sha1: str  # of the decompressed profile
+
+    @property
+    def version_str(self):
+        return ".".join(map(str, self.version))
+
+
+def parse_manifest(data):
+    """Returns the premium (NMT) package profiles listed in a manifest, oldest first."""
+    refs = []
+    groups = parse_pb(data).get(2, [])
+    for raw in (r for g in groups for r in parse_pb(g).get(2, [])):
+        m = parse_pb(raw)
+        url = _str(m, 2)
+        if "translate_premium" not in url:
+            continue
+        v = parse_pb(m[1][0])
+        version = tuple(v.get(k, [0])[0] for k in (1, 2, 3))
+        refs.append(ProfileRef(version, url, _hash(parse_pb(m[3][0]))[0]))
+    return sorted(refs, key=lambda r: r.version)
+
+
+@dataclass(frozen=True)
+class PackFile:
+    name: str       # file name on disk, e.g. encoder_0.tflite
+    url: str
+    sha1: str       # of the decompressed file
+    size: int       # decompressed bytes
+    download_size: int  # zlib-compressed bytes
+
+
+@dataclass(frozen=True)
+class Package:
+    name: str       # e.g. "en_iw" (direction src_tgt)
+    files: tuple
+
+    @property
+    def source(self):
+        return self.name.split("_", 1)[0]
+
+    @property
+    def target(self):
+        return self.name.split("_", 1)[1]
+
+
+@dataclass
+class Pack:
+    """A downloadable language: both directions to/from English."""
+    lang: str
+    packages: list = field(default_factory=list)
+    # False for older packs (nmt/8/1, WordPiece tokenizer) that the engine can't run yet.
+    supported: bool = True
+
+    @property
+    def download_size(self):
+        return sum(f.download_size for p in self.packages for f in p.files)
+
+    @property
+    def size(self):
+        return sum(f.size for p in self.packages for f in p.files)
+
+
+def parse_profile(data):
+    """Returns {lang_code: Pack} for every language with an NMT model (see Pack.supported)."""
+    top = parse_pb(data)
+    files = {}
+    for raw in top.get(3, []):
+        m = parse_pb(raw)
+        sha1, size = _hash(parse_pb(m[3][0]))
+        _, zsize = _hash(parse_pb(m[4][0]))
+        files[_str(m, 1)] = PackFile(_str(m, 5), _str(m, 2), sha1, size, zsize)
+
+    packages = {}
+    for raw in top.get(2, []):
+        m = parse_pb(raw)
+        if m[2][0] != PKG_NMT:
+            continue
+        pfiles = tuple(files[f.decode()] for f in m.get(4, []))
+        packages[_str(m, 1)] = Package(_str(m, 1).removesuffix("_premium_nmt"), pfiles)
+
+    packs = {}
+    for raw in top.get(1, []):
+        m = parse_pb(raw)
+        langs = [_str(parse_pb(x), 2) for x in parse_pb(m[3][0])[1]]
+        lang = next((l for l in langs if l != "en"), None)
+        pkgs = [packages[p.decode()] for p in m.get(2, []) if p.decode() in packages]
+        if lang and pkgs:
+            ok = all(NMT_FILES <= {f.name for f in p.files} for p in pkgs)
+            packs[lang] = Pack(lang, pkgs, ok)
+    return packs

offline_translate-0.1.0/src/offline_translate/engine.py

@@ -0,0 +1,118 @@
+"""Runs one direction of a Google Translate on-device NMT model (TFLite encoder + LSTM decoder)."""
+import os
+import re
+from pathlib import Path
+
+os.environ.setdefault("TF_CPP_MIN_LOG_LEVEL", "3")
+
+import numpy as np
+import sentencepiece as spm
+from ai_edge_litert.interpreter import Interpreter
+
+BOS, EOS = 1, 2
+STATE_KEYS = [f"beam_state:rnn_state_{i}_{x}:00" for i in (0, 1) for x in ("c", "m")]
+# The models are trained on single sentences; split on sentence-final punctuation + whitespace.
+SENT_RE = re.compile(r"(?<=[.!?…。！？])\s+")
+
+
+def split_sentences(text):
+    """Lines of sentences: line breaks are kept, each line is split into sentences."""
+    return [[s for s in SENT_RE.split(line.strip()) if s] for line in text.split("\n")]
+
+
+FINAL_PUNCT = ".!?…。！？:;\"'”’)»"
+
+
+def add_period_limit(processor_spec):
+    """max_sent_len of the pack's PreprocAddPeriod step, or None if the pack doesn't use it."""
+    m = re.search(rb"PreprocAddPeriod/[^:]*:max_sent_len=(\d+)", processor_spec)
+    return int(m.group(1)) if m else None
+
+
+def with_period(sentence, limit):
+    """Short sentences without final punctuation translate badly ("car" -> "tram"); the app adds
+    a period first. Returns (text to translate, whether a period was added)."""
+    if limit and len(sentence.split()) <= limit and sentence[-1:] not in FINAL_PUNCT:
+        return sentence + ".", True
+    return sentence, False
+
+
+def strip_period(translation):
+    return translation[:-1] if translation[-1:] in ".。" else translation
+
+
+class NmtModel:
+    def __init__(self, path):
+        path = Path(path)
+        self.src_sp = spm.SentencePieceProcessor(model_file=str(path / "src_spm_model.model"))
+        self.tgt_sp = spm.SentencePieceProcessor(model_file=str(path / "tgt_spm_model.model"))
+        runner = lambda name: Interpreter(model_path=str(path / f"{name}.tflite")).get_signature_runner()
+        self.encoder = runner("encoder_0")
+        self.dec_init = runner("decoder_init_0")
+        self.dec_step = runner("decoder_step_0")
+        spec = path / "processor_spec.pb"
+        self.period_limit = add_period_limit(spec.read_bytes()) if spec.exists() else None
+
+    def translate(self, text, beam=4):
+        return "\n".join(" ".join(self.translate_sentence(s, beam=beam) for s in line)
+                         for line in split_sentences(text))
+
+    def translate_sentence(self, text, beam=4):
+        text, added = with_period(text, self.period_limit)
+        out = self._decode(text, beam=beam)
+        return strip_period(out) if added else out
+
+    def _decode(self, text, beam=4, max_len=72, alpha=1.0):
+        # Source vocabularies are lowercase-only; the model restores casing on output.
+        ids = self.src_sp.encode(text.lower()) + [EOS]
+        src = np.array([ids], np.int32)
+        pad = np.zeros_like(src, np.float32)
+        encs = self.encoder(**{"feeds:src_ids:00": src, "feeds:src_paddings:00": pad})["fetches:src_encs:00"]
+        init = self.dec_init(**{
+            "feeds:num_hyps:00": np.array([1], np.int32),
+            "feeds:src_enc_paddings:00": pad,
+            "feeds:src_encs:00": encs,
+        })
+        vecs = init["atten_init:source_vecs:00"]
+        state = {k: np.zeros((1, 512), np.float32) for k in STATE_KEYS}
+        state["beam_state:atten_context:00"] = init["beam_state:atten_context:00"]
+
+        # Each live hypothesis: (tokens, cumulative log prob). Batch rows of `state` align with `hyps`.
+        hyps = [([], 0.0)]
+        done = []
+        max_len = min(max_len, 2 * len(ids) + 10)
+        for _ in range(max_len):
+            n = len(hyps)
+            last = np.array([[t[-1] if t else BOS] for t, _ in hyps], np.int32)
+            out = self.dec_step(**{
+                "atten_init:source_contexts:00": np.repeat(vecs, n, 0),
+                "atten_init:source_vecs:00": np.repeat(vecs, n, 0),
+                "atten_init:source_padding:00": np.repeat(pad, n, 0),
+                "feeds:step_ids:00": last,
+                **state,
+            })
+            logp = out["fetches:log_probs:00"]
+            scores = np.array([s for _, s in hyps])[:, None] + logp
+            flat = np.argsort(scores, axis=None)[::-1][: 2 * beam]
+            new_hyps, rows = [], []
+            for idx in flat:
+                row, tok = divmod(int(idx), logp.shape[1])
+                toks, sc = hyps[row][0] + [tok], float(scores[row, tok])
+                if tok == EOS:
+                    done.append((toks[:-1], sc / (((5 + len(toks)) / 6) ** alpha)))
+                elif len(new_hyps) < beam:
+                    new_hyps.append((toks, sc))
+                    rows.append(row)
+            if not new_hyps:
+                break
+            # Stop when the best finished hypothesis beats any live one's optimistic score.
+            if len(done) >= beam:
+                best_done = max(s for _, s in done)
+                best_live = max(s for _, s in new_hyps) / (((5 + max_len) / 6) ** alpha)
+                if best_done > best_live:
+                    break
+            hyps = new_hyps
+            state = {k: out[k][rows] for k in state}
+        if not done:
+            done = hyps
+        return self.tgt_sp.decode(max(done, key=lambda h: h[1])[0])