react-native-jieba 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -0,0 +1,51 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## 0.3.0
+
+### Breaking changes
+
+- **`initJieba` has been renamed to `prepareJieba`.** The old name implied a
+  mandatory global init step, which it no longer is on native. Update imports and
+  calls:
+
+  ```diff
+  - import { initJieba } from 'react-native-jieba';
+  - await initJieba();
+  + import { prepareJieba } from 'react-native-jieba';
+  + await prepareJieba();
+  ```
+
+  On **web** `prepareJieba()` is still required (it loads the `jieba-wasm`
+  binary). On **iOS/Android** it is now optional — see below.
+
+  `InitJiebaOptions` is likewise renamed to `PrepareJiebaOptions`.
+
+### Added
+
+- **`prepareJieba()` is now optional on native.** You can call `cut`, `tag`, etc.
+  directly without any setup — the dictionary is located/extracted automatically on
+  first use (iOS resolves it from the app bundle; Android lazily extracts the bundled
+  assets to `filesDir/jieba-dict/` via an fbjni bridge). Calling `prepareJieba()`
+  remains useful as an async warm-up to avoid a one-time blocking pause on Android's
+  first call.
+- **`isJiebaReady(): boolean`** — a synchronous readiness check, backed by the real
+  native engine state (a new `isReady()` JSI method) rather than a JS flag, so it
+  stays correct even when the dictionary is resolved lazily on the first call.
+
+## 0.2.0
+
+### Added
+
+- Web support via [`jieba-wasm`](https://github.com/fengkx/jieba-wasm) for
+  react-native-web, exposing the same API as the native build.
+
+## 0.1.0
+
+- Initial release: cppjieba C++ Turbo Module for iOS and Android with `cut`,
+  `cutAll`, `cutForSearch`, `cutHMM`, `cutSmall`, `tag`, `extract`, `insertUserWord`,
+  and `find`.

package/Jieba.podspec

@@ -14,6 +14,8 @@ Pod::Spec.new do |s|
   s.source       = { :git => "https://github.com/leonsilicon/react-native-jieba.git", :tag => "#{s.version}" }
 
   s.source_files = "ios/**/*.{h,m,mm}", "cpp/*.{hpp,cpp,c,h}", "ios/generated/*.{h,cpp,mm}"
+  # JiebaDictAndroid.{cpp,h} depend on fbjni and are Android-only.
+  s.exclude_files = "cpp/JiebaDictAndroid.{cpp,h}"
   s.private_header_files = "ios/**/*.h"
 
   s.resources = ["cpp/cppjieba/dict/*.utf8"]

package/README.md

@@ -48,11 +48,21 @@ export default {
 
 ## Usage
 
-Call `initJieba()` once before the first segmentation call. On iOS the dictionary path is resolved from the app bundle automatically; on Android it extracts the bundled assets to `filesDir/jieba-dict/` on first run.
+On **native (iOS/Android)** there is no setup — just call `cut`, `tag`, etc. The dictionary is located/extracted automatically: on iOS the path is resolved from the app bundle, and on Android the bundled assets are extracted to `filesDir/jieba-dict/` on the first call.
+
+```ts
+import { cut } from 'react-native-jieba';
+
+cut('我来到北京清华大学');
+// ['我', '来到', '北京', '清华大学']
+```
+
+`prepareJieba()` is an **optional async warm-up** on native: on Android's first run the asset extraction happens off the call site (asynchronously), avoiding a one-time blocking pause on the first `cut`/`tag`. On **web** it is **required** — it loads the `jieba-wasm` binary, which cannot happen synchronously. Use `isJiebaReady()` to check synchronously whether segmentation can run.
 
 ```ts
 import {
-  initJieba,
+  prepareJieba,
+  isJiebaReady,
   cut,
   cutAll,
   cutForSearch,
@@ -60,7 +70,10 @@ import {
   extract,
 } from 'react-native-jieba';
 
-await initJieba();
+// Native: optional warm-up. Web: required (loads wasm).
+await prepareJieba();
+
+isJiebaReady(); // true
 
 cut('我来到北京清华大学');
 // ['我', '来到', '北京', '清华大学']
@@ -80,11 +93,12 @@ extract('我是拖拉机学院手扶拖拉机专业的。', 5);
 
 ## API
 
-On native, all segmentation calls are synchronous JSI calls — no Promises. On web they are synchronous too, but `initJieba()` must finish first (it loads the wasm module).
+On native, all segmentation calls are synchronous JSI calls — no Promises. On web they are synchronous too, but `prepareJieba()` must finish first (it loads the wasm module).
 
 | Function | Signature | Notes |
 | --- | --- | --- |
-| `initJieba(options?)` | `(options?: { wasmUrl?: string \| URL \| Request }) => Promise<void>` | Must be awaited once before any other call. Idempotent. `wasmUrl` is web-only and rarely needed (override where the `.wasm` is served). |
+| `prepareJieba(options?)` | `(options?: { wasmUrl?: string \| URL \| Request }) => Promise<void>` | **Web:** must be awaited once before any other call. **Native:** optional warm-up — calling segmentation directly works too. Idempotent. `wasmUrl` is web-only and rarely needed (override where the `.wasm` is served). |
+| `isJiebaReady()` | `() => boolean` | Synchronous check for whether segmentation can run now. Native-backed (reflects the real engine state, not whether `prepareJieba` was called): `true` on iOS from launch, `true` on Android once the dict is resolved (eagerly via `prepareJieba` or lazily on the first call). |
 | `cut(sentence, hmm?)` | `(string, boolean) => string[]` | Precise mode. `hmm` defaults to `true`. |
 | `cutAll(sentence)` | `(string) => string[]` | Full mode (every possible word). |
 | `cutForSearch(sentence, hmm?)` | `(string, boolean) => string[]` | Search-engine mode. |
@@ -130,15 +144,15 @@ export default defineConfig({
 // App.tsx
 import { useEffect, useState } from 'react';
 import { Platform, Text, View } from 'react-native';
-import { initJieba, cut, cutForSearch, tag, extract } from 'react-native-jieba';
+import { prepareJieba, cut, cutForSearch, tag, extract } from 'react-native-jieba';
 
 export default function App() {
   const [words, setWords] = useState<string[] | null>(null);
 
   useEffect(() => {
-    // On web this loads + instantiates the wasm module (async).
-    // On native it's a near-instant no-op / dict-path setup.
-    initJieba()
+    // On web this loads + instantiates the wasm module (async) and is required.
+    // On native you can skip this and call cut() directly.
+    prepareJieba()
       .then(() => {
         setWords(cut('我来到北京清华大学'));
         // → ['我', '来到', '北京', '清华大学']
@@ -169,16 +183,17 @@ export default function App() {
 To self-host the binary (CDN or custom path), pass `wasmUrl`:
 
 ```ts
-await initJieba({ wasmUrl: 'https://cdn.example.com/jieba_rs_wasm_bg.wasm' });
+await prepareJieba({ wasmUrl: 'https://cdn.example.com/jieba_rs_wasm_bg.wasm' });
 ```
 
 ## How it works
 
 - The Turbo Module lives in `cpp/JiebaImpl.{h,cpp}` and wraps `cppjieba::Jieba` as a JSI Cxx module.
-- iOS resolves the dictionary directory from `NSBundle` in `ios/OnLoad.mm` before the module is registered, so `initJieba()` is a no-op on iOS.
-- Android ships the dictionary as AAR assets and extracts them on demand via `JiebaAndroidHelperModule`, which then hands the path to the C++ module via the codegen-exposed `setDictPath` JSI method.
+- iOS resolves the dictionary directory from `NSBundle` in `ios/OnLoad.mm` before the module is registered, so segmentation works immediately and `prepareJieba()` is a no-op on iOS.
+- Android ships the dictionary as AAR assets and extracts them to `filesDir/jieba-dict/`. This happens automatically: if `prepareJieba()` is never called, the C++ module extracts them synchronously on the first segmentation call via an fbjni call into `JiebaDict.extractDictDirFromNative` (a one-time cost). `prepareJieba()` does the same extraction asynchronously up front (through `JiebaAndroidHelperModule` → the codegen-exposed `setDictPath` JSI method) so that first call doesn't block.
+- `isJiebaReady()` is backed by the codegen-exposed `isReady()` JSI method, which reads the native engine state directly — so it stays correct even when the dictionary is resolved lazily on the first call.
 - cppjieba and its `limonp` dependency are vendored as git submodules under `cpp/cppjieba/`. The published npm tarball contains only the headers and dictionaries that are actually needed at build time.
-- On web, `react-native-web`'s bundler resolution picks up `src/NativeJieba.web.ts` and `src/init.web.ts`. `initJieba()` lazily imports `jieba-wasm`, awaits its async wasm initializer, and routes the JS API to the wasm exports.
+- On web, `react-native-web`'s bundler resolution picks up `src/NativeJieba.web.ts` and `src/init.web.ts`. `prepareJieba()` lazily imports `jieba-wasm`, awaits its async wasm initializer, and routes the JS API to the wasm exports.
 
 ## Contributing
 

package/android/CMakeLists.txt

@@ -6,6 +6,7 @@ set (CMAKE_VERBOSE_MAKEFILE ON)
 add_library(
     react-native-jieba    STATIC
     ../cpp/JiebaImpl.cpp
+    ../cpp/JiebaDictAndroid.cpp
 )
 
 set_target_properties(
@@ -26,4 +27,5 @@ target_link_libraries(
     react-native-jieba    jsi
     reactnative
     react_codegen_JiebaSpec
+    fbjni
 )

package/android/generated/java/com/jieba/NativeJiebaSpec.java

@@ -32,6 +32,10 @@ public abstract class NativeJiebaSpec extends ReactContextBaseJavaModule impleme
     return NAME;
   }
 
+  @ReactMethod(isBlockingSynchronousMethod = true)
+  @DoNotStrip
+  public abstract boolean isReady();
+
   @ReactMethod
   @DoNotStrip
   public abstract void setDictPath(String path);

package/android/generated/jni/JiebaSpec-generated.cpp

@@ -12,6 +12,11 @@
 
 namespace facebook::react {
 
+static facebook::jsi::Value __hostFunction_NativeJiebaSpecJSI_isReady(facebook::jsi::Runtime& rt, TurboModule &turboModule, const facebook::jsi::Value* args, size_t count) {
+  static jmethodID cachedMethodId = nullptr;
+  return static_cast<JavaTurboModule &>(turboModule).invokeJavaMethod(rt, BooleanKind, "isReady", "()Z", args, count, cachedMethodId);
+}
+
 static facebook::jsi::Value __hostFunction_NativeJiebaSpecJSI_setDictPath(facebook::jsi::Runtime& rt, TurboModule &turboModule, const facebook::jsi::Value* args, size_t count) {
   static jmethodID cachedMethodId = nullptr;
   return static_cast<JavaTurboModule &>(turboModule).invokeJavaMethod(rt, VoidKind, "setDictPath", "(Ljava/lang/String;)V", args, count, cachedMethodId);
@@ -64,6 +69,7 @@ static facebook::jsi::Value __hostFunction_NativeJiebaSpecJSI_find(facebook::jsi
 
 NativeJiebaSpecJSI::NativeJiebaSpecJSI(const JavaTurboModule::InitParams &params)
   : JavaTurboModule(params) {
+  methodMap_["isReady"] = MethodMetadata {0, __hostFunction_NativeJiebaSpecJSI_isReady};
   methodMap_["setDictPath"] = MethodMetadata {1, __hostFunction_NativeJiebaSpecJSI_setDictPath};
   methodMap_["cut"] = MethodMetadata {2, __hostFunction_NativeJiebaSpecJSI_cut};
   methodMap_["cutAll"] = MethodMetadata {1, __hostFunction_NativeJiebaSpecJSI_cutAll};

package/android/generated/jni/react/renderer/components/JiebaSpec/JiebaSpecJSI.h

@@ -22,6 +22,7 @@ public:
 
 protected:
   NativeJiebaCxxSpec(std::shared_ptr<CallInvoker> jsInvoker) : TurboModule(std::string{NativeJiebaCxxSpec::kModuleName}, jsInvoker) {
+    methodMap_["isReady"] = MethodMetadata {.argCount = 0, .invoker = __isReady};
     methodMap_["setDictPath"] = MethodMetadata {.argCount = 1, .invoker = __setDictPath};
     methodMap_["cut"] = MethodMetadata {.argCount = 2, .invoker = __cut};
     methodMap_["cutAll"] = MethodMetadata {.argCount = 1, .invoker = __cutAll};
@@ -35,6 +36,13 @@ protected:
   }
   
 private:
+  static jsi::Value __isReady(jsi::Runtime &rt, TurboModule &turboModule, const jsi::Value* /*args*/, size_t /*count*/) {
+    static_assert(
+      bridging::getParameterCount(&T::isReady) == 1,
+      "Expected isReady(...) to have 1 parameters");
+    return bridging::callFromJs<bool>(rt, &T::isReady,  static_cast<NativeJiebaCxxSpec*>(&turboModule)->jsInvoker_, static_cast<T*>(&turboModule));
+  }
+
   static jsi::Value __setDictPath(jsi::Runtime &rt, TurboModule &turboModule, const jsi::Value* args, size_t count) {
     static_assert(
       bridging::getParameterCount(&T::setDictPath) == 2,

package/android/src/main/java/com/jieba/JiebaAndroidHelperModule.kt

@@ -3,31 +3,24 @@ package com.jieba
 import com.facebook.react.bridge.Promise
 import com.facebook.react.bridge.ReactApplicationContext
 import com.facebook.react.module.annotations.ReactModule
-import java.io.File
-import java.io.FileOutputStream
 
 @ReactModule(name = JiebaAndroidHelperModule.NAME)
 class JiebaAndroidHelperModule(reactContext: ReactApplicationContext) :
   NativeJiebaAndroidHelperSpec(reactContext) {
 
+  init {
+    // Capture an application context so the C++ side can lazily extract the
+    // dictionaries on first use, even if JS never calls prepareJieba().
+    JiebaDict.setContext(reactContext)
+  }
+
   override fun getName(): String = NAME
 
   override fun prepareDictDir(promise: Promise) {
     val context = reactApplicationContext
     Thread {
       try {
-        val dictDir = File(context.filesDir, DICT_DIR_NAME)
-        if (!dictDir.exists()) dictDir.mkdirs()
-        for (filename in DICT_FILES) {
-          val target = File(dictDir, filename)
-          if (target.exists() && target.length() > 0) continue
-          context.assets.open(filename).use { input ->
-            FileOutputStream(target).use { output ->
-              input.copyTo(output)
-            }
-          }
-        }
-        promise.resolve(dictDir.absolutePath)
+        promise.resolve(JiebaDict.extractDictDir(context))
       } catch (t: Throwable) {
         promise.reject("E_JIEBA_DICT_EXTRACT", t.message, t)
       }
@@ -36,13 +29,5 @@ class JiebaAndroidHelperModule(reactContext: ReactApplicationContext) :
 
   companion object {
     const val NAME = "JiebaAndroidHelper"
-    private const val DICT_DIR_NAME = "jieba-dict"
-    private val DICT_FILES = arrayOf(
-      "jieba.dict.utf8",
-      "hmm_model.utf8",
-      "user.dict.utf8",
-      "idf.utf8",
-      "stop_words.utf8",
-    )
   }
 }

package/android/src/main/java/com/jieba/JiebaDict.kt

@@ -0,0 +1,72 @@
+package com.jieba
+
+import android.content.Context
+import java.io.File
+import java.io.FileOutputStream
+
+/**
+ * Shared dictionary extraction logic for Android.
+ *
+ * The cppjieba dictionaries ship as compressed APK assets, but cppjieba needs
+ * real filesystem paths. This object extracts them once into `filesDir/jieba-dict`
+ * and returns the directory.
+ *
+ * It is used from two places:
+ *  - [JiebaAndroidHelperModule.prepareDictDir] — the async warm-up exposed to JS
+ *    via `prepareJieba()`.
+ *  - [extractDictDirFromNative] — a synchronous entry point invoked from C++
+ *    (`JiebaImpl::getJieba`) on the very first segmentation call if `prepareJieba()`
+ *    was never awaited, so that callers don't have to call `prepareJieba` at all.
+ */
+object JiebaDict {
+  const val DICT_DIR_NAME = "jieba-dict"
+
+  val DICT_FILES = arrayOf(
+    "jieba.dict.utf8",
+    "hmm_model.utf8",
+    "user.dict.utf8",
+    "idf.utf8",
+    "stop_words.utf8",
+  )
+
+  /**
+   * Application context captured at module construction so the native (C++)
+   * fallback can extract assets without a `ReactApplicationContext` handle.
+   */
+  @Volatile
+  private var appContext: Context? = null
+
+  @JvmStatic
+  fun setContext(context: Context) {
+    if (appContext == null) {
+      appContext = context.applicationContext
+    }
+  }
+
+  /** Synchronously extract the dictionary assets and return the directory path. */
+  @JvmStatic
+  fun extractDictDir(context: Context): String {
+    val dictDir = File(context.filesDir, DICT_DIR_NAME)
+    if (!dictDir.exists()) dictDir.mkdirs()
+    for (filename in DICT_FILES) {
+      val target = File(dictDir, filename)
+      if (target.exists() && target.length() > 0) continue
+      context.assets.open(filename).use { input ->
+        FileOutputStream(target).use { output ->
+          input.copyTo(output)
+        }
+      }
+    }
+    return dictDir.absolutePath
+  }
+
+  /**
+   * Called from C++ via JNI. Returns the dict directory path, or an empty
+   * string if no context has been captured yet (extraction not possible).
+   */
+  @JvmStatic
+  fun extractDictDirFromNative(): String {
+    val context = appContext ?: return ""
+    return extractDictDir(context)
+  }
+}

package/android/src/main/java/com/jieba/JiebaPackage.kt

@@ -8,6 +8,10 @@ import com.facebook.react.module.model.ReactModuleInfoProvider
 
 class JiebaPackage : BaseReactPackage() {
   override fun getModule(name: String, reactContext: ReactApplicationContext): NativeModule? {
+    // Capture an application context as early as possible so the C++ module can
+    // lazily extract the bundled dictionaries on first use without requiring a
+    // call to prepareJieba().
+    JiebaDict.setContext(reactContext)
     return if (name == JiebaAndroidHelperModule.NAME) {
       JiebaAndroidHelperModule(reactContext)
     } else {

package/cpp/JiebaDictAndroid.cpp

@@ -0,0 +1,26 @@
+#include "JiebaDictAndroid.h"
+
+#include <fbjni/fbjni.h>
+
+namespace facebook::react {
+
+std::string extractJiebaDictDirAndroid() {
+  using namespace facebook::jni;
+
+  // Calling a JNI method requires a thread attached to the JVM. cut()/tag()
+  // run on the JS thread, which fbjni attaches; ThreadScope makes this robust
+  // for any caller thread.
+  ThreadScope ts;
+
+  static const auto cls = findClassStatic("com/jieba/JiebaDict");
+  static const auto method =
+    cls->getStaticMethod<jstring()>("extractDictDirFromNative");
+
+  local_ref<jstring> result = method(cls);
+  if (!result) {
+    return "";
+  }
+  return result->toStdString();
+}
+
+}

package/cpp/JiebaDictAndroid.h

@@ -0,0 +1,14 @@
+#pragma once
+
+#include <string>
+
+namespace facebook::react {
+
+// Android-only: synchronously extracts the bundled dictionary assets (via a
+// JNI call into Kotlin) and returns the directory path. Returns an empty string
+// if extraction is not possible yet (e.g. no application context captured).
+//
+// Defined in JiebaDictAndroid.cpp and only compiled on Android.
+std::string extractJiebaDictDirAndroid();
+
+}

package/cpp/JiebaImpl.cpp

@@ -1,5 +1,9 @@
 #include "JiebaImpl.h"
 
+#ifdef __ANDROID__
+#include "JiebaDictAndroid.h"
+#endif
+
 #include <cppjieba/Jieba.hpp>
 
 #include <memory>
@@ -58,6 +62,14 @@ void JiebaImpl::setDictPathFromNative(const std::string& dictPath) {
   }
 }
 
+bool JiebaImpl::isReady(jsi::Runtime& rt) {
+  std::lock_guard<std::mutex> lock(dictMutex());
+  // Ready once the engine is constructed, or once a dict path is resolved (the
+  // engine is then built lazily on first use). On iOS the path is set at +load;
+  // on Android it is set eagerly by prepareJieba or lazily on the first call.
+  return jiebaStorage() != nullptr || !dictPathStorage().empty();
+}
+
 void JiebaImpl::setDictPath(jsi::Runtime& rt, jsi::String path) {
   setDictPathFromNative(path.utf8(rt));
 }
@@ -65,6 +77,17 @@ void JiebaImpl::setDictPath(jsi::Runtime& rt, jsi::String path) {
 cppjieba::Jieba& JiebaImpl::getJieba() {
   std::lock_guard<std::mutex> lock(dictMutex());
   if (!jiebaStorage()) {
+#ifdef __ANDROID__
+    // If prepareJieba() was never awaited, extract the bundled assets now (a
+    // one-time, synchronous cost on the very first call) so callers don't have
+    // to call prepareJieba at all.
+    if (dictPathStorage().empty()) {
+      std::string extracted = extractJiebaDictDirAndroid();
+      if (!extracted.empty()) {
+        dictPathStorage() = extracted;
+      }
+    }
+#endif
     const std::string& base = dictPathStorage();
     if (base.empty()) {
       throw std::runtime_error(

package/cpp/JiebaImpl.h

@@ -19,6 +19,7 @@ public:
 
   static void setDictPathFromNative(const std::string& dictPath);
 
+  bool isReady(jsi::Runtime& rt);
   void setDictPath(jsi::Runtime& rt, jsi::String path);
   jsi::Array cut(jsi::Runtime& rt, jsi::String sentence, bool hmm);
   jsi::Array cutAll(jsi::Runtime& rt, jsi::String sentence);

package/cpp/cppjieba/CHANGELOG.md

@@ -0,0 +1,18 @@
+# CHANGELOG
+
+## v5.6.7
+
++ docs: document dictionary file formats
++ test: add `了解` segmentation regression coverage
++ test: replace copyrighted benchmark test data with synthetic text
+
+## v5.6.6
+
++ fix: add `习总书记` to the main dictionary for the reported `和习` segmentation case
++ chore: remove stray `韩玉鉴赏` entry from the default user dictionary
+
+## v5.6.5
+
++ deps: upgrade limonp to v1.0.2
+
+Older release history is preserved in git tags.