@gg-web-engine/ammo 0.0.59 → 0.0.60

package/build_gg_ammo/README.md

@@ -1,13 +1,37 @@
 This is a place where I build proprietary ammo.js for gg-web-engine.
 
-Reasons to build own ammo.js instead of using [up-to-date one](https://github.com/i12345/ammo.js):
-1) The crucial feature, which bullet does not have: use collision groups for raycast vehicle. I already made a 
-[PR](https://github.com/bulletphysics/bullet3/pull/4559) to bullet.
-1) Manually added functions "getPointer" and "compare" to type declarations. 
-[PR](https://github.com/anotherpit/webidl2ts/pull/2) to webidl2ts is also on it's way
-1) Expose `m_erp2` property of `btContactSolverInfo`, needed for fighting with object clipping problem,
-which appeared in bullet 2.84. Also made a [PR](https://github.com/i12345/ammo.js/pull/2)
+The build fetches [AndyGura/ammo.js](https://github.com/AndyGura/ammo.js) (my own fork, itself
+originally forked from the now-deleted `i12345/ammo.js`, which was in turn a more up-to-date fork
+of the unmaintained upstream [kripken/ammo.js](https://github.com/kripken/ammo.js)) at the pinned
+commit in `Makefile`, then applies the patches below on top before compiling. Whichever ammo.js
+repo `AMMOJS_GIT_REPO` points at only supplies the JS/TS build tooling (Emscripten/CMake/
+webidl2ts wiring) - the actual Bullet source is fetched separately from `BULLET_GIT_REPO` at its
+own pinned commit (see `Makefile`), and `ammo.idl` plus the vehicle raycaster source are replaced
+wholesale by `patch_files/` regardless of what the fetched repo ships, so this build isn't
+sensitive to the fetched ammo.js repo's own Bullet submodule pin or `ammo.idl` version.
 
+Reasons to build own ammo.js instead of using a stock upstream one:
+1) The crucial feature, which bullet does not have: use collision groups for raycast vehicle. I
+   already made a [PR](https://github.com/bulletphysics/bullet3/pull/4559) to bullet.
+1) Expose `m_erp2` property of `btContactSolverInfo`, needed for fighting with object clipping
+   problem, which appeared in bullet 2.84. Also made a PR to the (now-deleted) `i12345/ammo.js`
+   about it.
+
+(A third reason used to live here: `ammo.d.ts` was missing the `getPointer`/`compare` module
+functions, hand-patched into the generated output. That's no longer necessary - the
+[PR](https://github.com/anotherpit/webidl2ts/pull/2) fixing it in `webidl2ts` merged, and
+`AndyGura/ammo.js`'s `package.json` depends on `github:anotherpit/webidl2ts` unpinned, so a fresh
+`npm i` already pulls the fix.)
+
+`patch_files/CMakeLists.txt` and `patch_files/Dockerfile` are a different kind of patch: not a
+gg-web-engine feature, but keeping the build itself compiling against a moving Emscripten. The
+`Dockerfile` pins `emscripten/emsdk` to an exact tag instead of `:latest`, and `CMakeLists.txt`
+merges `EXTRA_EXPORTED_RUNTIME_METHODS` into `EXPORTED_RUNTIME_METHODS` and links the final
+`ammo.js`/`ammo.wasm.js` with `em++` instead of `emcc` - both required once this build was re-run
+against a current Emscripten (removed `EXTRA_EXPORTED_RUNTIME_METHODS` outright, and `emcc` no
+longer defaults to C++ linkage the way it used to, so linking Bullet's C++ objects with plain
+`emcc` fails with "undefined symbol: operator new"). If a future emsdk bump breaks the build again,
+expect it to show up as one of these two symptoms first.
 
 All changed file diffs in ammo.js and bullet can be found [here](./ammo_patches.txt) and [here](./bullet_patches.txt) appropriately
 
@@ -15,3 +39,11 @@ All changed file diffs in ammo.js and bullet can be found [here](./ammo_patches.
 ### How to build
 1) Docker has to be installed and started
 1) enter this directory and run `make all`
+1) commit the resulting changes under `../src/ammo.js` along with any `Makefile`/`*_patches.txt`
+   changes
+
+Or trigger the `Rebuild self-built ammo.js` GitHub Actions workflow (`workflow_dispatch`) instead -
+it runs the same `make all` in CI and commits the result back to the branch, so this no longer
+requires a local Docker install. That workflow also runs automatically (without committing) on any
+PR touching this directory, as a check that the build recipe still works - see
+`.github/workflows/build_ammo.yml`.

package/build_gg_ammo/ammo_patches.txt

@@ -1,3 +1,59 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 0512013..39cf63c 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -35,15 +35,13 @@ set(ALLOW_MEMORY_GROWTH 0 CACHE STRING "Allow Memory Growth")
+ 
+ set(EMCC_ARGS
+   --extern-pre-js ${AMMO_FRONT_MATTER_FILE}
+-  --llvm-lto 1
+   --post-js glue.js
+   --post-js ${AMMO_ONLOAD_FILE}
+   -O3
+   -s ALLOW_MEMORY_GROWTH=${ALLOW_MEMORY_GROWTH}
+   -s ALLOW_TABLE_GROWTH=1
+   -s EXPORTED_FUNCTIONS=["_malloc","_free"]
+-  -s EXPORTED_RUNTIME_METHODS=["UTF8ToString"]
+-  -s EXTRA_EXPORTED_RUNTIME_METHODS=["addFunction"]
++  -s EXPORTED_RUNTIME_METHODS=["UTF8ToString","addFunction"]
+   -s EXPORT_NAME="Ammo"
+   -s MODULARIZE=1
+   -s NO_EXIT_RUNTIME=1
+@@ -103,7 +101,10 @@ add_custom_target(ammo-bindings ALL DEPENDS glue.js glue.o)
+ #######################################
+ add_custom_command(
+   OUTPUT ammo.js
+-  COMMAND emcc glue.o ${BULLET_LIBS} ${EMCC_JS_ARGS} -o ammo.js
++  # em++ (not emcc): the final link pulls in libBulletCollision/Dynamics/etc, which are C++ and
++  # need operator new/delete resolved from libc++ - modern emscripten's emcc no longer defaults
++  # to C++ linkage the way it used to, and fails with "undefined symbol: operator new" otherwise.
++  COMMAND em++ glue.o ${BULLET_LIBS} ${EMCC_JS_ARGS} -o ammo.js
+   DEPENDS ammo-bindings ${AMMO_FRONT_MATTER_FILE} ${AMMO_ONLOAD_FILE} ${BULLET_TARGETS}
+   COMMENT "Building ammo javascript"
+   VERBATIM)
+@@ -113,7 +114,7 @@ add_custom_target(ammo-javascript ALL DEPENDS ammo.js)
+ #######################################
+ add_custom_command(
+   OUTPUT ammo.wasm.js ammo.wasm.wasm
+-  COMMAND emcc glue.o ${BULLET_LIBS} ${EMCC_WASM_ARGS} -o ammo.wasm.js
++  COMMAND em++ glue.o ${BULLET_LIBS} ${EMCC_WASM_ARGS} -o ammo.wasm.js
+   DEPENDS ammo-bindings ${AMMO_FRONT_MATTER_FILE} ${AMMO_ONLOAD_FILE} ${BULLET_TARGETS}
+   COMMENT "Building ammo webassembly"
+   VERBATIM)
+diff --git a/Dockerfile b/Dockerfile
+index e7f427c..a582408 100644
+--- a/Dockerfile
++++ b/Dockerfile
+@@ -1,4 +1,8 @@
+-FROM emscripten/emsdk
++# Pinned (not :latest): an unpinned emsdk silently drifts to whatever Emscripten ships that day,
++# which has broken this build before (removed -s EXTRA_EXPORTED_RUNTIME_METHODS, changed emcc's
++# default C/C++ linkage - see CMakeLists.txt). Bump deliberately, and re-run the full build to
++# confirm CMakeLists.txt's EMCC_ARGS/em++ usage still matches whatever version you bump to.
++FROM emscripten/emsdk:6.0.8
+ ENV PYTHONUNBUFFERED 1
+ RUN apt-get update \
+     && apt-get install -y --no-install-recommends \
 diff --git a/ammo.idl b/ammo.idl
 index 1f6b880..9c2e5cf 100644
 --- a/ammo.idl

package/build_gg_ammo/patch_files/CMakeLists.txt

@@ -0,0 +1,121 @@
+cmake_minimum_required(VERSION 3.13 FATAL_ERROR)
+set(CMAKE_POLICY_DEFAULT_CMP0077 NEW)
+
+FIND_PACKAGE(Python3)
+set(PYTHON ${Python3_EXECUTABLE} CACHE STRING "Python path")
+set(EMSCRIPTEN_ROOT $ENV{EMSDK}/upstream/emscripten CACHE STRING "Emscripten path")
+set(CMAKE_TOOLCHAIN_FILE ${EMSCRIPTEN_ROOT}/cmake/Modules/Platform/Emscripten.cmake)
+set(WEBIDL_BINDER_SCRIPT ${EMSCRIPTEN_ROOT}/tools/webidl_binder.py)
+set(AMMO_FRONT_MATTER_FILE ${CMAKE_CURRENT_SOURCE_DIR}/front-matter.js)
+set(AMMO_HEADER_FILE ${CMAKE_CURRENT_SOURCE_DIR}/ammo.h)
+set(AMMO_IDL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/ammo.idl)
+set(AMMO_ONLOAD_FILE ${CMAKE_CURRENT_SOURCE_DIR}/onload.js)
+set(BULLET_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/bullet/src/)
+
+
+set(BULLET_TARGETS BulletCollision BulletDynamics BulletSoftBody LinearMath)
+foreach(_TARGET ${BULLET_TARGETS})
+  list(APPEND BULLET_LIBS $<TARGET_FILE:${_TARGET}>)
+endforeach()
+
+# Disable bullet graphical benchmarks
+set(USE_GRAPHICAL_BENCHMARK OFF)
+
+# Disable bullet GLUT
+set(USE_GLUT OFF)
+
+# Build Release by default
+set(CMAKE_BUILD_TYPE Release CACHE STRING "Build Type")
+
+# Default is 64*1024*1024 = 64MB
+set(TOTAL_MEMORY 67108864 CACHE STRING "Total Memory")
+
+# Enable for resizable heap, with some amount of slowness
+set(ALLOW_MEMORY_GROWTH 0 CACHE STRING "Allow Memory Growth")
+
+set(EMCC_ARGS
+  --extern-pre-js ${AMMO_FRONT_MATTER_FILE}
+  --post-js glue.js
+  --post-js ${AMMO_ONLOAD_FILE}
+  -O3
+  -s ALLOW_MEMORY_GROWTH=${ALLOW_MEMORY_GROWTH}
+  -s ALLOW_TABLE_GROWTH=1
+  -s EXPORTED_FUNCTIONS=["_malloc","_free"]
+  -s EXPORTED_RUNTIME_METHODS=["UTF8ToString","addFunction"]
+  -s EXPORT_NAME="Ammo"
+  -s MODULARIZE=1
+  -s NO_EXIT_RUNTIME=1
+  -s NO_FILESYSTEM=1
+  -s TOTAL_MEMORY=${TOTAL_MEMORY})
+
+if(${CLOSURE})
+  # Ignore closure errors about the bullet Node class
+  # (Node is a DOM thing too)
+  LIST(APPEND EMCC_ARGS
+    --closure 1
+    -s IGNORE_CLOSURE_COMPILER_ERRORS=1)
+else()
+  LIST(APPEND EMCC_ARGS
+    -s NO_DYNAMIC_EXECUTION=1)
+endif()
+
+set(EMCC_JS_ARGS ${EMCC_ARGS}
+  -s AGGRESSIVE_VARIABLE_ELIMINATION=1
+  -s ELIMINATE_DUPLICATE_FUNCTIONS=1
+  -s LEGACY_VM_SUPPORT=1
+  -s SINGLE_FILE=1
+  -s WASM=0)
+
+set(EMCC_WASM_ARGS ${EMCC_ARGS}
+  -s BINARYEN_IGNORE_IMPLICIT_TRAPS=1
+  -s WASM=1)
+
+set(EMCC_GLUE_ARGS
+  -c
+  -I${BULLET_SRC_DIR}
+  -include${AMMO_HEADER_FILE})
+
+
+#######################################
+project("ammo")
+add_subdirectory(bullet EXCLUDE_FROM_ALL)
+
+
+#######################################
+add_custom_command(
+  OUTPUT glue.cpp glue.js
+  BYPRODUCTS parser.out WebIDLGrammar.pkl
+  COMMAND ${PYTHON} ${WEBIDL_BINDER_SCRIPT} ${AMMO_IDL_FILE} glue
+  DEPENDS ${AMMO_IDL_FILE}
+  COMMENT "Generating ammo bindings"
+  VERBATIM)
+add_custom_command(
+  OUTPUT glue.o
+  COMMAND emcc glue.cpp ${EMCC_GLUE_ARGS} -o glue.o
+  DEPENDS glue.cpp ${AMMO_HEADER_FILE}
+  COMMENT "Building ammo bindings"
+  VERBATIM)
+add_custom_target(ammo-bindings ALL DEPENDS glue.js glue.o)
+
+
+#######################################
+add_custom_command(
+  OUTPUT ammo.js
+  # em++ (not emcc): the final link pulls in libBulletCollision/Dynamics/etc, which are C++ and
+  # need operator new/delete resolved from libc++ - modern emscripten's emcc no longer defaults
+  # to C++ linkage the way it used to, and fails with "undefined symbol: operator new" otherwise.
+  COMMAND em++ glue.o ${BULLET_LIBS} ${EMCC_JS_ARGS} -o ammo.js
+  DEPENDS ammo-bindings ${AMMO_FRONT_MATTER_FILE} ${AMMO_ONLOAD_FILE} ${BULLET_TARGETS}
+  COMMENT "Building ammo javascript"
+  VERBATIM)
+add_custom_target(ammo-javascript ALL DEPENDS ammo.js)
+
+
+#######################################
+add_custom_command(
+  OUTPUT ammo.wasm.js ammo.wasm.wasm
+  COMMAND em++ glue.o ${BULLET_LIBS} ${EMCC_WASM_ARGS} -o ammo.wasm.js
+  DEPENDS ammo-bindings ${AMMO_FRONT_MATTER_FILE} ${AMMO_ONLOAD_FILE} ${BULLET_TARGETS}
+  COMMENT "Building ammo webassembly"
+  VERBATIM)
+add_custom_target(ammo-wasm ALL DEPENDS ammo.wasm.js ammo.wasm.wasm)

package/build_gg_ammo/patch_files/Dockerfile

@@ -0,0 +1,13 @@
+# Pinned (not :latest): an unpinned emsdk silently drifts to whatever Emscripten ships that day,
+# which has broken this build before (removed -s EXTRA_EXPORTED_RUNTIME_METHODS, changed emcc's
+# default C/C++ linkage - see CMakeLists.txt). Bump deliberately, and re-run the full build to
+# confirm CMakeLists.txt's EMCC_ARGS/em++ usage still matches whatever version you bump to.
+FROM emscripten/emsdk:6.0.8
+ENV PYTHONUNBUFFERED 1
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends \
+          libgeos-dev ed \
+          automake autoconf libtool \
+    && rm -rf /var/lib/apt/lists/*
+RUN mkdir code
+WORKDIR /code