nvs2colmap 0.3.2__tar.gz → 0.4.1__tar.gz

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nvs2colmap
-Version: 0.3.2
+Version: 0.4.1
 Summary: Utilities for converting novel view synthesis datasets to COLMAP format.
 Author-email: Howard Yin <yindaheng98@gmail.com>
 Maintainer-email: Howard Yin <yindaheng98@gmail.com>
@@ -31,6 +31,8 @@ Dynamic: license-file
 
 Utilities for converting novel view synthesis datasets to COLMAP format.
 
+**These scenes are prepared for [InstantSplat](https://github.com/yindaheng98/InstantSplat). InstantSplat initializes one frame at a time and does not undistort a whole sequence, so this repository undistorts every frame. The default commands write known-pose `sparse/0` text models and do not call COLMAP. `--use-colmap` runs feature extraction, matching, triangulation, mapping, `image_undistorter`, and `mask_undistorter` on every frame. Undistorted images go to `images/`, and undistorted masks go to `image_masks/`.**
+
 ## Supported Formats
 
 - **Neural 3D Video Dataset**: scenes with `poses_bounds.npy` and one `mp4`
@@ -65,11 +67,13 @@ pip install numpy plyfile torch
 ```
 
 For Neural 3D Video scenes, the command also needs `ffmpeg` and `ffprobe` on
-`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. If you want to run
-the full COLMAP pipeline, also provide a COLMAP executable via
-`--colmap-executable`.
+`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. To run the COLMAP
+reconstruction that
+[InstantSplat](https://github.com/yindaheng98/InstantSplat) continues from,
+also provide a COLMAP executable via `--colmap-executable`.
 
-Extract a Neural 3D Video scene and write per-frame COLMAP text models:
+Extract a Neural 3D Video scene and write per-frame COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -90,7 +94,8 @@ python -m nvs2colmap.n3dv \
   --n-frames 300
 ```
 
-Run the full COLMAP pipeline for each frame:
+Run COLMAP reconstruction and undistortion for each frame, for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -105,7 +110,8 @@ python -m nvs2colmap.n3dv \
 ```
 
 Link a Dynamic 3D Gaussians scene, including its masks, and write per-frame
-COLMAP text models:
+COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.dynamic3dgs \
@@ -116,16 +122,19 @@ python -m nvs2colmap.dynamic3dgs \
 For Neural 3D Video scenes, decoded frames are written to `frame*/images` by
 default, and the command also writes `frame*/sparse/0` text models. With
 `--use-colmap`, decoded frames are written to `frame*/input`, and each frame
-additionally gets the standard COLMAP outputs such as `distorted/`, `images/`,
-`sparse/`, and `stereo/`. When `--start-number N` is provided, decoding starts
-from source video frame `N`, and the generated folders/images are also numbered
-from `N`.
+gets `distorted/sparse/0` plus undistorted `images/`, `sparse/`, and `stereo/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). When
+`--start-number N` is provided, decoding starts from source video frame `N`,
+and the generated folders/images are also numbered from `N`.
 
 For Dynamic 3D Gaussians scenes, images from both `train_meta.json` and
 `test_meta.json` are hardlinked into `frame*/images`, and included masks are
 hardlinked into `frame*/image_masks` (`cam01.jpg` pairs with `cam01.jpg.png`).
 `--no-train-camera` and `--no-test-camera` drop one split. The same
-`--use-colmap` switch writes images to `frame*/input`, masks to
-`frame*/input_mask`, and runs COLMAP.
-`--start-number` uses the same 1-based output numbering; source file
-`000000.jpg` is frame `1`.
+`--use-colmap` switch writes images to `frame*/input` and masks to
+`frame*/input_mask`, then undistorts them to `images/` and `image_masks/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). That switch
+does not write `init_pt_cld.npz` into `sparse/0/points3D.ply`; the
+reconstructed points stay in `distorted/sparse/0` and the undistorted
+`sparse/`. `--start-number` uses the same 1-based output numbering; source
+file `000000.jpg` is frame `1`.

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/README.md

@@ -2,6 +2,8 @@
 
 Utilities for converting novel view synthesis datasets to COLMAP format.
 
+**These scenes are prepared for [InstantSplat](https://github.com/yindaheng98/InstantSplat). InstantSplat initializes one frame at a time and does not undistort a whole sequence, so this repository undistorts every frame. The default commands write known-pose `sparse/0` text models and do not call COLMAP. `--use-colmap` runs feature extraction, matching, triangulation, mapping, `image_undistorter`, and `mask_undistorter` on every frame. Undistorted images go to `images/`, and undistorted masks go to `image_masks/`.**
+
 ## Supported Formats
 
 - **Neural 3D Video Dataset**: scenes with `poses_bounds.npy` and one `mp4`
@@ -36,11 +38,13 @@ pip install numpy plyfile torch
 ```
 
 For Neural 3D Video scenes, the command also needs `ffmpeg` and `ffprobe` on
-`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. If you want to run
-the full COLMAP pipeline, also provide a COLMAP executable via
-`--colmap-executable`.
+`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. To run the COLMAP
+reconstruction that
+[InstantSplat](https://github.com/yindaheng98/InstantSplat) continues from,
+also provide a COLMAP executable via `--colmap-executable`.
 
-Extract a Neural 3D Video scene and write per-frame COLMAP text models:
+Extract a Neural 3D Video scene and write per-frame COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -61,7 +65,8 @@ python -m nvs2colmap.n3dv \
   --n-frames 300
 ```
 
-Run the full COLMAP pipeline for each frame:
+Run COLMAP reconstruction and undistortion for each frame, for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -76,7 +81,8 @@ python -m nvs2colmap.n3dv \
 ```
 
 Link a Dynamic 3D Gaussians scene, including its masks, and write per-frame
-COLMAP text models:
+COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.dynamic3dgs \
@@ -87,16 +93,19 @@ python -m nvs2colmap.dynamic3dgs \
 For Neural 3D Video scenes, decoded frames are written to `frame*/images` by
 default, and the command also writes `frame*/sparse/0` text models. With
 `--use-colmap`, decoded frames are written to `frame*/input`, and each frame
-additionally gets the standard COLMAP outputs such as `distorted/`, `images/`,
-`sparse/`, and `stereo/`. When `--start-number N` is provided, decoding starts
-from source video frame `N`, and the generated folders/images are also numbered
-from `N`.
+gets `distorted/sparse/0` plus undistorted `images/`, `sparse/`, and `stereo/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). When
+`--start-number N` is provided, decoding starts from source video frame `N`,
+and the generated folders/images are also numbered from `N`.
 
 For Dynamic 3D Gaussians scenes, images from both `train_meta.json` and
 `test_meta.json` are hardlinked into `frame*/images`, and included masks are
 hardlinked into `frame*/image_masks` (`cam01.jpg` pairs with `cam01.jpg.png`).
 `--no-train-camera` and `--no-test-camera` drop one split. The same
-`--use-colmap` switch writes images to `frame*/input`, masks to
-`frame*/input_mask`, and runs COLMAP.
-`--start-number` uses the same 1-based output numbering; source file
-`000000.jpg` is frame `1`.
+`--use-colmap` switch writes images to `frame*/input` and masks to
+`frame*/input_mask`, then undistorts them to `images/` and `image_masks/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). That switch
+does not write `init_pt_cld.npz` into `sparse/0/points3D.ply`; the
+reconstructed points stay in `distorted/sparse/0` and the undistorted
+`sparse/`. `--start-number` uses the same 1-based output numbering; source
+file `000000.jpg` is frame `1`.

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap/colmap.py

@@ -12,6 +12,7 @@ from nvs2colmap.utils.colmap import (
     feature_extractor,
     image_undistorter,
     mapper,
+    mask_undistorter,
     point_triangulator,
     read_db,
 )
@@ -39,6 +40,7 @@ def run_colmap(
     image_extension: str = ".png",
     colmap_executable: str = "colmap",
     use_gpu: str = "1",
+    has_mask: bool = False,
 ) -> None:
     folder = Path(folder)
     colmap_cameras, colmap_images = build_colmap_records(cameras, image_extension)
@@ -68,6 +70,10 @@ def run_colmap(
     # To fit sparse init in instantsplat
     if image_undistorter(str(folder), colmap_executable=colmap_executable) != 0:
         raise RuntimeError("Image undistortion failed")
+    if has_mask:
+        image_names = [f"{camera.name}{image_extension}" for camera in cameras]
+        if mask_undistorter(folder, image_names, colmap_executable=colmap_executable) != 0:
+            raise RuntimeError("Mask undistortion failed")
 
 
 def run_video_colmap(
@@ -78,6 +84,7 @@ def run_video_colmap(
     image_extension: str = ".png",
     colmap_executable: str = "colmap",
     use_gpu: str = "1",
+    has_mask: bool = False,
 ) -> None:
     output_pattern = str(output_pattern)
     for frame in range(start_number, start_number + n_frames):
@@ -87,4 +94,5 @@ def run_video_colmap(
             image_extension=image_extension,
             colmap_executable=colmap_executable,
             use_gpu=use_gpu,
+            has_mask=has_mask,
         )

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap/dynamic3dgs/__main__.py

@@ -57,7 +57,7 @@ def parse_args() -> argparse.Namespace:
         action="store_true",
         help=(
             "Run COLMAP feature extraction, matching, triangulation, mapping, "
-            "and undistortion instead of only writing sparse/0 text models."
+            "and undistortion of images and masks instead of only writing sparse/0 text models."
         ),
     )
     parser.add_argument(
@@ -115,6 +115,12 @@ def main() -> None:
             start_number=args.start_number,
             image_extension="",
         )
+        write_init_point_cloud(
+            npz_path=folder / "init_pt_cld.npz",
+            output_pattern=frame_output_pattern / "sparse" / "0" / "points3D.ply",
+            n_frames=n_frames,
+            start_number=args.start_number,
+        )
     else:
         run_video_colmap(
             output_pattern=frame_output_pattern,
@@ -124,15 +130,9 @@ def main() -> None:
             image_extension="",
             colmap_executable=args.colmap_executable,
             use_gpu=args.colmap_use_gpu,
+            has_mask=True,
         )
 
-    write_init_point_cloud(
-        npz_path=folder / "init_pt_cld.npz",
-        output_pattern=frame_output_pattern / "sparse" / "0" / "points3D.ply",
-        n_frames=n_frames,
-        start_number=args.start_number,
-    )
-
     print(f"Done: {folder}")
 
 

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap/n3dv/__main__.py

@@ -74,7 +74,7 @@ def parse_args() -> argparse.Namespace:
         action="store_true",
         help=(
             "Run COLMAP feature extraction, matching, triangulation, mapping, "
-            "and undistortion instead of only writing sparse/0 text models."
+            "and image undistortion instead of only writing sparse/0 text models."
         ),
     )
     parser.add_argument(

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap/utils/__init__.py

@@ -6,6 +6,7 @@ from .colmap import (
     feature_extractor,
     image_undistorter,
     mapper,
+    mask_undistorter,
     model_converter_bin,
     model_converter_txt,
     point_triangulator,
@@ -23,6 +24,7 @@ __all__ = [
     "feature_extractor",
     "image_undistorter",
     "mapper",
+    "mask_undistorter",
     "matrix_to_quaternion",
     "model_converter_bin",
     "model_converter_txt",

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap/utils/colmap.py

@@ -1,8 +1,10 @@
 """COLMAP command helpers."""
 
+import os
+import shutil
 import sqlite3
 import subprocess
-import os
+from pathlib import Path
 
 
 def execute(cmd):
@@ -62,7 +64,6 @@ def mapper(folder, mapper_input_path, colmap_executable="colmap"):
         colmap_executable, "mapper",
         "--database_path", os.path.join(folder, "distorted", "database.db"),
         "--image_path", os.path.join(folder, "input"),
-        "--Mapper.ba_global_function_tolerance=0.000001",
         "--input_path", mapper_input_path,
         "--output_path", os.path.join(folder, "distorted", "sparse", "0")
     ]
@@ -102,3 +103,46 @@ def image_undistorter(folder, colmap_executable="colmap"):
         "--output_type=COLMAP",
     ]
     return execute(cmd)
+
+
+def mask_undistorter(folder, image_names, colmap_executable="colmap"):
+    folder = Path(folder)
+    image_names = [Path(name) for name in image_names]
+    tmp_mask = folder / "tmp_mask"
+    shutil.rmtree(tmp_mask, ignore_errors=True)
+    linked = False
+    for image_name in image_names:
+        src = folder / "input_mask" / image_name.with_name(image_name.name + ".png")
+        if not src.is_file():
+            continue
+        dst = tmp_mask / image_name
+        dst.parent.mkdir(parents=True, exist_ok=True)
+        os.link(src, dst)
+        linked = True
+    if not linked:
+        shutil.rmtree(tmp_mask, ignore_errors=True)
+        return 0
+    tmp_sparse = folder / "tmp_mask_sparse"
+    shutil.rmtree(tmp_sparse, ignore_errors=True)
+    ret = execute([
+        colmap_executable, "image_undistorter",
+        "--image_path", os.fspath(tmp_mask),
+        "--input_path", os.fspath(folder / "distorted" / "sparse" / "0"),
+        "--output_path", os.fspath(tmp_sparse),
+        "--output_type=COLMAP",
+    ])
+    shutil.rmtree(tmp_mask, ignore_errors=True)
+    if ret != 0:
+        shutil.rmtree(tmp_sparse, ignore_errors=True)
+        return ret
+    for image_name in image_names:
+        src = tmp_sparse / "images" / image_name
+        if not src.is_file():
+            continue
+        dst = folder / "image_masks" / image_name.with_name(image_name.name + ".png")
+        dst.parent.mkdir(parents=True, exist_ok=True)
+        if dst.exists():
+            dst.unlink()
+        os.link(src, dst)
+    shutil.rmtree(tmp_sparse, ignore_errors=True)
+    return 0

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/nvs2colmap.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nvs2colmap
-Version: 0.3.2
+Version: 0.4.1
 Summary: Utilities for converting novel view synthesis datasets to COLMAP format.
 Author-email: Howard Yin <yindaheng98@gmail.com>
 Maintainer-email: Howard Yin <yindaheng98@gmail.com>
@@ -31,6 +31,8 @@ Dynamic: license-file
 
 Utilities for converting novel view synthesis datasets to COLMAP format.
 
+**These scenes are prepared for [InstantSplat](https://github.com/yindaheng98/InstantSplat). InstantSplat initializes one frame at a time and does not undistort a whole sequence, so this repository undistorts every frame. The default commands write known-pose `sparse/0` text models and do not call COLMAP. `--use-colmap` runs feature extraction, matching, triangulation, mapping, `image_undistorter`, and `mask_undistorter` on every frame. Undistorted images go to `images/`, and undistorted masks go to `image_masks/`.**
+
 ## Supported Formats
 
 - **Neural 3D Video Dataset**: scenes with `poses_bounds.npy` and one `mp4`
@@ -65,11 +67,13 @@ pip install numpy plyfile torch
 ```
 
 For Neural 3D Video scenes, the command also needs `ffmpeg` and `ffprobe` on
-`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. If you want to run
-the full COLMAP pipeline, also provide a COLMAP executable via
-`--colmap-executable`.
+`PATH`, or explicit paths via `--ffmpeg` and `--ffprobe`. To run the COLMAP
+reconstruction that
+[InstantSplat](https://github.com/yindaheng98/InstantSplat) continues from,
+also provide a COLMAP executable via `--colmap-executable`.
 
-Extract a Neural 3D Video scene and write per-frame COLMAP text models:
+Extract a Neural 3D Video scene and write per-frame COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -90,7 +94,8 @@ python -m nvs2colmap.n3dv \
   --n-frames 300
 ```
 
-Run the full COLMAP pipeline for each frame:
+Run COLMAP reconstruction and undistortion for each frame, for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.n3dv \
@@ -105,7 +110,8 @@ python -m nvs2colmap.n3dv \
 ```
 
 Link a Dynamic 3D Gaussians scene, including its masks, and write per-frame
-COLMAP text models:
+COLMAP text models for
+[InstantSplat](https://github.com/yindaheng98/InstantSplat):
 
 ```bash
 python -m nvs2colmap.dynamic3dgs \
@@ -116,16 +122,19 @@ python -m nvs2colmap.dynamic3dgs \
 For Neural 3D Video scenes, decoded frames are written to `frame*/images` by
 default, and the command also writes `frame*/sparse/0` text models. With
 `--use-colmap`, decoded frames are written to `frame*/input`, and each frame
-additionally gets the standard COLMAP outputs such as `distorted/`, `images/`,
-`sparse/`, and `stereo/`. When `--start-number N` is provided, decoding starts
-from source video frame `N`, and the generated folders/images are also numbered
-from `N`.
+gets `distorted/sparse/0` plus undistorted `images/`, `sparse/`, and `stereo/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). When
+`--start-number N` is provided, decoding starts from source video frame `N`,
+and the generated folders/images are also numbered from `N`.
 
 For Dynamic 3D Gaussians scenes, images from both `train_meta.json` and
 `test_meta.json` are hardlinked into `frame*/images`, and included masks are
 hardlinked into `frame*/image_masks` (`cam01.jpg` pairs with `cam01.jpg.png`).
 `--no-train-camera` and `--no-test-camera` drop one split. The same
-`--use-colmap` switch writes images to `frame*/input`, masks to
-`frame*/input_mask`, and runs COLMAP.
-`--start-number` uses the same 1-based output numbering; source file
-`000000.jpg` is frame `1`.
+`--use-colmap` switch writes images to `frame*/input` and masks to
+`frame*/input_mask`, then undistorts them to `images/` and `image_masks/`
+for [InstantSplat](https://github.com/yindaheng98/InstantSplat). That switch
+does not write `init_pt_cld.npz` into `sparse/0/points3D.ply`; the
+reconstructed points stay in `distorted/sparse/0` and the undistorted
+`sparse/`. `--start-number` uses the same 1-based output numbering; source
+file `000000.jpg` is frame `1`.

{nvs2colmap-0.3.2 → nvs2colmap-0.4.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "nvs2colmap"
-version = "0.3.2"
+version = "0.4.1"
 description = "Utilities for converting novel view synthesis datasets to COLMAP format."
 readme = "README.md"
 authors = [{ name = "Howard Yin", email = "yindaheng98@gmail.com" }]