[Geometry] Endoscopic scene reconstruction

Reference

Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

A Survey on 3D Gaussian Splatting

Endoscopic scene reconstruction의 개요는 A Survey on 3D Gaussian Splatting를 참조하기 바랍니다.

---

Deform3DGS official github

StereoMIS dataset을 사용할 때 EndoNeRF와 똑같은 depth, masks, images, intrinsics, extrinsics 파라미터를 얻습니다.

stereo depth는 specular & textureless surface에서 robust한 optical flow 기반 stereo depth estimation model인 RAFT를 사용하여 얻습니다.

EndoNeRF와 마찬가지로 StereoMIS 데이터도 fixed-view settings로 합니다. (single-view point video reconstructios)

Datasets

To use the StereoMIS dataset, please follow this github repo to preprocess the dataset.

After that, run the provided script stereomis2endonerf.py to extract clips from the StereoMIS dataset and organize the depth, masks, images, intrinsic and extrinsic parameters in the same format as EndoNeRF.

In our implementation, we used RAFT to estimate the stereo depth for StereoMIS clips.

Following EndoNeRF dataset, this script only supports fixed-view settings.