TWINGS: Thin Plate Splines Warp-aligned Initialization for Sparse-View Gaussian Splatting

Novel view synthesis from sparse-view inputs poses a significant challenge in 3D computer vision, particularly for achieving high-quality scene reconstructions with limited viewpoints. We introduce TWINGS, a framework that enhances 3D Gaussian Splatting (3DGS) by directly addressing point sparsity. We employ Thin Plate Splines (TPS), a smooth non-rigid deformation model that minimizes bending energy to estimate a globally coherent warp from control-point correspondences, to align backprojected points from estimated depth with triangulated 3D control points, yielding calibrated backprojected points. By sampling these calibrated points near the control points, TWINGS provides a fast and geometrically accurate initialization for 3DGS, ultimately improving structural detail preservation and color fidelity in reconstructed scenes. Extensive experiments on DTU, LLFF, and Mip-NeRF360 demonstrate that TWINGS consistently outperforms existing methods, delivering detailed and accurate reconstructions under sparse-view scenarios.

Visualization of 3D reconstructed points. Highly accurate 3D reconstructed points are required to constrain 3DGS.

TWINGS Pipeline. Our method consists of three key components: Multi-view Correspondences: We establish multi-view correspondences among query and key images. Using correspondences with known camera parameters, we reconstruct 3D points that correspond to desired control points (pink). By backprojecting the estimated depth, we generate backprojected points (green). TPS deformation: We define a TPS model that deforms the backprojected points to obtain TPS deformed points, which are referred to as CBP. CBPS: The CBP are then strategically sampled near desired control points (pink) to initialize 3DGS training.

Qualitative results on the DTU dataset. Novel view synthesis results rendered by CoR-GS, DNGaussian, DropGaussian, our approach, and ground truth for comparison.

Qualitative results on the LLFF dataset. Novel view synthesis results rendered by CoR-GS, DropGaussian, CoMapGS, our approach, and ground truth for comparison.

Qualitative results on the LLFF dataset. Novel view synthesis results rendered by CoR-GS, DropGaussian, CoMapGS, our approach, and ground truth for comparison.