Distilled-3DGS: Distilled 3D Gaussian Splatting

Lintao Xiang*1, 2, Xinkai Chen*2, Jianhuang Lai3, Guangcong Wang2
1The University of Manchester, 2Vision, Graphics, and X Group, Great Bay University,
3Sun Yat-Sen University
Arxiv 2025
* Indicates Equal Contribution
Paper arXiv Code

TL;DR: Our Distilled-3DGS distills large teacher 3DGS models' knowledge to a small student 3DGS model. On average, Distilled-3DGS requires only 12% of the Gaussians used in the original 3DGS to reconstruct scenes while achieving rendering quality comparable to or surpassing SOTA baselines.

3DGS vs. Distilled-3DGS (Ours) on Bicycle Scene of Mip-NeRF 360 Dataset

Abstract

3D Gaussian Splatting (3DGS) has exhibited remarkable efficacy in novel view synthesis (NVS). However, it suffers from a significant drawback: achieving high-fidelity rendering typically necessitates a large number of 3D Gaussians, resulting in substantial memory consumption and storage requirements. To address this challenge, we propose the first knowledge distillation framework, Distilled-3DGS, for 3DGS, featuring various teacher models, including vanilla 3DGS, noise-augmented variants, and dropout-regularized versions. The outputs of these teachers are aggregated to guide the optimization of a lightweight student model. To distill the hidden geometric structure, we propose a structural similarity loss to boost the consistency of spatial geometric distributions between the student and teacher model. Through comprehensive quantitative and qualitative evaluations across diverse datasets, the proposed Distilled-3DGS—a simple yet effective framework without bells and whistles—achieves promising rendering results in both rendering quality and storage efficiency compared to state-of-the-art methods.

Motivation

3DGS necessitates a large number of 3D Gaussians to ensure high-fidelity image rendering, particularly in the presence of complex scenes. This limits their applicability on platforms and devices with constrained computational resources and limited memory capacity. On the other hand, knowledge distillation has proven highly effective for model compression. We employ it to transfer knowledge from various teacher models with more Gaussians to a student model with fewer Gaussians, thereby reducing the total number of Gaussians in the scene representation.

Demo

Method

Multi-teacher knowledge distillation framework of Distilled-3DGS consists of two stages. First, a standard teacher model Gstd is trained, along with two variants: Gperb with random perturbation and Gdrop with random dropout. Then, a pruned student model Gstd is supervised by the outputs of these teachers. Additionally, a spatial distribution distillation strategy is introduced to help the student learn structural patterns from the teachers.

Citation

@article{Xiang2025Distilled3DGaussianSplatting,
    title={Distilled-3DGS: Distilled 3D Gaussian Splatting},
    author={Lintao Xiang and Xinkai Chen and Jianhuang Lai and Guangcong Wang},
    journal={arxiv},
    year={2025}}