Hyperspectral 3D imaging aims to acquire both depth and spectral information of a scene. However, existing methods are either prohibitively expensive and bulky or compromise on spectral and depth accuracy. In this paper, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL modifies a traditional projector-camera system by placing a sub-millimeter thick diffraction grating film in front of the projector. This configuration enables dispersing structured light based on light wavelength. To utilize the dispersed structured light, we devise a model for dispersive projection image formation and a per-pixel hyperspectral 3D reconstruction method. We validate DSL by instantiating a compact experimental prototype. DSL achieves spectral accuracy of 18.8 nm full-width half-maximum (FWHM) and depth error of 1 mm, outperforming prior work on practical hyperspectral 3D imaging. DSL promises accurate and practical hyperspectral 3D imaging for diverse application domains, including computer vision and graphics, cultural heritage, geology, and biology.
We devise the proposed imaging system with the goals of compactness and affordability. We combine a conventional trichromatic camera with a trichromatic projector and a diffraction-grating film mounted in front. This configuration makes light from the projector undergo dispersion, and, as such, patterns emitted from the projector are spatially dispersed depending on wavelength.
Our prototype consists of an RGB projector equipped with a diffraction grating film, and an RGB camera, as shown in the left and middle images. The right image shows an example projector pattern and its corresponding captured image, exhibiting clear first-order diffraction.
White scan line pattern for hyperspectral imaging and dispersed structured light for depth imaging.
@inproceedings{shin2024dispersed,
title={Dispersed Structured Light for Hyperspectral 3D Imaging},
author={Shin, Suhyun and Choi, Seokjun and Heide, Felix and Baek, Seung-Hwan},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2024}
}