Dispersed Structured Light for Hyperspectral 3D Imaging

1POSTECH, 2Princeton University
CVPR 2024
Teaser.

We introduce dispersed structured light, a low-cost high-quality hyperspectral 3D imaging method. By placing a diffraction grating film on a conventional camera-projector setup, we disperse structured-light patterns. Analyzing the images captured under the dispersed structured light enables accurate hyperspectral 3D reconstruction. (a) Capture configuration, (b) estimated hyperspectral image in sRGB, (c) comparison with spectroradiometer measurements, (d) estimated depth map, (e) estimated hyperspectral image.

Abstract

Hyperspectral 3D imaging aims to acquire both depth and spectral information of a scene. However, existing methods are either prohibitively expensive and bulky or compromise on spectral and depth accuracy. In this paper, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL modifies a traditional projector-camera system by placing a sub-millimeter thick diffraction grating film in front of the projector. This configuration enables dispersing structured light based on light wavelength. To utilize the dispersed structured light, we devise a model for dispersive projection image formation and a per-pixel hyperspectral 3D reconstruction method. We validate DSL by instantiating a compact experimental prototype. DSL achieves spectral accuracy of 18.8 nm full-width half-maximum (FWHM) and depth error of 1 mm, outperforming prior work on practical hyperspectral 3D imaging. DSL promises accurate and practical hyperspectral 3D imaging for diverse application domains, including computer vision and graphics, cultural heritage, geology, and biology.

Imaging System

We devise the proposed imaging system with the goals of compactness and affordability. We combine a conventional trichromatic camera with a trichromatic projector and a diffraction-grating film mounted in front. This configuration makes light from the projector undergo dispersion, and, as such, patterns emitted from the projector are spatially dispersed depending on wavelength.

Network architecture

Our prototype consists of an RGB projector equipped with a diffraction grating film, and an RGB camera, as shown in the left and middle images. The right image shows an example projector pattern and its corresponding captured image, exhibiting clear first-order diffraction.

Hyperspectral 3D Imaging using Dispersed Structured Light

White scan line pattern for hyperspectral imaging and dispersed structured light for depth imaging.

White Scanline Pattern

Dispersed Structured Light

Hyperspectral intensity of color checker

Network architecture

Hyperspectral 3D Imaging Results

White Scanline Pattern

Dispersed Structured Light

Reconstructed Results

White Scanline Pattern

Dispersed Structured Light

Reconstructed Results

White Scanline Pattern

Dispersed Structured Light

Reconstructed Results

DSL Presentation

BibTeX


      @inproceedings{shin2024dispersed,
        title={Dispersed Structured Light for Hyperspectral 3D Imaging},
        author={Shin, Suhyun and Choi, Seokjun and Heide, Felix and Baek, Seung-Hwan},
        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
        year={2024}
      }