Most cameras are inherently designed to mimic what is seen by the human eye: they have three channels of RGB and can achieve up to around 30 frames per second (FPS).
However, some cameras are designed to capture other modalities: some may have the ability to capture spectra from near UV to near IR rather than RGB, polarimetry, different times of light travel, etc. Such modalities are as yet unknown, but they can also collect robust data of the scene they are capturing.
This book will focus on the emerging computer vision techniques known as computational imaging. These include capturing, processing and analyzing such modalities for various applications of scene understanding.
Introduction xiii Takuya FUNATOMI and Takahiro OKABE Part 1 Transient Imaging and Processing 1 Chapter 1 Transient Imaging 3 Adrian JARABO 1.1 Introduction 3 1.2.Mathematical formulation 5 1.2.1 Analysis of transient light transport propagation 7 1.2.2 Sparsity of the impulse response function T (x, t) 8 1.3.Capturinglight in flight 9 1.3.1 Single-photon avalanche diodes (SPAD) 11 1.4.Applications 14 1.4.1.Range imaging 14 1.4.2.Material estimation and classification 14 1.4.3 Light transport decomposition 15 1.5.Non-line-of-sight imaging 15 1.5.1.Backprojection 17 1.5.2 Confocal NLOS and the light-cone transform 17 1.5.3 Surface-based methods 18 1.5.4.Virtualwaves and phasor fields 19 1.5.5.Discussion 21 1.6.Conclusion 22 1.7.References 22 Chapter 2 Transient Convolutional Imaging 29 Felix HEIDE 2.1 Introduction 29 2.2.Time-of-flight imaging 30 2.2.1.Correlationimage sensors 32 2.2.2 Convolutional ToF depth imaging 32 2.2.3 Multi-path interference 34 2.3.Transient convolutional imaging 35 2.3.1 Global convolutional transport 37 2.3.2 Transient imaging using correlation image sensors 37 2.3.3 Spatio-temporal modulation 40 2.4.Transient imaging in scattering media 41 2.5.Present and future directions 43 2.6.References 43 Chapter 3 Time-of-Flight and Transient Rendering 45 Adithya Kumar PEDIREDLA 3.1 Introduction 45 3.2.Mathematicalmodeling 46 3.2.1 Mathematical modeling for time-of-flight cameras 47 3.3.Howto render time-of-flight cameras? 50 3.3.1 Challenges and solutions in time-of-flight rendering 51 3.4.Open-source implementations 56 3.5.Applicationsof transient rendering 57 3.6.Future directions 61 3.7.References 62 Part 2 Spectral Imaging and Processing 69 Chapter 4 Hyperspectral Imaging 71 Nathan HAGEN 4.1 Introduction 71 4.2.2D(raster scanning) architectures 75 4.2.1.Czerny-Turnergratingspectrometers 76 4.2.2 Transmission grating/prism spectrometers 78 4.2.3.Coded aperture spectrometers 79 4.2.4.Echelle spectrometers 80 4.3.1Dscanningarchitectures 81 4.3.1.Dispersive spectrometers 82 4.3.2 Interferometric methods 83 4.3.3 Interferometric filter methods 83 4.3.4 Polarization-based filter methods 86 4.3.5 Active illumination methods 88 4.4.Snapshot architectures 88 4.4.1.Bowen-Walravenimage slicer 89 4.4.2 Image slicing and imagemapping 90 4.4.3 Integral field spectrometry with coherent fiber bundles (IFS-F) 93 4.4.4 Integral field spectroscopy with lens let arrays (IFS-L) 94 4.4.5 Filter array camera (FAC) 94 4.4.6 Computed tomography imaging spectrometry (CTIS) 96 4.4.7 Coded aperture snapshot spectral imager (CASSI) 97 4.5.Comparisonof snapshot techniques 98 4.5.1.The disadvantages of snapshot 100 4.6.Conclusion 101 4.7.References 102 Chapter 5 Spectral Modeling and Separation of Reflective-Fluorescent Scenes 109 Ying FU, Antony LAM, Imari SATO, Takahiro OKABE, and Yoichi SATO 5.1 Introduction 109 5.2.RelatedWork 111 5.3.Separationof reflection and fluorescence 113 5.3.1.Reflection and fluorescence models 113 5.3.2 Separation using high-frequency illumination 114 5.3.3 Discussion on the illumination frequency 116 5.3.4.Error analysis 118 5.4.Estimating the absorption spectra 119 5.5.Experiment results and analysis 122 5.5.1.Experimental setup 122 5.5.2 Quantitative evaluation of recovered spectra 122 5.5.3.Visual separation and relighting results 126 5.5.4 Separation by using high-frequency filters 130 5.5.5 Ambient illumination 134 5.6.Limitations and conclusion 137 5.7.References 137 Chapter 6 Shape from Water 141 Yuta ASANO, Yinqiang ZHANG, Ko NISHINO, and Imari SATO 6.1 Introduction 141 6.2.Relatedworks 143 6.3.Light absorption in water 145 6.4 Bispectral light absorption for depth recovery 146 6.4.1.Bispectral depth imaging 146 6.4.2 Depth accuracy and surface reflectance 147 6.5.Practical shape from water 148 6.5.1 Non-collinear/perpendicular light-camera configuration 148 6.5.2 Perspective camera with a point source 150 6.5.3.Non-idealnarrow-bandfilters 151 6.6 Co-axial bispectral imaging system and experiment results 151 6.6.1.Systemconfigurationand calibration 151 6.6.2 Depth and shape accuracy 152 6.6.3 Complex static and dynamic objects 154 6.7 Trispectral light absorption for depth recovery 155 6.7.1.Trispectraldepthimaging 156 6.7.2 Evaluation on the reflectance spectra database 157 6.8.Discussions 157 6.9.Conclusion 158 6.10.References 158 Chapter 7 Far Infrared Light Transport Decomposition and Its Application for Thermal Photometric Stereo 161 Kenichiro TANAKA 7.1 Introduction 161 7.1.1.Contributions 162 7.2.Relatedwork 163 7.2.1 Light transport decomposition 163 7.2.2.Computational thermal imaging 164 7.2.3.Photometricstereo 165 7.3.Far infrared light transport 165 7.4 Decomposition and application 171 7.4.1 Far infrared light transport decomposition 171 7.4.2 Separating the ambient component 172 7.4.3.Separatingreflectionand radiation 172 7.4.4 Separating diffuse and global radiations 172 7.4.5.Other options 173 7.4.6 Thermal photometric stereo 173 7.5.Experiments 174 7.5.1 Decomposition result 175 7.5.2.Surfacenormal estimation 177 7.6.Conclusion 179 7.7.References 180 Chapter 8 Synthetic Wavelength Imaging: Utilizing Spectral Correlations for High-Precision Time-of-Flight Sensing 187 Florian WILLOMITZER 8.1 Introduction 187 8.2.Syntheticwavelengthimaging 189 8.3.Synthetic wavelength interferometry 193 8.4 Synthetic wavelength holography 197 8.4.1 Imaging around corners with synthetic wavelength holography 199 8.4.2 Imaging through scattering media with synthetic wavelength holography 200 8.4.3 Discussion and comparison with the state of the art 203 8.5 Fundamental performance limits of synthetic wavelength imaging 205 8.6.Conclusionand future directions 210 8.7.Acknowledgment 210 8.8.References 211 Part 3 Polarimetric Imaging and Processing 219 Chapter 9 Polarization-Based Shape Estimation 221 Daisuke MIYAZAKI 9.1 Fundamental theory of polarization 221 9.2 Reflection component separation 225 9.3.Phase angle of polarization 226 9.4 Surface normal estimation from the phase angle 228 9.5.Degree of polarization 233 9.6 Surface normal estimation from the degree of polarization 236 9.7.Stokes vector 236 9.8 Surface normal estimation from the Stokes vector 237 9.9.References 239 Chapter 10 Shape from Polarization and Shading 241 Thanh-Trung NGO, Hajime NAGAHARA, and Rin-ichiro TANIGUCHI 10.1 Introduction 241 10.2.Relatedworks 243 10.2.1.Shadingand polarization fusion 243 10.2.2 Shape estimation under uncalibrated light sources 244 10.3 Problem setting and assumptions 245 10.4.Shadingstereoscopic constraint 246 10.5.Polarizationstereoscopic constraint 248 10.6.Normal estimation with two constraints 249 10.6.1 Algorithm 1: Recovering individual surface points 250 10.6.2 Algorithm 2: Recovering shape and light directions 251 10.7.Experiments 252 10.7.1 Simulation experiments with weights for two constraints 253 10.7.2.Real-world experiments 254 10.8.Conclusionand future works 263 10.9.References 263 Chapter 11 Polarization Imaging in the Wild Beyond the Unpolarized World Assumption 269 Jérémy Maxime RIVIERE 11.1 Introduction 269 11.2.Mueller calculus 271 11.3.Polarizingfilters 273 11.3.1.Linear polarizers 273 11.3.2.Reflectors 274 11.4.Polarizationimaging 275 11.5 Image formation model 277 11.5.1 Partially linearly polarized incident illumination 277 11.5.2 Unpolarized incident illumination 279 11.5.3.Discussion 280 11.6.Polarization imaging reflectometry in the wild 282 11.7.DigitalSingle-Lens Reflex (DSLR) setup 283 11.7.1 Data acquisition 283 11.7.2.Calibration 285 11.7.3.Polarizationprocessingpipeline 285 11.8.Reflectance recovery 287 11.8.1.Surface normal estimation 287 11.8.2.Diffuse albedo estimation 288 11.8.3 Specular component estimation 288 11.9.Results and analysis 291 11.9.1.Results 291 11.9.2.Discussion and error analysis 293 11.10.References 296 Chapter 12 Multispectral Polarization Filter Array 299 Kazuma SHINODA 12.1 Introduction 299 12.2 Multispectral polarization filter array with a photonic crystal 302 12.3 Generalization of imaging and demosaicking with multispectral polarization filter arrays 306 12.4.Demonstration 311 12.5.Conclusion 313 12.6.References 313 List of Authors 317 Index 319
Takuya Funatomi is an associate professor in Division of Information Science at the Nara Institute of Science and Technology (NAIST) in Japan. He has received a bachelor's degree in Engineering and a master’s degree and PhD in Informatics from Kyoto University in 2002, 2004 and 2007, respectively. Takahiro Okabe is a professor at Kyushu Institute of Technology in Japan. He received a bachelor's and a master's degree in Physics and a PhD in Information Science and Technology from the University of Tokyo in 1997, 1999 and 2011, respectively.
