Back to my home page.
Panorama Viewer for Generic Camera Models
Interactive Image Segmentation Based On GPU Cellular Automata
Frank Nielsen
Abstract Panorama Viewer:
Digital panoramas are nowadays omnipresent in Internet virtual tours (see http://www.world-heritage-tour.org/). A panorama basically stores light information arriving at a single position from a wide field of view. Panoramas are particular light fields that conveniently sample the plenoptic function in image-based rendering systems at discrete positions. We distinguish between spherical panoramas that cover the full sphere of directions ( steradians) from cylindrical panoramas that cover only 360 horizontal degrees over a partial vertical field of view. Panoramas are not only widely used in computer graphics as backdrops (also called skyboxes), but also as environment maps for dynamic reflections and more recently as light probes for the rendering of@convincing lighting. In the old days, environment maps were merely captured using a tele-lens camera (approximating the required orthographic projection) capturing the reflections of a spherical mirror ball [Nielsen05a]. Nowadays, stitching allows us to calibrate and register precisely a sequence of pictures acquired from a same nodal point: the center of projection (COP) [Nielsen05b]. Thus an environment map is a complete panorama raymap obtained from a single COP, while a pinhole image is interpreted as a local raymap sampling partially the environment map from the same COP. The most common environment maps used for simulating real-time reflections in computer graphics are the latitude-longitude map (also called equirectangular map), the cubic map (6 quad faces), and the dual paraboloid map (2 images) [Nielsen05a]. Wong et al. further introduced the HEALPix map (12 quads) [Wong05] that improves the spherical ray sampling distribution over the cube map for computing real-time reflections on the GPU.
Video of real-time GPU panorama viewer (2 MB)
Abstract GPU Segmentation:
Image segmentation consists in grouping pixels into homogeneous regions representing perceptual units using local and/or global cues. Although the very first segmentation algorithms were introduced in the early 1970s, segmentation is still nowadays a hot topic of computer vision. Segmentation can be handled at different image understanding scales [Nielsen05]: (1) low-level segmentation uses local pixel neighborhood cues such as color or texture information to infer the global image partition, (2) middle-level segmentation considers elements of the Gestalt theory such as symmetry detection and rules thereof to improve the pixel grouping, and (3) high-level segmentation that relies on (re)cognition to improve the overall global segmentation (e.g., recognizing first categories of objects let us refine later their segmentations). Foreground-background segmentation is a simplified segmentation task that seeks to decompose the image into two planes: the foreground mask plane and the background mask plane. Finally, we distinguish between hard and soft segmentation depending on whether masks have only 0/1 binary values or potentially float values (by analogy to hard/soft clustering). Segmentation algorithms are well-known to be time consuming that may work or simply fail depending on input images. Thus it is crucial to allow for a rectification mechanism by letting the user steer the segmentation using prior cues [Nock05]. To reduce segmentation computation times, GPU has already been successfully used in the past for fully automatic segmentation, using for example level sets [Lefohn05]. Here, we present a GPU shader implementation for a recent simple yet efficient cellular automata-based foreground/background hard segmentation: GrowCut [Vezhnevets05].
Video of real-time GPU segmentation (2 MB)
BibTex entry:
@incollection{bc-panoramagpu-2007,
author="Frank Nielsen",
title="Panorama Viewer for Generic Camera Models",
booktitle="ShaderX$^5$: Advanced Rendering Techniques",
pages = {543-552}
year= 2007,
publisher = "Charles River Media (\url{http://www.charlesriver.com/})"
}
@incollection{bc-segmentationgpu-2007,
author="Frank Nielsen",
title="Interactive Image Segmentation Based On GPU Cellular Automata",
booktitle="ShaderX$^5$: Advanced Rendering Techniques",
year= 2007,
pages={511-518},
publisher = "Charles River Media (\url{http://www.charlesriver.com/})",
}
Last updated, February 2007.