Patent application number | Description | Published |
20090041381 | Method and Apparatus for Radiance Processing by Demultiplexing in the Frequency Domain - Method and apparatus for radiance processing by demultiplexing in the frequency domain. A frequency domain demultiplexing module obtains a radiance image captured with a lens-based radiance camera. The image includes optically mixed spatial and angular frequency components of light from a scene. The module performs frequency domain demultiplexing on the radiance image to generate multiple parallax views of the scene. The method may extract multiple slices at different angular frequencies from a Fourier transform of the radiance image, apply a Fourier transform to each of the multiple slices to generate intermediate images, stack the intermediate images to form a 3- or 4-dimensional image, apply an inverse Fourier transform along angular dimension(s) of the 3- or 4-dimensional image, and unstack the transformed 3- or 4-dimensional image to obtain the multiple parallax views. During the method, phase correction may be performed to determine the centers of the intermediate images. | 02-12-2009 |
20090041448 | Method and Apparatus for Radiance Capture by Multiplexing in the Frequency Domain - An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene. | 02-12-2009 |
20090102956 | Fast Computational Camera Based On Two Arrays of Lenses - Method and apparatus for a fast (low F/number) computational camera that incorporates two arrays of lenses. The arrays include a lenslet array in front of a photosensor and an objective lens array of two or more lenses. Each lens in the objective lens array captures light from a subject. Each lenslet in the lenslet array captures light from each objective lens and separates the captured light to project microimages corresponding to the objective lenses on a region of the photosensor under the lenslet. Thus, a plurality of microimages are projected onto and captured by the photosensor. The captured microimages may be processed in accordance with the geometry of the objective lenses to align the microimages to generate a final image. One or more other algorithms may be applied to the image data in accordance with radiance information captured by the camera, such as automatic refocusing of an out-of-focus image. | 04-23-2009 |
20090185801 | Methods and Apparatus for Full-Resolution Light-Field Capture and Rendering - Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods. | 07-23-2009 |
20090268970 | Method and Apparatus for Block-Based Compression of Light-field Images - A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking scheme of a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the angular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image. | 10-29-2009 |
20090295829 | Methods and Apparatus for Full-Resolution Light-Field Capture and Rendering - Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to flats captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods. | 12-03-2009 |
20100020187 | PLENOPTIC CAMERA - One embodiment of the present invention provides a plenoptic camera which captures information about the direction distribution of light rays entering the camera. Like a conventional camera, this plenoptic camera includes a main lens which receives light from objects in an object field and directs the received light onto an image plane of the camera. It also includes a photodetector array located at the image plane of the camera, which captures the received light to produce an image. However, unlike a conventional camera, the plenoptic camera additionally includes an array of optical elements located between the object field and the main lens. Each optical element in this array receives light from the object field from a different angle than the other optical elements in the array, and consequently directs a different view of the object field into the main lens. In this way, the photodetector array receives a different view of the object field from each optical element in the array. | 01-28-2010 |
20110211824 | Methods and Apparatus for Full-Resolution Light-Field Capture and Rendering - Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods. | 09-01-2011 |
20110305447 | METHOD AND APPARATUS FOR RADIANCE CAPTURE BY MULTIPLEXING IN THE FREQUENCY DOMAIN - An external mask-based radiance camera may be based on an external, non-refractive mask located in front of the main camera lens. The mask modulates, but does not refract, light. The camera multiplexes radiance in the frequency domain by optically mixing different spatial and angular frequency components of light. The mask may, for example, be a mesh of opaque linear elements, which collectively form a grid, an opaque medium with transparent openings, such as circles, or a pinhole mask. Other types of masks may be used. Light may be modulated by the mask and received at the main lens of a camera. The main lens may be focused on a plane between the mask and the main lens. The received light is refracted by the main lens onto a photosensor of the camera. The photosensor may capture the received light to generate a radiance image of the scene. | 12-15-2011 |
20120177356 | Methods and Apparatus for Full-Resolution Light-Field Capture and Rendering - Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than ƒ from the photosensor, where ƒ is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to light-fields captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods. | 07-12-2012 |
20120183232 | Method and Apparatus for Block-Based Compression of Light-field Images - A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking scheme of a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the angular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image. | 07-19-2012 |
20120229679 | Methods and Apparatus for Full-Resolution Light-Field Capture and Rendering - Method and apparatus for full-resolution light-field capture and rendering. A radiance camera is described in which the microlenses in a microlens array are focused on the image plane of the main lens instead of on the main lens, as in conventional plenoptic cameras. The microlens array may be located at distances greater than f from the photosensor, where f is the focal length of the microlenses. Radiance cameras in which the distance of the microlens array from the photosensor is adjustable, and in which other characteristics of the camera are adjustable, are described. Digital and film embodiments of the radiance camera are described. A full-resolution light-field rendering method may be applied to flats captured by a radiance camera to render higher-resolution output images than are possible with conventional plenoptic cameras and rendering methods. | 09-13-2012 |
20120281072 | Focused Plenoptic Camera Employing Different Apertures or Filtering at Different Microlenses - Methods and apparatus for capturing and rendering images with focused plenoptic cameras employing different filtering at different microlenses. In a focused plenoptic camera, the main lens creates an image at the focal plane. That image is re-imaged on the sensor multiple times by an array of microlenses. Different filters that provide different levels and/or types of filtering may be combined with different ones of the microlenses. A flat captured with the camera includes multiple microimages captured according to the different filters. Multiple images may be assembled from the microimages, with each image assembled from microimages captured using a different filter. A final image may be generated by appropriately combining the images assembled from the microimages. Alternatively, a final image, or multiple images, may be assembled from the microimages by first combining the microimages and then assembling the combined microimages to produce one or more output images. | 11-08-2012 |
20130120356 | Methods, Apparatus, and Computer-Readable Storage Media for Depth-Based Rendering of Focused Plenoptic Camera Data - Methods, apparatus, and computer-readable storage media for rendering focused plenoptic camera data. A depth-based rendering technique is described that estimates depth at each microimage and then applies that depth to determine a position in the input flat from which to read a value to be assigned to a given point in the output image. The techniques may be implemented according to parallel processing technology that renders multiple points of the output image in parallel. In at least some embodiments, the parallel processing technology is graphical processing unit (GPU) technology. | 05-16-2013 |
20130120605 | Methods, Apparatus, and Computer-Readable Storage Media for Blended Rendering of Focused Plenoptic Camera Data - Methods, apparatus, and computer-readable storage media for rendering focused plenoptic camera data. A rendering with blending technique is described that blends values from positions in multiple microimages and assigns the blended value to a given point in the output image. A rendering technique that combines depth-based rendering and rendering with blending is also described. Depth-based rendering estimates depth at each microimage and then applies that depth to determine a position in the input flat from which to read a value to be assigned to a given point in the output image. The techniques may be implemented according to parallel processing technology that renders multiple points of the output image in parallel. In at least some embodiments, the parallel processing technology is graphical processing unit (GPU) technology. | 05-16-2013 |
20130121615 | Method and Apparatus for Managing Artifacts in Frequency Domain Processing of Light-Field Images - Various methods and apparatus for removing artifacts in frequency domain processing of light-field images are described. Methods for the reduction or removal of the artifacts are described that include methods that may be applied during frequency domain processing and a method that may be applied during post-processing of resultant angular views. The methods may be implemented in software as or in a light-field frequency domain processing module. The described methods include an oversampling method to determine the correct centers of slices, a phase multiplication method to determine the correct centers of slices, a method to exclude low-energy slices, and a cosmetic correction method. | 05-16-2013 |
20130127901 | Methods and Apparatus for Calibrating Focused Plenoptic Camera Data - Methods, apparatus, and computer-readable storage media for calibrating focused plenoptic camera data. A calibration technique that does not modify the image data may be applied to raw plenoptic images. Calibration parameters, including but not limited to tilt angle, corner crops, main lens distance from the microlens array, sensor distance from the microlens array, and microimage size, may be specified. Calibration may include scaling down the input texture coordinates for the plenoptic image so that the new coordinate range fits the size of the texture with crops taken into account. These coordinates may be further transformed by one or more of a matrix performing a scaling, to correct for lens distortion; a rotation, to correct for tilts; and a translation that finalizes the necessary corner crops. A transformation matrix is generated that can be applied to the raw image by radiance processing techniques such as super-resolution techniques. | 05-23-2013 |
20130128030 | Thin Plenoptic Cameras Using Solid Immersion Lenses - Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as solid immersion lens (SIL) technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques. Hemispherical SIL technology, along with multiple main lenses and a mask on the photosensor, may be employed in some thin plenoptic cameras. Other thin cameras may include a layer between hemispherical SILs and the photosensor that effectively implements superhemispherical SIL technology in the camera. | 05-23-2013 |
20130128068 | Methods and Apparatus for Rendering Focused Plenoptic Camera Data using Super-Resolved Demosaicing - A super-resolved demosaicing technique for rendering focused plenoptic camera data performs simultaneous super-resolution and demosaicing. The technique renders a high-resolution output image from a plurality of separate microimages in an input image at a specified depth of focus. For each point on an image plane of the output image, the technique determines a line of projection through the microimages in optical phase space according to the current point and angle of projection determined from the depth of focus. For each microimage, the technique applies a kernel centered at a position on the current microimage intersected by the line of projection to accumulate, from pixels at each microimage covered by the kernel at the respective position, values for each color channel weighted according to the kernel. A value for a pixel at the current point in the output image is computed from the accumulated values for the color channels. | 05-23-2013 |
20130128069 | Methods and Apparatus for Rendering Output Images with Simulated Artistic Effects from Focused Plenoptic Camera Data - Methods, apparatus, and computer-readable storage media for simulating artistic effects in images rendered from plenoptic data. An impressionistic-style artistic effect may be generated in output images of a rendering process by an “impressionist” 4D filter applied to the microimages in a flat captured with focused plenoptic camera technology. Individual pixels are randomly selected from blocks of pixels in the microimages, and only the randomly selected pixels are used to render an output image. The randomly selected pixels are rendered to generate the artistic effect, such as an “impressionistic” effect, in the output image. A rendering technique is applied that samples pixel values from microimages using a thin sampling kernel, for example a thin Gaussian kernel, so that pixel values are sampled only from one or a few of the microimages. | 05-23-2013 |
20130128077 | Thin Plenoptic Cameras Using Microspheres - Methods and apparatus for capturing and rendering high-quality photographs using relatively small, thin plenoptic cameras. Plenoptic camera technology, in particular focused plenoptic camera technology including but not limited to super-resolution techniques, and other technologies such as microsphere technology may be leveraged to provide thin form factor, megapixel resolution cameras suitable for use in mobile devices and other applications. In addition, at least some embodiments of these cameras may also capture radiance, allowing the imaging capabilities provided by plenoptic camera technology to be realized through appropriate rendering techniques. | 05-23-2013 |
20130128081 | Methods and Apparatus for Reducing Plenoptic Camera Artifacts - Methods and apparatus for reducing plenoptic camera artifacts. A first method is based on careful design of the optical system of the focused plenoptic camera to reduce artifacts that result in differences in depth in the microimages. A second method is computational; a focused plenoptic camera rendering algorithm is provided that corrects for artifacts resulting from differences in depth in the microimages. While both the artifact-reducing focused plenoptic camera design and the artifact-reducing rendering algorithm work by themselves to reduce artifacts, the two approaches may be combined. | 05-23-2013 |
20130128087 | Methods and Apparatus for Super-Resolution in Integral Photography - Methods and apparatus for super-resolution in integral photography are described. Several techniques are described that, alone or in combination, may improve the super-resolution process and/or the quality of super-resolved images that may be generated from flats captured with a focused plenoptic camera using a super-resolution algorithm. At least some of these techniques involve modifications to the focused plenoptic camera design. In addition, at least some of these techniques involve modifications to the super-resolution rendering algorithm. The techniques may include techniques for reducing the size of pixels, techniques for shifting pixels relative to each other so that super-resolution is achievable at more or all depths of focus, and techniques for sampling using an appropriate filter or kernel. These techniques may, for example, reduce or eliminate the need to perform deconvolution on a super-resolved image, and may improve super-resolution results and/or increase performance. | 05-23-2013 |