Patent application number | Description | Published |
20090285544 | Video Processing - A method and apparatus for processing video is disclosed. In an embodiment, image features of an object within a frame of video footage are identified and the movement of each of these features is tracked throughout the video footage to determine its trajectory (track). The tracks are analyzed, the maximum separation of the tracks is determined and used to determine a texture map, which is in turn interpolated to provide an unwrap mosaic for the object. The process may be iterated to provide an improved mosaic. Effects or artwork can be overlaid on this mosaic and the edited mosaic can be warped via the mapping, and combined with layers of the original footage. The effect or artwork may move with the object's surface. | 11-19-2009 |
20110228976 | PROXY TRAINING DATA FOR HUMAN BODY TRACKING - Synthesized body images are generated for a machine learning algorithm of a body joint tracking system. Frames from motion capture sequences are retargeted to several different body types, to leverage the motion capture sequences. To avoid providing redundant or similar frames to the machine learning algorithm, and to provide a compact yet highly variegated set of images, dissimilar frames can be identified using a similarity metric. The similarity metric is used to locate frames which are sufficiently distinct, according to a threshold distance. For realism, noise is added to the depth images based on noise sources which a real world depth camera would often experience. Other random variations can be introduced as well. For example, a degree of randomness can be added to retargeting. For each frame, the depth image and a corresponding classification image, with labeled body parts, are provided. 3-D scene elements can also be provided. | 09-22-2011 |
20110268316 | MULTIPLE CENTROID CONDENSATION OF PROBABILITY DISTRIBUTION CLOUDS - Systems and methods are disclosed for identifying objects captured by a depth camera by condensing classified image data into centroids of probability that captured objects are correctly identified entities. Output exemplars are processed to detect spatially localized clusters of non-zero probability pixels. For each cluster, a centroid is generated, generally resulting in multiple centroids for each differentiated object. Each centroid may be assigned a confidence value, indicating the likelihood that it corresponds to a true object, based on the size and shape of the cluster, as well as the probabilities of its constituent pixels. | 11-03-2011 |
20110293180 | Foreground and Background Image Segmentation - Foreground and background image segmentation is described. In an example, a seed region is selected in a foreground portion of an image, and a geodesic distance is calculated from each image element to the seed region. A subset of the image elements having a geodesic distance less than a threshold is determined, and this subset of image elements are labeled as foreground. In another example, an image element from an image showing at least a user, a foreground object in proximity to the user, and a background is applied to trained decision trees to obtain probabilities of the image element representing one of these items, and a corresponding classification assigned to the image element. This is repeated for each image element. Image elements classified as belonging to the user are labeled as foreground, and image elements classified as foreground objects or background are labeled as background. | 12-01-2011 |
20120162354 | Remote Workspace Sharing - Existing remote workspace sharing systems are difficult to use. For example, changes made on a common work product by one user often appear abruptly on displays viewed by remote users. As a result the interaction is perceived as unnatural by the users and is often inefficient. Images of a display of a common work product are received from a camera at a first location. These images may also comprise information about objects between the display and the camera such as a user's hand editing a document on a tablet PC. These images are combined with images of the shared work product and displayed at remote locations. Advance information about remote user actions is then visible and facilitates collaborative mediation between users. Depth information may be used to influence the process of combining the images. | 06-28-2012 |
20120194516 | Three-Dimensional Environment Reconstruction - Three-dimensional environment reconstruction is described. In an example, a 3D model of a real-world environment is generated in a 3D volume made up of voxels stored on a memory device. The model is built from data describing a camera location and orientation, and a depth image with pixels indicating a distance from the camera to a point in the environment. A separate execution thread is assigned to each voxel in a plane of the volume. Each thread uses the camera location and orientation to determine a corresponding depth image location for its associated voxel, determines a factor relating to the distance between the associated voxel and the point in the environment at the corresponding location, and updates a stored value at the associated voxel using the factor. Each thread iterates through an equivalent voxel in the remaining planes of the volume, repeating the process to update the stored value. | 08-02-2012 |
20120194517 | Using a Three-Dimensional Environment Model in Gameplay - Use of a 3D environment model in gameplay is described. In an embodiment, a mobile depth camera is used to capture a series of depth images as it is moved around and a dense 3D model of the environment is generated from this series of depth images. This dense 3D model is incorporated within an interactive application, such as a game. The mobile depth camera is then placed in a static position for an interactive phase, which in some examples is gameplay, and the system detects motion of a user within a part of the environment from a second series of depth images captured by the camera. This motion provides a user input to the interactive application, such as a game. In further embodiments, automatic recognition and identification of objects within the 3D model may be performed and these identified objects then change the way that the interactive application operates. | 08-02-2012 |
20120194644 | Mobile Camera Localization Using Depth Maps - Mobile camera localization using depth maps is described for robotics, immersive gaming, augmented reality and other applications. In an embodiment a mobile depth camera is tracked in an environment at the same time as a 3D model of the environment is formed using the sensed depth data. In an embodiment, when camera tracking fails, this is detected and the camera is relocalized either by using previously gathered keyframes or in other ways. In an embodiment, loop closures are detected in which the mobile camera revisits a location, by comparing features of a current depth map with the 3D model in real time. In embodiments the detected loop closures are used to improve the consistency and accuracy of the 3D model of the environment. | 08-02-2012 |
20120194650 | Reducing Interference Between Multiple Infra-Red Depth Cameras - Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment. | 08-02-2012 |
20120195471 | Moving Object Segmentation Using Depth Images - Moving object segmentation using depth images is described. In an example, a moving object is segmented from the background of a depth image of a scene received from a mobile depth camera. A previous depth image of the scene is retrieved, and compared to the current depth image using an iterative closest point algorithm. The iterative closest point algorithm includes a determination of a set of points that correspond between the current depth image and the previous depth image. During the determination of the set of points, one or more outlying points are detected that do not correspond between the two depth images, and the image elements at these outlying points are labeled as belonging to the moving object. In examples, the iterative closest point algorithm is executed as part of an algorithm for tracking the mobile depth camera, and hence the segmentation does not add substantial additional computational complexity. | 08-02-2012 |
20120196679 | Real-Time Camera Tracking Using Depth Maps - Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used. | 08-02-2012 |
20120239174 | Predicting Joint Positions - Predicting joint positions is described, for example, to find joint positions of humans or animals (or parts thereof) in an image to control a computer game or for other applications. In an embodiment image elements of a depth image make joint position votes so that for example, an image element depicting part of a torso may vote for a position of a neck joint, a left knee joint and a right knee joint. A random decision forest may be trained to enable image elements to vote for the positions of one or more joints and the training process may use training images of bodies with specified joint positions. In an example a joint position vote is expressed as a vector representing a distance and a direction of a joint position from an image element making the vote. The random decision forest may be trained using a mixture of objectives. | 09-20-2012 |
20120306876 | GENERATING COMPUTER MODELS OF 3D OBJECTS - Generating computer models of 3D objects is described. In one example, depth images of an object captured by a substantially static depth camera are used to generate the model, which is stored in a memory device in a three-dimensional volume. Portions of the depth image determined to relate to the background are removed to leave a foreground depth image. The position and orientation of the object in the foreground depth image is tracked by comparison to a preceding depth image, and the foreground depth image is integrated into the volume by using the position and orientation to determine where to add data derived from the foreground depth image into the volume. In examples, the object is hand-rotated by a user before the depth camera. Hands that occlude the object are integrated out of the model as they do not move in sync with the object due to re-gripping. | 12-06-2012 |
20130163857 | MULTIPLE CENTROID CONDENSATION OF PROBABILITY DISTRIBUTION CLOUDS - Systems and methods are disclosed for identifying objects captured by a depth camera by condensing classified image data into centroids of probability that captured objects are correctly identified entities. Output exemplars are processed to detect spatially localized clusters of non-zero probability pixels. For each cluster, a centroid is generated, generally resulting in multiple centroids for each differentiated object. Each centroid may be assigned a confidence value, indicating the likelihood that it corresponds to a true object, based on the size and shape of the cluster, as well as the probabilities of its constituent pixels. | 06-27-2013 |
20130244782 | REAL-TIME CAMERA TRACKING USING DEPTH MAPS - Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used. | 09-19-2013 |
20140126821 | FOREGROUND AND BACKGROUND IMAGE SEGMENTATION - Foreground and background image segmentation is described. In an example, a seed region is selected in a foreground portion of an image, and a geodesic distance is calculated from each image element to the seed region. A subset of the image elements having a geodesic distance less than a threshold is determined, and this subset of image elements are labeled as foreground. In another example, an image element from an image showing at least a user, a foreground object in proximity to the user, and a background is applied to trained decision trees to obtain probabilities of the image element representing one of these items, and a corresponding classification assigned to the image element. This is repeated for each image element. Image elements classified as belonging to the user are labeled as foreground, and image elements classified as foreground objects or background are labeled as background. | 05-08-2014 |