Method and Device for Removing Objects from a Video Feed in Real-time

Abstract

According to embodiments of the invention, systems, methods and devices are directed to removal of objects in a video feed that are captured continuously through video sources in real-time. Various embodiments of the invention implement several novel features, including, in one embodiment, a device that is coupled to cameras has an input device for selecting the objects to be removed from the feed. The selected object is removed and replaced with pixels neighboring the location where the pixels were taken from. Contour may also be controlled via the input, and sensitivity may be adjusted based on the application. Removed objects may also be selectively reintroduced into the video feed via the input device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of application Ser. No. 14/225,381, filed Mar. 25, 2014.

FIELD OF THE INVENTION

[0002] This invention generally relates to multimedia. Specifically, this invention relates to removing unwanted and/or sensitive objects/images/people from a video feed in real-time.

BACKGROUND OF THE INVENTION

[0003] Video capturing and surveillance has segmented itself into many facets of everyday life. No matter where an individual goes or what an individual does, there is always a video being captured or generated. The most common reason for the multitude of cameras in public spaces is security. However, the growing number of cameras has led to an increase in privacy concerns.

[0004] Video editing is well known in the art. Video processing techniques are capable of processing of images so that some images look very differently than the original source. There is no question that video processing can be used in protecting certain information from being disclosed to the general public or to the wrong audience. For example, these techniques are used extensively in security checking in virtually every building where security is a concerned, such as hospitals, police stations, and airports. In most instances, some of these videos are captured continuously, 24 hours a day. The challenge is, there are hardly any technologies to remove certain parts of a video due to privacy concerns. For example, in a video captured in an airport, certain individuals may appear on camera who posed a security risk or may be at risk if their whereabouts are known to others. To protect those people in terms of privacy, there exists a need in the art to remove certain moving objects from a live video feed.

SUMMARY OF THE INVENTION

[0005] According to embodiments of the invention, systems, methods and devices are directed to removal of objects in a video feed that are captured continuously through video sources in real-time. Various embodiments of the invention implement several novel features, including, in one embodiment, a device that is coupled to cameras has an input device for selecting the objects to be removed from the feed. The selected object is removed and replaced with pixels neighboring the location where the pixels were taken from. Contour may also be controlled via the input, and sensitivity may be adjusted based on the application. Removed objects may also be selectively reintroduced into the video feed via the input device.

[0006] In an embodiment of the disclosed invention, a multimedia black box is used for protecting certain sensitive information from display by removing selective objects from video streams captured by cameras in real-time. The multimedia black box may have one or more of the following components: a) a first input element connected to a first device that has a display, a camera, a video and audio input interface and a video and audio output interface, the first device being a traditional image or video capturing device; b) a second input element connected to a second device that has a display, a camera, a video and audio input interface and a video and audio output interface, the second device being a traditional image or video capturing device; c) a first output element connected to the first device for outputting processed signals back to the first device; d) a second output element connected to the second device for outputting processed signals back to the second device; and e) a touch screen having a user interface operable to select objects on a video stream being captured by the first and second devices:

[0007] The user interface facilitates and/or carries out the following: a) video signals captured from both the first device and the second device are transmitted simultaneously to the black box via the first and second input elements; b) the video signals can be paused when the signals arrive at the touch screen of the black box such that contours of objects are definable by a user to be removed; c) the defined contour of objects and the video are further sent securely to a remote server via a network; d) the video signals of both the first and second devices are screened in real-time to detect and remove any instances of the defined contour of objects are removed from each of the video signals and missing areas vacated by the removed objects are filled using neighboring pixels; and e) the processed video signals are subsequently sent back to the respective devices by the black box through the first and second outputs, the processed videos signals appearing on the displays of the first and second devices.

[0008] In a further embodiment, sensitivity of the defined contour of objects may be configurable via a remote server, so that one object can be screened from the video signals when the detected object reaches a lower matching sensitivity level whereas another object stays in the video signals when the object fails to reach a defined matching sensitivity level. Further, other parameters of the black box may be configurable via a web page, the parameters including choices of processing means, locations of the remote server or the cloud, authentication information, security certificates, and various types of processing choices of the audio and video signals, further wherein a processing choice includes mixing the video and audio signals so that the processed video and audio signals appear to originate from a single source.

[0009] Still further, the processed video signals may be outputted to a pairing output, the pairing output being an output channel having video and audio signals that are synchronized simultaneously with a second device. The output element may be connected to both the first or second devices through a KVM switch.

[0010] In still a further embodiment, the user may undo the removal of one defined contour of object so that the one defined contour of object reappears but the rest of the defined contour of objects remain removed from the video signals. Further, a user may configure parameters of the black boxes through a web page by selecting black boxes. The parameters may include choices of processing means, locations of the remote server or the cloud, authentication information, security certificates, and various types of processing choices of the audio and video signals. The processing choice may include mixing the video and audio signals so that the processed video and audio signals appear to originate from only a single source. Furthermore, the first and second black boxes may be located in separate premises. The processed video and audio signals may be outputted to a pairing output, the pairing output being an output channel whose video and audio signals are synchronized simultaneously with a third device.

[0011] In accordance with these and other objects which will become apparent hereinafter, the invention will now be described with particular reference to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1 is a top plan view of an exemplary configuration according to an embodiment of the disclosed technology.

[0013] FIG. 2 shows the configuration of FIG. 1 with the selected object removed.

[0014] FIG. 3 is a top plan view of another exemplary configuration according to an embodiment of the disclosed technology.

[0015] FIG. 4 shows the configuration of FIG. 3 with the selected object removed.

[0016] FIG. 5 is a high-level block diagram of a microprocessor device that may be used to carry out the disclosed technology.

DETAILED DESCRIPTION

[0017] According to embodiments of the invention, systems, methods and devices are directed to removal of objects in a video feed that are captured continuously through video sources in real-time. Various embodiments of the invention implement several novel features, including, in one embodiment, a device that is coupled to cameras has an input device for selecting the objects to be removed from the feed. The selected object is removed and replaced with pixels neighboring the location where the pixels were taken from. Contour may also be controlled via the input, and sensitivity may be adjusted based on the application. Removed objects may also be selectively reintroduced into the video feed via the input device.

[0018] Referring now to the figures, systems, methods and devices are provided for selectively removing objects from a real-time video feed using a black box device to pre-specify the contours of the object to be removed when detected in the feed. A "black box" may be any device having one or more video inputs and outputs, and is used to exemplary purposes. Thus, the term "black box" is not meant to be limiting in any manner. FIG. 1 is a top plan view of an exemplary configuration accord. A multimedia black box is provided for protecting certain sensitive information from display by removing selective objects 11 from video streams captured by cameras 1 and 2 in real-time. Such a device comprises a first input element connected to a first device that has a display, a camera 1, a video and audio input interface and a video and audio output interface, the first device being a traditional image or video capturing device. Further, a second input element is demonstrated. It is connected to a second device that has a display, a camera 2, a video and audio input interface and a video and audio output interface. As one understood by skilled in the art, the second device can be a traditional image or video capturing device.

[0019] Output elements are included in the black box. For example, a first output element can be connected to the first device 10 for outputting processed signals back to the first device, whereas a second output element is connected to the second device for outputting processed signals back to the second device. FIG. 3 is a top plan view of another exemplary configuration according to an embodiment of the disclosed technology. The exemplary black box device can further include a touch screen having a user interface operable to select objects on a video stream being captured by the first and second devices 10, wherein video signals are captured from both the first device and the second device are transmitted simultaneously to the black box via the first and second input elements. Functions can be performed on the video signals. For example, the video signals can be configured to be paused when the signals arrive at the touch screen of the black box such that contours of objects are definable by a user to be removed. The defined contour of objects and the video are further sent securely to a remote server via a network, so that the video signals of both the first and second devices 10 are screened in real-time to detect and remove any instances of the defined contour of objects 11 are removed from each of the video signals and missing areas vacated by the removed objects 11 are filled using neighboring pixels, as shown on FIG. 2, which shows the configuration of FIG. 1 with the selected object removed.

[0020] The processed video signals are subsequently sent back to the respective devices by the black box through the above-stated first and second outputs. These videos signals can appear on multiple displays of the first and second devices. FIG. 4 shows the configuration of FIG. 3 with the selected object removed. According to another embodiment consistent with the present invention, the sensitivity of the defined contour of objects is configurable via a remote server, so that one object can be screened from the video signals when the detected object 14 reaches a lower matching sensitivity level whereas another object 15 stays in the video signals when the object fails to reach a defined matching sensitivity level.

[0021] Further into the embodiment, other parameters of the black box are configurable via many user interfaces such as a web page. In one additional embodiment, such parameters may include choices of processing means, locations of the remote server or the cloud, authentication information, security certificates, and various types of processing choices of the audio and video signals, further wherein a processing choice includes mixing the video and audio signals so that the processed video and audio signals appear to originate from a single source. Subsequently, the processed video signals are outputted to a pairing output that can serve as an output channel having video and audio signals that are synchronized simultaneously with a second device. The embodiment can be implemented so that the output element can be connected to both the first or second devices through a KVM switch allowing some flexibility to the black box user.

[0022] FIG. 5 is a high-level block diagram of a microprocessor device that may be used to carry out the disclosed technology. The device 300 may or may not be a computing device. The device 300 comprises a processor 350 that controls the overall operation of a computer by executing the reader's program instructions which define such operation. The device's program instructions may be stored in a storage device 320 (e.g., magnetic disk, database) and loaded into memory 330 when execution of the console's program instructions is desired. Thus, the device's operation will be defined by its program instructions stored in memory 330 and/or storage 320, and the console will be controlled by the processor 350 executing the console's program instructions.

[0023] The device 300 may also include one or a plurality of input network interfaces for communicating with other devices via a network (e.g., the internet). The device 300 further includes an electrical input interface for receiving power and data from a power or RFID source. The device 300 may also include one or more output network interfaces 310 for communicating with other devices. The device 300 may also include input/output 340 representing devices which allow for user interaction with a computer (e.g., display, keyboard, mouse, speakers, buttons, etc.).

[0024] One skilled in the art will recognize that an implementation of an actual device will contain other components as well, and that FIG. 5 is a high level representation of some of the components of such a device for illustrative purposes. It should also be understood by one skilled in the art that the devices depicted and described with respect to FIGS. 1 through 4 may be implemented on a device such as is shown in FIG. 5.

[0025] While the disclosed invention has been taught with specific reference to the above embodiments, a person having ordinary skill in the art will recognize that changes can be made in form and detail without departing from the spirit and the scope of the invention. The described embodiments are to be considered in all respects only as illustrative and not restrictive. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope. Combinations of any of the methods, systems, and devices described herein are also contemplated and within the scope of the invention.

Claims

1. A method for selectively removing specific objects in real-time from a video feed, the method comprising: capturing a scene using a first and a second camera, the first and the second cameras being placed substantially close to one another; receiving, at a multimedia device, a real-time video feed from each camera via an audio and video interface, wherein the multimedia black box has a display, a processor, memory and an input device; displaying, on the display, the real-time video feeds from each of the cameras; defining and identifying objects present in the video feeds by detecting the contours of each object in three dimensions using the different perspectives of each camera; receiving via the input device, one or more selections of defined objects to be removed from an outputted video feed; and removing the selections of defined objects from the video feed and outputting a final real-time video feed from the multimedia black box.

2. The method of claim 1, wherein the sensitivity of the contour detecting may be adjusted via the input device.

3. The method of claim 1, wherein the sensitivity of the contour detecting may be adjusted via a remote server.

4. The method of claim 3, wherein the remote server is accessible by a web page.

5. The method of claim 1, wherein the input device is a touch screen.

6. The method of claim 1, wherein the final real-time video feed is outputted via a KVM switch.

7. The method of claim 1, wherein the final real-time video feed is outputted via an HDMI output.

8. The method of claim 4, further wherein other parameters of the black box are configurable via the web page by pointing to the black box, wherein the other parameters include choices of processing means, locations of the remote server or the cloud, authentication information, security certificates, and various types of processing choices of the audio and video signals, further wherein asaid processing choice includes mixing the video and audio signals such that the processed video and audio signals appear to originate from only a single source.

9. The method of claim 1, further comprising a step of: filling areas vacated by the removed objects with neighboring pixels from a vicinity of the removed object.

10. The method of claim 1, wherein the selections of objects to be removed are made by a user on a freeze frame of the scene from the first and the second cameras.

11. The method of claim 1, wherein the defining of object contours is facilitated by a user outlining a visualization of the object on the display using the input device.

12. A method for removing certain sensitive objects from a video feed in real-time, the method comprising: receiving, at a multimedia black box having at least a touch screen, video feeds from at least two cameras, the cameras capturing substantially the same scene, having substantially the same objects in a periphery, further wherein the at least two cameras have associated input and output elements for transmitting and receiving audio and video; displaying the video feeds from the at least two cameras on the touch screen; pausing the video feeds momentarily such that contours of objects present in the scene are selectively definable via gestures made on the touch screen; combining the video feeds to produce a resultant video feed with the defined objects removed or censored; and outputting the resultant video feed.

13. The method of claim 12, wherein the resultant video feed is outputted via an HDMI port.

14. The method of claim 12, wherein the resultant video feed is sent securely to a remote server through cloud computing via network cables, bluetooth, wifi, 4G, and other wireless technologies.

15. The method of claim 12, further comprising a step of: removing the defined objects and filling in the void with neighboring pixels from the vicinity of each object.

16. The method of claim 12, further comprising a step of: selectively reinstating a previously removed object based on gestures performed on the touch screen.

Images