A 360° camera that sees in 3D

It will be the ideal tool for videoconferences, video surveillance, movie making, and creating backgrounds for video games. Researchers from two EPFL laboratories have invented a revolutionary camera that can film everything around it, simultaneously and in real time, and then reproduce the images in three dimensions, distortion-free. A patent application has been filed.

The camera was inspired by the structure of a fly’s eye, and works without resorting to mirrors or mechanical parts of any kind. Over one hundred cameras, similar to those used in mobile phones, are crowded onto a metallic hemisphere the size of an orange. Because they are so close together, their range of vision overlaps slightly. A second, miniature prototype has also been developed. It’s about the size of a golf ball and has 15 cameras. The user can choose to have them all work together to obtain a panoramic image that covers a 360° range of vision, or individually to capture a particular point of view. The cameras were designed and built at EPFL in a collaboration between the Signal Processing Laboratory, led by Professor Pierre Vandergheynst, and the Microelectronic Systems Laboratory, led by Professor Yusuf Leblebici.

“With this invention, we solved two major problems with traditional cameras: the camera angle, which is no longer limited thanks to the camera’s ability to film in 360° and in real time; and the depth of field, which is no longer limiting thanks to the 3D reconstruction,” explains Vandergheynst.

Close collaboration

Vandergheynst’s lab wrote algorithms to calculate the distance between the camera and objects being filmed in order to do the 3D reconstruction, as well as the algorithms that assemble the images taken by all the different cameras into a single panoramic image. The Microelectronics Systems Laboratory developed the material and electronic apparatus that make it possible to collect and process the multi-gigabits of data that stream in at the rate of 30 images per second from the various cameras.

“The results that we have achieved to date have only been possible with the very close collaboration between our two groups, combining our expertise and managing the hardware design and algorithm/software design together. The outcome of this work is likely to change the entire field of image acquisition, with a huge range of potential applications,” adds Leblebici.