Week 5 Summary : KinectFusion and Going Out
KinectFusion: Realtime 3D Reconstruction and Interaction Using a Moving Depth Camera
This paper discussed one of the most recent and talked about technology currently: KinectFusion. It reconstructs a 3-D model of an object or environment using the data it receives from the Kinect sensor. The depth data from Kinect is used to track the 3D pose of the sensor and reconstruct the 3D model of the physical scene in real-time.
Even though depth cameras are not new, the Kinect due to its low-cost and real-time nature unlike a number of offline reconstruction systems has made it popular. The most basic yet compelling use for KinectFusion is its use as a low-cost object scanner. Also, any change in the physical location of the scene object is reflected in the 3D model reconstruction. KinectFusion enables interactions between the virtual world and real world objects, a true sense of Augmented Reality.
According to me the most compelling feature of KinectFusion is its ability to enable virtual objects to cast shadows, reflect, simulate rigid-body collisions and depict physics properties like any similar real world object. There is no shortage of applications that can be built to take advantage of these very properties.
This technology can be used to reconstruct physics based interactive experiments. Fine tuning the segmentation, tracking and multi-touch capabilities further I personally feel that KinectFusion has a lot to offer.
However, I would like to know more about the memory constraints that the system might face especially in large scene coverage and how much of an impact does continuous scene motion have on the output. The author mildly touches on the above but it would be interesting to know more on the limitations of the technology as well.
Going out: Robust Model-based Tracking for Outdoor Augmented Reality –
Gerhard Reitmayr, Tom W. Drummond
Handheld devices and other types of wearable computing provide an excellent platform for mobile visualization for outdoor environment. With augmented reality it is possible to display 3-dimensional information over a view of the real world.
However, to correctly overlay such information, the position and direction of the video camera within the environment needs to be known very accurately. To make this possible folks at Cambridge University developed a tracking system that combines input from video images, electronic sensors measuring rate of rotation, gravity and the Earth’s magnetic field and GPS.
This system overcame the limitation of the traditional GPS which was not accurate in a typical urban environment. Additionally it was prone to magnetic sensors and fields encountered in the urban environment. “Going out” solves these shortcomings. It is an edge-based tracking system which extracts the pose of a camera mounted to the handheld AR system. The system makes use of a textured 3D model instead of a pure edge model whose complexity is much lower. Firstly a view is rendered from the camera pose. Then, the edges are extracted from the grayscale image. The prior pose is overlaid on the video image for a match and is updated with measurements.
For demonstration the system was tested in a game, in which the goal was to locate the window in the environment and reach it via the ladder. In terms of accuracy the building appears slightly closer than it is and the camera trajectory deviates from the line. However, it is quite robust in handling disturbances and occlusions. The system operates at about 15-17 frames per second in terms of performance.
It would be interesting to see how this technology can be used for commercial applications despite the few limitations which it has currently.
Comments are closed.