Week 5 Summaries
KinnectFusion
Kinnect is now widely used and cost effective RGB-D sensor. Many researchers are working with Kinnect nowadays. KinnectFusion is one of the 3D reconstruction works that is based on Kinnect’s depth sensor. As a depth sensor, Kinnect’s cost is compelling but the performance is not as compelling compared to other depth cameras since Kinnect’s depth map is noisier and has many holes in the depth map. In order to compensate for the accuracy, KinnectFusion tracks camera position and make use of the camera position and previous depth map to reconstruct the 3D model of the scene. Implementing them in GPU made this work run in interactively real time.
The author then talks about possible uses of KinnectFusion, such as low-cost handheld scanning, geometry-aware augmented reality, interacting with the scene, and etc. These various examples are related not only to 3D reconstruction of the scene but also to the interaction between the scene and the user. The authors claim that by using segmentation methods, we can find intersection of user and an object, thus we can find the interaction going on in the scene.
Then, the authors describe their GPU algorithm pipeline. When the raw depth map from Kinnect comes in, the map is converted from image coordinate to 3D points in camera coordinate. Then, the algorithm tracks camera with 6DOF using ICP. After computing the position of camera, the points are converted into global coordinates and fed to 3D voxel grid. Then finally, raycasting algorithm is applied to 3D voxel grid to extract implicit surface and render the image for output. Each step is implemented using CUDA GPU language. Then the authors describe some examples such as how to find interaction in the scene using data generated by KinnectFusion.
Using noisy Kinnect depth map, this work successfully generated fine 3D reconstruction of the scene in real time. Furthermore, the algorithm gives the users the clues to interaction going on in the scene. There are a lot of possible extensions of this work done in field of AR regardless of its dependency on Kinnect device.
Going out: Robust Model-based Tracking for Outdoor Augmented Reality
Going out is a model-based tracking system for scenes in outdoor. This work can enhance the traditional noisy location tracking used in augmented reality based purely on GPS. In urban environment, the signals for GPS are blocked by high buildings and GPS performance becomes significantly bad. Other sensors such as inertial sensors and compass are prone to error and accumulation of local errors. Thus, vision based tracking can help a lot in urban outdoor scenes.
The tracking algorithm is described as follows. First, the algorithm tracks the pose of a camera using edge-based tracking system. The tracking system calculates the motion from measurements of edge displacements along the edge normal. Using sample points, the algorithm searches in the direction of the edge. The authors use samples from a coarse textured 3D model of urban environment to generate more realistic view. Then, the algorithm takes inertial measurements and merges information using EKF. The tracking is improved by additional sensor information if the edge tracking is not performing well. Then, the authors describe the recovery strategy the algorithm takes when the tracking fails. First, they describe the condition of finding tracking failure. Then, they describe how to recover from failure using point matching against frame store. They extract features from each frame and take sum-of-squared-distances for features in query frame and each stored frame.
The result of this work is demonstrated by a location based game delivering an item to a location. The tracking is robust and the recovery strategy dealt well with occlusions in the scene. The results look promising but to be able to run this algorithm in real time, more engineering or improvements are needed, especially while in recovery status.
Comments are closed.