Week 5 Sumary : Kinect Fusion & More
KinectFusion is a great example of SLAM (Simultaneous localization and Mapping Systems).
In most AR systems it was assumed prior knowledge of the user’s environment.
Maybe a map of the city. Or use co-ordinates to merge a Google earth 3D model with the real world. Maybe a point of interest , Eiffel Tower, or the printer AR example we had seen in class. However a comprehensive map is not always available For example, If i want to do AR in my room or the classroom I don’t have a map available for that. SLAM systems map it in real time and track my position in it.
Some salient features of the KinectFusion system, that set it apart from the previous work are:
– Tracking: Real Time (both tracking and geometry reconstruction)
– Mapping: High quality reconstruction of geometry
– Geometry aware AR – embed the 3D model into the screen to enable physics with embedded objects in the screen
– Enable dynamic interaction
– Infrastructure-less (small Kinect, and not fixed so can be used anywhere.)
– Large scale rooms (No just a small desktop or a physical object)
Some projects that came before it like 2007 paper of “Parallel Tracking and Mapping for Small AR Workspaces” by Georg Klein and David Murray tried to tackle the same problem of trying to create a 3D real time reconstruction and AR implementation of a small workspace. Their system wasn’t real time in its true sense, processing 1 in 20 frames. In this project tracking and mapping are separated, and run in two parallel threads. The calculations are optimized by not looking at each frame – they take keyframes when a significant change has happened. Atleast 20 keyframes, or a minimum change in position. For tracking, they look at 50 top coarse features, like edges, corners. Decide the pose on the camera based on this. Look at the linear and the angular transformation.
Knowing the new pose of the camera, it optimizes the relocation of the other points. They use epipolar search algorithm to measure the depth in space of any given point.
Beyond the KinectFusion, the Kinect has been used in varied applications to create 3D reconstruction of humans and objects, faster and as a cheaper alternative to the existing solutions. For example, in the paper “Scanning 3D Full Human Bodies Using Kinects” by Jing Tong et al, 3 Kinects have been used to create a detailed 3D representation in a few minutes. The ease of creating these realistic models can have many applications in VR in the future.
Comments are closed.