week2 Comments turned off

Week 2 summaries

By on January 16th, 2013

Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient

The paper describes how a small video camera mounted on a HMD can be useful for seeing ultrasound imagery within the patient. This approach basically merges virtual images with the real world view and can be extended for use in image guided surgical procedures, seeing through smoke in burning buildings, on site architecture preview and so on. However ultrasound imagery has several disadvantages as well such as low SNR, poor spatial resolution, speckle, increase in attenuation with frequency, phase aberration and reflection and refraction artifacts. It is yet apt for fetal examination, cardiac study etc because ultrasound imagery is safe and generates images in real time.

The system deals with 3 major areas such as:

  1. Real time ultrasound data acquisitions and rendering – The system incrementally visualizes a 3d volume dataset from multiple 2d data slices because when the system was constructed only 2d ultrasound scanners were available for real time volume data acquisition.  They sample the target function at irregular points and then reconstruct a 3d volume from this time series of spatially irregular sample points. This is useful as it limits the storage and computation requirements. The reconstruction is a 4d convolution process i.e a 3d Gaussian kernel is used for spatial reconstruction followed by a temporal reconstruction based on ARMA filtering. The reconstructed volume is then rendered with an incremental volume rendering technique. A hierarchical ray cache that stores ray samples is used for fast rendering of polygons.
  2. Creating a virtual environment that displays this ultrasound data in real time –  As each echography image is acquired by the ultrasound scanner, the scanner’s and HMD’s position and orientation in 3d world space are tracked with 6 DOF. Using this geometry, 3d renderings of the 2d ultrasound images is constructed by an image generation system. A small TV camera mounted on the HMD captures real world images which are video mixed with the 3d ultrasound image renderings.  The resulting composite image in turn shows the 2d ultrasound data registered in its true 3d location.
  3. Recovering structural information for volume rendering – This is concerned with surface definition and not covered in this paper.

The systems hardware is composed of 3 major components:

  1. Image acquisition and tracking system- It includes the ultrasound scanner and a Polheums tracking system which has 2 receivers, one tracks the HMD and other tracks the ultrasound transducer.
  2. Image generation system – It includes Pixel Planes 5 graphics multicomputer
  3. Video-see through HMD – It includes a portable TV camera, a video mixer and VPL EyePhone.

After testing it in the lab, the first live experiment was on a 38 week pregnant volunteer. The results were quite impressive though there were technical problems their system couldn’t resolve such as conflicting visual cues, system lag, tracking system range and stability etc. Most of these problems were due to the technology constraints and hence their approach seems to be great contribution to Computer graphics and visualization.

Question: Considering the number of applications of this technique the cost of HMD’s should reduce due to mass production. But even in 2013 good quality HMD’s are still too expensive for public use. Why? Why are most of the technological constraints faced by Bajura still existing?

Virtual Environments for Treating the Fear of Heights

The paper is quite interesting as it deals with how virtual environments can be useful for treating the fear of heights.  Acrophobia is treated by exposing the subject to anxiety producing stimuli. Studies show that in vivo(real) graded exposure is more effective than systematic desensitization(imaginary). What distinguishes VE ‘s from Computer Graphics and multimedia is the sense of presence and immersion. The paper validates that this sense of presence can be positively used to produce the same results as in vivo graded exposure. While VE’s had been applied to psychology prior to this paper, no study documented clinical data or analytical results. The authors of this paper so focus on proving how VE’s are effective in treating acrophobia rather than the technical aspects of developing such a Virtual Environment.  For the therapy, three virtual environments were created: an elevator, a series of balconies and a series of bridges of which most subjects founds bridges as least scary. Tactile feedback provided in each of these environments increased the subject’s sense of presence and immersion.  Questionnaires were used to find suitable subjects and record the study results and subjects feelings. The good feature about the experiment I feel was that the system was flexible as it allowed the subjects to spend as much time as they need in each situation to overcome their fears.  In general, subject’s anxiety increases as they experience more threatening situations and decreases as they spend more time in the same situation. The study shows that subjects experienced similar change in anxiety even in the Virtual Environment situations. Thus, using VE for psychological treatment is useful as it is less expensive in terms of both time and money as compared to in-vivo graded exposures. Also VR exposures can be made extreme that can be life threatening in real world.  The only two problems with VR exposure is nausea and being ineffective for people with poor eyesight. They treated nausea by limiting sessions to 30-40 minutes which seems as a reasonable solution. VR is ineffective for people with poor eyesight as they lose the sense of presence and immersion and this problem can be tackled by using specifically programmed images for them.  In totality, I feel Virtual Environments are apt for treating different forms of fears and for various psychological treatments.

Question: The results show that it reduces a small group of peoples’ anxiety but has/can it be scientifically proven as a therapy?

Designing Interactive Theme Park Rides

This paper is all about the design considerations Disney made to make their Theme Park ride fun. Disney realized that people will pay for the ride only if it is entertaining enough and it involves social interaction.  To ensure that the ride is fun they needed to prevent people from steering to dull places. By having sneak attack ships suddenly appear, ensuring at least one of the islands is in view, using water force to redirect the ship Disney ensured that people sailed to interesting spots. Since 5 minutes is too less to guarantee that guests would experience an exciting climax, they scripted the game such that the guests have to battle with Jolly Roger in the last half minute to win/lose the game.  Since 5 minutes are too less for guests to understand complex user interfaces they used steering wheel and cannons which every player can intuitively use. 3d surround sound, tactile speakers, strobe lights, 3d stereo glasses made the virtual world experience more real i.e increased guests sense of presence and immersion.  Since people visiting theme parks are always in group, this ride involving 4 players helps them experience a shared social interaction. Guest testing assured that the game is balanced properly and allowed to deal with unpredicted social behaviors.  All these features make the ride different than a video-game or an experience that can be replicated in the living room and ensure that the players enjoy the ride.

Question: Considering the technological innovations, why can’t such an experience be re-created in one’s living room even if on a small scale?

week2 Comments turned off

Week 2: Summary

By on January 16th, 2013

Merging Virtual Objects with Real World

One of the earliest and perhaps the most widely used result of live ultrasound is the visualization of a pregnant human subject. According to the current advances made and the work going on in this field clinical use of this technology is an area of intense research activity particularly in fetal anomaly scanning.

 The paper talks about this goal of creating an “ultimate” 3D ultrasound system which displays 3D volume data in real time.
In order to achieve this the two core areas in which advances need to made are those of
 1.) 3D volume data Acquisition
In order to reconstruct a 3D image from images of a lesser dimension, the location and orientation of the imaging primitive must be known. One of the main concerns in today’s realtime 3D imaging systems is the large amount of active channels needed to scan the volume. The paper touches upon the use of Explososcan  which is a parallel processing technique used to return echoes to boost data acquisition rate to solve this very problem.
 2.) 3D Volume data display
There is more than one way in which 3D images can be presented. This not only includes visual form but also presentation in the form of calculated values. Inorder to display the volume data low pass filtering techniques are used
Advantages of 3D Ultrasound :
Relatively safe compared to other imaging modalities.
Images are generated in real time.
Improvements in modalities  such as MRI.
Disadvantages of 3D Ultrasound :
Low signal to noise ratio
Poor spatial resolution
Attenuation that increases with frequency and reflection and refraction problems
Various applications :
From my understanding the application still remain centered more or less toward medical implementations. The author lists the following,
Emergency Scenarios:
In case of mishaps, or natural calamities 3D ultrasound can be used to see through areas in the interior of building to improve rescue and support operations especially in places inaccessible and hazardous to human lives.
Surgery application:
Neurosurgery, foetal imaging, cardiology applications. It can used to image arteries other small sensitive organs that must be avoided during surgery.etc
Electronic application:
But according to me the greatest achievement will be made when 2D technology on portable devices like laptops,PDAs will be replaced by 3D.
Final Conclusions – Although technology advancements have led to high quality 2D imaging, a real-time 3D imaging system has not yet been implemented with a quality on par with 2D imaging. The author agrees with this when he states that this field is yet to flourish and is only at the initial stages with a lot to do yet.
Virtual Environment for Treating the Fear of Heights
This paper on Virtual Environment for Treating Fear of Heights according to me is a novel way of using Virtual Environments. It is interesting to know how Acrophobia is not only studied but too an extent cured by using Virtual Environments.
After evaluating which heights situation cause a patient anxiety a therapist would arrange therapy sessions for the patient to go through a number of exposures and adjustments to help the patient overcome his fear. The threat scale with each passing session would increment, starting from say a third floor window to a higher floor window virtual environment setup later on.
The author mentions the implementation of 3 environment for therapy including:
1.)An elevator
2.)A series of balconies
3.)A series of bridges (considered to be less scary)
The study seems robust and complete in itself because a series of experiments were conducted which included a series of screening questionnaire and evaluation of their panic level progressively.
 The most remarkable aspect about this study is that it has helped prove that a person’s perceptions of physical world can be modified by experience in a virtual world. This can be used in treating numerous medical conditions and better understand human behavior and psychology in a controlled virtual environment which would replicate the behavior in the real world.
One question I would like to ask the author is if Virtual Environment is truly that efficient in solving certain medical conditions why haven’t we seen more of it. Is it because of the cost involved or scalability?
Designing Interactive Theme Park Rides
This paper describes an interactive theme park ride based on the classic Pirates of the Caribbean at Disneyland.  Its a 4 player immersive adventure game in which 1 guest steers the ship while the rest use cannons to attack virtual enemy pirate ships, sea monsters etc in order to defend as much gold as possible.
While the players think that they are completely in control in the game environment the developers have used interesting concepts like architectural weenies, guide ships, sneak attacks etc to guide the game play and establish some level ground. The 5 minute experience starts of with first a non-interactive sequence with Jolly Roger the ghost pirate and then climaxes in the last 30 seconds ending in two ways: victory or defeat both having their highlights.
The highlights of the game are intuitive user interface, easy understanding of what’s expected and the immersive environment coupled with the motion base to feel every cannon hit, every wave along with proper balance, script, audio and artwork are its star features.It also emphasizes on the fact that a visit to ant theme park is fun if you go with a small group.
The attention given to detail is very evident from the fact that it took almost an entire year perfecting the underlying dynamics of the gameplay.
We have seen how interactive and immersive environment prove to be successful in the entertainment field one question I would like to ask the author is will it ever be feasible to get such an experience in day to day commercial application or is there no need for that as of now.
week2, Weekly Paper Summaries Comments turned off

[week2 summaries]

By on January 16th, 2013

Virtual environments for treating the fear of heights

This paper mainly talks about using VE to help acrophobia patients to overcome the fear of heights. Acrophobia, according to wiki, is an extreme or irrational fear of heights. By setting up a virtual environment and expose those patients to the virtual height situations, the patients is expected to habituate.

In order to setting up the stage, firstly a graphics workstations was designed with numbers of virtual heights situation. The real-time response was set to 10-fps with detailed monoscopic images. To better observe and analysis the patients’ reaction, 3 virtual environments were created. They were an elevator with only waist high guard rail, a series of balconies which stretched outside a building and 3 canyon bridges varying in hight and steadiness.

Then, according to the American Psychiatric Association’s diagnostic and statistical manual and fear questionnaire specifically for this study, appropriate subjects were selected and randomly assigned to either a treatment group or a control group. A pre-treatment assessment was conducted for both group and after 7 weeks, a similar post-treatment assessment took place as comparison.

During therapy sessions, subjects would be asked every 5 minutes to evaluate their feelings in several related aspects. This was conducted by an advanced clinical psychology graduate student and the whole session was videotaped to be reviewed by a supervising licensed psychologist.

According to theory, the way to cure acrophobia is to activate the fear memory and modify it. Using virtual height situation for immersion was aimed at activation the memory for reconstruction. For acrophobia, the anxiety usually increases with higher exposure and decreases if time is spent in the situation. In virtual height environment, same reaction had be detected as evidence for the performance effect.

After therapy session, a table with PRE and POST assessment for both groups had shown the significant decrease of passive emotion in treatment group but not the other. Furthermore, some subjects reported progress in real life after the session, which evidences the session to be meaningful. The noticeable change showed the success in using VE to activate and modify negative attitudes toward height.

Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient

This paper described a system which can display ultrasound echography in real time. By doing that, there were two aspects we need to dig into: 3D volume data acquisition and 3D volume data display. The research here mainly concentrated on 3D volume data display.

The standard volume rendering techniques is not well suited in this case. This techniques relies on preprocessing while data we acquire is dynamic and needs to be rendered in real time. A incremental, interactive 3D ultrasound visualization technique is introduced to solve this problem. Firstly, they irregularly sample points from the targets. Since the targets is updating over time, the sampled points keep changing. Time stamps are used to mark different sample source. Then, the technique reconstructs a 3D volume based on the samples in corresponding time frame. So each time frame will get a unique 3D volume with incremental time steps. At last, a reconstructed 3D volume is rendered based on those time-specific 3D volumes.

To visualize the ultrasound image in place where the image is acquired, a system is developed to facilitate. The system has 3 components: image-acquisition and tracking; image-generation and a HMD. Firstly the ultrasound images are acquired by the scanners for later rendering. Simultaneously, HMD is taking a real-world image. The tracking system will track the position and orientation information both of the scanners and the HMD for calibration. After the ultrasound images get rendered based on the geometry provided by HMD, using rigid body rotation, a video mixed image is generated and sent to a VPL EyePhone (attached to the HMD) for display.

At the end, an experiment result of scanning the abdomen of a pregnant woman is shown and some remaining technical problems are listed for future research.

Designing Interactive Theme Park Rides

This paper mainly talks about how they designed the interactive game Pirates and what should be paid extra attention to when designing it.

As a interactive game which gives the guests freedom to explore, it doesn’t mean we can set them as a puma at large. The designer carefully placed some guidance (like architectural weenies, guide ships) or push (like sneak attacks or the waterspout) to keep or force the guests inside the game play area. Also, the architectural weenies, guide ships and sneak attack ship, by carefully design, together with scenario attached islands, will keep a balanced pacing between action and calm. Furthermore, the climax at the last 30 seconds is triggered for an ultimate battle between guests and Jolly Roger the Ghost Pirate. The open ended storyline will lends to a desire for a second round. In all, though looks like the players get the total freedom for destiny, the adventure pacing is actually under control to guarantee the fun.

Usability is another elements that impacts an interactive games’ quality. Pirates utilizes quite intuitive user interfaces: a steering wheel to steer and actual cannons to fire. To make it more handy, once on board, guests can play around with those facilities to acclimate. Apart from these physical fun, the game designers bend reality into a more show-like way, which gives the guests a fancy and breath-taking visual effect.

To make it more realistic, the designers mixed the motions (like cannon ball hit) and acoustic effect together, thus gamers can better immerse into the environment.

At last, a successful game is always designed and planned to satisfy people’s psychological needs. Noticing nobody goes to a theme park alone and people in small groups has the intention of enjoying shared experiences, a game like Pirates can always bring the participants tons of joy to share in between.