Pop-Through Buttons as 3DUI Devices

This article describes a method for interaction design that combines a novel use of available hardware to better match a users needs of 3DUI in AR worlds. The hardware trick is to combine two buttons with different tactile force requirements layered on one contact point. The researchers call this style of layering “pop-through buttons” in a reference to how one pops from the unpressed state into an action state and then continues further by popping through to the third state of the compound button.

The interaction is similar to a camera shutter button. The technique is a common idea even on older cameras (like my Canon AE-1, made in the late seventies). Pressing in the first low force range enables the light metering for focus, aperture, and light metering. Depressing the button harder into the second force range triggers the shutter mechanism.

The researchers built two prototypes. FingerSleeve is a cylindrical glove-finger with two touch points, each made from two tactile buttons with different actuation forces layered one on the other. The buttons are mounted on the sleeve, positioned so that they are over the interior pads of the finger. The buttons are activated by pressing the thumb against the finger, in a pinching style. Reference is made to the PinchGlove, a similar configuration which does not have the multiple force levels. A second prototype is the TriggerGun. The principle is the same, but the switch type configuration is different. The device is shaped like a handgun grip. There is a trigger-like switch at the index finger position made with two lever switched. At the top of the grip at the thumb position, there is a tactile button mounted over a lever switch.

In their stated Iterative Development process, the researchers explain that the prototypes are a work in progress, but that careful consideration went into choosing the size, force levels, location, and shapes of the materials for the button housings and the pop-through buttons themselves. Application of orientation controls to the devices needs improvement. As of this paper, the trigger gun uses a 6DOF inertial and magnetic orientation sensor with a wired connection to motion interpreting software.

I think that the current capabilities of MEMS on-chip devices could probably support wireless 9DOF sensing incorporating accelerometers, gyros, and compass in a small enough form to suit the TriggerGun. The paper mentions optical methods for motion tracking, but admits that this is a robust but limited way to do it. The required cameras make it useful for indoor work. Likewise, ultrasonic tracking would be limited to indoor areas with the appropriate sensor arrays. If a model can assume the device is connected to the wearer, a combination of computer vision (like Kinect) to create a skeletal model, fused with the motion data from a device-mounted MEMS could provide a high quality motion tracking system.

The researchers developed several interactions in real software applications. For each, the system follows a control paradigm to match the metaphor of stronger force means more deliberate action. Light pressure previews, firm pressure commits.

For ZoomBack, the tracked orientation of the device aims at an area of interest in the AR space. Light pressure on the button zooms the viewer to that point in the scene, and hard pressure commits the user to stay at that point. This continues the preview-commit metaphor.

Other interactions such as LaserGrab, SnapShop, CavePainting use similar techniques with additional context. CavePainting is a very good example of how restricting the domain of possible actions to a stricter metaphor (using a paintbrush in a virtual environment) allows the developers to extract additional usage cues from context to trigger different actions. They combine the users intuition about proprioceptive sense of nearness of the devices (the trigger control and the paintbrush) and make the button actions sensitive to that nearness.

What makes these devices such a good improvement over other 3DUI interaction devices is that they work well outside of the users gaze, but they aren’t clunky or awkwardly overloaded with buttons. In addition, by using durable switches, they correct some of the problems with other wearables like the PinchGlove which wears out significantly with constant use. Tactile buttons and lever switches are long-life devices where fabrics are less durable and gesture recognition or computer vision are much more computation heavy.

Overall, the significance of this paper is that it reinforces 3DUI control design ideas which reduce user effort, are suitably decoupled from the visibility and head tracking problems, and make intuitive sense.

What would the killer 3DUI device look like? A lot of ideas seem to be focused on extending what we know already from video game control design. This pop through button method, in particular, seems to be very much like the ‘analog’ buttons of several game consoles. Context sensitive menus and right-clicking seem to be very acceptable ways of overloading a device with few buttons. In any case, buttons with tactile feedback seem to be the best for blind and intuitive, sure use. I think the best new device will have tactile feel, motion sensitivity, contextual sensitivity such as proximity or orientation, and I think the perfect metaphor will be the keyring or swiss army knife idea. Every tool is in one place, and employing a given tool or key places the tool in specific context within the environment.

Comments are closed.