Recent advancements in mobile processors (CPUs and GPUs) have enabled researchers to explore 3D augmented reality applications on handheld devices. Art of Defense (AoD) is a novel AR game for a commodity phone (the Nokia N95). The goal of Art Of Defense is to explore game design and interaction techniques that are well suited to mobile AR on a small screen device. One principle guiding AoD is to uses the phones limitations as part of the game itself; in this case, designing assuming the player has a limited view of a larger game space.
AoD is based on two core mechanics that well suited to handheld AR: dynamic multi-marker map building and sketch-based interaction. Both mechanics encourage physical movement by the player and tangible interaction with the game pieces, which are two key differentiators between AR games and other phone-based games. The dynamic multi-marker building technique not only enhances and encourages movement by the player, it enables the game to be mobility and portable because the gameboard is disassembled when not in use. By requiring the player to sketch on the erasable game tiles (where are also laser-engraved with markers), our game is very tangible and enhances the content relationship between the physical and virtual worlds.
AoD also demonstrates a new type of AR exploration game, where the player controls what parts of the space they will, and can, explore.
Click here to view a video of the game.
12 comments
4 pings
Skip to comment form
I posted this on toucharcade as well.
Wow. This kind of stuff gets me so excited! I love your game project, can’t wait to see the newer version, let us know when the new video is available.
I am a huge board game player and this fusion of real-virtual gaming environments seems to add so much to the experience. I’d love to pick your brain just to get a taste of what is coming down the mobile AR pipeline. I am a forum member at toucharcade under Johannvonstranovic if you want to PM me. I am wondering:
Is it possible to use an existing game board that people already have, like settlers for instance, and program the AR to recognize those tiles?
Would your game come with a stand? It would help out a lot in making it possible for one player to play. For a multiplayer game board I am imagining a circular board with a track on the outside edge that a stand holding the device could rotate around like a lazy susan. You could have turn based games or coop games where you must quickly rotate the device around to the person with the best view to complete a certain task etc… Or vice versa, the stand holding the device could be stationary, and the board could rotate around in a circle. This would be ideal for one player games.
Is the virtual sword/lightsaber possible? Can one 3d projection controlled directly by moving the marker effect the other 3d projected elements? Sorry about the poor wording, but hopefully you know what I mean.
Thanks for listening and for the great AR work you are doing!
Author
Glad you like it! I think you’ll like the newer video and version of the system, which we will post soon.
Eventually, yes, you could build a system that works off existing board or hex tiles (like from Settlers). We have tons of board games in the lab that we use for brainstorming and thinking about games; I’ve been referring to the idea of using existing boards as “game recycling” (our “Recycled Games Project”)! 🙂
I am not really excited about using a stand for the phone, but I could see it for some games. What makes handheld games existing is the ability to dynamically control your viewpoint by moving it around. In the newer version of this game, we moved away from sketching, and use more pieces and tokens, specifically so it’s more natural one handed.
One of the students in our class last semester designed a game that used a lazy susan, with a stand, actually. It was interesting, but it seems very constraining, especially with the small screen (now you have to bring everything “under” the small area seen by the phone). In addition, it forces the player to sit in one place.
I don’t think a “sword” would work, just because of the limits of what the camera can see and the kinds of motions you’d want …
Cheers, and thanks for the comments. Nice to see others excited by this too.
Thanks for the quick response!
Recycled Games, I love it. I think this is a great idea especially due to the added difficulties/risks of mass producing a game board, distribution expenses etc… The power of the iphone distribution model is instant gratification, which is why I think from a money making standpoint, the best way to convert people over to the AR possibilities would be to officially license an existing popular board game and create a stand alone iphone app that people could network and play and enjoy solely on their iphones. Then provide the easter egg ability to pull out the board game that these purchasers most likely already own and introduce them to the AR playing possibilities. I’m already sold on the idea, but this might be a way to ease the consumer base into thinking about buying the actual physical board games you and your students will undoubtedly make. After all, that requires more effort than hitting the buy now button : )
I see your point about the stand. It would have to be the right type of game for sure. I am a graphic designer/videographer so my inspiration was to come up with some sort of an inexpensive “mini jib / steady cam” that allows free movement. This is easier said than done obviously, but I imagine it giving the camera a smooth almost cinematic feel to the game while not constricting movement. Like the arm on the old lamp imacs
http://www.youtube.com/watch?v=eCHblz_LsKc&feature=player_embedded
or the pixar anglepoise lamps.
If implemented in the right type of game, this “mini-camera jib” could also be used to help the player quickly choose optimal zoom lengths for game play and leave one or even both hands free for more frantic physical game board interactions. It doesn’t necessarily have to be more frantic, maybe the player needs to hold cards/tokens and role dice and still would like a virtual view of the board…
I’m sad about the “sword” 🙁 but see what you mean.
I can’t wait to see what you and your students come up with next. I know I am late to the conversation, but I am excited by the possibilities.
I tried emailing this to you but I don’t think I have the correct address, so pardon that is off topic, and hopefully you find it here.
Since you were so kind to indulge my curiosity and took the time to answer my questions, naturally I wondered if I could bug you some more with a few additional questions/ideas 🙂
Having only recently begun to research what is being done in the field of AR. I hope you’ll forgive my ignorance on what is possible and or already been considered or done.
You mention the new vuzix 920AV wrap to be very exciting, if it lives up to the proposed specs, and I know it will be compatible with the iphone. This obviously is exciting for the potential AR games that can result with this combination. However, even if the new iphone is released this summer with increased graphical processing power, I imagine there are severe limitations on how much (markerless) scene processing could occur simultaneously with overlaying the actual game elements. However, the new iphone is rumored to support higher 3g bandwidth (7.2 Mbps) and support 802.11n wireless networks which causes me to wonder if a service with OnLive games might be able to work. OnLive is untested as yet, but proposes you’ll only need 5mbps for high def streaming. It does require a cheap hardware add-on that may require a unique solution to this proposed set up.
OnLive game service article link below
http://news.bbc.co.uk/2/hi/technology/7976206.stm
Could the iphone be used to gather enough telemetry and send it to a service like this fast enough to make it work? I honestly would not have believed OnLive’s service to be possible in any scenario, but early beta reviewers seem quite positive on the zero latency.
My other thought involves the 6 dof attachment for the vuzix wrap, you mentioned it might be to cheap to be really useful, but let’s assume it’s pretty good. Also, I believe that with the addition of wii motionplus to the wiimote, it will also gain 6 dof. I may display my ignorance here, but once a 3d environment is mapped using stereoscopic cameras and all sorts of other tricks, won’t the headset be able to know exactly where it is within that 3d space? If so, once a physical “gameplay area” is defined to the system, won’t it be much easier to add the gameplay elements to that space without having to continuously track visual cues from the cameras?
I know it is not ideal but due to the difficulty in mapping out an actual 3d environment, why not enlist the users help? The market that will buy the vuzix wrap, will be tech savy enough to do it. Plus this market would all love to kill zombies, or fight storm troopers in their own home no matter what it takes.
My thought is to first calibrate the wiimote(with motionplus) to the glasses by displaying a target area that you simply align the wiimote with and press A. Now in theory I imagine that these devices could keep track of each other’s relative positions in 3d space. Next have a game experience that is very much user controlled. Using the stereoscopic cameras the user could walk around his house to define a “gameplay area”, the system could try it’s best to determine the polygons that best describe the scene and the user could assist it in trouble areas via mapped buttons in the wiimote to simplify certain polygons, readjust certain nodes, define immoveable/moveable objects, etc… Once the area is defined the user could use the wiimote to set up the level to his liking, hide ammo under the couch, place three zombies behind this door, etc… The enemies would have to have some AI to know what to do in ad-hoc environments and situations. Or the program could randomize placements. Then, hit start, hit A to extend the lightsaber from the wiimote and see if you can survive your own creation in your own home.
I don’t know if this video is real or not, but regardless it is almost exactly what I am hoping for.
http://www.youtube.com/watch?v=M0drf5mHIH0
I realize that all of the products and services I mentioned have not been released yet, so who knows. Also I might be fundamentally flawed from the get go on these ideas, but I would love to hear your take on them. Regardless of how it’s done, I think AR is going to be big in the coming years! Maybe someday Rainbow’s End big, but I will try to keep my hype under control.
Thanks,
Author
Hi, I’ll try to reply to some of this here.
In theory, I suppose a system like onlive could work. We actually did something very much like this for our mobile AR secondlife project, shown in this video: http://www.youtube.com/watch?v=vAWnFgwxxbI
The big problem is the latency; not the encoding/decoding, but the simple network latency. Also, actually dealing with scale, assuming it’s successful. It’s one thing to have the hardware and network bandwidth for a small-scale test, it’s quite another thing to support tens of thousands or more of users!
And, with AR, the bandwidth problem is worse because (a) it’s two way (sending video and receiving it back) and (b) latency is more obvious when you have ground truth (your view of the world) to compare it to.
But, for some applications it would be useful.
Regarding the vuzix, etc. In theory, having orientation and stereo cameras will enable a lot of things, going from those raw inputs to “knowing where you are in the room and having it mapped out” is a HUGE engineering endeavor. Nobody can, as yet, do natural feature tracking “for real”, with or without stereo, and with or without other sensors. It’s just “too hard” as of yet. Doing it on a low end computer just makes everything worse.
What’s exciting about the Vuzix display is that, if it really does work as we all hope it will, it’s very existence will spur more people on to solve this problem. Right now, everyone says “why bother, nobody can make affordable displays and nobody will wear them anyway.” … once we have the displays, and IF people show interest, things will change.
Linking in wiimotes, etc., won’t really help. The wiimote is actually a lot more limited than the various uses of it imply, and it’s especially not useful for mobile work. The biggest problem, for example, is the need for line-of-sight to the sensor bar, but there are other limitations.
In theory, your scenario is what many of us dream about, but the technical issues of really making it work are still a long way from being solved; the existence of this new hardware isn’t really going to change that … yet.
The zombie video is “real” for what it is. It’s just a tech demo, in a pre-defined space. But, it’s cute. 🙂
Thanks so much. I really appreciate you indulging my curiosity, and taking the time to respond. Best of luck to you and your students!
Guys please… Take this idea, team up with Wizards of the Coast and do a D&D simulation for RPG players, where we move our characters in a dungeon and have different animations for our Attacks, it can even implement the rules, so we just move our avatars around and it can do all the calculations and u will sell at least one for every RPG player in the world!!!
Author
Ha! Yes, that would be fun. But, the problem with handheld games right now is the fragmentation of devices. The only device that interests big game companies is the iPhone, and they don’t have a video API.
WotC is not that big of a player, and they have the Insider going on, im pretty sure its not a hard idea to sell. Im not talking about selling software, but the hardware, with exclusive software for its purpose. Man, we pay hundreds of dolars on miniatures, im sure there is a market for augmented reality and RPG out there… Take a look here “http://www.penpaperpixel.org/tutorials/tabletopprojection/” its the kind of solution we have nowdays and augmented reality is way cool, and maybe cheap.
Hello guys. What are you doing here is one of the most amazing thing…hope you will get the funding and human resources to keep improving it. I was just wondering if you have considered the new snapdragon platform…which seems pretty powerful and decent at 3D rendering. not to mention a pretty big screen. Good luck
Author
We haven’t gotten our hands on any SnapDragon systems yet, but there are a lot of other modern things we could port too as well (including the new Nokias coming out, the Tegra, the iPhone, etc). It’s going to be interesting to see how these sorts of games feel on these more powerful platforms.
Nice blog. I have added it to my favourites, greetings
thank you
aljawalat.com
[…] world. We’ve found this with the two studies we’ve done of tabletop AR games (Art of Defense and Bragfish); one key to those games is that the graphics were tightly aligned with […]
[…] The team at the AEL research lab at Georgia Tech brings us a video of an AR table top game called Art of Defense. […]
[…] in the interview). Curiously enough, a video of one of the games mentioned in the article, “Art of Defense”, was uploaded to Youtube today. It’s an interesting research in how people interact […]
[…] cette vidéo, vous observez Art of Defense (AoD), un jeu de plateau coopératif basé sur la réalité augmentée. Il cumule plusieurs […]