By merging graphics with props in the physical world, handheld Augmented Reality games pull the player through the small screen and into a larger merged play-space.

Our primary motivation for this AR game was to explore fast-action first-person augmented reality, where the camera controls and movement that would typically require a mouse and keyboard are handled directly by simply moving the device.

The tracking technology used here does not require ARToolKit-like black and white patterns as most tracking libraries do. Instead any picture with enough feature points can be used for tracking. For this particular game we used a city map (left image) as a tracking target as well as base graphics for our game. We designed the map as a star-shaped road junction centered on a fountain that plays a major role in our gameplay.

This advanced tracking technology allows the player to quickly zoom in and out and view the world at steep angles, making this a highly interactive and engaging game.

In this game, you find yourself in a helicopter flying over a city infested with zombies. The player has to move around the map to target a zombie and shoot it. If the player stays static for too long, zombies will start throwing things at him. That provides a strong incentive to keep moving.

The player also has to protect the civilians who are trying to get to the safe zone located at the middle of the map. A zombie will instantly attack any civilian that comes into his sight and kill him before he gets to the portal.

Still following our goal to merge the augmented and the real world even more, we decided to use real objects to interact with the augmented game. The player can use colored sweets Skittles as bombs in the game by simply placing it on the map and shoot it.

Orange Skittles act like proximity bombs which explode as soon as a zombie or a civilian gets close to it. Green bombs need to be armed before use, after a few seconds shoot it to set it off.

Proximity bombs are easier to manage (only one shoot necessary) but can't be deactivated and can therefore easily kill a civilian who gets in the way. Green bombs offer more control but requires more attention from the user.

We leverage the graphics capabilities of the nVidia Tegra (the first of his kind in 2009) for things such as dynamic lighting, vertex skinning and particle systems computed on the GPU.

The project became the topic of a talk given at GDC 2009, and was part of the video program at IEEE VR2010. It also appeared in most of the popular tech news websites (Engadget, Slashdot, Reddit...) and was featured in a TV program on BBC news (second video at 2:20).

Download the video

English Francais