The Next-Generation Gaming Experience is immersive, social, mobile, and physical. It raises a number of technology challenges to make it real, but researchers love a challenge! Here is our view of the next-gen gaming experience and a few projects that our researchers are working on to get us there. We showed them at the HP Gaming Summit last week in San Francisco.
There are many press articles and blog posts on the gaming summit. Kate Greene of Technology Review did a great job capturing my thoughts in this article.
Full disclosure: I manage some of these projects and I think they're cool!
The Next-Generation Gaming Experience
Immersive visual experience with pixels anywhere and pixels everywhere.
 Pixels anywhere- including on walls, on floors, on ceilings, on your watch, and on table tops. When pixels are on your coffee table, there is a technology challenge of creating new interaction models to share, control, and interact. When many people are looking at an upright display, there is a shared view of what is up-down-left-right. When many people are sitting around a coffee table, this assumption needs to be revisited, since each person sitting around the table has their own view of up-down-left-right. There is a challenge in inventing new interfaces and interaction models to make sure everyone around the table gets a first class experience. And, there is a challenge in designing applications that work well in this experience.
We showed Misto, our research project that has a touch-screen display embedded into a coffee table. It has applications such as solving jigsaw puzzles, web browsing (including Google earth fly-by's), and photo sharing, where you can slide photos across the table and spin them to be upright for each person.
 Pixels everywhere means that we have flexible pixels, on flat and curved surfaces, with any shape and size. Why limit yourself to a 4:3 or 16:9 aspect ratios? You can use many displays along with real-time video processing algorithms to make displays of any shape, size, and quality. The challenge comes from making many displays, such as projectors, look like one seamless display. Since each individual display has different display characteristics, you need to do some real-time video processing to adapt the color and geometry of the rendered pixels to produce one seamless display.
We showed Panoply, our research project that uses multiple projectors pointed side-by-side on a curved screen to give you an immersive visual experience that covers your entire field of view. We use real-time photometric and geometric calibration algorithms to create one seamless immersive display.
We also showed Pluribus, our research project that uses multiple projectors pointing at one large screen to give you a better-than-theater visual experience. (At the gaming summit we showed Pluribus with 12 projectors, but you can use any number.) We use real-time video processing algorithms to modify the pixels so that they add up to one super-projector, one that has higher image quality in terms of brightness, contrast, resolution, and color quality.
Social experience. It's a social experience that brings people together and forms communities, with a broader audience than what we have today (maybe including me!). Technologies such as motion sensors are making games easier and more intuitive to play, which will expand the gaming demographics. Also, games will be more integrated with communications such as instant messaging, chatrooms, voice, and video. The challenges lie in integrating these different modalities into one session in the context of a game, and making sure the network priotizes the different types of traffic accordingly. For example, you need very quick response for the game controls, but could delay a half second on a chat message.
Mobile experience. It's a mobile experience that that lets people play games together on any network, on any device, anywhere in the world. By mobile, we don't only mean playing a game on your cell phone. We mean that people are mobile- you will game on your TV, on your computer, on your portable player, or on your cell phone- but you still want to have your gaming experience wherever you are. Of course, your experience will be different depending on your device and your connectivity, so your gaming experience may be different in each situation, but it should still be fun!
Challenges come from rendering the game on any device through any network. It is useful to consider different device-network paradigms. For example, if there is enough network bandwidth you could go to an in-network rendering model, where the players view is rendered on a machine in the network, and the resulting video is streamed to the player through a regular video streaming connection. This way, the device only needs to decode a video stream (e.g., an MPEG) rather than have full graphics rendering capabilities. On the other hand, if your device has plenty of graphics capabilities but little network bandwidth, you could go to mode where you only send commands over the network but render the game on the device itself.
Physical experience that maps the physical and virtul worlds. It senses your physical context and triggers experiences accordingly. Your physical context can be your location, your movements, and even your heartrate or who you are next to. Based on your context, it then triggers a multmedia experience. Technology challenges lie in appropriately sensing your context, which requires having sensors that are accurate and power-efficient. Challenges also lie in developing applications and experiences that use context.
We showed Mscape, a.k.a. Mediascapes, our research project aimed at creating and sharing context-aware multimedia experiences. We've had Mediascapes deployed around the world, most recently at the Tower of London and Yosemite National Park.
What is your view of the next-gen gaming experience?
What technologies and projects are there out there that will make it real? (include URLs)
Tags: gaming, next generation gaming experience, HP Labs, VooDoo, HP Gaming Summit, misto, panoply, pluribus, mscape, mediascape, HP
Feel free to include a URL in your comments. |
Printable version
