Developing Interactive Content for the Holodome – Part 1
May 6, 2019
Over the last few years, Vulcan has been developing the Holodome, an immersive virtual experience system. The project pushed the boundaries of VR/AR technology – and has blazed new trails for creating immersive experience content.

Earlier this year we released Dome of the Dead: Escape the Bayou (DotD), the Holodome’s first official piece of interactive content, . DotD is a 4-person cooperative shooter game and was open to the public to play at MoPOP in Seattle alongside a number of fantastic cinematic pieces, the initial installation closed on April 28th. The game has been very well received and has achieved our highest satisfactions score yet, and everyone on the Holodome team is very proud of it.
 
DotD.jpg
 
Video of the game and Holodome can be seen here.
 
The road to realizing high quality interactive content has been a long one - this will be a multipart blog post to ensure the full story gets told.

I joined Vulcan as a contract developer in August 2017 on a team called The Forge. The Forge is an agile R&D strike team that prototypes, AR/VR apps and other tech to evaluate and ultimately prove / disprove the viability of Paul Allen’s vision of our augmented future. The Forge is great, and I was fortunate to meet Paul while a member. I’d heard about the dome from co-workers who said it was pretty impressive. I was intrigued, but based on my VR background was also skeptical about how impressive it really could be. One day The Forge made the trek, down to Seattle’s Georgetown neighborhood for a demo.
 
This visit changed everything for me, I became a believer that Vulcan really was building a Holodome. After talking to Joe Laurino, Mars Tanumihardja and other Holodome members it was clear The Forge could help take the Holodome to the next level. Soon Shamyl Zakariya, Ajay Desai, myself and others from The Forge found ourselves helping accelerate the Holdome renderer, and realize a viable SDK based on the Unity3D game engine and HTC Vive. These advancements were both necessary to enable real-time interactive content. I was so taken with its capabilities that I spent my entire Christmas 2017 break working with my desk setup inside the Holodome, running as many experiments and content tests as possible. It became clear the Holodome was where I could add the most value and soon joined the team along with Jon McElroy, a graphics guru also from The Forge. A few months later I was recruited to join the Vulcan’s Holodome team full time and accepted. 
 
My job entails more than designing / developing content for the dome, but this is a content blog and content is king. Without content the Holodome is simply a technical novelty, like a Lamborghini with no gas in the tank. It’s cool to look at and peer under the hood, but what everyone really wants to see what all that power and advanced design can do. I’m fortunate to be the guy who pushed the gas pedal. 

Game Mechanics are what it’s all about

Mechanics are the actions you can do in a game. Examples include running, jumping, shooting, pointing, clicking, driving, pushing, pulling, selecting, picking up, putting down, throwing fireballs, powering up, tiger uppercuts, crouching and ducking to name a few. Even playing a wild card, running out of ammo, and moving a chess piece are examples of game mechanics.  
 
Mechanics are the main ingredients, and most important feature, of any game. The foundation of every great game’s success is based primarily on the quality of its game mechanics and how they’re used, combined, and even limited to derive a fun, rewarding, challenging and memorable ‘gameplay’ experience. This holds just as true for Scrabble and Monopoly as it does for Pong and Pac Man, as it does for Grand Theft Auto and Halo as it does for Tilt Brush, Beat Saber and Dance Dance Revolution. There are very few examples of truly amazing, memorable games that were built primarily on something other than great mechanics. Quality story, narrative, graphics and soundtrack etc. are hugely important to many games but are often over emphasized as to what makes a game great. High quality in these areas unquestionably carry specific games to ‘masterpiece’ level, but without great game mechanics these experiences would just be movies, where story, visuals and soundtrack the main course.
 
The Holodome opened to the public at MoPOP on May 4th, 2018 exclusively with cinematic content. By this time Matt Milios, Susan Grella and others in Vulcan Productions had proven that the Holodome could deliver amazing visual and aural cinematic experiences, and convinced many that the Holodome may be the best platform ever devised for immersive storytelling. I made it my life’s goal to try out as many different types of game mechanics as I possibly could, because I believed the Holodome could also be best platform ever created for providing fully immersive, interactive experiences.

Overcoming doubt and preconceived notions

After joining the team, I compiled a matrix of mechanics and game ideas. I can’t begin to count how many times I heard “That doesn’t work in VR” or “That will make people sick” as if the Holodome was just some 6 figure VR headset. There are similarities, but it was my position that the Holodome is something different than ‘ultra premium VR’ and I set out to prove it by testing as many mechanics as possible. Here’s a table of many of the mechanics tested:
 
Player
Movement
Object
Interaction
Shooting
Pointing
Aiming
Positional Detection Multi
player
Horror Audio
Driving / Karts Stacking High accuracy
Rifle
Lazer
Sniper
Move
to specific place in dome
RC Cars Player Vs Player
 
Silhouettes 3D Spatial Audio - Ambionics
Walking / Running Carrying Low accuracy
Flashlight
Shotgun
Flame Thrower
FireHose
Sinking ship - everyone get to one side Cooperative
Game Play
Shadows behind screen Floor Haptics
Motor Simulation
Footsteps
Driving A Tank Pick up / put down.  Various manipulation and rotations modes Fake High Accuracy
Auto Aim
Musical chairs Split screen Jumpscares  
Teleport Pull / Push   Step on tiles in specific order   Simulate ripping the screen
and showing things behind it
 
Swinging /
spiderman
Throwing / Catching       Character watching and following you around the dome
 
 
Jumping Building          
Bouncing Destroying          
Boats            
Flying            
 
Most mechanics worked much better than expected, with locomotion being one of the biggest wins and biggest surprises. As part of this process I built a go cart simulator, a SpiderMan-esque sim where you jump between buildings, a motor boat sim with real-time wave simulation and multiple experiences where you ‘walk’ around using a controller and interact with objects in the environment, to name a few. Barring ‘Room Scale VR’, I believe we’ve shown the Holodome to be the most comfortable immersive platform on the planet. It’s definitely the most comfortable with respect to moving the player within the simulation, something very difficult to do well in VR. In the Holodome, unlike VR, you are grounded in the real world, you can see own body, your companions and projectors and therefore you have something to lock onto while the simulated world is moving. Vehicle locomotion works extremely well when a static platform is projected on the floor for you to stand on, such as the bed of pickup truck, or the deck of a boat, especially when paired with rumbling effects powered by the Holodome’s incredible vibroacoustic haptic floor. Simulated motor sounds that rumble the floor go a long way to make the sensation of being in/on a vehicle feel believable. To be fair it’s very easy to make people feel uncomfortable or sick if you’re not careful. By following a few guidelines Holodome experiences can comfortably move people in ways that VR can’t. Matt and Susan had learned a lot about this while developing Death Planet Rescue and other cinematics, I leveraged that knowledge.

In a nutshell the guidelines for comfortable locomotion are:
  • Limit rotating, tipping and rolling the simulation under people’s feet. 
    • People will naturally turn towards action and areas of interest and positional audio cues
  • When simulating vehicle movement keep the vehicles heading and position fixed within the dome
  • Project fixed platforms under people’s feet when simulating vehicles
  • Limit non-vehicle movement, i.e. ‘walking’, to mostly straight lines
  • Use the vibroacoustic haptic floor to evoke a sensation of movement
Most of the other mechanics worked well, with the notable exception of the ‘Positional Detection’ category. At this point the only positional input available were the HTC Vive Controllers and Trackers. We know exactly where and how these devices are oriented inside the Holodome, and they work great for hand held peripherals such as guns and pointing devices. Sadly they’ve proven to be unreliable proxies for tracking a person’s position, let alone pose. I’ve strapped trackers to people’s shoulders, hands and feet and had player move side to side to keep a boat upright, or move between tiles projected onto the floor in a specific sequence. i.e dancing / musical chairs. These didn’t work very well due to technical challenges. Fortunately, on the occasions these tests did work they were compelling enough to show that mechanics involving a player’s actual position and pose merits further investment in technologies such as computer vision, depth cameras and machine learning, and investment in these types of inputs are currently happening.
 
This post focused primarily on the nuts and bolts and creation of the Holodome’s new interactive features and how we convinced ourselves of their potential to enable quality games and experiences. We hoped you’ve enjoyed it!  In an upcoming post I will detail the multiple mini-games and technology advancements we designed and developed along the way to Dome of the Dead as well as a post on the design and development of the game itself.  Stay tuned!
About the Author
Patrick A.
Senior Software Engineer
Patrick is Interactive Content Lead for Vulcan’s Holodome, where leads creative and technical design for multiple projects including cooperative shooter game ‘Dome of the Dead’ as well as immersive interactive content for enterprise partners.  Patrick also leads development of the Holodome SDK, which is used to power all interactive Holodome experiences.  Prior to joining Vulcan Patrick was technical and creative lead behind multiple critically acclaimed games and VR products including GrooVR, Camp Pokemon and Republique, and also worked on advanced AR and VR hardware and software including Microsoft’s Hololens.  Patrick spent 9 years at Microsoft Game Studios / Xbox as an original member of the Kinect team, engineer on advanced Xbox projects in Microsoft Research, and contributor to over a dozen game franchises, including Halo, Age of Empires, Flight Simulator and more.

Category Tags
AR/VR
Holodome
Immersive Reality
OnlyAtVulcan
About the Author
Patrick A.
Senior Software Engineer
Patrick is Interactive Content Lead for Vulcan’s Holodome, where leads creative and technical design for multiple projects including cooperative shooter game ‘Dome of the Dead’ as well as immersive interactive content for enterprise partners.  Patrick also leads development of the Holodome SDK, which is used to power all interactive Holodome experiences.  Prior to joining Vulcan Patrick was technical and creative lead behind multiple critically acclaimed games and VR products including GrooVR, Camp Pokemon and Republique, and also worked on advanced AR and VR hardware and software including Microsoft’s Hololens.  Patrick spent 9 years at Microsoft Game Studios / Xbox as an original member of the Kinect team, engineer on advanced Xbox projects in Microsoft Research, and contributor to over a dozen game franchises, including Halo, Age of Empires, Flight Simulator and more.

Category Tags
AR/VR
Holodome
Immersive Reality
OnlyAtVulcan
Build a better future
Working at Vulcan