It was clear as soon as we saw the first DK2 demos that VR video wanted to somehow be integrated into a 3d environment with interactivity and head tracking. So in celebration of the DK2’s release, and the frankly convincing way it tracks the subtle side to side motion of a little lean or a curious look under that virtual table, we collaborated on a shiny new interactive video experience with Christopher Hart, game developer and VR bubble builder extraordinaire.
Come see our new world, or at least a place that has lots of worlds, each a planet, a hovering orb of permeable video like the atmosphere of a gas giant. Explore a new style of storytelling in this vast black space spotted with spheres of video. Hunt each new bubble and notice the forced perspective with some bubbles looming huge in front of you or appearing like tiny specks until you cross the threshold and pass inside, only to discover an entire world inside fit just to your size. Other bubbles may look just right from the outside but once you enter your perspective shifts and you rise giant-like through the lush green garden within.
This is our first step toward a large unexplored field of combined spaces that use rendered environments and real-world capture. We’re working towards:
Stories meant to be experienced in disparate chunks, letting people wander through the timeline of a narrative like one might wander through an unknown neighborhood on a foggy evening.
VR experiences with hand controls in which you spoon bulbs of video from your morning cereal, and when you, or at least your avatar, swallows them whole, the captured world swells around you trapping you inside until the drug of that little pill wears off, fading away, and returning you to the real rendered world.
Virtual museums where gazing at an object can bring up a video bubble with actual footage and more information on the history and context of its origins.
Virtual topographic maps you can roam around on, with immersive video bubbles of real footage from specific locations.
And of course, bubble-bubbles, underwater 😀
As always, our little prototype taught us a lot:
The size of the bubble matters surprisingly little, once you are inside of it. What matters is where you are within the bubble. Being at the bottom of a small bubble makes the bubble feel bigger than if you are in the middle of a large bubble. Head-tracking is awesome for letting you change your height and location in an intuitive way.
When you import a video asset into Unity, it looks like it’s crashing, but really you just need to wait a half an hour. Also, exporting the video in quicktime format seems to work best.
Bubbles-within-bubbles are super cool, and the layers add to the immersion factor.
We tried a prototype where bubbles are combined with a 3D virtual world, and the combination is really compelling (we can’t actually share it due to use of 3rd party material, but hopefully soonish!).
And of course, alongside our learning we are also running in to yet-unsolved challenges:
Getting the frame rate to work properly without juddering the entire world.
Getting different video textures to render for each eye, for stereo video.
Importing large high-quality h.264-encoded video files into Unity.
We’re gonna have to wait a bit before we work out all our own issues for licensing other people’s stuff and working on more demos, but in the mean time, if you’re developing for VR, we hope you’ll try downloading some of our creative commons spherical videos and putting them in your own virtual world to share. Head over to Chris’ site to download this piece and have fun!