I love cartoons. My Little Pony, Gravity Falls, Adventure Time, Bob’s Burgers, Steven Universe, Legend of Korra all delicious. So because I am a cartoon nerd and because every single immersive animation I have seen is 3D, I developed a technique to make 2D animation as immersive video.
First go watch (and listen) to the cutest little short about dreaming ever: Suki Sleeping. This is a stereo video (that’s kinda the whole point) so if you can watch it on your handy dandy headset.
I know, that is some high potency adorableness right? Just let it all soak in. Its good for you, cause science.
Ok so first I’ll walk you step by step through how Suki Sleeping was made and how you can do this with you next animation project then we’ll talk about a tool that will need to be built to facilitate this kind of project on a larger scale.
First build up your materials.
Animate a flat character doing a thing. Draw/collage an awesome place for you character to live. Just keep each piece, each chair or tree or door or character separate from each other and on transparent backgrounds.
Make a sequence in Premiere that is the full size of the finished product. I went with 4k x 2K because it makes the arithmetic easier later on. Keep in mind that if you choose a 2 x 1 sequence like I did you will need to divide the height of each picture element (like a tree or a character) by 2 while keeping the same width. This is because we’ll be squishing it into half the frame. Alternatively you can make your sequence a 1 x 1 and skip this step. Just depends on how big you want your file sizes to be.
Now you need a depth measuring grid because we want the final product to have stereo depth. This is essentially just an arbitrary but evenly spaced set of lines that are numbered lowest to highest top to bottom. The grid helps you organize objects at different depths. The top line should be placed at the horizon and the height, but not the width, should be adjusted to fit your scene. Here’s the one I used:
Now in the bottom half of the sequence you are going to arrange all of your elements. It is really useful to do the layout phase with the measuring grid visible like this:
Since you are doing this in a spherically unaware editor: avoid the poles. The closer you place things to the top or bottom the more distorted they will be. One method to get around this problem can be seen in the veranda scene in Suki Sleeping. I traced over an equirectangular photograph so my drawing would be in the correct projection. This is an easy stop gap technique until spherically aware animation software is a thing.
Once you have a whole scene arranged and you have all the characters movements in the scene animated its time for the tedious part. Select everything in the scene and duplicate it. Your timeline should look like the one below with both copies of the stereo measure at the very top and two identical chunks of clips below. It is fine to instead nest your scene and work with two copies of that nest but I found this to be more annoying than just having everything in the same sequence.
First step is to move all of the copied clips and images to the top half of the video. This is where that easy arithmetic lives. Under video effects, subtract 1/2 of the total height of the video from the y position of each clip.
Then add the stereo measure to the x position of the same clip. For example take the pineapple in the image below. The bottom edge of the pineapple is on the 30 line.
So we need to add 30 to the top pineapples x position.
Simple! Now just do that for every single thing you put in the scene! You should end up with some thing like this. You can get even fancier by animating the disparity between the two copies to make something move forward or backward in the scene.
This is a super time consuming process. Suki Sleeping took weeks to make and while I would be much faster next time as I wouldn’t need to also invent the entire process from scratch, it makes more sense to just have a better tool. There are tons of great 2D animation tools out there: Synfig if you want to go open source, Toom Boom and Anime Studio if not, but none of these do either spherical or stereo.
In a real immersive stereo 2D animation tool would have users do composition work not on a screen but in a head mounted display so the stereo can be seen as they go. In the headset the animator would have a bin of materials to composite, a bucket of trees characters etc to place. In the center of the virtual working space would be a stage on which these elements could be placed. The animator would have the ability to see from either the center of the stage, the spot from which the video will be rendered, or be able to walk around the stage seeing things from the outside. They would be back stage. Once a tree for example is placed the animator would be able to bring it closer or farther away with minimal effort.
This would remove the entire duplication phase and instead generate the stereo effect with a combination of built in stereo rules and depth drawing tools to let users manipulate those rules. Stereo rules would be things the software automatically handles like: farther away objects have reduced disparity, a tall object’s disparity changes with height, and an object moving from one depth to another has tweening disparity etc. Each image/clip/piece of the animation would have one float included in it’s standard effects called distance. This would set the base distance for any clip and could be animated over time.
As for drawing type tools: it would need intra-plane distance definitions, for example the veranda in the above image is actually three images that each have their own stereo disparity. This wouldn’t be necessary in a system that simply allows you to change the disparity of an image using close and far point pins or hot spots. Eventually even animating this disparity as well so a character could seem to reach back for a mug behind them.
This is just a sketch of a tool to bring 2D animation into immersive platforms but its pointing in the right direction. If you try out this animation technique or dive into the animation software making business I love to hear about your adventures. Find me on Twitter @emilyeifler and happy stereoing!