Editing spherical 3D in Premiere and After Effects, A Design Document

Editing spherical 3D in Premiere and After Effects, A Design Document

posted in: Cinematography and Editing | 0

We are interested in working with a plug-in developer to create a plug-in for working with virtual reality video in Premiere and After Effects so I put together this little walk though to describe what we need. Here goes the excitement!

The plug-in needs two distinct branches of functionality: an editor and a viewer. Both branches will need to work with two formats: a 2D spherical video or a Top Bottom stereoscopic spherical video (left eye on top). Both of these formats are exclusively 1×2. (We currently use the equirectangular projection for all our videos, it is defined below.)

image04

Typical 2D spherical equirectangular video as seen in standard playback

image03

Typical top bottom stereoscopic spherical equirectangular video as seen in standard playback

Branch time!

The Viewer:

One of the difficult parts of editing video for VR is not being able to see the results until after exporting the video out. The viewer would use Mercury Transmit to make playback visible in the Oculus Rift (and hopefully eventually other HMDs as well). This would mean taking head tracking information from the HMD and integrating it with playback to allow me to look around inside the sphere of video I am working on in AE or Premiere.

The Editor:

Now that I can see the video as a projected sphere instead of just a warped whole I need to be able to edit titles and effects in the same projection as the videos. Currently a title in Premiere, for example, stays the say size and skew no matter where in the video frame you put it. This works great if your video is flat but I need a deformation mesh to distort the shape and size of the title as I move it across the frame. Here is the current projection we use and an example of how this would look.

 

What stuff means time!

Definition: Equirectangular Projection (with pretty pictures!)

An equirectangular projection is a method of mapping a sphere on to a flat plane. It transforms both the meridians of the sphere to vertical straight lines with constant spacing and transforms the spheres latitudes to horizontal straight lines of constant spacing. This is commonly seen on maps of Earth.

image02

To maintain this alignment the surface of the sphere is warped, maintaining its original aspect ratio near the equator and smearing horizontally as it approached the poles.  The circles on the map below are an example of how the titles effects etc would change depending on their placement.

image00

The deformation can be seen more clearly with larger shapes such as this area of coverage map of an orbiting satellite. As the shape approaches the pole the horizontal stretch is increased.

image01

Definition: Equirectangular Projection (Again, but with math!)

Given a spherical model

x = λcosφ1

y = φ

 

where

λ is the longitude;

φ is the latitude;

φ1 are the standard parallels (north and south of the equator) where the scale of the projection is true;

x is the horizontal position along the map;

y is the vertical position along the map.

The point (0,0) is at the center of the resulting projection.

 

If you or someone you know is interested in working on this project shoot us an email! We can compensate you for your time but you should know that we are a research group, not a production company, so the plug ins and well as all associated source code will also be released online open source. All contributors will of course be credited for their super extra awesome contribution!

-Emily