Natural Language and VR Programming

Natural Language and VR Programming

posted in: webVR | 0

Hi everyone!  This is Evelyn Eastmond again, now in regular rotation at the eleVR blog.  I’d like to share with you a cool tool that I recently used to generate simple VR content.

GuriVR

As part of my exploration into the current state of VR applications for creativity, I have been tentatively interested in available VR programming toolkits. I say tentatively because, as a former software developer, I am weary of delving into early-stage development frameworks that are unstable, untested or undocumented. These days, I’m not very interested in the frustrating details and setbacks of an emerging programming framework. Instead, I am more interested in what that framework can offer me in terms of creativity and flexibility. I haven’t yet programmed a VR website, but I was hoping to learn how to do that as easily and seamlessly as possible.

A few weeks ago, our friend Nicky Case shared a project with me called GuriVR, a VR content generator geared towards journalists. GuriVR was created by Dan Zajdband, an OpenNews fellow at the Knight Foundation/Mozilla, with the hopes of bringing VR content publishing to non-programmers in the journalism world. To that end, he made a programming tool that lets you create time-based, 360-degree web content using natural language as the scripting interface. Instead of dealing with JavaScript libraries, syntax or obscure calculations, you simply write what you’d like to see in your scene.  The final VR creations are called “stories”, made up of “scenes”. Here are some examples of how story creation in GuriVR works:

 

VR Scene Generation in GuriVR

 

An example story in GuriVR with one scene:
This is my first scene. It lasts 10 seconds, has a skyblue background and shows text saying "Hello world"

 

Click on this image to launch the story in a website:

I was excited to see that I could create an immersive website without looking up JavaScript commands.  GuriVR even lets you create multiple scenes that display one after the other.

 

Here is another story, with two scenes:

My first scene lasts 5 seconds, has a skyblue background and has text that says "Hello world".
My second scene lasts 7 seconds, has a pink background, and has text that says "Look over here!" behind me.

 

Click on this image to launch the story in a website (and remember to turn your head to see behind you):

screen-shot-2016-09-26-at-3-26-38-pm

 

VR Scene Generation in GuriVR in Spanish

After playing around some more, I noticed that Dan also offers the tool with natural language processing of Spanish commands!  Being a native Spanish speaker, and having worked on multilingual programming capabilities (for Scratch) in the past, I was eager to test out generating VR stories in Spanish.

As part of my personal artistic explorations, I am trying to learn more about my cultural background.  I’m interested in the socio-political history of Colombia, especially in the topics of interpersonal violence and trauma.  I’m beginning that line of research by reading a well known literary work from Colombia called ‘Cien Ańos de Soledad’ or ‘100 Years of Solitude’ by Gabriel García Márquez.  In order to test GuriVR in Spanish, I decided to make VR scenes based on passages from that book.  Within the first two pages, I found two quotes that seemed fitting for experimental VR content.  The quotes are referring to an ocular, magnifying lens contraption given to one of the characters:

 

La ciencia ha eliminado las distancias.
Dentro de poco, el hombre podrá ver lo que ocurre en cualquier lugar de la tierra.

 

Loosely translated, these sentences are about the great affordances of this contraption:

 

Science has eliminated distances.
Soon, man will be able to see what happens anywhere on earth.

 

I like how these quotes remind me of what continues to attract me to digital technology: the Internet and accessibility to media creation tools and devices.  So, I decided to create a GuriVR story that shows these quotes, but this time, by writing the scripts in Spanish:

Mi primera escena dura 5 segundos, tiene un fondo #87CEEB y muestra un texto que dice "La ciencia ha eliminado las distancias.".
Mi segunda escena dura 7 segundos, tiene un fondo #FFC0CB, y muestra un texto que dice
"Dentro de poco, el hombre podra ver lo que ocurre en cualquier lugar de la tierra." atrás de mi.

 

Click on this image to launch the story in a website:
screen-shot-2016-09-26-at-3-55-47-pm

 

The scripting works just as well in Spanish as it did in English.  Some hexadecimal color codes were needed numerically instead of “naturally”, since presumably HTML color codes are possible only in English, RGB or Hex (but why is that, I wonder?).

At one point when creating the above script for my Spanish VR scenes, I noticed that the story worked fine even if I combined English and Spanish commands in a single script.  All of a sudden, I was programming in Spanglish!  The idea had never occurred to me, and the program I was writing was now not only natural language, but even in hybrid language.

 

Just for fun, here is what that script looked like:

Mi primera escena dura 5 segundos, has a skyblue background y muestra un texto that says "La ciencia ha eliminado las distancias."

 

I’ve never programmed a computer in Spanglish before, I’m not sure if anyone has!  It felt very reassuring, somehow.

 

Cien Ańos de Soledad en Colombia

GuriVR has many more features that I didn’t mention in this post, including addition of panoramic background images, audio playback, and many other features.  I recommend checking out the guide page if you are curious for more. For my Colombian VR story, I found an equirectangular image from Zipaquira to place behind the quotes.

 

Click below to see my GuriVR story in progress:

screen-shot-2016-09-26-at-4-54-37-pm

 

I look forward to testing 3D object loading into my stories, among many other of GuriVR’s features.  I’ll make another post in the future when I make more progress.  For now, check out GuriVR for yourself!