All about the context

All about the context

posted in: Media Theory | 0

Here’s something I don’t understand yet. Does it matter where you watch an immersive video? What’s the difference between drawing with a tool like TiltBrush in a cookie cutter demo space vs in your studio? How do similarities or contrasts between watch space and immersive space change the meaning of the media experience? Basically, does context matter?


My bedroom in your bedroom

The first time I tripped over the idea that immersive media is sensitive to context dependent meaning was in a conversation with collaborator and recent guest blogger Evelyn Eastmond. She mentioned that she’d watched my daily spherical video ‘H is for Husband’ while laying in bed. The video was shot in my bedroom and with Evelyn’s watch context suddenly I realized I could nest my bedroom inside of hers.



Then I started watching spherical videos everywhere I went: A video from the park while on a plane, videos I shoot in the studio while in my studio, watching indoor videos outside and vice versa. I found that for me the most effective were when the video place and the physical place were more similar. It intensified the sensation of presence with textures and smells while also leaving sticky traces of time travel in the physical room. One exception being when I was on a plane, then all I wanted to do was watch outside lovely bird sounds videos to transport me away.

I also made pieces that wanted to be watched only after you had been a particular somewhere. ‘Touch Me’ was shot in a gallery in SFMOMA. The video works ok if you have never been in that room but all the power and strangeness and the ahha moment are dependent on you having been in that room prior to watching the video. It might work the other way around as well, watching the video first then seeing the room, but I have my doubts the video would be as powerful in this order.



So if the context and the viewer’s history with the space was having such strong impact on the reception of the media why not just steal a space entirely and stick it inside itself? For ‘How to Bootleg’ I used a handheld 3D scanner to copy two rooms in SFMOMA, a pair of neighboring galleries housing work from Sol Lewitt and Frank Stella. From those scans I reconstructed a 3D model of the space in Maya and animated a camera walking through the space. This gave me a spherical video moving through the space I could watch on a phone in a Wearality Sky.



Again the video works ok on its own but I have a Vi, my ever willing test subject, and easy access to SFMOMA so I dragged her over there and got her to watch the piece in the space it was made.



With no verbal instruction she does all this cool stuff: follows the camera path with her body, looking back and forth between the video and the space, comparing the two, navigating both the people in the physical space and the one in the virtual space as well.

Immediately after watching the video she reports an intense feeling of awareness of the walls and geometry of the space. When the camera gets closer to a virtual wall then she would get to a physical wall her proximity warnings go off, but when the virtual world slips in orientation to the physical she has to work extra hard to map between the two. The space/media pair challenges her spatial awareness so much she becomes consciously active in her mapping process. Later we also tried it without the wearality sky just using the flat phone screen. Because in this setup you can easily reorient the spherical video it was easy to keep the video oriented correctly to the space which allowed us to, for example, line up the screen with the door way as the camera passed through revealing the lamination between the walls in the model.


HoloLens at Home

Of our pile of immersive hardware the Hololens is of course the most directly interactive with and influenced by the physical space its in. Because it is so portable it allows you to do things like… install holographic graffiti art in a big museum but what if the HoloLens was treated as both the camera and the viewing device?

In “Pretending to be on the Moth” I wore the Hololens and recorded myself improvising a story about my life. I recorded it in my living room which was still sprinkled with the holographic detritus of Neveryday Unnaturals. But instead of just using that video as the finished piece I added it back to the room it was recorded in with all the other holograms. Then I had Vi come over and watch the video in the headset in the room, again nesting spaces inside themselves.



Vive in the Livingroom

Last weekend we installed a Vive in my livingroom. I wasnt expecting a simple thing like domestic context to change the way I thought about it but wow… the difference between using the Vive in the Lab and using it in my own house is huge. Just the added social component of my husband and little brother sitting the couch peanut gallerying my game play via the shared screen of the TV brought it out of the domain of ‘socially separating and loney VR’ and made it warm and communal. Multiplayer games designed for this context, one person in VR and collaborators using a screen with either controllers or mouse and keyboard, will greatly expand the usability of the hardware for home use.

For me this new context meant exploring a new way of making in and relating to my home. While the Hololens was built for this kind of physical and digital influencing each other thing, with a few recording tricks I can bring context into the Vive too.



This video was made using OBS recording from both the Vive headset camera and a window capture of TiltBrush. I used OBS’s live color keying functionality to remove the white background from TiltBrush leaving behind just the drawn lines. I did my best to align them but there is not enough fish eye distortion on the TiltBrush window capture to get it to match the scene exactly but it’s close.

The experiencing of making this piece was amazing. I stood in a completely white room and reached out to touch the physical world with my left hand. I traced along edges I felt and slowly, line by line, the world came into the virtual space to greet me.