posted in: Media Theory | 0

Hyperlinks as object videos

“There is no end to the game.” Playern

Play/Room is a library, a database, an art installation, a Youtube channel, a memory palace and an artist’s studio. Play/Room is a room-scale mixed reality installation in which every physical object is linked/related to a spherical video. The links are printed and attached to each object and can be inspected using a handheld screen. Each tag also has the title of the video printed below the code. When an object’s code is inspected by a player the associated spherical videos show up on screen. The view point can be changed both by moving the screen and swiping with one finger.

This is an example of mixed or hybrid reality, meaning partially simulated, two overlapping, as opposed to adjacent, places. The seam, the API, between these two poorly compatible worlds, being a 3 tap clunky code reader and a web based spherical video player.

Play/Room started with a simple idea: when the web is spatial, when we are used to navigating networked VR, where do my videos want to live? And over the course of its one month existence I got to watch 13 individual play-throughs and one 3 person multiplayer run.



Here’s everything I learned making, thinking, and researching about this project. First I will cover some current models for interacting with groups of content online, then with that context I will talk about the specifics of the structure of Play/Room covering the types of object/video relationships that appeared in the piece, types of interactions observed during playtesting including trace leaving and the effects of visible titles on player choice, and the three main places conflict and frustration arose for players. All the videos from the setup and playtesting phases can be seen on the Play/Room playlist.


The Past

Search, surf, feed and playlist

“This is nothing like links on a screen.” Playern

I am bored with search and surf, feed and playlist. So bored. Homogeny of interface styles is like literally the worst. Google, Baidu, Yahoo, Amazon, Wikipedia, and Qq are primarily search based at the start of interactions (google “red panda,” click on more images) then switch to surfing, whiling your way through different pages via direct hyperlinking (256 pictures of adorable red pandas later…). That was all there was before RSS: a search field and a link and a huge pile of bookmarks of places you have already been and know you need to get back to. Then came today’s giants: Facebook, Youtube, and Twitter. These behemoths have some of that old school search and link quality about them but also rely heavily on feeds, reverse chronological lists of available content derived from good old fashioned mail bags: newest on top. Feeds, first as rss feeds, developed online to give users seamless access to frequently updated content without having to check all their favorite sites individually. They meant you wouldn’t miss out on something great, cause FOMO am I right.

But that built in anxiety, the catered too fear of the missed opportunity, of being out of touch, of being left behind has led to a boredom of its own. I sit on the train everyday watching people swipe up, swipe up, swipe up, swipe up, swipe up, swipe up. Their feeds filled with tiny rectangular images and strips of text. A brief glimpse of an acquaintance’s birthday party, above a mood gif from a favorite band, above a new podcast episode listen request from a prominent new writer, above a new hashtag campaign, above a baby picture from a total stranger, above a musing from a celebrity, above a breaking news story that might be related to that hashtag from before, above a video from a producer you followed once but rarely watch and on and on. Each creeping up to the top of the little hand held screen then disappearing, slipping below the surface and out of sight. Depending on how many outputs you follow the pile often grows unmanageable and is being added to so quickly it is inexhaustible.

But you are always missing out on something. Get over it.

The one externally curated model currently in wide use today is the playlist. Collected by an artist or user generated, played sequentially or shuffled, playlists remove newness as the primary selection factor but still limit interaction to ordered lists.

So what comes after search and surf, feed and playlist? Same as what came before: Places.


Organizing physical stuff in physical places.

“How do places work again?” Me, just now.

Take books. Books are great. People love organizing books. They make information browseable and thus non-linear and then we stick them on bookshelves which increases the density of information even further and facilitates inter-book browsing.  Then we stick those in libraries: stacks of bookshelves, which are stacks of books. You can pull them down and read them and stick them back somewhere new and everything works great until you have lots and lots of books and everyone is putting them back all higgledy piggledy. Then we have to make up book putting back science like the Universal Decimal Classification. It is a fantastic organization system if your goals are consistency and coverage across all branches of human knowledge. Classes of knowledge are categorized by number with longer numbers being associated with rising specificity of the classification. Meaning 5 is Mathematics and Natural Science while 539.120.2 is Symmetries in Quantum Physics.

All of this spatial organizing increases the findableness of the books but also fixes them in static relationships to one another. This is kind of ok though because books are unfortunately already restricted by their utilitarian form. If I could make you books that bounced like a soccer ball, or yipped like a tiny dog on a short leash, or hung the air like a soap bubbles until you popped them and their pages coalesced from the shattered iridescent skin I would totally do that but alas the physical world is kinda a stickler for physics and consistency and shit. Libraries are fantastic homes for books. The strictness in spatial organization methods of books fits their form, but without that restriction how we organize space loosens up.


Object/video relationships

“Everything’s a ladleful of spacetime in here.” Playern


Now we have ourselves some context, back to Play/Room. While unplanned the object/video pairs conformed to four general categories. They are listed from the most concrete, realistic and recognizable to the most abstract and conceptual.

  • Literal: A plastic skull, a toy train car, a pink shawl and Tumblr hat, a laser cut wood mesh, all objects that were in the video they linked to. This was the most obvious type of connection.
  • Diagrammatic: The main example was a scale model of my house with tags positioned where the videos had actually been shot. It diagrammed a specific systematic, in this case geographical, relationship between the videos linked.
  • Representational: Similar to the diagrammatic if more metaphorical these objects included a pair of teddy bears that stored a pair of videos of my husband and I, a t-shirt from a conference where a group of videos were shot, printouts of gallery websites linked to videos of visiting those galleries etc. These objects represented what they contained but did not appear in the video directly nor diagram any systematic relationship between them.
  • Conceptual: Time intensive paper sculptures linked to time intensive capital A art videos. Flimsy paper sculptures linked to less time intensive partially successful video experiments. Crumpled print outs of video screenshots tossed in a waste paper basket linked to the the videos that were utter failures. This category focused on quality similarities across different mediums.

The combination of all these different types of object video relationships was part of what made Play/Room so flexible for lots of different kinds of exploration. Let’s talk about that next.




Modes of interaction

“I want to do chaos to it.” Playern

The most surprising thing about watching my playtesters was the sheer variety of approaches players had to the room. Turns out spacial arrangements are super flexible.

While watching videos players sat on the floor or the poof, knelt, stood, paced, laid on the floor, danced around, and even played along on the ukulele that linked to several music videos. Several players even set down the handheld and wandered the room on their own first. Stacking the paper sculptures in various piles, pouring out the trash, or putting on the clothes before engaging with the links. But it wasn’t just posture or trajectory that varied widely from player to player, organization methods were all over the place. One players took special care, after inspecting an object’s link, to line it up with the others she’d finished in a straight line beside her on the floor like her own physical playlist. Another simply spread things out haphazardly as he moved from object to object. One refused to move anything calling it meaningless, while the next tossed all seen object videos in a pile in the corner. Another player posed various still lives around the room with object videos she felt went together, the paper fish head with the skull, the glass and the bottle and the spoon and the toy train all together under the stool, the bears went the gallery print out. As the playtests proceeded I learned that the more visibly organized the room was the more players pieced delicately though the objects touching as little as possible while visibly messy arrangements gave players permission to contribute to the mess.


Players often multitasked, propping the handheld up next to them, occasionally swiping to a new viewpoint, while they left their hands wander: picking up and feeling various objects, playing with playdoh, putting hats on bears, balancing piles of sculptures, posing still lives of emoji cards and dishes and dresses and the screen itself. Making and playing in a relaxed unfocused multi-modal state. The objects gave tactile feedback to wandering fingers while eyes and ears were trained on a video environment. The ceramic island object, linked to a video tour of a friends studio called “Ken’s Ceramic Studio,” was particularly enticing for fingers with nothing to do while other senses were occupied. In this case the object and the video environment have a literal link and together give the player to simultaneous entry points to the same world.

Two players made a game of discovering hidden tags. The tag on the bottom of a glass, on a tucked away power outlet, and the inside of the stool leg were all particularly engaging to these players. Three of the players took selfies, some with the handheld itself and some with the spherical camera that was included as an object video. One highlight from the multi user test was that each user had their own handheld and they would each scan a different object then lay the screens on the floor, collaging them together, using swipe to navigate to different views.

The most consistent note from all the playtests was that players rarely choose to use the higher immersion clip on headset option, a Wearality sky, when watching videos. Though initially interested players quickly abandoned it once they realized it slowed down the already clunky process of scanning the tags. At least half the players did return to it once when they found a video they wanted to experience more immersively then set it aside again.

Despite all this variance two fuzzy categories did seemed to emerge: long watches of few videos vs short watchers of many. Where as long watchers tended to make up their own decision criteria for navigating from object to object short watchers tended to want more structure, more guidance for a “good” trajectory through the installation. I could see these players enjoying a guided tour if a similar piece was installed in a museum context.



Effect of titles

“Wait, they have titles?”Playern

Titles were an unexpectedly polarizing aspect of Play/Room. For one player in particular the titles more than the objects themselves caught his eye and led him to choose which tag to scan. However the exact opposite was true for at least 3 other players, each of whom commented during their playtest that they hadn’t even noticed that the tags had titles at all. In the case of the vidcon shirt which was plastered with many tags, the tag titled “Sausage legs at Vidcon” was the most often inspected by players.

For abstract and representational style links the titles helped players ground the objects. The title is a top down classifying force and informational guide and while they seemed to support object oriented discovery for those who do not easily engage with the object directly, in some cases however the presence of language seemed to completely override the communication being done by the objects themselves rendering the room just an awkward unordered list. But even with this down side, ultimately the titles contributed to interactive flexibility.


Leaving Traces

“I want to leave a thing in the world for others to find and I want to find secrets they leave for me.” Playern

Players wanted to contribute little notes or their own paper sculptures for others to find or to leave traces that would let another player know what a certain video had made them think of. These types of contributions work well in the lab but I am unsure how that behavior would change outside our little sheltered cove. Texted based online trace leaving quickly devolves into bullying and all manner of discrimination but there is evidence for harmless object based trace leaving in art contexts. Sarah Sze, an American artist who makes large scale installations from ordinary objects, spoke in a 2012 interview with PBS’s Art 21 that museum visitors often left small objects from their pockets, like paper clips and coins, in her installations. The left behind objects may have been anonymous and even entirely invisible to other visitors as not part of the original piece but people still felt compelled to leave them. I will be exploring socially constructive ways of engaging this impulse in future projects.


Failures and Frustrations

“When an object is not literally linked to the video I think I am missing something.” Playern

There were three main pain points for players: Play/Room didn’t effectively teach people how to play with it, some players were disconnected from touching or moving the objects at all, and despite my expectations, many players struggled with Play/Room’s lack of clear structure.

Let’s take these in turn.

Players were given very few instructions upon entering other than a brief tutorial on how to use the scanning software. This meant it was not obvious to players what was and was not allowed. Many of the objects seemed fragile and were thus not considered play objects. In a completely virtual environment players would likely explore knowing they couldn’t actually damage anything, but in a mixed reality context players have the understandable fear that they will do it wrong and cause some irreversible damage to clearly unique objects. I’m not sure this is avoidable unless I used only objects that can be easily replaced if damaged or if installed in a public context place all tags in plain view so nothing is actually touched in the process of viewing. But eww, all looking and no touching is just eww.

The second hurdle to seamless use several players mentioned centered around the meaning of moving an object. “I would move things around but it doesn’t seem to mean anything,” said one player who also later mentioned a desire to see his trajectory laid out behind him showing which he had already watched. To my consternation, this player did not see the ability to move things as equivalent to the desired functionality. This type of spacial arrangement may never work seamlessly for some players or it may just be an audience behavior that needs to be taught more directly. I don’t know yet.




Lastly about a third of players wanted more structure than was provided. This came in two forms. First, players who wanted all object video relationships to be the literal type, and second, players who wanted the organization of the room and the meaning of the videos to have an overarching cohesive narrative structure.  Dissatisfaction with the link types because of feelings of missing something was justified. Players who focused on literal type links were missing something. One player framed it like this: “With books, I can tell what I am about to read when I see the cover.” And while I disagree with the judging books by their covers I think it points to a framing problem that could be solved with different environmental expectation setting. If installed in an art museum for example players would likely be better primed for a spectrum of link types. It should be noted that for these players the model of my house was the most satisfying object to inspect.

The latter group were less frustrated by any individual link than the lack of underlying discoverable narrative. The games Her Story and Gone Home were both mentioned as what players expected Play/Room to be like. (Her Story is a game in which you sift through unordered library of old police interview video files to discover a sordid tale of your family’s past. Gone home is game in which you wander your families unoccupied house examining ordinary household objects to figure out what’s been going on since you left.) Sure, these players were happy to wander the room watching videos and examining objects, but they wanted it to all mean something, to have some clear purpose, something to give their explorations focus. This is a need I hadn’t even considered when building Play/Room. It was designed to be a growable libraryesk place but there’s no reason an installation structurally identical Play/Room couldn’t fulfil this itch. Layering time also would be a very effective use of Play/Room style linking. Immersive theatre perhaps in which actors currently inhabit the space at one point in time but the space lets you look backward and forward to other scenes that happen in the same space.



Now what?

“It gives you privileged access to a private space.” Playern

I’m starting to realize that spherical videos are somewhere between an thing and a place. Like globes spherical videos let you see context instead of cropped portions of the world dissected. So where should they live? Where would they feel at home? Somewhere you could touch them like globes, somewhere you could feel the ridges of their mountains. Metaphorically or whatever.

A place you feel comfortable picking things up, touching things, going slow, a place like home.  Clothes in the closet, pans in the cupboards, book on the shelves, and creative messes everywhere (at least at my house) with things most often or most recently used migrating naturally to the front. Being invited into someone’s home is super intimate. You learn something close and peculiar about the person by what’s in their space: a shelf packed with vintage perfume bottles, a collection of gregarious high heels, piles of board games or religious paraphernalia. Play/Room successfully imparted this kind of personality on the players. Many mentioned feeling like they were visiting my artist’s studio or even poking around inside of my brain. One player even described the Play/Room as a physical memory palace. Like going through your grandmother’s attic or dad’s self storage unit while they are still around to tell you the stories of the objects and your family with them: hand sewn quilts with butterflies motifs made by great great great etc grandmothers, chachkies bought on trips to faraway places, the last piece of unbroken crockery from a long lost set, each with a story to tell. This response was strongest with the most documentary style videos but while players could look back, dipping into a stream of my memories, the objects also activated their own.

“With objects thumbnails could totally remember which ones I’ve seen and which I haven’t.” Playern

All of this touching stuff and memory talk has got me thinking and reading more about embodied knowledge and how our bodies and their sensory motor capacities understand stuff and how we can make better mixed physical virtual interfaces for them. So that’s where I am headed in the next post. Check back soon!