After learning how to do some marker-based AR with A-Frame a couple of weeks ago, I felt inspired to pickup where I left off in my experiments in building sculptures in AR back when I was at Banff. Especially since I am currently in between studios, and without easy access to an HTC Vive, AR felt like a good way to quickly test some more ideas of sculptures I want to make with objects from my memories, like the ones I have been making in Anyland.
I started by using the marker-based, web-enabled AR and my Pixel smartphone, putting my objects in various contexts and still life arrangements, and taking screenshots and screen recordings to document ideas. Pretty soon, though, I felt compelled to work larger and more spatially, making use of architectural and landscape elements around me to test even bigger, more embodied experiments. So at some point, I switched to using the Microsoft HoloLens to create sketches in space, and took documentation screenshots and videos on there.
In this post, I will share some of my results from these new experiments in using AR for sculptural still lifes and landscape interventions, and my continued impressions of what it feels like to work in mixed-reality.
Augmented Still Lifes
The first object I wanted to play with in AR was a stack of cakes that I had baked and used for sculptures at Banff. Before throwing the stack of cakes away at the end of my residency, I took a 3D scan of it, so I had the 3D scan ready to go and put into an AR website.
Original cake stack and 3D scan made with Scann3D on Android
I loaded the AR website with the cake stack model on my Pixel’s browser and experimented with putting the cake in various locations around an apartment:
Cake stack re-contextualized in various apartment scenes
I liked how the augmented cake stack had a real presence and “felt” like it had a weight to it in these screenshots. Even though it seems like just two superimposed digital photographs, the process of creating these images was much different from working in a flattened, 2D image editor. By having the cake stack “appear” in front of me in three dimensions (through the smartphone view-pane), I was actually able to walk around it, spin it around, and place it exactly where I wanted, taking the physical properties of the settings in mind. I think that somehow, being able to work in these three dimensions, let me make photos that feel more physically real (or even physically unreal, as in the case where the cake stack is balanced precariously on a plant).
I wanted to try this again with a different memory-object, so I scanned an ice cream cone and made a new AR website which I used to arrange and photograph some more still lifes. Here is a GIF showing what that mixed-reality rearranging process felt like:
Rearranging an augmented ice cream cone within a still life arrangement
As with the cake stacks, working in this way really emphasized the feeling of “Photoshopping in space” to me. I was able to move the augmented ice cream cone around freely and easily and place it where I wanted. It reminded me of rearranging layers in Photoshop, except the layers were the hybrid physical / virtual objects instead of purely 2D digital ones. It also let me work quickly and intuitively, letting me be expressive and spontaneous with the still lifes and shoot many images without having to spend lots of time setting them up:
Many still life experiments made quickly and intuitively
Applying some color correction helped to merge the virtual and physical objects even more:
Ice cream cone still life with fruit, vegetables and bowls
Making these collages that combined AR objects with ordinary, physical ones reminded me of a series of artworks called “One Minute Sculptures” by the artist Erwin Wurm. In those sculptures, people are placed with ordinary objects in surprising poses and tasked to hold the poses temporarily, for about a minute. They are documented as photographs and the resulting sculptures only exist in the documentation. The idea is for these sculptures to be expressive, spontaneous and ephemeral.
One Minute Sculptures by Erwin Wurm
Even though I am not using human figures, my quick AR still lifes documented through photographic screenshots reminded me of those one minute sculptural experiments. I’ve also been thinking about Cristóbal Cea’s work with mixed-reality renderings of draped figures inspired by news imagery of traumatic events in Chile, for example in works like Ghosts and Glorias. For these works, Cea carefully combines 3D renderings and regular video footage to make haunting mixed-reality videos. Although his videos are made with traditional 2D editing techniques, they still achieve a pretty convincing mixed-reality world. I would be interested in seeing how Cea’s work would change if he used AR techniques in space rather than on a computer screen.
Ghosts by Cristóbal Cea
I was excited by these initial experiments with AR still lifes, especially because the process of making them was so easy and portable, only needing a small printout of an AR marker and a Pixel phone. I plan to carry the AR marker around with me from now on so I can shoot spontaneous augmented reality still lifes in other locations.
Augmented Landscape Interventions
While I was experimenting with the AR still lifes, I went on a trip to visit a family farm in upstate New York that has a few acres of land. Since there was so much space to work with there, I felt inspired to take the AR still lifes outdoors to see what would happen when working at a much larger scale. However, the AR markers I was using for the still lifes was too small to use for larger objects since I would need to stand too far away. So, I switched to the HoloLens where I knew I could create some primitive shapes for simple sculptures without the need for a marker. The first object I made, using HoloStudio, was a large, semi-transparent cylinder, about the height of a small barn. I wanted to place it in a field by itself and walk around to see how it felt as a large virtual structure:
Walking around AR cylinders in the middle of a field
The cylinders had a solid presence and felt satisfying, so I tried to make them even larger to see what kind of photographs I could make, using the entire landscape as a setting:
A large, virtual cylinder breaking the horizon line
After noticing that the augmented cylinders I was playing were visually interacting with the horizon and surrounding landscape, I was reminded of the work of the artists Christo and Jeanne Claude, the famous landscape artists known for their large-scale, environmental interventions. One of my favorite works of theirs is Surrounded Islands, a project where they surrounded eleven islands in Biscayne Bay in Miami with 6.5 million square feet of pink fabric extended out over the water. The immensity and simplicity of that kind of landscape artwork feels overwhelming to me, giving me a sinking feeling in my stomach that I resonate with strongly.
Surrounded Islands by Christo and Jeanne-Claude, 1983
I have always wanted to experiment with that kind of work but didn’t know how to think about such large-scale ideas without actually testing them in space. After making the AR cylinders, though, I discovered that I could actually start sketching out these kinds of ideas. So, I tried a few more sculptures, placing large, geometric shapes to make large marks on the landscape.
Large-scale, augmented geometric interventions in the environment
Using HoloStudio for these large scale experiments felt a bit awkward, since I think it was designed for small scale use, so I looked in the Windows App Store to see if I could find any other 3D drawing app to experiment with. I ended up downloading 3dDraw for HoloLens by Case Western Reserve University and found it simpler to use than HoloStudio. With 3dDraw, I could just make a quick, natural hand motion to create cubes freely in space. Immediately, I felt compelled to walk around the farmland and leave trails of cubes as I walked, as sort of AR breadcrumbs:
Leaving a trail of AR “breadcrumbs” to trace my movement through the landscape
After leaving a long trail of breadcrumbs, I took some pictures of them in the environment:
AR breadcrumbs in the landscape
Working in the larger scale outdoors led me to want to try AR architectural interventions as well. For a while, I have had an idea to fill a doorway with pink cement, as a symbolic barrier between two rooms that is heavy but appears light. With the 3dDraw app, I went indoors and prototyped the pink cement barrier quickly between two rooms. I had a bit of trouble with the occlusion geometry not quite letting me fill the doorway completely with the virtual shape, but the prototype gave me a rough idea of what the barrier might feel like in space:
Prototyping a pink cement barrier in a doorway
I took a couple of photos to document the experiment as I think about completing this installation sometime in the future:
Documenting an architectural intervention prototype in AR
I really loved the experience of working at a much larger scale with AR sculptural experiments than I ever had before. I was happy to be able to test landscape and architectural art ideas that I have had for a while, but hadn’t been able to prototype easily. Since the summer is coming, I hope to create more large-scale AR tests outdoors. I also look forward to seeing more of this type of work in experimental exhibits, such as this outdoor AR sculpture exhibit hosted by the National Park Service at Boston Cyberarts Gallery in Boston this summer.
I tested my augmented still life website in the Microsoft Edge browser within the HoloLens and discovered an interesting AR-within-AR world!