Sculptural Experiments in AR

Sculptural Experiments in AR

posted in: AR, Uncategorized | 0

Sculptures in Physical Materials

This week I am writing from the Banff Centre for Arts and Creativity in Alberta, Canada, where I am a resident artist in the Body as Site visual arts residency program.  The Body as Site residency brings together artists and writers whose work uses the idea of the “body” as a source of inspiration.  My main project here was to gain a fluency in working with physical materials to make sculptures, while working intuitively and learning to allow my body’s natural response to space and composition guide the artworks. Some sculptors that inspired me to do this project included Judith Scott, Rachel Whiteread and Doris Salcedo, whose works all engage bodies in various ways.

When I arrived at Banff three weeks ago, I was intimidated by the size of the studio I was provided. I had never worked in such a large space and wasn’t sure how I would use it, especially since I hadn’t brought many materials with me.  I spent a couple of weeks learning how to make silicone molds and also gathering found objects and trash to use as materials for sculptures.  Next, I set a challenge for myself to make experimental sculptures that were roughly the same size as my own body and explored issues like tension and weight.  The process was new to me, and I learned about some of the considerations that go into making life-sized material works including the physical limitations of three-dimensional forms that are affected by gravity and other forces, the various sensual qualities that different materials can have, and the presence that large artworks can have merely by taking up space in a room.  I enjoyed trying to make a ‘body’ of trash and some cakes I baked that hold symbolic value for me, and came up with some various arrangements:



Bodily sculptural experiments in found objects and cake


Sculptures in Augmented Reality

Because I worked on these physical sculptures so quickly, I ran out of materials pretty fast.  While I could make iterations and reuse the materials, I found myself instead wishing I had infinite materials to be able to make many more sculptures without having to trash them and start over every time.  At this point, I remembered that I had a Microsoft HoloLens with me with infinitely-available holographic material instead of limited physical material.  So, I decided to see what it feels like to work on bodily-sized sculptures in AR, and whether that process would be able to feel as natural and intuitive as working with physical materials.

To try this sculptural-AR experiment, I decided to use Microsoft HoloStudio app, since it features three-dimensional primitives.  As creative constraints, I set myself the following limitations:

  • create a life-sized AR sculpture, roughly the size of my body
  • use only 6 cuboid forms, one each for head, body, two arms and two legs
  • use only the color red
  • make sure all the cuboids touch each other

I also decided to do 10-15 iterations, similar to the iterative creative exercises that Vi and M have recently done.   As visual inspiration for the experiment, I chose to look at the works of Joel Shapiro, one of my favorite contemporary sculptors who makes highly dynamic and playful abstract sculptures.  Here at Banff, sculptor Michael Doerksen pointed out to me that Joel Shapiro’s large, geometric sculptures actually represent falling, human figures, which was surprising to me and felt like an even better reason to use them as inspiration. The sculptures are also large, unwieldy, and metallic and have never been logistically or financially possible for me to attempt to make, so attempting them in AR seemed fitting.



Joel Shapiro, Untitled – inspiration for sculptural AR


After clearing my studio to make space for the augmented reality sculpture, I launched HoloStudio on the HoloLens and placed a single, red, rectangular form that was roughly my height in the middle of my studio:


A life-sized augmented reality form in my studio


Next, I added three-dimensional forms for the head, arms and legs:


A wonky, augmented reality human figure sculpture


Once I had the six primitives in place, I used the “resize”, “rotate” and “move” commands to reconfigure the sculpture intuitively.  I tried to work as quickly as I would with physical materials, to see if I could get compositions that “felt right” to me.  Below are 8 of the augmented reality sculptures that I made, getting increasingly more dynamic and unique as I became more comfortable in the user interface.  I could tell that I was able to work intuitively because one of the sculptures inadvertently came out looking like a dog:









Eight augmented reality body-based sculptures


After working on 15 or so sculptures, I finally created one that felt right to me:


My favorite iteration of the AR sculptures I made


Working on 15 quick iterations allowed me to focus on the process to really understand how the augmented reality sculpting experience felt in comparison to sculpting with physical materials.  I enjoyed the process very much, especially given the large amount of space in the studio and the quick ability to create large forms with seemingly-infinite material.  I had never been able to create such a large artwork in such a short amount of time, and it was thrilling to be able to sketch out large sculptural ideas so effectively.  Here is a video of me walking around the sculpture to understand it from all sides:



I was surprised by how much of a sense of presence the augmented reality sculpture had, even though it was not actually materialized in my studio.  Walking around it, I got the sense that it was really there, and I could evaluate how the forms felt in terms of (simulated) weight, space, color and composition.

While the advantages of working quickly and cheaply in AR were very apparent, there were also some obvious setbacks to working with material that isn’t really there, and is only represented as simulated visuals and weight.  Reflecting on this experiment, I came up with a personal list of pros and cons of working on sculptures in AR:


  • work on free-floating forms, free of gravity constraints
  • structural integrity is not an issue
  • “infinite” material to work with
  • cheaper if you don’t have to keep buying materials
  • easier to recolor material
  • easier to scrap work and start over
  • “undo” and “redo” actions
  • iteration can happen more quickly
  • less physically demanding on the sculptor


  • spotty object occlusion and a limited viewport makes the experience awkward at times
  • structural integrity isn’t automatically part of your creation
  • there is no actual material to push back on you as you create
  • there are no sensual qualities to the material like thickness, texture and tactility
  • there is no actual object produced to take up space in a room
  • less physically demanding on the sculptor also means your “body knowledge” is less applied


One of my favorite surprises when working in AR came when I decided to see what the sculpture would feel like if it were a few feet taller.  Normally, resizing an entire sculpture would require destroying the original, remaking all of the individual components and re-attaching them into the new, bigger sculpture.  In AR, however, resizing the sculpture was as easy as “gluing” the components to each other, and then applying the resize tool to the composite.  Here is a video of how quickly I was able to grow the sculpture by a few feet:


There is a joke among artists which goes: “If you can’t make it good, make it big. If it doesn’t look good big, make it red. If you want it to sell, make it shiny.”  With augmented reality, it’s very easy to make things big, red and shiny if you decide to. Since my sculpture was already red, I decided to quickly test it out in other colors to see how they felt:




Quickly changing the sculpture from green to gold to black

I liked the black version, it gave a heavier feeling to the piece and felt welcome to me at the time.  Here are some screenshots of how it felt as a dark presence in my studio space:





The black augmented reality sculpture held a real presence in the studio

When I finally closed the HoloStudio app, the sculpture I had been working on disappeared, and I felt an emptiness in the studio. This meant to me that the AR sculpture definitely had had a real presence and had helped me think through my artmaking in a successful way.

Future Experiments in Augmented Reality Artmaking

These initial experiments in AR sculptures were very exciting to me.  I didn’t expect to get to work intuitively, and that’s what matters to me most in my process.  I would love to explore this further to create more imagery, even just captured as photos.  I am also interested in mixed-reality sculpting tests, mixing holographic and physical materials in single sculptures.  At eleVR recently we’ve also been playing with mixed-reality line drawing using the Crayon app for the HoloLens, and it would be interesting to combine those with sculptures as well!  Stay tuned for more mixed-reality artistic experiments!