8 AR Programming Blocks

8 AR Programming Blocks

posted in: AR, webAR | 0

Our recent exploration into embodied programming with VR has inspired me to think about embodied programming out in the real world. What would it look like for these VR programs and programming blocks to co-exist with objects in the real world? Would real life situations, interactions and locations inspire ways to think of programming that aren’t immediately obvious?

To begin exploring this question, and inspired by the 8 programming blocks exercise for Scratch, we have exported 8 of the basic programming blocks from our VR programming language in Anyland as 3D models to be viewed in AR. The first 8 VR programming blocks we chose were:

Screen Shot 2017-06-30 at 1.19.09 PM


Screen Shot 2017-06-30 at 1.18.33 PM


Screen Shot 2017-06-30 at 1.17.54 PM


Screen Shot 2017-06-30 at 1.17.31 PM


Screen Shot 2017-06-30 at 1.17.10 PM


Screen Shot 2017-06-30 at 1.16.36 PM


Screen Shot 2017-06-30 at 1.15.38 PM

When Touched

Screen Shot 2017-06-30 at 1.15.03 PM

Increase Health


We then put these blocks into into a freely available WebAR app, using fiducial markers for each programming “block”. This let us place the paper markers anywhere in the real world and cause the blocks to appear via a smartphone screen:


ar-programming-blocksHello World! 8 programming blocks out in the real world

As a design exercise, we’ve begun ‘writing’ some simple programs with these blocks, combining them with objects we might wish to program:



This AR program might be interpreted as:

When ukulele touched > Hear music


This AR program might be interpreted as:

If dinner is ready > Say 'Dinner's Ready!'


To mimic our ‘brick and mortar’ code library from our VR programming environment, we put these code blocks on real life library shelves:


Screen Shot 2017-06-30 at 1.31.26 PMThe AR Programming Library

It’s interesting to think about open-source programming libraries as things that are more feasible, accessible and educational, much in the way that traditional libraries are. We could physically visit coding libraries, talk to trained ‘documentation librarians’, and check out a function or two, just to try them on for size. The AR capability of these 8 blocks is useful for programming on the go, or quickly sketching up programming ideas, as well as taking programming to places you may not have considered before, like in public places, or out in nature.

These first quick experiments also made me wonder what it might be like to feel these programming blocks at larger scales. While AR fiducials on a webapp limit how large you can view these, we realized that perhaps with the HoloLens we could view them larger and place them more freely in the landscape. I took the same 8 models and brought them into the HoloLens and placed them outdoors:


When pet dog > Increase health by 1



A branching loop



A larger-than-life loop

Try It Yourself!

If you’d like to try assembling your own AR programs, feel free to print out this image and cut the markers out individually:




Then, launch the app below on your non-iOS smartphone (WebRTC required), point your phone’s camera at the markers, and start coding. Also, feel free to remix the app on glitch and add your own programming blocks into the mix!

8 Blocks AR Programming App


Click to Launch

Remix on Glitch



If you make any of your own AR programs, share them with us, we’d love to see what you come up with!
– Evelyn