Projecting augmented reality

This week I went to Social Media Cafe in Manchester, which is an event where local creatives can show off some of the interesting stuff they’ve been working on and do some networking. One of the most interesting things I saw this time was a set of demos by Elliot Woods from Kimchi and Chips of their work on installations that are based on the idea of “pixel mapping” using data projectors and cameras.

The basic principle is rather similar to augmented reality technology – use a camera and feed it to software that can use markers and features in the environment to help position virtual objects on the image. The difference is that while augmented reality is mostly focussed on putting the virtual onto the real image within the camera view (for example, in a mobile device), what Elliot and his studio do is to map those pixels onto the external world and use a data projector (or several) to project an image into 3d space.

So, rather than peering through a smartphone at an overlay on the camera image, the overlay is projected into the environment. You can see this at work in the Link and Lit Tree installations.

As Elliot describes it, the challenge they wanted to meet was to go beyond projecting onto large, flat, fairly simple spaces (as typified by the current trend of projecting ads onto building facades) to instead be able to map a pixel from a projector onto more complex objects that are a little closer and more intimate with the viewer, such as objects in the room.

To make this work, the studio developed some open source  software called MapTools: “Structured light scanning software for use in detecting the 3D position of projector pixels when scattered by a non-planar object”. This is used in conjunction with a regular data projector. They also developed an iPad-based controller so that the artist can walk around the installation and control it, for example shifting the projection onto a different space.

One interesting side-effect is that it makes fine control of a projector possible without expensive servo controls – for example, to shift a projection from going onto a wall to the inside of a rectangular space on a table you can simply pull the projector downwards to encompass the area, and the software can pick up where it needs to be projecting within that space. This could make for very robust but flexible classroom technologies where we can move quickly from working in presentation mode down into “huddle” mode around tables – even, as the Link installation shows – projecting onto multiple locations from a single projector with a wide field using the 3D mapping software.

A photo of the Link installation in Korea: a data projector is showing movies on multiple cardboard boxes at various sizes and angles in a single space.

A photo of the Link installation in Korea: data projectors are showing movies on multiple cardboard boxes in a single space. Although multiple projectors are used, each one is showing a large number of videos on lots of different "screens" of different sizes and at different angles within its field using pixel mapping.

This does place some constraints on the environment in terms of positioning and lighting, but could be interesting to explore as a way of connecting personal and shared screens in a face to face environment without having to buy lots of expensive and fragile hardware.


This entry was posted in augmented reality. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s