This week I went to Social Media Cafe in Manchester, which is an event where local creatives can show off some of the interesting stuff they’ve been working on and do some networking. One of the most interesting things I saw this time was a set of demos by Elliot Woods from Kimchi and Chips of their work on installations that are based on the idea of “pixel mapping” using data projectors and cameras.
The basic principle is rather similar to augmented reality technology – use a camera and feed it to software that can use markers and features in the environment to help position virtual objects on the image. The difference is that while augmented reality is mostly focussed on putting the virtual onto the real image within the camera view (for example, in a mobile device), what Elliot and his studio do is to map those pixels onto the external world and use a data projector (or several) to project an image into 3d space.
As Elliot describes it, the challenge they wanted to meet was to go beyond projecting onto large, flat, fairly simple spaces (as typified by the current trend of projecting ads onto building facades) to instead be able to map a pixel from a projector onto more complex objects that are a little closer and more intimate with the viewer, such as objects in the room.
To make this work, the studio developed some open source software called MapTools: “Structured light scanning software for use in detecting the 3D position of projector pixels when scattered by a non-planar object”. This is used in conjunction with a regular data projector. They also developed an iPad-based controller so that the artist can walk around the installation and control it, for example shifting the projection onto a different space.
One interesting side-effect is that it makes fine control of a projector possible without expensive servo controls – for example, to shift a projection from going onto a wall to the inside of a rectangular space on a table you can simply pull the projector downwards to encompass the area, and the software can pick up where it needs to be projecting within that space. This could make for very robust but flexible classroom technologies where we can move quickly from working in presentation mode down into “huddle” mode around tables – even, as the Link installation shows – projecting onto multiple locations from a single projector with a wide field using the 3D mapping software.
This does place some constraints on the environment in terms of positioning and lighting, but could be interesting to explore as a way of connecting personal and shared screens in a face to face environment without having to buy lots of expensive and fragile hardware.