Researchers from Stanford University and Cornell University have put together a projector-camera system that can pull off a classic magic trick: it can read a playing card that is facing away from the camera.

The dual-photography system gains information from a subject by analyzing the way projected patterns of light bounce off it.The system can show a scene from the point of view of the projector as well as that of the camera. It could eventually be used to quickly add lighting effects in movie scenes, including the ability to realistically integrate actors who are shot separately and computer graphics into previously shot scenes.

The work also advances efforts aimed at collecting all of the visual information about a scene by sensing light scattered off objects within it and using the information to create views of the scene from any angle under any lighting condition. The ultimate goal of this area of imaging research is photorealistic virtual reality.

The system consists of a digital camera and digital projector. The projector beams a series of black and white pixels at a scene and the camera captures the way the light bounces off objects in the scene. The heart of the system is a computer algorithm that continually monitors the data and changes the patterns in order to gain the needed information.

What happens to the patterns from the time they leave the projector to the time they are picked up by the camera "tells us how the light... interacts with the scene," said Pradeep Sen, an electrical engineering researcher at Stanford University. Each pixel of light coming from the projector might bounce off a surface, refract or hit nothing. "Each of these interactions will modify the ray of light," said Sen.

For example, imagine a ray of white light hitting a red object, said Sen. "The reflected ray will be red, which is why the object will appear red to our eyes," he said. This is the change the camera measures. "Thus whatever color we measured at the camera, we know that this is also the color we would get at the projector if we shine a white light from the position of the camera," said Sen.

This allows the researchers to measure the light changes from the projector to the camera, then reverse the light to provide a picture from the point of view of the projector. The method works because the properties of a ray of light are unchanged when the ray is reversed, a characteristic of light termed Helmholtz reciprocity.

The trick to reading a playing card that is facing away from the camera is picking up light that is reflected off of a surface behind the card. "In the card experiment, the camera cannot see the card directly, but it can see the surface of the book [behind the card]; the light from the projector bounces off the card, then bounces off the book and hits the camera," said Sen.

When the projector shines on a red part of the card, like the heart of the suit, the light gets a red tint. "The camera observes it and our algorithm determines that the projector saw something red at that position," said Sen. When the camera shines on a blue part of the card, the light is blue. "In this manner, we put together the projector image pixel-by-pixel and can see the card," he said.

Excerpt from Technology Research News, www.trnmag.com, 6/18/05