The HyperMask Project
Description
HyperMask is a system which projects an animated
face onto a physical mask, worn by an actor. As the mask moves within
a prescribed area (the stage), its position and orientation are detected
by a camera, and the projected image moves with it. Also, if the
orientation of the mask changes, the projected image changes with respect
to the viewpoint of the audience. The lips of the projected face are automatically
synchronized in real time with the voice of the actor, who also controls
the face's expressions.
As a theatrical tool, HyperMask enables
a new style of storytelling, and we plan to demonstrate this style in a
short performance. However, the same technology also has applications in
offices, schools and theme parks. For example, in the so-called ``Office
of the Future'', we would like to be able to project dynamically images
and information onto moving, irregularly-shaped objects.
Technologies in use
-
Calibration (camera, projector)
-
Tracking system
-
Movement prediction & noise reduction
-
Pose estimation
-
Real-time Lip Synchronization
-
Face animation system, emotion controller
-
etc.
Team and contributors
-
Kim BINSTED, Sony CSL Inc.
-
Shigeo MORISHIMA, Seikei University
-
Frank NIELSEN, Sony CSL Inc.
-
Claudio PINHANEZ, MIT MediaLab
-
Tatsuo YOTSUKURA, Seikei University
-
Will UTHER, Carnegie Mellon University
Publications
-
HyperMask: Virtual Reactive Faces for Storytelling, Emerging
Technologies: The Millenium Motel, Siggraph 1999.
-
Projecting Computer Graphics on Moving Surfaces: A Simple Calibration
and Tracking Method , Sketch & Applications, Novel Projection Methods
track, Siggraph 1999.
-
Hypermask: Projection onto 3d moving surfaces, The Interactive Experiences,
Interact
1999.
Live demos at Siggraph 1999, Interact 1999 and Sony CSL.
Photo & Movies album. Soon!
(c) August 1999, Comments or remarks welcome: nielsen@csl.sony.co.jp