Skip to main content
COMMUNITY:
Reference Materials

Gesture Coder: A Tool for Programming Multi-Touch Gestures by Demonstration

April 12, 2005 | Media and Technology
Multi-touch gestures have become popular on a wide range of touchscreen devices, but the programming of these gestures remains an art. It is time-consuming and error prone for a developer to handle the complicated touch state transitions that result from multiple fingers and their simultaneous movements. In this paper, we present Gesture Coder, which by learning from a few examples given by the developer automatically generates code that recognizes multi-touch gestures, tracks their state changes and invokes corresponding application actions. Developers can easily test the generated code in Gesture Coder, refine it by adding more examples and, once they are satisfied with its performance, integrate the code into their applications. We evaluated our learning algorithm exhaustively with various conditions over a large set of noisy data. Our results show that it is sufficient for rapid prototyping and can be improved with higher quality and more training data. We also evaluated Gesture Coder’s usability through a within subject study in which we asked participants to implement a set of multi-touch interactions with and without Gesture Coder. The results show overwhelmingly that Gesture Coder significantly lowers the threshold of programming multi-touch gestures.

TEAM MEMBERS

  • Hao Lü
    Author
    University of Washington
  • Yang Li
    Author
    Google Research
  • Citation

    Resource Type: Report
    Discipline: Computing and information science | Education and learning science
    Audience: Museum/ISE Professionals | Scientists | Evaluators
    Environment Type: Media and Technology

    If you would like to edit a resource, please email us to submit your request.