Multimodal Signal Processing

LECTURER: Wim Pouw
August 19-23, 16.00-17.30, Room 3.105

The study of naturalistic communicative movement has reached new potential with advancements in computer vision and other computational innovations. With such advancements in for example pose estimation from 2D video, or kinematic pattern analyses for gesture understanding, old questions can be studied in new ways, but also new questions arise that were simply out of reach under more traditional modes of inquiry. In this short course, you will learn to use new tools and will be invited to engage in a broader discussion about a vision toward the future of a social science and how it can be assisted by multimodal signal processing pipelines. We will work mainly from analysis pipelines designed for the beginner programmer as presented on our open science platform www.envisionbox.org, but new user-friendly tools for non-programmers are also introduced by special guests from the Hasso Plattner Institute Potsdam (e.g., maskanyone, a tool that masks body and voice data while enriching it with kinematic information for privacy-aware data sharing and archiving). We will reserve time for discussion, and time for you to experiment with a envisionbox module of your choice, while the instructor assists you along the way.

With presentations from:
Martin Schilling (Hasso Plattner Institute, Potsdam)
Bjoern Benedikt Heyder (Hasso Plattner Institute, Potsdam)
Babajide Owoyele (Hasso Plattner Institute, Potsdam)

Requirements (please feel free to reach out)
Laptop
Git + Anaconda installed on your system
For help see here: https://envisionbox.org/gettingstarted.html

Preliminary Schedule
Day 1:  Introduction + How to mask your video and audio data for privacy-aware data sharing and archiving
Day 2: Looking to the future of multimodal signal processing via data dashboards
Day 3: Introduction Envisionbox and motion tracking
Day 4: Running a module, do it yourself session
Day 5: Going through a envisionbox module together, or another do it yourself session