At the SAT Metalab we’ve just started on a project called LivePose, which uses multiple popular pose estimation algorithms to track user poses and movements, and outputs the data in OSC and Websocket formats.

Check out our demo video in which we use LivePose to control a virutal reality avatar in Mozilla Hubs. Hubs is an entirely browser-based VR platform, and using our websockets addon the user can control their hubs avatar using just a webcam, with no additional hardware required.

Currently the user can choose between the OpenPose and Posenet pose estimation algorithms.

The user can also add any of several available “filters” to the output. The “armup filter,” for example, outputs whether the user has either of their arms raised above their head. These filter outputs can be used to trigger different events through user actions, adding a “hands-free” user interface based solely on movements and gestures.

All code for the project is free and open source on GitLab.

---

Photograph taken from Pexels.