At the Coachella Valley Music and Arts Festival 2019, one of the largest outdoor music festivals in the U.S., AR effects were implemented for streaming of three songs, including ‘Daijobanai.’
Using OpenPose, the poses of the three performers were estimated in real-time from camera images. By combining these pose estimates with pre-captured 3D scan data of the performers and pre-programmed positional data, a unique 3D mesh generation algorithm was used to create high-precision full 3D models that matched the camera images in real-time. This real-time generated 3D model, along with a pre-created 3D model of the venue, enabled high-precision mask processing, morphing effects, and scribble effects through real-time rendering.
