Machine learning for pose detection and matching - Demo

Machine learning and AI have come a long way. We are able to run powerful algorithms right on the browser. Incredibly this even works on browsers of mobile devices like iPhones and Android devices. This can lead to interesting applications: Multi-factor authentication, Captcha systems, Login & many others.

THE DEMO

In this demo, the system waits for you to match the pose of the picture you select. There is a variety of poses to try out.

- Click on the "Enable camera access" picture to get started.
- None of the images are sent to the server. Your data remains on your browser.
- You have to pick one of the images to get started. You have to match the pose on the reference image.
- You will need to position yourself so that your body is visible as per the reference image.
- This works on Desktop or mobile.

Click here to enable camera access


Images remain on your browser only

Try to match the poses as much as you can. The algorithm here tried to match your overall pose. The orientation of the palm and facial features are not considered. It is a lot of fun!

EXPANDING ON THE DEMO

In this demo, we tried to demonstrate machine learning models working on the browser. We rely heavily on the Google Creative Lab release of TensorFlow.js of PoseNet, a machine learning model which allows for real-time human pose estimation in the browser. Pose estimation refers to computer vision techniques that detect human figures in images and video, so that one could determine, for example, where someone’s elbow or knee shows up in an image. This technology is not recognizing who is in an image, there is no personal identifiable information associated to pose detection. The algorithm is simply estimating where key body joints are.

For this demo, privacy and simplicity, all the images are only on the browser side. For real world applications in a captcha, login or other experiments it will be a mix of browser and server side.

RELATED PROJECTS

Also take a look at this project, with palm detection, a demo where you login with the vulcan hand salute.

ABOUT WEBDIGI

Webdigi is a web development company in London. We specialise in developing bespoke web applications & unique web platforms. Over the years we have helped clients build great web applications. From time to time we publish open source tools, projects and demos like this to raise awareness and contribute back to the open source community that we rely on so much!