Login with the Vulcan hand salute 🖖 - TensorFlow on the browser

Machine learning and AI have come a long way. We are able to run powerful algorithms right on the browser. Incredibly this even works on browsers of mobile devices like iPhones and Android devices. This can lead to interesting applications: Multi-factor authentication, Captcha systems, Login & many others.

THE DEMO

In this demo, the system waits until it detects the Vulcan hand salute.
- Click on the "Enable camera access" picture to get started.
- None of the images are sent to the server. Your data remains on your browser.
- Try to position your hand so that it is fully visible.
- This model requires the right hand to be used.

Click here to enable camera access


Images remain on your browser only

Not every hand is very dexterous. Half of the office team here have no issues at all while the remaining struggle a little. It is a lot of fun nevertheless.

EXPANDING ON THE DEMO

In this simple demo, we tried to demonstrate machine learning models working on the browser. We rely heavily on the Tensorflow model which consists of a palm detection model and a hand-skeleton finger tracking model. Given an input, the model predicts whether it contains a hand. If so, the model returns coordinates for the bounding box around the hand, as well as 21 key points within the hand, outlining the location of each finger joint and the palm. We then compare the image with our reference vulcan hand salute to compare how far off the input image is.

The gestures can be changed very quickly by swapping the base reference image. The hand gestures can also be placed in sequence, the Vulcan hand salute followed by the thumbs up sign, etc.

For this demo, privacy and simplicity, all the images are only on the browser side. For real world applications in a captcha, login or other experiments it will be a mix of browser and server side.

RELATED PROJECTS

Also take a look at this project, with pose detection, a demo where you login with a pose.

ABOUT WEBDIGI

Webdigi is a web development company in London. We specialise in developing bespoke web applications & unique web platforms. Over the years we have helped clients build great web applications. From time to time we publish open source tools, projects and demos like this to raise awareness and contribute back to the open source community that we rely on so much!

Finally in the true Vulcan spirit. Live long and prosper!