Read the World Around You
A vision experiment with Shiseido and Google
Project Theia is an ongoing collaboration between Google and
Shiseido to explore innovative ways technology and cosmetics
can enhance the lives of the visually impaired.
Braille Nails is the first of these explorations.
Inspired by Japan’s fascination with nail art, Braille Nails combines the latest in material design and computer vision technologies to make this decorative artform more expressive and useful for the visually impaired.
Throughout the Braille Nails project we held interviews and focus groups with visually impaired people and experts in accessibility to better understand our users, their daily pain points and ultimately their needs. It was through this research and their feedback that we began to develop Braille Nails, leveraging technology to solve real-life problems.
The Braille Nails kit consists of two parts - a set of ten distinctive nails and a companion camera-badge.
The nails double as a marker, allowing the camera to track the user’s hands movements even in visually complex environments.
We’ve combined 3D printing, textural design and distinctive colours to create a range of nail designs that evoke different moods through both sight and touch.
The camera-badge is made from the same materials as the nails themselves, to compliment the wearers outfit, not distract from it. This also means that the badge can be customized alongside the users nails at a nail-salon, allowing infinite possibilities for self-expression.
Different hand gestures trigger different functions, extending the users understanding of their environment beyond the cane - for example:
Hold your thumb up and Braille Nails will describe the scene to you.
“There is a straight path in front of you with two benches to the left”
Point at the sky and it will give you a weather report.
“It’s cloudy and might rain soon. You may want a coat”
Hold your hand up to get a situation report.
“You are standing on Komazawa Dori opposite Ebisu Station in Ebisu, Tokyo”
You can even mark your favourite locations with a thumbs up gesture
“I’ve added this location to your list of favourite places.”
Braille Nails is a working prototype, but there’s still a lot to do to get it ready for mass production.
We have produced a range of four different tactile nail designs for field testing. Each was selected by a panel of visually impaired users.
Computer vision models require vast amounts of training data, and much of the relevant data is currently only available in English. We are contributing to building, retraining and localizing data in Japanese for this project and to assist further projects in Japan.
We’re experimenting with adding basic phrases to the Cloud Vision model that make the responses more conversational.
Braille Nails can currently identify objects and environments with 85% accuracy and works inside and outdoors.
The Braille Nails vision engine is being tested on mobile devices as we work on increasing the resolution and latency of the hands-free camera device.
The next stage of Braille Nails will involve implementing a wide range of additional features.
Braille Nails is an experimental collaboration between many different companies.
Creative Direction / Product Design / Technology Support
Product Design / Device Development
Camera Device Design / Production
Nail Design / Production