Enabling Smarter Skincare with Machine Learning

---

Olay

---

Overview

Olay, the Procter & Gamble (P&G) skincare leader, is committed to the science of skin care. Olay collaborated with us to develop a deep learning-powered skin analysis and recommendation platform to enable smarter, more personalized skin care for women. 

Objective

Develop a platform that enables smarter skin care choices
Making the right skin care choices can be difficult. For many women today, an overabundance of products on the market, together with a lack of deep scientific skincare knowledge, can result in uninformed experimentation with skin care products. This can lead to frustration, wasted money, and undesirable results. Olay wanted to develop an easy-to-use platform that would make skincare smarter and more personalized for everyday consumers.

Why us?

Deep capabilities in building advanced machine learning systems and user experience design
How do you take the knowledge found inside a lab or behind a counter to personalize product selection for every woman’s unique face? The answer is machine learning. Machine learning uses algorithms to learn from and make predictions from large amounts of data, in this case, skin features. 

Olay partnered with us because of their deep experience and capabilities in building advanced machine learning systems across industries. Using our user experience design capability, together we could co-develop the right platform to effectively communicate that skincare knowledge and create a sustained relationship with its users. 

Design-led, Rigorous Exploration of Accessible Markets (DREAM)

MULTI-PHASED USER EXPERIENCE RESEARCH

User experience probes
To address complex audience needs, new business models, and emerging technologies, we developed an iterative, hypothesis-driven research technique to explore the value of skin care advice solutions based on sensors and selfies.

We interviewed and observed Olay’s target consumers to better understand:

• Attitudes toward personal appearance and skin care 
• How and when users took and shared selfies
• User expectations and reactions around a skin care advice solution
• How to foster user trust when presenting advice with the intent to sell

We worked with Olay to co-construct a strategic roadmap for creating a technology-based solution to address the unique needs of the customer.

Technology development
Working closely with our user experience design team, our computer vision scientists developed software to guide the user to capture selfies under optimal conditions by controlling variables such as lighting, camera distance, and facial expression. 

Our machine learning scientists:

• Developed deep learning algorithms to make predictions of skin condition from selfies
• Trained models to detect the presence of target skin features, using human-graded image databases as ground truth

To ensure that the deep learning algorithms could robustly classify what was most valuable to consumers, we worked with Olay to design the annotation requirements for the image datasets that were being built.

Prototype design
We designed and built increasingly higher fidelity “looks-like” and “works-like” prototypes for Olay, enabling further user validation to occur while the platform was being developed. Prototypes were shown to Olay customers to evaluate their overall experience. 

To optimize the user experience and minimize disengagement, we:

• Designed an interface that guides customers through taking a selfie to maximize the quality of input data for deep learning analysis (rather than simply being a good-looking selfie to the user)
• Explored the optimal way to deliver key information back to the user with personalized, easy-to-understand recommendations

Solution

Olay Skin Advisor machine learning-powered platform
After an initial round of user studies and experience probes, together with Olay we agreed to start with a web-based platform with three forms of artificial intelligence—computer vision, deep learning, and adaptive recommendations—that could deliver personalized skin diagnosis using selfies, and recommend individualized product and regimen changes. Repeated use would improve results and provide deeper insights, as well as deepen the sustained relationship between each user and Olay. 

Results

Over 4 million site visits and continued co-development
We delivered to Olay a suite of algorithms and user experience elements and, in September of 2016, the Olay Skin Advisor platform was released. The platform enabled accurate analysis of users’ skin, informed users of what was happening with their skin, suggested product and regimen changes, and provided a compelling user interaction flow.

Since launch, the platform has reached several notable milestones for Olay, including: 

• Over 4 million site visits worldwide 
• Users of the platform exhibited 2x the conversion rate and 40% larger basket size upon checkout as compared to regular Olay.com visitors. They also yielded a 3x lower bounce rate and 4x time spent as engagement measures.
• Local versions in ten countries

Today, we continue to explore how to grow and expand the feature set of the Olay Skin Advisor platform, extending the strategic roadmap established at the start of the partnership.