Middle East
Behind The Scenes Of The Skin Analyzer Apps
How Research & Innovation drew on years of work to develop the algorithms
For over two years, our brands have been offering skin analyzer apps so that consumers can know more about their skin and how to care for it. But how were these tools developed, and what technologies made them possible?
Behind SkinconsultAI from Vichy and Skin Genius by L’Oréal Paris lie years of research and collaboration among our teams. L’Oréal's Research & Innovation teams reveal how we put together a diagnostic app.
Creating an app from years of dermatological research into skin aging
The 4,000 people in Research & Innovation work in about 50 different disciplines. Their aim is not just to better understand skin and hair and how they function. It's also to create new products and approaches that will help them invent the Beauty of tomorrow. In particular, some of these specialists focus their researches on studying the skin and its aging mechanisms.
The vital role of “Atlases of the signs of cutaneous aging”
One of these research outcomes is the publication since 2007 of six “Atlases of the signs of cutaneous aging.” Designed for those who work in the field, the Atlases provide a systematic characterization and classification of women and men’s skins, according to their age and geography.
By studying a range of facial characteristics, professionals in the sector can identify and quantify aging signs as they appear. The books define the relevant criteria – the pores' openness, the depth of wrinkles on the forehead and the corner of the eye – and provide a severity score that runs from zero to nine.
These atlases represent just one aspect of the knowledge on which our analyzer apps are based. However, this research led to creating algorithms to make the analysis results available in a more systematic way. These algorithms have now been approved by dermatologists and can be used to evaluate new products. They can also help to create new clinical knowledge. Analyzing hundreds of thousands of photographs to reveal aging signs is just one example of its potential.
Using AI to create simple tools that can evolve over time
On the surface, our analyzer apps are simple. The user takes a selfie – without wearing makeup – and the picture is analyzed using different criteria (pores, wrinkles, skin texture, marks, etc.). The result is an overall score based on all those various parameters. Behind the scenes, these apps are a combination of advanced technology and close collaboration between the Research department’s evaluation and digital teams.
15,000
Skin images analyzed by dermatologists
600,000
Photos analyzed by researchers
> 12
Researchers involved in the project
> 12
Algorithms developed
4 apps
L'Oréal Paris: Skin Genius,
Lancôme: Youth Finder,
Vichy: SkinconsultAI,
Biotherm: Skin Age Scan
Automating the diagnosis process
The creation of these applications went through several stages. First, the atlas data was used to feed the algorithms, associating a photo with a score. The algorithms are then trained to score the different points of observation automatically. It’s a major challenge. The clinical photos used were highly standardized, having been taken in the same light, at the same angle, and with the same background. They were a long way from the kind of pictures taken in everyday life with smartphones. It meant that new algorithms would have to be developed based on selfies taken in normal conditions.
Adapting the technology to handle any situation
The acquisition of Modiface in 2018 was a turning point in our projects. “Our challenge was to apply these algorithms to selfies – which are of varying quality, and where the light isn’t always great – and to get the same results as the algorithm working on clinical photos. The other challenge was that, with a smartphone, the distance between a person’s face and their device is very variable, which creates many discrepancies,” explains Frédéric Flament, International Director – Instrument Evaluation. Fortunately, this is where Artificial Intelligence comes in.
Thanks to the study of thousands of selfies worldwide, the algorithms based on AI have learned to recognize the signs of aging on a person’s face from any photo. The final step was to offer a truly augmented beauty experience: being able to predict the future for someone’s skin.
The starting point is to cross-reference the data from selfies with information about the user’s lifestyle, sleep patterns, and general environment. From there, an estimated ‘aging curve’ could be created, based on the average for someone of the same age. The ultimate aim was to give users a better understanding of their skin and state of health, to adapt their care routines.
“ While the tool measures skin aging at a given point in time, our wealth of earlier analysis means that we can make a prediction about aging that also takes into account the amount of sleep someone gets, the level of pollution around them, where they live, etc. ”
Frédéric Flament, Global Head of Claims Science - Product Performance Evaluation at L’Oréal R&I
Services that are continuously evolving:
Anticipating the needs of our consumers
Improving our studies with Deep Learning
Increasing our knowledge of all skin types
As the expectations of consumers, our apps are continuing to evolve. To make advances in dermatology, as for Beauty Tech, we need to reinvent ourselves and push the boundaries of knowledge.
As part of their work, our teams are also using Deep Learning. This technology enables algorithms to learn as they go along, making their output more relevant and more accurate over time. Today, with AI, we can analyze not just thousands of photos – but hundreds of thousands of them to improve our diagnostics.
Thanks to our clinical research and our algorithms' evolution, new versions of existing applications are being unveiled in Fall 2020. They can now analyze 16 or more characteristics of aging. And since the algorithms need new data to compare and analyze, our research is continuing too. Our teams are now studying a more diverse range of skin types – in Brazil, South Korea, Japan, and India – to collect ever more accurate data and make those analyses even more comprehensive.
These apps are the tangible result of our work in Beauty Tech, and the increasing role of digital in our work. They show that the world of tomorrow will be made more than ever together by decompartmentalizing teams and skills.