AI Meets Beauty

Perfect Half Million Beauty Product Image Recognition Challenge

As the revolution of artificial intelligence (AI) is underway around the globe, AI has already begun to reshape industries including the multibillion dollar beauty and personal care industry. In order to promote impactful research and problem solving in this disruptive AI beauty space, the leading beauty app developer, Perfect Corp. in collaborating with CyberLink Corp. and Academia Sinica to provide a large-scale image dataset of over half million images of beauty and personal care products, namely the Perfect-500K dataset. Perfect-500K is vast in scale, rich and diverse in content in order to collect as many as possible beauty and personal care items from major e-commerce sites.

*Reaching 580 million app downloads for its YouCam Apps suite, installs worldwide as of March 2018

AI Meets Beauty

Perfect Half Million Beauty Product Image Recognition Challenge

As the revolution of artificial intelligence (AI) is underway around the globe, AI has already begun to reshape industries including the multibillion dollar beauty and personal care industry. In order to promote impactful research and problem solving in this disruptive AI beauty space, the leading beauty app developer, Perfect Corp. in collaborating with CyberLink Corp. and Academia Sinica to provide a large-scale image dataset of over half million images of beauty and personal care products, namely the Perfect-500K dataset. Perfect-500K is vast in scale, rich and diverse in content in order to collect as many as possible beauty and personal care items from major e-commerce sites.

*Reaching 580 million app downloads for its YouCam Apps suite, installs worldwide as of March 2018

The first edition of “AI Meets Beauty” Challenge, exploring the Perfect-500K dataset was held at ACM Multimedia 2018 (https://challenge2018.perfectcorp.com). It was a very successful event and received excellent media coverage (https://www.businesswire.com/news/home/20181025005400/en/). As an inaugural event, the challenge attracted a strong number of participants, with a total of 97 teams from 13 countries around the world. Figure 1 shows the breakdown of the location of participants and Table 1 shows a ranking list of the top-10 teams in terms of evaluation metrics in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Following the success of our first “AI Meets Beauty” Challenge, this year we will organize the second “AI Meets Beauty” Challenge at ACM Multimedia 2019. A new test dataset for evaluations will be released for this challenge. Furthermore, we will publish all information related to this challenge on a website, and commit to maintaining it for at least the next three years.

Figure 1. The breakdown of the location of participants in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Rank Team Name Score(mAP)
1 VCA 0.577567
2 GAN 0.29135
3 WISLAB 0.270676
4 CISIP 0.251895
5 ivlab2017 0.229133
6 ML Artists 0.214105
7 Team ViPr 0.205836
8 USTC_316 0.185988
9 vyzuer 0.152462
10 Toan Vu 0.143374

Table 1. A ranking list of the top-10 teams in terms of evaluation metrics in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Following the success of our first “AI Meets Beauty” Challenge, this year we will organize the second “AI Meets Beauty” Challenge at ACM Multimedia 2019. A new test dataset for evaluations will be released for this challenge. Furthermore, we will publish all information related to this challenge on a website, and commit to maintaining it for at least the next three years.


Participation

The “AI Meets Beauty” Challenge 2019 is a team-based competition. Each team can have one or more members, and each individual can only be part of one team. This challenge is open to the public. Interested individuals, researchers/developers of tertiary education, research institutes or organizations from different sectors and fields are welcome to take part independently or as a team.
At the end of the challenge, all teams will be ranked based on both objective evaluation and subjective human evaluation criteria described above. The top three performing teams will receive award certificates and cash prizes:

At the same time, all accepted submissions will qualify for the conference’s grand challenge award competition. Each winning team for the cash prizes will be required to open source their proposed solution on Github before qualifying to receive the cash prizes.



Timeline

Unless otherwise stated, all deadlines are at 23:59 Anywhere on Earth (AoE), UTC-12.
please follow the instructions on the main conference website







Citation


If you are reporting results of the challenge or using the dataset, please cite with:

					  	@misc{perfecthalfmillionchallenge2018,title={Perfect Corp. Challenge 2018: 
Half Million Beauty Product Image Recognition,author={Wen-Huang Cheng,
Jia Jia, Si Liu, Jianlong Fu, Johnny Tseng, and Jau Huang,howpublished=
{\url{https://challenge2018.perfectcorp.com/index.html},year={2018}}





Paper Submission


We will follow the guideline of ACM Multimedia 2019 Grand Challenge for the paper submission.




Main Organizers

  • Jau Huang

    Jau Huang

    CyberLink Corp.

  • Johnny Tseng

    Johnny Tseng

    Perfect Corp.

  • Wen-Huang Cheng

    Wen-Huang Cheng

    National Chiao Tung University

  • Jia Jia

    Shintami Chusnul Hidayati

    Academia Sinica

  • Jia Jia

    Jiaying Liu

    Peking University

  • Jia Jia

    Jia Jia

    Tsinghua University

  • Si Liu

    Si Liu

    Beihang University

  • Jianlong Fu

    Jianlong Fu

    Microsoft Research

Sponsors


PerfectCorp is dedicated to transforming how consumers, content creators and beauty brands interact together. Our unique platform is the premier online destination for all beauty lovers. Our PERFECTTM apps, YouCam Makeup, recently voted Best Fashion & Apparels App* of 2015, and market leading YouCam Perfect redefine virtual beauty for tens of millions of users.
*YouCam Makeup is the winner of 2015 Appy Award of Best Fashion & Apparels App


Founded in 1996, CyberLink Corp. is the world’s leading multimedia software company and pioneer in video and audio technologies on PCs and portable devices including tablets and smartphones. From what began as a mission to create superb digital multimedia products for consumers by a group of National Taiwan University students led by Professor Huang, CyberLink has since grown into a global and award-winning brand with nearly 30 products and a solid reputation for efficiently delivering innovative, interoperable solutions.

Task Chairs & Main Contact

Dataset

Perfect-500K is a collection of beauty and personal care items from 14 popular e-commerce sites, including Amazon (USA, India), Cult Beauty, Flipkart, Galleria, Gmarket, JD.COM, Nordstrom, Sephora, Strawberrynet, Target, Walgreens, Walmart, and Yahoo Shopping Mall. Table 2 shows the detailed statistics of the dataset.

Source # of images
Amazon (USA) 129,192
Amazon (India) 208,936
Cult Beauty 3,139
Flipkart 38,726
Yahoo Shopping Mall 22,407
Target 11,896
JD.COM 5,084
Sephora 7,099
Strawberrynet 16,040
Walgreens 7,273
Walmart 34,428
Gmarket 19,579
Nordstrom 8,786
Galleria 8,142
TOTAL 520,727

Download Link

Note that the datasets will ONLY be released to participants who have registered for the challenge during the competition period. After the challenge is completed, we will make the data publically available to the whole research community.

Task Description

This year we will focus on one particular task: beauty and personal care product recognition. Meanwhile, we are open to innovative self-proposed topics.


Main Task: Beauty and personal care product recognition

Given a real-world image containing one beauty or personal care item (some samples are given below), the task is to match the real-world example of this item to the same item in the Perfect-500K data set. This is a practical but extremely challenging task, given the limitation that only images from e-commerce sites are available in Perfect-500K and no real-world examples will be provided in advance.

sample
sample
sample
sample
sample


Additional Task: Open topics

To encourage the exploration of the Perfect-500k application scope, we also accept innovative topics proposed by the participants themselves, e.g., object localization, data labeling, etc. For the open topics, the participants need to clearly define the topic, demonstrate the technical advancement of their proposed solutions, specify the evaluation protocols, and provide Perfect-500k based experimental results.

Submission Format & Evaluation Metrics

Each team is allowed to submit the results of at most five runs, selecting one run as the primary run of the submission (we do not guarantee that we will evaluate additional runs), which will be measured for performance comparison across teams. Each submission is required to be formatted as a csv file, where each line contains an image id followed by seven predicted labels sorted by confidence in descending order. It looks as follows:

<Test Image ID id#1>, <Training Image ID id#1_1>, <Training Image ID id#1_2>, ..., <Training Image ID id#1_6>, <Training Image ID id#1_7>
<Test Image ID id#2>, <Training Image ID id#2_1>, <Training Image ID id#2_2>, ..., <Training Image ID id#2_6>, <Training Image ID id#2_7>

<Test Image ID id#N>, <Training Image ID id#N_1>, <Training Image ID id#N_2>, ..., <Training Image ID id#N_6>, <Training Image ID id#N_7>

For each test image, the evaluation metric is Mean Average Precision@7 (MAP@7). The performance of each individual test image is then averaged to evaluate the submissions to this challenge.

Leader Board




Chart will update
according to participant performance