AI Meets Beauty

Perfect Half Million Beauty Product Image Recognition Challenge

As the revolution of artificial intelligence (AI) is underway around the globe, AI has already begun to reshape industries including the multibillion dollar beauty and personal care industry. In order to promote impactful research and problem solving in this disruptive AI beauty space, the leading beauty app developer, Perfect Corp. in collaborating with CyberLink Corp. and National Chiao Tung University to provide a large-scale image dataset of over half million images of beauty and personal care products, namely the Perfect-500K dataset. Perfect-500K is vast in scale, rich and diverse in content in order to collect as many as possible beauty and personal care items from major e-commerce sites.

*Reaching 700 million app downloads for its YouCam Apps suite, installs worldwide as of December 2018

AI Meets Beauty

Perfect Half Million Beauty Product Image Recognition Challenge

As the revolution of artificial intelligence (AI) is underway around the globe, AI has already begun to reshape industries including the multibillion dollar beauty and personal care industry. In order to promote impactful research and problem solving in this disruptive AI beauty space, the leading beauty app developer, Perfect Corp. in collaborating with CyberLink Corp. and National Chiao Tung University to provide a large-scale image dataset of over half million images of beauty and personal care products, namely the Perfect-500K dataset. Perfect-500K is vast in scale, rich and diverse in content in order to collect as many as possible beauty and personal care items from major e-commerce sites.

*Reaching 580 million app downloads for its YouCam Apps suite, installs worldwide as of March 2018


The first edition of “AI Meets Beauty” Challenge, exploring the Perfect-500K dataset was held at ACM Multimedia 2018 (https://challenge2018.perfectcorp.com). It was a very successful event and received excellent media coverage (https://www.businesswire.com/news/home/20181025005400/en/). As an inaugural event, the challenge attracted a strong number of participants, with a total of 97 teams from 13 countries around the world. Figure 1 shows the breakdown of the location of participants and Table 1 shows a ranking list of the top-10 teams in terms of evaluation metrics in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Following the success of our first “AI Meets Beauty” Challenge, this year we will organize the second “AI Meets Beauty” Challenge at ACM Multimedia 2019. A new test dataset for evaluations will be released for this challenge.

Figure 1. Statistics of the teams by country in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Rank Team Name Score(mAP)
1 VCA 0.577567
2 GAN 0.29135
3 WISLAB 0.270676
4 CISIP 0.251895
5 ivlab2017 0.229133
6 ML Artists 0.214105
7 Team ViPr 0.205836
8 USTC_316 0.185988
9 vyzuer 0.152462
10 Toan Vu 0.143374

Table 1. Top-10 best teams in the first “AI Meets Beauty” Challenge at ACM Multimedia 2018.


Participation

The “AI Meets Beauty” Challenge 2019 is a team-based competition. Each team can have one or more members, and each individual can only be part of one team. This challenge is open to the public. Interested individuals, researchers/developers of tertiary education, research institutes or organizations from different sectors and fields are welcome to take part independently or as a team.
At the end of the challenge, all teams will be ranked based on both objective evaluation and subjective human evaluation criteria described below. The top three performing teams will receive award certificates and cash prizes:

At the same time, all accepted submissions will qualify for the conference’s grand challenge award competition. Each winning team for the cash prizes will be required to open source their proposed solution on Github and make a Challenge paper submission to ACM Multimedia 2019 to explain in detail its methodology before qualifying to receive the cash prizes.



Timeline

Unless otherwise stated, all deadlines are at 23:59 Anywhere on Earth (AoE), UTC-12.
please follow the instructions on the main conference website







Citation


If you are reporting results of the challenge or using the dataset, please cite with:

					  	@misc{perfecthalfmillionchallenge2019, title={Perfect Corp. Challenge 2019: 
Half Million Beauty Product Image Recognition}, author={Wen-Huang Cheng,
Jia Jia, Si Liu, Jianlong Fu, Jiaying Liu, Shintami Chusnul Hidayati,
Johnny Tseng, and Jau Huang}, howpublished=
{\url{https://challenge2019.perfectcorp.com/}}, year={2019}}





Paper Submission


We will follow the guideline of ACM Multimedia 2019 Grand Challenge for the paper submission.




Main Organizers

  • Jau Huang

    Jau Huang

    CyberLink Corp.

  • Johnny Tseng

    Johnny Tseng

    Perfect Corp.

  • Wen-Huang Cheng

    Wen-Huang Cheng

    National Chiao Tung University

  • Jia Jia

    Shintami Chusnul Hidayati

    Academia Sinica

  • Jia Jia

    Jiaying Liu

    Peking University

  • Jia Jia

    Jia Jia

    Tsinghua University

  • Si Liu

    Si Liu

    Beihang University

  • Jianlong Fu

    Jianlong Fu

    Microsoft Research

Sponsors


PerfectCorp is dedicated to transforming how consumers, content creators and beauty brands interact together. Our unique platform is the premier online destination for all beauty lovers. Our PERFECTTM apps, YouCam Makeup, recently voted Best Fashion & Apparels App* of 2015, and market leading YouCam Perfect redefine virtual beauty for tens of millions of users.
*YouCam Makeup is the winner of 2015 Appy Award of Best Fashion & Apparels App


Founded in 1996, CyberLink Corp. is the world’s leading multimedia software company and pioneer in video and audio technologies on PCs and portable devices including tablets and smartphones. From what began as a mission to create superb digital multimedia products for consumers by a group of National Taiwan University students led by Professor Huang, CyberLink has since grown into a global and award-winning brand with nearly 30 products and a solid reputation for efficiently delivering innovative, interoperable solutions.

Task Chairs & Main Contact

Dataset

Perfect-500K is a collection of beauty and personal care items from 14 popular e-commerce sites, including Amazon (USA, India), Cult Beauty, Flipkart, Galleria, Gmarket, JD.COM, Nordstrom, Sephora, Strawberrynet, Target, Walgreens, Walmart, and Yahoo Shopping Mall. Table 2 shows the detailed statistics of the dataset.

Source # of images
Amazon (USA) 216,042
Amazon (India) 178,775
Flipkart 38,419
Walmart 34,015
Strawberrynet 14,985
Gmarket 13,569
Target 9,839
Nordstrom 8,720
Galleria 6,355
Sephora 5,927
Walgreens 5,663
JD.COM 5,074
Cult Beauty 3,123
TOTAL 540,506

Download Link

Note that the datasets will ONLY be released to participants who have registered for the challenge during the competition period. After the challenge is completed, we will make the data publically available to the whole research community.

Task Description

This year we will focus on one particular task: beauty and personal care product recognition. Meanwhile, we are open to innovative self-proposed topics.


Main Task: Beauty and personal care product recognition

Given a real-world image containing one beauty or personal care item (some samples are given below), the task is to match the real-world example of this item to the same item in the Perfect-500K data set. This is a practical but extremely challenging task, given the limitation that only images from e-commerce sites are available in Perfect-500K and no real-world examples will be provided in advance.

sample
sample
sample
sample
sample


Additional Task: Open topics

To encourage the exploration of the Perfect-500k application scope, we also accept innovative topics proposed by the participants themselves, e.g., object localization, data labeling, etc. For the open topics, the participants need to clearly define the topic, demonstrate the technical advancement of their proposed solutions, specify the evaluation protocols, and provide Perfect-500k based experimental results.

Submission Format & Evaluation Metrics


Submission

We require every participant to submit the Docker images for our evaluation. This method enables us to keep the testset secret and to easily reproduce the result. Each team is allowed to submit the results of at most 3 runs, selecting one run as the primary run of the submission. When submitting, you should specify 3 URLs at most for us to download your images.

The format of the output CSV file as a result of the execution is the same as that for last year’s challenge: each line contains an image id followed by 7 predicted labels sorted by confidence in descending order. It looks as follows:

<Test Image ID id#1>, <Training Image ID id#1_1>, <Training Image ID id#1_2>, ..., <Training Image ID id#1_6>, <Training Image ID id#1_7>
<Test Image ID id#2>, <Training Image ID id#2_1>, <Training Image ID id#2_2>, ..., <Training Image ID id#2_6>, <Training Image ID id#2_7>

<Test Image ID id#N>, <Training Image ID id#N_1>, <Training Image ID id#N_2>, ..., <Training Image ID id#N_6>, <Training Image ID id#N_7>

Preparation of the Image

We suggest you base your image on ubuntu:16.04 or above and test your solution on it. Once you are ready to submit the image, perform the following steps:

1. docker commit --change 'ENTRYPOINT ["<main_executable>"]' <container_id> <your_image>
2. docker save <your_image> | gzip -c > <saved_image_file>
3. upload the file to Google Drive and submit the downloadable URL to us

In the first command , the <main_executable> should be an executable, e.g., bash script, which accepts 2 arguments described below.

We provide a sample image at https://hub.docker.com/r/aimeetsbeauty/sample-submission.

The ENTRYPOINT in the sample is set to ["/challenge/predict"]. The /challenge/predict script calls random_predict.py to actually perform the prediction and write the output to the given path.

How We Run the Image

1. curl -L -o <downloaded_image> <url>
2. docker load < <downloaded_image>
3. docker run –-network none -ti -v <testset_path>:/testset:ro -v /result --name <container_name> <your_image> /testset /result/predictions.csv
4. docker cp <container_name>:/result/predictions.csv <all_results_path>

Note that:

  • The third command accepts 2 arguments:
    a. /testset is the path to the folder of the test images:
    /testset/t000001.jpg
    /testset/t000002.jpg
    /testset/t000003.jpg

    b. /result/predictions.csv is the path to the output CSV file
  • The image is run under CPU mode
  • There is no internet access in the container
  • The final command basically copies the produced CSV file to our local path so that we can evaluate your result.

Please test your image with these commands before submitting.

The total execution time must not be over 1 hour. The score will be 0 if the execution can not produce the expected output or not finish within 1 hour.

Hardware Used to Run the Image

AWS EC2 instance of type r4.4xlarge:

  • Dual socket Intel Xeon E5 Broadwell Processors (2.3 GHz), 16 vCPUs
  • DDR4 memory, 122GB

Evaluation Metrics

For each test image, the evaluation metric is Mean Average Precision@7 (MAP@7). The performance of each individual test image is then averaged to evaluate the submissions to this challenge.

Leader Board




Chart will update
according to participant performance