By Jinzhong from F(x) Team of Taobao Technology Department, Alibaba Group
The design and display of the front pages cannot be separated from pictures. In imgcook, icons of many pictures are a very important part. An icon is often composed of simple lines and has a distinct meaning, so it is widely used in the expression of common action points, such as home page, search, settings, return, and users. In addition, icons can convey the meaning in the text vividly by its highly standardized design.
In the comprehensive procedure of imgcook, about 6,156 icons are detected every month on average and about 2.67 icons are used for every new module. Therefore, to some extent, knowing which icons are used in the design helps to make clear the semantics within the scope of the icon. Thus, this helps imgcook achieve more intelligent judgments. In this case, icon semantization is very important.
In the previous imgcook procedure, the F(x) Team used resnet to classify and identify more than 80 icons and extract their semantic information. Moreover, the team also clustered similar pictures as a group to predict the semantics of icons based on the similarity. However, the result of online verification is not particularly satisfactory.
First of all, there are many types and large numbers of icons. The effect of direct model training is not particularly ideal. Therefore, scientific classification and sample management are needed to improve the quality.
Secondly, although most icons follow certain design principles such as simplicity and intuition, icon design is still a very creative process. The style of icons is constantly upgrading. Therefore, data and model with wider coverage and stronger generalization capability are needed.
To sum up, the identification and semantization of icons are indispensable parts of the entire imgcook procedure. The optimization of icon identification not only requires to optimizing the model, but also focusing on data sources and iteration. To achieve the auto iteration, the icon identification model has been upgraded from level 3 (L3) to level 4 (L4).
(Levels of Icon Identification Capabilities)
This article will describe how imgcook optimizes data and model by building a complete closed-loop procedure for the icon model.
Data is the most important and consistent part of the closed-loop procedure for the icon identification model. If the data cannot update and iterate by itself, the procedure is rather an experiment about model change than an engineered procedure. In the closed-loop procedure of imgcook, two data collection schemes are provided to ensure that the data is up-to-date and can supplement by itself.
If your data is already categorized, you can directly import the data via the panel:
Manually Importing Data in Batches
In addition, the icon automatically extracted by imgcook is directly collected and automatically labeled:
Automatically Labeled Data
By doing so, the background data set keeps collecting newly designed and categorized icons, thus expanding the coverage of the icon data set.
For automatically collected icons, the system will perform a prediction. If the prediction succeeds, the system automatically adds labels to the icons. If the prediction fails, the system notifies maintainers to add labels manually. Maintainers can label these icons with a simple click.
After the labeling, the system sends the data to OSS for model training then.
As the training data is updated in real time, the training layer of the model only needs to synchronize the data in OSS and labeled data to generate training samples. After data synchronization and image data enhancement, the archive files can be directly imported to pipcook for training. After the training, the current model will be accurately compared with the online model for grey release. After the grey release, the new model will replace the old one and enter into the main procedure of imgcook.
Closed Loop of the Icon Model
The overall quality of icon identification has been improved after the new model procedure is deployed.
Firstly, as the icons in the procedure have been strictly cleaned, deduplicated, and normalized, the bad sample rate has been greatly reduced. Since the new procedure was launched, the total number of new training sets has reached 2,729, and the average number of data retrieved monthly has reached 909. The number of categories has also increased from 97 to 105.
The following table shows the model comparison after two times of iteration. The accuracy of model test sets has reached 83%.
Icon quantity | Model accuracy | |
The first iteration in August | 14,912 | 80.152% |
The fifth iteration in November | 17,540 | 83.035% |
The online identification results, including the accuracy of bound class names have also been improved to some extent.
The core idea of this system is to continuously optimize the model by constantly supplementing data and ensuring data reliability. The data procedure is the core of the entire closed-loop procedure for the icon identification model. However, optimizations of the entire process of icon identification is not only restricted in the data layer. In the future, the F(x) team will make further studies on a more intelligent way to filter and clean the data and on more suitable models for training predictions.
Imgcook 3.0 Series: How Does Design-based Code Generation Recognize Components?
66 posts | 3 followers
FollowAlibaba F(x) Team - June 9, 2021
Alibaba F(x) Team - June 21, 2021
Alibaba F(x) Team - June 20, 2022
Alibaba F(x) Team - February 25, 2021
Alibaba F(x) Team - June 3, 2021
Alibaba F(x) Team - June 20, 2022
66 posts | 3 followers
FollowOffline SDKs for visual production, such as image segmentation, video segmentation, and character recognition, based on deep learning technologies developed by Alibaba Cloud.
Learn MoreExplore Web Hosting solutions that can power your personal website or empower your online business.
Learn MoreExplore how our Web Hosting solutions help small and medium sized companies power their websites and online businesses.
Learn MoreAlibaba Cloud 1688 Cloud Hub is a cloud-based solution that allows you to easily interconnect your 1688.com store with your backend IT systems across different geographic regions in a secure, data-driven, and automated approach.
Learn MoreMore Posts by Alibaba F(x) Team