Full Text:   <350>

Summary:  <45>

CLC number: TP391.4

On-line Access: 2023-02-27

Received: 2022-10-31

Revision Accepted: 2023-02-27

Crosschecked: 2022-11-28

Cited: 0

Clicked: 275

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Meiqin LIU

https://orcid.org/0000-0003-0693-6574

Chaofan ZHOU

https://orcid.org/0000-0003-0807-6539

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2023 Vol.24 No.2 P.203-216

http://doi.org/10.1631/FITEE.2200524


A graph-based two-stage classification network for mobile screen defect inspection


Author(s):  Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, Ping WEI, Badong CHEN

Affiliation(s):  State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   zhouchaofan@zju.edu.cn, liumeiqin@zju.edu.cn, slzhang@zju.edu.cn, pingwei@mail.xjtu.edu.cn, chenbd@mail.xjtu.edu.cn

Key Words:  Graph-based methods, Multi-label classification, Mobile screen defects, Neural networks


Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, Ping WEI, Badong CHEN. A graph-based two-stage classification network for mobile screen defect inspection[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(2): 203-216.

@article{title="A graph-based two-stage classification network for mobile screen defect inspection",
author="Chaofan ZHOU, Meiqin LIU, Senlin ZHANG, Ping WEI, Badong CHEN",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="24",
number="2",
pages="203-216",
year="2023",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2200524"
}

%0 Journal Article
%T A graph-based two-stage classification network for mobile screen defect inspection
%A Chaofan ZHOU
%A Meiqin LIU
%A Senlin ZHANG
%A Ping WEI
%A Badong CHEN
%J Frontiers of Information Technology & Electronic Engineering
%V 24
%N 2
%P 203-216
%@ 2095-9184
%D 2023
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2200524

TY - JOUR
T1 - A graph-based two-stage classification network for mobile screen defect inspection
A1 - Chaofan ZHOU
A1 - Meiqin LIU
A1 - Senlin ZHANG
A1 - Ping WEI
A1 - Badong CHEN
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 24
IS - 2
SP - 203
EP - 216
%@ 2095-9184
Y1 - 2023
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2200524


Abstract: 
Defect inspection, also known as defect detection, is significant in mobile screen quality control. There are some challenging issues brought by the characteristics of screen defects, including the following: (1) the problem of interclass similarity and intraclass variation, (2) the difficulty in distinguishing low contrast, tiny-sized, or incomplete defects, and (3) the modeling of category dependencies for multi-label images. To solve these problems, a graph reasoning module, stacked on a classification module, is proposed to expand the feature dimension and improve low-quality image features by exploiting category-wise dependency, image-wise relations, and interactions between them. To further improve the classification performance, the classifier of the classification module is redesigned as a cosine similarity function. With the help of contrastive learning, the classification module can better initialize the category-wise graph of the reasoning module. Experiments on the mobile screen defect dataset show that our two-stage network achieves the following best performances: 97.7% accuracy and 97.3% F-measure. This proves that the proposed approach is effective in industrial applications.

用于手机屏缺陷检测的基于图的两阶段分类网络

周超凡1,2,刘妹琴3,2,1,张森林1,2,魏平3,陈霸东3
1浙江大学工业控制技术国家重点实验室,中国杭州市,310027
2浙江大学电气工程学院,中国杭州市,310027
3西安交通大学人工智能与机器人研究所,中国西安市,710049
摘要:缺陷检测是手机屏质量控制的重要环节。手机屏缺陷的特性带来了一些具有挑战性的问题,包括:(1)类间相似性和类内差异性;(2)低对比度、微小尺寸或不完整缺陷的识别带来的困难;(3)针对多标签图像的类别相关性建模。为了解决这些问题,本文提出一种图推理模块,它可以堆放在常规的分类模块上。该推理模块利用类别间的依赖性、图像间的关系以及类别图像之间的相互作用来扩展特征维度,并且达到改进低质量图像特征的目的。为了进一步提高分类性能,分类模块的分类器被设计为一个余弦相似度函数。在对比学习的帮助下,分类模块可以更好地初始化推理模块的类别图。在手机屏缺陷数据集上的实验表明,所提出的两阶段网络取得了最佳性能:准确率为97.7%,F-measure为97.3%。这证明了本文所提出的方法在工业应用中是有效的。

关键词:基于图的方法;多标签分类;手机屏缺陷;神经网络

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Bottou L, 2010. Large-scale machine learning with stochastic gradient descent. Proc 19th Int Conf on Computational Statistics, p.177-186.

[2]Chen T, Kornblith S, Norouzi M, et al., 2020. A simple framework for contrastive learning of visual representations. Proc 37th Int Conf on Machine Learning, p.1597-1607.

[3]Chen ZM, Wei XS, Wang P, et al., 2019. Multi-label image recognition with graph convolutional networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5177-5186.

[4]Gidaris S, Komodakis N, 2018. Dynamic few-shot visual learning without forgetting. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4367-4375.

[5]Haurum JB, Moeslund TB, 2021. Sewer-ML: a multi-label sewer defect classification dataset and benchmark. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.13451-13462.

[6]He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770-778.

[7]He KM, Fan HQ, Wu YX, et al., 2020. Momentum contrast for unsupervised visual representation learning. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9726-9735.

[8]Hjelm RD, Fedorov A, Lavoie-Marchildon S, et al., 2019. Learning deep representations by mutual information estimation and maximization. Proc 7th Int Conf on Learning Representations.

[9]Khosla P, Teterwak P, Wang C, et al., 2020. Supervised contrastive learning. Proc 34th Conf on Neural Information Processing Systems, p.18661-18673.

[10]Kong YH, Liu X, Zhao ZB, et al., 2022. Bolt defect classification algorithm based on knowledge graph and feature fusion. Energy Rep, 8(Suppl 1):856-863.

[11]Lei J, Gao X, Feng ZL, et al., 2018. Scale insensitive and focus driven mobile screen defect detection in industry. Neurocomputing, 294:72-81.

[12]Li CS, Zhang XM, Huang YJ, et al., 2020. A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision. Comput Ind Eng, 146:106530.

[13]Lu Y, Ma L, Jiang HQ, 2020. A light CNN model for defect detection of LCD. In: Hung JC, Yen NY, Chang JW (Eds.), Frontier Computing. Springer, Singapore, p.10-19.

[14]Park JY, Hwang Y, Lee D, et al., 2020. MarsNet: multi-label classification network for images of various sizes. IEEE Access, 8:21832-21846.

[15]Paszke A, Gross S, Chintala S, et al., 2017. Automatic differentiation in PyTorch. Proc 31st Conf on Neural Information Processing Systems.

[16]Simonyan K, Zisserman A, 2015. Very deep convolutional networks for large-scale image recognition. Proc 3rd Int Conf on Learning Representations.

[17]Szegedy C, Vanhoucke V, Ioffe S, et al., 2016. Rethinking the inception architecture for computer vision. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2818-2826.

[18]Wang T, Zhang C, Ding RW, et al., 2021. Mobile phone surface defect detection based on improved faster R-CNN. Proc 25th Int Conf on Pattern Recognition, p.9371-9377.

[19]Wang Y, He DL, Li F, et al., 2020. Multi-label classification with label graph superimposing. Proc AAAI Conf Artif Intell, 34(7):12265-12272. doi:

[20]Wang YC, Gao L, Li XY, et al., 2021a. A new graph-based method for class imbalance in surface defect recognition. IEEE Trans Instrum Meas, 70:5007816.

[21]Wang YC, Gao L, Gao YP, et al., 2021b. A new graph-based semi-supervised method for surface defect classification. Rob Comput Integr Manuf, 68:102083.

[22]Wei B, Hao KR, Gao L, et al., 2021. Bioinspired visual-integrated model for multilabel classification of textile defect images. IEEE Trans Cognit Dev Syst, 13(3):503-513.

[23]Wu ZR, Xiong YJ, Yu SX, et al., 2018. Unsupervised feature learning via non-parametric instance discrimination. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3733-3742.

[24]Xiao WW, Song KC, Liu J, et al., 2022. Graph embedding and optimal transport for few-shot classification of metal surface defect. IEEE Trans Instrum Meas, 71:5010310.

[25]Xu H, Jiang CH, Liang XD, et al., 2019. Reasoning-RCNN: unifying adaptive global reasoning into large-scale object detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.6412-6421.

[26]Yuan ZC, Zhang ZT, Su H, et al., 2018. Vision-based defect detection for mobile phone cover glass using deep neural networks. Int J Precis Eng Manuf, 19(6):801-810.

[27]Zhang JB, Su H, Zou W, et al., 2021. CADN: a weakly supervised learning-based category-aware object detection network for surface defect detection. Patt Recogn, 109:107571.

[28]Zhao JW, Yan K, Zhao YF, et al., 2021. Transformer-based dual relation graph for multi-label image recognition. Proc IEEE/CVF Int Conf on Computer Vision, p.163-172.

[29]Zhou BL, Khosla A, Lapedriza A, et al., 2016. Learning deep features for discriminative localization. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2921-2929.

[30]Zhu Y, Ding RW, Huang WB, et al., 2022. HMFCA-Net: hierarchical multi-frequency based channel attention net for mobile phone surface defect detection. Patt Recogn Lett, 153:118-125.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2023 Journal of Zhejiang University-SCIENCE