Journal of Zhejiang University

ENGINEERING Information Technology & Electronic Engineering 2026 Vol.27 No.10 P.1-10

http://doi.org/10.1631/ENG.ITEE.2026.0151

Implicit multi-teacher distillation for unsupervised industrial anomaly detection

Author(s): Lanlan WU, Gaoming YANG, Xiujun WANG, Ji ZHANG, Zhi LIU
Affiliation(s): 1. State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science and Technology, Huainan 232001, China more
Corresponding email(s): gmyang@aust.edu.cn, Ji.Zhang@usq.edu.au
Key Words: Industrial anomaly detection, Knowledge distillation, Multi-teacher ensemble, Implicit reconstruction, False-negative-aware maximum response aggregation (FN-MRA)

<<< Previous Article \|Next Article >>>

Lanlan WU, Gaoming YANG, Xiujun WANG, Ji ZHANG, Zhi LIU. Implicit multi-teacher distillation for unsupervised industrial anomaly detection[J]. Journal of Zhejiang University Science C, 2026, 27(10): 1-10.

@article{title="Implicit multi-teacher distillation for unsupervised industrial anomaly detection",
author="Lanlan WU, Gaoming YANG, Xiujun WANG, Ji ZHANG, Zhi LIU",
journal="Journal of Zhejiang University Science C",
volume="27",
number="10",
pages="1-10",
year="2026",
publisher="Zhejiang University Press & Springer",
doi="10.1631/ENG.ITEE.2026.0151"
}

%0 Journal Article
%T Implicit multi-teacher distillation for unsupervised industrial anomaly detection
%A Lanlan WU
%A Gaoming YANG
%A Xiujun WANG
%A Ji ZHANG
%A Zhi LIU
%J Frontiers of Information Technology & Electronic Engineering
%V 27
%N 10
%P 1-10
%@ 1869-1951
%D 2026
%I Zhejiang University Press & Springer
%DOI 10.1631/ENG.ITEE.2026.0151

TY - JOUR
T1 - Implicit multi-teacher distillation for unsupervised industrial anomaly detection
A1 - Lanlan WU
A1 - Gaoming YANG
A1 - Xiujun WANG
A1 - Ji ZHANG
A1 - Zhi LIU
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 27
IS - 10
SP - 1
EP - 10
%@ 1869-1951
Y1 - 2026
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/ENG.ITEE.2026.0151

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: Industrial anomaly detection is important for intelligent manufacturing since defects can degrade product reliability. Existing methods based on knowledge distillation typically use a single teacher, limiting feature diversity and robustness. Multi-teacher methods often require explicit feature alignment across different network architectures, which may introduce semantic conflicts and weaken anomaly responses. We propose implicit multi-teacher distillation (IMTD) for industrial anomaly detection. IMTD employs three complementary teacher networks, namely ResNet18, ResNet34, and Wide-ResNet50, to capture texture, structural, and semantic representations from different feature spaces. Each teacher is paired with an independent decoder trained only on normal samples, enabling implicit knowledge transfer without cross-architecture feature alignment. A false-negative-aware maximum response aggregation (FN-MRA) module is further introduced to preserve the strongest anomaly evidence during fusion. Experiments on MVTec anomaly detection (MVTec AD), visual anomaly benchmark (VisA), and metal part defect dataset (MPDD) show that IMTD consistently outperforms recent state-of-the-art methods in both image-level and pixel-level anomaly detection. The results demonstrate the effectiveness of combining complementary teachers, implicit reconstruction, and maximum-response aggregation under anomaly-scarce conditions.

用于无监督工业异常检测的隐式多教师蒸馏

吴兰兰^1,2，杨高明^1,2,3，王修君⁴，张吉⁵，刘志⁶
¹安徽理工大学煤炭无人化开采数智技术全国重点实验室，中国淮南市，232001
²安徽理工大学计算机科学与工程学院，中国淮南市，232001
³安徽理工大学第一附属医院，中国淮南市，232001
⁴安徽工业大学计算机科学与技术学院，中国马鞍山市，243032
⁵南昆士兰大学数学、物理与计算学院，澳大利亚图文巴，4350
⁶电气通信大学计算机与网络工程系，日本东京，182-8585
摘要：工业异常检测在智能制造中具有重要意义，因为缺陷会降低产品可靠性。现有基于知识蒸馏的方法通常采用单一教师网络，限制了特征多样性和鲁棒性。多教师方法往往需要在不同网络架构之间进行显式特征对齐，这可能引入语义冲突并削弱异常响应。本文提出一种面向工业异常检测的隐式多教师蒸馏（IMTD）。IMTD采用3个具有互补性的教师网络，即ResNet18、ResNet34和Wide-ResNet50，从不同特征空间中提取纹理、结构和语义表征。每个教师网络均配备一个仅在正常样本上训练的独立解码器，从而实现无需跨架构特征对齐的隐式知识迁移。进一步引入一种假阴性感知最大响应聚合（FN-MRA）模块，以在融合过程中保留最强的异常证据。在MVTec异常检测（MVTec AD）、视觉异常基准（VisA）和金属部件缺陷数据集（MPDD）上的实验表明，IMTD在图像级和像素级异常检测中均持续优于近期先进方法。结果证明了在异常样本稀缺条件下，融合互补教师网络、隐式重建和最大响应聚合策略的有效性。

关键词：工业异常检测；知识蒸馏；多教师集成；隐式重建；假阴性感知最大响应聚合（FN-MRA）

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Ahmad M, Rehman A, 2025. Multi-source information fusion for anomaly detection in smart grids using federated learning. Chin J Inform Fus, 2(2):157-170.

[2]Batzner K, Heckler L, König R, 2024. EfficientAD: accurate visual anomaly detection at millisecond-level latencies. Proc IEEE/CVF Winter Conf on Applications of Computer Vision, p.127-137.

[3]Bergmann P, Fauser M, Sattlegger D, et al., 2019. MVTec AD—a comprehensive real-world dataset for unsupervised anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9584-9592.

[4]Cai YX, Liang DK, Luo DL, et al., 2024. A discrepancy aware framework for robust anomaly detection. IEEE Trans Ind Inform, 20(3):3986-3995.

[5]Cao YK, Zhang JN, Frittoli L, et al., 2024. AdaCLIP: adapting CLIP with hybrid learnable prompts for zero-shot anomaly detection. Proc 18^th European Conf on Computer Vision, p.55-72.

[6]Chen QY, Luo HY, Lv CK, et al., 2025. A unified anomaly synthesis strategy with gradient ascent for industrial anomaly detection and localization. Proc European Conf on Computer Vision, p.37-54.

[7]Cheng H, Luo JX, Zhang XY, 2025. Multimodal industrial anomaly detection via UNI-modal and cross-modal fusion. IEEE Trans Ind Inform, 21(6):5000-5010.

[8]Deng HQ, Li XY, 2022. Anomaly detection via reverse distillation from one-class embedding. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.9727-9736.

[9]Gu ZH, Zhang JN, Liu L, et al., 2024. Rethinking reverse distillation for multi-modal anomaly detection. Proc 38^th AAAI Conf on Artificial Intelligence, 38(8):8445-8453.

[10]Hendria WF, Kim H, Seo D, 2025. Multi-model anomaly detection for industrial inspection with dynamic loss weighting and soft-hard features loss. Neur Comput Appl, 37(21):17031-17054.

[11]Jeong J, Zou Y, Kim T, et al., 2023. WinCLIP: zero-/few-shot anomaly classification and segmentation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.19606-19616.

[12]Jezek S, Jonak M, Burget R, et al., 2021. Deep learning-based defect detection of metal parts: evaluating current methods in complex conditions. Proc 13^th Int Congress on Ultra Modern Telecommunications and Control Systems and Workshops, p.66-71.

[13]Li N, Liu AJ, Zhu ZW, et al., 2025. Knowledge distillation-based anomaly detection via adaptive discrepancy optimization. IEEE Trans Ind Inform, 21(9):7176-7187.

[14]Li XR, Huang ZM, Xue F, et al., 2024. MuSc: zero-shot industrial anomaly classification and segmentation with mutual scoring of the unlabeled images. Proc Int Conf on Learning Representations.

[15]Liu Y, Zhang CS, Dong XJ, et al., 2025. A multimodal industrial anomaly detection method based on mask training and teacher–student joint memory. Eng Appl Artif Intell, 161:112299.

[16]Luo JZ, Cai Y, Cai J, et al., 2025. STMBAD: spatio-temporal multimodal behavior anomaly detector for industrial control systems. IEEE Trans Ind Inform, 21(5):3746-3755.

[17]Luo W, Yao HM, Yu WY, et al., 2025. AMI-Net: adaptive mask inpainting network for industrial anomaly detection and localization. IEEE Trans Autom Sci Eng, 22:1591-1605.

[18]Ma Y, Jiang X, Guan N, et al., 2023. Anomaly detection based on multi-teacher knowledge distillation. J Syst Archit, 138:102861.

[19]Ma ZQ, Li JX, Wong WK, 2025. Patch distance based auto-encoder for industrial anomaly detection. Expert Syst Appl, 270:126537.

[20]Sun H, Cao YK, Dong H, et al., 2025. Unseen visual anomaly generation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.25508-25517.

[21]Tien TD, Nguyen AT, Tran NH, et al., 2023. Revisiting reverse distillation for anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.24511-24520.

[22]Wahid JA, Ayoub M, Xu ML, et al., 2025. NN2ViT: neural networks and vision transformers based approach for visual anomaly detection in industrial images. Neurocomputing, 615:128845.

[23]Wang RX, Li Z, Zheng L, et al., 2025. Deep feature clustering for multi-class industrial image anomaly detection. Knowl-Based Syst, 311:113134.

[24]Wei CY, Han H, Xia Y, et al., 2025. TDAD: self-supervised industrial anomaly detection with a two-stage diffusion model. Comput Ind, 164:104192.

[25]Xu MH, Zhu CP, Feng G, et al., 2025. Multitask hybrid knowledge distillation for unsupervised anomaly detection. IEEE Trans Ind Inform, 21(7):5666-5676.

[26]Xu MH, Qu LL, Nie ZH, et al., 2026. Towards reliable medical unsupervised anomaly detection: a benchmark and dataset for PET/CT with cross-modality knowledge distillation network. Inform Fus, 133:104341.

[27]Zhang H, Wang Z, Zeng D, et al., 2025. DiffusionAD: norm-guided one-step denoising diffusion for anomaly detection. IEEE Trans Patt Anal Mach Intell, 47(8):7140-7152.

[28]Zhang HL, Chen DF, Wang C, 2023. Adaptive multi-teacher knowledge distillation with meta-learning. Proc IEEE Int Conf on Multimedia and Expo, p.1943-1948.

[29]Zhang X, Li SY, Li X, et al., 2023. DeSTSeg: segmentation guided denoising student-teacher for anomaly detection. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3914-3923.

[30]Zhang ZL, Zhao ZB, Zhang XW, et al., 2023. Industrial anomaly detection with domain shift: a real-world dataset and masked multi-scale reconstruction. Comput Ind, 151:103990.

[31]Zhong YZ, Guo B, Sun ZM, et al., 2026. MSDCIR-AD: unsupervised anomaly detection via multi-criteria semantic distances and constrained image reconstruction. Comput Ind, 178:104467.

[32]Zou Y, Jeong J, Pemula L, et al., 2022. Spot-the-difference self-supervised pre-training for anomaly detection and segmentation. Proc 17^th European Conf on Computer Vision, p.392-408.

Full Text: <19>