Journal of Zhejiang University

ENGINEERING Information Technology & Electronic Engineering 2026 Vol.27 No.9 P.1-15

http://doi.org/10.1631/ENG.ITEE.2026.0160

Decoupled prompt-guided mixture-of-experts dynamic distillation for multimodal recommendation

Author(s): Lili WU, Renmin ZHANG, Bin ZHANG, Jincheng ZHANG, Xi CHEN, Yingjing QIAN
Affiliation(s): 1. School of Communication and Electronic Engineering, Jishou University, Jishou 416000, China more
Corresponding email(s): rzhang1981@163.com
Key Words: Multimodal recommendation, Knowledge distillation (KD), Prompt tuning, Mixture-of-experts (MoE), Dynamic temperature scheduling

<<< Previous Article \|Next Article >>>

Lili WU, Renmin ZHANG, Bin ZHANG, Jincheng ZHANG, Xi CHEN, Yingjing QIAN. Decoupled prompt-guided mixture-of-experts dynamic distillation for multimodal recommendation[J]. Journal of Zhejiang University Science C, 2026, 27(9): 1-15.

@article{title="Decoupled prompt-guided mixture-of-experts dynamic distillation for multimodal recommendation",
author="Lili WU, Renmin ZHANG, Bin ZHANG, Jincheng ZHANG, Xi CHEN, Yingjing QIAN",
journal="Journal of Zhejiang University Science C",
volume="27",
number="9",
pages="1-15",
year="2026",
publisher="Zhejiang University Press & Springer",
doi="10.1631/ENG.ITEE.2026.0160"
}

%0 Journal Article
%T Decoupled prompt-guided mixture-of-experts dynamic distillation for multimodal recommendation
%A Lili WU
%A Renmin ZHANG
%A Bin ZHANG
%A Jincheng ZHANG
%A Xi CHEN
%A Yingjing QIAN
%J Frontiers of Information Technology & Electronic Engineering
%V 27
%N 9
%P 1-15
%@ 1869-1951
%D 2026
%I Zhejiang University Press & Springer
%DOI 10.1631/ENG.ITEE.2026.0160

TY - JOUR
T1 - Decoupled prompt-guided mixture-of-experts dynamic distillation for multimodal recommendation
A1 - Lili WU
A1 - Renmin ZHANG
A1 - Bin ZHANG
A1 - Jincheng ZHANG
A1 - Xi CHEN
A1 - Yingjing QIAN
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 27
IS - 9
SP - 1
EP - 15
%@ 1869-1951
Y1 - 2026
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/ENG.ITEE.2026.0160

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: Multimodal recommendation aims to enrich preference modeling by leveraging visual and textual features. However, integrating high-dimensional pretrained features introduces substantial computational overhead. While knowledge distillation provides an effective compression strategy, existing frameworks face three intertwined challenges: rank bottlenecks caused by low-dimensional projections, cross-modal interference induced by shared fusion spaces, and optimization instability under static distillation temperatures. To address these issues, we propose ProMoE-DTS, a decoupled prompt-guided mixture-of-experts framework with dynamic temperature scheduling. Using an asymmetric teacher–student architecture, the teacher model leverages modality-aware soft prompts as semantic anchors to route heterogeneous features into parameter-disjoint expert networks, thereby alleviating cross-modal conflicts and resolving the rank bottlenecks. To ensure stable knowledge transfer, a feedback-driven dynamic temperature scheduler adaptively regulates the distillation intensity based on epoch-wise signals. This asymmetric design confines intensive multimodal operations to the offline teacher, leaving the online student model with a highly efficient, pure identifier-based structure. Extensive experiments on three benchmark datasets demonstrate that ProMoE-DTS improves Recall@20 by 2.24%–3.96% over state-of-the-art baselines, while requiring only 3.28%–3.55% of the teacher’s parameters.

针对多模态推荐的解耦提示引导混合专家动态蒸馏机制

吴李莅¹，张仁民¹，章滨¹，张锦程³，陈曦²，钱莹晶⁴
¹吉首大学通信与电子工程学院，中国吉首市，416000
²腾讯公司，中国深圳市，518057
³蚂蚁集团，中国长沙市，410000
⁴怀化学院电子信息工程学院，中国怀化市，418000
摘要：多模态推荐旨在利用视觉和文本特征来丰富用户偏好建模。然而，集成高维的预训练特征不可避免地引入了显著的计算开销。尽管知识蒸馏提供了一种有效的模型压缩策略，但现有框架仍面临3个相互交织的挑战：由低维投影引发的表示秩瓶颈、由共享融合空间导致的跨模态干扰以及在静态蒸馏温度下固有的优化不稳定性。为解决上述问题，本文提出了一种带有动态温度调度机制的解耦提示引导混合专家框架—ProMoE-DTS。在非对称的师生架构中，教师模型将感知模态的软提示作为语义锚点，引导异构特征独立路由至参数互斥的专家网络中，从而有效缓解跨模态冲突并打破秩瓶颈。为确保知识平稳传递，本文设计了一个反馈驱动的动态温度调度器，能够根据逐轮的训练误差信号自适应调节蒸馏强度。这种非对称设计将繁重的多模态计算完全限制在离线教师端，使得在线部署的学生模型仅保留极其高效的纯标识符结构。在3个基准数据集上的大量实验表明，相较于当前最先进的基线模型，ProMoE-DTS的Recall@20指标提升了2.24%–3.96%，且仅需占用教师模型3.28%–3.55%的参数量。

关键词：多模态推荐；知识蒸馏；提示微调；混合专家系统；动态温度调度

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Bao KQ, Zhang JZ, Zhang Y, et al., 2023. TALLRec: an effective and efficient tuning framework to align large language model with recommendation. Proc 17^th ACM Conf on Recommender Systems, p.1007-1014.

[2]Chen JY, Zhang HW, He XN, et al., 2017. Attentive collaborative filtering: multimedia recommendation with item- and component-level attention. Proc 40^th Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.335-344.

[3]Dai BQ, Du ZC, Zhu JM, et al., 2024. UniEmbedding: learning universal multi-modal multi-domain item embeddings via user-view contrastive learning. Proc 33^rd ACM Int Conf on Information and Knowledge Management, p.4446-4453.

[4]Deng JX, Wang SY, Cai K, et al., 2025. OneRec: unifying retrieve and rank with generative recommender and iterative preference alignment. https://arxiv.org/abs/2502.18965

[5]Dong XM, Huang O, Thulasiraman P, et al., 2023. Improved knowledge distillation via teacher assistants for sentiment analysis. Proc IEEE Symp Series on Computational Intelligence, p.300-305.

[6]Du YP, Sun Z, Wang ZY, et al., 2025. Active large language model-based knowledge distillation for session-based recommendation. Proc 39^th AAAI Conf on Artificial Intelligence, p.11607-11615.

[7]Geng X, Zhang HW, Bian JW, et al., 2015. Learning image and user features for recommendation in social networks. Proc IEEE Int Conf on Computer Vision, p.4274-4282.

[8]He RN, McAuley J, 2016. VBPR: visual Bayesian personalized ranking from implicit feedback. Proc 30^th AAAI Conf on Artificial Intelligence, p.144-150.

[9]He XN, Deng K, Wang X, et al., 2020. LightGCN: simplifying and powering graph convolution network for recommendation. Proc 43^rd Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.639-648.

[10]Hinton G, Vinyals O, Dean J, 2015. Distilling the knowledge in a neural network. https://arxiv.org/abs/1503.02531

[11]Hu XH, Zhang HT, 2025. Invariant representation learning in multimedia recommendation with modality alignment and model fusion. Entropy, 27(1):56.

[12]Huo FS, Xu WC, Guo JC, et al., 2024. C²KD: bridging the modality gap for cross-modal knowledge distillation. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.16006-16015.

[13]Islam SU, Ahad JI, Rahman F, et al., 2025. Dynamic temperature scheduler for knowledge distillation. https://arxiv.org/abs/2511.13767

[14]Jian M, Wang T, Yang MJ, et al., 2026. Hierarchy-aware multimodal distillation for recommendation. IEEE Trans Multim, 28:2279-2290.

[15]Kang S, Kweon W, Lee D, et al., 2023. Distillation from heterogeneous models for top-K recommendation. Proc ACM Web Conf, p.801-811.

[16]Lan PX, Xu HY, Yang EN, et al., 2025. Efficient and effective prompt tuning via prompt decomposition and compressed outer product. Proc Conf of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, p.4406-4421.

[17]Lee JW, Choi M, Lee J, et al., 2019. Collaborative distillation for top-N recommendation. Proc IEEE Int Conf on Data Mining, p.369-378.

[18]Li L, Zhang YF, Chen L, 2023. Personalized prompt learning for explainable recommendation. ACM Trans Inform Syst, 41(4):103.

[19]Liang JH, Zhao XY, Li MY, et al., 2023. MMMLP: multi-modal multilayer perceptron for sequential recommendations. Proc ACM Web Conf, p.1109-1117.

[20]Lin JH, Chen B, Wang HY, et al., 2024. ClickPrompt: CTR models are strong prompt generators for adapting language models to CTR prediction. Proc ACM Web Conf, p.3319-3330.

[21]Lin YF, He JY, 2026. A reinforcement learning framework for multimodal recommendation explanation generation. Authorea.

[22]Liu F, Chen HL, Cheng ZY, et al., 2023. Disentangled multimodal representation learning for recommendation. IEEE Trans Multim, 25:7149-7159.

[23]Liu MR, Zhang SX, Long C, 2025. Facet-aware multi-head mixture-of-experts model for sequential recommendation. Proc 18^th ACM Int Conf on Web Search and Data Mining, p.127-135.

[24]Liu YF, Zhang KN, Ren XY, et al., 2024. AlignRec: aligning and training in multimodal recommendations. Proc 33^rd ACM Int Conf on Information and Knowledge Management, p.1503-1512.

[25]Loshchilov I, Hutter F, 2019. Decoupled weight decay regularization.

[26]Ma WY, Xia HB, Liu Y, 2025. DiffKD: collaborative graph diffusion with knowledge distillation for multimodal recommendation. J Intell Inform Syst, 63(5):1487-1510.

[27]McAuley J, Targett C, Shi QF, et al., 2015. Image-based recommendations on styles and substitutes. Proc 38^th Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.43-52.

[28]Mo F, Xiao L, Song QY, et al., 2025. FGCM: modality-behavior fusion model integrated with graph contrastive learning for multimodal recommendation. IEEE Multim, 32(3):27-37.

[29]Nguyen NH, Nguyen TA, Nguyen T, et al., 2024. Towards efficient communication and secure federated recommendation system via low-rank training. Proc ACM Web Conf, p.3940-3951.

[30]Qiu RH, Wang S, Chen Z, et al., 2021. CausalRec: causal inference for visual debiasing in visually-aware recommendation. Proc 29^th ACM Int Conf on Multimedia, p.3844-3852.

[31]Reimers N, Gurevych I, 2019. Sentence-BERT: sentence embeddings using Siamese BERT-networks. Proc Conf on Empirical Methods in Natural Language Processing and the 9^th Int Joint Conf on Natural Language Processing, p.3982-3992.

[32]Rendle S, Freudenthaler C, Gantner Z, et al., 2009. BPR: Bayesian personalized ranking from implicit feedback. Proc 25^th Conf on Uncertainty in Artificial Intelligence, p.452-461.

[33]Tang JX, Wang K, 2018. Ranking distillation: learning compact ranking models with high performance for recommender system. Proc 24^th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.2289-2298.

[34]Tian YJ, Zhang CX, Guo ZC, et al., 2022. NOSMOG: learning noise-robust and structure-aware MLPs on graphs.

[35]Wang J, Zhu L, Dai T, et al., 2021. Low-rank and sparse matrix factorization with prior relations for recommender systems. Appl Intell, 51(6):3435-3449.

[36]Wang QY, Yin HZ, Wang H, et al., 2019. Enhancing collaborative filtering with generative augmentation. Proc 25^th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.548-556.

[37]Wang X, He XN, Wang M, et al., 2019. Neural graph collaborative filtering. Proc 42^nd Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.165-174.

[38]Wei RX, Lan JM, Li KK, et al., 2025. MFDB: multimodal feature fusion and dynamic behavior modeling for interactive recommendation systems. Knowl-Based Syst, 326:114047.

[39]Wei W, Huang C, Xia LH, et al., 2023. Multi-modal self-supervised learning for recommendation. Proc ACM Web Conf, p.790-800.

[40]Wei W, Tang JB, Xia LH, et al., 2024. PromptMM: multi-modal knowledge distillation for recommendation with prompt-tuning. Proc ACM Web Conf, p.3217-3228.

[41]Wei YW, Wang X, Nie LQ, et al., 2019. MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video. Proc 27^th ACM Int Conf on Multimedia, p.1437-1445.

[42]Wei YW, Wang X, Nie LQ, et al., 2020. Graph-refined convolutional network for multimedia recommendation with implicit feedback. Proc 28^th ACM Int Conf on Multimedia, p.3541-3549.

[43]Wei YW, Wang X, He XN, et al., 2022. Hierarchical user intent graph network for multimedia recommendation. IEEE Trans Multim, 24:2701-2712.

[44]Xun JH, Zhang SY, Zhao Z, et al., 2021. Why do we click: visual impression-aware news recommendation. Proc 29^th ACM Int Conf on Multimedia, p.3881-3890.

[45]Yu HF, Qi ZY, Jang LK, et al., 2024. MMoE: enhancing multimodal models with mixtures of multimodal interaction experts. Proc Conf on Empirical Methods in Natural Language Processing, p.10006-10030.

[46]Zhang JH, Zhu YQ, Liu Q, et al., 2021. Mining latent structures for multimedia recommendation. Proc 29^th ACM Int Conf on Multimedia, p.3872-3880.

[47]Zhang ZJ, Liu SC, Yu JA, et al., 2024. M³oE: multi-domain multi-task mixture-of-experts recommendation framework. Proc 47^th Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.893-902.

[48]Zhou X, Zhou HY, Liu Y, et al., 2023. Bootstrap latent representations for multi-modal recommendation. Proc ACM Web Conf, p.845-854.

[49]Zhu NJ, Ren YQ, Liu Y, et al., 2025. MMGCL: multi-scale and multi-channel graph contrastive learning for flight anomaly detection. Knowl-Based Syst, 329:114275.

Full Text: <372>