Abstract: The rapid and accurate detection of concrete sand moisture content (MC) is crucial for ensuring concrete quality. However, existing unimodal detection methods are constrained by limited representative features and lack robustness. Multimodal operations often involve simple concatenation of features from different modalities, lacking potential interactivity among features. To address this issue, a novel robust cross-modal integration fusion model, which uses five branches to extract the features of images, near-infrared spectrum, and dielectric constant and a multilevel cross-modal integration fusion network to fuse these features, is proposed for the rapid detection of MC in concrete sand. Specifically, the multilevel cross-modal integration fusion network comprises a feature attention module, a cross-modal self-attention fusion module, and an integrated output module. The feature attention module enhances the feature representation from each modality, reducing the interference from redundant features and noise. The cross-modal self-attention fusion module employs a residual self-attention mechanism to deeply mine and fuse interactions between modalities while retaining low-level features, improving model accuracy and stability. The integrated output module is utilized to obtain more robust prediction results. The results show that the proposed model outperforms unimodal, traditional multimodal, and cross-modal methods on our concrete sand dataset, achieving excellent and robust prediction results for both machine-made sand (root mean square error E_RMS=0.458, coefficient of determination R²=0.983, and residual predictive deviation D_RP=7.900) and natural sand (E_RMS=0.705, R²=0.984, and D_RP=7.931). The detection time was within 71 s, significantly enhancing the detection frequency and efficiency, which provides a reliable solution for the rapid detection of MC in concrete sand.

一种用于混凝土砂含水率快速检测的新型鲁棒跨模态集成融合模型

[1]AbasiS, MinaeiS, JamshidiB, et al., 2019. Rapid measurement of apple quality parameters using wavelet de-noising transform with Vis/NIR analysis. Scientia Horticulturae, 252:7-13.

[2]AmlashiAT, AbdollahiSM, GoodarziS, et al., 2019. Soft computing based formulations for slump, compressive strength, and elastic modulus of bentonite plastic concrete. Journal of Cleaner Production, 230:1197-1216.

[3]CaoYY, YangW, LiH, et al., 2024. Development of a vehicle-mounted soil organic matter detection system based on near-infrared spectroscopy and image information fusion. Measurement Science and Technology, 35(4):045501.

[4]ChengMH, JiaoXY, LiuYD, et al., 2022. Estimation of soil moisture content under high maize canopy coverage from UAV multimodal data and machine learning. Agricultural Water Management, 264:107530.

[5]ChungJ, GulcehreC, ChoK, et al., 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555.

[6]DeanTJ, BellJP, BatyAJB, 1987. Soil moisture measurement by an improved capacitance technique, Part I. Sensor design and performance. Journal of Hydrology, 93(1-2):67-78.

[7]FanCC, ZhengYX, WangSQ, et al., 2023. Prediction of bond strength of reinforced concrete structures based on feature selection and GWO-SVR model. Construction and Building Materials, 400:132602.

[8]FragkosA, LoukatosD, KargasG, et al., 2024. Response of the TEROS 12 soil moisture sensor under different soils and variable electrical conductivity. Sensors, 24(7):2206.

[9]GanCQ, TangY, FuX, et al., 2024. Video multimodal sentiment analysis using cross-modal feature translation and dynamical propagation. Knowledge-Based Systems, 299:111982.

[10]GavelaS, NikoloutsopoulosN, PapadakosG, et al., 2018. Multifactorial experimental analysis of concrete compressive strength as a function of time and water-to-cement ratio. Procedia Structural Integrity, 10:135-140.

[11]GuoZ, ZhangJ, MaCY, et al., 2023. Application of visible-near-infrared hyperspectral imaging technology coupled with wavelength selection algorithm for rapid determination of moisture content of soybean seeds. Journal of Food Composition and Analysis, 116:105048.

[12]HanZJ, CaoDF, ZhuHH, et al., 2022. A field test to investigate spatiotemporal distribution of soil moisture under different cropland covers in the semiarid Loess Plateau of China. Paddy and Water Environment, 20(3):339-353.

[13]HeXY, WangY, ZhaoS, et al., 2023. Co-attention fusion network for multimodal skin cancer diagnosis. Pattern Recognition, 133:108990.

[14]HuaCJ, SunMC, ZhuY, et al., 2022. Pedestrian detection network with multi-modal cross-guided learning. Digital Signal Processing, 122:103370.

[15]HuangZX, SanaeifarA, TianY, et al., 2021. Improved generalization of spectral models associated with Vis-NIR spectroscopy for determining the moisture content of different tea leaves. Journal of Food Engineering, 293:110374.

[16]InthiyazS, AltahanBR, AhammadSH, et al., 2023. Skin disease detection using deep learning. Advances in Engineering Software, 175:103361.

[17]JaafarN, LachiriZ, 2023. Multimodal fusion methods with deep neural networks and meta-information for aggression detection in surveillance. Expert Systems with Applications, 211:118523.

[18]KhalkhoD, ThakurS, TripathiMP, 2024. Soil moisture determination by normalized difference index based on drone images analysis. Journal of the Indian Society of Remote Sensing, 52(7):1623-1632.

[19]KingmaDP, BaJ, 2015. Adam: a method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations.

[20]KrzeminskaD, BloemE, StarkloffT, et al., 2022. Combining FDR and ERT for monitoring soil moisture and temperature patterns in undulating terrain in south-eastern Norway. CATENA, 212:106100.

[21]LiF, LuoJS, WangLL, et al., 2023. GCF²-Net: global-aware cross-modal feature fusion network for speech emotion recognition. Frontiers in Neuroscience, 17:1183132.

[22]LiH, SongYM, WangZY, et al., 2024. Development of an online prediction system for soil organic matter and soil moisture content based on multi-modal fusion. Computers and Electronics in Agriculture, 227:109514.

[23]LiuZQ, GuoD, LacasseS, et al., 2020. Algorithms for intelligent prediction of landslide displacements. Journal of Zhejiang University-SCIENCE A, 21(6):412-429.

[24]LiuZY, ZhangRT, YangCS, et al., 2022. Research on moisture content detection method during green tea processing based on machine vision and near-infrared spectroscopy technology. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 271:120921.

[25]MiaoXX, MiaoY, LiuY, et al., 2023. Measurement of nitrogen content in rice plant using near infrared spectroscopy combined with different PLS algorithms. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 284:121733.

[26]MOC (Ministry of Construction of the People’s Republic of China), 2006. Standard for Technical Requirements and Test Method of Sand and Crushed Stone (or Gravel) for Ordinary Concrete, JGJ 52-2006. Industry Standards of the People’s Republic of China in Chinese.

[27]MouazenAM, Al-AsadiRA, 2018. Influence of soil moisture content on assessment of bulk density with combined frequency domain reflectometry and visible and near infrared spectroscopy under semi field conditions. Soil and Tillage Research, 176:95-103.

[28]NawarS, MouazenAM, 2019. On-line Vis-NIR spectroscopy prediction of soil organic carbon using machine learning. Soil and Tillage Research, 190:120-127.

[29]NgW, MinasnyB, MontazerolghaemM, et al., 2019. Convolutional neural network for simultaneous prediction of several soil properties using visible/near-infrared, mid-infrared, and their combined spectra. Geoderma, 352:251-267.

[30]NiC, WangDY, TaoY, 2019. Variable weighted convolutional neural network for the nitrogen content quantization of Masson pine seedling leaves with near-infrared spectroscopy. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 209:32-39.

[31]NijagunaGS, ManjunathDR, AbouhawwashM, et al., 2023. Deep learning-based improved WCM technique for soil moisture retrieval with satellite images. Remote Sensing, 15(8):2005.

[32]RadicaF, IezziG, TrottaO, et al., 2024. Characterization of CDW types by NIR spectroscopy: towards an automatic selection of recycled aggregates. Journal of Building Engineering, 88:109005.

[33]RahmanA, StreetJ, WootenJ, et al., 2025. MoistNet: machine vision-based deep learning models for wood chip moisture content measurement. Expert Systems with Applications, 259:125363.

[34]ShaheedK, MaoAH, QureshiI, et al., 2022. DS-CNN: a pre-trained Xception model based on depth-wise separable convolutional neural network for finger vein recognition. Expert Systems with Applications, 191:116288.

[35]ShuT, WangXK, WangRT, et al., 2023. Mutilmodal feature extraction and attention-based fusion for emotion estimation in videos. arXiv:2303.10421.

[36]SomkunwarR, GuptaAK, AnandA, et al., 2024. CNN-based soil image analysis for enhanced crop prediction in smart agriculture. Proceedings of 2024 MIT Art, Design and Technology School of Computing International Conference, p.1-5.

[37]SongY, WangXZ, XieHL, et al., 2021. Quality evaluation of Keemun black tea by fusing data obtained from near-infrared reflectance spectroscopy and computer vision sensors. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 252:119522.

[38]SongY, YiWQ, LiuY, et al., 2025. A robust deep learning model for predicting green tea moisture content during fixation using near-infrared spectroscopy: integration of multi-scale feature fusion and attention mechanisms. Food Research International, 203:115874.

[39]SzegedyC, LiuW, JiaYQ, et al., 2015. Going deeper with convolutions. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, p.1-9.

[40]TanMX, LeQ, 2019. EfficientNet: rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning, p.6105-6114.

[41]TobiszewskiM, VakhC, 2023. Analytical applications of smartphones for agricultural soil analysis. Analytical and Bioanalytical Chemistry, 415(18):3703-3715.

[42]TongL, QianK, 2025. HFNet: high-precision robotic grasp detection in unstructured environments using hierarchical RGB-D feature fusion and fine-grained pose alignment. Measurement, 253:117775.

[43]VaswaniA, ShazeerN, ParmarN, et al., 2017. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, p.6000-6010.

[44]WangQQ, ZhangH, LiFD, et al., 2021. Assessment of calibration methods for nitrogen estimation in wet and dry soil samples with different wavelength ranges using near-infrared spectroscopy. Computers and Electronics in Agriculture, 186:106181.

[45]WangYT, LiMZ, JiRH, et al., 2021a. Construction of complex features for predicting soil total nitrogen content based on convolution operations. Soil and Tillage Research, 213:105109.

[46]WangYT, LiMZ, JiRH, et al., 2021b. A deep learning-based method for screening soil total nitrogen characteristic wavelengths. Computers and Electronics in Agriculture, 187:106228.

[47]WatanabeA, TokudaS, MizutaY, et al., 2021. Toward automated non-destructive diagnosis of chloride attack on concrete structures by near infrared spectroscopy. Construction and Building Materials, 305:124796.

[48]WeiYY, CaoL, DongYL, et al., 2025. MCNN-CMCA: a multiscale convolutional neural networks with cross-modal channel attention for physiological signal-based mental state recognition. Digital Signal Processing, 156:104856.

[49]WuCW, ZhengY, YangH, et al., 2021. Effects of different particle sizes on the spectral prediction of soil organic matter. CATENA, 196:104933.

[50]WuZJ, CuiNB, ZhangWJ, et al., 2024. Estimation of soil moisture in drip-irrigated citrus orchards using multi-modal UAV remote sensing. Agricultural Water Management, 302:108972.

[51]YangJC, WangXL, WangRH, et al., 2020. Combination of convolutional neural networks and recurrent neural networks for predicting soil properties using Vis–NIR spectroscopy. Geoderma, 380:114616.

[52]YouZY, WangX, XuJW, et al., 2025. Signal generation for bolt loosening detection with unbalanced datasets based on the CBAM-VAE. Measurement, 240:115589.

[53]YuXX, ZhangZG, TangBP, et al., 2024. A multi-head self-attention autoencoder network for fault detection of wind turbine gearboxes under random loads. Measurement Science and Technology, 35(8):086137.

[54]YuanQ, WangJJ, ZhengMW, et al., 2022. Hybrid 1D-CNN and attention-based Bi-GRU neural networks for predicting moisture content of sand gravel using NIR spectroscopy. Construction and Building Materials, 350:128799.

[55]YuanQ, WangJJ, WuBP, et al., 2024. Deep multimodal fusion model for moisture content measurement of sand gravel using images, NIR spectra, and dielectric data. Measurement, 227:114270.

[56]ZhangXL, LinT, XuJF, et al., 2019. DeepSpectra: an end-to-end deep learning approach for quantitative spectral analysis. Analytica Chimica Acta, 1058:48-57.

[57]ZhangXL, YangJ, LinT, et al., 2021. Food and agro-product quality evaluation based on spectroscopy and deep learning: a review. Trends in Food Science & Technology, 112:431-441.

[58]ZhongH, WangJJ, JiaHJ, et al., 2019. Vector field-based support vector regression for building energy consumption prediction. Applied Energy, 242:403-414.

[59]ZhuCM, DingJL, ZhangZP, et al., 2022. Exploring the potential of UAV hyperspectral image for estimating soil salinity: effects of optimal band combination algorithm and random forest. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 279:121416.

[60]ZuoZY, MuJD, LiWJ, et al., 2023. Study on the detection of water status of tomato (Solanum lycopersicum L.) by multimodal deep learning. Frontiers in Plant Science, 14:1094142.

Similar articles