Abstract: Automated analysis of sports video summarization is challenging due to variations in cameras, replay speed, illumination conditions, editing effects, game structure, genre, etc. To address these challenges, we propose an effective video summarization framework based on shot classification and replay detection for field sports videos. Accurate shot classification is mandatory to better structure the input video for further processing, i.e., key events or replay detection. Therefore, we present a lightweight convolutional neural network based method for shot classification. Then we analyze each shot for replay detection and specifically detect the successive batch of logo transition frames that identify the replay segments from the sports videos. For this purpose, we propose local octa-pattern features to represent video frames and train the extreme learning machine for classification as replay or non-replay frames. The proposed framework is robust to variations in cameras, replay speed, shot speed, illumination conditions, game structure, sports genre, broadcasters, logo designs and placement, frame transitions, and editing effects. The performance of our framework is evaluated on a dataset containing diverse YouTube sports videos of soccer, baseball, and cricket. Experimental results demonstrate that the proposed framework can reliably be used for shot classification and replay detection to summarize field sports videos.

体育视频摘要中的镜头分类和回放检测

[1]Bagheri-Khaligh A, Raziperchikolaei R, Moghaddam ME, 2012. A new method for shot classification in soccer sports video based on SVM classifier. Proc IEEE Southwest Symp on Image Analysis and Interpretation, p.109-112.

[2]Chen CM, Chen LH, 2014. Novel framework for sports video analysis: a basketball case study. Proc Int Conf on Image Processing, p.961-965.

[3]Chen CM, Chen LH, 2015. A novel method for slow motion replay detection in broadcast basketball video. Multimed Tools Appl, 74(21):9573-9593.

[4]Choroś K, Gogol A, 2016. Improved method of detecting replay logo in sports videos based on contrast feature and histogram difference. Proc 8^th Int Conf on Computational Collective Intelligence, p.542-552.

[5]Dang ZH, Du J, Huang QM, et al., 2007. Replay detection based on semi-automatic logo template sequence extraction in sports video. Proc 4^th Int Conf on Image and Graphics, p.839-844.

[6]Duan LY, Xu M, Tian Q, et al., 2004. Mean shift-based video segment representation and applications to replay detection. Proc 29^th IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.709-712.

[7]Ekin A, Tekalp AM, Mehrotra R, 2003. Automatic soccer video analysis and summarization. IEEE Trans Image Process, 12(7):796-807.

[8]Eldib MY, Zaid BSA, Zawbaa HM, et al., 2009. Soccer video summarization using enhanced logo detection. Proc 16^th IEEE Int Conf on Image Processing, p.4345-4348.

[9]Fani M, Yazdi M, Clausi DA, et al., 2017. Soccer video structure analysis by parallel feature fusion network and hidden-to-observable transferring Markov model. IEEE Access, 5:27322-27336.

[10]Javed A, Bajwa KB, Malik H, et al., 2016. An efficient framework for automatic highlights generation from sports videos. IEEE Signal Process Lett, 23(7):954-958.

[11]Javed A, Irtaza A, Khaliq Y, et al., 2019. Replay and keyevents detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Appl Intell, 49(8):2899-2917.

[12]Javed A, Malik KM, Irtaza A, et al., 2020. A decision tree framework for shot classification of field sports videos. J Supercomput, 76(9):7242-7267.

[13]Jiang H, Zhang M, 2011. Tennis video shot classification based on support vector machine. Proc IEEE Int Conf on Computer Science and Automation Engineering, p.757-761.

[14]Kapela R, McGuinness K, O’Connor NE, 2017. Real-time field sports scene classification using colour and frequency space decompositions. J Real-Time Image Process, 13(4):725-737.

[15]Li W, Chen SJ, Wang HB, 2009. A rule-based sports video event detection method. Proc Int Conf on Computational Intelligence and Software Engineering, p.1-4.

[16]Minhas RA, Javed A, Irtaza A, et al., 2019. Shot classification of field sports videos using AlexNet convolutional neural network. Appl Sci, 9(3):483.

[17]Murala S, Maheshwari RP, Balasubramanian R, 2012. Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process, 21(5):2874-2886.

[18]Nguyen N, Yoshitaka A, 2012. Shot type and replay detection for soccer video parsing. Proc IEEE Int Symp on Multimedia, p.344-347.

[19]Pan H, Van Beek P, Sezan MI, 2001. Detection of slow-motion replay segments in sports video for highlights generation. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.1649-1652.

[20]Pan H, Li BX, Sezan MI, 2002. Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.IV-3385-IV-3388.

[21]Raventós A, Quijada R, Torres L, et al., 2015. Automatic summarization of soccer highlights using audio-visual descriptors. SpringerPlus, 4(1):301.

[22]Su PC, Lan CH, Wu CS, et al., 2013. Transition effect detection for extracting highlights in baseball videos. EURASIP J Image Video Process, 2013(1):27.

[23]Tavassolipour M, Karimian M, Kasaei S, 2014. Event detection and summarization in soccer videos using Bayesian network and copula. IEEE Trans Circ Syst Video Technol, 24(2):291-304.

[24]Tien MC, Chen HT, Chen YW, et al., 2007. Shot classification of basketball videos and its application in shooting position extraction. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1085-1088.

[25]Wang DH, Tian Q, Gao S, et al., 2004. News sports video shot classification with sports play field and motion features. Proc IEEE Conf on Image Processing, p.2247-2250.

[26]Wang JJ, Chang E, Xu CS, 2005. Soccer replay detection using scene transition structure analysis. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.433-436.

[27]Wang L, Liu X, Lin S, et al., 2004. Generic slow-motion replay detection in sports video. Proc Int Conf on Image Processing, p.1585-1588.

[28]Wu X, He R, Sun ZN, et al., 2018. A light CNN for deep face representation with noisy labels. IEEE Trans Inform Forens Secur, 13(11):2884-2896.

[29]Xu W, Yi Y, 2011. A robust replay detection algorithm for soccer video. IEEE Signal Process Lett, 18(9):509-512.

[30]Zhao F, Dong Y, Wei Z, et al., 2012. Matching logos for slow motion replay detection in broadcast sports video. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1409-1412.

[31]Zhao Z, Jiang SQ, Huang QM, et al., 2006. Highlight summarization in sports video based on replay detection. Proc Int Conf on Multimedia and Expo, p.1613-1616.