Full Text:   <1825>

CLC number: TP391

On-line Access: 2022-05-19

Received: 2020-08-16

Revision Accepted: 2022-05-19

Crosschecked: 2021-03-25

Cited: 0

Clicked: 3026

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Ali JAVED

https://orcid.org/0000-0002-1290-1477

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2022 Vol.23 No.5 P.790-800

http://doi.org/10.1631/FITEE.2000414


Shot classification and replay detection for sports video summarization


Author(s):  Ali JAVED, Amen ALI KHAN

Affiliation(s):  Department of Software Engineering, University of Engineering and Technology, Taxila 47050, Pakistan

Corresponding email(s):   ali.javed@uettaxila.edu.pk

Key Words:  Extreme learning machine, Lightweight convolutional neural network, Local octa-patterns, Shot classification, Replay detection, Video summarization


Ali JAVED, Amen ALI KHAN. Shot classification and replay detection for sports video summarization[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(5): 790-800.

@article{title="Shot classification and replay detection for sports video summarization",
author="Ali JAVED, Amen ALI KHAN",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="23",
number="5",
pages="790-800",
year="2022",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2000414"
}

%0 Journal Article
%T Shot classification and replay detection for sports video summarization
%A Ali JAVED
%A Amen ALI KHAN
%J Frontiers of Information Technology & Electronic Engineering
%V 23
%N 5
%P 790-800
%@ 2095-9184
%D 2022
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2000414

TY - JOUR
T1 - Shot classification and replay detection for sports video summarization
A1 - Ali JAVED
A1 - Amen ALI KHAN
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 23
IS - 5
SP - 790
EP - 800
%@ 2095-9184
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2000414


Abstract: 
Automated analysis of sports video summarization is challenging due to variations in cameras, replay speed, illumination conditions, editing effects, game structure, genre, etc. To address these challenges, we propose an effective video summarization framework based on shot classification and replay detection for field sports videos. Accurate shot classification is mandatory to better structure the input video for further processing, i.e., key events or replay detection. Therefore, we present a lightweight convolutional neural network based method for shot classification. Then we analyze each shot for replay detection and specifically detect the successive batch of logo transition frames that identify the replay segments from the sports videos. For this purpose, we propose local octa-pattern features to represent video frames and train the extreme learning machine for classification as replay or non-replay frames. The proposed framework is robust to variations in cameras, replay speed, shot speed, illumination conditions, game structure, sports genre, broadcasters, logo designs and placement, frame transitions, and editing effects. The performance of our framework is evaluated on a dataset containing diverse YouTube sports videos of soccer, baseball, and cricket. Experimental results demonstrate that the proposed framework can reliably be used for shot classification and replay detection to summarize field sports videos.

体育视频摘要中的镜头分类和回放检测

Ali JAVED, Amen ALI KHAN
塔克西拉工程技术大学软件工程系,巴基斯坦塔克西拉市,47050
摘要:由于摄像机、回放速度、光照条件、剪辑效果、比赛结构和类型等方面的差异,体育视频摘要的自动分析具有挑战性。为了解决这些问题,针对户外运动视频,本文提出一种基于镜头分类和回放检测的有效视频摘要框架。准确的镜头分类对于更好地安排输入视频从而进行进一步处理(关键事件或回放检测)是必要的。因此,提出一种基于轻量级卷积神经网络的镜头分类方法。该方法对每一个镜头进行回放检测;特别地,检测出从体育视频中识别出回放片段的连续标识转换帧。为此,提出局部八元模式特征来表示视频帧,并训练极限学习机分为回放或非回放两类。所提框架对于摄像机、回放速度、镜头速度、光照条件、比赛结构、运动类型、广播公司、标识设计和位置、帧转换和剪辑效果具有鲁棒性。基于YouTube体育视频集中的足球、棒球和板球运动对所提框架进行性能评估。实验结果证明所提框架能够可靠地用于户外运动视频摘要的镜头分类和回放检测。

关键词:极限学习机;轻量级卷积神经网络;局部八元模式;镜头分类;回放检测;视频摘要

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Bagheri-Khaligh A, Raziperchikolaei R, Moghaddam ME, 2012. A new method for shot classification in soccer sports video based on SVM classifier. Proc IEEE Southwest Symp on Image Analysis and Interpretation, p.109-112.

[2]Chen CM, Chen LH, 2014. Novel framework for sports video analysis: a basketball case study. Proc Int Conf on Image Processing, p.961-965.

[3]Chen CM, Chen LH, 2015. A novel method for slow motion replay detection in broadcast basketball video. Multimed Tools Appl, 74(21):9573-9593.

[4]Choroś K, Gogol A, 2016. Improved method of detecting replay logo in sports videos based on contrast feature and histogram difference. Proc 8th Int Conf on Computational Collective Intelligence, p.542-552.

[5]Dang ZH, Du J, Huang QM, et al., 2007. Replay detection based on semi-automatic logo template sequence extraction in sports video. Proc 4th Int Conf on Image and Graphics, p.839-844.

[6]Duan LY, Xu M, Tian Q, et al., 2004. Mean shift-based video segment representation and applications to replay detection. Proc 29th IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.709-712.

[7]Ekin A, Tekalp AM, Mehrotra R, 2003. Automatic soccer video analysis and summarization. IEEE Trans Image Process, 12(7):796-807.

[8]Eldib MY, Zaid BSA, Zawbaa HM, et al., 2009. Soccer video summarization using enhanced logo detection. Proc 16th IEEE Int Conf on Image Processing, p.4345-4348.

[9]Fani M, Yazdi M, Clausi DA, et al., 2017. Soccer video structure analysis by parallel feature fusion network and hidden-to-observable transferring Markov model. IEEE Access, 5:27322-27336.

[10]Javed A, Bajwa KB, Malik H, et al., 2016. An efficient framework for automatic highlights generation from sports videos. IEEE Signal Process Lett, 23(7):954-958.

[11]Javed A, Irtaza A, Khaliq Y, et al., 2019. Replay and keyevents detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Appl Intell, 49(8):2899-2917.

[12]Javed A, Malik KM, Irtaza A, et al., 2020. A decision tree framework for shot classification of field sports videos. J Supercomput, 76(9):7242-7267.

[13]Jiang H, Zhang M, 2011. Tennis video shot classification based on support vector machine. Proc IEEE Int Conf on Computer Science and Automation Engineering, p.757-761.

[14]Kapela R, McGuinness K, O’Connor NE, 2017. Real-time field sports scene classification using colour and frequency space decompositions. J Real-Time Image Process, 13(4):725-737.

[15]Li W, Chen SJ, Wang HB, 2009. A rule-based sports video event detection method. Proc Int Conf on Computational Intelligence and Software Engineering, p.1-4.

[16]Minhas RA, Javed A, Irtaza A, et al., 2019. Shot classification of field sports videos using AlexNet convolutional neural network. Appl Sci, 9(3):483.

[17]Murala S, Maheshwari RP, Balasubramanian R, 2012. Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process, 21(5):2874-2886.

[18]Nguyen N, Yoshitaka A, 2012. Shot type and replay detection for soccer video parsing. Proc IEEE Int Symp on Multimedia, p.344-347.

[19]Pan H, Van Beek P, Sezan MI, 2001. Detection of slow-motion replay segments in sports video for highlights generation. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.1649-1652.

[20]Pan H, Li BX, Sezan MI, 2002. Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.IV-3385-IV-3388.

[21]Raventós A, Quijada R, Torres L, et al., 2015. Automatic summarization of soccer highlights using audio-visual descriptors. SpringerPlus, 4(1):301.

[22]Su PC, Lan CH, Wu CS, et al., 2013. Transition effect detection for extracting highlights in baseball videos. EURASIP J Image Video Process, 2013(1):27.

[23]Tavassolipour M, Karimian M, Kasaei S, 2014. Event detection and summarization in soccer videos using Bayesian network and copula. IEEE Trans Circ Syst Video Technol, 24(2):291-304.

[24]Tien MC, Chen HT, Chen YW, et al., 2007. Shot classification of basketball videos and its application in shooting position extraction. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1085-1088.

[25]Wang DH, Tian Q, Gao S, et al., 2004. News sports video shot classification with sports play field and motion features. Proc IEEE Conf on Image Processing, p.2247-2250.

[26]Wang JJ, Chang E, Xu CS, 2005. Soccer replay detection using scene transition structure analysis. Proc IEEE Int Conf on Acoustics, Speech, and Signal Processing, p.433-436.

[27]Wang L, Liu X, Lin S, et al., 2004. Generic slow-motion replay detection in sports video. Proc Int Conf on Image Processing, p.1585-1588.

[28]Wu X, He R, Sun ZN, et al., 2018. A light CNN for deep face representation with noisy labels. IEEE Trans Inform Forens Secur, 13(11):2884-2896.

[29]Xu W, Yi Y, 2011. A robust replay detection algorithm for soccer video. IEEE Signal Process Lett, 18(9):509-512.

[30]Zhao F, Dong Y, Wei Z, et al., 2012. Matching logos for slow motion replay detection in broadcast sports video. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1409-1412.

[31]Zhao Z, Jiang SQ, Huang QM, et al., 2006. Highlight summarization in sports video based on replay detection. Proc Int Conf on Multimedia and Expo, p.1613-1616.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE