Publishing Service

Polishing & Checking

Frontiers of Information Technology & Electronic Engineering

ISSN 2095-9184 (print), ISSN 2095-9230 (online)

Shot classification and replay detection for sports video summarization

Abstract: Automated analysis of sports video summarization is challenging due to variations in cameras, replay speed, illumination conditions, editing effects, game structure, genre, etc. To address these challenges, we propose an effective video summarization framework based on shot classification and replay detection for field sports videos. Accurate shot classification is mandatory to better structure the input video for further processing, i.e., key events or replay detection. Therefore, we present a lightweight convolutional neural network based method for shot classification. Then we analyze each shot for replay detection and specifically detect the successive batch of logo transition frames that identify the replay segments from the sports videos. For this purpose, we propose local octa-pattern features to represent video frames and train the extreme learning machine for classification as replay or non-replay frames. The proposed framework is robust to variations in cameras, replay speed, shot speed, illumination conditions, game structure, sports genre, broadcasters, logo designs and placement, frame transitions, and editing effects. The performance of our framework is evaluated on a dataset containing diverse YouTube sports videos of soccer, baseball, and cricket. Experimental results demonstrate that the proposed framework can reliably be used for shot classification and replay detection to summarize field sports videos.

Key words: Extreme learning machine; Lightweight convolutional neural network; Local octa-patterns; Shot classification; Replay detection; Video summarization

Chinese Summary  <19> 体育视频摘要中的镜头分类和回放检测

Ali JAVED, Amen ALI KHAN
塔克西拉工程技术大学软件工程系,巴基斯坦塔克西拉市,47050
摘要:由于摄像机、回放速度、光照条件、剪辑效果、比赛结构和类型等方面的差异,体育视频摘要的自动分析具有挑战性。为了解决这些问题,针对户外运动视频,本文提出一种基于镜头分类和回放检测的有效视频摘要框架。准确的镜头分类对于更好地安排输入视频从而进行进一步处理(关键事件或回放检测)是必要的。因此,提出一种基于轻量级卷积神经网络的镜头分类方法。该方法对每一个镜头进行回放检测;特别地,检测出从体育视频中识别出回放片段的连续标识转换帧。为此,提出局部八元模式特征来表示视频帧,并训练极限学习机分为回放或非回放两类。所提框架对于摄像机、回放速度、镜头速度、光照条件、比赛结构、运动类型、广播公司、标识设计和位置、帧转换和剪辑效果具有鲁棒性。基于YouTube体育视频集中的足球、棒球和板球运动对所提框架进行性能评估。实验结果证明所提框架能够可靠地用于户外运动视频摘要的镜头分类和回放检测。

关键词组:极限学习机;轻量级卷积神经网络;局部八元模式;镜头分类;回放检测;视频摘要


Share this article to: More

Go to Contents

References:

<Show All>

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





DOI:

10.1631/FITEE.2000414

CLC number:

TP391

Download Full Text:

Click Here

Downloaded:

7478

Download summary:

<Click Here> 

Downloaded:

428

Clicked:

5077

Cited:

0

On-line Access:

2022-05-19

Received:

2020-08-16

Revision Accepted:

2022-05-19

Crosschecked:

2021-03-25

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952276; Fax: +86-571-87952331; E-mail: jzus@zju.edu.cn
Copyright © 2000~ Journal of Zhejiang University-SCIENCE