In the context of indoor team sport events, our goal is to detect and characterize objects of interest for game strategy analysis and autonomous production and broadcasting. Convolutional Neural Networks are considered for this purpose, taking panoramic views of sport scenes as inputs. Basketball images are challenging due to the lack of contrast between players and the background, high dynamics, occlusions and strong light reflections.
Team discrimination with associative embedding
Team classification of players based on deep learning generally requires a fine-tuning to characterize the teams of each new game. To circumvent this constraint, we propose in (Istasse et al. 2019) to extend a real-time players segmentation network to assign a descriptor vector, named embedding, to each pixel so as to support team clustering. Embeddings are trained to assign similar embeddings to teammates (same jersey color) while pushing away embeddings of opponents.
$K$-fold cross-game and cross-arena validation reveals that our method largely overcomes a standard clustering based on RGB player histograms, achieving more than 90% accuracy.
Efficient ball detection
In the context of sport analytics, knowing the position of the ball is of major interest. But more specifically, knowing its position with high accuracy and with a small delay becomes critical for real-time automated production of sports events. This paper considers the task of ball detection from a single viewpoint in the challenging but common case where the ball interacts frequently with the players while being poorly contrasted with respect to the scene background. We propose in (Van Zandycke and De Vleeschouwer 2019) to formulate the problem as a segmentation problem and adopt a multi-scale Convolutional Neural Network (CNN) architecture to find the ball whatever its actual size in the image.
Our inference model runs on a single capture, without temporal information. To take advantage of the ball dynamics, the difference to the previous image is added to the network input.
