Improved Actor Relation Graph based Group Activity Recognition

Video understanding is to recognize and classify different actions or activities appearing in the video. A lot of previous work, such as video captioning, has shown promising performance in producing general video understanding.However, it is still challenging to generate a fine-grained description of human actions and their interactions using state-of-the-art video captioning techniques. The detailed description of human actions and group activities is essential information, which can be used in real-time CCTV video surveillance, health care, sports video analysis, etc. This study proposes a video understanding method that mainly focused on group activity recognition by learning the pair-wise actor appearance similarity and actor positions. We propose to use Normalized cross-correlation (NCC) and the sum of absolute differences (SAD) to calculate the pair-wise appearance similarity and build the actor relationship graph to allow the graph convolution network to learn how to classify group activities. We also propose to use MobileNet as the backbone to extract features from each video frame. A visualization model is further introduced to visualize each input video frame with predicted bounding boxes on each human object and predict individual action and collective activity.

基于改进的Actor关系图的群体活动识别

视频理解是对视频中出现的不同动作或活动进行识别和分类。许多以前的工作，例如视频字幕，在产生一般的视频理解方面显示出令人鼓舞的性能。.. 但是，使用最先进的视频字幕技术生成对人类行为及其交互的详细描述仍然是一项挑战。对人类行为和小组活动的详细描述是必不可少的信息，可用于实时CCTV视频监视，医疗保健，运动视频分析等。本研究提出了一种视频理解方法，该方法主要侧重于通过学习识别小组活动成对演员外观相似度和演员位置。我们建议使用归一化互相关（NCC）和绝对差之和（SAD）计算成对的外观相似度，并建立参与者关系图，以使图卷积网络学习如何对群体活动进行分类。我们还建议使用MobileNet作为骨干从每个视频帧中提取特征。进一步引入了可视化模型，以使用每个人类对象上的预测边界框来可视化每个输入视频帧，并预测单个动作和集体活动。 （阅读更多）