Crowd Attention Estimation Automatisation Based on Semi-Automatic Image Semantic Segmentation by Using UNet and CRF Networks

Abstract

Semantic segmentation of crowd images plays a pivotal role in various applications such as crowd management, surveillance, and urban planning. In this paper, we propose an approach for dense and sparse crowd image semantic segmentation based on semi-automatic labeling by employing a combination of UNet and Conditional Random Field (CRF). We introduce a technique for generating segmentation maps for crowd images. We utilize UNet for initial rough segmentation followed by refi nement using CRF. Experimental results demonstrate the model performs better in binary segmentation (crowd and on-crowded regions) rather than ternary segmentation (dense crowds, sparse crowds, and non-crowded areas). However the latter shows better results in terms of crowd detection (regardless of its type). Besides, we show the CRF refi nement is signifi cant in ternary segmentation. Also, we highlight some crowd behavior patterns based on the proposed segmentation model. They diff er in people’s attention types, connections within and between crowds, and possibilities of emergencies.