inliers

ANR Project MOHICANS : Towards Modelling High-density Crowds for Assisting Planning and Safety


In brief


threecamsflowcolor

One of the most active recent developments in computer vision has been the analysis of crowded scenes. The interest that this specific field has raised may be explained from two different perspectives. In terms of applicability, continuous surveillance of public and sensitive areas has benefited from the advancements in hardware and infrastructure, and the bottleneck moved towards the processing level, where human supervision is a laborious task which often requires experienced operators. Other circumstances involving the analysis of dense crowds are represented by large scale events (sport events, religious or social gatherings) which are characterized by very high densities (at least locally) and an increased risk of congestions. From a scientific perspective, the detection of pedestrians in different circumstances, and fur- thermore the interpretation of their actions involve a wide range of branches of computer vision and machine learning.

Single camera analysis This represents the typical setup for a broad range of applications related to prevention and detection in public and private environments. Although some cam- era networks may contain thousands of units, it is quite common to perform processing tasks separately in each view. However, single view analysis is limited by the field of view of indi- vidual cameras and furthermore by the spatial layout of the scene; also, frequent occlusions in crowded scenes hamper the performance of standard detection algorithms and complexify tracking.

Multiple camera analysis Multiple camera analysis has the potential to overcome problems related to occluded scenes, long trajectory tracking or coverage of wider areas. Among the main scientific challenges, these systems require mapping different views to the same coordi- nate system; also, solutions for the novel problems they address (detection in dense crowds, object and track association, re-identification etc.) may not be obtained simply by employing and extending previous strategies used in single camera analysis.

In our study, we focus on solving the problem of analyzing the dynamics of a high-density crowd. The goal of the present proposal is to tackle the major challenge of detecting and tracking simultaneously as particles thousands of pedestrians forming a high-density crowd, and based on real data observations, to assist in proposing and validating a particle interaction model for crowd flow. Our project is original in its aim of performing particle level analysis, as well as through its emphasis on wide area multiple camera tracking. The strategy we intend to follow is based on a feedback loop involving particle segmentation and tracking, which aims to address the main difficulty of this problem, the uncertainty of data association. The value of such a study rests on the need for better solutions for human urban environments and for transport infrastructures, that not only improve the efficiency of the flows involved, but also do it in such a way as to increase and not diminish the quality of life. Another important prerogative of such research is to prevent fatalities during large scale events and gatherings.

Main objectives Toward the end of the project, we intend to propose a methodology for the analysis of high-density crowds which benefits from the recent developments in single camera tracking, and also proposes effective data association solutions among multiple cameras. Secondly, we intend to support the research community by providing a multi-camera dataset which would also allow for a stronger implication of additional fields involved in the general study of crowds, mainly physics, control, simulations and sociology.


People involved


Researchers:
  • Emanuel Aldea: project coordinator
  • Sylvie Le Hégarat-Mascle: collaborator, image processing
  • Séverine Dubuisson: collaborator, tracking
  • Khurom Kiyani: collaborator, physical models and analysis
Current and past students:
  • Jennifer Vandoni: PhD student
  • Nicola Pellicanò: PhD student
  • Camille Palmier: Master intern
  • Raphaël Guegan: Master intern
  • Huiqin Chen: Master intern
  • Davide Marastoni: Master intern


Papers published or accepted for publication

An Evidential Framework for Pedestrian Detection in High-Density Crowds, Jennifer Vandoni, Emanuel Aldea and Sylvie Le Hégarat-Mascle, Proceedings of the 14th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS), 2017 final draft, bib
@inproceedings{vandoni17avss,
 author = {Vandoni, Jennifer and Aldea, Emanuel and Le Hégarat-Mascle, Sylvie},
 booktitle = {Proceedings of the 14th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS)},
 title = {An Evidential Framework for Pedestrian Detection in High-Density Crowds},
 year = {2017}
}
Active Learning for High-Density Crowd Count Regression, Jennifer Vandoni, Emanuel Aldea and Sylvie Le Hégarat-Mascle, Proceedings of the 14th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS) - 2nd Workshop on Signal Processing for Understanding Crowd Dynamics, 2017 final draft, bib
@inproceedings{vandoni17spcrowd,
 author = {Vandoni, Jennifer and Aldea, Emanuel and Le Hégarat-Mascle, Sylvie},
 booktitle = {Proceedings of the 14th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS) - 2nd Workshop on Signal Processing for Understanding Crowd Dynamics},
 title = {Active Learning for High-Density Crowd Count Regression},
 year = {2017}
}
Geometry-Based Multiple Camera Head Detection in Dense Crowds, Nicola Pellicanò, Emanuel Aldea and Sylvie Le Hégarat-Mascle, Proceedings of the 28th British Machine Vision Conference (BMVC) - 5th Activity Monitoring by Multiple Distributed Sensing Workshop, 2017 final draft, bib
@inproceedings{pellicano17ammds,
 author = {Pellicanò, Nicola and Aldea, Emanuel and Le Hégarat-Mascle, Sylvie},
 booktitle = {Proceedings of the 28th British Machine Vision Conference (BMVC) - 5th Activity Monitoring by Multiple Distributed Sensing Workshop},
 title = {Geometry-Based Multiple Camera Head Detection in Dense Crowds},
 year = {2017}
}
Pressure Estimation In A High-Density Crowd Using A Multi-Scale Count Regressor, Emanuel Aldea and Khurom H. Kiyani, Proceedings of the 12th International Conference on Traffic and Granular Flow (TGF), 2017 bib
@inproceedings{aldea17tgf,
 author = {Aldea, Emanuel and Kiyani, Khurom H.},
 booktitle = {Proceedings of the 12th International Conference on Traffic and Granular Flow (TGF)},
 title = {Pressure Estimation In A High-Density Crowd
Using A Multi-Scale Count Regressor},
 year = {2017}
}
Robust Wide Baseline Pose Estimation from Video, Nicola Pellicanò, Emanuel Aldea and Sylvie Le Hégarat-Mascle, Proceedings of the International Conference on Pattern Recognition (ICPR), 2016 final draft, supp. material, bib
@inproceedings{pellicano16icpr,
 author    = {Nicola Pellicano and
               Emanuel Aldea and
               Sylvie Le H{\'{e}}garat{-}Mascle},
  title     = {Robust wide baseline pose estimation from video},
  booktitle = {23rd International Conference on Pattern Recognition, {ICPR} 2016,
               Canc{\'{u}}n, Mexico, December 4-8, 2016},
  pages     = {3820--3825},
  year      = {2016}
}


Related code and data

The files below support some of the publications. They are provided with a documentation and demo examples, and a short intro is also provided here.
Data

Part of the work is performed on a dataset containing three synchronized overlapping views of Regents Park Mosque sahn, featuring a sparse to medium density crowd. We also work on some high-density data recorded in Makkah. Depending on your needs (i.e. crowd density, low or high dynamics), an adapted sequence may be provided under the following agreement:
  • strict academic research use
  • no redistribution
Along with the images, we can provide intrinsic calibration files (and the distance between the cameras measured with a handheld laser device), fundamental matrices between camera views, and a manual annotation of corner matches between pairs of views which may be used as ground truth.
Pose estimation from video inliers

Code: available on GitHub
Data: a subset of the images we used in the evaluation is provided along with the code, in order to facilitate testing. If you would like to get the entire set, contact me.

The code allows you to refine the estimation of the relative pose between two synchronized cameras based on the video streams recorded by the cameras. For the two synchronized video streams (with people/cars/etc. moving around), the algorithm requires as input only the undistorted images. This algorithm might help you if:

  • your scene is difficult (wide baseline, homogeneous areas, repetitive patterns), and a baseline pose estimation using an image pair fails consistently
  • your scene does not allow for object based calibration
  • object based calibration has been used, but you need better accuracy in some parts of the scene.
inliers Beside the code for pose estimation, an additional annotation package will be provided. This package allows you to choose manually and refine accurately a set of matches in order to get a ground truth pose estimation. You may use the ground truth:
  • to check the result of our algorithm (in the form of an error distribution in the image space, as in the image to the right)
  • in an extreme case, to get a very accurate pose manually if nothing else works
Further details are provided in the following publication(s):
Robust Wide Baseline Pose Estimation from Video, Nicola Pellicanò, Emanuel Aldea and Sylvie Le Hégarat-Mascle, Proceedings of the International Conference on Pattern Recognition (ICPR), 2016 final draft, supp. material, bib
@inproceedings{pellicano16icpr,
 author = {Pellican{\`o}, Nicola and Aldea, Emanuel and Le H{\'e}garat-Mascle, Sylvie },
 booktitle = {Proceedings of the International Conference on Pattern Recognition (ICPR)},
 title = {Robust Wide Baseline Pose Estimation from Video},
 year = {2016}
}

Copyright © 2015-2017 E Aldea. Project logo created in 15' with Logojoy!