More and more cameras have been applied in surveillance to monitor activities over an extended area. One problem associated with a multi-camera system is to automatically analyze and fuse information gathered from multiple cameras so that human intervention is reduced to a maximum extent.
We assume that the problem of single camera tracking is already solved. Main problems associated with a multi-camera system are object representation and matching, topology estimation and data association.
Experimental Setup:
The experimental setup consists of three cameras with non-overlapping fields of view. The cameras are widely separated, including two outdoor settings and one indoor setting.
Some example blobs: each column shows blobs of the same person under three disjoint views.
We also use VIPeR (Viewpoint Invariant Pedestrian Recognition) database in our experiments. The VIPeR database contains a significant amount of viewpoint and illumination variation and contains 632 pedestrian image pairs taken from arbitrary viewpoints under varying illumination conditions.
Object representation and matching
Since the fields of view of multiple cameras are non-overlapping, the appearances of moving objects under these cameras may exhibit significant differences due to variations of illumination conditions, poses and camera parameters. Therefore, the key problem of matching objects across cameras is to build a representation for the appearance of objects which is robust to these variations. The appearance matching problem is similar to the query-by-example problem in CBIR.
ACCV:
Diffusion distance is proposed to histogram matching to compensate for illumination changes and camera distortions.
Related Publications:
1. Yinghao Cai, Wei Chen, Kaiqi Huang, Tieniu Tan, “Continuously Tracking Objects Across Multiple Widely Separated Cameras”. The 8th Asian Conference on Computer Vision (1) 2007: 843-852. (ACCV poster).
ICIP:
Unlike snapshot-based appearance matching which matches objects by a single image, we focus on sequence matching to alleviate the uncertainties brought by segmentation errors and partial occlusions.
We partition the blob into regular patches for localization of color components. Each patch is represented by its dominant colors and frequencies of occurrence these colors appearing in the patch on the target.
At the training stage, given the sequence of a queried person under one camera, the appearance model is formulated by concatenating feature vectors with the majority of votes over the sequence.
At the testing stage, Bayesian inference is incorporated into the identification framework to accumulate the temporal information in the sequence.
Related Publication:
1. Yinghao Cai, Kaiqi Huang and Tieniu Tan, “Matching Tracking Sequences Across Widely Separated Cameras”, accepted by IEEE International Conference on Image Processing, 2008. (Oral presentation).
ICPR:
Objects of interest, pedestrians are represented by a set of region signatures centered at points sampled from edges. The problem of frame-to-frame matching is formulated as finding corresponding points in two images as minimization of a cost function over the space of correspondence. The correspondence problem is solved under integer optimization framework where the cost function is determined by similarity of region signatures as well as geometric constraints between points.
A demo:
Corresponding points in two images
Related Publication:
1. Yinghao Cai, Kaiqi Huang and Tieniu Tan, “Human Appearance Matching Across Multiple Non-overlapping Cameras”, Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, Tampa, USA. (Oral presentation).
Topology estimation
A topology graph is an abstract network of nodes and connections as shown in Figure. Why is topology graph necessary?~~~
(1) Automatically learn a model of activity for a multi-camera surveillance network.
(2) Since moving objects do not move around the scene randomly. The topology graph provides an understanding of paths that can be followed by moving objects moving within the field of view of the network of cameras.
(3) Spatial relations between cameras help to coordinate object tracking between multiple non-overlapping views.
Publishable results are expected soon
Data association:
The problem of data association is, given sequences as follows, how to optimally assign each object so that error rate is reduced?
