Clement Creusot's Home Page

A Machine-learning Approach to Keypoint Detection and Landmarking on 3D Meshes
Clement Creusot, Nick Pears, and Jim Austin
Special Issue on 3D Imaging, Processing and Modeling Techniques - International Journal of Computer Vision (IJCV), 2013

Paper [pdf, 5.1MB]
Supplemental materials
- Landmarking results on the FRGC database: [tar.gz, 2.8 MB]
- Landmarking results on the Bosphorus database: [tar.gz, 2.3 MB]
Springer Link: 10.1007/s11263-012-0605-9
Abstract

We address the problem of automatically detecting a sparse set of 3D mesh vertices, likely to be good candidates for determining correspondences, even on soft organic objects. We focus on 3D face scans, on which single local shape descriptor responses are known to be weak, sparse or noisy. Our machine-learning approach consists of computing feature vectors containing D different local surface descriptors. These vectors are normalized with respect to the learned distribution of those descriptors for some given target shape (landmark) of interest. Then, an optimal function of this vector is extracted that best separates this particular target shape from its surrounding region within the set of training data. We investigate two alternatives for this optimal function: a linear method, namely Linear Discriminant Analysis, and a non-linear method, namely AdaBoost. We evaluate our approach by landmarking 3D face scans in the FRGC v2 and Bosphorus 3D face datasets. Our system achieves state-of-the-art performance while being highly generic.

Citations

Real-time small obstacle detection on highways using compressive RBM road reconstruction
Clement Creusot and Asim Munawar
IEEE Intelligent Vehicles Symposium (IV), 2015, Seoul, South Korea.

Paper [pdf, 1.4 MB]
Abstract

Small objects on the road can become hazardous obstacles when driving at high speed. Detecting such obstacles is vital to guaranty the safety of self-driving car users, especially on highways. Such tasks cannot be performed using existing active sensors such as radar or LIDAR due to their limited range and resolution at long distances. In this paper we propose a technique to detect anomalous patches on the road from color images using a Restricted Boltzman Machine neural network specifically trained to reconstruct the appearance of the road. The differences between the observed and reconstructed road patches yield a more relevant segmentation of anomalies than classic image processing techniques. We evaluated our technique on texture-based synthetic datasets as well as on real video footage of anomalous objects on highways.

Citations

Real-time Barcode Detection in the Wild
Clement Creusot and Asim Munawar
IEEE Winter conference on Applications of Computer Vision (WACV), 2015, Waikoloa Beach, Hawaii.

Paper [pdf, 3.7 MB]
Abstract

The linear 1D barcode is the main tagging system for billions of products sold each day. Barcodes have many advantages but require a laser scanner for fast and robust scanning. Solutions exist to read barcodes from cellphones but they assume a carefully framed image within the field of view. This undermines the true potential of barcodes in a wide range of scenarios. In this paper we present a real-time technique to detect barcodes in the wild from video streams. Our technique outperforms the state-of-the-art passive techniques both in accuracy and speed. Potential commercial applications enabled by such passive scanning system are also discussed in this paper.

Citations

Local Segmentation for Pedestrian Tracking in Dense Crowds
Clement Creusot
Multimedia Modeling, 20th International Conference, MMM 2014, Dublin, Ireland, January 8-10, 2014

Paper [pdf, 2MB]
Supplemental materials
Abstract

People tracking in dense crowds is challenging due to the high levels of inter-pedestrian occlusions occurring continuously. After each successive occlusion, the surface of the tracked object that has never been hidden reduces. If not corrected, this shrinking problem eventually causes the system to stop as the area to track become too small. In this paper we investigate how hidden parts of one target object can be recovered after occlusions and propose challenging data to evaluate such segmentation-tracking technique in dense crowds. The segmentation/tracking problem is particularly difficult to solve for non-rigid objects. Here, we focus on pedestrians whose limbs and lower body parts often get occluded in crowded scene. We first investigate the unmet challenges of pedestrian tracking in crowds and propose a challenging video to evaluate segmentation-tracking robustness to inter-pedestrian occlusions. We then detail a fast segmentation-based method to overcome some aspects of the tracking-under-occlusion problem. We finally compare our results with two existing tracking methods.

Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local Shapes
Clement Creusot, Nick Pears, and Jim Austin
3DIMPVT 2011, pp. 204-211, Hangzhou, China.

Paper [pdf, 4.2 MB]
Slides [pdf, 3.4 MB]
IEEE Link: 10.1109/3DIMPVT.2011.33
Abstract

Keypoints on 3D surfaces are points that can be extracted repeatably over a wide range of 3D imaging conditions. They are used in many 3D shape processing applications; for example, to establish a set of initial correspondences across a pair of surfaces to be matched. Typically, keypoints are extracted using extremal values of a function over the 3D surface, such as the descriptor map for Gaussian curvature. That approach works well for salient points, such as the nose-tip, but can not be used with other less pronounced local shapes. In this paper, we present an automatic method to detect keypoints on 3D faces, where these keypoints are locally similar to a set of previously learnt shapes, constituting a 'local shape dictionary'. The local shapes are learnt at a set of 14 manually-placed landmark positions on the human face. Local shapes are characterised by a set of 10 shape descriptors computed over a range of scales. For each landmark, the proportion of face meshes that have an associated keypoint detection is used as a performance indicator. Repeatability of the extracted keypoints is measured across the FRGC v2 database.

Citations

Talk: Reading robot minds, literally.
Clement Creusot and Asim Munawar
Memory Inspired Robot Architecture (MemX) Workshop at the Robot and Human Interactive Communication Symposium (RO-MAN) , Kobe, Japan, August 31th, 2015

Abstract

Conversation, the action of talking in turns between two or more agents, is yet to be fully achieved in robotic systems. Some limited aspects of the problem such as speech-to-text conversion, semantic analysis or question answering have seen tremendous improvement in recent years. However, solving context-dependent ambiguities, showing common sense, or being able to improvise in non-predefined situations are still daunting challenges for the community. While humans use vision to evaluate other people faces and body language, the amount of information conveyed by such medium is relatively limited. This can be explained by the fact humans compete with each others and might have evolutionary interests not to share their thoughts entirely. Humans can show emotions with their face, but cannot communicate higher level semantic using such display system. As far as robots are concerned, there is no reason to limit such communication channel while communicating with humans. We can of course give faces to robot to display emotions, but we can also give them other display organs to show even more of their mind set, memory states and expectations. Here we discuss possible real-time visualizations of a robot mind on a screen display to help the human agent interact more efficiently with the machine. We argue that the ability of a robot to display his current memory states and processes in a human readable form, and consequently, to have his mind read by humans, is a very important tool to facilitate human-machine conversations.

Ground Truth for Pedestrian Analysis and Application to Camera Calibration
Clement Creusot and Nicolas Courty
Ground Truth Workshop at the Computer Vision and Pattern Recognition Conference (CVPR) 2013, Portland, Oregon.

Paper [pdf, 0.9 MB]
Supplemental materials
- Example of Per-frame Results [avi, 39 MB]
Abstract

This paper investigates the use of synthetic 3D scenes to generate ground truth of pedestrian segmentation in 2D crowd video data. Manual segmentation of objects in videos is indeed one of the most time-consuming type of assisted labeling. A big gap in computer vision research can not be filled due to this lack of temporally dense and precise segmentation ground truth on large video samples. Such data is indeed essential to introduce machine learning techniques for automatic pedestrian segmentation, as well as many other application involving occluded people. We present a new dataset of 1.8 million pedestrian silhouettes presenting human-to-human occlusion patterns likely to be seen in real crowd video data. To our knowledge, it is the first publicly available large dataset of pedestrian in crowd silhouettes. Solutions to generate and represent this data are detailed. We discuss ideas of how this ground truth can be used for a large number of computer vision applications and demonstrate it on a camera calibration toy problem.

Citations

3D Landmark Model Discovery from a Registered Set of Organic Shapes
Clement Creusot, Nick Pears, and Jim Austin
Point Cloud Processing (PCP) Workshop at the Computer Vision and Pattern Recognition Conference (CVPR) 2012, Providence, Rhode Island.

Paper [pdf, 4.9 MB]
Slides [pdf, 5.7 MB]
IEEE Link: 10.1109/CVPRW.2012.6238915
Abstract

We present a machine learning framework that automatically generates a model set of landmarks for some class of registered 3D objects: here we use human faces. The aim is to replace heuristically-designed landmark models by something that is learned from training data. The value of this automatically generated model is an expected improvement in robustness and precision of learning-based 3D landmarking systems. Simultaneously, our framework outputs optimal detectors, derived from a prescribed pool of surface descriptors, for each landmark in the model. The model and detectors can then be used as key components of a landmark-localization system for the set of meshes belonging to that object class. Automatic models have some intrinsic advantages; for example, the fact that repetitive shapes are automatically detected and that local surface shapes are ordered by their degree of saliency in a quantitative way. We compare our automatically generated face landmark model with a manually designed model, employed in existing literature.

Citations

3D face landmark labelling
Clement Creusot, Nick Pears, and Jim Austin
In Proc. ACM Workshop on 3D Object Retrieval, pp. 27-32, Firenze, Italy, Oct. 2010.

Win the SAIC UK Biometric Research Competition - Press release 1 Press release 2
Paper [pdf, 459 KB]
Slides [pdf, 1.3 MB]
ACM Link: 10.1145/1877808.1877815
Abstract

Most 3D face processing systems require feature detection and localisation, for example to crop, register, analyse or recognise faces. The three features often used in the literature are the tip of the nose, and the two inner corner of the eyes. Failure to localise these landmarks can cause the system to fail and they become very difficult to detect under large pose variation or when occlusion is present. In this paper, we present a proof-of-concept for a face labelling system, capable of overcoming this problem, as a larger number of landmarks are employed. A set of points containing hand-placed landmarks is used as input data. The aim here is to retrieve the landmark's labels when some part of the face is missing. By using graph matching techniques to reduce the number of candidates, and translation and unit-quaternion clustering to determine a final correspondence, we evaluate the accuracy at which landmarks can be retrieved under changes in expression, orientation and in the presence of occlusions.

Citations

Automatic Landmarking for Non-cooperative 3D Face Recognition
Clement Creusot
PhD thesis, University of York, Dept. of Computer Science, Dec. 2011.

Paper [pdf, 64 MB]
Slides [pdf, 25 MB]
White Rose Online link
Abstract

This thesis describes a new framework for 3D surface landmarking and evaluates its performance for feature localisation on human faces. This framework has two main parts that can be designed and optimised independently. The first one is a keypoint detection system that returns positions of interest for a given mesh surface by using a learnt dictionary of local shapes. The second one is a labelling system, using model fitting approaches that establish a one-to-one correspondence between the set of unlabelled input points and a learnt representation of the class of object to detect. Our keypoint detection system returns local maxima over score maps that are generated from an arbitrarily large set of local shape descriptors. The distributions of these descriptors (scalars or histograms) are learnt for known landmark positions on a training dataset in order to generate a model. The similarity between the input descriptor value for a given vertex and a model shape is used as a descriptor-related score. Our labelling system can make use of both hypergraph matching techniques and rigid registration techniques to reduce the ambiguity attached to unlabelled input keypoints for which a list of model landmark candidates have been seeded. The soft matching techniques use multi-attributed hyperedges to reduce ambiguity, while the registration techniques use scale-adapted rigid transformation computed from 3 or more points in order to obtain one-to-one correspondences. Our final system achieves better or comparable (depending on the metric) results than the state-of-the-art while being more generic. It does not require pre-processing such as cropping, spike removal and hole filling and is more robust to occlusion of salient local regions, such as those near the nose tip and inner eye corners. It is also fully pose invariant and can be used with kinds of objects other than faces, provided that labelled training data is available.

Citations

Clement Creusot, PhD

Publications (First Authored)

Journals

Conferences

Workshops

Others