Computer vision researchers are beginning to tackle the highly salient field of phrenology. Every human culture has believed that inferring character from physical features is possible. Whether humans are merely superstitious or employing useful mental heuristics, they tend to agree with each other about which faces look dominant, attractive, trustworthy, or extroverted. This paper recreates this ability with objective computational pattern-matching, and the results generally agree with human intuition.
The particular concern of this paper is to build a computer program that predicts whether a person is a criminal or an ordinary citizen, using only an image of their face. Though it ought to be fairly easy to distinguish “normal”-looking faces from abnormal ones, this subject has received little recent attention due to its historical association with the horrors perpetrated by adherents to social Darwinism. The objectivity of electronics eliminates problems like ideological bias and human incompetence, and the field of computer vision has finally reached a level where computers can match or exceed human abilities in facial recognition.
This paper uses standard ID photographs rather than computer-generated face models, which would not be representative of actual criminal and non-criminal populations. The dataset consisted of 1,856 Chinese men, 1,126 of whom were non-criminals. Many aspects of the photos were normalized to avoid confounding variables, such as lighting intensity.
Four machine learning methods were employed: Nearest Neighbor, Logistic Regression, Support Vector Machine, and Convolutional Neural Network. The first three of these rely on a vector of facial landmark points, a PCA vector thereof, an LBP vector thereof, and finally a concatenation of these vectors. Each of the four methods was then tested several times against random samples from the population and the results were averaged to yield its predictive power.
The Convolutional Neural Network had extraodinary predictive power, averaging 89.51% accuracy, and the others performed with 78% accuracy or better.
(This takes a bit longer than I expected. To be continued…)