Yibie.Log

3D Face Reconstruction from a Single Image

Machine Learning research from University of Nottingham School of Computer Science can generate a 3D model of a human face from an image using neural networks:

3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, we propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions.       

There is an online demo which will let you upload an image to convert and even save as a 3D model here

Back in May I shared a list of books that TED speakers had mentioned on Twitter. That list was a little uneven because a mention isn’t necessarily a recommendation and some speakers aren’t on Twitter while others tweet books constantly. This list of 101 books to dive into this summer compiled by TED is much more cohesive and useful. Among the picks that sounded interesting:

Pocket Atlas of Remote Islands: Fifty Islands I Have Not Visited and Never Will by Judith Schalansky.

The Atlas of Remote Islands, Judith Schalansky’s beautiful and deeply personal account of the islands that have held a place in her heart throughout her lifelong love of cartography, has captured the imaginations of readers everywhere. Using historic events and scientific reports as a springboard, she creates a story around each island: fantastical, inscrutable stories, mixtures of fact and imagination that produce worlds for the reader to explore.

The More They Disappear by Jesse Donaldson.

There’s a lot of talk about the opioid crisis these days, but what’s missing from the statistics is the human story, the understanding of why people are making the choices they do. This novel, which focuses on Kentucky in the 1990s, gave me that understanding. After I finished it – which didn’t take long because I couldn’t put it down – I felt like I had physically been transported to that time and place.

Placing Outer Space: An Earthly Ethnography of Other Worlds by Lisa Messeri.

In Placing Outer Space Lisa Messeri traces how the place-making practices of planetary scientists transform the void of space into a cosmos filled with worlds that can be known and explored. Making planets into places is central to the daily practices and professional identities of the astronomers, geologists, and computer scientists Messeri studies.

Behave: The Biology of Humans at Our Best and Worst by Robert Sapolsky.

Due to my work filming with former jihadis, I’ve become very interested in understanding more about human interaction. Behave explores human nature, from the firing of a synapse all the way to the broader effects of culture. Based on a wide and multidisciplinary knowledge of science, this book provides a fascinating exploration of humanity, which might give us some important information on how we can work towards a better future for us all.

The Monster at the End of This Book by Jon Stone.