#semantic labelling

Fast Pix2Pix - Update

Interesting additions to project by Zaid Alyafeai which implements neural network image translation speedily in-browser. As well as improved training sets for older subjects, you can now doodle with a Pokemon dataset.

Most impressive is the new Faces v2 addition, which employs semantic labelling method to specify certain facial details within your drawing for more accurate results.

Uncanny Rd

Project by Anastasis Germanidis and Cristóbal Valenzuela is an online Neural Network semantic painting tool. Trained on visual road data, you can doodle or place preset markers which will be visually translated into another image:

Uncanny Road is an experimental tool for collectively synthesizing a never-ending road using Generative Adversarial Neural Networks. It is based on the pix2pixHD project, published by @nvidia and UC Berkeley (Project, Paper and Code), that allows for photorealistic image-to-image translation. The pix2pix model was trained using adversarial learning on the Cityscapes dataset, containing thousands of street images.

To synthesize street images, draw on the colormap of the scene. Each color represents a different kind of object label (e.g. road, building, vegetation, etc.) that the neural network can understand

pix2pixHD

Research from @nvidia and UC Berkeley have updated their neural network image translation framework to run with a semantic interface and produce results at a much higher resolution:

We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs). Conditional GANs have enabled a variety of applications, but the results are often limited to low-resolution and still far from realistic. In this work, we generate 2048x1024 visually appealing results with a novel adversarial loss, as well as new multi-scale generator and discriminator architectures. Furthermore, we extend our framework to interactive visual manipulation with two additional features. First, we incorporate object instance segmentation information, which enables object manipulations such as removing/adding objects and changing the object category. Second, we propose a method to generate diverse results given the same input, allowing users to edit the object appearance interactively. Human opinion studies demonstrate that our method significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing. 

More Here

CASENet: Deep Category-Aware Semantic Edge Detection

Computer Vision paper from Mitsubishi Electric Research Labs (MERL) uses machine learning to break down a visual feed into identifiable parts:

Boundary and edge cues are highly beneficial in improving a wide variety of vision tasks such as semantic segmentation, object recognition, stereo, and object proposal generation. Recently, the problem of edge detection has been revisited and significant progress has been made with deep learning. While classical edge detection is a challenging binary problem in itself, the category-aware semantic edge detection by nature is an even more challenging multi-label problem. We model the problem such that each edge pixel can be associated with more than one class as they appear in contours or junctions belonging to two or more semantic classes. To this end, we propose a novel end-to-end deep semantic edge learning architecture based on ResNet and a new skip-layer architecture where category-wise edge activations at the top convolution layer share and are fused with the same set of bottom layer features. We then propose a multi-label loss function to supervise the fused activations. We show that our proposed architecture benefits this problem with better performance, and we outperform the current state-of-the-art semantic edge detection methods by a large margin on standard data sets such as SBD and Cityscapes.