#webtoy

My favorite memories from surfing the internet as a kid was finding weird webtoys that people built.

Fast Pix2Pix - Update

Interesting additions to project by Zaid Alyafeai which implements neural network image translation speedily in-browser. As well as improved training sets for older subjects, you can now doodle with a Pokemon dataset.

Most impressive is the new Faces v2 addition, which employs semantic labelling method to specify certain facial details within your drawing for more accurate results.

Pix2Pix Edges2Pichaku Example

New interactive Pix2Pix visual translation example from ml5js transforms your doodles based on a dataset trained on a Pokemon character.

1. Press your mouse to draw a Pikachu on the canvas.

2. Click 'Transfer' button.

3. A colored Pikachu image will appear in ~5s.

4. Click 'Clear' button to clear the canvas and draw again.

You can try it out for yourself here

Semantris

Online games from Experiments with Google where you score more points when you have typed words with the strongest association that appear on the screen:

Semantris is a set of word association games powered by machine-learned, natural language understanding technology.  Each time you enter a clue, the AI looks at all the words in play and chooses the ones it thinks are most related. Because the AI was trained on billions of examples of conversational text that span a large variety of topics, it’s able to make many types of associations. 

Appearing in two forms (Arcade for speed and Blocks for in your own pace), it is oddly addictive (though occasionally disappointing when you know you are right - if ‘webpage’ and ‘cat’ are present and you type ‘internet’, the game says ‘cat’ wins ....)

FaceVTuber

Japanese online avatar puppetry app lets you become an anime character for live online streaming. While not as sophisticated as paid apps with similar functionality, if you have a PC and can work with Japanese tutorials, you can use the PMCA character maker to customize and use your own avatar.

The video below is a Japanese tutorial so recommend to turn on translated captions:

Links to tools mentioned in the above tutorial can be found here

Real-time Human Pose Estimation in the Browser with TensorFlow.js

Machine Learning web demo from Google Creative Lab and Dan Oved can detect human poses via a webcam with web code:

In collaboration with Google Creative Lab, I’m excited to announce the release of a TensorFlow.js version of PoseNet¹,² a machine learning model which allows for real-time human pose estimation in the browser.

So what is pose estimation anyway? Pose estimation refers to computer vision techniques that detect human figures in images and video, so that one could determine, for example, where someone’s elbow shows up in an image. To be clear, this technology is not recognizing who is in an image — there is no personal identifiable information associated to pose detection. The algorithm is simply estimating where key body joints are.

Ok, and why is this exciting to begin with? Pose estimation has many uses, from interactive installations that react to the body to augmented reality, animation, fitness uses, and more. We hope the accessibility of this model inspires more developers and makers to experiment and apply pose detection to their own unique projects. While many alternate pose detection systems have been open-sourced, all require specialized hardware and/or cameras, as well as quite a bit of system setup. With PoseNet running on TensorFlow.js anyone with a decent webcam-equipped desktop or phone can experience this technology right from within a web browser. And since we’ve open sourced the model, Javascript developers can tinker and use this technology with just a few lines of code. What’s more, this can actually help preserve user privacy. Since PoseNet on TensorFlow.js runs in the browser, no pose data ever leaves a user’s computer.

You can find more background to the project here

3D Face Reconstruction from a Single Image

Machine Learning research from University of Nottingham School of Computer Science can generate a 3D model of a human face from an image using neural networks:

3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, we propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions.       

There is an online demo which will let you upload an image to convert and even save as a 3D model here

Beat Blender

Musical project from Google Creative Lab lets you generate beats with the assistance of Machine Learning, exploring variations with output support:

This experiment lets you explore and create beats in a fun new way using machine learning. Just drag the circle, or draw a path to discover beats. It's built using a neural network trained on over 3.8 Million drum beats. Built by Torin Blankensmith and Kyle Phillips with friends on the Magenta team at Google. Made with deeplearn.js.