#alphago zero

The AlphaGo Zero cheat sheet

Mastering Chess and Shogi by Self-Play with a General Reinforcement  Learning Algorithm

Google’s DeepMind Achieves Machine Learning Breakthroughs, Which is Terrifying

I wrote an article last week about the AlphaGo and AlphaGo Zero “robots” that used machine learning to beat humans in Go.  But I want to explore the technology behind what made all of that possible.  If you want to read more about that particular battle, check out my previous post.  Google is making progress in the field of machine learning at a startling rate.  Their AutoML recently made jaws…

Like the line of Alpha Go playing AIs, which play the board game Go! It was once imagined that a robot would never beat a human in chess, there were just too many moves. An AI would never be able to remember all possible “states” of the game, they thought. And yet it happened! AI beat man, and when it did, programmers didn't say "okay we won board games let's do something else!" They said, let's play another game! And that is just so cute.

So, Go playing AIs were developed, and initially, again, people laughed because of the rediculous complexity of Go. For quick reference, I found there to be roughly 3.8 x 10^46 possible boards in chess which is on an 8x8 board. Compare this to go which is a 19x19 board with states of black, white, empty. Therefore, roughly there are 3^361 states of a Go Board. That's impossible to properly imagine so back to the story.

Oh yeah, Alpha Go beat a human. What human? Lee sadol, the world's best Go player at the time. So, thanks to neural networks, and reinforcement learning, an AI was able to beat the best human at Go. Nice. We're done right? Finally humans would do something else with this ability? Nope, we made another AI that also plays Go, but this time we made it teach itself instead of telling it how to play.

That's right, through reinforcement learning, being told either it won or it lost, by initially random moves, and eventually figuring out tactics, it taught itself how to play Go. After I believe 40 hours of playing itself? It beat the previous version. Alpha Go Zero (instructions) beat the previous version Alpha Go, 100-0.

So while I'm sure we're now making an AI that will make another AI that plays Go, let's just enjoy how happy humans are to make bots that play board games.

In 2016, DeepMind’s AlphaGo program outsmarted the world’s top Go player. In 2017 AlphaGo Zero was created: unlike AlphaGo, AlphaGo Zero wasn’t trained using previous human games of Go, but was simply given the rules of Go—and in four days it defeated the AlphaGo program.

Our own biology is, of course, vastly more complex than the game of Go, and that, Metzl said, is our starting point. “The system of our own biology that we are trying to understand is massively, but very importantly not infinitely, complex,” he added.

(...)Multiple countries already starting to produce this data. The UK’s National Health Service recently announced a plan to sequence the genomes of five million Britons over the next five years. In the US the All of Us Research Program will sequence a million Americans. China is the most aggressive in sequencing its population, with a goal of sequencing half of all newborns by 2020.

“We’re going to get these massive pools of sequenced genomic data,” Metzl said. “The real gold will come from comparing people’s sequenced genomes to their electronic health records, and ultimately their life records.” Getting people comfortable with allowing open access to their data will be another matter; Metzl mentioned that Luna DNA and others have strategies to help people get comfortable with giving consent to their private information. But this is where China’s lack of privacy protection could end up being a significant advantage.

To compare genotypes and phenotypes at scale—first millions, then hundreds of millions, then eventually billions, Metzl said—we’re going to need AI and big data analytic tools, and algorithms far beyond what we have now. These tools will let us move from precision medicine to predictive medicine, knowing precisely when and where different diseases are going to occur and shutting them down before they start.

But, Metzl said, “As we unlock the genetics of ourselves, it’s not going to be about just healthcare. It’s ultimately going to be about who and what we are as humans. It’s going to be about identity.”

“We all need to be part of an inclusive, integrated, global dialogue on the future of our species,” Metzl said. “Healthcare professionals are essential nodes in this.” Not least among this dialogue should be the question of access to tech like IVG; are there steps we can take to keep it from becoming a tool for a wealthy minority, and thereby perpetuating inequality and further polarizing societies?