The concept of artificial intelligence, which seemed so much like science fiction a few decades ago, has made real, practical inroads in producing results that organizations can find useful. What’s making those results happen, though, isn’t esoteric pie-in-the-sky theory: it’s creating statistical models that have been trained to make decisions. And trained a lot.
Artificial intelligence itself is a term that, for now, has had less of a focus than the more results-oriented machine learning, where a computer system is given input and output data and then is directed to infer the mathematical rules that govern the transformation of that data.
“It’s like pointing a program to look at the solar system and then have it figure out the laws of motion that govern a planetary system,” explained Sanjay Arora.
Arora, a Data Scientist in Red Hat’s Office of the CTO, is particularly interested in one very specific sliver of machine learning technology: unsupervised deep learning. To understand what that is, it’s important to first understand two types of machine learning: supervised and unsupervised.
Types of machine learning
Supervised learning, Arora explained, is when a machine learning algorithm is given a set of data that consists of inputs to and outputs from a system. In this case, the data scientist “knows what the answer is. You have the answer key which is the set of outputs.” The aim of the machine-learning algorithm is to figure out the rules governing the mapping from inputs to outputs.
Supervised learning is a great tool to train machine learning models, because it can quickly be determined when a machine learning model is heading in the right direction, or not. But because of the dearth of labelled data and the high cost of manually labeling data, the benefits of supervised learning are often unrealized.
The next question for data scientists like Arora, then becomes, can we find the patterns within datasets without having the answers in the first place? This, then, is unsupervised learning: pointing an application at a set of data and seeing what, if any, patterns can be inferred from that data.
For example, perhaps a machine learning application could look at a set of reading habits for a population and break that population into clusters, where each cluster represents the college major of individuals in that grouping. Or perhaps look at all the financial data from early 2007 and determine what, if any, patterns might have been present that were harbingers of the Great Recession. And, more importantly, can such patterns be found again to predict future financial turmoil accurately?
Many organizations are collecting massive amounts of unlabeled data that could be made useful with unsupervised learning.
Deep learning, the art and science of using modern neural networks, provides very powerful tools to solve unsupervised problems. “These models don’t need much a priori information”, Arora said, which also makes them very good at image or natural-language analysis.
Pictures of cats
There are many ways machine learning researchers and data scientists are approaching unsupervised deep learning. One method Arora described is using so-called generative adversarial networks (GANs). These models can look at data–say, images of cats–and then understand the statistical patterns and correlations between the pixels of the images. They can then be used to generate new cat images that are completely different from anything the model saw but still look like cats to humans. The goal is to use these models and small amounts of labeled data to generate more instances of data from each label, which can then be used to train a supervised model.
Currently, Arora sees such techniques as being in their initial developmental stages. GANs still need a lot of data in order to learn the hidden structure but the hope is that with time, more sophisticated learning techniques as well as better neural network architectures will enable the training of GANs with much smaller datasets. The ultimate goal would be to need as little data as a human being does. After all, a human child only needs to look at a few cats to be able to recognize cats in the wild.
With the use of unsupervised and supervised data models, machine learning is growing up fast, getting ready for application in business and science sectors. Practice does indeed make perfect.
You can learn more about this topic at DevConf.US when Arora and his colleague Uli Drepper present “An Introduction to Unsupervised Deep Learning” on Aug 15 at DevConf.US.