One of the biggest misconceptions about artificial intelligence is the belief that today’s AIs possess generalized intelligence. We are really good at leveraging large data sets to accomplish specific tasks, but fall flat at replicating the breadth of human intelligence.
If we’re going to move toward generalized intelligence, Facebook wants to make sure we know how to evaluate progress. In a paper, Facebook’s AI Research (FAIR) lab outlines just that as part of its CommAI framework.
First, the team argues that a proper generalized AI should be able to fluently communicate in natural language with humans. Researchers in the field of natural language processing have been working on the problem of human computer interaction through language for some time now, but we have a long way to go before our computational systems can rival humans.
This is obvious if you’ve spent any time talking to the latest, greatest conversational AI systems on the market, like Apple’s Siri and Amazon’s Alexa. We will need our systems to be able to communicate and learn through language effectively, even when lacking context and discussing things in undefined terms.
Moreover, such a system should be capable of learning new skills easily. Facebook calls this skill set “learning to learn.” Today’s machine learning models can be trained on data and used for classifying defined objects. We can use transfer learning to quickly adapt a model to accomplish the same task on new data, but our machines can’t teach themselves completely new skills without heavy intervention from developers.
“It’s generally agreed that, in order to generalize across tasks, a program should be capable of compositional learning, that is, of storing and re-combining solutions to sub-problems across tasks,” the team adds.
In a nod to the growing sub-field of reinforcement learning, Facebook also notes that a generalized AI should resemble a human’s ability to master new tasks with decreasing explicit rewards. And with respect to interface, modern systems should be able to take in new information and express themselves in a variety of ways that fit with the demands of diverse situations.
Facebook considers these capabilities to be more of a prerequisite to generalized AI than a true Turing test. The original Turing test was created by Alan Turing in 1950. It’s generally understood to be a means of evaluating machine intelligence with respect to human intelligence.
But as the field of AI has matured, the test has lost much of its relevance. Facebook provides a nice alternative way to think about the important requirements of a modern generalized AI that should be less of a research distraction than the more rigid Turing test.
The FAIR team of Marco Baroni, Armand Joulin, Allan Jabri, Germán Kruszewski, Angeliki Lazaridou, Klemen Simonic and Tomas Mikolov also developed an open source platform for testing and training AI systems. Similar to OpenAI’s Gym, the CommAI-env tool focuses on more incremental tasks that build on previous accomplishments. It’s also designed to push models to their limits by emphasizing task variation.
Featured Image: GraphicaArtis/Getty Images
