When Google announced in February 2015 that its DeepMind AI beat 49 Atari video games, it was a major step forward for machine learning, but it left the team behind DeepMind with a frustrating situation. Even though the AI could beat the games, it needed to be retrained to beat each new game, and quickly forgot how to play a game after it had moved on to the next one.
The researchers at DeepMind, however, have developed an algorithm to solve the problem: Elastic Weight Consolidation (EWC).
As explained in a paper in the Proceedings of the National Academy of Sciences journal and an accompanying blog post, EWC allowed DeepMind’s AI to overcome the issues it encountered back in 2014. With EWC, the AI is now able to retain knowledge of the games and is able to learn multiple games in succession.
DeepMind developed EWC in order to overcome a phenomenon known as “catastrophic forgetting,” where new tasks and adaptations overwrite previously acquired knowledge and memories. The researchers explain in the paper that this usually occurs because the deep neural networks used for machine learning are typically only capable of learning multiple tasks when data is presented all at once. Human brains, in contrast, learn things sequentially.
While the researchers state that today’s “computer programs cannot learn from data adaptively and in real time,” they “hope that this research represents a step towards programs that can learn in a more flexible and efficient way.” They also say that their research “progresses our understanding of how consolidation happens in the human brain” by using neuroscience-based theories about acquired skills and memories in human brains. DeepMind’s further development of machine learning and memory represents progress toward more advanced AI, even though today’s AI tech still face major problems.
Previous DeepMind experiments include pitting AIs against each other to see if they would fight or cooperate, and defeating legendary Go player Lee Se-Dol.
