Download The Free eBook

The Skymind Wiki eBook: Part 5

The Bleeding Edge in Deep Learning

An overview of bleeding-edge architectures that are driving state-of-the-art advancements in AI.

Now that you understand the basics -- feed-forward, recurrent . and convolutional networks -- it's time to look at some of the more exotic varieties of deep learning.

Generative Adversarial Networks, or GANs, are a popular and fast evolving type of neural network, famous for their ability to generate visual art.

Some of the most cutting-edge architectures combine neural networks with other technologies. For example, DeepMind's AlphaGo combined deep reinforcement learning with convolutional networks to win the board game Go. Reinforcement learning is a type of goaloriented machine learning that can be applied to real-world problems where actions must be taken.

Finally, attention mechanisms, memory networks and transformers combine differentiable neural networks with keyvalue stores to better understand context in natural language. These are at the heart of the huge, recent advances in language models that have allowed researchers to transfer learning from one corpus to another, and greatly improved machine accuracy in NLP tasks.

Altogether, these cutting-edge algorithms represent our best attempts to approximate artificial general intelligence. While we're still a long way from AGI, some derivative, some combination, or some scaled-up version of these algorithms just might get us there.