Summary of Deep Learning History and Recent Timeline

00:00:00 - 00:20:00

This video provides a history of deep learning and a recent timeline of deep learning development. The video explains that deep learning is a kind of last word in image recognition, and that it is hard to imagine generating more accurate images than style gans is able to. It also shows that deep learning is able to generate variation in the position of hair in a person's head, shocking for imagenet. However, variational auto encoders were never competitive with gans in terms of the ability to generate images, but they have certain advantages with a gan. With a gan, you're never really sure if it's covering the space of all faces if there's some kind of face that's missing. Variational auto encoders have a quantitative measure of its ability to cover the whole space of images in the distribution, and to me that makes it in that aspect a much more compelling model.

• 00:00:00 The history of deep learning covers a long period of time, from the early days of neural networks to more recent developments. In the 1980s and 1990s, deep learning was in the shadows due to the dominance of symbolic methods, but developments in long-term memories (1997) and neural networks (1998) led to a revival.
• 00:05:00 In 2003, computer vision achieved a major breakthrough with the development of deep learning. In 2012, deep learning achieved another major breakthrough with the success of a computer vision competition called AlexNet. In 2014, deep learning achieved another major breakthrough with the development of neural machine translation. In 2015, deep learning achieved another major breakthrough with the development of residual connections. In 2016, deep learning achieved another major breakthrough with the development of alpha zero. In 2017, deep learning achieved another major breakthrough with the success of a game playing machine called Alphago.
• 00:10:00 In 2017, artificial intelligence (AI) made significant progress in a variety of areas, including self-play leading to superhuman abilities in computer chess, unsupervised machine translation, and gpt-3, a new language model that achieved a number one ranking on Hacker News. While it is unclear if these advances are objectively significant, they are garnering a lot of attention.
• 00:15:00 In recent years, deep learning has seen dramatic improvements in facial recognition and style generation. However, there is still some room for improvement.
• 00:20:00 This YouTube video provides a history of deep learning and a recent timeline of deep learning development. The video explains that deep learning is a kind of last word in image recognition, and that it is hard to imagine generating more accurate images than style gans is able to. It also shows that deep learning is able to generate variation in the position of hair in a person's head, shocking for imagenet. However, variational auto encoders were never competitive with gans in terms of the ability to generate images, but they have certain advantages with a gan. With a gan, you're never really sure if it's covering the space of all faces if there's some kind of face that's missing. Variational auto encoders have a quantitative measure of its ability to cover the whole space of images in the distribution, and to me that makes it in that aspect a much more compelling model. In 2019, there was a development in variational autoencoders called the vector quantized variation of autoencoders, which suddenly became competitive with gans in terms of image generation. This video leaves the audience with the question of will Moore's law continue and where students will take us next in terms of deep learning development.