Groundbreaking Architectures

ImageNet

Over 1 million images of 1000 classes of images. Here are some models that performed extremely well with this dataset.

AlexNet (2012)

• Pioneered usage of ReLU and Dropout to avoid overfitting

• Had large convolutional filters (11x11)

• By UofT

VGG (2014)

• VGG-16, VGG-19

• Long sequence of 3x3 convolutional, 2x2 Pooling, 3 FC layers

• Pioneered usage of small 3x3 convolutional filters

• By Oxford University

ResNet (2015)

• ResNet-152

  • With the enormous # of layers, Vanishing Gradient Problem was discovered

  • They solved this by adding skip layers which shortened the route for gradients to travel (initially, the gradients had to travel the entire 152 layers which is a lot)

• By Microsoft Research