Deep Learning Fundamentals

ABOUT THIS COURSE

Get a crash course on the what there is to learn and how to go about learning more. Deep Learning presents a simplified explanation of some of the hottest topics in data science today:

• What is Deep Learning?
• What are are convolutional neural networks?
• Why is deep learning so powerful and what can it be used for?
• Be part of a rapidly growing field in data science; there's no better time than now to get started with neural networks.

COURSE SYLLABUS

• Module 1 - Introduction to Deep Learning
  1. Why Deep Learning?
  2. What is a neural network?
  3. Three reasons to go Deep
  4. Your choice of Deep Net
  5. An old problem: The Vanishing Gradient

• Module 2 - Deep Learning Models
  1. Restricted Boltzmann Machines
  2. Deep Belief Nets
  3. Convolutional Networks
  4. Recurrent Nets

• Module 3 - Additional Deep Learning Models
  1. Autoencoders
  2. Recursive Neural Tensor Nets
  3. Deep Learning Use Cases

• Module 4 - Deep Learning Platforms and Software Libraries
1. What is a Deep Learning Platform?
2. H2O.ai
3. Dato GraphLab
4. What is a Deep Learning Library?
5. Theano
6. Caffe
7. TensorFlow

GENERAL INFORMATION

• This course is free.
• It is self-paced.
• It can be taken at any time.
• It can be audited as many times as you wish.

Answers

Module 1: Introduction to Deep Learning Answers

Q1-Select the reason(s) for using a Deep Neural Network

• Some patterns are very complex and can’t be deciphered precisely by alternate means
• Deep Nets are great at recognizing patterns and using them as building blocks in deciphering inputs
• We finally have the technology – GPUs – to accelerate the training process by several folds of magnitude
• All of the above

Q2-What is TRUE about the functions of a Multi Layer Perceptron?

• The first neural nets that were born out of the need to address the inaccuracy of an early classifier, the perceptron.
• It predicts which group a given set of inputs falls into.
• It generates a score that determines the confidence level of the prediction.
• All of above.  

Q3-Why is the vanishing gradient a problem?

• Training is quick if the gradient is large and slow if its small
• With backprop, the gradient becomes smaller as it works back through the net
• The gradient is calculated multiplying two numbers between 0 and 1
• All of above.

Module 2: Deep Learning Models Answers

Q1-For Unsupervised Learning, which of the following deep nets would you choose?

• Autoencoder or Restricted Boltzmann Machines
• Deep Belief Nets
• Convolutional Nets
• Recurrent Nets

Q2-True or False: The RELU activation has no effect on back-propagation and the vanishing gradient

• True
• False

Q3-True or False: Convolutional Nets are the right model when dealing with data that changes over time because of their built-in feedback loop, allowing them to serve as a forecasting engine.

• True
• False

Module 3: Additional Deep Learning Models Answers

Q1- Which of the following are use cases of Deep nets?

• Sentiment Analysis of text data.
• Offering personalized ads based on user activity history.
• Flagging a transaction as fraudulent.
• Analyze and segment customers based on digital activity and footprint.
• Using satellite feeds and sensor data to detect changes in environmental conditions.
• All of the above.  

Q2- Which of the following are use cases of machine vision. Select all that apply.

• Image classification and tagging
• Sentiment Analysis
• Face Detection
• Video Recognition
• Speech Recognition

Q3- Which of the following is a good application of an RNTN?

• If the patterns change through time
• For general classification problems
• If there is an unknown hierarchy inherent in the input features
• For Supervised Fine-tuning
• To determine the relative importance in the input features

Module 4: Deep Learning Platforms & Libraries Answers

Q1- Which of the following is not an aspect of a deep net platform?

• Choice of deep net models
• Ability to integrate data from multiple sources
• Manage deep net models from the UI
• Under the hood performance enhancements to allow for fast training and execution
• Deriving the optimal hyper-parameter configuration

Q2- What are the different aspects of a Deep Learning Library?

• They are a set of pre-built functions and modules that you can call through your own programs
• Usually maintained by high-performance teams and are regularly updated
• Most are open source and have a large community that contribute to the code base
• All of above.

Q3-True or False: Theano, Caffe, and TensorFlow are examples of deep learning platforms.

• True
• False

Final Exam Answers – Deep Learning Fundamentals

Q1- For supervised learning, which of the following deep nets would you choose?

• Autoencoder
• Deep Belief Nets
• Convolutional Nets
• Restricted Boltzmann Machines
• Recurrent Nets

Q2- Which of the following is true with respect to the training process of a deep net?

• The Cost is the difference between the net’s predicted and actual outputs.
• The training process utilizes gradients which measure the rate at which the weights and biases change with respect to the cost.
• The objective of the training process is to make the cost as low as possible.
• The training process utilizes a technique called back-propagation.
• All of above.

Q3- True or False: With backprop, the early layers train slower than the later ones, making the early layers incapable of accurately identifying the pattern building blocks needed to decipher the full pattern.

• True
• False

Q4- For image recognition, which of the following deep nets would you choose? Select all that apply.

• Autoencoder
• Deep Belief Nets
• Convolutional Nets
• Restricted Boltzmann Machines
• Recurrent Nets

Q5- How does the Deep Belief Network (DBN) solve the vanishing gradient? Select all that apply.

• It uses a stack of RBMs to determine the initial weights and biases, where the output of any RBM forms the input to the next RBM.
• It uses a small labelled data set to associate patterns learned by the RBMs to classes.
• It utilizes supervised fine-tuning, resulting in tweaks in weights and biases and a slight improvement in accuracy.
• It quickly moves through solution states – set of weights and biases – going from one to another based on a reward.
• The complete process – RBMs for pre-training and supervised fine-tuning – results in a very accurate net which trains in an acceptable time.

Q6- True or False: To train, a DBN combines two Learning methods – supervised and unsupervised.

• True
• False

Q7- Which of the following is the most popular use of a Convolutional Net?

• Image Recognition
• Object Recognition in an Image
• Time Series Forecasting
• Supervised Fine Tuning
• General classification

Q8- Which of the following are True about a RBM? Select all that apply.

• The RBM is part of the first attempt at beating the vanishing gradient and uses unlabelled data.
• It improves its own accuracy through self-correction.
• Its purpose is to re-create inputs and in doing so has to make decisions about which input features are more important.
• It stores the relative importance of the features as weights and biases.
• It predicts which group a given set of inputs falls into.

Q9- Which of the following statements are true about the architecture of a CNN? Select all that apply.

• A CNN can only have two types of layers: CONV and RELU.
• A RELU layer has to always be followed by a POOL layer.
• FC layers are usually found at the end.
• A CONV layer has a theoretical maximum number of filters.
• A typical CNN implementation has multiple repetitions of CONV, RELU and POOL layers, with sub-repetitions.

Q10- True or False: By definition, the classifier in the nodes of an MLP cannot be anything other than the Perceptron.

• True
• False

Q11- Which of the following are differences between a Recurrent Net and a Feedforward Net? Select all that apply.

• Recurrent Nets feed the output of any time step back in as input for the next step.
• Recurrent Nets are used for time series forecasting.
• Recurrent Nets can output a sequence of values.
• Recurrent Nets are trained using back-propagation.
• The nodes in a recurrent nets have a classifier that activate and produce a score.

Q12- Which of the following statements are true about training a Recurrent Net? Select all that apply.

• Since RNNs use backprop, the vanishing gradient is a problem.
• The number of time steps used for training has no bearing on the severity of the vanishing gradient problem.
• The vanishing gradient can potentially lead to decay of information through time.
• The most popular technique to address the vanishing gradient is the use of gates.
• The only technique to address the vanishing gradient is the use of gates.

Q13- True or False: Deep Autoencoders are used for dimensionality reduction.

• True
• False

Q14- Which of the following are true about Autoencoders? Select all that apply.

• It improves its own accuracy through self-correction.
• Its purpose is to re-create inputs and in doing so has to make decisions about which input features are more important.
• A Restricted Boltzmann Machine is a type of Autoencoders.
• It stores the relative importance of the features as weights and biases.
• It predicts which group a given set of inputs falls into.

Q15- True or False: Given they are mainly about machine vision, Convolutional Nets don’t really find a home in the field of medicine.

• True
• False