16.4 Neural Networks

  • Deep learning uses artificial neural networks to learn
  • Similar to how scientists believe our brains work

Artificial Neural Network Diagram

  • The following diagram shows a three-layer artifical neural network
  • Circles represent neurons, lines between them simulate synapses—brain's connections between neurons
  • Output from one neuron becomes input to another
  • Diagram of a fully connected network
    • Not all neural networks are fully connected

Three-layer, fully connected neural network

How Artificial Neurons Decide Whether to Activate Synapses

  • During training, the network calculates weights for every connection between the neurons in one layer and those in the next
  • On a neuron-by-neuron basis, each of its inputs is multiplied by that connection’s weight—sum of those weighted inputs is passed to the neuron’s activation function
  • Activation function’s output determines which neurons to activate based on the inputs—just like neurons in your brain respond to inputs from your senses

How Artificial Neurons Decide Whether to Activate Synapses (cont.)

  • Diagram of a neuron receiving three inputs (black dots) and producing an output (hollow circle) that would be passed to all or some of neurons in the next layer, depending on the types of the neural network’s layers

Neuron receiving three inputs (the black dots) and producing an output (the hollow circle) that would be passed to all or some of neurons in the next layer, depending on the types of the neural network’s layers

  • w1, w2 and w3 are weights
  • In a new model that you train from scratch, these values are initialized randomly by the model

How Artificial Neurons Decide Whether to Activate Synapses (cont.)

  • As the network trains, tries to minimize error rate between network’s predicted labels and samples’ actual labels
  • Error rate is known as the loss
  • Calculation that determines the loss is the loss function
  • Backpropagation—During training, the network determines the amount that each neuron contributes to the loss, then adjusts the weights throughout the layers in an effort to minimize that loss

©1992–2020 by Pearson Education, Inc. All Rights Reserved. This content is based on Chapter 5 of the book Intro to Python for Computer Science and Data Science: Learning to Program with AI, Big Data and the Cloud.

DISCLAIMER: The authors and publisher of this book have used their best efforts in preparing the book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. The authors and publisher make no warranty of any kind, expressed or implied, with regard to these programs or to the documentation contained in these books. The authors and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of, the furnishing, performance, or use of these programs.