**Foreword**
Direct code is the most effective way to learn. This tutorial explains the BP backpropagation algorithm with a very simple example implemented in a short Python code.
Of course, the above procedure may be too concise. Below I will briefly break it down into several parts for discussion.
**Part 1: A Simple Neural Network**
A neural network trained with the BP algorithm attempts to predict the output based on input. Consider the scenario where given three columns of inputs, we try to predict the corresponding column of outputs. We can solve this problem by simply measuring the data of the input and output values. In this case, the leftmost column of input and output values are perfectly matched. Intuitively, the backpropagation algorithm uses this method to measure the statistical relationship between data and then obtain the model. Let's go straight to the topic and practice it.
**Layer 2 Neural Network**
After training, the output is as follows:
[[ 0.00966449]
[ 0.00786506]
[ 0.99358898]
[ 0.99211957]]
**Variable Definition**
- X: Input dataset in the form of a matrix with 1 training sample per row.
- y: Output dataset in the form of a matrix with 1 training sample per row.
- L0: Network layer 1, the input layer.
- L1: Layer 2 of the network, often referred to as the hidden layer.
- Syn0: First layer weight, connecting L0 and L1.
**Key Points to Understand the Program**
- Compare the state of the L1 layer at the first and last iteration.
- Look closely at the "nonlin" function, which gives us a probability value as an output.
- Carefully observe how L1_error changes during the iteration.
- Split the expression in line 36 to analyze; most of the secret weapons are inside.
- Carefully understand line 39 of the code, as all operations on the network are preparing for this step.
**Let’s Go Through the Code Line by Line**
Suggestion: Open this blog with two screens so you can read the article alongside the code. I did exactly that when writing this blog.
**Line 1:** Import numpy, a linear algebra tool library, which is the only external dependency in the program.
**Line 4:** Here is our "non-linear" part. The nonlinear mapping used is a function called "sigmoid." The Sigmoid function maps any value to a range between 0 and 1, allowing real numbers to be converted into probabilities. It also has nice features for neural network training.
**Line 5:** Note that the derivative of the sigmoid function is available through the "nonlin" function body (when the formal parameter deriv is True). One of the excellent features of the Sigmoid function is that its derivative can be obtained from its output value. If the output value of Sigmoid is represented by out, its derivative is simply out * (1 - out), which is very efficient.
**Line 10:** This line initializes our input dataset to a matrix in numpy. Each row is a "training instance" with one input node for each column. So, our neural network has 3 input nodes and 4 training instances.
**Line 16:** This line initializes the output dataset. To save space, I generated the dataset in a horizontal format. Using the transpose function ".T," the y matrix becomes 4 rows and 1 column, matching our input structure.
**Line 20:** Seeding your random number generator ensures consistent initial weights, making it easier to track how changes affect the training process.
**Line 23:** This line initializes the weight matrix for the neural network. The weight matrix dimension is (3,1) because the network has 3 inputs and 1 output. The random initialization has a mean of 0, which is common practice for simplicity.
**Line 25:** This line starts the neural network training loop. It iteratively adjusts the weights to better fit the training set.
**Line 28:** The first layer, L0, is our input data. We handle all instances at once, which is known as "batch" training. Even with multiple instances, the process remains the same.
**Line 29:** This is the forward prediction phase. The network tries to predict the output based on the given input. Then, adjustments are made to improve performance in each iteration.
**Matrix Multiplication:** The calculation involves multiplying L0 and Syn0, followed by applying the sigmoid function. The dimensions are (4x3) dot (3x1) = (4x1), resulting in four predictions.
**Line 32:** After predicting, we compare the results with the actual output. L1_error reflects the difference between the predicted and actual outputs.
**Line 36:** This is the core of the backpropagation algorithm. The error is multiplied by the derivative of the sigmoid function to adjust the weights effectively.
**Line 39:** The weight update is now ready. This step accumulates the updates for all training instances and adjusts the weights accordingly.
**Key Conclusions**
The neural network updates its weights based on the error and the slope of the activation function. When the prediction is confident, the update is small. When uncertain, the adjustment is larger. This allows the network to learn from its mistakes over time.
**Part II: A Slightly More Complex Issue**
Consider a scenario where the first two columns of inputs are used to predict the output. There is no direct relationship between individual columns and the output. However, the combination of the two columns determines the result. This is a non-linear pattern, similar to image recognition, where certain pixel combinations indicate specific objects.
**Our Strategy**
To handle such cases, we add an additional layer to the network. The first layer combines the inputs, and the second layer maps these combinations to the final output. This approach, known as deep learning, allows the network to model complex relationships by stacking layers.
**Part III: Summary**
If you want to truly understand neural networks, try reconstructing them from memory. While it sounds challenging, it helps solidify your understanding. This practice is especially useful when working with new academic papers or open-source frameworks like Torch, Caffe, or Theano. Understanding the basics thoroughly makes it easier to build and adapt more complex models.
Various types of rotors, including air-cooled rotors, water-cooled rotors, high-speed rotors, etc
YIWU JINGHONG AUTO PARTS CO.,LTD , https://en.jhauto.ru