From developers to project managers, anyone that works with software should at least understand the basics of neural networks, as many problems can be solved by intelligently implementing a neural network machine learning model.
What’s best is that modern computers can easily handle these algorithms and structures, first devised in the 1960s, as computers are a thousand times faster than they were back then. All it takes is a little effort on our part to put in the time and see just what neural networks are, and which are some of their applications. Let’s quickly review the history of the neural network and the most basic of examples.
The first neural network
Like many early computation milestones, the neural network was born out of military interests. The first neural network was called perceptron; first conceived in 1958 as a machine that would receive light-based inputs, to be processed through potentiometers, which would finally be adjusted by an electric motor as a primitive way of “learning.”
Years later, the workings of the perceptron would be formalized in an algorithm, given:
A vector of inputs:
x = (x1, x2, …, xn)
A vector of weights:
w = (w1, w2, …, wn)
An activation function:
and a bias b, which can be zero.
In essence, a perceptron would be an algorithm where the values within a vector of inputs are given a degree of importance. Finally, the activation function tells us whether the input fits our classification label or not by producing, in this case, a 0 or a 1, respectively.
A neural network on paper
Let’s illustrate an example using the simplest neural network. Using the algorithmic definition of a perceptron, we’ll build a very simple machine learning model to deduce whether the input is a vegetable or a fruit. No coding needed!
First, let’s say that our classification label is 1 for fruits and 0 for vegetables (or not fruits). Now, doing a bit of research, we can find some interesting rules:
- Fruits come from the flower of the plant, vegetables do not
- These are fruits, but they are sometimes considered vegetables because they often go in salads:
- Tomatoes
- Squash
- Avocado
- Olives
- Zucchini
- Eggplant
- Most fruits are sweet.
Because the “Grows from flower or bulb” condition is a surefire way of knowing if it’s a fruit, we’ll use it to validate our prediction. Let’s try to make a model to see if we can predict whether the input is a fruit or vegetable, based on only two other parameters.
In basic perceptrons, the values inside the input vector are either 0 or 1. There are also non-linear activation functions, which benefit greatly from input vector parameters between 0 and 1. For this example, we’ll just use 0 for “No” and 1 for “Yes.”
Let’s create the following input vectors:
i
Name
Oftentimes goes in salads (x1)
Is sweet (x2)
Grows from flower or bulb = Is a fruit
1
Cucumber
1
1
1
2
Carrot
1
1
0
3
Banana
0
1
1
4
White onion
1
0
0
5
Tomato
1
1
1
Now, let’s try and assign some weights to the activation functions:
- Oftentimes seen as vegetable (w1) = 0.5
- Is sweet (w2) = 0.5
Let’s set a bias (b) = -0.5
This is our neural network:
And this is the result:
i
Name
Oftentimes goes in salads (x1)
x1*w1
Is sweet (x2)
x2 *w2
Prediction
(x1*w1+x2 *w2+b)
Validation
1
Cucumber
1
0.5
1
0.5
0.5 → 1
1
2
Carrot
1
0.5
1
0.5
0.5 → 1
0
3
Banana
0
0
1
0.5
0 → 0
1
4
White onion
1
0.5
0
0
0 → 0
0
5
Tomato
1
0.5
1
0.5
0.5 → 1
1
Our model has an accuracy of 60%, not bad for our first try!
The following step would be to fine-tune the weights and the bias to see if we can make it more precise. But studying our input data, we see that the cucumber and the carrot share the same vector but with different outputs, so the model can’t have an accuracy greater than 80%, but that’s alright! This happens in real life. Machine learning engineers and data miners find these situations all the time, where either there isn’t enough data or there is too much to make sense of it. Can you guess which weight you need to adjust to get that accuracy to 80%?
This is just the most basic instance of a neural network. There are innumerable applications and more complex ways of structuring these algorithms to solve difficult problems, such as image recognition and automated driving. It’s all a matter of being creative and studying more diverse types of neural networks and machine learning models.
Note: perceptrons are no longer used because the new neural networks are more effective
