Artificial Neural Network (ANN) using Python

Step 1: Importing Libraries For our calculations we will be using numpy

In [53]:

from IPython.core.display import display, HTML
display(HTML("<style>.container { width:103% !important;margin-left: -2%; }</style>"))

In [ ]:

import numpy as np

Step 2:Creating a dummy representative input and output Lets create an input array.This will be a simple binary input with all combinations of 0 and 1

In [2]:

input_features=np.array([[0,1],[1,0],[0,0],[1,1]])
input_features

Out[2]:

array([[0, 1],
       [1, 0],
       [0, 0],
       [1, 1]])

In [3]:

# It has 4 rows and 2 colums
input_features.shape 

Out[3]:

(4, 2)

In [4]:

target_output = np.array([[0,1,1,1]]).T
target_output

Out[4]:

array([[0],
       [1],
       [1],
       [1]])

Step 3:Assigning random wegiths to the input Since we have two input elements, so there will be two weights involved

In [5]:

weights = np.array([[0.1],[0.2]])
weights

Out[5]:

array([[0.1],
       [0.2]])

Step 4:Adding a Bias value to the input By default, the bias is assigned a value equal to 1.However, the weight assigned to it will be random initially

In [6]:

bias=0.3

Step 5:Defining Learning Rate It is another configurable parameter used in training neural network. It often ranges between 0-1 and is given a small value.It controls how quickly model is adapted to the problem at hand

In [7]:

lr=0.05

Step 6:Applying a Sigmoid Function The problem we are trying to model is a classification problem.Whatever inputs we are going to have, the output will always be restricted between 0 and 1.So sigmoid function fits the bill here.It take an input and generates the output between 0 and 1

In [8]:

def sigmoid(x):
    z=1/(1+np.exp(-x))
    return(z)

In [9]:

# Testing the function by giving it a value of 2
sigmoid(2)

Out[9]:

0.8807970779778823

Step 7:Derivative of the Sigmoid function

In [10]:

def sigmoid_der(x):
    z=sigmoid(x)*(1-sigmoid(x))
    return(z)

Step 8:Main Logic for ANN Here we will be calculating the following things

• Feedforward Inputs
• Feedforward Output
• Backpropagation(Error Calculation)
• Calculating Derivative
• Multiplying Derivative
• Updating Weights
• Updating Bias Weights

We will run the code for 10,000 times to get the weights and bias right.Each training cycle of the data is known as an epoch

In [36]:

weights = np.array([[0.1],[0.2]])
weights

bias=0.3
lr=0.05 

In [37]:

 

for epochs in range(10000):
    input = input_features
    
    # Feedforward inputs
    in_o = np.dot(input,weights) + bias
    
    # Feedforward output
    out_o = sigmoid(in_o)
    
    # Backpropagation:Calculation of error
    error = out_o - target_output # error will be an array of shape (4,1)
    x= error.sum()
    #print(x)
    
    # Calculating derivative of the error wrt weights
    # 1.Derivative of Error wrt transormed output
    derror_dout = error
    
    # 2.Derivative of transormed output wrt transformed input
    dout_dino = sigmoid_der(out_o)
    
    # 3.input component of the derivative chain
    inputs = input_features.T
    
    # Multipying 1 and 2
    deriv = derror_dout * dout_dino
    
    # Multiplying deriv with 3 component
    deriv_final = np.dot(inputs,deriv)
    #print(out_o)
    
    # Updating the weights value
    weights -= lr*deriv_final
    
    # Updating the bias value
    for i in deriv:
        bias -= lr*i

In [38]:

in_o
#sigmoid(in_o)

Out[38]:

array([[-3.40015087],
       [10.74262174],
       [ 3.46549005],
       [ 3.87698081]])

In [39]:

# Checking weights and bias value
print(weights)
print(bias)
    

[[ 7.27733325]
 [-6.8658431 ]]
[3.46559021]

Step 9:Predicting Values Lets look at the first row of inputs and output and predict the output based on the model parameters obtained

In [40]:

input_features[0]

Out[40]:

array([0, 1])

In [41]:

expected_output = target_output[0]
expected_output

Out[41]:

array([0])

In [42]:

# inputs with weight and bias applied
result1 = np.dot(input_features[0],weights) + bias
result1

Out[42]:

array([-3.40025289])

In [43]:

# transfomed input
sigmoid(result1)

Out[43]:

array([0.03228756])

In [ ]:

# We can see that the output is around 0 which is equal to the expected output

Step 10:Putting it all together

In [44]:

Result1=np.dot(input_features,weights) + bias
Result1

Out[44]:

array([[-3.40025289],
       [10.74292346],
       [ 3.46559021],
       [ 3.87708036]])

In [45]:

# transfomed input
sigmoid(Result1)

Out[45]:

array([[0.03228756],
       [0.9999784 ],
       [0.96969269],
       [0.97970904]])

So it can be seen that the first element of the output is 0.03 which is close to 0. The rest of the outputs are close to 1. Hence our output is [0,1,1,1].

Through this simple example we saw how we can train an Artificial Neural Network (ANN) and understand how various components are set up and initialised