Feed Forward Neural Network (FFNN) and Backpropagation

 

Feed Forward Neural Network (FFNN) Understanding Information Flow in Artificial Neural Networks

Introduction

In the previous sessions, we learned about Artificial Neural Networks (ANNs), Single-Layer Perceptrons, Multi-Layer Perceptrons, and why hidden layers are required to solve non-linear problems such as XOR and XNOR.

An important question now arises:

How does information actually travel inside a neural network?

Does the information move randomly?

Can it move backward?

Can one neuron send information back to the previous neuron?

The answer is No.

Information always moves in one direction.

This type of network is called a Feed Forward Neural Network (FFNN).

Why is it called Feed Forward?

Let's understand the name.

Feed

Means giving input to the network.

Forward

Means information moves only towards the output.

Therefore,

Feed Forward

means

Feed the input and move forward until the prediction is obtained.

---

Basic Architecture of a Feed Forward Neural Network

Draw this on the board.

Input Layer

↓

Hidden Layer

↓

Output Layer  

Explain:

• Input Layer receives the data.
• Hidden Layer performs mathematical calculations.
• Output Layer produces the prediction.

Notice one important thing.

There is no arrow pointing backward.

---

Real-World Example 1: Student Placement Prediction

Suppose we want to predict whether a student will be placed.

Input Features

• CGPA
• Aptitude Score
• Communication Score

These values enter the ANN.

Input Layer

↓

Hidden Layer performs calculations.

↓

Output Layer

Prediction

Placed

or

Not Placed

Once the prediction is generated,

the data never returns to the input layer.

That is why it is called

Feed Forward.

Real-World Example 2: Hotel Booking Cancellation

Input

• Lead Time
• Average Daily Rate
• Customer Type
• Previous Cancellations

The ANN performs

Input

↓

Hidden Layer

↓

Output

Booking Cancelled

or

Booking Not Cancelled

Again,

information only moves

Forward.

---

Real-World Example 3: Breast Cancer Prediction

Input

• Radius
• Texture
• Area
• Perimeter
• Compactness

↓

Hidden Layer

↓

Output

Benign

or

Malignant

The information never travels backward while making a prediction.

Step-by-Step Working of Feed Forward Neural Network

Step 1

Input features are received.

Example

Feature	Value
Lead Time	90
Room Price	4500
Special Requests	0

---

Step 2

Each input is multiplied by its weight.

$$z=(x_1w_1+x_2w_2+x_3w_3)+b$$

Step 3

The weighted sum is calculated.

---

Step 4

Activation Function

Example

Sigmoid

ReLU

Tanh

---

Step 5

The output is generated.

Example

Booking Cancelled

---

Important Observation

Notice

During prediction

Information moves

Input

↓

Hidden Layer

↓

Output

Only once.

There is

No Loop.

No Recursion.

No Feedback.

---

Why is Feed Forward Network Important?

Because

It is

• Easy to design
• Easy to train
• Computationally efficient
• Suitable for most classification problems

Examples

• Student Result Prediction
• Hotel Booking Prediction
• Breast Cancer Prediction
• Loan Approval
• Spam Detection
• Customer Churn Prediction

---

Limitation of Feed Forward Network

Suppose you want to predict

The next word in a sentence.

Example

"I am going to ____"

Can the ANN predict

"School"

without remembering previous words?

No.

Similarly,

Speech Recognition

needs memory.

Language Translation

needs memory.

Stock Price Prediction

needs previous days' prices.

A Feed Forward Network has

No Memory.

It cannot remember previous inputs.

This limitation led to the development of

Recurrent Neural Networks (RNNs),

which you will study later.

---

What is a Deep Feed Forward Neural Network?

Until now,

we have seen only

One Hidden Layer.

Suppose we increase the number of hidden layers.

Input

↓

Hidden Layer 1

↓

Hidden Layer 2

↓

Hidden Layer 3

↓

Output

This becomes

Deep Feed Forward Neural Network.

---

Why is it called "Deep"?

The word

Deep

does not mean

Large Dataset.

It means

More Hidden Layers.

Examples

One Hidden Layer

Simple ANN

Three Hidden Layers

Deep Neural Network

Ten Hidden Layers

Very Deep Neural Network

---

Real-World Example

Suppose we want to recognize a human face.

Input

Face Image

Hidden Layer 1

Learns

Edges

Hidden Layer 2

Learns

Eyes

Nose

Ears

Hidden Layer 3

Learns

Complete Face Structure

Output

Person Identified

Each hidden layer learns

More complex information.

That is why

Deep Networks perform better.

---

Another Example

Medical Diagnosis

Input

MRI Image

↓

Hidden Layer 1

Edges

↓

Hidden Layer 2

Tissues

↓

Hidden Layer 3

Tumor Shape

↓

Hidden Layer 4

Cancer Pattern

↓

Output

Benign

or

Malignant

What Happens After Feed Forward Neural Network?

In the previous section, we learned that a Feed Forward Neural Network (FFNN) processes the input data by moving information in only one direction:

Input Layer → Hidden Layer(s) → Output Layer

The network performs mathematical calculations at each neuron and finally generates a prediction.

For example, consider a student placement prediction system.

CGPA	Aptitude Score	Communication Skills	Actual Result	Predicted Result
8.5	90	Good	Placed	Not Placed

In this example, the neural network predicts "Not Placed", whereas the actual result is "Placed." This indicates that the prediction is incorrect and an error has occurred.

At this stage, the neural network must improve itself by identifying the error and adjusting its internal parameters. This learning process is known as Backpropagation.  

Why is Backpropagation Required?

A Feed Forward Neural Network is capable of making predictions, but it does not automatically know whether those predictions are correct or incorrect.

After generating the output, the predicted result is compared with the actual result. The difference between these two values is called the error or loss.

To improve future predictions, the neural network must:

• Calculate the prediction error.
• Identify which weights contributed to the error.
• Adjust the weights to reduce the error.
• Repeat the learning process until the prediction becomes more accurate.

This entire process is performed using Backpropagation.

---

Real-World Example: Student Learning Process

The concept of Backpropagation can be understood by comparing it with the learning process of a student.

Suppose a student appears for a mathematics examination.

First Attempt

The student scores 45 marks out of 100.

After checking the answer sheet, the teacher identifies the mistakes.

The student studies the incorrect topics and prepares again.

Second Attempt

The student scores 60 marks out of 100.

The remaining mistakes are corrected through additional practice.

Third Attempt

The student scores 78 marks out of 100.

Fourth Attempt

The student scores 92 marks out of 100.

The student's performance improves because they continuously learn from previous mistakes.

Similarly, a neural network improves its prediction accuracy by learning from prediction errors. This learning mechanism is called Backpropagation.

Feed Forward vs Backpropagation

Feed Forward	Backpropagation
Generates a prediction	Learns from prediction errors
Information moves from Input to Output	Error moves from Output to Input
Uses the current weights	Updates the weights
Produces the output	Improves future predictions

Information Flow in Feed Forward

During the Feed Forward process, information moves only in one direction.

Input Layer

      ↓

Hidden Layer(s)

      ↓

Output Layer

Information Flow in Backpropagation

Once the prediction is generated, the output is compared with the actual value.

If an error exists, the error is propagated backward through the network.

Output Layer

      ↑

Hidden Layer(s)

      ↑

Input Layer  

During this backward movement, the weights connecting the neurons are updated to reduce the prediction error.