Convolution, Padding, Stride, Pooling and Activation

A Convolutional Neural Network (CNN) is a neural network mainly used for working with images. CNNs learn visual patterns step by step:

Edges
  ↓
Shapes
  ↓
Textures
  ↓
Objects

A typical CNN looks like:

Input Image
    ↓
Convolution
    ↓
ReLU
    ↓
Pooling
    ↓
Convolution
    ↓
ReLU
    ↓
Pooling
    ↓
Flatten
    ↓
Fully Connected Layer
    ↓
Output

1. Convolution

Convolution is the main operation in a CNN. A small matrix called a filter or kernel moves across the image and detects patterns. For example, consider this image:

2  3
5  6
8  9

A 2 × 2 filter:

1  0
0  1

The filter first looks at:

1  2
4  5

Multiply corresponding values:

(1 × 1) + (2 × 0) + (4 × 0) + (5 × 1)

= 1 + 0 + 0 + 5

= 6

Move the filter one position:

2  3
5  6

Calculate again:

(2 × 1) + (3 × 0) + (5 × 0) + (6 × 1)

= 2 + 0 + 0 + 6

= 8

Continue this process:

6   8
12  14

This output is called a feature map.

Why is Convolution Useful?

Different filters can learn different features:

Filter 1 → Edges
Filter 2 → Corners
Filter 3 → Shapes
Filter 4 → Textures

The CNN learns the filter values automatically during training.

Convolution Output Formula

The output size of a convolution is:

O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1

where:

$O$ = output size
$N$ = input size
$F$ = filter size
$P$ = padding
$S$ = stride

For example:

Input size  = 5
Filter size = 3
Padding     = 0
Stride      = 1

Using the formula:

O = \left\lfloor \frac{5 - 3 + 2(0)}{1} \right\rfloor + 1

O = 3

Therefore:

5 × 5
  ↓
3 × 3 Filter
  ↓
3 × 3 Output

PyTorch

import torch
import torch.nn as nn

x = torch.randn(1, 1, 5, 5)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=2,
    kernel_size=3
)

output = conv(x)

print("Input shape :", x.shape)
print("Output shape:", output.shape)

Output:

Input shape : torch.Size([1, 1, 5, 5])
Output shape: torch.Size([1, 2, 3, 3])

2. Padding

Padding means adding extra pixels around the border of an image. Usually, zeros are added. Original image:

2  3
5  6
8  9

With padding of 1:

Why Do We Use Padding?

Padding helps to:

Preserve the image size.
Give border pixels more importance.
Prevent the feature map from becoming too small too quickly.

Without Padding

Input
5 × 5
  ↓
3 × 3 Filter
  ↓
Output
3 × 3

With Padding

For a 3 × 3 filter, padding 1, and stride 1:

Input
5 × 5
  ↓
3 × 3 Filter
Padding = 1
  ↓
Output
5 × 5

Using the formula:

O = \left\lfloor \frac{5 - 3 + 2(1)}{1} \right\rfloor + 1

O = 5

So the spatial size stays the same.

PyTorch

import torch
import torch.nn as nn

x = torch.randn(1, 1, 5, 5)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    padding=1
)

output = conv(x)

print("Input :", x.shape)
print("Output:", output.shape)

Output:

Input : torch.Size([1, 1, 5, 5])
Output: torch.Size([1, 1, 5, 5])

3. Stride

Stride tells us how many pixels the filter moves at each step.

Stride = 1

The filter moves one pixel at a time:

→
→
→

This produces a larger feature map.

Stride = 2

The filter moves two pixels at a time:

→ →
→ →

This produces a smaller feature map.

Example

Suppose:

Input size  = 7
Filter size = 3
Padding     = 0
Stride      = 2

Using the formula:

O = \left\lfloor \frac{7 - 3 + 2(0)}{2} \right\rfloor + 1

O = \left\lfloor 2 \right\rfloor + 1

O = 3

Therefore:

7 × 7
  ↓
3 × 3 Filter
Stride = 2
  ↓
3 × 3

PyTorch

import torch
import torch.nn as nn

x = torch.randn(1, 1, 7, 7)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    stride=2
)

output = conv(x)

print("Input :", x.shape)
print("Output:", output.shape)

Output:

Input : torch.Size([1, 1, 7, 7])
Output: torch.Size([1, 1, 3, 3])

Important Formula

Always remember:

O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1

This formula is used to calculate the output size of a convolution layer.

4. Pooling

Pooling reduces the spatial size of a feature map. It helps to:

Reduce computation
Reduce the number of parameters
Keep important information
Make the model less sensitive to small changes

The two common types are:

Max Pooling
Average Pooling

Max Pooling

Max pooling selects the largest value from each region. For example:

2  8
3  4

The maximum value is:

Consider this feature map:

Using a 2 × 2 max pooling window:

5 | 2  3
8 | 7  6
-----+-----
1 | 9  3
4 | 2  8

Take the maximum from each region:

8  7
5  9

So:

4 × 4
  ↓
2 × 2 Pooling
  ↓
2 × 2

PyTorch

import torch
import torch.nn as nn

x = torch.tensor([
    [[
        [1., 5., 2., 3.],
        [4., 8., 7., 6.],
        [2., 1., 9., 3.],
        [5., 4., 2., 8.]
    ]]
])

pool = nn.MaxPool2d(
    kernel_size=2,
    stride=2
)

output = pool(x)

print(output)

Output:

tensor([[[[8., 7.],
          [5., 9.]]]])

Average Pooling

Average pooling calculates the average value of each region. For example:

1  5
4  8

The average is:

\frac{1 + 5 + 4 + 8}{4}

= 4.5

PyTorch

import torch.nn as nn

pool = nn.AvgPool2d(
    kernel_size=2,
    stride=2
)

Easy Difference

Max Pooling
    ↓
Take the largest value


Average Pooling
    ↓
Take the average value

5. Activation Function

After convolution, we usually apply an activation function. The most common activation function in CNNs is ReLU. ReLU stands for: Rectified Linear Unit

ReLU Formula

ReLU(x) = \max(0, x)

This means:

Negative value → 0
Positive value → Same value

For example:

Input:

-5  -2   0   3   7

After ReLU:

0   0    0   3   7

Why Do We Need ReLU?

Without activation functions, multiple neural network layers would behave like a linear transformation. ReLU adds non-linearity. This allows the CNN to learn complex patterns.

Without Activation
        ↓
Mostly Linear Relationships


With ReLU
        ↓
Non-linear Learning
        ↓
Complex Patterns

PyTorch

import torch
import torch.nn as nn

x = torch.tensor([
    -3.,
    -1.,
    0.,
    2.,
    5.
])

relu = nn.ReLU()

output = relu(x)

print(output)

Output:

tensor([0., 0., 0., 2., 5.])

You can also use:

output = torch.relu(x)

6. Convolution + ReLU + Pooling

A common CNN block is:

Input
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling

This block can be repeated multiple times. For example:

Input Image
     ↓
Convolution
     ↓
ReLU
     ↓
Max Pooling
     ↓
Convolution
     ↓
ReLU
     ↓
Max Pooling
     ↓
Flatten
     ↓
Fully Connected Layer
     ↓
Output

PyTorch

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):

    def __init__(self):
        super().__init__()

        self.features = nn.Sequential(

            nn.Conv2d(
                in_channels=1,
                out_channels=16,
                kernel_size=3,
                padding=1
            ),

            nn.ReLU(),

            nn.MaxPool2d(
                kernel_size=2,
                stride=2
            ),

            nn.Conv2d(
                in_channels=16,
                out_channels=32,
                kernel_size=3,
                padding=1
            ),

            nn.ReLU(),

            nn.MaxPool2d(
                kernel_size=2,
                stride=2
            )
        )

    def forward(self, x):
        return self.features(x)


model = SimpleCNN()

x = torch.randn(1, 1, 28, 28)

output = model(x)

print("Input :", x.shape)
print("Output:", output.shape)

Shape Changes

Start with:

1 × 28 × 28

First convolution:

1 × 28 × 28
     ↓
16 × 28 × 28

ReLU:

16 × 28 × 28
     ↓
16 × 28 × 28

Max pooling:

16 × 28 × 28
     ↓
16 × 14 × 14

Second convolution:

16 × 14 × 14
     ↓
32 × 14 × 14

ReLU:

32 × 14 × 14
     ↓
32 × 14 × 14

Second pooling:

32 × 14 × 14
     ↓
32 × 7 × 7

Final output:

32 × 7 × 7

Including the batch dimension:

[1, 32, 7, 7]

7. Understanding Channels

An image can have different numbers of channels.

Grayscale Image

A grayscale image usually has one channel:

1 × Height × Width

For a 28 × 28 image:

1 × 28 × 28

RGB Image

An RGB image has three channels:

3 × Height × Width

For a 224 × 224 image:

3 × 224 × 224

The three channels are:

Red
Green
Blue

Convolution Channels

Consider:

nn.Conv2d(
    in_channels=3,
    out_channels=32,
    kernel_size=3
)

This means:

Input channels  = 3
Output channels = 32
Filter size     = 3 × 3

The layer produces 32 feature maps. So:

RGB Image
3 channels
    ↓
Convolution
    ↓
32 Feature Maps

8. Complete CNN Example

Here is a simple CNN for MNIST digit classification.

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader


# Dataset

transform = transforms.ToTensor()

train_data = datasets.MNIST(
    root="./data",
    train=True,
    download=True,
    transform=transform
)

train_loader = DataLoader(
    train_data,
    batch_size=64,
    shuffle=True
)


# CNN Model

class CNN(nn.Module):

    def __init__(self):
        super().__init__()

        self.conv1 = nn.Conv2d(
            in_channels=1,
            out_channels=32,
            kernel_size=3,
            padding=1
        )

        self.conv2 = nn.Conv2d(
            in_channels=32,
            out_channels=64,
            kernel_size=3,
            padding=1
        )

        self.pool = nn.MaxPool2d(
            kernel_size=2,
            stride=2
        )

        self.relu = nn.ReLU()

        self.fc1 = nn.Linear(
            64 * 7 * 7,
            128
        )

        self.fc2 = nn.Linear(
            128,
            10
        )

    def forward(self, x):

        x = self.conv1(x)
        x = self.relu(x)
        x = self.pool(x)

        x = self.conv2(x)
        x = self.relu(x)
        x = self.pool(x)

        x = x.view(x.size(0), -1)

        x = self.relu(self.fc1(x))
        x = self.fc2(x)

        return x


# Create Model

model = CNN()

criterion = nn.CrossEntropyLoss()

optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)


# Training

for epoch in range(5):

    for images, labels in train_loader:

        predictions = model(images)

        loss = criterion(
            predictions,
            labels
        )

        optimizer.zero_grad()

        loss.backward()

        optimizer.step()

    print(
        f"Epoch {epoch + 1}, "
        f"Loss: {loss.item():.4f}"
    )

9. Understanding the MNIST Shape Changes

An MNIST image has:

1 × 28 × 28

First convolution:

nn.Conv2d(
    1,
    32,
    kernel_size=3,
    padding=1
)

The spatial size stays the same:

1 × 28 × 28
      ↓
32 × 28 × 28

Then max pooling:

32 × 28 × 28
      ↓
32 × 14 × 14

Second convolution:

32 × 14 × 14
      ↓
64 × 14 × 14

Second pooling:

64 × 14 × 14
      ↓
64 × 7 × 7

Then flatten:

64 × 7 × 7

Number of values:

64 \times 7 \times 7 = 3136

Therefore:

nn.Linear(64 * 7 * 7, 128)

is used.

10. Important Formulas

Convolution Output Size

O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1

where:

N = Input size
F = Filter size
P = Padding
S = Stride
O = Output size

Example

N = 28
F = 3
P = 1
S = 1

O = \left\lfloor \frac{28 - 3 + 2(1)}{1} \right\rfloor + 1

O = 28

Therefore:

28 × 28
   ↓
3 × 3 Filter
Padding = 1
Stride = 1
   ↓
28 × 28

Pooling Output Size

The same general formula can be used for pooling:

O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1

For:

N = 28
F = 2
P = 0
S = 2

O = \left\lfloor \frac{28 - 2}{2} \right\rfloor + 1

O = 14

Therefore:

28 × 28
   ↓
2 × 2 Pooling
Stride = 2
   ↓
14 × 14

11. Quick Revision

Convolution

Finds useful features

Remember: Convolution = Find features Examples:

Edges
Corners
Shapes
Textures

Padding

Adds pixels around the border

Remember: Padding = Protect the borders Main purpose:

Preserve spatial size

Stride

Controls how far the filter moves

Remember: Stride = Movement

Stride 1 → Move one pixel
Stride 2 → Move two pixels

Pooling

Reduces feature-map size

Remember: Pooling = Reduce size

Max Pooling
→ Take maximum


Average Pooling
→ Take average

Activation

Adds non-linearity

Remember: ReLU = Remove negative values

ReLU(x) = \max(0,x)

12. Easy Way to Remember CNN

Image
  ↓
Convolution
  ↓
Find Features
  ↓
ReLU
  ↓
Add Non-Linearity
  ↓
Pooling
  ↓
Reduce Size
  ↓
Repeat
  ↓
Flatten
  ↓
Classification

The five concepts can be remembered as:

Convolution
    ↓
Find features

Padding
    ↓
Handle borders

Stride
    ↓
Control movement

Pooling
    ↓
Reduce size

Activation
    ↓
Learn complex patterns

Final Summary

Convolution finds useful patterns in an image. Padding adds pixels around the border and can preserve the spatial size. Stride controls how far the filter moves. Pooling reduces the size of feature maps. Activation adds non-linearity so the network can learn complex patterns. The basic CNN flow is:

Image
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling
  ↓
Flatten
  ↓
Fully Connected Layer
  ↓
Prediction

One-Line Memory Trick

Convolution finds features, Padding handles borders, Stride controls movement, Pooling reduces size, and Activation helps the network learn complex patterns.

​Convolution, Padding, Stride, Pooling and Activation

​1. Convolution

​Why is Convolution Useful?

​Convolution Output Formula

​PyTorch

​2. Padding

​Why Do We Use Padding?

​Without Padding

​With Padding

​PyTorch

​3. Stride

​Stride = 1

​Stride = 2

​Example

​PyTorch

​Important Formula

​4. Pooling

​Max Pooling

​PyTorch

​Average Pooling

​PyTorch

​Easy Difference

​5. Activation Function

​ReLU Formula

​Why Do We Need ReLU?

​PyTorch

​6. Convolution + ReLU + Pooling

​PyTorch

​Shape Changes

​7. Understanding Channels

​Grayscale Image

​RGB Image

​Convolution Channels

​8. Complete CNN Example

​9. Understanding the MNIST Shape Changes

​10. Important Formulas

​Convolution Output Size

​Example

​Pooling Output Size

​11. Quick Revision

​Convolution

​Padding

​Stride

​Pooling

​Activation

​12. Easy Way to Remember CNN

​Final Summary

​One-Line Memory Trick

Convolution, Padding, Stride, Pooling and Activation

1. Convolution

Why is Convolution Useful?

Convolution Output Formula

PyTorch

2. Padding

Why Do We Use Padding?

Without Padding

With Padding

PyTorch

3. Stride

Stride = 1

Stride = 2

Example

PyTorch

Important Formula

4. Pooling

Max Pooling

PyTorch

Average Pooling

PyTorch

Easy Difference

5. Activation Function

ReLU Formula

Why Do We Need ReLU?

PyTorch

6. Convolution + ReLU + Pooling

PyTorch

Shape Changes

7. Understanding Channels

Grayscale Image

RGB Image

Convolution Channels

8. Complete CNN Example

9. Understanding the MNIST Shape Changes

10. Important Formulas

Convolution Output Size

Example

Pooling Output Size

11. Quick Revision

Convolution

Padding

Stride

Pooling

Activation

12. Easy Way to Remember CNN

Final Summary

One-Line Memory Trick