> ## Documentation Index
> Fetch the complete documentation index at: https://ai.tharung.in/llms.txt
> Use this file to discover all available pages before exploring further.

# CNN Fundamentals

# Convolution, Padding, Stride, Pooling and Activation

A **Convolutional Neural Network (CNN)** is a neural network mainly used for working with images.

CNNs learn visual patterns step by step:

```text theme={null}
Edges
  ↓
Shapes
  ↓
Textures
  ↓
Objects
```

A typical CNN looks like:

```text theme={null}
Input Image
    ↓
Convolution
    ↓
ReLU
    ↓
Pooling
    ↓
Convolution
    ↓
ReLU
    ↓
Pooling
    ↓
Flatten
    ↓
Fully Connected Layer
    ↓
Output
```

***

# 1. Convolution

Convolution is the main operation in a CNN.

A small matrix called a **filter** or **kernel** moves across the image and detects patterns.

For example, consider this image:

```text theme={null}
1  2  3
4  5  6
7  8  9
```

A `2 × 2` filter:

```text theme={null}
1  0
0  1
```

The filter first looks at:

```text theme={null}
1  2
4  5
```

Multiply corresponding values:

```text theme={null}
(1 × 1) + (2 × 0) + (4 × 0) + (5 × 1)

= 1 + 0 + 0 + 5

= 6
```

Move the filter one position:

```text theme={null}
2  3
5  6
```

Calculate again:

```text theme={null}
(2 × 1) + (3 × 0) + (5 × 0) + (6 × 1)

= 2 + 0 + 0 + 6

= 8
```

Continue this process:

```text theme={null}
6   8
12  14
```

This output is called a **feature map**.

### Why is Convolution Useful?

Different filters can learn different features:

```text theme={null}
Filter 1 → Edges
Filter 2 → Corners
Filter 3 → Shapes
Filter 4 → Textures
```

The CNN learns the filter values automatically during training.

### Convolution Output Formula

The output size of a convolution is:

$$
O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1
$$

where:

* $O$ = output size
* $N$ = input size
* $F$ = filter size
* $P$ = padding
* $S$ = stride

For example:

```text theme={null}
Input size  = 5
Filter size = 3
Padding     = 0
Stride      = 1
```

Using the formula:

$$
O = \left\lfloor \frac{5 - 3 + 2(0)}{1} \right\rfloor + 1
$$

$$
O = 3
$$

Therefore:

```text theme={null}
5 × 5
  ↓
3 × 3 Filter
  ↓
3 × 3 Output
```

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

x = torch.randn(1, 1, 5, 5)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=2,
    kernel_size=3
)

output = conv(x)

print("Input shape :", x.shape)
print("Output shape:", output.shape)
```

Output:

```text theme={null}
Input shape : torch.Size([1, 1, 5, 5])
Output shape: torch.Size([1, 2, 3, 3])
```

***

# 2. Padding

Padding means adding extra pixels around the border of an image.

Usually, zeros are added.

Original image:

```text theme={null}
1  2  3
4  5  6
7  8  9
```

With padding of `1`:

```text theme={null}
0  0  0  0  0
0  1  2  3  0
0  4  5  6  0
0  7  8  9  0
0  0  0  0  0
```

### Why Do We Use Padding?

Padding helps to:

1. Preserve the image size.
2. Give border pixels more importance.
3. Prevent the feature map from becoming too small too quickly.

### Without Padding

```text theme={null}
Input
5 × 5
  ↓
3 × 3 Filter
  ↓
Output
3 × 3
```

### With Padding

For a `3 × 3` filter, padding `1`, and stride `1`:

```text theme={null}
Input
5 × 5
  ↓
3 × 3 Filter
Padding = 1
  ↓
Output
5 × 5
```

Using the formula:

$$
O = \left\lfloor \frac{5 - 3 + 2(1)}{1} \right\rfloor + 1
$$

$$
O = 5
$$

So the spatial size stays the same.

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

x = torch.randn(1, 1, 5, 5)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    padding=1
)

output = conv(x)

print("Input :", x.shape)
print("Output:", output.shape)
```

Output:

```text theme={null}
Input : torch.Size([1, 1, 5, 5])
Output: torch.Size([1, 1, 5, 5])
```

***

# 3. Stride

Stride tells us how many pixels the filter moves at each step.

### Stride = 1

The filter moves one pixel at a time:

```text theme={null}
→
→
→
```

This produces a larger feature map.

### Stride = 2

The filter moves two pixels at a time:

```text theme={null}
→ →
→ →
```

This produces a smaller feature map.

### Example

Suppose:

```text theme={null}
Input size  = 7
Filter size = 3
Padding     = 0
Stride      = 2
```

Using the formula:

$$
O = \left\lfloor \frac{7 - 3 + 2(0)}{2} \right\rfloor + 1
$$

$$
O = \left\lfloor 2 \right\rfloor + 1
$$

$$
O = 3
$$

Therefore:

```text theme={null}
7 × 7
  ↓
3 × 3 Filter
Stride = 2
  ↓
3 × 3
```

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

x = torch.randn(1, 1, 7, 7)

conv = nn.Conv2d(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    stride=2
)

output = conv(x)

print("Input :", x.shape)
print("Output:", output.shape)
```

Output:

```text theme={null}
Input : torch.Size([1, 1, 7, 7])
Output: torch.Size([1, 1, 3, 3])
```

### Important Formula

Always remember:

$$
O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1
$$

This formula is used to calculate the output size of a convolution layer.

***

# 4. Pooling

Pooling reduces the spatial size of a feature map.

It helps to:

* Reduce computation
* Reduce the number of parameters
* Keep important information
* Make the model less sensitive to small changes

The two common types are:

1. Max Pooling
2. Average Pooling

***

## Max Pooling

Max pooling selects the largest value from each region.

For example:

```text theme={null}
2  8
3  4
```

The maximum value is:

```text theme={null}
8
```

Consider this feature map:

```text theme={null}
1  5  2  3
4  8  7  6
2  1  9  3
5  4  2  8
```

Using a `2 × 2` max pooling window:

```text theme={null}
1  5 | 2  3
4  8 | 7  6
-----+-----
2  1 | 9  3
5  4 | 2  8
```

Take the maximum from each region:

```text theme={null}
8  7
5  9
```

So:

```text theme={null}
4 × 4
  ↓
2 × 2 Pooling
  ↓
2 × 2
```

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

x = torch.tensor([
    [[
        [1., 5., 2., 3.],
        [4., 8., 7., 6.],
        [2., 1., 9., 3.],
        [5., 4., 2., 8.]
    ]]
])

pool = nn.MaxPool2d(
    kernel_size=2,
    stride=2
)

output = pool(x)

print(output)
```

Output:

```text theme={null}
tensor([[[[8., 7.],
          [5., 9.]]]])
```

***

## Average Pooling

Average pooling calculates the average value of each region.

For example:

```text theme={null}
1  5
4  8
```

The average is:

$$
\frac{1 + 5 + 4 + 8}{4}
$$

$$
= 4.5
$$

### PyTorch

```python theme={null}
import torch.nn as nn

pool = nn.AvgPool2d(
    kernel_size=2,
    stride=2
)
```

### Easy Difference

```text theme={null}
Max Pooling
    ↓
Take the largest value


Average Pooling
    ↓
Take the average value
```

***

# 5. Activation Function

After convolution, we usually apply an activation function.

The most common activation function in CNNs is **ReLU**.

ReLU stands for:

**Rectified Linear Unit**

### ReLU Formula

$$
ReLU(x) = \max(0, x)
$$

This means:

```text theme={null}
Negative value → 0
Positive value → Same value
```

For example:

```text theme={null}
Input:

-5  -2   0   3   7
```

After ReLU:

```text theme={null}
0   0    0   3   7
```

### Why Do We Need ReLU?

Without activation functions, multiple neural network layers would behave like a linear transformation.

ReLU adds **non-linearity**.

This allows the CNN to learn complex patterns.

```text theme={null}
Without Activation
        ↓
Mostly Linear Relationships


With ReLU
        ↓
Non-linear Learning
        ↓
Complex Patterns
```

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

x = torch.tensor([
    -3.,
    -1.,
    0.,
    2.,
    5.
])

relu = nn.ReLU()

output = relu(x)

print(output)
```

Output:

```text theme={null}
tensor([0., 0., 0., 2., 5.])
```

You can also use:

```python theme={null}
output = torch.relu(x)
```

***

# 6. Convolution + ReLU + Pooling

A common CNN block is:

```text theme={null}
Input
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling
```

This block can be repeated multiple times.

For example:

```text theme={null}
Input Image
     ↓
Convolution
     ↓
ReLU
     ↓
Max Pooling
     ↓
Convolution
     ↓
ReLU
     ↓
Max Pooling
     ↓
Flatten
     ↓
Fully Connected Layer
     ↓
Output
```

### PyTorch

```python theme={null}
import torch
import torch.nn as nn

class SimpleCNN(nn.Module):

    def __init__(self):
        super().__init__()

        self.features = nn.Sequential(

            nn.Conv2d(
                in_channels=1,
                out_channels=16,
                kernel_size=3,
                padding=1
            ),

            nn.ReLU(),

            nn.MaxPool2d(
                kernel_size=2,
                stride=2
            ),

            nn.Conv2d(
                in_channels=16,
                out_channels=32,
                kernel_size=3,
                padding=1
            ),

            nn.ReLU(),

            nn.MaxPool2d(
                kernel_size=2,
                stride=2
            )
        )

    def forward(self, x):
        return self.features(x)


model = SimpleCNN()

x = torch.randn(1, 1, 28, 28)

output = model(x)

print("Input :", x.shape)
print("Output:", output.shape)
```

### Shape Changes

Start with:

```text theme={null}
1 × 28 × 28
```

First convolution:

```text theme={null}
1 × 28 × 28
     ↓
16 × 28 × 28
```

ReLU:

```text theme={null}
16 × 28 × 28
     ↓
16 × 28 × 28
```

Max pooling:

```text theme={null}
16 × 28 × 28
     ↓
16 × 14 × 14
```

Second convolution:

```text theme={null}
16 × 14 × 14
     ↓
32 × 14 × 14
```

ReLU:

```text theme={null}
32 × 14 × 14
     ↓
32 × 14 × 14
```

Second pooling:

```text theme={null}
32 × 14 × 14
     ↓
32 × 7 × 7
```

Final output:

```text theme={null}
32 × 7 × 7
```

Including the batch dimension:

```text theme={null}
[1, 32, 7, 7]
```

***

# 7. Understanding Channels

An image can have different numbers of channels.

### Grayscale Image

A grayscale image usually has one channel:

```text theme={null}
1 × Height × Width
```

For a `28 × 28` image:

```text theme={null}
1 × 28 × 28
```

### RGB Image

An RGB image has three channels:

```text theme={null}
3 × Height × Width
```

For a `224 × 224` image:

```text theme={null}
3 × 224 × 224
```

The three channels are:

```text theme={null}
Red
Green
Blue
```

### Convolution Channels

Consider:

```python theme={null}
nn.Conv2d(
    in_channels=3,
    out_channels=32,
    kernel_size=3
)
```

This means:

```text theme={null}
Input channels  = 3
Output channels = 32
Filter size     = 3 × 3
```

The layer produces 32 feature maps.

So:

```text theme={null}
RGB Image
3 channels
    ↓
Convolution
    ↓
32 Feature Maps
```

***

# 8. Complete CNN Example

Here is a simple CNN for MNIST digit classification.

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader


# Dataset

transform = transforms.ToTensor()

train_data = datasets.MNIST(
    root="./data",
    train=True,
    download=True,
    transform=transform
)

train_loader = DataLoader(
    train_data,
    batch_size=64,
    shuffle=True
)


# CNN Model

class CNN(nn.Module):

    def __init__(self):
        super().__init__()

        self.conv1 = nn.Conv2d(
            in_channels=1,
            out_channels=32,
            kernel_size=3,
            padding=1
        )

        self.conv2 = nn.Conv2d(
            in_channels=32,
            out_channels=64,
            kernel_size=3,
            padding=1
        )

        self.pool = nn.MaxPool2d(
            kernel_size=2,
            stride=2
        )

        self.relu = nn.ReLU()

        self.fc1 = nn.Linear(
            64 * 7 * 7,
            128
        )

        self.fc2 = nn.Linear(
            128,
            10
        )

    def forward(self, x):

        x = self.conv1(x)
        x = self.relu(x)
        x = self.pool(x)

        x = self.conv2(x)
        x = self.relu(x)
        x = self.pool(x)

        x = x.view(x.size(0), -1)

        x = self.relu(self.fc1(x))
        x = self.fc2(x)

        return x


# Create Model

model = CNN()

criterion = nn.CrossEntropyLoss()

optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)


# Training

for epoch in range(5):

    for images, labels in train_loader:

        predictions = model(images)

        loss = criterion(
            predictions,
            labels
        )

        optimizer.zero_grad()

        loss.backward()

        optimizer.step()

    print(
        f"Epoch {epoch + 1}, "
        f"Loss: {loss.item():.4f}"
    )
```

***

# 9. Understanding the MNIST Shape Changes

An MNIST image has:

```text theme={null}
1 × 28 × 28
```

First convolution:

```python theme={null}
nn.Conv2d(
    1,
    32,
    kernel_size=3,
    padding=1
)
```

The spatial size stays the same:

```text theme={null}
1 × 28 × 28
      ↓
32 × 28 × 28
```

Then max pooling:

```text theme={null}
32 × 28 × 28
      ↓
32 × 14 × 14
```

Second convolution:

```text theme={null}
32 × 14 × 14
      ↓
64 × 14 × 14
```

Second pooling:

```text theme={null}
64 × 14 × 14
      ↓
64 × 7 × 7
```

Then flatten:

```text theme={null}
64 × 7 × 7
```

Number of values:

$$
64 \times 7 \times 7 = 3136
$$

Therefore:

```python theme={null}
nn.Linear(64 * 7 * 7, 128)
```

is used.

***

# 10. Important Formulas

## Convolution Output Size

$$
O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1
$$

where:

```text theme={null}
N = Input size
F = Filter size
P = Padding
S = Stride
O = Output size
```

### Example

```text theme={null}
N = 28
F = 3
P = 1
S = 1
```

$$
O = \left\lfloor \frac{28 - 3 + 2(1)}{1} \right\rfloor + 1
$$

$$
O = 28
$$

Therefore:

```text theme={null}
28 × 28
   ↓
3 × 3 Filter
Padding = 1
Stride = 1
   ↓
28 × 28
```

***

## Pooling Output Size

The same general formula can be used for pooling:

$$
O = \left\lfloor \frac{N - F + 2P}{S} \right\rfloor + 1
$$

For:

```text theme={null}
N = 28
F = 2
P = 0
S = 2
```

$$
O = \left\lfloor \frac{28 - 2}{2} \right\rfloor + 1
$$

$$
O = 14
$$

Therefore:

```text theme={null}
28 × 28
   ↓
2 × 2 Pooling
Stride = 2
   ↓
14 × 14
```

***

# 11. Quick Revision

## Convolution

```text theme={null}
Finds useful features
```

Remember:

**Convolution = Find features**

Examples:

```text theme={null}
Edges
Corners
Shapes
Textures
```

***

## Padding

```text theme={null}
Adds pixels around the border
```

Remember:

**Padding = Protect the borders**

Main purpose:

```text theme={null}
Preserve spatial size
```

***

## Stride

```text theme={null}
Controls how far the filter moves
```

Remember:

**Stride = Movement**

```text theme={null}
Stride 1 → Move one pixel
Stride 2 → Move two pixels
```

***

## Pooling

```text theme={null}
Reduces feature-map size
```

Remember:

**Pooling = Reduce size**

```text theme={null}
Max Pooling
→ Take maximum


Average Pooling
→ Take average
```

***

## Activation

```text theme={null}
Adds non-linearity
```

Remember:

**ReLU = Remove negative values**

$$
ReLU(x) = \max(0,x)
$$

***

# 12. Easy Way to Remember CNN

```text theme={null}
Image
  ↓
Convolution
  ↓
Find Features
  ↓
ReLU
  ↓
Add Non-Linearity
  ↓
Pooling
  ↓
Reduce Size
  ↓
Repeat
  ↓
Flatten
  ↓
Classification
```

The five concepts can be remembered as:

```text theme={null}
Convolution
    ↓
Find features

Padding
    ↓
Handle borders

Stride
    ↓
Control movement

Pooling
    ↓
Reduce size

Activation
    ↓
Learn complex patterns
```

## Final Summary

**Convolution** finds useful patterns in an image.

**Padding** adds pixels around the border and can preserve the spatial size.

**Stride** controls how far the filter moves.

**Pooling** reduces the size of feature maps.

**Activation** adds non-linearity so the network can learn complex patterns.

The basic CNN flow is:

```text theme={null}
Image
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling
  ↓
Convolution
  ↓
ReLU
  ↓
Pooling
  ↓
Flatten
  ↓
Fully Connected Layer
  ↓
Prediction
```

### One-Line Memory Trick

> **Convolution finds features, Padding handles borders, Stride controls movement, Pooling reduces size, and Activation helps the network learn complex patterns.**