> ## Documentation Index
> Fetch the complete documentation index at: https://ai.tharung.in/llms.txt
> Use this file to discover all available pages before exploring further.

# MNIST Detail Info

Problem Statement

* **Develop a neural network using PyTorch to recognize handwritten digits (0–9) from MNIST images.**
* **Train the model using labeled images so it can learn patterns and classify unseen handwritten digits correctly.**
* **Evaluate the model using test data and measure its performance using classification accuracy.**

***

# 1. Importing PyTorch

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim

from torch.utils.data import DataLoader
from torchvision import datasets, transforms
```

### `import torch`

```python theme={null}
import torch
```

PyTorch is the main deep-learning library we're using.

It provides:

* Tensors
* GPU/CPU computation
* Automatic differentiation
* Neural-network operations
* Model training utilities

Think of a **tensor** as a multidimensional array.

For example:

```python theme={null}
x = torch.tensor([1, 2, 3])
```

is a 1-dimensional tensor.

You can also have:

```python theme={null}
x = torch.tensor([
    [1, 2],
    [3, 4]
])
```

which is a 2D tensor.

***

### `torch.nn`

```python theme={null}
import torch.nn as nn
```

`torch.nn` contains tools for creating neural networks.

For example:

```python theme={null}
nn.Linear(...)
nn.ReLU()
nn.Conv2d(...)
nn.Dropout(...)
```

Your code uses:

```python theme={null}
nn.Linear
```

to create fully connected layers.

We use `nn` as a shorter name for `torch.nn`.

***

### `torch.optim`

```python theme={null}
import torch.optim as optim
```

This contains optimization algorithms.

Your code uses:

```python theme={null}
optim.Adam
```

Adam is responsible for **updating the neural-network weights** after calculating the error.

***

### `DataLoader`

```python theme={null}
from torch.utils.data import DataLoader
```

`DataLoader` takes a dataset and gives it to the neural network in **batches**.

Instead of giving 60,000 images to the network at once:

```text theme={null}
60,000 images → Neural Network
```

we can give:

```text theme={null}
64 images → Neural Network
64 images → Neural Network
64 images → Neural Network
...
```

This is called **mini-batch training**.

***

### `datasets` and `transforms`

```python theme={null}
from torchvision import datasets, transforms
```

`torchvision` contains datasets and computer-vision utilities.

You're using:

```python theme={null}
datasets.MNIST
```

to download/load the MNIST handwritten-digit dataset.

And:

```python theme={null}
transforms.ToTensor()
```

to convert images into PyTorch tensors.

***

# 2. Selecting CPU or GPU

```python theme={null}
device = torch.device(
    "cuda" if torch.cuda.is_available() else "cpu"
)
```

This determines where your neural network will run.

The logic is:

```text theme={null}
Is CUDA/GPU available?
       |
      Yes
       ↓
     "cuda"
       |
      No
       ↓
     "cpu"
```

So:

```python theme={null}
torch.cuda.is_available()
```

checks whether PyTorch can use an NVIDIA CUDA GPU.

If yes:

```python theme={null}
device = torch.device("cuda")
```

Otherwise:

```python theme={null}
device = torch.device("cpu")
```

***

### Why GPU?

Neural networks perform lots of mathematical operations.

A GPU can perform many operations in parallel and is generally much faster for deep learning.

For MNIST, however, a CPU is usually sufficient because the network is very small.

***

```python theme={null}
print("Using", device)
```

This might print:

```text theme={null}
Using cuda
```

or:

```text theme={null}
Using cpu
```

***

# 3. Image transformation

```python theme={null}
transform = transforms.ToTensor()
```

MNIST images are originally image data.

`ToTensor()` converts them into PyTorch tensors.

It also scales pixel values.

Originally, a pixel is generally represented from:

```text theme={null}
0 → 255
```

After `ToTensor()`:

```text theme={null}
0 → 1
```

For example:

```text theme={null}
0     → 0.0
255   → 1.0
128   → ~0.502
```

This makes the data easier for a neural network to work with.

***

# 4. Loading the training dataset

```python theme={null}
train_dataset = datasets.MNIST(
    root = "./data",
    train = True,
    download = True,
    transform = transform
)
```

This loads the **training portion** of MNIST.

Let's understand every argument.

***

### `root`

```python theme={null}
root="./data"
```

This tells PyTorch where to store the dataset.

Your project might look like:

```text theme={null}
project/
│
├── your_script.py
│
└── data/
    └── MNIST/
```

***

### `train=True`

```python theme={null}
train=True
```

MNIST contains two main portions:

```text theme={null}
Training dataset → 60,000 images
Testing dataset  → 10,000 images
```

`train=True` means:

> Give me the training data.

***

### `download=True`

```python theme={null}
download=True
```

If MNIST isn't already present, PyTorch downloads it.

If it's already downloaded, it normally won't download it again.

***

### `transform=transform`

```python theme={null}
transform=transform
```

This applies:

```python theme={null}
transforms.ToTensor()
```

to every image when it is retrieved.

***

# 5. Loading the test dataset

```python theme={null}
test_dataset = datasets.MNIST(
    root="./data",
    train=False,
    download=True,
    transform=transform
)
```

This is almost identical.

The important difference:

```python theme={null}
train=False
```

means:

> Give me the testing dataset.

So now you have:

```text theme={null}
train_dataset → 60,000 images
test_dataset  → 10,000 images
```

The model learns using the training data.

The test data is kept separate to evaluate whether the model can recognize images it wasn't trained on.

***

# 6. Creating DataLoaders

## Training DataLoader

```python theme={null}
train_loader = DataLoader(
    train_dataset,
    batch_size=64,
    shuffle=True
)
```

This creates batches of training images.

### `batch_size=64`

Instead of processing one image:

```text theme={null}
image → network
```

the model processes:

```text theme={null}
64 images → network
```

at a time.

***

### `shuffle=True`

Before every training epoch, the training data is shuffled.

For example, imagine your dataset contains:

```text theme={null}
1, 1, 1, 1, 2, 2, 2, 3, 3...
```

Shuffling might make it:

```text theme={null}
7, 2, 9, 1, 4, 3, 8...
```

This generally helps training because the model doesn't repeatedly see the data in the same order.

***

# 7. Test DataLoader

```python theme={null}
test_loader = DataLoader(
    test_dataset,
    batch_size=64,
    shuffle=False
)
```

Again, we use batches of 64.

But:

```python theme={null}
shuffle=False
```

because we don't need random ordering during testing.

The model is simply being evaluated.

***

# 8. Checking dataset sizes

```python theme={null}
print("Training Samples: ", len(train_dataset))
print("Test samples: ", len(test_dataset))
```

Expected output:

```text theme={null}
Training Samples: 60000
Test samples: 10000
```

***

# 9. Looking at one image

```python theme={null}
image, label = train_dataset[0]
```

This retrieves the first training example.

There are two things:

```text theme={null}
image → actual handwritten digit
label → correct answer
```

For example:

```text theme={null}
image → picture of "5"
label → 5
```

***

```python theme={null}
print("Image Shape:", image.shape)
```

MNIST images are:

```text theme={null}
28 × 28
```

Because `ToTensor()` adds a channel dimension, you'll typically get:

```text theme={null}
torch.Size([1, 28, 28])
```

The three dimensions mean:

```text theme={null}
1     → number of channels
28    → height
28    → width
```

MNIST is grayscale, so it has only **one channel**.

For RGB images you'd usually have:

```text theme={null}
3 × height × width
```

because RGB has:

```text theme={null}
Red
Green
Blue
```

***

```python theme={null}
print("Label:", label)
```

Could output:

```text theme={null}
Label: 5
```

***

# 10. Building the neural network

Now comes the important part.

```python theme={null}
class SimpleNN(nn.Module):
```

You're defining your own neural-network class.

`nn.Module` is the base class for PyTorch neural networks.

Your network inherits from it.

Think:

```text theme={null}
nn.Module
    ↓
SimpleNN
```

***

# 11. Constructor

```python theme={null}
def __init__(self):
    super().__init__()
```

`__init__()` runs when you create:

```python theme={null}
model = SimpleNN()
```

***

### Why `super().__init__()`?

Because `SimpleNN` inherits functionality from `nn.Module`.

Calling:

```python theme={null}
super().__init__()
```

initializes the parent `nn.Module` properly.

This allows PyTorch to track things like:

* model parameters
* weights
* gradients
* layers

***

# 12. First neural-network layer

```python theme={null}
self.fc1 = nn.Linear(28*28, 128)
```

This is a fully connected layer.

`28*28` is:

```text theme={null}
784
```

because:

```text theme={null}
28 × 28 = 784
```

An MNIST image contains:

```text theme={null}
784 pixels
```

So the first layer takes:

```text theme={null}
784 inputs
```

and produces:

```text theme={null}
128 outputs
```

Therefore:

```text theme={null}
784 → 128
```

***

## What does `Linear` actually do?

Mathematically:

```text theme={null}
output = input × weights + bias
```

More formally:

```text theme={null}
y = xWᵀ + b
```

The layer learns:

```text theme={null}
weights
biases
```

during training.

Initially, the weights are not useful.

Training gradually changes them.

***

# 13. Second layer

```python theme={null}
self.fc2 = nn.Linear(128, 10)
```

This takes the 128 values from the first layer and produces 10 outputs.

So:

```text theme={null}
128 → 10
```

Why 10?

Because MNIST has 10 classes:

```text theme={null}
0
1
2
3
4
5
6
7
8
9
```

Therefore, the final output contains 10 numbers.

***

# 14. Understanding the complete network

Your network is:

```text theme={null}
MNIST image
    ↓
28 × 28
    ↓
Flatten
    ↓
784
    ↓
Linear layer
    ↓
128
    ↓
ReLU
    ↓
128
    ↓
Linear layer
    ↓
10
    ↓
Predictions
```

***

# 15. The `forward()` function

```python theme={null}
def forward(self, x):
```

This defines how data flows through the network.

When you write:

```python theme={null}
outputs = model(images)
```

PyTorch effectively calls:

```python theme={null}
model.forward(images)
```

***

# 16. Flattening the image

```python theme={null}
x = x.view(x.size(0), -1)
```

This is extremely important.

Your images initially have shape:

```text theme={null}
batch_size × 1 × 28 × 28
```

For example:

```text theme={null}
64 × 1 × 28 × 28
```

But `nn.Linear` expects something like:

```text theme={null}
64 × 784
```

So we flatten each image.

***

### `x.size(0)`

```python theme={null}
x.size(0)
```

returns the batch size.

For example:

```python theme={null}
x.size(0)
```

might be:

```text theme={null}
64
```

***

### `-1`

```python theme={null}
x.view(64, -1)
```

The `-1` means:

> PyTorch, calculate this dimension automatically.

Since:

```text theme={null}
1 × 28 × 28 = 784
```

the result becomes:

```text theme={null}
64 × 784
```

So:

```text theme={null}
64 × 1 × 28 × 28
            ↓
64 × 784
```

***

# 17. First layer

```python theme={null}
x = self.fc1(x)
```

Now:

```text theme={null}
64 × 784
```

goes through:

```text theme={null}
784 → 128
```

Result:

```text theme={null}
64 × 128
```

***

# 18. ReLU activation

```python theme={null}
x = torch.relu(x)
```

ReLU means:

**Rectified Linear Unit**

Mathematically:

```text theme={null}
ReLU(x) = max(0, x)
```

Examples:

```text theme={null}
x = -5 → 0
x = -2 → 0
x = 0  → 0
x = 3  → 3
x = 8  → 8
```

So negative values become zero.

***

## Why do we need ReLU?

Without activation functions, stacking linear layers doesn't give the network much additional expressive power.

ReLU introduces **non-linearity**.

This allows the network to learn more complicated patterns.

***

# 19. Final layer

```python theme={null}
x = self.fc2(x)
```

Now:

```text theme={null}
64 × 128
```

becomes:

```text theme={null}
64 × 10
```

Each image now has 10 output values.

For example, the network might output:

```text theme={null}
[-2.1, 0.3, -1.2, 5.8, 0.2, 1.1, -3.0, 0.5, -1.7, 0.1]
```

The largest value is:

```text theme={null}
5.8
```

which corresponds to digit:

```text theme={null}
3
```

So the model predicts:

```text theme={null}
3
```

***

# 20. Creating the model

```python theme={null}
model = SimpleNN().to(device)
```

First:

```python theme={null}
SimpleNN()
```

creates the neural network.

Then:

```python theme={null}
.to(device)
```

moves it to the selected device.

If:

```text theme={null}
device = cuda
```

the model goes to GPU.

If:

```text theme={null}
device = cpu
```

it stays on CPU.

***

```python theme={null}
print(model)
```

You'll see something similar to:

```text theme={null}
SimpleNN(
  (fc1): Linear(in_features=784, out_features=128, bias=True)
  (fc2): Linear(in_features=128, out_features=10, bias=True)
)
```

***

# 21. Loss function

```python theme={null}
criterion = nn.CrossEntropyLoss()
```

The loss function tells us:

> How wrong is the model?

Your model predicts 10 scores.

Suppose the correct answer is:

```text theme={null}
7
```

but the model gives a high score to:

```text theme={null}
2
```

The loss will be relatively high.

If the model strongly predicts:

```text theme={null}
7
```

the loss will be lower.

***

## Why CrossEntropyLoss?

`CrossEntropyLoss` is commonly used for **multi-class classification**.

MNIST is a multi-class classification problem because there are 10 possible classes:

```text theme={null}
0–9
```

An important detail:

Your final layer should **not** have `softmax` when using `CrossEntropyLoss`.

`CrossEntropyLoss` internally handles the necessary log-softmax operation.

***

# 22. Optimizer

```python theme={null}
optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)
```

The optimizer updates the network's weights.

***

### `model.parameters()`

This gives Adam access to the parameters that need to be learned.

Your network contains parameters such as:

```text theme={null}
fc1 weights
fc1 biases
fc2 weights
fc2 biases
```

***

### Learning rate

```python theme={null}
lr=0.001
```

Learning rate controls how large the updates are.

Conceptually:

```text theme={null}
new weight = old weight - learning_rate × gradient
```

A very large learning rate can make training unstable.

A very small learning rate can make training slow.

`0.001` is a common starting point for Adam.

***

# 23. Number of epochs

```python theme={null}
epochs = 10
```

An **epoch** means:

> The model has processed the entire training dataset once.

You have:

```text theme={null}
60,000 training images
```

So:

```text theme={null}
1 epoch = 60,000 training images processed
```

You're doing:

```text theme={null}
10 epochs
```

Therefore the network gets 10 passes through the training dataset.

***

# 24. Starting the training loop

```python theme={null}
for epoch in range(epochs):
```

Since:

```python theme={null}
epochs = 10
```

this runs:

```text theme={null}
epoch 1
epoch 2
epoch 3
...
epoch 10
```

***

# 25. Training mode

```python theme={null}
model.train()
```

This tells PyTorch:

> The model is currently being trained.

This matters particularly for models containing layers such as:

```text theme={null}
Dropout
BatchNorm
```

Your current network doesn't use them, but it's still good practice to explicitly call:

```python theme={null}
model.train()
```

before training.

***

# 26. Tracking loss

```python theme={null}
total_loss = 0
```

This starts a counter.

During the epoch, you'll accumulate all batch losses:

```text theme={null}
batch 1 loss
+
batch 2 loss
+
batch 3 loss
+
...
```

***

# 27. Getting batches

```python theme={null}
for images, labels in train_loader:
```

`train_loader` gives you batches.

With:

```python theme={null}
batch_size=64
```

you might get:

```text theme={null}
images → [64, 1, 28, 28]
labels → [64]
```

So each iteration processes 64 images.

***

# 28. Moving images to GPU/CPU

```python theme={null}
images = images.to(device)
labels = labels.to(device)
```

Suppose you're using GPU.

The model is on GPU:

```python theme={null}
model.to(cuda)
```

Therefore the input data must also be on GPU.

You can't normally do:

```text theme={null}
model → GPU
images → CPU
```

and expect the operation to work.

Both need to be on the same device.

***

# 29. Clearing old gradients

```python theme={null}
optimizer.zero_grad()
```

This is very important.

PyTorch accumulates gradients by default.

Imagine:

```text theme={null}
Batch 1 → gradient
Batch 2 → gradient
Batch 3 → gradient
```

If you don't clear them, they accumulate.

So before calculating the gradients for the current batch:

```python theme={null}
optimizer.zero_grad()
```

clears the previous gradients.

***

# 30. Forward pass

```python theme={null}
outputs = model(images)
```

This is called the **forward pass**.

Data flows through:

```text theme={null}
images
  ↓
flatten
  ↓
fc1
  ↓
ReLU
  ↓
fc2
  ↓
outputs
```

For a batch of 64 images:

```text theme={null}
Input:
64 × 1 × 28 × 28

Flatten:
64 × 784

fc1:
64 × 128

ReLU:
64 × 128

fc2:
64 × 10
```

***

# 31. Calculate loss

```python theme={null}
loss = criterion(outputs, labels)
```

Now we compare:

```text theme={null}
model predictions
       VS
correct labels
```

The loss tells us how poorly the model performed on that batch.

For example:

```text theme={null}
Loss = 2.31
```

could be early in training.

Later:

```text theme={null}
Loss = 0.15
```

might indicate the model has learned much better.

***

# 32. Backpropagation

```python theme={null}
loss.backward()
```

This is where PyTorch calculates the gradients.

The basic process is:

```text theme={null}
Prediction
    ↓
Loss
    ↓
Backward propagation
    ↓
Gradients
```

A gradient tells us approximately:

> If I change this parameter, how will the loss change?

PyTorch's **autograd** system automatically calculates these gradients.

You don't have to manually derive all the calculus.

***

# 33. Updating the weights

```python theme={null}
optimizer.step()
```

Now Adam uses the calculated gradients to update the model parameters.

Conceptually:

```text theme={null}
weights
   ↓
calculate gradients
   ↓
optimizer
   ↓
updated weights
```

The model is therefore learning.

***

# 34. Accumulating the loss

```python theme={null}
total_loss += loss.item()
```

`loss` is a PyTorch tensor.

```python theme={null}
loss.item()
```

extracts the ordinary Python number from it.

For example:

```python theme={null}
loss.item()
```

might return:

```text theme={null}
0.3421
```

Then:

```python theme={null}
total_loss
```

keeps adding the losses from all batches.

***

# 35. Printing average loss

```python theme={null}
print(
    f"Epoch {epoch+1}/{epochs}, "
    f"Loss: {total_loss/len(train_loader): .4f}"
)
```

Suppose:

```text theme={null}
total_loss = 100
```

and:

```text theme={null}
len(train_loader) = 938
```

Then:

```python theme={null}
total_loss / len(train_loader)
```

gives the average batch loss.

***

### Why `epoch+1`?

Python starts counting from zero:

```python theme={null}
range(10)
```

produces:

```text theme={null}
0
1
2
...
9
```

But humans usually want:

```text theme={null}
1
2
3
...
10
```

Therefore:

```python theme={null}
epoch + 1
```

is used.

***

### `:.4f`

```python theme={null}
:.4f
```

means:

> Display the number with 4 digits after the decimal point.

For example:

```text theme={null}
0.234567
```

becomes:

```text theme={null}
0.2346
```

***

# 36. Testing the model

After training:

```python theme={null}
model.eval()
```

This puts the model into evaluation mode.

Again, this is particularly important for layers such as Dropout and BatchNorm.

***

# 37. Initialize counters

```python theme={null}
correct = 0
total = 0
```

We need to calculate:

```text theme={null}
accuracy =
correct predictions / total predictions
```

So we start with zero.

***

# 38. Disable gradient calculation

```python theme={null}
with torch.no_grad():
```

During testing, we're not training.

We don't need gradients.

So:

```python theme={null}
torch.no_grad()
```

tells PyTorch:

> Don't calculate/store gradients here.

This saves memory and computation.

***

# 39. Loop through test batches

```python theme={null}
for images, labels in test_loader:
```

Again, we're receiving batches of 64 images.

***

# 40. Move test data to device

```python theme={null}
images = images.to(device)
labels = labels.to(device)
```

Same reason as during training.

The data needs to be on the same device as the model.

***

# 41. Get predictions

```python theme={null}
outputs = model(images)
```

The model produces:

```text theme={null}
64 × 10
```

For each of the 64 images, there are 10 output scores.

For example:

```text theme={null}
Image 1 → [0.1, -2.1, 4.8, ...]
Image 2 → [5.4, 0.2, -1.1, ...]
...
```

***

# 42. Find predicted class

```python theme={null}
_, predicted = torch.max(outputs, 1)
```

This finds the largest score in each row.

Suppose:

```python theme={null}
outputs =
[
    [1.2, 5.8, 0.4],
    [7.1, 2.2, 1.0]
]
```

The maximum values are:

```text theme={null}
5.8
7.1
```

Their indices are:

```text theme={null}
1
0
```

Therefore:

```text theme={null}
Prediction 1 → class 1
Prediction 2 → class 0
```

***

### What is `_`?

`torch.max()` returns two things:

```text theme={null}
maximum values
indices
```

You only need the indices.

So:

```python theme={null}
_, predicted = torch.max(outputs, 1)
```

means:

```text theme={null}
_          → ignore maximum values
predicted  → keep indices
```

***

# 43. Count total images

```python theme={null}
total += labels.size(0)
```

`labels.size(0)` gives the number of labels in the current batch.

Usually:

```text theme={null}
64
```

So:

```python theme={null}
total += 64
```

Eventually:

```text theme={null}
total = 10,000
```

***

# 44. Count correct predictions

```python theme={null}
correct += (predicted == labels).sum().item()
```

Let's break this down.

Suppose:

```text theme={null}
predicted = [7, 2, 3, 5]
labels    = [7, 1, 3, 5]
```

Comparison:

```python theme={null}
predicted == labels
```

gives:

```text theme={null}
[True, False, True, True]
```

Then:

```python theme={null}
.sum()
```

counts the `True` values:

```text theme={null}
3
```

So:

```python theme={null}
correct += 3
```

***

# 45. Calculate accuracy

```python theme={null}
accuracy = 100 * correct/total
```

Suppose:

```text theme={null}
correct = 9,800
total = 10,000
```

Then:

```text theme={null}
accuracy = 100 × 9800 / 10000
         = 98%
```

***

# 46. There's a small bug in your final print

You wrote:

```python theme={null}
print("Test Accuracy: {accuracy: .2f}%")
```

This will literally print:

```text theme={null}
Test Accuracy: {accuracy: .2f}%
```

because you forgot the `f` before the string.

You need:

```python theme={null}
print(f"Test Accuracy: {accuracy:.2f}%")
```

Now if:

```python theme={null}
accuracy = 98.12
```

you'll get:

```text theme={null}
Test Accuracy: 98.12%
```

***

# 47. Code

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim

from torch.utils.data import DataLoader
from torchvision import datasets, transforms


# --------------------------------------------------
# 1. Select device
# --------------------------------------------------

device = torch.device(
    "cuda" if torch.cuda.is_available() else "cpu"
)

print("Using:", device)


# --------------------------------------------------
# 2. Image transformation
# --------------------------------------------------

transform = transforms.ToTensor()


# --------------------------------------------------
# 3. Load training dataset
# --------------------------------------------------

train_dataset = datasets.MNIST(
    root="./data",
    train=True,
    download=True,
    transform=transform
)


# --------------------------------------------------
# 4. Load test dataset
# --------------------------------------------------

test_dataset = datasets.MNIST(
    root="./data",
    train=False,
    download=True,
    transform=transform
)


# --------------------------------------------------
# 5. Create DataLoaders
# --------------------------------------------------

train_loader = DataLoader(
    train_dataset,
    batch_size=64,
    shuffle=True
)

test_loader = DataLoader(
    test_dataset,
    batch_size=64,
    shuffle=False
)


# --------------------------------------------------
# 6. Check dataset
# --------------------------------------------------

print("Training Samples:", len(train_dataset))
print("Test Samples:", len(test_dataset))


# Check one image

image, label = train_dataset[0]

print("Image Shape:", image.shape)
print("Label:", label)


# --------------------------------------------------
# 7. Define neural network
# --------------------------------------------------

class SimpleNN(nn.Module):

    def __init__(self):
        super().__init__()

        # 28 x 28 = 784 input pixels
        self.fc1 = nn.Linear(28 * 28, 128)

        # 128 hidden neurons -> 10 output classes
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):

        # Flatten image
        x = x.view(x.size(0), -1)

        # First layer
        x = self.fc1(x)

        # Activation function
        x = torch.relu(x)

        # Output layer
        x = self.fc2(x)

        return x


# --------------------------------------------------
# 8. Create model
# --------------------------------------------------

model = SimpleNN().to(device)

print(model)


# --------------------------------------------------
# 9. Loss function
# --------------------------------------------------

criterion = nn.CrossEntropyLoss()


# --------------------------------------------------
# 10. Optimizer
# --------------------------------------------------

optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)


# --------------------------------------------------
# 11. Training
# --------------------------------------------------

epochs = 10

for epoch in range(epochs):

    model.train()

    total_loss = 0

    for images, labels in train_loader:

        # Move data to CPU/GPU
        images = images.to(device)
        labels = labels.to(device)

        # Clear old gradients
        optimizer.zero_grad()

        # Forward pass
        outputs = model(images)

        # Calculate loss
        loss = criterion(outputs, labels)

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()

        # Add batch loss
        total_loss += loss.item()

    # Average loss
    average_loss = total_loss / len(train_loader)

    print(
        f"Epoch {epoch + 1}/{epochs}, "
        f"Loss: {average_loss:.4f}"
    )


# --------------------------------------------------
# 12. Evaluation
# --------------------------------------------------

model.eval()

correct = 0
total = 0

with torch.no_grad():

    for images, labels in test_loader:

        images = images.to(device)
        labels = labels.to(device)

        # Get predictions
        outputs = model(images)

        # Get class with highest score
        _, predicted = torch.max(outputs, 1)

        # Count samples
        total += labels.size(0)

        # Count correct predictions
        correct += (predicted == labels).sum().item()


# --------------------------------------------------
# 13. Calculate accuracy
# --------------------------------------------------

accuracy = 100 * correct / total

print(f"Test Accuracy: {accuracy:.2f}%")
```

***

# 48. The entire process in one picture

The most important thing is to understand the **flow of data**.

Your program does this:

```text theme={null}
                 MNIST DATASET
                      │
             ┌────────┴────────┐
             ↓                 ↓
        Training Data       Test Data
        60,000 images       10,000 images
             │                 │
             ↓                 │
        DataLoader             │
             │                 │
             ↓                 │
        64 images/batch        │
             │                 │
             ↓                 │
       ┌─────────────┐         │
       │ Neural      │         │
       │ Network     │         │
       └─────────────┘         │
             │                 │
             ↓                 │
          784 inputs            │
             │                 │
             ↓                 │
        Linear 784→128          │
             │                 │
             ↓                 │
            ReLU                │
             │                 │
             ↓                 │
        Linear 128→10           │
             │                 │
             ↓                 │
        10 class scores         │
             │                 │
             ↓                 │
       Calculate Loss           │
             │                 │
             ↓                 │
       Backpropagation          │
             │                 │
             ↓                 │
       Adam updates weights     │
             │                 │
             └───────┐         │
                     │         │
                     ↓         │
               Repeat batches  │
                     │         │
                     ↓         │
               Repeat epochs   │
                               │
                               ↓
                         Test the model
                               │
                               ↓
                          Predictions
                               │
                               ↓
                           Accuracy
```

***

# 49. The most important concepts to remember

If you're learning PyTorch, focus on these concepts first:

| Concept              | Meaning                                        |
| -------------------- | ---------------------------------------------- |
| **Tensor**           | Numerical data structure used by PyTorch       |
| **Dataset**          | Collection of training/testing examples        |
| **DataLoader**       | Provides data in batches                       |
| **Batch**            | Small group of training examples               |
| **Model**            | Neural network that makes predictions          |
| **Layer**            | Transformation of data inside the network      |
| **Forward pass**     | Input → prediction                             |
| **Loss**             | Measures how wrong the prediction is           |
| **Backward pass**    | Calculates gradients                           |
| **Gradient**         | Direction/magnitude used to improve parameters |
| **Optimizer**        | Updates model parameters                       |
| **Epoch**            | One complete pass through training data        |
| **ReLU**             | Non-linear activation function                 |
| **CrossEntropyLoss** | Classification loss                            |
| **Evaluation**       | Testing how well the trained model performs    |

***

# 50. The training cycle you should memorize

The core of virtually every PyTorch training loop is:

```python theme={null}
optimizer.zero_grad()

outputs = model(images)

loss = criterion(outputs, labels)

loss.backward()

optimizer.step()
```

Think of it as:

```text theme={null}
1. Clear old gradients
          ↓
2. Make prediction
          ↓
3. Calculate error
          ↓
4. Calculate gradients
          ↓
5. Update weights
```

Or even more simply:

```text theme={null}
PREDICT
   ↓
COMPARE
   ↓
CALCULATE GRADIENT
   ↓
UPDATE
   ↓
REPEAT
```