> ## Documentation Index
> Fetch the complete documentation index at: https://ai.tharung.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Pytorch - Important

This note covers the core PyTorch concepts you need for machine learning and deep learning:

1. What is PyTorch?
2. Installation
3. PyTorch workflow
4. Tensors
5. Tensor operations
6. Dataset and DataLoader
7. Transforms
8. Neural networks
9. Optimizers
10. Autograd
11. Backpropagation
12. Loss functions
13. Complete training loop
14. Complete example

***

# 1. What is PyTorch?

**PyTorch** is an open-source deep learning framework developed by Meta AI.

It is mainly used for:

* Machine Learning
* Deep Learning
* Neural Networks
* Computer Vision
* Natural Language Processing
* Generative AI
* Reinforcement Learning

PyTorch provides:

* Tensor computation
* GPU acceleration
* Automatic differentiation
* Neural network building blocks
* Optimizers
* Dataset and DataLoader utilities

### Simple idea

```text theme={null}
Data
 ↓
Tensor
 ↓
Neural Network
 ↓
Prediction
 ↓
Loss
 ↓
Backpropagation
 ↓
Optimizer
 ↓
Updated Weights
```

***

# 2. Install PyTorch

For a basic installation:

```bash theme={null}
pip install torch torchvision torchaudio
```

Check the installation:

```python theme={null}
import torch

print(torch.__version__)
```

Example output:

```text theme={null}
2.x.x
```

***

# 3. Import PyTorch

```python theme={null}
import torch
```

You will commonly use:

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
```

***

# 4. PyTorch Workflow

A typical PyTorch deep-learning project follows this workflow:

```text theme={null}
1. Load data
      ↓
2. Convert data to tensors
      ↓
3. Create Dataset
      ↓
4. Create DataLoader
      ↓
5. Define neural network
      ↓
6. Define loss function
      ↓
7. Define optimizer
      ↓
8. Forward pass
      ↓
9. Calculate loss
      ↓
10. Backpropagation
      ↓
11. Update weights
      ↓
12. Repeat for multiple epochs
      ↓
13. Evaluate model
```

***

# 5. PyTorch Tensors

A **tensor** is the fundamental data structure in PyTorch.

You can think of a tensor as a generalization of:

* Scalar → 0D tensor
* Vector → 1D tensor
* Matrix → 2D tensor
* Higher-dimensional array → 3D, 4D, etc.

***

## 5.1 Scalar

A scalar contains one value.

```python theme={null}
import torch

x = torch.tensor(10)

print(x)
print(x.ndim)
```

Output:

```text theme={null}
tensor(10)
0
```

***

## 5.2 Vector

```python theme={null}
x = torch.tensor([10, 20, 30])

print(x)
print(x.ndim)
print(x.shape)
```

Output:

```text theme={null}
tensor([10, 20, 30])
1
torch.Size([3])
```

***

## 5.3 Matrix

```python theme={null}
x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x)
print(x.ndim)
print(x.shape)
```

Output:

```text theme={null}
tensor([[1, 2, 3],
        [4, 5, 6]])

2
torch.Size([2, 3])
```

***

# 6. Creating Tensors

## `torch.tensor()`

```python theme={null}
x = torch.tensor([1, 2, 3])

print(x)
```

Output:

```text theme={null}
tensor([1, 2, 3])
```

***

## `torch.zeros()`

Creates a tensor filled with zeros.

```python theme={null}
x = torch.zeros(2, 3)

print(x)
```

Output:

```text theme={null}
tensor([[0., 0., 0.],
        [0., 0., 0.]])
```

***

## `torch.ones()`

```python theme={null}
x = torch.ones(2, 3)

print(x)
```

Output:

```text theme={null}
tensor([[1., 1., 1.],
        [1., 1., 1.]])
```

***

## `torch.full()`

```python theme={null}
x = torch.full((2, 3), 5)

print(x)
```

Output:

```text theme={null}
tensor([[5, 5, 5],
        [5, 5, 5]])
```

***

## `torch.arange()`

```python theme={null}
x = torch.arange(0, 10)

print(x)
```

Output:

```text theme={null}
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
```

***

## `torch.linspace()`

Creates evenly spaced values.

```python theme={null}
x = torch.linspace(0, 1, 5)

print(x)
```

Output:

```text theme={null}
tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
```

***

## Random tensor

```python theme={null}
x = torch.rand(2, 3)

print(x)
```

Example:

```text theme={null}
tensor([[0.32, 0.71, 0.12],
        [0.83, 0.45, 0.91]])
```

Values are between `0` and `1`.

***

# 7. Tensor Data Types

```python theme={null}
x = torch.tensor([1, 2, 3])

print(x.dtype)
```

Output:

```text theme={null}
torch.int64
```

Float tensor:

```python theme={null}
x = torch.tensor([1.0, 2.0, 3.0])

print(x.dtype)
```

Output:

```text theme={null}
torch.float32
```

Convert datatype:

```python theme={null}
x = torch.tensor([1, 2, 3])

x = x.float()

print(x.dtype)
```

Output:

```text theme={null}
torch.float32
```

***

# 8. Tensor Shape

```python theme={null}
x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x.shape)
```

Output:

```text theme={null}
torch.Size([2, 3])
```

This means:

```text theme={null}
2 rows
3 columns
```

***

# 9. Tensor Indexing

```python theme={null}
x = torch.tensor([10, 20, 30, 40])

print(x[0])
print(x[2])
```

Output:

```text theme={null}
tensor(10)
tensor(30)
```

For a matrix:

```python theme={null}
x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x[0])
print(x[0, 1])
```

Output:

```text theme={null}
tensor([1, 2, 3])
tensor(2)
```

***

# 10. Tensor Slicing

```python theme={null}
x = torch.tensor([10, 20, 30, 40, 50])

print(x[1:4])
```

Output:

```text theme={null}
tensor([20, 30, 40])
```

***

# 11. Tensor Arithmetic

```python theme={null}
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

print(a + b)
print(a - b)
print(a * b)
print(a / b)
```

Output:

```text theme={null}
tensor([5, 7, 9])
tensor([-3, -3, -3])
tensor([ 4, 10, 18])
tensor([0.2500, 0.4000, 0.5000])
```

***

# 12. Matrix Multiplication

Matrix multiplication is extremely important in neural networks.

```python theme={null}
a = torch.tensor([
    [1, 2],
    [3, 4]
])

b = torch.tensor([
    [5, 6],
    [7, 8]
])

result = torch.matmul(a, b)

print(result)
```

Output:

```text theme={null}
tensor([[19, 22],
        [43, 50]])
```

You can also use:

```python theme={null}
result = a @ b
```

***

# 13. Reshaping Tensors

Use `.reshape()` to change the tensor shape.

```python theme={null}
x = torch.tensor([1, 2, 3, 4, 5, 6])

x = x.reshape(2, 3)

print(x)
```

Output:

```text theme={null}
tensor([
    [1, 2, 3],
    [4, 5, 6]
])
```

***

# 14. Flatten

Flatten converts multiple dimensions into one dimension.

```python theme={null}
x = torch.tensor([
    [1, 2],
    [3, 4]
])

print(x.flatten())
```

Output:

```text theme={null}
tensor([1, 2, 3, 4])
```

This is commonly used before fully connected layers.

***

# 15. Tensor Device — CPU and GPU

PyTorch can execute tensor operations on:

* CPU
* GPU

Check GPU availability:

```python theme={null}
import torch

print(torch.cuda.is_available())
```

Output might be:

```text theme={null}
True
```

Create device:

```python theme={null}
device = torch.device(
    "cuda" if torch.cuda.is_available() else "cpu"
)

print(device)
```

Move tensor to device:

```python theme={null}
x = torch.tensor([1, 2, 3])

x = x.to(device)
```

Move model:

```python theme={null}
model = model.to(device)
```

During training, both model and data must be on compatible devices.

***

# 16. Dataset

A **Dataset** represents your data.

PyTorch provides:

```python theme={null}
torch.utils.data.Dataset
```

A custom dataset normally implements:

```python theme={null}
__init__()
__len__()
__getitem__()
```

***

# 17. Creating a Custom Dataset

Suppose we have:

```text theme={null}
Hours studied → Exam score

1 → 20
2 → 35
3 → 50
4 → 65
5 → 80
```

Create the dataset:

```python theme={null}
import torch
from torch.utils.data import Dataset

class StudentDataset(Dataset):

    def __init__(self):
        self.x = torch.tensor(
            [[1.0], [2.0], [3.0], [4.0], [5.0]]
        )

        self.y = torch.tensor(
            [[20.0], [35.0], [50.0], [65.0], [80.0]]
        )

    def __len__(self):
        return len(self.x)

    def __getitem__(self, index):
        return self.x[index], self.y[index]
```

Create dataset:

```python theme={null}
dataset = StudentDataset()

print(len(dataset))
```

Output:

```text theme={null}
5
```

Get one sample:

```python theme={null}
x, y = dataset[0]

print(x)
print(y)
```

Output:

```text theme={null}
tensor([1.])
tensor([20.])
```

***

# 18. Why Dataset?

Dataset provides a standard way to:

* Store data
* Access individual samples
* Separate data from model logic
* Work with DataLoader

***

# 19. DataLoader

A `DataLoader` loads data in batches.

```python theme={null}
from torch.utils.data import DataLoader

loader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)
```

Loop through batches:

```python theme={null}
for x, y in loader:
    print(x)
    print(y)
```

Instead of processing:

```text theme={null}
5 samples
```

all at once, DataLoader can process:

```text theme={null}
Batch 1 → 2 samples
Batch 2 → 2 samples
Batch 3 → 1 sample
```

***

# 20. Important DataLoader Parameters

```python theme={null}
DataLoader(
    dataset,
    batch_size=32,
    shuffle=True
)
```

### `batch_size`

Number of samples processed at once.

```python theme={null}
batch_size=32
```

means:

```text theme={null}
32 samples → model
```

### `shuffle`

Randomizes the training data.

```python theme={null}
shuffle=True
```

This helps prevent the model from learning based on the original ordering of training data.

***

# 21. Dataset vs DataLoader

| Component  | Purpose                                   |
| ---------- | ----------------------------------------- |
| Dataset    | Stores and retrieves samples              |
| DataLoader | Creates batches and iterates over dataset |

Simple way to remember:

```text theme={null}
Dataset = Data

DataLoader = How data is delivered
```

***

# 22. Transforms

Transforms are used to preprocess or modify data.

Commonly used with:

* Images
* Computer vision
* Data augmentation

Import:

```python theme={null}
from torchvision import transforms
```

***

# 23. `ToTensor()`

Converts image data into a PyTorch tensor.

```python theme={null}
transform = transforms.ToTensor()
```

Example:

```python theme={null}
from torchvision import transforms

transform = transforms.ToTensor()
```

***

# 24. Normalize

Normalization changes the scale of input data.

```python theme={null}
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.5,),
        std=(0.5,)
    )
])
```

For RGB images:

```python theme={null}
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.5, 0.5, 0.5),
        std=(0.5, 0.5, 0.5)
    )
])
```

***

# 25. Resize

Resize images to a fixed size.

```python theme={null}
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor()
])
```

This converts images to:

```text theme={null}
224 × 224
```

***

# 26. Data Augmentation

Data augmentation creates variations of training images.

Example:

```python theme={null}
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ToTensor()
])
```

This can help reduce overfitting.

***

# 27. `Compose`

`Compose` combines multiple transformations.

```python theme={null}
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(
        (0.5, 0.5, 0.5),
        (0.5, 0.5, 0.5)
    )
])
```

Execution:

```text theme={null}
Image
 ↓
Resize
 ↓
ToTensor
 ↓
Normalize
 ↓
Model
```

***

# 28. Neural Network in PyTorch

PyTorch provides:

```python theme={null}
torch.nn
```

Import:

```python theme={null}
import torch.nn as nn
```

A neural network usually inherits from:

```python theme={null}
nn.Module
```

***

# 29. Simple Neural Network

```python theme={null}
import torch
import torch.nn as nn

class SimpleModel(nn.Module):

    def __init__(self):
        super().__init__()

        self.layer = nn.Linear(1, 1)

    def forward(self, x):
        return self.layer(x)
```

Create model:

```python theme={null}
model = SimpleModel()

print(model)
```

Output:

```text theme={null}
SimpleModel(
  (layer): Linear(in_features=1, out_features=1, bias=True)
)
```

***

# 30. Understanding `nn.Linear`

```python theme={null}
nn.Linear(3, 2)
```

means:

```text theme={null}
3 input features
        ↓
Linear Layer
        ↓
2 output features
```

Mathematically:

```text theme={null}
y = xW + b
```

where:

* `x` = input
* `W` = weights
* `b` = bias
* `y` = output

***

# 31. Multiple Layers

```python theme={null}
class NeuralNetwork(nn.Module):

    def __init__(self):
        super().__init__()

        self.network = nn.Sequential(
            nn.Linear(4, 16),
            nn.ReLU(),
            nn.Linear(16, 8),
            nn.ReLU(),
            nn.Linear(8, 1)
        )

    def forward(self, x):
        return self.network(x)
```

Architecture:

```text theme={null}
4 input features
       ↓
Linear(4 → 16)
       ↓
ReLU
       ↓
Linear(16 → 8)
       ↓
ReLU
       ↓
Linear(8 → 1)
       ↓
Prediction
```

***

# 32. Activation Function

Activation functions introduce non-linearity.

Common activations:

```python theme={null}
nn.ReLU()
nn.Sigmoid()
nn.Tanh()
nn.Softmax()
```

Example:

```python theme={null}
x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])

relu = nn.ReLU()

print(relu(x))
```

Output:

```text theme={null}
tensor([0., 0., 0., 1., 2.])
```

***

# 33. Loss Function

Loss measures how wrong the model's prediction is.

Concept:

```text theme={null}
Prediction
    ↓
Compare with actual value
    ↓
Loss
```

Common loss functions:

| Task                       | Loss                        |
| -------------------------- | --------------------------- |
| Regression                 | MSELoss                     |
| Binary classification      | BCELoss / BCEWithLogitsLoss |
| Multi-class classification | CrossEntropyLoss            |

***

# 34. MSE Loss

Mean Squared Error is commonly used for regression.

```python theme={null}
loss_function = nn.MSELoss()
```

Example:

```python theme={null}
prediction = torch.tensor([10.0])
actual = torch.tensor([12.0])

loss = loss_function(prediction, actual)

print(loss)
```

Output:

```text theme={null}
tensor(4.)
```

Because:

```text theme={null}
(10 - 12)² = 4
```

***

# 35. Optimizers

An **optimizer updates the model's parameters to reduce the loss**.

Common optimizers:

* SGD
* Adam
* RMSprop
* AdamW

***

# 36. SGD Optimizer

SGD = Stochastic Gradient Descent.

```python theme={null}
optimizer = torch.optim.SGD(
    model.parameters(),
    lr=0.01
)
```

Here:

```text theme={null}
lr = learning rate
```

The optimizer uses gradients to update weights.

Conceptually:

```text theme={null}
weight = weight - learning_rate × gradient
```

***

# 37. Adam Optimizer

Adam is one of the most commonly used optimizers.

```python theme={null}
optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.001
)
```

Adam adapts the update for each parameter using information from past gradients.

A common starting point is:

```python theme={null}
lr=0.001
```

***

# 38. Optimizer Comparison

| Optimizer | Description                              |
| --------- | ---------------------------------------- |
| SGD       | Simple gradient descent                  |
| Adam      | Adaptive and commonly used               |
| RMSprop   | Adaptive learning rate                   |
| AdamW     | Adam with improved weight decay handling |

***

# 39. What is Autograd?

**Autograd** is PyTorch's automatic differentiation system.

It automatically calculates gradients.

Example:

```python theme={null}
x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 2

y.backward()

print(x.grad)
```

Output:

```text theme={null}
tensor(4.)
```

Why?

We have:

```text theme={null}
y = x²
```

Derivative:

```text theme={null}
dy/dx = 2x
```

At:

```text theme={null}
x = 2
```

therefore:

```text theme={null}
dy/dx = 4
```

PyTorch calculates this automatically.

***

# 40. `requires_grad=True`

When you write:

```python theme={null}
x = torch.tensor(
    2.0,
    requires_grad=True
)
```

you tell PyTorch:

> Track operations involving this tensor because I may need its gradient.

Check:

```python theme={null}
print(x.requires_grad)
```

Output:

```text theme={null}
True
```

***

# 41. Gradient

A gradient tells us how much a value changes when a parameter changes.

Example:

```python theme={null}
x = torch.tensor(
    3.0,
    requires_grad=True
)

y = x ** 2

y.backward()

print(x.grad)
```

Output:

```text theme={null}
tensor(6.)
```

Because:

```text theme={null}
y = x²

dy/dx = 2x

x = 3

gradient = 6
```

***

# 42. `.backward()`

The `.backward()` function calculates gradients.

Example:

```python theme={null}
x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 3

y.backward()

print(x.grad)
```

Mathematically:

```text theme={null}
y = x³

dy/dx = 3x²

x = 2

gradient = 12
```

Output:

```text theme={null}
tensor(12.)
```

***

# 43. Computational Graph

PyTorch creates a computational graph when operations are performed on tensors that require gradients.

Example:

```python theme={null}
x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 2

z = y + 3
```

Graph:

```text theme={null}
x
↓
x²
↓
y
↓
+ 3
↓
z
```

When you call:

```python theme={null}
z.backward()
```

PyTorch calculates gradients through this graph.

***

# 44. Backpropagation

**Backpropagation** is the process of calculating how much each model parameter contributed to the error.

Basic flow:

```text theme={null}
Input
  ↓
Forward Pass
  ↓
Prediction
  ↓
Loss
  ↓
Backward Pass
  ↓
Gradients
  ↓
Optimizer
  ↓
Updated Weights
```

***

# 45. Forward Pass

Suppose:

```text theme={null}
Input = 2
Weight = 3
Bias = 1
```

The neuron calculates:

```text theme={null}
y = wx + b
```

Therefore:

```text theme={null}
y = 3(2) + 1
y = 7
```

In PyTorch:

```python theme={null}
x = torch.tensor([[2.0]])

model = nn.Linear(1, 1)

prediction = model(x)

print(prediction)
```

***

# 46. Complete Autograd Example

```python theme={null}
import torch

x = torch.tensor(
    2.0,
    requires_grad=True
)

w = torch.tensor(
    3.0,
    requires_grad=True
)

b = torch.tensor(
    1.0,
    requires_grad=True
)

y = w * x + b

target = torch.tensor(10.0)

loss = (y - target) ** 2

loss.backward()

print("Prediction:", y.item())
print("Loss:", loss.item())

print("dw:", w.grad.item())
print("db:", b.grad.item())
```

Let's understand it.

### Forward calculation

```text theme={null}
y = wx + b

y = 3 × 2 + 1

y = 7
```

Target:

```text theme={null}
10
```

Loss:

```text theme={null}
(7 - 10)²
= 9
```

Then:

```python theme={null}
loss.backward()
```

calculates:

```text theme={null}
∂Loss/∂w
∂Loss/∂b
```

***

# 47. Gradient Descent Manually

Suppose:

```text theme={null}
weight = 3
gradient = -12
learning_rate = 0.01
```

Update:

```text theme={null}
new_weight
=
weight - learning_rate × gradient
```

Therefore:

```text theme={null}
new_weight
=
3 - (0.01 × -12)

= 3.12
```

PyTorch's optimizer performs this update automatically.

***

# 48. The Three Important Optimizer Steps

During training, you commonly see:

```python theme={null}
optimizer.zero_grad()
```

then:

```python theme={null}
loss.backward()
```

then:

```python theme={null}
optimizer.step()
```

These are extremely important.

***

## Step 1 — `zero_grad()`

```python theme={null}
optimizer.zero_grad()
```

Clears previously stored gradients.

PyTorch accumulates gradients by default.

***

## Step 2 — `backward()`

```python theme={null}
loss.backward()
```

Calculates gradients.

***

## Step 3 — `step()`

```python theme={null}
optimizer.step()
```

Updates model parameters using those gradients.

***

# 49. Complete Training Loop

```python theme={null}
for epoch in range(epochs):

    for x, y in dataloader:

        # Forward pass
        prediction = model(x)

        # Calculate loss
        loss = loss_function(
            prediction,
            y
        )

        # Clear old gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()
```

This is the core PyTorch training pattern.

***

# 50. Complete Regression Example

Let's build a simple model that learns:

```text theme={null}
y = 2x + 1
```

Dataset:

```text theme={null}
x → y

1 → 3
2 → 5
3 → 7
4 → 9
5 → 11
```

***

## Step 1 — Import libraries

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader
```

***

## Step 2 — Create data

```python theme={null}
X = torch.tensor([
    [1.0],
    [2.0],
    [3.0],
    [4.0],
    [5.0]
])

y = torch.tensor([
    [3.0],
    [5.0],
    [7.0],
    [9.0],
    [11.0]
])
```

***

## Step 3 — Create Dataset

```python theme={null}
dataset = TensorDataset(X, y)
```

`TensorDataset` is convenient when your data is already stored as tensors.

***

## Step 4 — Create DataLoader

```python theme={null}
dataloader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)
```

***

## Step 5 — Create Model

```python theme={null}
model = nn.Linear(1, 1)
```

The model contains:

```text theme={null}
1 input
↓
Linear layer
↓
1 output
```

***

## Step 6 — Loss Function

```python theme={null}
loss_function = nn.MSELoss()
```

***

## Step 7 — Optimizer

```python theme={null}
optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)
```

***

## Step 8 — Training

```python theme={null}
epochs = 1000

for epoch in range(epochs):

    for X_batch, y_batch in dataloader:

        # Forward pass
        prediction = model(X_batch)

        # Calculate loss
        loss = loss_function(
            prediction,
            y_batch
        )

        # Clear gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()

    if (epoch + 1) % 100 == 0:
        print(
            f"Epoch {epoch + 1}, "
            f"Loss: {loss.item():.4f}"
        )
```

***

# 51. Make a Prediction

After training:

```python theme={null}
test = torch.tensor([[6.0]])

prediction = model(test)

print(prediction)
```

Expected result:

```text theme={null}
tensor([[~13.]])
```

Because the model learned approximately:

```text theme={null}
y = 2x + 1
```

For:

```text theme={null}
x = 6
```

we expect:

```text theme={null}
y = 13
```

***

# 52. Complete PyTorch Example

Here is the same example in one place:

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader


# -------------------------
# 1. Create data
# -------------------------

X = torch.tensor([
    [1.0],
    [2.0],
    [3.0],
    [4.0],
    [5.0]
])

y = torch.tensor([
    [3.0],
    [5.0],
    [7.0],
    [9.0],
    [11.0]
])


# -------------------------
# 2. Dataset
# -------------------------

dataset = TensorDataset(X, y)


# -------------------------
# 3. DataLoader
# -------------------------

dataloader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)


# -------------------------
# 4. Model
# -------------------------

model = nn.Linear(1, 1)


# -------------------------
# 5. Loss
# -------------------------

loss_function = nn.MSELoss()


# -------------------------
# 6. Optimizer
# -------------------------

optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)


# -------------------------
# 7. Training
# -------------------------

epochs = 1000

for epoch in range(epochs):

    for X_batch, y_batch in dataloader:

        # Forward pass
        prediction = model(X_batch)

        # Calculate loss
        loss = loss_function(
            prediction,
            y_batch
        )

        # Clear gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()


    if (epoch + 1) % 100 == 0:

        print(
            f"Epoch: {epoch + 1}, "
            f"Loss: {loss.item():.4f}"
        )


# -------------------------
# 8. Prediction
# -------------------------

test = torch.tensor([[6.0]])

prediction = model(test)

print("Prediction:", prediction.item())
```

Expected:

```text theme={null}
Prediction: approximately 13
```

***

# 53. Training Loop Explained

The most important part is:

```python theme={null}
prediction = model(X_batch)
```

This is the **forward pass**.

Then:

```python theme={null}
loss = loss_function(
    prediction,
    y_batch
)
```

This calculates the error.

Then:

```python theme={null}
optimizer.zero_grad()
```

Clears previous gradients.

Then:

```python theme={null}
loss.backward()
```

Calculates gradients using backpropagation.

Finally:

```python theme={null}
optimizer.step()
```

Updates weights.

So remember:

```text theme={null}
Forward
   ↓
Loss
   ↓
zero_grad
   ↓
Backward
   ↓
Optimizer step
```

***

# 54. Why Do We Need `zero_grad()`?

Consider:

```python theme={null}
loss.backward()
```

PyTorch accumulates gradients.

For example:

```text theme={null}
Step 1:
gradient = 2

Step 2:
gradient = 3

Without zero_grad:
gradient = 5
```

Usually we want each training iteration to use its own gradients.

Therefore:

```python theme={null}
optimizer.zero_grad()
```

is used before:

```python theme={null}
loss.backward()
```

***

# 55. `model.train()`

During training:

```python theme={null}
model.train()
```

Example:

```python theme={null}
model.train()

for X_batch, y_batch in dataloader:
    ...
```

This puts the model into training mode.

It matters especially for layers such as:

* Dropout
* Batch Normalization

***

# 56. `model.eval()`

During evaluation:

```python theme={null}
model.eval()
```

Example:

```python theme={null}
model.eval()

with torch.no_grad():
    prediction = model(X_test)
```

***

# 57. `torch.no_grad()`

During prediction, gradients usually aren't required.

Use:

```python theme={null}
with torch.no_grad():

    prediction = model(X_test)
```

Advantages:

* Less memory usage
* Faster inference
* No unnecessary gradient calculations

***

# 58. Training vs Evaluation

### Training

```python theme={null}
model.train()

optimizer.zero_grad()

prediction = model(X)

loss = loss_function(
    prediction,
    y
)

loss.backward()

optimizer.step()
```

### Evaluation

```python theme={null}
model.eval()

with torch.no_grad():

    prediction = model(X_test)
```

***

# 59. Classification Example

For a multi-class classification problem:

```python theme={null}
model = nn.Sequential(
    nn.Linear(4, 16),
    nn.ReLU(),
    nn.Linear(16, 3)
)
```

Suppose:

```text theme={null}
4 input features
3 classes
```

The output is:

```text theme={null}
[class 0 score,
 class 1 score,
 class 2 score]
```

Use:

```python theme={null}
loss_function = nn.CrossEntropyLoss()
```

***

# 60. Classification Training

```python theme={null}
model = nn.Sequential(
    nn.Linear(4, 16),
    nn.ReLU(),
    nn.Linear(16, 3)
)

loss_function = nn.CrossEntropyLoss()

optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)
```

Training:

```python theme={null}
for epoch in range(10):

    for X_batch, y_batch in dataloader:

        prediction = model(X_batch)

        loss = loss_function(
            prediction,
            y_batch
        )

        optimizer.zero_grad()

        loss.backward()

        optimizer.step()
```

***

# 61. Important PyTorch Concepts

| Concept         | Meaning                            |
| --------------- | ---------------------------------- |
| Tensor          | Data structure used by PyTorch     |
| Dataset         | Represents dataset samples         |
| DataLoader      | Loads data in batches              |
| Transform       | Preprocesses/augments data         |
| `nn.Module`     | Base class for neural networks     |
| Layer           | Performs computation               |
| Activation      | Adds non-linearity                 |
| Loss            | Measures prediction error          |
| Autograd        | Calculates gradients automatically |
| Backpropagation | Computes gradients from loss       |
| Optimizer       | Updates model parameters           |
| Epoch           | One complete pass through dataset  |
| Batch           | Group of samples                   |
| Learning rate   | Size of parameter update           |

***

# 62. Epoch vs Batch vs Iteration

Suppose:

```text theme={null}
Dataset = 1,000 samples
Batch size = 100
```

Then:

```text theme={null}
1 epoch = 1,000 samples
```

Number of batches:

```text theme={null}
1000 / 100 = 10
```

Therefore:

```text theme={null}
1 epoch = 10 iterations
```

***

# 63. Learning Rate

Learning rate controls how much the model changes its weights.

Example:

```python theme={null}
optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)
```

Small learning rate:

```text theme={null}
0.0001
```

May train slowly.

Large learning rate:

```text theme={null}
1.0
```

May overshoot the optimal solution.

Typical values depend on the optimizer and model.

***

# 64. Model Parameters

A neural network contains learnable parameters.

Example:

```python theme={null}
model = nn.Linear(2, 1)
```

It has:

```text theme={null}
2 weights
1 bias
```

Inspect them:

```python theme={null}
for name, parameter in model.named_parameters():

    print(name)
    print(parameter)
```

***

# 65. Saving a Model

Save model parameters:

```python theme={null}
torch.save(
    model.state_dict(),
    "model.pth"
)
```

Load them:

```python theme={null}
model.load_state_dict(
    torch.load("model.pth")
)
```

`state_dict()` contains the model's learned parameters.

***

# 66. Complete Conceptual Picture

The complete process can be visualized as:

```text theme={null}
                 TRAINING DATA
                       |
                       v
                  DataLoader
                       |
                       v
                    Tensor
                       |
                       v
               +---------------+
               | Neural Network|
               +---------------+
                       |
                       v
                  Prediction
                       |
                       v
                Loss Function
                       |
                       v
                     Loss
                       |
                       v
              loss.backward()
                       |
                       v
                  Gradients
                       |
                       v
                  Optimizer
                       |
                       v
              Updated Weights
                       |
                       +----------+
                                  |
                                  v
                              Next Batch
```

***

# 67. Most Important Code to Remember

If you're learning PyTorch for interviews or practical ML, remember this pattern:

```python theme={null}
# Model
model = nn.Linear(
    input_features,
    output_features
)

# Loss
loss_function = nn.MSELoss()

# Optimizer
optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.001
)

# Training
for X_batch, y_batch in dataloader:

    # Forward pass
    prediction = model(X_batch)

    # Loss
    loss = loss_function(
        prediction,
        y_batch
    )

    # Reset gradients
    optimizer.zero_grad()

    # Backpropagation
    loss.backward()

    # Update parameters
    optimizer.step()
```

The four lines you should especially remember are:

```python theme={null}
prediction = model(X)
loss = loss_function(prediction, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
```

***

# 68. PyTorch Cheat Sheet

## Import

```python theme={null}
import torch
import torch.nn as nn
import torch.optim as optim
```

## Tensor

```python theme={null}
x = torch.tensor([1, 2, 3])
```

## Random Tensor

```python theme={null}
x = torch.rand(2, 3)
```

## Shape

```python theme={null}
x.shape
```

## Reshape

```python theme={null}
x.reshape(2, 3)
```

## Device

```python theme={null}
device = torch.device(
    "cuda" if torch.cuda.is_available()
    else "cpu"
)
```

## Dataset

```python theme={null}
from torch.utils.data import Dataset
```

## DataLoader

```python theme={null}
from torch.utils.data import DataLoader
```

## Neural Network

```python theme={null}
class Model(nn.Module):

    def __init__(self):
        super().__init__()

    def forward(self, x):
        return x
```

## Linear Layer

```python theme={null}
nn.Linear(10, 5)
```

## ReLU

```python theme={null}
nn.ReLU()
```

## Loss

```python theme={null}
nn.MSELoss()
```

## Classification Loss

```python theme={null}
nn.CrossEntropyLoss()
```

## Optimizer

```python theme={null}
optim.Adam(
    model.parameters(),
    lr=0.001
)
```

## Gradient

```python theme={null}
loss.backward()
```

## Update

```python theme={null}
optimizer.step()
```

## Clear Gradient

```python theme={null}
optimizer.zero_grad()
```

## Training Mode

```python theme={null}
model.train()
```

## Evaluation Mode

```python theme={null}
model.eval()
```

## Disable Gradients

```python theme={null}
with torch.no_grad():
    ...
```

***

# 69. The Big Picture

The concepts you asked about are connected:

```text theme={null}
                    PyTorch
                       |
       +---------------+----------------+
       |               |                |
    Tensors         Dataset          Neural Network
       |               |                |
       |           DataLoader           |
       |               |                |
       +---------------+----------------+
                       |
                       v
                 Forward Pass
                       |
                       v
                  Prediction
                       |
                       v
                 Loss Function
                       |
                       v
                   Autograd
                       |
                       v
                Backpropagation
                       |
                       v
                   Gradients
                       |
                       v
                  Optimizer
                       |
                       v
               Updated Weights
                       |
                       v
                  Better Model
```