This note covers the core PyTorch concepts you need for machine learning and deep learning:

What is PyTorch?
Installation
PyTorch workflow
Tensors
Tensor operations
Dataset and DataLoader
Transforms
Neural networks
Optimizers
Autograd
Backpropagation
Loss functions
Complete training loop
Complete example

1. What is PyTorch?

PyTorch is an open-source deep learning framework developed by Meta AI. It is mainly used for:

Machine Learning
Deep Learning
Neural Networks
Computer Vision
Natural Language Processing
Generative AI
Reinforcement Learning

PyTorch provides:

Tensor computation
GPU acceleration
Automatic differentiation
Neural network building blocks
Optimizers
Dataset and DataLoader utilities

Simple idea

Data
 ↓
Tensor
 ↓
Neural Network
 ↓
Prediction
 ↓
Loss
 ↓
Backpropagation
 ↓
Optimizer
 ↓
Updated Weights

2. Install PyTorch

For a basic installation:

pip install torch torchvision torchaudio

Check the installation:

import torch

print(torch.__version__)

Example output:

2.x.x

3. Import PyTorch

import torch

You will commonly use:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader

4. PyTorch Workflow

A typical PyTorch deep-learning project follows this workflow:

1. Load data
      ↓
2. Convert data to tensors
      ↓
3. Create Dataset
      ↓
4. Create DataLoader
      ↓
5. Define neural network
      ↓
6. Define loss function
      ↓
7. Define optimizer
      ↓
8. Forward pass
      ↓
9. Calculate loss
      ↓
10. Backpropagation
      ↓
11. Update weights
      ↓
12. Repeat for multiple epochs
      ↓
13. Evaluate model

5. PyTorch Tensors

A tensor is the fundamental data structure in PyTorch. You can think of a tensor as a generalization of:

Scalar → 0D tensor
Vector → 1D tensor
Matrix → 2D tensor
Higher-dimensional array → 3D, 4D, etc.

5.1 Scalar

A scalar contains one value.

import torch

x = torch.tensor(10)

print(x)
print(x.ndim)

Output:

tensor(10)
0

5.2 Vector

x = torch.tensor([10, 20, 30])

print(x)
print(x.ndim)
print(x.shape)

Output:

tensor([10, 20, 30])
1
torch.Size([3])

5.3 Matrix

x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x)
print(x.ndim)
print(x.shape)

Output:

tensor([[1, 2, 3],
        [4, 5, 6]])

2
torch.Size([2, 3])

6. Creating Tensors

`torch.tensor()`

x = torch.tensor([1, 2, 3])

print(x)

Output:

tensor([1, 2, 3])

`torch.zeros()`

Creates a tensor filled with zeros.

x = torch.zeros(2, 3)

print(x)

Output:

tensor([[0., 0., 0.],
        [0., 0., 0.]])

`torch.ones()`

x = torch.ones(2, 3)

print(x)

Output:

tensor([[1., 1., 1.],
        [1., 1., 1.]])

`torch.full()`

x = torch.full((2, 3), 5)

print(x)

Output:

tensor([[5, 5, 5],
        [5, 5, 5]])

`torch.arange()`

x = torch.arange(0, 10)

print(x)

Output:

tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

`torch.linspace()`

Creates evenly spaced values.

x = torch.linspace(0, 1, 5)

print(x)

Output:

tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])

Random tensor

x = torch.rand(2, 3)

print(x)

Example:

tensor([[0.32, 0.71, 0.12],
        [0.83, 0.45, 0.91]])

Values are between 0 and 1.

7. Tensor Data Types

x = torch.tensor([1, 2, 3])

print(x.dtype)

Output:

torch.int64

Float tensor:

x = torch.tensor([1.0, 2.0, 3.0])

print(x.dtype)

Output:

torch.float32

Convert datatype:

x = torch.tensor([1, 2, 3])

x = x.float()

print(x.dtype)

Output:

torch.float32

8. Tensor Shape

x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x.shape)

Output:

torch.Size([2, 3])

This means:

2 rows
3 columns

9. Tensor Indexing

x = torch.tensor([10, 20, 30, 40])

print(x[0])
print(x[2])

Output:

tensor(10)
tensor(30)

For a matrix:

x = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print(x[0])
print(x[0, 1])

Output:

tensor([1, 2, 3])
tensor(2)

10. Tensor Slicing

x = torch.tensor([10, 20, 30, 40, 50])

print(x[1:4])

Output:

tensor([20, 30, 40])

11. Tensor Arithmetic

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

print(a + b)
print(a - b)
print(a * b)
print(a / b)

Output:

tensor([5, 7, 9])
tensor([-3, -3, -3])
tensor([ 4, 10, 18])
tensor([0.2500, 0.4000, 0.5000])

12. Matrix Multiplication

Matrix multiplication is extremely important in neural networks.

a = torch.tensor([
    [1, 2],
    [3, 4]
])

b = torch.tensor([
    [5, 6],
    [7, 8]
])

result = torch.matmul(a, b)

print(result)

Output:

tensor([[19, 22],
        [43, 50]])

You can also use:

result = a @ b

13. Reshaping Tensors

Use .reshape() to change the tensor shape.

x = torch.tensor([1, 2, 3, 4, 5, 6])

x = x.reshape(2, 3)

print(x)

Output:

tensor([
    [1, 2, 3],
    [4, 5, 6]
])

14. Flatten

Flatten converts multiple dimensions into one dimension.

x = torch.tensor([
    [1, 2],
    [3, 4]
])

print(x.flatten())

Output:

tensor([1, 2, 3, 4])

This is commonly used before fully connected layers.

15. Tensor Device — CPU and GPU

PyTorch can execute tensor operations on:

Check GPU availability:

import torch

print(torch.cuda.is_available())

Output might be:

True

Create device:

device = torch.device(
    "cuda" if torch.cuda.is_available() else "cpu"
)

print(device)

Move tensor to device:

x = torch.tensor([1, 2, 3])

x = x.to(device)

Move model:

model = model.to(device)

During training, both model and data must be on compatible devices.

16. Dataset

A Dataset represents your data. PyTorch provides:

torch.utils.data.Dataset

A custom dataset normally implements:

__init__()
__len__()
__getitem__()

17. Creating a Custom Dataset

Suppose we have:

Hours studied → Exam score

→ 20
→ 35
→ 50
→ 65
→ 80

Create the dataset:

import torch
from torch.utils.data import Dataset

class StudentDataset(Dataset):

    def __init__(self):
        self.x = torch.tensor(
            [[1.0], [2.0], [3.0], [4.0], [5.0]]
        )

        self.y = torch.tensor(
            [[20.0], [35.0], [50.0], [65.0], [80.0]]
        )

    def __len__(self):
        return len(self.x)

    def __getitem__(self, index):
        return self.x[index], self.y[index]

Create dataset:

dataset = StudentDataset()

print(len(dataset))

Output:

Get one sample:

x, y = dataset[0]

print(x)
print(y)

Output:

tensor([1.])
tensor([20.])

18. Why Dataset?

Dataset provides a standard way to:

Store data
Access individual samples
Separate data from model logic
Work with DataLoader

19. DataLoader

A DataLoader loads data in batches.

from torch.utils.data import DataLoader

loader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)

Loop through batches:

for x, y in loader:
    print(x)
    print(y)

Instead of processing:

5 samples

all at once, DataLoader can process:

Batch 1 → 2 samples
Batch 2 → 2 samples
Batch 3 → 1 sample

20. Important DataLoader Parameters

DataLoader(
    dataset,
    batch_size=32,
    shuffle=True
)

`batch_size`

Number of samples processed at once.

batch_size=32

means:

32 samples → model

`shuffle`

Randomizes the training data.

shuffle=True

This helps prevent the model from learning based on the original ordering of training data.

21. Dataset vs DataLoader

Component	Purpose
Dataset	Stores and retrieves samples
DataLoader	Creates batches and iterates over dataset

Simple way to remember:

Dataset = Data

DataLoader = How data is delivered

22. Transforms

Transforms are used to preprocess or modify data. Commonly used with:

Images
Computer vision
Data augmentation

Import:

from torchvision import transforms

23. `ToTensor()`

Converts image data into a PyTorch tensor.

transform = transforms.ToTensor()

Example:

from torchvision import transforms

transform = transforms.ToTensor()

24. Normalize

Normalization changes the scale of input data.

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.5,),
        std=(0.5,)
    )
])

For RGB images:

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.5, 0.5, 0.5),
        std=(0.5, 0.5, 0.5)
    )
])

25. Resize

Resize images to a fixed size.

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor()
])

This converts images to:

224 × 224

26. Data Augmentation

Data augmentation creates variations of training images. Example:

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ToTensor()
])

This can help reduce overfitting.

27. `Compose`

Compose combines multiple transformations.

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(
        (0.5, 0.5, 0.5),
        (0.5, 0.5, 0.5)
    )
])

Execution:

Image
 ↓
Resize
 ↓
ToTensor
 ↓
Normalize
 ↓
Model

28. Neural Network in PyTorch

PyTorch provides:

torch.nn

Import:

import torch.nn as nn

A neural network usually inherits from:

nn.Module

29. Simple Neural Network

import torch
import torch.nn as nn

class SimpleModel(nn.Module):

    def __init__(self):
        super().__init__()

        self.layer = nn.Linear(1, 1)

    def forward(self, x):
        return self.layer(x)

Create model:

model = SimpleModel()

print(model)

Output:

SimpleModel(
  (layer): Linear(in_features=1, out_features=1, bias=True)
)

30. Understanding `nn.Linear`

nn.Linear(3, 2)

means:

3 input features
        ↓
Linear Layer
        ↓
2 output features

Mathematically:

y = xW + b

where:

x = input
W = weights
b = bias
y = output

31. Multiple Layers

class NeuralNetwork(nn.Module):

    def __init__(self):
        super().__init__()

        self.network = nn.Sequential(
            nn.Linear(4, 16),
            nn.ReLU(),
            nn.Linear(16, 8),
            nn.ReLU(),
            nn.Linear(8, 1)
        )

    def forward(self, x):
        return self.network(x)

Architecture:

4 input features
       ↓
Linear(4 → 16)
       ↓
ReLU
       ↓
Linear(16 → 8)
       ↓
ReLU
       ↓
Linear(8 → 1)
       ↓
Prediction

32. Activation Function

Activation functions introduce non-linearity. Common activations:

nn.ReLU()
nn.Sigmoid()
nn.Tanh()
nn.Softmax()

Example:

x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])

relu = nn.ReLU()

print(relu(x))

Output:

tensor([0., 0., 0., 1., 2.])

33. Loss Function

Loss measures how wrong the model’s prediction is. Concept:

Prediction
    ↓
Compare with actual value
    ↓
Loss

Common loss functions:

Task	Loss
Regression	MSELoss
Binary classification	BCELoss / BCEWithLogitsLoss
Multi-class classification	CrossEntropyLoss

34. MSE Loss

Mean Squared Error is commonly used for regression.

loss_function = nn.MSELoss()

Example:

prediction = torch.tensor([10.0])
actual = torch.tensor([12.0])

loss = loss_function(prediction, actual)

print(loss)

Output:

tensor(4.)

Because:

(10 - 12)² = 4

35. Optimizers

An optimizer updates the model’s parameters to reduce the loss. Common optimizers:

SGD
Adam
RMSprop
AdamW

36. SGD Optimizer

SGD = Stochastic Gradient Descent.

optimizer = torch.optim.SGD(
    model.parameters(),
    lr=0.01
)

Here:

lr = learning rate

The optimizer uses gradients to update weights. Conceptually:

weight = weight - learning_rate × gradient

37. Adam Optimizer

Adam is one of the most commonly used optimizers.

optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.001
)

Adam adapts the update for each parameter using information from past gradients. A common starting point is:

lr=0.001

38. Optimizer Comparison

Optimizer	Description
SGD	Simple gradient descent
Adam	Adaptive and commonly used
RMSprop	Adaptive learning rate
AdamW	Adam with improved weight decay handling

39. What is Autograd?

Autograd is PyTorch’s automatic differentiation system. It automatically calculates gradients. Example:

x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 2

y.backward()

print(x.grad)

Output:

tensor(4.)

Why? We have:

y = x²

Derivative:

dy/dx = 2x

At:

x = 2

therefore:

dy/dx = 4

PyTorch calculates this automatically.

40. `requires_grad=True`

When you write:

x = torch.tensor(
    2.0,
    requires_grad=True
)

you tell PyTorch:

Track operations involving this tensor because I may need its gradient.

Check:

print(x.requires_grad)

Output:

True

41. Gradient

A gradient tells us how much a value changes when a parameter changes. Example:

x = torch.tensor(
    3.0,
    requires_grad=True
)

y = x ** 2

y.backward()

print(x.grad)

Output:

tensor(6.)

Because:

y = x²

dy/dx = 2x

x = 3

gradient = 6

42. `.backward()`

The .backward() function calculates gradients. Example:

x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 3

y.backward()

print(x.grad)

Mathematically:

y = x³

dy/dx = 3x²

x = 2

gradient = 12

Output:

tensor(12.)

43. Computational Graph

PyTorch creates a computational graph when operations are performed on tensors that require gradients. Example:

x = torch.tensor(
    2.0,
    requires_grad=True
)

y = x ** 2

z = y + 3

Graph:

x
↓
x²
↓
y
↓
+ 3
↓
z

When you call:

z.backward()

PyTorch calculates gradients through this graph.

44. Backpropagation

Backpropagation is the process of calculating how much each model parameter contributed to the error. Basic flow:

Input
  ↓
Forward Pass
  ↓
Prediction
  ↓
Loss
  ↓
Backward Pass
  ↓
Gradients
  ↓
Optimizer
  ↓
Updated Weights

45. Forward Pass

Suppose:

Input = 2
Weight = 3
Bias = 1

The neuron calculates:

y = wx + b

Therefore:

y = 3(2) + 1
y = 7

In PyTorch:

x = torch.tensor([[2.0]])

model = nn.Linear(1, 1)

prediction = model(x)

print(prediction)

46. Complete Autograd Example

import torch

x = torch.tensor(
    2.0,
    requires_grad=True
)

w = torch.tensor(
    3.0,
    requires_grad=True
)

b = torch.tensor(
    1.0,
    requires_grad=True
)

y = w * x + b

target = torch.tensor(10.0)

loss = (y - target) ** 2

loss.backward()

print("Prediction:", y.item())
print("Loss:", loss.item())

print("dw:", w.grad.item())
print("db:", b.grad.item())

Let’s understand it.

Forward calculation

y = wx + b

y = 3 × 2 + 1

y = 7

Target:

Loss:

(7 - 10)²
= 9

Then:

loss.backward()

calculates:

∂Loss/∂w
∂Loss/∂b

47. Gradient Descent Manually

Suppose:

weight = 3
gradient = -12
learning_rate = 0.01

Update:

new_weight
=
weight - learning_rate × gradient

Therefore:

new_weight
=
3 - (0.01 × -12)

= 3.12

PyTorch’s optimizer performs this update automatically.

48. The Three Important Optimizer Steps

During training, you commonly see:

optimizer.zero_grad()

then:

loss.backward()

then:

optimizer.step()

These are extremely important.

Step 1 — `zero_grad()`

optimizer.zero_grad()

Clears previously stored gradients. PyTorch accumulates gradients by default.

Step 2 — `backward()`

loss.backward()

Calculates gradients.

Step 3 — `step()`

optimizer.step()

Updates model parameters using those gradients.

49. Complete Training Loop

for epoch in range(epochs):

    for x, y in dataloader:

        # Forward pass
        prediction = model(x)

        # Calculate loss
        loss = loss_function(
            prediction,
            y
        )

        # Clear old gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()

This is the core PyTorch training pattern.

50. Complete Regression Example

Let’s build a simple model that learns:

y = 2x + 1

Dataset:

x → y

→ 3
→ 5
→ 7
→ 9
→ 11

Step 1 — Import libraries

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader

Step 2 — Create data

X = torch.tensor([
    [1.0],
    [2.0],
    [3.0],
    [4.0],
    [5.0]
])

y = torch.tensor([
    [3.0],
    [5.0],
    [7.0],
    [9.0],
    [11.0]
])

Step 3 — Create Dataset

dataset = TensorDataset(X, y)

TensorDataset is convenient when your data is already stored as tensors.

Step 4 — Create DataLoader

dataloader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)

Step 5 — Create Model

model = nn.Linear(1, 1)

The model contains:

1 input
↓
Linear layer
↓
1 output

Step 6 — Loss Function

loss_function = nn.MSELoss()

Step 7 — Optimizer

optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)

Step 8 — Training

epochs = 1000

for epoch in range(epochs):

    for X_batch, y_batch in dataloader:

        # Forward pass
        prediction = model(X_batch)

        # Calculate loss
        loss = loss_function(
            prediction,
            y_batch
        )

        # Clear gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()

    if (epoch + 1) % 100 == 0:
        print(
            f"Epoch {epoch + 1}, "
            f"Loss: {loss.item():.4f}"
        )

51. Make a Prediction

After training:

test = torch.tensor([[6.0]])

prediction = model(test)

print(prediction)

Expected result:

tensor([[~13.]])

Because the model learned approximately:

y = 2x + 1

For:

x = 6

we expect:

y = 13

52. Complete PyTorch Example

Here is the same example in one place:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader


# -------------------------
# 1. Create data
# -------------------------

X = torch.tensor([
    [1.0],
    [2.0],
    [3.0],
    [4.0],
    [5.0]
])

y = torch.tensor([
    [3.0],
    [5.0],
    [7.0],
    [9.0],
    [11.0]
])


# -------------------------
# 2. Dataset
# -------------------------

dataset = TensorDataset(X, y)


# -------------------------
# 3. DataLoader
# -------------------------

dataloader = DataLoader(
    dataset,
    batch_size=2,
    shuffle=True
)


# -------------------------
# 4. Model
# -------------------------

model = nn.Linear(1, 1)


# -------------------------
# 5. Loss
# -------------------------

loss_function = nn.MSELoss()


# -------------------------
# 6. Optimizer
# -------------------------

optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)


# -------------------------
# 7. Training
# -------------------------

epochs = 1000

for epoch in range(epochs):

    for X_batch, y_batch in dataloader:

        # Forward pass
        prediction = model(X_batch)

        # Calculate loss
        loss = loss_function(
            prediction,
            y_batch
        )

        # Clear gradients
        optimizer.zero_grad()

        # Backpropagation
        loss.backward()

        # Update weights
        optimizer.step()


    if (epoch + 1) % 100 == 0:

        print(
            f"Epoch: {epoch + 1}, "
            f"Loss: {loss.item():.4f}"
        )


# -------------------------
# 8. Prediction
# -------------------------

test = torch.tensor([[6.0]])

prediction = model(test)

print("Prediction:", prediction.item())

Expected:

Prediction: approximately 13

53. Training Loop Explained

The most important part is:

prediction = model(X_batch)

This is the forward pass. Then:

loss = loss_function(
    prediction,
    y_batch
)

This calculates the error. Then:

optimizer.zero_grad()

Clears previous gradients. Then:

loss.backward()

Calculates gradients using backpropagation. Finally:

optimizer.step()

Updates weights. So remember:

Forward
   ↓
Loss
   ↓
zero_grad
   ↓
Backward
   ↓
Optimizer step

54. Why Do We Need `zero_grad()`?

Consider:

loss.backward()

PyTorch accumulates gradients. For example:

Step 1:
gradient = 2

Step 2:
gradient = 3

Without zero_grad:
gradient = 5

Usually we want each training iteration to use its own gradients. Therefore:

optimizer.zero_grad()

is used before:

loss.backward()

55. `model.train()`

During training:

model.train()

Example:

model.train()

for X_batch, y_batch in dataloader:
    ...

This puts the model into training mode. It matters especially for layers such as:

Dropout
Batch Normalization

56. `model.eval()`

During evaluation:

model.eval()

Example:

model.eval()

with torch.no_grad():
    prediction = model(X_test)

57. `torch.no_grad()`

During prediction, gradients usually aren’t required. Use:

with torch.no_grad():

    prediction = model(X_test)

Advantages:

Less memory usage
Faster inference
No unnecessary gradient calculations

58. Training vs Evaluation

Training

model.train()

optimizer.zero_grad()

prediction = model(X)

loss = loss_function(
    prediction,
    y
)

loss.backward()

optimizer.step()

Evaluation

model.eval()

with torch.no_grad():

    prediction = model(X_test)

59. Classification Example

For a multi-class classification problem:

model = nn.Sequential(
    nn.Linear(4, 16),
    nn.ReLU(),
    nn.Linear(16, 3)
)

Suppose:

4 input features
3 classes

The output is:

[class 0 score,
 class 1 score,
 class 2 score]

Use:

loss_function = nn.CrossEntropyLoss()

60. Classification Training

model = nn.Sequential(
    nn.Linear(4, 16),
    nn.ReLU(),
    nn.Linear(16, 3)
)

loss_function = nn.CrossEntropyLoss()

optimizer = optim.Adam(
    model.parameters(),
    lr=0.001
)

Training:

for epoch in range(10):

    for X_batch, y_batch in dataloader:

        prediction = model(X_batch)

        loss = loss_function(
            prediction,
            y_batch
        )

        optimizer.zero_grad()

        loss.backward()

        optimizer.step()

61. Important PyTorch Concepts

Concept	Meaning
Tensor	Data structure used by PyTorch
Dataset	Represents dataset samples
DataLoader	Loads data in batches
Transform	Preprocesses/augments data
`nn.Module`	Base class for neural networks
Layer	Performs computation
Activation	Adds non-linearity
Loss	Measures prediction error
Autograd	Calculates gradients automatically
Backpropagation	Computes gradients from loss
Optimizer	Updates model parameters
Epoch	One complete pass through dataset
Batch	Group of samples
Learning rate	Size of parameter update

62. Epoch vs Batch vs Iteration

Suppose:

Dataset = 1,000 samples
Batch size = 100

Then:

1 epoch = 1,000 samples

Number of batches:

1000 / 100 = 10

Therefore:

1 epoch = 10 iterations

63. Learning Rate

Learning rate controls how much the model changes its weights. Example:

optimizer = optim.SGD(
    model.parameters(),
    lr=0.01
)

Small learning rate:

0.0001

May train slowly. Large learning rate:

1.0

May overshoot the optimal solution. Typical values depend on the optimizer and model.

64. Model Parameters

A neural network contains learnable parameters. Example:

model = nn.Linear(2, 1)

It has:

2 weights
1 bias

Inspect them:

for name, parameter in model.named_parameters():

    print(name)
    print(parameter)

65. Saving a Model

Save model parameters:

torch.save(
    model.state_dict(),
    "model.pth"
)

Load them:

model.load_state_dict(
    torch.load("model.pth")
)

state_dict() contains the model’s learned parameters.

66. Complete Conceptual Picture

The complete process can be visualized as:

                 TRAINING DATA
                       |
                       v
                  DataLoader
                       |
                       v
                    Tensor
                       |
                       v
               +---------------+
               | Neural Network|
               +---------------+
                       |
                       v
                  Prediction
                       |
                       v
                Loss Function
                       |
                       v
                     Loss
                       |
                       v
              loss.backward()
                       |
                       v
                  Gradients
                       |
                       v
                  Optimizer
                       |
                       v
              Updated Weights
                       |
                       +----------+
                                  |
                                  v
                              Next Batch

67. Most Important Code to Remember

If you’re learning PyTorch for interviews or practical ML, remember this pattern:

# Model
model = nn.Linear(
    input_features,
    output_features
)

# Loss
loss_function = nn.MSELoss()

# Optimizer
optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.001
)

# Training
for X_batch, y_batch in dataloader:

    # Forward pass
    prediction = model(X_batch)

    # Loss
    loss = loss_function(
        prediction,
        y_batch
    )

    # Reset gradients
    optimizer.zero_grad()

    # Backpropagation
    loss.backward()

    # Update parameters
    optimizer.step()

The four lines you should especially remember are:

prediction = model(X)
loss = loss_function(prediction, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()

68. PyTorch Cheat Sheet

Import

import torch
import torch.nn as nn
import torch.optim as optim

Tensor

x = torch.tensor([1, 2, 3])

Random Tensor

x = torch.rand(2, 3)

Shape

x.shape

Reshape

x.reshape(2, 3)

Device

device = torch.device(
    "cuda" if torch.cuda.is_available()
    else "cpu"
)

Dataset

from torch.utils.data import Dataset

DataLoader

from torch.utils.data import DataLoader

Neural Network

class Model(nn.Module):

    def __init__(self):
        super().__init__()

    def forward(self, x):
        return x

Linear Layer

nn.Linear(10, 5)

ReLU

nn.ReLU()

Loss

nn.MSELoss()

Classification Loss

nn.CrossEntropyLoss()

Optimizer

optim.Adam(
    model.parameters(),
    lr=0.001
)

Gradient

loss.backward()

Update

optimizer.step()

Clear Gradient

optimizer.zero_grad()

Training Mode

model.train()

Evaluation Mode

model.eval()

Disable Gradients

with torch.no_grad():
    ...

69. The Big Picture

The concepts you asked about are connected:

                    PyTorch
                       |
       +---------------+----------------+
       |               |                |
    Tensors         Dataset          Neural Network
       |               |                |
       |           DataLoader           |
       |               |                |
       +---------------+----------------+
                       |
                       v
                 Forward Pass
                       |
                       v
                  Prediction
                       |
                       v
                 Loss Function
                       |
                       v
                   Autograd
                       |
                       v
                Backpropagation
                       |
                       v
                   Gradients
                       |
                       v
                  Optimizer
                       |
                       v
               Updated Weights
                       |
                       v
                  Better Model

​1. What is PyTorch?

​Simple idea

​2. Install PyTorch

​3. Import PyTorch

​4. PyTorch Workflow

​5. PyTorch Tensors

​5.1 Scalar

​5.2 Vector

​5.3 Matrix

​6. Creating Tensors

​torch.tensor()

​torch.zeros()

​torch.ones()

​torch.full()

​torch.arange()

​torch.linspace()

​Random tensor

​7. Tensor Data Types

​8. Tensor Shape

​9. Tensor Indexing

​10. Tensor Slicing

​11. Tensor Arithmetic

​12. Matrix Multiplication

​13. Reshaping Tensors

​14. Flatten

​15. Tensor Device — CPU and GPU

​16. Dataset

​17. Creating a Custom Dataset

​18. Why Dataset?

​19. DataLoader

​20. Important DataLoader Parameters

​batch_size

​shuffle

​21. Dataset vs DataLoader

​22. Transforms

​23. ToTensor()

​24. Normalize

​25. Resize

​26. Data Augmentation

​27. Compose

​28. Neural Network in PyTorch

​29. Simple Neural Network

​30. Understanding nn.Linear

​31. Multiple Layers

​32. Activation Function

​33. Loss Function

​34. MSE Loss

​35. Optimizers

​36. SGD Optimizer

​37. Adam Optimizer

​38. Optimizer Comparison

​39. What is Autograd?

​40. requires_grad=True

​41. Gradient

​42. .backward()

​43. Computational Graph

​44. Backpropagation

​45. Forward Pass

​46. Complete Autograd Example

​Forward calculation

​47. Gradient Descent Manually

​48. The Three Important Optimizer Steps

​Step 1 — zero_grad()

​Step 2 — backward()

​Step 3 — step()

​49. Complete Training Loop

​50. Complete Regression Example

​Step 1 — Import libraries

​Step 2 — Create data

​Step 3 — Create Dataset

​Step 4 — Create DataLoader

​Step 5 — Create Model

​Step 6 — Loss Function

​Step 7 — Optimizer

​Step 8 — Training

​51. Make a Prediction

​52. Complete PyTorch Example

​53. Training Loop Explained

​54. Why Do We Need zero_grad()?

​55. model.train()

1. What is PyTorch?

Simple idea

2. Install PyTorch

3. Import PyTorch

4. PyTorch Workflow

5. PyTorch Tensors

5.1 Scalar

5.2 Vector

5.3 Matrix

6. Creating Tensors

`torch.tensor()`

`torch.zeros()`

`torch.ones()`

`torch.full()`

`torch.arange()`

`torch.linspace()`

Random tensor

7. Tensor Data Types

8. Tensor Shape

9. Tensor Indexing

10. Tensor Slicing

11. Tensor Arithmetic

12. Matrix Multiplication

13. Reshaping Tensors

14. Flatten

15. Tensor Device — CPU and GPU

16. Dataset

17. Creating a Custom Dataset

18. Why Dataset?

19. DataLoader

20. Important DataLoader Parameters

`batch_size`

`shuffle`

21. Dataset vs DataLoader

22. Transforms

23. `ToTensor()`

24. Normalize

25. Resize

26. Data Augmentation

27. `Compose`

28. Neural Network in PyTorch

29. Simple Neural Network

30. Understanding `nn.Linear`

31. Multiple Layers

32. Activation Function

33. Loss Function

34. MSE Loss

35. Optimizers

36. SGD Optimizer

37. Adam Optimizer

38. Optimizer Comparison

39. What is Autograd?

40. `requires_grad=True`

41. Gradient

42. `.backward()`

43. Computational Graph

44. Backpropagation

45. Forward Pass

46. Complete Autograd Example

Forward calculation

47. Gradient Descent Manually

48. The Three Important Optimizer Steps

Step 1 — `zero_grad()`

Step 2 — `backward()`

Step 3 — `step()`

49. Complete Training Loop

50. Complete Regression Example

Step 1 — Import libraries

Step 2 — Create data

Step 3 — Create Dataset

Step 4 — Create DataLoader

Step 5 — Create Model

Step 6 — Loss Function

Step 7 — Optimizer

Step 8 — Training

51. Make a Prediction

52. Complete PyTorch Example

53. Training Loop Explained

54. Why Do We Need `zero_grad()`?

55. `model.train()`