Skip to main content
Machine Learning (ML) is a branch of Artificial Intelligence where systems learn patterns from data and improve predictions or decisions without being explicitly programmed for every rule.

​
What is Machine Learning?

Traditional programming: 
Input + Rules → Output
Machine Learning: 
Input + Output Data → Model learns Rules
Example: Spam Email Detection Traditional approach: 
if email contains "lottery":
    spam=True
Machine Learning approach: 
Provide thousands of spam and non-spam emails

Model learns patterns automatically

​
Why Machine Learning?

Machine learning is useful when:
  • Rules are too complex
  • Huge amounts of data exist
  • Patterns continuously change
  • Manual programming becomes difficult
Examples:
  • Netflix recommendations
  • Self-driving cars
  • Fraud detection
  • Face recognition
  • ChatGPT
  • YouTube recommendations

​
Machine Learning Pipeline

Typical workflow: 
Collect Data
      ↓
Clean Data
      ↓
Feature Engineering
      ↓
Split Data
      ↓
Train Model
      ↓
Evaluate Model
      ↓
Tune Model
      ↓
Deploy Model

​
Types of Machine Learning

Machine Learning mainly has three types: 
1. Supervised Learning

2. Unsupervised Learning

3. Reinforcement Learning

​
1. Supervised Learning

​
What is Supervised Learning?

The model learns from labeled data. Labeled data: 
Input → Output
Example: 
House Area → House Price

1000 sqft → ₹20 lakh
1200 sqft → ₹25 lakh
1500 sqft → ₹30 lakh
The model learns: 
Area → Price relationship
After learning: 
1700 sqft → ?
Model predicts answer.

​
Types of Supervised Learning

Two major types: 
1. Regression

2. Classification

​
Regression

Predicts continuous numerical values. Examples:
  • House price prediction
  • Stock price prediction
  • Temperature prediction
  • Sales forecasting
Output: 
₹25,43,000

72.5°C

50.8
Algorithms:
  • Linear Regression
  • Random Forest Regressor
  • XGBoost
  • Decision Tree Regressor

​
Classification

Predicts categories/classes. Examples:
  • Spam or Not Spam
  • Disease Detection
  • Fraud Detection
  • Cat vs Dog
Output: 
Spam

Fraud

Cat
Algorithms:
  • Logistic Regression
  • Decision Trees
  • Random Forest
  • SVM
  • Neural Networks

​
Supervised Learning Flow

Input Data

↓

Learn patterns

↓

Predict outputs
Example: Student marks prediction:
Hours StudiedMarks
240
460
675
Predict: 
8 hours → ?

​
Advantages

  • Easy to evaluate
  • High accuracy with enough labeled data
  • Clear target output

​
Disadvantages

  • Requires large labeled datasets
  • Labeling data is expensive
  • Can overfit

​
2. Unsupervised Learning

​
What is Unsupervised Learning?

Learns from unlabeled data. No correct outputs exist. Input: 
Only data
Example: Customer purchase data: 
Age
Salary
Products Bought
Model discovers hidden patterns.

​
Types of Unsupervised Learning

Main categories: 
1. Clustering

2. Dimensionality Reduction

3. Association Rule Learning

​
Clustering

Groups similar data points. Example: Customer segmentation: 
Group 1 → Students

Group 2 → Working professionals

Group 3 → Senior citizens
Algorithms:
  • K-Means
  • DBSCAN
  • Hierarchical Clustering
Applications:
  • Customer segmentation
  • Fraud detection
  • Image segmentation

​
Dimensionality Reduction

Reduces features while preserving information. Example: Dataset: 
Height
Weight
Age
BMI
Some columns may be highly related. Reduce: 
4 features → 2 features
Algorithms:
  • PCA
  • t-SNE
Applications:
  • Faster training
  • Visualization
  • Noise reduction

​
Association Rule Learning

Finds relationships among items. Example: 
People buying bread also buy milk
Applications:
  • Product recommendation
  • Market basket analysis
Algorithms:
  • Apriori
  • FP Growth

​
Advantages

  • No labeled data needed
  • Finds hidden patterns

​
Disadvantages

  • Hard to evaluate
  • Interpretation may be difficult

​
3. Reinforcement Learning (RL)

​
What is Reinforcement Learning?

Learning by interacting with an environment. Agent learns through: 
Actions → Rewards → Learning

​
RL Components

​
Agent

The learner Examples: 
Robot
Game AI
Self-driving car

​
Environment

Where agent acts. Examples: 
Chess board

Road

Game world

​
State

Current situation. Example: 
Robot location

​
Action

What agent can do. Example: 
Move left

Move right

Jump

​
Reward

Feedback after action. Example: 
+10 → good move

−5 → bad move

​
Reinforcement Learning Flow

Agent

↓

Take Action

↓

Environment changes

↓

Reward received

↓

Learn from reward

↓

Repeat

​
Example: Self-driving Car

State: 
Traffic signal = Red
Action: 
Stop
Reward: 
+10
Wrong action: 
Move forward
Reward: 
−100

​
Algorithms

  • Q-Learning
  • Deep Q Network (DQN)
  • SARSA
  • PPO
  • Policy Gradient

​
Applications

  • Robotics
  • Self-driving vehicles
  • Game AI
  • Trading systems
  • Recommendation systems

​
Advantages

  • Learns automatically
  • Handles complex environments

​
Disadvantages

  • Requires large training time
  • High computational cost

​
Comparison of ML Types

FeatureSupervisedUnsupervisedReinforcement
Labeled DataYesNoReward Signal
GoalPredict outputFind patternsMaximize reward
ExampleSpam detectionCustomer groupingSelf-driving cars
OutputPredictionGroups/patternsActions

​
Important ML Terms

​
Features

Input variables used for prediction. Example: 
Area
Bedrooms
Age

​
Labels

Expected output. Example: 
House price

​
Dataset

Collection of records. Example:
AreaBedroomsPrice
1200325L
1500435L

​
Training Data

Data used to train model.

​
Test Data

Data used to evaluate model.

​
Model

Mathematical representation learned from data.

​
Prediction

Output produced by model.

​
Overfitting

Model memorizes training data. Problem: 
High training accuracy

Low testing accuracy

​
Underfitting

Model fails to learn patterns. Problem: 
Low training accuracy

Low testing accuracy

​
Train / Validation / Test, Overfitting, Bias–Variance Tradeoff

​
Why Do We Split Data?

Suppose we train a model using all available data: 
Entire Data

↓

Train Model

↓

Evaluate on same data
Problem: 
Model may memorize data
The model can show: 
Training Accuracy = 99%
but for new unseen data: 
Real-world Accuracy = 50%
To avoid this problem, data is split.

​
Dataset Split

Usually: 
Dataset

↓

Train Data
Validation Data
Test Data
Common split: 
Training → 70%

Validation → 15%

Testing → 15%
or 
Training → 80%

Validation → 10%

Testing → 10%

​
Train Dataset

​
What is it?

Training data is used to teach the model patterns. Example: House Price Dataset:
AreaBedroomsPrice
1200325L
1500435L
1800440L
The model learns: 
Area + Bedrooms

↓

Price

​
Purpose

  • Learn relationships
  • Update model weights
  • Fit the model

​
Example

X_train=[
    [1200,3],
    [1500,4],
    [1800,4]
]

y_train=[
    25,
    35,
    40
]

​
Validation Dataset

​
What is it?

Validation data is used during training to check model performance and tune settings. Model does not learn from this data.

​
Purpose

Used for:
  • Hyperparameter tuning
  • Model selection
  • Detecting overfitting
Examples of hyperparameters: 
Learning rate

Epochs

Tree depth

Batch size

​
Example

Try different values: 
learning_rate=0.001

learning_rate=0.01

learning_rate=0.1
Choose the best validation performance.

​
Test Dataset

​
What is it?

Test data evaluates final performance. The model should never see test data during training.

​
Purpose

Measures: 
Real-world performance
Example: 
X_test=[
    [2000,5],
    [2500,6]
]

y_test=[
    50,
    65
]
Prediction: 
Predicted:

48

63

​
Complete Flow

Raw Data

↓

Train Data

↓

Train Model

↓

Validation Data

↓

Tune Hyperparameters

↓

Final Model

↓

Test Data

↓

Final Accuracy

​
Train-Test Split Using Scikit-Learn

Code: 
from sklearn.model_selection import train_test_split

X=[1,2,3,4,5,6,7,8,9,10]

y=[0,1,0,1,0,1,0,1,0,1]

X_train,X_test,y_train,y_test= train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

print(X_train)

print(X_test)
Possible Output: 
X_train:

[6,1,8,3,10,5,4,7]

X_test:

[9,2]

​
Train / Validation / Test Split

Code: 
from sklearn.model_selection import train_test_split

X_train,X_temp,y_train,y_temp = train_test_split(
        X,
        y,
        test_size=0.3,
        random_state=42
)

X_val,X_test,y_val,y_test = train_test_split(
        X_temp,
        y_temp,
        test_size=0.5,
        random_state=42
)

print(len(X_train))
print(len(X_val))
print(len(X_test))
Output: 
7
1
2

​
Overfitting

​
What is Overfitting?

The model memorizes training data instead of learning general patterns.
Example: Training accuracy: 
99%
Testing accuracy: 
55%
Graph intuition: 
Training Error

\
 \
  \
   \

Validation Error

\
 \
  \
   \__
       \
        \
Model becomes too complex and starts fitting noise.

​
Causes

  • Very complex model
  • Small dataset
  • Too many epochs
  • Too many features

​
Symptoms

High training accuracy

Low testing accuracy

​
Example

Training data:
HoursMarks
130
240
350
460
Model learns: 
Perfect curve passing every point
Instead of: 
General trend

​
Reduce Overfitting

​
1. More data

Increase training examples

​
2. Regularization

Adds penalty. Examples: 
L1 Regularization

L2 Regularization

​
3. Dropout

Randomly removes neurons. Used in: 
Neural Networks

​
4. Reduce complexity

Example: 
Reduce tree depth

​
5. Early stopping

Stop training when validation performance stops improving.
Example code: 
from tensorflow.keras.callbacks import EarlyStopping

early=EarlyStopping(
        monitor='val_loss',
        patience=3
)

​
Underfitting

​
What is Underfitting?

Model fails to learn patterns.
Example: Training accuracy: 
50%
Testing accuracy: 
48%
Symptoms: 
Low training accuracy

Low testing accuracy
Causes:
  • Model too simple
  • Too few features
  • Insufficient training
Solutions: 
Increase complexity

Train longer

Add features

​
Bias–Variance Tradeoff

This is one of the most important concepts in ML.

​
What is Bias?

Bias measures error due to overly simple assumptions. High bias: 
Model oversimplifies
Example: Trying to fit: 
Curved data
using: 
Straight line
Characteristics: 
Low variance

High error

​
What is Variance?

Variance measures sensitivity to small data changes. High variance: 
Model memorizes noise
Characteristics: 
Low bias

High error on new data

​
Relationship

High Bias

↓

Underfitting



Balanced Bias + Variance

↓

Good model



High Variance

↓

Overfitting

​
Bias–Variance Curve

Error

^
|
|\
| \
|  \
|   \____
|        \__
|
+-----------------> Model Complexity
Explanation: Left side: 
High bias

Underfitting
Middle: 
Optimal point
Right side: 
High variance

Overfitting

​
Example

Suppose actual data: 
Hours Studied → Marks

​
High Bias

Model: 
Marks = Hours
Too simple.

​
Balanced

Model: 
Learns trend

​
High Variance

Model: 
Fits every tiny fluctuation

​
Visual Example with Polynomial Regression

Code: 
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

x=np.array([1,2,3,4,5]).reshape(-1,1)

y=np.array([1,4,9,16,25])

poly=PolynomialFeatures(degree=4)

x_poly=poly.fit_transform(x)

model=LinearRegression()

model.fit(x_poly,y)

pred=model.predict(x_poly)

plt.scatter(x,y)

plt.plot(x,pred)

plt.show()
Output: 
Shows fitted curve
Image
If degree becomes very high: 
degree=20
Model starts overfitting.

​
Training vs Validation Loss

Good model: 
Epoch      Train Loss      Val Loss

1              0.8            0.9
2              0.6            0.7
3              0.4            0.5
4              0.3            0.35
Overfitting: 
Epoch      Train Loss      Val Loss

1              0.8            0.9
2              0.5            0.6
3              0.2            0.8
4              0.1            1.4
Training improves but validation worsens.

​
Quick Notes

Supervised:
Input + Correct Output

Regression:
Predict numbers

Classification:
Predict categories

Unsupervised:
Find hidden patterns

Clustering:
Group similar data

PCA:
Reduce dimensions

Reinforcement:
Learn using rewards

Features:
Inputs

Labels:
Outputs

Training Data:
Learn patterns

Validation Data:
Tune hyperparameters

Testing Data:
Final evaluation

Overfitting:
High train accuracy
Low test accuracy

Underfitting:
Low train accuracy
Low test accuracy

Bias:
Model too simple

Variance:
Model too sensitive

Bias + Variance:
Need balance

​
PTR

Why validation set? 
To tune hyperparameters without touching test data
Why not use test data for training? 
Causes data leakage and unrealistic performance
Signs of overfitting? 
High training accuracy
Low testing accuracy
How to reduce overfitting? 
Regularization
Dropout
More data
Early stopping
Reduce complexity
Bias vs Variance 
Bias → Underfitting

Variance → Overfitting