What is Feature Engineering?

Feature Engineering is the process of creating, modifying, or selecting features so that machine learning models can learn better from the data. Example: Suppose we have:

Age	Gender	Salary	City
22	Male	30000	Chennai
25	Female	40000	Mumbai
30	Male	50000	Chennai

Before giving this data to a machine learning model, we may need to:

Missing values
      ↓
Encoding
      ↓
Scaling
      ↓
Feature Selection
      ↓
ML Model

1. Missing Value Handling

Definition

A missing value means some data is not available. Example:

Age	Salary
22	30000
25	NaN
30	50000

NaN means the value is missing.

Common Methods

A. Remove rows

df.dropna()

Removes rows containing missing values.

B. Fill with mean

df["Salary"] = df["Salary"].fillna(df["Salary"].mean())

Replaces missing values with the average.

C. Fill with median

df["Salary"] = df["Salary"].fillna(df["Salary"].median())

Useful when data contains outliers.

D. Fill categorical values with mode

df["City"] = df["City"].fillna(df["City"].mode()[0])

Uses the most frequently occurring value.

Example

import pandas as pd

df = pd.DataFrame({
    "Age": [20, 25, 30],
    "Salary": [30000, None, 50000]
})

df["Salary"] = df["Salary"].fillna(df["Salary"].mean())

print(df)

Output

   Age   Salary
 20  30000.0
 25  40000.0
 30  50000.0

Remember

Missing values → Remove or fill them appropriately.

2. Encoding Categorical Variables

Definition

Machine learning models generally work with numbers, but real-world data often contains text. Example:

Gender
------
Male
Female
Male

We convert categorical values into numerical values. Two common methods:

Label Encoding
One-Hot Encoding

A. Label Encoding

Converts categories into numbers.

Male   → 1
Female → 0

Code

from sklearn.preprocessing import LabelEncoder

data = ["Male", "Female", "Male"]

encoder = LabelEncoder()

result = encoder.fit_transform(data)

print(result)

Output

[1 0 1]

Use

Generally useful for ordinal categories or binary categories.

B. One-Hot Encoding

Creates separate columns for each category. Original:

City
----
Chennai
Mumbai
Delhi

After encoding:

Chennai  Mumbai  Delhi
     0       0
     1       0
     0       1

Code

import pandas as pd

df = pd.DataFrame({
    "City": ["Chennai", "Mumbai", "Delhi"]
})

result = pd.get_dummies(df, dtype=int)

print(result)

Output

   City_Chennai  City_Delhi  City_Mumbai
     1            0           0
     0            0           1
     0            1           0

Remember

Categorical data → Convert text into numbers.

3. Feature Scaling

Definition

Feature Scaling puts numerical features into a similar range. Example:

Age       → 20 to 60
Salary    → 20,000 to 1,00,000

Salary has much larger values than Age. Some algorithms can be affected by this difference.

Common Scaling Methods

Standardization
Min-Max Scaling

A. StandardScaler

Standardization transforms data so that it generally has:

Mean = 0
Standard deviation = 1

Code

from sklearn.preprocessing import StandardScaler
import numpy as np

X = np.array([
    [20, 20000],
    [30, 40000],
    [40, 60000]
])

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

print(X_scaled)

Output

[[-1.2247 -1.2247]
 [ 0.      0.    ]
 [ 1.2247  1.2247]]

B. MinMaxScaler

Converts values generally into the range:

0 to 1

Code

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()

X_scaled = scaler.fit_transform(X)

print(X_scaled)

Output

[[0.  0. ]
 [0.5 0.5]
 [1.  1. ]]

When is scaling important?

Especially for algorithms based on distance or magnitude, such as:

K-Means
KNN
SVM
PCA
Neural Networks

Remember

Feature Scaling → Put numerical features on comparable scales.

4. Feature Selection

Definition

Feature Selection means selecting the most useful features and removing unnecessary ones. Example: Suppose we have:

Age
Salary
Experience
Height
Favorite Color
Customer ID

Maybe only these are useful:

Age
Salary
Experience

We remove irrelevant features.

Why?

Feature selection can:

Reduce model complexity
Reduce training time
Remove irrelevant information
Reduce overfitting
Sometimes improve model performance

Simple Example

import pandas as pd

df = pd.DataFrame({
    "Age": [20, 25, 30],
    "Salary": [30000, 40000, 50000],
    "Customer_ID": [101, 102, 103]
})

selected = df[["Age", "Salary"]]

print(selected)

Output

   Age  Salary
 20   30000
 25   40000
 30   50000

Customer_ID was removed because it usually doesn’t provide useful predictive information.

Common Feature Selection Methods

Filter Methods
    ↓
Correlation, Chi-Square

Wrapper Methods
    ↓
RFE

Embedded Methods
    ↓
Lasso, Decision Tree

Complete Feature Engineering Flow

              Raw Data
                  ↓
        ┌─────────────────┐
        │ Missing Values                │
        └────────┬────────┘
                 ↓
        Handle Missing Data
                 ↓
        ┌─────────────────┐
        │ Categorical Data              │
        └────────┬────────┘
                 ↓
              Encoding
                 ↓
        ┌─────────────────┐
        │ Numerical Data                │
        └────────┬────────┘
                 ↓
              Scaling
                 ↓
          Feature Selection
                 ↓
          Machine Learning
              Model

Quick Comparison

Technique	Purpose
Missing Value Handling	Deal with missing data
Encoding	Convert categorical data to numbers
Feature Scaling	Put numerical features on similar scales
Feature Selection	Keep useful features and remove unnecessary ones

Easy way to remember

Missing values → Fix the data Encoding → Convert text to numbers Scaling → Normalize numerical ranges Selection → Keep important features

​What is Feature Engineering?

​1. Missing Value Handling

​Definition

​Common Methods

​A. Remove rows

​B. Fill with mean

​C. Fill with median

​D. Fill categorical values with mode

​Example

​Output

​Remember

​2. Encoding Categorical Variables

​Definition

​A. Label Encoding

​Code

​Output

​Use

​B. One-Hot Encoding

​Code

​Output

​Remember

​3. Feature Scaling

​Definition

​Common Scaling Methods

​A. StandardScaler

​Code

​Output

​B. MinMaxScaler

​Code

​Output

​When is scaling important?

​Remember

​4. Feature Selection

​Definition

​Why?

​Simple Example

​Output

​Common Feature Selection Methods

​Complete Feature Engineering Flow

​Quick Comparison

​Easy way to remember

What is Feature Engineering?

1. Missing Value Handling

Definition

Common Methods

A. Remove rows

B. Fill with mean

C. Fill with median

D. Fill categorical values with mode

Example

Output

Remember

2. Encoding Categorical Variables

Definition

A. Label Encoding

Code

Output

Use

B. One-Hot Encoding

Code

Output

Remember

3. Feature Scaling

Definition

Common Scaling Methods

A. StandardScaler

Code

Output

B. MinMaxScaler

Code

Output

When is scaling important?

Remember

4. Feature Selection

Definition

Why?

Simple Example

Output

Common Feature Selection Methods

Complete Feature Engineering Flow

Quick Comparison

Easy way to remember