> ## Documentation Index
> Fetch the complete documentation index at: https://ai.tharung.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Feature Engineering

### What is Feature Engineering?

**Feature Engineering** is the process of **creating, modifying, or selecting features** so that machine learning models can learn better from the data.

Example:

Suppose we have:

| Age | Gender | Salary | City    |
| --: | ------ | -----: | ------- |
|  22 | Male   |  30000 | Chennai |
|  25 | Female |  40000 | Mumbai  |
|  30 | Male   |  50000 | Chennai |

Before giving this data to a machine learning model, we may need to:

```text theme={null}
Missing values
      ↓
Encoding
      ↓
Scaling
      ↓
Feature Selection
      ↓
ML Model
```

***

# 1. Missing Value Handling

### Definition

A **missing value** means some data is not available.

Example:

| Age | Salary |
| --: | -----: |
|  22 |  30000 |
|  25 |    NaN |
|  30 |  50000 |

`NaN` means the value is missing.

### Common Methods

#### A. Remove rows

```python theme={null}
df.dropna()
```

Removes rows containing missing values.

#### B. Fill with mean

```python theme={null}
df["Salary"] = df["Salary"].fillna(df["Salary"].mean())
```

Replaces missing values with the **average**.

#### C. Fill with median

```python theme={null}
df["Salary"] = df["Salary"].fillna(df["Salary"].median())
```

Useful when data contains outliers.

#### D. Fill categorical values with mode

```python theme={null}
df["City"] = df["City"].fillna(df["City"].mode()[0])
```

Uses the most frequently occurring value.

### Example

```python theme={null}
import pandas as pd

df = pd.DataFrame({
    "Age": [20, 25, 30],
    "Salary": [30000, None, 50000]
})

df["Salary"] = df["Salary"].fillna(df["Salary"].mean())

print(df)
```

### Output

```text theme={null}
   Age   Salary
0   20  30000.0
1   25  40000.0
2   30  50000.0
```

### Remember

> **Missing values → Remove or fill them appropriately.**

***

# 2. Encoding Categorical Variables

### Definition

Machine learning models generally work with **numbers**, but real-world data often contains text.

Example:

```text theme={null}
Gender
------
Male
Female
Male
```

We convert categorical values into numerical values.

Two common methods:

1. **Label Encoding**
2. **One-Hot Encoding**

***

## A. Label Encoding

Converts categories into numbers.

```text theme={null}
Male   → 1
Female → 0
```

### Code

```python theme={null}
from sklearn.preprocessing import LabelEncoder

data = ["Male", "Female", "Male"]

encoder = LabelEncoder()

result = encoder.fit_transform(data)

print(result)
```

### Output

```text theme={null}
[1 0 1]
```

### Use

Generally useful for **ordinal categories** or binary categories.

***

## B. One-Hot Encoding

Creates separate columns for each category.

Original:

```text theme={null}
City
----
Chennai
Mumbai
Delhi
```

After encoding:

```text theme={null}
Chennai  Mumbai  Delhi
   1       0       0
   0       1       0
   0       0       1
```

### Code

```python theme={null}
import pandas as pd

df = pd.DataFrame({
    "City": ["Chennai", "Mumbai", "Delhi"]
})

result = pd.get_dummies(df, dtype=int)

print(result)
```

### Output

```text theme={null}
   City_Chennai  City_Delhi  City_Mumbai
0       1            0           0
1       0            0           1
2       0            1           0
```

### Remember

> **Categorical data → Convert text into numbers.**

***

# 3. Feature Scaling

### Definition

**Feature Scaling** puts numerical features into a similar range.

Example:

```text theme={null}
Age       → 20 to 60
Salary    → 20,000 to 1,00,000
```

Salary has much larger values than Age.

Some algorithms can be affected by this difference.

### Common Scaling Methods

* **Standardization**
* **Min-Max Scaling**

***

## A. StandardScaler

Standardization transforms data so that it generally has:

```text theme={null}
Mean = 0
Standard deviation = 1
```

### Code

```python theme={null}
from sklearn.preprocessing import StandardScaler
import numpy as np

X = np.array([
    [20, 20000],
    [30, 40000],
    [40, 60000]
])

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

print(X_scaled)
```

### Output

```text theme={null}
[[-1.2247 -1.2247]
 [ 0.      0.    ]
 [ 1.2247  1.2247]]
```

***

## B. MinMaxScaler

Converts values generally into the range:

```text theme={null}
0 to 1
```

### Code

```python theme={null}
from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()

X_scaled = scaler.fit_transform(X)

print(X_scaled)
```

### Output

```text theme={null}
[[0.  0. ]
 [0.5 0.5]
 [1.  1. ]]
```

### When is scaling important?

Especially for algorithms based on **distance or magnitude**, such as:

* K-Means
* KNN
* SVM
* PCA
* Neural Networks

### Remember

> **Feature Scaling → Put numerical features on comparable scales.**

***

# 4. Feature Selection

### Definition

**Feature Selection** means selecting the **most useful features** and removing unnecessary ones.

Example:

Suppose we have:

```text theme={null}
Age
Salary
Experience
Height
Favorite Color
Customer ID
```

Maybe only these are useful:

```text theme={null}
Age
Salary
Experience
```

We remove irrelevant features.

### Why?

Feature selection can:

* Reduce model complexity
* Reduce training time
* Remove irrelevant information
* Reduce overfitting
* Sometimes improve model performance

***

## Simple Example

```python theme={null}
import pandas as pd

df = pd.DataFrame({
    "Age": [20, 25, 30],
    "Salary": [30000, 40000, 50000],
    "Customer_ID": [101, 102, 103]
})

selected = df[["Age", "Salary"]]

print(selected)
```

### Output

```text theme={null}
   Age  Salary
0   20   30000
1   25   40000
2   30   50000
```

`Customer_ID` was removed because it usually doesn't provide useful predictive information.

### Common Feature Selection Methods

```text theme={null}
Filter Methods
    ↓
Correlation, Chi-Square

Wrapper Methods
    ↓
RFE

Embedded Methods
    ↓
Lasso, Decision Tree
```

***

# Complete Feature Engineering Flow

```text theme={null}
              Raw Data
                  ↓
        ┌─────────────────┐
        │ Missing Values                │
        └────────┬────────┘
                 ↓
        Handle Missing Data
                 ↓
        ┌─────────────────┐
        │ Categorical Data              │
        └────────┬────────┘
                 ↓
              Encoding
                 ↓
        ┌─────────────────┐
        │ Numerical Data                │
        └────────┬────────┘
                 ↓
              Scaling
                 ↓
          Feature Selection
                 ↓
          Machine Learning
              Model
```

# Quick Comparison

| Technique                  | Purpose                                          |
| -------------------------- | ------------------------------------------------ |
| **Missing Value Handling** | Deal with missing data                           |
| **Encoding**               | Convert categorical data to numbers              |
| **Feature Scaling**        | Put numerical features on similar scales         |
| **Feature Selection**      | Keep useful features and remove unnecessary ones |

### Easy way to remember

> **Missing values → Fix the data**
> **Encoding → Convert text to numbers**
> **Scaling → Normalize numerical ranges**
> **Selection → Keep important features**