> ## Documentation Index
> Fetch the complete documentation index at: https://ai.tharung.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Supervised ML Algorithms & Model Evaluation

# Machine Learning: Regression, Classification & Model Evaluation

## 1. Machine Learning Overview

Machine Learning (ML) allows a computer to learn patterns from data and make predictions without explicitly programming every rule.

Two common supervised learning tasks are:

| Task           | Output            | Examples                         |
| -------------- | ----------------- | -------------------------------- |
| Regression     | Continuous number | House price, salary, temperature |
| Classification | Category/class    | Spam/Not Spam, Pass/Fail         |

***

# 2. Regression

Regression is used when the target/output is a **continuous numerical value**.

### Example

Predict house price:

```text theme={null}
Input:
Area = 1500 sq.ft
Bedrooms = 3

Output:
Price = ₹75,00,000
```

Common regression algorithms:

1. Linear Regression
2. Ridge Regression
3. Lasso Regression

***

# 3. Linear Regression

Linear Regression finds a relationship between input features and a continuous target.

The basic equation is:

```text theme={null}
y = mx + b
```

Where:

* `y` = predicted value
* `x` = input
* `m` = slope/coefficient
* `b` = intercept

For multiple features:

```text theme={null}
y = b + w1x1 + w2x2 + ... + wnxn
```

## Example

Predict salary based on years of experience.

```python theme={null}
from sklearn.linear_model import LinearRegression

X = [[1], [2], [3], [4], [5]]
y = [30000, 40000, 50000, 60000, 70000]

model = LinearRegression()
model.fit(X, y)

prediction = model.predict([[6]])

print(prediction)
```

Output:

```text theme={null}
[80000.]
```

### Important Properties

```python theme={null}
model.coef_
model.intercept_
```

* `coef_` → learned slope/weight
* `intercept_` → value of `y` when all features are 0

### When to use

Use Linear Regression when:

* Target is continuous
* Relationship is approximately linear
* You want a simple and interpretable model

***

# 4. Ridge Regression

Ridge Regression is Linear Regression with **L2 regularization**.

It adds a penalty for large coefficients.

Conceptually:

```text theme={null}
Loss = MSE + α × Σ(coefficient²)
```

Where:

* `α` = regularization strength
* Larger `α` → stronger regularization

## Example

```python theme={null}
from sklearn.linear_model import Ridge

X = [[1], [2], [3], [4], [5]]
y = [30000, 40000, 50000, 60000, 70000]

model = Ridge(alpha=1.0)

model.fit(X, y)

prediction = model.predict([[6]])

print(prediction)
```

### Why use Ridge?

Ridge helps when:

* Features are highly correlated
* The model is overfitting
* You have many features

### Important

Ridge generally **shrinks coefficients toward zero**, but usually does not make them exactly zero.

***

# 5. Lasso Regression

Lasso Regression uses **L1 regularization**.

Conceptually:

```text theme={null}
Loss = MSE + α × Σ|coefficient|
```

Unlike Ridge, Lasso can make some coefficients exactly zero.

## Example

```python theme={null}
from sklearn.linear_model import Lasso

X = [[1], [2], [3], [4], [5]]
y = [30000, 40000, 50000, 60000, 70000]

model = Lasso(alpha=1.0)

model.fit(X, y)

prediction = model.predict([[6]])

print(prediction)
```

### Why use Lasso?

Lasso is useful for:

* Feature selection
* Reducing unnecessary features
* Preventing overfitting

### Ridge vs Lasso

| Feature                        | Ridge      | Lasso               |      |    |
| ------------------------------ | ---------- | ------------------- | ---- | -- |
| Regularization                 | L2         | L1                  |      |    |
| Penalty                        | `coef²`    | \`                  | coef | \` |
| Coefficients become zero       | Usually no | Yes                 |      |    |
| Feature selection              | No         | Yes                 |      |    |
| Useful for correlated features | Yes        | Can select one/some |      |    |

***

# 6. Classification

Classification predicts a **category/class**.

### Examples

```text theme={null}
Email → Spam / Not Spam

Transaction → Fraud / Not Fraud

Student → Pass / Fail

Image → Cat / Dog
```

Common classification algorithms:

1. Logistic Regression
2. K-Nearest Neighbors
3. Decision Tree
4. Random Forest
5. Support Vector Machine
6. Naive Bayes

***

# 7. Logistic Regression

Despite its name, Logistic Regression is mainly used for **classification**.

It predicts the probability of a class.

The sigmoid function converts a value into a probability between 0 and 1.

```text theme={null}
σ(z) = 1 / (1 + e^-z)
```

Example:

```text theme={null}
Probability = 0.85

0.85 >= 0.5

Prediction = Class 1
```

## Example

```python theme={null}
from sklearn.linear_model import LogisticRegression

X = [[1], [2], [3], [4], [5], [6]]
y = [0, 0, 0, 1, 1, 1]

model = LogisticRegression()

model.fit(X, y)

prediction = model.predict([[5]])

probability = model.predict_proba([[5]])

print(prediction)
print(probability)
```

### Important methods

```python theme={null}
model.predict(X)
```

Returns predicted classes.

```python theme={null}
model.predict_proba(X)
```

Returns class probabilities.

### When to use

Use Logistic Regression when:

* Target is categorical
* You need probability estimates
* You want a simple and interpretable classification model

***

# 8. K-Nearest Neighbors (KNN)

KNN predicts a data point based on its nearest training examples.

The basic idea:

```text theme={null}
1. Choose K
2. Calculate distance
3. Find K nearest points
4. Look at their classes
5. Majority class becomes prediction
```

Example:

```text theme={null}
K = 3

Nearest neighbors:

Point 1 → Cat
Point 2 → Dog
Point 3 → Cat

Majority = Cat

Prediction = Cat
```

## Example

```python theme={null}
from sklearn.neighbors import KNeighborsClassifier

X = [[1], [2], [3], [10], [11], [12]]
y = [0, 0, 0, 1, 1, 1]

model = KNeighborsClassifier(n_neighbors=3)

model.fit(X, y)

prediction = model.predict([[4]])

print(prediction)
```

### Choosing K

Small `K`:

```text theme={null}
More sensitive to noise
Possible overfitting
```

Large `K`:

```text theme={null}
Smoother decision
Possible underfitting
```

### Important

KNN is distance-based, so **feature scaling is usually important**.

***

# 9. Decision Tree

A Decision Tree makes predictions using a sequence of questions.

Example:

```text theme={null}
             Age > 30?
             /       \
           Yes        No
           /           \
      Income > 50K?    No
       /      \
     Yes      No
```

The tree contains:

* Root node
* Internal nodes
* Branches
* Leaf nodes

## Example

```python theme={null}
from sklearn.tree import DecisionTreeClassifier

X = [[20], [25], [30], [35], [40], [45]]
y = [0, 0, 0, 1, 1, 1]

model = DecisionTreeClassifier(max_depth=3)

model.fit(X, y)

prediction = model.predict([[38]])

print(prediction)
```

### Important parameters

```python theme={null}
DecisionTreeClassifier(
    max_depth=3,
    min_samples_split=2,
    min_samples_leaf=1
)
```

### Advantages

* Easy to understand
* Little preprocessing required
* Can model non-linear relationships

### Disadvantage

A deep tree can easily **overfit**.

***

# 10. Random Forest

Random Forest is an ensemble of multiple Decision Trees.

Instead of relying on one tree:

```text theme={null}
Tree 1 → Class A
Tree 2 → Class B
Tree 3 → Class A
Tree 4 → Class A
Tree 5 → Class B

Majority → Class A
```

For classification, trees vote.

For regression, predictions are generally averaged.

## Example

```python theme={null}
from sklearn.ensemble import RandomForestClassifier

X = [[20], [25], [30], [35], [40], [45]]
y = [0, 0, 0, 1, 1, 1]

model = RandomForestClassifier(
    n_estimators=100,
    random_state=42
)

model.fit(X, y)

prediction = model.predict([[38]])

print(prediction)
```

### Important parameters

```python theme={null}
n_estimators
max_depth
min_samples_split
min_samples_leaf
```

`n_estimators` controls the number of trees.

### Advantages

* Usually more robust than a single tree
* Handles non-linear relationships
* Works with many features
* Less prone to overfitting than a single unrestricted tree

***

# 11. Support Vector Machine (SVM)

SVM finds a decision boundary that separates classes.

The best boundary tries to maximize the **margin** between classes.

```text theme={null}
Class A       |       Class B

● ● ●         |         ○ ○ ○
● ● ●         |         ○ ○ ○

             ↑
        Decision boundary
```

The closest points to the boundary are called **support vectors**.

## Example

```python theme={null}
from sklearn.svm import SVC

X = [[1], [2], [3], [10], [11], [12]]
y = [0, 0, 0, 1, 1, 1]

model = SVC(kernel="linear")

model.fit(X, y)

prediction = model.predict([[5]])

print(prediction)
```

### Common kernels

```text theme={null}
linear
poly
rbf
sigmoid
```

Example:

```python theme={null}
model = SVC(kernel="rbf")
```

### Important

SVM is sensitive to feature scale.

Scaling is commonly performed before training:

```python theme={null}
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.svm import SVC

model = make_pipeline(
    StandardScaler(),
    SVC()
)
```

***

# 12. Naive Bayes

Naive Bayes is a probabilistic classification algorithm based on **Bayes' theorem**.

Bayes' theorem:

```text theme={null}
P(A|B) = P(B|A)P(A) / P(B)
```

The "naive" assumption is that features are conditionally independent given the class.

## Example

```python theme={null}
from sklearn.naive_bayes import GaussianNB

X = [[1, 20], [2, 25], [3, 30],
     [10, 40], [11, 45], [12, 50]]

y = [0, 0, 0, 1, 1, 1]

model = GaussianNB()

model.fit(X, y)

prediction = model.predict([[4, 28]])

print(prediction)
```

### Common Naive Bayes types

| Type          | Common use                |
| ------------- | ------------------------- |
| GaussianNB    | Continuous numerical data |
| MultinomialNB | Word counts/text          |
| BernoulliNB   | Binary features           |

***

# 13. Model Evaluation

After training a model, we need to determine:

```text theme={null}
How good is the model?
```

Different metrics are used for different problems.

***

# 14. Confusion Matrix

A confusion matrix summarizes classification predictions.

For binary classification:

|                 | Predicted Positive | Predicted Negative |
| --------------- | -----------------: | -----------------: |
| Actual Positive |                 TP |                 FN |
| Actual Negative |                 FP |                 TN |

Where:

* **TP** = True Positive
* **TN** = True Negative
* **FP** = False Positive
* **FN** = False Negative

Example:

```text theme={null}
Actual:     [1, 1, 0, 0]
Predicted:  [1, 0, 0, 1]
```

Results:

```text theme={null}
TP = 1
FN = 1
TN = 1
FP = 1
```

## Python

```python theme={null}
from sklearn.metrics import confusion_matrix

y_true = [1, 1, 0, 0]
y_pred = [1, 0, 0, 1]

cm = confusion_matrix(y_true, y_pred)

print(cm)
```

Output:

```text theme={null}
[[1 1]
 [1 1]]
```

***

# 15. Accuracy

Accuracy tells us the percentage of predictions that were correct.

Formula:

```text theme={null}
Accuracy = (TP + TN) / (TP + TN + FP + FN)
```

## Example

```python theme={null}
from sklearn.metrics import accuracy_score

y_true = [1, 1, 0, 0]
y_pred = [1, 0, 0, 1]

accuracy = accuracy_score(y_true, y_pred)

print(accuracy)
```

Output:

```text theme={null}
0.5
```

So:

```text theme={null}
Accuracy = 50%
```

### Problem with Accuracy

Accuracy can be misleading for **imbalanced datasets**.

Example:

```text theme={null}
1000 transactions
990 = Not Fraud
10 = Fraud
```

A model predicting every transaction as "Not Fraud" gets:

```text theme={null}
Accuracy = 99%
```

But it detects:

```text theme={null}
0 fraud cases
```

Therefore, other metrics may be more useful.

***

# 16. Precision

Precision answers:

> Of all the samples predicted as positive, how many were actually positive?

Formula:

```text theme={null}
Precision = TP / (TP + FP)
```

Example:

```text theme={null}
TP = 80
FP = 20

Precision = 80 / (80 + 20)
          = 0.80
```

So:

```text theme={null}
Precision = 80%
```

## Python

```python theme={null}
from sklearn.metrics import precision_score

y_true = [1, 1, 1, 0, 0]
y_pred = [1, 1, 0, 1, 0]

print(precision_score(y_true, y_pred))
```

### When precision matters

Precision is important when **false positives are costly**.

Example:

```text theme={null}
Spam detection
```

You don't want many legitimate emails to be incorrectly marked as spam.

***

# 17. Recall

Recall answers:

> Of all the actual positive samples, how many did the model find?

Formula:

```text theme={null}
Recall = TP / (TP + FN)
```

Example:

```text theme={null}
TP = 80
FN = 20

Recall = 80 / (80 + 20)
       = 0.80
```

So:

```text theme={null}
Recall = 80%
```

## Python

```python theme={null}
from sklearn.metrics import recall_score

y_true = [1, 1, 1, 0, 0]
y_pred = [1, 1, 0, 1, 0]

print(recall_score(y_true, y_pred))
```

### When recall matters

Recall is important when **false negatives are costly**.

Example:

```text theme={null}
Disease detection
Fraud detection
Security threat detection
```

Missing a positive case can be expensive.

***

# 18. Precision vs Recall

Remember:

```text theme={null}
Precision → "When I predict positive, am I correct?"

Recall → "Did I find most of the actual positives?"
```

| Metric    | Focus                  |
| --------- | ---------------------- |
| Precision | Reduce False Positives |
| Recall    | Reduce False Negatives |

***

# 19. F1 Score

F1 Score combines Precision and Recall.

It is the harmonic mean of precision and recall.

Formula:

```text theme={null}
F1 = 2 × (Precision × Recall)
     ----------------------------
       Precision + Recall
```

Example:

```text theme={null}
Precision = 0.80
Recall = 0.60

F1 = 2 × (0.80 × 0.60) / (0.80 + 0.60)

F1 ≈ 0.686
```

## Python

```python theme={null}
from sklearn.metrics import f1_score

y_true = [1, 1, 1, 0, 0]
y_pred = [1, 1, 0, 1, 0]

print(f1_score(y_true, y_pred))
```

### When to use

F1 is useful when:

* Classes are imbalanced
* Both precision and recall matter
* You want one metric balancing both

***

# 20. ROC-AUC

ROC-AUC evaluates how well a classification model separates positive and negative classes across different classification thresholds.

### ROC

ROC stands for:

```text theme={null}
Receiver Operating Characteristic
```

It plots:

```text theme={null}
True Positive Rate
vs
False Positive Rate
```

Where:

```text theme={null}
TPR = TP / (TP + FN)
```

and

```text theme={null}
FPR = FP / (FP + TN)
```

### AUC

AUC means:

```text theme={null}
Area Under the Curve
```

General interpretation:

|     AUC | Interpretation    |
| ------: | ----------------- |
|     1.0 | Perfect           |
|    0.9+ | Excellent         |
| 0.8–0.9 | Good              |
| 0.7–0.8 | Fair              |
|     0.5 | Random            |
|  \< 0.5 | Worse than random |

## Python

```python theme={null}
from sklearn.metrics import roc_auc_score

y_true = [0, 0, 1, 1]
y_probability = [0.1, 0.3, 0.7, 0.9]

auc = roc_auc_score(y_true, y_probability)

print(auc)
```

Output:

```text theme={null}
1.0
```

### Important

ROC-AUC generally requires **probabilities or decision scores**, not just final class labels.

For example:

```python theme={null}
model.predict_proba(X)[:, 1]
```

***

# 21. MAE

MAE stands for:

```text theme={null}
Mean Absolute Error
```

It measures the average absolute difference between actual and predicted values.

Formula:

```text theme={null}
MAE = Σ|Actual - Predicted| / n
```

Example:

```text theme={null}
Actual:     [100, 200, 300]
Predicted:  [110, 180, 310]

Errors:
10, 20, 10

MAE = (10 + 20 + 10) / 3
    = 13.33
```

## Python

```python theme={null}
from sklearn.metrics import mean_absolute_error

y_true = [100, 200, 300]
y_pred = [110, 180, 310]

mae = mean_absolute_error(y_true, y_pred)

print(mae)
```

Output:

```text theme={null}
13.333333333333334
```

### Interpretation

If MAE is:

```text theme={null}
13.33
```

the model is off by about **13.33 units on average**.

***

# 22. MSE

MSE stands for:

```text theme={null}
Mean Squared Error
```

Formula:

```text theme={null}
MSE = Σ(Actual - Predicted)² / n
```

Example:

```text theme={null}
Actual:     [100, 200, 300]
Predicted:  [110, 180, 310]

Errors:
10, -20, 10

Squared errors:
100, 400, 100

MSE = (100 + 400 + 100) / 3
    = 200
```

## Python

```python theme={null}
from sklearn.metrics import mean_squared_error

y_true = [100, 200, 300]
y_pred = [110, 180, 310]

mse = mean_squared_error(y_true, y_pred)

print(mse)
```

Output:

```text theme={null}
200.0
```

### Important

MSE gives more importance to large errors because errors are squared.

***

# 23. RMSE

RMSE stands for:

```text theme={null}
Root Mean Squared Error
```

It is the square root of MSE.

Formula:

```text theme={null}
RMSE = √MSE
```

For the previous example:

```text theme={null}
MSE = 200

RMSE = √200
     ≈ 14.14
```

## Python

```python theme={null}
from sklearn.metrics import root_mean_squared_error

y_true = [100, 200, 300]
y_pred = [110, 180, 310]

rmse = root_mean_squared_error(y_true, y_pred)

print(rmse)
```

Output:

```text theme={null}
14.1421356237
```

### Version note

If your scikit-learn version does not provide `root_mean_squared_error`, use:

```python theme={null}
import numpy as np
from sklearn.metrics import mean_squared_error

rmse = np.sqrt(mean_squared_error(y_true, y_pred))

print(rmse)
```

***

# 24. R² Score

R² is called the **coefficient of determination**.

It measures how well a regression model explains the variation in the target.

A simplified interpretation:

```text theme={null}
R² = 1
```

Perfect prediction.

```text theme={null}
R² = 0
```

Model is roughly as good as predicting the mean.

```text theme={null}
R² < 0
```

Model can be worse than simply predicting the mean.

## Python

```python theme={null}
from sklearn.metrics import r2_score

y_true = [10, 20, 30, 40]
y_pred = [12, 18, 31, 39]

r2 = r2_score(y_true, y_pred)

print(r2)
```

### Interpretation

Higher R² is generally better for a regression model, but it should not be used alone to judge model quality.

***

# 25. Classification Metrics Summary

| Metric           | Formula         | Main Question                             |
| ---------------- | --------------- | ----------------------------------------- |
| Accuracy         | `(TP+TN)/Total` | How many predictions are correct?         |
| Precision        | `TP/(TP+FP)`    | How many predicted positives are correct? |
| Recall           | `TP/(TP+FN)`    | How many actual positives were found?     |
| F1               | `2PR/(P+R)`     | How balanced are precision and recall?    |
| ROC-AUC          | Area under ROC  | How well are classes separated?           |
| Confusion Matrix | TP/TN/FP/FN     | What types of errors occurred?            |

***

# 26. Regression Metrics Summary

| Metric | Meaning                | Lower/Better |
| ------ | ---------------------- | ------------ |
| MAE    | Average absolute error | Lower        |
| MSE    | Average squared error  | Lower        |
| RMSE   | Square root of MSE     | Lower        |
| R²     | Explained variance     | Higher       |

***

# 27. Complete Classification Example

A typical ML workflow looks like this:

```python theme={null}
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    accuracy_score,
    precision_score,
    recall_score,
    f1_score,
    confusion_matrix,
    roc_auc_score
)

# Data
X = [
    [1, 20],
    [2, 21],
    [3, 22],
    [8, 40],
    [9, 41],
    [10, 42]
]

y = [0, 0, 0, 1, 1, 1]

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.3,
    random_state=42,
    stratify=y
)

# Scaling
scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Model
model = LogisticRegression()

model.fit(X_train, y_train)

# Prediction
y_pred = model.predict(X_test)

# Probability for positive class
y_prob = model.predict_proba(X_test)[:, 1]

# Evaluation
print("Accuracy:", accuracy_score(y_test, y_pred))
print("Precision:", precision_score(y_test, y_pred))
print("Recall:", recall_score(y_test, y_pred))
print("F1:", f1_score(y_test, y_pred))
print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))
print("ROC-AUC:", roc_auc_score(y_test, y_prob))
```

***

# 28. Complete Regression Example

```python theme={null}
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import (
    mean_absolute_error,
    mean_squared_error,
    r2_score
)
import numpy as np

# Data
X = [[1], [2], [3], [4], [5], [6], [7], [8]]
y = [10, 20, 30, 40, 50, 60, 70, 80]

# Split data
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42
)

# Model
model = LinearRegression()

# Train
model.fit(X_train, y_train)

# Predict
y_pred = model.predict(X_test)

# Evaluation
mae = mean_absolute_error(y_test, y_pred)
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, y_pred)

print("MAE:", mae)
print("MSE:", mse)
print("RMSE:", rmse)
print("R²:", r2)
```

***

# 29. Which Algorithm Should I Choose?

## Regression

### Linear Regression

Use when:

```text theme={null}
Relationship ≈ Linear
Need simple model
Need interpretability
```

### Ridge

Use when:

```text theme={null}
Many features
Multicollinearity
Overfitting
```

### Lasso

Use when:

```text theme={null}
Many features
Need feature selection
Want some coefficients → 0
```

***

## Classification

### Logistic Regression

```text theme={null}
Simple classification
Need probabilities
Need interpretability
```

### KNN

```text theme={null}
Small/medium dataset
Similarity-based prediction
Non-linear boundaries
```

### Decision Tree

```text theme={null}
Need interpretability
Non-linear relationships
Minimal preprocessing
```

### Random Forest

```text theme={null}
Strong general-purpose baseline
Non-linear data
Many features
Want better robustness than one tree
```

### SVM

```text theme={null}
Small/medium datasets
High-dimensional data
Clear class boundaries
```

### Naive Bayes

```text theme={null}
Text classification
Fast classification
Probabilistic problems
```

***

# 30. Quick Algorithm Comparison

| Algorithm           | Type           | Main Idea                 | Scaling Usually Needed? |
| ------------------- | -------------- | ------------------------- | ----------------------- |
| Linear Regression   | Regression     | Fit linear relationship   | Sometimes               |
| Ridge               | Regression     | Linear + L2 penalty       | Yes, usually            |
| Lasso               | Regression     | Linear + L1 penalty       | Yes, usually            |
| Logistic Regression | Classification | Probability using sigmoid | Yes, usually            |
| KNN                 | Classification | Nearest neighbors         | Yes                     |
| Decision Tree       | Classification | If/else splits            | No                      |
| Random Forest       | Classification | Many decision trees       | No                      |
| SVM                 | Classification | Maximum-margin boundary   | Yes                     |
| Naive Bayes         | Classification | Bayes probability         | Depends                 |

***

# 31. Important Concepts to Remember

## Overfitting

Model performs well on training data but poorly on unseen data.

```text theme={null}
Training accuracy → Very high
Test accuracy     → Low
```

Possible solutions:

```text theme={null}
More training data
Regularization
Simpler model
Cross-validation
Feature selection
Tree pruning
```

***

## Underfitting

Model is too simple to learn the underlying pattern.

```text theme={null}
Training performance → Poor
Test performance     → Poor
```

Possible solutions:

```text theme={null}
More complex model
Better features
Reduce excessive regularization
Train longer where applicable
```

***

# 32. Regularization

Regularization prevents a model from becoming unnecessarily complex.

### Ridge

```text theme={null}
L2 → coefficient²
```

### Lasso

```text theme={null}
L1 → |coefficient|
```

Remember:

```text theme={null}
Ridge → Shrinks coefficients

Lasso → Shrinks + can remove features
```

***

# 33. Train/Test Workflow

The standard supervised ML workflow is:

```text theme={null}
Raw Data
   ↓
Clean Data
   ↓
Feature Selection
   ↓
Train/Test Split
   ↓
Preprocessing
   ↓
Train Model
   ↓
Predict
   ↓
Evaluate
   ↓
Tune Model
   ↓
Final Model
```

Example:

```python theme={null}
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

model.fit(X_train, y_train)

y_pred = model.predict(X_test)
```

***

# 34. Most Important Formulas

## Linear Regression

```text theme={null}
y = mx + b
```

## Logistic Regression

```text theme={null}
σ(z) = 1 / (1 + e^-z)
```

## Ridge

```text theme={null}
Loss = MSE + αΣw²
```

## Lasso

```text theme={null}
Loss = MSE + αΣ|w|
```

## Accuracy

```text theme={null}
(TP + TN) / (TP + TN + FP + FN)
```

## Precision

```text theme={null}
TP / (TP + FP)
```

## Recall

```text theme={null}
TP / (TP + FN)
```

## F1

```text theme={null}
2 × Precision × Recall
----------------------
Precision + Recall
```

## MAE

```text theme={null}
Σ|Actual - Predicted| / n
```

## MSE

```text theme={null}
Σ(Actual - Predicted)² / n
```

## RMSE

```text theme={null}
√MSE
```

## R²

```text theme={null}
1 - (Residual Sum of Squares / Total Sum of Squares)
```

***

# 35. One-Minute Revision

```text theme={null}
REGRESSION
│
├── Linear Regression → Straight-line relationship
├── Ridge → L2 regularization
└── Lasso → L1 regularization + feature selection


CLASSIFICATION
│
├── Logistic Regression → Probability + classification
├── KNN → Nearest neighbors
├── Decision Tree → If/else splits
├── Random Forest → Many decision trees
├── SVM → Maximum-margin boundary
└── Naive Bayes → Bayes probability


CLASSIFICATION EVALUATION
│
├── Accuracy → Overall correctness
├── Precision → Correct positive predictions
├── Recall → Actual positives found
├── F1 → Precision + Recall balance
├── Confusion Matrix → TP/TN/FP/FN
└── ROC-AUC → Class separation


REGRESSION EVALUATION
│
├── MAE → Average absolute error
├── MSE → Average squared error
├── RMSE → Error in original units
└── R² → Explained variance
```

***

# 36. Key Mental Model

The easiest way to remember the algorithms:

```text theme={null}
"Draw a line"
        ↓
Linear Regression


"Draw a line, but control huge weights"
        ↓
Ridge


"Draw a line and remove useless features"
        ↓
Lasso


"Predict probability of a class"
        ↓
Logistic Regression


"Look at nearby examples"
        ↓
KNN


"Ask a sequence of questions"
        ↓
Decision Tree


"Ask many decision trees and vote"
        ↓
Random Forest


"Find the best separating boundary"
        ↓
SVM


"Calculate probabilities using Bayes"
        ↓
Naive Bayes
```

The most important evaluation distinction is:

```text theme={null}
Classification
    ↓
Accuracy / Precision / Recall / F1 / ROC-AUC
    ↓
Confusion Matrix

Regression
    ↓
MAE / MSE / RMSE / R²
```