What are the Models of Supervised Learning

 

In Supervised Learning, the Model is trained using labeled data, meaning the input data is paired with the correct output. The objective is to learn an association between input data samples and corresponding outputs after performing multiple training data instances.

The process involved in this Machine Learning category is as follows—

1. Collecting Labeled Data: Each data point includes an input (features) and the correct output (label).
2. Splitting the Data: The dataset is usually divided into a training set and a test set.
3. Training the Model: The algorithm learns from the training set by minimizing errors between predicted and actual labels.
4. Evaluating Performance: The model is tested on unseen data to measure accuracy.

There are 2 main types of Models in Supervised Learning—

    1. Classification:

• Used when the target variable is a category or class.
• Model learns from labeled data and predicts which class a new observation belongs to.

          2. Regression:

• Used when the target variable is a continuous numerical value.
• Model predicts a quantity rather than a category.

CLASSIFICATION MODELS

A. Logistic Regression

This model is primarily used for binary classification tasks that can help answer questions like—

• Is this email spam or not?
• Will this customer buy the product or not?
• Does this patient have the disease or not?

At its core, Logistic Regression predicts the probability that a given observation belongs to a particular class, and its probability score varies between 0 and 1.

Logistic Regression derives its name from the logistic/sigmoind function, which transforms a linear combination of input features into probabilities. This enables the algorithm to handle classification tasks effectively while remaining interpretable and computationally efficient.

The process workflow behind this Model is as follows—

a)     Collect training data.

b)     Compute a linear combination of input features:

c)     Apply the sigmoid function:

The sigmoid function converts any real value into a probability between 0 and 1.

d)     Make a prediction:

• If probability ≥ 0.5 → Class 1
• If probability < 0.5 → Class 0

B. Decision Tree

Decision tree is a hierarchical tree-based model that is used to classify or predict outcomes based on a set of rules. It consists of mainly 3 parts—

a) Root Node: Represents the entire dataset and the first feature used for splitting.

b) Decision Nodes: Internal nodes where the data is split based on feature values.

c) Leaf Nodes: Final nodes that represent the predicted class.

While there are various algorithms in Machine Learning, Decision Tree is used because of below reasons—

• Decision Trees usually mimic human thinking ability while making a decision, so it is easy to understand.
• The logic behind the decision tree can be easily understood because it shows a tree-like structure.

C. Random Forest

This classification model uses an ensemble of decision trees to make predictions. The algorithm was first introduced by Leo Breiman and Adele Cutler in 2001 with the key idea of creating a large number of decision trees, each of which is trained on a different subset of the data.

Below are some points that explain why Random Forest algorithm is used:

• It takes less training time than other algorithms.
• It predicts output with high accuracy, even for the large dataset it runs efficiently.
• It can also maintain accuracy when a large proportion of data is missing.

The areas where this model can be applied are as follows—

• Customer churn prediction
• Fraud detection
• Stock price prediction
• Medical diagnosis
• Image recognition

D. K-Nearest Neighbors (KNN)

KNN is a lazy learning algorithm, meaning it does not build a model during training. Instead, it stores the training data and performs computations only when making predictions.

In this model, K is just a number that tells the algorithm how many nearby points or neighbors to look at when it makes a decision.

KNN algorithm works in the following manner—

a)     Choose the value of K (the number of nearest neighbors).

b)     Calculate the distance between the new data point and all training data points.

c)     Select the K closest neighbors.

d)     For classification, assign the class with the majority vote among the neighbors.

The areas where this model is applied are—

• Recommendation Systems
• Spam Detection
• Customer Segmentation
• Speech Recognition

REGRESSION MODELS

A. Linear Regression

Defined as the statistical model, it analyzes the linear relationship between a dependent variable and a given set of independent variables. This model makes predictions for continuous/real or numeric variables such as sales, salary, age, and product price.

The linear regression model provides a sloped straight line representing the relationship between the variables. Consider the image below:

Mathematically, linear regression can be represented as:

y= a0+a1x+ ε

Here,

Y= Dependent Variable (Target Variable)

X= Independent Variable (predictor Variable)

a0= intercept of the line (Gives an additional degree of freedom)

a1 = Linear regression coefficient (scale factor to each input value).

ε = random error

The values for x and y variables are training datasets for Linear Regression model representation.

Linear Regression model can be classified as follows—

a) Simple Linear Regression:

• Uses one independent variable.
• Example: Predicting salary based on years of experience.

            b) Multiple Linear Regression:

• Uses two or more independent variables.
• Example: Predicting house prices using area, number of bedrooms, and age of the house.

B. Decision Tree Regressor

The goal of this model is to predict continuous values such as prices or scores using a tree-like structure. Unlike linear regression, decision trees partition the feature space in a hierarchical, rule-based way that enables them to capture complex, non-linear relationships.

This model continuously splits data into subsets, based on the features that result in the lowest prediction error, forming a tree-like structure where:

• Each internal node represents a decision rule on a feature.
• Each branch represents the outcome of a decision.
• Each leaf node provides the predicted value

Decision Tree Regressors use one of the following criteria for splitting the data:

• Mean Squared Error (MSE) – Measures the average squared difference between actual and predicted values.
• Mean Absolute Error (MAE) – Measures the average absolute difference between actual and predicted values.
• Friedman MSE – A variant of MSE commonly used in gradient boosting.

C. Random Forest Regressor

It is an ensemble learning method that combines multiple decision trees to produce more accurate and stable predictions. Unlike Decision Tree Regressor, which relies on a single tree, this model builds many decision trees and combines their predictions by averaging.

Random Forest Regressor works in the following manner:

1. Draw multiple bootstrap samples (random samples with replacement) from the training dataset.
2. Train a Decision Tree Regressor on each sample.
3. At each split, each tree considers only a random subset of features.
4. Each tree predicts a numerical value for a new data point.
5. The final prediction is the average of all tree predictions.

The areas where this model can be applied are:

• House price prediction
• Sales forecasting
• Demand forecasting
• Stock market prediction
• Energy consumption forecasting
• Weather prediction

CONCLUSION

The study of supervised learning demonstrates how different algorithms can be applied to solve a wide range of business and real-world problems. Classification models help organizations make informed decisions, while regression models support forecasting and trend analysis. Selecting the most suitable algorithm based on data characteristics significantly improves prediction accuracy and overall model performance.