What are the Models of Self-Supervised Learning

 

Self-Supervised Learning (SSL) is a machine learning process where the model trains itself to learn one part of the input from another part. It is also known as Unsupervised Representation Learning and is a subset of Unsupervised Learning.

The idea behind this Learning approach is to generate supervisory signals by making sense of the unlabeled data provided to it in an unsupervised fashion on the first iteration. Then, the model uses the high confidence data labels among those generated to train the model in the next iterations like any other supervised learning model via backpropagation.

SSL is particularly valuable for LLMs, computer vision, speech recognition, medical imaging, and other domains where huge amounts of unlabeled data are available but labeled data are expensive.

In this article, we are going to explore different models of Self-Supervised Learning that can applied on the data.

Autoencoder (AE)

An autoencoder (AE) is a neural network that learns to represent input data in a compact form without requiring manually labeled data. In self-supervised setups, the model is trained to solve a pretext task—like reconstructing corrupted or masked input.

The architecture behind Autoencoder (AE) consists of following three main components—

a) Encoder:

Takes the input data and reduces it into smaller form while preserving essential information. It has three layers which are:

• Input Layer: original data enters the network.
• Hidden Layers: These layers apply transformations on the input data, using weights and activation functions to capture important patterns
• Output: The encoder outputs a compressed vector known as encoding

b) Bottleneck (Latent Space):

This is the smallest layer in network that holds a compressed representation of the input data. It forces the model to focus on the most important features

c) Decoder:

It takes the compressed representation from the latent space and recreates it back into the original data form.

Generative Pre-trained Transformer (GPT) models

GPT models belong to the class of large language model (LLM) that is widely used in generative artificial intelligence chatbots. These models are based on the transformer architecture and are pre-trained on large datasets of unlabeled content, enabling them to generate novel content.

The transformer architecture is the core technology of a GPT and had solved many of the performance issues that were associated with older recurrent neural network (RNN) designs for natural language processing (NLP).

GPT models use a task called next-token prediction in Self-Supervised Learning where the model learns to predict the probability of the next token. The Learning pipeline behind Generative Pre-trained Transformer Model is as follows--

1. The process starts with the large collection of text which is unlabeled data.
2. Then, the text is converted to tokens.
3. Next, create training examples.
4. GPT converts each token into a vector called an embedding.
5. The embeddings are passed through multiple Transformer layers.
6. The attention mechanism in GPT allows the model to examine relationships between tokens.
7. The Transformer produces a representation for each position. A final linear layer converts that representation into scores for the vocabulary.
8. Then the model's prediction is compared with the actual next token. Here, the cross-entropy loss is used where the model is encouraged to increase the probability of the correct next token.
9. After calculating the loss, Backpropagation process is applied for updating the model parameters.

Bidirectional Encoder Representations from Transformers (BERT)

BERT is a language model introduced by Google where the key idea is that the model learns from unlabeled text by hiding some words and trains itself to predict the hidden words. This model was primarily designed to understand text. 

In Self-Supervised Learning, BERT is a self-supervised Transformer encoder that learns contextual language representations by predicting masked tokens from their surrounding context. The model is being trained on basis of the following tasks—

a) Masked Language Modeling (MLM):

• BERT randomly hides 15% of tokens in a sentence and learns to predict the hidden words based on their surrounding context.

b) Next Sentence Prediction (NSP):

• BERT learns to determine whether two sentences logically follow each other.
• This helps the model understand relationships between sentences, which is useful for tasks like question answering or document summarization.

Bootstrap Your Own Latent (BYOL)

BYOL is a new approach to Self-Supervised Learning that enhances image representation learning. The key idea behind BYOL is that it learns useful representations without negative examples or labeled data.

The BYOL architecture is built on basis of below Neural Networks both of which learn and interact with each other—

a) Online Network:

• Predicts the target network's representation of an image under different augmented views
• Contains the below stages—
  > Encoder
  > Projector
  > Predictor

b) Target Network:

• It updates with slow moving average of the online network
• Contains the below stages—
  > Encoder
  > Projector

Self-Distillation with No Labels (DINO)

DINO is a vision-specific self-supervised learning framework introduced by Meta AI researchers primarily built for learning powerful visual representations without labeled data. 

The key idea behind the Model is that a student network learns to match the output of a teacher network using different augmented views of the same image. It is particularly valuable in scenarios where labeled datasets are scarce or expensive to create.

DINO employs a student-teacher architecture that has the following networks—

a) Student Network:

• This network learns through backpropagation
• It has following major components:
  > Encoder: Extracts meaningful features from the image.
  > Projection Head: Transforms the encoder representation into a space where the self-supervised learning objective is applied.

b) Teacher Network:

• This network is updated using the student's parameters through an Exponential Moving Average (EMA).
• It has following components:
  > Encoder: Extracts features from its input image.
  > Projection Head: In this component, the teacher output becomes the target that the student tries to reproduce.

Conclusion

The models discussed above demonstrate the remarkable ability of Self-Supervised Learning to learn without relying heavily on human-labeled data. Whether reconstructing information, predicting tokens, or comparing different representations, they offer powerful approaches. But how far can machines go when they learn primarily from the data itself?

What are the Models of Reinforcement Learning

 

Reinforcement learning (RL) is concerned with how the machine learning models can train agents to make decisions in an environment by performing actions and monitoring the results. For every good action, the agent gets rewards, and for every bad action, the agent gets penalties. In short, RL models learn by trial and error to make better decisions in the future.

There are 5 key components in Reinforcement learning:--

a) Agent: An entity which performs action
b) Environment: The world or system in which the agent operates.
c) State: The situation or condition the agent is currently in.
d) Action: The different choices that the agent can make
e) Reward: The result from the environment based on the agent’s action.

This Machine Learning category can be implemented by following the below approaches--

• Value-Based Models
• Policy-Based Models
• Model-Based Reinforcement Learning
• Deep Reinforcement Learning Models

In this article, we will talk about the functionality of the Models used in each of the above categories.

Value-Based Models

This RL category focuses on learning the value of states or state-action pairs to derive an optimal policy. The goal is to maximize the value function, which represents the long-term cumulative reward.

Q-Learning

Q-Learning is a model free RL algorithm that helps an agent learn how to make the best decisions by interacting with the environment. “Q” stands for Quality which represents how valuable the action is in maximizing future rewards. 

The working of this model is through a set of key components as illustrated in the diagram below:

The functionality of these components can be explained as follows—

a) Agent: A learner or decision maker who observes the environment and takes actions.

b) Action: The move that the agent can make based on a specific strategy.

c) Environment: Defines the problem's rules and determines the outcome of each action.

d) State: Represents the agent's current condition or situation.

SARSA

SARSA stands for State-Action-Reward-State-Action, which helps an agent to learn an optimal policy by interacting with the environment. Being the modified version of the Q-learning, it learns from the actual action taken by agent.

The main idea behind SARSA is trial and error. The agent takes an action in a situation, observes the result, and adjusts its strategy based on the outcome. This process is repeated many times, leading to improvements in the agent’s decisions over time.

The Update Equation rule used in SARSA is—

Where:

• Q(s,a) = Current estimate of action value
• α = Learning rate
• γ = Discount factor
• r = Immediate reward
• Q(s',a') = Value of the next state-action pair actually chosen

Policy-Based Models

The models in this segment directly learn and optimize the policy mapping states to actions, often using gradient-based methods for continuous or stochastic action spaces.

REINFORCE

REINFORCE is a policy gradient algorithm based on Monte Carlo methods, which directly learns a policy by adjusting its parameters to maximize the expected cumulative reward. Introduced by Ronald Williams in 1992, it is considered as the foundation of modern policy-based RL methods.

The key working steps involved in this algorithm are as follows:--

a) Collect Episodes: The agent interacts with the environment, generating trajectories of states, actions, and rewards.
b) Calculate Returns: For each time step, compute the discounted cumulative reward (return) from that point onward.
c) Policy Gradient Update: Update the policy parameters using the gradient of the log-probability of actions, weighted by the returns.
d) Repeat: Iterate over multiple episodes to refine the policy.

Vanilla Policy Gradient (VPG)

VPG is one of the most fundamentally pure algorithms in policy-based Reinforcement Learning. This model operates by explicitly modeling the policy and updating its parameters in the direction that improves long-term performance.

The working process of the VPG algorithm is as follows—

a) Initialize Policy: Start with a random policy network.

b) Generate Episodes: Interact with the environment and collect states, actions and rewards.

c) Compute Returns: Calculate cumulative future rewards.

d) Compute Policy Gradient: Estimate the gradient using collected experiences.

e) Update Policy: Adjust policy parameters to increase expected rewards.

f) Repeat: Continue until the policy converges.

Model-Based Reinforcement Learning

In this type of Reinforcement Learning, the agent uses the model of environment to predict what will happen after taking actions, and then uses those predictions to choose better actions.

Dyna-Q

Developed by Richard Sutton in 1990, the core idea behind Dyna-Q is to combine real experience with simulated experience generated by a learned model. When the agent interacts with the environment, it stores:

• Estimated value of each state-action 
• What it has learned about the environment

The key steps involved in the working process of Dyna-Q algorithm are as follows—

a) Initialize the Q-table and an empty model (which will store transitions).

b) Take an action in the environment → observe reward and next state.

c) Real Experience Update: Update the Q-value using the Q-learning rule.

d) Update the Model: Record the observed transition

e) Planning Step:

• Randomly sample a few previously observed state-action pairs.
• Use the model to simulate what would happen next.
• Apply the same Q-learning update using the simulated data.

f) Repeat steps b)–e) until convergence.

World Models

This approach involves creating a model that simulates the environment's dynamics using past experiences. It often employs recurrent neural networks (RNNs) to learn the transition function f(s,a) from state s and action a.

World Models has following categories—

a) Explicit models: The model directly predicts environmental quantities. This is useful when the state is well defined.
b) Latent models: The model predicts future latent representations, rather than reconstructing the complete environment.
c) Probabilistic models: Instead of predicting one future, the model represents uncertainty. This is useful when the environment is stochastic.

Deep Reinforcement Learning Models

Deep Reinforcement Learning (Deep RL) combines reinforcement learning (RL) with deep neural networks. It enables an agent to learn how to make decisions directly from complex data such as images, sensor readings, or large state spaces.

Deep Q Network (DQN)

DQN in reinforcement learning is a combination of deep neural networks and Q-learning that enables agents to learn optimal policies in complex environments. Unlike tabular Q-learning, which stores Q-values in a table, DQN learns a function approximation.

The core components used in DQN architecture are as follows—

a) Input Layer: receives state information from the environment in the form of numerical values.
b) Hidden Layers: consist of multiple fully connected neuron that transform the input data into more complex features that are more suitable for predictions.
c) Output Layer: Each possible action in the current state is represented by a single neuron in the DQN's output layer. The output values of these neurons represent the estimated value of each action within that state.

d) Memory: DQN utilizes a memory replay to store the training events of the agent.

e) Loss Function: the DQN computes the difference between the actual Q-values form replay memory and predicted Q-values to determine loss.

f) Optimization: It involves adjusting the network's weights in order to minimize the loss function.

Proximal Policy Optimization (PPO)

PPO is a policy optimization algorithm that trains an agent to make better decisions by directly improving its policy (the mapping from states to actions). The key idea is to update the policy gradually and prevent it from changing too much in a single training step.

The training cycle behind PPO is as follows—

a) Agent interacts with environment.
b) Collect trajectories (state, action, reward).
c) Compute advantages (how good an action was).
d) Update policy network using PPO objective.
e) Repeat until convergence.

Conclusion

Overall, Reinforcement Learning is a powerful Machine Learning technique that allows agents to learn from rewards and penalties. The models discussed in this article demonstrate different ways of achieving effective decision-making.

What are the Models of Semi-Supervised Learning

 

Semi-Supervised Learning (SSL) combines both supervised and unsupervised learning by using small amount of labelled data and huge amount of unlabelled data for improving model performance. This approach is beneficial when the labelled data is time-consuming.

Initially, the SSL Models find insights in the labelled data, and then further refine their understanding using unlabelled data. During their usage, they carry the following benefits—

• Cost-effective: Reduces the need for manual labeling.
• Scalable: Handles large datasets.
• Efficient: Achieves better generalization compared to using small, labelled dataset alone.

In this post, we will understand different techniques used in Semi- Supervised Learning (SSL), explaining how they are applied on the data.

Self-Training

The basic idea is to first train the model on labelled data and then predict labels for unlabelled data. Then the predictions with high confidence level are selected and added back to the training set.

Working Process:

1. Begin with small set of labelled data.
2. Train the supervised model.
3. Predict on the unlabelled dataset.
4. Add high-confidence predictions to the labelled set.
5. Repeat

Pros:

• Easy to implement.
• Works efficiently with most classifiers.

Cons:

• Requires a mechanism for confidence scoring.

Co-Training

This machine learning technique works best for Classification problems where the core idea is to use different views on classified data. Every view indicates another angle of same data. Here, two individual classifiers are trained based on two views of data to improve learning performance.

Working process:

1. Separate features into two distinct views.
2. Train two classifiers.
3. Each classifier labels the most confident instances for the other.

Co-training is useful when the data lends itself to multiple views that provide complementary information, such as medical images and clinical data paired to the same patient. This approach helps in reducing the risk of reinforcing incorrect predictions, as the two models can correct each other.

Generative Models

The models in this machine learning approach combine the benefits of both labelled and unlabelled data. The idea is to characterise the distribution of data. Once the distribution is known, the model can recreate data and judge how likely it belongs to any particular class.

Generative Models involves the following methods—

A. Generative adversarial networks (GANs):

• Designed to imitate the structure and functionality of a human brain.
• Consists of below neural networks:
  

  > Generator- Takes a random noise vector as input and produces a synthetic data point
  > Discriminator- Evaluates the generated data point and tells the generator whether it's real or not.

• The adversarial process between Generator and Discriminator continues, with each model striving to outperform the other.

B. Variational Auto Encoders (VAEs):

• An artificial neural network architecture designed to capture the underlying probability distribution of a given dataset and generate novel samples.
• It is part of the families of probabilistic graphical models and variational Bayesian methods.

Graph-Based Models

In Graph‑Based Semi‑Supervised Learning, data points are treated as nodes and edges represent similarities, allowing labels to propagate through the graph based on structural connections. The main idea is to represent the entire dataset as a graph, then use the relationships between similar data points to propagate the known labels to unlabeled points.

Working process:

1. Construct a graph based on data similarities.
2. Propagate labels through the graph.
3. Assign labels to unlabeled nodes based on their connections.

Consistency-Based Models

The models in this approach focus on ensuring that the predictions remain consistent across different inputs. This is achieved through techniques such as consistency regularization, which trains the model to be robust to small disturbances of its inputs and parameters.

The main idea behind these Models is that they produce consistent predictions for the same unlabeled example even when the input is slightly changed.

Pros:

• Makes effective use of large unlabeled data.
• Improves robustness to noise.
• Can be combined with pseudo-labeling.
• Reduces dependence on expensive labelled datasets.

Semi-Supervised Support Vector Machines

This technique is an extension of Support Vector Machine (SVM) that includes both labeled and unlabeled data during the training process. The core idea is to use labelled data to find a good decision boundary, while using unlabelled data to place the boundary in a low-density region between classes.

Pros:

• Can achieve high performance when labelled data is expensive.
• Works well when the low-density assumption is appropriate.
• Extends the powerful margin-based idea of SVMs to semi-supervised settings.

Conclusion

Overall, Semi-Supervised Learning represents an important area of machine learning because it enables models to learn effectively from both labelled and unlabelled data. As industries continue to generate massive amounts of unlabeled data, SSL techniques can help transform the data into useful knowledge while reducing labelling costs and improving scalability, efficiency, and generalization.

What are the Models of UnSupervised Learning

 

In Unsupervised Learning, the model understands the data without any supervision. The techniques applied in this Learning cluster unlabeled data based on similarities and differences.

This Machine Learning category follows the below working process—

1. Collect Unlabeled Data
2. Select an Algorithm
3. Train the Model on Raw Data
4. Group or Transform Data
5. Interpret and Use Results

Unsupervised Machine Learning can be broken down into following categories—

• Clustering
• Association rules
• Dimensionality reduction.

Each of these categories consists of models whose functionality we will discuss in this article

Clustering

These algorithms find the relationship patterns among data samples and then cluster those samples into groups having similarity based on features. Most of the industries use clustering models, from airlines to healthcare and beyond.

K-Means Clustering

• It is also called as Flat Clustering algorithm.
• This algorithm divides the dataset into K clusters based on feature similarity.
• The objective is to group similar data points while keeping different groups separately.
• K-Means Clustering follows the below working process:

a)     Choosing Number of Clusters (k): The first step is to decide total clusters (k) you want to create. Choosing k is essential for meaningful clustering.

b)     Initial Partitioning: After k is chosen, the model partitions the data points into k subsets.

c)     Computing Centroids: After the initial partition, the model determines the centroid for each cluster.

d)     Reassigning Points: Each data point is assigned to the cluster whose centroid is nearest, typically with the help of Euclidean distance.

e)     Iteration Until Convergence: Steps c) and d) are repeated until the cluster assignments no longer change

Hierarchical Clustering

• Groups similar data points into hierarchy of clusters, allowing us to explore data at multiple levels of granularity.
• The output is represented in the form of a tree called Dendrogram.
• Unlike K-Means, it does not require preselecting the number of clusters.
• Follows the below main approaches—

a)     Agglomerative (Bottom-Up):

>  Also called AGNES (Agglomerative Nesting)

>  This approach starts with taking all data points as single clusters and merging them until one cluster is left.

b)     Divisive (Top-Down):

>  Also called DIANA (Divisive Analysis)

>  All the data points are treated as one big cluster and the process of clustering involves dividing one big cluster into various small clusters.

DBSCAN

• DBSCAN, which stands for Density-Based Spatial Clustering of Applications with Noise is a density-based clustering algorithm that groups together data points that are closely packed while marking points in low-density regions as outliers.
• This Clustering method uses 2 important parameters—

a)     Epsilon (ε): The max distance within which two data points are considered neighbors.

b)     MinPts: The min number of neighboring points required to form a dense region (cluster).

• DBSCAN algorithm classifies the data points into 3 types—

a)     Core point: Have sufficient number of neighbors within a given radius.

b)     Border point: Close to the core point but does not meet the density requirement themselves.

c)     Noise point: Which are isolated and do not belong to any cluster.

 

Association Rules

This Learning technique finds relationships, patterns or associations between variables in large datasets. It is commonly used in Market Basket Analysis for identifying products that are most frequently purchased.

Apriori Algorithm

• It was introduced by Rakesh Agrawal and Ramakrishnan Srikant in 1994.
• Finds frequent itemsets in a transaction database and generate association rules based on those itemsets.
• This algorithm follows the principle— “If an itemset is frequent, then all its subsets must also be frequent.”
• The key metrics in this algorithm are--

a)     Support: measures how frequently an item or item-set appears in the dataset relative to the total transactions.

b)     Confidence: measures the likelihood that item Y is purchased when item X is purchased.

c)     Lift: measures how likely the two items are purchased together compared to random chance.

FP (Frequent Pattern)-Growth Algorithm

• This technique is an improvement to Apriori algorithm, since it efficiently discovers frequent itemsets in transactional databases without generating candidate itemsets.
• FP Tree in this algorithm is a tree data structure created from the transaction data while generating frequent itemsets.
• FP-Growth algorithm works in the following manner:

a)     First, it compresses the input database creating an FP-tree instance to represent frequent items.

b)     Then, it divides the compressed database into a set of conditional databases, each associated with one consistent pattern.

c)     Finally, each of these databases are mined separately.

Dimensionality Reduction 

Dimensionality reduction reduces the input features in a dataset while preserving important information. It transforms high-dimensional data into a lower-dimensional data for simpler representation.

Principal Component Analysis (PCA) 

• PCA was introduced by Karl Pearson in 1901 as a statistical method to analyze data variation and relationships.
• This technique transforms large set of correlated features into a smaller set of uncorrelated features called principal components, while retaining as much information as possible.
• The working process of this technique is as follows—

a)     Standardize the data

b)     Compute the covariance matrix

c)     Compute the eigenvectors and eigenvalues of the covariance matrix

d)     Select the principal components

e)     Project the data onto the new feature space

t-SNE

• t-SNE (T-distributed Stochastic Neighbor Embedding)  is a non-linear dimensionality reduction technique used for visualizing high-dimensional data in a lower-dimensional space mainly in 2D or 3D.
• This technique was introduced by Laurens van der Maaten and Geoffrey Hinton in 2008.
• Unlike PCA, which preserves overall variance, t-SNE focuses on preserving the local relationships between data points.

Conclusion

As industries continue to generate large amounts of unlabeled data, Unsupervised Learning becomes increasingly important across various areas such as healthcare, finance, retail, cybersecurity, and manufacturing. The models covered in this article provide the foundation for extracting actionable insights, improving decision-making, and enabling data-driven innovation without relying on manually labeled datasets.

What are the Models of Supervised Learning

 

In Supervised Learning, the Model is trained using labeled data, meaning the input data is paired with the correct output. The objective is to learn an association between input data samples and corresponding outputs after performing multiple training data instances.

The process involved in this Machine Learning category is as follows—

1. Collecting Labeled Data: Each data point includes an input (features) and the correct output (label).
2. Splitting the Data: The dataset is usually divided into a training set and a test set.
3. Training the Model: The algorithm learns from the training set by minimizing errors between predicted and actual labels.
4. Evaluating Performance: The model is tested on unseen data to measure accuracy.

There are 2 main types of Models in Supervised Learning—

    1. Classification:

• Used when the target variable is a category or class.
• Model learns from labeled data and predicts which class a new observation belongs to.

          2. Regression:

• Used when the target variable is a continuous numerical value.
• Model predicts a quantity rather than a category.

CLASSIFICATION MODELS

A. Logistic Regression

This model is primarily used for binary classification tasks that can help answer questions like—

• Is this email spam or not?
• Will this customer buy the product or not?
• Does this patient have the disease or not?

At its core, Logistic Regression predicts the probability that a given observation belongs to a particular class, and its probability score varies between 0 and 1.

Logistic Regression derives its name from the logistic/sigmoind function, which transforms a linear combination of input features into probabilities. This enables the algorithm to handle classification tasks effectively while remaining interpretable and computationally efficient.

The process workflow behind this Model is as follows—

a)     Collect training data.

b)     Compute a linear combination of input features:

c)     Apply the sigmoid function:

The sigmoid function converts any real value into a probability between 0 and 1.

d)     Make a prediction:

• If probability ≥ 0.5 → Class 1
• If probability < 0.5 → Class 0

B. Decision Tree

Decision tree is a hierarchical tree-based model that is used to classify or predict outcomes based on a set of rules. It consists of mainly 3 parts—

a) Root Node: Represents the entire dataset and the first feature used for splitting.

b) Decision Nodes: Internal nodes where the data is split based on feature values.

c) Leaf Nodes: Final nodes that represent the predicted class.

While there are various algorithms in Machine Learning, Decision Tree is used because of below reasons—

• Decision Trees usually mimic human thinking ability while making a decision, so it is easy to understand.
• The logic behind the decision tree can be easily understood because it shows a tree-like structure.

C. Random Forest

This classification model uses an ensemble of decision trees to make predictions. The algorithm was first introduced by Leo Breiman and Adele Cutler in 2001 with the key idea of creating a large number of decision trees, each of which is trained on a different subset of the data.

Below are some points that explain why Random Forest algorithm is used:

• It takes less training time than other algorithms.
• It predicts output with high accuracy, even for the large dataset it runs efficiently.
• It can also maintain accuracy when a large proportion of data is missing.

The areas where this model can be applied are as follows—

• Customer churn prediction
• Fraud detection
• Stock price prediction
• Medical diagnosis
• Image recognition

D. K-Nearest Neighbors (KNN)

KNN is a lazy learning algorithm, meaning it does not build a model during training. Instead, it stores the training data and performs computations only when making predictions.

In this model, K is just a number that tells the algorithm how many nearby points or neighbors to look at when it makes a decision.

KNN algorithm works in the following manner—

a)     Choose the value of K (the number of nearest neighbors).

b)     Calculate the distance between the new data point and all training data points.

c)     Select the K closest neighbors.

d)     For classification, assign the class with the majority vote among the neighbors.

The areas where this model is applied are—

• Recommendation Systems
• Spam Detection
• Customer Segmentation
• Speech Recognition

REGRESSION MODELS

A. Linear Regression

Defined as the statistical model, it analyzes the linear relationship between a dependent variable and a given set of independent variables. This model makes predictions for continuous/real or numeric variables such as sales, salary, age, and product price.

The linear regression model provides a sloped straight line representing the relationship between the variables. Consider the image below:

Mathematically, linear regression can be represented as:

y= a0+a1x+ ε

Here,

Y= Dependent Variable (Target Variable)

X= Independent Variable (predictor Variable)

a0= intercept of the line (Gives an additional degree of freedom)

a1 = Linear regression coefficient (scale factor to each input value).

ε = random error

The values for x and y variables are training datasets for Linear Regression model representation.

Linear Regression model can be classified as follows—

a) Simple Linear Regression:

• Uses one independent variable.
• Example: Predicting salary based on years of experience.

            b) Multiple Linear Regression:

• Uses two or more independent variables.
• Example: Predicting house prices using area, number of bedrooms, and age of the house.

B. Decision Tree Regressor

The goal of this model is to predict continuous values such as prices or scores using a tree-like structure. Unlike linear regression, decision trees partition the feature space in a hierarchical, rule-based way that enables them to capture complex, non-linear relationships.

This model continuously splits data into subsets, based on the features that result in the lowest prediction error, forming a tree-like structure where:

• Each internal node represents a decision rule on a feature.
• Each branch represents the outcome of a decision.
• Each leaf node provides the predicted value

Decision Tree Regressors use one of the following criteria for splitting the data:

• Mean Squared Error (MSE) – Measures the average squared difference between actual and predicted values.
• Mean Absolute Error (MAE) – Measures the average absolute difference between actual and predicted values.
• Friedman MSE – A variant of MSE commonly used in gradient boosting.

C. Random Forest Regressor

It is an ensemble learning method that combines multiple decision trees to produce more accurate and stable predictions. Unlike Decision Tree Regressor, which relies on a single tree, this model builds many decision trees and combines their predictions by averaging.

Random Forest Regressor works in the following manner:

1. Draw multiple bootstrap samples (random samples with replacement) from the training dataset.
2. Train a Decision Tree Regressor on each sample.
3. At each split, each tree considers only a random subset of features.
4. Each tree predicts a numerical value for a new data point.
5. The final prediction is the average of all tree predictions.

The areas where this model can be applied are:

• House price prediction
• Sales forecasting
• Demand forecasting
• Stock market prediction
• Energy consumption forecasting
• Weather prediction

CONCLUSION

The study of supervised learning demonstrates how different algorithms can be applied to solve a wide range of business and real-world problems. Classification models help organizations make informed decisions, while regression models support forecasting and trend analysis. Selecting the most suitable algorithm based on data characteristics significantly improves prediction accuracy and overall model performance.