What are the Models of Semi-Supervised Learning

 

Semi-Supervised Learning (SSL) combines both supervised and unsupervised learning by using small amount of labelled data and huge amount of unlabelled data for improving model performance. This approach is beneficial when the labelled data is time-consuming.

Initially, the SSL Models find insights in the labelled data, and then further refine their understanding using unlabelled data. During their usage, they carry the following benefits—

• Cost-effective: Reduces the need for manual labeling.
• Scalable: Handles large datasets.
• Efficient: Achieves better generalization compared to using small, labelled dataset alone.

In this post, we will understand different techniques used in Semi- Supervised Learning (SSL), explaining how they are applied on the data.

Self-Training

The basic idea is to first train the model on labelled data and then predict labels for unlabelled data. Then the predictions with high confidence level are selected and added back to the training set.

Working Process:

1. Begin with small set of labelled data.
2. Train the supervised model.
3. Predict on the unlabelled dataset.
4. Add high-confidence predictions to the labelled set.
5. Repeat

Pros:

• Easy to implement.
• Works efficiently with most classifiers.

Cons:

• Requires a mechanism for confidence scoring.

Co-Training

This machine learning technique works best for Classification problems where the core idea is to use different views on classified data. Every view indicates another angle of same data. Here, two individual classifiers are trained based on two views of data to improve learning performance.

Working process:

1. Separate features into two distinct views.
2. Train two classifiers.
3. Each classifier labels the most confident instances for the other.

Co-training is useful when the data lends itself to multiple views that provide complementary information, such as medical images and clinical data paired to the same patient. This approach helps in reducing the risk of reinforcing incorrect predictions, as the two models can correct each other.

Generative Models

The models in this machine learning approach combine the benefits of both labelled and unlabelled data. The idea is to characterise the distribution of data. Once the distribution is known, the model can recreate data and judge how likely it belongs to any particular class.

Generative Models involves the following methods—

A. Generative adversarial networks (GANs):

• Designed to imitate the structure and functionality of a human brain.
• Consists of below neural networks:
  

  > Generator- Takes a random noise vector as input and produces a synthetic data point
  > Discriminator- Evaluates the generated data point and tells the generator whether it's real or not.

• The adversarial process between Generator and Discriminator continues, with each model striving to outperform the other.

B. Variational Auto Encoders (VAEs):

• An artificial neural network architecture designed to capture the underlying probability distribution of a given dataset and generate novel samples.
• It is part of the families of probabilistic graphical models and variational Bayesian methods.

Graph-Based Models

In Graph‑Based Semi‑Supervised Learning, data points are treated as nodes and edges represent similarities, allowing labels to propagate through the graph based on structural connections. The main idea is to represent the entire dataset as a graph, then use the relationships between similar data points to propagate the known labels to unlabeled points.

Working process:

1. Construct a graph based on data similarities.
2. Propagate labels through the graph.
3. Assign labels to unlabeled nodes based on their connections.

Consistency-Based Models

The models in this approach focus on ensuring that the predictions remain consistent across different inputs. This is achieved through techniques such as consistency regularization, which trains the model to be robust to small disturbances of its inputs and parameters.

The main idea behind these Models is that they produce consistent predictions for the same unlabeled example even when the input is slightly changed.

Pros:

• Makes effective use of large unlabeled data.
• Improves robustness to noise.
• Can be combined with pseudo-labeling.
• Reduces dependence on expensive labelled datasets.

Semi-Supervised Support Vector Machines

This technique is an extension of Support Vector Machine (SVM) that includes both labeled and unlabeled data during the training process. The core idea is to use labelled data to find a good decision boundary, while using unlabelled data to place the boundary in a low-density region between classes.

Pros:

• Can achieve high performance when labelled data is expensive.
• Works well when the low-density assumption is appropriate.
• Extends the powerful margin-based idea of SVMs to semi-supervised settings.

Conclusion

Overall, Semi-Supervised Learning represents an important area of machine learning because it enables models to learn effectively from both labelled and unlabelled data. As industries continue to generate massive amounts of unlabeled data, SSL techniques can help transform the data into useful knowledge while reducing labelling costs and improving scalability, efficiency, and generalization.