What is Class Imbalance?  

Class imbalance occurs when the number of samples in one class (called the majority class) significantly outnumbers those in another class (called the minority class) in classification problems. This imbalance can cause models to be biased towards predicting the majority class, resulting in poor detection of minority class cases, which are often the critical ones (e.g., detecting fraud or diagnosing rare diseases).

Why is it Important?  

If imbalance is not handled, models may:

• Achieve high overall accuracy, but fail to detect minority class cases
• Produce many false negatives (missed detections), which can be dangerous in fields like healthcare
• Have misleading evaluation metrics because accuracy is dominated by the majority class
• Struggle to generalize well on new data

Techniques to Handle Imbalance 

Class imbalance handling refers to a set of techniques used to deal with classification problems where one class (the majority) has significantly more samples than another (the minority).

Oversampling

Oversampling is used to address the issue of class imbalance in datasets—particularly common in binary classification problems where one class significantly outnumbers the other.

In an imbalanced dataset, the majority class dominates, which can lead machine learning models to be biased toward predicting that class more often, ignoring the minority class entirely. This results in poor recall and precision for the minority class, which is often the class of interest.

What does oversampling do?

Oversampling increases the representation of the minority class by adding more samples, either by duplicating existing examples or generating new synthetic ones. The goal is to balance the class distribution, so the model receives equal learning exposure to both classes.

Common techniques:

• SMOTE (Synthetic Minority Over-sampling Technique): Creates synthetic minority class samples by interpolating between existing minority examples.
• RandomOverSampler: Simply duplicates minority class samples randomly.
• BorderlineSMOTE: Generates synthetic samples only near the decision boundary where classes overlap.
• ADASYN: Similar to SMOTE but focuses more on creating samples for harder-to-learn minority cases.

Example:

Suppose you have a fraud detection dataset:

Using SMOTE, you generate synthetic fraud cases to increase minority class samples from 200 to 10,000. This balanced dataset helps the model better learn fraud patterns, reducing missed fraud cases (false negatives) and improving detection rates.

Undersampling

Undersampling is a technique used to handle class imbalance in machine learning datasets by reducing the size of the majority class so that it is comparable to the size of the minority class. This helps create a more balanced dataset, ensuring that the model pays equal attention to both classes during training.

In a typical imbalanced scenario, one class (usually the one we care less about) dominates the dataset. For example, in an email classification task, the number of “non-spam” emails might vastly outnumber the “spam” ones. Without balancing, a model might learn to predict only the majority class to optimize accuracy, while completely neglecting the minority class.

What does undersampling do? 

Undersampling reduces the skewed distribution by randomly or strategically removing samples from the majority class, thus shrinking it to match or get closer to the size of the minority class. This forces the model to learn more equally from both classes, which can improve its performance on the minority class.

Common techniques:

• RandomUnderSampler: Randomly removes samples from the majority class.
• TomekLinksUnderSampler: Removes majority samples that are very close to minority samples (cleaning noisy overlaps).
• EditedNNUnderSampler: Removes majority samples misclassified by nearest neighbors, reducing noise.
• NearMissUnderSampler: Keeps majority samples that are close to minority samples (focuses on difficult boundary cases).

Example:

In a medical diagnosis dataset:

Using RandomUnderSampler, the healthy samples are reduced from 5,000 to 300 to match the disease samples. This helps the model avoid bias towards the healthy class and better detect the disease.