ML Algorithms in QuickML

ML algorithms are programs that can learn from data and improve from experience, without any external intervention. The following algorithms and operations are all available in QuickML as stages that can be configured in one or more pipeline executions.

The most widely used algorithms in the data science domain are,

1. Classification algorithms
2. Regression algorithms

Classification Algorithms

Classification is the task of predicting a discrete class label. QuickML features several classification algorithms, including:

AdaBoost Classification

This classification begins by fitting a classifier on the original dataset, followed by additional copies of the classifier on the same dataset. The weights of these instances are adjusted according to the error of the current prediction. As such, subsequent classifiers focus more on difficult cases. AdaBoost is a machine-learning algorithm that builds a series of small, one-step (one level) decision trees, adapting each tree to predict difficult cases missed by the previous trees and combining all trees into a single model.

Hyper Parameters:

CatBoost Classification

CatBoost is based on gradient-boosted decision trees. During training, a set of decision trees is built consecutively. Each successive tree is built with reduced loss compared to the previous trees. The number of trees is controlled by the starting parameters.

This classification has much less prediction time compared to others.

Hyper Parameters:

Decision-Tree Classification

Decision tree builds classification or regression models in the form of a tree structure. It breaks down a dataset into smaller and smaller subsets, while at the same time an associated decision tree is incrementally developed.

Decision trees can handle both categorical and numerical data. when predicting the output value of a set of features, it will predict the output based on the subset that the set of features falls into.

Hyper Parameters

GB Classification

Gradient-boosting classification calculates the difference between the current prediction and the known correct target value. This difference is called residual. After finding this value, the gradient-boosting classifier trains a weak model (Decision Tree) that maps features to that residual. This residual predicted by a weak model is added to the existing model input and thus this process nudges the model towards the correct target. Repeating this step multiple times improves the overall model prediction.

Hyper Parameters:

KNN Classification

KNN works by finding the distances between a query (data instance) and all the examples in the data, selecting the specified number examples (K) closest to the query, then voting for the most frequent label in the neighbourhood.

Hyper Parameters:

LGBM Classification

LGBM works by starting with an initial estimate that is updated using the output of each tree. The learning parameter controls the magnitude of this change in the estimates. It can be used on any data and provides a high degree of accuracy, as it contains many built-in preprocessing steps.

The LightGBM algorithm grows vertically, meaning it grows leaf-wise, while other algorithms grow level-wise. LightGBM chooses the leaf with the largest loss to grow. It can lower more loss than a level-wise algorithm when growing the same leaf.

Hyper Parameters:

Logistic Regression

When the target is binary value, we can use logistic classification. It maps the value between 0 and 1.

Hyper Parameters:

Naive Bayes Classification

Naive Bayes is a classifier that uses the Bayes Theorem. It predicts membership probabilities for each class, such as the probability that a given record or data point belongs to a particular class. The class with the highest probability is considered as the most likely class.

Random-Forest Classification

The random forest is a classification algorithm consisting of many decisions trees. It uses bagging and feature randomness when building individual trees to try to create an uncorrelated forest of trees whose prediction by committee is more accurate than that of any individual tree.

A Bagging is an ensemble meta-estimator that fits base classifiers/regressors on random subsets of the original dataset, then aggregates their individual predictions (either by voting or by averaging) to form a final prediction.

Hyper Parameters:

SVM Classification

SVM, or Support Vector Machine, is a linear model for classification and regression problems. It can solve linear and non-linear problems and work well for many practical problems. The idea of SVM is simple: The algorithm creates a line or a hyperplane that separates the data into classes.

Hyper Parameters:

XGB Classification

XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine-learning algorithms under the Gradient Boosting framework. It provides a parallel tree boosting to solve many data science problems quickly and accurately. It uses L1 and L2 regularisation to predict points and it fast in training.

Each model has Fit (to train model), predict (to predict new data), get metrics (to get model’s accuracy and other metrics), and feature_importances (importances of the input features for the prediction).

Adaboost, CatBoost, Decision tree, Gradient boost(GB), LGBM, RandomForest, SVM, and XGB’s basic working principles are almost identical for both regression and classification.

Hyper Parameters:

POSSIBLE VALUES FOR “OBJECTIVE” PARAM :

{binary:logistic, binary:logitraw, binary:hinge, multi:softmax, multi:softprob}