---
title: "Classification Algorithms"
url: https://docs.catalyst.zoho.com/en/quickml/help/ml-algorithms/classification-algorithms/
description: "QuickML is a fully no-code ML pipeline builder service in the Catalyst development platform for creating machine-learning pipelines with end-to-end solutions."
last_updated: 2023-10-09T12:48:15Z
service: QuickML
type: general
---

<!-- ### ML Algorithms
QuickML ML operations are categorized into multiple sections. All of these operations are available as stages in Pipeline Builder as mentioned and the main categories are:
1. **Algorithms**
     - Classification algorithms
     - Regression algorithms
2. **Encoding**
3. **Imputation**
4. **Binning**
5. **Transformers**
6. **Data type detection**
7. **DType transformation**
8. **Feature generation**

All the operations listed above have detailed categories and their own use cases in the ML pipeline environment. Those can be checked in detail in upcoming pipeline articles. -->

# ML Algorithms in QuickML

QuickML mainly focuses on powering ML Pipelines with Machine Learning Operations in an effortless manner to provide smooth pipeline execution environment. Hence, it has been integrated with a wide range of ML algorithms & features to provide the best analytical results out of data. 

ML algorithms are programs that can learn from data and improve from experience, without any external intervention. The following algorithms and operations are all available in **QuickML as stages** that can be configured in one or more pipeline executions.

The most widely used algorithms in the data science domain are,

1. **Classification algorithms**
2. **Regression algorithms**

# Classification Algorithms

Classification is the task of predicting a discrete class label. QuickML features following classification algorithms

1. ### AdaBoost Classification

    AdaBoost is a machine-learning algorithm that builds a series of small, one-step (one level) decision trees, adapting each tree to predict difficult cases missed by the previous trees and combining all trees into a single model.
    This classification begins by fitting a classifier on the original dataset, followed by additional copies of the classifier on the same dataset. The weights of these instances are adjusted according to the error of the current prediction. So that, subsequent classifiers focus more on difficult cases.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
            <tr>
                <th class="w15p">Parameter</th>
                <th class="w35p">Description</th>
                <th class="w15p">Data Type</th>
                <th class="w20p">Possible Values</th>
                <th class="w15p">Default Values</th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td>`base_estimator`</td>
                <td>The base estimator from which the boosted ensemble is built. If none, then the base estimator is `DecisionTreeClassifier` initialized with **max_depth=1**.</td>
                <td>object</td>
                <td>Any classification model except KNN Classification model</td>
                <td>None</td>
            </tr>
            <tr>
                <td>`n_estimators` 
(number of estimators)</td>
                <td>The maximum number of estimators at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early.</td>
                <td>int</td>
                <td>[1, 500]</td>
                <td>50</td>
            </tr>
            <tr>
                <td>`learning_rate`</td>
                <td>Weight applied to each classifier at each boosting iteration. A higher learning rate increases the contribution of each classifier.</td>
                <td>float</td>
                <td>(0.0, +Inf)</td>
                <td>1.0</td>
            </tr>
            <tr>
                <td>`algorithm`</td>
                <td>If ‘SAMME.R’ then use the SAMME.R real boosting algorithm. base_estimator must support calculation of class probabilities. If ‘SAMME’ then use the SAMME discrete boosting algorithm. The SAMME.R algorithm typically converges faster than SAMME, achieving a lower test error with fewer boosting iterations.</td>
                <td>string</td>
                <td>{‘SAMME’, ‘SAMME.R’}</td>
                <td>’SAMME.R’</td>
            </tr>
        </tbody>
    </table>

2. ### CatBoost Classification

    CatBoost is based on gradient-boosted decision trees. During training, a set of decision trees is built consecutively. Each successive tree is built with reduced loss compared to the previous trees. The number of trees is controlled by the starting parameters. 

    This classification has much **less prediction time** compared to others.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
            <tr>
                <th class="w25p">Parameter</th>
                <th class="w25p">Description</th>
                <th class="w15p">Data Type</th>
                <th class="w20p">Possible Values</th>
                <th class="w15p">Default Values</th>
            </tr>
        </thead>
        <tbody>
        <tr>
        <td>`learning_rate`</td>
        <td>Used for reducing the gradient step.</td>
        <td>float</td>
        <td>(0,1]</td>
        <td>0.03</td>
        </tr>
        <tr>
        <td>`l2_leaf_reg` (l2_leaf_regularization)</td>
        <td>Coefficient at the L2 regularization term of the cost function.</td>
        <td>float</td>
        <td>[0,+inf)</td>
        <td>3.0</td>
        </tr>
        <tr>
        <td>`rsm` (random subspace method)</td>
        <td>The percentage of features to use at each split selection, when features are selected over again at random</td>
        <td>float</td>
        <td>(0,1]</td>
        <td>None</td>
        </tr>
        <tr>
        <td>`loss_function`</td>
        <td>The metric to use in training. The specified value also determines the machine learning problem to solve. Some metrics support optional parameters.</td>
        <td>string</td>
        <td>{&#39;Logloss&#39;, &#39;CrossEntropy&#39;, &#39;MultiClass&#39;, &#39;MultiClassOneVsAll&#39;}</td>
        <td>&#39;MultiClass&#39;</td>
        </tr>
        <tr>
        <td>`nan_mode`</td>
        <td>The method for processing missing values in the input dataset.</td>
        <td>string</td>
        <td>{&#39;Forbidden&#39;, &#39;Min&#39;, &#39;Max&#39;}</td>
        <td>Min</td>
        </tr>
        <tr>
        <td>`leaf_estimation_method`</td>
        <td>The method used to calculate the values in leaves.</td>
        <td>string</td>
        <td>{&quot;Newton&quot;, &quot;Gradient&quot;}</td>
        <td>None</td>
        </tr>
        <tr>
        <td>`score_function`</td>
        <td>The score type used to select the next split during the tree construction.</td>
        <td>string</td>
        <td>{L2, Cosine}</td>
        <td>Cosine</td>
        </tr>
        <tr>
        <td>`max_depth`</td>
        <td>Maximum depth of the tree.</td>
        <td>int</td>
        <td>[1,+Inf)</td>
        <td>None</td>
        </tr>
        <tr>
        <td>`n_estimators`
 (number of estimators)</td>
        <td>The maximum number of trees that can be built when solving machine learning problems. When using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter</td>
        <td>int</td>
        <td>[1, 500]</td>
        <td>None</td>
        </tr>
        </tbody>
    </table>

3. ### Decision-Tree Classification

    Decision tree builds classification or regression models in the form of a tree structure. It breaks down a dataset into smaller and smaller subsets, while at the same time an associated decision tree is incrementally developed.

    Decision trees can handle **both categorical and numerical data**. when predicting the output value of a set of features, it will predict the output based on the subset that the set of features falls into.

    #### Hyper Parameters

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w30p">Description</th>
            <th class="w15p">Data Type</th>
            <th class="w20p">Possible Values</th>
            <th class="w10p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`criterion`</td>
            <td>The function to measure the quality of a split.</td>
            <td>string</td>
            <td>{“gini”, “entropy”}</td>
            <td>”gini”</td>
        </tr>
        <tr>
            <td>`splitter`</td>
            <td>The strategy used to choose the split at each node.</td>
            <td>string</td>
            <td>{“best”, “random”}</td>
            <td>”best”</td>
        </tr>
        <tr>
            <td>`max_depth`</td>
            <td>The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.</td>
            <td>int</td>
            <td>(0, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_samples_split`</td>
            <td>The minimum number of samples required to split an internal node.</td>
            <td>int  or float</td>
            <td>[2, +Inf) or (0, 1.0]</td>
            <td>2</td>
        </tr>
        <tr>
            <td>`min_samples_leaf`</td>
            <td>The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches.</td>
            <td>int or float</td>
            <td>[1, +Inf) or (0, 0.5]</td>
            <td>1</td>
        </tr>
        <tr>
            <td>`min_weight_fraction_leaf`</td>
            <td>The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node.</td>
            <td>float</td>
            <td>[0, 0.5]</td>
            <td>0</td>
        </tr>
        <tr>
            <td>`max_features`</td>
            <td>The number of features to consider when looking for the best split</td>
            <td>int, float or string</td>
            <td>(0, n_features] or { “sqrt”, “log2”}</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`max_leaf_nodes`</td>
            <td>Grow a tree with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.</td>
            <td>int</td>
            <td>(1, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_impurity_decrease`</td>
            <td>A node will be split if this split induces a decrease of the impurity greater than or equal to this value.</td>
            <td>float</td>
            <td>[0, +Inf)</td>
            <td>0.0</td>
        </tr>
        </tbody>
        </table>

4. ### GB Classification

    Gradient-boosting classification calculates the difference between the current prediction and the known correct target value. This difference is called residual. After finding this value, the gradient-boosting classifier trains a weak model (Decision Tree) that maps features to that residual. This residual predicted by a weak model is added to the existing model input and thus this process nudges the model towards the correct target. Repeating this step multiple times improves the overall model prediction.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w30p">Description</th>
            <th class="w15p">Data Type</th>
            <th class="w15p">Possible Values</th>
            <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`loss`</td>
            <td>The loss function to be optimized. ‘deviance’ refers to deviance (= logistic regression) for classification with probabilistic outputs.</td>
            <td>string</td>
            <td>{&#39;deviance&#39;, &#39;exponential&#39;}</td>
            <td>&#39;deviance&#39;</td>
        </tr>
        <tr>
            <td>`learning_rate`</td>
            <td>Learning rate shrinks the contribution of each tree by learning_rate. There is a trade-off between learning_rate and n_estimators.</td>
            <td>float</td>
            <td>(0.0, +Inf)</td>
            <td>0.1</td>
        </tr>
        <tr>
            <td>`n_estimators`
 (number of estimators)</td>
            <td>The number of boosting stages to perform.</td>
            <td>int</td>
            <td>[1, 500]</td>
            <td>100</td>
        </tr>
        <tr>
            <td>`criterion`</td>
            <td>The function to measure the quality of a split.</td>
            <td>string</td>
            <td>{&#39;friedman_mse&#39;, &#39;mse&#39;, &#39;mae&#39;}</td>
            <td>’friedman_mse’</td>
        </tr>
        <tr>
        <td>`subsample`</td>
            <td>The fraction of samples to be used for fitting the individual base learners.</td>
            <td>float</td>
            <td>(0.0, 1.0]</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`max_depth`</td>
            <td>The maximum depth of the individual regression estimators.</td>
            <td>int</td>
            <td>(0, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_samples_split`</td>
            <td>The minimum number of samples required to split an internal node</td>
            <td>int  or float</td>
            <td>[2, +Inf) or (0, 1.0]</td>
            <td>2</td>
        </tr>
        <tr>
            <td>`min_samples_leaf`</td>
            <td>The minimum number of samples required to be at a leaf node.</td>
            <td>int or float</td>
            <td>[1, +Inf) or (0, 0.5]</td>
            <td>1</td>
        </tr>
        <tr>
            <td>`min_weight_fraction_leaf`</td>
            <td>The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node.</td>
            <td>float</td>
            <td>[0, 0.5]</td>
            <td>0</td>
        </tr>
        <tr>
            <td>`max_features`</td>
            <td>The number of features to consider when looking for the best split</td>
            <td>int, float or string</td>
            <td>(0, n_features] or { “sqrt”, “log2”}</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`max_leaf_nodes`</td>
            <td>Grow trees with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity.</td>
            <td>int</td>
            <td>(1, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_impurity_decrease`</td>
            <td>A node will be split if this split induces a decrease of the impurity greater than or equal to this value.</td>
            <td>float</td>
            <td>[0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`init`</td>
            <td>An estimator object that is used to compute the initial predictions.</td>
            <td>object or string</td>
            <td>estimator (Any classification model except SVM classification and catboost) or ‘zero’</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`warm_start`</td>
            <td>When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just erase the previous solution.</td>
            <td>bool</td>
            <td>True or False</td>
            <td>False</td>
        </tr>
        <tr>
            <td>`tol` (tolerance)</td>
            <td>Tolerance for the early stopping. When the loss is not improving by at least tol for n_iter_no_change iterations (if set to a number), the training stops.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1e-4</td>
        </tr>
        </tbody>
        </table>

5. ### KNN Classification

    KNN works by finding the distances between a query (data instance) and all the examples in the data, selecting the specified number examples (K) closest to the query, then voting  for the most frequent label in the neighbourhood.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
        <th class="w15p">Parameter</th>
                <th class="w35p">Description</th>
                <th class="w15p">Data Type</th>
                <th class="w20p">Possible Values</th>
                <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`n_neighbors`
(number of neighbours)</td>
            <td>Number of neighbors to use by default for kneighbors queries.</td>
            <td>int</td>
            <td>[1, n]
 n = Total number of records in dataset</td>
            <td>5</td>
        </tr>
        <tr>
            <td>`weights`</td>
            <td>Weight function used in prediction. Possible values</td>
            <td>string</td>
            <td>{‘uniform’, ‘distance’}</td>
            <td>’uniform’</td>
        </tr>
        <tr>
            <td>`algorithm`</td>
            <td>Algorithm used to compute the nearest neighbors.</td>
            <td>string</td>
            <td>{‘auto’, ‘ball_tree’, ‘kd_tree’, ‘brute’}</td>
            <td>’auto’</td>
        </tr>
        <tr>
            <td>`leaf_size`</td>
            <td>Leaf size passed to BallTree or KDTree. This can affect the speed of the construction and query, as well as the memory required to store the tree.</td>
            <td>int</td>
            <td> (1, +Inf)</td>
            <td>30</td>
        </tr>
        <tr>
            <td>`p`</td>
            <td>Power parameter for the Minkowski metric.</td>
            <td>int</td>
            <td>[1,3]</td>
            <td>2</td>
        </tr>
        <tr>
            <td>`metric`</td>
            <td>Metric to use for distance computation. Default is “minkowski”, which results in the standard Euclidean distance when p = 2.</td>
            <td>string</td>
            <td>{‘cityblock’, ‘cosine’, &#39;euclidean&#39;, &#39;l1&#39;, &#39;l2&#39;, &#39;manhattan&#39;, &#39;nan_euclidean&#39;, ’minkowski’}</td>
            <td>’minkowski’</td>
        </tr>
    </tbody>
    </table>

6. ### LGBM Classification
    
    LGBM works by starting with an initial estimate that is updated using the output of each tree. The learning parameter controls the magnitude of this change in the estimates. It can be used on any data and provides a high degree of accuracy, as it contains many built-in preprocessing steps.

    The LightGBM algorithm grows vertically, meaning it grows leaf-wise, while other algorithms grow level-wise. LightGBM chooses the leaf with the largest loss to grow. It can lower more loss than a level-wise algorithm when growing the same leaf.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w30p">Description</th>
            <th class="w10p">Data Type</th>
            <th class="w20p">Possible Values</th>
            <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`boosting_type`</td>
            <td>Method of boosting.</td>
            <td>string</td>
            <td>{‘gbdt’, ‘dart’, ‘goss’ }</td>
            <td>&#39;gbdt&#39;</td>
        </tr>
        <tr>
            <td>`num_leaves`</td>
            <td>Maximum tree leaves for base learners.</td>
            <td>int</td>
            <td>(1, +Inf)</td>
            <td>31</td>
        </tr>
        <tr>
            <td>`max_depth`</td>
            <td>Maximum tree depth for base learners, &lt;= 0 means no limit.</td>
            <td>int</td>
            <td>(-Inf, +Inf)</td>
            <td>-1</td>
        </tr>
        <tr>
            <td>`learning_rate`</td>
            <td>Boosting learning rate. You can use callbacks parameter of fit method to shrink/adapt learning rate in training using reset_parameter callback.</td>
            <td>float</td>
            <td>(0.0, +Inf)</td>
            <td>0.1</td>
        </tr>
        <tr>
            <td>`n_estimators`
 (number of estimators)</td>
            <td>Number of boosted trees to fit.</td>
            <td>int</td>
            <td>[1, 500]</td>
            <td>100</td>
        </tr>
        <tr>
            <td>`subsample_for_bin`</td>
            <td>Number of samples for constructing bins.</td>
            <td>int</td>
            <td>(0, +Inf)</td>
            <td>200000</td>
        </tr>
        <tr>
            <td>`min_split_gain`</td>
            <td>Minimum loss reduction required to make a further partition on a leaf node of the tree.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`min_child_weight`</td>
            <td>Minimum sum of instance weight (Hessian) needed in a child (leaf).</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1e-3</td>
        </tr>
        <tr>
            <td>`min_child_samples`</td>
            <td>Minimum number of data needed in a child (leaf).</td>
            <td>int</td>
            <td>[0, +Inf)</td>
            <td>20</td>
        </tr>
        <tr>
            <td>`subsample`</td>
            <td>Subsample ratio of the training instance.</td>
            <td>float</td>
            <td>(0.0, 1.0]</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`subsample_freq` (subsample_frequency)</td>
            <td>Frequency of subsample, &lt;= 0 means no enable.</td>
            <td>int</td>
            <td>(-Inf, +Inf)</td>
            <td>0</td>
        </tr>
        <tr>
            <td>`colsample_bytree` (column sample by tree)</td>
            <td>Subsample ratio of columns when constructing each tree.</td>
            <td>float</td>
            <td>(0.0, 1.0]</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`reg_alpha` (alpha)</td>
            <td>L1 regularization term on weights.</td>
            <td>float</td>
            <td>(0.0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`reg_lambda` (lambda)</td>
            <td>L2 regularization term on weights.</td>
            <td>float</td>
            <td>(0.0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`importance_type`</td>
            <td>The type of feature importance to be filled into feature<em>importances</em>. If ‘split’, result contains numbers of times the feature is used in a model. If ‘gain’, result contains total gains of splits which use the feature.</td>
            <td>string</td>
            <td>{ ‘gain’, &#39;split&#39;}</td>
            <td>&#39;split&#39;</td>
        </tr>
        </tbody>
        </table>

7. ### Logistic Regression

    When the target is **binary value**, we can use logistic classification. It maps the value between 0 and 1.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w20p">Parameter</th>
            <th class="w35p">Description</th>
            <th class="w10p">Data Type</th>
            <th class="w20p">Possible Values</th>
            <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`penalty`</td>
            <td>Specify the norm of the penalty:<ul style="text-align: left;"><li><strong>&#39;none&#39;</strong>: no penalty is added;<li><strong>&#39;l2&#39;</strong>: add a L2 penalty term and it is the default choice;<li><strong>&#39;l1&#39;</strong>: add a L1 penalty term;<li><strong>&#39;elasticnet&#39;</strong>: both L1 and L2 penalty terms are added.</ul></td>
            <td>string</td>
            <td>{ ‘l1’, ‘l2’, ‘elasticnet’, ‘none’}</td>
            <td>’l2’</td>
        </tr>
        <tr>
            <td>`dual`</td>
            <td>Dual or primal formulation. Dual formulation is only implemented for l2 penalty with liblinear solver.</td>
            <td>bool</td>
            <td>True or False</td>
            <td>False</td>
        </tr>
        <tr>
            <td>`tol` (tolerance)</td>
            <td>Tolerance for stopping criteria.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1e-4</td>
        </tr>
        <tr>
            <td>`C`</td>
            <td>Inverse of regularization strength; must be a positive float.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`solver`</td>
            <td>Algorithm to use in the optimization problem.</td>
            <td>string</td>
            <td>{ ‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’ }</td>
            <td>’lbfgs’</td>
        </tr>
        <tr>
            <td>`fit_intercept`</td>
            <td>Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function.</td>
            <td>bool</td>
            <td>True or False</td>
            <td>True</td>
        </tr>
        <tr>
            <td>`l1_ratio`</td>
            <td>The Elastic-Net mixing parameter, with
0 &lt;= l1_ratio &lt;= 1. Only used if penalty=&#39;elasticnet&#39;.</td>
            <td>float</td>
            <td>[0, 1]</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`multi_class`</td>
            <td>If the option chosen is ‘ovr’, then a binary problem is fit for each label. For ‘multinomial’ the loss minimised is the multinomial loss fit across the entire probability distribution, even when the data is binary.</td>
            <td>string</td>
            <td>{&#39;auto&#39;, &#39;ovr&#39;, &#39;multinomial&#39;}</td>
            <td>&#39;auto&#39;</td>
        </tr>
        <tr>
            <td>`intercept_scaling`</td>
            <td>Useful only when the solver ‘liblinear’ is used and self.fit_intercept is set to True. The intercept becomes intercept_scaling * synthetic_feature_weight.</td>
            <td>float</td>
            <td>(0, +Inf)</td>
            <td>1.0</td>
        </tr>
        </tbody>
        </table>

> **Note:** The values in the "solver" parameter support only few of the values in the "penalty" parameter. So the supported penalties by the solver are mentioned below:

    * newton-cg’ - [‘l2’, ‘none’]
    * ‘lbfgs’ - [‘l2’, ‘none’]
    * ‘liblinear’ - [‘l1’, ‘l2’]
    * ‘sag’ - [‘l2’, ‘none’]
    * ‘saga’ - [‘elasticnet’, ‘l1’, ‘l2’, ‘none’]

8. ### Naive Bayes Classification

    Naive Bayes is a classifier that uses the **Bayes Theorem**. It predicts membership probabilities for each class, such as the probability that a given record or data point belongs to a particular class. The class with the highest probability is considered as the most likely class.

9. ### Random-Forest Classification

    The random forest is a classification algorithm consisting of many decisions trees. It uses bagging and feature randomness when building individual trees to try to create an uncorrelated forest of trees whose prediction by committee is more accurate than that of any individual tree.

    A Bagging is an ensemble meta-estimator that fits base classifiers/regressors on random subsets of the original dataset, then aggregates their individual predictions (either by voting or by averaging) to form a final prediction.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w30p">Description</th>
            <th class="w10p">Data Type</th>
            <th class="w20p">Possible Values</th>
            <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`n_estimators`
 (number of estimators)</td>
            <td>The number of trees in the forest.</td>
            <td>int</td>
            <td>[1, 500]</td>
            <td>100</td>
        </tr>
        <tr>
            <td>`criterion`</td>
            <td>The function to measure the quality of a split.</td>
            <td>string</td>
            <td>{“gini”, “entropy”}</td>
            <td>”gini”</td>
        </tr>
        <tr>
            <td>`max_depth`</td>
            <td>The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.</td>
            <td>int</td>
            <td>(0, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_samples_split`</td>
            <td>The minimum number of samples required to split an internal node</td>
            <td>int or float</td>
            <td>[2, +Inf) or (0, 1.0]</td>
            <td>2</td>
        </tr>
        <tr>
            <td>`min_samples_leaf`</td>
            <td>The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches.</td>
            <td>int or float</td>
            <td>[1, +Inf) or (0, 0.5]</td>
            <td>1</td>
        </tr>
        <tr>
            <td>`min_weight_fraction_leaf`</td>
            <td>The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node.</td>
            <td>float</td>
            <td>[0, 0.5]</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`max_features`</td>
            <td>The number of features to consider when looking for the best split</td>
            <td>int, float or string</td>
            <td>(0, n_features] or { “sqrt”}</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`max_leaf_nodes`</td>
            <td>Grow trees with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity.</td>
            <td>int</td>
            <td>(1, +Inf)</td>
            <td>None</td>
        </tr>
        <tr>
            <td>`min_impurity_decrease`</td>
            <td>A node will be split if this split induces a decrease of the impurity greater than or equal to this value.</td>
            <td>float</td>
            <td>[0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`bootstrap`</td>
            <td>Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree.</td>
            <td>bool</td>
            <td>True or False</td>
            <td>True</td>
        </tr>
        <tr>
            <td>`oob_score`(out of bag score)</td>
            <td>Whether to use out-of-bag samples to estimate the generalization score. Only available if bootstrap=True.</td>
            <td>bool</td>
            <td>True or False</td>
            <td>False</td>
        </tr>
        <tr>
            <td>`warm_start`</td>
            <td>When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest.</td>
            <td>bool</td>
            <td>True or  False</td>
            <td>False</td>
        </tr>
    </tbody>
    </table>

10. ### SVM Classification

    SVM, or Support Vector Machine, is a linear model for classification and regression problems. It can solve linear and non-linear problems and work well for many practical problems. The idea of SVM is simple: The algorithm creates a line or a hyperplane that separates the data into classes.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w35p">Description</th>
            <th class="w10p">Data Type</th>
            <th class="w15p">Possible Values</th>
            <th class="w15p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`C`</td>
            <td>Regularization parameter. The strength of the regularization is inversely proportional to C.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`kernel`</td>
            <td>Specifies the kernel type to be used in the algorithm. If none is given, ‘rbf’ will be used.</td>
            <td>string</td>
            <td>{‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’}</td>
            <td>’rbf’</td>
        </tr>
        <tr>
            <td>`degree`</td>
            <td>Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.</td>
            <td>int</td>
            <td>[0, +Inf)</td>
            <td>3</td>
        </tr>
        <tr>
            <td>`gamma`</td>
            <td>Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.</td>
            <td>string or float</td>
            <td>{‘scale’, ‘auto’} or (0.0, +Inf)</td>
            <td>’scale’</td>
        </tr>
        <tr>
            <td>`coef0`</td>
            <td>Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.</td>
            <td>float</td>
            <td>(-Inf, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`shrinking`</td>
            <td>Whether to use the shrinking heuristic.</td>
            <td>bool</td>
            <td>True or  False</td>
            <td>True</td>
        </tr>
        <tr>
            <td>`probability`</td>    
            <td>Whether to enable probability estimates.</td>
            <td>bool</td>
            <td>True or  False</td>
            <td>False</td>
        </tr>
        <tr>
            <td>`tol` (tolerance)</td>
            <td>Tolerance for stopping criterion.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>1e-3</td>
        </tr>
        <tr>
            <td>`decision_function_shape`</td>
            <td>Whether to return a one-vs-rest (‘ovr’) decision function of shape (n_samples, n_classes) as all other classifiers, or the original one-vs-one (‘ovo’) decision function of libsvm which has shape (n_samples, n_classes * (n_classes - 1) / 2).</td>
            <td>string</td>
            <td>{‘ovo’, ‘ovr’}</td>
            <td>’ovr’</td>
        </tr>
        <tr>
            <td>`break_ties`</td>
            <td>If true, decision_function_shape=&#39;ovr&#39;, and number of classes &gt; 2, predict will break ties according to the confidence values of decision_function; otherwise the first class among the tied classes is returned.</td>
            <td>bool</td>
            <td>True or  False</td>
            <td>False</td>
        </tr>
    </tbody>
    </table>

11. ### XGB Classification

    XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine-learning algorithms under the Gradient Boosting framework. It provides a parallel tree boosting to solve many data science problems quickly and accurately. It uses L1 and L2 regularisation to predict points and it fast in training.

    Each model has Fit (to train model), predict (to predict new data), get metrics (to get model's accuracy and other metrics), and feature_importances (importances of the input features for the prediction).

    Adaboost, CatBoost, Decision tree, Gradient boost(GB), LGBM, RandomForest, SVM, and XGB's basic working principles are almost identical for both regression and classification.

    #### Hyper Parameters:

    <table class="content-table quickml-content-table">
        <thead>
        <tr>
            <th class="w25p">Parameter</th>
            <th class="w30p">Description</th>
            <th class="w10p">Data Type</th>
            <th class="w25p">Possible Values</th>
            <th class="w20p">Default Values</th>
        </tr>
        </thead>
        <tbody>
        <tr>
            <td>`booster`</td>
            <td>Decides which booster to use.</td>
            <td>string</td>
            <td>{‘gbtree&#39;, &#39;gblinear&#39;, &#39;dart&#39; }</td>
            <td>’gbtree’</td>
        </tr>
        <tr>
            <td>`learning_rate`</td>
            <td>Step size shrinkage used in update to prevents over fitting. After each boosting step, we can directly get the weights of new features, and eta shrinks the feature weights to make the boosting process more conservative.</td>
            <td>float</td>
            <td>[0,1]</td>
            <td>0.1</td>
        </tr>
        <tr>
            <td>`n_estimators`
 (number of estimators)</td>
            <td>Number of trees to fit.</td>
            <td>int</td>
            <td>[1, 500]</td>
            <td>100</td>
        </tr>
            <tr>
            <td>`objective`</td>
            <td>Logistic regression for binary classification.</td>
            <td>string</td>
            <td>Mentioned below the table .</td>
            <td>&quot;binary:logistic&quot;</td>
        </tr>
        <tr>
            <td>`subsample`</td>
            <td>Control the sample&#39;s proportion.</td>
            <td>int</td>
            <td>(0,1]</td>
            <td>1</td>
        </tr>
        <tr>
            <td>`max_depth`</td>
            <td>Maximum depth of a tree.</td>
            <td>int</td>
            <td>(0, +Inf)</td>
            <td>3</td>
        </tr>
        <tr>
            <td>`max_delta_step`</td>
            <td>If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced.</td>
            <td>int  or float</td>
            <td>[0, +Inf)</td>
            <td>0</td>
        </tr>
        <tr>
            <td>`colsample_bytree` (column sample by tree)</td>
            <td>Column&#39;s fraction of random samples.</td>
            <td>float</td>
            <td>(0.0, 1.0]</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`colsample_bylevel` (column sample by level)</td>
            <td>It is the subsample ratio of columns for each level. Subsampling occurs once for every new depth level reached in a tree. Columns are subsampled from the set of columns chosen for the current tree.</td>
            <td>float</td>
            <td>(0.0, 1.0]</td>
            <td>1.0</td>
        </tr>
        <tr>
            <td>`min_child_weight`</td>
            <td>Minimum sum of weights.</td>
            <td>int</td>
            <td>[0, +Inf)</td>
            <td>1</td>
        </tr>
        <tr>
            <td>`reg_alpha` (alpha)</td>
            <td>L1 regularization term on weights.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`reg_lambda` (lambda)</td>
            <td>L2 regularization term on weights.</td>
            <td>float</td>
            <td>[0.0, +Inf)</td>
            <td>0.0</td>
        </tr>
        <tr>
            <td>`scale_pos_weight` (scale positive weight)</td>
            <td>Control the balance of positive and negative weights, useful for unbalanced classes.</td>
            <td>int</td>
            <td>[0, +Inf)</td>
            <td>1</td>
        </tr>
        </tbody>
        </table>

    **POSSIBLE VALUES FOR "OBJECTIVE" PARAM :**

    {binary:logistic, binary:logitraw, binary:hinge, multi:softmax, multi:softprob}