Browser Grid Features

This section explores the various features that make up the Browser Grid component.

Hub

Hub is a virtual machine that acts as a controller. When a user sends requests to the headless browser present in a browser grid, the Hub will allot a node to run the headless browser to execute the request. The functionality of the Hub is entirely handled by the Catalyst SmartBrowz service. This ensures your grid can scale seamlessly to handle your request load.

Nodes

Nodes are virtual machines that run the required headless browsers. Based on the configuration, you can dictate the number of nodes that need to be created in your browser grid and the number of headless browsers that need to be spawned in each node.

Headless Browsers

Headless browsers are browsers that run without a Graphical User Interface (GUI). These types of browsers are the entities or instances that execute and process the requests sent by the user.

The Browser Grid component provides you with the option to effortlessly connect to multiple secure remote browsers using popular automation libraries like Puppeteer, Playwright, and Selenium.

Based on your requirement, you have the option to choose your automation library and launch your required secure headless browser using a CDP/Webdriver endpoint generated by the Catalyst SmartBrowz service.

The endpoint is made secure using an API KEY which will be uniquely generated for you.

Browser Grid Configuration

In a browser grid, you can define the number of nodes and the number of browsers that can be created in a singular node. The configuration also allows you to set the number of browsers that can be run concurrently. When you create or update an existing grid, you have the option to choose:

• Basic Configuration: This is a type of configuration that is best suited to handle an ideal load of requests, with a processing power of 1GiB Memory, 1vCPU.
• Advanced Configuration: This type of configuration allows you the option to choose the number of nodes and the number of browsers allotted per node. You will have the following options:
  • Light Node Type: You can choose to create a Grid that has a processing power of 1GiB Memory, 1vCPU.
  • Moderate Node Type: You can choose to create a Grid that has a processing power of 2GiB Memory, 2vCPU.
  • Heavy Node Type: You can choose to create a Grid that has a processing power of 4GiB Memory, 4vCPU.

The following table will further detail the differences between the various configurations:

Hub-Node-Browser Relation

The following diagram illustrates the relation between hubs, nodes, and browsers.

Based on the volume of requests being sent, the hub will compute allocate nodes individually. When the first allotted node attains its maximum concurrency limit, with requests still pending, only then will the hub allot the next node.

These nodes will run the required headless browsers and process your requests. The number of nodes and browsers that are created by the hub will be based on the configuration you chose while creating your Browser Grid.

Alerts

Alerts is a feature in Browser Grid that will alert you to any error or configuration failure the grid encountered during its operations by populating the Alerts section in theDashboard.

The following are the five possible alerts that you may receive if your grid is not functioning as intended:

• Browser Creation Rejected: This alert will be created if the grid is operating at maximum configuration, and you still have requests that need to be processed in the queue beyond the queue time (Selenium: 30 seconds, Puppetter/Playwright: 5 minutes). The queued requests demand new browsers be created, but none can be created as the configuration does not allow for it or there are no more browsers to be spawned, leaving the requests in the queue to be killed.

• 90% Disk Exhausted: This alert will occur if even one node’s allotted disk capacity has reached 90%. The default size allocated for the disk is 10GB.

• 80% Memory Exhausted: This alert will occur if even one node’s allotted memory is 80% consumed.

• 80% CPU Exhausted: This alert will occur if even one node reached 80% of the CPU capacity allotted to it.

• Nodes Crashed: This will only occur in severe times, when the node is operating at the maximum configuration allotted to it and is severely strained to process the volume of requests being sent to it.

You can click on a specific alert for more information about the alert notification.

To deal with these alerts:

• Try reducing the number of requests being sent for processing.
• Try updating the browser grid’s configuration.

Browser Grid States

• Active: This is a state the Browser Grid enters into when you connect to the remote browser using its endpoint through code. While in this state, the browser grid spawns the required number of nodes and browsers to process the requests.

• Inactive: This is a state the Browser Grid enters into when you stop the browser grid. This is the default the grid is in when you create a browser grid.

• Idle: The Browser Grid is an auto-scaling component. Based on your demand, the required number of nodes and respective browsers are spawned (as per your configurations), and when there are no requests to be processed, the browsers and nodes are killed. Once the grid has scaled completely down and is not processing any requests, the browser grid will enter the Idle state. There will be no requests being processed in the grid.