# Automation Testing -------------------------------------------------------------------------------- title: "Introduction" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.552Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/introduction/" service: "DevOps" -------------------------------------------------------------------------------- # Automation Testing ### Introduction Catalyst Automation Testing is a Catalyst Devops component that serves as an API testing tool to automate the testing process of APIs in your Catalyst application. Using this feature, you can write test cases based on your requirements, schedule tests to be executed automatically, and manage the entire process of API testing with minimal manual intervention. Automation Testing will provide you with real-time detailed reports and debug results, which enable you to analyze the root causes of failure in your APIs and resolve issues faster. You can learn more about the other components of the DevOps service from {{%link href="/en/devops" %}}this page{{%/link%}}. Automation Testing is a convenient platform for you to manage all your testing needs, without needing to code or configure elaborate backend setups. This feature increases your overall testing efficiency, and saves you from investing valuable time and effort into performing repetitive manual testing. Catalyst servers are highly scalable, and can therefore accommodate high demands without affecting your testing process. Automation Testing enables you to test all the accessible end points of your application, i.e., the {{%link href="/en/serverless/help/functions/introduction" %}}Basic I/O and Advanced I/O functions{{%/link%}}. You can also test any {{%bold%}}third-party URL{{%/bold%}} that you need to, and obtain responses. This feature is provided as a {{%link href="/en/devops/help/automation-testing/pricing" %}}partially paid component{{%/link%}}, that requires you to pay after you exhaust the free limits for each project. {{%note%}}{{%bold%}}Note:{{%/bold%}} * Automation Testing is not accessible in the Production environment, as testing is ideally performed in the Development environment. However, you can execute tests for the Production environment from the Development environment, and check Production responses from there. Refer to the {{%link href="/en/deployment-and-billing/environments/introduction" %}}Environments help page{{%/link%}} to learn more about them. * Automation Testing is currently not available to Catalyst users accessing from the EU, AU, IN, or CA data centers. {{%/note%}} The sequence of working with Automation Testing can be summed up as follows: 1. Create a module to include test cases that execute related actions. 2. Create a test case and configure individual requests in it. 3. Create a test suite to facilitate the execution of various test cases and modules in parallel or in a sequence. 4. Create a test plan to schedule the automated execution of one or more test suites. <br> ### Benefits 1. {{%bold%}}Efficient Application Development Process as a Result of Complete Automation{{%/bold%}}<br /> Automation in testing is highly useful if your microservice demands repeated and frequent tests for a variety of different test conditions. Automation becomes crucial in minimizing the high-effort, time-consuming, and monotonous nature of manual testing, that is often prone to human errors. Automation Testing enables complete automation, but it also offers you the flexibility to intervene manually whenever needed. 2. {{%bold%}}Scalable and Reliable Platform equipped with a host of Functionalities{{%/bold%}}<br /> Automation Testing comes equipped with a host of features that satisfy all your fundamental testing needs, and facilitates a sophisticated testing experience. You can write individual test cases for precise actions, organize relevant test cases together, and reuse them whenever needed. You can also orchestrate chained API executions and sequential test runs and pass dynamic values between them, as well as parallel test runs that speed up testing. 3. {{%bold%}}Ease of Usability in configuring Testing{{%/bold%}}<br /> Automation Testing provides an integrated platform that enables you to set up your testing environment easily in minutes. You can configure everything with just a few button clicks. You can easily configure a test case, ascertain the values that you expect to be returned in the response, and quickly schedule an automated test run for it. Automation Testing will handle the rest, and deliver results with detailed insights. You can even facilitate tests involving different sets of data for both Development and Production environments, without performing any additional configurations. 4. {{%bold%}}Enabling Ease of Debug and Problem Solving{{%/bold%}}<br /> In addition to pinpointing the exact problem areas, Automation Testing also decodes issues for you that will immensely help you while you fix them. The results contain the exact responses received for each request in each test case, along with crucial metrics and performance statistics of the setup overall. You can measure the performance of your service using these insightful metrics, determine the functionality and reliability of your its integration, and ensure meeting the high standards to your end-users. <br> ### Enable Automation Testing Before we discuss the features of Automation Testing, let's enable it for a project in the Catalyst console for your project. 1. Navigate to {{%bold%}}DevOps{{%/bold%}} in the left pane of the Catalyst console and click {{%bold%}}Automation Testing.{{%/bold%}} 2. Click {{%bold%}}Enable Now{{%/bold%}} in the Automation Testing feature page and click {{%bold%}}Proceed{{%/bold%}} in the confirmation window that appears. Automation Testing is now enabled for your project. <br> ### Disable Automation Testing You can disable Automation Testing in the following way: 1. Click the ellipsis icon from the top-right corner of the Automation Testing page, then click {{%bold%}}Disable{{%/bold%}}. 2. Type "DISABLE" in the confirmation pop-up to confirm, then click {{%bold%}}Yes, Proceed{{%/bold%}}. Automation Testing is now disabled for your project. You can disable and re-enable it based on your needs. -------------------------------------------------------------------------------- title: "Modules and Test Cases" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.552Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/modules-test-cases/" service: "DevOps" -------------------------------------------------------------------------------- # Modules and Test Cases ### Modules Modules enable you to categorize the test cases that you create, based on the actions they execute or their logic. Modules can also represent the same process or function. However, you can use them to divide the process into various parts that focus on individual activities. For example, you can group all the test cases related to submitting a particular form in your application in one module. Make sure to remember the following points about modules in Automation Testing: * You must create a test case in a module, because a test case must be associated with a module. Therefore, before you {{%link href="#create-a-test-case" %}}create a test case{{%/link%}}, you must either create a module for it or add it to an existing module. * You can create sub-modules in a module up to one level. * Modules cannot contain test cases and sub-modules at the same level. For example, _Module A_ cannot contain _Test Case 1_ and _Submodule B_ as separate entities on the same level. You can create _Test Case 1_ inside _Submodule B_, in a different sub-module, or create it in a different module altogether. <br> ### Test Cases Test cases contain the definitions and particulars of an action to be performed. An API test case involves invoking one or more request URLs, and passing parameters, headers, or data in the request body. {{%note%}}{{%bold%}}Note:{{%/bold%}} * You can test both the Development and Production environment function URLs of your application in a request, as well as any third-party URLs. * You can also test the APIs that you configured in {{%link href="/en/cloud-scale/help/api-gateway/introduction" %}}API Gateway{{%/link%}} for your function endpoints. {{%/note%}} In API testing, we typically send a request to access a function's endpoint or a third-party resource to receive a response. The testing is done to observe the responses that will be received for various test conditions, request methods, environments, and values that you set for the request. If you obtain the expected responses for the values and conditions that you pass in all the requests you make in a test case, the testing can be deemed successful. Keep the following points in mind about the test cases in Automation Testing: * A test case can contain one or more requests that you can configure individually. * A test case can only be automated after it is added to a {{%link href="/en/devops/help/automation-testing/test-suites" %}}test suite{{%/link%}}, and scheduled to be executed in a {{%link href="/en/devops/help/automation-testing/test-plans" %}}test plan{{%/link%}}. * You can orchestrate a sequence of requests in a single test case to execute automatically, and pass the dynamic values obtained from one request's response into the next. This can be done by {{%link href="/en/devops/help/automation-testing/variables" %}}defining a variable for a request{{%/link%}} and passing that variable as the {{%link href="#configure-a-request" %}}parameter{{%/link%}} in the next request. * You must define the expected responses in a request using {{%link href="#add-assertions" %}}Assertions{{%/link%}}. Upon the execution of a request, Catalyst will automatically validate the received response with the expected values, determine the result of the test case accordingly, and display it in the {{%link href="/en/devops/help/automation-testing/results" %}}Results section{{%/link%}}. Each individual request must be successful for a test case execution to be successful. * The requests made in Automation Testing can be {{%link href="#configure-a-request" %}}authenticated{{%/link%}} through a _Custom Authentication_ method. You can set up a connection using Deluge Connections and authenticate the request through it. <br> ### Create a Test Case You can begin working with Automation Testing by creating a test case. Each action involved in setting up the test case's configuration is described in this section in detail. 1. After you enable Automation Testing from the _Add-on Services_, you can click {{%bold%}}Create Test Case{{%/bold%}} from the _Test Cases_ section.<br /> 2. Enter a name and a description for the test case. <br /> <br /> Since a test case needs to be associated with a {{%link href="#Module" %}}module{{%/link%}} and we have not created a module yet, click the drop-down and type a name for a new module and click {{%bold%}}Create{{%/bold%}} in the search bar. <br /> <br /> If a module or a sub-module already exists, you can simply select it from the list. 3. Click {{%bold%}}Create{{%/bold%}}. The test case will be created in the specified module. The test case's tab will be opened, and you can configure requests in it. We will discuss the UI elements {{%link href="#manage-a-test-case-and-its-requests" %}}later{{%/link%}} in this documentation. <br> ### Configure a Request You can configure a request in a test case in the following way: 1. Click {{%bold%}}Add Request{{%/bold%}} in the test case's section.<br /> <br /> 2. Enter the URL of the API endpoint that needs to be tested. <br /> <br /> You can enter a Basic I/O or an Advanced I/O function URL of either environment, an API you configured for them in {{%link href="/en/cloud-scale/help/api-gateway/introduction" %}}API Gateway{{%/link%}}, or any third-party URL.<br /><br /> You can also pass an Environment or Global variable that you configured, as the value for the request URL. You can learn about these in the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}}. 3. Select a HTTP request method for the request from the dropdown list. You can select from: {{%badge%}}GET{{%/badge%}}, {{%badge%}}PUT{{%/badge%}}, {{%badge%}}POST{{%/badge%}}, {{%badge%}}PATCH{{%/badge%}}, {{%badge%}}DELETE{{%/badge%}}, {{%badge%}}HEAD{{%/badge%}}, {{%badge%}}OPTIONS{{%/badge%}}. 4. {{%bold%}}Params:{{%/bold%}} You can configure query strings to be passed appended to your request URL in a key-value format under the _Params_ section. Once you enter a parameter name, a new column will appear to add another parameter. You can click the close icon to remove a parameter.<br /> <br /> You can also pass an {{%link href="/en/devops/help/automation-testing/variables" %}}Environment or Global variable{{%/link%}} in the {{%bold%}}{{$\<variable\>}}{{%/bold%}} format, or {{%link href="#chain-requests" %}}assign a value from a previous request's response{{%/link%}} to a variable and pass that as the parameter of a request in the {{%bold%}}{{\<variable\>}}{{%/bold%}}. 5. {{%bold%}}Authentication:{{%/bold%}} To set up the authentication method for this particular request, click the {{%bold%}}Authentication{{%/bold%}} tab under the request URL. If you opt to have no authentication in your request, select {{%bold%}}None{{%/bold%}}.<br /><br /> You can select {{%bold%}}Custom Auth{{%/bold%}} to configure a {{%link href="https://www.zoho.com/deluge/help/deluge-connections.html" %}}Deluge connection{{%/link%}}. Deluge connections enable you to establish authentication with a Zoho or a third-party service. This connection will allow Catalyst to invoke the third party service's API, and obtain access to the authenticated data. In this case, the request you are configuring is the third-party service. <br /><br /> When you select {{%bold%}}Custom Auth{{%/bold%}}, a drop-down box will list any pre-existing connections that were configured. To create a new connection, click {{%bold%}}Create New{{%/bold%}}. <br /><br /> <br /> This will open the Connections configuration page in the request section. You can configure the connection here, and choose one of the authentication types available. <br /><br /> <br /> You can learn about these steps in detail from the {{%link href="https://www.zoho.com/crm/developer/docs/connectors/set-up.html" %}}Zoho CRM Connections help page{{%/link%}}. CRM implements the same Deluge connections service. A connection with Zoho CRM, for example, can enable you to perform actions such as inserting a lead or obtaining a contact from your CRM account, in the request. 6. {{%bold%}}Headers:{{%/bold%}} You can configure headers for the request URL from the _Headers_ section. Click the {{%bold%}}Headers{{%/bold%}} tab. Enter the names and corresponding values of the headers to be passed. Catalyst suggests header keys as you type.<br /> <br /> Once you enter a header name, a new column will appear to add another header. You can click the close icon to remove a header. 7. {{%bold%}}Body:{{%/bold%}} You can configure data to be passed in the body of the request from the Body section. A request body is typically passed in {{%badge%}}POST{{%/badge%}} or {{%badge%}}PUT{{%/badge%}} requests. Click the {{%bold%}}Body{{%/bold%}} tab. Automation Testing supports three types of request bodies: _form-data_, _x-www-form-urlencoded_, and _raw_. Select a type from the drop-down list.<br /><br /> If you select _form-data_ or _x-www-form-urlencoded_ you can enter the payload in a key-value format. You can also pass {{%link href="/en/devops/help/automation-testing/variables" %}}variables{{%/link%}} as the value for keys in these formats. If you select _raw_, you can pass it as a JSON.<br /><br /> After you configure all the necessary parts of the request, you can add assertions for the request. <br> ### Add Assertions Assertions are used to validate the response received for a request. You can define the values that you expect in various parts of the response here. Automation Testing will automatically compare the expected values with the received values. If the values match and all assertions are satisfied, the request execution is considered successful. Otherwise, it is considered a failure. {{%note%}}{{%bold%}}Note:{{%/bold%}} Assertions are not mandatory for a request in a test case. If you don't add any assertions in a request, the test run will be considered a success automatically. You will be able to view detailed reports of automated test runs from the {{%link href="/en/devops/help/automation-testing/results" %}}_Results_{{%/link%}} section.{{%/note%}} Click the {{%bold%}}Assertions{{%/bold%}} tab in the request configuration window, then click {{%bold%}}Add Assertion{{%/bold%}} to configure assertions for the test case. An assertion consists of the following elements that you can configure: * {{%bold%}}Source:{{%/bold%}} The location the assertion needs to be applied on.You can select one of these sources for an assertion from the drop-down list: * {{%bold%}}Header:{{%/bold%}} You can define the headers and their corresponding values that are expected in the response. * {{%bold%}}Status Code:{{%/bold%}} You can define the HTTP status code that is expected for the request in the response. For example, if you expect your request to be processed successfully, you can define an assertion indicating the status code equals 200. * {{%bold%}}XML Body:{{%/bold%}} You can extract a specific data item received in the XML body of a response by specifying its path and defining its expected value. * {{%bold%}}JSON Body:{{%/bold%}} You can extract a particular data item received in the JSON body of a response by specifying its path and defining its expected value. * {{%bold%}}Property:{{%/bold%}} The specific attribute of the source that the assertion needs to be applied for<br /><br /> For example, if you select {{%bold%}}Header{{%/bold%}} as the source, you can enter the key of a specific header whose value needs to be compared in the Property text box.<br /><br /> Similarly, if you select {{%bold%}}XML Body{{%/bold%}} or {{%bold%}}JSON Body{{%/bold%}} as the source, you can specify the path of the data item to be extracted in the Property text box. You must ensure that you adhere to the JSON or XML syntax in defining the path. {{%note%}}{{%bold%}}Note:{{%/bold%}} The Property attribute is not available for a Status Code source type, since there are no attributes in it.{{%/note%}} * {{%bold%}}Comparison:{{%/bold%}} Operator that compares the expected value with the defined value<br /><br /> You can select any one of these operators from the drop-down list: _Equals_, _Not Equals_, _Contains_, _Not Contains_, _Is Empty_, _Is Not Empty_, _Lesser Than_, _Greater Than_, _Lesser Than Or Equal To_, _Greater Than Or Equal To_, _Has Key_, _Array Count_.<br /><br /> The _Has Key_ operator can verify if a particular key is present in a header, or an XML or JSON path. You can use the _Array Count_ operator to specify the count of array items you expect in a header, or for an XML or JSON key. {{%note%}}{{%bold%}}Note:{{%/bold%}} These operators are not applicable for a Status Code source type: Contains, Not Contains, Is Empty, Is Not Empty, Has Key, Array Count. {{%/note%}} * {{%bold%}}Value:{{%/bold%}} The value that is expected for the selected property, based on the comparison operator {{%note%}}{{%bold%}}Note:{{%/bold%}} You will not be able to add values for the Is Empty and Is Not Empty operators.{{%/note%}} For example, if you expect the response status code not to be any of the failure codes, you can define an assertion for the it using Is Lesser Than and entering the value as "400", as the HTTP status codes above 400 indicate errors. You can remove an assertion by clicking the corresponding close icon. {{%bold%}}Example:{{%/bold%}} Assume you expect the response of a certain request to contain data in a JSON body that fulfil these conditions: * Key _id_ must be greater than or equal to "10000" * Key _location.country_ must definitely be present * Key _location.country_ must not contain an empty value * Key _location.city_ must not contain the value "New York" Therefore, you must add these assertions after selecting {{%bold%}}JSON Body{{%/bold%}} as the source each time: * {{%bold%}}Assertion 1:{{%/bold%}} _Property_\= "id", _Comparison_\= "Greater Than Or Equal To", _Value_\= "10000" * {{%bold%}}Assertion 2:{{%/bold%}} _Property_\= "location", _Comparison_\= "Has Key", _Value_\= "country" * {{%bold%}}Assertion 3:{{%/bold%}} _Property_\= "location.country", _Comparison_\= "Is Not Empty" * {{%bold%}}Assertion 4:{{%/bold%}} _Property_\= "location.city", _Comparison_\= "Not Equals", _Value_\= "New York" Now assume that you receive this JSON response upon execution: {{%code class="language-json line-numbers"%}}{ "id" : "10993", "location": { "country":"US", "city":"Austin" } } {{%/code%}} Because the response satisfies all the assertions that you defined, the test is considered to be successful. <br> ### Configure Variables in a Request You can extract data from the response that will be received for a request, and assign it to a variable to pass it on to the next request as a parameter value in a chain request. The variables that you configure inside a test case are different from the Environment or Global variables that you use to store the required details of your Catalyst project. The syntax for using these variables in the parameter or request URL field are different. You can learn about these in detail from the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}}. We will discuss adding variables for a chain request alone in this section. Click the {{%bold%}}Variables{{%/bold%}} tab in the request configuration window, then click {{%bold%}}Add Variable{{%/bold%}} for each variable you want to configure for the test case. <br /> This contains the same elements as {{%link href="#add-assertions" %}}Assertions{{%/link%}}. 1. Specify the location that the data needs to be extracted from, by selecting the {{%bold%}}Source{{%/bold%}}. 2. Define the specific attribute of the source that the variable needs to be assigned for, as the _Property_. 3. Enter the variable name that the data needs to be assigned to, under the Variable column. Refer to the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}} for detailed syntax. {{%note%}}{{%bold%}}Note:{{%/bold%}} The scope of a variable created during API testing is limited to the specific test case. The scopes of the Environment and Global variables are global to all test cases. {{%/note%}} You can remove variables by clicking the close icons. The {{%link href="#chain-requests" %}}Chain Requests section{{%/link%}} illustrates an example of passing data in a chained request using a variable that you create. <br> ### Manage a Test Case and its Requests Before we discuss configuring a chain request, here are a few actions that you can perform on a request from the console: <br /> * {{%bold%}}Rename a Request:{{%/bold%}} Click the edit icon next to its name on the request's configuration window, then enter a new name. Press {{%bold%}}Enter{{%/bold%}} to save the changes. * {{%bold%}}Delete a Request:{{%/bold%}} Click the delete icon, then click {{%bold%}}Yes{{%/bold%}} in the confirmation pop-up window.<br /> <br /> You can also perform the following actions on the test case from the tool bar and from the options below: * {{%bold%}}Run a Test:{{%/bold%}} You can instantly run a test and view a live preview of the response. Click {{%bold%}}Run{{%/bold%}} from the bottom of the page, after saving all the changes by clicking the {{%bold%}}Save{{%/bold%}} button next to it. {{%note%}}{{%bold%}}Note:{{%/bold%}} This option is used to quickly check if the configurations of your test case are working properly. You can automate a test run by adding the test case to a {{%link href="/en/devops/help/automation-testing/test-suites" %}}suite{{%/link%}} and scheduling its execution using a {{%link href="/en/devops/help/automation-testing/test-plans" %}}test plan{{%/link%}}. {{%/note%}} You can also optionally select a default environment that the test must be run in, from the pop-up window that opens. You can also click the arrow next to the run icon, then click {{%bold%}}Customize{{%/bold%}} to choose or change previously set configurations. <br /> You must select the environment variables from the drop-down window. Refer to the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}} for details on configuring environment variables. You can click the checkbox to save this as the preferred configuration. Click {{%bold%}}Run{{%/bold%}}.<br /> <br /> A Live Preview window will open displaying the progress. It will show the run summary after the execution is complete. You can click {{%bold%}}Restart{{%/bold%}} to restart the run.<br /> <br /> You can open collapse and expand the live preview anytime from the bottom of the screen by clicking {{%bold%}}Show Less{{%/bold%}} or {{%bold%}}Show More{{%/bold%}} respectively. If your run includes multiple requests, all their responses will be displayed in the Live Preview section. {{%note%}}{{%bold%}}Note:{{%/bold%}} * The summary of an instant run will not contain detailed reports. * These summaries will also not be available in the {{%link href="/en/devops/help/automation-testing/results" %}}Results section{{%/link%}}. The Results section only displays the detailed reports of the automated test runs executed through test plans. * The summary shown in this Live Preview section will disappear if you navigate away from this section. {{%/note%}} * {{%bold%}}Save a Test Case:{{%/bold%}} You must ensure that you save the changes you make in a test case periodically by clicking {{%bold%}}Save{{%/bold%}} from the bottom of the page. * {{%bold%}}Instantly Add or Remove a Test Case from a Suite:{{%/bold%}} Click the {{%bold%}}suite icon{{%/bold%}} on the tool bar of the test case to add it to a {{%link href="/en/devops/help/automation-testing/test-suites" %}}test suite{{%/link%}} instantly. Click {{%bold%}}Add to Test Suite{{%/bold%}} in the pop-up window, and select the existing suites from the list.<br /> <br /> You can also remove the test case from a suite it is already added in, by clicking the {{%bold%}}remove icon{{%/bold%}} that appears when you hover over the suite name in this window. Click {{%bold%}}Yes, Proceed{{%/bold%}}.<br /> * {{%bold%}}Edit a Test Case:{{%/bold%}} Click the {{%bold%}}edit icon{{%/bold%}} to edit the test name and description, or move the test case to a different module. Make the changes and click {{%bold%}}Update{{%/bold%}}. Read more about the left panel in {{%link href="#manage-all-test-cases-and-modules" %}}this section{{%/link%}}. <br> ### Chain Requests After you create and configure all the necessary requests in a test case, you can choose the sequence that these requests must be executed in. Whenever that test case runs, the requests will be executed in the configured sequence alone. By default, the requests are listed in the order of creation in the test case, with the first request you created on top, followed by the next request you created, and so on. You can arrange the requests in a sequence that you require, by clicking the drag icon on the top right corner of the request window, and dragging it to new location. Chain requests enable you to pass dynamic values between the requests using {{%link href="/en/devops/help/automation-testing/variables" %}}variables{{%/link%}}. Let's discuss this with an example. Assume we configure _Request 1_ to invoke a function and fetch some data using the {{%badge%}}GET{{%/badge%}} method. The response will return a JSON body. We need to extract the values of the keys _id_, _email_, and _city_ from this response and pass them to _Request 2_ as its parameters. We can do this by configuring the variables for these values to be extracted in _Request 1_ as shown below. If you wish to pause for a certain amount of time after executing the first request and before executing the next, you can configure the pause time by clicking {{%bold%}}Pause{{%/bold%}} below the first request. You must configure this before creating the second request. {{%note%}}{{%bold%}}Note:{{%/bold%}} You can use configure the pause time after a request if you expect any wait time in receiving its response. This will make Automation Testing wait for that time before initiating the execution of the next request.{{%/note%}} Let us now create another request and pass the extracted values to it dynamically. We must add corresponding parameters in _Request 2_ to pass these variables as the parameter values. We can do this as shown below. <br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} You can define the local variables in a request by directly declaring their names, and pass them as parameter values in the {{%bold%}} {{\<variable\>}} {{%/bold%}} format. Refer to the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}} for the syntax of Global and Environment variables.{{%/note%}} If we initiate an {{%link href="#manage-a-test-case-and-its-requests" %}}instant run{{%/link%}} after saving the configurations, _Request 1_ will execute first. Assume it returns with the expected JSON response. The values sent for the defined keys _id_, _email_, and _city_ will be extracted from it and assigned to the variables _customer\_id_, _customer\_email_, and _city\_name_ for _Request 1_. These will be passed as the values to the parameters _customerID_, _customerEmail_, and _city_, defined in _Request 2_ respectively, while the request is invoked. You can configure and orchestrate chain requests in this manner. <br> ### Manage All Test Cases and Modules The left panel in the _Test Cases_ section enables you to perform several actions. It lists all the modules that you have created, and all the test cases present in those modules. You can search for a test case using the search bar, and open any test case by navigating to its module from here. #### Create a Module You can create any number of modules to add test cases in. 1. Click {{%bold%}}Add Module{{%/bold%}} on the bottom of the left panel. 2. Enter a name and a description for the module.<br /> <br /> 3. If you are creating a sub-module, click the drop-down list to choose an existing parent module. If you wish to create a new module, ignore this and click {{%bold%}}Create{{%/bold%}}.<br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} {{%link href="#modules-and-testcases" %}}As mentioned earlier{{%/link%}}, you can create sub-modules upto one level. A module cannot contain test cases and sub-modules at the same level. Therefore, you can only choose a parent module if it does not have a test case directly in its root.{{%/note%}} The module will be created and listed in your directory. #### Test Cases Schema You can access a detailed schema of all your modules, sub-modules, and test cases that you configure by clicking {{%bold%}}Manage{{%/bold%}} on the top of the left panel. This will open the details page. You can expand or collapse directories and view the schema of the test cases. You can search for a test case using the search bar, click the list icon corner to view only a list of the test cases, or quickly create a new test case or module from the top-right corner of this section. Click the {{%bold%}}ellipsis icon for a module{{%/bold%}} to perform the following actions: * You can create a new sub-module or a new test case inside the module by selecting {{%bold%}}+Module{{%/bold%}} or {{%bold%}}+Test Case{{%/bold%}} from the menu respectively. * {{%bold%}}Edit a Module:{{%/bold%}} Click {{%bold%}}Edit{{%/bold%}} to update a module's name, description, or move it under a different module or out of a parent module.<br /> <br /> Make the necessary changes and click {{%bold%}}Update{{%/bold%}}. * {{%bold%}}Delete a Module:{{%/bold%}} Click {{%bold%}}Delete{{%/bold%}} to delete a module permanently. <br /> <br /> Click {{%bold%}}Yes{{%/bold%}} in the confirmation pop-up window. {{%note%}}{{%bold%}}Note:{{%/bold%}} A module cannot be deleted if it contains a test case. You must delete all the test cases in a module first before deleting the module.{{%/note%}} * {{%bold%}}Move a Module:{{%/bold%}} You can also quickly move a module under a different module or out of a parent module by clicking {{%bold%}}Move to{{%/bold%}}. This will open a panel on the right. Choose a module or the root, then click {{%bold%}}Move{{%/bold%}}.<br /> * {{%bold%}}Add a Module to a Suite:{{%/bold%}} Click {{%bold%}}Add to Suite{{%/bold%}} to quickly add a module to one or more suites. This will open a panel on the right where you can select the suites. Click {{%bold%}}Apply{{%/bold%}} to save the changes.<br /> <br /> Similarly, if you click the ellipsis icon for a test case from this page, you can access the options shown below. <br /> * {{%bold%}}Delete a Test Case:{{%/bold%}} Click {{%bold%}}Delete{{%/bold%}} from the menu, then click {{%bold%}}Yes{{%/bold%}} in the confirmation pop-up window.<br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} A test case cannot be deleted if it has been referred in a test suite or a test plan. You must first remove the test suite from the test plan, followed by {{%link href="/en/devops/help/automation-testing/test-suites/#manage-the-test-cases-in-a-suite" %}}removing the test case from the test suite{{%/link%}}, and only then delete the test case.{{%/note%}} The _Edit_, _Move To_, and _Add to Suite_ options are the same as discussed {{%link href="#manage-a-test-case-and-its-requests" %}}in this section{{%/link%}}. You can also select multiple test cases using the check boxes in the Manage section to delete, move, or add to a suite in bulk. -------------------------------------------------------------------------------- title: "Test Suites" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.553Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/test-suites/" service: "DevOps" -------------------------------------------------------------------------------- # Test Suites ### Introduction Test suites are the next level of organization of the {{%link href="/en/devops/help/automation-testing/modules-test-cases" %}}test cases{{%/link%}} you create in Automation Testing. A suite facilitates a group of test cases, and a group of test suites can be automated together using a test plan. You can group multiple test cases in a suite and execute them parallelly or sequentially, based on your needs. For instance, if a group of test cases involves sequential actions, such as fetching data from a process and passing it on to another process, you can order the sequence of their execution. If a group of test cases are not related to one another, you can execute them parallelly. Automation Testing enables you to add test cases in any combinations into a suite. That is, you can add test cases belonging to the same module or sub-module, or different modules together. {{%note%}}{{%bold%}}Note:{{%/bold%}} * A test suite can only be created after you have created atleast one test case. * A test case cannot be automated without first adding it to a suite and configuring a test plan for it. {{%/note%}} ### Create a Test Suite You can create a test suite in the following manner: 1. Navigate to the {{%bold%}}Test Suites{{%/bold%}} tab in _Automation Testing_, then click {{%bold%}}Create Suite{{%/bold%}}. 2. Enter a name for the test suite, and select the execution as {{%bold%}}Sequence{{%/bold%}} or {{%bold%}}Parallel{{%/bold%}} from the drop-down list. Select all the test cases to be added in the suite from the schema. You can select any test case from any module. You can arrange the test cases based on the order they must be executed in by dragging and dropping from the right panel. 3. Click {{%bold%}}Create{{%/bold%}}. The suite will be created and the details page will open. ### Manage a Test Suite A test suite's details page will contain the configurations and information of that suite. <br /> After you create a test plan and include this suite in it to be automated for execution, you will be able to view the statistics of this suite. The top section of this page shows the details of the suite, its success rate, and the success stats of the last five test runs. #### Edit a Suite 1. You can edit a suite's name or execution type by clicking the edit icon next to its name on the top of the details page. 2. Make the necessary changes in the pop-up, then click {{%bold%}}Update{{%/bold%}}. #### Add Test Cases or Modules to a Suite 1. You can add test cases or modules by searching for it from the search bar on the left side of the suite's details page. Select the ones to add from the list.<br /> The modules you include will be shown in blue and the test cases in green. 2. You can optionally add these items after a specific execution item, by checking the {{%bold%}}Add cases after{{%/bold%}} box and entering the {{%bold%}}Order ID{{%/bold%}} of the previous execution item. You can find the Order IDs of all execution items on the right side of this page. If you don't enter this, the test cases and/or modules will be added after the last item in the order, by default. 3. Click {{%bold%}}Add{{%/bold%}}. #### Manage the Test Cases in a Suite You can view a list of all the test cases in the suite, and their details on the right side of this section.v This also shows the following reports of the suite's execution: * {{%bold%}}Frequently Failed:{{%/bold%}} Test cases that frequently failed in descending order, based on the number of times they failed * {{%bold%}}Failed in Last Run:{{%/bold%}} Test cases failed in the last execution of the suite {{%bold%}}Configure the Order of Test Case Execution:{{%/bold%}} The test cases will be listed in the configured order of execution. In a sequential suite, they will be executed one after the other in that order. You can change a test case's order, by clicking the {{%bold%}}move{{%/bold%}} icon for it. Enter the Order ID where it should be moved to, then click {{%bold%}}Move{{%/bold%}}. The Order IDs are mentioned in this table's first column. Alternatively, you can click the drag icon on the left for a test case to drag and drop it in a new location. {{%bold%}}Remove a Test Case from a Suite:{{%/bold%}} You can remove a test case from the suite by clicking the remove icon for it in this table. Click {{%bold%}}Confirm{{%/bold%}} in the confirmation pop-up. #### Delete a Test Suite You can find a list of all the test suites you create by clicking the back arrow next to this suite's name in its details page. This page shows important details of each suite. <br /> 1. You can delete a test suite by clicking the delete icon that appears when you hover over the suite in this list. 2. Click {{%bold%}}Yes{{%/bold%}} in the confirmation pop-up window.<br /> You can also select multiple suites from the list using the checkboxes and delete them in bulk. -------------------------------------------------------------------------------- title: "Test Plans" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.554Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/test-plans/" service: "DevOps" -------------------------------------------------------------------------------- # Test Plans A test plan enables the automated execution of one or more {{%link href="/en/devops/help/automation-testing/test-suites" %}}test suites{{%/link%}}. You can configure various details of an execution in a test plan, such as the order of execution of the suites, the {{%link href="/en/devops/help/automation-testing/variables" %}}environment variables{{%/link%}} that need to be applied for the run, the execution schedule, and notification alerts. The detailed report of a test plan's execution can be viewed from the {{%link href="/en/devops/help/automation-testing/results" %}}Results section.{{%/link%}} ### Create a Test Plan You can configure a test plan in the following manner: 1. Click the {{%bold%}}Test Plan{{%/bold%}} tab in Automation Testing, then click {{%bold%}}Create Test Plan{{%/bold%}}.<br /> 2. Enter a name and description for the test plan.<br /> 3. {{%bold%}}Environment Variables:{{%/bold%}} You can select the environment variables to apply for this test run. {{%note%}}{{%bold%}}Note:{{%/bold%}} If you had defined a variable with the same key as both a global and an environment variable, Catalyst will use the value of the environment variable over the value of the global variable during the test plan execution. Refer to the {{%link href="/en/devops/help/automation-testing/variables" %}}Variables section{{%/link%}} for more information.{{%/note%}} 4. {{%bold%}}Test Suites:{{%/bold%}} Click {{%bold%}}Add Test Suites{{%/bold%}} to include at least one test suite that must be executed in this test plan. You can select the test suites from the pop-up window and click {{%bold%}}Save{{%/bold%}}.<br /> You can rearrange the test suites to define their order of execution after they are added, by dragging and dropping them using the {{%bold%}}drag icon{{%/bold%}} on the left. You can also delete a test suite you included by clicking the {{%bold%}}delete icon{{%/bold%}}.<br /> 5. {{%bold%}}Schedule:{{%/bold%}} Configure the schedule of the test run by selecting one of the following {{%bold%}}run types{{%/bold%}}: * {{%bold%}}Nightly:{{%/bold%}} You can configure the run time between 6 pm and 6 am for this type. This will be executed every night. You can select the days of the week this must be executed on, and a test plan start date from the calender. * {{%bold%}}Day:{{%/bold%}} You can configure the run time between 6 am and 6 pm for this type. This will be executed every day. You can select the days of the week this must be executed on, and a test plan start date from the calender. * {{%bold%}}Custom:{{%/bold%}} You can configure this to execute on a specific start date and at any run time in 24 hours. You will not be able to select the days of a week for this. * {{%bold%}}Starts Now:{{%/bold%}} This will initiate the execution right after the test plan is created. You will not be able to configure a schedule for this.<br /> 6. {{%bold%}}Notifications{{%/bold%}}: Click {{%bold%}}Add Notifications{{%/bold%}} to configure notification alerts to be sent for various events in the test plan's execution. This will open a pop-up window.<br /> You can select one of the execution events from the {{%bold%}}When{{%/bold%}} drop-down list: * {{%bold%}}Started:{{%/bold%}} When the test plan execution begins * {{%bold%}}Completed:{{%/bold%}} When the test plan execution completes * {{%bold%}}Success:{{%/bold%}} If the test plan execution is successful * {{%bold%}}Fail:{{%/bold%}} If the test plan execution is successful * {{%bold%}}Not executed (delay):{{%/bold%}} If the test plan does not start at the scheduled time * {{%bold%}}Terminated:{{%/bold%}} If the test plan terminates during its progress without completion The _Success_ and _Fail_ options provide an additional field to specify the percentage of success or failure at which the notification should be triggered.<br /><br /> Select the email addresses of the recipients that must be alerted from the {{%link href="/en/getting-started/set-up-a-catalyst-project/collaborators" %}}collaborators{{%/link%}} that are listed when you click _Recipients email address_ text field. You can type in the email addresses of any other recipients as well.<br /><br /> Enter the notification message to be sent, then click {{%bold%}}Save{{%/bold%}}. You can add multiple notifications for different events, and edit or delete them after they are created. 7. After configuring the test plan, click {{%bold%}}Create{{%/bold%}}. The test plan will be created and listed in the _Test Plans_ section. A unique {{%bold%}}Test Plan ID{{%/bold%}} will be created for it. ### Manage a Test Plan You can find all your test plans in the _Test Plans_ section along with their details. You will be able to perform several actions on a test plan from here. * {{%bold%}}Disable or Enable a Test Plan:{{%/bold%}} Click the toggle under the _Status_ column to enable or disable the scheduled test plan. {{%note%}}{{%bold%}}Note:{{%/bold%}} This will only disable or enable the schedule of the test plan. You can still perform an instant run of the test plan as specified below.{{%/note%}} * {{%bold%}}Run a Test Plan:{{%/bold%}} You can perform an instant run of the test plan to verify its outcome by clicking the run icon. You will be able to view the results of this run from the {{%link href="/en/devops/help/automation-testing/results" %}}Results section{{%/link%}}. A unique {{%bold%}}Run ID{{%/bold%}} will be generated for each automated and instant test plan execution, that will be shown in the Results section. * {{%bold%}}Delete a Test Plan:{{%/bold%}} Click the delete icon, then click {{%bold%}}Yes{{%/bold%}} on the confirmation pop-up to delete the test plan. You can select multiple test plans using the checkboxes to delete them in bulk. -------------------------------------------------------------------------------- title: "Results" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.554Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/results/" service: "DevOps" -------------------------------------------------------------------------------- # Results The Results section displays a detailed and comprehensive report of all the test plans that were executed. This shows the results of the automated test plan executions that were scheduled, and the instant runs performed from the _Test Plans_ section. This will not show the results of the {{%link href="/en/devops/help/automation-testing/modules-test-cases/#manage-a-test-case-and-its-requests" %}}live previews{{%/link%}} executed in the _Test Cases_ section. You can check those results from the respective sections. Click the {{%bold%}}Results{{%/bold%}} tab in Automation Testing to access the results. The Results section displays results in three levels: ### All Test Runs All test runs of all test plans will be displayed in the initial Results page. You can view stats such as the number of completed and currently running executions, the overall success rate of all test plan executions, and the number of environments that were involved in the all runs. The last one is based on the environment variables that you select while {{%link href="/en/devops/help/automation-testing/test-plans/#create-a-test-plan" %}}creating the test plans{{%/link%}}. You can find a list of all test runs along with details like their unique Run IDs, number of test cases included in a run, the success percentage of the run, its start time and execution duration. You can sort results based on the criteria shown below, to display runs accordingly. You can also filter the results by clicking the filter button and selecting various filters. * {{%bold%}}Date:{{%/bold%}} You can select one of these options for the date filter: _Last 24 hours_, _Last 7 days_, _Last 1 month_, _Custom_ to view results only of that time period. The _Custom_ option enables you select a date window based on your needs. * {{%bold%}}Status:{{%/bold%}} You can choose any one of these execution statuses to view relevant results: _Completed_, _Running_, _Queued_, _Terminated_. * {{%bold%}}Test Plan:{{%/bold%}} You can also filter the results based on a specific test plan. All executions of that test plan alone will be displayed. Click {{%bold%}}Apply{{%/bold%}} after selecting the filters. If the results take some time to reflect, click {{%bold%}}Refresh{{%/bold%}} on the top of this page. <br> ### Details of a Particular Test Run You can open a specific test run from the test runs list to view the details of the test cases included in the test plan. You can view stats like the total number of test cases, number of test cases that are completed, remaining, or inactive, and the overall success rate of that test run. It lists all the test cases that were included in the test plan, along with the overall results and other details of each test case. You can also sort the results based on the duration of the test case execution. You can filter results by clicking the filter icon and selecting various details. You can view the test cases of a particular suite alone, or test cases that executed with any one of these statuses: _Completed_, _Passed_, _Failed_, _Inactive_, _Remaining_. Click {{%bold%}}Apply{{%/bold%}} after selecting the filters. <br> ### Details of a Particular Test Case You can open a particular test case from the test case list to view the execution details of the individual requests in it. This section provides a summary of the particular test case along with useful statistics such as the overall result and duration of the test case execution, and the overall number of assertions that were passed in the test case. It lists the details of individual requests in the test case, along with the detailed response received for each request. You can expand a request's section to view the assertions that were passed in that request and the response status code. The report also shows the response body, the response headers, and the request details. You can view the {{%bold%}}Response Body{{%/bold%}} in the raw or pretty format by selecting a choice from the drop-down on the right. Click the {{%bold%}}Response Headers{{%/bold%}} or {{%bold%}}Request Details{{%/bold%}} tab to view those information. This drill down enables you to easily identify the exact request a test case failed on, with the specific reason. You can navigate to view the details of the next test case using the arrows on the top-right corner. <br> ### Details of a Failed Test Run As discussed earlier, in the event of a failed test run, an automatic re-run can be configured. The status and the details of the re-run can be viewed in the *Results* section. <br /> In the *Results* section you can expand the failed test plan and get a useful overview of the number of test runs that were performed, number of test cases that were tested, their status, the duration, and the overall results. The overview of all the test runs and re-runs can be viewed here. <br /> **Re-Run 1** in this image denotes that an automatic re-run on failure was performed, and the overview of that test run is also displayed alongside. You can click a particular failed test run to view a detailed report of it. <br> ### Details of a Particular Failed Test Run Once you click a failed test run, you can view all the overview stats present in the *Results* section, and view the status and the results of the **Pre-Condition** and the **Post-Condition** test cases as well, along with their details. <br /> At the top of the screen, beside the name of the test plan, you can click the dropdown and select **Re-Run 1** to view the status and results of the re-run that was automatically executed upon failure. <br /> ### Manually Run a Test Plan with a Failed Re-Run 1. If the re-run is also a failure, you can manually run the failed test plan by clicking the **Run** button on the top right-hand corner. <br /> 2. Click **Proceed** to start a manual re-run. <br /> Click the respective test plan on which you performed the manual re-run to view its status. <br /> This drill down enables you to easily identify the exact request a test case failed on with the details. You can re-run a failed test plan any number of times in the same way, until its execution is a success. If its execution succeeds, the run button will no longer be displayed in this section. The detailed log of all the previous test cases be viewed by clicking the drop-down selection and choosing the required test run. <br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} You can still manually run any test plan from the Test Plan section, whether or not you configured an automatic re-run of failed test plans.{{%/note%}} -------------------------------------------------------------------------------- title: "Variables" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.555Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/variables/" service: "DevOps" -------------------------------------------------------------------------------- # Variables ### Introduction A variable is essentially a placeholder for a dynamic value that you might pass in your request. Storing a frequently used value as a variable and referring to it in your test cases makes it easier when you have to change its value, because you need not change it in each test case you have used it in. You can just edit the configured variable. You can configure different types of variables in Automation Testing that are used for different purposes. One way of using variables is defining and storing values for fixed parameters in the _Variables_ section, that you can easily refer to in your requests. This involves the following two types of variables: * {{%bold%}}Global Variables:{{%/bold%}} The global variables is a single set of variables that you can define and store to be used in any of your test cases. These variables have a global scope, and can therefore be applied to all test cases. * {{%bold%}}Environment Variables:{{%/bold%}} Environment variables help you categorize and configure values of the same keys for different environments. When you configure a {{%link href="/en/devops/help/automation-testing/test-plans" %}}test plan{{%/link%}} or perform an {{%link href="/en/devops/help/automation-testing/modules-test-cases/#manage-a-test-case-and-its-requests" %}}instant run of a test case{{%/link%}}, you can choose the environment that must be used for the execution. The variables defined in that environment will be applied. These variables can also be used globally in all test cases. Another way of using variables in Automation Testing is extracting values received from the responses of individual requests in test cases, and assigning them to variables to pass them as parameters in the next requests in a chained set up. The scope of such variables defined within a request is only limited to that test case, and they cannot be referred outside of that test case. This method is discussed in detail with an example in the {{%link href="/en/devops/help/automation-testing/modules-test-cases/#chain-requests" %}}Chain Requests section{{%/link%}}. #### An Example for Global and Environment Variables: You can store different values of one variable in each single environment. For example, the value of ZAID in Catalyst is different for the Development and Production environments. In that case, you can first create two environments named "Development" and "Production". You can then create a variable with the name "zaid" in both of these environments, and define its respective value in both separately. While configuring a test plan, if you choose _Development_, the value defined for zaid in the development environment will be applied, wherever you have used the variable zaid in your requests. You can also define the same variable as a global variable, as well as in each environment separately. For example, you can create a variable _baseURL_ as a global variable and define its value as "https://globaltest.com". You can then create the same _baseURL_ variable in the Development and Production environments with the values "https://developmenttest.com" and "https://productiontest.com" respectively. Now, assume you refer to _baseURL_ as the request URL while {{%link href="/en/devops/help/automation-testing/modules-test-cases" %}}configuring the request of a test case{{%/link%}}. If you choose _Development_ as the environment variables to use while performing an {{%link href="/en/devops/help/automation-testing/modules-test-cases/#manage-a-test-case-and-its-requests" %}}instant run{{%/link%}}, the value https://developmenttest.com will be used as the request URL. If you choose _Production_, the value "https://productiontest.com" will be used. If you don't select any environment for the instant run, the global value "https://globaltest.com" will be used by default. #### Important points to remember: * Creating the same variable as a global variable and as different environment variables is assigning multiple values to the same key. * You can create a global variable, and have corresponding environment variables. If you create a new variable only in a specific environment, it will still be listed in the global variables section with the same value. * Global variables are used by default in a test plan or an instant run, when no environment is chosen. Only when you select an environment, the environment's values will override the global values of the variable. * The local variables you create inside a specific request can be declared directly by their names, and they must be {{%link href="/en/devops/help/automation-testing/modules-test-cases/#configure-a-request" %}}passed as parameter values{{%/link%}} in other requests in the {{%bold%}}\{\{\<variable \>\}\} {{%/bold%}} format. * You must pass the Global and Environment variables in the {{%bold%}}\{\{$\<variable \>\}\}{{%/bold%}} format inside a request. ### Create Global Variables You can create global variables in the following manner: 1. Click the {{%bold%}}Variables{{%/bold%}} tab in _Automation Testing_, then click {{%bold%}}Create Variable{{%/bold%}}.<br /> <br /> 2. Enter the names and values for the variables, then click {{%bold%}}Save{{%/bold%}}.<br /> <br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} Variable names must not include spaces or start with a number. Variable names are also case-sensitive.{{%/note%}} You can also add multiple variables in bulk by clicking {{%bold%}}Bulk Add{{%/bold%}} on the top-right corner. Define the variables in the {{%badge%}}key:value{{%/badge%}} format in the pop-up window, then click {{%bold%}}Save{{%/bold%}}.<br /> All these variables will be added and listed.<br /> ### Create Environment Variables Before you create environment variables, you must create an environment. 1. Click {{%bold%}}Environment Variables{{%/bold%}} from the _Variables_ section. 2. Enter a name for the environment in the pop-up window, then click {{%bold%}}Create{{%/bold%}}.<br /> 3. You can now create variables in that environment in the same way you created Global variables. Enter the names and values of the variables, then click {{%bold%}}Save{{%/bold%}}.<br /> In an environment's section, all your global variables will be listed when you click the Name text box. You can select one of them to assign a different value for that variable for this environment, or you can type in the text box to create a completely new variable. This variable will also be listed in the global section. You can also add multiple environment variables in bulk by clicking {{%bold%}}Bulk Add{{%/bold%}} on the top right corner, in the same as adding global variables in bulk. ### Manage Global and Environment Variables You can choose to open the global variables or a specific environment's variables, by clicking the drop-down on top. Select the necessary option. * {{%bold%}}Edit an Environment's name:{{%/bold%}} Click the edit icon next to the environment's name. Make the changes in the pop-up window and click {{%bold%}}Update{{%/bold%}}.<br /> <br /> * {{%bold%}}Delete an Environment:{{%/bold%}} Click the delete icon next to the environment's name. Click {{%bold%}}Yes{{%/bold%}} in the pop-up window.<br /> {{%note%}}{{%bold%}}Note:{{%/bold%}} If you delete an environment or a Global or Environment variable, it might affect your configured test cases and result in data loss. Make sure that you proceed with caution.{{%/note%}} * {{%bold%}}Delete a Variable:{{%/bold%}} You can delete a Global or an Environment variable by clicking the edit icon that appears when you hover over its name.<br /> <br /> Click {{%bold%}}Yes{{%/bold%}} in the confirmation pop-up window.<br /> <br /> -------------------------------------------------------------------------------- title: "Pricing" description: "Automation Testing enables you to execute efficient, automated API tests on function end-points or any third-party URL and resolve problem areas." last_updated: "2026-03-18T07:41:08.555Z" source: "https://docs.catalyst.zoho.com/en/devops/help/automation-testing/pricing/" service: "DevOps" -------------------------------------------------------------------------------- # Pricing All other Catalyst components are offered free of charge for you to configure and work with in the {{%link href="/en/deployment-and-billing/environments/development-environment" %}}Development environment{{%/link%}}. You will generally be asked to set up {{%link href="/en/deployment-and-billing/billing/introduction" %}}Catalyst payments{{%/link%}} while you deploy an application to Production. However, as mentioned in the Introduction, since Automation Testing is only accessible in the Development environment and not in Production, Catalyst provides this component as partially paid in the Development environment. The criteria on which Automation Testing is price is the number of {{%bold%}}test runs{{%/bold%}} that are executed in this component. A single test run is the execution of one test case. In other words, each test case execution listed in the Results section in the {{%link href="/en/devops/help/automation-testing/results/#details-of-a-partiular-test-run" %}}_Details of a Particular Test Run level_{{%/link%}} is counted as one test run. For example, if _Test Plan A_ includes a few suites that contain 20 test cases in total, and _Test Plan A_ has executed 3 times overall, then that would make the total number of test runs as 60\. In this way, the number of test cases executed in each test plan during each execution is counted, to determine the total number of test runs. Catalyst offers the {{%bold%}}first 500 test runs for free{{%/bold%}}. The _Free trial runs left_ count on the bottom of Automation Testing page in all sections constantly displays the number of free test runs remaining out of 500. This will show the updated count after each test run has executed. When this limit is exhausted, Catalyst will begin charging for each test run from thereon. You can view the exact cost and other details from the {{%link href="https://www.zoho.com/catalyst/pricing.html" %}}Pricing section on our website{{%/link%}}. If you have not previously set up Catalyst payments before, you must add your card to enable Catalyst payments. You can click on the _Free trial runs left_ count to open a pop-up window. You can then click on {{%bold%}}Add Card{{%/bold%}} button to set up the payment method. Refer to the {{%link href="/en/deployment-and-billing/billing/introduction" %}}Billing help document{{%/link%}} for detailed help on setting up Catalyst payments and managing billing. Once you have exhausted your 500 free trial runs limit, Catalyst will also display a banner on top of the Automation Testing page with an {{%bold%}}Add Card{{%/bold%}} button. After you have set up the payment method, the free trial runs will not be displayed in the bottom. --- ## API Reference — Automation Testing -------------------------------------------------------------------------------- title: "Execute Test Plan" description: "This API enables you to trigger the execution of a test plan in Automation Testing that is already configured in the remote console." last_updated: "2026-03-18T07:41:08.561Z" source: "https://docs.catalyst.zoho.com/en/api/code-reference/devops/automation-testing/execute-test-plan/" service: "DevOps" -------------------------------------------------------------------------------- {{% section path="/en/api/code-reference/devops/automation-testing/execute-test-plan/" id="ExecuteTestPlan" %}} {{%divtag class="LeftContent"%}} # Automation Testing {{%link href="/en/devops/help/automation-testing/introduction/"%}}Automation Testing{{%/link%}} is an API testing tool that automates the testing process of the APIs in your application. You can test the Basic I/O and Advanced I/O functions in your application, or any third-party URL, and obtain responses. You can write test cases based on your requirements, schedule tests to be executed automatically, and manage the entire process of API testing with minimal manual intervention. {{%note%}}{{%bold%}}Note:{{%/bold%}} Automation Testing is currently not available to Catalyst users accessing from the EU, AU, IN, or CA data centers. {{%/note%}} # Execute a Test Plan ### Description A {{%link href="/en/devops/help/automation-testing/test-plans/"%}}test plan{{%/link%}} enables the automated execution of one or more {{%link href="/en/devops/help/automation-testing/test-suites/"%}}test suites{{%/link%}} in Automation Testing. This API enables you to trigger the execution of a test plan that is already configured in the remote console in your project. This is equivalent to the {{%link href="/en/devops/help/automation-testing/test-plans/#manage-a-test-plan"%}}instant run option{{%/link%}} available for a test plan in the console. ### Request Details #### Request URL {{%request class="apiyellow" req="POST" %}}<p>{api-domain}/baas/v1/project/{project_id}/automation-testing/{test_plan_id}/execute</p>{{%/request%}} {{%request_id id="api-domain" data="URL" %}}The DC-specific domain to access Catalyst API{{%/request_id%}} {{%request_id id="project_id" data="Numerical" %}}The {{%link href="/en/getting-started/set-up-a-catalyst-project/general-settings/"%}}unique ID{{%/link%}} of the project{{%/request_id%}} {{%request_id id="test_plan_id" data="Numerical" %}}The {{%link href="/en/devops/help/automation-testing/test-plans/#manage-a-test-plan"%}}unique ID{{%/link%}} created for a test plan.{{%/request_id%}} #### Request Headers {{%scope%}} **Authorization:** Zoho-oauthtoken {oauth_token} <br> **Environment:** Development {{%/scope%}} **Optional Headers** <br> {{%scope%}} **CATALYST-ORG:** {org_id} {{%/scope%}} {{%scope%}} **Environment:** Development {{%/scope%}} #### Scope {{%scope%}} ZohoCatalyst.automationtesting.execute {{%/scope%}} ### Response Details The response contains the status of the test plan execution, along with details such as the unique Run ID that will be generated for each automated or instant test plan execution, the Test Plan ID that was passed, and the {{%badge%}}is_schedule_started{{%/badge%}} parameter that contains the status of the execution initiation as {{%badge%}}true{{%/badge%}} or {{%badge%}}false{{%/badge%}}. {{%/divtag%}} {{%divtag class="RightContent"%}} {{% panel_without_adjustment header="Sample Request" class="language-bash" %}}curl -X POST \ https://api.catalyst.zoho.com/baas/v1/project/3359000000016812/automation-testing/537000000022025/execute \ -H "Authorization: Zoho-oauthtoken 1000.25*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*e.3e*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*3" \ {{% /panel_without_adjustment %}} {{% panel_without_adjustment header="Sample Response" class="language-json line-numbers"%}} { "status": "success", "data": { "id": 537000000034053, "test_plan_id": 537000000022025, "is_schedule_started": true } }{{% /panel_without_adjustment %}} {{%/divtag%}} {{%/section%}}