Third-Party Migration

Introduction

Catalyst provides the ability to migrate NoSQL databases from third-party sources. This migration is supported only for databases of the NoSQL type and not for relational or other databases.

The provision to migrate a database from an external source enables you to easily switch to Catalyst for your NoSQL needs. If you had been maintaining NoSQL databases for Catalyst applications or microservices in an external environment, you can benefit from integrating it within your Catalyst ecosystem and manage the entire setup here.

You can perform the migration from the Catalyst console with just the click of a few buttons. All you would need to do to migrate databases is to provide the required credentials to access the third-party database, and choose the tables to migrate. Catalyst ensures you face zero downtime or latency issues during the migration process. After the database is migrated, you can access the tables in the NoSQL section in console inside your project.

Catalyst currently supports migration from this third-party source: AWS DynamoDB

There are two different migration options that you can utilize:

• Migrating Metadata: This option will only migrate the schema and metadata of your tables from the third-party source. This is available only in the development environment in Catalyst.

• Migrating Data: This option will migrate the data of your tables along with the configurations from the third-party source. This is available only in the production environment in Catalyst.

You can choose the appropriate option you require and perform migration from the suitable environment.

Migrating from AWS DynamoDB

To perform migrations from AWS, you will be required to configure AWS Access Key Credentials with Catalyst. The AWS access key credentials allow you to access AWS resources through their CLI or API.

AWS access keys are composed of two parts: Access Key ID and Secret Access Key. You will need to configure both in Catalyst, and also provide information about the AWS region your AWS account fall under.

Migrate Metadata from DynamoDB

You can migrate metadata of the tables in your AWS DynamoDB while you work in the development environment of Catalyst. This is the default environment in Catalyst and therefore, you can access this migration option by default. You can use this option if you prefer creating new data for your database, and only require the table configurations.

The metadata migration will migrate the table configurations and index configurations of the tables you select, such as their configured partition keys and sort keys, as well as any other meta information or configurations that Catalyst NoSQL supports.

To migrate metadata from AWS DynamoDB into Catalyst NoSQL in your project:

1. Navigate to NoSQL from the CloudScale section from the development environment in the console, then click Third-Party Meta Migration.

2. This will open a pop-up. Enter the Access Key ID, Secret Access Key, and the region of your AWS account. Then, click Next.

3. Select the tables whose metadata you want to migrate from your DynamoDB. Then, click Import.

Catalyst will then schedule the migration process. The time taken to complete the migration depends on the number of tables and their sizes.

After the tables are migrated, you will be notified.

You can then access the imported tables in your project by refreshing the console. You will find the imported tables with their configurations in NoSQL, and you can then proceed to add data.

Migrate Data from DynamoDB

To migrate data of the tables in your AWS DynamoDB, you will first need to access the production environment in Catalyst. If you have never deployed your Catalyst project to production, you must first do that to enable production. Refer to this help page for deploying projects to the production environment.

Once you are able to access the production environment of your project, you can switch to the production environment in the console from the header.

You can use the option to migrate data along with the schema if you require the entire database to be migrated. This will import the data of the tables you select, as well as the configurations and metadata of the tables you select. This includes the table configurations and index configurations of the tables you select, such as their configured partition keys and sort keys, as well as any other meta information or configurations that Catalyst NoSQL supports.

To migrate data from AWS DynamoDB into Catalyst NoSQL in your project:

1. Navigate to NoSQL from the CloudScale section from the production environment in the console, then click Third-Party Data Migration.

2. This will open a pop-up. Enter the Access Key ID, Secret Access Key, and the region of your AWS account. Then, click Next.

3. Select the tables whose data you want to migrate from your DynamoDB. Then, click Import.

Catalyst will then schedule the migration process. The time taken to complete the migration depends on the number of tables, and their sizes based on the volume of the data.

After the tables are migrated, you will be notified.

You can then access the imported tables with their data in your project. You can update or delete items, or process the data as required.