Components of NoSQL

To understand the architecture of Catalyst NoSQL databases and learn about working with them, you must first understand the components that make up NoSQL databases.

Basic Components

The fundamental components involved in Catalyst NoSQL are explained below.

Terminology

Sample Table Representation

Table Name: Travel

    

        
copy
        

            

        
    
    

         {
 "item": {
  "DestinationName": {
		"S": "Honolulu"
			},
 	"Distance": {
		"N": 4960
		},
	 "BestTimeToVist": {
		"L":
		   [
			{
  "S": "September"
			},
			{
  "S": "October"
			},
			{
  "S": "November"
			}
   ]
			}, 
"EstimatedCost": {
		"N": 2500
			}
},
	"item": {
 "DestinationName": {
		"S": "Prague"
			},
 	"Distance": {
		"N": 4081
			},
	"Location": {
		"M": {
  	"Country": {
			"S": "Czech Republic"
				},
 	"Continent": {
			"S": "Europe"
				}
			}
  	},
 "EstimatedCost": {
			"N": 3000
			}
 },
"item": { 
	"DestinationName": {
		"S": "Marrakesh"
			},
 "Distance": {
		"N": 3700
			},
 "BestTimeToVist": {
		"L":
		[
			{
   "S": "March"
			},
			{
   "S":	"April"
			},
			{
   "S": "September"
			},
			{
 	"S": "October"
			}
  	]
	}, 
 "Location": {
			"M": {
  	"Country": {
			"S": "Morocco"
			},
		"Continent": {
			"S": "Africa"
			}
  	}
		},
 "EstimatedCost": {
		"N": 1700
		}
 }
}

    

The Travel table represents a simple NoSQL table in the Custom JSON format. The structure includes nested attributes and arrays: the BestTimeToVist is present as an array of the data type list (“L”), and the Location attribute is a nested attribute of the data type map (“M”) containing the keys Country and Continent.

Table Keys

Following a typical NoSQL architecture, Catalyst NoSQL stores data in partitions across clusters, also known as shards. These shards can also be replicated over many servers in mirrors. This distributed storage across multiple nodes breaks down datasets into smaller chunks and promotes optimization.

While querying data in a NoSQL table, we must be able to locate the partition a specific item is stored in. Moreover, sorting through a partition to locate an item also involves additional efforts. These are accomplished with the help of distinct keys that identify partitions or sort through them, as described below.

Terminology

Sample Table Representation

Table Name: Menu

    

        
copy
        

            

        
    
    

         {
 "item": {
	"Category": {
		"S": "Pizza"
			},
 "DishName": {
		"S": "Vegan"
			},
 "Calories": {
		"N": 350
			},
 "PreparationTime": {
		"N": 20
			},
 	"Cost": {
		"N:" 25
			},
 "DiscountPercentage": {
		"N": 5
			},
 "KeyIngredients": {
		"L": [
 		{
		"S": "Pizza Base"
		},
		{
 		"S": "Pizza Sauce"
		},
		{
 		"S": "Olives"
		},
		{
 		"S": "Bell Peppers"
		},
		{
 		"S": "Corn"
		},
		{
 		"S": "Onions"
		}
 		]
		}
	},
	"item": {
 	"Category": {
		"S": "Pizza"
			},
 "DishName": {
		"S": "Chicken Overload"
			},
 "Calories": {
		"N": 450
			},
 "PreparationTime": {
		"N": 30
			},
 "KeyIngredients": {
 	"L":	[
			{
   "S": "Pizza Base"
			},
			{
 		"S": "Pizza Sauce"
			},
			{
 		"S": "Fried Chicken"
			},
			{
		"S": "Paprika"
			}
 	]
	}
	},
	"item": {
 "Category": {
		"S": "Sandwich"
			},
 "DishName": {
		"S": "Vegan"
			},
 "Calories": {
		"N": 300
			},
	"PreparationTime": {
		"N": 20
			},
 	"Cost": {
		"N": 25
		},
"DiscountPercentage": {
		"N": 15
		}
}
}

    

The Menu table includes various attributes of which Category is the partition key, DishName is the sort key, and PreparationTime and Calories are additional sort keys.

The composite primary key enhances data querying. If this table was configured with just the simple primary key, i.e., with Category (example: “Pizza”), then querying it will fetch all items of that Category (all the “pizzas” in the menu). However, because it is configured with a composite primary key, i.e, with Category and DishName, then querying it with both the keys will retrieve a unique item of those values (example: “Pizza” and “Vegan”).

Similarly, the additional sort keys provide further flexibility while querying the table within the same partition. You can choose to query the table with the additional sort key PreparationTime in combination with the partition key Category (example: “Sandwich” and “20”).

Before discussing indexing, let’s understand a special attribute that Catalyst NoSQL provides for all tables.

Time To Live (TTL)

Time To Live (TTL) provides an automated way of deleting items in the Catalyst NoSQL tables after a specific point of time. You can configure an attribute to hold the expiration timestamp in your table while you create it. You can then add the expiration timestamp of each item in the table in the Unix epoch time format while adding the data. Catalyst will permanently delete the items in the table past their expiration time automatically, saving you from performing manual updates and consuming write throughputs.

The TTL attribute is handy in removing items that will not be needed beyond a specific point of time or whose validity is short-lived. The items that are deleted by Catalyst automatically after they are past their expiration date will be removed from the table and from anywhere they are indexed.

Catalyst operates a built-in scheduler once every 24 hours to check for any items whose TTL attribute values indicate expired. This check is performed in all NoSQL tables of all your projects, and the items past their expiration are deleted permanently.

Let’s understand TTL with an example. Assume we create an attribute named Expiry to store the TTL timestamp in the Menu table given in the previous section. We can then define the expiration timestamp for all the items in the table in the following manner.

Sample Table Representation

Table Name: Menu

    

        
copy
        

            

        
    
    

         {
 "item": {
	"Category": {
 	"S": "Pizza"
  	},
 "DishName": {
 	"S": "Vegan"
  	},
 "Calories": {
 	"N": 350
  	},
 "PreparationTime": {
 	"N": 20
  	},
 	"Cost": {
 	"N:" 25
  	},
 "DiscountPercentage": {
 	"N": 5
  	},
"Expiry": {
		"N": 1715755926
		}
	},
	"item": {
 	"Category": {
 	"S": "Pizza"
  	},
 "DishName": {
 	"S": "Chicken Overload"
  	},
 "Calories": {
 	"N": 450
  	},
 "PreparationTime": {
 	"N": 30
  	},
	"Expiry": {
 	"N": 1715565610
 	}
	},
	"item": {
 "Category": {
 	"S": "Sandwich"
  	},
 "DishName": {
 	"S": "Vegan"
  	},
 "Calories": {
 	"N": 300
  	},
	"PreparationTime": {
 	"N": 20
  	},
 	"Cost": {
 	"N": 25
 	},
"DiscountPercentage": {
 	"N": 15
 	}
"Expiry": {
 		"N": 1715700869
 		}
}
}


    

Points to remember:

• You will be able to update the TTL attribute value for items in a NoSQL table anytime you require. When the built-in Catalyst scheduler runs, it will identify and process the items based on the latest TTL value.

• You can also delete the TTL attribute for items as per your needs, based on the given values