Configure the Cron Function

Next, we’ll begin coding the news application by configuring the cron function component.

The Cron function is initialized in Node.js its directory, functions/NewsApp, contains:

• The index.js main function file
• The catalyst-config.json configuration file
• Node modules
• package.json and package-lock.json dependency files

We will be adding code in the index.js file. You can use any IDE to configure the function.

As mentioned in the introduction, the cron function performs two tasks: making the API calls to the NewsAPI to fetch news, populating the news in Catalyst Data Store. The API calls are made using an API key provided by NewsAPI.

Register with NewsAPI

Before you code the cron function, you must register for a free developer subscription with NewsAPI and obtain the API key in the following way:

1. Visit https://newsapi.org/register.

2. Provide the required details and click Submit.

After your registration is complete, NewsAPI will provide you with an API key. You must use this in your cron function, as instructed after the code section.

Install Packages for Node.js

The Node.js Cron function requires two packages to be installed: express and axios.

express

We will use the Express framework to manage the routing operations that enable us to fetch and delete files.

To install express, navigate to the function’s directory (functions/NewsApp) in your terminal and execute the following command:

This will install the Express module and save the dependencies.

axios

axios is a promise-based HTTP client that we will use to fetch data by executing the NewsAPI.

To install axios, navigate to the Node function’s directory (functions/NewsApp) and execute the following command:

This will install the module.

Information about these packages will also be updated in the package.json file of the Cron function. You can now add the code in the function file.

Copy the code below and paste it in index.js located in functions/NewsApp directory and save the file.

    

        
index.js
        

            
copy
            

                

            
        
    
    

    

         "use strict";

const catalyst = require("zcatalyst-sdk-node");
const axios = require("axios");

const HOST = "https://newsapi.org";
const TABLENAME = [
  "HEADLINES",
  "BUSINESS",
  "ENTERTAINMENT",
  "HEALTH",
  "SCIENCE",
  "SPORTS",
  "TECHNOLOGY",
];
const COUNTRY = "IN";
const APIKEY = "Your_API_Key";
const DATASTORE_LOAD = 5;

const pushNewstoDatastore = async ({ table, rowIds, articles }) => {
  return Promise.all(
    articles.map(async (article, idx) => {
      const payload = {
        title: article.title,
        url: article.url,
      };
      if (rowIds.length === 0) {
        //insert the new row
        return table.insertRow(payload);
      }
      return table.updateRow({ ...payload, ROWID: rowIds[idx] });
    })
  );
};

module.exports = async (_cronDetails, context) => {
  try {
    const catalystApp = catalyst.initialize(context);
    const zcqlAPI = catalystApp.zcql();
    const datastoreAPI = catalystApp.datastore();
    //async fetch news and query table
    const metaArr = await Promise.all(
      TABLENAME.map(async (table) => {
        //construct request path to newsapi
        let url = `${HOST}/v2/top-headlines?country=${COUNTRY}&apiKey=${APIKEY}`;
        if (table !== TABLENAME[0]) {
 		url += "&category=" + table;
        }
        const response = await axios({
 		method: "GET",
 		url
 	  });
 	let data = response.data;

        //query using zcql to check if row exists
        const queryResult = await zcqlAPI.executeZCQLQuery(
          `SELECT ROWID FROM ${table}`
        );
        return {
          table_name: table,
          table: datastoreAPI.table(table),
          zcql_response: queryResult,
          articles: data.articles.splice(0, 15),
        };
      })
    );
    //sync insert/update to datastore
    for (const meta of metaArr) {
      let rowIds = [];
      while (meta.articles.length > 0) {
        const chunk = meta.articles.splice(0, DATASTORE_LOAD);
        if (meta.zcql_response.length > 0) {
          rowIds = meta.zcql_response
            .splice(0, DATASTORE_LOAD)
            .map((row) => row[meta.table_name].ROWID);
        }
        await pushNewstoDatastore({ ...meta, articles: chunk, rowIds });
      }
      console.log(
        `${meta.table} table ${
          rowIds.length === 0 ? "inserted" : "updated"
        } with current news'`
      );
    }
    context.closeWithSuccess();
  } catch (err) {
    console.log(err);
    context.closeWithFailure();
  }
};

         
View more
    

The Cron function is now configured. We will discuss the application’s architecture after you configure the client.