Bulk Write Rows

# Bulk Write Rows

Catalyst enables you to perform bulk write jobs on a specific table present in the Data Store. The bulk write operation can fetch thousands of records from a CSV file uploaded in Stratus and insert them in a specific table.

The table is referred to by its unique table ID that is generated by Catalyst during creation. The column in which the write operation must be performed is referred to by its unique column ID.

Note: To perform a bulk write operation, you must first upload the required data as a CSV file in Stratus.

<table class="content-table">
	<thead>
		<tr>
			<th class="w20p">Parameter Name</th>
      <th class="w20p">Data Type</th>
      <th class="w60p">Definition</th>
		</tr>
	</thead>
	<tbody>
		<tr>
			<td>tableID</td>
      <td>String</td>
			<td>A Mandatory parameter. Will hold the ID of the table to which data has to be written.</td>
		</tr>
        <tr>
			<td>find_by</td>
      <td>String</td>
			<td>A Mandatory parameter. Will hold the name of the column to which data has to be written.</td>
		</tr>
        <tr>
			<td>fk_mapping</td>
      <td>Array</td>
			<td>A Mandatory parameter. Will hold the local_column and reference_column details.</td>
		</tr>
        <tr>
			<td>operation</td>
      <td>String</td>
			<td>A Mandatory parameter. The value of this parameter should be insert.</td>
		</tr>
    <tr>
      <td>object_details</td>
      <td>JSON Object</td>
      <td>
        <ul>
          <li>bucket_name: The name of the bucket, where the object is stored.</li>
          <li>object_key: Can contain the path or the Object URL of the required object.</li>
          <li>version_id: If the bucket has versioning enabled, then the specific versionID of the file will be stored in this attribute.</li>
        </ul>
      </td>
    </tr>
	</tbody>
</table>

Copy the SDK snippet below to perform a bulk write job on a particular table. To know more about the component instance datastore_service used below, please refer to this help section.

datastore_service = app.datastore()
bulk_write = datastore_service.table("Sample").bulk_write()
object_details = {
  "bucket_name": "zcstratus12345",
  "object_key": "sample.csv",
  "version_id": "64832huidksnd83"
}
#create bulk write job
bulk_write_job = bulk_write.create_job(object_details, {
  "find_by": "S1",
  "fk_mapping": [
    {"local_column": "EmployeeID", "reference_column": "EmpId"},
    {"local_column": "DepartmentID", "reference_column": "DepId"}
  ],
  "operation": "insert"
})
#get bulk write status
status = bulk_write.get_status('6759000000167103')
#get bulk write result
result = bulk_write.get_result('6759000000167103')

<br />

Note: A maximum of 100,000 rows can be written at one time.

Info : Refer to the SDK Scopes table to determine the required permission level for performing the above operation.

The table is referred to by its unique table ID that is generated by Catalyst during creation. The column in which the write operation must be performed is referred to by its unique column ID.

Note: To perform a bulk write operation, you must first upload the required data as a CSV file in Stratus.

Parameter Name	Data Type	Definition
tableID	String	A Mandatory parameter. Will hold the ID of the table to which data has to be written.
find_by	String	A Mandatory parameter. Will hold the name of the column to which data has to be written.
fk_mapping	Array	A Mandatory parameter. Will hold the local_column and reference_column details.
operation	String	A Mandatory parameter. The value of this parameter should be insert.
object_details	JSON Object	bucket_name: The name of the bucket, where the object is stored. object_key: Can contain the path or the Object URL of the required object. version_id: If the bucket has versioning enabled, then the specific versionID of the file will be stored in this attribute.

Copy the SDK snippet below to perform a bulk write job on a particular table. To know more about the component instance datastore_service used below, please refer to this help section.

copy

         datastore_service = app.datastore()
bulk_write = datastore_service.table("Sample").bulk_write()
object_details = {
  "bucket_name": "zcstratus12345",
  "object_key": "sample.csv",
  "version_id": "64832huidksnd83"
}
#create bulk write job
bulk_write_job = bulk_write.create_job(object_details, {
  "find_by": "S1",
  "fk_mapping": [
    {"local_column": "EmployeeID", "reference_column": "EmpId"},
    {"local_column": "DepartmentID", "reference_column": "DepId"}
  ],
  "operation": "insert"
})
#get bulk write status
status = bulk_write.get_status('6759000000167103')
#get bulk write result
result = bulk_write.get_result('6759000000167103')

    

Note: A maximum of 100,000 rows can be written at one time.

Info : Refer to the SDK Scopes table to determine the required permission level for performing the above operation.

Last Updated 2025-11-11 12:11:26 +0530 IST