Pipeline Builder Interface

QuickML provides easy to use, convenient Drag and Drop editor powered with data and Machine learning capabilities as Stages.

Stage

A stage is a single task or unit of function that provides particular results based on the nature of computation assigned to it. A stage can be any data-related or ML related task that can be sequentially executed in a workflow. For example, a stage can be a data filter that takes raw data as input and provides filtered data as output based on the filter type.

Pipeline

A pipeline is a sequential combination of stages which is intended to do multiple operations and provide a collective result as output. Any data pipelines or ML pipelines can be created using QuickML service with provided functionalities.

Pipeline Builder Interface

The Pipeline Builder interface is a drag-and-drop, in-browser editor component provided in QuickML to construct pipelines with multiple or single stages of desired operations. This editor provides all the basic create, copy, edit, undo, redo and delete operations in the pipeline building process. Each data or ML module is segregated as stages in the editor and listed under data and ML categories.

Each pipeline will have a mandatory source and destination stages. Other stages are customizable as per your pipeline requirements. Any stage that has been added to the editor needs to be connected with next or previous stages in the editor in order to form a pipeline execution flow. Once the mapping is done with stages, a preview will be shown in the editor for better understanding of the pipeline flow.

Data Operations in the Editor

The data-related operations are listed in the pipeline builder to add in the data pipeline construction. The details of each operation in the editor are provided in the data pipelines chapter of this manual. For each data operation available in the pipeline editor, a custom-configuration panel will be open to fill and feed configurations for that stage.

Criteria Editor

A criteria editor component has been provided in the builder to apply custom criteria over the resultant data of any stage based on your requirements.

ML operations in the Editor The ML operations will be available in the builder when the model pipeline needs to be constructed. The ML pipeline builder contains both data-related operations and ML operations to form an execution pipeline.

These options are categorized as algorithms and operations for ML tasks. ML stages can be any algorithms or Zia features that are integrated as stages into the builder, and can be added to pipeline by dragging them into the builder.

Once the pipeline is built, it can be saved and kept for future development, or can be published to execute in the background. These executions are versioned and execution results will be stored to keep track of the status. Any version of the pipeline can be viewed at any time and promoted as production-ready pipelines.