Execute Vision Model EndpointAdmin Scope

Vision Language Model

QuickML hosts Vision Language Models in the LLM Serving module under the Generative AI section in the console. VLMs are multimodal models that accept a file object along with a text prompt and return a natural language response. The model interprets the image according to the text prompt instruction and generates an appropriate response.

Interaction with Vision Language Model works similar to single shot interaction mode with LLM. Each request must carry its own file and prompt and is processed independently, with no memory of previous requests. To analyze the same image again with a different instruction, send a new request.

Execute Vision Model Endpoint

VLM is served with its own set of parameters, giving you control over the response before publishing it. You can test a model with sample images and prompts in the test interface, save the configuration, and create an endpoint from it to integrate with your application.

A Vision Model endpoint is created from a saved parameter configuration, so the model, system prompt, instructions, and generation parameters you tested in the console are exactly what your application calls. The endpoint must be created from a configuration that uses a Vision Language Model, text-only LLM endpoints do not accept image input. Open the image in binary read mode and pass the file object to the method.

a. Analyze an Image

The analyzeImage(imageEndpointKey, imageBase64, imagePrompt) method requires the endpoint key, a Base64-encoded image, and an input prompt. The endpoint key is generated when the VLM endpoint is created.

This method sends an image and an accompanying prompt to a published Vision Language Model (VLM) endpoint and returns the model’s response.

Parameters used

Allowed file formats: .jpg, .jpeg, .png

File size limit: 500 KB

Sample Code Snippet

 // -------------------------------------------------------------------
// Vision Model - Analyze Image
// -------------------------------------------------------------------
const endpointKey = “<ENDPOINT_KEY>”;
// Replace with the path to your image.
const imagePath = “<IMAGE_PATH>”;
const image = fs.createReadStream(imagePath);
const prompt = “”;
const result = await quickML.analyzeImage(imageEndpointKey, imageBase64, imagePrompt);
console.log(result);

The syntax of the response received is shown below:

{
    "status": "success",
    "result": [
        {
            "content": "A description of the image, as instructed by the prompt.",
            "finish_reason": "stop",
            "usage": {
                "input_tokens": 1064,
                "output_tokens": 74,
                "total_tokens": 1138
            }
        }
    ]
}

Use Vision Language Model for tasks such as image description, document and receipt understanding, chart interpretation, and visual question answering, etc.

Where to find the endpoint information

Create an endpoint for your Saved VLM configuration and access the endpoint details page to view the Endpoint URL, required headers and a sample request response.