Overview

Each model trained with Roboflow Train is deployed as a custom API you can use to make predictions from any device that has an internet connection. Inference is done on the server so you don't need to worry about the edge device's hardware capabilities.

We automatically scale this API up and down and do load balancing for you so that you can rest assured that your application will be able to handle sudden spikes in traffic without having to pay for GPU time you're not using. Our hosted prediction API has been battle-hardened to handle even the most demanding production applications (including concurrently surviving through the famous Hacker News and Reddit "hugs of death" without so much as batting an eye).

The Example Web App

The easiest way to familiarize yourself with the inference endpoint is to visit the Example Web App. To use the Web App, simply input your model , version and api_key. These will be pre-filled for you after training completes if you click through via the web UI under your versions "Training Results" section.

Then select an image via Choose File. After you have chosen the settings you want, click Run Inference.

On the left side of the screen, you will see example JavaScript code for posting a base64-encoded image to the inference endpoint. Within the form portion of the Web App, you can experiment with changing different API parameters when posting to the API.

Obtaining Your Model Endpoint

To use the inference API, you will need your model url slug, version number and the API key for the workspace the project belongs to. Your API key can be retrieved from in your workspace's settings page under the "Roboflow API" section. Your model url slug is the unique and url-safe version of your dataset name. The easiest way to retrieve it is via the web UI by clicking the "curl command" link:

{
    "predictions": [{
        "x": 234.0,
        "y": 363.5,
        "width": 160,
        "height": 197,
        "class": "hand",
        "confidence": 0.943
    }, {
        "x": 504.5,
        "y": 363.0,
        "width": 215,
        "height": 172,
        "class": "hand",
        "confidence": 0.917
    }, {
        "x": 1112.5,
        "y": 691.0,
        "width": 139,
        "height": 52,
        "class": "hand",
        "confidence": 0.87
    }, {
        "x": 78.5,
        "y": 700.0,
        "width": 139,
        "height": 34,
        "class": "hand",
        "confidence": 0.404
    }]
}

{
    "Message": "User is not authorized to access this resource"
}

Code Snippets

For your convenience, we've provided code snippets for calling this endpoint in various programming languages. If you need help integrating the inference API into your project don't hesitate to reach out.

All examples upload to an example dataset with a model-endpoint of your-dataset-slug/your-version. You can easily find your dataset's identifier by looking at the curl command shown in the Roboflow web interface after your model has finished training.

base64 YOUR_IMAGE.jpg | curl -d @- \
"https://detect.roboflow.com/your-model/42?api_key=YOUR_KEY"

curl -X POST "https://infer.roboflow.com/your-model/42?\
api_key=YOUR_KEY&\
image=https%3A%2F%2Fi.imgur.com%2FPEEvqPN.png"

import requests
import base64
import io
from PIL import Image
​
# Load Image with PIL
image = Image.open("YOUR_IMAGE.jpg").convert("RGB")
​
# Convert to JPEG Buffer
buffered = io.BytesIO()
image.save(buffered, quality=90, format="JPEG")
​
# Base 64 Encode
img_str = base64.b64encode(buffered.getvalue())
img_str = img_str.decode("ascii")
​
# Construct the URL
upload_url = "".join([
    "https://detect.roboflow.com/your-model/42",
    "?api_key=YOUR_KEY",
    "&name=YOUR_IMAGE.jpg"
])
​
# POST to the API
r = requests.post(upload_url, data=img_str, headers={
    "Content-Type": "application/x-www-form-urlencoded"
})
​
# Output result
print(r.json())

import requests
import urllib.parse
​
# Construct the URL
img_url = "https://i.imgur.com/PEEvqPN.png"
upload_url = "".join([
    "detect.roboflow.com/your-model/42",
    "?api_key=YOUR_KEY",
    "&image=" + urllib.parse.quote_plus(img_url)
])
​
# POST to the API
r = requests.post(upload_url)
​
# Output result
print(r.json())

const axios = require("axios");
const fs = require("fs");
​
const image = fs.readFileSync("YOUR_IMAGE.jpg", {
    encoding: "base64"
});
​
axios({
    method: "POST",
    url: "https://detect.roboflow.com/your-model/42",
    params: {
        api_key: "YOUR_KEY"
    },
    data: image,
    headers: {
        "Content-Type": "application/x-www-form-urlencoded"
    }
})
.then(function(response) {
    console.log(response.data);
})
.catch(function(error) {
    console.log(error.message);
});

const axios = require("axios");
​
axios({
    method: "POST",
    url: "https://detect.roboflow.com/your-model/42",
    params: {
        api_key: "YOUR_KEY",
        image: "https://i.imgur.com/PEEvqPN.png"
    }
})
.then(function(response) {
    console.log(response.data);
})
.catch(function(error) {
    console.log(error.message);
});

Video Inference

To get predictions from your model on a video, split it into frames, perform inference on each frame, then composite the predictions back into a rendered video.

We have an open source video inference utility script that performs these steps with ffmpeg and the Roboflow Inference API:

Using Your Webcam

You can also pipe frames from your webcam to your Roboflow model for predictions. We've created a tutorial with example code to show you how:

Displaying the Response Image with format=image

If you pass format=image in the query string the inference API will return a base64 encoded string of your image with the inference detections drawn on top. You can decode this with your favorite image processing library - here we provide an example with cv2 and numpy

    # Get prediction from Roboflow Infer API
    resp = requests.post(upload_url, data=img_str, headers={
        "Content-Type": "application/x-www-form-urlencoded"
    }, stream=True).raw
​
    # Parse result image
    image = np.asarray(bytearray(resp.read()), dtype="uint8")
    image = cv2.imdecode(image, cv2.IMREAD_COLOR)

Drawing a Box from the Inference API JSON Output

Note: to receive and image with predicted box output - just pass "format=image" as a query parameter. This section shows an example if you want to double check and parse your JSON output.

Let's suppose you've created an inference url to return JSON predictions like so:

parts = []
url_base = 'https://infer.roboflow.com/'
endpoint = '[YOUR-MODEL]'
access_token = '?access_token=[YOUR_TOKEN]'
format = '&format=json'
confidence = '&confidence=10'
stroke='&stroke=5'
parts.append(url_base)
parts.append(endpoint)
parts.append(access_token)
parts.append(format)
parts.append(confidence)
parts.append(stroke)
url = ''.join(parts)

And you've drawn an inference from your API like so:

import glob
import requests
import base64
from base64 import decodebytes
import io
from PIL import Image
import time
import cv2
from io import BytesIO
​
​
f = '[YOUR-IMAGE].jpg'
image = Image.open(f)
buffered = io.BytesIO()
image = image.convert("RGB")
image.save(buffered, quality=90, format="JPEG")
img_str = base64.b64encode(buffered.getvalue())
img_str = img_str.decode("ascii")
​
headers = {'accept': 'application/json'}
start = time.time()
r = requests.post(url, data=img_str, headers=headers)
print('post took ' + str(time.time() - start))
​
# print(r)
print(r.json())
preds = r.json()
detections = preds['predictions']

Then you can draw the box on your image like so:

from PIL import Image, ImageDraw, ImageFont
f = './[YOUR-IMAGE].jpg'
image = Image.open(f)
​
draw = ImageDraw.Draw(image)
font = ImageFont.load_default()
​
for box in detections:
    color = "#4892EA"
    x1 = box['x'] - box['width'] / 2
    x2 = box['x'] + box['width'] / 2
    y1 = box['y'] - box['height'] / 2
    y2 = box['y'] + box['height'] / 2
​
    draw.rectangle([
        x1, y1, x2, y2
    ], outline=color, width=5)
​
    if True:
        text = box['class']
        text_size = font.getsize(text)
​
        # set button size + 10px margins
        button_size = (text_size[0]+20, text_size[1]+20)
        button_img = Image.new('RGBA', button_size, color)
        # put text on button with 10px margins
        button_draw = ImageDraw.Draw(button_img)
        button_draw.text((10, 10), text, font=font, fill=(255,255,255,255))
​
        # put button on source image in position (0, 0)
        image.paste(button_img, (int(x1), int(y1)))
image.show()