Web Browser

Realtime predictions at the edge with roboflow.js

For most business applications, the Hosted API is suitable. But for many consumer applications and some enterprise use cases, having a server-hosted model is not workable (for example, if your users are bandwidth constrained or need lower latency than you can achieve using a remote API).

inferencejs is a custom layer on top of Tensorflow.js to enable real-time inference via JavaScript using models trained on Roboflow.

Learning Resources

Try Your Model With a Webcam

Once you have a trained model, you can easily test it using your webcam using the "Try with Webcam" button.

The webcam demo is a sample app that is available for you to download and tinker with the "Get Code" link.

You can try out a webcam demo of a hand-detector model here (it is trained on the public EgoHands dataset).

Interactive Replit Environment

We have published a "Getting Started" project on Repl.it with an accompanying tutorial showing how to deploy YOLOv8 models using our Repl.it template.

GitHub Template

The Roboflow homepage uses inferencejs to power the COCO inference widget. The README contains instructions on how to use the repository template to deploy a model to the web using GitHub Pages.

Documentation

If you would like more details regarding specific functions in inferencejs, check out our documentation page or click on any mention of a inferencejs method in our guide below to be taken to the respective documentation.

Installation

To add inference to your project, simply install using npm or add the script tag reference to your page's <head> tag.

npm install inferencejs

<script src="https://cdn.jsdelivr.net/npm/inferencejs"></script>

Initalizing `inferencejs`

Authenticating

You can obtain your publishable_key from the Roboflow workspace settings.

Note: your publishable_key is used with inferencejs, not your private API key (which should remain secret).

Start by importing InferenceEngine and creating a new inference engine object

inferencejs uses webworkers so that multiple models can be used without blocking the main UI thread. Each model is loaded through the InferenceEngine our webworker manager that abstracts the necessary thread management for you.

import { InferenceEngine } from "inferencejs";
const inferEngine = new InferenceEngine();

Now we can load models from roboflow using your publishable_key and the model metadata (model name and version) along with configuration parameters like confidence threshold and overlap threshold.

const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]");

inferencejs will now start a worker that runs the chosen model. The returned worker id corresponds with the worker id in InferenceEngine that we will use for inference. To infer on the model we can invoke the infer method on the InferenceEngine.

Let's load an image and infer on our worker.

const image = document.getElementById("image"); // get image element with id `image`
const predictions = await inferEngine.infer(workerId, image); // infer on image

This can take in a variety of image formats (HTMLImageElement, HTMLVideoElement, ImageBitmap, or TFJS Tensor).

This returns an array of predictions (as a class, in this case RFObjectDetectionPrediction )

Configuration

If you would like to customize and configure the way inferencejs filters its predictions, you can pass parameters to the worker on creation.

const configuration = {scoreThreshold: 0.5, iouThreshold: 0.5, maxNumBoxes: 20};
const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]", configuration);

Or you can pass configuration options on inference

const configuration = {
    scoreThreshold: 0.5, 
    iouThreshold: 0.5, 
    maxNumBoxes: 20
};
const predictions = await inferEngine.infer(workerId, image, configuration);

PreviousPython inference-sdk NextLens Studio

Last updated 26 days ago

Was this helpful?

Web Browser

Realtime predictions at the edge with roboflow.js

inferencejs is a custom layer on top of Tensorflow.js to enable real-time inference via JavaScript using models trained on Roboflow.

Learning Resources

Try Your Model With a Webcam

Once you have a trained model, you can easily test it using your webcam using the "Try with Webcam" button.

The webcam demo is a sample app that is available for you to download and tinker with the "Get Code" link.

You can try out a webcam demo of a hand-detector model here (it is trained on the public EgoHands dataset).

Interactive Replit Environment

We have published a "Getting Started" project on Repl.it with an accompanying tutorial showing how to deploy YOLOv8 models using our Repl.it template.

GitHub Template

The Roboflow homepage uses inferencejs to power the COCO inference widget. The README contains instructions on how to use the repository template to deploy a model to the web using GitHub Pages.

Documentation

Installation

To add inference to your project, simply install using npm or add the script tag reference to your page's <head> tag.

npm install inferencejs

<script src="https://cdn.jsdelivr.net/npm/inferencejs"></script>

Initalizing `inferencejs`

Authenticating

You can obtain your publishable_key from the Roboflow workspace settings.

Note: your publishable_key is used with inferencejs, not your private API key (which should remain secret).

Start by importing InferenceEngine and creating a new inference engine object

import { InferenceEngine } from "inferencejs";
const inferEngine = new InferenceEngine();

Now we can load models from roboflow using your publishable_key and the model metadata (model name and version) along with configuration parameters like confidence threshold and overlap threshold.

const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]");

Let's load an image and infer on our worker.

const image = document.getElementById("image"); // get image element with id `image`
const predictions = await inferEngine.infer(workerId, image); // infer on image

This can take in a variety of image formats (HTMLImageElement, HTMLVideoElement, ImageBitmap, or TFJS Tensor).

This returns an array of predictions (as a class, in this case RFObjectDetectionPrediction )

Configuration

If you would like to customize and configure the way inferencejs filters its predictions, you can pass parameters to the worker on creation.

const configuration = {scoreThreshold: 0.5, iouThreshold: 0.5, maxNumBoxes: 20};
const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]", configuration);

Or you can pass configuration options on inference

const configuration = {
    scoreThreshold: 0.5, 
    iouThreshold: 0.5, 
    maxNumBoxes: 20
};
const predictions = await inferEngine.infer(workerId, image, configuration);

PreviousPython inference-sdk NextLens Studio

Last updated 26 days ago

Was this helpful?

Learning Resources

Try Your Model With a Webcam

Interactive Replit Environment

GitHub Template

Documentation

Installation

Initalizing inferencejs

Authenticating

Configuration

Learning Resources

Try Your Model With a Webcam

Interactive Replit Environment

GitHub Template

Documentation

Installation

Initalizing inferencejs

Authenticating

Configuration

Initalizing `inferencejs`

Initalizing `inferencejs`