Web Browser

Realtime predictions at the edge with roboflow.js

Roboflow provides JavaScript packages for deploying computer vision models in web browsers. For most applications, we recommend @roboflow/inference-sdk for cloud-based inference with full model support, GPU acceleration, and Roboflow Workflows. For specialized use cases requiring offline functionality or strict data privacy, inferencejs provides edge-based inference.

Overview

Package

Use Case

Processing Location

Model Support

@roboflow/inference-sdk

Recommended: ✅ Real-time streaming, workflows, full model access

Roboflow Cloud/Server (WebRTC)

All models & workflows

inferencejs

Specialized: Offline functionality, strict data privacy

Browser (TensorFlow.js)

Limited (YOLOv8, YOLOv5, Gaze)

Comparison Table

Feature

@roboflow/inference-sdk (Cloud)

inferencejs (Edge)

Typical Use Case

Recommended for most applications

Specialized offline/privacy scenarios

Processing

Server (GPU-accelerated)

Browser (TensorFlow.js)

Model Support

All Roboflow models + workflows

Limited (3 models: YOLOv8, YOLOv5, Gaze)

Internet Required

Yes, continuous

Only for initial load

Network Latency

Low (WebRTC optimized)

Zero network latency

Processing Latency

Cloud (optimized)

Dependent on user's device

Choosing the Right Package

For Most Applications: Use `@roboflow/inference-sdk` (Cloud)

The cloud-based SDK is recommended as the default choice for web applications:

Full model access - Use any Roboflow model type and version
Roboflow Workflows - Run complex multi-step computer vision pipelines
GPU acceleration - Leverage powerful server-side processing
Real-time streaming - Low-latency WebRTC for video applications
No device limitations - Works on any device regardless of processing power
Always up-to-date - Access the latest models without client updates

For Specialized Use Cases: Use `inferencejs` (Edge)

Use edge inference only when you have specific requirements that require browser-side processing:

Offline functionality is mandatory - Application must work without internet
Strict data privacy - Regulatory requirements prevent any cloud transmission
Supported models only - You're exclusively using YOLOv8, YOLOv5, or Gaze Detection
Device capability - Users have devices capable of running TensorFlow.js models

Real-Time Streaming with @roboflow/inference-sdk

What is WebRTC Streaming?

@roboflow/inference-sdk enables real-time video streaming from your browser to Roboflow's inference servers using WebRTC. This allows you to:

Execute Workflows - Run complex multi-step computer vision pipelines
Access All Models - Use any Roboflow model type
Server-Side Processing - Leverage powerful GPUs
Low Latency - WebRTC provides near-real-time results
Bidirectional Communication - Send and receive data during streaming

Installation

npm install @roboflow/inference-sdk

Quick Start

Take a look at the video / sample code below to get started:

import { connectors, webrtc, streams } from '@roboflow/inference-sdk';

// ⚠️ Use withApiKey for development only
// ⚠️ Do not use this in production, because it will expose your API key
// For production, use a backend proxy (see next section)
const connector = connectors.withApiKey("your-api-key");

// Get camera stream
const stream = await streams.useCamera({
  video: {
    facingMode: { ideal: "environment" },
    width: { ideal: 640 },
    height: { ideal: 480 }
  }
});

// Start WebRTC connection
const connection = await webrtc.useStream({
  source: stream,
  connector,
  wrtcParams: {
    workspaceName: "your-workspace",
    workflowId: "your-workflow",
    imageInputName: "image",
    streamOutputNames: ["output"],
    dataOutputNames: ["predictions"]
  },
  onData: (data) => {
    console.log("Inference results:", data);
  }
});

// Display processed video
const videoElement = document.getElementById('video');
videoElement.srcObject = await connection.remoteStream();

// Clean up when done
await connection.cleanup();

🔐 Security Best Practices

NEVER expose your API key in frontend code for production applications.

The connectors.withApiKey() method is convenient for demos but exposes your API key in the browser. For production, always use a backend proxy:

Secure Production Pattern

Frontend:

import { connectors, webrtc, streams } from '@roboflow/inference-sdk';

// Use proxy endpoint instead of direct API key
const connector = connectors.withProxyUrl('/api/init-webrtc');

const stream = await streams.useCamera({ video: true });
const connection = await webrtc.useStream({
  source: stream,
  connector,
  wrtcParams: { /* ... */ }
});

Backend (Express):

import { InferenceHTTPClient } from '@roboflow/inference-sdk/api';

app.post('/api/init-webrtc', async (req, res) => {
  const { offer, wrtcparams } = req.body;

  // API key stays secure on the server
  const client = InferenceHTTPClient.init({
    apiKey: process.env.ROBOFLOW_API_KEY
  });

  const answer = await client.initializeWebrtcWorker({
    offer,
    workspaceName: wrtcparams.workspaceName,
    workflowId: wrtcparams.workflowId,
    config: {
      imageInputName: wrtcparams.imageInputName,
      streamOutputNames: wrtcparams.streamOutputNames,
      dataOutputNames: wrtcparams.dataOutputNames
    }
  });

  res.json(answer);
});

Key Features

Dynamic Output Reconfiguration

Change stream and data outputs at runtime without restarting:

// Switch to different visualization
connection.reconfigureOutputs({
  streamOutput: ["blur_visualization"]
});

// Enable all data outputs
connection.reconfigureOutputs({
  dataOutput: ["*"]
});

// Change both at once
connection.reconfigureOutputs({
  streamOutput: ["annotated_image"],
  dataOutput: ["predictions", "counts"]
});

Complete Working Example

For a full working example with both frontend and backend code, see the sample application repository. The sample app demonstrates:

Proper backend proxy setup for API key security
Camera streaming integration
Error handling and connection management
Production-ready patterns

Resources

Example Application - Complete working demo
Package on NPM
WebRTC Streaming Guide - Detailed documentation
API Reference

Edge Inference with inferencejs

inferencejs is a custom layer on top of Tensorflow.js to enable real-time inference via JavaScript using models trained on Roboflow.

See the inferencejs reference here

For most business applications, the Hosted API is suitable. But for many consumer applications and some enterprise use cases, having a server-hosted model is not workable (for example, if your users are bandwidth constrained or need lower latency than you can achieve using a remote API).

Learning Resources

Try Your Model With a Webcam: You can try out a webcam demo of a hand-detector model here (it is trained on the public EgoHands dataset).
Interactive Replit Environment: We have published a "Getting Started" project on Repl.it with an accompanying tutorial showing how to deploy YOLOv8 models using our Repl.it template.
GitHub Template: The Roboflow homepage uses inferencejs to power the COCO inference widget. The README contains instructions on how to use the repository template to deploy a model to the web using GitHub Pages.
Documentation: If you would like more details regarding specific functions in inferencejs, check out our documentation page or click on any mention of a inferencejs method in our guide below to be taken to the respective documentation.

Supported Models

inferencejs currently supports these model architectures:

YOLOv8
YOLOv5
Gaze Detection

Installation

To add inference to your project, simply install using npm or add the script tag reference to your page's <head> tag.

npm install inferencejs

<script src="https://cdn.jsdelivr.net/npm/inferencejs"></script>

Initalizing `inferencejs`

Authenticating

You can obtain your publishable_key from the Roboflow workspace settings.

Note: your publishable_key is used with inferencejs, not your private API key (which should remain secret).

Start by importing InferenceEngine and creating a new inference engine object

inferencejs uses webworkers so that multiple models can be used without blocking the main UI thread. Each model is loaded through the InferenceEngine our webworker manager that abstracts the necessary thread management for you.

import { InferenceEngine } from "inferencejs";
const inferEngine = new InferenceEngine();

Now we can load models from roboflow using your publishable_key and the model metadata (model name and version) along with configuration parameters like confidence threshold and overlap threshold.

const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]");

inferencejs will now start a worker that runs the chosen model. The returned worker id corresponds with the worker id in InferenceEngine that we will use for inference. To infer on the model we can invoke the infer method on the InferenceEngine.

Let's load an image and infer on our worker.

const image = document.getElementById("image"); // get image element with id `image`
const predictions = await inferEngine.infer(workerId, image); // infer on image

This can take in a variety of image formats (HTMLImageElement, HTMLVideoElement, ImageBitmap, or TFJS Tensor).

This returns an array of predictions (as a class, in this case RFObjectDetectionPrediction )

Configuration

If you would like to customize and configure the way inferencejs filters its predictions, you can pass parameters to the worker on creation.

const configuration = {scoreThreshold: 0.5, iouThreshold: 0.5, maxNumBoxes: 20};
const workerId = await inferEngine.startWorker("[model name]", "[version]", "[publishable key]", configuration);

Or you can pass configuration options on inference

const configuration = {
    scoreThreshold: 0.5,
    iouThreshold: 0.5,
    maxNumBoxes: 20
};
const predictions = await inferEngine.infer(workerId, image, configuration);

PreviousPython inference-sdk Nextinferencejs Reference

Last updated 4 days ago

Was this helpful?

Overview

Choosing the Right Package

For Most Applications: Use @roboflow/inference-sdk (Cloud)

For Specialized Use Cases: Use inferencejs (Edge)

Real-Time Streaming with @roboflow/inference-sdk

What is WebRTC Streaming?

Installation

Quick Start

🔐 Security Best Practices

Secure Production Pattern

Key Features

Dynamic Output Reconfiguration

Complete Working Example

Resources

Edge Inference with inferencejs

Learning Resources

Supported Models

Installation

Initalizing inferencejs

Authenticating

Configuration

For Most Applications: Use `@roboflow/inference-sdk` (Cloud)

For Specialized Use Cases: Use `inferencejs` (Edge)

Initalizing `inferencejs`