Roboflow
ProductForumComputer Vision DatasetsComputer Vision Models
Search
⌃K
Links

Enterprise NVIDIA Jetson

Deploy your Roboflow model on the edge to the NVIDIA Jetson

Run the NVIDIA Jetson TRT Server

You can take the edge acceleration version of your model to the NVIDIA Jetson, where you may need realtime speeds with limited hardware resources.
Step 1 - Flash Jetson: Ensure that your Jetson is flashed with Jetpack 4.5 or 4.6. You can check you existing with this repository from Jetson Hacks
git clone https://github.com/jetsonhacks/jetsonUtilities.git
cd jetsonUtilities
python jetsonInfo.py
Step 2 - Start Server:
sudo docker run --privileged --net=host --gpus all --mount source=roboflow,target=/cache -e NUM_WORKERS=1 roboflow/inference-server-trt-jetson:latest
Step 3 - Request Server:
base64 your_img.jpg | curl -d @- "http://localhost:9001/[YOUR MODEL]/[YOUR VERSION]?api_key=[YOUR API KEY]"
When you request the server for the first time, your model will compile on your Jetson device for 5-10 minutes.

Attach Video Capture and UDP Socket to Jetson Inference Container

Plug a UDP socket into your Jetson deployment method when you require realtime network configuration between your Jetson and another machine in your network.
sudo docker run --privileged --device=/dev/video0:/dev/video0 --net=host --gpus all -e DATASET="YOUR_MODEL" -e VERSION="YOUR_VERSION" -e API_KEY="YOUR_API_KEY" -e VIDEO_DEVICE="/dev/video0" -e IP_BROADCAST_ADDR="0.0.0.0" -e IP_BROADCAST_PORT="8080" -e PYTHONUNBUFFERED=1 --mount source=roboflow,target=/cache roboflow/inference-server-jetson-udp:latest

Explanation of required environment variables:

DATASET - the dataset you want to launch
VERSION - the version of your dataset you want to launch
API_KEY - your Roboflow api key
VIDEO_DEVICE - the video device you want to capture frames from, be sure it's forwarded to docker with the --device flag as well
IP_BROADCAST_ADDR - the IP address you want to broadcast predictions to via UDP
IP_BROADCAST_PORT - the port to broadcast predictions to

Explanation of optional environment variables:

CONFIDENCE - the confidence to filter predictions from, default 0.4
OVERLAP - the IoU overlap at which to filter predictions during NMS, default 0.3
CAP_WIDTH , CAP_HEIGHT - the resolution to capture from your camera, default 640, 380 . Inference is performed at the resolution that you trained your model on.
Previous
Enterprise NVIDIA GPU
Next
License Server
Last modified 5mo ago