Enterprise NVIDIA Jetson
Deploy your Roboflow model on the edge to the NVIDIA Jetson
You can take the edge acceleration version of your model to the NVIDIA Jetson, where you may need realtime speeds with limited hardware resources.
Step 1 - Flash Jetson: Ensure that your Jetson is flashed with Jetpack 4.5 or 4.6. You can check you existing with this repository from Jetson Hacks
git clone https://github.com/jetsonhacks/jetsonUtilities.git
Step 2 - Start Server:
sudo docker run --privileged --net=host --gpus all --mount source=roboflow,target=/cache -e NUM_WORKERS=1 roboflow/inference-server-trt-jetson:latest
Step 3 - Request Server:
base64 your_img.jpg | curl -d @- "http://localhost:9001/[YOUR MODEL]/[YOUR VERSION]?api_key=[YOUR API KEY]"
When you request the server for the first time, your model will compile on your Jetson device for 5-10 minutes.
Plug a UDP socket into your Jetson deployment method when you require realtime network configuration between your Jetson and another machine in your network.
sudo docker run --privileged --device=/dev/video0:/dev/video0 --net=host --gpus all -e DATASET="YOUR_MODEL" -e VERSION="YOUR_VERSION" -e API_KEY="YOUR_API_KEY" -e VIDEO_DEVICE="/dev/video0" -e IP_BROADCAST_ADDR="0.0.0.0" -e IP_BROADCAST_PORT="8080" -e PYTHONUNBUFFERED=1 --mount source=roboflow,target=/cache roboflow/inference-server-jetson-udp:latest
DATASET- the dataset you want to launch
VERSION- the version of your dataset you want to launch
API_KEY- your Roboflow api key
VIDEO_DEVICE- the video device you want to capture frames from, be sure it's forwarded to docker with the
--deviceflag as well
IP_BROADCAST_ADDR- the IP address you want to broadcast predictions to via UDP
IP_BROADCAST_PORT- the port to broadcast predictions to
CONFIDENCE- the confidence to filter predictions from, default
OVERLAP- the IoU overlap at which to filter predictions during NMS, default
CAP_HEIGHT- the resolution to capture from your camera, default
380. Inference is performed at the resolution that you trained your model on.