> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server/install/jetson.md).

# Install on NVIDIA Jetson

## Overview

Jetson is NVIDIA's line of compact, power-efficient modules designed to run AI and deep learning workloads at the edge. They combine a GPU, CPU, and neural accelerators on a single board, which makes them a good fit for robotics, drones, smart cameras, and other embedded applications that need real-time computer vision without a cloud connection. For more details, see [NVIDIA's Jetson overview](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/).

## Prerequisites

* **Disk space:** allocate at least 10 GB free for the Roboflow Jetson image (8.14 GB).
* **JetPack version:** a supported JetPack (5.x or 6.x).
* **Recommended hardware:** an NVIDIA Orin NX 16 GB or above for best performance.
* **Docker and the NVIDIA Container Toolkit:** containers need the Docker engine plus the NVIDIA runtime to access the GPU. Follow the [Docker install guide](https://docs.docker.com/engine/install/ubuntu/) and the [NVIDIA Container Toolkit guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).

Roboflow publishes specialized containers built with hardware acceleration support for JetPack L4T. To detect your JetPack version automatically and start the right container with good defaults, run:

```bash
pip install inference-cli
inference server start
```

## Manually starting the container

If you want more control over the container settings, start it yourself. Jetson devices with NVIDIA JetPack are pre-configured with the NVIDIA container runtime and are hardware accelerated out of the box.

{% tabs %}
{% tab title="JetPack 6.2" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-6.2.0:latest
```

{% endtab %}

{% tab title="JetPack 6.0" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-6.0.0:latest
```

{% endtab %}

{% tab title="JetPack 5" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-5.1.1:latest
```

{% endtab %}

{% tab title="JetPack 4 (deprecated)" %}
{% hint style="warning" %}
JetPack 4 is deprecated and will not receive future updates. Please migrate to JetPack 6.
{% endhint %}

Use the same command as JetPack 5 with the JetPack 4 image tag: `roboflow/roboflow-inference-server-jetson-4.6.1:latest` for JetPack 4.6, or `roboflow/roboflow-inference-server-jetson-4.5.0:latest` for JetPack 4.5.
{% endtab %}
{% endtabs %}

## TensorRT

You can optionally enable [TensorRT](https://developer.nvidia.com/tensorrt), NVIDIA's model optimization runtime. It greatly increases your models' speed at the expense of a heavy compilation and optimization step (sometimes 15+ minutes) the first time you load each model.

Enable TensorRT by adding `TensorrtExecutionProvider` to the `ONNXRUNTIME_EXECUTION_PROVIDERS` environment variable on any of the commands above:

```bash
    -e ONNXRUNTIME_EXECUTION_PROVIDERS="[TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider]" \
```

Mounting a persistent cache volume (as in the commands above) keeps the compiled TensorRT engines between restarts, so you only pay the compilation cost once per model.

## Docker Compose

If you use Docker Compose for your application, the equivalent YAML is below. Swap the image tag for your JetPack version: `jetson-6.2.0`, `jetson-6.0.0`, `jetson-5.1.1`, `jetson-4.6.1`, or `jetson-4.5.0`.

```yaml
version: "3.9"

services:
  inference-server:
    container_name: inference-server
    image: roboflow/roboflow-inference-server-jetson-6.2.0:latest

    read_only: true
    ports:
      - "9001:9001"

    volumes:
      - "${HOME}/.inference/cache:/tmp:rw"

    runtime: nvidia

    # Optionally: uncomment the following lines to enable TensorRT:
    # environment:
    #   ONNXRUNTIME_EXECUTION_PROVIDERS: "[TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider]"

    security_opt:
      - no-new-privileges
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
```

{% hint style="info" %}
Roboflow Enterprise plans add [a Helm chart](https://github.com/roboflow/inference/tree/main/inference/enterprise/helm-chart) for Kubernetes deployments, networking solutions for OT networks, customized support and installation packages, and [a pre-configured Jetson-based edge device](https://roboflow.com/hardware). [Contact the sales team](https://roboflow.com/sales) to learn more.
{% endhint %}

## Next steps

* [Run a model](/deployment/self-hosted/self-hosted.md#run-a-model) against your new server.
* [Deployment Manager](/deployment/self-hosted/enterprise/deployment-manager.md) (Enterprise) to manage a fleet of Jetson devices remotely.
* [Securing a self-hosted server](/deployment/self-hosted/inference-server/configuration/security.md) before you expose it beyond localhost.