> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server/install/jetson.md).

# Install on NVIDIA Jetson

Install the Roboflow Inference Server on an NVIDIA Jetson device with JetPack-specific containers, TensorRT acceleration, and Docker Compose.

## Overview

Jetson is NVIDIA's line of compact, power-efficient modules designed to run AI and deep learning workloads at the edge. They combine a GPU, CPU, and neural accelerators on a single board, which makes them a good fit for robotics, drones, smart cameras, and other embedded applications that need real-time computer vision without a cloud connection. For more details, see [NVIDIA's Jetson overview](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/).

## Prerequisites

* **Disk space:** allocate at least 10 GB free for the Roboflow Jetson image (8.14 GB).
* **JetPack version:** a supported JetPack (5.x or 6.x).
* **Recommended hardware:** an NVIDIA Orin NX 16 GB or above for best performance.
* **Docker and the NVIDIA Container Toolkit:** containers need the Docker engine plus the NVIDIA runtime to access the GPU. Follow the [Docker install guide](https://docs.docker.com/engine/install/ubuntu/) and the [NVIDIA Container Toolkit guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).

Roboflow publishes specialized containers built with hardware acceleration support for JetPack L4T. To detect your JetPack version automatically and start the right container with good defaults, run:

```bash
pip install inference-cli
inference server start
```

{% hint style="warning" %}
**The server is reachable from your whole network.** Unlike other targets, which are published on `127.0.0.1`, Jetson images are published on `0.0.0.0` so you can drive the device from another machine, and the manual `docker run` examples on this page publish port 9001 the same way. That means anything that can route to the Jetson can use the server, which has no authentication by default and runs Workflows Custom Python blocks - remote code execution on the device. Use `inference server start --bind-address 127.0.0.1` for local-only access, or configure [server authentication and network restrictions](/deployment/self-hosted/inference-server/configuration/security.md) before serving remote clients.
{% endhint %}

## Manually starting the container

If you want more control over the container settings, start it yourself. Jetson devices with NVIDIA JetPack are pre-configured with the NVIDIA container runtime and are hardware accelerated out of the box.

{% tabs %}
{% tab title="JetPack 6.2" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-6.2.0:latest
```

{% endtab %}

{% tab title="JetPack 6.0" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-6.0.0:latest
```

{% endtab %}

{% tab title="JetPack 5" %}

```bash
sudo docker run -d \
    --name inference-server \
    --runtime nvidia \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-jetson-5.1.1:latest
```

{% endtab %}

{% tab title="JetPack 4 (deprecated)" %}
{% hint style="warning" %}
JetPack 4 is deprecated and will not receive future updates. Please migrate to JetPack 6.
{% endhint %}

Use the same command as JetPack 5 with the JetPack 4 image tag: `roboflow/roboflow-inference-server-jetson-4.6.1:latest` for JetPack 4.6, or `roboflow/roboflow-inference-server-jetson-4.5.0:latest` for JetPack 4.5.
{% endtab %}
{% endtabs %}

## TensorRT

You can optionally enable [TensorRT](https://developer.nvidia.com/tensorrt), NVIDIA's model optimization runtime. It greatly increases your models' speed at the expense of a heavy compilation and optimization step (sometimes 15+ minutes) the first time you load each model.

Enable TensorRT by adding `TensorrtExecutionProvider` to the `ONNXRUNTIME_EXECUTION_PROVIDERS` environment variable on any of the commands above:

```bash
    -e ONNXRUNTIME_EXECUTION_PROVIDERS="[TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider]" \
```

Mounting a persistent cache volume (as in the commands above) keeps the compiled TensorRT engines between restarts, so you only pay the compilation cost once per model.

## Docker Compose

If you use Docker Compose for your application, the equivalent YAML is below. Swap the image tag for your JetPack version: `jetson-6.2.0`, `jetson-6.0.0`, `jetson-5.1.1`, `jetson-4.6.1`, or `jetson-4.5.0`.

```yaml
version: "3.9"

services:
  inference-server:
    container_name: inference-server
    image: roboflow/roboflow-inference-server-jetson-6.2.0:latest

    read_only: true
    ports:
      - "9001:9001"

    volumes:
      - "${HOME}/.inference/cache:/tmp:rw"

    runtime: nvidia

    # Optionally: uncomment the following lines to enable TensorRT:
    # environment:
    #   ONNXRUNTIME_EXECUTION_PROVIDERS: "[TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider]"

    security_opt:
      - no-new-privileges
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
```

{% hint style="info" %}
Roboflow Enterprise plans add [a Helm chart](https://github.com/roboflow/inference/tree/main/inference/enterprise/helm-chart) for Kubernetes deployments, networking solutions for OT networks, customized support and installation packages, and [a pre-configured Jetson-based edge device](https://roboflow.com/hardware). [Contact the sales team](https://roboflow.com/sales) to learn more.
{% endhint %}

## Next steps

* [Run a model](/deployment/self-hosted/self-hosted.md#run-model-locally) against your new server.
* [Deployment Manager](/deployment/self-hosted/enterprise/deployment-manager.md) (Enterprise) to manage a fleet of Jetson devices remotely.
* [Securing a self-hosted server](/deployment/self-hosted/inference-server/configuration/security.md) before you expose it beyond localhost.