Getting started with Roboflow Inference on Kubernetes
Roboflow’s inference docker containers can be easily hosted within Kubernetes.
The Kubernetes manifest below shows a simple example of creating a single CPU-based roboflow infer pod and attaching a cluster-IP service to it.
- name: roboflow
- containerPort: 9001
- name: rf-svc-port
(the above example assumes your Kubernetes cluster can download images from Docker hub)
Save the blurb of yaml above as
roboflow.yamland use the
kubectlcli to deploy the pod and service into the default namespace of your Kubernetes cluster.
kubectl apply -f roboflow.yaml
A service (of type ClusterIP) will be created; you can access Roboflow inference from within the Kubernetes cluster at this URI:
Kubernetes gives you the power to incorporate several advanced features and extensions into your Roboflow inference service. For example, you could extend the above example for more advanced use-cases such as
- Using nodeSelectors to host the pod(s) on GPU machines node pools within your Kubernetes environments and using the
- Creating Kubernetes deployments to horizontally autoscale the Roboflow inference service and setting up auto-scaling triggers based on specific metrics like CPU usage.
- Using different service types like nodePort and LoadBalancer to serve the Roboflow inference service externally
- Use ingress controllers to expose Roboflow inference over TLS (HTTPs) etc.
- Add monitoring and alerting to your Roboflow inference service
- Integrating the license server and offline modes