Load Balancers on OCI

Customize load balancers on OCI for Verrazzano system and application endpoints

Verrazzano sets up the following load balancers on Kubernetes at installation:

  • Load balancer for NGINX ingress
  • Load balancer for Istio ingress

Verrazzano allows customizing the load balancers allocated by Oracle Cloud Infrastructure (OCI) using annotations defined by the OCI Cloud Controller Manager (OCI-CCM). For a detailed description of different load balancer customization annotations, see the documentation here.

This document describes how to use these annotations to customize the following settings for Verrazzano load balancers:

  • Load balancer shape
  • Private IP address and subnet placement

Customize the load balancer shape

At installation, Verrazzano lets you customize the shape and size of the load balancers created. Oracle Cloud Infrastructure offers a flexible load balancer which uses Dynamic Shape:

  • 10 Mbps
  • 100 Mbps
  • 400 Mbps
  • 8,000 Mbps

For more details on service limits and shape, see here.

For example, you can set up an NGINX load balancer with 10Mbps as follows:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  environmentName: default
  components:
    ingress:
      type: LoadBalancer
      nginxInstallArgs:
      - name: controller.service.annotations."service\.beta\.kubernetes\.io/oci-load-balancer-shape"
        value: "10Mbps"

For example, you can set up an Istio load balancer with 10Mbps as follows:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  environmentName: default
  components:
    ingress:
      type: LoadBalancer
      istioInstallArgs:
      - name: gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/oci-load-balancer-shape"
        value: "10Mbps"

Use private IP addresses with a load balancer

At installation, Verrazzano lets you customize the IP address and subnet of the load balancers created. This is achieved using OCI-CCM annotations on the NGINX and Istio load balancer services, as documented here.

The following example configures the NGINX load balancer service to have a private load balancer IP address on the private subnet identified by OCID ocid1.subnet.oc1.phx.aaaa..sdjxa, and uses the default (public) load balancer configuration for Istio:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  environmentName: default
  components:
    ingress:
      type: LoadBalancer
      nginxInstallArgs:
      - name: controller.service.annotations."service\.beta\.kubernetes\.io/oci-load-balancer-internal"
        value: "true"    
      - name: controller.service.annotations."service\.beta\.kubernetes\.io/oci-load-balancer-subnet1"
        value: "ocid1.subnet.oc1.phx.aaaa..sdjxa"

The following example configures the Istio ingress gateway service to have a private load balancer IP address on the private subnet identified by OCID ocid1.subnet.oc1.phx.aaaa..sdjxa, and uses the default (public) load balancer configuration for NGINX:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  environmentName: default
  components:
    ingress:
      type: LoadBalancer      
    istio:
      istioInstallArgs:
        - name: gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/oci-load-balancer-internal"
          value: "true"
        - name: gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/oci-load-balancer-subnet1"
          value: "ocid1.subnet.oc1.phx.aaaa..sdjxa"

The following example configures both NGINX and Istio to have a private load balancer IP address on the private subnet identified by OCID ocid1.subnet.oc1.phx.aaaa..sdjxa:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
  environmentName: default
  components:
    ingress:
      type: LoadBalancer
      nginxInstallArgs:
      - name: controller.service.annotations."service\.beta\.kubernetes\.io/oci-load-balancer-internal"
        value: "true"
      - name: controller.service.annotations."service\.beta\.kubernetes\.io/oci-load-balancer-subnet1"
        value: "ocid1.subnet.oc1.phx.aaaa..sdjxa"
    istio:
      istioInstallArgs:
      - name: gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/oci-load-balancer-internal"
        value: "true"
      - name: gateways.istio-ingressgateway.serviceAnnotations."service\.beta\.kubernetes\.io/oci-load-balancer-subnet1"
        value: "ocid1.subnet.oc1.phx.aaaa..sdjxa"