Generic Kubernetes

Instructions for setting up a generic Kubernetes cluster for Verrazzano

Prepare for the generic install

To use a generic Kubernetes implementation, there are two main areas you can configure: ingress and storage.


You can achieve ingress configuration using Helm overrides. For example, to use the nginx-controller for ingress on KIND, apply the following customization to the Verrazzano CRD.

spec:
 components:
  ingress:
   nginxInstallArgs:
   - name: controller.kind
     value: DaemonSet
   - name: controller.hostPort.enabled
     value: "true"
   - name: controller.nodeSelector.ingress-ready
     value: "true"
     setString: true
   - name: controller.tolerations[0].key
     value: node-role.kubernetes.io/master
   - name: controller.tolerations[0].operator
     value: Equal
   - name: controller.tolerations[0].effect
     value: NoSchedule

By default, each Verrazzano install profile has different storage characteristics. Some components have external storage requirements (expressed through PersistentVolumeClaim declarations in their resources/helm charts):

  • MySQL
  • Elasticsearch
  • Prometheus
  • Grafana

By default, the prod profile uses 50Gi persistent volumes for each of the above services, using the default storage class for the target Kubernetes platform. The dev profile uses ephemeral emptyDir storage by default. However, you can customize these storage settings within a profile as desired.

To override these settings, customize the Verrazzano install resource by defining a VolumeSource on the defaultVolumeSource field in the install CR, which can be one of:

Configuring emptyDir for the defaultVolumeSource forces all persistent volumes created by Verrazzano components in an installation to use ephemeral storage unless otherwise overridden. This can be useful for development or test scenarios.

You can use a persistentVolumeClaim to identify a volumeClaimSpecTemplate in the volumeClaimSpecTemplates section via the claimSource field. A volumeClaimSpecTemplate is a named PersistentVolumeClaimSpec configuration. A volumeClaimSpecTemplate can be referenced from more than one component; it merely identifies configuration settings and does not result in a direct instantiation of a persistent volume. The settings are used by referencing components when creating their PersistentVolumeClaims at install time.

If the component supports it, then you can override the defaultVolumeSource setting at the component level by defining a supported VolumeSource on that component. At present, only the keycloak/mysql component supports a volumeSource field override.

Examples

The following example shows how to define a dev profile with different persistence settings for the monitoring components and the Keycloak/MySQL instance.

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: kind-verrazzano-with-persistence
spec:
  profile: dev
  defaultVolumeSource:
    persistentVolumeClaim:
      claimName: default  # Use the "default" volume template
  components:
    keycloak:
      mysql:
        volumeSource:
          persistentVolumeClaim:
            claimName: mysql  # Use the "mysql" PVC template for the MySQL volume configuration
  volumeClaimSpecTemplates:
  - metadata:
      name: default      # "default" is a known template name, and will be used by Verrazzano components by default if no other template is referenced explicitly
    spec:
      resources:
        requests:
          storage: 2Gi
  - metadata:
    spec:
      resources:
        requests:
          storage: 5Gi  # default

The following example shows how to define a dev profile where all resources use emptyDir by default.

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: storage-example-dev
spec:
  profile: dev
  defaultVolumeSource:
    emptyDir: {}  # Use ephemeral storage for dev mode for all Components

Next steps

To continue, see the Installation Guide.


Last modified March 8, 2021: fix up docs (#104) (85b1d53)