Velero - Kubernetes Backup and Restore Made Simple

Pre-requisites

• Azure Kubernetes Service (AKS)
• Azure CLI
• kubectl
• Velero CLI

Are you concerned about protecting your Kubernetes workloads and data? If so, this article will guide you through implementing a robust backup and disaster recovery strategy. We’ll explore how to accomplish this using Velero, the open-source tool for backing up and restoring Kubernetes clusters.

Now let’s dive into the practical demonstration. I’ve prepared a complete setup that shows how to configure Velero in an Azure environment, specifically focusing on backing up both Kubernetes resources and persistent volumes. To get started, we’ll need to set up an Azure Storage Account, configure RBAC permissions, and install Velero with its Azure plugins.

How it works?

Storage Components

• Azure Blob Storage: Serves as the backup location where Velero stores all your Kubernetes resource configurations and persistent volume snapshots.
• Azure Managed Disks: When backing up persistent volumes, Velero creates snapshots of your Azure Managed Disks automatically.

Step 1: Provision prerequisites on Azure

• Azure Storage Account: Create a new storage account to store Velero backups.
• Azure AD Service Principal: Create a new service principal to authenticate Velero with Azure.
• Resource Group: Create a new resource group to manage all the resources.

Step 2: Install Velero on AKS Cluster via Helm

resource "helm_release" "velero" {
  name             = "velero"
  repository       = "https://vmware-tanzu.github.io/helm-charts"
  chart            = "velero"
  namespace        = "velero"
  create_namespace = true
  version          = "7.0.0"

  values = [
    templatefile("${path.module}/charts/velero/velero-values.yaml", {
      VELERO_STORAGE_ACCOUNT   = azurerm_storage_account.velero.name
      AZURE_CLIENT_ID          = azuread_service_principal.velero.application_id
      AZURE_CLIENT_SECRET      = azuread_service_principal_password.velero.value
      AZURE_RESOURCE_GROUP     = azurerm_resource_group.this.name
      AZURE_AKS_RESOURCE_GROUP = data.azurerm_resources.aks-nsg.resource_group_name
      AZURE_SUBSCRIPTION_ID    = data.azurerm_client_config.current.subscription_id
      AZURE_TENANT_ID          = data.azurerm_client_config.current.tenant_id
    })
  ]
}

Values for Velero Helm Chart:

credentials:
  secretContents:
    cloud: |
      AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID}
      AZURE_TENANT_ID=${AZURE_TENANT_ID}
      AZURE_CLIENT_ID=${AZURE_CLIENT_ID}
      AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}
      AZURE_RESOURCE_GROUP=${AZURE_AKS_RESOURCE_GROUP}
      AZURE_CLOUD_NAME=AzurePublicCloud

configuration:
  backupStorageLocation:
    - name: azure
      provider: azure
      bucket: velero-backups-prod
      config:
        resourceGroup: ${AZURE_RESOURCE_GROUP}
        storageAccount: ${VELERO_STORAGE_ACCOUNT}
        subscriptionId: ${AZURE_SUBSCRIPTION_ID}

  volumeSnapshotLocation:
    - name: azure
      provider: azure
      config:
        resourceGroup: ${AZURE_RESOURCE_GROUP}
        subscriptionId: ${AZURE_SUBSCRIPTION_ID}

initContainers:
  - name: velero-plugin-for-azure
    image: velero/velero-plugin-for-microsoft-azure:v1.5.0
    volumeMounts:
      - mountPath: /target
        name: plugins

Step 3: Provision Scheduled Backups via Custom Resource Definition (CRD)

• Velero Schedule: Create a new schedule to backup your Kubernetes resources and persistent volumes. In includedResources field, you can specify the resources you want to backup.

resource "kubectl_manifest" "velero_schedule" {
  yaml_body = yamlencode({
    apiVersion = "velero.io/v1"
    kind       = "Schedule"
    metadata = {
      name      = "daily-pvc-backup"
      namespace = "velero"
    }
    spec = {
      schedule = "0 13 * * *" # 1 PM UTC
      template = {
        snapshotVolumes         = true
        includeClusterResources = null # This means "auto"
        includedResources = [
          "persistentvolumeclaims",
          "persistentvolumes"
        ]
        storageLocation         = "azure"
        volumeSnapshotLocations = ["azure"]
        includedNamespaces      = ["<put your namespace here>"]
        ttl                     = "720h" # 30 days
      }
    }
  })
}

Step 3: Backup and Restore

• Backup: Create a backup of your Kubernetes resources and persistent volumes.

1. First, check the available backups and schedule:

main X φ k get Schedule -n velero
NAME               STATUS    SCHEDULE     LASTBACKUP   AGE    PAUSED
daily-pvc-backup   Enabled   0 13 * * *   21h          124d   
main X φ 

main X φ k get Backup -n velero
NAME                              AGE
daily-pvc-backup-20241216130031   45h
daily-pvc-backup-20241217130032   21h

Optionally you could use velero backup get

Execute the Restore Script

• Replace <backup_name> with the actual name of the backup you want to restore
• In your terminal, type:

velero restore create --from-backup "${BACKUP_NAME}" --include-resources persistentvolumeclaims,persistentvolumes

Monitor Restore Progress

• Use the command: velero restore get
• For more detailed information: velero restore describe <restore_name>

Verify Restored Resources

• Check if PersistentVolumeClaims and PersistentVolumes were restored correctly
• Use kubectl commands to verify the state of restored resources
• Check backups and restores in Azure Portal: Storage Account and Container

Summary

Velero is a reliable backup and disaster recovery tool for Kubernetes clusters in Azure, offering both opportunities and challenges.

Key Benefits

• Seamless Azure integration with Blob Storage and Managed Disks
• Flexible backup options (full/selective)
• Cross-cluster restoration and migration capabilities
• Automated scheduled backups

Challenges

• Initial setup complexity with Azure permissions
• Resource-intensive for large backups
• Storage costs with frequent backups

Common Use Cases

• Disaster recovery and data protection
• Cluster migration and environment promotion
• Compliance requirements and audit needs
• Development and testing environment management

Despite some complexity in setup, Velero remains essential for protecting Kubernetes workloads in Azure environments, particularly for organizations requiring robust backup and recovery solutions.