ArangoBackup Custom Resource

Full CustomResourceDefinition reference ->

The ArangoBackup Operator creates and maintains ArangoBackups in a Kubernetes cluster, given a Backup specification. This deployment specification is a CustomResource following a CustomResourceDefinition created by the operator.

Defining a secret for backup upload or download

credentialsSecretName in spec.download and spec.upload expects the next structure for secret:

apiVersion: v1
data:
  token: <json token>
kind: Secret
metadata:
  name: <name>
type: Opaque

JSON Token options are described on the rclone page. We can define more than one protocols at same time in one secret.

This field is defined in json format:

{
  "<protocol>": {
    "type":"<type>",
    ...parameters
    }
}

AWS S3 example - based on rclone S3 documentation and interactive process:

{
  "S3": {
    "type": "s3", # Choose s3 type
    "provider": "AWS", # Choose one of the providers
    "env_auth": "false", # Define credentials in next step instead of using ENV
    "access_key_id": "xxx",
    "secret_access_key": "xxx",
    "region": "eu-west-2", # Choose region
    "location_constraint": "eu-west-2", # Match the region (see notes below)
    "no_check_bucket": "true"
  }
}

and you can from now use S3://bucket/path.

Note the following when configuring rclone:

• acl: AWS buckets created since April 2023 default to Bucket owner enforced Object Ownership, which rejects requests with an ACL header. Omit the acl key (or set it to "") for such buckets. It may still be required for some S3-compatible providers and older AWS buckets with ACLs re-enabled.
• Region: For AWS S3 with a region other than us-east-1, set the location_constraint to the region, "no_check_bucket": "true", or both. Otherwise rclone (v1.68.0 and later) sends an unspecified location constraint that AWS rejects with an IllegalLocationConstraintException.
• Checksums: For S3-compatible providers (e.g. GCS, Ceph, MinIO, Wasabi), uploads may fail unless you set "use_data_integrity_protections": "false", because rclone (v1.68.0 and later) defaults to CRC32/CRC64 checksums while these providers may expect MD5.
• Provider quirks: rclone auto-handles quirks for known providers (e.g. use_x_id, sign_accept_encoding, use_multipart_uploads). You may need to set these manually if your provider is not recognized.

Use IAM with Amazon EKS

Instead of creating and distributing your AWS credentials to the containers or using the Amazon EC2 instance’s role, you can associate an IAM role with a Kubernetes service account and configure pods to use the service account.

1. Create a Policy to access the S3 bucket.

   aws iam create-policy \
   --policy-name S3-ACCESS_ROLE \
   --policy-document \
   '{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": "s3:ListAllMyBuckets",
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": "*",
           "Resource": "arn:aws:s3:::MY_BUCKET"
       },
       {
           "Effect": "Allow",
           "Action": "*",
           "Resource": "arn:aws:s3:::MY_BUCKET/*"
       }
   ]
   }'
   

2. Create an IAM role for the service account (SA).

   eksctl create iamserviceaccount \
     --name SA_NAME \
     --namespace NAMESPACE \
     --cluster CLUSTER_NAME \
     --attach-policy-arn arn:aws:iam::ACCOUNT_ID:policy/S3-ACCESS_ROLE \
     --approve
   

3. Ensure that you use that SA in your ArangoDeployment for dbservers and coordinators.

   apiVersion: database.arangodb.com/v1
   kind: ArangoDeployment
   metadata:
     name: cluster
   spec:
     image: arangodb/enterprise
     mode: Cluster
   
     dbservers:
       serviceAccountName: SA_NAME
     coordinators:
       serviceAccountName: SA_NAME
   

4. Create a Secret Kubernetes object with a configuration for S3.

   apiVersion: v1
   kind: Secret
   metadata:
     name: arangodb-cluster-backup-credentials
   type: Opaque
   stringData:
     token: |
       {
         "s3": {
           "type": "s3",
           "provider": "AWS",
           "env_auth": "true",
           "location_constraint": "eu-central-1",
           "region": "eu-central-1",
           "no_check_bucket": "true"
         }
       }
   

5. Create an ArangoBackup Kubernetes object with upload to S3.

   apiVersion: "backup.arangodb.com/v1alpha"
   kind: "ArangoBackup"
   metadata:
     name: backup
   spec:
     deployment:
       name: MY_DEPLOYMENT
     upload:
       repositoryURL: "s3:MY_BUCKET"
       credentialsSecretName: arangodb-cluster-backup-credentials
   

Examples:

Create simple Backup

apiVersion: "backup.arangodb.com/v1"
kind: "ArangoBackup"
metadata:
  name: "example-arangodb-backup"
  namespace: "arangodb"
spec:
  deployment:
    name: "my-deployment"

Action:

Create Backup on ArangoDeployment named my-deployment

Create and upload Backup

apiVersion: "backup.arangodb.com/v1"
kind: "ArangoBackup"
metadata:
  name: "example-arangodb-backup"
  namespace: "arangodb"
spec:
  deployment:
    name: "my-deployment"
  upload:
    repositoryURL: "S3:test/kube-test"
    credentialsSecretName: "my-s3-rclone-credentials"

Action:

Create Backup on ArangoDeployment named my-deployment and upload it to S3://test/kube-test.

Download Backup

apiVersion: "backup.arangodb.com/v1"
kind: "ArangoBackup"
metadata:
  name: "example-arangodb-backup"
  namespace: "arangodb"
spec:
  deployment:
    name: "my-deployment"
  download:
    repositoryURL: "S3:test/kube-test"
    credentialsSecretName: "my-s3-rclone-credentials"
    id: "backup-id"

Download Backup with id backup-id from S3://test/kube-test on ArangoDeployment named my-deployment

Restore

To restore a data for deployment for specific backup, use spec.restoreFrom field of ArangoDeployment.