Elasticsearch Snapshots with Azure Part 4: Create Elastic Snapshot Policy

Elasticsearch Snapshots with Azure Part 4: Create Elastic Snapshot Policy

Elasticsearch Snapshot Policy

  • schedule: What frequency and time to snapshot our data. You can make this as frequent as you require, without worrying too much about storage constraints. Snapshots are incremental, meaning that only the differences between the last snapshot and current snapshot need to be stored. If almost nothing changed between yesterday’s backup and today’s data, then the Elasticsearch snapshot will require negligible storage space, meaning that even if you have gigabytes of data, the snapshot might require just a few megabytes or less.
  • name: Defines the pattern to use when naming the snapshots
  • repository: Specifies where to store our snapshots
  • config.indices: List of the indices to include
  • retention: Is an optional parameter we can use to define when SLM can delete some of the snapshots. We can specify three options here:
  • expire_after: This is a time-based expiration. For example, a snapshot created on January 1, with expire_after set to 30 days will be eligible for deletion after January 31.
  • min_count: Tells Elasticsearch to keep at least this number of snapshots, even if all are expired.
  • max_count: Tells Elasticsearch to never keep more than this number of snapshots. For example, if we have 100 snapshots and only one is expired, but max_count is set to 50, then 50 of the oldest snapshots will be deleted – even the unexpired ones.

Step 1 - Click on "Snapshot and Restore"

Step 2 - Select "Policies"

Step 3 - Click "Create a policy"

Step 4 - Fill Out Logistics Information

Step 5 - Adding custom "cron" expression

59 59 05 1* ?

Step 6 - Snapshot Settings

For my setup I will just be going with defaults

Step 7 - Snapshot Retention

Snapshots for me will be deleted after 180 days

Step 8 - Review Policy

Step 9 - Run a snapshot to verify the setup works

Step 10 - Snapshots

Click on snapshots to see the snapshot you just made

Step 11 - Snapshot completed

Step 12 - Back in Azure Cloud

You will see the backup of indices in your blob container.