Create an Elasticsearch index lifecycle management (ILM) policy

Already configured Elastic for ODP?

If you have already configured Elastic for earlier ODP Elastic samples that use an old index naming pattern:

Keep the ILM policy for those old indices, to continue managing them while they still exist. Delete that policy only after all of the indices that refer to it have been deleted.
For new indices for the current starter dashboards, use a policy with a different name, as shown in the following starter sample.
The following starter policy is named omegamon-ds-ilm-policy (note the ds-), which is deliberately different from the omegamon-ilm-policy name used in earlier samples.
The different policy name reflects the fact that the current starter Elastic configuration uses data streams, and also enables old and new indices to be managed separately.

Warning

If you don't use an ILM policy, and you keep forwarding data to Elastic, then you will eventually run out of disk space.

Here is a starter policy:

PUT _ilm/policy/omegamon-ds-ilm-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "10gb",
            "max_age": "10d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "10d",
        "actions": {
          "set_priority": {
            "priority": 50
          }
        }
      },
      "delete": {
        "min_age": "20d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

Tips

You can paste this definition into the Kibana console. In Kibana, select Management ▶ Dev Tools ▶ Console.

This starter policy is designed for use with data streams. The policy contains a rollover action that triggers the creation of a new backing index for the data stream either when the primary shard size reaches 10 GB, or 10 days after the current backing index was created, whichever occurs first.

This starter policy deletes indices 20 days after rollover.

This policy is a starter only. Create a policy that matches your organization's requirements and available disk space.

For details on creating index lifecycle policies and associating them with indices, see the Elastic Stack ILM documentation.