[okfn-labs] ElasticSearch backup/restore

[Nick Stenning]

Dear all,

As part of work on AnnotateIt, I've had a need to copy data from
ElasticSearch between different indices (for example, copying data
from the production to the staging index, in order to do proper
acceptance testing on what was once live data).

It's actually a little tricky to do this in a nice way (or so I
found), but I've now got a workable solution which I thought might be
useful for some of you. It's committed to the sysadmin repo:

  https://bitbucket.org/okfn/sysadmin/changeset/0ed32c94c47f

And you use it thusly:

  backup-es.sh localhost:9200 index_name

Which will generate a mappings file, and a line-delimited file with
all your documents in. You can then:

  restore-es.sh localhost:9200 another_index_name mappings.json documents.json

It's not fast or clever, but I hope some of you find this useful nonetheless.

Best,
Nick