[ckan-dev] Backup strategy for CKAN

Wed Apr 9 18:12:15 UTC 2014

Hi Florian,

That's really cool. Is your CKAN instance already online? Where is it? Any
caveats on using it with Docker?

Cheers,

2014-04-09 0:15 GMT-03:00 Florian May <florian.wendelin.mayer at gmail.com>:

> Hi Stéphane,
>
> we're deploying our CKAN using Docker linux containers. In our docker
> image build process we copy out storage folders and the database from the
> (non persistent) container into persistent directories within a BTRFS
> snapshotting file system.
> That simplifies a few things for us:
> - All read-only files (software, config, dependencies) are located within
> the Docker image, which also contains all installed extensions,
> - All read/write files, the installed / set up / populated database, plus
> uploaded attachments are located within the persistent folder, making
> migration a "build the image and copy the persist folder" job,
> - the snapshotting file system allows us to roll back the CKAN instance to
> a sane state, should bad things happen, instead of having to
> migrate/install.
>
> Out backup strategy is:
> - manual backup of dev/test servers: synchronise persistent folder to a
> save place using rsync.
> - automated backup of prod servers: configure snapshotting intervals of
> BTRFS appropriately, and schedule a cron job to rsync persistent folder for
> good measure.
> We'll be updating our Docker image in the next weeks, but feel free to
> have a squiz at the code (rsync backup settings) at
> https://bitbucket.org/dpaw/dpaw_docker/src/tip/ckan/
>
> Cheers,
> Florian Mayer
> Department of Parks & Wildlife, Western Australia
>
>
>
> On Wed, Apr 9, 2014 at 8:43 AM, Hayden Waring <hayden at opengovgear.com>wrote:
>
>> Hey Stéphane,
>>
>> I recently did a CKAN migration. I simply dumped the database, copied the
>> storage folders, and took notes of which extensions I had installed.
>> I loaded the database onto a second instance of CKAN, swapped in my
>> storage folders, and installed my extensions from git.
>> With a basic script I was able to reboot the exact machine in under ten
>> minutes. Depending on how much content you have I'm sure that would be able
>> to get comparable results restoring a failed machine.
>>
>> DB synchronization is an option that we have been working with but it is
>> slightly more expensive.
>>
>> I would recommend it if you are in a situation where downtime is not an
>> option or you are dealing with large amounts of data that cannot be moved
>> and loaded quickly.
>>
>> Hayden Waring
>> OpenGovGear
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>

-- 

Vítor Baptista

Developer  |  http://vitorbaptista.com |
LinkedIn<http://www.linkedin.com/in/vitorbaptista>|
@vitorbaptista <http://twitter.com/vitorbaptista>

The Open Knowledge Foundation <http://okfn.org>

*Empowering through Open Knowledge*

http://okfn.org/  |  @okfn <http://twitter.com/okfn>  |  OKF on
Facebook<https://www.facebook.com/OKFNetwork> |
Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140409/e6666a95/attachment-0003.html>