[ECODP-dev] Script for dumping the live database

Fri Mar 1 13:31:35 UTC 2013

Hello

I have a created a version of the backup script with user data removed and
tested in on odp.tenforce.com

This really is important if we are going to find out the performance issues
with their site.

Thanks

David

On Wed, Feb 27, 2013 at 3:53 PM, Bert Van Nuffelen <
bert.van.nuffelen at tenforce.com> wrote:

> Hi David,
>
> Thanks for the reply.
> Indeed the script is doing more than just creating a backup of the
> postgreSQL but that does not harm. Moreover it is necessary if you want to
> test CKAN w.r.t. the PO data.
> The intension of the script is creating a restoreable backup. And
> eventually migrate this backup on another machine. That is an outstanding
> task.
>
> You refer to the new script, but I do not find an attachment. Am I
> overlooking something?
>
> kind regards,
>
> Bert
>
>
>
> 2013/2/27 David Raznick <david.raznick at okfn.org>
>
>> Hello
>>
>> That particular dump script not only dumps the database but all the whole
>> site including all source code and all uploaded files.  This is really
>> unnecessary if we are just want a backup of the database for this purpose.
>>
>> The supplied new script needs to be run on the db server as root.
>>
>> David
>>
>>
>>
>> On Mon, Feb 25, 2013 at 10:00 AM, Bert Van Nuffelen <
>> bert.van.nuffelen at tenforce.com> wrote:
>>
>>> Hi John,
>>>
>>> have a look to the scripts I send you. They are based on Ian Murray's
>>> information last autumn. If you extend that one with an option to exclude
>>> the user info that would great.
>>>
>>> kind regards,
>>>
>>> Bert
>>>
>>>
>>>
>>> 2013/2/25 John Glover <john.glover at okfn.org>
>>>
>>>> Hi Bert,
>>>>
>>>> The script is only 3 lines so I'm not sure how much documentation
>>>> should be necessary. It needs to be able to access the CKAN database with
>>>> whatever user can read the data on whatever machine the PO has the database
>>>> running on. I can add variables to the top to specify these.
>>>>
>>>> The only information taken from the user table is the id, name and
>>>> creation date (which could be useful in debugging). Do we need to restrict
>>>> this further?
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>
>>>> On 25 February 2013 10:19, Bert Van Nuffelen <
>>>> bert.van.nuffelen at tenforce.com> wrote:
>>>>
>>>>> Hi Darwin,
>>>>>
>>>>> A what late comment on this script: This script will not be accepted
>>>>> by PO. The obvious are which user; which machine; no docs....
>>>>> Can you extend this script(s) with an option to deal with anonymizing
>>>>> the users? This are the backup scripts we have already submitted to PO and
>>>>> which satisfy already 70% of their concerns.
>>>>>
>>>>> kind regards,
>>>>>
>>>>> Bert
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2013/2/13 Darwin Peltan <darwin.peltan at okfn.org>
>>>>>
>>>>>>  Hi Bert,
>>>>>>
>>>>>> Please see attached the script to dump the live database. Please can
>>>>>> you test it yourself on the 10F test server and then send it to the PO.
>>>>>>
>>>>>> It blocks out any sensitive data and produces 3 files that need to be
>>>>>> supplied back.
>>>>>>
>>>>>> Many thanks,
>>>>>>
>>>>>> Darwin
>>>>>>
>>>>>> Darwin Peltan
>>>>>> Project Manager
>>>>>>
>>>>>> The Open Knowledge Foundation
>>>>>> http://www.okfn.org
>>>>>>
>>>>>> Skype: darwinp
>>>>>> Twitter: @darwin
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ecodp-dev mailing list
>>>>>> Ecodp-dev at lists.okfn.org
>>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Bert Van Nuffelen
>>>>>
>>>>> Semantic Technologies Software Architect at TenForce
>>>>> www.tenforce.be
>>>>>
>>>>> Bert.Van.Nuffelen at tenforce.com
>>>>> Office: +32 (0)16 31 48 60
>>>>> Mobile:+32 479 06 24 26
>>>>> skype: bert.van.nuffelen
>>>>> _______________________________________________
>>>>> Ecodp-dev mailing list
>>>>> Ecodp-dev at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ecodp-dev mailing list
>>>> Ecodp-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Bert Van Nuffelen
>>>
>>> Semantic Technologies Software Architect at TenForce
>>> www.tenforce.be
>>>
>>> Bert.Van.Nuffelen at tenforce.com
>>> Office: +32 (0)16 31 48 60
>>> Mobile:+32 479 06 24 26
>>> skype: bert.van.nuffelen
>>>
>>> _______________________________________________
>>> Ecodp-dev mailing list
>>> Ecodp-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>
>>>
>>
>> _______________________________________________
>> Ecodp-dev mailing list
>> Ecodp-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>
>>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130301/6c700bc5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: backup_ckan_anon.sh
Type: application/x-sh
Size: 1330 bytes
Desc: not available
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130301/6c700bc5/attachment-0001.sh>