[ECODP-dev] Script for dumping the live database

Fri Mar 1 13:49:06 UTC 2013

Hi David,

One thing you have to realize is that the performance issue is part CKAN
(how the engine is build/DB schema) but certainly a part the proxy/caching
configuration.
So expect on the second level to provide advice.

Bert

2013/3/1 David Raznick <david.raznick at okfn.org>

> Hello
>
> I have a created a version of the backup script with user data removed and
> tested in on odp.tenforce.com
>
> This really is important if we are going to find out the performance
> issues with their site.
>
> Thanks
>
> David
>
>
>
>
> On Wed, Feb 27, 2013 at 3:53 PM, Bert Van Nuffelen <
> bert.van.nuffelen at tenforce.com> wrote:
>
>> Hi David,
>>
>> Thanks for the reply.
>> Indeed the script is doing more than just creating a backup of the
>> postgreSQL but that does not harm. Moreover it is necessary if you want to
>> test CKAN w.r.t. the PO data.
>> The intension of the script is creating a restoreable backup. And
>> eventually migrate this backup on another machine. That is an outstanding
>> task.
>>
>> You refer to the new script, but I do not find an attachment. Am I
>> overlooking something?
>>
>> kind regards,
>>
>> Bert
>>
>>
>>
>> 2013/2/27 David Raznick <david.raznick at okfn.org>
>>
>>> Hello
>>>
>>> That particular dump script not only dumps the database but all the
>>> whole site including all source code and all uploaded files.  This is
>>> really unnecessary if we are just want a backup of the database for this
>>> purpose.
>>>
>>> The supplied new script needs to be run on the db server as root.
>>>
>>> David
>>>
>>>
>>>
>>> On Mon, Feb 25, 2013 at 10:00 AM, Bert Van Nuffelen <
>>> bert.van.nuffelen at tenforce.com> wrote:
>>>
>>>> Hi John,
>>>>
>>>> have a look to the scripts I send you. They are based on Ian Murray's
>>>> information last autumn. If you extend that one with an option to exclude
>>>> the user info that would great.
>>>>
>>>> kind regards,
>>>>
>>>> Bert
>>>>
>>>>
>>>>
>>>> 2013/2/25 John Glover <john.glover at okfn.org>
>>>>
>>>>> Hi Bert,
>>>>>
>>>>> The script is only 3 lines so I'm not sure how much documentation
>>>>> should be necessary. It needs to be able to access the CKAN database with
>>>>> whatever user can read the data on whatever machine the PO has the database
>>>>> running on. I can add variables to the top to specify these.
>>>>>
>>>>> The only information taken from the user table is the id, name and
>>>>> creation date (which could be useful in debugging). Do we need to restrict
>>>>> this further?
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>>
>>>>> On 25 February 2013 10:19, Bert Van Nuffelen <
>>>>> bert.van.nuffelen at tenforce.com> wrote:
>>>>>
>>>>>> Hi Darwin,
>>>>>>
>>>>>> A what late comment on this script: This script will not be accepted
>>>>>> by PO. The obvious are which user; which machine; no docs....
>>>>>> Can you extend this script(s) with an option to deal with anonymizing
>>>>>> the users? This are the backup scripts we have already submitted to PO and
>>>>>> which satisfy already 70% of their concerns.
>>>>>>
>>>>>> kind regards,
>>>>>>
>>>>>> Bert
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/2/13 Darwin Peltan <darwin.peltan at okfn.org>
>>>>>>
>>>>>>>  Hi Bert,
>>>>>>>
>>>>>>> Please see attached the script to dump the live database. Please can
>>>>>>> you test it yourself on the 10F test server and then send it to the PO.
>>>>>>>
>>>>>>> It blocks out any sensitive data and produces 3 files that need to
>>>>>>> be supplied back.
>>>>>>>
>>>>>>> Many thanks,
>>>>>>>
>>>>>>> Darwin
>>>>>>>
>>>>>>> Darwin Peltan
>>>>>>> Project Manager
>>>>>>>
>>>>>>> The Open Knowledge Foundation
>>>>>>> http://www.okfn.org
>>>>>>>
>>>>>>> Skype: darwinp
>>>>>>> Twitter: @darwin
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ecodp-dev mailing list
>>>>>>> Ecodp-dev at lists.okfn.org
>>>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Bert Van Nuffelen
>>>>>>
>>>>>> Semantic Technologies Software Architect at TenForce
>>>>>> www.tenforce.be
>>>>>>
>>>>>> Bert.Van.Nuffelen at tenforce.com
>>>>>> Office: +32 (0)16 31 48 60
>>>>>> Mobile:+32 479 06 24 26
>>>>>> skype: bert.van.nuffelen
>>>>>> _______________________________________________
>>>>>> Ecodp-dev mailing list
>>>>>> Ecodp-dev at lists.okfn.org
>>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ecodp-dev mailing list
>>>>> Ecodp-dev at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Bert Van Nuffelen
>>>>
>>>> Semantic Technologies Software Architect at TenForce
>>>> www.tenforce.be
>>>>
>>>> Bert.Van.Nuffelen at tenforce.com
>>>> Office: +32 (0)16 31 48 60
>>>> Mobile:+32 479 06 24 26
>>>> skype: bert.van.nuffelen
>>>>
>>>> _______________________________________________
>>>> Ecodp-dev mailing list
>>>> Ecodp-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Ecodp-dev mailing list
>>> Ecodp-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>>
>>>
>>
>>
>> --
>> Bert Van Nuffelen
>>
>> Semantic Technologies Software Architect at TenForce
>> www.tenforce.be
>>
>> Bert.Van.Nuffelen at tenforce.com
>> Office: +32 (0)16 31 48 60
>> Mobile:+32 479 06 24 26
>> skype: bert.van.nuffelen
>>
>> _______________________________________________
>> Ecodp-dev mailing list
>> Ecodp-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>>
>>
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130301/279e431d/attachment.html>