[ECODP-dev] Production database size

Mon Sep 30 13:25:57 UTC 2013

Thank you John,

that clarifies it more

Best
Dimitrios

On 30/09/2013 15:22, John Glover wrote:
> Hi Dimitrios,
>
> Oh I see. No this data should not be produced by clients that are 
> doing simple reads of data such as crawlers. It is produced by changes 
> to datasets and resources, in particular the twice daily update of all 
> Eurostat resources is likely to be the main producer of data.
>
> No the purge command just does a SQL delete, so they should not need 
> exclusive access to do this. If they want to run a full VACUUM 
> afterwards then they will need a lock on each table, so it would be 
> best to make sure that the old extensions have been removed first (so 
> that they are not trying to write more data during this process) [1].
>
> I am not sure how long it should take to run, there are a large number 
> of factors involved there. It is probably best to try it on a test 
> instance and see.
>
> [1] http://www.postgresql.org/docs/9.1/static/sql-vacuum.html
>
> Regards,
> John
>
>
> On 30 September 2013 15:09, Dimitrios Mexis 
> <dimitrios.mexis at tenforce.com <mailto:dimitrios.mexis at tenforce.com>> 
> wrote:
>
>     John,
>
>     Hi, I didn't exactly asked for that.
>
>     My question is more like:
>
>     is it because google, bing and whatever crawls in the site ? as
>     well people making clicks in ckan portal ? So that means the
>     tracking gives these?
>
>     is it happening due to upload of datasets ?
>
>     is it happening due to download of datasets ?
>
>
>     Regards
>     Dimitrios
>
>
>     On 30/09/2013 15:06, John Glover wrote:
>
>         Some unnecessary data is also in the resource revision tables.
>
>
>
>     _______________________________________________
>     Ecodp-dev mailing list
>     Ecodp-dev at lists.okfn.org <mailto:Ecodp-dev at lists.okfn.org>
>     http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>
>
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130930/ae0d5d86/attachment.html>