[ECODP-dev] Production database size

Thu Sep 26 15:06:26 UTC 2013

Hi John,

thanks for the investigation.

Is the api call

api.purge=http://localhost:8008/data/api/action/purge_revision_history

equal to the purge-package-extra-revision paster command?

kind regards,

Bert

2013/9/26 John Glover <john.glover at okfn.org>

> Hi Bert,
>
> I have had a look at the PO database dump. The sizes of the largest tables
> are as follows:
>
> public.kombu_message                           | 15 GB
> pg_toast.pg_toast_78586                        | 3662 MB
> public.task_status                             | 1548 MB
> public.package_extra_revision                  | 811 MB
> public.task_status_entity_id_task_type_key_key | 670 MB
> public.task_status_pkey                        | 458 MB
>
> Apart from package_extra_revision, the other large tables all refer to
> tables used by the old CKAN extensions (such as ckanext-archiver,
> ckanext-datastorer and ckanext-qa). These should not be installed any more.
>
> In the previous release, we supplied a paster command to purge the
> task_status and kombu_message tables:
>
> paster --plugin=ckanext-ecportal ecportal purge-task-data -c <config>
>
> Running this should get rid of most of this unnecessary data (although you
> will probably to do a VACUUM in postgres afterwards to reclaim space). This
> should only have to be run once if the extensions have been correctly
> uninstalled. If these tables continue to grow, then at least one of the
> extensions is still installed. So, make sure that none of the following
> appear in the ckan.plugins config list: archiver, qa, datastorer.
>
> From looking at the timestamps of the task_status entries, it would seem
> at least 1 of these extensions is still installed, as the last write was by
> the datastorer on the 4th September.
>
>  The package revision table is also quite large, but this is expected due
> to the daily updating of all Eurostat packages. This should be regularly
> cleared with the purge-package-extra-revision paster command as in previous
> releases.
>
> Regards,
> John
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130926/baaada3c/attachment.html>