[ECODP-dev] Production database size

Thu Sep 26 11:46:10 UTC 2013

Hi Bert,

I have had a look at the PO database dump. The sizes of the largest tables
are as follows:

public.kombu_message                           | 15 GB
pg_toast.pg_toast_78586                        | 3662 MB
public.task_status                             | 1548 MB
public.package_extra_revision                  | 811 MB
public.task_status_entity_id_task_type_key_key | 670 MB
public.task_status_pkey                        | 458 MB

Apart from package_extra_revision, the other large tables all refer to
tables used by the old CKAN extensions (such as ckanext-archiver,
ckanext-datastorer and ckanext-qa). These should not be installed any more.

In the previous release, we supplied a paster command to purge the
task_status and kombu_message tables:

paster --plugin=ckanext-ecportal ecportal purge-task-data -c <config>

Running this should get rid of most of this unnecessary data (although you
will probably to do a VACUUM in postgres afterwards to reclaim space). This
should only have to be run once if the extensions have been correctly
uninstalled. If these tables continue to grow, then at least one of the
extensions is still installed. So, make sure that none of the following
appear in the ckan.plugins config list: archiver, qa, datastorer.

>From looking at the timestamps of the task_status entries, it would seem at
least 1 of these extensions is still installed, as the last write was by
the datastorer on the 4th September.

The package revision table is also quite large, but this is expected due to
the daily updating of all Eurostat packages. This should be regularly
cleared with the purge-package-extra-revision paster command as in previous
releases.

Regards,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130926/c92d467f/attachment.html>