[ckan-dev] Bootup performance

[Ian Ward]

Hi Alex,

What you're describing sounds unusual. Web start up time should not
depend on the number of datasets, and should be measured in seconds,
not minutes.

Are you running anything else during start up? Have you tried
disabling plugins in your ini file? Have you made any changes to ckan?

Ian


On Tue, Apr 7, 2015 at 6:14 AM, Alex Corbi <a.corbi at gmail.com> wrote:
> Hi Everyone,
>
> I have a performance question depending on amount of stored datasets and
> bootup times.
>
> In the context of http://data.opendevelopmentmekong.net/, which is an
> instance based on CKAN v2.2.1 deployed through uwsgi on a server with 2core
> and 4GB of memory (using docker containers), currently with VERY low traffic
> and 13 datasets hosted. In this scneario, bootup times after restart of the
> docker container for CKAN are quick and does not present any issue.
>
> However, the bootup time and derivated Issues increase considerably with the
> number of datasets. On the very same setup, but being populated with ~2000
> datasets, the CKAN instance takes up to 20 minutes to boot and sometimes
> shows an erratic behaviour after rebooting ( Internal Server Errors, random
> URLs and resources not being loaded).
>
> So, here my questions:
> - Do the characteristics of the described system (number of datasets,
> traffic) comply with the "small to medium" instance type mentioned on
> https://github.com/ckan/ckan/wiki/Hardware-Requirements? Are 2 core/4GB mem
> ok?
>
> - During the bootup process, activity on the CKAN and Postgresql side can be
> detected. both components take a big percentage of the CPU during the bootup
> (~20 minutes). What is supposed to be happening behind the scenes? Solr
> reindexing everytime CKAN restarts?
>
> - Is there any possible action to be done in order to reduce the booting
> time of a restarted CKAN instance? (DB/Solr/ckan conf.)
>
> Thanks in advance, any help is appreciated,
>
> --
> Alex Corbi
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>