[ckan-dev] Preview from datastore, count(*) takes very long time on large resources.

Wed Dec 11 09:15:26 UTC 2013

Hello

The issue here is that if you are going to get pagination correct you
need to count all the rows for each query.  Doing the count in the
above way is the quickest solution as it does not mean 2 queries.

I am not sure of a solution to this apart from using postgres
statistics table to make an estimate of the size instead of the actual
count.

David

On 11 December 2013 09:00, Henrik Aagaard Sørensen <BU1G at tmf.kk.dk> wrote:
> I’ve got several large resources where the datastore contains 8-11 mio.
> rows.
>
>
>
> Previewing those takes a very long time, which seems to be caused by the:
>
> count(*) over() AS "_full_count"
>
> which occurs in the SQL called.
>
>
>
> Could this call be placed elsewhere or could something else be done?
>
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>