[ckan-discuss] CKAN is slowwwwww

Sun Oct 3 16:31:57 BST 2010

On 2 Oct 2010, at 16:11, Rufus Pollock wrote:
> For this kind
> of bulk analysis it may be worth reinstating our daily json dumps of
> the entire db.

This would be quite handy, so if it is easy to do, then please! It  
could remove some of the API load.

Richard

>
> Rufus
>
> 2010/9/30 David Read <david.read at okfn.org>:
>> Christophe,
>>
>> Yes it shouldn't be this slow doing 1500 queries. We've suffered
>> performance problems in the past 24 hours and this is probably
>> related. Having said that, I've opened a ticket to take a proper look
>> at this:
>> http://knowledgeforge.net/ckan/trac/ticket/667
>>
>> This particular problem sounds ilke a job for the 'resource search'
>> feature, which achieves what you want in one query, taking under a
>> second:
>> http://ckan.net/api/search/resource?format=api/sparql
>>
>> and you could add &all_fields=1 to get all the package properties  
>> to process.
>>
>> I'm afraid this is a new feature so has been put into the ckanclient
>> yet, but should not be too hard to add in, as package search is  
>> almost
>> identical. Do write back to the list to let us know how you get on  
>> and
>> if you want any more help.
>>
>> David
>>
>> 2010/9/30 Christophe Guéret <cgueret at few.vu.nl>:
>>>  Hello!
>>>
>>> I've made a small script (attached to this mail) using the python  
>>> CKAN API
>>> to browse the content of CKAN in search for SPARQL end points.
>>> Everything works fine apart from the fact that this script takes  
>>> at least 2h
>>> to run! I was hoping that it would take no more than a few  
>>> seconds, or maybe
>>> a minute or so. But not hours ;-)
>>>
>>> Is it normal that CKAN is so slow to browse?
>>>
>>> Cheers,
>>> Christophe
>>>
>>>
>>> --
>>> Dr. Christophe Guéret (cgueret at few.vu.nl)
>>> http://cgueret.net
>>> Postdoc working on SOKS (http://www.few.vu.nl/soks)
>>> Knowledge Representation&  Reasoning Group
>>> Computational Intelligence Group
>>> Department of Computer Science, AI
>>> VU University Amsterdam
>>>
>>>
>>> _______________________________________________
>>> ckan-discuss mailing list
>>> ckan-discuss at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>>>
>>>
>>
>> _______________________________________________
>> ckan-discuss mailing list
>> ckan-discuss at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>>
>
>
>
> -- 
> Open Knowledge Foundation
> Promoting Open Knowledge in a Digital Age
> http://www.okfn.org/ - http://blog.okfn.org/
>
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss