[ckan-dev] API Performance

Tue Jun 25 09:50:09 UTC 2013

Hi Toby,

As Ross (T) raised API performance in another thread, I thought I'd ask - is anything happening about API performance generally? Read as well as write.

The APIController can be quite memory hungry when returning large results and I've had some small success in using a generator instead of passing around large strings. I imagine that there might be an efficient way of using a generator to return each item for the results (as it is encoded) rather than directly encoding a blob as json in memory. 

Ross (J).

On 24 June 2013 16:58, Ross Thompson <ross.thompson.ca at gmail.com> wrote:
> 2. Performance issues when dealing with 180000+ datasets. Loading and
> updating larges numbers of datasets through the API can take a weekend.