[ckan-discuss] dataproxy timeout

Fri Mar 15 14:18:14 GMT 2013

On 15 March 2013 13:45, Ross Jones <ross at servercode.co.uk> wrote:

> Hi Davide,
>
> On data.gov.uk we're now using a modified version of the jsonpdataproxy
> that will serve the same format response from /data/preview/<resource_id>
> which is working fairly well.  It's modified to run with the local archived
> version of the resource should you also be running ckanext-archiver.
>

This is great stuff Ross :-) - see comments below ...

@Davide: if you want the plain dataproxy that runs
jsonpdataproxy.appspot.com you can get it here
https://github.com/okfn/dataproxy

I should point out that the main reason jsonpdataproxy is slow is that it
runs on google app engines free tier (it is after all a free service).
Responsiveness and performance could no doubt be dramatically improved by
dedicating more resources to it. Alternatively if you use the datastore you
no longer need the dataproxy at all!

> It requires a change to recline.js, and it might be more useful to modify
> the current recline backend to accept any url as long as the target
> understands the 'protocol'.  As we're using an old version of recline (for
> now) we haven't yet done that.
>
> I am currently in the process of modifying it so that it uses
> github.com/okfn/messytables, and so is dependant on a branch within that.
>

Strong recommendation: use https://github.com/okfn/dataconverters rather
than plain messytables. It's easier to use and provides a good wrapper.
It's what dataproxy now uses.

> Now that I've got the disclaimers out of the way, it is at
> https://github.com/datagovuk/ckanext-datapreview if you want to give it a
> try. If you have any problems, do let me know (or add tickets to github).
>

Would it be worth trying to collaborate a bit here - perhaps produce a
common conversion app (and possibly a common service?)

Rufus

> Ross
>
>
>
> On Fri, Mar 15, 2013 at 1:17 PM, Davide <davide.savazzi at gmail.com> wrote:
>
>> Hi,
>>
>> I'm configuring 3 new CKAN 1.8 instances. I have an issue about CSV
>> file preview, at first it didn't work because DataProxy doesn't answer
>> after 5 seconds:
>>
>> Request Error: Backend did not respond after 5 seconds
>>
>> To solve this I've edited
>> ckan/public/scripts/vendor/recline/recline.js and increased the
>> timeout.
>>
>> Is there a way to avoid DataProxy usage or is there a DataProxy
>> version that I can install on my own server?
>>
>> Thanks,
>> --
>> ==
>> Our support, Your Success! Visit http://opensdi.geo-solutions.it for
>> more information.
>> ==
>>
>> Davide Savazzi
>> @svzdvd
>> Senior Software Engineer
>>
>> GeoSolutions S.A.S.
>> Via Poggio alle Viti 1187
>> 55054  Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax:      +39 0584 1660272
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>>
>> _______________________________________________
>> ckan-discuss mailing list
>> ckan-discuss at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-discuss
>>
>
>
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20130315/2b5a6c7c/attachment.htm>