[ckan-dev] Future, flask, breaking things, funding.

Mon Sep 14 13:15:05 UTC 2015

Hi Denis,

> On 14 Sep 2015, at 13:57, Denis Zgonjanin <deniszgonjanin at gmail.com> wrote:
> Now that government is (slowly) catching on, more stream, API, and even real-time data is being published. CKAN doesn't do a great job here. The biggest obstacle to creating nice extensions to CKAN for non-file data is that Pylons is still firmly stuck within the HTTP request-response lifecycle. 

Sure, and I don't think that it's anything that was really intentional, just something that sort of happened - there isn't a huge amount of choice, where we are, for doing this without introducing queues and associated dependencies, config, extra monitoring etc.  It annoys me at least twice a week.

> Porting CKAN to flask is no small feat, so let's make sure we do it right. Now that we're not using CKAN to just host static files anymore, we need to have better, built-in async support in CKAN. Perhaps this means moving to Python 3 where we'll have asyncio (and hopefully a future version of flask will work well with it). Other frameworks, like tornado, are also quite lightweight and support this out of the box for python 2.x.

I will avoid making the unpopular comment about language choice ;) but I think it is definitely worth the effort to look at alternative frameworks, and to start thinking about what 3.0 looks like.  It isn't imminent, and so we should definitely have plenty of time to consider the options before choosing one.  We also need to consider what the key factors are, is it just increasing contributions, is it a focus on more support for concurrency, is it just moving to py3?

Hesitating to suggest a tech + user working group, but maybe it's the best thing to do?  Would be great to have user involvement in making these decisions as they're not 100% technical ones ... you never know, maybe the steering group might chip in ;)

Cheers

Ross.