[ckan-dev] What would you remove from CKAN core and why?

Fri Aug 3 14:27:47 UTC 2018

Hi all,

Sorry it took me a while to answer this but thanks a lot Kevin to
start such important discussion.

On the items mentioned so far:

* DataProxy: yes please, it must go, see
https://github.com/ckan/ideas-and-roadmap/issues/217#issuecomment-410227350

* stats/tracking: +1 We actually ship stats it enabled by default and
I'm sure most people is not aware of this. I doubt there's much people
that knows how the tracking code works. They both would benefit
immensely of a rewrite by someone that wants to make them actually
useful.

* model/context. I'm not sure about the models vs dict, I think the
"logic layer" using dicts and separated from the models has worked
quite well. I think we all agree that the "passing the context around"
pattern has been abused and leads to confusion so I'd be happy to see
that refactored. The problem is that it touches pretty much all plugin
hooks and how controllers/blueprints interact with the actions, so it
might be quite a disruptive change. But a major version change might
be the time to do that

I think is fair to say that for a while the approach favored by the
tech team has been to try an keep core as lean as possible and
implement extra features to extensions. There is a natural tendency in
open source projects to grow the codebase and the number of users,
which is not correlated with a number of resources (ie tech team devs)
devoted to maintaining it (see [1]). CKAN is no exception so shifting
some of the features to an extension lowers the entry barrier for
contributors and allows a faster development pace. That's not to say
that code can make the opposite path of course. I think it's fair to
say that many people would like to see ckanext-scheming, fluent or
others shipped as part of core. And of course there is the danger and
has happened in some occasions that extensions get also neglected.

Adrià

[1] https://twitter.com/NumFOCUS/status/1017418416725155840

[2] https://github.com/ckan/ckan/issues/4212
[3] https://lists.okfn.org/pipermail/ckan-dev/2018-April/011386.html

On 31 July 2018 at 16:24, Kevin Brochet-Nguyen <kevin at viderum.com> wrote:
> CKAN’s context has caused me some headaches in the past, too. I’d put
> context and models vs. dicts in the “improvable” rather than the “removable”
> column, though.
>
> As for dataproxy and anything that could be described as not part of core
> but required by core—it's worth considering removing it and converting into
> an external add-on as you suggest.
>
> And yes, maybe only “considering removing” and not "actually removing". I
> understand what you say about breaking things gently over time and that it’s
> possible that a feature you or I might not like is well-used by others.
>
> At the same time, maybe thinking about drastic changes, even if we don’t
> intend to follow through with them, will inspire us. Flask and Python 3 are
> right around the corner. We’re definitely going to be breaking some things
> in the next year, with the benefit that we’ll be working with a modern
> framework on modern Python. Users are already going to be faced with the
> questions of what and when and how to upgrade.
>
> What do you think about the idea of removing tracking and stats from core?
> Just the principle: we know some (even many) users will be inconvenienced,
> but what if that also drives them to adopt a better solution, or even to
> contribute to or fund work on a better solution?
>
> (And to be clear tracking/stats isn’t my main concern. I just think it’s a
> good example of something that could be removed and the removal would spur
> the community towards adopting/creating something better. Your datapoxy
> suggestion falls into the same category. Maybe cutting it free from core
> would make it easier to improve it or to replace it with something better. I
> also have things like Python’s requests and attrs libraries in mind here:
> they could be part of core Python, but they’re not because putting them in
> core would slow down their pace of development. Not being in core doesn’t
> necessarily mean not being available.)
>
> On 24. July 2018 at 17:39:59, ross at mailbolt.com (ross at mailbolt.com) wrote:
>
> Okay, I’ll bite.
>
>
>
> The dataproxy used for previews of CSV/XLS not in the datastore.  It’s
> running on Google AppEngine, it only works during a full moon, and doesn’t
> help with data stored on an internal network.  It’s not technically part of
> core, but core depends on it, and it should probably be an external thing
> you can install alongside CKAN.
>
>
>
> Number 2 would be the general abuse of ckan.model models vs dicts in various
> places, and the way context is used throughout the logic layer (see
> https://github.com/ckan/ideas-and-roadmap/issues/53).
>
>
>
> While I think we all have bits of CKAN that we’d rather not be there, there
> are just as many people using them – there’s nothing quite so frustrating as
> a point release breaking things, so the general approach is to break things
> gently, over time, and ideally with advance warning. This is not a bad
> thing, but it’s kind of hard to just remove big chunks of functionality
> when/if people are depending on them.  I guess a separate extension is the
> answer where it’s possible.
>
>
>
> I actually put together a quick survey for contributors, and
> non-contributors to try and learn a bit more about the reasons people do, or
> do not contribute.  I’ll send it out later.
>
>
>
> On a related note, has the steering group come up with a roadmap? Presumably
> customers are telling them what they need… it’d be good to see what their
> expectations are over the next 12 months.
>
>
>
> Ross.
>
>
>
> From: ckan-dev <ckan-dev-bounces at lists.okfn.org> On Behalf Of Kevin
> Brochet-Nguyen
> Sent: 24 July 2018 08:41
> To: ckan-dev at lists.okfn.org
> Subject: Re: [ckan-dev] What would you remove from CKAN core and why?
>
>
>
> I'll go first: we should strip out the tracking middleware and also remove
> ckanext-stats. No more tracking or stats in CKAN core.
>
> Why? Analytics doesn't belong in CKAN core and if someone needs analytics,
> they can get better results from a dedicated analytics application or
> service.
>
> Right now, if someone asks "Does CKAN show usage stats?" the answer is
> "Well, sort of, but only in certain cases, and not for API or M2M usage, and
> there's no dashboard...well, there is a stats dashboard, but it's actually
> something different, and...well, maybe you should use the Google analytics
> extension...but that doesn't do everything you want either, so..."
>
> The answer to that question should just be "No." And that's not to say users
> shouldn't have access to analytics. They should just get them from some
> other plugin or service that really specializes in analytics. They'll get
> better results, developers who do want to focus on CKAN-related analytics
> will have more motivation to make polished plugins, and we as a community
> will have more time to devote to features that only CKAN can provide.
>
> So what do you think CKAN core would be better off without?
>
> -Kevin
>
>
>
> On 23.07.2018 18:00, Kevin Brochet-Nguyen wrote:
>
> Hi fellow devs, contributors, and users,
>
> If you could remove any feature from CKAN core, what would you remove? What
> code just doesn't belong in CKAN core? What is CKAN trying to do that it
> just shouldn't do? ...and why?
>
> Why am I asking?
>
> We're transitioning to Python 3 and Flask. We're modernizing and
> refactoring. And maybe there are some things that we just shouldn't take
> with us when we cross the threshold.
>
> Maybe we can save ourselves a lot of work and at the same time build a
> better application if CKAN just does less.
>
> So what would you remove, if you could remove anything?
>
> Rules of the game:
> - You can remove any feature or functionality you want.
> - You can put the deleted feature or functionality into a separate plugin or
> service outside of CKAN core if you want to.
> - But you don't have to—you can also just remove it and let the community
> find a new (and probably better solution).
>
> - Kevin
>
> --
> Kevin Brochet-Nguyen
> CKAN developer @ Viderum GmbH
> NextGEOSS Data Hub developer
>
>
>
>
> _______________________________________________
>
> ckan-dev mailing list
>
> ckan-dev at lists.okfn.org
>
> https://lists.okfn.org/mailman/listinfo/ckan-dev
>
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> --
>
> Kevin Brochet-Nguyen
>
> Sofware Developer
>
>
>
> Viderum – ein Unternehmen der Open Knowledge Foundation
>
> http://www.viderum.de/  |  @videruminc
>
>
>
> Viderum GmbH
>
> Marchlewskistr. 101
>
> 10243 Berlin
>
>
>
> Eingetragen beim Amtsgericht Charlottenburg, Registernummer HRB 176512 B
>
> Geschäftsführer: Sebastian Moleski
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>