[openspending-dev] Modularisation

Sun Mar 10 19:47:47 UTC 2013

On Sun, Mar 10, 2013 at 12:11 PM, Friedrich Lindenberg
<friedrich.lindenberg at okfn.org> wrote:
>> Of course this is complete speculation. This is not even in the plans
>> and I'm not proposing it, but you prove my point. If swapping out one
>> module for another requires refactoring and re-designing the rest of the
>> software there is something illogical in the design that needs to be
>> fixed. That's exactly why I feel modularisation is good.
>
> What is the inherent value of "swapability"? My point is that OS right now

The value of specialisation, and the interchangeability of parts, has
been understood since the writing of Adam Smith in the 18th century.

> has a number of integrated components (a data modelling interface that
> generates models in a specific format, a data validation service that
> evaluates these models, a query service that builds aggregations based on
> this model). Swapping out one of these would necessitate adapting the other
> parts.

This is somewhat begging the question, though, isn't it?

A modular system is one in which you don't have to change one
component in order to change another. It's common ground that
OpenSpending does not have this property.

The observation that "swapping out one of [the components] would
necessitate adapting the other parts" is a statement that OpenSpending
is not modular, rather than an argument that it should not be made
more modular.

It's not clear to me that the components you refer to in the paragraph
above should be disaggregated from each other. However, whether it
makes sense to keep them together does not determine questions such as
whether the OLAP backend should be modular.

> It's a bit like complaining that Wordpress is horrible because it can't
> render content from a Drupal database (and that we all need to define a
> common concept of content so that all CMS can access the same database -
> Java people actually did this in proposals like JSR 168 and 170, not with
> very much success. It's infrastructure for its own sake.)

This is an analogy, not a deductive argument.

> Let me be clear: I didn't say this is generic, it's just the thing that
> doesn't exist yet. I personally believe that's because in order to make it
> good to use for non BI-folks, you have to make it specific (model your data

What's the basis for this belief? (I'm not saying it's untrue, just
wondering where it comes from)

> ... for what?). If you want to simplify data loading for OpenSpending, add
> more guidance and specificity - don't rip it out and dump all CKAN use cases
> onto it just so it becomes completely impossible to keep simple.

This seems like a straw man - is anyone proposing to rip out and dump
all CKAN use cases onto OS? (Someone may be, of course, I just wasn't
aware; you'll recall I was bit proponent of the "optional" compromise
between the "mandatory" and "prohibited" approaches to the use of CKAN
for loading data into OS).

> Nah, most of it just doesn't generate value to users. This is not
> short-sightedness, its just keeping the eye on the ball.

Did changing from SQL to Mongo and back to SQL again generate value for users?

Sometimes you need to do things which are totally irrelevant to users.
Focus on user needs should be a high priority, not a rule which
excludes all other activity.

Mk