[openspending-dev] Modularisation

Sat Mar 9 22:30:55 UTC 2013

Hey Tryggvi,

On Sat, Mar 9, 2013 at 10:55 PM, Tryggvi Björgvinsson <
tryggvi.bjorgvinsson at okfn.org> wrote:

> Having a modularised system can actually help us focus on our actual
> problem (explaining government spending). Having a big software that
> tries to do "all the things!" can actually hurt the project. Even though
> we're good there might be others who are interested to add features but
> are turned away when they start messing with the code.
>

Well, I'm not entirely sure we're facing the "really big software" issue so
much: we're talking about something around 7000 lines of Python (including
tests). I've worked on individual Java classes that had half that line
count. Out of this, roughly 500 lines are actually the cubes code - most of
the effort goes into user guidance and data loading.

Bernhard and Hannes (along with their group) set up their own instance
> and have talked about specific Austria-based features. I would like to
> see support for these in OpenSpending but they might be turned away by
> the fact that the technical platform is too coupled with
> openspending.org (I guessing here - Bernhard and Hannes can correct me).
>

Sorry, but if you really want to start supporting deployments, you're in
deep, deep trouble. That would actually require releases and release
management. People can do it, but do you really want to make it your
responsibility to fix it? This will drain your time to work on stuff
immediately while not fixing any issue other than bad communication.

> Knowing where to add single-entry bookkeeping support without messing
> with a Solr instance on their dev computer can help OpenSpending
> development (faster and more agile).

Side-note: most German budgets are single-entry bk and last time I checked
we had about 50 of these loaded. There is very little that I can think of
that can be added to OpenSpending core to make support for these better.

> In their case they're going to pay
> for some development and could divert that dev attention to the
> "spending analytical machine (spam?)" and leave things like what does
> openspending.content_root in the .ini file do to the people who work on
> offenerhaushalt.at

Wut? Not sure I follow this?

The biggest benefit I see with having a modularised system is that it
> makes it easier for us to swap out the modules. What if the python cubes
> project is good enough for us? A modularised system would allow us to
> drop our homebuilt solution (which is probably better than cubes for our
> needs at the moment) and replace it with cubes. We would become
> contributors to cubes but active maintenance would be somewhere else
> (raising some other issues which are I think easier to deal with than
> maintaining things on our own). This would actually help us focus more
> on our mission and real problems (although it's always easy to hack
> together a working piece of software, maintaining it is difficult -
> especially when it's not the core focus of the project).
>

YAGNI until proven otherwise. This is complete speculation. Cubes has cool
stuff in it - no question - but the actual effort for using it would be in
re-architecting most of the rest of OS to use it: it doesn't have data
loading/table generation, input validation or some component for data
modelling in the browser.

Re-architecting your entire codebase based on the feeling that you may or
may not want to do something for one of any number of reasons at some point
later down the road is just a really bad idea.

So not a top priority and we would have to figure out how we want to
> modularise the code base (it may not be necessary to divide them into
> different projects although I feel that it would be the best).
>

Can you give a sketch of what modules you envision?

You haven't really convinced me that it's a bad idea. You've just
> convinced me that this may not be a short term goal (and I didn't really
> need to be convinced of that since I never thought it was a short-term
> goal).
>

What I really wanted to convince you of is that the environment you're
working in has a special dynamic with regards to infrastructure work. I'm
trying to say that I'm concerned about OS getting distracted and tech-happy
instead of serving its users.

> Just out of curiosity, what is this "satellite contributors problem" you
> mentioned (without wanting to divert this newly created thread)?
>

So we've gone through three partial rewrites of the platform over the last
few years, but in terms of user-facing outputs all it does is bubbles,
treemaps and tables (this is early 2009: http://bund.offenerhaushalt.de/,
we couldn't even do the bar charts out of the box now). Vitor is our hope
here (yay, linegraphs!).

Getting other people to actually do cool new vis types like maps, project
trackers, tell a story, run a statistical analysis on some data or to have
fun with the data we have in some other way is a much more interesting and
realistic than to hope that we will get someone to add roll-ups to our
backend for no particular reason.

Cheers,

 - Friedrich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20130309/3408588f/attachment.html>