[openspending-dev] OpenSpending's future architecture

Tue Feb 24 07:34:49 UTC 2015

Just chiming in here to say a big +1 on this decision and on the clear path
going forward.

@Tryggvi: this is an excellent summary and I think it would be worth
posting on the OpenSpending blog and cross-posting to the main list.

Rufus

On 19 February 2015 at 21:53, Tryggvi Björgvinsson <
tryggvi.bjorgvinsson at okfn.org> wrote:

> Hi all,
>
> I wanted to give all of you update on list about where we're at with
> OpenSpending. As some of you might know, others might have deduced and
> some maybe not known about, we've been going through a trial period
> since the beginning of the year, to try to understand better what
> architecture we'd go for with OpenSpending in the future.
>
> There are two approaches we've wanted to choose between:
>
> * Continue with the monolithic code base we have but port it from the
> obsolete pylons to flask (something Friedrich has done to a large extent
> -- thanks Friedrich!)
> * Go for the micro-services based approach proposed in the OpenSpending
> Enhancement Proposal #1:
>
> https://github.com/openspending/osep/blob/gh-pages/01-approach-and-architecture-of-openspending.md
>
> Ultimately this boils down to preference. With micro-services we
> increase deployment complexity at the cost of more flexibility. It is
> hard for me to make an unbiased decision as I was involved in the
> micro-services approach proposal (OSEP#1) so it should not come as a
> surprise that I am going to propose that we *accept* OSEP#1.
>
> This does not mean the flask port gets thrown out the window and never
> looked at again. It means that it won't be *the* service. There are
> definitely things in it we can use so we might strip out a few things
> (This is one of the reasons I settled for flask as the preferred web
> framework for OpenSpending services - as discussed at the February dev
> meeting)
>
> I came to this conclusion by looking at both users and developers but I
> do admit (like I said before) that this might be a biased view.
>
> # Users
>
> Users of OpenSpending or potential users want to be able to do different
> things. Some would like to be able to compare different datasets, some
> would just like to provide datasets, some want to visualise datasets on
> their site without having to rely on an external service, some want to
> be able to find source data.
>
> People want to do different things and they come to OpenSpending for
> different reasons. Currently OpenSpending is kind of designed to be an
> analysis service. You model a dataset, based on a source file, you load
> it, you extract the data you want to make pretty visualisations. Nobody
> other than you is likely to use your dataset.
>
> We could of course modify the existing code base to better serve the
> users' needs and provide different contact points to the process but I
> think the micro-service approach, which inevitably has to be built
> around conformed or standardised data passing between the services,
> allows for better flexibility and gives people a better chance to hook
> into the process based on their needs (key thing here being standardised
> or conformed datasets).
>
> # Developers
>
> Now with developers I am mostly thinking about the OpenSpending
> "services" developers. I see some users as being developers (those who
> want to re-use some part or all of the OpenSpending services to fulfill
> their needs) but I'm not focussing on those. However I interpret the
> OpenSpending "services" developers more broadly than I do for the
> current OpenSpending site.
>
> There are a lot of very interesting projects/services that we could
> easily re-use, build around or incorporate without having to merge them
> into our code base, or others could re-use our services:
>
> * Just today Paul Walsh announced a tabular validator project on the
> okfn-labs mailing list (disclaimer: I'm involved in that project).
> * Friedrich Lindenberg has a really great idea to have a central civic
> app login system (something he calls CitizenID) and there are other such
> services (more general) like lastuser by hasgeek.
> * Nathan Hilbert has been using I believe some parts of OpenSpending to
> create a data management platform in a project he's working on.
>
> These can be a part of OpenSpending but can also stand as services on
> their own and developers can work on them without having to understand
> or worry about the whole OpenSpending project (like how to do a data
> warehouse or how to provide search).
>
> So I believe that with this, we can be more flexible in what we use and
> do to create a central repository for conformed budget data and provide
> different services around it, based on users' needs. Just like the
> single statement summary says:
>
> > We want to centralize data but decentralize "presentation" ("views")
>
> It's not something we do tomorrow. It will take us a long time to get
> there but this is what I believe will make OpenSpending more valuable to
> users (and developers).
>
> It's been a really difficult decision to make but I wanted to share this
> with you in advance and give you a chance to comment before I
> "officially" accept OSEP#1 this Saturday.
>
> /Tryggvi
>
> _______________________________________________
> openspending-dev mailing list
> openspending-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/openspending-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/openspending-dev
>

-- 

*Rufus PollockFounder and President | skype: rufuspollock | @rufuspollock
<https://twitter.com/rufuspollock>Open Knowledge <http://okfn.org/> - see
how data can change the world**http://okfn.org/ <http://okfn.org/> | @okfn
<http://twitter.com/OKFN> | Open Knowledge on Facebook
<https://www.facebook.com/OKFNetwork> |  Blog <http://blog.okfn.org/>*

The Open Knowledge Foundation is a not-for-profit organisation.  It is
incorporated in England & Wales as a company limited by guarantee, with
company number 05133759.  VAT Registration № GB 984404989. Registered
office address: Open Knowledge Foundation, St John’s Innovation Centre,
Cowley Road, Cambridge, CB4 0WS, UK.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20150224/d079595c/attachment-0002.html>