[openspending-dev] OpenSpending visualizations in CKAN
Michael Bauer
michael.bauer at okfn.org
Tue Apr 29 05:09:54 UTC 2014
Friedrich,
I thought about doing aggregation client-side. That would require every
user to download the whole dataset though (which is unrealistic for some
datasets, e.g. the UK spending one).
So the only way would be to integrate some parts of OpenSpending (modelling
and aggregation API) into CKAN - sounds like a steep plan.
Michael
On Mon, Apr 28, 2014 at 08:19:33AM -0400, Friedrich Lindenberg wrote:
> Hey Michael,
>
> that’s kind of possible, but you somehow need to do aggregation. Since that’s the most computationally intense bit we do, I’m really not sure you want to do it client-side. So you’d need to remotely control OpenSpending to load the dataset, then use it’s API. In the process you’d probably end up re-implementing both the data modeller and the views designer in CKAN (which might be a nice reason to re-do them as JS-only apps, but now this is getting out of hand in terms of scope).
>
> In general, it seems to me like embedding the vis as a resource view iframe might be the best integration scenario for the moment :)
>
> - Friedrich
>
> On 27 Apr 2014, at 06:50, Michael Bauer <michael.bauer at okfn.org> wrote:
>
> > Hi there,
> >
> > Just to chip in to the discussion:
> >
> > If we would have modeling that happens client-side (e.g. tell us which
> > column is which) there could be an aggregator.js that works from the CKAN
> > API. This would still require the people creating OpenSpending
> > visualisations to know about the modeling step and do it.
> >
> > (There is no real way to do this automatically as I see now - unless
> > everyone adopts the same spending standard and we can autodetect this is
> > expenditure data e.g.)
> >
> > Michael
> >
> > On Fri, Apr 25, 2014 at 12:11:54PM -0300, Vitor Baptista wrote:
> >> Hi Stéphane,
> >>
> >> As you probably know, we've been working on building new "resource views"
> >> on CKAN. This wasn't merged yet on master, but you can start playing with
> >> it using the branch at https://github.com/ckan/ckan/pull/1251.
> >>
> >> To do what you want, you'd either have to develop an extension that creates
> >> a new API endpoint which mimicks OpenSpending's API, or change the
> >> OpenSpendingJS's Aggregation code to support CKAN API as well. Having that,
> >> you'll have to develop the actual treemap resource view, which shouldn't be
> >> too hard.
> >>
> >> In the mean time, you already can embed OpenSpending's visualizations on
> >> CKAN using the WebPage resource view (that uses an iframe).
> >>
> >> Cheers,
> >>
> >>
> >> 2014-04-25 11:39 GMT-03:00 Stéphane Guidoin <stephane at opennorth.ca>:
> >>
> >>> Thanks for the quick answer Tryggvi!
> >>>
> >>> I'll look at the technical detail a bit later, but to answer quickly to
> >>> the question of "why":
> >>>
> >>> The purpose is too allow a city with a open data portal to show
> >>> visualization of its budget data on its open data portal. Usually they want
> >>> to limit as much a possible to use multiple tools. Fact is that maintaining
> >>> correctly an open data portal is already a challenge because most cities
> >>> work on limited budget to do that. So the target is really that people from
> >>> the city uploads the dataset where they are used to manage their data,
> >>> choose the type of visualization they want in CKAN and boom, they have it.
> >>>
> >>> The idea is really to avoid playing with several tools and make it as
> >>> simple and short as possible.
> >>>
> >>> Does it make sense?
> >>>
> >>> Stéphane
> >>>
> >>>
> >>> On Fri, Apr 25, 2014 at 10:31 AM, Tryggvi Björgvinsson <
> >>> tryggvi.bjorgvinsson at okfn.org> wrote:
> >>>
> >>>> On fös 25.apr 2014 14:06, Stéphane Guidoin wrote:
> >>>>
> >>>> I would like to directly integrate a budget treemap based on a CKAN budget
> >>>> dataset _in CKAN_ (so without loading the dataset in openspending.org or an
> >>>> openspending instance).
> >>>>
> >>>>
> >>>> That's great stuff (although I don't understand why you'd want to skip
> >>>> the openspending loading step since that's what openspending.org is all
> >>>> about enabling). I'm really interested in hearing why :)
> >>>>
> >>>>
> >>>> I've had a look to the github account of openspendingjs and found some
> >>>> traces of work done to include openspending in CKAN, but nothing
> >>>> compelling. CKAN has started to implement custom visualization plugins so
> >>>> it should be possible to create a custom viz module.
> >>>>
> >>>>
> >>>> We have recently been moving away from the heavy integration of
> >>>> openspendingjs with openspending. The interesting code is in src/ where
> >>>> we've tried to make things configurable (so you don't have to interface
> >>>> with openspenidng.org but at the moment what you interface with must
> >>>> return same structure are openspending.org.
> >>>>
> >>>> If you like to change that to be more inclusive then that's great. Just
> >>>> submit a pull request so we can all benefit :-)
> >>>>
> >>>> With these changes our hope has been to enable users to create treemaps
> >>>> with a simple jQuery extension:
> >>>>
> >>>> $('#myawesomeidentifier').treemap({
> >>>> data: { url: 'https://openspending.org',
> >>>> dataset: 'thedatasetidentifier',
> >>>> ... }
> >>>> more configurations ...
> >>>> });
> >>>>
> >>>> (also via a shortcut method of automatically creating treemaps for every
> >>>> dom element with the class treemap (and configurable via the html
> >>>> data-attributes)).
> >>>>
> >>>> If you make changes please try to confine yourself to this setup so that
> >>>> openspendingjs can continue to be useful as a standalone library.
> >>>>
> >>>> Everything in the src/ folder is in the new versioned releases where this
> >>>> convention is used (vendor/ is external dependencies and almost everything
> >>>> else is stuff we're trying to move away from so sorry for an unclean repo
> >>>> but we don't want to break stuff that relies on openspendingjs as it was).
> >>>>
> >>>>
> >>>> My questions:
> >>>> - can openspending be used outside of openspending?
> >>>>
> >>>>
> >>>> Yes (I assume you're talking about openspendingjs). As I said, it's about
> >>>> configuring the data.url to something else but at the moment that is
> >>>> restricted to the openspending API (because that's what we know so feel
> >>>> free to add something that can use CKAN).
> >>>>
> >>>>
> >>>> - Is there some examples of such use of openspendingjs?
> >>>>
> >>>>
> >>>> Not that I know of so I refer this to others on the list.
> >>>>
> >>>>
> >>>> - I seached as much as I could but couldn't figure how and in which format
> >>>> openspendingjs reads the data... (I have to admit I understand JS but I'm
> >>>> not really proficient)
> >>>>
> >>>>
> >>>> The magic happens in src/utils/openspending.aggregator.js which
> >>>> interfaces with the openspending aggregation API. You can read more about
> >>>> that here:
> >>>> http://docs.openspending.org/en/latest/api.html
> >>>>
> >>>> Just let us know if you run into trouble. I think this is a great project
> >>>> to help us separate openspendingjs from openspending more -- which is what
> >>>> we want to do) and pleeease let us know when you have something cool to
> >>>> show :)
> >>>>
> >>>> --
> >>>>
> >>>> Tryggvi Björgvinsson
> >>>>
> >>>> Technical Lead, OpenSpending
> >>>>
> >>>> The Open Knowledge Foundation <http://okfn.org>
> >>>>
> >>>> *Empowering through Open Knowledge*
> >>>>
> >>>> http://okfn.org/ | @okfn <http://twitter.com/OKFN> | OKF on Facebook<https://facebook.com/OKFNetwork>|
> >>>> Blog <http://blog.okfn.org/> | Newsletter<http://okfn.org/about/newsletter>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Stéphane Guidoin
> >>> Director, Transportation
> >>> Open North
> >>> 514-862-0084
> >>> http://opennorth.ca
> >>> Twitter: @opennorth / @hoedic
> >>>
> >>> _______________________________________________
> >>> openspending-dev mailing list
> >>> openspending-dev at lists.okfn.org
> >>> https://lists.okfn.org/mailman/listinfo/openspending-dev
> >>> Unsubscribe: https://lists.okfn.org/mailman/options/openspending-dev
> >>>
> >>>
> >>
> >>
> >> --
> >>
> >> Vítor Baptista
> >>
> >> Developer | http://vitorbaptista.com |
> >> LinkedIn<http://www.linkedin.com/in/vitorbaptista>|
> >> @vitorbaptista <http://twitter.com/vitorbaptista>
> >>
> >> The Open Knowledge Foundation <http://okfn.org>
> >>
> >> *Empowering through Open Knowledge*
> >>
> >> http://okfn.org/ | @okfn <http://twitter.com/okfn> | OKF on
> >> Facebook<https://www.facebook.com/OKFNetwork> |
> >> Blog <http://blog.okfn.org/> | Newsletter<http://okfn.org/about/newsletter/>
> >
> >> _______________________________________________
> >> openspending-dev mailing list
> >> openspending-dev at lists.okfn.org
> >> https://lists.okfn.org/mailman/listinfo/openspending-dev
> >> Unsubscribe: https://lists.okfn.org/mailman/options/openspending-dev
> >
> >
> > --
> > Data Diva | skype: mihi_tr | @mihi_tr
> > Open Knowledge | School of Data
> > http://okfn.org | http://schoolofdata.org
> > GPG/PGP key: http://tentacleriot.eu/mihi.asc
> > _______________________________________________
> > openspending-dev mailing list
> > openspending-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/openspending-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/openspending-dev
>
--
Data Diva | skype: mihi_tr | @mihi_tr
Open Knowledge | School of Data
http://okfn.org | http://schoolofdata.org
GPG/PGP key: http://tentacleriot.eu/mihi.asc
More information about the openspending-dev
mailing list