[open-government] Open Data Tooling - should the OKF/others promote a set of tools for open data?

Mon Aug 22 10:57:07 UTC 2011

Hi Tim(s),

I think we're trying to answer two slightly separate questions in one
here: how do we (as a part of the open data developer community)
establish best practices amongst ourselves and how do we train
newcomers? Both are necessary, but slightly different strategies might
make sense in each case: there is certainly no reason for anyone to
proclaim a standard toolset for open data, but it might make sense to
set a common starting point in order to reduce sources of confusion
for beginners.

This is where I think Python etc. makes sense: if you don't even know
programming, its a nice place to start and if you do, it's close
enough to pseudocode to not matter too much. Last week I visited a
team of CS students who are about to build an open data web app in
Java - and looking at the state of Java tooling, they're in for some
pain.

2011/8/21 Tim Davies <tim at timdavies.org.uk>:
> Releasing any source code/spreadsheets/how to guides they've written for
> working with particular datasets, and making sure details of these are
> linked to when the dataset is published; ideally licensing such resources to
> allow for their re-use and modifications based on them to be shared freely.

This is an aspect where language matters. You don't just want to share
your data, you want to share the scraper, cleanup scripts etc. as well
- both for someone else to improve them and as a means of
documentation. Things like https://bitbucket.org/pudo/dpkg-eu-budget
are OK, but they only work for someone who is familiar with Python.
The other end of the spectrum is a Word document describing the data -
I just received one, which has been incredibly useful but it requires
a lot of time to search and "parse".

Is there a good way to (also) give a more general description of the
process in non-textual form that is sufficiently detailed (unlike
things like OPMV, which just chains black boxes) - or do we all have
to write this in JavaScript because everyone has it?

> Developing open routes for users of data to feed back potential
> modifications and updates;

This to me is the interesting one if we want to have true data bazaar.
We know how to manage and share code, we're beginning to see how to
manage and share data; what's missing is a good way to describe the
link between the two. This means cleanly describing inputs and outputs
of processing in a place that is easy to contribute to...
understandable both to machines and to humans. This is harder as it
may seem if its supposed to really enable a sharing process, I think.

- Friedrich

> Effectively advocating for open data as the starting point for "open all the
> way down", and from that, talking about linux / R / python etc. as possible,
> but not the only, tools for that.
> Tim
> On Sun, Aug 21, 2011 at 8:28 AM, Tim McNamara <tim.mcnamara at okfn.org> wrote:
>>
>> Alex Howard (O'Reilly Radar's government) has written a nice
>> feature[0] on Chicago city's moves towards open data. There's lots of
>> great content in the article. An excerpt I would like to draw people's
>> attention to is reproduced below:
>>
>> "To fully embrace this vision, however, Chicago is going to have to
>> build out its data capabilities to become a smarter city. "The first
>> step is moving over to a more open platform," said Goldstein. "You
>> don't have to make a multi-million-dollar investment to get a fancy
>> GUI and something meaningful. If you bring something over to Linux,
>> between Python and R you can produce some remarkable outcomes. These
>> are some really low-cost solutions.""
>>
>> One of the things he mentions on Twitter[1] is the interesting
>> coincidence of Python & R in Chicago's plans and a post I wrote in
>> early August about open data tooling[2]. It has made me wonder if
>> there should be calls for a standard set of tools for introducing
>> people to the open data. The OKF has been actively supporting
>> Python[3]. Is that the right approach?
>>
>>
>> Tim McNamara
>> Open Knowledge Foundation
>>
>>
>> [0]
>> http://radar.oreilly.com/2011/08/chicago-data-apps-open-government.html
>> [1] https://twitter.com/#!/digiphile/status/104973421519179776
>> [2] http://notebook.okfn.org/2011/08/01/open-data-tooling/
>> [3] http://datapatterns.org/
>>
>> _______________________________________________
>> open-government mailing list
>> open-government at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/open-government
>
>
>
> --
>
>
> +44 (0)7834 856 303
> @timdavies
> http://www.timdavies.org.uk
>
>
> _______________________________________________
> open-government mailing list
> open-government at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/open-government
>
>

-- 
Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/

http://twitter.com/pudo
http://pudo.org