[open-government] Defining Open Government Data?

Wed Oct 20 07:55:03 UTC 2010

Hello all,

I'd just like to give my 2 cents to this thread which, although it is
extremely interesting, remains very UK and US-centered.

As Jonathan wrote, I work in "countries where open government data isn't on
the agenda at all". Having a definition would be great, but going into too
many specific details will scare off any enthusistic civil servant.

Considering the discussion I have with those, who usually want to do
something but have no idea how to proceed, I think the most important points
you guys addressed are license (as close to public domain as possible should
do it) and format. On this last point, I don't think that "machine-readable"
should be in the definition so much as "collaboration with requestors".

Very few public servants or decision-makers know which formats are
machine-readable and which are not. Now, if we - requestors - have access to
the list of formats the administration has, we can help them organize it in
a way that's machine readable.

To give you an example, the French law is made is such way that we cannot
request access to data in a certain format, even if we know they have it, if
the administration published it in the first place. That's how I spend days
scraping data from their SQL tables into my SQL tables :-)

Hope this helps!

Best wishes

nkb.
--
Datajournalist at OWNI.fr
Berlin: +49 174 472 2256
Paris: +336 50 57 53 80

2010/10/20 Tim Davies <tim at timdavies.org.uk>

> Hello all,
>
> This is a really useful discussion. Some thoughts below...
>
> *On the question of a definition*
> *I'm sceptical about the value of a solid-line definition which says some
> things are in - some things are out - when it comes to open government data.
> The Open Definition already provides a solid definition of a particular
> notion of openness - and at most a short FAQ on how this applies to
> government data should cover providing a sense of a gold-standard for
> something being 'formally' open.*
> *
> *
> *Models like the 'Five Stars of Open Linked Data' (
> http://inkdroid.org/journal/2010/06/04/the-5-stars-of-open-linked-data/)
> are far more useful in both helping people assess their current openness,
> and providing a motivational structure for making data available. *
> *
> *
> *An adapted version of the 5-stars, talking about licenses in place of
> linked-data etc., and adding a 'social openness' step at the end may be one
> route to a definition. *
> *
> *
> *A good definition for the end-user should be able to be re-formulated
> into a set of questions, such as:*
>
>    - *(License) *Is your data published under a license that allows it to
>    be re-used by anyone, or placed into the public domain so there are no
>    restrictions on re-use?
>
>    - *(Format) *Is your data accessible to humans and machines in a
>    structured way?
>    (As a good rule of thumb, if it's possible for a re-user to take a copy
>    of your data, load it into standard software, and edit that copy easily,
>    it's machine-readable).
>
>    - *(Social) *Have you worked to ensure that citizens and other
>    potential re-users of your data have access to the additional information,
>    tools and resources that they would need to make effective use of your data?
>
> *Social openness*
> The last point there is my attempt at some sort of social openness clause.
> Clearly it isn't unambiguous (what's 'effective', or enough effort in
> 'working to ensure'?) but it tries to capture what might be the steps
> governments (and wider communities are encouraged to take) to ensure the
> data is usable and used in practice.
>
> The practical openness of any dataset is not a property only of that
> dataset, but also of the tools-chains available; access to knowledge and
> skills; access to meta-data; etc. - and government clearly has a role to
> play in promoting access to and development of those resources - but the
> responsibility is shared with civil society / citizens / communities /
> business.
>
> The one point in here which might be slightly separate, around providing
> 'additional information' (in practice, meta-data and guides/handbooks
> etc.).
>
> Would a separate meta-data term of the 'definition' be useful?
> *
> *
> *Different sorts of openness: commercial and civic?*
> I'm sure it's a debate that's been over many times, and one it seems OKF
> have a fairly settled position on - but I do think it's worth the
> distinction between: 'civic openness' and 'commercial openness' being made -
> particularly for the broadest possible use of a definition.
>
> If a government does not wish to make data available for commercial re-use,
> but accepts free access to machine-readable data for citizens to use in
> non-commercial ways - that has significant potential benefits for democracy
> - and should be recognised as an open data policy; albeit only providing
> 'civic/democratic openness' and clearly shown to fail on 'commercial
> openness'.
>
> Of course - this moves from single unified definition more towards
> 'framework' - but, as above, my sense is that definitional frameworks,
> rather than exclusionary definitions, are a better route to go...
>
> *Other points*
> One other point that might have a place in a framework would be around
> 'Making Connections'. Perhaps *(connected) *is the top of a five-stars of
> open government data?
>
>
>    - *(Connecting Data to Information)* When you publish information
>    (charts / tables / reports) based on your data, do you provide a clear link
>    back to the original data, and any other information re-users would need to
>    understand how the information was generated?
>
>    (Drawing on
>    http://practicalparticipation.co.uk/odi/report/2010/2-3-data-and-information/
>    )
>    *
>    *
>    - *(Connecting Data)* Do you use linked-data approaches to make
>    connections between your data and other datasets.
>
>
> Hope these are useful inputs...
>
> All the best
>
> Tim
> -
>
> +44 (0)7834 856 303
> @timdavies
> http://www.timdavies.org.uk
>
>
>
> On Tue, Oct 19, 2010 at 9:32 PM, Jonathan Gray <jonathan.gray at okfn.org>wrote:
>
>> On Tue, Oct 19, 2010 at 6:49 PM, Ton Zijlstra <ton.zijlstra at gmail.com>
>> wrote:
>> > I agree with keeping things simple.
>> > However, a minimalistic way of adding some 'social open' notions could
>> be
>> > enough for now:
>> >
>> > findability (such as datasets described in a way that my average self
>> can
>> > find it, without learning Dept X particular lingo)
>> > such as having a contactperson and e-mail address mentioned with a
>> dataset
>> > e.g.,
>> > a way of giving feedback on data sets etc,
>> > showing contextual provenance other than 'Dept X published this' and
>> more
>> > along the lines: this was collected for task x by body y, and used in z
>> way,
>> > and things like when it will be next updated.
>>
>> Okay -- I agree that this is useful. Lets try and formulate it. Say a
>> local government body puts a spreadsheet (tick: machine readable,
>> technically open) online on their website at <some.gov.xa/data> under
>> CC0 (tick: public domain, legally open). It is *nearly* there -- but
>> how do we know whether the material they've uploaded is open
>> government data? Do they need to do one of a short list of things to
>> make sure its socially open? All of a short list of things? Is their
>> URL enough? Say they put a news item on their press section? Or Tweet
>> it? Do they need to have an event? Should they solicit for feedback?
>> Bear in mind we are focusing on open government *data* rather than
>> open *government*, per se. How can we capture the social openness in a
>> sentence or two, and ensure that it is clear enough that a non-expert
>> could apply the rule in a majority of cases (like a dataset being
>> machine readable or not, or a license being open or not).
>>
>> > none of those are tech-aspects or legal aspects, but important
>> nonetheless
>> > to render a data set useful.
>> > the whole 'stay in touch with all your stakeholders' 'community
>> building'
>> > 'being a platform for re-users' can be part of the natural growth path
>> on
>> > top of the minimalistic definitions.
>> >>Also are we saying that governments should do social stuff on PSB
>> >>websites
>> >
>> > My answer would be yes. It's called interacting with citizens, and a
>> primary
>> > ingredient of having a public sphere at all. I'd say 'doing social
>> stuff' is
>> > a core task of gov :)
>>
>> Yes indeed! Sorry should have said: is it essential for *open
>> government data* that government does social stuff. I.e. should PSIH's
>> be required to 'do social stuff' *in order* for their data to be
>> considered open? A very different question from should governments 'do
>> social stuff' full stop. ;-)
>>
>> > Also indications are pretty strong that it's the 'socially open' aspects
>> > that ultimately drive the adoption of re-use.
>>
>> I think it can really depend on the context. E.g. in the UK there was
>> a flourishing civic hacker community *before* the Cabinet Office
>> started funding hackdays, or, indeed, before it launched data.gov.uk.
>> Folks had to look a lot harder for the data in those days -- but the
>> absence of social openness wasn't necessarily the main blocker. I know
>> that in several countries where open government data isn't on the
>> agenda at all, there are communities who are keen to get hold of
>> certain datasets. I'd be interested in hearing more anecdotes about
>> this but I get the impression that in many cases prospective reusers
>> know what they are looking for, and the key thing is getting it under
>> an open license in a form which isn't unusable (e.g. PDF, weird legacy
>> database, ...).
>>
>> Of course if governments who don't have a flourishing (prospective)
>> re-user community already want to see results fast, they may do stuff
>> to catalyse uptake, or increase impact of opening up. The question is
>> do we want to *require* this in a definition of open government data?
>> Could this not be setting the bar quite high for, e.g. some countries
>> where governments may have very limited budget?
>>
>> > As well as it seems the way to
>> > take away unarticulated fears of data holders.
>> > These data sets become objects of sociality, creating and sustaining
>> > conversations with and around gov. To not make sure there's a conduit
>> for
>> > that interaction is setting it up to fail. As the example of opening
>> > landownership data in Bangladesh shows us.
>>
>> Indeed -- but my impression is that this is not necessarily something
>> that an email address, feedback form or data catalogue would fix. But
>> point taken. ;-)
>>
>> > All in all, I think 'social stuff' is key.
>> > It may very well be that part of the resulting interaction need not be
>> > connected to a singular dataset but rather to a corpus of datasets, such
>> as
>> > a data catalogue.
>> > Maybe my point is that if you posit this as a technology or legal driven
>> > thing only, gov's will miss why it's important and that will make the
>> open
>> > definition become self-defeating to a certain extent.
>>
>> *Absolutely* agree that social stuff is important. My question is
>> whether this should be dealt with in our minimalist bare-bones
>> definition, or in ancillary material. E.g. on opengovernmentdata.org,
>> the open data manual, etc. I feel I could be persuaded either way and
>> would love to hear what other folks think!
>>
>> As an analogy: what of the free/open source software approach
>> (collaborative development, methodology, etc) can you find in the
>> free/open source definition?
>>
>> All edits/comments most welcome! ;-)
>>
>>  http://opengovernmentdata.okfnpad.org/definition
>>
>> All the best,
>>
>> Jonathan
>>
>> > best,
>> > Ton
>> > On Tue, Oct 19, 2010 at 7:29 PM, Jonathan Gray <jonathan.gray at okfn.org>
>> > wrote:
>> >>
>> >> Yes agree this is very important, and we wrote about aspects of this
>> >> in several recent reports [1].
>> >>
>> >> However, I strongly feel that for present purposes the definition
>> >> should be (i) *very very* simple (as easy as possible to determine
>> >> compliance) and (ii) unambiguous to evaluate. How would one determine
>> >> if something is socially open? Would it be clear cut in every case?
>> >> Also thinking of free/open source software definitions do we perhaps
>> >> want to separate between subject matter (data) and surrounding
>> >> processes (how it is published, social openness) for purposes of a
>> >> definition, even though both are important?
>> >>
>> >> Also are we saying that governments should do social stuff on PSB
>> >> websites, or do also want to enable and encourage innovation from
>> >> outside government? A major point in Tom Steinberg/Ed Mayo's excellent
>> >> Power of Information report [2].
>> >>
>> >> Jonathan
>> >>
>> >> [1] cf. e.g. http://writetoreply.org/beyondaccess/4-1-discoverability/
>> >> and http://www.unlockingaid.info/3/
>> >> [2] http://www.opsi.gov.uk/advice/poi/power-of-information-review.pdf
>> >>
>> >> On Tue, Oct 19, 2010 at 6:16 PM, Ton Zijlstra <ton.zijlstra at gmail.com>
>> >> wrote:
>> >> > Hi Jonathan,
>> >> > Maybe we can add a component 'socially open' as well? Just this week
>> I
>> >> > saw
>> >> > the results of a study about municipal websites in the Netherlands,
>> that
>> >> > had
>> >> > as a result that while information and service were nominally
>> available
>> >> > as
>> >> > the law dictates, it was all very well hidden deep in websites to the
>> >> > point
>> >> > of uselessness. No 'social openness' in short, as in findable,
>> connected
>> >> > to
>> >> > contexts etc., and absence of dialogue with re-users, feedback
>> >> > possibilities
>> >> > for re-users towards PSB's etc.
>> >> > Those three components, legally open, technically open, socially open
>> >> > were
>> >> > also the components that floated to the foreground while we were
>> writing
>> >> > on
>> >> > the Open Data Manual in Berlin earlier this month.
>> >> > best,
>> >> > Ton
>> >> > -------------------------------------------
>> >> > Interdependent Thoughts
>> >> > Ton Zijlstra
>> >> >
>> >> > ton at tonzijlstra.eu
>> >> > +31-6-34489360
>> >> >
>> >> > http://zylstra.org/blog
>> >> > -------------------------------------------
>> >> >
>> >> >
>> >> > On Tue, Oct 19, 2010 at 7:00 PM, Jonathan Gray <
>> jonathan.gray at okfn.org>
>> >> > wrote:
>> >> >>
>> >> >> We'd like to start a process to encourage key stakeholders in the
>> >> >> (rapidly growing!) world of open government data to have some
>> >> >> consensus on what 'open government data' means. This would be a
>> 'bare
>> >> >> minimum' that would need to be complied with in order to be called
>> >> >> OGD, not a wish list in an ideal world in perfect conditions.
>> >> >>
>> >> >> We already have several sets of principles [1], but many of these
>> are
>> >> >> quite jurisdiction specific -- e.g. according to 8 principles the
>> >> >> Australian, New Zealand and UK governments don't have any open
>> >> >> government data as it isn't 'license free', and the UK principles
>> are
>> >> >> clearly only intended for the UK (and it would be good not to have a
>> >> >> different set of standards for each country!).
>> >> >>
>> >> >> We'd like something *really* simple that we can start to try to
>> build
>> >> >> consensus around. Hence I'd like to start discussion around a basic
>> >> >> definition/standard that we can all start to encourage the adoption
>> >> >> of, to distinguish open government data from e.g. a bunch of PDFs
>> >> >> published on a website with no information about reuse, or an API
>> with
>> >> >> restrictive terms of use.
>> >> >>
>> >> >> I envisage this as having two key components:
>> >> >>
>> >> >>  (i) legally open (as in opendefinition.org)
>> >> >>  (ii) technically open (i.e. machine readable, available to download
>> in
>> >> >> bulk)
>> >> >>
>> >> >> (i) would be to make sure that we don't start calling stuff 'open
>> >> >> government data' which:
>> >> >>
>> >> >>  * doesn't explicitly let the public reuse it for any purpose
>> >> >> (whether as a result of national copyright law, or departmental
>> >> >> policy)
>> >> >>  * doesn't permit derivative works
>> >> >>  * doesn't permit commercial reuse
>> >> >>
>> >> >> (ii) would be to make sure that material is not *only*:
>> >> >>
>> >> >>  * available via an API
>> >> >>  * available in non-machine readable formats, where machine readable
>> >> >> copies exist
>> >> >>
>> >> >> I've started a draft along these lines at:
>> >> >>
>> >> >>  http://opengovernmentdata.okfnpad.org/definition
>> >> >>
>> >> >> Any input/comments would be very much appreciated! We'd ideally like
>> >> >> something ready at or just before Open Government Data Camp in
>> London!
>> >> >>
>> >> >>  http://opengovernmentdata.org/camp2010/
>> >> >>
>> >> >> All the best,
>> >> >>
>> >> >> Jonathan
>> >> >>
>> >> >> [1]
>> >> >>
>> >> >>
>> http://sunlightfoundation.com/policy/documents/ten-open-data-principles/
>> >> >> http://resource.org/8_principles.html
>> >> >> http://razor.occams.info/pubdocs/opendataciviccapital.html
>> >> >>
>> >> >>
>> >> >>
>> http://blog.okfn.org/2010/06/28/new-uk-transparency-board-and-public-data-principles/
>> >> >>
>> >> >> --
>> >> >> Jonathan Gray
>> >> >>
>> >> >> Community Coordinator
>> >> >> The Open Knowledge Foundation
>> >> >> http://blog.okfn.org
>> >> >>
>> >> >> http://twitter.com/jwyg
>> >> >> http://identi.ca/jwyg
>> >> >>
>> >> >> _______________________________________________
>> >> >> open-government mailing list
>> >> >> open-government at lists.okfn.org
>> >> >> http://lists.okfn.org/mailman/listinfo/open-government
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Jonathan Gray
>> >>
>> >> Community Coordinator
>> >> The Open Knowledge Foundation
>> >> http://blog.okfn.org
>> >>
>> >> http://twitter.com/jwyg
>> >> http://identi.ca/jwyg
>> >
>> >
>>
>>
>>
>> --
>> Jonathan Gray
>>
>> Community Coordinator
>> The Open Knowledge Foundation
>> http://blog.okfn.org
>>
>> http://twitter.com/jwyg
>> http://identi.ca/jwyg
>>
>> _______________________________________________
>> open-government mailing list
>> open-government at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/open-government
>>
>
>
>
> _______________________________________________
> open-government mailing list
> open-government at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/open-government
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-government/attachments/20101020/6efdd446/attachment-0001.html>