[euopendata] A critical look at the Open Data Index methodology

Tue Nov 13 19:59:09 UTC 2012

Important!

Thank you for noting, and for sharing.
This should be a good opportunity to refine the methodology.
Is it open? Can it be massaged into something more precise?

One way to build a sample of 'real data'  (even a small onem maybe testing
other countries too, surely if Spain is swayed ,others could be too) and
see what figures comes up
when analyzed, then challenging the outcome from there, and see how the
methodology can be fixed. Take the exercise as 'evaluation' and feed it
back into the work.

I take a quick look here
http://thewebindex.org/documents/2012-webindex-dataset#Q22
and have some doubts about the indicators and the datasources, possibly not
sufficiently representative or other possible bias

PDM

On Tue, Nov 13, 2012 at 7:01 PM, David Cabo <david.cabo at gmail.com> wrote:

>   Hi,
>
>  When the Web Foundation's Open Data Index was published last September a
> number of Open Data activists here in Spain thought that the good score
> given to Spain was unjustified, as everyone aware of the situation on the
> ground knows there's no consistent national policy on the matter, and that
> the critical data sets (health, education, crime, spending…) are published
> in non-reusable formats, or not at all. We've written an open letter [1] -
> reproduced below - where we elaborate our criticism of the Index
> methodology in general, and Spain's score in particular.
>
>  Regards,
>
> /david
>
> PS: The letter is available online at:
> http://www.access-info.org/es/open-government-data/302-spain-is-a-world-leader-in-open-data-says-who
>
> ----
>
> Spain is a world leader in Open Data. Says who?
>
> In September 2012 the Web Foundation published the first edition of its
> Open Data Index, “a specific set of 14 indicators directly targeted at
> measuring open data worldwide”. Many open data and transparency activists
> in Spain were surprised to find Spain in the leading pack, since Spain
> still doesn't have an access to information law and there is no coherent
> national Open Data policy or practice. The only actively maintained Open
> Data initiatives are those started by a few local and regional governments,
> with no coordination or support from the national level. More importantly,
> key datasets about health, education, public procurement or official
> agendas are still being withheld by the administration,with no plan to
> release them.
>
> When asked for an explanation and rationale for these results, the Web
> Foundation responded that these results are "based on perception". In
> particular, in the perception of one person in Spain, who asked not to be
> identified, and about whom we know nothing.
>
> Also, we've been told the questions measure “availability”, not
> “openness”. If so, the name of the Open Data Index is seriously misleading.
> When the Spanish government was asked in March 2012 by member of parliament
> Alberto Garzon to release the national budget in machine-readable format,
> the official government response said "transparency is about the extent of
> the information provided, not about formal aspects of presentation" [1]. In
> spite of this, and of the fact budget execution (spending) data has little
> detail, procurement data is fragmented across many sources - most often in
> non-reusable formats -, and of the fact citizens have no access whatsoever
> to actual invoices, Spain gets a score of 8/10 on spending data [2].
>
> The question about crime data [3] is also particularly interesting. The
> Spanish government promised in its Open Government Partnership action plan
> to release the data in April 2012 [4], but once published it fell short
> compared with the detailed information available in other countries: only a
> summary of provincial level crime figures is made available once per
> quarter, in PDF [5]. According to the unidentified expert who contributed
> to the Open Data Index, Spain deserves a score of 10/10 for availability of
> crime data.
>
> Reviewing all the index scores falls beyond the scope of this open letter,
> but similar arguments could be made for the health or education datasets.
> Because of this, we call on the Web Foundation to:
>
>  * Review the Open Data Index score for Spain.
>
>  * Identify the expert who carried out the analysis for Spain.
>
>  * Revise its methodology to:
>    - Move away from perception-based scoring of one or two experts to
> results which are subject to fact-checking and empirical verification;
>    - Base the results on a full open data standards which includes that
> the data is made available in machine-readable, open formats, is
> comprehensive, raw data and is regularly updated in a timely manner;
>    - Provide links to all data sets and other sources used in the Index so
> that others can review and assess the scores.
>
> Signed:
>
> Victoria Anderica, Access Info Europe
> David Cabo, Civio Foundation
> Javier de la Cueva, Lawyer
> Helen Darbishire, Access Info Europe
> Jacobo Elosua, Civio Foundation
> José Luis Marin, EuroAlert
>
> [1]: “En todo caso, el concepto de transparencia se refiere a la
> suficiencia de información suministrada y no al mero aspecto formal de
> presentación de datos.” http://www.agarzon.net/?p=1758
> [2]: Question Q23b - “To what extent are there Government data on the Web
> in the following areas: detailed data on budgeted and actual spending on
> different departments”
> [3]: Question Q23j - “To what extent are there Government data on the Web
> in the following areas: data and statistics on crime”
> [4]: http://www.access-info.org/es/coalicion-pro-acceso/237-spain-in-ogp
> [5]: http://www.interior.gob.es/file/58/58149/58149.pdf
>
>
> _______________________________________________
> euopendata mailing list
> euopendata at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/euopendata
> Unsubscribe: http://lists.okfn.org/mailman/options/euopendata
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/euopendata/attachments/20121113/7af300b0/attachment-0001.html>