[open-government] [open-development] Summary "Open Knowledge Index" Discussion

[Jose M. Alonso]

Dear Guo, all,

I have not followed the discussion closely but wanted to add my two cents.

I'm sure most of you are aware of the Open Data feasibility studies in Ghana and Chile by the Web Foundation and CTIC:
http://www.webfoundation.org/2011/05/open-government-data-reports-published/

You'll find there the methodology that was used and an extensive questionnaire developed to run the in-country interviews that gives some hints on what should be measured in terms of indicators for an index.

Before approaching the studies this way, we tried to build an experimental Open Data Index; we even had close to 200 indicators on various categories such as technical, socio-economic and others.

It was an interesting experiment but got push back from several members of our advisory group for very good reasons. For example, it's still too early to say what variables relate to Open Data, how and how they correlate (although we tried as a start), i.e. you will find lots of issues around 2) as you already mentioned but most could not be solved. The best you could do is to experiment (as we did).
Additionally, you would need a big sample if you want to test and index like this, not just two countries or a subset of EU countries. Not an statistician at all, but heard this from those with that expertise and I believe they're right.

The Web Foundation has certainly interest in this area. We are developing a Web Index, designed to measure growth and usage of the Web, and guide future investment in the Web. We might want to revisit the Open Data Index.

One of our goals of the Open Data Initiative I'm leading is to develop a Monitoring and Evaluation framework for Open Data programs that would allow those investing on them to measure the impact across dimensions; this would have to do with the Index.

Said all of that, I'm restricting my comments to Open Data, so can imagine that an Open Knowledge Index will be even much more difficult to develop.

Hope this helps.
Josema.


-- 

Jose M. Alonso
Program Manager, Open Data
World Wide Web Foundation
email: josema at webfoundation.org
twitter: @josemalonso
skype: josema.alonso
http://www.webfoundation.org



El 04/07/2011, a las 20:58, Guo Xu escribió:
> Dear list,
> 
> Wow! I did not expect to get such a great number of feedback - let me
> briefly summarize the discussion:
> 
> 1) There seems to be a consensus that a "Open Knowledge Index" would
> be a valuable asset in both for researchers and policymakers alike.
> - Open Government would be a part of the index, but we would like to
> capture civil society as well).
> 
> 2) Ideally, we would like to have an index for the country-level as
> well as the subnational (say, NUTS-1) level
> - The subnational index would obviously need to capture more "within"
> variation as many legislations are national and don't vary across
> administrative regions.
> 
> In terms of concrete steps, we would need to:
> 
> 1) Define the dimensions and variables we want to measure:
> - I have set up a pad where we people can contribute their ideas: We
> might want to take some data from the  epsiplatform-scoreboards for
> the prototype... feel free to add variables. Let's brainstorm and then
> think about how this works with theory
> http://okfnpad.org/openknowledgeindex
> - Collecting data from governments using surveys (as Lucas proposed)
> is a great idea to get comparable data. For the time being, however, I
> feel the realistic approach is to collect the data that is readily
> available on the web (for example, dummy whether country X has a
> "Freedom of Information Act" etc.)
> 
> 2) Define the weights we assign to each dimension and variable
> - This will invariably cause a lot of discussion - this is the
> standard problem in indices
> 
> 3) Calculate this for a range of countries. I would propose to do this
> for a fairly restricted sample of EU countries as a cross-section, say
> 2005 (so we have all the data available)
> 
> Also, I would like to invite you all for a sprint so we can overcome
> coordination problems and get something running. This would not be a
> code-sprint, so everyone can contribute, either conceptually defining
> the index or looking for the appropriate proxies.
> 
> Let me know what you think! And thanks for all the helpful answers and
> suggestions!
> 
> Guo
> 
> _______________________________________________
> open-development mailing list
> open-development at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/open-development