[Open-data-census] [okfn-discuss] A question about the Open Data census scoring system

Tue Oct 1 15:59:30 BST 2013

Thanks Pierre for your reply, and I agree with your points.

regarding the data quality issue, i think it is important to make it clear
that the current version of census does NOT include the data quality
assessment. But it should be taken into account in the future version. The
problem Daniela mentioned above also puzzles me for a while and perhaps
minimum requirements on dataset  ( For instance, we need bus station's geo
coordination as well as bus station's name in a bus route dataset) should
be specified in the future version. The question is shall we go to define
standard for each dataset or not? if not, how can we avoid too much
subjectivity (on data quality) in the census?

Feng

On Tue, Oct 1, 2013 at 10:46 PM, Pierre Chrzanowski <
pierre.chrzanowski at gmail.com> wrote:

> Thanks FENG,
>
> I think we should test different methodologies having in mind that the
> score calculation should :
>
> 1/ be very simple and easy to understand
>
> 2/ really value openess of a dataset as defined by the opendefinition
>
> 3/ clearly express what is assessed and what is not (for instance quality
> of the dataset is not)
>
>
> On Tue, Oct 1, 2013 at 4:12 PM, FENG GAO <gaofeng860918 at gmail.com> wrote:
>
>> Hi, Pierre:
>>
>> I agree that the current scoring system is problematic
>>
>> but I'm not sure whether 0 is appropriate for non-open dataset. Since a
>> country having the dataset publicly available (but not open) is different
>> from a country having no such data at all. The scoring system should be
>> able to differentiate them from each other.
>>
>> so perhaps only when the dataset does not exist then it is scored 0,
>> otherwise, it depends on whether it is publicly available and whether it is
>> open. The scoring system proposed by @stefenw (
>> https://github.com/okfn/opendatacensus/issues/33?source=cc) seems to be
>> reasonable.
>>
>> Feng
>>
>>
>> On Tue, Oct 1, 2013 at 8:39 PM, Pierre Chrzanowski <
>> pierre.chrzanowski at gmail.com> wrote:
>>
>>> Hi All, I already got some feedbacks about the new census results and
>>> most of the time people are quite surprised with the total score and
>>> position of France. I think the fact that France (whose most of datasets
>>> are openly licensed) and China (whose datasets are not open) share almost
>>> the same score is one of the main reasons.
>>>
>>> I already expressed my concerns to Christian that the scoring system
>>> could be misleading, or even irrelevant.
>>> And regarding the first feedbacks I got, I must say I am now quite sure
>>> it is misleading ;)
>>>
>>> Then, I would suggest to :
>>>
>>> 1/ remove the actual score which does not reflect openess of datasets in
>>> the countries
>>>
>>> and
>>>
>>> 2/ create a scoring system that really value openess (i.e score 0 if
>>> dataset not open)
>>>
>>> That being said, I am also wondering if it would not be better not have
>>> a score at all ...
>>>
>>> What do you think ?
>>>
>>> I found a discussion on the scoring calculation methodology and how it
>>> has evolved here
>>> https://github.com/okfn/opendatacensus/issues/33?source=cc
>>>
>>>
>>>
>>> On Fri, Aug 23, 2013 at 12:49 PM, Christian Villum <
>>> christian.villum at okfn.org> wrote:
>>>
>>>> Dear Tatyana,
>>>>
>>>> Thank you for offering to help, we will definitely be explaining the
>>>> methodology on the site as part of the coming updates.
>>>> We'll also look into the EXMO tool, thanks for the tip.
>>>>
>>>> Let's stay in touch!
>>>>
>>>> -Christian
>>>>
>>>> --
>>>>
>>>> Christian Villum
>>>>
>>>> Community Manager, Open Government Data + Local Groups Network
>>>> skype: christianvillum  |  @villum <http://www.twitter.com/villum>
>>>>
>>>> The Open Knowledge Foundation <http://okfn.org/>
>>>> *Empowering through Open Knowledge
>>>> *http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on
>>>> Facebook <https://www.facebook.com/OKFNetwork>  |  Blog<http://blog.okfn.org/> |
>>>> Newsletter <http://okfn.org/about/newsletter>
>>>>
>>>> *Have you registered for OKCon 2013 <http://okcon.org/>?*
>>>>
>>>>
>>>> On Wed, Aug 21, 2013 at 11:49 AM, Tatyana Tolsteneva <
>>>> tatyanatolsteneva at gmail.com> wrote:
>>>>
>>>>> Dear Rufus,
>>>>>
>>>>> thanks a lot for your respond.
>>>>> I would like suggest to join to the idea of developing and publishing
>>>>> full methodology of the Census.
>>>>> We in the Freedom of Information Foundation (Russia) have already done
>>>>> tis job for our monitoring (please see our methodology on our web-site in
>>>>> English http://www.svobodainfo.org/en/node/1014).
>>>>> We would be happy to provide translation support and translate the
>>>>> Census methodology in Russian.
>>>>>
>>>>> Also we would be pleasant to offer you our ICT monitoring tool - EXMO
>>>>> system, specially developed for government web-sites monitoring. Please
>>>>> find the article about EXMO work here (in English)
>>>>> http://www.svobodainfo.org/en/node/2162
>>>>> Just note, that unique quantity of EXMO is it's interaction part,
>>>>> which allows not only measure government openness level, but also improve
>>>>> it via internet communication with officials.
>>>>>
>>>>>   Yours sincerely,
>>>>>
>>>>> Tatyana Tolsteneva
>>>>> Development manager
>>>>> tatyana.tolsteneva at svobodainfo.org
>>>>>
>>>>> Freedom of Information Foundation
>>>>> (formerly known as Institute for Information Freedom Development)
>>>>> P.O. Box 527, St.-Petersburg, 192007, Russia
>>>>> Phone:  +7 812 766-03-66
>>>>> Fax: +7 812 766-52-61
>>>>> Email: info at svobodainfo.org <info at svobodainfo.org>
>>>>> www.svobodainfo.org
>>>>>
>>>>>
>>>>> 01.08.2013, в 14:06, Rufus Pollock написал(а):
>>>>>
>>>>> On 1 August 2013 10:24, Tatyana Tolsteneva <
>>>>> tatyanatolsteneva at gmail.com> wrote:
>>>>>
>>>>>> Dear Rufus,
>>>>>>
>>>>>> thanks a lot for your explanation, i have already found it here
>>>>>> https://github.com/okfn/opendatacensus/issues/33.
>>>>>> Please take my apologies for "OKNF", it was just a misprint.
>>>>>>
>>>>>
>>>>> No worries :-) (Just making sure not a confusion as opposed to typo!)
>>>>>
>>>>>
>>>>>> Here in Russia the Census it considered to be most authoritative
>>>>>> comparative study of open data around the world. And here we really feel
>>>>>> some lack of official information about the Census methodology.
>>>>>> It seems, this problem could be solved by official publication of
>>>>>> the methodology on the Census website.
>>>>>> Could it be possible? What do you think about the methodology
>>>>>> publication?
>>>>>>
>>>>>
>>>>> Yes this is a very good idea and we'll get moving on this right away.
>>>>> Any suggestions for specific things to include or language are very welcome!
>>>>>
>>>>> Rufus
>>>>>
>>>>> Yours sincerely,
>>>>>>
>>>>>> Tatyana Tolsteneva
>>>>>> Development manager
>>>>>> tatyana.tolsteneva at svobodainfo.org
>>>>>>
>>>>>> Freedom of Information Foundation
>>>>>> (formerly known as Institute for Information Freedom Development)
>>>>>> P.O. Box 527, St.-Petersburg, 192007, Russia
>>>>>> Phone:  +7 812 766-03-66
>>>>>> Fax: +7 812 766-52-61
>>>>>> Email: info at svobodainfo.org <info at svobodainfo.org>
>>>>>> www.svobodainfo.org
>>>>>>
>>>>>> 31.07.2013, в 22:28, Rufus Pollock написал(а):
>>>>>>
>>>>>> [changing mailing list from okfn-discuss to open-data-census]
>>>>>>
>>>>>> On 23 July 2013 12:06, Tatyana Tolsteneva <
>>>>>> tatyanatolsteneva at gmail.com> wrote:
>>>>>>
>>>>>>> Dear colleagues,
>>>>>>> my name is Tatyana and i'm a development manager of the Freedom of
>>>>>>> Information Foundation - Russian FOI-advocasy NGO based in St. Petersburg.
>>>>>>> I'm interested in the OKNF Open Data Census scoring system because i
>>>>>>> can't catch it's idea of it http://census.okfn.org/G8/
>>>>>>>
>>>>>>
>>>>>> OKFN (Open Knowledge Foundation (Network))
>>>>>>
>>>>>>
>>>>>>>  At first i had a hypothesis, that each "yes" costs 1 poin and
>>>>>>> points just should to be summarized.
>>>>>>>
>>>>>>
>>>>>> That's almost the case but not quite: whether data is up to date does
>>>>>> not count towards the total score.
>>>>>>
>>>>>>
>>>>>>> This idea has been perfect, if maximum for all questions would have
>>>>>>> been 7.
>>>>>>> But we have 6 as maximum.
>>>>>>> So my question is about "data existence" column
>>>>>>>
>>>>>> IN case all  "yes" should cost 1 points, it should be 7 as maximum.
>>>>>>>
>>>>>>
>>>>>> See above.
>>>>>>
>>>>>>
>>>>>>> Form the table http://census.okfn.org/G8/ i can see that some
>>>>>>> times "yes" per "data existence" is taken into the account.
>>>>>>>
>>>>>>
>>>>>> It should be always taken into account.
>>>>>>
>>>>>>
>>>>>>>  Fo example,  Transport timetable is Russia, Government spending is
>>>>>>> France and in Italy.
>>>>>>> But in the majority of cases YES per "data existence" is not
>>>>>>> included into the account. For example Zip-codes Russia
>>>>>>>
>>>>>>
>>>>>> It is included there - it's up to date that is not included.
>>>>>>
>>>>>>
>>>>>>> And speaking about Russian case, company register for Russia  has
>>>>>>> 2/6 and Transport timetable has 2/6, but Company register is up to date
>>>>>>> and Transport timetable is not.
>>>>>>>
>>>>>>
>>>>>> See above.
>>>>>>
>>>>>>
>>>>>>> Would you be so kind to explain this scoring and situation with
>>>>>>> points for "data existence" and i woul be specially glad for explaining in
>>>>>>> the context of the Russian score.
>>>>>>>
>>>>>>
>>>>>> Hope the above clarifies things. Also thanks for asking as your
>>>>>> questions will encourage us to put up further documentation on the website!
>>>>>>
>>>>>> Rufus
>>>>>>  *
>>>>>>
>>>>>>
>>>>>> *
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *
>>>>> Rufus Pollock
>>>>> Founder and Executive Director | skype: rufuspollock | @rufuspollock<https://twitter.com/rufuspollock>
>>>>> The Open Knowledge Foundation <http://okfn.org/>
>>>>> Empowering through Open Knowledge
>>>>> http://okfn.org/ | @okfn <http://twitter.com/OKFN> | OKF on Facebook<https://www.facebook.com/OKFNetwork>|
>>>>> Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>
>>>>>
>>>>>
>>>>> *
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Open-data-census mailing list
>>>>> Open-data-census at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/open-data-census
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Open-data-census mailing list
>>>> Open-data-census at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/open-data-census
>>>>
>>>>
>>>
>>>
>>> --
>>> *Pierre Chrzanowski*
>>> *Open Knowledge Foundation France*
>>>
>>> Mail: pierre.chrzanowski at gmail.com
>>> Mobile: +33 (0)7 855 71 292 | Skype: pierre.chrzanowski | Twitter:
>>> @piezanowski <https://twitter.com/piezanowski>
>>>
>>> _______________________________________________
>>> Open-data-census mailing list
>>> Open-data-census at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/open-data-census
>>>
>>>
>>
>>
>> --
>> Feng GAO <http://about.me/feng_gao> | @okfncnfenggao<http://twitter.com/okfncnfenggao> (same
>> on Weibo)
>> Ambassador at China, Open Knowledge Foundation
>>
>> OKF China Local Group (开放知识基金会中国) <http://cn.okfn.org>
>> Weibo: @开放知识基金会中国 <http://weibo.com/okfncn> |Twitter: @okfncn<http://twitter.com/okfncn>
>> Join Discussion on the maillist<http://lists.okfn.org/mailman/listinfo/okfn-cn>or Google+
>> Community <https://plus.google.com/communities/107913594667636511717>
>> Join the Meetup Community<http://www.meetup.com/OpenKnowledgeFoundation/Shanghai-CN/>to follow events in Shanghai
>>
>>
>
>
> --
> *Pierre Chrzanowski*
> *Open Knowledge Foundation France*
>
> Mail: pierre.chrzanowski at gmail.com
> Mobile: +33 (0)7 855 71 292 | Skype: pierre.chrzanowski | Twitter:
> @piezanowski <https://twitter.com/piezanowski>
>

-- 
Feng GAO <http://about.me/feng_gao> |
@okfncnfenggao<http://twitter.com/okfncnfenggao> (same
on Weibo)
Ambassador at China, Open Knowledge Foundation

OKF China Local Group (开放知识基金会中国) <http://cn.okfn.org>
Weibo: @开放知识基金会中国 <http://weibo.com/okfncn> |Twitter:
@okfncn<http://twitter.com/okfncn>
Join Discussion on the
maillist<http://lists.okfn.org/mailman/listinfo/okfn-cn>or Google+
Community <https://plus.google.com/communities/107913594667636511717>
Join the Meetup
Community<http://www.meetup.com/OpenKnowledgeFoundation/Shanghai-CN/>to
follow events in Shanghai
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-data-census/attachments/20131001/b3c8be1e/attachment-0001.htm>