[open-science] IPCC report

Thu Oct 3 09:38:33 UTC 2013

Hi Daniel,

I agree validation is important - It's something we're missing at the
moment even in the main Open Access Button project.

With that in mind we could use the tool in a way which doesn't need
crowdcrafting, possibly like this:

1. A user reports a blocked *or open *paper with our tool. (All this
requires is change to our current form)
2. A server stores that info (I'd suggest storing this in a separate
database to our current one to avoid confusion)
3. People use the tool to go through the references (potentially on form
submit they state what reference they were hunting for). We'd then have a
event detailing the location of user, the url they were looking at, the
reference they were looking for and if it was open or not (+ anything else
asked in the form).
4. Our data set can be extracted and looked at fairly simply. You could
sort the data by reference and looking at agreement of users of whether
it's open or not.

At it's core the OAbutton can be used as a simple bookmarklet + form and
database feeding into a map. We've built a bunch of stuff around that but
it can be boiled down to that. Note though we don't do automatic detection
of if a paper is open or shut yet.

Thanks,

Joe

On 3 October 2013 10:03, Daniel Lombraña González <teleyinex at gmail.com>wrote:

> Hi Joe,
>
> I think the connections with OAButton and CrowdCrafting have a lot of
> possibilities. From my point of view, one that could be very useful for you
> is the validation of reported closed papers/journals:
>
> 1.- A user reports a blocked paper with your tool.
> 2.- Your server stores that info, and automatically creates a task in a
> CrowdCrafting app that you own.
> 3.- 30 (or more) people review that paper, so you can statistically
> analyzed the answers regarding if it is closed or not.
> 4.- You get all the answers for that specific paper, validate them, and
> publish them in your site as a story.
>
> In the case of Peter's idea:
>
> 1.- The 9K references are created as tasks in a CrowdCrafting app.
> 2.- People answer all the questions that he wants to address, plus one
> that is really relevant to your project.
> 3.- They gather all the answers, do the statistical analysis, and with the
> conclusions automatically push that data to your platform.
>
> If you want, they can include the button as part of the task. The idea is
> to report directly to your platform which papers are not accessible from
> CrowdCrafting.org. However, this may end up in generating too much noise
> for you, as if I'm not mistaken you would like to have "validated
> revisions". What do you think?
>
> From my experience, crowdsourced projects should include a validation
> step, otherwise the conclusions will not be really "useful". In other
> words, I think that you would like to say in your stories or results page,
> that a given paper was reported as closed and that N different people
> agreed that it is closed for public view.
>
> These are just random thoughts, I'm just brainstorming :-) Comments are
> welcomed!
>
> Cheers,
>
> Daniel
>
>
> On Thu, Oct 3, 2013 at 10:37 AM, Joseph Mcarthur <
> joseph.mcarthur.10 at gmail.com> wrote:
>
>> Hello,
>>
>> What an amazing email to receive. Climate and health are close to my
>> heart so would be amazing to see the OAbutton tool used for this.
>>
>> I think in it's current format the tool may not be optimal for you
>> unfortunately. BUT! Of course all our code is online, open, reuseable ext.
>> I'd suggest what you could do is fork our repo and make a few minor changes
>> (I think our basic infastructure could be very useful for you) to make it
>> more optimal for you. One of our initial goals was to encourage and support
>> people re-using the OAbutton tool for their own purposes, so keen to
>> support.
>>
>> How long are you wanting this to take? We're under a lot of pressure
>> already trying to get this main OAbutton tool out in time for Open Access
>> week (doesn't look like we'll manage at this point - so please don't chat
>> to much about this). I know one of our best developers, Andy (cc'ed) is
>> very into climate and health stuff though and may be up for helping do the
>> changes.
>>
>> Thanks all,
>>
>> Joe
>>
>>
>> On 3 October 2013 09:24, Peter Murray-Rust <pm286 at cam.ac.uk> wrote:
>>
>>> I know them well. They ran a hackathon last month with pizza etc. I've
>>> blogged them. I am kicking myself I didn't make the connection! But hey!
>>> that's why we have a collective approach to projects. I am copying them in.
>>>
>>> And they were at C4CC hack / maker on Tuesday.
>>>
>>> It's a wonderful synergy. Climate and Health are probably the two areas
>>> that Citizens care most about. They have different public appeals. The
>>> thing about Climate is it has a simple closure - we have 10000 references
>>> and that's the immediate goal. So yes, getting the OA button technology
>>> into analysing the climate references would be sensational!
>>>
>>> Meanwhile I am looking to see how to extract the references
>>> automatically. They won't be trivial to parse as there are references to a
>>> number of reports and chapters. But we should be able to poke something
>>> into Google scholar or Microsoft academic.
>>>
>>> P.
>>>
>>>
>>>
>>>
>>> On Thu, Oct 3, 2013 at 9:12 AM, Daniel Lombraña González <
>>> teleyinex at gmail.com> wrote:
>>>
>>>> Hi Cameron,
>>>>
>>>> I think I talked to them in the past OKCon13 actually :-) In any case,
>>>> you are right! Your idea is really good! Interaction between both projects
>>>> should be something to discuss by all means. Peter, do you know this group?
>>>>
>>>> Cheers,
>>>>
>>>> Daniel
>>>>
>>>>
>>>> On Thu, Oct 3, 2013 at 10:07 AM, Cameron Neylon <cn at cameronneylon.net>wrote:
>>>>
>>>>> One thing that you may want to consider is working with the OA Button
>>>>> group. Once you've got a list of articles and a way of distributing them
>>>>>  you could interact with the infrastructure they are building to make it
>>>>> easy to record failed access requests.
>>>>>
>>>>> https://oabutton.wordpress.com/
>>>>> https://github.com/oabutton
>>>>>
>>>>> Cheers
>>>>>
>>>>> Cameron
>>>>>
>>>>> On 3 Oct 2013, at 08:24, Daniel Lombraña González <teleyinex at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Marc,
>>>>>
>>>>>
>>>>> On Wed, Oct 2, 2013 at 5:09 PM, Couture Marc <marc.couture at teluq.ca>wrote:
>>>>>
>>>>>>  Daniel suggests the following for the crowdcrafting application:
>>>>>>
>>>>>>
>>>>>> >
>>>>>>
>>>>>> > * Each citizen is given a reference. and asked "can you read this"
>>>>>>
>>>>>> >
>>>>>>
>>>>>> > […]
>>>>>>
>>>>>> >
>>>>>>
>>>>>> > * they click questions such as "is this paper on a public site?"
>>>>>> "is it the publisher
>>>>>>
>>>>>> > site?" "can you access the full-text?" "if not, how much does it
>>>>>> cost?" "please save
>>>>>>
>>>>>> > the URL"
>>>>>>
>>>>>> >
>>>>>>
>>>>>
>>>>> Actually that was suggested by Peter :-) And I like his proposal.
>>>>>
>>>>>>
>>>>>>  I don’t know if this forum is the right place to discuss these
>>>>>> details; maybe one should put up an EtherPad or Google Doc  to that effet.
>>>>>> Anyway, I’ll go ahead.
>>>>>>
>>>>> Well,  I think that as the first e-mail included this list, it is fair
>>>>> to continue the discussion, unless someone says we should move to another
>>>>> place :-)
>>>>>
>>>>>>
>>>>>>
>>>>>> I don’t have any concrete experience with crowdcrafting, only a
>>>>>> knowledge derived from reading about some projects, mainly Galaxy Zoo, and
>>>>>> I had a look at Ross Mounce’s app for OA journals copyright policies (a
>>>>>> subject I know very well).
>>>>>>
>>>>>>
>>>>>> All I can say is that different people will come up with different
>>>>>> answers to these questions; some will find an OA version, others won’t. And
>>>>>> I know, because I do it on a regular basis for a journal, that OA versions
>>>>>> are sometimes quite hard to find.
>>>>>>
>>>>>
>>>>> IMHO this is also interesting :-) The perception of users regarding OA
>>>>> versions could be something really good to analyze too. It would be cool to
>>>>> show which references generated more disagreement regarding the chosen
>>>>> licenses. For example, for our ForestWatchers.net project, we analyze
>>>>> the disagreement for specific tasks<http://forestwatchers.net/pybossa/app/besttile/>(check the map, and select the Heat Map layer), and it reveals very useful
>>>>> information: the most cloudy areas, are the most difficult ones.
>>>>>
>>>>>>
>>>>>>
>>>>>> My point is that I don’t know what would be the quality of a survey
>>>>>> in which each reference is checked by only one person. In Galaxy Zoo, for
>>>>>> instance, they combine answers from many participants and empirically
>>>>>> determined thresholds (number of participants and % of agreement) for an
>>>>>> answer to be considered reliable (in their case, as reliable as what would
>>>>>> be obtained by a trained astrophysicist).
>>>>>>
>>>>>
>>>>> You have raised a very good point: using only one user is bad, really
>>>>> bad. For this reason, and by default, CrowdCrafting and any PyBossa server
>>>>> sends an application task to at least 30 different persons, so you can do
>>>>> the statistical analysis correctly.
>>>>>
>>>>> PyBossa allows you to change that value, to have a bigger sample, or a
>>>>> smaller one, but by default, if you do not touch anything you will get 30
>>>>> samples per task. In other words, each task is going to be reviewed by 30
>>>>> different people, so as you can see we are doing the same as Galaxy Zoo,
>>>>> and other citizen science projects.
>>>>>
>>>>> CrowdCrafting and PyBossa also supports "golden tasks" (in PyBossa
>>>>> terms I called them calibration tasks) which means that you can weight the
>>>>> answer of the users based on the answer given to the golden ones.
>>>>>
>>>>> Thus, the application is covered by the server unless the authors
>>>>> decide to change that value to something like one (i.e. for testing
>>>>> purposes). In summary, CrowdCrafting follows the same pattern as GalaxyZoo.
>>>>>
>>>>> I hope this is much clearer now :-)
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Daniel
>>>>>
>>>>> PS: PyBossa also supports task priorities, so you can change the
>>>>> priority of apps as you wish.
>>>>> PS2: PyBossa is the open-source software that powers CrowdCrafting.organd
>>>>> ForestWatchers.net
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Marc Couture
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *De :* open-science-bounces at lists.okfn.org [mailto:
>>>>>> open-science-bounces at lists.okfn.org] *De la part de* Daniel Lombraña
>>>>>> González
>>>>>> *Envoyé :* 2 octobre 2013 04:15
>>>>>> *À :* Peter Murray-Rust
>>>>>> *Cc :* open-science; okfn-discuss
>>>>>> *Objet :* Re: [open-science] IPCC report
>>>>>>
>>>>>>
>>>>>> Hi Peter,
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 2, 2013 at 9:57 AM, Peter Murray-Rust <pm286 at cam.ac.uk>
>>>>>> wrote:
>>>>>>
>>>>>> Great,
>>>>>>
>>>>>> I'll probably try to hack bits of the report today.
>>>>>>
>>>>>>
>>>>>> Nice!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> The way I see it is something like:
>>>>>>
>>>>>> * Crowdcrafting is given 9000 references:
>>>>>>
>>>>>> * Each citizen is given a reference. and asked "can you read this"
>>>>>>
>>>>>> * they are expected to paste the text into Google or some other
>>>>>> search engine (maybe Microsoft Academic Search)
>>>>>>
>>>>>>
>>>>>>
>>>>>> In the application I mentioned before, I basically avoid the
>>>>>> copy&paste action by giving a button that will directly do the search in
>>>>>> Google Scholar in a new tab. You can adapt it easily to any service :-)
>>>>>>
>>>>>>    * they click questions such as "is this paper on a public site?"
>>>>>> "is it the publisher site?" "can you access the full-text?" "if not, how
>>>>>> much does it cost?" "please save the URL"
>>>>>>
>>>>>>
>>>>>>
>>>>>> It makes a lot of sense :-) Once the user has all that info, they
>>>>>> will click a button with the Save URL or Send report, and a new task will
>>>>>> be loaded for them.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> and repeat.  It's up to our app to keep track of the results.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Yep! You will be able to get all the answers via JavaScript, or if
>>>>>> you prefer, download them, do the statistics first, and then generate the
>>>>>> output. This is up to you to decide how do you want to achieve it.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> There's slightly more cut and paste than normal, but many citizens
>>>>>> should have high motivation. My guess it will take about 0.5-3 mins. We may
>>>>>> need notes on how to navigate some journals. Their interfaces are awful.
>>>>>>
>>>>>>
>>>>>>
>>>>>> That's something you can address in the tutorial. Every
>>>>>> CrowdCrafting.org application can have one, so all you have to do is
>>>>>> to create one for yours :-)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> We also need a wiki/mail - e.g. how do we find the cost for Journal
>>>>>> X...
>>>>>>
>>>>>>
>>>>>>
>>>>>> That's something CrowdCrafting does not have, but there should be
>>>>>> free Wikis or Etherpads to coordinate yourself.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I think it could be exciting, rapid and very worthwhile.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Indeed! If promoted well, you will get a lot of people!
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Daniel
>>>>>>
>>>>>> _______________________________________________
>>>>>> open-science mailing list
>>>>>> open-science at lists.okfn.org
>>>>>> http://lists.okfn.org/mailman/listinfo/open-science
>>>>>> Unsubscribe: http://lists.okfn.org/mailman/options/open-science
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://daniellombrana.es
>>>>> http://citizencyberscience.net
>>>>> http://www.shuttleworthfoundation.org/fellows/daniel-lombrana/
>>>>>
>>>>> ··························································································································································
>>>>> Please do NOT use proprietary file formats to share files
>>>>> like DOC or XLS, instead use PDF, HTML, RTF, TXT, CSV or
>>>>> any other format that does not impose on the user the employment
>>>>> of any specific software to work with the information inside the files.
>>>>>
>>>>> ··························································································································································
>>>>> Por favor, NO utilice formatos de archivo propietarios para el
>>>>> intercambio de documentos, como DOC y XLS, sino PDF, HTML, RTF, TXT,
>>>>> CSV
>>>>> o cualquier otro que no obligue a utilizar un programa de un
>>>>> fabricante concreto para tratar la información contenida en él.
>>>>>
>>>>> ··························································································································································
>>>>>  _______________________________________________
>>>>> open-science mailing list
>>>>> open-science at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/open-science
>>>>> Unsubscribe: http://lists.okfn.org/mailman/options/open-science
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> http://daniellombrana.es
>>>> http://citizencyberscience.net
>>>> http://www.shuttleworthfoundation.org/fellows/daniel-lombrana/
>>>>
>>>> ··························································································································································
>>>> Please do NOT use proprietary file formats to share files
>>>> like DOC or XLS, instead use PDF, HTML, RTF, TXT, CSV or
>>>> any other format that does not impose on the user the employment
>>>> of any specific software to work with the information inside the files.
>>>>
>>>> ··························································································································································
>>>> Por favor, NO utilice formatos de archivo propietarios para el
>>>> intercambio de documentos, como DOC y XLS, sino PDF, HTML, RTF, TXT, CSV
>>>> o cualquier otro que no obligue a utilizar un programa de un
>>>> fabricante concreto para tratar la información contenida en él.
>>>>
>>>> ··························································································································································
>>>>
>>>
>>>
>>>
>>> --
>>> Peter Murray-Rust
>>> Reader in Molecular Informatics
>>> Unilever Centre, Dep. Of Chemistry
>>> University of Cambridge
>>> CB2 1EW, UK
>>> +44-1223-763069
>>>
>>
>>
>>
>> --
>> Joseph McArthur ¦ Open Access Button <http://oabutton.wordpress.com/>Project Lead ¦ Twitter:
>> @Mcarthur_Joe ¦ skype: joseph_mcarthur ¦ Medsin-UK is a registered
>> charity tackling global and local health inequalities. Registered charity
>> number: 1111824
>>
>
>
>
> --
> http://daniellombrana.es
> http://citizencyberscience.net
> http://www.shuttleworthfoundation.org/fellows/daniel-lombrana/
>
> ··························································································································································
> Please do NOT use proprietary file formats to share files
> like DOC or XLS, instead use PDF, HTML, RTF, TXT, CSV or
> any other format that does not impose on the user the employment
> of any specific software to work with the information inside the files.
>
> ··························································································································································
> Por favor, NO utilice formatos de archivo propietarios para el
> intercambio de documentos, como DOC y XLS, sino PDF, HTML, RTF, TXT, CSV
> o cualquier otro que no obligue a utilizar un programa de un
> fabricante concreto para tratar la información contenida en él.
>
> ··························································································································································
>

-- 
Joseph McArthur ¦ Open Access Button
<http://oabutton.wordpress.com/>Project Lead ¦ Twitter:
@Mcarthur_Joe ¦ skype: joseph_mcarthur ¦ Medsin-UK is a registered charity
tackling global and local health inequalities. Registered charity number:
1111824
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-science/attachments/20131003/51aae5f9/attachment-0001.html>