[open-science] Let us denonce the pseudo-open Public Library of Science

Tue Feb 14 15:26:55 UTC 2017

On 2017-02-14, at 9:58 AM, Paola Di Maio <paola.dimaio at gmail.com<mailto:paola.dimaio at gmail.com>> wrote:

Heather,
thanks -  I did not intend to start a challenge :-)
But let me reply quickly-

1. Does open data facilitate replicability? I argue that it does not. At most, open data permits repeat analysis of the same data. This is a good thing, but it is not replication. To replicate a study, one must repeat the study, sometimes with variations to eliminate limitations of prior studies, gather new data.

to replicate a study, one must repeate the study -
ssuming that by 'study' you mean the application of a methodology

 but to replicate the result of a study, one needs the  exact data that the study has used. what about if I get different results from the same study (method)?  what would that imply?

Let's take the equivalent of the salt water experiment and look at what we can do with closed and open data and replicability.

Closed data: I add salt to water, stir, taste it and conclude that it is salty. I pour out the water so no one else can confirm the taste.

Open data: I add salt to water, stir, taste it and conclude that it is salty. I invite anyone who is interested to come to my place and taste it too. This way others can confirm or not the salty taste (open data; re-analysis of the same data). This is a useful check on my assessment, if not particularly hygienic.

Replication: I add salt to water, stir, taste it and conclude that it is salty. I pour out the salty water. I start over with a new glass or cup, add salt and water, taste and conclude that it is salty. In case this is unique to my location, I go someplace else, add salt to water, taste and conclude that it is salty. I tell my colleague. My colleague, who lives in a different city, takes a glass, adds salt and water, tastes and concludes that it is salty. This is an experiment that has been replicated.

Key point: open data / re-analysis of the same data cannot take the place of replication in science.

2. Does replicability depend on open data? I argue that it does not. Studies have been replicated as long as science has existed. If I add salt to water, I get salt water. Anyone with salt and water can replicate this experiment. People figured this and many other things out a long time before open data became possible.

How much salt? what kind of water? (mineral analysis) under what conditions? (temperature, altitude, pressure, other conditions)
so grossly speaking yes, add salt to water and you get salt water, but subtly speaking, no, you need all the data of the experiment to replicate it exactly

I agree. I am deliberately simplifying the example to make the point clear.

 points 4. and 5, do not relate to anything I said  - not all science is replicable (nor falsifiable) yet some replicability may be essential to prove the scientific validity of some claims.

Can we agree that the argument of replicability does not apply to all types of research data?

Can all scientifically valid phenomena be replicated exactly?
Does all research produce scientifically valid findings?
I dont think so -

But these (interesting) points do not belong to this thread perhaps

(apologies for the typos btw-  crossing wires in the brain)

and my general apology for many typos in many messages  ~

h

:-0

p

4. Is replicability essential to all science? I argue that it is not. Some kinds of studies can be replicated, but not all. Many variables change over time. Salt plus water = salt water is an experiment that can be replicated. Measuring the temperature at one moment at one place in time can only be done in that moment, in that place in time. Estimating the temperature at that moment at that place in time can happen using different techniques, however this is triangulation, not replication.

4. Not all knowledge involves variables that are replicable. Consider opinion research. There is a reason why polls are run on a frequent basis; what people think about a politician today is often not what they thought about a politician yesterday. Some important knowledges are time and context-specific, and not at all replicable.

best,

Heather Morrison

On 2017-02-14, at 9:10 AM, Paola Di Maio <paola.dimaio at gmail.com<mailto:paola.dimaio at gmail.com>> wrote:

A lot of issues coming up in this thread!!

In my  first reply to you, I wanted to say that sometimes (as in the case of PLOS you mention) organisations deliberately keep their options open. to do or not to do what they say. which may or may not lead to the results we are trying to achieve.

True that innovations take time, good example with education, but also human rights, democracy  and a lot of other stuff that exists on paper  but in practice is actually not at all like on paper.

Unless open (research) data is accessible, its impossible to replicate or even verify/validate research findings, so the whole replicability of science becomes null. Maybe that's why Sidney Brenner, a nobel prize whom I interviewed both befor and after he got the prize says that 95% of all research is complete rubbish . (he said this as an opening comment of his inaugural speech at a systems biology lecture hall in 2010 in Edinburgh).  Without open research data, your hypothesis is as good as mine.

is ther a calatogue/analysis of what research data cannot be open?
I understand 'person data' but a given result can be replicated even if name, and other person data are deleted/not accessible.

The good news, as you say Heather, is that there is always more research to be done :-)  Or maybe that's the bad news.

PDM

I argue that it is not unusual for it to take some time to move from a good idea to full implementation, particularly at a global level.

It can be disheartening to realize that elementary education is not universal. However in teaching policy I have learned to emphasize just how much progress has been made since the 1948 UN Declaration on Human Rights, the first global commitment to universal education, a declaration that even today not all countries fully support.

Open access is more straightforward than open data because the focus is on a type of work that scholars were already publishing (making public). Research data has not traditionally been published. There are a great many issues to address along the road to open. There are different types of issues - policy (e.g. privacy, security, credit/attribution), technical, infrastructure including service, support, funding for service and support, and education. It is not realistic to expect a quick transition from commitment to practice.

To me this is good news for researchers; there will be plenty to keep researchers busy for many years to come. We need to understand the barriers to open. I argue that one of the barriers that applies to the open data movement itself is wrapping our minds around the concept that not all data can be made open. In some cases, not now; in other cases, not ever. Opening up data for research will require a range of open, from open to researchers with special clearance only (e.g. data where there are security issues) to somewhat open data (e.g. health or education data that can be made available to groups of researchers where a common set of ethical standards can be assumed), to fully open.

Among policy makers, my sense is that there is a strong commitment to open. I suggest that the most effective way to move forward will be to focus on understanding and overcoming barriers to open. In most cases, these will be specific to different types of data. Policies and procedures that work well for my research group on the OA APC project involving collection of data already freely available on the web will not work for my colleagues who do research on records management working with organizations like the Department of Defence and the Bank of Canada. An audit approach either will not help, or will have the unfortunate side-effect of rewarding those who are doing work that can be made open and discouraging the kinds of research where data cannot be made open.

best,

Heather Morrison

On 2017-02-14, at 7:21 AM, Paola Di Maio <paola.dimaio at gmail.com<mailto:paola.dimaio at gmail.com>>
 wrote:

Heather
my conclusion were derived from researching to a specific research problem - the lack of open data in publicly funded research despite full adherence of the research councils.

So on the one hand, the research councils heralded full support to open data, but I when to count the actual open data sets associated to each research grant, they never heard of it.

The conclusions and recommendations however, seem to be universal, or at least, apply to wide range of situations

when people say 'we do this' then when you audit what they do, they ve got nothing to show. especially in social innovation. full of hypochrisy and contradictions.

we then have to dig further, what is causing this ubiquity?

lack of integrated system view (with my systemist hat on) and transparent accountable throughput function - I dont know how else to put it.

to answer your questions
yes, I think that only when an organisational processes are coherent from beginning to end, we can expect the desired system functionality - in this case accountability and transformation - (the opposite is true, dysfunctionality. the product of lack of coherence, actually can kill)

My guess (hypothesis?) is that this applies to PLOS as well as to the rest of the universe

PDM

<https://about.me/paoladimaio?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=chrome_ext>

[https://thumbs.about.me/thumbnail/users/p/a/o/paoladimaio_emailsig.jpg?_1466089258_72]

Paola Di Maio
about.me/paoladimaio

On Tue, Feb 14, 2017 at 5:27 PM, Heather Morrison <Heather.Morrison at uottawa.ca<mailto:Heather.Morrison at uottawa.ca>> wrote:
hi Paolo,

Thank you for this insight. It is possible that your analysis applies to PLOS. I do not know enough about PLOS to comment.

Here is how I read your argument: the remedy that you propose is change in organizational structure, to align policy and practice. Am I reading this correctly? If so, is this your remedy for PLOS or do you mean to argue for universal organizational change?

best,

Heather Morrison

-------- Original message --------
From: Paola Di Maio <paola.dimaio at gmail.com<mailto:paola.dimaio at gmail.com>>
Date: 2017-02-14 1:26 AM (GMT-05:00)
To: open-science at lists.okfn.org<mailto:open-science at lists.okfn.org>
Subject: [open-science] Fwd: Let us denonce the pseudo-open Public Library of Science

Hi Heather

I have researched this kind of paradoxes extensively, including in my PhD thesis (2012) [1]

I have concluded that what you and I perceive as 'hypocrisy' can be called 'systemic deviation'  and ' pragmatic gap', which I explain charachterised and defined in some of my talks.

Fundamentally, the problem can be broken down to a lack of integration and consistency between the policies and the practice

The solution I propose to tackle this kind of paradox is a clearer and stronger integration between value statements (policies) and technical implementations (how things are done in practice), At the moment policy and practice are handled as separate things by separate departments in most organisation, using different logic - as if the left hand does not know what the right hand is doing

Organisational processes are deliberately designed like that, so that they can be double facing. This has to change.

Maybe work to be done

PDM

[1] http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.597113

<https://about.me/paoladimaio?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=chrome_ext>

[https://thumbs.about.me/thumbnail/users/p/a/o/paoladimaio_emailsig.jpg?_1466089258_72]

Paola Di Maio
about.me/paoladimaio

On Sun, Feb 12, 2017 at 7:40 PM, Heather Morrison <Heather.Morrison at uottawa.ca<mailto:Heather.Morrison at uottawa.ca>> wrote:
For the sake of argument let us imagine that I am now convinced that we cannot tolerate any person or organization that is somewhat but not perfectly open.

I submit that from this perspective no one deserves to be denounced more than PLOS.

PLOS uses open licensing for their articles, but their software is proprietary and their terms of use make their highly protective approach to their trademark very clear.

PLOS' advocacy for extremes in openness is clearly hypocritical.

I denounce thee, PLOS, hypocritical, intolerant advocate of openness whilst actually a developer of proprietary software!

No doubt all the members of this list dedicated to denouncing the impure in open will reply to the list supplying this perspective?

best,

Heather Morrison
Pseudo radical open cult member

_______________________________________________
open-science mailing list
open-science at lists.okfn.org<mailto:open-science at lists.okfn.org>
https://lists.okfn.org/mailman/listinfo/open-science
Unsubscribe: https://lists.okfn.org/mailman/options/open-science

_______________________________________________
open-science mailing list
open-science at lists.okfn.org<mailto:open-science at lists.okfn.org>
https://lists.okfn.org/mailman/listinfo/open-science
Unsubscribe: https://lists.okfn.org/mailman/options/open-science

_______________________________________________
open-science mailing list
open-science at lists.okfn.org<mailto:open-science at lists.okfn.org>
https://lists.okfn.org/mailman/listinfo/open-science
Unsubscribe: https://lists.okfn.org/mailman/options/open-science

_______________________________________________
open-science mailing list
open-science at lists.okfn.org<mailto:open-science at lists.okfn.org>
https://lists.okfn.org/mailman/listinfo/open-science
Unsubscribe: https://lists.okfn.org/mailman/options/open-science

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-science/attachments/20170214/38ea4d60/attachment-0003.html>