[Open-education] OER Quality? transatlantic and sectoral differences (HE or K-12)

Fri Sep 25 17:10:36 UTC 2015

Sören and Alex have raised an interesting issue - but one with a sectoral and indeed a transatlantic divide.

The following is too rough and ready, and probably controversial, to be a blog posting, but it could become one if felt useful.  Some might feel it is the kind of email people write at close of business (UK time) on a Friday – which it is. 

Cutting a long story short, in Europe if the OER is used as part of a module within an accredited higher education programme, then it falls within the remit of the relevant national HE quality agency, most of whom are now members of ENQA - the European Association for Quality Assurance in Higher Education.

I, and I suspect many others, including in quality agencies, would say that it is reasonable to apply similar approaches to OER used within non-accredited programmes (like open MOOCs) – if only because one might want to derive an accredited course from it later, or to accredit (via prior learning) the learning outcomes of the course, at a later stage. But this is more arguable and not mandatory.

Again, cutting a long second story short, the ENQA Standards and Guidelines encourage national quality agencies to focus on the quality of the PROCESS that institutions use to ensure high quality courses, not on CONTENT  - thus in these courses quality of content is one aspect, but usually just a small one. 

For example, it can be a perfectly reasonable strategy in some courses (especially higher-level ones) to encourage students to explore a wide range of resources (open and otherwise), of widely varying quality and bias. In networking courses at MSc level we encourage students to explore standards sites and vendor and trade association sites, as well as the strengths and weaknesses of Wikipedia entries. 

A very few national quality agencies in Europe have guidelines for quality in online courses, most do not, and most do not want to. UK and Sweden used to but do not now. Ireland is generating them now, and some other Member States have them, but developed from some other national body than the quality agency.

ICDE recently produced a report "Global overview of quality in online and open education", written by a team including Ebba Ossiannilsson, who is well-known in EU OER circles, which gives a good treatment of the issues at an international level. See http://www.icde.org/Global+overview+of+quality+in+online+and+open+education.b7C_wRvG43.ips 

At a European level, EADTU, in the SEQUENT project, produced an overview report and guidelines on what they recommend should happen within Europe, in the context of the ENQA Standards and Guidelines mentioned above. Some of the most useful parts of that, speaking as one of the authors, are the descriptions of what specific countries and institutions do. A lot of useful reports and references are brought together on the page http://www.sequent-network.eu/instruments 

As described in the SEQUENT Showcases report - http://www.sequent-network.eu/images/Guidelines/Sequent_Showcases.pdf -  several European institutions have their own guidelines for quality of online courses (including those with open content), but typically content would not get much attention. Some aspects should, especially those which relate to legal compliance with accessibility legislation (for students with visual or auditory difficulties).

As one example out of many, Northumbria University in England has a comprehensive set of guidelines - https://www.northumbria.ac.uk/static/worddocuments/ardocs/773481.doc - of which just one section is on content. It says:
  “3.2 Learning materials
    a..  Appropriateness of structure – fit for purpose 
    b..  Relevance to module aims and learning outcomes – how are these flagged up within the learning materials, how are learning packages structured – relevance, appropriateness, signposting for access 
    c..  Appropriateness of materials to academic level – relationship to subject benchmarks, style, level of language, intellectual rigour 
    d..  Ease of navigation –avoidance of dead-ends, clear signposting, indication of where learner is an how to get back to previous areas, ease of finding materials, adaptive release 
    e..  Quality of materials – appropriateness to learner approach; copyright adherence 
    f..  Accessibility – range and variety of means of access. Adherence to Special Educational Needs and Disability Act (2001)”
Anyone who is keen on this area is encouraged to come to the SEQUENT project session on quality in online and open learning at the EADTU conference in Germany in late October - http://conference.eadtu.eu/ – when the details are finalised I will let people know.

The situation in the US is somewhat different, despite the regional accrediting bodies, but I am not an expert in these. In Canada, it is very different, since universities appear in reality to be self-accrediting. 

While there is an international association of quality assurance agencies (INQAAHE) and a considerable and increasing level of commonality (especially within Commonwealth countries) there are still many differences, even before anyone considers the OER issues.

In other education sectors (like K-12) the above discussion does not apply – things re quality are on the whole completely different, and in my view usually pretty disorganised and/or very traditional (e.g. ignoring OER and MOOCs altogether). Despite fears of regimentation, UNESCO reports make it clear that in most advanced countries (though not all) institutions and even individual teachers within them have considerable autonomy in what they use for teaching resources and how they use them, even if there is (or is supposed to be) a national curriculum. 

Paul

Paul Bacsich
Senior Consultant, SeroHE

-----Original Message----- 
From: Sören Auer 
Sent: Friday, September 25, 2015 5:09 PM 
To: open-education at lists.okfn.org 
Subject: Re: [Open-education] OER Quality? 

Thanks Alex for starting this discussion!

I think quality is really an issue. We did some empirical evaluation of
OpenCourseWare quality and found that it often suffers:

OpenCourseWare Observatory -- Does the Quality of OpenCourseWare Live up
to its Promise?
http://arxiv.org/abs/1410.5694

>From my perspective the best way to improve quality while at the same
time progressing iteratively is following an open collaboration approach
(similar to Wikis, Wikipedia, OpenStreetMaps or GitHub). We also started
to work a bit in this direction with http://SlideWiki.org and are
currently preparing a larger project on the topic.

Best,

Sören

-- 

Project: BigDataEurope-Empowering Communities with Big Data technologies
http://big-data-europe.eu

Enterprise Information Systems, Computer Science, University of Bonn
http://eis.iai.uni-bonn.de/SoerenAuer

Fraunhofer-Institute Intelligent Analysis & Information Systems (IAIS)
Organized Knowledge -- http://www.iais.fraunhofer.de/Auer.html

Skype: soerenauer, Mobile +4915784988949

http://linkedin.com/in/soerenauer
https://twitter.com/SoerenAuer

On 9/25/2015 4:57 PM, Alexandre Enkerli wrote:
> How much should we emphasise quality, when we frame OERs to others?
> 
> Asked Rory McGreal about this, during (and after) a CIDER session.
> https://landing.athabascau.ca/groups/profile/289790/canadian-initiative-for-distance-education-research-cider/tab/359765/sessions
> 
> Missed most of the webinar but it sounded like much emphasis was on the "competition" between commercial textbooks and OERs. Got an issue with this.
> 
> Granted, if we all want to shift away from the offerings of commercial publishers, we need to do better than them by producing (selecting, reusing…) textbooks of equal production value. Which is extremely expensive, cumbersome, and time-consuming. To caricature: if the goal is to replace the current structure, we need to have the exact same structure as theirs, only more "open". The OER magic.
> 
> Clearly, many people do want textbooks of very high "production value". In some cases, people think specifically about the same type of page layout and use of amazing images as what McGraw-Hill and others have pushed down our learners’ throats. It could because people were burnt by the movement for desktop Linux never achieving the same type of user experience as other operating systems. So we need to do better than the competition to convince people to switch.
> 
> But maybe this perceived need comes from a skewed idea about learning? After all, are we so sure that learning happens through textbooks themselves? We could open a whole discussion about the value of textbooks, here, which could be quite useful. The reason for this discussion's usefulness is the dominance of the textbook model, especially here in North America (and especially among English-speakers). As a personal perspective: textbooks can fit a specific purpose but they may be equivalent to "using a sledgehammer to crack a nut" (or "killing a fly with a bazooka").
> 
> We need not get caught up in that textbook-based model, though. What about "small" OERs? Or "quick and dirty" ones? UNESCO mentions that:
> "OERs range from textbooks to curricula, syllabi, lecture notes, assignments, tests, projects, audio, video and animation."
> http://www.unesco.org/new/en/communication-and-information/access-to-knowledge/open-educational-resources/what-are-open-educational-resources-oers/
> 
> Quite a difference between syllabi and textbooks!
> 
> The aforementioned Tony Bates textbook example notwithstanding, much OER can be about the "stuff" we already create in the course of our usual work. Including the material created by learners.
> 
> Been particularly fond of a saying from software development, especially fitting in agile programming: "Release Early, Release Often". If you want to create the perfect product, chances are that you will get bogged down in details. A lot of developers end up not releasing their wares because they feel they're "not ready for primetime". Financially responsible, perhaps, but sad nonetheless.
> 
> A former student taught me a valuable lesson, there. After completing a very high quality version of the main project for the course (with friends seeing it), this student postponed submitting the final version in order to tweak it further. Weeks after the class ended, had to fail this student because the work had yet to reach me. (We eventually worked it out but it was quite stressful.)
> 
> In terms of learning material, there might even be a deeper lesson. As we polish up our work as much as possible, we chip away at some of the learning opportunities. For instance, as we fine-tune course notes too far in advance, we miss the chance to respond to learners' actions. And when we produce an impressive video, implicitly asking learners to do the same, we intimidate many of them.
> 
> Not to mention that the "remix" part of Open Content is more likely to be perceived as defacement if it starts with finished material than when it comes from a simple sketch. Or that learners and teachers may collaborate in creating material which only becomes well-edited after many iterations and interactions.
> 
> Not to be too one-sided about this. There is a clear case for extremely high quality offerings, accessible at no cost. Like there is a need for museum displays. 
> 
> But how about Opening Educational Resources as a "work in progress"?
> 
> Thanks.
> 
> — Alex
> 
> --
> Alex Enkerli, Learning Technology Advisor
> Vitrine technologie-éducation http://www.vteducation.org/en
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> open-education mailing list
> open-education at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/open-education
> 

_______________________________________________
open-education mailing list
open-education at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/open-education
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/open-education/attachments/20150925/fd1f5f83/attachment-0003.html>