The main reason for my visit to Uruguay was to attend the First International Workshop on New Metrics for Evaluation: Towards Innovation in Learning. This event was organised by the Centre for Research at the Ceibal Foundation in collaboration with INEEd, the ICT4V centre and the education division of the Inter-American Development Bank.
The workshop had four objectives, which the organisers framed as:
1. Using data for research and evaluation: towards an open and collaborative process for analysis, research and improving education.
2. Presenting experiences in the use of information systems for improving learning outcomes.
3. Presenting innovative approaches for evaluation and assessment of learning outcomes.
4. Policies, projects and programs for technology integration and data use in education.
It was a fascinating event, with representatives from countries across South and Central America, including speakers from Brazil, Chile, Colombia, Ecuador, Mexico, Nicaragua and Uruguay. Other speakers from outside the continent were Dragan Gasevic from Edinburgh, Neil Selwyn from Monash in Australia and Gilles Dowek from France.
I was particularly interested to find that Uruguay runs a ‘One Laptop per Child’ programme based on premises of equality and justice. Uruguay sees access to computers and the Internet as a right. You should have them in your classroom, just as you should have electricity in your classroom. Plan Ceibal has supplied 600,000 people (a fifth of the population) with laptops or tablets. Every child gets one when they start school, and they get a replacement every three years, with secondary school children now receiving Chromebooks. Internet is available nationwide – no one should be more than 400 metres from the Internet. There is a maintenance programme and a disposal programme, a teacher training programme, a learning management system, a suite of software, and a programme of video-conferenced English lessons, arranged in conjunction with the British Council.
- What are the potential gains, and what are the potential losses?
- What are the unintended consequences or second-order effects?
- What underlying values and agendas are implicit?
- In whose interests is this working? Who benefits, and in what ways?
- What are the social problems that data is being presented as a solution to?
- How responsive to a ‘data fix’ are these problems likely to be?
These wider questions of politics and power have not yet been taken up to any extent by the learning analytics community, but they look set to be bigger issues as the field matures.
My talk was on learning analytics, the state of the art and what the future might look like.
I also took part in a round-table discussion with Neil, Gilles and Dragan on issues related to learning analytics.
The back channel – mostly in Spanish – used the hashtag
During a visit to Uruguay, I was lucky enough to be invited to visit the Institute of Education at the ORT University in Montevideo. There, I gave a presentation to faculty members and postgraduate students on Innovating Pedagogy.
For the past four years, The Open University has produced an Innovating Pedagogy report annually. This series explores new forms of teaching, learning and assessment for an interactive world, to guide educators in productive innovation. As one of the report authors, I presented a quality enhancement lunchtime seminar on 23 March 2016 (part of the QELS series). In the seminar, I introduced the themes that have emerged from this series of reports – scale, connectivity, reflection, extension, embodiment and personalisation – and how these connect with modules (courses) run by the OU. The seminar included examples of innovative pedagogies in use at the OU, and identified others that could be used in future.
Fifty people attended the workshop, including invited experts (expert presentations), representatives of current European-funded projects in the field of learning analytics (project presentations), and representatives of the European Commission.
The workshop dealt with the current state of the art in learning analytics, the prospects for the implementation of learning analytics in the next decade, and the potential for European policy to guide and support the take-up and adaptation of learning analytics to enhance education.
The workshop began with a review of current learning analytics work by participants and went on to consider how learning analytics work can be taken forward in Europe (presentation on the LAEP project).
Participants at the workshop identified immediate issues for learning analytics in Europe. They set out considerations to be taken into account when developing learning analytics, made recommendations for learning analytics work in Europe and then identified both short- and long-term policy priorities in the area.
Immediate issues for LA in Europe
Framework for development: A European roadmap for learning analytics development would help us to build and develop a set of interoperable learning analytics tools that are tailored for the needs of Europe and that have been shown to work in practice.
Stakeholder involvement: There is a need to bring different people and stakeholders on board by reaching out to groups including teachers, students, staff, employers and parents. Our current engagement with stakeholders is too limited.
Data protection and surveillance: As legislation changes and individuals become more aware of data use, institutions need to understand their responsibilities and obligations with regard to data privacy and data protection
Empirical evidence and quality assurance: More empirical evidence is needed about the effects of learning analytics, in order to support a process of quality assurance.
Considerations for the development of LA
- Learning analytics can change or reinforce the status quo
- Learning analytics should enhance teaching, not replace it
- It is our duty to act upon the data we possess
- Desirable learning outcomes must be identified
- Be clear why we are collecting and analysing data
- Bring the data back to the learner
- Intelligent systems need human and cultural awareness
- Impressive data are not enough
Recommendations for LA work in Europe
- Undertake qualitative studies to understand how learning analytics can be aligned with the perceived purpose of education in different contexts, and which aspects of different educational contexts will support or constrain the use of learning analytics.
- Publicise existing evaluation frameworks for learning analytics and develop case studies that can be used to enrich and refine these frameworks
- Develop forms of quality assurance for learning analytics tools and for the evidence that is shared about these tools.
- Identify the limitations of different datasets and analytics and share this information clearly with end users.
- Explore ways of combining different datasets to increase the value of learning analytics for learners and teachers.
- Extend to different sectors of education the work currently being carried out in the higher education sector to identify the different elements that need to be taken into account when deploying learning analytics.
- Develop analytics, and uses for analytics, that delight and empower users.
Short-term policy priorities
Innovative pedagogy: Top priority is the need for novel, innovative pedagogy that drives innovation and the use of data to solve practical problems.
Evidence hub: Second priority is to secure continuing funding for a site that brings together evidence of what works and what does not in the field of learning analytics.
Data privacy: Participants considered that a clear statement is needed from privacy commissioners about controls to protect learners, teachers and society.
Orchestration of grants: The European grants system could better support the development of learning analytics if grants were orchestrated around an agreed reference model.
Crowd-sourced funding support: Set up a system for crowd-sourcing funding of tools teachers need, with EU top-up funding available for successful candidates.
21st-century skills: Focus on developing learning analytics for important skills and competencies that are difficult to measure, particularly 21st-century skills.
Open access standards: Standards need to be put into practice for analytics across Europe, with an open access forum that will enable the creation of standards from practice.
Ambassadors: We need more outreach, with ministries and politicians spreading the word and encouraging local communities and schools to engage.
Long-term policy priorities
Teacher education: Top priority in the longer term was for media competencies and learning analytics knowledge to be built into training for both new and existing teachers.
Decide which problems we want to solve: In order to develop the field of learning analytics we need to have collective discussions on the directions in which we want to go.
Facilitate data amalgamation: More consideration is needed of how to combine data sources to provide multi-faceted insights into the problems we seek to solve.
Identify success cases and methodologies that give us a solid foundation: We need a coordinated approach to quality assurance and to the identification of successful work.
Several accounts of the workshop are available online, dealing with the morning of day one, the afternoon of day one, day one as a whole, the morning of day two, the afternoon of day two and day two as a whole.
New paper out in the Journal of Learning Analytics Research, building on our previous papers dealing with how learners engage with MOOCs.
Massive open online courses (MOOCs) are being used across the world to provide millions of learners with access to education. Many who begin these courses complete them successfully, or to their own satisfaction, but the high numbers who do not finish remain a subject of concern. In 2013, a team from Stanford University analysed engagement patterns on three MOOCs run on the Coursera platform. They found four distinct patterns of engagement that emerged from MOOCs based on videos and assessments. Subsequent studies on the FutureLearn platform, which is underpinned by social-constructivist pedagogy, indicate that patterns of engagement in these massive learning environments are influenced by decisions about pedagogy and learning design. This paper reports on two of these studies of learner engagement with FutureLearn courses. Study One first tries, not wholly successfully, to replicate the findings of the Coursera study in a new context. It then uses the same methodological approach to identify patterns of learner engagement on the FutureLearn platform, and indicates how these patterns are influenced by pedagogy and elements of learning design. Study Two investigates whether these patterns of engagement are stable on subsequent presentations of the same courses. Two patterns are found consistently in this and other work: samplers who visit briefly, and completers who fully engage with the course. The paper concludes by exploring the implications for both research and practice.
Ferguson, Rebecca, & Clow, Doug. (2016). Consistent commitment: patterns of engagement across time in massive open online courses (MOOCs). Journal of Learning Analytics, 2(3), 63-88.
This is the fourth in a series of influential reports from The Open University exploring new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This report represents a collaboration with our colleagues in the Center for Technology and Learning at SRI International, the leading US research organisation.
This year, the focus is on:
- Crossover learning (connecting formal and informal learning)
- Learning through argumentation (developing skills of scientific argumentation)
- Incidental learning (harnessing unplanned or unintentional learning)
- Context-based learning (how context shapes and is shaped by the process of learning)
- Computational thinking (solving problems using techniques from computing)
- Learning by doing science with remote labs (guided experiments on authentic scientific equipment)
- Embodied learning (making mind and body work together to support learning)
- Adaptive teaching (adapting computer-based teaching to the learner’s knowledge and action)
- Analytics of emotions (responding to the emotional states of students)
- Stealth assessment (unobtrusive assessment of learning processes).
You can download the report at www.open.ac.uk/innovating