Leverhulme Lecture: Understanding Learning in the World of AI

On 21 February 2024, I hosted a CALRG (Computers and Learning research group)  and OpenTEL event with Professor David W Shaffer, Sears Bascom Professor of Learning Analytics and Vilas Distinguished Achievement at the University of Wisconsin-Madison.  

In this talk, Dr Shaffer looked at what AI is (and isn’t), its impact on what and how we learn, and how AI can change what it means to do research. 

This was followed in the afternoon by a Leverhulme workshop on epistemic network analysis.

This hands-on workshop introduced participants to the principles of Quantitative Ethnography (QE), an approach to analysing Big Data that goes beyond the old dichotomy of qualitative and quantitative methods and past simple mixtures of methods. The workshop focused on Epistemic Network Analysis (ENA), a tool for modelling complex and collaborative thinking within a QE framework. ENA models how humans make meaning of events in the world using in large- and small-scale datasets on many kinds, including logfiles, transcripts of structured and semi-structured interviews, simulations, chat, email, and social media. A

FLAN tenth anniversary

The first meeting of the FutureLearn Academic Network (FLAN) took place in London on 26 September 2013 with speakers from Glasgow, Exter and The Open University. Since then, the network has held around four events a year, in the UK, Ireland, Spain and Australia. These have included 247 presentations given by 201 different speakers.

The 33rd FLAN meeting, celebrating its Tenth Anniversary, took place at the University of Leeds on 23 October 2023. The talks (other than institutional updates) are all available in this YouTube playlist.

• Ayşe Sunar: Where we have progressed or stopped on the way to a personalised MOOC experience
• Keynote – Mike Sharples: ‘The changing pedagogy of FutureLearn, from “massive open social learning” to “professional progression with tailored learning”
• Duncan Kemp, FutureLearn Partnerships Director, FutureLearn update from
• Matt Cornock, Latest work at Digital Education Service, University of Leeds
• Monty King: Doing MOOCs in Dili
• Barbara Conde & Napat Jitpaisarnwattana: Bridging the gaps in language learning through LMOOCs.
• Michael Meaney: A Framework for Considering the Matthew Effect in MOOCs
• Fereshte Goshtasbpour & Katy Jordan (Katy was unable to attend): The Decade of the MOOC

ECTEL 2023

I joined the Eighteenth European Conference on Technology Enhanced Learning Responsive and Sustainable Educational Futures (ECTEL 2023) in Aveiro, Portugal from 4-8 September 2023 as a keynote panelist for a panel on ‘Responsible and Sustainable Learning Futures’. (I don’t think Slideshare likes the OU’s current font – the original slides looked better than the version embedded below!)

Responsive sustainable education futures.pptx from Rebecca Ferguson

The panel speakers were me, Paul Prinsloo, Marco Kalz, and Antonio Dias de Figueiredo and the panel was chaired by Marcus Specht. We discussed four questions / statements posed in advance by different experts. I’ve added my responses below.

“We’re all worried about the climate, but it’s simply weak-minded to suggest we all need to change our research. It has no bearing on most ECTEL work. It’s all equally valuable!” (Expert 1)

I don’t think we all need to change our research, but we do need to bear the climate emergency in mind. It’s down to what we think the purpose of education is. Something we should be preparing our learners for are the big challenges that need to be addressed and that will affect their lives. We can ask ourselves how our work is promoting sustainability, which skills we are developing in our learners, and we can work towards creating systems that solve problems rather than creating them.

“The current trajectory of AI development will powerfully support and advance individual approaches to learning to the detriment of important social, cultural and institutional aspects.” (Expert 2)

We have influence on how AI is applied in education. What we need to do is keep promoting the ways in which it can maintain and support the things that are good. We already know that it has made us question the uses and purposes of assessment, and to consider why the current proxies for learning have become so baked in that we are unwilling to give them up even when we see students’ mental health suffering, and inequalities being maintained.

“By running experiments that compare an experimental group using some piece of learning technology and a control group doing a non-technological activity, we make our research findings non-scalable to a broad community because in most classroom practices teachers integrate technology-based and technology-free activities. Worse, this comparison perpetuates the myth that the compared technology would have an intrinsic effect, i.e. an effect independent from previous activities in the same scenario and independent to the way the teacher orchestrates this scenario.” (Expert 3)

There are some things that you can investigate through a randomised control trial, but education contains so many variables that you have to pare it down considerably to make those experiments work. One of the reasons why we consistently see TEL and technology-free education coming out equal, is that to carry out an RCT we have to reduce our teaching to the lowest common denominator. For example, if I compared structured, teacher-supported science inquiry, using the sensors available in a smart phone, we don’t really have a non-tech alternative. So instead, we have to reduce our investigation to something that can be done without tech – for example, learning specific facts with textbooks and teacher demos – then we’ve taken out all the ways in which TEL could be better and, not surprisingly, we find that TEL is simply a more expensive way of doing the same thing.

“Currently, AI models tend to be evaluated primarily with respect to their technical predictions. What is required to facilitate such a shift to promote the adoption of a socio-technical and cultural perspective for understanding the design and impact of AI-powered educational decision support systems to advance equity and inclusion in educational practices?” (Expert 4)

We are looking for political will to do this. We are looking for funding to do this. We are looking for support to develop and test simple to use frameworks and models. We don’t want to support equity and inclusion solely in terms of AI models – we want to make them an integral part of all educational systems. And, importantly, we want to draw attention to the sustainability aspects of these technologies. According to Medium, a query on ChatGPT consumes the equivalent amount of energy of running a 5W LED bulb for 1hr 20min!  Each Google search would keep it going for 3 minutes.

Playful Learning: Monumental Consequence

Together with Mark Childs and Mike Collins, I ran a session on the Reacting to the Past game Monumental Consequence at Playful Learning 2023 in Leicester at the beginning of July.

Description

Is art ever worth dying for? In this role-playing game, an invading army has set up headquarters in a building containing priceless art, key to your cultural history. You must decide together – will you risk lives by trying to save the art, or will you simply bomb the church?

What will you be doing in this session:

This role-playing game, developed by Mary Beth Looney and Central Michigan University Press, introduces ‘Reacting to the Past’, an active learning pedagogy of role-playing games that promote engagement with big ideas. While participants must stick to the philosophical and intellectual beliefs of the historical figures they are assigned, they must also express those ideas persuasively.

Participants take on a citizen’s role, perhaps as a seamstress, tavern owner, merchant, widow, militia member or other local. Each role has priorities, viewpoints, and alliances. Only some can be on the winning side. Together they must decide whether to ask the civic militia to risk their lives by taking on the enemy to save the town’s cultural heritage, or bomb the church.

Once assigned a role, players view locations and treasures, then introduce themselves. After the opening prompt has been read, they discuss what action to take, focusing on why the art is important to save, or why lives are more important. They then vote. A two-thirds majority leads to action, otherwise a dice throw triggers a random event, followed by more discussion, another vote and action.

When attendees leave this session they will have

Played a great game, thought deeply about different perspectives on cultural history, and know more about the Reacting to the Past pedagogy.

References, web links and other resources:

• Reacting to the Past https://reacting.barnard.edu/ – browse the games, sign up for an event (they tend to be in the USA), join the community and/or sign up for the newsletter.
• Reacting to the past on Twitter https://twitter.com/reactingttpast?lang=en
• An overview of Reacting to the Past, with videos, from the American HIstorical Association https://www.historians.org/teaching-and-learning/teaching-resources-for-historians/reacting-to-the-past
• Chain Reactions – a reacting blog https://reactingconsortium.org/Blog
• Reacting to the Past gamebooks from the University of North Carolina Press
• Get your own copy of Monumental Consequence (25 USD) https://cmichpress.com/product/monumental-consequence-physical-edition/
https://uncpress.org/series/reacting-to-the-past/
• Artist and independent scholar Mary Beth Looney talks about her game, Monumental Consequence, and the value of art and monuments to the societies that create them (53 mins) https://www.youtube.com/watch?v=jy5idbhx-AE&ab_channel=BeyondSolitaire
• Barnard and Columbia undergraduates play “Defining a Nation: India on the Eve of Independence, 1945” at Barnard College (5-minute video) https://www.youtube.com/watch?v=_U6L9ERzw0U&ab_channel=ReactingtothePast
• Students—with costumes and trumpet in hand—assume roles in the Athenian Assembly to debate issues as Athens reconstructs after the Peloponnesian War (2-minute video) https://www.youtube.com/watch?v=clO0uvCgwlM&ab_channel=SmithCollege
• Webb, J. and Engar, A., 2016. Exploring Classroom Community: A Social Network Study of Reacting to the Past. Teaching & Learning Inquiry, 4(1), p.n1.
• Lazrus, P.K. and McKay, G.K., 2013. 21: The Reacting to the Past Pedagogy and Engaging the First‐year Student. To Improve the Academy, 32(1), pp.351-363.
• Joyce, K.E., Lamey, A. and Martin, N., 2018. Teaching philosophy through a role-immersion game: Reacting to the past. Teaching Philosophy, 41(2), pp.175-198.

Pedagodzilla at Playful Learning

Along with Mike, Mark and Puiyin Wong, I was also involved in the Pedagodzilla podcast activity at the Playful Learning conference (the picture shows MIke Collins in conversation with Professor Nicola Whitton at the conference).

Learning analytics: what can we achieve together?

Learning analytics – what can we achieve together.pptx from Rebecca Ferguson

I was invited to keynote at the Learning Analytics Summer Institute (LASI) co-hosted by the National University of Singapore’s Institute for Applied Learning Sciences and Educational Technology (NUS ALSET) and the Society for Learning Analytics Research (SoLAR). The event took place on the National University of Singapore’s University Town campus 6-8 June 2023.

Keynote abstract

‘Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.’ That’s the definition we have worked with since the first Learning Analytics and Knowledge (LAK) conference, back in 2011. Together, we have made great progress on dealing with data but have moved more slowly on understanding learning and learning environments. In this keynote, Professor Rebecca Ferguson will look at what we mean by ‘optimizing learning’, identify areas of research that are currently proving fruitful, and examine the challenges we need to face together.

Tutorial

In addition to my ketynote, I also led a workshop at LASI.

‘This tutorial is designed for everyone with an interest in increasing the impact of their learning analytics research. The tutorial will begin with a short introduction to the field and to the learning analytics community. It will go on to identify significant challenges that learning analytics needs to address, and factors that should be taken into account when implementing analytics, including ethical considerations related to development and implementation. As a participant, you’ll have opportunities to relate these challenges to your own work, and to consider how your research is situated in the field. You’ll be encouraged to reflect on how your work aligns with the learning analytics cycle, how it contributes to the evidence base in the field, and ways in which you can structure your work to increase its impact. The tutorial will include opportunities to share ideas and experiences.’

2023 Introduction to learning analytics.pptx from Rebecca Ferguson

Panel

My final contribution to LASI 2023 was as a panellist together with:

• Edna Chan, Singapore Polytechnic
  Tan Teck Kiang, National University of Singapore
• Abelardo Pardo, University of South Australia
• Dragan Gasevic, Monash University – Moderator

During the panel, we addressed the following questions:

1. Starting with what seems like a simple question but ends up being more difficult to operationalise in practise. What is blended learning and what are the key areas Learning Analytics can already play in shaping blended learning experiences?
2. The hallmark of Learning Analytics is that the human remains in the loop to ultimately make decisions. That human can be the learner themselves or someone who monitors the learner like an instructor. As we leap forward into a world filled with AI-generated and curated content and decision systems engineered to persuade or nudge the human to take particular actions, what is the role of the human in learning going forward?
3. Life transitions are major periods for learning. We saw a number of such transitions in the Educational Landscape President Tan showed on Tuesday. The one that stuck with me the most is that feedback loop between Work and Life and the Universities, Polytechnics and Institutes. How could we be using Learning Analytics to support people through those transitions and flows between formal and workplace learning?
4. Relatedly, if there was a set of measures you could collect from your learners after they leave your respective institutions, what would they include and what would you do with them?
5. As a final question, we’ve seen advances in coupling learning analytics systems to support experiential learning of physical skills in real-time in a similar fashion as cognitive tutors have for years. The systems monitor the completion of a task and highlight areas of deviation from a standard. The learner then attempts to correct their completion of the task during the next cycle. How do you see these systems playing out in both formal and informal learning environments? What then becomes the role of skill coaches?

Challenges for innovation and educational change in digital education in low resourced settings: a Kenyan example

Goshtasbpour, Fereshte; Pitt, Beck; Cross, Simon; Ferguson, Rebecca and Whitelock, Denise (2023). Challenges for Innovation and Educational Change in Digital Education in Low Resourced Settings: A Kenyan Example. In: European Distance and E-Learning Network (EDEN) 2023 Annual Conference, 18-20 Jun 2023, Dublin, Ireland.

Abstract

Many governments invest in digital education and deliver multi-faceted initiatives to develop the capacity of educational institutions and staff confidence in use of technology for enhanced learning and better student outcomes. However, the impact of these initiatives does not always result in improved technology-supported learning or successful digital educational products. Research shows many factors impede the achievement of such objectives. Yet, they are not well understood particularly in low-resourced educational settings. Thus, this paper as part of a larger study aimed to explore barriers to implementing TEL for higher education in low-resourced contexts and potential solutions from the perspective of educators, managers and support staff. The unique inclusion of support staff and management in this research provides a deeper understanding of current barriers to TEL, and the value of multi-stakeholder engagement to develop meaningful context-driven solutions. Using a qualitative oriented mixed-method approach underpinned by Beyond Prototypes conceptual model of TEL complex, the study found that most barriers and enablers of effective TEL implementation are related to the ecology in which it is practised, together with the influence of the salient communities. Findings also suggest that most stakeholders are aware and prioritise barriers related to their role, but have limited awareness of barriers faced by other stakeholders or how others perceived their role in supporting TEL. While the study supports findings of previous research, it adds further insight by considering the views of all implementing stakeholders and the relationships between them.

Adapting OER for more sustainable online course development

My colleagues presented a poster we had worked on together to the European Distance and E-Learning Network (EDEN) 2023 Annual Conference, 18-20 Jun 2023.

Introduction

As with many universities around the world, in Kenya the Covid-19 pandem- ic necessitated a move to what is often described as “emergency remote teaching” rather than a planned response and strategic shift to teaching and learning online (see e.g. Bozkurt et al, 2020). In Kenya, the move to emer- gency remote teaching began with the closure of educational institutions, in- cluding universities. In addition to the challenge of moving from face-to-face to online teaching, specific challenges for Kenyan learners and educators that were highlighted particularly during this early period of the pandemic in- clude connectivity, digital literacy, power supply and access to internet ena- bled devices and wi-fi when outside of the campus setting (see Pete & Soku (2020) and Omwenga (2022)). The Kenyan government has therefore priori- tised the development of HE sector expertise in designing and delivering digital education (online and blended) and The Open University (UK) is sup- porting the government to close the gap in required expertise.

How to build data literacy within faculty

On 19 April I took part in the Times Higher Education (THE) Digital Universities UK event, rather grandly styled a ‘world summit’. I was one of the panellists in an interactive workshop on data literacy.

Workshop description

• Eliza Compton, THE Campus
• Rebecca Ferguson, OU
• James Pickering, Leeds
• Bronwen Swinnerton, Leeds

Join this interactive workshop curated by THE Campus to hear from our panel of experts on the solutions to some of higher education’s most pressing challenges to building data literacy among faculty. Panellists will meet beforehand to discuss these key questions and come prepared with new thoughts and ideas:

• What are the issues that faculty have in data literacy?
• How to distinguish between data literacy and technical literacy
• How to create a skills-focused faculty development programme
• What software or programmes can facilitate data analysis?

Possible answers to those questions

What are the issues that faculty have in data literacy? 

• Use the Jisc Digital Capabilities Framework
• Wide range of stakeholders, levels of buy-in, types of data and data uses
• Preconceptions about data collection, bias, varied acceptance of data as a tool
• Questions over benefits of data analytics in addressing issues such as engagement (from researchers as well as faculty)
• Time required to learn and use new tools and systems (that keep changing)
• Governance, gatekeeping, code of practice
• Beliefs of faculty – for example about ease of use, utility, changes to workload or potential threats – are critical factors in acceptance and adoption

How to distinguish between data literacy and technical literacy  

• Use the European Digital Competencies framework
• Difficult to separate due to overlapping concerns about why we collect data, what does it mean, and how data is collected and analysed
• Data analytics is one tool among many to monitor engagement
• Also need to distinguish information literacy (capacity to find, evaluate, manage, curate, organise and share digital information) and media literacy (capacity to critically receive and respond to messages in a range of media)

How to create a skills-focused faculty development programme  

• Aim to develop a culture of informed decision-making
• Work with faculty to identify and support their priorities for data use
• Identify the main types of data available to faculty and provide examples of use
• Consider different stakeholders
• Training is not just functional; it’s about why we use data
• Distinguish between learning analytics (using data to support learning and teaching) and academic analytics (used to support management and reporting)
• Tried not to oversell it and maintained stance as ‘critical advocates’
• Consider statistical literacy (limits of what data can show)
• Include ethical considerations: responsibility, transparency and consent, privacy, validity, access, enabling positive interventions, minimising adverse impacts and stewardship of data
• Cover the multiple reasons why data may be incomplete
• Acknowledge difficulty of understanding how data sets intersect
• Evaluate the programme in relation to its aim
• Make it clear where faculty can go for support or development
• Use existing frameworks (see below) to shape your curriculum

What software or programs can facilitate data analysis?

• Leeds uses Solutionpath and future needs might include flexibility
• Software for quantitative analytics
• Statistical tests for research purposes
• OU uses the SAS statistical software suite.
• OU also uses Tableau for data visualisation
• Th OU has a Data Handbook online (internal accesthat covers getting started with data, funding and data impact, tools and best practice, data definitions, non-student data, and data in the public domain

 

Creating and using MOOCs in teaching: Singapore

In February, I was one of the examiners for Oi Lin Lian’s doctoral thesis, Creating and using massive open online courses in teaching: A study of the Singapore higher education sector, submitted for examination at the University of Western Australia. The doctorate was awarded on 23 May 2023 and the thesis is now available online. UWA theses are examined at a distance, without a viva, so I have no viva picture to share but I can share the abstract and thesis link.

Abstract

This qualitative interpretivist study sought to uncover the perspectives of MOOC instructors from Singapore universities on their MOOC experiences through semi-structured interviews with 10 instructors. Using reflective thematic analysis, four propositions were generated: (a) instructors experiment with MOOCs in order to ultimately benefit their university students; (b) the MOOC instructor role is to provide a well-designed and engaging course; (c) while instructors doubt that MOOCs can replace traditional courses, they find MOOCs useful; (d) institutional support is crucial for creating and sustaining high-quality MOOCs. Based on the findings, recommendations are made for MOOC instructor practice, institutional policy, and research.

CHatGPT and other artificial assistants

Innnovations in online teaching and learning: CHatGPT and other artificial assistants from Rebecca Ferguson

Talk given by Agnes Kukulska-Hulme and Rebecca Ferguson at a SciLab (a centre for pedagogical research and innovation in business and law) seminar at The Open University, Milton Keynes, UK on Wednesday 3 May 2023.

Abstract

Artificial Intelligence (AI) is revolutionizing the way we approach education, offering new and innovative ways to improve or enhance teaching, learning and assessment, while also posing multiple challenges. The recent interest in ‘generative AI’ tools such as ChatGPT has brought the potential of AI to the fore. ChatGPT is a natural language processing tool that uses machine learning to generate human-like responses to text input from humans. In this seminar we will first briefly review the growing field of ‘artificial assistants’ that are increasingly being used in education. We will then explore how generative-AI can be used in teaching, learning and assessment. The seminar will enable discussion of the potential benefits and challenges of using AI powered pedagogies, the ethical implications of using AI in education, and emerging opportunities and impacts in FBL [Facultyof Business and Law]  discipline areas.