Leverhulme Lecture: Understanding Learning in the World of AI

On 21 February 2024, I hosted a CALRG (Computers and Learning research group)  and OpenTEL event with Professor David W Shaffer, Sears Bascom Professor of Learning Analytics and Vilas Distinguished Achievement at the University of Wisconsin-Madison.  

In this talk, Dr Shaffer looked at what AI is (and isn’t), its impact on what and how we learn, and how AI can change what it means to do research. 

This was followed in the afternoon by a Leverhulme workshop on epistemic network analysis.

This hands-on workshop introduced participants to the principles of Quantitative Ethnography (QE), an approach to analysing Big Data that goes beyond the old dichotomy of qualitative and quantitative methods and past simple mixtures of methods. The workshop focused on Epistemic Network Analysis (ENA), a tool for modelling complex and collaborative thinking within a QE framework. ENA models how humans make meaning of events in the world using in large- and small-scale datasets on many kinds, including logfiles, transcripts of structured and semi-structured interviews, simulations, chat, email, and social media. A

Jacob Solomon viva

On 31 March 2022, I chaired a successful viva for EdD candidate Jacob Solomon. The viva was held online and was examined by Stephen Powell and Julia Sargent, with supervisor Anesa Hosein as an observer.

Abstract

This work assesses the efficacy of the educational practice of school faculties supporting the motivation of their students studying wholly-online-delivered academic courses in the pre-university final two years of secondary school. Its focus is on the pioneering 2015-2019 work of Pamoja Education, an official provider of IB (International Baccalaureate) fully-online subject courses whose target clientele is the international school community served by the IB, and whose results during that period consistently paralleled the range of IB grades as a whole. In anticipating motivation issues in regular online study, this company requires and formally enables a student-support structure that includes the designation, training, and liaisoning with a faculty member of the student’s school serving in an on-site facilitator role as a site-based coordinator (SBC).

This study considers the contribution that this SBC arrangement makes to allow the virtual learning environment to rival formal classroom instruction in terms of academic standards achieved. It investigated the range of motivational challenges faced by online students within the framework of self-determination theory, particularly within the extrinsic domains of student autonomy, competence, and relatedness. Students (14), SBCs (4), and Pamoja staff (2) responded within semi-structured interviews on student motivation and the types of support that the SBCs provide. This data was classified into themes and codes using thematic analysis, which indicated that chiefly, but not exclusively due to their proximity and thus their relatedness to the student, the SBCs at Pamoja do substantially contribute to student engagement and student performance in that virtual learning environment, with the optimum level of input, however, varying from student to student.

The findings indicate that the motivational role of the SBC is recognized by a large body of students as well as SBCs and Pamoja staff as being crucial to student motivational well-being in the virtual environment and that with such support, online delivery could well be a viable means of enabling schools to widen their curriculum without having to employ additional staff.

 

Learning Analytics and Knowledge 2019 (LAK19)

If it’s March, it must be the LAK conference. This year it was in sunny Tempe, at the Arizona State University (ASU), and I was lucky enough to be a Program chair once again – this time in an ex officio role as a previous program chair. The role involves a lot of hard work during the year, and it’s great to see it all coming together at the conference.

Highlights of the event for me (apart from the analytics research, of course) were the conference dinner in the Desert Botanical Garden, and the outdoor poster session around the fountain as night fell. ASU were great hosts, and many people told me it was the LAK that they had enjoyed the most (and that’s against some stiff competition).

The final day of the conference coincided with International Women’s Day, so we gathered on stage after the day’s keynote from Shirley Alexander for a photo that brings together the women of the Society for Learning Analytics Research (SoLAR).

2019 Google Scholar metrics are out! Our event, the International Conference on Learning Analytics and Knowledge #LAK20 is now #6 in the category of Educational Technology #learninganalytics pic.twitter.com/eaPnh8BSRb

— SoLAR (@SoLAResearch) July 20, 2019

FutureLearn Educators

The new module I’m leading, H880: TEL Foundations and Futures, is the first that The Open University is presenting on FutureLearn. The shift from the university’s Moodle virtual learning environment (VLE) to FutureLearn has meant many changes.

The module is using the conversational learning pedagogy supported by FutureLearn. Learners are encouraged and supported to converse about why things happen, offering conceptions of their learning and questioning the understanding of others, in attempts to reach agreement about their reflective understandings. They ask questions, and share experiences, interpretations and links to resources.

We know, from our experience with MOOCs, that conversational learning can generate an enormous amount of discussion. The first run of The Online Educator MOOC, for example, which ran for just four weeks, prompted comments with a similar word length to Crime and Punishment. H880 has 32 study weeks.

Many of these discussions are ‘water-cooler conversations’, like the ones that take place by an office water-cooler. People come in and out; some contribute, some simply listen. Some stay for a while, some are only there briefly. There’s no expectation that students or tutors will engage with the entire conversation, just with the most recent or the most popular comments.

This is a different model to VLE forum discussion, where students and tutors often read all comments posted, and the main learning activities take place elsewhere.

I therefore ran a one-day briefing on 17 November for the associate lecturers (tutors) who would be working on the module, outlining the differences between the VLE and FutureLearn, and suggesting ways of working. I also circulated the first draft of the ‘Tutor Guide’, a resource that brings together a set of information about the module structure, pedagogy, and ways of working, as well as information about available tools and resources.

The module team and tutors will be working together to put into practice a way of working that provides the same level of support for students as on any other OU postgraduate module, without overburdening tutors.

Student Support Team

Support for students at The Open University doesn’t only come from tutors. There’s a team on the LIbrary Helpdesk, where the ‘Chat to a Librarian’ facility is available 24/7. There’s the Computing Helpdesk team, who provide support every day of the year except Christmas Day, New Year’s Day, and Easter Sunday. There are also Student Support Teams, who answer general enquiries and provide specialist advice on students’ areas of study six days a week.

All three teams needed to know how a FutureLearn module would be different to an H880 module in order to be able to support students. They also all need to be able to access the module (the registration process currently has to take place manually, rather than going ahead automatically, as it would on the VLE). I spent a day in Nottingham, working with 19 members of the Student Support team who would be dealing with H880. As with the tutors, there were opportunities to discuss what would happen when the module went live, and to consider which established practices would change. And, as with the tutors, there was a determination to provide students with the same level of support as they would expect on any other OU module.

FutureLearn Academic Network: London

One of my roles is the Academic Coordinator of the International FutureLearn Academic Network (FLAN). On 22 June 2018, we ran a meeting at the FutureLearn offices in Camden, London.

The theme of the meeting was ‘New Research Directions’. This was the network’s annual opportunity to catch up on the work of the many doctoral students who spend their lives researching aspects of FutureLearn.

Keynote speaker Alyssa Wise, from New York University, encouraged FLAN members to pool ideas and research, identify overarching concepts, and develop a framework that can be used to structure research into learning at scale. She asked two questions: What are the MOOC learning outcomes that are valuable, and that are valued? What are the unique qualities, the core characteristics, of MOOCS?

Alyssa identified massive scale as one of these core characteristics. Massive scale brings its own opportunities, challenges, interactions and pedagogies – all of which need to be investigated. At the same time, it’s also important to look at the smaller elements that make up the massive, taking opportunities to examine what is happening at a smaller scale, and exploring the diversity that is another core characteristic of MOOCs.

With keynote speaker Alyssa WIse

Several speakers focused on what we can tell about learning and teaching by analysing discussion threads. Alyssa talked about her work on the MOOCeology project, which looks at how people interact in large-scale learning environments. Her team are currently working on ways of telling automatically whether an online discussion is focused on course content. They’re also using network analysis to study content-related discussion networks. The people in these networks interact with each other, they keep engaging, and they keep the discussions going. Alyssa suggested that how people engage in content-related discussion could be one of the most important indicators of their learning.

Apart from teaching, learning and discussion, the other main focus of the day was on diversity and accessibility. Janesh Sanzgiri compared the experience of Indian learners on FutureLearn and on the Indian MOOC platform NPTEL. Shahrzad Ardavani and Monty King examined a MOOC-based CPD course for English Language teachers from different perspectives, including the perspective of learners in East Timor. Francisco Iniesto looked at the motivations of FutureLearn’s disabled learners, as well as the barriers they have to overcome. Each of these speakers identified ways in which FutureLearn could be made more accessible and inclusive.

LAK18 – Sydney

A highlight of my year will surely be LAK18 – the annual Learning Analytics and Knowledge conference run by the Society for Learning Analytics Research (SoLAR). Together with Simon Buckingham Shum, Xavier Ochoa and Agathe Merceron, I was programme chair for the conference.

Our five days in Sydney were the culmination of more than a year’s hard work. We were really pleased with the attendance and the engagement at the conference, and the success of new initiatives such as double-blind peer review and the introduction of discussion around meta-reviews of the papers.

The next steps for us will be two special sections of the Journal of Learning Analytics – one related to the conference theme of human-centred design, and one including extended versions of the best papers. I shall also be ex officio programme chair of the next conference, at Arizona State in 2019.

The conference also provided a chance for many of the Learning Analytics Community Europe (LACE) organising team to meet up and make plans for the future.

 

Academic coordinator: FLAN

Following the retirement of Mike Sharples (who will return to The Open University as an Emeritus Professor in March). I have taken on the role of Academic Coordinator for the FutureLearn Academic Network (FLAN).

The network was established in 2013 by a group of academics in order to connect academics and research students based at FutureLearn partner institutions, share research, and explore shared research opportunities. These include: joint research bids and publications, comparative studies using shared FutureLearn data, course designs, and methods to analyse and evaluate courses.

The Network is open to staff and research students based at FutureLearn partner institutions with an interest in research related to the FutureLearn platform.

On 7 November, we held one of our quarterly meetings – this time at the British Council in Central London. Among the many interesting talks:

• Josh Underwood gave a detailed and considered account of the role of a mentor or facilitator within FutureLearn courses.
• Matthew Nicholls and Bunny Waring talked about their use of a virtual reality simulation of Rome in the 4th century CE.
• Phil Tubman introduced a tool for visualising discussion, which is now being used on a course from Lancaster University.
• Eileen Scanlon and I talked about research ethics on the platform and initiated discussion on changes to the terms and conditions.

The next meeting of FLAN is likely to be in Exeter at the end of February 2018. If you are eligible to be part of FLAN and would like to be involved either in person or remotely, do get in touch.

CALRG conference 2017

Together with Liz FitzGerald and Eileen Scanlon, I chaired the 38th annual conference of the Computers and Learning research group (CALRG), which took place at The Open University 16-18 June 2017. We enjoyed keynote presentations from Siân Bayne, Jenny Preece and Ben Shneiderman.

Full details of the conference, together with links to all the abstracts and to many of the presentations, are available on Cloudworks.

The third day of the conference was FutureLearn Academic Network day. This annual conference event prioritises the work of doctoral students within the FLAN Network. This year, it brought together presenters from Bath, Lancaster, Purdue (USA), Sheffield, Southampton, The Open University, and Warwick.

Our discussant was Professor Rupert Wegerif, University of Cambridge.

Members of FLAN can access the video of the event.

LAK17: it’s a wrap

Scattered between my research presentations at LAK17 was my work as a member of the executive for the Society for Learning Analytics Research (SoLAR). The executive met daily during the conference – it is the only chance we have each year for face-to-face meetings. The LAK conferences also provide a venue for the AGM of the society and, despite the size of the room, where the AGM was held, it was standing room only for most of the meeting.

The executive also have a role to play in decisions about the conference itself, as well as acting as reviewers on the programme committee and chairs for the different sessions. Next year, at LAK18 in Vancouver, I shall be taking on a bigger role, as one of the programme chairs for the conference.

The picture shows me with half the SoLAR Executive at the post-LAK17 review meeting.

LAK17: doctoral consortium

A very busy week in Vancouver at the LAK17 (learning analytics and knowledge) conference kicked off with the all-day doctoral consortium on 14 March (funded by SoLAR and the NSF). I joined Bodong Chen and Ani Aghababyan as an organiser this year and we enjoyed working with the ten talented doctoral students from across the world who gained a place in the consortium.

1. Alexander Whitelock-Wainwright: Students’ intentions to use technology in their learning: The effects of internal and external conditions
2. Alisa Acosta: The design of learning analytics to support a knowledge community and inquiry approach to secondary science
3. Daniele Di Mitri: Digital learning shadow: digital projection, state estimation and cognitive inference for the learning self
4. Danielle Hagood: Learning analytics in non-cognitive domains
5. Justian Knobbout: Designing a learning analytics capabilities model
6. Leif Nelson: The purpose of higher education in the discourse of learning analytics
7. Quan Nguyen: Unravelling the dynamics of learning design within and between disciplines in higher education using learning analytics
8. Stijn Van Laer: Design guidelines for blended learning environments to support self-regulation: event sequence analysis for investigating learners’ self-regulatory behavior
9. Tracie Farrell Frey: Seeking relevance: affordances of learning analytics for self-regulated learning
10. Ye Xiong: Write-and-learn: promoting meaningful learning through concept map-based formative feedback on writing assignments

The intention of the doctoral consortium was to support and inspire doctoral students in their ongoing research efforts. The objectives were to:

• Provide a setting for mutual feedback on participants’ current research and guidance on future research directions from a mentor panel
• Create a forum for engaging in dialogue aimed at building capacity in the field with respect to current issues in learning analytics ranging from methods of gathering analytics, interpreting analytics with respect to learning issues, considering ethical issues, relaying the meaning of analytics to impact teaching and learning, etc.
• Develop a supportive, multidisciplinary community of learning analytics scholars
• Foster a spirit of collaborative research across countries, institutions and disciplinary background
• Enhance participating students’ conference experience by connecting participants to other LAK attendees