Analytics To Identify Learning Dialogue in Online Discussions

On 11 April, I presented a full paper at the learning analytics and knowledge conference, LAK13, in Leuven, Belgium.

The paper, ‘An Evaluation of Learning Analytics To Identify Exploratory Dialogue in Online Discussions‘ was co-authored by Zhongyu Wei of the Chinese University of Hong Kong, Yulan He, now at Aston University, and Simon Buckingham Shum from The Open University.

Learning analytics to identify exploratory dialogue in online discussions from Rebecca Ferguson

Paper abstract:

Social learning analytics are concerned with the process of knowledge construction as learners build knowledge together in their social and cultural environments. One of the most important tools employed during this process is language. In this paper we take exploratory dialogue, a joint form of co-reasoning, to be an external indicator that learning is taking place. Using techniques developed within the field of computational linguistics, we build on previous work using cue phrases to identify exploratory dialogue within online discussion. Automatic detection of this type of dialogue is framed as a binary classification task that labels each contribution to an online discussion as exploratory or non-exploratory. We describe the development of a self-training framework that employs discourse features and topical features for classification by integrating both cue-phrase matching and k-nearest neighbour classification. Experiments with a corpus constructed from the archive of a two-day online conference show that our proposed framework outperforms other approaches. A classifier developed using the self-training framework is able to make useful distinctions between the learning dialogue taking place at different times within an online conference as well as between the contributions of individual participants.

Doug Clow liveblogged the presentation.

Photo from gr0uch0‘s excellent LAK13 conference set.

Checking grant bids

This week I shall be one of the presenters at a university Bidding for Funding workshop on ‘Building your track record with funders’. Below is the checklist for grant applications that I will be circulating to participants

Quick-check questions

1. What story am I telling?
2. Who is the audience?
3. Why does it matter?
4. Why now?
5. Why me / us?

Have I:

• Formulated the problem clearly?
• Established appropriate intellectual aims?
• Set the problem in the context of contemporary scientific and theoretical debates?
• Explained what the research will do – to whom or what – and why?
• Justified my selection of staff and / or collaborators?
• Demonstrated the ways in which this work will build on existing research?
• Clearly and concisely set out appropriate, practical and attainable aims / objectives?
• Shown how my research will relate to and deliver these aims and objectives?
• Developed a well thought-out research design in which there is a reasoned and realistic explanation of the scale, timing and resources necessary?
• Provided a full and detailed description of the proposed research methods?
• Defended my research design and shown why others are not appropriate?
• Highlighted any innovation in the methodology I am planning to use?
• Justified the quality, validity, reliability and relevance of this research?
• Considered the possibility of using existing data sources?
• Set out a clear and systematic approach to the analysis of data?
• Shown how my approach to analysis fits the research design?
• Thought about the ethics of what I plan to do?
• Addressed any sensitive issues or potential problems?
• Fully consulted on these issues and obtained approval if required?
• Provided written confirmation that access will be given where necessary?
• Identified and planned for the skills and competencies required?
• Highlighted potential difficulties and discussed how they will be handled?
• Demonstrated the ways in which this research will make a contribution to the area?
• Identified people outside the academic community who might use this research?
• Involved / consulted potential users of this research?
• Arranged for those users to continue to be involved in an appropriate way?
• Explained why this research will be of interest to this funder?
• Investigated possibilities for co-funding the research?
• Provided a good quality, up-to-date bibliography?
• Provided a clear dissemination strategy that will engage all interested parties?
• Demonstrated ways in which my research will make an impact?
• Considered ways of making my data and my publications open access?
• Checked the spelling, grammar and style of my proposal?
• Identified potential referees, and justified their selection?
• Conveyed my genuine interest in, understanding of and enthusiasm for the research?

Southern SoLAR Flare Keynote

On 29 November I gave a keynote on ‘Social Learning Analytics: Some Approaches’ at the Southern SoLAR Flare held at the University of Technology Sydney, Australia.

The SoLAR Southern Flare Conference (SSFC) introduced the potential of learning analytics to practitioners, academics, researchers, administrators, and anyone interested in learner-centred data driven practices. The attendees included representatives from over half the universities in Australia and New Zealand as well as participants from industry. The Flare was blogged by event chair Shirley Alexander.

Social Learning Analytics, Southern SoLAR Flare from Rebecca Ferguson

Keynote abstract

Universities around the world are currently exploring radical new approaches to post-compulsory education, based around online courses with no formal entry requirements or constraints on class sizes, combining the power of social networks for learning with open access learning materials (Kop, 2011; Kop, Fournier, & Sui Fai, 2011). These massive open online courses (MOOCs) include MITx, which offers a portfolio of MIT courses to a virtual community of learners around the world free of charge, and the Stanford ‘Introduction to Artificial Intelligence’ course that signed up 160,000 students from 190 countries in 2011 (Leckhart & Cheshire, 2012).
For these and similar courses to be educationally effective, they not only demand new methods of teaching, but also new approaches to providing individualized support, new ways of tracking and managing the learning of thousands of students, and new tools that will help learners to orient themselves in complex online settings and develop a coherent view of an information space (Siemens, 2011).
This presentation will showcase The UK Open University’s research into the development of social learning analytics – analytics that can be used to understand and support how learners build knowledge together in different cultural and social settings, both inside and outside formal education.

Confronting Reality with Big Data & Learning Analytics

Yesterday I was at ALT-C in Manchester, as one of a group from The Open University and JISC, running the symposium, Big Data & Learning Analytics: Confronting Reality with Big Data & Learning Analytics. Sheila MacNeill has blogged about the event and responses to it.

ALT-C2012 Learning Analytics Symposium from Simon Buckingham Shum

Abstract

We are experiencing an explosion in the quantity of data available online from archives and live streams. Learning Analytics is concerned with how educational research, and learning platform design, can make more effective use of such data (Long & Siemens, 2011). Improving outcomes through the analysis of data is of interest to researchers, administrators, systems architects, social media developers, educators and learners. Analytics are being held up by some as a way to confront, and tackle, the tough new realities of less money, less attention, and higher accountability for quality of learning.

Researchers and vendors are building reporting capabilities into tools that provide unprecedented levels of data on learners. This symposium will show what is possible, and what’s coming soon. What objections could possibly be raised to such progress?

However, information infrastructure embodies and shapes worldviews: classification schemes are not only systematic ways to capture and preserve, but also to forget, by virtue of what remains invisible (Bowker & Star, 1999). Learning analytics and recommendation engines are designed with a particular conception of ‘success’, driving the patterns deemed to be evidence of progress, the interventions that are deemed appropriate, the data captured and the rules that fire in software.

This symposium will air some of the critical arguments around the limits of decontextualised data and automated analytics, which often appear reductionist in nature, failing to illuminate higher order learning. There are complex ethical issues around data fusion, and it is not clear to what extent learners are empowered, in contrast to being merely the objects of tracking technology. Educators may also find themselves at the receiving end of a new battery of institutional ‘performance indicators’ that do not reflect what they consider to be authentic learning and teaching.

This Symposium will provide the opportunity to hear a series of brief presentations introducing contrasting perspectives, before the debate is opened to all. Speakers from a cross-section of The Open University will describe how we are connecting datasets, analysing student data and prototyping next generation analytics. Complementing this, JISC will present a national capability perspective, with an update on the JISC CETIS ‘landscape analysis’ of the field, which will clarify potential benefits, issues to consider, and help institutions to assess their current capability and possible next steps.

Participants will catch up with developments in this fast moving field, through exposure to the possibilities of analytics, as well as issues to be alert to.

SoLAR Storm Event

Social Learning Analytics symposium, Knowledge Media Institute, The Open University UK.

An afternoon presenting and discussing the work of SocialLearn interns Zhongyu Wei (The Chinese University of Hong Kong) and Shaofu Huang (Bristol University) on the development of social learning analytics. I re-presented the talk on social learning analytics that I presented at LAK 2012 earlier this year, setting the scene for the work of the interns, and painting the broader picture. Zhongyu talked about his work on discourse analytics; Shaofu talked about the progress of his work on disposition analytics and there was a question and answer session with members of the audience.

View the webinar.

Full details of the event are available on the SocialLearn Research blog.

This hybrid face-to-face / webinar event formed part of SoLAR Storm  — the virtual research lab convened by the Society for Learning Analytics Research to build research capacity in this new field by networking PhD researchers with each other and the wider community.

EnquiryBlogger

Together with Simon Buckingham Shum, I presented a Technology Coffee Morning on EnquiryBlogger at The Open University, which also formed part of the annual Computers and Learning Research Group (CALRG) conference.

We covered the rationale for EnquiryBlogger, its links with Authentic Inquiry and with learning dispositions, and reported on initial trials with Masters students at Bristol University and with primary school pupils at Bushfield School, Milton Keynes.

http://podcast-api.open.ac.uk/play/0KZvm8VxVLmSv_j4TWF2ew

Ienquiryblogger-analytics-calrg2012

View more presentations from Simon Buckingham Shum

SoLAR Summit

Following the LAK12 conference, I was invited to join the learning analytics summit organised by SoLAR. The Society for Learning Analytic Research (SoLAR) is taking the lead in the field by organising the LAK conferences and working to coordinate research plans and new initiatives.
The summit brought together 60 people, including foundations, grant agencies, organisations, university leaders, industry, experts and researchers.
Topics included:

• Why are analytics important in education?
• What are analytics from your perspective?
• Open learning analytics
• Learning analytics in the future

I was invited to give a talk based on my technical report: ‘The State of Learning Analytics in 2012: A Review and Future Challenges‘

SOLAR – learning analytics, the state of the art

View more PowerPoint from Rebecca Ferguson

Abstract

Learning analytics is a significant area of technology‐enhanced learning that has emerged during the last decade. This presentation begins with an examination of the technological, educational and political factors that have driven the development of analytics in educational settings, and the challenges posed by these factors. It goes on to summarise the emergence of learning analytics, including their origins in the 20th century, the development of data‐driven analytics, the rise of learning-focused perspectives and the influence of national economic concerns. Finally, it sets out the current state of research, and identifies a series of future challenges.

Social Learning Analytics: Five Approaches

2 May, and my main LAK12 conference paper on social learning analytics - co-authored with Simon Buckingham Shum.

This provides a context for the learning analytics research and development work that we are currently carrying out on SocialLearn.

Social learning analytics: LAK 2012

View more PowerPoint from Rebecca Ferguson

Video of the presentation.

Abstract

This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The paper introduces the broad rationale for SLA by reviewing some of the key drivers that make social learning so important today. Five forms of SLA are identified, including those which are inherently social, and others which have social dimensions. The paper goes on to describe early work towards implementing these analytics on SocialLearn, an online learning space in use at the UK’s Open University, and the challenges that this is raising. This work takes an iterative approach to analytics, encouraging learners to respond to and help to shape not only the analytics but also their associated recommendations.

Slideshare stats

Slideshare emailed me recently to point out that my presentations have had 10,000 views.

It’s difficult to know how seriously to take this figure – after all, many of these ‘views’ may be people or bots browsing through and not reading anything. Perhaps more meaningful is the fact that I have currently had my presentations and documents downloaded 117 times. The reality probably lies somewhere between the two – but 117 to 10,000 is a very wide ballpark. Compared with publication in a journal with an impact factor of 1 or 2, though, either one is looking good.

Mentoring Analytics

In Vancouver for LAK 2012, the second Learning Analytics and Knowledge conference. Lots of good representation from the Open University: Simon Buckingham Shum taking the lead as one of the programme chairs, Doug Clow joining the panel on building a data governance model for learning analytics, Fenella Galpin and Sharon Slade leading a workshop on the ethics of labelling students.

On 1 May I presented ‘Exploring Qualitative Analytics for E-Mentoring Relationships Building in an Online Social Learning Environment’ I was third author on this paper, with the lead taken by Haiming Liu and Ronald Macintyre.

Watch the presentation video.

Mentoring analytics LAK 2012

View more PowerPoint fromRebecca Ferguson

Abstract

The language of mentoring has become established within the workplace and has gained ground within education. As work-based education moves online, we see an increased use of what is termed ‘e-mentoring’. In this paper we identify some of the challenges of forming and supporting mentoring relationships virtually, and explore the solutions afforded by online social learning and Web 2.0. Based on a conceptualisation of learning network theory derived from the literature and from qualitative learning analytics, we propose that an e-mentoring relationships is mediated by a connection with or through a person or learning objects. We provide an example to illustrate how this might work.