I’ve been a massive fan of MOOCs ever since they started, and have personally leveraged the opportunity to the fullest (though it did take some time to figure out how to complete a course, versus starting one!). Students get to learn from the best educators in a variety of different subjects (sometimes they (the teachers) are also from the top universities) for free or for a nominal cost. What’s not to like?

These days Universities are moving from offering individual courses, or ‘micro-master’s’ programs to offering full-fledged online Masters Degrees. These are equivalent to what one would obtain by physically attending the college (the certificate does not distinguish between online and in-person), but this also requires a lot more rigor in evaluating students who attend online, a point that I come back to below. 

I’ve seen quite a transition from a video recording of the teacher with slides/monitor on the side to attempts using Microsoft Kinect. And there’s the disconcerting trend where the instructor writes on a board facing us, but we see things the right way (I haven’t quite figured out how that is done). However, at its core, the model is unchanged: the lecturer lectures and we watch them online, rather than being physically present in the classroom. The watching is done at our convenience and pace, and we can rewind and rewatch what we want. MOOCs are making knowledge accessible, but I don’t think we’ve actually used technology to develop a new way of instruction.

I’d like to focus on a few aspects of scale — the ‘Massive’ part of MOOCs. How do these courses deal with the sheer number of students enrolled, and how do they impart the best possible instruction?

The first, obvious, observation is that there are numerous platforms available (edX, Coursera, Udemy, etc.) that take care of the bureaucratic stuff: accounts, logins, billing, tracking, etc.) and the technology (videos, tests, etc.), so these don’t have to be redeveloped for each course. And yes, there are no scaling limits here. 

The second aspect of scale is that of content delivery. Lectures are recorded and available at all times. We don’t need to have all students in the same location (or online) at the same time. Since the model is different from a physical classroom, the limitations of a physical classroom are dealt away with. This is a solved problem.

The third aspect of scale is where things get interesting, and is that of student evaluation. The original MOOCs (and I suspect, most of the current generation) used to limit themselves to quizzes that consisted of multiple-choice questions. Some were embedded in the videos. These are, of course, easy to grade automatically. Others allow text boxes to enter (numerical) values, and sometimes $\LaTeX$ formatted answers, but these take a bit of getting used to. I have also seen some that require essay-type answers, these are peer evaluation, which I thought was a good way to address the problems of scale, but this seems to have gone down – a failed experiment, perhaps?

Back to the degree-granting MOOCs (can I call them that?): they decided the only way of evaluating students was the traditional way of evaluating students, namely, have them write an exam they way they normally would. So…

students turn on their webcams, rotate them around the room to prove that there is nobody around helping them, and keep these on and focused on them for the duration of the test. I believe the microphone is kept on as well. And there is a person on the other end of the connection keeping an eye on the student! Holy 19th-century non-scalable solutions, Batman!!!

This is a great opportunity for an unskilled proctor who otherwise would not have gainful employment. I have no idea how much they make, and what trauma they go through, having to spend their waking hours watching students bent over exam books.

The final aspect of scaling is that when we have essay-type questions (say, ‘develop an algorithm to do x’), a human grader is going through these and assigning scores. Even if we overload a poor TA with 100 exams, the larger the course, the correspondingly larger staff needed to manage just the grading. 

I think we’ve been going about this the wrong way. We’ve been trying to replicate what we do offline to the online world, which leads to issues like the above. How might we look at this from a fresh perspective? Glad you asked!

• Goal: Provide feedback to the student on which areas they are doing well and areas for improvement. This is (or should be) the only reason to conduct exams, but that is fodder for another article. A secondary goal is to provide a normalized score that accurately captures the student’s capabilities in the subject. Ultimately, we would like to measure how much a student has learnt.
• The means of measurement have to be scalable. It should work across thousands or tens of thousands of students. 
• It should not be possible (too restrictive? Maybe it should be very difficult) to game the measurement

We can think about how technology can solve these issues once we have clarity on what we are trying to solve!

Thoughts?