Virtual Ethnography on Peer Assessment in Coursera Gamification MOOC

online-community

Ethnography

Tried to do the ethnography in Storify but it meant uploading all the headings and screen shots to flickr, which was incredibly fiddly and time-consuming. The pdf format, while not what I originally wanted, as least does the job!

8 thoughts on “Virtual Ethnography on Peer Assessment in Coursera Gamification MOOC”

  1. I really liked some of your initial comments here about the potential to be responsive and adaptive with ethnographic methods. For me, it seems important not assume we know what a ‘community’ *is* before we approach this kind of task. What we find through ethnography need not then be compared to our prescribed notions of what it is we are researching – although that may be a lot harder than it sounds!

    Focussing on the peer assessment was a good choice I think, its a really interest element of the MOOC. I’d agree though that your choice not to participate will have limited what you can study here, and getting involved might have provided you with richer material to consider.

    It seemed to me that your focus was more on the ‘issues’ or problems created by the peer assessment activity than on the community interactions around them. This seemed particularly so in your conclusion, which was more about how to improve future peer assessment rather than summarising findings about the ‘community’ itself. So my questions would be around the significance of the various issues to the sense of MOOC community. Did you feel the problems divided people, or brought them together. Was there a rift between students and instructors? Something I thought of when reading your snapshat was the ‘number of times read’ data that you get in the Coursera forums. Was this high on the threads mentioned? What might that signify? Did you get a sense that these issues were of high or low importance to the broader course group?

  2. Clare, you made analysis of peer assessment look easy, and I know that it was not. I would like to learn how you got such clean, crisp screen shots of the MOOC discussions. I guess that you also had to revert to a Plan B when Plan A (Storify) proved ‘troublesome.’ Bhatti had some nifty advice that the ethnographer needs to be “adaptable” and “to cultivate the ability to deal with uncertainty and self-doubt.” I got the impression that you made a conscious decision to assume a ‘non-participant observation’ approach. I would be interested if I understood that correctly, or if you also did peer assessment? or otherwise participate in the MOOC coursework. If not, how did you come to that decision to jsut observe, rather than also participate. Cheers, PJ

  3. Clare, really interesting subject. I have never had to peer assess in MOOC before and the idea might make me think twice depending how much time I had. Some strong comments as well, you can feel the emotion in the text.

  4. Hi Clare,

    Great ethnography. I agree with Dr Knox, that making the focal point of your ethnography on peer assessment was a great idea. I find it interesting that all 3 of the assignments in the MOOC are peer assessment. You suggest that the feedback be made mandatory but was there any evidence of incentives for peer assessing?

  5. Hi Jeremy, PJ, Em and Ben,

    Thanks for your feedback and questions.

    I had initially set out to explore how community is built on such a big MOOC but then for some reason started to focus on peer assessment and the whole process of being an ethnographer.

    PJ – yes I had made a conscious decision not to participate but that was when my aim was to study the community aspect. It would have been so much more logical to have participated when I shifted my focus as I would have not only been able to give peer feedback but to receive it and join in the discussions.

    Ben – I have never been involved in peer assessment either and this would have been a great opportunity!

    PJ – you’re right about having to be adaptable but at the same time I think we have to adjust our approach according to any changes we decide to make.

    Em – I guess the best incentive is giving constructive written feedback as you would wish to receive it!

    Jeremy – there was a strong sense of community right from the beginning of the course. One participant started a thread asking where people where and created a geolocator map. 199 people posted in the thread, which had 1774 views. The community also quickly divided itself into study groups – mainly according to nationality – and set themselves up on social media. The issues with feedback brought people together and not only did participants help translate assignments not written in English, but gave each other feedback when they were not satisfied with the feedback they had received. There was definitely a rift between the students and the instructors as the instructors didn’t really listen to what the students were asking. The community were aware of this and worked together to provide their own solutions.

    I’m really disappointed that I didn’t bring the ethnography together at the end with a focus on community and am annoyed with myself that I had assumed that Storify would be a quick copy and paste job.

    I’ve learnt several very important lessons from this experience, which will hopefully make me a better ethnographer in the future! :)

  6. Hi Claire, late comment here so apologies. This was an interesting focus (peer assessment) and one which is highly relevant for me within my professional field, the ‘L2′ issue was highly illuminating, and issues around use of language citerea, as was the absence of incorporating the peer feedback into the final assessment, or any kind of written feedback. There are lessons for me to take away here, in the event of designing on -line community based peer feedback. Thanks!

  7. Hi Clare
    I found this really interesting, especially as I was interested in how peer assessment worked out in the MOOC I was looking at, but decided on another focus in the end.

    I’m really interested by the language aspect and I wonder how many learners were constrained by the use of a non-native language. Think this really highlights some of the issues around MOOCs we touched on in the IDEl course, around MOOCs and globalisation.
    best wishes
    Nick

  8. Hi Clare,
    So many differences with the peer assessment on your mooc:

    The Metaliteracy Mooc took the opposite stance regarding language, grammar and spelling and incorporated it into the assessment criteria. This caused some debate on the forums, for example:

    “I would agree, Gerd. One of my peer assessments is for someone for whom English is not their native language as evidenced by their answers. I was ambivalent in answering the question on grammatical and spelling correctness – I understood what my peer was trying to say and thought they made good points. But I am having a hard time deducting a point from a person’s score because English is not their primary language.

    I, too, am taking my first MOOC and am really excited about working with this global group of peers – and I hope this creates a very important discussion on how to learn and collaborate in an English language centric MOOC where many who have joined do not have the same level of English language grammar and spelling skills as the course designers and professors. Is grammar and spelling a key learning objective that needs to be graded?”

    And,


    “It’s a little bit frustrating to evaluate the grammar since I’m not an english native speaker. And even most of us understand what’s the intention of the cuestion, the answer it’s very clear… you’re asking about the grammar… and since right now I know I’ll be lack of quality…”

    I guess this shows that you really can’t please all of the people all of the time.

Leave a Reply to Em Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>