Stop! Slow down and read this, marker

Standard

Whilst thinking for a topic for my blog this week, I decided I’d like to talk about something I’m passionate about. In a recent course rep meeting with members of staff, we were discussing the impact and accuracy of grades. This discussion made me realise that grades are a key factor in determining your future life choices. Your grades at school determine if you can pursue your interest at degree level, and your grades at university can determine the likelihood of achieving your dream job. Therefore over the upcoming weeks I will talk about whether grading is the most effective method for assessing students, and how grades impact students in a multitude of ways. However, as suggested by Boud (1995, as cited in Yorke, Bridges & Woolf, 2000), “Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition, if they want to graduate) escape the effects of poor assessment”. Grading is often a complex process, and the marker will have an impact on the grade before the grade has an impact on the student. Students often assume their grade is solely reflective of their performance, when in reality a number of other factors can influence the grade they are given.  

Lecturers who mark your work can be subjected to influence, just like any other individual. Personality traits, energy, time available, interest and experience of marking and the subject area can all affect a marker’s accuracy (Suto & Nadas, 2008; Yorke, Bridges & Woolf, 2000). In addition to this, another students’ work can influence an individual’s mark. Markers often struggle to treat each piece of work separately, and comparisons are often made to the previous assignment (Crisp, 2010). For example, if the assignment read before yours was outstanding, by comparison your work may not be graded as highly. In this respect, it seems that marking is vulnerable to subjectivity.

Marking assignments at University isn’t the only thing lecturers do, as they are often involved in lecturing, research and pastoral care. As a result, a team of markers, including postgraduate students, are often used for a quicker marking turnover (Bird & Yucel, 2010). This may be time-effective, but the inter-rater reliability between markers is often subsequently questioned (Suto & Nadas, 2008; Bird & Yucel, 2010). It is important for markers to have a mutual understanding of what is expected from an assignment, however self-reports have suggested they often use different judgement processes when marking. Some markers have learnt their assessment techniques from colleagues, others from how they were graded at school, or through training within their educational institution (Yorke, Bridges & Woolf, 2000).

There are methods for tackling this lack of inter-rater reliability. Bird & Yucel (2010) suggested an “integrated model of assessment programme”, which uses marking schemes, moderation sessions and examples of graded work. This particular programme resulted in a decrease of marker variability, and time taken to mark work was reduced. It is often assumed that there is a speed-accuracy trade off with marking, however it has been found that practice effects allow markers to assess the work more automatically, and occasionally more accurately (Nadas & Suto, 2010). Thus practice makes perfect, and inexperienced postgraduates in particular should be provided ample opportunity to practice their assessing ability.

Clear marking schemes will increase reliability amongst markers. However it is important to ensure this focus on reliability isn’t at the expense of validity. A valid assessment tests if students have an in-depth knowledge of the area, and education minister Michael Gove has supported this by suggesting our focus should shift from reliability to validity to truly stretch students’ abilities (Ofqual, 2013). Also, if marking schemes are expected to be followed strictly, what happens when a student produces something novel? Will their creativity be inhibited by the need to follow guidelines? I will discuss the effect grading has on creativity in a later blog.

Research has also investigated the scope of marks given for subjective assignments. In a subject such as Maths, there is a definitive answer, thus the full range of marks are often given. However in a subject such as Psychology where there is often no wrong or right answer, grades are often restricted to the middle quartile. The top marks (e.g. A*) are expected to be used for a ‘perfect’ answer, however due to subjective marking, ‘perfect’ is almost impossible to achieve (Yorke, Bridges & Woolf, 2000).

Some educators believe inaccuracy between markers and subjectivity is such a problem, that alternative methods have been suggested. For example automated assessment programmes have been created, that can be used to assess essay answers (Valenti, Neri & Cucchiarelli, 2003). This method removes the subjective element to human marking, however would a student’s motivation be impacted if the essay they slaved over for days was never read? Inevitably, computers cannot be inspired and entertained by an essay like a marker can (Rosa, 2013). This is a fascinating, controversial method which has sparked some debate amongst educators, and I will explore this issue further in next week’s post.

From this research it is clear that markers play a vital role in students’ grades. Regardless of training and strict criteria for markers, some human error is inevitable. Thus it is important education aims to minimise this error, even when they cannot eliminate it. Inevitably, subjectivity is what makes us human, and allows students to take a novel approach on a question. Our markers aren’t machines; therefore it would be illogical and suppressing on both the student and marker to have an idealism of completely objective marking. So after you have read this blog, how are you going to assess it? Will you follow a strict criteria of how an assignment should be structured, or allow novel ideas and subjectivity to suppress the guidelines?

Advertisements

7 responses »

  1. This is definitley a topic which needs close attention and you have raised some good key issues in your blog. It appears that there is a gap in knowledge when it comes to grading in the way that students often dont know the reasoning behind the grades they are assigned; feedback is lacking. I am without doubt that many students have applied all thier efforts to a task only for thier confidence to be knocked down by receiving a poor grade and worst of all, they often do not know why.

    This point is reitterated in a study by Sadler and Good (2006) which compared grades given by a teacher to a student and the grades that student either gave themselves or was given by a peer. When grading their peers,students were often found to assign lower scores than their teachers to the higher performing students, this demonstrates my previous point about the gap in knowledge between student and teacher. Furthmore, students who graded themselves assigned themselves higher grades than the teacher did.However, despite these defferences, research suggests that self grading can actually be valuable to students learning.

    Self grading results in much quicker feedback than if a teacher were to take the work, mark it then return it a few days later. The student can see first hand what mistakes they are making but also what they are doing well (McLeod, 2001). A student’s understanding of a topic can also be deepend by judging the correctness of thier own answers (Boud, 1989). In addition, self grading encourages the student to spend more time evaluating thier own work (Darling-Hammond, Ancess, & Faulk, 1995) and peers can often offer more time and more detailed feedback than a teacher (Weaver & Cotrell, 1986).

    According to Edwards (2007) Self grading has another major benefit. It demystifies grading, in other words it closes the gap in knowledge between the grade the teacher gives and the grade the student expects or believes they deserve. Therefore, it allows students to feel they have more control over thier own evaluation of thier work.

    Ofcourse, it is no absolute substitute for teacher to student grading but the research definitley highlights its benefits.

    References:

    Boud, D. (1989). The role of self-assessment in student grading. Assessment and Evaluation in Higher
    Education, 14, 20–30.

    Darling-Hammond, L., Ancess, J., & Faulk, B. (1995). Authentic assessment in action: Studies of
    schools and students at work. New York: Teachers College Press.

    Edwards, N. M. (2007). Student self-grading in social statistics. College Teaching, 55 (2), 72-76.

    McLeod, A. (2001). In lieu of tests. Retrieved August 4, 2005, from National Teaching and Learning
    Forum’s Frequently Asked Questions at http://www.ntlf.com/html/lib/faq/al-ntlf.htm

    Sadler, P. and Good, E. (2006) The Impact of Self- and Peer-Grading on Student Learning,
    Educational Assessment, 11(1), 1–31

    Weaver, R. L.,&Cotrell, H.W. (1986). Peer evaluation:Acase study. Innovative Higher Education, 11,
    25–39.

    • Sadler and Good (2006) also found evidence that children who mark their own work improved significantly when taking the same test again a week later, unannounced. If we can get students to view their own work after taking a test, mark it and criticise it, it will give them the opportunity to learn from their mistakes. Instead of just giving them a grade with no or little explanation for their grade, we can allow them to see first hand where they can improve, hopefully giving them a better chance in the future. Self-graded work also had a stronger correlation (0.976) with teacher grades than peer-graded work (0.905), meaning self marking is very valid form of marking.

      The study then goes on to discuss the future possibilities of peer- grading, as there is still a strong correlation in teacher and peer-grading (Sadler & Good, 2006). In order to counteract the slight difference in student and teacher grading, it may be possible to introduce some sort of moderating procedure (Boud, 1989). It may be more time saving for students to mark each others work, with the oversight of their teacher. So the students will save the teacher some time with the marking, and students will also gain the advantage of having two people mark their work (peers and students).

      Self marking appears to be a very strong and possibly superior alternative to teacher marking, but peer-marking is still very effective and can be applicable with teacher oversight.

      Boud, D. (1989). The role of self-assessment in student grading. Assessment and Evaluation in Higher Education, 14, 20–30.

      Sadler, P. M., & Good, E. (2006). The impact of self-and peer-grading on student learning. Educational assessment, 11(1), 1-31.

  2. Very interesting blog and I agree that there are many factors which affect how a piece of work is graded, I have often found myself wondering why I got a particular mark for a piece of work, Subjectivity in marking is almost inevitable, as different people would have a different view of the subject, and ultimately the choices made, even in a tight set of criteria are ultimately due to a human judgement, which as we all know can be flawed. One way in which this is addressed is though second making, (which is common in most universities) and external evaluation where an outside party grades a random selection of work to see if this matches the grades given by the tutor. However even this system is not fool proof, since universities are businesses they are also driven by the need to ‘process’ students through their courses and with league tables showing which are the highest achievers, the academic judgment on grade boundaries may end up being overruled by the need to keep published results high. Sadler, 2009)

    Does grading get in the way of our learning? I would say yes because we write to what we believe our teachers want to read, mould our learning to get the correct answers on the test, even this module is not immune, I have spent ages thinking about what to write on my blogs and comments to get the best grade… ideally we would get rid of grades and learn for the fun of it. Sadly the administration would not let us do this, Denis Rancourt a physics professor was dismissed from his post for giving the entire class an A+ grade, because he believed that stress over grades got in the way of learning. (Hill, 2011) so what is the answer? If we get rid of grades how do we measure achievement and learning?

    References:

    Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159-179.

    Hill, D. (2011). A contentious triangle: Grading and academic freedom in the academy. Higher Education Quarterly, 65(1), 3-11.

  3. Grades are becoming a powerful social force where they no longer just a form of feedback from teacher to student (Rediehs, 2009) In fact grades now link to parental/friendship approval; scholarship’s and participation in extra-curricular activities. Therefore an increased amount of pressure is being placed on students, in turn impacting an individual’s self-esteem. Crocker et al (2003) found when a student received a good, satisfactory grade an increase in self-esteem was measured; a bad grade led to a decreased amount of self-esteem. If education is going to rely on grades, it needs to revert back to simply feedback format. Due to the increased amount of focus on grades and no longer being personal, it can lead to detrimental effects for students.
    References:
    Crocker, J., Karpinsky, A., Quinn, D. M., & Chase, S. K. (2003) When graduates determine self-worth: consequences of contingent self-worth for male and female engineering and psychology majors. Journal of Personality and Social Psychology, 85(3), 507-516.
    Rediehs, L. (2009). Trust and Distrust: The problem with Traditional Grading. Retrieved on 27th October 2013 from: http://it.stlawu.edu/~lrediehs/grading_files/problem.htm

  4. Another aspect to consider is that of peer assessment. Group work when applied to assessment is often only used to rate the input given by each individual in a group project (Goldfinch, 1994). However, the students themselves could be used to assess their peers. Post-graduate students are already used to help alleviate heavy workloads (Bird & Yucel, 2010), so it would only be a small step to involve the students directly.

    In an investigation involving 210 undergraduate students, Freeman (2010) found there was no significant difference between the grades given by students and given by teachers. It is suggested that peer assessors have less skill at efficient assessment, however they are likely to devote more time. This means they are likely to produce an assessment that is equally reliable and valid to that of a teacher (Topping, 2009). Due to this, peer assessment is a tool that could be implemented by educators to reduce the workload on teachers.

    Goldfinch, J. (1994). Further developments in peer assessment of group projects. Assessment & Evaluation in Higher Education, 19(1), 29-35.

    Bird, F., & Yucel, R. (2010). Building sustainable expertise in marking: integrating the moderation of first year assessment. ATN Assessment conference University of Technology Sydney. http://www. iml. uts. edu. au/atnassessment/poster. html.

    Freeman, M. (1995). Peer assessment by groups of group work. Assessment & Evaluation in Higher Education, 20(3), 289-300.

    Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20-27.

  5. I recently came across this spoken word video from Suli Breaks. I do not necessarily agree with everything he says in this song but I am a fan of spoken word and rap and this is related to science of education and marks.

  6. One of the things that I dislike about University is that we don’t get to see our exams after they’ve been marked, meaning that we never really see where we’ve done wrong, to me this seems detrimental to people who need to take resits as they don’t know in which area they are lacking.

    I think it would be a good idea to have an automated marking system as human error is otherwise unavoidable. Research has shown that if 2 people sit down and mark the same paper it is almost guaranteed that the paper will get 2 different marks. This is because no matter how stringent the guidelines, people just have different expectations and different interpretations of the guidelines.

    I find it laughable that human error isn’t really taken into account too much when it has a massive impact on people’s lives, for example, I failed my A Level English twice, it turned out to just be an error with the markers, but in the end they said I’d just have to resit and hope for the best, meaning that I changed my degree and entire career mindset, because some people couldn’t mark an exam paper competently.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s