“Using Rubrics to Develop and Apply Grading Criteria” by John Bean
“As teachers, our goal is to maximize the help we give students while keeping our own workloads manageable.” –John Bean
Professor John Bean provides a comprehensive and balanced analysis of rubrics in Chapter 14 (“Using Rubric to Develop and Apply Grading Criteria”) of his book, Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking, and Active Learning in the Classroom. Overall, Bean is in favor of using rubrics in the classroom, and he advises teachers to “communicate grading criteria to students at the outset” (267), which is quite valuable advice. No matter what type of grading system you have, it is imperative to communicate that information to students at the beginning of the course.
He proceeds to explain the controversy about evaluation criteria by acknowledging that is there is a “tangle of uncertainty” surrounding grading essays because teachers do not discuss grading practices with their colleagues (268). One teacher may consider her grading policy as “universal,” while another teacher may consider it as “idiosyncratic” (268). Hence, there is an amount of “subjective judgments” in terms of grading writing since a random set of teachers can assign a paper different grades. As shown in the Paul Diedrich (1974) experiment, where he collected three hundred essays written by first-year students at three different universities and had them graded by fifty-three professionals in six different occupational fields, he concluded that they were a range of grades with “no essay receiving less than five different grades” (268). Five different grades for the same paper is quite a discrepancy. What would have been the results if he had fifty-three English instructors grading the papers? There may have been a narrower range of grades. Nonetheless, Diedrich made an important point: grading essays is not a precise science because there is a “wide disagreement” about constitutes good writing. According to the fifty-three professionals across disciplines, What is good writing? Using factor analysis, Diedrich identifies “five criteria of good writing such as ideas, organization, sentence structure, wording, and flavor (or voice)” and was able to train evaluators to look at writing based on these criteria; his research was instrumental in creating “norming sessions” that helped train teachers departmental or cross-disciplinary to create a “communal standard” of evaluating writing (269). Bean calls this “pro-rubric” strand, which is used to increase reliability in evaluating writing (269).
While acknowledging that not everyone is pro-rubric, Bean provides an overview of different kinds of rubrics before further explaining the controversy surrounding rubrics.
Different flavors of rubrics such as analytic, holistic, generic, task-specific (or primary trait), grid, or non-grid. It is up to the teacher to decide which rubric she wants to use in her classroom. This freedom of choice is an important buy-in for teachers since they can customize the rubrics based on their grading philosophies and adjust the rubrics based on the levels of their students. Customized rubrics are crucial when evaluating the writing of Special Education and ESL students’ writings. If a rubric places too much emphasis on grammar and diction, an ESL student may earn a low score. A steady diet of low scores send a demoralizing message to the students that they are not good writers. Therefore, the option of choice is crucial — especially within multicultural classrooms and with diverse writers.
Analytic versus Holistic. In an analytic rubric, there are separate scores for each criterion such as ideas, organization, syntax, diction, and voice. Some instructors prefer analytic rubrics because they are more detailed and concrete — and coupled with the instructor’s comments — provide more substantive feedback. In a holistic rubric, there is one score reflecting all the criteria. Some professors prefer holistic rubrics because “philosophically writing cannot be broken down to components” (270). Overall, analytic rubrics provide more detailed feedback for revisions; whereas, holistic evaluation is faster and more suited for rapidly scoring essays.
Analytic versus Holistic Rubric Chart
Generic Versus Task-Specific. Both analytic and holistic rubrics can be either general or specific. General rubrics can be applied to most writing assignments, while specific rubrics are particular to that one writing assignment. Bean prefers the task-specific rubric. However, the general rubric is useful as a starting point for new writing instructors or for writing tutors since the rubric conveys the department’s general grading expectations. In my English department, we are given a general rubric similar to Figure 14.1 (General Writing Rubric Using Analytic Method) and are encouraged to customize and adjust it based on individual assignments.
Different Methods of Describing Performance Level. Bean recommends “a simplified step-down approach to specify different levels of achievement for each criterion” (276), as shown: Meets Most Criteria Meets Some Criteria Meets Few Criteria
Grids Versus No Grids. Some rubrics are gridless. Instead of specific descriptors for each criterion, the instructor asks questions such as “Does the introduction effectively present the issue and the thesis, while evoking reader interest (10 points)?” (Figure 14.5) Gridless rubrics are helpful during peer-revision conferences since students can see what the instructor values in their writing. The problem with gridless rubrics is that it is difficult to assign a score within the range of 10 points. Do I give the introduction a 7? An 8? A 5? Scales can be problematic.
After Bean provides an overview of the different types of rubrics, he summarizes the controversies about rubrics; then he provides his own approach to rubrics.
Controversies About Rubrics. Composition researcher Bob Broad raises the concern of the “false notion of the universal reader” in that students believe that there is “agreed-upon standard for writing” (277). Broad argues that students need to grapple with the fact that “readers read in different ways” (277). In response to Broad, instructors need to explain to the students that there is no universal standard for good writing and that there may be a need within a department to achieve reliability in grading by conducting norming sessions so that instructors’ grading of essays is not too strict nor too lenient. Students appreciate consistency in grading essays. Critics also argue that rubrics oversimplify the writing process. In response to this criticism, Crystal Sands in “Rubrics Oversimplify the Writing Process” acknowledges that rubrics may not be able to capture the complexity of writing but does not mean that we should dismiss them (268). She argues that rubrics coupled with the instructor’s written feedback are effective. She makes a valid point by arguing that rubrics should not replace marginal notes, end comments, and other written commentary.
The Problem of the Generic Rubric. The one-size-fits-all does not work, which was proven in a cross-disciplinary research project conducted by Thaiss and Zawacki (2006), where they studied the criteria when grading students, and they found that “generic rubrics can’t accommodate the rhetorical contexts of different disciplines and genres” (279). Therefore, generic rubrics need to be revised to meet the rhetorical context of their discipline. To illustrate, a history teacher may be more concerned with historical accuracy than an English teacher.
The Problem of Implied Precision. Writing is art, and by reducing it to grids, categories, and scales implies a sense of precision. However, that is not the case. Good writing, like beauty, is elusive. We cannot assess writing, art, or beauty with precision based on rubrics, no matter how good they are.
John Bean’s Left-Brain, Right-Brain Approach to Rubrics. Bean selects a task-specific, analytic rubric with a simplified step-down approach. He reads over a small sample of papers to get a range and to assess problems in the students’ writing. Then he gives the paper a holistic letter grade based on overall impression, which is his right brain score. He writes end comments and then makes recommendations for revisions. Then he staples a copy of the rubric and circles a score based on each criterion. The sum of the score provides him with a left-brain score. This process may seem time-consuming to use both holistic and analytic evaluations; however, Bean asserts that this left brain, right brain approach leads to a “fairer and more thoughtful grades,” and the payoff is that “students never challenge the numbers” (281). Overall, rubrics need to clearly communicate to the students the instructor’s judgment of their writing and offer recommendations to improve it. Rubrics serve as a bridge between the instructor’s and student’s expectations.
Finding What Works for You –From Simple to Elaborate Rubrics. There is an assumption that elaborate rubrics are more specific and better for providing concrete feedback. To illustrate, the College Board streamlined its original 9-point AP English Language and Composition Holistic Rubric to a simplified 6-point Analytic Rubric. Please see the video on the change to the AP English Language and Composition rubric: https://images.app.goo.gl/g6Z9nfV9wjG158mYA. The shift from an elaborate 9-point rubric to a less elaborate 6-point rubric resulted in a streamlined rubric that is more concrete and more student-friendly. I am a fan of simple rubrics, and in my 10th grade English classes, I use single-point rubrics. Please see figure below:
Conducting a Group Norming Session. After scoring the essays, Bean recommends a norming session where instructors can discuss writing scores and identify problems in students’ papers. Thus, having a conversation about grading writing is an integral part of any English department. It is also important to involve students in this conversation.
My Final Thoughts. With increased class sizes, it is not feasible to provide marginal notes, end comments, and extensive commentary on students’ papers. Therefore, rubrics — along with brief end comments and suggestions for revision — are effective tools in providing students with meaningful feedback and suggestions to help guide them through the revision process. It is also beneficial to involve students in the process of creating rubrics so that their voices and opinions can be heard.
- Are you always given grading requirements and/or rubrics at the beginning of your graduate courses?
- Have you ever received a paper with just a grade and minimal comments? Based on these minimal comments, did you learn how to improve for the next paper?
- What is the difference between a rubric and a grading guide?
- Are you in favor of rubrics? Why or why not?
- Predict your grade for this course. Provide a rationale for the grade.