Evaluating Rubrics
Why should you evaluate the rubrics you use?
Once my rubrics are developed, it is important that I evaluate them, to ensure they are reliable and valid.
As I learned in the module, I will also want to ensure that my rubrics provide more than one form of feedback. I can see where this might be easily overlooked in the development process so I was grateful for that reminder.
The term metarubric was new to me, but I do recognize the value it holds for evaluating my rubrics. What I consider a quality rubric may not be as “quality†as it needs to be and the metarubric would make this clear. I need to ensure the content is complete, that the criteria are clearly defined, the rubric is manageable (or it may not be utilized by students), and as mentioned previously, is it reliable and valid?
This module was a good reminder for me to go back to my current rubrics and evaluate them to ensure they are the useful tool I assume them to be.
Jennifer ,
Isn't amazing that once we create something that we don't go back to review them! I think I need to check my own rubrics for my own classes. . . .
Dr. Kelly Wilkinson
To ensure that the rubrics are properly assessing what they were developed for. I think it is easy to become complacent and continue to use rubrics that could be ineffective.
Julie,
You have a great point. We must continue to review and assess our assignments and how we assess them! Complacency can be our worst enemy.
Dr. Kelly Wilkinson
You should evaluate the rubrics you use because both learning and teaching are continuous processes. I don't think there is such a thing as a "perfect rubric." It depends on the assignment, the content, and many other things. All of these factors typically change, or should change, as the instructor develops and revises the course materials. This includes rubrics. I know when I am grading assignments using a rubric, I often come across things I need to change, or areas that I think could be improved. I look forward to revising my rubrics, because I know that I will be that much more efficient and effective the next time around I grade that particular assignment. And then I'll find even more things that I can change!
Debra,
Great answer! Yes, you do find "holes" in your rubric when you are grading.
Dr. Kelly Wilkinson
My online teaching involves utilizing software that is continuously updated yearly. While most of the software does not change, it can affect certain projects which would mandate a change in the rubric.
Fred,
True, do you change they way you assess certain projects based on industry standards or changes in procedures or skills?
Dr. Kelly Wilkinson
Course content and objectives change as the class develops. The rubric should reflect shifting course outcomes.
I also think it, if possible, a rubric should be altered based upon the level of the course and prior knowledge of the student.
A master level course should be weighted and graded with the same rubric an entry level course does.
Additionally, should a student or class show strong mastery for a certain area, it may be benficial to exchange the rubric for another that emphasis another area to encourage improvement.
Christina,
You are correct about the changing the rubric.
Dr. Kelly Wilkinson
Rubrics should always be evaluated to ensure that they are relevant to the course that is being taught. For example, a math rubric may not be applicable to a history rubric. Additionally, if course content changes, then rubrics must also concurrently change.
Ed,
Yes. We need to know what we are assessing and if we are doing it correctly. It is good to assess our assess,ent tools. :-)
Rubrics are developed based upon the way assignments are set up at one point in time. Over time, the assignments and/or course/assignment objectives may change and would require the rubrics to change.
I have found that I change rubrics based upon the "highest" level of work that student's present in a course. As student's submit "improved" content on assignments, I review and update the rubrics to meet their challenge!
I have also found that I like change and work to improve the content/wording of rubrics. This can only happen as rubrics are tested in different scenarios and with varying level of student's assignment submissions.
Laurie
To make sure that they effectively evalute a students performace regardless of the level of competence in the work. You want all students to be graded fairly without any advantage to any one student.
Laurie,
Good point. I definitely edit rubrics on an ongoing basis. You learn as you go and the students always provide excellent feedback to help you make those edits. Thanks!
Francis,
Yes, consistency in the grading - very helpful. Thanks!
It is necesary to determine if it does what it is suppose to do. Does it contain the correct content, is it clear, is it useful, and is it reliabe/valid? These criteria need to be evaluated. Does the rubric produce the same grading results when multiple instructors grade the same assignment? If not, the rubric isn't useful, or the instructor's aren't using it. Does the rubric make grading easier for the instructor? Does the rubric make it easier for the student to understand the assignment & to learn? If not, then the rubric is not doing what it is suppose to do, and it needs to be evaluated & modified. If the assignment changes, the rubric needs to be evaluated to see if it still meets the assignment criteria or do you need to modify or make a new rubric. Also do you need to have instructors, students, or both evaluate the rubric? The contain or the grading results might answer this. If student answers are not good & grades show poor results, you may need to have students evaluate the rubric to see how they ubderstand what is needed. It may be that different instructors understand the rubric differently & grade differently. If so the rubric is not doing what you want it to do. Instructors may need to be evaluated to see what they understand from the rubric.
William,
You have provided much essential information in this post. Excellent. Thank you for your input.
Dr. Kelly Wilkinson,
The reason one should evaluate the rubrics before one uses it, is to familiarize oneself with the mechanics and to see if you agree that it is relevant to your course. This is expecially true since in my experience the rubrics is already developed by someone else. Knowledge of the rubrics can also help the instuctor teach teh criteria of the assignments.
Sincerely,
Walter
9-7-12
