Evaluating Learning
Why is it important to evaluate the process of learning in addition to the learning outcomes?
I have a basic answer for that. There is a process in manufacturing called the value chain.
raw materials--->machine work----> human work= finished products.
Let use this allegory for learning:
learning---> learning outcomes
If you don't apply the process of learning, how will you learn the outcomes?
Hi Jack,
Yes, there is a logical connection , however I would suggest that often the learning process is interrupted or diminished through external influences that are not part of the instructional design - things like pre-set testing or ineffective assignments, or teacher-driven content. Therefore, instructors are central in supporting the process throughout.
We can't always know if a student has learned anything without looking at process. A student may cheat on exam, have some one help them with a writing assignment, have someone write their writing assignment for them, etc. In terms of classroom participation using a textbook, a student may have a used book in which someone has already written the answers. And ultimately, all students come to classes already knowing about the subject at various levels. If we want to evaluate what a student has learned, or is learning, we need an evaluation system that values process rather than simply learning outcomes.
Yes, Scott. Do you have ideas of ways to do that? What sorts of evidences might we want to evaluate in order to capture the process? It is more difficult to evaluate process and it usually takes a variety of methods in order to truly see process over task.
Process is reflective of "critical thinking" in action. The end result or answer may be right but is tha due to the student copying information? Is it due to the student memorizing information?
If instructors view the process, even if the answer is wrong, the instructor is better able to help the student identify where he/she may have made an error. The instructor is truly helping the student learn how to apply what is being presented.
The process is what will sustain the student and help them become more effective in using life skills. Thus the learning outcomes become far more reaching than the "instant" answer in the class.
Yes, I agree, Glenda. It is vital that instructors are fully aware of the dynamic nature of teaching and learning and that students may take various paths to reach their conclusions or to fully grasp a concept. If we then value the process, we reinforce the importance of process of learning over simply providing an answer (not be confused with a simple answer :) ...)
You did a good job Glenda!
Your response brings the term "holistic", which has been used in this training class to mind. Anybody remember what holistic means??
John
Indeed - holistic is that inclusive approach to teaching and learning :)
I see any “Process” as a tool; and as with any tool, learning to use / apply the tool properly is essential in task completion. By not only understanding the process but being able to implement the process successfully, the student is left with a tool that may be successfully utilized in other applications.
The process must be assessed at every step to ensure consistency in the output, allow the student to troubleshoot the process when they are on their own, and better comprehend the processes capability for use in other applications.
I agree, Donald. More often than not, however, we evaluate tasks rather than process. So students could actual be involved in an excellent learning process but may not quite complete a task "correctly", therefore, they lose all points rather than receiving a percentage of the points.
I think this is one of the most important aspects to consider with all the of changes in moving toward a more student-led approach. As the course suggested, it is important to be able to come to the correct answer (if there really is one) at some point but the process if getting there is equally as critical. For me and the subjects I teach (management mainly), there is usually no one right anwser and therefore, it is the thinking process and how one assesses and analyzes the situation and what conclusion they arrive at (truly process based). It is an unfolding almost like the peeling of an onion. You make an educated suggestion and attempt an intervention of some kind and then reassess where you are at and try to figure out the next move. Layer by layer, you peel the onion and ideally arrive at a new place with a myriad of concepts and new ways of thinking, etc.
What I do is make every attempt to connect to learning outcomes to the process of learning itself and I typically use and do most everything suggested here in the training. It is good to read and review this kind of material to keep the idea in my mind that I am doing the right things. If my people leave my class unable to think for themselves and come up with intelligent solutions to unique and creative problems, then I have failed.
It is hard to get many of them to accept this method of teaching and assessment and as was suggested in the course (at first). I also have students come up after the course or later on and tell me that they "get it" and are excited about what we are doing. This is gratifying but frustrating at first when they look at me like I am lost. :-) Thanks for the thought provoking trigger.
Many old sayings and analogies come to mind with this one; "catch a fish FOR a man and he eats for a day, teach HIM to fish and he eats for a lifetime." Something along those lines anyway.
For my courses, and for those I mentor, I challenge us as instructors to figure out, what is the best learning outcome for this course? What do you want students to be able to take away from the course REGARDLESS of major? That focuses course design and activities on developing a process (or processes) that will then meet that goal(s).
It's like having a destination on a map. There may be many little side roads, short cuts, scenic routes, main highways to get there, but yet in the end, we still end up at our desired destination. Meaning there is more than one way to skin a cat! Some ways may work better than others, but the core point, is that students can access different types of processes to meet any one single outcome.
If my students cannot develop a working process with what they've learned that allows them to apply it to some aspect of their personal life or business, then I feel I've not done my job. "Book learning" is a foundation, being able to learn in a process and project based fashion, helps them bring the textbook foundation to life and apply it in the real world. It helps us better assess true learning and skill building; it allows them to process and apply information in ways that are meaningful and beneficial to them.
For example, psychotherapy is a process. There are numerous ways to help a client along in their therapeutic journey to get where they need to be. As I tell my future therapists, the client won't care how many big words you use, or how much psychobabble you can spit from memorization. They want to know- can you relate to me, right here, right now, as I am and help me get better? Teaching a subject and assessing through process based assessments is really the same thing. Which process will best serve the student to get to where they need/want to be with the subject matter.
In nursing we often describe a task, demonstrate the task, then have the learner show us how they would do it. Again, assessing the process to see if they are just mimicking what you've said, vs. really being able to understand and apply what you've taught them, is critical to helping the patient stay healthy and the medical practitioner work effectively and efficiently.
My nephew is autistic. One of the hallmarks of autism is the inability to predict the consequences or social responses of others to their actions. So he may demonstrate or parrot something back to me, but that by no means indicates he has truly "learned" what to do or how to apply something I've tried to teach him.
Given the same circumstance, on a different day, he may simply repeat and/or mimic what he "thinks" is appropriate from our previous conversation (if he even remembers it, lol), vs. really knowing that what he is doing, is the right thing for that situation.
As Mischel says, one can repeat an action, but that doesn't mean they know how to apply the action in different circumstances, or even what it means to use it in a purposeful, problem solving way. Evaluating the process brings us closer to accurately measuring whether or not what we've covered is now a part of a new skill set that can be used in new and creative ways depending on the sitation. Being able to apply what they've learned in the "real world," is always one of my main goals for the students I teach.
Being a QA/QC Manager myself for a company, it's my job to develop a quality management system that promotes continuous improvement. The basis of this is a simple, fundamental four-step process called "Plan-Do-Check-Act" developed by quality guru, W.E. Deming. For any given process used in developing a product, 1) the process is planned, then 2) the process is done, then 3) the process is checked, then 4) action is taken to correct it.
In terms of the learning process, the steps are very similar...the process is planned, then performed, then it should be evaluated, then acted upon to improve upon it, then the cycle perpetuates. The outcomes should be used as a tool to evaluate how to do it better the next time. I do this with my online courses already.
Good points, Donald. I agree that the learning outcomes must drive the process...I think, however, looking at the skills that students develop while using new technology and the potential skills that could be developed, we may have to rethink and redirect some of the learning outcomes to be more holistic in concept.
Thank you, Theresa. The problem with evaluating the process is that it takes longer and requires more detailed attention from the instructor. In larger size classes, this can be a problem, so instructors should set up systems of evaluation that do not totally depend on them. That is, various forms of evaluation along the way, so that the process can be truly evaluated and not just from the perspective of "the right answer". Great discussion!
Yes, Bryan, and if this approach you describe is applied in any subject area, more students would feel empowered and not feel excluded from a subject area. For example, so many of us grew up feeling we "could't do math" rather than we couldn't find the right answer. The process in getting there required strengthening rather than so much focus on the end result.
Outcomes are transitory; the process endures. Since new knowledge and new technology is being generated at a very, very fast pace; whatever the outcomes of our training programs, our students will need to adjust to the demand for capabilities, skills and knowledge. Yes, I (and the company) hope that when I teach a student to maintain that tool, e.g., an induction furnace, I am very interested that they can diagnose and correct problems. However, next year that furnace will have a new component or be replaced with a newer model. If I have aided the student in learning the process of diagnosing and correcting problems with equipment in general, they will be able to work with the new system faster...ideally, training themselves in how this new system is different and how that affects their maintenance processes.
Excellent points, Thomas. Indeed the process of teaching and learning is dynamic and as such does not end with the immediate confines of whatever course we happen to be teaching. It is more important that students learn how to learn and how to continue to learn - new technology does add even more pressure with constant changes and developments.
Evaluating the process helps to deliver on the outcomes. I have often struggled with a number of students who were not meeting the outcome, whether it's a specific grade, or grasping of key knowledge, etc. Outcomes or expectations don't necessarily need to be changed. If outcomes are not being met, then somewhere in the process needs to be improved.
