Evaluation
When we finally have the prototype of our program up and running, I am planning on having a graduated student help with the evaluation. This person would have taken the program in class and so would be familiar with the program. Do you see any downfalls with this? From a learner's point of view, I am hopeful that they would recognize what is missing as opposed to someone who does not know the material.
Suzanne:
I do not see a problem with this kind of evaluation once the purpose of the evaluation is not to determine the achievement of the learning outcomes. If it is, the graduated student will, as a minimum, not be representative of the student population for which you are developing the course.
I do not follow the logic of the graduate student helping to identify what is missing from the course. If this course was designed from a Course Design Specifications generated by following sound course design and development principles, then nothing should be missing. At least that is my opinion. Moreover, your isntitution must have provided the Curriculum, a Syllabus or at least statements of the desired learning outcomes. The Instructor is the best person to determine the content domain in terms of relevance and adequacy; not the graduated student.
Satrohan
A graduate might make assumptions about the course as they have already taken the equivalent of the course. An alternative would be to use students that have not taken and will not take the course evaluate it. I say this so that you can get a true picture of the course dynamics. Then use students from the program that have not taken the course yet.
You could incorporate graduates in a survey to see what they would recommend to improve the course.
Surfiene:
Two levels of evaluation are usually used in the academic environment----formative and summative. Formative precedes summative and involves one-on-one trials, as well as small group trials. In both of these trials, the participants are representative of the target student population. Summative evaluation involves running the course the way it was intended with actual students, and obtaining feedback through course evaluations. This is what typical ISD models recommend.
Real world scenarios may dictate differently. Budgetary constraints, time constraints, and availability of participants are the three main factors which can prevent conducting these levels of evaluation. Some institutions find it difficult to find participants even when they offer compensation.
Satrohan
This is good in one way and bad in another, yes this student will know the material and might have some good questions for an evaluation. This student might have a problem with a certain part of the material that another student might not have a problem with. best bet is to incorpate your own questions along with the graduate's questions.
Suzanne,
I believe someone below alluded to the fact that the student used for the assessment may make assumptions based on the fact that they have already been exposed to the material. Here is a key point which we discussed in another topic. Visual materials generated for instructor led training may not be as detailed as is necessary for online courses. In the IL courses the instructor is there to fill in the blanks, expand upon and even explain in detail the visuals meaning. Consequently since the student has already seen the visuals being used, they grasp the concept from prior exposure where in reality the visual may not be detailed enough for someone without previous understanding. This is one example but could relate to other parts of the course as well. Just my two cents.
Donald,
Good point - from what I have seen, it will vary from publisher to publisher and class to class on IL materials. Some are good enough to essentially stand alone, while others are really just cue points for the instructor and have gaps. Its therefore important to determine what your materials are like before you make decisions on how much you can rely on them.
Jon
That is a great idea for a post grad student for your primary evaluation.
I think this is very beneficial, especially because you are obtaining a real-time learner perspective. This will help you align your course to run smoother for future course students.
Adib Shakir
Adib,
Great point - thank you for sharing!
Jon
I think that is a great idea. An evaluation from a former student would be valuable. Actually, the more observations the better. If you could also get a few faculty memebers to review from their perspective it would help. A sample size of one is hardly enough to get a sense of the course content and retention.
