I’ve been trying to put some further reflection into my testing methods in the upper year pathophysiology class I teach. As I have mentioned briefly, the course has traditionally been tested using high-stakes multiple choice tests. These tests have been designed to follow the structure of the provincial certifying exam written after graduation. My program offers the highest ranking on this provincial test for the past 13 years, 100% pass rate, and no other school is close to boasting that. I understand however, that just because this is the way we’ve always done it, this does not mean that it is the right way. In my last post I looked at an alternative grading scheme, that could even continue to utilize the current tests, but that would perhaps give students the best opportunity to do well. On one hand it doesn’t penalize a single poor performance if one achieves well elsewhere, on the other hand, in the field of paramedicine failing to do well on a single call may have drastic consequences, and hence we’ve always been okay with high-stakes testing. I’ve also heard of a multiple-choice scheme where the test is given in class, which is worth two-thirds, and then the test is taken home until next class for the students to write again using their notes and resources, worth the remaining third, in an attempt to deepen learning. Reviewing a multiple choice guide from the University of Texas, it gives a few tips to creating multiple choice tests. It was interesting to see how many of the “do’s” I don’t do. Why? For the same reason I’ve brought up previously, the provincial certification exam is written in a confusing method with often poorly written questions, so as a program we take it upon ourselves as a duty to prepare students for this exam, which will then allow them to find gainful employment and a fulfilling career. After all, is this not why they came to college in the first place? Within the Texas document, it does discuss some positives of multiple choice testing, which I have found highly beneficial. Of these, namely the quick feedback, and the ability to statistically analyze the data. It does warn however that learning can be lessened if the test is written poorly, as students will study for recognition, not recall. Additionally, poorly written tests will allow test-wise students to find answers throughout the test, which I do on purpose to teach students how to better write tests. It suggests to avoid tricks, minimize reading, and ensure there is just one correct answer. I incorporate small degrees of all of these, again to teach students how to write such tests, as not only will their provincial exam be like this, but the majority of their hiring-competition written tests as well. It is as though they are often written by those trying to prove how smart they are, not ascertain the knowledge of the applicants. Since I took this course over now two years ago, I’ve taken the degree of multiple choice down to 70% of each test, with 30% short-answer, to further emphasize a deeper learning and engagement with the material. I’m brainstorming with ideas to incorporate more case-based learning and inquiry through assignments, to supplement the traditional exam style of the class. My thoughts are this may increase engagement, give the students more ownership of their learning, improving material contact, while still administering the now lower-stakes multiple choice tests to ensure post-graduation readiness. As I’m in the reflection / brainstorming phase, I would greatly appreciate any feedback and experience with those under the pressures of standardized tests, who have been influenced to teach to a test, and from those that successfully made the leap away from the traditional. https://facultyinnovate.utexas.edu/teaching/check-learning/question-types/multiple-choice
4 Comments
2/14/2017 12:01:37 pm
Thanks for sharing Jason, I also have experience teaching to standardized provincial examinations - English language and literature exams for BC.
Reply
2/16/2017 03:15:04 pm
Hi Ryan, this is my 2nd "reply" to your comment, for whatever reason the first one was not posted - browser timeout, lost connection, forgot to "submit", or maybe it's just out there floating around and will appear in three days...
Reply
Lesley
2/16/2017 08:34:34 am
1. Thank you for this interesting reflection on multiple choice tests. As someone who studied English literature, I certainly have a bias in favour of other means of testing for the subjects that I have pursued and that I have taught. However, I found your justification for for such tests compelling given that "in the field of paramedicine failing to do well on a single call may have drastic consequences." There is a TedTalk you may appreciate by Dr. Martin Bush titled, "Multiple-choice tests without the guesswork". In it he discusses negative marking schemes designed to reduce guesswork, sub-set selection, and other diversified formats for multiple choice tests that have been studied and executed through the past half a century. His comment about multiple choice tests being quite difficult to create reminds me of a story a friend recently told me. When she was in high school, she remembers being given the option in one of her classes to either take the existing multiple choice test for the subject OR create her own multiple choice test. Many students opted for creating their own test thinking that it would be easier and guarantee them a better grade. However, designing a good multiple choice test demands such mastery of the material that many found this process to be, in fact, more rigorous. This may be a worthwhile experiment with your students in order to promote that "deeper learning and engagement with the material."
Reply
2/16/2017 12:40:56 pm
Hi Lesley, thanks for the added resource! I tried to get the article, but watched the TedTalk and decided to elaborate my thoughts about that in a subsequent post that's up as of now, thanks again!
Reply
Leave a Reply. |
AuthorI teach Paramedicine at Cambrian College, in Sudbury, Ontario. I also continue to work as a paramedic, and ride bikes. This is my third semester in the PME program, and I look forward to learning with everyone! Archives
March 2017
Categories |