How can we provide assessment options that engage students while connecting them to each other and the broader world? While a well-crafted exam can be a powerful evaluation tool, U of T instructors are creating innovative assessment solutions to support and evaluate their students within a constantly evolving teaching and learning landscape.
A key concept in creating meaningful assessments is backwards design – where assessment, pedagogical strategies and curriculum are designed with “the end in mind.” In other words, assessing what students should know or be able to do at the end of a learning experience.
In February 2016, the Centre for Teaching Support & Innovation (CTSI) hosted a group of faculty to meet with Emeritus Professor Geoff Scott of Western Sydney University to showcase their powerful assessment strategies. As part of his Australian National Teaching Fellowship, Scott is collecting best practices and creating a searchable online resource that showcases international examples of assessment, including many from U of T.
Brenda Mori, Director, Clinical Education & Community Affairs and Associate Professor, Teaching Stream, Physical Therapy, is the lead on a team that developed the Canadian Physiotherapy Assessment of Clinical Performance (ACP). The ACP will be used by students and instructors to describe their clinical performance behaviors, relative to what is expected of a practicing physiotherapist in Canada. After rigorous testing, Mori and her team determined that this assessment could be used to evaluate entry-level physiotherapy students.
In 2015, Ken Derry (Associate Professor, Teaching Stream, Religious Studies, UTM) introduced a problem-based fourth year seminar course on “Advanced Topics in Christianity.” The class was divided into groups, and each group was assigned a literary text and asked to design an undergraduate course on Christianity and modern literature showcasing their text. Students were provided with teaching and learning resources on course and assignment design, and presented a final project comprised of an annotated syllabus, including resources, and a lecture on the assigned text. Students greatly appreciated this novel learning opportunity, and in an end-of-course reflection, fourth year student Saadia Salamath wrote: “I am grateful to have been given the opportunity to design a course in this class, to figure things out on our own, to reach that aha! moment. I have learned so much about myself and others and my worldview has expanded based on class discussions as well as the books we read. I have learned things that no content-based course would ever be able to teach and it was even kind of fun. I learned to compromise as well as to stick by the ideas that I think could potentially work. I learned that I’m actually pretty good at diffusing tense situations in a group setting. I learned how to understand myself as a human in this world better and you taught me that maybe being shy doesn’t have to be such a terrible thing.”
Beth Fischer, Assistant Professor, Teaching Stream, at Woodsworth College teaches an interdisciplinary course for first-year students called “Order and Disorder: Global Affairs and Emerging Technologies.” Throughout the course, a series of role-play exercises challenge students to think through practical problems and situations, resulting in a deeper understanding of theoretical concepts concerning global affairs. For example, a role-play exercise that simulates a nuclear crisis challenges students to reason through their response. The class is divided into two groups – Soviet and American leaders. The students receive a series of “intelligence reports” that they are required to respond to as a group, considering what they know, what they need to know, and their options. In this way, they come to a deeper understanding of a difficult theoretical concept – Mutually Assured Destruction. Dr. Fischer’s goal has been to create a learning environment that engages students, cultivates their curiosity, and, through novel assessments, deepens their understanding while building community. The student reaction to these assessments has been very positive – former students have expressed to Fischer that, in upper-year courses, they have an easier time grasping complex concepts by relating them to the role-playing exercises.
Patricia Sheridan, a doctoral student at the Institute for Leadership Education in Engineering has developed the Team-effectiveness Learning System. Using an online tool, students give each other analytic and holistic feedback at the mid and end-points of team projects, which is provided anonymously with the goal of creating more effective teams.
“Teamwork is critical because engineering is a team-oriented profession,” Sheridan said. “Many technically exceptional students struggle with the industry-demanded skills of how to work effectively with others.”
In addition to providing students with a look at their own teamwork skills, team and class-level diagnostics are provided to the instructors, allowing them to tailor their teaching on team effectiveness to the specific needs of their students.
The work of these instructors reflects a burgeoning interest in rethinking assessment at U of T. Many of their approaches are considered by researchers to be “authentic assessment,” comprised of processes and products that mirror real-world applications of learning.
“When students are able to demonstrate competency in real world situations that require high-level reflection, inventiveness and collaboration, they gain confidence in their skills and knowledge and better understand the value of their education. Importantly, that confidence also supports their self-directed learning once they complete their university programs,” notes Carol Rolheiser, Professor of Curriculum, Teaching and Learning, OISE, and Director of CTSI.
U of T is committed to high quality learning experiences for its students. To meet this priority, the new course evaluation framework adopts a cascaded assessment model that, in addition to the institution’s core items, provides opportunities for individual faculties, departments, and instructors to include their own assessment items.
Dr. Cherie Werhun, Teaching Assessment & Course Evaluation Coordinator, CTSI, elaborates, “The real innovation of our new course evaluation framework lies in its commitment to meaningful and purposeful assessment of the learning experiences of U of T’s students. Data collected reflects students’ perceptions of the extent to which intended teaching and learning expectations of our institution and of each Faculty are actually part of students’ learning experiences in their courses. This purposeful design means that the data collected can begin to answer important questions about whether we are living up to these expectations.”
To support this more nuanced approach to the design and collection of course evaluation data, U of T adopted an online application, which not only manages the unique pairings of course evaluation items to faculties and departments, it hosts a 200 item course evaluation bank for instructors to choose from; and sends out and collects students’ responses to hundreds of thousands evaluations each year.
Currently, implementation of the new framework within a Faculty takes approximately one year as Faculties work with CTSI through a rigorous design and implementation process. Now at 10 Faculties, with the remaining soon to follow, once implemented across the entire institution, the data analysis possibilities are endless.
“Good assessment means good data; and ultimately, good decision-making about teaching and learning at the University of Toronto – and since we began implementation in 2011, we have had some of the best conversations about teaching,” Dr. Werhun said.
Higher education continues to respond to shifts at both global and local levels, using responsive, accurate and authentic feedback and powerful assessment processes. These feedback and assessment processes provide us with data to help us answer questions about what factors contribute to excellence in both the student learning experience and teaching at U of T.
Additional files from Brenda Mori, Patricia Sheridan, U of T Engineering News