We Need to Measure What Students Learn

GUEST COLUMN – Accounting for learning

Facebooktwitterpinterestlinkedintumblrmail

This is the first in a series of guest columns written by Prof. Maureen Mancuso, U of G provost and vice-president academic, for University Affairs magazine, a publication of the Association of Universities and Colleges of Canada. Reprinted with permission.

Provost Maureen Mancuso

At Canadian universities, administrators and faculty face an ever-brewing, sometimes boiling conflict between unconstrained academic freedom and the increasing demands for accountability and proof of “return on investment” in higher education.

In some ways we are victims of our own success: as academics we know that education is as much about the journey as the destination, but there’s no escaping the fact that many of the people involved value that destination pretty highly. Education is an excellent investment – for students, communities and governments – but, as with any program that pays off in social and intellectual as well as mere monetary capital, it can be challenging to demonstrate that fact convincingly.

What annoys many academics, including me, about the baldly economic description of education as an “investment” is that modelling teachers as producers and students as consumers not only demeans both parties to the relationship but also reduces learning to a purely passive activity: mere absorption of knowledge, pre-digested and individually wrapped. The problem is that budgets are budgets, and if all we can provide are fiscal metrics, there will always be a tendency to use a model that conforms to what can be measured.

Figures showing the enhanced career prospects of graduates are compelling, but they measure something (usually income) that is only distantly connected to the actual educational experience, and they are generational metrics. Students, parents and especially governments don’t want to wait until a cohort retires to decide whether its youthful promise was wasted or amplified.

Nor should we. “Continuous improvement” is one of those weaselly phrases that echo around admin buildings, but it also describes good pedagogy and indeed all academic pursuits. What are we, as educators, other than a culture dedicated to experimenting, testing, questioning, challenging, assessing, tweaking and improving our understanding and our approach to learning?

Even if you recoil at the idea of a transactional metaphor overtaking the idealistic pursuit of knowledge, you still care whether you are ennobling young minds or just helping them kill their time, like a less histrionic form of reality TV playing in the background while they update their Facebook pages. But how can anyone be sure this is happening if what we (as a system) count is just inputs, like the number of hours per week spent in class (a metric close in spirit to TV ratings), or opaque summaries like grades and marks? How can students communicate their achievements to future instructors expecting evidence of requisite knowledge, or to employers demanding useful skills?

The answer to “what did this student learn?” is not “1.5 credit hours” or “a B+.” The registrar’s office may consider those metrics significant, but few outside the academy will be able to decipher them. The answer is not a score on a standardized test, which measures test-taking skill and test preparation above all else. The answer should be measurable in terms of some specific kind of enhanced skills or mastery.

For example, transcript designations that identify courses that are particularly writing- or research- or numeracy-intensive offer a more transparent view of just what a student gained by achieving a good mark. Methods like rubrics, portfolios, analytics and curriculum maps identify tangible learning outcomes, and they give students, employers and governments a more evidentiary and transparent basis on which to assess the value of a university degree – a basis that does not encourage the transactional fallacy.

We all want students’ learning experiences to be highly “productive.”  We want that investment of time, effort, intellectual wattage and, yes, money to yield a good return because it’s through just that sort of productivity enhancement in our own lives that we came to our calling as teachers. Outcomes-based learning assessment practices simply externalize the multidimensional evaluation we already do as an essential part of teaching. They are, in a way, just a higher-level application of some familiar exhortations: “show your work” and “discuss, with specific examples.”

Professors and governments may not share all the same motivations, but they agree on the goal of enhancing student learning. It’s just that without documented reason to believe that graduates have actually expanded their critical faculties in specific ways, we must demand stakeholders take it on faith, and faith isn’t enough to justify the investment we hope to see in the system. With learning outcomes assessment, we can give those stakeholders a way to satisfy their need for accountability while still acknowledging the pedagogical values we respect and want to preserve.