UW News

Archive

April 12, 2007

Standardized testing of college students won’t work, says new book by UW researchers

Inside the Undergraduate Experience was published in March as results of the UW Study of Undergraduate Learning (UW SOUL).

Catharine Hoffman Beyer

Researchers on the UW Study of Undergraduate Learning project with four of the undergraduates who helped them. From left, they are Amy Lu, Andrew Fisher (researcher), Idrus Syed, Jerry Gillmore (researcher), Catharine Beyer (researcher), Oluwatope Fashola, Tula Habb.

When Jeremy Nolan came to the UW in 1999, he thought he’d major in business and become a stockbroker, but four years later, after a research project in Indonesia got him interested in nonhuman primates, he graduated with bachelor’s degrees in biology and psychology.

UW researchers who noticed changes in that young man and other students like him have produced a comprehensive study of UW students, the first longitudinal assessment of learning among a large group of American college students.

Inside the Undergraduate Experience appears as a federal commission proposes standardized testing of college students similar to that required by No Child Left Behind.

Published in March, the book says learning is mediated by the academic disciplines, particularly the major the student chooses. Office of Educational Assessment researchers Catharine Hoffman Beyer, Gerald M. Gillmore and Andrew T. Fisher say meaningful assessment of undergraduate learning must be conducted at the departmental level, rather than centralized.

The book grew out of the UW Study of Undergraduate Learning, which followed 304 students through four years at the UW. Three or more times each year, they sent students open-ended e-mail questions and surveys. Additionally, they interviewed half the students each year, conducted focus groups, and collected portfolios of student work. They wanted to know what and how students had learned.

Inside the Undergraduate Experience blends quantitative and qualitative analysis with case studies, including stories told primarily in the students’ own voices. Researchers measured critical thinking, writing and quantitative skills, personal growth, diversity awareness and knowledge of information technology.

A universitywide forum on the book and the study has been scheduled for 3:30 to 5:30 p.m. Thursday, May 10, in the Walker-Ames room of Kane.

Results from the study show that writing, critical thinking, and quantitative reasoning are not generic skills and that even among freshmen, such skills are mediated by the disciplines. Thus Nolan’s report on monkeys in Indonesia will be different in many ways from a chemistry lab report or an English essay. What counts as good thinking, writing, quantitative reasoning, and information literacy practices in college is closely aligned with the professional practices in those fields.

The UW researchers also learned:



  • Gaps between high school and college learning were most obvious in writing, critical thinking, and quantitative reasoning.
  • Learning about others and oneself is a central part of college and may be affected by choice of major.
  • Peers are important, but faculty are the key to students’ learning.
  • To a great extent, students meet their learning goals, and self-assessment is an important tool.
  • Students make gains in critical thinking and writing during college.


Higher education has functioned through most of its history without sufficient data to make good choices about college, said Thomas Toch, co-founder and co-director of Education Sector, an education policy think tank in Washington, D.C. U.S. News & World Report issues rankings, but they’re based on fame, wealth and exclusivity, he said, not necessarily quality of learning.

About 165 colleges and universities around the country use the Collegiate Learning Assessment, a generic test of critical thinking and writing ability, to determine what their students have absorbed, but those institutions are reluctant make results public.

The Spellings Commission favors such a test, but Beyer and her colleagues question its worth, citing deficits in the questions themselves. In the critical thinking section of the test, students could be asked, for example, to determine whether a company should purchase an airplane recently involved in an accident. In the writing exam, they might be asked to write an essay on whether public figures should expect to lose their privacy.

Beyer and her colleagues argue that if a student such as Nolan scored poorly, one might mistakenly conclude that he is a poor thinker or writer, but a low score might actually mean that he had no practice writing recommendations for businesses and no interest in lives of the rich and famous.

College learning “is mediated by the academic disciplines,” so a standardized test won’t capture students’ real academic experience, Beyer said. Also, that sort of test can’t measure the rich mix of experience inside and outside the classroom that results in an educated person, she and her colleagues say.

College, the authors say, “is a huge experience for most students, calling their beliefs and values into question, asking them to make independent decisions that have long-term consequences, sometimes for the first time in their lives.”