June 26, 2003
Study of undergraduate learning probes thoughts, feelings of students
For the last four years she’s been listening, and now it’s Catharine Beyer’s turn to speak. Beyer, together with Gerald Gillmore, director emeritus of the Office of Educational Assessment; Andrew Fisher, their research assistant; and four undergraduate researchers, has been probing the thoughts and feelings of UW students in an effort to find out how their schooling here affects their lives.
It’s called the UW Study of Undergraduate Learning, or UW SOUL, and as far as Beyer can tell it’s the only longitudinal (following the same group over time) study of the college student experience in recent times. First-year students — both freshmen and transfer — were enrolled in the study four years ago and then contacted periodically to answer questions about their experience. They were followed in this way through graduation.
For Beyer, it was a new way of interacting with students. She has taught for 18 years in the Interdisciplinary Writing Program, but a few years back she worked on several longitudinal studies of student writing. That’s where she met Gillmore.
“Jerry was thinking that it would be interesting to retain a panel of students that you could question every now and then as part of an assessment study,” Beyer says. “So you could find out, for example, whether they thought their classes were challenging and whether they thought they were learning critical thinking skills.”
Half the group of 300 students, who were recruited from among those who filled out the entering student survey, completed online surveys and answered e-mail questions, while the other half did that and also submitted to interviews, participated in focus groups and compiled a yearly portfolio with an essay. The students were paid for their efforts, giving them incentive to remain in the study. The money came from state assessment funds with an additional contribution from the Office of Undergraduate Education.
“The study was not hypothesis driven,” Beyer says. “Its purpose was to describe what the undergraduate experience is. We wanted to find out what the University is doing well and what might be improved. We didn’t have a clue what we would find.”
UW SOUL sought information in six categories. Four were the categories that the state mandates be used for assessment: writing, critical thinking/problem solving, quantitative reasoning and information literacy. Gillmore and Beyer added understanding and appreciating diversity because past student surveys suggested that was an area of growth for students. Personal growth was added because students in previous studies talked about how they had changed as people as a result of their schooling.
So what are UW SOUL’s results? Difficult to encapsulate and incomplete at this point are two ways to answer that question. Although the study did ask some quantifiable survey questions, more often the questions were open-ended and the answers had to be analyzed and categorized in a slow, labor-intensive process. This means the researchers are still working on analysis even though the study has officially ended. It also means the results for the most part can’t be broken down into standard categories like “strongly agree, somewhat agree,” etc.
Which doesn’t mean that there is nothing to say. Much has been learned already, especially about the first two years, and Beyer has begun to talk about it.
“The bottom line is that students change while they are here,” she told the audience at a recent presentation. “They change because they are challenged. They are changed by what we ask, by what they demand of themselves and by what life imposes on them. They leave different than when they arrived.”
The first of these changes, for traditional college-age students, is a huge gain in independence. Away from their high school identities, they learn who they really are. “They learn what they want, not what someone else wants for them,” Beyer says.
But students also receive — usually in their first quarter — what Beyer calls a “hammering.” They come here having been the best in their high schools or community colleges. Suddenly they discover that everyone around them is just as smart and they can’t count on outstanding grades with little effort.
This is a blow but it’s also a turning point, Beyer says. “They get over that and when they come back they come back really strong.”
For some the hammering continues in other forms. Beyer told the story of one young man who had come to the UW with an interest in computers and an intention to major in business. But when he applied to the business school he was rejected — twice. He then took a class in informatics and enjoyed it so he applied to major in that. Again he was rejected — twice.
Bowed but not beaten, the young man took a geography class that involved service learning. He told Beyer: “Before I came here, I was interested in making money. Geography changed this. Now I have this desire to change the world. After I graduate, I’m going to do Teach for America. I want to help empower people with education.”
Young students are not the only ones who are changed. Beyer told of an older student who had returned to school with the idea of becoming a history teacher. He described to her how he was moved nearly to tears in an ethnic studies class, learning about the people who had been left out of the history classes he had taken when he was younger.
Results in the specific areas of the study include the following:
Quantitative Reasoning: Work in this area differs significantly from what students have had in the past. They’re used to working problems. Here they have to figure out what they are being asked, which formulas will work and then do the problems. Beyer says they find this “enormously difficult.”
Writing/Critical Thinking: Students do not know there are differences in the types of thinking and writing required in different disciplines. They are used to writing only in English class. They have particular difficulties writing papers in which they must take a position and present supporting arguments.
Information Literacy: As students turn to the Web for information, the University has less control over what they are learning. “Somebody tosses out something in a lecture and students go to the Web to look it up,” Beyer says. “It’s exciting because they are ratcheting up their learning this way.”
Diversity: Students are excited about the prospect of more diversity when they come to the UW. However, diversity isn’t always what we think it is — “shuffling people so they’re mixed.” For many students of color, Beyer says, diversity means finally finding more people like themselves — something they didn’t have in high school.
Asked what the University could do to improve the undergraduate experience, Beyer says the most glaring need is for faculty to make the “bones of our disciplines” clear to first-year students, especially freshmen.
“They need to understand what kinds of methods we’re using, how an argument is constructed in anthropology as opposed to art history or literature,” she says. “Then they’ll have an easier time writing the papers we assign. We could accomplish that in part by showing them good student papers from the previous year.”
Although making classes smaller would help too, Beyer knows that’s not going to happen in budget-cutting times. And the students feel differently about this than one might expect. “The thing students think is most surprising in their first year is how small the UW seems,” Beyer says. “Already by their second quarter they feel they have a little community here, and that says a lot about the efforts we’ve made, such as the Freshman Interest Groups.”
In that tradition, Beyer is currently working with new- student orientation to plan a new event called Faculty Seminars. Between 150 and 300 faculty will meet with about 15 students each on the Friday before classes. Both teachers and students will have a booklet containing some of the results of UW SOUL that they’ll be able to discuss. What should be in those booklets is the question Beyer is wrestling with now.
There are a number of reports outlining specific results of UW SOUL already available on the Web at http://www.washington.edu/oea/soul.htm (see sidebar below). Beyer and Gillmore continue to prepare new reports with the help of their research assistant Fisher. And the trio is working on a book.
“I feel a moral obligation to what the students told me,” Beyer says. “I feel like I’m haunted by voices, that they told me stuff about their experience here, and I need to get that word out there to people. I feel missionary-like about that.”
UW SOUL reports available
Reports available from UW SOUL include student responses to specific questions as well as more general explorations of a topic. Titles are below:
What personal skills/abilities do you want to develop while you are at the UW?
Describe a high school or community college project or assignment that required you to use problem-solving and critical- thinking skills.
What was the hardest paper you wrote before you came to UW?
Thinking about what you expected when you came to the UW, what, if anything, has surprised you the most?
Student perceptions of academic expectations at the UW
Critical thinking and problem solving
Did you learn something important outside your classes this year?
Attitudes about diversity
Did you accomplish what you set out to accomplish this year?
What do your professors do that leads you to think they care about your learning?
Critical thinking and problem solving in the first two years
Writing at the UW: The first year
