October 16, 2003
UW part of effort to measure students’ information technology skills
The UW is part of an ambitious effort to develop new tools for assessing students’ skills in information and communication technologies (ICT).
Educational Testing Service, the world’s largest private educational testing and measurement organization, is working with seven college and university systems, which collectively enroll 25 percent of the nation’s college students, on the initial design phase.
In effect, the group is dealing with the question of defining a new kind of literacy and how to measure it.
“This is a compelling issue,” says Anne Zald, information literacy coordinator for UW Libraries. “We’re living in an age with expanding access to all kinds of information in a variety of formats. We are creating things we couldn’t create before, new types of scholarship, new ways of working, new ways of thinking, in academia and the workplace. Yet, there’s a sense of a gap — between the availability of technology and the ability to use it in a critical way.”
A previous panel convened by ETS attempted to define this new literacy. The report of the panel is available at http://www.ets.org/research/ictliteracy/index.html. The panel concluded that “mastery alone does not define ICT literacy,” and that true literacy involves integration of technology and cognitive skills.
Although the “Phase I Group,” as Zald has characterized it, has met just twice, with two more meetings planned, ETS already is planning to roll out some measurement instrument in 2005. What shape it will take, and how it will be applied, are important but unanswered questions. Nevertheless, Zald sees great potential in the approach ETS is taking.
“This could be a powerful tool,” Zald says. “We could use it to analyze populations of entering students, to determine their capabilities as a group and what kinds of curricula we need to offer them. Or the tool could be used to track individual students, to place them in the appropriate classes, so that they have the option of improving their skills. This could go as far as becoming a means of certification, which seems to interest some of the California schools.”
The first phase of this project is scheduled to end in November. The second phase could involve testing of some prototype tools, possibly on the UW campus. Whatever instrument is designed, it will be delivered online.
“Our work thus far has been to try and identify the components of ICT literacy,” Zald says, “and to develop questions that test this ability. It’s pretty clear that the instrument will be more than a multiple choice test. It will be scenario based and taken online so that test scoring may include not only the answer but the pathways taken to get to that answer.”
The UW community has been interested in the subject of ICT literacy for some time — almost since online technology became widely available. Earlier this year, the Program for Educational Transformation through Technology (PETTT) conducted a survey of UW students and faculty to assess their current uses of educational technology (see http://depts.washington.edu/pettt/tech_report.pdf).
One of the survey’s findings was that students consider themselves highly skilled at evaluating information they find on the Web — but faculty strongly disagree and would like to see the university build ICT literacy training into more courses. At a forum earlier this year at which the PETTT report was presented, a group of faculty and staff discussed the desirability of developing measures to test ICT literacy. But many participants were skeptical that measurements with wide applicability were even possible.
Other universities also are looking at ways to measure ICT literacy, but Zald believes the ETS effort is likely to be the most comprehensive.
In addition to the UW, other participating institutions are: the California Community College System, California State University, UCLA, University of Louisville, University of North Alabama, and the University of Texas System.
