Time: the neglected variable in TIMSS?
by John L. Roeder
At the 1999 Summer Institute for the Physics Teaching Resource Agents (PTRAs) of the American Association of Physics Teachers (AAPT) at Trininty University in San Antonio, TX, former AAPT President John Layman presented 15 physics questions from the Third International Math and Science Study (TIMMS). He asked us to review the questions and then answer the following questions about the test questions:
My response was that these questions required a high level of physics knowledge, more comparable to that achieved in an Advanced Placement course than to the algebra-based course I teach many of the seniors at The Calhoun School, and my response appeared to be typical of the other PTRAs.
- What are the general characteristics of the test items?
- What student knowledge and skills are expected?
- Are these appropriate to your classes?
- Would your students do well?
- What is your reaction to the performance expectations?
Layman then went on to have us read an article that Michael Neuschatz of the American Institute of Physics had written for the January 1999 issue of The Science Teacher. In the published version of that article (Sci. Teach., 66(1), 23-26 (Jan 99)) Neuschatz writes:
The content of the TIMSS itself is not the issue. An examination of the physics questions that comprise the released set of test items (more than half of all the questions actually used) shows a series of carefully crafted and creatively presented problems that address the ability of students to think scientifically and manipulate concepts, not just equations. Instead the problem with the TIMSS results is the comparability of the students who took the test.
No wonder I responded the way I did! The cohort of students examined in physics by TIMSS in most of the other countries had studied physics for at least two years. On the other hand, of the 775,000 American seniors who have studied physics, only 35,000 have studied it for a second year: this is only 4.5% of the total number of physics students, while 14% were examined by TIMSS.
The role of time was later brought up by Jose Mestre of the University of Massachusetts (Amherst) in the opening paper of the Physics Education Research Conference held a week later in conjunction with the AAPT summer meeting. In the course of reviewing a typically-structured project studying the effect of a particular educational treatment in physics, Mestre asked "Do students spend more time outside of class as a result of the treatment? Are students spending time more efficiently?"
Here for the second time in as many weeks the role of time and its efficient use surfaced with regard to learning physics. Yet any hope for a brighter interpretation of the U.S. 12th grade TIMSS scores was short-lived. In another article in the January 1999 issue of The Science Teacher (by Pascal Forigone, David Nohara, Christine Welch, and Laura Salganik, pp. 38-41) I read the following:
Looking only at the scores of the most advanced physics students in the United States -- those who had completed or were completing Advanced Placement physics -- the news was slightly better: U.S. AP physics students' average score was similar to the average scores of advanced physics students in 8 of the 16 participating nations, and these students were outperformed by students in only 7 nations. Nonetheless, U.S. AP physics students still scored below the international average.
Forigone, et al., also reported that this year the TIMSS assessment will be "given to eighth graders who are the same cohort who were in fourth grade when the original TIMSS assessments were given." Layman also reported to the PTRAs that there will be additional TIMSS retesting in classes whose teachers attended National Science Foundation (NSF)-supported programs, after NSF raised concern that students taking the TIMSS test were taught by teachers not participating in these programs. The results of this latter retesting are to be presented at the AAPT winter meeting next January in Orlando, FL.
(Editor's Note: Two other viewpoints on TIMSS were presented on the front page of our Winter 1999 issue. Additional points of view can be found in the January 1999 issue of The Science Teacher (reference #6, this issue).)
Home Fall 99 Full Screen
The TEACHERS CLEARINGHOUSE FOR SCIENCE AND SOCIETY EDUCATION