First- and second-order reactivity to verbal protocols: an example from a study on strategy variability

Verbal reports are a common method of data collection in studies of mathematics learning, often in studies with a longitudinal component or those employing microgenetic methods where several observations of problem-solving are made over a short period of time. Whilst there is a fairly substantial literature on reactivity to verbal reports, addressing concerns that the production of verbal reports will change the behaviour that participants are reporting on, little of this literature addresses the question of how verbal reports might affect subsequent trials (what we would like to call second-order reactivity). In order to address the issues of both first- and second-order reactivity, ninety-nine 14-year-old participants were asked to provide answers to a set of six questions on two occasions, one week apart. Fifty-eight participants were asked for written answers and explanations, while 41 participants were asked for both concurrent and retrospective verbal reports. The results show evidence for both first- and second-order reactivity. The distribution of strategies employed in the first session was different for the two conditions and there were some interesting differences in patterns of strategy selection across the two sessions. These findings have some important implications for the use of verbal reports over multiple trials.