To return to part 1 of this paper, click here.
To skip to part 3 of this paper, click here.
To skip to part 4 of this paper, click here.
The cognitive structures that we have referred to as "student expectations" clearly are complex and contain many facets. We decided to focus on six issues or dimensions along which we might categorize student attitudes towards the appropriate way to do physics. Three of these are taken from Hammer's study and we have added three of our own.
Building on the work of Perry and Songer and Linn cited earlier, Hammer proposed three dimensions along which to classify student beliefs about the nature of learning physics:
In the MPEX survey, we seek to probe three additional dimensions:
The extreme views associated with each of these variables are given in Table 2. We refer to the extreme view that agrees with that of most mature scientists as the expert or favorable view, and the view that agrees with that of most beginning students as the novice or unfavorable view. The survey items that have been selected to probe the six attitudes are given in the right hand column of the table. We refer to the collection of survey items designed to probe a particular dimension as a cluster. Note that there is some overlap, as these dimensions are not independent variables.
Although we believe the attitudes that we have defined as expert correspond to those attitudes needed by most creative, intuitive, and successful scientists, we note that they are not always predictors of success in introductory physics classes. In an earlier study, Hammer studied two students in the algebra-based physics course at Berkeley. One student possessed many novice characteristics but was doing well in the course. The other student possessed many of the characteristics preferred by experts but was having trouble. The second student's desire to make sense of the physics for herself was not supported and she did not begin to succeed until she switched her approach to memorization and pattern matching. In this case the course supported an attitude and an approach to learning that most physics instructors would not endorse and one which certainly would cause her trouble if she were to try to take more advanced science courses.
We conducted more than 100 hours of videotaped student interviews in order to validate that our interpretation of the survey items matched the way they were read and interpreted by students. We asked students (either individually or in groups of two or three) to describe their interpretations of the statements and to indicate why they responded in the way that they did. In addition, students were asked to give specific examples from class to justify their responses.
From these interviews, we have found that students are not always consistent with their responses to what appear to us to be similar questions and situations. We feel that this does not represent a failure of the survey, but properly matches these students' ill-defined understanding of the nature of physics. One reason for this was described by Hammer. He observed that some students in his study believed that professional physics operated under the favorable conditions, but that it sufficed for them to behave in the unfavorable fashion for the purposes of the course. He referred to this by adding the marker "apparent" to the characteristic. This is only one aspect of the complex nature of human cognition. We must also be careful not to assume that a student exists in one extreme state or another. A student's attitude may be modified by an additional attitude, as in Hammer's observations, or even exist simultaneously in both extremes, depending on the situation that triggers the response. One must therefore use considerable care in applying the results of a limited probe such as our survey to a single student.
We are also aware that students' self-reported perceptions may not match the way they actually behave. However, the interviews suggest that if a student's self-perception of the learning characteristics described in Table 2 differs from the way that student actually functions, the self-perception has a strong tendency to be closer to the side chosen by experts. We therefore feel that while survey results for an individual student may be misleading, survey results of an entire classroom might understate unfavorable student characteristics.
|independence||takes responsibility for constructing own understanding||takes what is given by authorities (teacher, text) without evaluation|
14, 17, 27
|coherence||believes physics needs to be considered as a connected, consistent framework||believes physics can be treated as unrelated facts or "pieces"||
|concepts||stresses understanding of the underlying ideas and concepts||focuses on memorizing and using formulas|
|reality link||believes ideas learned in physics are relevant and useful in a wide variety of real contexts||believes ideas learned in physics has little relation to experiences outside the classroom||
|math link||considers mathematics as a convenient way of representing physical phenomena||views the physics and the math as independent with little relationship between them||
|effort||makes the effort to use information available and tries to make sense of it||does not attempt to use available information effectively||
In order to test whether the survey correctly represents elements of the hidden curriculum, we gave it to a variety of students and physics instructors. We defined as "expert" the response that was given by a majority of experienced physics instructors who have a high concern for educational issues and a high sensitivity to students. We conjectured that experts, when asked what answers they would want their students to give, would respond consistently.
We tested the response of a wide range of respondents by comparing five groups:
The University of Maryland students are a fairly typical diverse group of engineering students at a large research university. The entering class average on the FCI is around 50%. The number of students in the sample is N=445.
The US International Physics Olympics Team (USIPOT) is a group of high school students selected from applicants throughout the USA. After a two week training session, five are chosen to represent the US in the International Physics Olympics. In 1995 and 1996, this group trained at the University of Maryland in College Park and we took the opportunity to have them complete survey forms. The total number of respondents in this group is N=56. Although they are not teachers, they have been selected by experts as some of the best high school physics students in the nation. Our hypothesis was that they would prove to be more expert than the average university physics student, but not as expert as our groups of experienced instructors.
The physics instructors who served as our test groups were all visiting Dickinson College. Attendees came from a wide variety of institutions. Many have had considerable experience in teaching, and all of them were sufficiently interested in educational development to attend a workshop. We separated them into three groups: Group 3 high school teachers attending a two-week summer seminar (N=26), Group 4 college and university teachers attending the two-week summer seminar (N=56), and Group 5 college and university teachers implementing Workshop Physics in their classroom (N=19). The teachers in Group 5 were committed to implementing an interactive engagement model of teaching in their classroom. We asked the three groups of instructors to respond with the answer they would prefer their students to give. We expected these five groups to show an increasing level of agreement with answers we preferred.
The group we expected to be the most sophisticated, the group 5 instructors, agreed strongly as to what were the responses they would like to hear from their students. On all but three items, ~80% or more of this group agreed with a particular position . Three items, numbers 7, 9, and 34, had a strong plurality of agreement, but between and of the respondents chose neutral. We define the preferred response of Group 5 as the expert response. We define a response in agreement with the expert response as favorable and a response in disagreement with the expert response as unfavorable. For the analysis in this paper, the agree and strongly agree responses (4 and 5) are added together, and the disagree and strongly disagree responses (1 and 2) are added together. A list of the favorable responses to the survey items is presented in Table 3.
To display our results in a concise and easily interpretable manner, we introduce an agree-disagree (A-D) plot. In this plot, the percentage of respondents in each group answering favorably are plotted against the percentage of respondents in each group answering unfavorably. Since the fraction of students agreeing and disagreeing must add up to less than or equal to 100%, all points must lie in the triangle bounded by the corners (0,0), (0,100), (100,0). The distance from the diagonal line is a measure of the number of respondents who answered neutral or chose not to answer. The closer a point is to the upper left corner of the allowed region, the better the group's agreement with the expert response.
The results on the overall survey are shown in Fig. 1. In this plot, the percentages are averaged over all of the items of the survey, using the preferred responses of calibration group 5 as favorable. The groups' responses are distributed from less to more favorable in the predicted fashion.
Although the overall results support our contention that our survey correlates well with an overall sophistication of attitudes towards doing physics, the cluster results show some interesting deviations from the monotonic ordering. These deviations are quite sensible and support our use of clusters as well as overall results. In order to save space and simplify the interpretation of results, we present the data in Table 4. Displayed in this table are the percentages of each group's favorable and unfavorable responses (in the form favorable/unfavorable). The percentage of neutrals and not answering can be obtained by subtracting the sum of the favorable and unfavorable responses from 100.
From the table we see that most of the fraction of respondents agreeing with the favorable response tends to decrease monotonically from group 1-5 with a few interesting exceptions. The high school teachers (group 3) are farther than their average from the favorable corner in the coherence and math clusters, while the Physics Olympics team is closer to the favorable corner in those categories than their average. These results are plausible if we assume that high school teachers are less concerned with their students forming a coherent and a mathematically sophisticated view of physics than are university teachers. The results also agree with our personal observations that the members of the USIPOT are unusually coherent in their views of physics and exceptionally strong in their mathematical skills.
Note also that the Olympics team results are very far from the favorable corner in the effort cluster. The main discrepancies are in items 3 and 7. We suggest that reader peruse the survey items of that cluster (3, 6, 7, 24, 31). These items represent highly traditional measures of effort (reading the textbook, going over one's lecture notes) which we conjecture are not yet part of the normal repertoire of the best and brightest high school physics students before they enter college. We also conjecture that most of them will have to learn to make these kinds of efforts as they progress to increasingly sophisticated materials and the level of challenge rises.
This analysis of both the overall responses of the calibration groups and the variations in the ordering confirms that the MPEX survey provides a quantitative measure of characteristics which experts hope and expect their students to have.
|University of Maryland||Physics Department||PERG UMD||The MPEX Project|