Student Expectations in Introductory Physics: Part 2

Edward F. Redish, Jeffery M. Saul, and Richard N. Steinberg

Department of Physics, University of Maryland, College Park, MD 20742

To return to part 1 of this paper, click here.
To skip to part 3 of this paper, click here.
To skip to part 4 of this paper, click here.

Choosing the Items of the MPEX Survey

The cognitive structures that we have referred to as "student expectations" clearly are complex and contain many facets. We decided to focus on six issues or dimensions along which we might categorize student attitudes towards the appropriate way to do physics. Three of these are taken from Hammer's study and we have added three of our own.

Building on the work of Perry and Songer and Linn cited earlier, Hammer proposed three dimensions along which to classify student beliefs about the nature of learning physics:[18]

1. Independence beliefs about learning physics - whether it means receiving information or involves an active process of reconstructing one's own understanding;
2. Coherence beliefs about the structure of physics knowledge - as a collection of isolated pieces or as a single coherent system;
3. Concepts beliefs about the content of physics knowledge - as formulas or as concepts that underlie the formulas.

   In the MPEX survey, we seek to probe three additional dimensions:

4. Reality Link beliefs about the connection between physics and reality - whether physics is unrelated to experiences outside the classroom or whether it is useful to think about them together;
5. Math Link beliefs about the role of mathematics in learning physics - whether the mathematical formalism is just used to calculate numbers or is used as a way of representing information about physical phenomena;
6. Effort beliefs about the kind of activities and work necessary to make sense out of physics - whether they expect to think carefully and evaluate what they are doing based on available materials and feedback or not.

The extreme views associated with each of these variables are given in Table 2. We refer to the extreme view that agrees with that of most mature scientists as the expert or favorable view, and the view that agrees with that of most beginning students as the novice or unfavorable view. The survey items that have been selected to probe the six attitudes are given in the right hand column of the table. We refer to the collection of survey items designed to probe a particular dimension as a cluster. Note that there is some overlap, as these dimensions are not independent variables.[19]

Although we believe the attitudes that we have defined as expert correspond to those attitudes needed by most creative, intuitive, and successful scientists, we note that they are not always predictors of success in introductory physics classes. In an earlier study, Hammer studied two students in the algebra-based physics course at Berkeley.[20] One student possessed many novice characteristics but was doing well in the course. The other student possessed many of the characteristics preferred by experts but was having trouble. The second student's desire to make sense of the physics for herself was not supported and she did not begin to succeed until she switched her approach to memorization and pattern matching. In this case the course supported an attitude and an approach to learning that most physics instructors would not endorse and one which certainly would cause her trouble if she were to try to take more advanced science courses.[21]

Validating the Survey: Interviews

We conducted more than 100 hours of videotaped student interviews in order to validate that our interpretation of the survey items matched the way they were read and interpreted by students. We asked students (either individually or in groups of two or three) to describe their interpretations of the statements and to indicate why they responded in the way that they did. In addition, students were asked to give specific examples from class to justify their responses.

From these interviews, we have found that students are not always consistent with their responses to what appear to us to be similar questions and situations. We feel that this does not represent a failure of the survey, but properly matches these students' ill-defined understanding of the nature of physics. One reason for this was described by Hammer. He observed that some students in his study believed that professional physics operated under the favorable conditions, but that it sufficed for them to behave in the unfavorable fashion for the purposes of the course. He referred to this by adding the marker "apparent" to the characteristic. This is only one aspect of the complex nature of human cognition. We must also be careful not to assume that a student exists in one extreme state or another. A student's attitude may be modified by an additional attitude, as in Hammer's observations, or even exist simultaneously in both extremes, depending on the situation that triggers the response.[22] One must therefore use considerable care in applying the results of a limited probe such as our survey to a single student.

We are also aware that students' self-reported perceptions may not match the way they actually behave.[23] However, the interviews suggest that if a student's self-perception of the learning characteristics described in Table 2 differs from the way that student actually functions, the self-perception has a strong tendency to be closer to the side chosen by experts. We therefore feel that while survey results for an individual student may be misleading, survey results of an entire classroom might understate unfavorable student characteristics.

Table 2

	Favorable	Unfavorable	MPEX Items
independence	takes responsibility for constructing own understanding	takes what is given by authorities (teacher, text) without evaluation	1, 8, 13, 14, 17, 27
coherence	believes physics needs to be considered as a connected, consistent framework	believes physics can be treated as unrelated facts or "pieces"	12, 15, 16, 21, 29
concepts	stresses understanding of the underlying ideas and concepts	focuses on memorizing and using formulas	4, 19, 26, 27, 32
reality link	believes ideas learned in physics are relevant and useful in a wide variety of real contexts	believes ideas learned in physics has little relation to experiences outside the classroom	10, 18, 22, 25
math link	considers mathematics as a convenient way of representing physical phenomena	views the physics and the math as independent with little relationship between them	2, 6, 8, 16, 20
effort	makes the effort to use information available and tries to make sense of it	does not attempt to use available information effectively	3, 6, 7, 24, 31

Table 2: Dimensions of student expectations.

IV. EXPERT EXPECTATIONS: THE CALIBRATION GROUPS

In order to test whether the survey correctly represents elements of the hidden curriculum, we gave it to a variety of students and physics instructors. We defined as "expert" the response that was given by a majority of experienced physics instructors who have a high concern for educational issues and a high sensitivity to students. We conjectured that experts, when asked what answers they would want their students to give, would respond consistently.

A. The Calibration Groups

We tested the response of a wide range of respondents by comparing five groups:

• Group 1: engineering students entering the calculus-based physics sequence at the University of Maryland,
• Group 2: members of the US International Physics Olympics Team
• Group 3: high school teachers attending the two-week Dickinson College Summer Seminar on new approaches in physics education
• Group 4: university and college teachers attending the two-week Dickinson College Summer Seminar on new approaches in physics education
• Group 5: college faculty who are part of a multi-university FIPSE-sponsored project to implement Workshop Physics at their home institutions.

The University of Maryland students are a fairly typical diverse group of engineering students at a large research university. The entering class average on the FCI is around 50%. The number of students in the sample is N=445.

The US International Physics Olympics Team (USIPOT) is a group of high school students selected from applicants throughout the USA. After a two week training session, five are chosen to represent the US in the International Physics Olympics. In 1995 and 1996, this group trained at the University of Maryland in College Park and we took the opportunity to have them complete survey forms. The total number of respondents in this group is N=56. Although they are not teachers, they have been selected by experts as some of the best high school physics students in the nation. Our hypothesis was that they would prove to be more expert than the average university physics student, but not as expert as our groups of experienced instructors.

The physics instructors who served as our test groups were all visiting Dickinson College. Attendees came from a wide variety of institutions. Many have had considerable experience in teaching, and all of them were sufficiently interested in educational development to attend a workshop. We separated them into three groups: Group 3 high school teachers attending a two-week summer seminar (N=26), Group 4 college and university teachers attending the two-week summer seminar (N=56), and Group 5 college and university teachers implementing Workshop Physics in their classroom (N=19). The teachers in Group 5 were committed to implementing an interactive engagement model of teaching in their classroom. We asked the three groups of instructors to respond with the answer they would prefer their students to give. We expected these five groups to show an increasing level of agreement with answers we preferred.

B. The Responses of the Calibration Groups

The group we expected to be the most sophisticated, the group 5 instructors, agreed strongly as to what were the responses they would like to hear from their students. On all but three items, ~80% or more of this group agreed with a particular position . Three items, numbers 7, 9, and 34, had a strong plurality of agreement, but between and of the respondents chose neutral. We define the preferred response of Group 5 as the expert response. We define a response in agreement with the expert response as favorable and a response in disagreement with the expert response as unfavorable. For the analysis in this paper, the agree and strongly agree responses (4 and 5) are added together, and the disagree and strongly disagree responses (1 and 2) are added together. A list of the favorable responses to the survey items is presented in Table 3.

Table 3

1	D	8	D	15	D	22	D	29	D
2	D	9	(D)	16	D	23	D	30	A
3	A	10	D	17	D	24	D	31	A
4	D	11	A	18	A	25	A	32	A
5	A	12	D	19	D	26	A	33	D
6	A	13	D	20	D	27	D	34	(A)
7	(A)	14	D	21	D	28	D

Table 3: Prevalent responses of our expert group. Where the respondents did not agree
at the >80% level, the item is shown in parentheses and the majority response is shown.
The response "A" indicates agree or strongly agree. The response "D" indicates disagree
or strongly disagree.

To display our results in a concise and easily interpretable manner, we introduce an agree-disagree (A-D) plot. In this plot, the percentage of respondents in each group answering favorably are plotted against the percentage of respondents in each group answering unfavorably. Since the fraction of students agreeing and disagreeing must add up to less than or equal to 100%, all points must lie in the triangle bounded by the corners (0,0), (0,100), (100,0). The distance from the diagonal line is a measure of the number of respondents who answered neutral or chose not to answer. The closer a point is to the upper left corner of the allowed region, the better the group's agreement with the expert response.[24]

The results on the overall survey are shown in Fig. 1. In this plot, the percentages are averaged over all of the items of the survey, using the preferred responses of calibration group 5 as favorable. The groups' responses are distributed from less to more favorable in the predicted fashion.[25]

Fig. 1: A-D plot for validation groups, average of all items. The percentage of respondents agreeing with the majority of experts’ views
(favorable responses) is plotted against the percentage disagreeing with those views (unfavorable responses).

Although the overall results support our contention that our survey correlates well with an overall sophistication of attitudes towards doing physics, the cluster results show some interesting deviations from the monotonic ordering. These deviations are quite sensible and support our use of clusters as well as overall results. In order to save space and simplify the interpretation of results, we present the data in Table 4. Displayed in this table are the percentages of each group's favorable and unfavorable responses (in the form favorable/unfavorable). The percentage of neutrals and not answering can be obtained by subtracting the sum of the favorable and unfavorable responses from 100.

From the table we see that most of the fraction of respondents agreeing with the favorable response tends to decrease monotonically from group 1-5 with a few interesting exceptions. The high school teachers (group 3) are farther than their average from the favorable corner in the coherence and math clusters, while the Physics Olympics team is closer to the favorable corner in those categories than their average. These results are plausible if we assume that high school teachers are less concerned with their students forming a coherent and a mathematically sophisticated view of physics than are university teachers. The results also agree with our personal observations that the members of the USIPOT are unusually coherent in their views of physics and exceptionally strong in their mathematical skills.

Note also that the Olympics team results are very far from the favorable corner in the effort cluster. The main discrepancies are in items 3 and 7. We suggest that reader peruse the survey items of that cluster (3, 6, 7, 24, 31). These items represent highly traditional measures of effort (reading the textbook, going over one's lecture notes) which we conjecture are not yet part of the normal repertoire of the best and brightest high school physics students before they enter college. We also conjecture that most of them will have to learn to make these kinds of efforts as they progress to increasingly sophisticated materials and the level of challenge rises.

This analysis of both the overall responses of the calibration groups and the variations in the ordering confirms that the MPEX survey provides a quantitative measure of characteristics which experts hope and expect their students to have.

Notes

18. Hammer, "Epistemological beliefs in introductory physics," Cognition and Instruction 12, 151-183 (1994); D. Hammer, "Students' beliefs about conceptual knowledge in introductory physics," Int. J Sci. Ed. 16, 385-403 (1994).
19. In addition to the items representing these clusters, the survey contains additional items whose results (and shifts) we believe are also interesting, but which are associated with a student's style of approaching physics. Items 5, 9, 23, 28, 30, 33, and 34 fall into this category.
20. Hammer, "Two approaches to learning physics," The Physics Teacher 27, 664-670 (1989).
21. Classes such as the one described by Hammer may appear to satisfy both the teacher and some students, but they can do damage if they focus on a superficial success at manipulation of a poorly understood content while neglecting the "hidden" curriculum of meta-concept development.
22. The ability of an individual to hold conflicting views depending on circumstances is a fundamental tenet of our learning model. See ref. 5 and R. Steinberg and M. Sabella, "Student performance on multiple choice questions vs. open-ended exam problems", The Physics Teacher 35 (3) 150-155 (1997) for more discussion of this point.
23. How students think, and how students think they think, are not necessarily the same: Cf. the chapter "The Tune's My Own Invention" from Through the Looking Glass, by Lewis Carroll.
24. The device of plotting three numbers whose sum is fixed in a triangle is well known in elementary particle physics as a Dalitz plot. In our case, the percentage responding agree, disagree, and neutral must add up to 100%.
25. Note that we have included all items, including those marked with parentheses in Table 3. As remarked above, even though the agreement on these items is not as strong, there is still a strong plurality of our experts in favor of the indicated responses. The shift in the position of the overall items resulting from removing these items is on the order of a few percent and the relative order of the groups is not modified.

RETURNS

University of Maryland	Physics Department	PERG UMD	The MPEX Project

page prepared by E. F. Redish
email:redish@quark.umd.edu
last updated 6/17/97