Fig. 2a: A-D plot for large schools, average of all items...... Fig. 2b: A-D plot for small schools, average of all items.
To return to part 1 of this paper, click here.
To return to part 2 of this paper, click here.
To go on to part 4 of this paper, click here.
In this section, we discuss the results obtained from giving the MPEX survey at the beginning and end of the first term of introductory calculus-based physics at six different institutions. In each case, the subject covered was Newtonian mechanics. The schools involved include the flagship research institutions of three large state universities: the University of Maryland (UMCP), Ohio State (OSU), and Minnesota (UMN); plus three smaller schools: Dickinson College (DC), a small public liberal arts college (PLA), and a public two year college (TYC). At the named colleges, we have data from multiple instructors. In the case of the last two institutions, data was only collected from a small number of instructors and students. These are included in order to demonstrate how the MPEX survey can be used as a diagnostic tool, but are kept anonymous to protect the identity of the instructors and institutions involved.
At Maryland, Ohio State, and Minnesota, classes were presented in the traditional lecture-lab-recitation framework with some modifications. At Maryland, there is no laboratory in the first semester and some of the recitation sections were done with University of Washington style tutorials.[26] Results for tutorial and recitation classes were comparable. At Minnesota, the laboratory and recitations involve carefully designed problem-solving group work.[27] At Ohio State, lectures are traditional but are enhanced by use of various interactive elements, while recitation and laboratory are done in a group problem-solving format somewhat similar to that developed at Minnesota. At Dickinson College and at the public liberal arts institution, the classes surveyed were done in the Workshop Physics environment which replaces lectures with a combined lab and class discussion.3d The two-year college used a purely traditional lecture-recitation framework. Like Maryland, they have no lab in the first semester. The schools involved, the structure of their courses, and the number of students in our sample are summarized in Table 1.
In order to eliminate the confounding factor of differential drop-out rates, we only include students who completed the survey both at the beginning and at the end of the term. We say that the data is matched. Our results show some differences among different classes at the same institution, but the variation is statistically consistent with the sample size. Therefore, we have combined results for similar classes at a given institution.
The overall survey results for the six schools are presented in an A-D plot in Fig. 2. In order to simplify the reading of the graphs, we have displayed the results from the three large research universities in one part of the figure (Fig. 2a) and those from the smaller schools in another (Fig. 2b). The pre-course results are shown by filled markers and the post-course results by open markers. The results of the expert group are shown by a cross.
We make two observations.
The overall survey includes items that represent a variety of characteristics, as displayed in Table 2. In order to better understand what is happening in the classes observed, let us consider the initial state and the change of student expectations in our various clusters. The results are presented in Table 4.
| Overall | Indepen-dence | Coherent | Concept | Reality Link | Math Link | Effort | |
| Experts | |||||||
| College | |||||||
| HS | |||||||
| POT | |||||||
| UMCP pre | |||||||
| UMCP post | |||||||
| UMN pre | |||||||
| UMN post | |||||||
| OSU pre | |||||||
| OSU post | |||||||
| DC pre | |||||||
| DC post | |||||||
| PLA pre | |||||||
| PLA post | |||||||
| TYC pre | |||||||
| TYC post |
One characteristic of the binary thinker, as reported by Perry and BGCT, is the view that answers come from an authoritative source, such as an instructor or a text, and it is the responsibility of that authority to convey this knowledge to the student. The more mature students understand that developing knowledge is a participatory process. Hammer classifies these two extreme views as "by authority" and. "independent." Survey items 1, 8, 13, 14, 17, and 27 probe students' views along this dimension. On this cluster, students' initial views were favorable in a range from 41% (TYC) to 62% (DC). All groups showed essentially no significant change as a result of one term of instruction. For comparison, the USIPOT showed favorable views on these items 81% of the time.
Survey items 1 and 14 are particularly illuminating and show the largest gaps between experts and novices.
#1: All I need to do to understand most of the basic ideas in this course is just read the text, work most of the problems, and/or pay close attention in class.
#14:. Learning physics is a matter of acquiring new knowledge that is specifically located in the laws, principles, and equations given in the textbook and in class.
The expert group was in 100% agreement that students should disagree with item 1 and in 84% agreement that they should disagree with item 14. Disagreeing with these items represents a rather sophisticated view of learning, but favorable shifts on these items are exactly the sort of changes that indicate the start of a transition between a binary and a more constructivist thinker. The interviews strongly support this view. Students who disagreed with these items were consistently the most vigorous and active learners.
This cluster of items, and items 1 and 14 in particular, appear to confirm that most students in university physics enter with at least some characteristics of binary learners, agreeing that learning physics is simply a matter of receiving knowledge in contrast to constructing one's own understanding. We would hope that if a university education is to help students develop more sophisticated views of their own learning, that the introductory semester of university physics would begin to move students in the direction of more independence. Unfortunately, this does not appear to have been the case. In the touchstone items of 1 and 14, the only significant improvement was DC on item 14 (26% to 53%), and overall, only DC showed improvement.
Most physics faculty feel strongly that students should see physics as a coherent, consistent structure. A major strength of the scientific worldview is its ability to describe many complex phenomena with a few simple laws and principles. Students who emphasize science as a collection of facts fail to see the integrity of the structure, an integrity that is both epistemologically convincing and useful. The lack of a coherent view can cause students many problems, including a failure to notice errors in their reasoning and an inability to evaluate a recalled item through cross-checks. Survey items 12, 15, 16, 21, and 29 have been included in order to probe student views along this dimension.
Our expert group was in agreement as to what responses were desirable on the elements of this cluster 85% of the time. The initial views of students at our six schools were only favorable between 50% and 58% of the time. Most classes showed a small deterioration on this cluster, except for UMN (slight improvement from 57% to 61% favorable responses) and DC (improvement of 58% to 66% favorable responses).
Two specific items in this cluster are worthy of an explicit discussion.
#21: If I came up with two different approaches to a problem and they gave different answers, I would not worry about it; I would just choose the answer that seemed most reasonable. (Assume the answer is not in the back of the book.)
#29: A significant problem in this course is being able to memorize all the information I need to know.
Item 21 is a touchstone. Coming up with two different answers using two different methods indicates something is seriously wrong with at least one of your solutions and perhaps with your understanding of the physics and how to apply it to problems. Our expert group and USIPOT students feel strongly that students should disagree with item #21 at the 85% level. Initially, only 42-53% of students produced a favorable response for this item, and only DC showed any significant improvement on this item (52% to 59%). One school (PLA) showed a substantial deterioration (42% to 17%).
The interpretation of item #29 may depend significantly on the details of the examination structure of the course being probed. A sophisticated student will realize that the large number of different equations and results discussed in a physics text can be structured and organized so that only a small amount of information needs to be memorized and the rest can be easily rebuilt as needed. Item #29 is part of a probe into whether or not students see this structure or are relying on memorizing instead of rebuilding. However, if students are permitted to use a formula sheet or if exams are open book, they may not perceive memorization as a problem. This does not mean that they see the coherence of the material.[28] If extensive information is made available to students during exams, item #29 needs to be interpreted carefully. A variety of examination aids were used for the classes of this study, ranging from open-book exams (DC) to no aids (UMCP). Omission of item #29 does not change the distributions in this cluster significantly.
The group of items selected for the concepts cluster (items 4, 19, 26, 27, and 32), are intended to probe whether students are viewing physics problems as simply a mathematical manipulation of an equation, or whether they are aware of the more fundamental role played by physics concepts in complex problem solving. For students who had high-school physics classes dominated by simple "problem solving" (find the right equation, perhaps manipulate it, then calculate a number), we might expect largely unfavorable responses on our items. We would hope, however, for substantial improvement, even as the result of a single college physics course.
Our experts agree on their responses to the items of this cluster 89% of the time. The initial views of the students at the six schools were favorable between 30% (TYC) and 47% (DC) of the time. All schools showed some improvement on this cluster except OSU which showed a small deterioration (37% to 35% favorable responses). The two Workshop Physics schools showed the largest gains in favorable responses (DC 47% to 58%, PLA 38% to 45%).
Within this cluster, the results on items 4 and 19 are particularly interesting.
#4: "Problem solving" in physics basically means matching problems with facts or equations and then substituting values to get a number.
#19: The most crucial thing in solving a physics problem is finding the right equation to use.
While these items are similar, they are not identical. Agreeing with item 4 indicates a naive view of physics problems or a lack of experience with complex problems. A more experienced student could reject 4 but still agree with 19 because of the phrase "most crucial". One would, however, hope that increased experience with complex physics problems would lead a student to disagree with this item as well. For example, 54% of the USIPOT students gave a favorable response on this item as compared to only 22% of beginning students at UMCP. Our personal observations of these students indicate that as expected, the USIPOT students have considerably more experience with complex problem solving than the typical beginning engineering student.
| University of Maryland | Physics Department | PERG UMD | The MPEX Project |
|---|---|---|---|
 |
 |
 |
 |