Research, Field Testing and Evaluation
An Example of Effective Sequencing of Problems
CMP: A Curriculum Co-Developed with Teachers and Students
Research, Field Testing and Evaluation
Before starting the design phase of the materials, we commissioned individual reviews of the first edition of CMP units from 84 individuals in 17 states and comprehensive reviews from more than 20 schools in 14 states.
Individual Reviews These reviews focused on particular strands over all three grades (such as number, algebra, or statistics) on particular sub- populations (such as students with special needs or those who are commonly underserved), or on topical concerns (such as language use and readability).
Comprehensive Reviews These reviews were conducted in groups that included teachers, administrators, curriculum supervisors, mathematicians, experts in special education, language, and reading level analyses, English language learners, issues of equity, and others. Each group reviewed an entire grade level of the curriculum. All responses were coded and entered into a database that allowed reports to be printed for any issue or combination of issues that would be helpful to an author or staff person in designing a unit.
In addition, CMP issued a call to schools to serve as pilot schools for the development of CMP2. We received 50 applications from districts for piloting. From these applications we chose 15 that included 49 school sites in 12 states and the District of Columbia. We received evaluation feedback from these sites over the five-year cycle of development.
Based on the commissioned reviews, what the authors had learned from CMP schools over a 6-year period, and input from our Advisory Board, the authors started with grades 6 and 7 and systematically revised and restructured the units and their sequence for each grade-level to create a first draft of the revision. These were sent to our pilot schools to be taught during the second year of the project. These initial grade level unit drafts were the basis for substantial feedback from our trial teachers.
Examples of the kinds of questions we asked the trial teachers following each iteration of a unit or grade level are given below.
Big Picture Unit Feedback
- Is the mathematics of the unit important for students at this grade level? Explain.
- Are the mathematical goals of the unit clear to you?
- Overall, what are the strengths and weaknesses in this unit?
- Please comment on your students' achievement of mathematics understanding at the end of this unit. What concepts/skills did they "nail"? Which concepts/skills are still developing? Which concepts/skills need a great deal more reinforcement?
- Is there a flow to the sequencing of the Investigations? Does the mathematics develop smoothly throughout the unit? Are there any big leaps where another problem is needed to help students understand a big idea in an Investigation? What adjustments did you make in these rough spots?
Problem-by-Problem Feedback
- Are the mathematical goals of each problem/investigation clear to you?
- Is the language and wording of each problem understandable to students?
- Are there any grammatical or mathematical errors in the problems? (Please be as specific as possible.)
- Are there any problems that you think can be deleted?
- Are there any problems that needed serious revision?
Applications- Connections-Extensions Feedback
- Does the format of the ACE exercises work for you and your students? Why or why not?
- Which ACE exercises work well, which would you change, and why?
- What needs to be added to or deleted from the ACE exercises? Is there enough practice for students? How do you supplement and why?
- Are there sufficient ACE exercises that challenge your more interested and capable students? If not, what needs to be added and why?
- Are there sufficient ACE exercises that are accessible to and helpful to students that need more scaffolding for the mathematical ideas?
Mathematical Reflections and Looking Back-Looking Ahead Feedback
- Are these reflections useful to you and your students in identifying and making more explicit the "big" mathematical ideas in the unit? If not, how could they be improved?
Assessment Materials Feedback
- Are the check-ups, quizzes, unit tests, and projects useful to you? If not, how can they be improved? What should be deleted and what should be added? (Please give specifics.)
- How do you use the assessment materials? Do you supplement the materials? If so, how and why?
Teacher's Guides Feedback
- Is the Teacher's Guide useful to you? If not, what changes do you suggest and why?
- Which parts of the Teacher's Guide help you and which do you ignore or seldom use?
- What would be helpful to add or expand in the Teacher's Guide?
Year-End Grade Level Feedback
- Are the mathematical concepts, skills and processes in the units appropriate for the grade level?
- Is the grade level placement of units optimal for your school district? Why or why not?
- Does the mathematics flow smoothly for the students over the year?
- Once an idea is learned, is there sufficient reinforcement and use in succeeding units?
- Are connections made between units within each grade level?
- Does the grade level sequence of units seem appropriate? If not, what changes would you make and why?
- Overall, what are the strengths and weaknesses in the units for the year? (Please be as specific as possible.)
Big Picture Question
- What three to five things would you have us seriously improve, change, or drop at each grade level? Please be specific about exactly what you suggest and why you would like to see this change.
Development Summary
CMP development followed the very rigorous design, field-test, evaluate loop pictured in the diagram below.
The units for each grade level went through at least three cycles of field trials-data feedback- revision. If needed, units had four rounds of field trials. This process of (1) commissioning reviews from experts, (2) using the field trials- feedback loops for the materials, (3) conducting key classroom observations by the CMP staff of units being taught, and (4) monitoring student performance on state and local tests by trial schools comprises research-based development of curriculum. This process takes five years to produce the final drafts of units that are sent to the publisher. Another 6 to 18 months is needed for editing, design, and layout for the published units. This process produces materials that are cohesive and effectively sequenced.
An Example of Effective Sequencing of Problems
To be effective, problems must be carefully sequenced to help students develop appropriate understanding and skill. The following set of problems from the Grade 6 unit, Covering and Surrounding, develops methods for finding the circumference and area of a circle.
The first problem uses the context of irregularly-shaped lakes to explore possible relationships between the perimeter of a curved figure and its area. Using a square grid to estimate perimeter and area helps students understand the meaning of perimeter and area before using formulas.
Covering and Surrounding - page 71
In the second problem, students measure the diameter and circumference of several circular objects. They create a table and a graph of their data and look for a pattern relating the two measurements. Students should discover that the circumference is the diameter times "a little bit more than 3." With the help of the teacher, students are introduced to the idea of pi and find a closer approximation of its value.
Covering and Surrounding - page 73
The third problem asks students to estimate the area of a circle. Students are encouraged to think of several different methods and to explain their thinking. This problem is intended to motivate the need for a shortcut for calculating the area. To answer Parts C and D, students must consider the relationships between each of the measurements-radius, diameter, circumference, and area-and the possible price of each pizza.
Covering and Surrounding - page 75
In the fourth problem, students estimate the number of "radius squares" (squares with side length equal to the radius of the circle) it takes to cover a circle. This problem helps students discover the formula for the area of a circle and to understand why it makes sense. Students should find that the area of a circle is "a little bit more than 3" radius squares. With the help of the teachers, students relate "a little bit more than 3" to the number π, and develop the area formula A=πr2. Mental images such as the square embedded in a circle trigger a way for students to recall the formula for the area of a circle and to remember why the formula makes sense.
Covering and Surrounding - page 77
The sequencing of this set of problems and its effectiveness is reflective of the interactions between the authors and the teachers and students at our trial sites.
CMP: A Curriculum Co-Developed With Teachers and Students
Developing a curriculum with a complex set of interrelated goals takes time and input from many people. As authors, our work was based on a set of deep commitments we had about what would constitute a more powerful way to engage students in making sense of mathematics. Our Advisory Board took an active role in reading and critiquing units in their various iterations. In order to enact our development principles, we found that three full years of field trials in schools was essential for each year of the materials. This feedback from teachers and students across the country is the key element in the success of the Connected Mathematics Project materials. The final materials comprised the ideas that stood the test of time in classrooms across the country. Nearly 200 teachers in 15 trial sites around the country (and their thousands of students) are a significant part of the team of professionals that made these materials happen. The scenarios of teacher and student interactions with the materials became the most compelling parts of the Teacher's Guides.
Without the bravery of these teachers in using materials that were never perfect in their first versions, CMP would have been a set of ideas that lived in the brains and imaginations of the five authors. Instead, they are materials with classroom heart because our trial teachers and students made them so. We believe that such materials have the potential to dramatically change what students know and are able to do in mathematical situations. The emphasis on thinking and reasoning, on making sense of ideas, on making students responsible for both having and explaining their ideas, on developing sound mathematics habits gives students opportunities to learn in ways that can change how they think of themselves as learners of mathematics.
From the authors' perspectives, our hope is to develop materials that play out deeply held beliefs and firmly grounded theories about what mathematics is important for students to learn and how they should learn it. We hope that we have been a part of helping to challenge and change the curriculum landscape of our country. Our students are worth the effort. Authors and Staff
