Conducting CFIP

STEP 3B: IDENTIFY THE MAJOR PATTERNS OF STUDENTS' STRENGTHS AND NEEDS AT THE CLASS LEVEL, USING MULTIPLE DATA SOURCES.

The second part of CFIP's step 3 asks team members to identify patterns of strengths and weaknesses for the class as a whole from several data sources.

Because no single piece of information about student achievement gives a complete and accurate picture of a school, team, or student, multiple data sources are necessary to help provide a broader and more balanced look at the results. Triangulation is the process of bringing together data from several sources (such as external, benchmark, and classroom assessments) to form logical conclusions that may not be evident when only one data type is considered.

Using multiple sources of data on which to base decisions:

• Reduces the anxiety (for teachers and students) of relying on a single high-stakes measure as the sole definition of student success.
• Reduces mistakes and gives team decisions increased validity. The accuracy of student achievement data is higher the more opportunities that students are given to show in varied formats what they know and can do.
• Provides useful insights that are often not evident in single data sources. For example, students who do not have to justify their responses in class may have difficulty when asked to do so on a benchmark assessment, poor writers may know the content but not be able to explain it clearly in a brief constructed response (BCR), and students who are accustomed to constructing their own responses may have difficulty choosing the one best answer in multiple choice items.
• Maintains a desirable balance between decisions based on teacher judgment in classroom assessments and those resulting from externally-developed more standardized measurements.
• Provides more frequent evidence of student learning on which teachers can act.

Educators are often warned that it is not possible to mix data types, such as external assessments with high validity and reliability with classroom assessments that lack a similar rigor in their design and implementation.

However, data expert Stephen White has noted that "the problem of data in the public schools is the fact that the data available are seldom perfect. . . . The triangulation process has been employed for centuries for the very purpose of gleaning meaning from imperfect and incomplete data,. . .data that are varied, unrelated, and collected at different times for different purposes."³¹

White continues that, "though there are certainly instances in which data should not be compared in the statistical sense, the complexity of education compels us to look for patterns and trends in the practical sense that lead us to decisions that improve student achievement, regardless of the type of data."³²

He concludes that "triangulation is a messy process. It requires teams to make assumptions, draw inferences, and come to conclusions without total certainty. When data are triangulated, each point serves as a check on the other dimensions, with the desired outcome . . . always being the realization of new insights that are not available from examining only one type of data or one perspective."³³

Here are two key questions to include in the search for patterns from multiple data sources:

• What knowledge and skills are the most important overall class strengths, according to more than one source?
• What knowledge and skills are the most important overall class weaknesses, according to more than one source?

This conversation also gives school teams the opportunity to dialogue about inconsistencies or the lack of patterns when findings from multiple sources are compared. Questions such as these could be used:

• Are these the results we expected from each source? Why or why not?
• What is puzzling about the data from multiple sources?
• What conflicting results emerge when multiple data sources are considered?
• To what might we attribute differences among student results in the varying data sources?
• In what ways can we provide students additional opportunities to show what they know and can do in a variety of assessment formats and types?