Evaluating The Cognitive, Psychometric, And Instructional Affordances Of Curriculum-Embedded Assessments: A Comprehensive Validity-Based Approach

Project Description

The overarching goal of this project is to evaluate the embedded assessment components within standards-based K-12 mathematics curricula. To do this we will apply a comprehensive assessment validity perspective that integrates a variety of empirical evidence regarding the cognitive, psychometric, and instructional affordances of multiple assessments embedded in curricula as part of their overall instructional design. We illustrate this evaluation process by working with two NSF-supported mathematics curricula: Everyday Mathematics and Math Trailblazers.

A key assumption underlying this research is that effective implementation of quality mathematics curricula depends on the presence of embedded, high quality and informative assessments of student learning. Empirical data that establishes such characteristics for the assessments woven into these two curricula is currently lacking. Thus, the project’s specific objectives are to determine:

1. What the curriculum-embedded assessments reveal about student learning, including what students understand and how they understand it, as well as particular and unique assessment issues that arise with linguistically and ethnically diverse populations;
2. the assessment activity types and scoring approaches that are most informative for purposes of instructional decision making, improving student learning and/or for projecting performance on external summative assessments;
3. what teachers understand about student learning based on the information produced from curriculum-embedded assessments, and how teachers can make use of that information to assist student learning;
4. ways to improve and enhance assessment activities and scoring approaches and develop teacher professional development materials that can facilitate effective use of assessment in instruction; and
5. a broad dissemination strategy that is targeted to cross-disciplinary researchers as well as practitioners including curriculum designers, teacher preparation programs, school and district personnel responsible for managing and evaluating curricula and teacher professional development, and teachers.

The project’s overall design represents an innovative collaboration of curriculum designers, cognitive scientists and psychometricians. The five-year work plan will be carried out in the context of the Chicago Public School’s Mathematics-Science Initiative, a comprehensive, multiyear effort to improve mathematics teaching and learning. Multiple schools, classrooms, teachers, and students will be part of the sampling design and data collection, representing the full ethnic and linguistic diversity of Chicago’s K-8 population. The research design and data collection plan includes in-depth cognitive and curricular analyses of the curricular structure and the curricular and instructional role played by embedded assessments, protocol studies that will include observation and interview of students performing embedded assessment activities, classroom observation and interviews with teachers, and large scale data collection and analysis of student embedded assessment performance gathered from students in numerous classrooms.

Project findings are expected to contribute to understanding: (a) what constitutes quality assessment in elementary school mathematics and how it supports student learning as part of an ongoing and coherent instructional process tied to nationally defined standards (NCTM, 2000); (b) what teachers need to know to make effective use of such assessments in their practice; and (c) the research foundations for the coordinated cognitive, curricular, instructional, and psychometric study of embedded assessments. The results will contribute to curriculum developers’ understandings about how embedded assessments can reveal important variation in students’ knowledge and understanding, especially among diverse student groups. Little is currently understood about how to provide all students with good opportunities to “show what they know,” to support students’ understandings and build on their own ways of knowing, including providing opportunities: (1) to reference prior experiences and background knowledge; (2) to enable ready use of a combination of forms of representations; and (3) to explain their own understandings using linguistic forms that serve them well such as, e.g., first language or dialect. Our findings, situated in the ethnically and linguistically diverse Chicago Public Schools, are expected to provide a strong contribution to the research, curriculum design, and professional development literatures relevant to providing equitable and principled opportunities for all students in mathematics learning.

• Kimberly Gomez