Constructing Padi Measurement Models for the BEAR Scoring Engine (Padi Technical Report 7)

Abstract

Representing complex science inquiry tasks for item response modeling (IRM) presents a number of challenges for the assessment designer. Typically, such tasks provide evidence of multiple aspects of learning and involve sequential or interdependent responses. The BEAR Scoring Engine is introduced as a software tool to compute proficiency estimates for such tasks. In addition, reusable data structures containing measurement model specifications are presented as a technique to enhance internal assessment coherence, improve the interpretability of statistical analyses of student responses, and speed the process of developing new assessment tasks that are consistent with articulated learning goals. This report begins by defining an assessment system as comprising task design and delivery components. The Principled Assessment Designs for Inquiry (PADI) design system is then introduced and positioned within the assessment system framework. Next, we describe the role of measurement models in operationalizing the manner in which inferences are drawn from observations and interpreted in an assessment system. Connections among the task design, student work products, evaluation, and inferential reasoning are highlighted, and the BEAR Scoring Engine is presented as one example of how multidimensional item response modeling can be integrated with the PADI design system. Finally, several examples of assessment tasks common to science inquiry are developed to illustrate how they would be implemented with this particular scoring engine.