Expanding The Model Of Item Writing Expertise: Cognitive Processes And Requisite Knowledge Structures

Background

A number of studies have attempted to establish a research base for writing assessment items (e.g., Wesman, 1971; Millman & Greene, 1989; Bejar, 1993; Gorin, 2006). In these studies, an understanding of test takers’ response processes and knowledge structures is used to predict the psychometric properties of the items. This same knowledge of test takers’ responses and knowledge structures is used to construct items that measure specific aspects of the test framework, e.g., reasoning versus problem solving.

However, research supporting a research-based approach to writing items has tended to overlook the item writers themselves. Few studies have examined cognitive models of item writers’ writing processes and knowledge structures. The development of a cognitive model of item-writing expertise holds promise for improving the quality of the written items. Careful examination of item writers’ cognitive processes and knowledge structures should provide insight into yet another aspect of item writing and thus further inform efforts to improve the quality of items at an early phase of development by addressing and resolving areas of need in item writers’ knowledge and skills related to item construction. Such an improvement effort may be especially useful in relation to writing innovative and technology-enhanced items. One example of how this improvement might be effected is through incorporation of information from an item-writing cognitive model into item writer training workshops and guides.

Several studies of studies of item writers’ cognition were completed by Fulkerson and his colleagues. In an earlier study of experienced item writers’ cognition, Fulkerson, Mittelholtz, and Nichols (2009) found that expert item writers engaged in three phases of problem solving.