Collaboration in the Development of Concept Inventories

Alexey Leontyev
Adams State University

As chemists always need high-quality measurement tools, chemistry educational researchers also find themselves in the situation where they need instruments that produce reliable data and valid inferences. However, while the measurements in chemistry are objective and can be observed directly (e.g., melting point), in educational research, variables are latent and cannot be measured as a result of direct observation. Quite often a need for a measurement tool is crucial, for example in quasi-experimental and experimental studies, in which performances of two or more groups are compared to determine the effect of a certain pedagogical intervention.

Among many assessment tools, there are concept inventories. Concept inventories are standardized diagnostic instruments that assess how well students’ conceptual knowledge fits the commonly accepted knowledge in the discipline. From a methodological perspective, a concept inventory is a multiple-choice assessment test that probes the understanding of a single topic. Distracters for questions are composed typically from students’ misconceptions. Misconceptions are any concept that differs from the commonly accepted scientific understanding of the term. Once integrated into cognitive structures, misconceptions become incredibly stable and interfere with subsequent learning. One of the most persistent misconceptions in chemistry is that breaking molecular bonds can release energy. Concept inventories can be instrumental in revealing the information about students’ misconceptions.

For practitioners, concept inventories may be used as a brief formative assessment tool at the end of the corresponding module. Instructors can administer concept inventories to students in their classes, look at the results, and quickly decide if any additional instruction on certain concepts is needed. For educational researchers, concept inventories can be used as a measure of cognitive outcomes for experimental designs that, for example, compare active learning with traditional lecture instruction or any other instructional strategies that are intended to create a student-centered environment. These research studies can contribute to the body of evidence-based literature and inform the chemistry community. One of the most influential chemistry education researchers, Melanie Cooper (2007), reported that faculty who develop educational strategies often try to promote them even if there is only anecdotal evidence of their effectiveness. Quite often they do not engage in collecting evidence to support the effectiveness of their method of instruction.

Treagust (1988) in his seminal paper described a developmental process for diagnostic tests, including concept inventories, which has been implemented to create a number of assessment instruments. He presented three broad areas required for creating these tests: “defining the content,” “obtaining information about students’ misconceptions,” and “developing a diagnostic test.” The testing process follows an iterative design in which a version of the instrument is developed, administered to a sample of students, and analyzed. Usually, revisions are made based upon the outcome of the analysis. The subsequent version is then administered to another sample of participants, and the process iterates. Many researchers who have developed diagnostic tools modified or eliminated steps proposed by Treagust. No two instruments are alike, neither in the development process nor in the implication for practice.

Chemistry Collaborations, Workshops, and Community of Scholars is an initiative sponsored by the National Science Foundation intended to facilitate faculty development workshops, including the Active Learning in Organic Chemistry (ALOC) workshop. The ALOC workshop is offered every year for 20-25 organic chemistry instructors from all other the country. The diverse ALOC community played a big role in the development of two concept inventories, the Aldehyde and Ketones Concept Inventory and the Alkene Structure and Reactivity Concept Inventory. By giving voice to the community of experts, we enhance a credibility of newly born concept inventories. We encourage participants of the future workshops to try concept inventories in their classes and contribute to the development of high-quality assessment tools.

References

• Cooper, M. M. (2007). Data-driven education research. Science, 317, 1171.
• Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10(2), 159–169.