Design and Validation of a Multimethod Assessment of Metacognition and Study of the Effectiveness of Metacognitive Interventions

Abstract

The central role of metacognition in learning and problem solving, in general and in chemistry in specific, has been substantially demonstrated and has raised pronounced interest in its study. However, the intrinsic difficulties associated with the inner processes of such a non-overt behavior have delayed the development of appropriate assessment instruments. The first research question addressed in this work originates from this observation: Is it possible to reliably assess metacognition use in chemistry problem solving? This study presents the development, validation, and application of a multimethod instrument for the assessment of metacognition use in chemistry problem solving. This multimethod is composed of two independent methods used at different times in relation to the task performance: (1) the prospective Metacognitive Activities Inventory, MCA-I; and (2) the concurrent Interactive MultiMedia Exercises software package, IMMEX. This work also includes the design, development, and validation of the MCA-I; evidence is discussed that supports its robustness, reliability and validity. Even though IMMEX is well-developed, its utilization as a metacognition assessment tool is novel and explained within this work. Among the benefits of utilizing IMMEX are: the automation of concurrent evidence collection and analysis which allows for the participation of large cohorts, the elimination of subjective assessments, and the collection of data in the absence of observers which presumably favors a more realistic deployment of skills by the participants. The independent instruments produced convergent results and the multimethod designed was proven to be reliable, robust and valid for the intended purpose. The second guiding question refers to the development of metacognition: Can regulatory metacognition use be enhanced by learning environments? Two interventions were utilized to explore this inquiry: a Collaborative Metacognitive Intervention and a Cooperative Problem-Based Laboratory Project. The former was designed and developed as part of this study; the latter is part of the curriculum of a two-semester cooperative General Chemistry Laboratory course. Both interventions rely on two main axes to promote metacognition development: intense social interaction and induced reflection. In the first case, it is through small group collaborative work and guided and peer prompting; in the second one through cooperation and inquiry in the laboratory. The effect of both interventions was investigated using pre and posttest, control and treatment type experiments. The choice of assessment was the multimethod developed in the first part of this same study. Despite the differences between the interventions (length, nature of prompting, and relation to chemistry contents) both learning environments succeeded in enhancing the awareness and use of metacognition in chemistry problem solving. Findings support the assertion that the mechanisms that define the learning environments under study--social interaction and reflection--promote the enhancement of metacognitive skills. A significant corollary of this research is that it offers evidence of the laboratory as a learning environment where students can acquire high order thinking skills and develop content knowledge and understanding.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 22 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.