Measuring Complex STEM Thinking Using Epistemic Network Analysis

This project attempts to provide a transformative and novel answer to a foundational issue for science, technology, engineering and mathematics (STEM) learning: How can we measure the development of complex STEM thinking?

Young people today need to develop complex thinking in STEM in order to participate in the social, economic, and cultural life of a globalized world. Yet we cannot create curricula, programs, or activities to teach complex STEM thinking unless we can measure and show whether students have developed it.

In this project, we propose to develop epistemic network analysis (ENA) as a toolkit for measuring complex STEM thinking. We propose to develop and test the toolkit initially by measuring the extent to which a person is thinking like an engineer in STEM learning games, engineering courses, and in the engineering profession. Importantly, however, our goal is to develop tools and techniques that will be applicable to any form of complex STEM thinking, and thus our research and dissemination plan is explicitly designed to extend ENA to other STEM domains.

This project is based on the psychological theory of epistemic frames, which suggests that complex thinking in any field requires more than simply knowing basic facts and having basic skills. Epistemic frame theory suggests that complex STEM thinking requires a shared base of knowledge and skills, but also an understanding of the values that guide the use of those skills and of how to make decisions and justify actions to solve problems.

More important, complex thinking does not just mean having a set of knowledge, skills, values, and ways of making decisions. It means understanding how these different elements of problem solving are connected: which values to consider before taking a certain action, what knowledge to gather before making a particular kind of decision, and so on. This means the development of STEM thinking can be quantified by a model of that network of connections.

The project is organized around a design cycle consisting of three key activities: (1) research on the psychometric properties of ENA, (2) research on the most effective means of visualizing (and thus communicating) the significance of ENA analyses based on those psychometric properties, and (3) disseminating the ENA toolkit to potential users.