After Design Testing and Failure, then What? Using Elementary Student Avatars to Practice Teaching a Post-Testing Engineering Argumentation Discussion

Workshop Focus and Relevance

This workshop simultaneously focuses on two challenges for science teacher education:
1) How to facilitate a post-testing engineering argumentation discussion with students.
2) How to provide opportunities for teachers to practice leading these kinds of discussions.

Challenge 1: Post-Testing Engineering Argumentation Discussions

Performance expectations in the Next Generation Science Standards encourage teachers to include engineering design within science instruction (NGSS Lead States, 2013). Engineering design involves students defining problems and solving them through the creation of technologies (i.e., things, processes, or systems). Within science classrooms, these engineering problems and/or their solutions reinforce or apply scientific knowledge. Students solve problems via an engineering design process (EDP). While there are various EDPs, all of them involve essential elements such as problem identification, brainstorming, planning, creating or implementing, testing, and improving (Crismond & Adams, 2012; Moore et al., 2014; Lottero-Perdue, 2017a). Students work in teams to solve problems, each team moving through the process to create their own unique solution to the problem. There is no singularly correct solution (i.e., no ‘one right answer’).

One critical point in an EDP is when students consider the results of their first designs (i.e., their first attempt at solving the problem), analyze design performance, and consider how to improve. Ideally, this thinking should be rooted in evidence-based reasoning (Rynearson, Moore, Tank, & Gajdzik, 2018). Students should draw from their testing results and their knowledge of relevant science ideas to inform their ideas about the next design in the iterative EDP (Siverling, Suazo-Flores, Moore, 2018). However, teams’ initial analyses, responses to design failures (i.e., when designs fail to meet one or more criteria for the challenge), and ideas about improvement range from being naïve to robust (Wilson-Lopez & Garlick, 2017; Lottero-Perdue & Parry, 2017a, 2017b). They may or may not employ evidence-based reasoning. Further, in a classroom setting, teams can learn how to better analyze their own design performance or consider how to improve after listening to other teams’ descriptions of design performance and improvement.

This critical point in the EDP is an opportunity for students to listen to and learn from one another, perhaps changing their initial ideas based on new perspectives and information. It is not meant to have all teams decide on one design to pursue; rather, it is an opportunity for each team to move their ideas forward to solve the problem. This is what we will call henceforth a “post-testing engineering argumentation discussion.” Although argumentation within science education has been identified as an essential practice in science education (Berland & McNeill, 2006; Cartier, Smith, Stein, & Ross, 2013; Chinn, 2006; Duschl & Osborne, 2002; Simon, Erduran, & Osborne, 2006; Windschitl, Thompson, Braaten, & Stroupe, 2012), very little work has addressed what argumentation looks like in engineering or how teachers might practice leading argumentation discussions in engineering (Mathis, Siverling, Glancy, Guzey, & Moore, 2016).

Challenge 2: Practicing Leading Discussions

It is important for preservice and inservice teachers learning new skills and strategies to have opportunities to practice their craft and receive support and feedback as they work towards expertise (Ball & Cohen, 1999; Ball & Forzani, 2009; Benedict, Holdheide, Brownell, & Marshall Foley, 2016; Grossman, Hammerness, & McDonald, 2009; Lampert, 2009; Zeichner, 2012). Traditionally in preservice teacher education, practice teaching is most conspicuous during student teaching. However, many have argued that it is not sufficient for all of the practicing to be done during student teaching (Carrington, Kervin, & Ferry, 2011). Similarly, for inservice teachers, practice is likely to take the form of trying out the new skill or strategy in their classrooms with their students. However, those experiences can be limiting due to their particular classroom contexts, including the students they teach, and the curriculum materials they are required to use. Other forms of practice teaching have included: microteaching or peer teaching (Zhou & Xu, 2017); teaching small groups of K-12 students instead of whole classes (Lottero-Perdue, 2017b; Sandifer, Hermann, Cimino & Selway, 2015); and – more recently – using simulated classroom environments (Dieker, Rodriguez, Lignugaris, Hynes & Hughes, 2013).

This latest form of practice teaching, which can be employed in preservice teacher education or in professional learning experiences, is the focus of this workshop. The simulated classroom environment that we use in our research includes a small group of five upper-elementary (fifth grade) student avatars divided into three smaller engineering design teams. Live actors playing the roles of the students enable the discussion to happen responsively and in real time as would occur in a discussion with real students; that said there are limitations (e.g., the students cannot get up from their desks). We have used this simulated classroom environment in our ongoing research examining how preservice and inservice teachers lead argumentation discussions in science (Mikeska & Lottero-Perdue, 2019). One of the benefits of this simulated classroom environment is that each teacher who leads a discussion in this environment can do so: 1) for the same “scenario” or topic; and 2) with the same students who have the same dispositions, background knowledge and ideas. Thus, comparisons can be drawn across teachers who lead the same discussion with the same students.

In this workshop, we share example data from ongoing work using digital simulations to engage inservice teachers in leading small group, post-testing engineering argumentation discussions. Each teacher enters the simulator and facilitates a 20-minute discussion with the five elementary student avatars. The discussion is video recorded, capturing the teacher, avatars, and a shared workspace on which the teachers and avatars can write.

Anticipated Participants

This workshop will be most pertinent to teacher educators and professional learning providers who: 1) work directly with preservice and inservice elementary and middle-level science teachers, 2) aim to provide coursework and/or professional learning experiences about how to teach science-integrated engineering, and 3) are interested in learning about innovative instructional technologies to support science teacher learning. We also suspect that those who conduct research in argumentation, evidence-based reasoning, and the use of simulated classroom environments would be interested in this session.

Workshop Presenters

Dr. Pamela Lottero-Perdue is a professor of science and engineering education at Towson University (TU). She is currently principal investigator on a TU School of Emerging Technologies grant focused on investigating how expert elementary teachers facilitate small group discussions in post-investigation science and post-testing engineering discussions with student avatars. Dr. Lottero-Perdue was a co-principal investigator on an NSF DRK-12 grant to examine the efficacy of Engineering is Elementary (EiE) and has published research and practitioner articles on engineering design failure.

Dr. Jamie Mikeska is a research scientist in the Student and Teacher Research Center at ETS and is currently principal investigator of two NSF-funded research studies, one of which is designed to develop, pilot, and validate a set of performance-based tasks delivered within a simulated classroom environment in order to improve preservice elementary teachers’ ability to facilitate discussions in science and mathematics. Dr. Mikeska also has experience teaching elementary science methods courses and has studied science teacher learning in professional development across multiple research studies.

Learning Objectives, Activities, Strategies, and Effectiveness

In this workshop session, participants will:
1) Review a classroom scenario that provides background information needed to lead a post-testing engineering argumentation discussion;
2) Consider how an experienced inservice teacher may lead the discussion; and
3) Use a rubric to analyze and evaluate video excerpts of teachers leading this discussion.
Participants’ analysis of these videos will be used to invite discussion about how simulated classroom environments can be leveraged most effectively to provide practice-based learning opportunities to teachers as they learn how to lead post-testing engineering argumentation discussions. Additionally, participants will be invited to discuss feedback and follow-up debrief activities that they could provide and facilitate to support teachers in learning how to engage productively in this teaching practice. The following guiding questions will frame our discussions:
• How might an experienced teacher lead this discussion?
• For each video excerpt of a teaching performance: What did you notice about teaching in the simulated classroom environment?
• What similarities or differences are there between the example teaching performances?
• If these discussions had be led by teachers in your courses or professional learning sessions, what would you do next to support their learning?
• How might you consider integrating these practice discussions into your courses or professional learning sessions?

The workshop agenda will be as follows (min = minutes):
10 min: Introduction to the two challenges of focus for the workshop
20 min: Introduction to the engineering design challenge and scenario
15 min: Discussion about how to approach the post-testing discussion
15 min: Introduction to the rubric
40 min: Video analysis
10 min: Considering how to provide feedback and lead debrief activities
10 min: Final questions and discussion

By engaging in the above activities, participants in this workshop will learn to:
1) Understand the goals of a post-testing engineering argumentation discussion.
2) Understand the affordances of simulated classroom environments for supporting elementary science teachers in learning how to lead a post-testing engineering argumentation discussion.
3) Use a scoring rubric to analyze video excerpts of elementary science teachers’ instruction within simulated classrooms.
4) Consider how to provide feedback and lead debrief activities to best support teachers’ learning how to lead a post-testing engineering argumentation discussion.

We will assess the workshop’s effectiveness in two ways. First, we will request that workshop attendees complete a written feedback form at the conclusion of the workshop to inform us about their perceptions on the activities and structure of the session, what they learned, and what and how they plan to apply what they learned in their own settings. Second, we will also observe the ways in which the participants analyze and discuss the example video recorded performances and the variety and complexity of the ideas that they generate regarding how these tools can be integrated into teacher education settings.


Following the workshop, we will create a shared folder (similar to a shared Google Drive folder) to post handouts and tools used during the workshop, enabling participants to consider and follow through on potential applications in their own contexts. We will also encourage workshop attendees to add any additional resources they have created, used, and/or learned about related to creating practice-based learning opportunities for teachers to the folder. We will also create a form within this folder where conference attendees can share ideas and pose questions to us and to one another about how to apply what they learned in their own settings.