Evaluating preservice science teachers: The development and implementation of an observation rubric

Supporting new teachers’ developing instruction is a vitally important aspect of teacher education. In response to major reform efforts in the late 20th century (AAAS, 1990; NRC, 1996), observational instruments for evaluating reform-based science instruction were developed and validated in order to aid research and observation in science instruction (Bodzin & Beerer, 2003; Piburn et al., 2000). More recently, classroom observation instruments that depict models of teaching, including goals and values of specific teacher education programs, have emerged (Caughlan & Jiang, 2014; Grossman, Cohen, Ronfeldt, & Brown, 2014). This approach is designed to illustrate a program’s vision of effective teaching, describing discrete practices in such a way that new teachers can comprehend and implement them in a progressively more expert manner. Used as a tool for communication between the observer (program faculty or classroom mentor) and the observed (preservice teacher), these observation instruments are designed to help preservice teachers imagine what it is that makes for effective teaching, while providing instructional goals as well as the means to reach those goals (Chen, Hendricks, & Archibald, 2011).

Our faculty has developed an Observation Rubric that illustrates the teaching practices at the center of the MAT program’s goal of preparing effective Earth science teachers for high-needs schools. Based in contemporary recommendations for science teaching (NRC, 2012; NRC, 2005) and in the core practices research in teacher education (Kloser, 2014; Stroupe & Windschitl, 2015; Windschitl & Calabrese Barton, 2016), the Observation Rubric is used to assess the development of preservice teachers’ abilities to implement ambitious and equitable science teaching (Windschitl, Thompson, & Braaten, 2018), recognize and respond to students’ cultural and cognitive needs and strengths (Ladson-Billings, 2009; Moll, Amanti, Neff, & Gonzalez, 1992; Tomlinson & Strickland, 2005), assess students authentically and continuously (Black & Wiliam, 2009; Keeley, 2015), use appropriate content knowledge in instruction (Hill, Ball, & Schilling, 2008; Lederman & Lederman, 2014), and explicitly attend to safety concerns in the classroom (NSTA, 2019). MAT faculty also use rubric data to inform continuous program improvement through effectively integrating academic courses with school-based experiences.

Utilizing a modified protocol from the National School Reform Faculty (NSRF, 2019), MAT faculty came together over several meetings in the 2016–2017 academic year to develop the Observation Rubric into an instrument that clearly illustrates the program’s vision for science instruction. Multiple sources of feedback and study of other program’s rubrics, along with video analysis, were used to support the descriptions of reasonable targets for novice teacher’s accomplished instruction and how to illustrate these in an Observation Rubric. The rich conversation that arose from this task allowed faculty to focus on the actual work of classroom teaching, specify which aspects of instruction could realistically and productively be included in an observation rubric, and determine how the rubric could be developed to better represent program expectations for preservice teachers’ instructional development over time. Several rubric criteria were removed entirely because they could not be assessed through classroom observations and were determined to be assessed through multiple course assignments. Likewise, several revisions were made in order to support a focus on observable instructional strategies. The MAT faculty determined inter-rater reliability for the Observation Rubric by observing classroom video together and individually scoring the teaching in the video using the Observation Rubric. We will model this process during the workshop, as described below.

Those within the ASTE membership who would be the most interested in our workshop are those involved in science teacher preparation at both the graduate and undergraduate levels such as teacher educators, methods instructors, curriculum instructors, clinical supervisors, mentor teachers, preservice teachers, and educational researchers. Additionally, the workshop will interest school leaders who support inservice teachers, especially in their first few years of teaching. The workshop will give these key stakeholders an opportunity to think about the attributes of high quality teaching and how they might articulate a vision of effective teaching through developing a rubric.

This workshop will be conducted by three presenters who are faculty in an urban residency-based MAT program and work with preservice and inservice science teachers. Presenter 1 has led workshops at ASTE for the last five years, taught high school earth science and biology for six years, and teaches methods and science content courses in addition to supervising preservice teachers in schools for the last seven years. In her current role, presenter 1 observes and reflects with preservice teachers using the Observation Rubric and leads professional development with the mentor teachers in order to deepen their understanding of the rubric components and to norm the rubric. Presenter 2 has led four workshops at ASTE, taught high school biology for four years, and has been working with preservice and inservice teachers for over 15 years. In her current role, presenter 2 teaches curriculum and instruction and foundations courses, and leads professional development with new graduates of the program in order to deepen their understanding of the instructional approaches represented in the Observation Rubric. Presenter 3 has led four workshops at ASTE over the last few years, was an elementary science teacher, coach, and district instructional specialist for 13 years, and has been teaching methods and research courses along with supervising preservice teachers in schools for the last seven years. In her current role, presenter 3 observes and reflects with preservice teachers using the Observation Rubric and leads professional development with mentor teachers in order to deepen their understanding of the rubric components and to norm the rubric. All three presenters were contributors to the development of the Observation Rubric along with the other faculty in their MAT program.

After attending this workshop, participants will be able to:
1. Describe and discuss practices for observing and providing feedback to preservice teachers.
2. Evaluate a preservice teacher using the observation rubric and provide evidence and next steps.
3. Apply their new understanding of rubric development and use this understanding in their home institution.
We will assess objective 1 as participants engage in discussions throughout the workshop about current practices used in their own as well as in other institutions. Participants will meet objective 2 when they complete an Observation Rubric while watching a video of a preservice science teacher in a classroom. Objective 3 will be assessed through whole group discussion on how participants can apply their new understanding and experiences in their home institutions.

The following is an outline of the workshop that shows the sequence and duration of the activities:
1. Surface current practices (10 min)
2. History of Observation Rubric (10 min)
3. Examine specific criteria of Observation Rubric (30 min)
4. Score video using Observation Rubric and discuss (45 min)
5. How Observation Rubric is used (10 min)
6. Share future plans for Observation Rubric and how participants can apply in home institution (15 min)
The entire duration of this workshop is 120 minutes. The workshop will begin by surfacing participants’ current practices regarding observations and feedback for preservice science teachers. This will be followed by introducing our MAT Program and Observation Rubric including the history of its development. Next, participants will focus on a subset of criteria from the Observation Rubric in order to develop their understanding of the rubric and identify appropriate evidence. They will then watch a video of a preservice science teacher in a classroom and score them on this subset of criteria using the Observation Rubric. Participants will discuss their observations, evidence, and scores in small groups and then as a whole group. We will then share how the Observation Rubric is used among our faculty and with preservice teachers and classroom mentor teachers. Future plans for the Observation Rubric will be shared and participants will discuss how they might apply what they have learned in their own settings.

The presenters will obtain and share the email addresses of the participants so that they might continue to assist and support each other in preservice science teacher development. We will create a Google Folder in Google Drive in order to share updated versions of the Observation Rubric with participants and participants will be encouraged to share their own materials and rubrics within this folder. We will also add videos to the Google Folder that could be used to practice norming an observation rubric.