Using Evidence from Students for Teaching Evaluation
To make soliciting and using feedback from students a positive and productive experience, faculty need to start over. They need to leave behind all the negative evaluative experiences and begin with this premise: feedback from students can be used to improve teaching and learning. The kind of feedback that promotes teacher growth is diagnostic, detailed, and descriptive... When it is [exchanged constructively], feedback from students grows and develops teachers and students.
(Weimer, 2010, p.15)

To assist with the use of student perspectives and learning outcomes as data sources for teaching evaluation, we reviewed the ample literature on the topic as well as processes from other institutions. This overview:

• Describes why students are a useful source of data.
• Outlines the limitations associated with student ratings.
• Describes two types of student data: students' perceptions and learning outcomes.
• Makes connections throughout between collaborating with and collecting data from students, and advancing our FIU vision of teaching excellence.

Students are uniquely positioned to provide useful insights on their perceptions our instruction because they spend the greatest amount of time engaging with our teaching. In contrast to our peers, they can offer feedback from the perspective of the naive learner (Supiano, 2017; Weimer 2010).
Although student course evaluation data (i.e., SPOTs) have been historically used in teaching evaluations, this type of data is also associated with a bevy of biases and limitations (Nilson, 2006). However, when student course evaluation data is combined with additional data on their perceptions and/or evidence relating to their learning, growth, and development, students can be a highly effective data source for refining good teaching practices (Clayson, 2009; Finelli et al., 2008).

What counts as student data for evaluating teaching?
The most commonly-used source of student data for teaching evaluation are student course evaluations—or as they were recently renamed at FIU—SPOTs (Student Perceptions of Teaching survey). Although a large proportion of departments have historically relied on SPOTs results for teaching evaluations, we recognize the limitations of the SPOTs as measures of teaching quality (Theall & Franklin, 2001). For a more detailed discussion of the utility of and limitations of using SPOTs for evaluating teaching please see the SPOTs Guide developed by the Center for the Advancement of Teaching.

Beyond SPOTs
In addition to SPOTs, there are other ways with which faculty can collect useful data on students’ perceptions, which can help them  gain nuanced and/or course-specific insights into their learning experience.

Data on students’ perceptions
Whether you are interested in conducting a mid-semester feedback session or are collecting other student perception data in addition to SPOTs, there are several activities that can provide you with students’ perceptions of their learning experience. In our student data collection table, we provide some suggestions of activities for each method, as well as examples of data/information that you might collect and ideas as to how you might share that evidence. Additionally, the table provides information as to how these data collection activities can be tailored to align with one or more of the pillars of excellent teaching at FIU.

Data on students’ learning outcomes
Although data on students’ perceptions can yield beneficial insights, they are limited, in that they do not provide direct evidence of their learning and development. Analyzing data on students’ learning outcomes can complement their perception data and paint a more comprehensive picture of their experience in the classroom.

The good news is that you are already collecting data on students’ learning outcomes in your course exams, projects, etc. One caveat here: The extent to which exam and project/essay data are effective as measures of learning outcomes depends on how well the exam was designed (including question wording, alignment between outcomes and questions, etc.) or the design of the assignment. With this in mind, Nilson (2006) recommends the use of culminating assessments such as comprehensive exams or final papers. She also suggests that faculty administer a measure that is specifically designed to assess student learning—ideally one that measures students’ learning directly through a pre- and post-test design. Our student data collection table provides examples of activities that can be used to collect both direct and indirect evidence of students’ learning outcomes.

This said, it is important to consider the limitations associated with this type of data as well. For instance, oftentimes, students’ performance on tests, projects, and other measures of learning can be influenced by factors other than their mastery of the material—some of which may be out of the instructor’s control (Berk, 2005). However, the impact of external factors on students’ performance can be minimized by using certain methods of data collection—such as pre- and post-tests. See our student data collection table for examples of pre- and post-test data collection activities.

Final Thoughts
Overall, our review of the research and best practices on the use of student ratings in teaching evaluations yielded several important findings. One, although student data is associated with several limitations, students are a powerful source for formative feedback that can help faculty improve their teaching, and in turn, boost students’ learning outcomes. Two, both researchers and practitioners agree that providing faculty with guidance on using their SPOTs results can result in more effective use of that data (i.e., check out CATs SPOTs guide, “Interpreting and Working with Your SPOTs Results: A Guide for Faculty”). The use of students as a source of data for teaching evaluation is enhanced by the careful use of additional data on students’ learning outcomes. By combining data on students’ perceptions with data on their learning outcomes, faculty can draw connections between students’ learning and the various aspects of instruction and course design that are measured in SPOTs and other perceptions evidence.

Using Peer Feedback
Colleagues can keep the life blood flowing in a teacher’s veins. They can offer encouragement, critique, advice, ideas, and inspiration... We share so much that makes us equal. We all have burgeoning amounts of content to organize and explain, not always well-prepared students to motivate and educate, multiple demands on our time, and tough institutional environments in which teaching is not always valued. When we join forces with colleagues, these challenges are more easily faced and more successfully mastered.
(Weimer, 2010, p. 105)

To assist with the use of peer feedback as a data source for teaching evaluation, we reviewed the ample literature on the topic as well as processes from other institutions. This overview:

1. Describes why peers are a good source of evidence.
2. Expands the notion of who counts as a peer.
3. Shares conclusions and insights from decades of research on the topic.
4. Summarizes best practices in peer collaboration.
5. Makes connections throughout between collaborating with and collecting data from peers, and advancing our FIU vision of teaching excellence.

Note: We know many of us have aversions to or are skeptical of class observations, so we want to state explicitly that we do not equate evidence from peers with peer evaluation of a class session and we do not suggest that peer evaluation be included as an evaluation tool. Instead, as we outline throughout this document—there are a variety of ways peers can provide useful feedback that is focused on improvement and growth. It will also be important to make clear in the departmental guidelines that faculty will determine what information about the peer feedback process to share (i.e. the peers are not completing anything in P180).

Why are peers a useful data source for teaching evaluation?
Our colleagues are in a unique position to provide expert feedback and help us improve our teaching. For instance, Paulsen (2002) points out that “peer review brings content-based contextuality to the evaluation of teaching” (p. 10). Peers are especially equipped to provide feedback on course materials (syllabi, assignment instructions/descriptions, etc) and measures of content knowledge (quizzes, project, test, completed student work). Berk (2005) extends this argument, drawing a parallel between peer feedback on teaching and the peer review process used in other forms of scholarship.

In a study of peer feedback, Donnelly (2007) found that instructors whose teaching was observed and received peer feedback were more likely to apply theory to practice, reflect on the rationale behind their practices, and develop increased confidence and feelings of self-efficacy in teaching. Most recently, Fletcher (2018) found that engineering faculty who developed and implemented a collaborative model for peer review not only used the feedback to improve their teaching, but also cited an increased sense of collegiality within their department as a key outcome.

Who counts as a peer?
Weimer (2010) argues that many established systems of peer evaluation have failed because they insufficiently considered the implications of the question “Who counts as peer?”. For instance, she writes that many peer feedback programs were designed based on the common and flawed assumption that “more experienced faculty (those tenured and promoted) are qualified to judge the teaching effectiveness of those less experienced” (p. 107).

Instead of selecting peers based on seniority or rank, Weimer suggests that, “faculty need...a diverse collection of colleagues with whom they explore a variety of roles and activities” (p. 106). Specifically, this might include a departmental colleague, a colleague from another department at the same institution, a good teacher, someone from the Center for the Advancement of Teaching, a colleague who shares a pedagogical interest, and/or someone to practice teaching on (i.e., anyone willing to play the role of learner).

How might peers provide each other with useful feedback?
Weimer (2010) expands this question, asking us to consider the activities in relation to the varied roles our peers can play and the goals at hand: “When colleagues are collaborators, what kind of roles and activities accomplish the goals of ongoing growth and vitality for the teacher and improved learning experiences for students?” (p. 116-127). She responds with seven possible peer roles:
Colleague as:

1. Collaborator (working on a shared project such as designing a new assignment)
2. Co-learner (of teaching scholarship, a new instructional practice or tool, etc.)
3. Student (offering possible student reactions to course materials, exercises)
4. Questioner (asking about pedagogical beliefs or course policies, for ex.)
5. Critic (constructively disagreeing, identifying practices that may limit learning)
6. Advocate (speaking publicly about policies that enhance or compromise learning)
7. Confidant (listening to one’s joys and struggles)

In a Faculty Focus blog post, Weimer (2017) offers concrete activities aligned with these roles, several of which we adapt in our table of activities for peer data collection. For each activity listed in this table, we also offer suggestions for aligning each activity with FIUs vision of teaching excellence, and list examples of evidence that can be collected and how it might be shared.

Best Practices in Peer Feedback: Formative Peer Review
Formative peer review of teaching is the long-term enhancement of teaching and learning focused on growth and improvement. The process should be primarily driven and guided by the faculty member's personal goals, by feedback from students and/or colleagues, and/or by a desire to address problems in a specific course or academic context (Arreola, 2007). For some faculty, this can mean working with a teaching mentor over the course of several semesters or it can be faculty pairing up to share teaching concerns and working through them together in critical assessments of course materials.
Repeating Smith’s (2014) synthesis of the research: “The overall consensus in the literature on [peer review of teaching] (Bernstein, Jonson, & Smith, 2000; Blackmore, 2005; Bovill, 2008) is that formative peer feedback is best with:

• Faculty Owernship. The faculty member’s own concerns and goals for her/his teaching should guide the process. Ideally, some consensus between and among the faculty about what “good teaching” actually is should be achieved through a critical and collegial dialogue. At FIU, we have articulated a “macro” vision of excellent teaching and invited departments to examine this vision in the context of their department and discipline.
• Confidentiality. Discussions of teaching should be kept between the faculty and the peer they have chosen to work with. Faculty reporting on the results of this peer feedback choose what to upload in P180.
• Relationship of equals . Colleagues need to work together as equals, rather than a peer serving as an expert and the colleague being reviewed as an object of scrutiny.
• Collegial feedback. The colleague making an in-class observation, reviewing a course syllabus, or performing other feedback tasks, provides information that is constructive and collegial rather than evaluative. This ideally takes place within a dialogic format, where issues can be discussed, problems addressed, and plans made for steps toward improvement (Byrne, Brown, & Challen, 2010; Roxa & Martensson, 2009).
• Open-ended processes. Teaching in higher education can be viewed as a form of scholarship (Bastow, 2008; Richlin & Cox, 2004; Simpson & Anderson, 2010) and like all forms of scholarly endeavors, it occurs over time with cycles of practice, review, and application of feedback. As a result, peer review of teaching is an ongoing process of improvement.

Final Thoughts
We recognize that, despite the emphasis on the use of peer feedback focused on growth and improvement in this overview, peer observations are often used for decision-making. It is important to mention that there is consensus among teaching experts that peer observation data should be used for formative rather than for summative decisions (Berk, 2005).

If your department relies on peer observations for the evaluation of teaching, it is important to ensure that the evidence collected through observations is both valid (accurate/appropriate) and reliable (consistent). We strongly encourage working with the Center for the Advancement of Teaching or the STEM Transformation Institute to identify an effective observation tool and protocol that are discipline and course appropriate—and/or reviewing guidelines such as those generated by colleagues at Western Michigan University and Georgia Tech.

Self-Assessment and Reporting for Teaching Evaluation
“Planned, systematic self-evaluation is a self-strengthening process – it builds muscles for reflection and learning. And the more you reflect and learn and then act on your learning, the better you do it next time. Self-evaluation creates a habit for continuous improvement.”
(Chahine, 2008, p.4)

To assist with the use of self-assessment as data for teaching evaluation, we reviewed the literature on the topic as well as processes from other institutions.  This overview:

• Describes why faculty are in a unique position to provide evidence for their own teaching evaluation.
• Acknowledges some of the limitations of self-assessment.
• Makes connections between self-assessment and advancing our FIU vision of teaching excellence.

Why are faculty a useful data source for evaluating their teaching?
As teaching evaluation expert Peter Seldin (1999) points out, faculty have self-knowledge and beliefs that influence their interpretations of their teaching experiences, ones that are likely unknown by their peers or students. This includes what motivates them to teach, why they use certain instructional practices over others, the types of learning objectives they prioritize, and so much more. In their broad study of faculty assessment (including but not limited to teaching), Braskamp and Ory (1994) are even more emphatic, writing that “faculty themselves are the most important assessment source because only they can provide descriptions of their work, the thinking behind it, and their own personal reporting, appraisals, interpretations and goals” (p. 102).
As is the case with peer and student data, instructor’s self-reported data, is associated with some degree of bias—such that faculty often give themselves higher ratings than their students do (Seldin, 1999). For instance, Arreola (1995) found that faculty assigned themselves higher ratings than their students did, particularly in regard to their rapport with students and the quality of their feedback. However, Seldin (1999) contends that faculty may be more open and honest in their self-assessments if the risks of doing so are minimal, such that it is made clear that the primary goal of collecting this data is to improve their teaching-- which is the intention at FIU.  Given these limitations, Berk (2005) affirms that, for personnel decisions, the information provided by faculty “should be … reviewed and compared with the other sources of evidence” (p. 52). Other teaching evaluation scholars not only echo this recommendation, they stress the value of using self-evaluation to complement the feedback yielded from other sources, in our case, students and peers (Arreola, 1995; Braskamp & Ory, 1994; Seldin,1998).

Self-assessment and reporting presents a significant opportunity for FIU faculty to shed light on their many contributions to student learning and success that often go unnoticed. As our colleagues from the Department of History wrote in their proposal for enhanced annual teaching evaluation, our new processes “allow [faculty] to make [their] presently invisible work around teaching visible.”

How can faculty go about self-assessing?
Most faculty evaluate their teaching—albeit informally—through their students’ reactions in the classroom and their performance on exams or assignments. However, purposeful, self-reflective processes are most likely to yield information that can facilitate improvements in teaching (Blumberg, 2014). In our self-assessment data collection table, we present several methods for collecting and reporting self-assessment data that align with the FIUs Vision of Teaching Excellence.

“Best Practices” in Self-Assessment & Reporting
Seldin (1999) argues that self-evaluations should be structured to uncover teaching specifics and that a standardized format for the information is necessary. He also offers two recommendations for what a faculty member can contribute that will benefit a multisource evaluation system.
In specific terms, an instructor’s self-evaluation should:

1. Address his/her objectives, activities, accomplishments, and failures for the year
2. Contain illustrative material and hard evidence of accomplishments (p. 99)

   
Final Thoughts
In closing, we would like to stress that self-assessment of one’s teaching is not necessarily limited to one’s own perceptions and experiences of teaching. Blumberg (2014) and Brooksfield (2017) argue that a critical self-analysis of one’s teaching can help faculty consider the perspective of their students and/or contrast their instructional strategies to best practices in that discipline/field. That said, self-evaluation is an important source of evidence to consider in formative and summative decisions.